Estimation of different types of entropies for the Kumaraswamy distribution

The estimation of the entropy of a random system or process is of interest in many scientific applications. The aim of this article is the analysis of the entropy of the famous Kumaraswamy distribution, an aspect which has not been the subject of particular attention previously as surprising as it may seem. With this in mind, six different entropy measures are considered and expressed analytically via the beta function. A numerical study is performed to discuss the behavior of these measures. Subsequently, we investigate their estimation through a semi-parametric approach combining the obtained expressions and the maximum likelihood estimation approach. Maximum likelihood estimates for the considered entropy measures are thus derived. The convergence properties of these estimates are proved through a simulated data, showing their numerical efficiency. Concrete applications to two real data sets are provided.


Introduction
Information theory provides natural mathematical tools for measuring the uncertainty of random variables and the information shared by them. In this regard, entropy and mutual information are two fundamental concepts. More precisely, the probability distribution of a random variable is associated with some sort of uncertainty, and entropy is used to quantify it. The concept of entropy was formerly proposed by [1]. Since that publication, many areas of study such as statistics, neurobiology, cryptography, bioinformatics, quantum computer science and linguistics, have developed various entropy-based measures. Modern and exhaustive reviews on the 'entropy universe' can be found in [2][3][4][5][6].
In applied probability and statistics, many authors have conducted their studies for diverse and important distributions based on entropy. The essential references in this regard are briefly presented below. Reference [7] used the concept of entropy to communicate on the probability distribution of electric charge between atoms observed in a certain condition.
Reference [8] derived the entropy for the Feller-Pareto family and presented the entropy ordering property for some related sample minimum and maximum. Reference [9] estimated the entropy of the Weibull distribution by considering different loss functions based on a generalized progressively hybrid censoring scheme. Reference [10] discussed the entropy for the generalized half-logistic distribution based on the type II censored samples. References [11] and [12] proposed estimates for the entropy of absolutely continuous random variables. Reference [13] presented an indirect method using a decomposition to simplify the entropy's calculation under the progressive type II censoring. Reference [14] derived a nonparametric kernel estimator for the general Shannon entropy. Reference [15] estimated the entropy for several exponential distributions and extended the results to other circumstances. Reference [16] estimated the Shannon entropy of the Rayleigh model under doubly generalized type-II hybrid censoring, and evaluated its performance by two criteria. Reference [17] derived a nonparametric wavelet estimator for the general Shannon entropy. Reference [18] provided an exact expression for entropy information contained in both types of progressively hybrid censored data and applied it in the setting of the exponential distribution. Reference [19] investigated entropy measures for weighted and truncated weighted exponential distributions. Reference [20] presented the estimation of entropy for inverse Weibull distribution under multiple censored data. Reference [21] introduced estimation of entropy for inverse Lomax distribution under the multiple censored scheme. Reference [22] examined Bayesian and non-Bayesian methods to estimate the dynamic cumulative residual Rényi entropy for the Lomax distribution.
Surprisingly, to our knowledge, the entropy of the famous Kumaraswamy distribution has not been studied in depth. In this article, we fill this gap both probabilistically and statistically. The specificities and interests of the Kumaraswamy distribution are described below. First, it was introduced by [23], and was motivated as an alternative to the beta distribution which are (i) mathematically simpler, without special function in particular, and (ii) more suited to the modeling of various hydrological phenomena observed at low frequency (daily rainfall, daily flow of rivers, etc.). Mathematically, the probability density function (pdf) of the Kumaraswamy distribution is specified by in the same way as the beta distribution. The corresponding cumulative distribution and quantile functions are quite simple; they are defined without special function contrary to those of the beta distribution. Special cases of the Kumaraswamy distribution correspond to the distribution of minimum or maximum of uniform samples. We may refer the reader to [24] for all the known features of this distribution. Also, the kumaraswamy distribution has generated many flexible distributions with various domains and number of parameters through the generalized Kumaraswamy class elaborated by [25].
In a sense, this study complements the work of [24] by investigating the overall concept of entropy of the Kumaraswamy distribution, which has never been studied before. More precisely, we consider six well-referenced entropy measures. We derive their analytical expressions by using the well-known beta function. We compare them numerically by considering different parameter values. Then, we propose an efficient strategy based on the maximum likelihood approach to estimate these entropy measures. A simulation study is done to see how effective our strategy is. Graphical and numerical comparisons are performed. We end the study by two illustrative examples on real data sets, showing how the methodology can be applied in a concrete statistical setting.
The following sections make up the document. Section 2 presents a result on a special integral, and shows how it is related to important entropy measures of the Kumaraswamy distribution. Numerical values of these entropy measures with different values of the parameters are also given. Section 3 studies the estimation of these entropy measures. Then, using generated values from the Kumaraswamy distribution, graphical and numerical comparisons are discussed. The entropy of the random characteristics behind two real data sets is investigated. Finally, conclusions are presented in section 4.

An integral result
The following result shows that a certain integral involving the pdf of the Kumaraswamy distribution can be expressed in terms of the classical beta function. The connection between this integral and the considered entropy measures will be developed later.
where B (u, v) , which is integrable in the neighborhood of 0 if and only if δ(1 − a)<1 by the Riemann integral criteria. Similarly, when x tends to 1, we have This ends the proof of Proposition 1.
In fact, the beta function is implemented in most of the mathematical software (see the function beta of the package stat of R, the Beta function of Mathematica, etc.). Therefore, thanks to Proposition 1, the computation of I δ (a, b) can be done quite efficiently with little effort. Also, the existing results on the beta functions allow a mathematical control of this integral. Some related results are presented below.
• Through the use of the standard Euler gamma function given as GðuÞ ¼ R þ1 0 x uÀ 1 e À x dx, one can write Gðdðb À 1=aÞ þ 1=a þ 1Þ : • Also, assuming that δ(1 − 1/a) + 1/a and δ(b − 1) + 1 are positive integers, the following formula holds: • By virtue of the main result in [26], if δ(a − 1) � a − 1 and b � 1, then we have with the best possible constants α = 0 and β = 0.08731. . .. Therefore, for not too large value of δ, the following numerical approximation seems acceptable: In our study, the interest of Proposition 1 is that I δ (a, b) is the main ingredient in the definitions of various entropy measures of the Kumaraswamy distribution, as developed in the next part.

Various entropy measures
The entropy of the Kumaraswamy distribution can be measured in different manners. The most useful entropy measures of the literature are recalled in Table 1 for a general distribution with pdf denoted by f(x;φ), φ representing a possible vector of parameters. Also, we suppose that δ > 0 and δ 6 ¼ 1 as basic assumptions in this general case.
For the two entropy measures proposed by [27], it is supposed that sup x2R f ðx; φÞ is finite and well identified.
From Table 1, we see that the integral R þ1 À 1 f ðx; φÞ d dx is central to determine the considered entropy measures. Now, we present the corresponding entropy measures of the Kumaraswamy distribution. Based on Proposition 1, it is supposed that a, b and δ satisfy min(a, b)>max(1 − 1/δ, 0). Table 1, Eq (1) and Proposition 1, the Rényi entropy of the Kumaraswamy distribution can be expressed as

Rényi entropy. Based on
Havrda and Charvát entropy. From Table 1, Eq (1) and Proposition 1, the Havrda and Charvát entropy of the Kumaraswamy distribution can be expressed as Arimoto entropy. Again, from Table 1, Eq (1) and Proposition 1, the Arimoto entropy of the Kumaraswamy distribution is specified by Tsallis entropy. Based on Table 1, Eq (1) and Proposition 1, the Tsallis entropy of the Kumaraswamy distribution can be expressed as

Name of the entropy Reference Notation Expression
Rényi Havrda and Charvat [29] HC δ (φ) Awad and Alawneh 1 entropy. From Table 1, Eq (1) and Proposition 1, the Awad and Alawneh 1 entropy of the Kumaraswamy distribution is given as Before going further, we need to determine sup 0<x<1 f(x;a, b). The following lemma provides the necessary in this regard.
Lemma 2 Let f(x;a, b) be given as Eq (1). Then, sup 0<x<1 f(x;a, b) is finite if and only if a � 1 and b � 1 with ab 6 ¼ 1, and in this case, we have Proof. We have Note that, for a = 1, with the convention This ends the proof of Lemma 2.
Based on Lemma 2, if a > 1 and b > 1, Eq (2) becomes Awad and Alawneh 2 entropy. From Table 1, Eq (1), Proposition 1 and Lemma 2, the Awad and Alawneh 2 entropy of the Kumaraswamy distribution is given as Theoretically, it is complicated to study the behavior of these entropy measures. For this reason, a numerical study is proposed in the next section.

Numerical values
We now investigate the numerical values for the six entropy measures presented in Subsection 2.2 under the following configuration of the parameters: Configuration 1:  Tables 2-7, respectively.
In view of Tables 2-7, the following comments can be formulated. First, we recall that Tables 2-4 indicate the values of the entropy measures of the Kumaraswamy distribution for a fixed value of a and different values for b and δ. In this context, Table 2. Numerical values of the considered entropy measures of the Kumaraswamy distribution at a = 2 and δ = 0.5.  Table 3. Numerical values of the considered entropy measures of the Kumaraswamy distribution at a = 2 and δ = 1 • the Rényi, Havrda and Charvat, Arimoto and Tsallis entropy measures are decreasing when δ is increasing while the Awad and Alawneh1 entropy is increasing when δ is increasing, but the Awad and Alawneh2 entropy is decreasing and increasing when δ is increasing.  Table 5. Numerical values of the considered entropy measures of the Kumaraswamy distribution at b = 2 and δ = 0.5. • the Rényi, Havrda and Charvat, Arimoto and Tsallis entropy measures are decreasing when δ is increasing while the Awad and Alawneh1 entropy is increasing when δ is increasing, but the Awad and Alawneh2 entropy is decreasing and increasing when δ is increasing.

Maximum likelihood estimation
The inference on the six considered entropy measures of the Kumaraswamy distribution is now investigated via the maximum likelihood technique. This technique is well-known and has proved itself in various modern studies such as those in [32][33][34].

Estimation of the entropy measures
The estimation of the parameters of the Kumaraswamy model through the maximum likelihood technique is well-known and the details can be found in [24]. The minimal theory is recalled below. Based on n values x 1 , . . ., x n supposed to be observed from a random variable X with the Kumaraswamy distribution with parameters a and b, the maximum likelihood estimates (MLEs) of a and b, sayâ andb, are defined by ðâ;bÞ ¼ argmax ða;bÞ2ð0;þ1Þ 2 'ða; bÞ; where ℓ(a, b) denotes the log-likelihood function specified by These MLEs are also the solutions of the two following equations according to a and b: That is,â andb satisfy the following simple relation: Then, the properties of these MLEs follow from the usual maximum likelihood theory. In particular, thanks to the functional invariance of the MLEs, one can deduce easily the MLEs of the entropy measures. More concretely, based on the six entropy measures described in Subsection 2.2, R d ðâ;bÞ is the MLE of R δ (a, b), HC d ðâ;bÞ is the MLE of HC δ (a, b), A d ðâ;bÞ is the A δ (a, b), T d ðâ;bÞ is the MLE of T δ (a, b), AA1 d ðâ;bÞ is the MLE of AA1 δ (a, b), and AA2 d ðâ;bÞ is the MLE of AA2 δ (a, b).

Simulation
We now investigate the numerical behavior of the MLEs of the entropy measures via the use of simulated values. That is, we consider N = 5000 samples of values from a random variable X with the Kumaraswamy distribution of parameters a and b with different samples sizes; n = 100, 200, 300 and 1000 are considered. The following configurations on the parameters are considered: Configuration1: a = 3, b = 3 and δ 2 X with X = {0.5, 1.5, 2.5}, and Configuration 2: a = 3, b = 5 and δ 2 X.
In each configuration, for each sample, the MLEsâ andb are determined. Then, based on the N samples of fixed size, we determine the average of the N MLEs and use it to define the entropy estimates. The corresponding mean squared error (MSE) and mean deviation (MD) defined by the following generic formulas: MSE = sum(exact value-estimate) 2 / N and MD = sum abs(exact value-estimate) / N, respectively, are also calculated. These assessment criteria are often used quite effectively to make a full comparison of models. In this regard, we can refer the reader to the useful works of [35][36][37].
The results on the Rényi entropy under Configurations 1 and 2 are given in Tables 8 and 9, respectively, results on the Havrda and Charvat entropy under Configurations 1 and 2 are indicated in Tables 10 and 11, respectively, results on the Arimoto entropy under Configurations 1 and 2 are presented in Tables 12 and 13, respectively, results on the Tsallis entropy  Tables 18 and 19.  Based on Tables 8-19, in all the situations, we see that the MLEs of the entropy measures are close to the target values and, as anticipated, the MSEs and MDs decrease and approach 0 as n increases. This proves the accuracy of the proposed estimation methods in the context of the Kumaraswamy distribution. Also, one can notice that the MSEs and MDs increase as δ increases.  1 (a = 3, b = 3 1 (a = 3, b = 3).  1 (a = 3, b = 3).

Illustrative examples
In this Section, two real life data sets are used to illustrate the proposed methodology. The considered data sets are described below.
The first data set. The data set consists of 48 rock samples from an oil reservoir. It corresponds to twelve oil tank cores that were sampled by four cross sections. Each core was measured for permeability and each cross section has the following variables: total pore area, total pore perimeter, and shape. We analyze the perimeter of the shape by a squared variable (area). It has been analyzed by [38], among others. Explicitely, the data set is:   The second data set. This data set contains 20 observations of flood data. It was analyzed by [39]. In order to check the adequateness of the Kumaraswamy distribution to these data, we apply the Kolmogorov-Smirnov test. We find p-value 0.2092 and p-value = 0.3359 for the first and second data sets, respectively. Since both satisfy p-values >0.05, the two considered data set are in adequateness with the Kumaraswamy distribution.   Table 9. https://doi.org/10.1371/journal.pone.0249027.g002 Now, Tables 20 and 21 present the estimations of the six entropy measures considered in Subsection 2.2, following the methodology described in Subsection 3.1, for the first and second data sets, respectively.
We can notice that, under our framework, the Rényi, Havrda and Charvat, Arimoto, Tsallis, Awad and Alawneh 2 entropy measures are decreasing when δ is increasing while the Awad and Alawneh 1 entropy is increasing when δ is increasing.  Table 11.
https://doi.org/10.1371/journal.pone.0249027.g004 Tour knowledge, it is the first time that the entropy of the uncertainty behind these data sets are evaluated. They can be taken into account for further statistical analysis in the future.

Conclusion
For the first time, this article proposed a special focus on the entropy of the Kumaraswamy distribution. Both theoretical and practical aspects were covered, though complementary works. In particular, six different entropy measures were investigated. After determining the  Table 14.
https://doi.org/10.1371/journal.pone.0249027.g007 closed-form expressions of these measures, an estimation strategy was developed to evaluate them in a practical setting. A simulation study ensured the convergence of the obtained estimates. Two real-life data sets are used to show how the related entropy can be concretely estimated. The finding of this study aims to be applied by the statistician to assess the entropy of diverse data with values on the unit interval, such as modern rate, percentage and proportion type data.  Table 15.
https://doi.org/10.1371/journal.pone.0249027.g008 The limitation of current research remains on the classicity of the statistical framework considered. Directions for future research include the estimation of the entropy of the Kumaraswamy distribution in more sophisticated statistical schemes with physical motivations, such as the progressive type II censoring scheme, generalized progressively hybrid censoring scheme, etc., or taking into account generalized versions of the Kumaraswamy distribution, such as the one proposed by [40].  Table 19. https://doi.org/10.1371/journal.pone.0249027.g012