Towards Information Inequalities for Generalized Graph Entropies

In this article, we discuss the problem of establishing relations between information measures for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the Rényi entropy have been considered for this study. Our main results involve establishing formal relationships, by means of inequalities, between these two kinds of measures. Further, we also state and prove inequalities connecting the classical partition-based graph entropies and partition-independent entropy measures. In addition, several explicit inequalities are derived for special classes of graphs.


Introduction
Complexity of a system, in general, deals with the intricate design and complex interrelations among the components of the system.One of the plausible categorization of complexity analysis is based on the functional behavior, topological properties, and/or at the compositional level of a system [1].Over the years, all these categories have been implemented and contemplated concurrently in all branches of science and social science.However, here we restrict our attention to the topological complexity of network-based systems.This relates to determining the complexity of the underlying graph structure of a network.The quantitative estimation (using measures/indices) of topological complexity has been proven useful in characterizing the networks and has widely spread into all branches of natural sciences, mathematics, statistics, economics and sociology; for e.g., see [2][3][4][5][6][7][8][9][10][11].In the study of complexity, information theory has been playing a predominant role.That is, the measures based on Shannon entropy has been very powerful and useful in determining the topological complexity of networks; see [1,9,12,13].Apart from Shannon entropy, its generalizations such as Rényi entropy [14], Daròczy entropy [15] have also been identified as potential contributors in characterizing network-based systems; see [16].
In this paper, we deal with an interesting and significant aspect when analyzing the complexity of network-based systems.Namely, we establish relations between information-theoretic complexity measures [17,18].Investigating relations (in the form of inequalities) among measures, by and large is very useful when studying large scale networks where evaluating the exact value of a measure is computationally hard and time-consuming.In addition, they also serve as a tool for solving problems.In communication theory, inequalities have paved way to the development of so-called algebra of information, various rules involving mutual information and entropy measure.For example, inequalities such as Young's inequality, Brunn-Minkowski inequality, Fisher's information inequalities have been main contributors to the mentioned development [19][20][21].
Inequalities involving information measures are also referred to as information inequalities [22].They are classified in two types, namely the implicit information inequalities and explicit information inequalities.In particular, when information measures are present on either side of the inequality, we call it an implicit information inequality [22], while in the latter, the information measure is bounded by a function of parameters (or constants) involved.For some of the recent works in this direction, we refer to [17,[22][23][24][25].
The main contribution of this paper, is to establish implicit information inequalities involving the Shannon entropy and the Rényi entropy measures when applied to network graphs.Secondly, we also present implicit inequalities between Rényi entropy measures having two different types of probability distributions with additional assumptions.To achieve this, we analyze and establish the relation between classical partition-based graph entropies [13,26,27] and the non-partition based (or the functional) based entropies [28].Finally, we apply the obtained inequalities to specific graph classes and derive simple explicit bounds for the Rényi entropy.
Measures are based on two major classifications, namely partition based and partition-independent measures.Some basic results on inequalities on real numbers [21,33] are also presented towards the end of the section.Let G = (V, E) be a graph on N vertices where V = {v 1 , v 2 , . . ., v N } and E ⊆ V × V .Throughout this article, G denotes a simple undirected graph.Let X be a collection of subsets of G representing a graph object.Let Γ be an equivalence relation that partitions is the value of probability on each of the partition.
For graphs, the Shannon's entropy measure [34] is also referred to as "the information content of graphs" [35,36] and is defined as follows: Definition 1.The mean information content, H Γ (G), of G with respect to Γ is given by Note that while the above definition is based on partitioning a graph object, another class of Shannon entropy has been defined by [29] where the probability distribution is independent of partitions.That is, probabilities were defined for every vertex of the graph using the concept of information functionals.
Suppose f : V → R + is an arbitrary information functional [29] that maps a set of vertices to the non-negative real numbers and let denote the probability value of v ∈ V .Here, Definition 2. The graph entropy, H f (G), representing the structural information content of G [29,30] is then given by, As a follow-up to Shannon's seminal work [32], many generalizations of the entropy measure were proposed in the literature [14,15,37].These generalized entropies were recently [16], extended to study the class of graphs.In the following, we present one such generalization from [16], namely the Rényi entropy for graphs.Definition 3. The Rényi entropy H α,Γ (G), for 0 < α < ∞ and α = 1, of a graph G [16] is given by, Here, Γ is the equivalence relation on a graph object and p i (1 ≤ i ≤ k) denotes the probabilities defined on the partition induced by Γ.
It has been proved that Rényi entropy is a generalization of Shannon entropy and in the limiting case when α → 1, the Rényi entropy equals the Shannon entropy [37].
Similar to expression (3), the Rényi entropy can be immediately extended [16] to partition-independent probability distributions defined on G. Definition 4. Let H α,f (G), for 0 < α < ∞ and α = 1, denote the Rényi entropy [16]defined using an information functional f .Then Next we state some interesting inequalities from the literature that are crucial to prove our main results.One of the well-known result for the real numbers is stated as follows [33].Lemma 1. [33] Let x, y > 0 and x = y be real numbers.Then if r < 0 or r > 1, rx r−1 (x − y) < x r − y r < ry r−1 (x − y), if 0 < r < 1.
A simplified form of Minkowski's inequality is given by [33]: where As an extension of discrete Jensen's inequality, the following inequality was derived in [21].

Results
In this section, we present our main results on implicit information inequalities.To begin with, we establish the bounds for Rényi entropy in terms of Shannon entropy.
Theorem 4. Let p(v 1 ), p(v 2 ), . . ., p(v N ) be the probability values on the vertices of a graph G. Then the Rényi entropy can be bounded by the Shannon entropy as follows: Proof: It is well known [37] that the Rényi entropy satisfies the following relation with the Shannon entropy . and Next let us prove the other bound of . Consider, the inequality (9) from Lemma 3 with (14) Now we prove the theorem by considering intervals for α.Case 1: When 0 < α < 1.
Dividing by (1 − α) on either side of the expression ( 14), we get Using inequality (6) from Lemma 1, with r = α − 1 < 0 in the following sum we get, Thus, expression (15) becomes which is the required upper bound in (10).
Remark 6. Observe that the Theorem 4, in general holds for any arbitrary probability distribution with non-zero probability values.The following theorem illustrates this fact with the help of a probability distribution obtained by partitioning a graph object.
Theorem 7. Let p 1 , . . ., p k be the probabilities of the partitions obtained using an equivalence relation Γ as stated before.Then when 0 < α < 1, and when α > 1.
Proof: Proceeding similar to Theorem 4, we get the desired result.
In the next theorem, we establish bounds between like-entropy measures, by considering the two different probability distributions. Proof: By raising either side of the expression to the power α, and by applying summation over i from 1 to k on either side we get, Taking logarithms on either side, we obtain Now we make two cases, depending on α as follows: Case 1: When 0 < α < 1, dividing by 1 − α on either side of equation (36), we get Case 2: When α > 1, dividing by 1 − α on either side of equation (36), we get Expressions (37) and (38) are the required inequalities.
Remark 9. A similar relation by considering H Γ (G) and H f (G) has been derived in [24].
We focus our attention to the Rényi entropy measure defined using information functionals (given by equation ( 5)) and present various bounds when two different functionals and their probability distributions satisfy certain initial conditions.A similar study has been performed in the case of Shannon entropy; see [17,22].
Let f 1 and f 2 be two information functionals defined on a graph G = (V, E).
Let p f1 (v) and p f2 (v) denote the probabilities of f 1 and f 2 , respectively, on a vertex v ∈ V .Let H α,f1 (G) and H α,f2 (G) denote the Rényi entropy based on the functionals f 1 and f 2 respectively.
Theorem 10.Suppose p f1 (v) ≤ ψ * p f2 (v), ∀v ∈ V and ψ > 0 a constant, then . By raising to the power of α and by applying summation over the vertices of G, we get, Taking logarithms on either side, we get, = α log ψ + log Case 1: When 0 < α < 1. Dividing either side of the equation by (1 − α), we get the desired expression (39).Case 2: When α > 1.In this case, dividing either side of the equation by (1 − α), we get the expression (40) as desired.
Proof: By the assumption, we have p f1 (v) ≤ S 2 S 1 p f2 (v).Therefore, the corollary follows by letting in the above theorem.
By applying Lemma 2 to the expression (48) we get, Now, taking logarithms on either side, we get = log Dividing by (1 − α), we get the desired expression (47).
and for α > 1, Here, Now raising to the power α and summing it over all the vertices of G, we get Now let us make the cases of the theorem.
Case 1: 0 < α < 1. Applying Lemma 2, we get Taking logarithms on either side, we get Dividing by 1 − α, we get the required expression (60).Case 2: α > 1. Applying Lemma 2 yields, Taking logarithms on either side, we get Dividing by 1 − α, we get the required expression (61). (80) Here, . Proof: The proof follows similar to Theorem 13.In the case of 0 ≤ α < 1, the equation ( 69) is expressed as follows: Now proceeding as before and by simplifying each of the terms in the above equation, we get the required expression (80).
Similarly, in the case of α > 1, the expression (76) is expressed as, As before upon simplification of the above equation, we get the required expression (81).

Applications to chemical graphs
In this section, we consider various classes of simple chemical graphs and illustrate the results from previous section.To this purpose we consider a specific example of the equivalence relation Γ on G and an information functional f P .In order to define concrete graph entropies, we need to specify graph invariants and information functionals to determine a probability distribution.
For the graph invariant we use the automorphism group of a graph.We use this invariant due to their extensive investigations available in the literature; for example see [38].Note that there are various other invariants such as distance, degrees and paths that could be used.Observe that each graph belongs to an automorphism group, where an automorphism is a permutation of the vertices such that the adjacency relation of the graph is preserved.An automorphism group divides the vertex set into orbits where a vertex orbit is a collection of topologically equivalent vertices.

Definition 5. Let Γ be an automorphism (equivalence relation) that partitions the vertex set
For the information functional, we reproduce the definitions of two information functionals based on metrical properties of graphs [29][30][31].
Let G = (V, E) be a simple, undirected graph on n vertices and let d(u, v) denote the distance between two vertices u and v, and let η(G) = max{d(u, v) : u, v ∈ V }.Let S j (u; G) denote the j-sphere of a vertex u defined as S j (u; G) = {x ∈ V : d(u, x) = j}.Definition 6. Parameterized linear information functional using j-spheres: where Definition 7. Parameterized exponential information functional using j-spheres: where β > 0 and c k > 0 for 1 ≤ k ≤ η(G).
Note that the constants c j 's in the above expressions contribute to the weight of the j-spheres and hence the functionals could be meaningfully interpreted when the c j 's are all distinct.

Stars
A Star S n is a tree on n vertices where there is exactly one vertex of degree n − 1 and n − 1 vertices of degree n.Star graphs have been of considerable interest, since they represent trees with smallest possible diameter among all trees on n vertices.
Theorem 15.Let Γ be an automorphism defined on S n such that Γ partitions V (S n ) into two orbits, X 1 and X 2 , with and for α > 1, Observe that, Now by Theorem 7, we have for 0 ≤ α ≤ 1.Hence, Similarly, for α > 1, we have by Theorem 7, That is, Hence the theorem.
Theorem 16.Let Γ be an automorphism on V (S n ) and let f be any information functional defined on for some i and j, 1 ≤ i = j ≤ n.Then, for 0 < α < 1, and for α > 1, Here S = v∈V f (v).
Remark 17.For instance, when f = f ′ P defined by equation ( 88) with constants c 1 , c 2 ≥ 1.We have . By substituting the value of S in equations ( 99) and (100), we get the bounds for H α,f ′ P (S n ).Remark 18.At this point, observe that another class of graph that possesses the same automorphism group as the stars is the wheel graph.A wheel W n is a graph obtained by joining a new vertex v to every vertex of an (n − 1)-cycle C n−1 .That is, W n = C n−1 + {v}.While studying the inequalities for this class of graph, we derived similar expressions as of theorems 15 and 16.Hence we conclude that the theorems 15 and 16 also holds for the wheel W n .

Paths
A path graph, denoted by P n , are the only trees with maximum diameter among all the trees on n vertices.This class of graph has received considerable attention in chemistry when studying the hydrogen-depleted hydrocarbon molecules.Let Γ be an automorphism defined on P n , where Γ partitions the vertices of P n into n 2 orbits (X i ) of size 2, when n is even, and n−1 2 orbits of size 2 and one orbit of size 1, when n is odd.
Theorem 19.Let f be any information functional such that f (v) > 2 for at least n 2 vertices of P n and let Γ be as defined above.Then where S = v∈V f (v).
Proof: H α,Γ (P n ) follows directly from the definition of Rényi entropy.Next, by using the value of H α,Γ (P n ) in Theorem 8, we get the desired expression for H α,f (P n ).

Connected graphs
In this section, we consider any general connected graph G and the functionals f P and f ′ P given by equations ( 89) and (88) respectively.In the next two theorems, we present the explicit bounds for the Rényi entropy H α,f (G), when we choose the two information functionals in particular.Theorem 20.Let f = f ′ P given by equation (88).Let c max = max{c i : 1 ≤ i ≤ η(G)} and c min = min{c i : 1 ≤ i ≤ η(G)} where c i is defined in f ′ P .Then the value of H α,f ′ P (G) lies within the following bounds. (104) Similarly, Therefore, combining the Equations ( 107) and (108) and by adding over all the vertices of G, we get Hence, By raising to the power α and adding over all the vertices of G, we have Taking logarithms we get, Dividing the expression (112) by (1 − α), and simplifying we get the desired expressions given by ( 105) and (106) depending on the value of α. Hence, Let X = c max − c min .Now, by raising p f (v) to the power α and adding over all the vertices of G, we have, Taking logarithms we get,

Conclusions and Summary
In this article, we have studied one of the most daunting problems in the study of information measures, that is, establishing relations between graph entropy measures.Among all the entropy measures, we have considered the graph entropies defined using the classical Shannon entropy and the Rényi entropy.Further, we have also considered two major types of probability distributions in the definitions of the entropies, namely the classical partition-based distributions and the recent non-partition-based distribution defined on the vertices of the graph by using information functionals.Thus, we have established analytical relations when the two different types of distribution are used for the measures.
In general, the results could be used in various branches of science including mathematics, statistics, information theory, biology, chemistry and social sciences.Further, the determination of analytical relations between measures gains practical importance when dealing with large scale networks.Moreover, relations involving quantitative network measures could be fruitful when determining the information content of large complex networks.