1. Introduction

Hierarchies are ubiquitous forms of organization. Not surprisingly, early comparative studies between administrative and economic hierarchies, and other studies of hierarchies in the the context of political organization, date back to the beginning of the twentieth century [1, 2]. Since then, their study has encompassed a large variety of disciplines like, for example, labor and organizational economics [3–5], moral and evolutionary psychology [6–9], anthropology [10] and sociology [11, 12]. We remark that in the economics literature, the importance of hierarchies also comes from the study of firms: the structure of firms can be understood as hierarchical structures in production organizations (e.g., superiors having veto power over subordinates to access productive assets) [13, 14]. In general, the empirical study of the emergence of hierarchical structures in social networks has benefited from advancements in the field of sociometry [15], and, theoretically, graph theoretical analysis has proven to be a useful tool for structural analysis of general hierarchical structures [16]. In this paper, we adopt a game theoretical perspective to study the emergence of hierarchical structures through what is known as network formation games. We propose a game and analyze how its play leads to the emergence of hierarchies. The importance of our approach stems from the fact that this game theoretical study allows us to investigate the underlying structures in the individual utilities that promote the emergence of hierarchies from self-interested individuals. Then, the importance of this work is that it defines micro-mechanisms on the individual level that can lead to the emergence of hierarchical structures on the macro-level.

To the best of our knowledge, there is no previous study about the emergence of hierarchical network structures from a network formation perspective. However, we briefly mention that there has been a considerable body of literature on the study of hierarchies from other game theoretical perspectives [17]. For example, there are studies in the literature about allocation rules that take into account explicit hierarchical relationships among the agents (e.g., [18]), and about the analysis of games where agents have veto power over the actions of other players as a result of an underlying hierarchical structure (e.g., [19]).

On the other hand, there is a mature body of work in the network formation game literature related to other applications. Generally speaking, in a network formation game, self-interested or utility-maximizing agents decide to form or severe links (interpreted as some sort of relationship) with other agents so that they can form network structures that maximize some utility or allocation value. The structure of connections resulting from playing the game induces an emergent network whose topology can have different interpretations according to the specific application the game is modeling. The literature on network formation games has a long trajectory which historically started from a cooperative perspective, and has later devised a huge boom from a non-cooperative perspective [17, 20]. We present an overview of the literature on network formation games according to a classification of different previous approaches, this classification also introduces relevant technical terms that will be useful in this paper. For other more algorithmic perspectives to the subject in terms of solutions concepts and efficiency, not mentioned in this review, we refer to the textbook treatment of the subject in [21, Chapter 19]. Network formation games have been analyzed via a static analysis, that is, the analysis of solution concepts that describes the emergence of particular “stable” networks as a result of the agents playing the game. Additionally, they have also been analyzed via a dynamic analysis, that is, the study of the emergence of a subset (or possibly all) of the networks defined by the static analysis (also known as equilibrium selection) as the result of some defined dynamic rules that describe the evolution of the agents’ actions [22, 23]. For example, [24] uses Nash equilibrium as solution concept, whereas [25] uses the solution concept called pairwise stability (first introduced in [26]); and both works establish a dynamical analysis of convergence towards “stable” networks in their respective solution concept. The two solution concepts mentioned above are the ones that have been typically used in applications where the efficiency of the formed network was the main focus of study; however, in general, solution concepts are proposed (and sometimes renamed) depending on the specific application to study from the network formation perspective (whether there is only a static analysis or a dynamic one too). For example, other solution concepts exist like the generalization of pairwise stability called strong stability also studied in the context of network efficiency [27], or like the solution concept introduced in [28] which is particularly tailored for the problem of routing traffic on networks.

Broadly, the literature on network formation games can be classified according to the nature of the agents that play the game and/or the nature of the connections they can establish. We discuss first the possible nature of the agents. Traditionally, a first distinction is whether the agents are non-consensual or consensual. A non-consensual agent i can establish links unilaterally with agent j without j’s consent, i.e., without j having to agree to accept such link (see, e.g., [24]); and a consensual i is the one who needs j’s consent or approval to establish a link (see, e.g., [25, 29, 30]). Another second major distinction regarding the agents is whether they all have the same structure in their utility or payoff function, i.e., the rewards or costs of making/maintaining/severing a link is the same for any agent, which is know as the homogeneous case; or whether they have different structures in their utility function, which is the heterogeneous case [31].

Regarding the distinction on the nature of the connections that agents can establish, network formation games can be classified depending on whether the formed network is undirected (i.e., the edge or link between agents i and j has no direction) or directed (i.e., every edge or link has an intrinsic direction, so that an edge can be directed from i to j). The interpretation is that an undirected edge between i and j gives a two-way benefit of flows, i.e., the utility generated by this edge benefits both i and j. On the contrary, a directed edge from i to j has a one-way flow of benefits since the utility generated by this edge benefits i but may or may not benefit j [24, 31]. Examples of undirected relationships are friendships, collaborations (e.g., co-authorship), mutual insurance, etc. Examples of directed relationships arise naturally in communication networks (e.g., someone send an email to another person), loan relationships, subordinate-authority relationships, buyer-seller relationships, etc. We refer to [32, Chapter 5] for more discussion on the use of undirected vs directed graphs in game theoretical modeling.

We stress that not all network formation games in the literature have been studied from a dynamic analysis (indeed, all the classifications presented above only have to do with the static analysis aspect). A first classification from a dynamic analysis is to consider whether the agents, at any time-step of the (discrete) evolution of the game, are myopic (e.g., [24, 25, 30]), i.e., they want to only maximize their current utility at any time step, or, otherwise, foresighted (e.g., [33–35]), i.e., they also want to maximize their (expected cumulative) future utility. Myopic agents are a more reasonable assumption in contexts where agents live in a big and/or complex network, where it is virtually impossible for a single agent to completely predict how her actions would influence over the rest of the whole network and thus how current actions will affect future utility.

1.2 Notation and preliminary modeling

Let N = {1, …, n} be the finite set of agents, and, throughout the paper, consider n ≥ 3. Let ij ≔ (i, j)∈N × N. Any element of N is a node of the directed graph (digraph) G, and G is thus defined by a set of ordered pairs between different elements of N. (Strictly speaking, a graph is a pair (N, E), where N is the node set N and E is the edge set E; however, for simplicity, we adopt a notation traditionally used in the network formation literature (e.g., see [20]) and refer to the graph by its edge set; we let the context specify the node set.) An element ij ∈ G is called an edge from i to j. Let G + ij ≔ G∪{ij} and G − ij ≔ G\{ij}.

We define and , with the in-degree of i being . Given G and i ∈ N, if () then i is a source (sink). If ji ∈ G and ij ∉ G, then we say ij is a single edge; and, if ij, ji ∈ G then we say i and j have an undirected edge. A graph is complete if there is an undirected edge between any pair of nodes.

Whenever there exists a sequence of edges ik₁, …, k_m−1 j that connects i with j with no edge appearing more than once and no node appearing more than in two edges, we say that there exists a path of length m between i and j. Whenever j = i in the previous path, we say that this path is a cycle containing i. We denote by C(G) the condensation graph of G, and by Γ(i, G) the connected component of i ∈ N. Graph G is acyclic if it has no cycles, i.e., it has no paths of one node to itself. A directed cycle is any cycle that contains at least one single edge.

We refer to Section 6 for basic graph theoretical terminology and concepts that will be used throughout the paper.

1.2.1 New notation and definitions.

Given a directed acyclic graph G′ and a node i of G′, the level of i, denoted by ℓ_i(G′), is the largest number of arcs or edges from any source node of G′ to i. Also, for digraph G, the level of i denoted by ℓ_i(C(G)), is the largest number of arcs in C(G) from any source of C(G) to the strongly connected component containing i. See Fig 1(a).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Graphs and notation.

In (a), we number the levels of all nodes in a directed acyclic graph. In (b), given two agents i and j, we represent relationships of subordination in the upper figure (here, i is j’s subordinate), and collaboration in the lower figure. In (c), , , , , P_i(G, m) = P_j(G, m) = P_i(G) = P_j(G) = {i, j}.

https://doi.org/10.1371/journal.pone.0255990.g001

We also define and .

Consider a graph G and any i ∈ N. Given a node , we define P_i(G, k) = {i}∪{j ∈ N|kj ∈ G and ij, ji ∈ G} and . We use the term undirected connected component whenever all edges of the connected component are undirected, and we call it complete connected component when additionally it is a complete subgraph.

Given an undirected connected component that contains nodes i and j, we say the undirected edge ij, ji ∈ G is critical if Γ(i, G)≠Γ(i, G − ij).

Finally, throughout the paper, we use the terms agent, node and individual interchangeably, as well as the terms network and graph, and the terms edge and link.

2 Proposed model and game

In our model, any single edge ij is a pairwise relationship that describes an unambiguous hierarchical or authority relationship where agent j has a higher hierarchy or exercises some authority over i; so we say j is a supervisor of i, and, respectively, that agent i is a subordinate of j. This simple concept has been used extensively in the game theory literature [32]. On the other hand, whenever i and j have an undirected edge, we understand that they have a collaborative or cooperative relationship, and thus, they must share the same hierarchical status and no agent is the subordinate of the other one. Accordingly, we have that is the set of j’s subordinates, P_j(G, i) is the set of j’s collaborators who also have i as a common subordinate, and P_j(G) is the set of j’s collaborators who have the same subordinates as j. See Fig 1(b) and 1(c).

2.1 Utility definition

We first specify the costs and rewards that any agent incurs according to our proposed model.

1. (i). Subordinate-collaborative reward: From an organizational perspective, this represents the positive reward an individual obtains by being the subordinate or collaborator of another competent agent. For example, consider agent i wants to pursue her own goals (e.g., as an employee), then she will seek direction and help from perhaps an equally or better skilled person (e.g., a competent boss or co-worker). This reward can also be interpreted as an exogenous reward (e.g., a salary) that an agent receives by working for or with other people inside an organization. We represent the subordinate-collaborative reward per link by the positive constant γ.
2. (ii). Hierarchical reward: Individuals who have a higher hierarchical status or position receive an inherent reward for being in a higher position of power. For example, in companies, political organizations, and public institutions, individuals with higher hierarchical positions can exert more influence than those in lower positions (e.g., leaders or directives have more influence over ordinary individuals in the society or over their political base). Then, the reward for these individuals comes from the strength of their influence and the reach of the consequences of their decisions. Mathematically, it is natural that if i and j both appear in some cycle, then agent i has no authority or any hierarchical power over j, and so i must have the same hierarchical reward as j. Therefore, two agents in the same strongly connected component have the same reward. In summary, we model the hierarchical reward for agent i in the graph G as a (strictly) monotonic function which takes as an argument the level of i. Formally, the hierarchical reward for an agent i situated in the network G is (1)
   We call H the reward function, and we assume it is non-negative and (strictly) monotonic (e.g., we have that H(0)≥0). Intuitively, the monotonic property of H means that individuals with higher positions of status are better rewarded.
3. (iii). Management cost: An individual i who is in charge of a group of people incurs on a management cost directly proportional to the number of her subordinates. There are two possible reasons for this. First, leading or directing more subordinates requires more managerial or logistic work, thus demanding more energy than doing it with a smaller group, since i has to respond to the demands of more people. Naturally, people who collaborate with i (i.e., any j such that ij, ji ∈ G) do not incur a management cost to i. Second, from an organizational perspective, a supervisor i is responsible for the group she leads, and she also needs to report to her immediate superiors (i.e., the people, if any, to whom she is a subordinate) about how the people she is responsible for are performing. This reporting task, obviously, requires more work when i is directing larger groups. Observe that management costs must be reduced if i is part of a team of people who also manage or direct her same subordinates since this team will collaborate with her and thus reduce the workload on i. Formally, we model the management cost as (observe that this quantity is well-defined) for some positive constant c > 0. The chosen functional form for the management cost captures the qualitative behaviors mentioned above: the larger the number of subordinates of i (i.e., the larger the set ), the larger the cost; and the larger the number of collaborators (i.e., the larger the set P_i(G, k) with ), the smaller the cost.

We conclude that the utility of an individual is the sum of her subordinate-collaborative and hierarchical rewards minus her management cost.

Definition 2.1 (Utility function). The utility that an individual i incurs given a graph G is (2)

Clearly, if i has no connections and no other agent is her subordinate in G, then u_i(G) = 0.

Example 1: We present a couple of graphs for which we compute the utilities of some agents to better illustrate the calculation using Eq (2). Consider Fig 2. If G is as in (a), then

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Network examples.

https://doi.org/10.1371/journal.pone.0255990.g002

If G is as in (b), then

3 Static analysis

3.1 Hierarchical structures

We introduce a useful class of networks that abstracts and represents the concept of hierarchies. Intuitively, agents with a higher level have higher positions of power, and formally, this is true, since agents with a higher level have a greater hierarchical reward (see Eq (1)). When k different connected components have the same level ℓ, we say there are k components per level ℓ. If the maximum level that any agent in the network has is ℓ, then we say the network has ℓ + 1 levels.

Definition 3.1 (Hierarchical structures). A (weakly) connected digraph G is a hierarchical structure if each connected component is an undirected subgraph, and it is additionally a sequential hierarchy if there is only one connected component per any level, and, if there is more than one level, then any node has a single edge towards any other node of a higher level.

A hierarchical structure G has the particularity that any of its connected components are composed of agents that form strong collaborative units or teams, i.e., all agents in a connected component are collaborators. Thus, G is naturally a hierarchical representation of the organization, power and/or influences among teams of people (with the understanding that a team may be composed of a single individual). Now, a sequential hierarchy G is a hierarchical representation in which there is only one team at each rank or level of the hierarchy. Observe that a particular trivial case of a hierarchical structure which is also a sequential hierarchy is when the network is complete, i.e., all agents form a unique team. See Fig 3 for examples of hierarchical structures.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Examples of hierarchical structures.

Consider each subfigure, (a) and (b), as an example of a digraph G which has its condensation graph C(G) represented by the graph with white-colored nodes on the left, and which has a subgraph corresponding to the nodes A and B of C(G) represented by a graph with gray-colored nodes on the right. In (a) we have a particular example of a hierarchical structure which is not a sequential hierarchy, whereas in (b) we have a hierarchical structure which is a sequential hierarchy. In both cases, all agents corresponding to node A in C(G) have level 2, whereas all agents corresponding to node B in C(G) have level 1.

https://doi.org/10.1371/journal.pone.0255990.g003

Observe that a pyramidal structure is formed by a hierarchical structure with two or more levels, generally with one connected component per any level, and such that the number of agents that have a lower level is greater than the ones that have a higher one. This type or class of graphs is easily found in real life hierarchies; for example, military organizations (there are less generals than colonels, but less colonels than lieutenants, etc.) and clerical hierarchies in the Catholic church [37] (there is only one Pope, and more bishops, but there are less bishops than priests) have this type of hierarchies. Moreover, it is known that many companies naturally adopt a pyramidal structure for its management organizations [38], with the CEO or president at the top, followed by a small executive leadership or vice-presidents, followed by tiers of middle managers, and all the way down to the lowest level employees.

3.2 Solution analysis

Now, we introduce the main results for this section. All relevant proofs can be found in Section 7.

Theorem 3.1 (Non-consensual agents). Consider non-consensual agents. A network G is an equilibrium network for some game if and only if it is a sequential hierarchy that satisfies the following conditions:

1. (i). If the network has k > 1 levels, then the following condition holds: (3) for any i, j ∈ N such that ℓ_i(C(G)) = ℓ and ℓ_j(C(G)) = ℓ − 1 with 0 < ℓ ≤ k − 1.
2. (ii). If there exists some critical edge ij, ji ∈ G, then (4)

Moreover, there exist sequential hierarchies that are not equilibrium networks for any hierarchical network formation game.

Remark 3.2 (Interpretation of Theorem 3.1). The importance of Eqs (3) and (4) is that they relate the agents’ utility parameters (γ, H, c) with the specific topology of the sequential hierarchy G resulting from the hierarchical network formation game with non-consensual agents.

We start by interpreting Eq (3). A first observation is that this equation specifies how the increment in the hierarchical reward between the current level ℓ and the previous level ℓ − 1, i.e., Δ(H)≔ H(ℓ) − H(ℓ − 1), restricts the number of agents that can exist at level ℓ, i.e., |P_i(G)|. Interestingly, the larger the increment Δ(H), the smaller the smallest possible number of agents with level ℓ, that can exist in an equilibrium network. (To analyze this fact in the right-hand side of Eq (3) observe that the function with x > 0 and positive constant a, is monotonically decreasing.) From an organizational point of view, this can be interpreted as follows: whenever there is a large difference in power between two consecutive levels in the hierarchy of an organization, we can then expect to find less people in the higher level. For example, in a company, there are fewer executives than managers. Therefore, the equilibrium networks from Theorem 3.1 are able to model a pyramidal structure with N levels by, for example, enforcing monotonic increments in the hierarchical reward (e.g., H(1) − H(0)<H(2) − H(1)<…<H(N) − H(N − 1)). Moreover, this reflects a natural organizational phenomenon: the salary difference between higher positions in an organization are more accentuated than between lower positions.

A second observation of Eq (3) is the following: if inequality (3) is satisfied for greater values of the management cost parameter c, then it is possible to find a greater number of agents with level ℓ. This has an intuitive explanation: more agents can be found on level ℓ because this implies that any agent in this level has more collaborators with whom to reduce the management cost that otherwise would be larger due to the larger value of c.

We now interpret Eq (4). First observe that, since the equilibrium networks are sequential hierarchies, all connected components are complete subgraphs. Therefore, whenever there is a critical edge, it must belong to a connected component of two nodes. Then, Eq (4) says that agent i with level ℓ and who has a critical edge, has no incentive in severing its collaborative tie with j if, for example, her subordinate-collaborative reward γ is large enough to counteract the increase on the hierarchical reward that she would get by severing her critical edge. This condition seems restrictive because it implies an upper bound on Δ(H), however, we remark that this condition is only present whenever a critical edge exists, i.e., whenever a group of only two people exist for some level. Indeed, later in this paper, in Theorem 4.4, a result is presented where agents dynamically play the hierarchical network formation game and a network with no critical edges is formed.

Theorem 3.3 (Consensual agents). Consider consensual agents. A network G is an equilibrium network for some game if and only if G is composed by one or multiple hierarchical structures and G satisfy the following conditions:

1. (i). If a hierarchical structure in G has k > 1 levels, then (5) for any i, j ∈ N belonging to this hierarchical structure and such that ji ∈ G, with ℓ_i(C(G)) = ℓ ≤ k − 1 and ℓ_j(C(G)) = ℓ − m for any appropriate 0 ≤ m ≤ ℓ.
2. (ii). If there exists i, j ∈ N such that ℓ_i(C(G)) = ℓ ≥ 0 and ℓ_j(C(G)) = ℓ − m ≥ 0 for any appropriate 0 ≤ m ≤ ℓ, and such that there exists no path from i to j (whether i and j belong to the same hierarchical structure or not), then (6)
3. (iii). If there exists a critical edge ij, ji ∈ G such that ℓ_i(C(G)) = ℓ ≥ 0 and ℓ_i(C(G − ij)) = ℓ + 1, then (7)

Moreover, there exist hierarchical structures, which are possibly sequential hierarchies, that are not equilibrium networks for any hierarchical network formation game.

Remark 3.4 (Interpretation of Theorem 3.3). From Theorem 3.3 we conclude that consensual agents allow the formation of a wider variety of hierarchical structures than non-consensual agents. Indeed, given a game with non-consensual agents, any of its equilibrium networks is also an equilibrium network for the same game but with consensual agents. We also remark that consensual agents allow equilibrium networks that are not connected. See Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Condensation graphs of equilibrium networks.

We depict the condensation graph of a network G containing three isolated hierarchical structures that are possible to result as a solution to a hierarchical network formation game with consensual agents (provided the adequate utility parameters exist). Notice that each isolated network has the topology of another possible equilibrium network for consensual agents, but not for non-consensual ones.

https://doi.org/10.1371/journal.pone.0255990.g004

Therefore, we note that Eqs (5) and (7) from Theorem 3.3 are the more general versions of Eqs (3) and (4) from Theorem 3.1, respectively, and, despite having a more intricate expression, inherit a similar qualitative interpretation from their counterparts (as explained in Remark 3.2).

Eq (6) can only be present in the case of consensual agents (i.e., it has no counterpart for non-consensual agents) for two reasons. The first reason is that it must be satisfied, for example, when the equilibrium network consists of two or more hierarchical structures, or if the equilibrium is a hierarchical structure whose condensation graph has multiple sources. The second reason is that this condition is a result of the non-consensual interaction: considering the statement (ii) of the theorem, if agent i wants to establish a connection with j (observe that i has a higher level than j), j will not consent i’s intention of becoming her subordinate if the increase of management cost is larger than the increase of hierarchical reward that j would obtain from having i as a subordinate.

It is easy to construct games with consensual agents in which one or multiple sequential hierarchies (including complete networks) are equilibrium networks. The next proposition gives other non-trivial examples of equilibrium networks for a game with consensual agents.

Proposition 3.5 (Examples of equilibrium networks for consensual agents). Consider any hierarchical structure G with more than one level, one complete connected component per any level whose size is greater than two, and such that the only existing edges in the graph are as follows: for any i, j ∈ N with ℓ_i(C(G)) = ℓ and ℓ_j(C(G)) = ℓ + 1, pk ∈ G for any p ∈ Γ(i, G), k ∈ Γ(j, G). Then,

1. (i). G, or
2. (ii). any network composed by two or more (isolated) hierarchical structures with the same properties as G and the additional property that |P_i(G)| ≥ |P_j(G)| for any i, j ∈ N with ℓ_j(C(G)) = ℓ_i(C(G)) − 1 > 0.

is an equilibrium network for some game with consensual agents.

Remark 3.6 (Existence of hierarchical structures that are not solutions of the game). As mentioned by both Theorems 3.1 and 3.3, there exist hierarchical structures that cannot be equilibrium networks of any hierarchical network formation game, i.e., there exist hierarchical structures which are distinct from the equilibrium networks of all possible tuples that define a game with either consensual or non-consensual agents.

4 Dynamic analysis

We now explore the emergence of hierarchical structures when agents play a hierarchical network formation game dynamically, which we call the network formation process.

Definition 4.1 (Network formation process). Consider a hierarchical network formation game and any fixed graph G_o. At each time step t = 0, 1, …:

1. (i). Consider the action profile at time t as a(t) = (a₁(t), …, a_n(t)) and the respective emergent network as G(t). If t = 0, then consider G(0) = G_o.
2. (ii). Select a pair of agents (i, j), i ≠ j, randomly and independently from N × N with all possible pairs of different agents only having positive probabilities of being selected according to some time-invariant distribution.
3. (iii). Then:
   1. (a). For non-consensual agents: (8)
   2. (b). For consensual agents: let first i consider the action as if she was a non-consensual agent. If ij ∈ G(t), then . Now, if ij ∉ G(t) and , then since i is a consensual agent, we have that a_ij(t + 1) = 1 if and only if u_j(G(t) + ij)≥u_j(G(t)) (otherwise, obviously, a_ij(t + 1) = 0).

Remark 4.1 (Agents in the network formation process). The network formation process, as clearly seen in Definition 4.1, implements a better-response dynamics for each agent, in which each utility-maximizing agent plays the game sequentially and myopically, i.e., they maximize their current utility (given the constraint of their actions) without any foresightedness or being forward-looking. We remark that the network formation process implements a better-response dynamics because an agent i can only consider the addition or deletion of a single edge per time step. If, on the other hand, i is able to select the best pure strategy in her action space per time step, i.e., being able to alter multiple edges at the same time, we would say the network formation process implements a best-response dynamics.

Note, however, that when the agent’s utility is the same irrespective of what action she can take in the next time step, she will prefer changing her action in the next time step (since the action space has only two states, this simply means switching her current action in the next time step). Myopic agents are a natural assumption when networks are big or complex, since in this case it is hard for an agent to trace all the past actions from every other member of the network or predict what the repercussion of her actions will be over the rest of agents. Likewise, myopic agents have the least cognitive and informational requirements about the history of the played game: they only need to remember what happened in the previous time-step. These assumptions have been widely used in the literature, e.g., [24]. We also remark that pairwise selection of agents in dynamic settings of network formation games (as in step (ii) from Definition 4.1) has been widely used in the literature of network formation games, e.g., [25].

Formally speaking, the network formation process induces a homogeneous Markov chain whose state space is the set of all possible graphs with N nodes (defined by all possible action profiles the agents can take), and whose transition probabilities are immediately defined as an induced measure from the underlying pairwise selection process defined in step (ii) from Definition 4.1. See Fig 5 for an example of a one-step transition in the network formation process.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Example of a one-step transition in the network formation process.

At some time t ≥ 0, we have an emergent network G(t) with its condensation graph G_C(t). Assume the pair of agents (i, j) is chosen at time t by the network formation process, with i such that Γ(i, G(t)) corresponds to node A and j such that Γ(j, G(t)) corresponds to node B in G_C(t). Then G_C(t + 1), which is the condensation graph of G(t + 1), is as shown in the figure for non-consensual agents, and for consensual agents if and only if H(2) − H(0)≥c.

https://doi.org/10.1371/journal.pone.0255990.g005

We proceed to analyze the convergence properties of the network formation process. All relevant proofs can be found in Section 8.

Definition 4.2 (Invariant networks). Consider the network formation process. An invariant network G* is any network such that if G(t) = G* then, with probability one, G(t + 1) = G*.

Proposition 4.2. A network is invariant for the network formation process if and only if it is an equilibrium network.

Proof. Consider any invariant network G* and any (i, j)∈N × N (with i ≠ j). Consider first non-consensual agents. If ij ∈ G* then it must follow that u_i(G*)>u_i(G* − ij); otherwise, if u_i(G)≤u_i(G* − ij), then item (iii) from Definition 4.1 implies that i will change her action in the next time step and thus the topology of G*, which gives a contradiction. A similar proof can be done to show that if ij ∉ G*, then u_i(G*)>u_i(G* + ij); and by Definition 2.3, conclude that G* is an equilibrium network. Now, for consensual agents, it only suffices to analyze what happens when ij ∉ G* and u_i(G*)≤u_i(G* + ij), i.e., the case in which, agent i’s action in the next time step would be 1 if she was non-consensual. In such case, according to item (iii) from Definition 4.1, it must follow that u_j(G* + ij)<u_j(G*), so that i has no consent from j and will not change her current action 0 in the next time step. This implies, according to Definition 2.3, that G* is an equilibrium network.

Given an equilibrium network, it is easy to show it is also an invariant network from Definition 2.3 and the fact that only a pair of agents is selected per time step in the network formation process.

Formally speaking, Proposition 4.2 tells us that a network G is an absorbing state of the Markov chain if and only if it is an equilibrium network.

Problem 4.3 (Important problems). The first and most important problem to solve is whether (i) the network formation process, which results from the agents playing (myopic) better-response dynamics starting from a fixed arbitrary initial graph, can lead convergence to an equilibrium network, or (ii) if the individuals may never settle down in forming an invariant network so that their formed networks are continually oscillating between different configurations. Once convergence to an equilibrium network is proved, a second natural problem is the further exploration of the specific structures or topology of the formed equilibrium networks.

Theorem 4.4 (Convergence of the network formation process). Consider the network formation process, with an arbitrary initial network G_o, for a game with a fixed type of agents and such that H(ℓ + 1) − H(ℓ)>γ + cN for any integer 0 ≤ ℓ ≤ N − 1. With probability one, G(t) one, converges in finite time to an equilibrium network with no critical edges and such that it is connected (according to the conditions stipulated in Theorem 3.1 or Theorem 3.3 depending on the type of agents).

The following corollary follows from the proof construction of Theorem 4.4.

Corollary 4.5 (Convergence starting from an empty graph). Consider the network formation process with an empty initial network G_o = ∅, for a game with a fixed type of agents and such that H(ℓ + 1) − H(ℓ)>γ + cN for any integer 0 ≤ ℓ ≤ N − 1. With probability one, G(t) converges in finite time to an equilibrium network with only single edges and such that it is connected (according to the conditions stipulated in Theorem 3.1 or Theorem 3.3 depending on the type of agents).

Remark 4.6 (Analysis of results). There are important remarks to make about the convergence results for consensual agents. In general, the final or converged network from the network formation process (for any type of agents) depends on both the initial network and the specific realization path of the underlying stochastic process, i.e., the final network is a random function of the initial conditions. This is known as path dependence [24]: depending on the initial graph, there may be a positive probability of converging to more than one invariant network, provided this is allowed by the utility parameters and the topology of the initial graph. See Example 2 for an illustration of this phenomenon. Finally, notice how starting the network formation process from an empty graph greatly reduces the final topology of the possible converged equilibrium network (e.g., compare Corollary 4.5 to Theorem 4.4).

We provide a small example to illustrate the observations made in Remark 4.6 regarding path dependence.

Example 2: (Simple examples of network formation processes and path dependence) Consider the network formation process starting from an empty graph with three agents, i.e., N = {1, 2, 3} for some game.

Consider first consensual agents in a game with utility parameters such that H(1) − H(0) = H(2) − H(1)>γ + c, and consider (a, b, c) to be any permutation of elements of N. Then, if the first three selected pairs are {(a, b), (a, c), (c, b)}, then the process converges to the invariant network G₁(a, b, c) = {ab, ac, cb} with probability one. However, if the first two selected pairs are {(a, b), (b, c)}, then the process converges to the invariant network G₂(a, b, c) = {ab, bc} with probability one. We conclude that, when starting from an empty graph, the equilibrium networks G₁ and G₂ (for different values of (a, b, c)) have a positive probability of being the invariant network towards which the network converges to, i.e., there is path dependence.

Now, consider non-consensual agents. If the utility parameters of the game are such that 0 < H(1) − H(0)<γ + c, then the converged network is a complete network with probability one, i.e., no matter what is the realization of the stochastic process, the network always converges to a complete one. Now, assume H(1) − H(0)>γ + c. First consider H(2) − H(1)>γ + c. Then, the converged network has a structure of the form G = {ab, ac, bc}, where any possible permutation (a, b, c) of elements of N has a positive probability of appearing. Similarly, if H(2) − H(1)<γ + c, then the converged network has a structure of the form G = {ab, bc, cb, ac} where any possible permutation (a, b, c) of elements of N has a positive probability of appearing. In conclusion, if H(1) − H(0)<γ + c then there is path dependency; otherwise, if H(1) − H(0)>γ + c then there is no path dependence.

5 Conclusion

In this paper we propose, to the best of our knowledge, the first network formation game model that explains the emergence of hierarchical structures in a society. We consider both non-consensual and consensual agents, and study the different topologies of hierarchies that can emerge as solutions of a game. Moreover, we develop a network formation process in which the agents play stochastically the game sequentially and myopically, and characterize the possible final formed networks. We learn that the introduction of hierarchical rewards that increasingly benefit the agents of higher hierarchy or power, is crucial in determining both the total number of levels in the final hierarchical structure and the maximum number of individuals that can share the same level. Moreover, we learn that it is plausible that individuals within the same level cooperate with each other.

Our paper motivates future work on the analysis of hierarchical structures, like, for example, the proposal of new models that can display either a narrower or a wider variety of other forms of hierarchies than the ones presented in this paper. One way to possibly achieve this, is by the judicious incorporation of heterogeneous agents in the game that may have different structures of subordinate-collaborative or hierarchical reward; see, for example, Remark 2.3. Another possible future work is to define different dynamic processes of network formation that perhaps can drive the agents to different equilibrium selection outcomes and study why such phenomena may happen.

As pointed out in Remark 2.1, a direction of future research is to extend our proposed game to allow agents to severe incoming connections from her subordinates. Moreover, a second research direction is to study the effect of stability of the equilibrium networks of the game, and further study the conditions that could lead more or less efficient equilibrium networks in terms of the game parameters. Finally, from an organizational point of view, it is relevant to theoretically understand under what conditions are flat or steep hierarchies preferable in terms of the game’s parameters.

6 Basic graph theoretical definitions

Consider a directed graph (digraph) G. Graph G is strongly connected whenever there exist a path between any two of its nodes; and, by convention, any graph composed by a single node is in itself strongly connected. We say that i and j are weakly connected when, if we consider all edges of G being undirected, there exists a path from one of these nodes to the other, and we say that G is (weakly) connected if any two nodes are (weakly) connected. If G is not connected, then any connected subgraph such that no other node of G is connected to any of its nodes, is called an isolated network.

A (strongly) connected component of G is a subgraph of G that is strongly connected and such that no additional edges or nodes from G can be included in the subgraph without violating its property of being strongly connected. We can partition the nodes of G according to which connected component they belong to. The condensation graph of G is the graph resulting from contracting every strongly connected component of G into a single node, and it is an acyclic graph.

7 Proofs for the static analysis section

We first introduce a couple of useful technical results.

Lemma 7.1. For both non-consensual and consensual agents, an equilibrium network cannot have directed cycles, and thus, only has undirected connected components. In particular, for the case of non-consensual agents, all connected components are complete subgraphs.

Proof. By contradiction, consider an equilibrium network G and a single edge ij ∈ G such that a directed cycle involving the node i exists with the edge ij belonging to such cycle. We analyze whether j has an incentive to start a collaboration with i. Observe that H(ℓ_j(G)) = H(ℓ_j(G + ji)), |P_j(G, k)| ≤ |P_j(G + ji, k)| for any with k ≠ i. Then,

Thus, j has an incentive to establish the edge ji. For non-consensual agents, this would contradict the fact that G is an equilibrium network, and so G cannot have single edges in any connected component, which implies that all connected components are undirected subgraphs. Now, assume that agents are consensual and continue the analysis to see if i has an incentive in accepting a link from j. We can easily show that since |P_i(G + ij, k)| ≥ |P_i(G, k)| for any . Thus, i will accept collaborating with j, and this immediately implies that G must only have undirected connected components for consensual agents.

Now, assume we have any i, j ∈ N with ij, ji ∉ G belonging to the same undirected connected component of G and that this connected component is not a complete subgraph. Then, it is easy to show that u_i(G + ij) − u_i(G) = γ > 0, and so i has an incentive to form the connection ij. For non-consensual agents, this contradicts G being an equilibrium network, from which we immediately conclude that all connected components must be complete subgraphs. Now, consider consensual agents. Then, it is easy to show that , and so j has no incentive in accepting a single edge from i. Thus, the connected components of G are not necessarily complete for consensual agents.

Corollary 7.1. Consider any graph G without directed cycles. Then, no agent has an incentive to sever any single edge.

Proof. Given the graph G, consider any i, j ∈ N such that ij ∈ G and ji ∉ G; and observe that u_i(G − ij) − u_i(G) = −γ < 0.

Now, we introduce the important proofs of our Solution Analysis section.

Theorem 3.1. Throughout the proof, let . We denote by i → j the fact that there exists a path between i and j, and, with a slight abuse of notation, we will also use it to denote the set of edges that compose some path between i and j, so that clearly i → j ⊂ G. (Note that there might be multiple paths between i and j and so there is some ambiguity as to which specific path i → j refers to, but this will not be a problem throughout the paper).

Part I: We first prove that any given sequential hierarchy as stipulated in items (i) and (ii) of the theorem statement, is the equilibrium network for some game . Consider a given sequential hierarchy G. Now, we choose any c, γ > 0 and fix their values. Then, the remaining problem is to find a reward function H so that G is an equilibrium network for the game . Then, for G to be an equilibrium network, we need to ensure that no agent is able to establish new links or severe existing ones as pointed out in Definition 2.3. Let k be the number of levels of G.

If k = 1, then the network is complete with u_i(G − ij) = γ(n − 2)+ H(0) − c/(n − 1) and u_i(G) = γ(n − 1)+ H(0) for any i, j ∈ N; which gives u_i(G − ij) − u_i(G) = −γ − c/(n − 1)<0 since n ≥ 3. Thus, there is no incentive for any agent of the network to severe a link, irrespective of the function H. Thus, any non-negative monotonic function H can be chosen as the reward function.

Let us assume now that we have k > 1.

Case 1: Assume i has level 0 < ℓ ≤ k − 1 and that she considers establishing a link with some agent j that has level ℓ − 1 (recall that ij ∈ G), which we call an immediate backward connection. Then, and , and thus

Having this previous expression being less than zero would suffice for a non-consensual agent to not have an incentive to establish immediate backward links, which is the same as choosing a function H that satisfies condition (i) of the theorem statement (observe that S must be, indeed, strictly increasing since the sequence is also strictly increasing for k > 2).

Case 2: Assume i has level 0 < ℓ ≤ k − 1 and that she considers establishing a connection with an agent j that has level ℓ − m for m > 1 (assuming, obviously, that such j exists). Let q_ℓ(G) = |{j|ℓ_j(C(G)) = ℓ}| be the number of agents that have rank ℓ in the graph G. Then, and ; and thus

Now, if (3) holds, then for any 0 ≤ p ≤ ℓ − 1; from which it follows that and so i has no incentive to establish such new link. In conclusion, if condition (i) from the theorem statement holds, then no agent has in incentive to establish a link with any agent that has a lower level.

Case 3: Assume i has any level 0 ≤ ℓ ≤ k − 1 and that she considers severing any of her existing edges. From Corollary 7.1, we know that she has no incentive to sever any single edge. Now, consider any agent j such that ij ∈ G belongs to some undirected edge. If the undirected edge is a critical edge, i.e., |P_i(G)| = 2, then it is easy to show that . Having this previous expression being less than zero would suffice for i to not have an incentive to severe the edge ij, which is the same as choosing a function H that satisfies condition (ii) from the theorem statement. On the other hand, if the undirected edge is not a critical edge, so that |P_i(G)| > 2, then it is easy to show that ; and thus, i has no incentive in severing the edge ij.

In conclusion, from all the previous analyzed cases, if G is a sequential hierarchy with more than one level, then we only need to define the non-negative monotonic function H as stipulated by the conditions (i) and (ii) of the theorem statement in order to finally obtain the sought game such that G is an equilibrium network for .

Part II: Now, we need to prove that any equilibrium network is a sequential hierarchy as stipulated in items (i) and (ii) of the theorem statement. Consider the given equilibrium network G. From Lemma 7.1 we immediately conclude that G is a hierarchical structure with complete connected components. Note that this lemma does not specify that G must be connected. However, from the proof of the same lemma, if we assume G is not connected, then there is always an incentive for some agent to form a single edge with an agent from another complete connected component of the same level, which by contradiction implies that there is only one connected component per any level in G. Now, we claim that any agent in G must have single edges towards any other agent with a higher rank than her. Assume this is not the case and take any i, j ∈ N such that ij ∉ G and ℓ_i(C(G)) < ℓ_j(C(G)), then it follows easily that i has an incentive to form links with j since such action would increase her utility by γ > 0, giving a contradiction. Thus, in observance of Definition 3.1, we have proved that G is a sequential hierarchy. It should be noted that, from our previous analysis in Part I, the number of agents in each complete connected component of G (when there is more than one level) is determined by the utility parameters (γ, H, c) satisfying the conditions (i) and (ii) of the theorem statement. This concludes the proof that any equilibrium network is a sequential hierarchy as stipulated in items (i) and (ii) of the theorem statement.

Finally, assume we have a sequential hierarchy G such that there exists some level ℓ > 1 such that: 1) for any i ∈ N that has level ℓ, we have ; and 2) for any p ∈ N that has level ℓ − 1, we have |P_p(G)| = 2. Then, we claim that G cannot be an equilibrium network for any game. To see this, observe condition 1) implies , which immediately implies both (3) and (4) cannot hold simultaneously because of condition 2), for any utility parameters (γ, H, c).

Proof of Theorem 3.3. Throughout the proof, let . We adopt the same notation introduced at the beginning of the proof of Theorem 3.1, and we additionally introduce the following notation: given nodes i, j ∈ N, let denote the fact that there does not exist any path between i and j in G, and let i − j denote that i and j are (weakly) connected, i.e., i → j or j → i.

Part I: We first analyze the conditions under which any given hierarchical structure, as in items (i), (ii) and (iii) of the theorem statement, is the equilibrium network for some game . Consider a given network G that has one or more hierarchical structures. The remaining problem is to find the utility parameters (γ, H, c) so that G is an equilibrium network for the game . Now, we analyze the conditions that ensure no agent has an incentive and/or is able to form a new link or severe any existing link, so that G can be properly defined as an equilibrium network of .

Case 1: Assume any agent i that has level 0 < ℓ ≤ k − 1, with k > 1 being the number of levels in the hierarchical structure for which i belongs to, is considering establishing a link with any agent j that has rank ℓ − m for any appropriate m ≥ 1, i.e., establish a backward connection. First, assume ji ∈ G. Then, following a similar procedure as for Case 1 of Part I in the proof of Theorem 3.1, we obtain with |P_i(G + ij, p)| ≥ |P_i(G, p)| for any , so that i does not have an incentive to establish a link with j if the inequality (5) from condition (i) of the theorem statement is satisfied. Assume i has an incentive to establish this link with j, then since agents are consensual, we need to analyze if j would consent to such action from i. Then, it is easy to show that since |P_i(G + ij, p)| ≥ |P_i(G, p)| for any , and thus j would accept such proposition. Then, the sufficient and necessary condition for not incentivizing a backward connection from i is the satisfaction of condition (i) from the theorem statement. Second, assume ji ∉ G and j → i. Then, it is easy to see that u_i(G + ij) − u_i(G) = γ + H(ℓ − m) − H(ℓ). Assume the worst case, i.e., i has an incentive in making such connection. Then, we find that u_j(G + ij) − u_j(G) = −c < 0, and so j will not accept a new link from i. Third, assume ji ∉ G and (notice that i and j may or may not belong to the same hierarchical structure). Then, it is easy to show that i has an incentive in establishing such link, and that for j we have u_j(G + ij) − u_j(G) = H(ℓ + 1) − H(ℓ − m) − c, and so j will not accept such connection if and only if (9)

Case 2: Assume any agent i that has level 0 ≤ ℓ < k − 1, with k > 1 being the number of levels in the hierarchical structure for which i belongs to, considers making a forward connection with any agent j that has rank ℓ + m for any appropriate m ≥ 1. We already know from Part I in the proof of Theorem 3.1 that i has an incentive to establish a link with j. If m = 1 and i → j, then it follows that . If m = 1 and , or m > 1 (whether i − j or not), then u_j(G + ij) − u_j(G) = −c < 0. Then, we conclude that j has no incentive in accepting a connection from i.

Case 3: Assume any agent i that has level 0 ≤ ℓ ≤ k − 1, with k ≥ 1 being the number of levels in the hierarchical structure for which i belongs to, considers making a connection with any agent j of rank ℓ (so that ij, ji ∉ G). Then, we already know i has an incentive for making such connection. If j ∈ Γ(i, G), then we have and thus j will not accept such link from i. If j ∉ Γ(i, G) (whether i belongs or not to the same hierarchical structure as j), then we obtain u_j(G + ij) − u_j(G) = H(ℓ + 1) − H(ℓ) − c and so j has no incentive in accepting such link from i if and only if (10)

Case 4: Assume any agent i that has any rank ℓ considers severing any of her existing edges. From Corollary 7.1, we know that she has no incentive to sever any single edge. Now, consider any agent j such that ij ∈ G belongs to some undirected edge. If the undirected edge is a critical edge and i can increase its level by severing the edge ij, then it is easy to show that . Then, the condition would suffice for i to not have an incentive to sever the edge ij, which is expressed in Eq (7) from condition (iii) of the theorem statement. On the other hand, if the undirected edge is not a critical edge, then |Γ(i, G)| > 2 and it is easy to show that since |P_i(G − ij, p)| ≤ |P_i(G, p)| for any ; and thus, i has no incentive in severing the edge ij.

Notice that enforcing inequalities (9) and (10) amounts to the condition (ii) of the theorem statement. Therefore, if there exist utility parameters (γ, H, c) that satisfy the conditions (i), (ii) and (iii) from the theorem statement, we conclude that the given sequential hierarchy is an equilibrium network.

Part II: Now, consider a given equilibrium network G. From Lemma 7.1 we immediately conclude that G is a hierarchical structure or is composed of isolated hierarchical structures (since this lemma does not specify whether G is connected or not). Then, from our previous analysis in Part I, it follows that the utility parameters (γ, H, c) must be such that the conditions (i), (ii) and (iii) in the theorem statement are satisfied. This concludes the proof that any equilibrium network is composed by one or multiple hierarchical structures.

Finally, we prove the existence of hierarchical structures that are not equilibrium networks of any possible hierarchical network formation game. For the case in which the hierarchical structures are sequential hierarchies, we can use the same construction given in the proof of Theorem 3.1. Now, we focus on the case in which the hierarchical structures are not sequential hierarchies. Consider a hierarchical structure G with more than one component per any level and with complete connected components, and such that the only single edges in the graph are as follows: for any i, j ∈ N with ℓ_j(C(G)) = ℓ_j(C(G)) − 1 ≥ 0, it holds pk ∈ G (and so kp ∉ G) for any p ∈ Γ(i, G), k ∈ Γ(j, G). Moreover, also assume |P_i(G)| > 2 for any i ∈ N, i.e., there are no critical edges in G. We immediately notice from the topology of G and our discussion above that the expression in Eq (5) from condition (i) of the theorem statement becomes (11) for any i, j ∈ N with ℓ_i(C(G)) = ℓ and ℓ_j(C(G)) = ℓ − 1, ℓ > 0. Assume G has more than one hierarchical structure, and that one of them has ℓ + 1 > 0 levels with i_p ∈ N denoting any agent of such hierarchical structure such that ℓ_i(C(G)) = p ∈ [0, ℓ − 1]. Consider any level 0 < ℓ ≤ k − 1. Now, observe that condition (i) (using (11)) and condition (ii) imply (12) (13) respectively. We claim that if there exists some level ℓ such that , then there are no utility parameters that can define a game such that G is its equilibrium network. To see this, observe that this condition implies for any γ, c > 0, which makes it impossible to choose a reward function H such that conditions (12) and (13) are satisfied, which in turn violates both conditions (i) and (ii) from the theorem.

Proof of Proposition 3.5. We analyze the conditions under which any given network as in (i) or (ii) is the equilibrium network for some game with consensual agents. Given the network, the remaining problem is to find the appropriate utility parameters (γ, H, c) so that it becomes the equilibrium network for the game .

Consider G to be a given hierarchical structure as in statement (i) with k > 1 levels. First, observe that since all connected components are complete subgraphs with more than two nodes, then there are no critical edges in G, so that we do not need to analyze the satisfaction of condition (iii) of Theorem 3.3. We immediately notice from the structure of G and the proof of Theorem 3.3 that Eq (6) of condition (ii) from Theorem 3.3 becomes (14) with ℓ > 0 and any i, j ∈ N with ℓ_i(C(G)) = ℓ > 0 and ℓ_j(C(G)) = ℓ − 1. Then, it follows immediately that we can first choose any γ, c > 0 and then an appropriate non-negative monotonic function H satisfying Eq (14) to form the sought game.

Now, consider G to be a given network as in statement (ii). Observe that all isolated networks in G have more than one level. First, we make the observation that, considering any agent i, j ∈ N with ℓ_i(C(G)) = ℓ > 0 and ℓ_j(C(G)) = ℓ − 1, then, from the conditions of G, the following inequalities hold: (15) since |P_i(G)| > 2.

Using the inequality in (14), we conclude that if there exists a tuple of utility parameters (γ, H, c) such that they satisfy the condition (16) then this will imply the satisfaction of conditions (i) and (ii) from Theorem 3.3 and thus create a game such that G is its equilibrium network. We claim that such tuple exist. To see this, notice that (15) implies cU(G)<c, and so we can choose a sufficiently small γ > 0 so that γ + cU(G)<c, and therefore we can squeeze in the difference H(ℓ) − H(ℓ − 1)>0 (which immediately defines the reward function H appropriately) so that (16) holds.

8 Proofs for the dynamic analysis section

Proof of Theorem 4.4. For any type of agents, let be the set of equilibrium networks. Consider a set of graphs (with, obviously, different elements). We say is an invariant set in our network formation process if the following holds: for any elements , if G(t)∈G_i then G(t′) = G_j for infinitely many t′ ≥ t with probability one. Notice that this implies that, if and , then there is zero probability that the network formation process starting at G_i can eventually be the network G_k. In particular, an invariant set has only one element if and only if this element is an invariant network (see Definition 4.2). Finally, we remark that in the literature of Markov chains, this concept of invariant set defined above is equivalent to the concept of an irreducible closed set of persistent states [39].

Now, according to the Decomposition Theorem of Markov Chains, the state space of the Markov chain can be partitioned uniquely as the union of a set of transient states (if such set exists) and one or more invariant sets. Now, from the fact that G(t) can only take a finite number of different topologies, or, equivalently, from the fact that the Markov chain is finite, it is known that, with probability one, the network G(t) enters an invariant set in finite time (indeed, each graph in this invariant set is a non-null persistent state [39]). Obviously, if the network formation process starts in an equilibrium network, we know from Proposition 4.2 that this finite time is actually zero. In conclusion, from Proposition 4.2, if any invariant set has only one element, then we conclude that G(t) converges to an equilibrium network with probability one.

For the rest of the proof, pick any type of agents for the game. Now, given a graph G, define the set as follows: .

Now, we make a couple of observations:

1. (a) Consider a single edge ij ∈ G(t) that belongs to a directed cycle at any time t, and that i has level ℓ. Then, it is easy to see that H(ℓ + k) is the hierarchical reward of i for the graph G(t) − ij, for some k ≥ 0. Now, observe that u_i(G(t) − ij) − u_i(G(t)) = −γ + H(ℓ + k) − H(ℓ). If k = 0, then G(t + 1) = G(t); otherwise, if k > 0, then, from the conditions on the increments of the reward function in the theorem statement, we get that u_i(G(t) − ij)>u_i(G(t)) and so G(t + 1) = G(t) − ij.
2. (b) From the proof of Lemma 7.1, we conclude that: if ij belongs to a directed cycle in G(t), then, with positive probability, ij, ji ∈ G(t + 1).

Let G(t_o) be the graph at any time t_o ≥ 0. Then, with these two observations, we can construct a finite sequence of pairs of agents such that they modify the network according to the following steps (a step can be skipped if not applicable):

1. eventually makes the graph connected at some time t₁ ≥ t_o, i.e., G(t₁) has no isolated networks (this is easy to see for non-consensual agents, and for consensual agents, it follows from the fact that inequality (6) in item (ii) of Theorem 3.3 cannot be satisfied under the conditions on the increments of the reward function H),
2. eventually makes the graph to have no directed cycles at some time t₂ ≥ t₁,
3. taking G(t₂), it starts establishing one or multiple single edges that may result in the formation of new directed cycles (if any), and then immediately turn these edges into undirected ones according to (b) (for example, for non-consensual agents, new directed edges may appear when an agent i forms a single edge with another agent j of the same level and such that there is a path from j to i); and repeat this process until the graph has no directed cycles and cannot form new directed cycles at some time t₃ ≥ t₂; notice that G(t₃) is a hierarchical structure,
4. severs any existing critical edge in G(t₃) (i.e., turns it into a single edge), and continues doing this process until no new critical edge is found in the graph at some time t₄ ≥ t₃ (this can always be done since the condition on the increments of the reward function H implies that the inequality (7) is always not satisfied, which, from the proof of Theorem 3.3, let us conclude that any existing critical edge can always be severed).

Notice that the graph G(t₄) is still a hierarchical structure, since severing critical edges (as in step 4) does not introduce directed cycles, nor introduce new isolated networks. Since G(t₄) is a hierarchical structure and the increments of the reward function H are large enough, there is no incentive for any agent to establish an edge with any agent with a lower level. Notice that for non-consensual agents, step 3) implies all agents with the same level form a complete subgraph. Notice also that for consensual agents, if a single edge is established between two agents i and j with the same level in G(t₄), then, for any t ≥ t₄, this single edge will not be an undirected edge with probability one, nor no new directed cycle that contains i and j will be formed with probability one. Therefore, we conclude that, for any type of agents, no new undirected edges nor new directed cycles can be formed for any G(t)≥G(t₄) with probability one.

So, given the arbitrary initial network G_o, let us define T to be the random time at which G(t) becomes a hierarchical structure as characterized by G(t₄). Then, it is easy to observe that T is a stopping time and we can use the first Borel-Cantelli lemma (as in [40, Theorem 3.1]) to prove that T < ∞ with probability one, since any finite sequence of pairs of agents that implements the appropriate changes in the steps 1) to 4) (if any) always exists with positive probability at any time t. Then, we can use the strong Markov property, and assume that the network formation process starts at T with the graph G(T). Since only new single edges may appear at future time-steps and these single edges cannot form directed cycles, we conclude that, with probability one, for any t ≥ T, i.e., . Moreover, for any t ≥ T, we conclude that G(t) has a pair of agents that can establish a single edge at time t + 1 if and only if with positive probability. Now, consider any invariant set . It follow from these observations that if , then with probability one. However, this only occurs if and only if no new single edges can be added, i.e., when . Then, we conclude that any invariant set is an invariant network and so that the network converges to an equilibrium network.