Moderate Intra-Group Bias Maximizes Cooperation on Interdependent Populations

Evolutionary game theory on spatial structures has received increasing attention during the past decades. However, the majority of these achievements focuses on single and static population structures, which is not fully consistent with the fact that real structures are composed of many interactive groups. These groups are interdependent on each other and present dynamical features, in which individuals mimic the strategy of neighbors and switch their partnerships continually. It is however unclear how the dynamical and interdependent interactions among groups affect the evolution of collective behaviors. In this work, we employ the prisoner's dilemma game to investigate how the dynamics of structure influences cooperation on interdependent populations, where populations are represented by group structures. It is found that the more robust the links between cooperators (or the more fragile the links between cooperators and defectors), the more prevalent of cooperation. Furthermore, theoretical analysis shows that the intra-group bias can favor cooperation, which is only possible when individuals are likely to attach neighbors within the same group. Yet, interestingly, cooperation can be even inhibited for large intra-group bias, allowing the moderate intra-group bias maximizes the cooperation level.


Introduction
Cooperation is a widely observed phenomenon in social science, biology and economics [1,2]. However, cooperative behavior apparently contradicts the natural selection [3]: Selfish players always have a higher average fitness than that of cooperators, since selfish players enjoy the benefits from the cooperation of others without associated costs. Therefore, it has fascinated many interests from natural and social scientists to understand the emergence and the stability of cooperation.
Within the interdisciplinary field of evolutionary game theory, this puzzle benefits from techniques of biology, economy, computer sciences, and physics [4,5]. As a metaphor, the prisoner's dilemma (PD) game has attracted great attention in both theoretical and experimental studies to investigate the evolution of cooperation [6][7][8][9][10][11][12]. In a typical PD game, two players simultaneously decide whether they act as a cooperator (C) or a defector (D). Cs are willing to engage in cooperative tasks, while Ds prefer not to. They will receive the reward, R, if both cooperate, and the punishment, P, if both defect. However, if one player defects while the other decides to cooperate, the former will get the temptation, T, while the latter will get the sucker's payoff, S. Namely, the local interaction between C and D is given by the following payoff matrix: where these payoffs satisfy TwRwPwS and 2RwTzP. It is obvious that players prefer to defect if they wish to maximize their own payoffs, whatever the opponent's decision.
Various mechanisms to support the evolution of cooperation have been identified recently [13][14][15][16][17][18], such as direct reciprocity, indirect reciprocity, group selection and network reciprocity [19]. The most famous context is spatial games introduced by Nowak and May [20], where players are arranged on a spatially structured population and interact with their neighbors only. It is shown that cooperators could survive by means of forming compact clusters, which minimize the exploitation of defectors. In the line of this seminal achievement, the role of spatial game and its underlying promoted mechanisms in evolutionary games have been intensively explored, such as the mobility of players [21][22][23], different evolutionary time scales [24][25][26][27], social diversity [28,29], heterogeneous ability and aspiration [30,31] (for comprehensive reviews refer to Ref. [32]).
Though large amounts of work upon spatial reciprocity are available, the main attention remains in an isolated and single structure. In human societies, empirical evidences have shown that the realistic structures are composed of many interactive groups, which interact with each other over time [33][34][35][36][37][38][39][40]. In this context, the evolution behavior traits have been considered underlying the interdependent populations, where populations are represented by group structures to account for different social types. Note that this framework is similar with previous studies that have addressed the structure on interdependent networks, in the sense that the success of one node in a given group not only depends on the nodes in the same group, but also replies on the states of other nodes in other groups. Taking some examples more specifically, in a recent paper [41], where the biased utility function on interdependent networks were implemented, it was shown that the stronger the bias in the utility function, the higher the level of public cooperation. While in [42], a replicator such as evolutionary game dynamics took place on interdependent populations, cooperative behaviors are fixed on the system (even if the system is well-mixed). Moreover, it was also a remarkable hint that only an intermediate density of sufficiently strong interactions between groups could lead the optimal resolution of social dilemmas [43,44].
Aside from the effect of spatial structure and its various promoted mechanisms, the co-evolution of game models also attracts numerous attention [45][46][47][48][49][50][51][52], which not only reflects the evolving of strategies over time, but also characterizes the adaptive development of topologies and/or update rules. In particular, the interdependent populations in our real social life are dynamical and changing over time. Besides, the essence of evolutionary game theory on interdependent populations remains unclear, especially for the question how the structure of dynamical and interdependent populations affects the evolution of cooperation. Therefore, we introduce an intra-group bias based rewiring probability, and focus on co-evolution of strategy and structure to investigate the evolution of cooperation on interdependent populations. Within the fast rewiring process, we derive a simple rule quantitatively revealing how the link breaking probability and intra-group bias are chosen to stabilize cooperation. Interestingly, though cooperation is favored by intra-group bias conditionally, it is precluded for a large intra-group bias, which uncovers that the moderate intra-group bias maximizes the cooperation level.

Model
We consider the co-evolution of strategy and topology structure of the PD game. Each player can be one of the two strategies, either cooperation (C) or defection (D), where Cs incur a cost c and provide a benefit b to its opponent (bwcw0), while Ds neither incur costs nor provide benefits. The local interaction between C and D is given by the payoff matrix M ij , which is a simplified version of Eq. (1): Initially, the whole population is located at interdependent populations, which consists of Group-1 and Group-2 (see Fig. 1). Each Group-i (i~1,2) is represented by a network structure with size N i , leading to the size of total population N~N 1 zN 2 . The average degree of Group-i is L i~2 H i =N i , where H i is the total number of links in Group-i. Assume N i &L i , which implies that each player has a limited number of neighbors compared with the population size of the group. Denote the number of links connecting two nodes via intra-group interactions as H ii (i~1,2), while the number of links connecting two nodes via inter-group interactions as H 12 (or H 21 ). Then, the total number of links is H~H 12 zH 11 zH 22 .
At each time step, the event of updating strategy takes place with probability v, otherwise link adjustment happens with probability 1{v. Here, v governs the dynamical timescales between strategy updating and topology evolution.
For the strategy updating, we adopt the Fermi dynamics [53,54] on interdependent populations. Each player is allowed to play with all its current neighbours, and obtains an accumulated payoff. Player g in the whole population is selected at random, subsequently player h is selected among g's current neighbors. Then, the strategy of focal player g tries to replace that of neighbor h with probability p~1 1ze {b(Pg{Ph) , where P g~P 2 i~1 P i,g represents the total payoff of player g (P i,g is the payoff of player g obtained from Group-i). b denotes the imitation intensity, measuring how strongly the imitation behavior depends on the payoff difference [55]. In this work, we consider the weak selection (i.e., b%1), i.e., one phenotype is slightly advantageous, and the effects of payoff differences are small, such that the evolutionary dynamics are mainly driven by random fluctuations.
For the linking dynamics, each link is assigned a label l[f1,2, Á Á Á ,Hg as its name. Assume players will leave or break interactions when they dissatisfy with the current situations. In fact, the social interactions between players in evolve with time based on aspiration payoff [30,31], reputation [56,57], and other Figure 1. The schematic presentation for the linking adjustment between interdependent populations. Two types of players, C and D, are arranged on the nodes of two interactive groups: Group-1 and Group-2, respectively. If the dashed link is selected in the topological evolution, it will be broken off with probability k CD . If the dashed link is broken, one of the two players (C or D) occupying the two extremes of the broken link is selected randomly. Subsequently, the selected player (marked by red circle) switches to another player who is not its current neighbor: it will choose the player in its own group with probability a, and choose the player in the other group with probability 1{a. doi:10.1371/journal.pone.0088412.g001 mechanisms [49,51]. To characterize the dynamics of structure with various kinds of relationship, we introduce probability k XY to estimate whether the XY -type link is broken. At each time of linking dynamics, link l t of type X i Y j is selected from the whole interdependent populations at random (X ,Y [fC,Dg, i,j[f1,2g). With probability 1{k XY , the selected link l t remains unchanged, otherwise, the selected link is broken. If the link is broken, then one node is selected randomly from the two, and it tries to find another partner to connect with. With probability a, the neighbor is only selected within the same group, otherwise, the potential neighbor is chosen from the other group (see Fig. 1). Here, a is the intragroup attaching bias between two interactive groups, which reflects the propensity to rewire neighbors via intra-group interactions.
It is worth noting that k XY is time-invariant and describes an intrinsic quantity of the linking dynamics. It is shown that the duration time of XY link obeys the geometric distribution with parameter k XY [58,59]. Therefore, the inverse of k XY can be taken as the the average interaction rate between X and Y . Besides, the total number of links remains constant during the linking dynamics of the interdependent populations as in [24,47].

Evolutionary dynamics on interdependent populations
Denoted the types of link l t as T(l t )[fC 1 Then, the dynamics of T(l t ) is captured by a Markov chain with transition matrix Q~½Q (XiYj )(ZmWn) , whose entry Q (XiYj )(ZmWn) is the transition probability that link l t of type X i Y j transforms to link l tz1 of type Z m W n . The transition matrix of such a Markov chain is given by Q~(1=H)Pz½(H{1)=HI 10 (See Text S1).
Since the Markov chain is irreducible and aperiodic, there exists a unique stationary distribution W~(W XiYj ) determined by equation WQ~W [58], where d indicates the Kronecker delta, x i,X is the frequency of strategy X in Group-i, and g( is the normalization factor. The normalized stationary distribution W X i Y j represents the fraction of X i Y j links in the whole population. Therefore, the average number of In the case of a fast rewiring process, i.e., w%1, the strategy updating occurs less frequently than linking adjustment [24,25], and the structure of groups is almost in the stationary state with the distribution described by Eq. (3) when the strategy evolution occurs. In this case, the average fitness function of strategies C and D in Group-i is given by where e 1~( 1,0), e 2~( 0,1), G~H : is a modified payoff matrix of Eq. (2), whose payoff entry is rescaled by the inverse of the breaking probability. Note that the first term in the bracket of Eq. (4) represents the payoff obtained via interactions in the same group, while the second term of Eq. (4) represents the payoff obtained via interactions belonging to different groups. Therefore, the payoff of each player in the interdependent populations relies on the neighbors in not only the same group but also the other group. Besides, as shown in Fig. 2, the change of x i,C in Group-i is due to the pairwise comparison between the focal player C in Group-i and player D in Group-i (or Group-j), which yields the transition probabilities is the transition probability caused by the pairwise comparison occurring in the same group, while is the transition probability caused by the pairwise comparison occurring in different groups.
For a large population, the stochastic process can be well approximated by a set of stochastic differential equations referring to Langevin dynamics [60]. To the pairwise comparison process occurring on interdependent populations, the Langevin dynamics yields _ x is the effective terms, and j is the uncorrelated Gaussian noise. Since minfN 1 ,N 2 g&maxfL 1 ,L 2 g, the stochastic term vanishes [61,62], which leads to Figure 2. An illustrative transition probability of interdependent populations. The left: probability that the number of C players in Group-i increases from k to kz1 is T z , which is caused by a player C in Group-i (a) (or Group-j (b)) replacing a player D in Group-i. The right: probability that the number of C players in Group-i decreases from k to k{1 is T { , which is caused by a player C in Group-i replaced by a player D in Group-i (c) (or Group-j (d)). doi:10.1371/journal.pone.0088412.g002 Especially, when N 1~N2 , the strategy evolution degenerates to an extension of the replicator dynamics, which yields where k~b G 16N 1 is a constant factor influencing the timescale a. Here, we assume N 1~N2 for simplicity without loss of generality, and the numerical results for N 1 =N 2 are consistent with this simplification. Note that the unit square 0ƒx 1,C ,x 2,C ƒ1 is the invariant set of 2-D plane. From Eq. (8), we obtain three possible equilibria E 1~( 0,0), E 2~( 1,1), and E 3~( ). For such three equilibria, the Jacobian matrix J of (8) has the form x 2,C (1{x 2,C )z 1 8 , J 22~k ½(1{2x 2,C )((2d 1 z2d 2 zd 3 zd 4 )x 2,C z3 d 5 a )z(2d 1 zd 3 zd 4 )x 2,C (1{x 2,C ){ 1 8 . The corresponding eigenvalues of Jacobian matrix J at E 1 , E 2 and E 3 are listed in Table 1.
, which implies that the eigenvalues of Jacobian matrix J are both negative for E 1 ; d 1 zd 5 w0 and 2k 1 a (d 1 zd 5 )z 1 4 w0, which implies that both the the eigenvalues of Jacobian matrix J are negative for E 2 . Therefore, both E 1 and E 2 are stable. For the interior equilibrium E 3 , when (2a{1)w{ 1 8k , the eigenvalues of Jacobian ), E 3 is an unstable equilibrium.

Results
Let us now consider how the co-evolution of strategy and link dynamics affects cooperation on the interdependent populations.
), the eigenvalues of the Jacobian matrix in Eq. (8) are negative for both E 1 and E 2 , yet positive for E 3 . Thus, both E 1 and E 2 are stable, and E 3 is unstable, which leads to the final state converging to E 1 or E 2 . Namely, Eq. (8) in the whole group is composed of all-C (E 2 ) or all-D (E 1 ) (See Fig. 3). On the other hand, when ava Ã , E 3 becomes a saddle-point. Therefore, strategies C and D are bistable on the interdependent populations. Besides, the equilibrium E 3~( x Ã 1,C ,x Ã 2,C ) determines the attraction basin of cooperation (x Ã 1,C ,1|(x Ã 2,C ,1. If the initial condition, x i,C (0), is more than the critical value of x Ã i,C (i.e., x 1,C (0)wx Ã 1,C and x 2,C (0)wx Ã 2,C ), then system (8) converges to all-C; otherwise, it reaches all-D. The effect of initialization on the frequency of strategy C in both groups is shown in Fig. 4. In other  words, the PD game with link dynamics corresponds to a coordination game in well-mixed populations, where both cooperation and defection are best replies to themselves [63]. Thus, cooperation is stable only when Note that Eq. (9) is necessary for emerging cooperation, namely, the co-evolution of strategy and link dynamics can favor cooperation if the benefit-to-cost ratio b=c exceeds lz1 ). This condition is intuitive: the critical benefitto-cost ratio is a decreasing function of k CD but an increasing function of k CC . Indeed, the evolution of cooperation is promoted if C{D links are more fragile than C{C links, which coincides with the results in [24,64]. Besides, quantity l measures the propensity for cooperators to form clusters, and k CD =k CC~( 1=l)z1 characterizes the fragility ratio between C{D link and C{C link. Decreasing l allows cooperators to spread more effectively [22]. In particular, when l?0, a cooperator is more likely to play with cooperators rather than defectors, and easier to form clusters. In this sense, l illustrates how likely a cooperator is to interact with a cooperator.  . Final fraction of cooperators as a function of the intra-group bias a. We set N 1~1 50, N 2~1 00, L 1~6 , L 2~9 , v~0:01, b~0:1, b~2, c~1, k CC~0 :2, k CD~0 :6 and k DD~0 :8. The analytical value of a Ã~0 :65, which is marked out by red arrow in the plot. Each data point is averaged over 30 independent runs, and in each realization, we set 10 6 time steps to ensure the evolution of dynamics in steady states. It is shown that there exists a moderate intra-group bias leading to maximum cooperation level in the whole populations. doi:10.1371/journal.pone.0088412.g005 More interestingly, there exists a lower bound of the intra-group bias between two groups for emerging cooperation. The intragroup bias between two groups hinders the invasion of defectors on the single group, thereby influences the evolution of cooperation. When the value of ava Ã , the interior equilibrium becomes a saddle point. Thus, small value of a is excluded to the model of coordination game, and the cooperation will never emerges when a is smaller than the critical value a Ã . For instance, with a small value of a, a defector on Group-1 might take advantage from the vicinity of cooperators on Group-2, because the corresponding interactions on Group-2 may supply enough resource to be exploited, which results in the prosperous of defection. On the other hand, big intra-group bias also ignores the inter-group interactions between two interdependent groups. In this way, increasing a narrows the attraction basin of cooperation (x Ã 1,C ,1|(x Ã 2,C ,1, and makes it difficult for cooperation to gain a foothold in the population. Therefore, the intra-group bias favors the cooperation for players are likely to switching to attach neighbors within the same group. However, too large intra-group bias hinders the prosperity of cooperation, allowing the moderate intra-group bias maximizes the cooperation level (See Fig. 5).
Till now, we have shown a simple rule telling how cooperation emerges with linking dynamics. Although, condition (9) guarantees the necessity of emerging cooperation, it's not sufficient to make cooperation advantageous. To make cooperators gain a foothold in the population of coordination game, the initial frequency of cooperators in the whole group should exceeds the unstable interior fixed point, which equals Similarly, x Ã C is a decreasing function of k CD and an increasing function of k CC . Thus, decreasing k CC and increasing k CD enlarges the attraction basin of cooperation (x Ã 1,C ,1|(x Ã 2,C ,1, and makes it easier for cooperation thrives. Fig. 6 shows that the critical value of unstable interior fixed point x Ã C increases with increasing k CC , i.e. a larger k CC leads to the larger x Ã C , which makes the flourishing of cooperation more difficult. Increasing the value of k CC to 0:4, cooperators are never advantageous compared to defectors. Thus, cooperators are never favored by selection. Besides, the critical unstable interior fixed point x Ã C decreases with increase of k CD (see Fig. 7), i.e., a larger k CD leads to the smaller x Ã C , which is beneficial to the flourishing of cooperation. Contrarily, a smaller k CD prevents the flourishing of cooperation. Specially, when decreases k CD to 0:4, cooperators are never advantageous compared to defectors.

Conclusions
To sum up, we have established a microscopic model on the coevolutionary dynamics of cooperation and interdependent populations. Under the assumption of fast structure evolving, we analytically arrived in the macro-dynamics at the population level: an extended replicator equation which incorporates both the interactions of groups and the strategy evolution. Based on this extended equation, it is shown that the less the fragile cooperatorcooperator links (or the more the fragile cooperator-defector links), the easier the emergence of cooperation. This result is consistent with previous findings that assortments of cooperators are likely to invade a defector population and escape from the exploitation of defector mutants [24,25,47], which paves the way for both emergence and stabilization of cooperation.
Interestingly, we have revealed that the dynamical interactions on interdependent populations can greatly affect the evolution of cooperation: cooperation can only emerge when intra-group bias is big enough. This is intuitive in the sense that intra-group bias can lead to cooperation [35,36], which indicates that it might be more likely to establish neighbors within the same group. However, counterintuitive results also arise: it is unlikely that the more possible individuals establishing neighbors within the same group, the higher cooperation level is. In fact, too large intragroup switching bias inhibits cooperation [42,43]. This reminds us with the migration effect: Neither too large nor too small mutation rate benefits cooperation. Thus, it would be beneficial for cooperators to move from time to time in order to hunt/establish a paradise to live, since this accidental moving can help the cooperators to escape from the nasty environment consisting of mainly defectors. Yet large migration rates make the population approximately well-mixed destroying the cooperation clustering, which deters cooperation. The switching rate of attaching neighbors outside the group can be viewed as the migration rate, thus leads to moderate switching rate maximizing cooperation. Our work shed light on how the dynamic of interdependent have an impact on the cooperation. This insight might also be constructive to other collective behaviors such as swarming and coordination and opinion formation. Works along those lines are in progress.

Supporting Information
Text S1 Embedded Markov chain approximation for linking dynamics. (PDF) Figure 7. Final fraction of cooperators as a function of initial fraction of cooperators with different k CD . According to Eq. (8), we obtain that when k CD~0 :6,0:8, the analytical results of x Ã C~0 :625,0:0:417 respectively, which are marked out by arrows in the plot. For all the three line in the plot, we set N 1~1 50, N 2~1 00, L 1~6 , L 2~9 , v~0:01, b~0:1, b~1:8, c~1, a~0:6, k CC~0 :2 and k DD~0 :8. The simulation results show that the initial frequency of cooperators x Ã C decreases with increasing of k CD , i.e., large k CD enlarges the attraction basin of cooperation and promotes the flourishing of cooperation. doi:10.1371/journal.pone.0088412.g007