Evolution of Cooperation in a Heterogeneous Graph: Fixation Probabilities under Weak Selection

It has been shown that natural selection favors cooperation in a homogenous graph if the benefit-to-cost ratio exceeds the degree of the graph. However, most graphs related to interactions in real populations are heterogeneous, in which some individuals have many more neighbors than others. In this paper, we introduce a new state variable to measure the time evolution of cooperation in a heterogeneous graph. Based on the diffusion approximation, we find that the fixation probability of a single cooperator depends crucially on the number of its neighbors. Under weak selection, a cooperator with more neighbors has a larger probability of fixation in the population. We then investigate the average fixation probability of a randomly chosen cooperator. If a cooperator pays a cost for each of its neighbors (the so called fixed cost per game case), natural selection favors cooperation if the benefit-to-cost ratio is larger than the average degree. In contrast, if a cooperator pays a fixed cost and all its neighbors share the benefit (the fixed cost per individual case), cooperation is favored if the benefit-to-cost ratio is larger than the harmonic mean of the degree distribution. Moreover, increasing the graph heterogeneity will reduce the effect of natural selection.


Introduction
Evolution of cooperation is one of the most important theoretical questions in evolutionary biology and social biology [1][2][3][4]. In the framework of evolutionary game theory, cooperation has been studied in a variety of game theoretical models such as the Prisoner's Dilemma [1][2][3]. It is well known that in unstructured populations, natural selection favors defectors over cooperators [2][3]. This means that in a population where all individuals have the same chance to interact with each other, defectors will have a higher average payoff than cooperators (i.e. natural selection will increase the relative abundance of defectors and drive cooperators to extinction). However, in more realistic populations, different individuals interact with different subsets of the whole population. This kind of structure can be described by means of complex networks (or graphs), in which players of an evolutionary game occupy the vertices of a network, and the edges denote the links between individuals in terms of game dynamical interaction.
A great deal of research has been devoted to explain the influence of network structure on the evolution of cooperation. Pioneering work by Nowak and May [5] unveiled that a spatial lattice where each player interacts only with his/her four (or eight) immediate neighbors is a viable mechanism for the emergence of cooperation. The success of their model has attracted considerable attention, subsequent studies including not only theoretical investigations [6][7][8][9][10][11][12][13][14] but also economic experiments [15][16][17]. In these studies, the spatial structure retains homogeneity (or regularity) since every individual is topologically identical to any other individual in the population. However, empirical evidence shows that most networks related to interactions in the real world are heterogeneous in that some individuals have many more neighbors (i.e. have higher degree) than others [18][19][20][21][22][23]. Representative networks include the random network [24][25], Small-World network [26][27] and scale-free network [28][29]. Many recent researches have revealed that graph heterogeneity dramatically enhances cooperation in complex networks [30][31][32][33][34][35][36], in particular for the scale-free network [37][38][39][40][41][42]. Although there are many differences in the models considered in this literature (e.g. different update rules, payoff structure, network clustering, etc.), one of the main reasons behind the increase of cooperation levels in the scale-free network is that hubs are usually occupied by cooperators, which ensures their long term success in the evolutionary process. The same occurs when the network structure is coevolving together with the strategy dynamics [43][44][45][46][47][48]. If neighboring pairs of cooperators are more stable than pairs that include at least one defector, cooperators are more likely to have higher degree, which gives rise to highly cooperative heterogeneous networks.
Recently, Ohtsuki et al. [49] extended the approach of Nowak and May [5] and developed a theoretical model based on the simplified Prisoner's Dilemma (PD) game to investigate the evolution of altruistic cooperation on a graph (or a social network) where a donor pays a cost, c, for the recipient to get a benefit, b. They alleged that natural selection favors the emergence of cooperation if the benefit of the altruistic act, b, divided by the cost, c, exceeds the average degree of the graph, k, i.e. b=cwk.
They considered a population of N individuals consisting of cooperators and defectors (where the population size N is fixed), and assumed that: (i) a cooperator helps all individuals to whom it is connected, i.e. if a cooperator is connected to k other individuals and i of those are cooperators, then its payoff is bi{ck; (ii) a defector does not provide any help, but it can receive the benefit from neighboring cooperators, i.e. if a defector is connected to j cooperators, then its payoff is bj; (iii) the fitness of an individual is given by its baseline fitness plus the payoff, where strong (weak) selection means that the payoff is large (small) compared to the baseline fitness; and (iv) for the evolutionary dynamics, in each time step, a random individual is chosen to die, and the neighbors compete for the empty site proportional to their fitness (this process is called 'death-birth' updating). Ohtsuki [49] is based only on a regular graph where all vertices have the same degree. Numerical simulations in their paper show that the fixation probabilities of cooperators in heterogeneous graphs deviate from the theoretical prediction. Thus, a more challenging question is to derive the relation between the fixation probability and the graph heterogeneity. In general, the heterogeneity of a graph (or a social network) can be described by the mean and variance of its degree distribution [34,50]. The degree distribution of a graph, denoted by p(k), gives the frequency of vertices with degree k for k~1,2, Á Á Á, or, alternatively, p(k) is the probability that a randomly chosen individual has exactly k neighbors.
In this paper, we develop and analyze the connection between p(k) and the fixation probabilities of cooperators when selection is weak that provides a new theoretical explanation for the effect of graph heterogeneity on the evolution of cooperation.

Methods
Consider a connected graph with N vertices and with degree distribution p(k). Similar to Ohtsuki et al. [49], the individual at each vertex is either a cooperator in interactions with all of the neighbors or a defector. We label these N vertices as vertex 1, vertex 2, Á Á Á, vertex N, respectively, and the degree of vertex i is denoted by k i for i~1,2, Á Á Á ,N. For an individual at vertex i, its degree k i can be also expressed as k i~ki,C zk i,D where k i,C is the number of its neighbors with strategy C (cooperation) and, similarly, k i,D the number of its neighbors with strategy D (defection) (see Figure 1). The average degree of the graph, denoted by z, is z~P N i~1 k i =N, or z~P k kp(k). On the other hand, the strategy of the individual at vertex i is denoted by s i with s i [fC,Dg for i~1,2, Á Á Á ,N, i.e. if this individual is a cooperator (or a defector), then s i~C (or s i~D ).
In our model, if there exists an edge between two vertices i and j for 1ƒi,jƒN, then we use (i,j) to denote a directed edge from vertex i to vertex j. For the interaction between two individuals at vertices i and j, we consider the two directed edges (i,j) and (j,i) equivalent. Notice that the total number of the directed edges in the graph is zN. Thus, the proportion of the directed edges starting from the cooperators in the total directed edges, denoted by Q C , can be given by Q C~Ps i~C k i =zN, i.e. Q C denotes a ratio of the total degree of cooperators to the total number of directed edges. Similarly, the proportion of directed edges starting from the defectors, denoted by Q D , is Q D~Ps i~D k i =zN~1{Q C . It is easy to see that if k i~z for all i~1,2, Á Á Á ,N (i.e. all vertices have the same degree), then Q C exactly represents the proportion of cooperators in the population (i.e. the frequency of strategy C in the population). For a general heterogeneous graph, we will use the time evolution of Q C to measure the evolution of cooperation.
We also assume that an individual is randomly chosen to die, and its neighbors compete for the empty site proportional to their fitness, i.e. the 'death-birth' updating process [49,[51][52]. For a randomly chosen defector, let f C denote the fitness of its Cneighbors and f D the fitness of its D-neighbors. These are given by f C~( 1{v)zvh C and f D~( 1{v)zvh D , respectively, where the parameter v measures the intensity of selection with 0ƒvƒ1 [49,[51][52], and h C (h D ) is the expected payoff of its C-neighbors (D-neighbors). Similarly, for a random chosen cooperator, let g C denote the fitness of its C-neighbors and g D the fitness of its Dneighbors, which are g C~( 1{v)zvl C and g D~( 1{v)zvl D , respectively, where l C (l D ) is the expected payoff of its C-neighbors (D-neighbors).
For the expected payoffs of both Cand D-individuals, two cases are considered in our model, called the fixed cost per game and the fixed cost per individual, respectively [33]. For the fixed cost per game, a cooperator pays a cost c for each of its neighbors to get a benefit b [49]; and for the fixed cost per individual, a cooperator with k neighbors pays a cost c=k for each of its neighbors to get a benefit b=k, i.e. a cooperator will provide a combined fixed benefit b to all its neighbors and pay a fixed cost c equally shared by all its neighbors [33]. Notice also that the conditional probability that a neighbor of a Y -individual is a Xindividual is given by q X jY~Ps i~Y k i,X = P si~Y k i for X ,Y [fC,Dg. Thus, for a randomly chosen D-individual with degree k, the expected payoffs of its Cand D-neighbors are h C~( z{1)q CjC b{zc and h D~( z{1)q CjD b, respectively, in the case of fixed cost per game. In the case of fixed cost per individual, these expected payoffs are h C~( z{1)q CjC vb{c and h D~( z{1)q CjD vb, respectively, where vb~Pk k (b=k k)p(k k) is the expected benefit from a C-neighbor (see Supporting Information S1). Similarly, for a randomly chosen C-individual with degree k, the expected payoffs of its Cand D-neighbors are l C~( z{1)q CjC bzb{zc and l D~( z{1)q CjD bzb, respectively, in the case of fixed cost per game, and l C~( z{1)q CjC vbzb=k{c and l D~( z{1)q CjD vbzb=k, respectively, in the case of fixed cost per individual (see Supporting Information S1).
If a defector with degree k is randomly chosen to die, then the probability that this individual has exactly k C neighbors with strategy C and k D neighbors with strategy D is k! kC !kD! (q CjD ) kC (q DjD ) kD . Thus, the probability that the change of Q C equals exactly DQ C~k =zN in one time step is Pr DQ C~k zN where Q k,C (Q k,D ) denotes the proportion of vertices with degree k that are in strategy C(D). Similarly, if a cooperator with degree k is randomly chosen to die, then the probability that the change of Q C equals exactly DQ C~{ k=zN is We next derive the time evolution of Q C under weak selection. Weak selection means that the payoff from the game represents only a small contribution to fitness [2]. In a heterogeneous graph, this requires that the payoff for the player with highest degree is small compared to the baseline fitness, i.e., vk max %1, where k max~m axfk i ji~1,:::,Ng. In the limit of weak selection, the expected change of Q C , denoted by DQ C , in the time interval (t,tzDt) is approximated by EfDQ C g~½v(z{2)Q C (1{Q C ) (b{zc)=N(z{1)Dt for the fixed cost per game and EfDQ C g~½v(z{2)Q C (1{Q C )(vb{c)=N(z{1)Dt for the fixed cost per individual (this follows from Eq (1) and Eq (2) as shown in Supporting Information S1). For both cases, the variance of DQ C is VarfDQ C g~½2Efk 2 g(z{2)Q C (1{Q C )=N 2 z 2 (z{1)Dt (see also Supporting Information S1). So, from the theory of diffusion approximation [53], if the initial value of Q C is s (with 0vsv1), then the fixation probability of cooperation, denoted by r(s), is for the case of fixed cost per game, and, similarly, for the case of fixed cost per individual (the mathematical proofs of these two equations are shown in Supporting Information S1). In both Eq.(3) and Eq.(4), c d is the square of the relative deviation of the degree distribution (i.e. c d~V arfkg=z 2 where Varfkg~Efk 2 g{z 2 is the variance of the degree distribution) and provides a measure of heterogeneity for the graph.

Results
It is easy to see that for both Eq.(3) and Eq.(4), if v~0, then r(s)~s. That is, under neutral selection, the fixation probability of a single cooperator with degree k exactly equals k=zN. To test the effect of graph heterogeneity on neutral selection, two graphs are considered in Figure 2; namely, the scale-free network and the random graph. The simulation results match the theoretical prediction well: if there is no selection, then the fixation probability of a single cooperator is uniquely determined by the number of its neighbors. In a heterogeneous graph, if there is no selection, then the fixation probability of a single cooperator with degree k is larger (less) than 1=N if kwz (kvz). This implies that the condition that the fixation probability of a randomly chosen cooperator is larger (or less) than 1=N cannot be used as a criterion to estimate whether cooperation will be favored by natural selection in a general heterogeneous graph, the degree of the cooperator must also be considered. This is different from the situation for a homogeneous graph [49].
When there is weak selection (i.e. vw0 but vk max %1), the term vN(b{zc)s(1{s)=2(1zc d ) in Eq.(3) denotes the effect of natural selection on cooperation for the case of fixed cost per game. In particular, cooperation will be favored by natural selection (i.e. The scale-free network (generated according to [28]) and the random graph (generated according to [62]) are used to test the effect of graph heterogeneity on neutral selection (v~0). For each of these two graphs, the total population size is N~1000 and the fixation probability of a single cooperator is measured using the fraction of runs where cooperators reached fixation out of 3|10 7 runs (based on 10 3 graphs and 3|10 4 runs per graph). The simulation results are plotted in Figure 2A for the scale-free network and in Figure 2B for the random graph. For both Figures 2A and 2B, the x-axis denotes the number of a single cooperator's neighbors and the y-axis the fixation probability of cooperation. In each of Figures 2A and 2B, the three solid lines represent the theoretical predictions of fixation probabilities for three average degrees, where z~4,6,8 in Figure 2A and z~10,20,30 in Figure 2B, and diamonds, squares and triangles denote the simulation results. It is clear that the simulation results match the theoretical prediction well. doi:10.1371/journal.pone.0066560.g002 r(s)ws) if the benefit-to-cost ratio b=c is larger than the average degree z of the graph. Although this result is similar to Ohtsuki et al. [49] (obviously, if all individuals have the same degree, then c d~0 and Eq.(3) is exactly the same as Ohtsuki et al.'s theoretical result), we also notice that the term 1=(1zc d ) represents how selection favoring cooperation is affected by the graph heterogeneity. Specifically, for given average degree z and weak selection parameter v, the absolute size of the term vN(b{zc)s(1{s)=2(1zc d ) will decrease with the increase of c d . That is, an increase in graph heterogeneity will reduce the effect of selection. Simulation results in Figure 3 show this tendency. The fixation probability approaches that of neutral selection as c d increases.
Similar to Eq.(3), r(s)ws if b=cw1=v in Eq.(4) for the case of fixed cost per individual where 1=v~1 P k p(k) k is the harmonic mean of the degree distribution. Since 1=vƒz with equality if and only if the graph is homogeneous, if selection is weak, then cooperation should be more easily favored by natural selection in a heterogeneous graph. To examine whether this theoretical prediction is true, simulations for four graphs are considered: two Small-World networks (denoted by SW-I and SW-II), a random graph (denoted by RD) and a scale-free graph (denoted by SF). For all four graphs, the total population size is taken as N~500 and three different average degrees are considered. The simulation results are plotted in Figure 4, which show the fixation probability of a single cooperator in the case of fixed cost per game for different values of the benefit-to-cost ratio, b=c. In order to satisfy the weak selection condition vk max %1, v~0:01 for SW-I, SW-II and RD ( Figure 4A) and v~0:001 for SF ( Figure 4B). The theoretical predictions present a good approximation to the numerical results, i.e. the relationship between the benefit-to-cost ratio and the harmonic mean of the degree distribution in a heterogeneous graph determines whether cooperation will be favored by natural selection.

Discussion
Ohtsuki et al. [49] argued that the following simple rule that arises from their theoretical analysis of the homogeneous graph should also hold in general: the fixation probability of a single cooperator is larger than 1=N if b=c is larger than the average degree of the graph. Later, Tomohiko [54] provided a more accurate approximation based on the average degree of the nearest neighbors. In this paper, we find that the fixation probability of a cooperator depends crucially on its degree. In our theoretical framework, a new state variable Q C is used to measure the time evolution of cooperation in a heterogeneous graph, which is the proportion of the total degree of cooperators in the total directed edges, instead of the frequency of cooperators in the population.
If all individuals have the same number of neighbors, then Q C equals the frequency of cooperators and so Ohtsuki et al.'s method is a special case of our model. On the other hand, for the expected payoff of an individual, two cases are considered, called the fixed cost per game and the fixed cost per individual, respectively. In the case of fixed cost per game, a cooperator pays a cost c for each of its neighbors to get a benefit b; and in the case of fixed cost per individual, a cooperator with k neighbors pays a cost c=k for each of its neighbors to get a benefit b=k, i.e. a cooperator will provide a combined fixed benefit b to its all neighbors and pay a fixed cost c equally shared by its all neighbors. For the evolution of cooperation in a heterogeneous graph, our main results show that (i) under neutral selection, the fixation probability of a single cooperator depends linearly on the number of its neighbors, i.e. the more neighbors a single cooperator has, the larger its fixation probability; (ii) under weak selection, cooperation is favored by natural selection if the benefit-to-cost ratio, b=c, is larger than the average degree, z, in the case of fixed cost per game or if the benefit-to-cost ratio, b=c, is larger than the harmonic mean of the degree distribution, 1=v, in the case of fixed cost per individual; and (iii) the heterogeneity of a graph is measured by the square of the relative deviation of the degree distribution, c d , and under weak selection, the effect of selection will decrease with the increase of c d .
Our analysis provides an insight for understanding the evolution of cooperation in a heterogeneous graph. In the case of fixed cost per game, Ohtsuki et al.'s simple rule [49] is still valid on average. However, numerical simulations in their paper show that the Figure 3. Effect of graph heterogeneity on the fixation probability under weak selection and fixed cost per game. Simulation results for the fixation probability in random graphs and scale-free graphs with different heterogeneities are shown in Figure 3A and 3B, respectively (see the generation of these graphs in Supporting Information S1). In Figure 3A, N~1000 and z~20. The benefit-to-cost ratio is taken as b=c~16, 20, 24, 28 and 32, respectively, and the selection intensity is v~0:0005. In Figure 3B, N~1000 and z~10. The benefit-to-cost ratio is taken as b=c~5, 10, 15, 20 and 25, respectively, and the selection intensity is v~0:001. In both Figures 3A and 3B, the x-axis denotes c d and the y-axis the fixation probability of a single cooperator with z neighbors. The fixation probability is measured using the fraction of runs where cooperators reached fixation out of 10 7 runs. The dash line r~0:0005 in Figure 3A and the dash line r~0:001 in Figure 3B denote respectively the fixation probability under neutral selection (v~0). Both Figures 3A and 3B show the tendency that for all different values of b=c, fixation probability approaches that of neutral selection as c d increases. doi:10.1371/journal.pone.0066560.g003 Evolution of Cooperation in a Heterogeneous Graph PLOS ONE | www.plosone.org fixation probabilities of cooperators in heterogeneous graphs such as the scale-free graph are lower than the theoretical prediction. There are two possible reasons. On the one hand, the strength of selection decreases as the graph heterogeneity increases, which implies that the fixation probability is more sensitive to random effects in a strongly heterogeneous graph. Therefore, compared to homogenous graphs, more simulation periods are needed to calculate the fixation probability. In our paper, fixation probabilities of cooperators were found by using either 10 7 or 5|10 7 runs (which is ten times the number of runs used in Ohtsuki et al.'s simulations) and our simulation results are closer to the theoretical prediction. On the other hand, the selection intensity (v~0:01) in Ohtsuki et al.'s simulations is not weak enough. In a heterogeneous graph, a cooperator with many neighbors pays significantly higher than a cooperator with few neighbors. This makes the survival of large degree cooperators very difficult.
In a real population, as pointed out by Santos et al. [33], there is no reason for every cooperator to contribute the same amount to each game in which he/she participates. Santos et al. [33] then introduced the concept of fixed cost per individual. When the contribution per individual is fixed, their simulations showed that there is an impressive boost of cooperation for scale-free graphs under the case of fixed cost per individual compared to the case of fixed cost per game. In fact, the promotion of cooperation is due to the " symmetry breaking" of the game [35]. That is, the payoff difference between cooperator and defector in a single game is no longer c, but inversely proportional to the number of games each player plays. This then gives an evolutionary advantage to large degree cooperators. Our theoretical results confirm Santos et al.'s finding. Since the harmonic mean of the degree distribution decreases as the graph heterogeneity increases, social diversity promotes the emergence of cooperation.
Ohtsuki et al. [49] indicated that in social networks, people might have a substantial number of connections, but only very few of them are strong. Hence, they thought that the 'effective' average degree of many relevant networks could be small, thereby making selection of cooperation on graphs a powerful option. Obviously, for the case of fixed cost per game, our results provide a reasonable theoretical explanation for Ohtsuki et al.'s conjecture. In fact, the low connectivity on average is more favorable for the emergence of cooperation for the case of fixed cost per individual since the harmonic mean of the degree distribution is less than the average degree in general. It is important to point out that, although cooperation will be favored by natural selection when the average degree is low, the fixation probability of a single given cooperator increases when it has more neighbors.
Graph heterogeneity should be one of the most important characteristics of natural and social networks. Our study analyzes the effect of graph heterogeneity under weak selection only, and a further question is how to explain the effect of strong selection. Evolution of cooperation under strong selection in a heterogeneous graph have been considered by several authors [30][31][32][33][34][55][56][57][58]. Unfortunately, our theoretical methods cannot be easily extended to the situation with strong selection. In fact, it has been shown that analytical predictions obtained in the pair approximation leads to an apparent contradiction with simulation results even in regular graphs [11,[59][60][61]. In our opinion, other theoretical methods such as the " macro-dynamics" approach developed by Pinheiro et al. [57][58] are then better able method to explain the evolution of cooperation under strong selection in a heterogeneous graph.

Supporting Information
Supporting Information S1 Supporting Information (containing one table) for Evolution of cooperation in a heterogeneous graph: Fixation probabilities under neutral selection.  Four heterogeneous graphs, SW-I (the Small-World graph generated according to [27] with rewiring probability 0.1), SW-II (the Small-World graph generated according to [27] with rewiring probability 1), RD (random graph generated according to [62]) and SF (scale-free graph generated according to [28]) are used to test the theoretical predictions. Simulation results for the fixation probability of a single cooperator with z neighbors in SW-I, SW-II and RD are shown in Figure 4A. For z~4,6,8, the harmonic means of the degree distribution are 1=v&3:90, 5:90, 7:90 in SW-I, 1=v&3:53, 5:51, 7:51 in SW-II, and 1=v&3:00, 4:82, 6:83 in RD, respectively. The vertical dash line represents the harmonic mean of degree distribution, 1=v, for each of z~4,6,8, where the red, green and blue vertical dash lines correspond to SW-I, SW-II and RD, respectively. Figure 4B shows the simulation results for the fixation probability of a single cooperator with 4z neighbors in SF. For z~8,12,16, the harmonic means of the degree distribution are 1=v&5:72, 8:76, 11:83, respectively. The vertical dash line represents the harmonic mean of degree distribution, 1=v, where the blue, green and red vertical dash lines correspond to z~8,12,16, respectively. In both Figures 4A and 4B, the x-axis denotes the benefit-to-cost ratio, b=c, the y-axis the fixation probability, and the horizontal dash-point line denotes the fixation probability of a single cooperator under neutral selection (i.e. v~0) which is k=zN. The fixation probability of a single cooperator is measured using the fraction of runs where cooperators reached fixation out of 5|10 7 runs (based on 10 3 graphs and 5|10 4 runs per graph). Both Figures 4A and 4B show that the theoretical predictions present a good approximation to the numerical results. doi:10.1371/journal.pone.0066560.g004 Evolution of Cooperation in a Heterogeneous Graph PLOS ONE | www.plosone.org