## Figures

## Abstract

In social dilemmas, cooperation among randomly interacting individuals is often difficult to achieve. The situation changes if interactions take place in a network where the network structure jointly evolves with the behavioral strategies of the interacting individuals. In particular, cooperation can be stabilized if individuals tend to cut interaction links when facing adverse neighborhoods. Here we consider two different types of reaction to adverse neighborhoods, and all possible mixtures between these reactions. When faced with a gloomy outlook, players can either choose to cut and rewire some of their links to other individuals, or they can migrate to another location and establish new links in the new local neighborhood. We find that in general local rewiring is more favorable for the evolution of cooperation than emigration from adverse neighborhoods. Rewiring helps to maintain the diversity in the degree distribution of players and favors the spontaneous emergence of cooperative clusters. Both properties are known to favor the evolution of cooperation on networks. Interestingly, a mixture of migration and rewiring is even more favorable for the evolution of cooperation than rewiring on its own. While most models only consider a single type of reaction to adverse neighborhoods, the coexistence of several such reactions may actually be an optimal setting for the evolution of cooperation.

**Citation: **Zhang C, Zhang J, Weissing FJ, Perc M, Xie G, Wang L (2012) Different Reactions to Adverse Neighborhoods in Games of Cooperation. PLoS ONE 7(4):
e35183.
doi:10.1371/journal.pone.0035183

**Editor: **Angel Sánchez, Universidad Carlos III de Madrid, Spain

**Received: **December 15, 2011; **Accepted: **March 9, 2012; **Published: ** April 23, 2012

**Copyright: ** © 2012 Zhang et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Funding: **This work was supported by the National Nature Science Foundation of China (grant nos. 60736022, 10972002, 60774089, 10972003, and 10926195). In addition, CZ and JZ acknowledge support from the Erasmus Mundus Action (EMA2 Lotus Grant), and MP acknowledges support from the Slovenian Research Agency (grant no. J1-4055). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests: ** The authors have declared that no competing interests exist.

## Introduction

Cooperation is a fascinating area of research since it touches upon so many different disciplines, ranging from biology to economics, sociology, and even theology [1]. The fact that in human societies cooperative behavior is common among unrelated people is puzzling from evolutionary point of view, since cooperation can easily be exploited by selfish strategies. Evolutionary game theory [2]–[4] provides a theoretical framework to address the subtleties of cooperation among selfish individuals. In particular, the prisoner's dilemma [5], [6] is a paradigm example for studying the emergence of cooperation in spite of the fact that self-interest seems to dictate defective behavior.

Past research has identified several key mechanisms (comprehensively reviewed in [7]) that promote the evolution of cooperation. In particular, spatial reciprocity [8] has launched a spree of activity aimed at disentangling the role of the spatial structure by the evolution of cooperation. The seminal works in this area focused on regular graphs and lattices [8]–[18]. Later attention shifted to more complex networks [19], [20], and, in particular, to scale-free networks. Evolutionary games on graphs and networks are thoroughly reviewed in [21]. More recent studies have elaborated on various aspects, including the dynamical organization [22], clustering [23] and mixing patterns [24], [25], as well as memory [26], robustness [27], phase transitions [28] and payoff normalization [29], [30].

While considering population structure is an important step for understanding the evolution of cooperation, a crucial ingredient is still missing. In real social networks the interaction structure is frequently not static but evolving in concern with the behavior of the interacting agents. As reviewed in [31] a vibrant new research area is emerging that studies the joint evolution of interaction structure and behavior. Many models have focused on the way players make (or break) links in reaction to the degree of cooperation they experienced from their interaction partners [32]–[46]. Other models have considered the possibility to leave uncooperative neighborhoods [47]–[54]. It is plausible that both mechanisms can promote the evolution of cooperation, but it is not obvious which of the two mechanisms is more efficient. Moreover, it is not self-evident that all individuals use the same rules for changing their interaction network in response to adverse conditions. In fact, everyday experience tells us that different people may react quite differently when they find themselves in a bad neighborhood: while some tend to migrate to another location, others tend to stay put and instead search for new friends (or get rid of old friends) in order to improve the situation.

Motivated by such considerations, we study the joint evolution of cooperation and interaction structure in the prisoner's dilemma game and in the snowdrift game. The players are of two types that differ in the way they react to an adverse neighborhood. A fixed fraction of the players consists of ‘migrants’, who in proportion to the number of defectors in their neighborhood tend to migrate to another (unoccupied) position in the network. The complementary fraction of the players consists of ‘rewirers’, who have the tendency to break their links with defectors and subsequently to reattach the free links to other players. By changing the parameter , our model allows to transverse smoothly from an adaptive linking model () to a migratory model (). In between these two extremes, we have a situation where different players react differently when finding themselves in an adverse neighborhood.

## Results

Our analysis is based on the prisoner's dilemma game and the snowdrift game, two paradigm models for the evolution of cooperation. We label the payoff parameters in line with the conventions for the prisoner's dilemma [5]: a cooperating player receives the “reward” in case of mutual cooperation and the “sucker's payoff” in case of being defected; a defecting player receives the “temptation to defect” when the other player cooperates and the “punishment” in case of mutual defection. By definition, a prisoner's dilemma game satisfies the payoff relationships . When played as a one-shot game in a well-mixed population, defect is the only evolutionarily stable strategy; despite of the fact that the payoff to both players can be considerably smaller than the payoff for mutual cooperation. The snowdrift game is characterized by and . When played as a one-shot game in a well-mixed population, none of the two pure strategies is evolutionarily stable and a mixed strategy is expected to result [16]. Without loss of generality, we normalize and to and . In all our graphs, is systematically varied from to . Hence, the game considered is a prisoner's dilemma game if and a snowdrift game if .

Fig. 1 shows how for four values of the payoff parameter the level of cooperation evolves in relation to the temptation to defect and the relative frequency of players reacting to adverse conditions by migration. By and large, the outcome is very similar in all four cases. Cooperation is more difficult to achieve for larger values of , but in general the outcome is dominated by the parameter . If migration is the only reaction to adverse conditions (; right-hand border of each panel), cooperation goes extinct not only in the prisoner's dilemma games (upper panels in Fig. 1) but also in the snowdrift games (bottom panels in Fig. 1). In contrast, cooperation can reach high levels or even go to fixation if all players react to adverse conditions by rewiring (; left-hand border of each panel). Interestingly, a combination of migration and rewiring is most favorable for the evolution of cooperation. For a broad range of -values (), cooperation tends to fixation, even in case of a relatively large temptation to defect. Apparently, cooperation is favored if a certain fraction of the players choose for a complete reset of their interactions when surrounded by defectors, while too high levels of migration mix up the population to such an extent that local structures providing a foothold for cooperation cannot develop.

Panels (a) and (b) depict the outcome for the prisoner's dilemma game with and , respectively, while panels (c) and (d) depict the outcome for the snowdrift game with and , respectively. Other payoff parameters are and in all four panels. The density of cooperators in the stationary state (after iteration steps) is color-coded from blue (full defection) to red (full cooperation), as indicated on the right of the figure. If all players are “migrants” (; right-hand border of each panel), defection is clearly dominant, while intermediate levels of cooperation evolve if all players are “rewirers” (; left-hand border of each panel). In all types of game, the highest level of cooperation evolves in mixed populations consisting of both migrants and rewirers. In this figure, the density of occupied nodes is .

To further analyze the mechanisms enhancing or impeding cooperation, we performed extensive numerical simulations for the case . This is a border case that is sometimes called a “weak” prisoner's dilemma game [6]. In general, the weak form of the prisoner's dilemma game can have other properties than the strong form [55], but Fig. 1 clearly demonstrates that this is not the case in our model.

Fig. 2 shows how the evolution of cooperation is affected by population density. At low densities (; upper panels), cooperation does not get off the ground and at best stays at the initial level. If the majority of the population reacts to adverse conditions by migration, cooperation goes extinct. Similar results were obtained when the value of was smaller than about . If the population density is too small, players only have few interaction partners, making it difficult for cooperators to form local clusters enforcing their success.

Panels depict results for all players (a,d), for players adopting rewiring (b,e), and for players adopting migration (c,f), each for two different population densities [ in (a,b,c) and in (d,e,f)]. As in Fig. 1, each panel shows the outcome in dependence on the fraction of players reacting to adverse conditions by migration () and the payoff parameter (), color coded as indicated on the right of the figure. Other payoff parameters are and in all six panels. See main text for further details.

Up to now, we have focused on the behavior of the system as a whole. We will now zoom in a bit and study the different types of player in more detail. The middle and right panels of Fig. 2 demonstrate that in the stationary state the strategy choice (cooperate versus defect) of a player becomes associated with the player's reaction to adverse conditions. In fact, players adopting rewiring (Fig. 2(b)(e)) show a markedly higher tendency to cooperate than players adopting migration (Fig. 2(c)(f)). The difference in cooperation tendency between both types of player is smallest at low population densities (here ), where for neither type of player the relative frequency of cooperation exceeds . One reason for this may be that the migration of players provides an opportunity for defectors to invade and destroy the sparse cooperative clusters in the scattered population, essentially creating a situation comparable to well-mixed conditions. In the low-density scenario, cooperation is only sustained (at intermediate level) when the exploitation from defectors is not too strong. At high densities, the situation is markedly different. Even for relatively large values of , there is a boost of cooperation even for large values of . Still, players adopting migration do worse than those adopting rewiring.

Fig. 3 shows how the reaction of a player to adverse conditions affects the player's degree of connectedness in the evolved stationary population. It is obvious that the topology of a player's neighborhood is at least partly shaped by the player's behavioral choices. It has been shown that the adaptive interplay between the players' strategies and the underlying network can lead to the emergence of heterogeneity from an initially homogeneous connectivity structure [8]. As before, the difference in connectivity between players adopting migration and players adopting rewiring is quite small at low population density (upper panels in Fig. 3). This difference becomes much more pronounced at high density. For players adopting rewiring (left panels in Fig. 3), the average degree first increases with (to reach highest levels for ), subsequently decreasing with a further increase of . In contrast, the average degree of players adopting migration only marginally depends on , staying close to the initial value of . For most parameter combinations, players adopting rewiring have a higher degree than players adopting migration. Presumably, rewiring leads to an accumulation of links between cooperators and thereby the formation of tightly connected cooperative clusters.

Panels depict results for players adopting rewiring (a,c) and for players adopting migration (b,d), each for two different population densities [ in (a,b) and in (c,d)]. The average initial degree of each occupied node (player) was . Graphical conventions and payoff parameters are the same in Fig. 2, only that the color bar encodes the average degree in the stationary state. See main text for further details.

This interpretation is corroborated by Fig. 4, which shows the difference in connectedness between cooperating and defecting individuals. The regions in parameter space where a high level of cooperation evolved (Fig. 2(d)) corresponds to those regions where cooperators are tightly connected (i.e. where cooperators have a high degree). As before, connectedness is low in the low-density situation (where cooperation did not get off the ground) and much higher (at least for cooperators) in the high-density situation. For low and intermediate values of , the connectedness of cooperators is markedly higher than the connectedness of defectors. The opposite is the case a high values of .

Panels depict results for cooperators (a,c) and defectors (b,d), each for two different population densities [ in (a,b) and in (c,d)]. Graphical conventions and payoff parameters are the same in Fig. 3. See main text for further details.

Finally, we consider the differences in the degree of cooperation experienced by cooperators and defectors, respectively (Fig. 5). Irrespective of population density and other parameters, defectors always ended up in adverse neighborhoods. In contrast, cooperators tended to interact only with other cooperators - at least as long as the fraction of players adopting rewiring was not too small (i.e. for ). For large values of , cooperative clusters did not emerge, corresponding to the collapse of cooperation under these conditions.

Panels depict results for the neighborhood of cooperators (a,c) and the neighborhood of defectors (b,d), each for two different population densities [ in (a,b) and in (c,d)]. Graphical conventions and payoff parameters are the same as in Fig. 2. See main text for further details.

Overall, our results confirm the importance of the formation of cooperative clusters. If population densities are not too low and if sufficiently many individuals adopt rewiring, cooperative clusters can emerge even under unfavorable conditions (e.g. a large value of ). Once clusters of cooperators have formed, selection against defective partners can effectively shield clusters of cooperators from the invasion of defectors. This is so because cooperators within the cluster attract interactions with cooperators at the cluster boundary. In this way the payoffs of cooperators both inside the cluster and at its fringe are enhanced and interactions with defectors are avoided. Defectors surrounding clusters of cooperators have limited opportunities for exploitation, allowing clusters of cooperators to expand. Thus, clusters uphold cooperative behavior even if the temptation to defect is large. Conversely, defectors are unable to claim lasting benefits from occupying the clusters, simply because they become very weak as soon as all the neighbors of the defecting cluster become defectors themselves.

## Discussion

In line with other studies, we have shown that in a network environment cooperation in social dilemmas does readily evolve if players have the opportunity to change their local interaction structure when surrounded by non-cooperative neighbors. Among the two reactions to adverse conditions considered, rewiring was clearly more favorable for the evolution of cooperation than migration. This was even the case in a model where the costs of rewiring and emigration were the same (zero), while migration will often be more costly in natural settings. Interestingly, the highest degree of cooperation evolved when the player population was polymorphic in the sense that both types of reaction to adverse neighborhoods (rewiring and migration) were present in non-negligible frequencies. We interpret this finding by the interplay of two factors: while migration induces a certain mixing of the population due to increased interaction ranges of the migrating players, rewiring may lead to strongly heterogeneous interactions networks, which are tightly associated with flushing cooperative states [38].

In our model, the reaction to adverse conditions was assumed as a fixed property of each player. Hence, this reaction did not evolve. Our results suggest that the joint evolution of the strategies in the cooperation game and the reaction to adverse condition would lead to a polymorphism in the reaction to adverse neighborhoods. It remains to be seen whether this is indeed the case.

## Methods

All simulations were run on a lattice of nodes with periodic boundary. In each simulation, a fraction of the lattice nodes was occupied by players, who initially were distributed randomly over the lattice. Initially, each player was connected with all players on adjacent lattice nodes. Accordingly, the initial degree of each player ranged from to , with an average of .

Throughout a simulation, each player had the fixed status of either adopting migration or rewiring when confronted with adverse conditions. This status was assigned to players at random at the start of a simulation, with being the fraction of migrants. In addition, players could at each time be classified as either cooperators or defectors in the cooperation game, but the strategy of each player could change during a simulation due to payoff-based learning. Initially, the strategies and were randomly assigned to the players with equal probability.

To simulate evolution, we employed event-based asynchronous updating where interactions, rewiring and migration all occurred on the same time scale. Whenever an “event” occurred, a focal player was chosen at random. This player had pairwise interactions with all “neighbors” (that is, all players connected with ), yielding a sum of all payoffs. For a randomly selected neighbor of , the payoff was determined in a similar way. Based on the payoff difference , the focal player switches to j's strategy with probability . If is large, payoff differences do not matter much for the direction of strategy change, while such differences are decisive in case of a small value of . Throughout this work we set , indicating that strategies of better performing players are readily, though not always, adopted.

Following the game interactions and the strategy change phase, the focal player reconsiders its interaction structure. If the focal individual adopts rewiring, it will cut a random tie with a defecting neighbor with probability proportional to the number of defectors; subsequently the free link will be reattached to another player randomly chosen from the entire population. If the focal player adopts migration, it will migrate to a randomly chosen empty target site with probability proportional to the number of defectors in its neighborhood; at the new site, it will establish new links with all players occupying adjacent sites on the network.

During a full iteration, the above process was repeated times. Hence on average each player was in the focal role once per iteration, and each player had on average once the opportunity to pass its strategy to one of its neighbors. The process was repeated until a stationary state was reached, where the distribution of strategies and the characteristics of neighborhoods did not change any more. Typically we ran each simulation for steps. For each parameter combination we ran replicate simulations. The results reported are averages over these replicates.

## Author Contributions

Conceived and designed the experiments: CZ JZ FJW MP GX LW. Performed the experiments: CZ JZ FJW MP GX LW. Analyzed the data: CZ JZ FJW MP GX LW. Contributed reagents/materials/analysis tools: CZ JZ FJW MP GX LW. Wrote the paper: CZ JZ FJW MP GX LW.

## References

- 1.
Nowak MA, Highfield R (2011) SuperCooperators: Altruism, Evolution, and Why We Need Each Other to Succeed. New York: Free Press.
- 2.
Smith JM (1982) Evolution and the Theory of Games. Cambridge, UK: Cambridge University Press.
- 3.
Hofbauer J, Sigmund K (1998) Evolutionary Games and Population Dynamics. Cambridge, UK: Cam-bridge University Press.
- 4.
McNamara JM, Weissing FJ (2010) Evolutionary Game Theory. In: Székeley T, Moore AJ, Komdeur J, editors. Social Behaviour. Genes, Ecology and Evolution. Cambridge, UK: Cambridge University Press. pp. 109–133.
- 5.
Axelrod R (1984) The Evolution of Cooperation. New York: Basic Books.
- 6. Roca CP, Cuesta JA, Sánchez A (2009) Evolutionary game theory: Temporal and spatial effects beyond replicator dynamics. Phys Life Rev 6: 208–249.
- 7. Nowak MA (2006) Five rules for the evolution of cooperation. Science 314: 1560–1563.
- 8. Nowak MA, May RM (1992) Evolutionary games and spatial chaos. Nature 359: 826–829.
- 9. Nowak MA, May RM (1993) The spatial dilemmas of evolution. Int J Bifurcat Chaos 3: 35–78.
- 10. Lindgren K, Nordahl MG (1994) Evolutionary dynamics of spatial games. Physica D 75: 292–309.
- 11. Durrett R (1994) The importance of being discrete (and spatial). Theor Popul Biol 46: 363–394.
- 12. Grim P (1995) The greater generosity of the spatialized prisoner's dilemma. J Theor Biol 173: 353–359.
- 13. Szabó G, Tőke C (1998) Evolutionary prisoner's dilemma game on a square lattice. Phys Rev E 58: 69–73.
- 14. Brauchli K, Killingback T, Doebeli M (1999) Evolution of cooperation in spatially structured popula- tions. J Theor Biol 200: 405–417.
- 15. Hauert C, Szabó G (2003) Prisoner's dilemma and public goods games in different geometries: compulsory versus voluntary interactions. Complexity 8: 31–38.
- 16. Hauert C, Doebeli M (2004) Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428: 643–646.
- 17. Szolnoki A, Perc M (2009) Promoting cooperation in social dilemmas via simple coevolutionary rules. Eur Phys J B 67: 337–344.
- 18. Szolnoki A, Perc M (2008) Coevolution of teaching activity promotes cooperation. New J Phys 10: 043036.
- 19. Albert R, Barabási AL (2002) Statistical mechanics of complex networks. Rev Mod Phys 74: 47–97.
- 20. Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang D (2006) Complex networks: Structure and dynamics. Phys Rep 424: 175–308.
- 21. Szabó G, Fáth G (2007) Evolutionary games on graphs. Phys Rep 446: 97–216.
- 22. Gómez-Gardeñes J, Campillo M, Moreno Y, Floría LM (2007) Dynamical organization of cooperation in complex networks. Phys Rev Lett 98: 108103.
- 23. Assenza S, Gómez-Gardeñes J, Latora V (2008) Enhancement of cooperation in highly clustered scale- free networks. Phys Rev E 78: 017101.
- 24. Rong Z, Li X, Wang X (2007) Roles of mixing patterns in cooperation on a scale-free networked game. Phys Rev E 76: 027101.
- 25. Poncela J, Gómez-Gardeñes J, Moreno Y (2011) Cooperation in scale-free networks with limited associative capacities. Phys Rev E 83: 057101.
- 26. Wang WX, Ren J, Chen G, Wang BH (2006) Memory-based snowdrift game on networks. Phys Rev E 74: 056113.
- 27. Poncela J, Gómez-Gardeñes J, Floría LM, Moreno Y (2007) Robustness of cooperation in the evolu- tionary prisoner's dilemma on complex systems. New J Phys 9: 184.
- 28. Floría LM, Gracia-Lőzaro C, Gómez-Gardeñes J, Moreno Y (2009) Social network reciprocity as a phase transition in evolutionary cooperation. Phys Rev E 79: 026106.
- 29. Santos FC, Pacheco JM (2006) A new route to the evolution of cooperation. J Evol Biol 19: 726–733.
- 30. Szolnoki A, Perc M, Danku Z (2008) Towards effective payoffs in the prisoner's dilemma game on scale-free networks. Physica A 387: 2075–2082.
- 31. Perc M, Szolnoki A (2010) Coevolutionary games – a mini review. BioSystems 99: 109–125.
- 32. Zimmermann MG, Eguíluz V, Miguel MS (2004) Coevolution of dynamical states and interactions in dynamic networks. Phys Rev E 69: 065102(R).
- 33. Equíluz VM, Zimmermann MG, Cela-Conde CJ, Miguel MS (2005) Cooperation and the emergence of role differentiation in the dynamics of social networks. Am J Sociology 110: 977–1008.
- 34. Pacheco JM, Traulsen A, Nowak MA (2006) Active linking in evolutionary games. J Theor Biol 243: 437–443.
- 35. Pacheco JM, Traulsen A, Nowak MA (2006) Coevolution of strategy and structure in complex networks with dynamical linking. Phys Rev Lett 97: 258103.
- 36. Santos FC, Pacheco JM, Lenaerts T (2006) Cooperation prevails when individuals adjust their social ties. PLoS Comput Biol 2: 1284–1290.
- 37. Hanaki N, Peterhansl A, Dodds PS, Watts DJ (2007) Cooperation in evolving social networks. Management Science 53: 1036–1050.
- 38. Szolnoki A, Perc M, Danku Z (2008) Making new connections towards cooperation in the prisoner's dilemma game. EPL 84: 50007.
- 39. Pacheco JM, Traulsen A, Ohtsuki H, Nowak MA (2008) Repeated games and direct reciprocity under active linking. J Theor Biol 250: 723–731.
- 40. Kun Á, Scheuring I (2009) Evolution of cooperation on dynamical graphs. Biosystems 96: 65–68.
- 41. Szolnoki A, Perc M (2009) Resolving social dilemmas on evolving random networks. EPL 86: 30007.
- 42. Tanimoto J (2009) Promotion of cooperation through co-evolution of networks and strategy in a 2×2 game. Physica A 388: 953–960.
- 43. Van Segbroeck S, Santos FC, Lenaerts T, Pacheco JM (2009) Reacting differently to adverse ties promotes cooperation in social networks. Phys Rev Lett 102: 058105.
- 44. Gräser O, Xu C, Hui PM (2009) Disconnected-connected network transitions and phase separation driven by co-evolving dynamics. EPL 87: 38003.
- 45. Szolnoki A, Perc M (2009) Emergence of multilevel selection in the prisoner's dilemma game on coe- volving random networks. New J Phys 11: 093033.
- 46. Gracia-Lázaro C, Quijandría F, Hernández L, Floría LM, Moreno Y (2011) Coevolutionary network approach to cultural dynamics controlled by intolerance. Phys Rev E 84: 067101.
- 47. Majeski SJ, Linden G, Linden C, Spitzer A (1999) Agent mobility and the evolution of cooperative communities. Complexity 5: 16–24.
- 48. Vainstein MH, Arenzon JJ (2001) Disordered environments in spatial games. Phys Rev E 64: 051905.
- 49. Vainstein MH, Silva ATC, Arenzon JJ (2007) Does mobility decrease cooperation? J Theor Biol 244: 722–728.
- 50. Helbing D, Yu W (2008) Migration as a mechanism to promote cooperation. Adv Compl Syst 11: 641–652.
- 51. Helbing D, Yu W (2009) The outbreak of cooperation among success-driven individuals under noisy conditions. Proc Natl Acad Sci USA 106: 3680–3685.
- 52. Droz M, Szwabinski J, Szabó G (2009) Motion of influential players can support cooperation in prisoner's dilemma. Eur Phys J B 71: 579–585.
- 53. Meloni S, Buscarino A, Fortuna L, Frasca M, Gomez-Gardeñes J, et al. (2009) Effects of mobility in a population of prisoner's dilemma players. Phys Rev E 79: 067101.
- 54. Roca CP, Helbing D (2011) Emergence of social cohesion in a model society of greedy, mobile individuals. Proc Natl Acad Sci USA 108: 11370–11374.
- 55. Ohtsuki H, Nowak MA (2006) The replicator equation on graphs. J Theor Biol 243: 86–97.