Adaptive Evolution of Cooperation through Darwinian Dynamics in Public Goods Games

The linear or threshold Public Goods game (PGG) is extensively accepted as a paradigmatic model to approach the evolution of cooperation in social dilemmas. Here we explore the significant effect of nonlinearity of the structures of public goods on the evolution of cooperation within the well-mixed population by adopting Darwinian dynamics, which simultaneously consider the evolution of populations and strategies on a continuous adaptive landscape, and extend the concept of evolutionarily stable strategy (ESS) as a coalition of strategies that is both convergent-stable and resistant to invasion. Results show (i) that in the linear PGG contributing nothing is an ESS, which contradicts experimental data, (ii) that in the threshold PGG contributing the threshold value is a fragile ESS, which cannot resist the invasion of contributing nothing, and (iii) that there exists a robust ESS of contributing more than half in the sigmoid PGG if the return rate is relatively high. This work reveals the significant effect of the nonlinearity of the structures of public goods on the evolution of cooperation, and suggests that, compared with the linear or threshold PGG, the sigmoid PGG might be a more proper model for the evolution of cooperation within the well-mixed population.


Introduction
The evolution of cooperation in social dilemmas has attracted broad interests across disciplines [1][2][3][4][5]. Social dilemmas are situations in which individual rationality leads to collective irrationality [6,7]. They are pervasive in all kinds of relationships, from the interpersonal to the international. For example, a public local library financed through donations benefits all people in the community. One can benefit most if he donates nothing. However, if everyone reasoned like this, the library would not keep running due to the lack of finance, and all people would be worst off [8]. This is a Public Goods dilemma. There exists another kind of social dilemma called commons dilemma. For example, farmers living in a common grassland can benefit more by raising as many cattle as they want. However, if every farmer reasoned like this, the grassland would be depleted very soon, and all farmers would worst off [6]. The same reasoning applies to these two kinds of social dilemmas, so we focus on the Public Goods dilemma, which is usually modeled as a Public Goods game (PGG).
In a traditional PGG experiment, some subjects form a group. Each subject is endowed with a certain amount of money, and they have to decide how much to invest in the public project, which is increased to a multiple of it and then split evenly among all subjects. So the gains of the subjects consist of two parts: the money left that they do not invest and the money gained from investing in the public project. For example, each of a fourmember group is given 20 money units (MUs), and the money invested in the public project is doubled. If all members invest 20 MUs, everyone will have 40 MUs. However, every invested MU only returns a half, and thus all members have an incentive to keep all money in pocket. If you defect by investing zero while every other member invests 20 MUs, you will have 50 MUs while other members 30 MUs per person. If all members defect, everyone ends up with 20 MUs and the benefit of the public project is forgone. Consequently a dilemma arises. Since every invested MU returns a half, from now on we call it a linear PGG, instead ( Fig. 1).
In the linear PGG, investing nothing is the only equilibrium. That is, no one can gain more by investing more than zero no matter how much others invest. However, whether in linear PGG experiments or in real life, people often invest more than zero [9]. To better understanding people's behaviors, the threshold PGG is extensively researched (Fig. 1). In the threshold PGG, there exits a provision point or threshold value. If the total sum of the contributions is less than it, all contributions are lost, whereas if the total sum exceeds it, a fixed amount of the public good is gained. In contrast to the linear PGG, the threshold PGG has other equilibria except investing nothing. That is, any combination of contributions that sum to the provision point is an equilibrium. For example, each of a four-member group is given 20 MUs, and when the money invested in the public project reached 60 MUs every member is given extra 40 MUs. Then every member invests 15 MUs is an equilibrium. Three investing 20 MUs and one investing zero is another equilibrium. A threshold PGG is a dilemma with a coordination game embedded in it [8].
However, most of social dilemmas in the real world are not with an obvious or clearly defined provision point. For example, in order to establish and maintain a public local library, those initial donations are important. Once the library starts to run, extra donations are also important for keep it running smoothly. But they are not as important as those that finally make possible the establishment of the library. Therefore, a tilted S-shaped continuous function such as a sigmoid function may provide a better model of many social dilemmas [8,[10][11][12]. We refer to a PGG with this kind of structure as a sigmoid PGG (Fig. 1). As pointed out in [11], the linear or threshold PGG is a simplification, or rather an extreme version of the sigmoid PGG.
So far, there have been very few efforts made to directly explore the effect of nonlinearity of the structures of public goods on the evolution of cooperation. In [10], a rather simple model was employed to independently analyze the accelerating, linear, and decelerating portions of the S-shaped function, so that the complexity of directly dealing with the S-shaped function itself was circumvented. In [12], the authors concluded by adopting replicator dynamics that the threshold PGG (therein is called the Volunteer's Dilemma) is a good approximation of any public goods games in which the public good is a nonlinear function of the number of cooperators (see further comparison to our analysis in section Results and Discussion). Here we will apply Darwinian dynamics [4,[13][14][15][16] to analyze the evolutionarily stable strategies (ESS) of these three kinds of PGGs, and try to show that the sigmoid PGG is really a more proper model for the evolution of cooperation within the well-mixed population, compared with the linear or threshold PGG in that it can reinforce our understanding of people's behaviors in the real world.

Analysis
The pioneering definition of ESS, which is originated by Maynard Smith and Price, refers to a strategy that, when common, can resist the invasion of a minority of any other strategy [17]. Resistance to invasion is a static concept, since it says nothing about what would happen if the population starts at (or is perturbed to) a nearby point [15]. Therefore, an ESS which does not require convergence stability may be unattainable through strategy dynamics by natural selection. This leads to the proliferation of related terminology such as evolutionarily unbeatable strategy, d-stability, internal stability, and evolutionarily singular strategy [18].
In contrast, Darwinian dynamics use a fitness-generating function (G-function) approach to continuous-trait evolutionary games [13,14]. The G-function allows for simultaneous consideration of population dynamics and strategy dynamics. An ESS is redefined as a coalition of strategies that is both convergentstable and resistant to invasion, which is a natural extension of the original definition of Maynard Smith and Price. Those strategies consisting of an ESS are evolutionarily stable maxima on the adaptive landscape [4]. Here we adopt this definition of ESS.
In the following, we first introduce Darwinian dynamics and the extended concept of ESS. Then we analyze these three kinds of PGGs in this context. After the relatively simple linear and sigmoid PGGs are analyzed, the threshold PGG, which is not continuously differentiable so that the G-function approach cannot be directly applied to, is approximated by analyzing a class of PGGs with the structure of power functions.

The G-function Approach
The G-function approach is mainly developed by Vincent, Brown, and their coauthors [4,13,14,16]. We begin with introducing the fitness-generating function (G-function). Assume that there are s populations, and that the i-th population adopts the strategy u i and its frequency is p i [P~½0,1. All strategies u i 's are limited in the evolutionarily feasible set U. We set u~½u 1 ,u 2 , . . . ,u s [U s and p~½p 1 ,p 2 , . . . ,p s [P s . The G-function G(v,u,p) represents the fitness of the i-th population when the virtual variable v[U is replaced with u i .
Darwinian dynamics consist of population dynamics and strategy dynamics. In terms of the G-function G(v,u,p), the population dynamics are given by where When strategies u i 's do not evolve with time, they are equivalent to the replicator dynamics [19,20]. The strategy dynamics are given by where h is a positive factor that influences the speed of the evolution of strategies [16]. In the special case that one extant strategy is invaded by one rare mutant strategy, they reduce to the adaptive dynamics [14,18,21,22]. A non-trivial equilibrium point p Ã~½ p Ã 1 , . . . ,p Ã s [P s (reorder the indexes if necessary) is called an ecologically stable equilibrium point, if it satisfies that for i~1, . . . ,s, and that every trajectory starting from a point which is in P s and near p Ã remains in P s for all time and converges to p Ã as time approaches infinity. The strategies corresponding to p Ã is denoted by The coalition of strategies u Ã c [U s is defined as an evolutionarily stable strategy (ESS), if p Ã is an ecologically stable equilibrium point for any u Ã m [U s{s . The adaptive landscape is simply a plot of G(v,u,p){G Â Ã versus the virtual variable v with u and p fixed. The ESS Maximum Principle [13] states that G(v,u,p){G Â Ã u~u Ã ,p~p Ã must take on its maximum value, 0, as a function of v[U at v~u Ã 1 , . . . ,u Ã s . Here we assume that the evolution of strategy is slower than that of population (but in all of the following invasion simulations we do not make this assumption), and focus on the ESS coalition of one strategy where u Ã c~u Ã 1 and p Ã 1~1 . On the adaptive landscape, a stable minimum indicates an evolutionary branching point. The population which evolves to branching points may diverge into two separate populations or species with distinct strategies [18,22]. Both unstable maxima and unstable minima are repelling points, and they should not be observed in nature [15]. An ESS is an global fitness maximum and convergently stable [14].
In the interior of U, a necessary condition for u Ã 1 to resist the invasion of rare mutant strategies is given by A necessary condition for the convergence stability of u Ã 1 is given by The linear PGG is played in a group of n interacting members. Each member is endowed with c units of utility, and they have to decide how much to invest in the public project. The total units of utility invested in the public project is multiplied by a positive number r and then split evenly among all members. If r §n, no member will lose anything no matter how much he invests. If rƒ1, no member can gain more no matter how much he invests. So the number r is restricted between one and n. Group members benefit most when all cooperate, but each has an incentive to contribute nothing because every invested unit of utility only returns r=n units of utility and thus cooperation incurs cost c(1{r=n) to himself. So the group will no doubt end up all members contributing nothing when they get experienced and the benefit of the public project is forgone. This is the dilemma all group members face. The interests of individuals totally contradict the interest of the group.
From now on we set c~1 with no loss of generality, since it has no effect on the nature of the dilemma. We subsequently apply this G-function approach to the aforementioned three kinds of PGGs, so as to analyze the dependence of cooperation levels on the structures of Public Goods.
For the PGG, if the populations are evolutionarily stable in the evolutionarily feasible set U~½0,1, the expected contribution from any random group member is X s i~1 p i u i . In a group of n members, if the focal member decides to contribute v[U, then the average contribution A s (v) is given by Thus the return from the public good for the focal member is rg½A s (v), and the G-function is given by where the function g(x) is supposed to represent the structure of the public good (Fig. 1).

The Linear PGG
In the special case of the linear PGG of our interest here (Fig. 1), we set and thus the G-function is It follows that which is independent of the composition of the population. Group members can always benefit more by reducing their contributions, so there exists no ESS in the interior of ½0,1.
However, this also gives us a hint that contributing nothing, where u Ã 1~0 and p Ã 1~1 , is the only possible ESS. Considering that the adaptive landscape reaches its global maximum, 0, in ½0,1 when v~0 (Fig. 2), contributing nothing is surely an ESS for the linear PGG. Similarly, we can conclude that another boundary value of ½0,1, contributing all, where u Ã 1~1 and p Ã 1~1 , is not an ESS, since the adaptive landscape reaches its global minimum, 0, in ½0,1 when v~1 (Fig. 2).
A simulation of altruistic cooperators who contribute all (i.e., v~1) invading the population of defectors who contribute nothing (i.e., v~0) is shown in Fig. 3. The result shows that the ESS v~0 is rather robust against invasion. Yet this contradicts the fact that the mean contributions usually end up with between 40% and 60% in experiments [9].

The Sigmoid PGG
In the special case of the sigmoid PGG (Fig. 1), we set Other functions with similar properties are of course possible, but not explored here for simplicity. Thereby the G-function is simplified as We examine the one-strategy ESS (coalition of one strategy); that is, s~1. When u Ã c~u Ã 1 and p Ã 1~1 , the G-function (Fig. 4) is It follows that If rv 2n p , LG(v,u Ã 1 ,p Ã 1 ) Lv v0. We can verify that v~0 is the global maximum in ½0,1 of the adaptive landscape Hence, if rv 2n p , contributing nothing is also an ESS for the sigmoid PGG, just as in the case of the linear PGG.
When r § 2n p , the equation has two solutions in ½0,1: and We can identify u Ã 1,2 as an ESS candidate by verifying the following two conditions, and Similarly, we can show that u Ã 1,1 is an unstable fitness minimum. Hence there exists a stable state of the population contributing u Ã 1,2 , which is more than half, if the return rate r is relatively high. A simulation of defectors who contribute nothing (i.e., v~0) invading the population of individuals who play the ESS (i.e., v~u Ã 1,2 ) is shown in Fig. 5. The result shows that the ESS is surely able to resist the invasion. The Threshold PGG For the special case of the threshold PGG (Fig. 1), we set Yet the discontinuity of g t (x) inhibits the application of Darwinian dynamics to our research into the process of evolution. Instead, we adopt a class of power functions g k (x), where k~1,2, . . ., to approach function g t (x) (Fig. 6); that is, Other functions with similar properties are of course possible, but not explored here for simplicity. Hence the G-function can be expressed as We still focus on the one-strategy ESS where s~1. When u Ã c~u Ã 1 and p Ã 1~1 , the adaptive landscape (Fig. 6) is It follows that The equation LG also has two solutions in ½0,1: and We can identify u Ã 1,2 as an ESS candidate by verifying the following two conditions, and Similarly, we can show that u Ã 1,1 is an unstable fitness minimum. With increasing k, u Ã 1,1 monotonically decreases, whereas u Ã 1,2 monotonically increases, and both approach the threshold value 1=2 (Fig. 6). Fig. 6 also shows that, in contrast to the ESS in the sigmoid PGG, here just on the left side of u Ã 1,2 there exists a global minimum, which makes the ESS is rather fragile. This point is fully exposed in Fig. 7, where only 0:1% invaders of defectors who contribute nothing (i.e., v~0) drove the whole population to the stable state of contributing nothing with a much faster speed relative to that in Fig. 3 or Fig. 5. Hence the threshold PGG basically does not have much advantage over the linear PGG.

Results and Discussion
In summary, by adopting Darwinian dynamics, we have explored the significant effect of nonlinearity of the structures of public goods on the evolution of cooperation within the wellmixed population. The threshold PGG does not have much advantage over the linear PGG, whereas in the sigmoid PGG there exists a one-strategy ESS of the whole population contributing more than half. This suggests that the sigmoid PGG might be a more proper mathematical model for the research of the evolution of cooperation within the well-mixed population, and thereby may release researchers from the shackles of the linear or threshold PGG.
In contrast to most work in which replicator dynamics or adaptive dynamics were applied to the evolution of cooperation in social dilemmas [12,22], here we adopt Darwinian dynamics mainly developed by Vincent, Brown, and their coauthors, which simultaneously consider the evolution of populations and strategies on a continuous adaptive landscape [4,13,14,16]. In Darwinian dynamics, the concept of ESS is extended as a coalition of strategies that is both convergent-stable and resistant to invasion, whereas the original definition of ESS by Maynard Smith and Price might be unattainable through strategy dynamics by natural selection. This well-developed framework provides us with another wonderful mathematical tool for the research related to natural selection.
To our knowledge the only systematic theoretical analysis until now of the effect of nonlinearity of the structures of public goods on the evolution of cooperation is [12], in which a series of functions 1=(1ze {kx ) were adopted to explore the sigmoid PGG and their limit function when k?? was used to approach the threshold PGG, and the authors concluded that the threshold PGG is a good approximation of any public goods games in which the public good is a nonlinear function of the number of cooperators. However, compared to Eqn. (28) we adopt here, 1=(1ze {kx ) is not a good approximation due to its asymptotic nature. For example, this series of functions cannot represent full cooperation (or full defection) even though all individuals are cooperators (or defectors).
Both in the sigmoid PGG approximated by 1=(1ze {kx ) and in the threshold PGG approximated either by lim k?? 1=(1ze {kx ) or by Eqn. (25), the ESS (note the different definition of ESS in our analysis from [12]) is accompanied by an unstable cooperation level (Figs. 6 and 7), which makes the ESS is rather fragile. In contrast, in the sigmoid PGG approximated here by Eqn. (15) the ESS is the only global extreme point in the interior of the evolutionarily feasible set U~½0,1 (Figs. 4 and 5). This suggests that the sigmoid PGG might be a more proper model for the evolution of cooperation within the well-mixed population, in that it hosts a non-trivial evolutionarily stable cooperation level when the return rate is relatively high, whereas the linear or threshold PGG never does.
Note that our results are reached within the well-mixed population. There exist different possibilities if we adopt other assumptions on the population, the group size, or the structure of the PGG. For example, within structured populations with different group sizes, the coexistence of cooperation and defection is possible even for the linear PGG due to noise underlying strategy adoptions [23]. The exploration of the linear PGG that requires a minimum collective investment to ensure any benefit shows that decisions within small groups under high risk significantly raise the chances of coordinating actions [24]. In addition, the relative size of the threshold value of the threshold PGG might also affect the evolution of cooperation within the structured population [25].
However, our work does show the significant effect of nonlinearity of the structures of public goods on the evolution of cooperation within the well-mixed population. Actually, when x increases from 0 to 1, the slope of the S-shaped function g(x) goes through a process from accelerating to decelerating. Simulations show that this property of g(x) plays a key role for the existence of a robust ESS in the PGG within a well-mixed population. Naturally, an interesting future work might be to search for the optimal structure of public goods in the sense that complete cooperation is a robust global ESS in the PGG with this kind of structure, and the way to implement it in the real world.