Table 1.
Base payoff values.
Fig 1.
The payoff matrix of the two-stage game.
The payoff matrix of the two-stage game can be constructed based on the payoffs of the first stage (prisoner’s dilemma) and second stage games (game B). The payoffs of the row player are shown.
Table 2.
Payoff values of the two-stage game when game B is a Snow Drift game (top) and a Stag Hunt game (bottom).
The first number shows the payoff of the row player, and the second number shows the payoff of the column player. Cooperative Nash equilibria are indicated by green and defective equilibria by red cells.
Fig 2.
Simple rules for the existence of a cooperative Nash equilibrium.
A: A cooperative Nash equilibrium in which defectors play softly with cooperators and cooperators play hard with defectors when the game B is a Snow Drift game exists when the cost of cooperation is smaller than the coordination asymmetry defined as the payoff difference of hard and soft strategies in the Snow Drift game. B: A cooperative Nash equilibrium, when the game B is a Stag Hunt game, exists when the cost of cooperation is smaller than the cost of coordinating on an inferior equilibrium, defined as the payoff difference of the superior and inferior equilibria in the Stag Hunt game. These conditions can be derived by requiring the payoff of Nash strategies (shown in white) to be larger than all the other payoffs in the same column.
Fig 3.
Frequency of different strategies in the fixed points of the replicator-mutator dynamics.
Form A to D the game B is respectively, Snow Drift, Battle of the Sexes, Leader, and Stag hunt game. For anti-coordination games (A to C) the replicator-mutator dynamic has two stable fixed points, a cooperative fixed point where cooperation evolves (top) and a defective fixed point where cooperation does not evolve. For the Stag Hunt game, D, the replicator-mutator dynamic has two cooperative fixed points (top) and a defective fixed point (bottom).
Fig 4.
The evolution of cooperation depends only on the cost of cooperation.
The cooperative fixed point of the evolutionary dynamics when the game B is a Snow Drift game is plotted for two different mutation rates and two different benefits of cooperation (as indicated in the legend). The mixed strategy equilibrium (MNE) composed of Cuu, Cdu, Ddu, and Ddd, which coincides with the cooperative fixed point for zero mutation rate, is also plotted (orange circles). A cooperative fixed point exists for small costs and becomes unstable for high costs. The dynamics settle in a fully cooperative fixed point for too small costs where only cooperative strategies survive. Here a helping game version of the Prisoner’s Dilemma with payoff values R = b − c, T = b, P = 0, and S = −c is used. A base payoff of π0 = 5 is added to all the individuals. For the Snow Drift game, the base payoff values, presented in Table 1, is used.
Fig 5.
The probability of settling in the cooperative fixed point.
A: The probability of settling in cooperative fixed point starting from random initial conditions for different structures of game B. While for anti-coordination games, the cooperative fixed point has a large basin of attraction, for the Stag Hunt game, cooperative fixed points can occur only for special initial conditions. B to C: The probability of settling in the cooperative fixed point as a function of the initial frequency of cooperators , A, and initial frequency of soft players,
, B, for different structures of game B is plotted. The replicator-mutator dynamic is solved for 107 different randomly generated initial conditions to derive the probabilities.
Fig 6.
Time evolution in the model with direct interactions.
A and B: The time evolution of different strategies resulted from the replicator-mutator dynamics A and a simulation in a finite population B. (c) and (d): The time evolution of the density of the cooperators ρC (up), and the density of the soft strategies ρd (bottom), resulted from the replicator-mutator dynamics C, and a simulation D. Cooperation favoring moral norms evolves through a rapid dynamical transition. The simulation is performed on a population of size N = 20000, and the mutation rate is ν = 0.005. The initial condition is a random assignment of strategies (for the replicator-mutator dynamics, this implies ρx = 1/8, for all strategies x). The base payoff values, presented in Table 1, are used.
Fig 7.
The direct interaction model with the three archetypal games.
The density of cooperators, ρC, A, the density of soft strategies in game B, ρd, B, the normalized payoff difference of cooperators and defectors in game B, Δπ, (c), and the correlation between the strategy of the individuals in the two games, 〈sAsB〉c, D, as a function of the temptation, T. Here, from top to bottom, the game B is the Snow Drift, the Battle of the Sexes, and the Leader game. The payoff values used for the games are presented in Table 1. The lines show the result of the replicator-mutator dynamics, and the markers show the results of simulations. The solid blue line shows the equilibrium fixed point, which occurs starting from an unbiased initial condition in which the density of all the strategies are equal, and the dashed red line shows the non-equilibrium fixed point, which can occur starting from certain initial conditions. The system is bistable and both a cooperative fixed point with a high level of cooperation A and soft strategies B, and a defective fixed point with a low level of cooperation and soft strategies are possible. In the cooperative phase, cooperators receive a higher payoff from game B C. Moreover, the strategies of individuals show an anti-correlation in the cooperative fixed point D, resulting from the fact that defectors play softly with cooperators and cooperators play hard with defectors in this fixed point. For the simulations, a sample of 80 simulations, in a population of size N = 10000 is used. The simulations start from random initial conditions. In each simulation, the dynamics settle in one of the two fixed points. The markers show the averages, and the error bars show the standard deviation in the sample of simulations that settle in the given fixed point, and the size of markers is proportional to the number of times that the given fixed point occurs. Here, the mutation rate, ν = 0.005. The simulations are run for 20000 time steps, and an average over the last 1000 time steps is taken.
Fig 8.
The reputation-based model with the three archetypal games.
The density of cooperators, ρC, A, the density of soft strategies in game B, ρd, B, the normalized payoff difference of cooperators and defectors in game B, Δπ, B, and the correlation between the strategy of the individuals in the two games, 〈sAsB〉c, D, as a function of the probability of error in inferring the PD strategy of the opponent, η. Here, from top to bottom, the game B is the Snow Drift, the Battle of the Sexes, and the Leader game. The payoff values used for the games are presented in Table 1. The lines show the result of the replicator-mutator dynamics, and the markers show the results of simulations. The solid blue line shows the equilibrium fixed point, which occurs starting from an unbiased initial condition in which the density of all the strategies are equal, and the dashed red line shows the non-equilibrium fixed point, which can occur for certain initial conditions. The system is bistable for small recognition noise, η and both a cooperative fixed point with a high level of cooperation A and soft strategies B, and a defective fixed point with a low level of cooperation and soft strategies are possible. In the cooperative phase, cooperators receive a higher payoff from game B C. Moreover, the strategies of individuals show an anti-correlation in the cooperative fixed point D, resulting from the fact that defectors play softly with cooperators and cooperators play hard with defectors in this fixed point. For the simulations, a sample of 80 simulations, in a population of size N = 10000 is used. The simulations start from random initial conditions. In each simulation, the dynamics settle in one of the two fixed points. The markers show the averages, and the error bars show the standard deviation in the sample of simulations that settle in the given fixed point. The size of the markers is proportional to the number of times that the given fixed point occurs. Here, the mutation rate, ν = 0.005. The simulations are run for 20000 time steps, and an average over the last 1000 time steps is taken.
Fig 9.
The behavior of the models under continuous variation of the structure of game B.
The color plot of the density of cooperators, ρC, in the direct interaction model A, and the reputation-based model B, in the SB − TB plane. The top panels show the result of the replicator-mutator dynamics and the bottom panels show the results of simulations in a population of 1000 individuals. In both cases, an unbiased initial condition (random assignment of strategies) is used. I have set R = 3, S = 0, P = 1, T = 5, RB = 3, and PB = 1. The boundaries of bistability are plotted as well. Below this boundary the dynamic is monostable, settling into a fixed point with a low level of cooperation. Above the boundary, a cooperative fixed point becomes stable and the dynamics become bistable. The two branches of the boundary meet at a critical point, where the transition becomes continuous. A comparison shows finite size effects strongly favor cooperation. Here, η = 0.1, and ν = 0.005.
Fig 10.
Time evolution of the system in a structured population when game B is the Stag Hunt game.
A to C: snapshots of the population during the evolution for different times, t, are presented. (d) and (e): The densities of different strategies as a function of time. Here, ν = 10−3, and game B is the Stag Hunt game. The payoff values of the games are presented in Table 1. The initial population is a uniformly distributed mixture of Cdd and Ddu. This initial condition favors the evolution of cooperation-favoring coordination norms in a mixed population. However, due to the formation of homogeneous blocks in a structured population, cooperation favoring coordination norms are unstable, and the system evolves to a defective state composed of Ddd and Dud types. The population resides on a 200 × 200 first nearest neighbor square lattice with von Neumann connectivity and periodic boundaries. The direct interaction model is used.
Fig 11.
The reputation-based model with three archetypal games in a structured population.
The density of cooperators, ρC, A, the normalized payoff difference of cooperators and defectors in game B, δπ, B, the correlation between the individuals’ strategies in the two games, 〈sAsB〉c, C, and the density of soft strategies in game B, ρd, D, as a function of the probability of error in inferring the PD strategy of the opponent, η, are plotted. The payoff values used for the games are presented in Table 1. Simulations are performed in a population of 40000 individuals residing on a 200 × 200 square lattice with first nearest neighbor von Neumann connectivity and periodic boundaries. The simulations are performed for 6000 time steps, and averages and standard deviations are calculated based on the last 4000 time steps. The simulations start from random initial conditions. Here, ν = 0.005.
Fig 12.
The density of different strategies in the reputation-based model with three archetypal games in a structured population.
The time average density of different strategies, as a function of the probability of error in inferring the PD strategy of the opponent, η, are plotted. The payoff values used for the games are presented in Table 1. Simulations are performed in a population of size 40000 individuals residing on a 200 × 200 square lattice with first nearest neighbor von Neumann connectivity and periodic boundaries. The simulations are performed for 6000 time steps, and averages and standard deviations are calculated based on the last 4000 time steps. The simulations start from random initial conditions. Here, ν = 0.005.
Fig 13.
Strategic response to cooperators and defectors in game B, in the reputation-based model with three archetypal games in the structured population.
The density of strategies who play up with cooperators u(C), down with cooperators d(C), up with defectors, u(D), and down with defectors, d(D), as a function of the probability of error in inferring the PD strategy of the opponent, η, are plotted. The payoff values used for the games are presented in Table 1. Simulations are performed in a population of size 40000 individuals residing on a 200 × 200 square lattice with first nearest neighbor von Neumann connectivity and periodic boundaries. The simulations are performed for 6000 time steps, and averages and standard deviations are calculated based on the last 4000 time steps. The simulations start from random initial conditions. Here, ν = 0.005.
Fig 14.
A to C: snapshots of the population during the evolution for different times, t, are presented. D and E: The densities of different strategies as a function of time. Here, ν = 10−4, and game B is the Snow Drift game. The payoff values of the games are presented in Table 1. The initial population is of type Dud. The population resides on a 400 × 400 first nearest neighbor square lattice with von Neumann connectivity and periodic boundaries.