Direct Sum Matrix Game with Prisoner's Dilemma and Snowdrift Game

A direct sum form is proposed for constructing a composite game from two games, prisoner's dilemma and snowdrift game. This kind of direct sum form game is called a multiple roles game. The replicator dynamics of the multiple roles game with will-mixed populations is explored. The dynamical behaviors on square lattice are investigated by numerical simulation. It is found that the dynamical behaviors of population on square lattice depend on the mixing proportion of the two simple games. Mixing SD activities to pure PD population inhibits the proportion of cooperators in PD, and mixing PD activities to pure SD population stimulates the proportion of cooperators in SD. Besides spatial reciprocity, our results show that there are roles reciprocities between different types of individuals.


Introduction
As mathematical framework, evolutionary game theory has been used to model evolutions in social, economical and biological systems widely [1][2][3]. One of the fundamental problems in this theory is the evolution and maintenance of cooperation among selfish individuals [4][5][6][7][8]. Two prominent mathematical metaphors have attracted most attention in theoretical and experimental studies of cooperation, the prisoner's dilemma and the snowdrift games [9][10][11]. In both games, each player decides whether to cooperate or defect. In the prisoner's dilemma(PD), cooperation of the players results to the highest payoff which is equally shared among the two players, yet individual defectors will do better if the opponent decides to cooperate. Since the selfish players want to maximize their own income and they both decide to defect. None of them gets a profit and instead of equally sharing the payoff received by mutual cooperation, they end up almost emptyhanded. The snowdrift(SD) game has different payoffs matrix compare with the PD. It is an interesting alternative for the study of cooperation, and individuals in such game can gain access to benefits for the pair at one individual cost. Cooperators have to bear the costs whereas defectors are not. Both games represent social dilemmas [12], in which defectors are prone to exploit cooperators, and have an evolutionary advantage over cooperators in populations. But cooperation is widespread in the real world and required for many levels of biological organization ranging from genes to groups of animals. Cooperation is also the decisive organizing principle of human societies. Therefore, the underlying mechanisms of cooperation are much needed and have been investigated extensively in different contexts, such as kin selection [13], direct reciprocity [4,9,[14][15][16], indirect reciprocity [17][18][19], group selection [20][21][22] and network reciprocity [23,24], which are summarized as five rules [25]. The network reciprocity could also be regarded as a generalization of spatial reciprocity.
To study the dynamical behaviors of a spatial game, the following points should be considered: the spatial structure on which the game runs, the interactive ways of individuals, the updating and mutation rules. The most common spatial structure is the square lattice [23], in which the cooperators can form clusters to protect themselves against the exploitation by defectors. Small-world networks, scale-free graphs and evolving networks may also serve as the spatial structures of a game [26]. By adopting coevolutionary rules, coevolutionary games can rearrange network structures [27]. For instance, Chen et al. [28] have proposed a coevolutionary rule in spatial public goods games. The interactive ways are determined by payoff functions, which are known as payoff matrices in matrix games, and the scope of opponents, such as Von Neumann and Moore neighborhood in square lattice, pairwise interaction, group interaction, etc. Strategy updating and mutation rules may also affect evolutionary dynamics. Some examples include the birth-death and imitation rule [29], the proportional imitation rule [30], the reinforcement learning adoption rule [31], or the Fermi rule [32]. Wang et al. [33] have recently considered an adaptive strategy-adoption rule in which the focal player evaluates its strategy by comparing the average payoff of each strategy in the neighborhood. They have shown that the survivability of cooperators has a significant increment in contrast with that of pairwise strategy updating. Based on winstay-lose-shift rule, Liu et al. [34] introduced a win-stay-lose-learn strategy updating rule in spatial prisoner's dilemma game, where the focal player attempts to update her strategy only when her payoff is less than her aspiration.
Most previous works about PD and SD game have been referred to the consideration of the individual interaction being one fold, which means that all interactions adopt either PD or SD payoff matrix [32][33][34][35]. That is all individuals participate in the same kind of game. However, the situation that individuals involved in multiple activities is happened frequently in economic and social activities or biological behaviors. In the context of game theory, individuals participate in multiple games and act different roles in these games. Such games can be called multiple roles game(MRG).
In the present work, a simple MRG model with PD and SG payoff matrix is constructed. In the modeling approach section, we describe the model for multiple roles games with prisoner's dilemma and snowdrift game. In the results section, the replicator dynamics of the multiple roles game with will-mixed populations is explored, and the dynamical behaviors on square lattice are calculated in a numerical way over the parameter space. In the last section, conclusions are presented and the potential clues to other general cases are predicated.

Modeling Approach
To elaborate the idea of MRG, let's focus on two-strategy games. The two strategies can be denoted as cooperation and defection, abbreviated as C and D, respectively. The payoff matrix has the form Here, a player with C strategy obtains profit R from another C player, S from a D player. Similarly, a D player obtains T from a C player, P from another D player. The Prisoner's dilemma(PD) game starts on the condition as following:

TwRwPwS: ð2Þ
The relationship RwP implies that mutual cooperation is superior to mutual defection, while the relationships TwR and PwS imply that defection is the dominant strategy regardless of opponent's strategy. The SD game has a different condition as shown below: In contrast to PD, the best strategy now depends on the opponent's strategy. The relationship TwR implies that cooperation is better if the opponent defects, while SwP implies that defection is better if the opponent cooperates. The general payoff matrix form(1) has too many parameters to be analyzed. To prevent this kind of situation, the payoff matrix form with two parameters is given for two-strategy game as following [36,37], with {1vSv1 and 0vTv2. If Tw1w0wS, (4) indicates PD payoff matrix, and if Tw1wSw0, it indicates SD payoff matrix. If a MRG model with PD and SD is constructed with the payoff matrix (4), it would have four parameters which will lead to complexity of analysis. Therefore, it is necessary to construct single parameter payoff matrices of PD and SD. One can rescale the payoff matrix (4) to single parameter forms [10,11,35,38] as shown in Eq.(5) for the PD game, and for the SD game. Both u and v are constrained to the interval ½0,1.
Obviously, (5) and (6) satisfy the corresponding inequalities (2) and (3) respectively. Now consider a population where each individual has two identities: prisoner for PD game and driver for SD game. Each individual participates in two games simultaneously. When two individuals interplay, the process is divided into two steps. In the first step, they have to determine what kind of game they will play, and in the second step, they play the selected game according to their own strategy. Let us assume that in a pairwise interplay they choose PD game with probability p, SD game with probability 1{p. In this way, a simply MRG model is constructed (Fig. 1). The left panel of Fig. 1 shows that there are two channels of interaction between the individual A and B, the PD channel with probability p and the SD channel with probability 1{p. The right panel of Fig. 1 shows the details of the payoff matrix of MRG. Here we denote strategies with capital letters C and D in PD game, and lowercase letters c and d in SD game. Formally, an extended payoff matrix M Ex can be obtained by direct sum for the MRG, Since it is impossible that an individual who opts for PD game interacts with the individual who has made a choice of SD game with c or d strategy, we express these cases with two sub-matrices 0 in (7).
To construct a spatial model of MRG, let's place each individual on one cell in a two-dimensional L|L square lattice with periodic boundary conditions being used. There are no empty cells on lattice. An individual interacts with its Moore neighborhoods, and its total payoff is the sum of payoffs obtained from its all neighbors. The population is asynchronously updated by the rule known as the replicator rule or proportional imitation rule [30,36,39]. Randomly selecting a focal individual i, let p i denotes its total payoff, s i its category, or its strategy set. Individual i updates its strategy set by comparing its total payoff p i to the total payoff p j of a randomly selected neighbor j. The focal individual i adopts j's strategy set s j with a probability P ij proportional to the payoff difference, provided that p i vp j . So the probability P ij is given by where W denotes a normalization constant to ensure P ij [½0,1. Here where k~8 represents the number of i's Moore neighbors. After updating its strategy set, the payoff of the focal individual i with that of its all neighbors are reset to zero. Note that the current MRG model does not take into account mutations in updating strategy sets, and individuals propagate every strategy of its each role to the next generation perfectly.

Replicator dynamics of the multiple roles game
According to the strategies individuals adopt, they can be divided into four categories, which can be recorded as Cc, Cd, Dc, Dd. With this convention, Cd means that the individual adopts strategy of cooperation in PD games, and defection in SD games, and the other three categories are similar to Cd. In the replicator dynamics, x 1 , x 2 , x 3 , x 4 represent the fractions of Cc, Cd, Dc, Dd respectively. Without arising confusions, sometimes Cc, Cd, Dc, Dd also represent the fractions of the corresponding types of the individuals. The payoff matrix for the four strategies is: The deterministic evolutionary game dynamics are given by the replicator equation [40,41] , it is not difficult to get eight equilibrium points: (1,0,0,0), (0,1,0,0), (0,0,1,0), (0,0,0,1), (1{v,v,0,0), (0,0,1{v,v), (1{ p 1{p u{v,0,0, p 1{p uzv), (0,v{ p 1{p u,1{vz p 1{p u,0). Apparently, only the last two are related to the parameter p. Consideration of the constraint 0ƒx i ƒ1, the seventh and eighth are only valid as they meet criterion 0ƒpuz(1{p)vƒ1{p and 0ƒ(1{p)v{puƒ1{p, respectively. It shows that the mixed probability p of PD and SD can influence the number of the equilibrium points.

Effect of the mixed probability p on population structure
Simulation runs on a 51|51 lattice with four categories individuals initial randomly uniform distributing over it equiprobably. To ensure a correct convergence, 11000 time steps are employed [36], where one time step means each individual updates its strategies one time averagely. Preliminary simulation results shows that, for p from 0 to 1 step 0:05, the two-dimensional u{v parameter plane can be divided into four areas roughly as shown in Fig. 2    indicate the population structure with pure SD game. The right endpoints of the curves correspond to the case p~1, which indicate the population structure with pure PD game. Comparing with these reference endpoints the average fractions of Dc and Dd form two convex curves, and the average fractions of Cc and Cd form two concave curves. It is quite obvious that from the global perspective, mixing SD and PD games will stimulate Dc and Dd, and inhibit Cc and Cd individuals.
For comparison with the pure PD and SD games, let PDC represents the average fraction of players with C strategy in PD games, which equals to CczCd, and SDc represents the average fraction of players with c strategy in SD games, which equals to CczDc. The average fractions of PDC and SDc are given in Fig. 4 as functions of p. The SDc curve increases with the increasing mixed probability p, which indicates cooperators of SD increase as p increases. It means that mixing PD game in SD population can stimulate cooperate fraction in SD. The average fraction of PDc forms a concave curve and stills bottoming out in the area p~0:2 to 0:4. From the view of PD, mixing SD game in PD population tends to inhibit the cooperate C strategy in PD game. The extent of inhibition reaches a maximum at p~0:2 to 0:4. The population fractions distributing in the phase plane can reveal more details of the evolutionary dynamics. Fig. 5 shows the fraction of SDc distributing in u{v parameter plane. As a frame of reference, Fig. 5A(p~0:0) represents the case of pure SD population, in which the SDc individuals are mainly distributed in the low value zone of v, or the bottom of u{v plane, and have no association with the parameter u. Fig. 5B,C,D(p~0:3,0:6,0:9 respectively) show that with the increase of p, the SDc region expands slowly to the zone of the larger v value. This is in consistency with the SDc curve in Fig. 4. Fig. 6 shows the the fraction of PDC distributing in u{v plane. As a frame of reference, Fig. 6A(p~1:0) represents the case of pure PD population, in which the PDC individuals are mainly distributed in the low value zone of u, or the left of u{v plane, and have no   association with the parameter v. Fig. 6B,C(p~0:7,0:4 respectively) show that with the decrease of p, the PDC region compresses to the zone of the smaller u value inhomogeneously. But one should note that for p~0:1 (Fig. 6 D), the PDC region has an anomalous expansion comparing with that of p~0:4( Fig. 6 C). This kind of anomalous expansion is consistent with the curve PDC in Fig. 4 which decreases first and then increases with p from 1:0 to 0. To explore the origin of the PDC anomalous expansion at lower p values, it should be noted that PDC~CczCd. The fractions of Cc and Cd in u{v parameter plane are shown in Fig. 7 A-F for several different p values. From Fig. 7 A,D we know that the anomalous expansion of PDC for p~0:1 is originated from that of Cd.
It is well known that if there is only one kind of game in population, such as PD or SD game, there exists effect of spatial reciprocity in lattice space [23,24]. But if an individual in lattice space is drawn into multiple games, its different strategies in each game may lead up to the situation: what one loses on the swings, she gets back on the roundabouts. In the current situation, it becomes that what one loses as a prisoner in PD, she gets back as a driver in SD, or vice verse. This phenomenon can be called as roles reciprocity. This effect superimposes on that of spatial reciprocity to influence the fractions of individuals on payoff matrix parameter space u{v plane.
In the PD and SD MRG case, from comparing the fractions of Cd and Dc (Fig. 7 D-F, G-I) in u{v parameter plane, it is found that the regions of Cd and Dc extend to the side of each other. This phenomenon is a typical demonstration of roles reciprocity. Roles reciprocity strengthens the survival ability of Cd and Dc players. Although individuals with strategies Cd or Dc have no maximal fractions in four categories (Fig. 3), they have more tolerance in unfavorable parameters (Fig. 4 CdzDc curve) in appropriate parameter intervals of p, u and v.
To illustrate the effect of roles reciprocity, we cancel the interaction between the Cd and Dc individuals for simulation with p~0:1. Comparing with roles reciprocity(the case of existing interaction between the Cd and Dc), the average fractions of Cd and Dc descend in the absence of interaction between them (Fig. 8).
In u{v parameter plane, the effect of roles reciprocity become ever more evident (Fig. 9). Without the roles reciprocity, the cross domain of Cd and Dc vanished completely (Fig. 9 B). It can be say that Cd and Dc individuals establish a ecological chain in a small parameter region with about u[½0,0:06 and v[½0:3,0:7 at p~0:1. The fact that the region is too small reflects the frangibility of that toy ecosystem.

Discussion
In the real world, it is common that an agent acts as multiple identities. It can be say that the agents participate in multiple games and have multiple roles, and it can be called multiple roles game, or MRG. To imitate these cases, a evolutionary game model is introduced, in which each agent has two identities, one for PD game and another for SD game. A parameter p is introduced to indicate the probability to select PD game for each pair interaction. It shows that the mixed probability p of PD and SD can influence the number of the equilibrium points in the deterministic evolutionary game dynamics. In the spatial MRG, the agents are placed on patches in a two-dimensional lattice with periodic boundary conditions being used. The number simulation shows that mixing SD and PD games will stimulate Dc and Dd, and inhibit Cc and Cd individuals from the global perspective. Comparison with the pure PD or SD games shows that SD cooperators ratio(SDc~CczDc) increases as p increase, and PD cooperators ratio(PDC~CczCd) is inhibited for p[(0,1).
The population fraction distributions in the u{v phase plane reveals that besides spatial reciprocity, there exist roles reciprocity, which means for some kind of agents their multiple identities will have help them to obtain benefits in population. In the MRG model with PD and SD games, because of the roles reciprocity, Cd and Dc individuals form a ecological chain in appropriate parameter region. These toy ecosystems are fragile on account of too small of the appropriate parameter areas.
To configure our multiple roles game model, we introduced direct sum form to construct the payoff matrix from simple game's payoff matrix. This method gives a way to construct complex MRG model from simple sub games. In game theory experiments, participators often influence by various factors. Or in biology, a group may have more than one function. If one need to construct a game theory frame in these cases, the MRG modeling method could be an option to consider.