Utility Evaluation Based on One-To-N Mapping in the Prisoner’s Dilemma Game for Interdependent Networks

In the field of evolutionary game theory, network reciprocity has become an important means to promote the level of promotion within the population system. Recently, the interdependency provides a novel perspective to understand the widespread cooperation behavior in many real-world systems. In previous works, interdependency is often built from the direct or indirect connections between two networks through the one-to-one mapping mode. However, under many realistic scenarios, players may need much more information from many neighboring agents so as to make a more rational decision. Thus, beyond the one-to-one mapping mode, we investigate the cooperation behavior on two interdependent lattices, in which the utility evaluation of a focal player on one lattice may not only concern himself, but also integrate the payoff information of several corresponding players on the other lattice. Large quantities of simulations indicate that the cooperation can be substantially promoted when compared to the traditionally spatial lattices. The cluster formation and phase transition are also analyzed in order to explore the role of interdependent utility coupling in the collective cooperation. Current results are beneficial to deeply understand various mechanisms to foster the cooperation exhibited inside natural, social and engineering systems.


Introduction
The persistence and emergency of cooperation is a widespread phenomenon in the nature and human being society, ranging from single cellular organisms to vertebrates, mammals, and even to populations [1,2]. Yet, according to Darwinian evolutionary theory [3], any behavior that does not contribute to himself will eventually lead to the extinction of cooperation, which is rightly contradictory to many real-world situations. Therefore, deeply comprehending the persuasive cooperation conduct has become a challenging comission, and it is also a long-networks is built and the role of interdependency in the promotion of cooperation is greatly impacted by the coupling way of utility or fitness assessment.
In previous works, the utility coupling is often implemented between one-to-one mapping agents on different networks, for example, Ref. [50] introduces the biased utility coupling for the PGG, in which the individual payoff on one network accounts for α and payoff of corresponding agent on the other networks occupies (1 − α), and finds that the aggregate level of cooperation on two networks is higher than that obtained on one isolated network; but for the PDG in Ref. [53], the utility function U x is formulated as the linear combination of payoffs (P x and P x 0 ) of corresponding agents: U x = αP x + (1 − α)P x 0 , and the results indicate that there exists a critical interdependent factor α C , where the cooperation symmetry will be kept if α < α C and spontaneous symmetry breaking of fraction of cooperators presents itself between different networks provided that α > α C . In Ref. [56], this kind of biased utility is not adopted, but an approach regarding the payoff superposition between different networks is applied, and it is unarguably uncovered that the cooperation phenomenon becomes richer and richer when compared to the cooperation behavior on single networks. In reality, the aforementioned works usually consider the utility computation only between corresponding players on distinct layers, that is, one-to-to mapping. However, under some real-world circumstances, much more information on the payoffs of multiple players from other networks will be beneficial for the focal one to make the decision. Thus, in this study, beyond the one-to-one mapping mode, we present an improved utility evaluation covering the combination of payoffs from corresponding partners on interdependent networks, in which the corresponding player and his nearest neighbors on the other network will be also taken into account when the player's utility is calculated, that is, the game concerning a focal agent will be mapped into himself and multiple partners on the other network. The current work will further help to understand the role of interdependent network reciprocity in the evolution of cooperation within the social and natural systems.

Methods and Materials
The Monte Carlo Simulation (MCS) is utilized to perform the evolution of cooperation behavior, and the system proceeds from two regular lattices with the linear size L, in which either lattice is satisfied with the conditions of periodic boundary. Initially, all players are allocated on the intersections of lattices, and designated as a cooperator (s x = C) or a defector (s x = D) with the equal probability (that is, being a cooperator or defector with the possibility of 50%). Thus, the system size or total number of players (N) is two times of the square of lattice linear size, i.e., N = 2 × L 2 .
After that, starting from a randomly chosen player from any network, the pairwise interaction and payoff computing will be performed according to the prisoner's dilemma as follows where R and P denote the payoff of reward or punishment when two players take the same action: cooperation or defection; and the cooperator gets the sucker's payoff (S), but the defector cannot resist the temptation and obtain the highest payoff when they choose the different strategy; Meanwhile, T > R > P > S and 2R > S + T must be satisfied for the standard PDG model. Since T > R and P > S, it is obvious that to defect is the best option for a player regardless of his opponent's choice, that is, defection becomes the Nash equilibrium of PDG. Without loss of generality, we adopt a class of weak PDG parameter setup: 1 < T = b < 2, R = 1 and P = S = 0, which is firstly applied by Nowak & May [16] and nearly captures all PDG's features during the strategy game. Henceforth, each player (say x) can calculate his or her payoff (P x ) by playing with 4 nearest neighbors as we can only discuss the typical von-Neumann neighborhood setup in this study. Then, the individual utility or fitness will be assessed for the evolutionary iteration, and the focal agent's utility evaluation will not only be related with his own payoff, but also with the average of payoff of players from the other lattice, which include the corresponding partner (say x 0 ) and his 4 immediate neighbors. Among them, player x 0 and his neighbors will also apply the identical method to compute their payoffs according to Eq (1), and their average value " P will be calculated as follows, where N x 0 denotes the von-Neumann neighborhood of player x 0 and i 0 represents one of members in the neighborhood (i.e., one of nearest neighbors of player x 0 ). It is worth mentioning that each agent's payoff can be only obtained by playing with 4 nearest neighbors on the same lattice, not be allowed to directly integrate the interaction between individuals on different lattices. Thus, we consider the focal player's utility as a linear combination of P x and " P in the following way, where α is a tunable parameter which means the extent of average payoff integration into the current utility assessment, without loss of generality, α usually lies between 0 and 1. Different from previous works [50][51][52][53][54][55][56], the utility computation will cover the focal one and multiple partners on the other network as shown in Fig 1, that is, beyond the usual one-to-one mapping assumption for the evolution of cooperation on interdependent networks. Next, the focal player will try to update his current strategy by comparing the utility between himself (say x) and one of nearest neighbors picked up at random (say y), in which player y will acquire his utility in the same method with the focal one according to the abovementioned procedure. The strategy (s x ) of player x will be changed into the strategy (s y ) of player y with the following probability where κ stands for the uncertainty of strategy adoption [57] or the individual irrationality for an agent to make a decision (κ is usually set to be 0.1 unless stated before), U x and U y denote the utilities of players x and y, respectively. Finally, one full MCS step will be completed provided that each agent will have a chance to update his strategy on average. In the following section, the lattice configuration is set to be 2 × 200 × 200, and larger lattice size (e.g., L = 300 or L = 400) is also tested and qualitatively similar results are obtained. Moreover, all numerical simulation results will be averaged over 20 independent runs for the same parameter setup.

Results and Analysis
Firstly, in order to deeply explore the role of utility coupling, we provide the evolution of fraction of cooperators at each time step for a fixed defection parameter (b = 1.07) in Fig 2, where the tunable parameter (α) can be set as 0, 0.2, 0.4, 0.6, 0.8 and 1.0, respectively. On the one hand, the simulation results convincingly turned out that the cooperation behavior can be greatly improved on account of the introduction of utility coupling; On the other hand, we can observe that the behavior of F C (t) inside two panels [panel (a) and (b)] is fully identical, that is, the cooperation symmetry always holds for various tunable parameters α. Moreover, for the first 10 time steps, we divide one MCS time step into 1000 sub-steps so that the evolutionary process of F C (t) at each time step can be carefully scrutinized. As we can see, the time course of cooperation can be divided into two evident phases: the enduring (END) period and the expanding (EXP) period. Initially, in the END period, the fraction of cooperators will gradually reduced regardless of whether the utility coupling exists, which shows that the defectors will dominate the population since the Nash equilibrium of PDG is the defection. However, after several steps, the cooperation exhibits a small bit different scenario and the EXP period emerges. From this period, the cooperators begin to win the superiority over the defectors for the utility coupling takes effect. Among them, α = 0 or α = 0.2, the eventual fate of the whole population is the full defection, which shows that only the spatial reciprocity or the very weak utility coupling is not enough to support the evolution of cooperation. However, α becomes larger and larger, for example, α is set to be equal to or higher than 0.4, the fraction of cooperators can be finally maintained at a stable level since the cooperators can organize into compact clusters to resist the invasion of defectors (which will be further illustrated in the following paragraph). In the meantime, the larger the tunable parameter α, the higher the stationary fraction of cooperators, and the current results are also consistent with those in Fig 3. Thus, the utility coupling between two interdependent lattices has altered the behavior of cooperation by a large margin when compared to the traditionally spatial lattice setup (i.e., α = 0).
Next, since the symmetry for the fraction of cooperators can be well kept on two panels under various α, we will record the average value of cooperator's density at the stationary state on two lattices as the fraction of cooperators (F C ) among the whole population. Here, we depict F C as a function of temptation to defect (b) in Fig 3 for various tunable parameters to check the role of utility coupling under the one-to-N mapping condition. It can be clearly observed that the level of cooperation can be generally promoted with the introduction of utility coupling from the corresponding partners on the other network. Among them, when α is equal to 0, the focal player's utility calculation does not refer to any other partner, the current model will be equivalent to the traditional PDG model and our simulation results are also identical with previous works. However, when α > 0, the focal player computes his utility by considering the average value of corresponding partners through the 1-to-N mapping, and the numerical simulations unamgiguously indicate that F C can be elevated as α increases. Meanwhile, the larger the adjustable parameter α, the higher the fraction of cooperators F C . As an example, α = 0., the fraction of cooperators still arrives at around 60% when b = 1.0375 where the cooperation becomes extinct in the standard PDG, the maximum defection parameter (b m ) leading to the extinction of cooperation arrives at b = 1.08; Of particular note is that the extinction threshold b m can be up to 1.16, which is often extra-ordinarily high as compared to the standard spatial PDG. From this point, only the spatial reciprocity can be not enough to create this kind of favorable situations to foster the cooperative individuals. Accordingly, the exterior information obtained from the other network can be added into the current utility evaluation, the focal agent will become much more rational for the strategy choice since he can hold the richer information to aid his decision making. Taking together, F C becomes higher and higher as the tunable parameter α increases under the identical defection parameter b since the novel utility coupling has been added into the current model.
Regarding the competition properties between cooperators and defectors during the evolution of system status, how the cooperators accumulated the advantage over the defectors is still to be resolved. We make a preliminary attempt to check the formation of cooperative clusters by providing the snapshots at several different time steps for a specified tunable parameter α = 0.8 in Fig 4. Here, the upper row panels characterize the snapshots on one lattice, and from left [panel (a)] to right [panel (e)], the time step is set to be MCS = 0, 10, 100, 1000, 50000,  Utility Evaluation Based on One-To-N Mapping in the PDG for Interdependent Networks respectively. It can be shown that the strategy of agents is randomly assigned at the initial time [panel (a)], hence the cooperators or defectors nearly occupy the equal space and they cannot be discerned visually. After 9 steps, the defectors obtained the superior position during the game playing and the light gray dots obviously occupy most of space in panel (b). However, some compact and cooperative clusters are sporadically organized so that the orange dots still exist within the population [panel (b)]. In the following steps [from panel (c) to (e)], the cooperators gradually accumulate the evolutionary advantage over defectors during the competition between them and eventually arrive at the stationary state. Accordingly, the corresponding snapshots on the lower panels depict the distribution of cooperators and defectors on the other lattice, and the identical evolving tendency can be observed. Furthermore, the similar pattern analysis is performed for another tunable parameter α = 0.2 in Fig 5, where the snapshots are taken at MCS = 0, 10, 100, 200 and 1000 (since the cooperators are fully extinct after 1000 steps), and it can be examined that the cooperators will lose the superiority during the process of game playing since the extent of utility coupling from the other lattice is not strong enough. At the same time, the cooperation symmetry can be kept substantially well between the corresponding upper and lower panels, which is also consistent with results displayed in Fig 2. To further account for the origin favoring the cooperation in the current PDG model, Figs 6 and 7 depict the evolution of characteristic snapshots for two different utility coupling strength α = 0.8 and α = 0.2, where the initial cooperators are preset in the middle of lattice and defectors are evenly distributed on both sides of lattice. In Fig 6, the coupling strength α is set to be 0.8, and all other simulation parameters are also wholly same as those in Fig 4. Two rows denote the setup of two-layered lattices, and 5 columns characterize the snapshots at different MCS time steps which are, from left to right, MCS = 0, 10, 100, 1000, 50000, respectively. Although the initial distribution of cooperators and defectors is utterly different from that in Fig 4, the evolutionary process is analogous, and cooperators can gradually organize into the compact clusters with the help of utility evaluation borrowing from the payoffs of players on the other lattice, to defend from the exploitation of defectors and lead to the situation in which two strategies can coexist at the stationary state. Henceforth, the final evolutionary outcome is identical with the case of randomly initial deployment. Similarly, in Fig 7, α = 0.2 is not enough to support the persistence of cooperation and free-riding from defectors thrives so that the cooperators cannot resist the invasion of defectors and the cooperators are finally extinct, the evolution of snapshot is also akin to Fig 5 where the snapshots are taken at MCS = 0, 10, 100, 200 and 1000 (from left to right). In addition, the evolutionary process still maintains the symmetry of cooperation between two lattices. Taking together, the collective cooperation evolution in the whole population is totally independent of their initial setup on two lattices, and the strength of utility coupling determines their final fate.
The impact of utility coupling between two interdependent lattices can be examined in depth by checking the full b − κ phase diagram in Fig 8, which fully demonstrates the robustness of cooperation against the noise strength (i.e., the uncertainty of strategy selection κ), unlike some previous works where the noise may substantially vary the collective cooperation ratio with the whole population [58]. In Fig 8, and from panel (a) to (d), the tunable parameter α is set to be 0, 0.4, 0.8 and 1.0, respectively, at the same time the pure cooperating (defecting) region is denoted by the capital C (D), while C + D characterizes the coexistence region containing cooperators and defectors within systems, and the upper triangle (crossed circle) symbol means the critical value standing for the transition from pure cooperators to mixed cooperators and defectors (mixed ones to pure defectors). From Fig 8, the general trend is that, as α increases, the pure defecting regions is greatly compressed, and the mixing or coexistence regions is largely broadened, but the pure cooperating regions is almost kept to be unchanged. Among them, in panel (a), α is equal to 0, which implies that the system is reduced to two traditional and independent lattice systems and our results reproduce the classical bellshaped ones regarding the phase transition, and current phase diagrams are consistent with many previous results. However, when α is greater than 0, and the utility evaluation need to consider the impact of corresponding partners on the other lattice, henceforth render that the phase diagram differs a little from the classical ones and the bell-shaped curve indicating the transition between D and C + D disappears, which can be observed from panel (b) to (d). Meanwhile, as κ lies between 0.01 and 1.0, it is easier for the cooperators and defectors to coexist; while κ > 1.0, the range of temptation b for them to coexist becomes very narrow, and it is particular for κ > 3.0, the coexistence between cooperators and defectors becomes impossible, that is, the system could only get into the full defection or cooperation status as b varies. Since the very large noise strength (κ) leads to the fact that the probability to select the cooperation or defection is almost not influenced by the utility difference and the system tends to trap into Utility Evaluation Based on One-To-N Mapping in the PDG for Interdependent Networks one of two states, full cooperation or complete defection. Taking together, introducing the payoff coupling into the individual utility evaluation will greatly change the behaviors of phase transition between the cooperation and defection on the square lattice.

Conclusions and Discussion
In summary, we discuss the evolution of cooperation behavior on two interdependent lattices through the utility coupling between corresponding players. In many previous works, this type of utility coupling is often implemented with one-to-one mode, that is, the evaluation of one player's utility on one lattice is only correlated with his own payoff and one corresponding partner's payoff on the other lattice. In reality, the individual decision may be dependent on many factors including his own or other individual's payoffs. Thus, in this paper, beyond oneto-one pattern, we propose a novel utility coupling way based on himself and several corresponding players on the other networks, that is, one-to-N mapping mode, and the numerical simulation results indicate that the cooperation behavior on the spatial lattices can be largely modified. Firstly, the total level of collective cooperation at the stationary state can be greatly promoted as the extent of payoff integration into the individual utility evaluation (i.e., tunable parameter α) increases, which means that the larger the tunable parameter α, the higher the level of stationary state. Secondly, the coupling of utility assessment concerning multiple agents on the other lattice is beneficial for the focal player to make a rational choice and helps to create the cooperative clusters to defend the invasion of defectors. Lastly, this novel utility coupling also alters the properties regarding the phase transition when compared to the classical spatial lattices. The current results further enrich the understanding of widespread cooperation phenomena within many natural, social and even engineering systems, especially aiding to illustrate the role of interdependency and network reciprocity in the promotion of cooperation.
The future works will extend the current one-to-N mapping mode into other spatial game models (e.g., snowdrift game and public goods game) or other heterogenous topologies (i.e., interdependent random graphs and scale free networks), to further explore whether the cooperation level can be enhanced within these scenarios by the one-to-N mapping mode. Furthermore, the evolution of cooperation within three or more networks can also deeply investigated since many real-world systems are often interwoven through multiple subsystems.