Strategy intervention for the evolution of fairness

The ‘irrational’ preference for fairness has attracted increasing attention. Although previous studies have focused on the effects of spitefulness on the evolution of fairness, they did not consider non-monotonic rejections shown in behavioral experiments. In this paper, we introduce a non-monotonic rejection in an evolutionary model of the Ultimatum Game. We propose strategy intervention to study the evolution of fairness in general structured populations. By sequentially adding five strategies into the competition between a fair strategy and a selfish strategy, we arrive at the following conclusions. First, the evolution of fairness is inhibited by altruism, but it is promoted by spitefulness. Second, the non-monotonic rejection helps fairness overcome selfishness. Particularly for group-structured populations, we analytically investigate how fairness, selfishness, altruism, and spitefulness are affected by population size, mutation, and migration in the competition among seven strategies. Our results may provide important insights into understanding the evolutionary origin of fairness.


Introduction
In the last thirty years, there has been substantial progress in understanding the evolution of fairness by studying the Ultimatum Game (UG). In a typical UG, a proposer and a responder allocate a fixed sum of money. The proposer makes a proposal about how to allocate the money and the responder decides to accept the proposal or not. If the proposal is accepted, both of them are paid accordingly. Otherwise, neither of them is paid. If each player in the game tries to maximize his own payoff, the responder should accept any non-zero offers [1]. In such case, the proposer should offer the minimum allowable proportion of the sum to the responder. However, this prediction contradicts with nearly all experimental observations, in which responders usually reject offers less than 30% of the sum and the most common offer of proposers is 50% of the sum (see reviews [2,3]).
Here, we denote an offer less than 50% of the sum as a low offer and an offer more than 50% as a high offer. Most experiments which are confined in typical student populations have shown that players reject low offers and rarely reject high offers. However, some experiments which are confined in non-student populations have found the rejection of high offers [4][5][6][7][8]. The rejection of an offer can be regarded as the behavior of costly punishment [9][10][11], because the responder voluntarily suffers the cost of the 'offer' to cause the proposer not to obtain the remaining amount. Some investigations argue that such rejection is motivated by individuals' prosocial preference for fairness [12,13]. The preference for spitefulness has been found to be another potential motivation [14][15][16]. In this paper, we will pay attention to the effects of spitefulness on the evolution of fairness.
Recently, the effects of spitefulness on the evolution of fairness have been studied by one evolutionary game model [17]. This model has focused on the replicator dynamics of four discrete strategies in infinite populations, which represent selfishness, fairness, altruism, and spitefulness, respectively. It has implicitly shown spitefulness promotes the evolution of fairness in infinite populations under certain conditions. Unlike the previous work, we will introduce non-monotonic rejections which reject low offers and high offers in finite populations, which have been found in behavioral experiments [6,7]. Besides the four strategies in the previous model, our model will adopt three new strategies, which represent altruism, spitefulness, and fairness, respectively. Moreover, we will use strategy intervention to explicitly study how spitefulness and altruism influence the evolution of fairness in finite populations. Specifically, we start by studying the competition between a selfish strategy and a fair strategy, and then add five strategies to them in sequence. Our study will go from the two-strategy competition to the seven-strategy competition. In particular, the four-strategy competition in our model can recover to the previous model but with finite populations.
We will analyze our model based on the well-known Tarnita-σ condition [46], which is a simple and general condition for strategy k 2 {1, 2, Á Á Á, S} to be favored by natural selection. Specifically, the average frequency of strategy k over the stationary distribution is greater than 1/S under weak selection if and only if where a ij is the payoff of an individual using strategy i when interacting with an individual using strategy j, " The condition implicates that strategy selection is simply the sum of two competition terms. One is evaluated in states of pairwise strategies and the other one is evaluated in the state of all strategies with the same frequency. The evolutionary process has a great number of possible states, each of which should indicate strategies and locations of all individuals. Therefore, it is surprising for the condition to be so simple. The condition holds for a large class of population structures and update rules satisfying some mild assumptions. The population structure could involve interactions between neighbor nodes on a graph [47], or between individuals of the same group, phenotype, or set [48][49][50]. The update rule could be the Moran process, the Wright-Fisher process, or the pairwise comparison process.
We will investigate the impacts of altruism and spitefulness on the evolution of fairness in general structured populations. In such case, the unknown parameters Γ 1 , Γ 2 , and Γ 3 in the Tarnita-σ condition do not have to be calculated. Particularly for group-structured populations, we will quantitatively analyze how the evolution of selfishness, fairness, altruism, and spitefulness is influenced by population size, mutation, and migration, respectively. Moreover, we will compare the results between the Moran process and the Wright-Fisher process. A necessary premise for these analyses is the calculation of Γ 1 , Γ 2 , and Γ 3 . We will calculate them based on the results in the previous literature [51], which have been used to analyze the multiple-strategy competition in general models. From a long-term perspective, the group-structured population without migration evolves just like the one-group population, because the absorbing state is that all individuals are located in the same group. The one-group population can also be seen as the well-mixed population. Accordingly, our results in the absence of migration are appropriate for well-mixed populations.

Model and method
In the UG, the proposer has to divide a certain amount of money, say 1, with the responder who can accept or reject the split. If the responder accepts the split, the money is shared accordingly; if not, both individuals remain empty handed. We focus on a simplified version of the UG in Fig 1. Proposers have three kinds of offers: the fair offer (0.5), a low offer (p > 0, p ! 0), and a high offer (0.5 + p). The first one is an equal offer for the proposer and the responder, whereas the latter two are unequal offers. Some experiments have found that many responders use non-monotonic rejections which reject low offers and high offers [6,7]. Besides the non-monotonic rejection, we assume that responders have three kinds of veto power: accept any offers (accept any), reject the low offer (reject low), and reject the two unequal offers (reject unequal). A strategy should denote what choice to make as a proposer and what choice to make as a responder. Seven representative discrete strategies will be used: S 1 = (p, accept any), S 2 = (0.5, reject low), S 3 = (0.5, accept any), S 4 = (p, reject low), S 5 = (0.5 + p, accept any), S 6 = (p, reject unequal), and S 7 = (0.5, reject unequal).
The strategies S 1 , S 3 , and S 5 have a common point, i.e., accepting the low offer as the responder. Such behavior has been found to be selfishness or altruism by a behavioral experiment [52]. Selfishness is a desire to maximize one's own payoff. The strategy S 1 displays selfishness by offering very little as the proposer and accepting any offers as the responder. Altruism is a desire to be kind to opponents using any strategies. The strategies S 3 and S 5 display altruism by giving the opponent a non-low offer and accepting any offers of the opponent. The strategies S 2 , S 7 , S 4 , and S 6 have the common behavior as responders, i.e., rejecting the low offer. This behavior has been found to be fairness or spitefulness by a behavioral experiment [15]. Fairness is a desire to sacrifice one's own payoff to pursue fairness. For S 2 and S 7 , the proposer gives up his priority to play fair to the opponent and the responder punishes the unequal offer of the opponent at a cost. Therefore, the strategies S 2 and S 7 represent fairness. Spitefulness is a desire to obtain an advantageous standing over one's opponent. For S 4 , the proposer is always trying to get a higher payoff than the opponent and the responder will not leave his own payoff below his opponent's. Therefore, the strategy S 4 represents spitefulness. S 6 is a complex strategy because it does not seem to fully fit in one of the above four preference types. It exhibits spitefulness from the perspective of the proposer, and it exhibits fairness from the perspective of the responder. Therefore, the strategy S 6 represents spitefulness and fairness. If a person causes others to suffer obvious loss by showing spitefulness and cannot help others obtain obvious benefit by showing fairness, we usually remember his spitefulness and neglect his fairness. In this sense, we label S 6 as spitefulness in this paper.
We will first consider general structural populations satisfying the Tarnita-σ condition. A given interaction is comprised of two games, in which two individuals play the roles of proposer and responder alternately. The payoff matrix for the simplified version of the UG is shown in Table 1. All interactions accumulate the payoff of individual i, p i , and further his fitness, f i = 1 + δp i , where δ is the selection intensity. Mutation may occur during reproduction. With probability u, mutation occurs on one of the offspring, and then he equi-probably chooses one of the possible strategies. Otherwise, the offspring inherits the strategy of his parent.
We will then consider group-structured populations. Specifically, all individuals are distributed over M groups which are located in a circle, and an individual only interacts with the others of the same group. The Moran process and the Wright-Fisher process will be studied, respectively. In the Moran process, all individuals compete to reproduce one offspring proportional to their fitness, and then one individual is equi-probably chosen from the whole population to die. In the Wright-Fisher process, all individuals compete to reproduce N (population size) offspring proportional to their fitness, and then they all are replaced by the newborn offspring. Besides mutation, migration is also introduced in our model. With probability 1 − v, the offspring remains in his parent's group. Otherwise, he moves to a new group according to the migration pattern of the range r shown in Fig 1. For the migration range r, all possible displacements generated by a single-step migration are contained in the set O(r) = {1, 2, Á Á Á, r}. We assume that all elements of O(r) are performed equi-probably.
The comparison among selfishness, fairness, altruism, and spitefulness is based on f 1 , f 2 , f 3 , and f 4 in Table 2. Take selfishness and fairness for example. Selfishness has an advantage over aÞ. When selfishness, fairness, altruism, or spitefulness is exhibited by a single strategy, we assume f i = F i with i 2 {1, 2, 3, 4}. Under weak selection, all possible strategies have similar frequencies. To guarantee that the comparison proceeds on the same scale, we assume f 2 = (F 2 + F 7 )/2, f 3 = (F 3 + F 5 )/2, or f 4 = (F 4 + F 6 )/2 when fairness, altruism, or spitefulness is exhibited by two strategies.

Structured populations satisfying the Tarnita-σ condition
When the selfish strategy S 1 and the fair strategy S 2 coexist in the population, they compete equally with each other, i.e., f 1 = f 2 . This is because their payoffs are identical for all population states. By sequentially adding S 3 , S 4 , S 5 , S 6 , and S 7 , we show how the competition between selfishness and fairness is influenced by altruism and spitefulness in Fig 2. When the altruistic strategy S 3 with the fair offer is introduced into the population, the selfish strategy S 1 gains an advantage over the fair strategy S 2 , i.e., f 1 > f 2 . This advantage can be removed, i.e., f 1 = f 2 , by introducing the spiteful strategy S 4 which rejects the low offer. When we continue to introduce the altruistic strategy S 5 with the high offer, the advantage of S 1 over S 2 shows up again. Then the advantage is removed when we continue to introduce the spiteful strategy S 6 which rejects unequal offers. Accordingly, the evolution of fairness is inhibited by altruism, but it is promoted by spitefulness. The reason is twofold: Compared with S 1 , S 2 obtains less payoffs from Three-strategy competition is the competition of s 1 , s 2 , s 3 , four-strategy competition is the competition of s 1 , s 2 , s 3 , s 4 , five-strategy competition is the competition of s 1 ,  Strategy intervention for the evolution of fairness two altruistic strategies and more payoffs from two spiteful strategies. It can be understood intuitively by comparing the row sum of S 1 with that of S 2 in Table 1. When the fair strategy S 7 which rejects unequal offers is introduced, the total row sum of S 2 and S 7 is greater than twice of S 1 's row sum in Table 1. Therefore, fairness which is measured by the average of S 2 and S 7 first gains an advantage over selfishness, i.e., f 2 > f 1 , in the seven-strategy competition. This means that the non-monotonic rejection helps fairness overcome selfishness. As shown in Table 2, the increase of p induces the frequencies of selfishness, fairness, and altruism to increase in the seven-strategy competition. However, the increase of p induces the frequency of spitefulness to decrease. Assume X is one of S 1 , S 4 , S 5 , and S 6 , then a larger p causes the proposer using X to obtain a less payoff once the responder using another strategy Y accepts the offer. Here, we define that X gives Y one score and draw one arrow which goes from X to Y in Fig 2. Fig 2 shows that S 1 gives two strategies two scores and three strategies give S 1 three scores. The score of the selfish strategy S 1 is positive, meaning that the increase of p helps S 1 obtain more payoffs. Accordingly, the increase of p raises the frequency of selfishness. By using a similar analysis, we can arrive at the following conclusions. The total score of two fair strategies S 2 and S 7 is positive, and thus the increase of p raises the frequency of fairness. The total score of two altruistic strategies S 3 and S 5 is positive, and thus the increase of p raises the frequency of altruism. The total score of two spiteful strategies S 4 and S 6 is negative, and thus the increase of p reduces the frequency of spitefulness. For the rest of the paper, we will only focus on the case of p = 0.01.

Group-structured populations
The seven-strategy competition is also investigated in group-structured populations. For the Moran process and the Wright-Fisher process, the average frequency of strategy k 2 {1, 2, Á Á Á, S} over the stationary distribution under weak selection (δ ! 0), hx k i δ ! 0 , is given by in which Γ 1 , Γ 2 , and Γ 3 are unknown. Let I ij be the total number of interactions between strategy i and strategy j. Then Γ 1 , Γ 2 , and Γ 3 can be expressed by hx i I jk i 0 which is the probabilityweighted average of x i I jk over all possible steady states under neutral selection: More details can be obtained from pages 1 − 2 of SI (Supplementary Information) in Ref. [51]. The calculation of hx i I jk i 0 can be transformed into calculating the probabilities that three randomly chosen individuals use given strategies and are located in given groups (please refer to pages 3 − 4 of SI in Ref. [51] for more details). These probabilities have been calculated for the Moran process and the Wright-Fisher process (please refer to pages 5 − 10 and pages 10 − 16 of SI in Ref. [51] for more details). We first take the known values of these probabilities into the expression of hx i I jk i 0 . Then we obtain the precise values of Γ 1 , Γ 2 , and Γ 3 according to Eq (3), which are summarized in Table 3. The expression of f(x) in Table 3, which corresponds to the migration range r, is given by where b M 2 c is the greatest integer no greater than M 2 . These results hold for arbitrary population sizes, non-zero mutation probabilities, migration probabilities, migration ranges, and group numbers. Fig 3 shows that analytical results agree well with simulated results for sufficiently small δ and display obvious disagreement with simulated results for other δ.
We show the effects of the population size on the seven-strategy competition in Fig 4. Irrespective of the population size, fairness has a higher frequency than the other three behaviors. Accordingly, the population size cannot impact the dominance of fairness. In small populations, selfishness has the second highest frequency. Meanwhile, spitefulness has an advantage over altruism or the opposite holds, i.e., In moderate populations, the former disappears and the latter holds for the whole area spanned by v and u. In Table 3. Γ 1 , Γ 2 and Γ 3 for the Moran process (G Mo 1 , G Mo 2 , and G Mo 3 respectively), and Γ 1 , Γ 2 and Γ 3 for the Wright-Fisher process (G WF 1 , G WF 2 , and G WF 3 respectively), where F i (f(x)), C i (f(x)), F 0 i ðf ðxÞÞ, and C 0 i ðf ðxÞÞ are abbreviated as https://doi.org/10.1371/journal.pone.0196524.t003 Strategy intervention for the evolution of fairness large populations, the (v, u) area for the latter diminishes. Meanwhile, a new phenomenon appears in which altruism gains an advantage over selfishness, i.e., f 2 > f 3 > f 1 > f 4 . Accordingly, we arrive at the following two conclusions. First, the increase of the population size raises the frequency ranking of altruism. It means that a larger population size enhances the evolution of altruism. Second, the increase of the population size reduces the frequency rankings of selfishness and spitefulness. It means that a larger population size weakens the evolution of selfishness and spitefulness. The above results hold for the Moran process and the Wright-Fisher process.
We demonstrate how mutation influences the seven-strategy competition in Fig 5. Independent of the migration probability v, selfishness and fairness exhibit inverted U-shaped curves with the mutation probability u. Accordingly, intermediate mutation maximizes selfishness and fairness. Spitefulness exhibits a U-shaped curve with u irrespective of v. It means that intermediate mutation minimizes spitefulness. Altruism exhibits an inverted U-shaped curve when v is intermediate. However, the curve is changed to a U-shaped curve with u when v is too low or too high (not shown in Fig 5). Therefore, intermediate mutation maximizes  We investigate the effects of migration on the seven-strategy competition when u is low in Fig 6 and high in Fig 7, respectively. Migration changes selfishness and fairness qualitatively similarly. They both have decreasing trends with v, which is independent of u. Compared with the case without migration, the existence of migration decreases selfishness and fairness. Therefore, migration inhibits the evolution of selfishness and fairness. The way that altruism changes with v is different for low u and high u. When u is low, there exists a moderate v  which maximizes altruism. When u is high, altruism increases with v. Compared with the case without migration, the existence of migration increases altruism, and thus migration promotes the evolution of altruism. The way that spitefulness changes with v is also different for low u and high u. When u is low, the curve of spitefulness with v has an increasing trend with a small perturbation near v = 0.01. When u is high, spitefulness increases with v. Compared with the case without migration, sufficient migration increases spitefulness, and thus it promotes the evolution of spitefulness. The above results are independent of the migration range, and they hold for the Wright-Fisher process and the Moran process. These two update rules have a qualitative difference for the smallest migration range and small u. Specifically, the curves of selfishness, fairness, altruism, and spitefulness with v have small perturbations at v = 1 for the Wright-Fisher process but not for the Moran process.

Discussion
We have focused on the effects of spitefulness and altruism on the evolution of fairness in general structured populations of finite size. We have used strategy intervention to explicitly study the effects. Specifically, we first study the competition between a selfish strategy and a fair strategy, and then add five strategies to them. Our study goes from the two-strategy competition to the seven-strategy competition. In the two-strategy competition, selfishness competes equally with fairness. The addition of altruism leads to the advantage of selfishness over fairness, and this advantage can be removed by the further addition of spitefulness. When the fair strategy with the non-monotonic rejection is added, fairness gains an advantage over selfishness. Accordingly, we arrive at the following conclusions: 1) The evolution of fairness is inhibited by altruism, but it is promoted by spitefulness; 2) The non-monotonic rejection helps fairness overcome selfishness. The four-strategy competition of our model corresponds to the finitepopulation version of the previous model [17]. In contrast to our work, the previous model has only studied the replicator dynamics of four strategies in infinite populations. It has implicitly demonstrated that spitefulness promotes the evolution of fairness in infinite populations under certain conditions. Moreover, the previous model has not considered non-monotonic rejections which have been shown in behavioral experiments [6,7].
Most previous studies about the UG, including the above-mentioned literature [17], have neglected the role of population finiteness in the evolution of fairness. Recently, a stochastic evolutionary model has demonstrated that fairness can evolve in finite populations without any other mechanisms [28]. It indicates that the finiteness of the population matters in the evolution of fairness. In this paper, we have focused on finite populations, including general structured populations and group-structured populations. Particularly for group-structured populations, we have studied the effects of the population size on the seven-strategy competition by the Tarnita-σ condition [46]. For the Moran process and the Wright-Fisher process, the population size cannot change the dominance of fairness. A larger population size enhances the evolution of altruism, but it weakens the evolution of selfishness and spitefulness. The unknown parameters in the Tarnita-σ condition have been obtained based on the results in the previous literature [51], which have been used to analyze the multiple-strategy competition in general models.
The effects of migration on the evolution of fairness have been previously studied by agentbased simulations [26]. In this paper, we have given the analytic results about how migration and mutation influence the evolution of fairness, selfishness, altruism, and spitefulness. The Moran process and the Wright-Fisher process have the following qualitatively similar results. Intermediate mutation maximizes selfishness and fairness, but it minimizes spitefulness. Intermediate mutation maximizes altruism for intermediate migration and minimizes altruism otherwise. Migration inhibits the evolution of selfishness and fairness, but it promotes the evolution of altruism. Only sufficient migration promotes the evolution of spitefulness. For the smallest migration range and small mutation probabilities, the Moran process and the Wright-Fisher process have the following qualitatively different results. The curves of selfishness, fairness, altruism, and spitefulness with the migration probability v have small perturbations at v = 1 for the Wright-Fisher process but not for the Moran process.