1 Supplementary Information : Extrapolating weak selection in evolutionary games

In evolutionary games, reproductive success is determined by payoffs. Weak selection means that even large differences in game outcomes translate into small fitness differences. Many results have been derived using weak selection approximations, in which perturbation analysis facilitates the derivation of analytical results. Here, we ask whether results derived under weak selection are also qualitatively valid for intermediate and strong selection. By “qualitatively valid” we mean that the ranking of strategies induced by an evolutionary process does not change when the intensity of selection increases. For two-strategy games, we show that the ranking obtained under weak selection cannot be carried over to higher selection intensity if the number of players exceeds two. For games with three (or more) strategies, previous examples for multiplayer games have shown that the ranking of strategies can change with the intensity of selection. In particular, rank changes imply that the most abundant strategy at one intensity of selection can become the least abundant for another. We show that this applies already to pairwise interactions for a broad class of evolutionary processes. Even when both weak and strong selection limits lead to consistent predictions, rank changes can occur for intermediate intensities of selection. To analyze how common such games are, we show numerically that for randomly drawn two-player games with three or more strategies, rank changes frequently occur and their likelihood increases rapidly with the number of strategies . In particular, rank changes are almost certain for , which jeopardizes the predictive power of results derived for weak selection.


Preliminaries
We use the following symbols: N population size n number of strategies in the game a ij payoff of strategy i player when facing a strategy j player β selection intensity ∆π(k) payoff difference between mutant and wild types when there are k mutants g(x) imitation function, where x = β∆π(k) φ ij fixation probability of a single mutant of type j taking over a wild population of type i In well-mixed populations, every individual plays equally likely with all the other players in the populations. In 2 × 2 games with payoff matrix (a ij ) 2×2 , the payoffs for strategy 1 and 2 are thus given by π 1 (k) = (a 11 (k − 1) + a 12 (N − k))/(N − 1) and π 2 (k) = (a 21 k + a 22 (N − k − 1))/(N − 1), where k is the number of strategy 1 individuals. The payoff difference ∆π(k) = π 1 (k) − π 2 (k) is given by (1) Note that the fixation probability φ ij depends only on g(β∆π). In turn, ∆π depends on u ij and v ij . Moreover, since u ji = u ij and v ji = −N u ij − v ij , G ij = φ ij − φ ji only depends on u ij , v ij and β, and can be written as a function G ij (β, u ij , v ij ). For convenience, when dealing with games of two strategies, we drop the indexes in u ij , v ij , G ij . In addition, the indexes are treated modulo the number of strategies n if we are considering more than two strategies.

Embedded Markov chain approximation for weak mutation
Under sufficiently weak mutation a mutant fixates or goes extinct before a new mutant arises [1,2]. This means that the population spends most of the time in monomorphic states where all the individuals are of the same strategy. The dynamics in this case is approximated by the transition probabilities φ ij between homogeneous states i and j. Thus, the dynamics is fully approximated by an embedded Markov chain with transition matrix M , given by where All i denotes the state consisting of only strategy i individuals, and φ ij is the fixation probability of a single mutant of type j in a population of N − 1 i individuals. Note that we will always follow the common assumption of uniform mutation kernels [3].
To compute the fixation probabilities φ ij we use a generalized pairwise comparison rule. Each individual interacts with the rest of the population equally likely and obtains a payoff. Then one individual is selected randomly to imitate another randomly chosen individual's strategy with a probability g(β∆π), where ∆π is the payoff difference between its opponent and the focal individual.
Let k denote the number of the mutants in the population. The transition probabilities to go from state k to k ± 1, T ± k are given by while the probability to stay in state k is The fixation probability is given by where i and j refer to the wild and mutant types [4][5][6].
Note that the fixation probabilities in M depend on the intensity of selection β. We compute the stationary distribution of M , given by the left eigenvector of M to the unit eigenvalue. This stationary distribution is also a function of selection intensity, a vector with n elements where each element is the long-term abundance of the corresponding strategy. We are interested in how the ranking of strategies according to abundance changes with increasing selection intensity.
3 Discussion of the ranking invariance property for two-strategy multiplayer games For 2 × 2 games, it has been shown that the ranking of strategies is invariant with increasing selection intensity [7]. This is valid for any pairwise comparison rule. No technical constraints in the imitation function g(x) are required, thus the result is robust if the payoffs are given by two-strategy two-player games. Let strategy 1 and 2 be the mutant and wild type respectively. For rare mutations, the transition matrix M in Eq. (2) is and the stationary distribution is ( φ21 φ12+φ21 , φ12 φ12+φ21 ) [2,8]. The ratio between the abundance of strategy 1 and 2 is φ 21 /φ 12 . Following [8][9][10] we have where, in the last step, we have used exp x = exp x. By Eq. (6), for D 1 > 0, strategy 1 is more abundant while strategy 2 is more abundant for D 1 < 0. For D 1 = 0, the two strategies are equally abundant. Thus the invariance of ranking with increasing selection intensity is equivalent to the invariance of sign for D 1 for all β > 0. For D 1 , we have the following properties: (i) For the Fermi imitation function, g(x) = 1/(1 + exp(−x)), D 1 = β N −1 k=1 ∆π(k), this leads to the invariance of signs for all the β > 0 for any two-strategy games. Thus for any two-strategy game, the ranking invariance property holds for the Fermi process; (ii) D 1 is invariant by rearranging the index k, thus the ranking is invariant by rearranging k such that the rearranged ∆π(k) is monotonic; (iii) For weak selection, D 1 is approximated by β(2g (0)/g(0)) N −1 k=1 ∆π(k). Since g (0)/g(0) > 0 for all imitation functions, the sign of D 1 is solely determined by the sum of the payoff differences. This illustrates that under weak selection, for any given two-strategy game, the ranking is the same for any imitation process.
In contrast, for games with more than two players, the abundance ranking may change as illustrated in the main text for a three player game. In addition to the Fermi function, we discuss the imitation function given by g(x) = (1 + erf(x))/2, where erf(x) = (2/ √ π) x 0 exp(−t 2 )dt is the error function. In this case, we find that the ranking can change with the selection intensity. It turns out that the criterion to determine the ranking differs between weak and strong selection for this imitation function. For strong selection, expanding g(1/t) around t = 0 + leads to the approximation of g(x) for x → +∞, i.e.
Thus under strong selection, or sufficiently large |x|, ln(g(x))−ln(g(−x)) = ln(g(x)/g(−x)) ≈ sgn(x) ln( 2 ) as x → +∞. Since the leading term is sgn(x)x 2 , for strong selection, the sign of D 1 is determined by N −1 k=1 sgn(∆π(k))(∆π(k)) 2 . For weak selection, however, the sign of D 1 is determined by N −1 k=1 ∆π(k) as aforementioned. In fact, the multiplayer game is constructed such that the sign differs between N −1 k=1 ∆π(k) and N −1 k=1 sgn(∆π(k))(∆π(k)) 2 . This example shows that for multiplayer games the ranking invariance property does not always hold as it does for pairwise games. Furthermore, D 1 is invariant by rearranging the index k in Eq. (6) such that the rearranged ∆π(k) is monotonic as in 2 × 2 games, hence even monotonicity in payoff differences is not sufficient to ensure the invariance of rank for any pairwise comparison rules. This implies that the invariance of rank is not robust for two-strategy multiplayer games.

Proof of Theorem 1
Theorem 1 Consider any imitation process with a strictly increasing, twice differentiable imitation function g(x). For a sufficiently large population size N and any selection intensity β * (0 < β * < ∞), there exists a 3 × 3 payoff matrix (a ij ) 3×3 with the following two properties: 1. The stationary distribution is uniform for β = 0 (as always) and for β → ∞.
Theorem 1 implies that the rank can still change for moderate selection intensity, even when weak and strong selection limits lead to the same rank. Based on the first three lemmas that follow, we construct a 3 × 3 payoff matrix that satisfies the two conditions in Theorem 1.

Construction of the 3 × 3 matrix
The intuition to establish these lemmas is as follows: We need to find a 3 × 3 matrix (or nine payoff entries) to satisfy the constraints in Theorem 1. Lemma 3 formally establishes that only six parameters u i,i+1 , v i,i+1 , i = 1, 2, 3 are necessary for pairwise comparison processes. To further reduce the number of parameters, Lemma 1 is introduced. Therein, it establishes a mapping between u i,i+1 and v i,i+1 , i.e., v i,i+1 = v(u i,i+1 ), i = 1, 2, 3. In addition, Lemma 2 is introduced to clarify the domain where v(u) is defined. As a consequence, only three parameters u i,i+1 , i = 1, 2, 3 are required. This is smaller than the dimension of a 3 × 3 matrix. It suggests that the matrices satisfying the constraints in Theorem 1 are located on a subspace with the dimension lower than that of a 3 × 3 matrix.
Lemma 2 assumes that the imitation function is twice continuously differentiable and population size large. Consequently, Theorem 1 requires the same assumptions to use the inverse function theorem. Yet the example shown in Figure. 1 suggests that assumption of large population size is not necessary.
Lemma 1 For every imitation process with strictly increasing continuously differentiable imitation function g(x) with lim β→−∞ g(x) = 0 and lim β→+∞ g(x) = 1, β * > 0, population size N and −1 < c * < 1, there is a δ > 0 and a continuously differentiable Lemma 2 For any imitation process with strictly increasing second order continuously differentiable imitation function g(x) and any Lemma 3 If there are 3 strategies in the population, for every u i,i+1 , v i,i+1 , i = 1, 2, 3, and population size N , there is an affine space of dimension 3, such that every element in such a space corresponds to a payoff matrix (a ij ) 3×3 . (The proof can be found in Section 5.3).
3. Based on u i,i+1 , v i,i+1 , i = 1, 2, 3, (six numbers in total), by Lemma 3, there exists an affine space, such that every element in such a space corresponds to a 3 × 3 matrix (a ij ).
Remark For the established game matrix, u i,i+1 and v i,i+1 (i = 1, 2, 3) are all positive. In fact, the second step explicitly illustrates that u i,i+1 > 0 for i = 1, 2, 3. In addition, considering that v(0) > 0, by the continuity Section 4.2 proves that any 3 × 3 game that follows the recipe above fulfills the conditions in Theorem 1. An example using the Fermi function is shown in Figure. 1.

Proof that the established matrix satisfies the constraints
In order to prove that the matrix established in Section 4.1 fulfills the conditions listed in Theorem 1, we list three more lemmas. They are sufficient conditions under which the constraints in the Theorem are fulfilled: Lemma 4 and Lemma 6 illustrate conditions under which the ranking is uniform and the ranking changes respectively, reflecting the two main constraints in Theorem 1. While Lemma 5 is a sufficient condition under which the uniform distribution is the stationary distribution for strong selection limit. Then we prove that the constructed matrix satisfies the conditions in these lemmas, which completes the proof. S1.pdf

Selec%on intensity
Average abundance We use a Fermi process with imitation function g(x) = 1/ (1 + exp(−x)). Let N = 30, c * = 0.1, β * = 0.1 and u i,i+1 = i/(2N ), where i = 1, 2, 3. Using the procedure above we find a game such that the uniform distribution is the stationary distribution for β = 0 and β → ∞, and two of the three strategies exchange the rank at β * = 0.1 as Theorem 1 states. Furthermore, we observe that (i) every two strategies exchange their rank at the β * = 0.1; (ii) the most abundant strategy becomes the least abundant when the selection intensity exceeds this critical value β * . (iii) β * = 0.1 is the unique positive selection intensity at which all three strategies are equal in abundance.
Lemma 4 For n = 3 and β * > 0, the uniform distribution is the stationary distribution of the transition matrix M at β * if and only if G 1,2 = G 2,3 = G 3,1 at β * . (For a proof, see Section 5.4.) This lemma demonstrates that the uniform distribution is the stationary distribution if and only if the relative transition rates between every two strategies are identical.
Lemma 6 For n = 3, assume that imitation function g(x) is strictly increasing and continuously dif- uniform distribution is the stationary distribution at β * > 0, then there are two strategies out of the three that exchange their ranking at β * . (For a proof, see Section 5.6.) This lemma shows that if the uniform distribution is the stationary distribution at a certain selection intensity, a ranking change occurs, provided the increase rates of the relative transition probabilities at the selection intensity differ from each other. We prove that the matrix established in Section 4.1 fulfills the conditions listed in Theorem 1.
This completes the proof.
Remark For the established matrix, the uniform distribution is also the stationary distribution when β = β * . This is why Figure. 1 shows that all the three lines are intersecting at the given selection intensity β * .

Lemma 1
Proof The outline of the proofs are in the following: Given the imitation function g(x), selection intensity β * > 0, population size N and −1 < c * < 1, for u = 0, there exits a unique v, denoted as v 0 , such that G(β * , 0, v 0 ) = c * . Then, based on implicit function theory, we prove the existence of the implicit function v(u) defined in a vicinity of u = 0 such that G(β * , u, v(u)) = c * . For c * ≥ 0, we need to prove that (i) G(β * , u, v) is continuously differentiable as a function of u and v; (ii) ∂G ∂v is not zero at point (0, v 0 ). For c * < 0, we convert it into a case with c * > 0.
In summary, we have proved that for arbitrary c * ∈ (−1, 1) and β * > 0, there exists a function defined in a vicinity of u = 0 such that G(β * , u, v(u)) = c * . Also by the implicit function theorem, v(u) is continuous. Furthermore, since g(x) is continuously differentiable, ∂F ∂u is continuous in the whole plane. This leads to that v(u) is also differentiable. In fact, since g(x) is differentiable, v (u) is continuous.
This completes the proof.

Lemma 2
Proof To prove the existence of the inverse function of H(u, v) around (0, v 0 ), where v 0 > 0, we are employing the inverse function theorem. What we need to prove is: Second, we show that if v 0 > 0, the Jacobian matrix of H(u, v) at (0, v 0 ) is non-degenerate. We rewrite the function G(β, u, v) asG(βu, βv), i.e., where The Jacobian matrix of H(u, v) at (0, v 0 ) is equivalent to that ofG(x, y) at (0, βv 0 ), and it is given by whereG (i,j) = ∂ i+jG ∂x i ∂y j . In order to prove that Eq. (16) is non-degenerate, we perform elementary transformations, which do not change the rank of matrix.
Since β is positive, we multiply the second row of Eq. (16) with factor −1/β, and add it to the first row. This leads to For Eq. (17), since v 0 > 0 by assumption, we multiply 1/(βv 0 ) to the first row then 1/β to the second row. This yields Taking into account Eqs. (14) and (15), Eq. (18) is where Taking Eqs. (15) into consideration leads to where Considering the following identities [11] and taking them into Eqs. (22) lead to On one hand, g(x) is increasing and positive, v 0 > 0 thus 0 < h < 1, where h = g(−v 0 β)/g(−v 0 β) in Eq. (23). On the other hand, a, b and h in Eq. (23) are not dependent on population size N , they are viewed as order 1. Therefore, for any i > 0, N i h N → 0 as the large population size N is sufficiently large. Since the population size N is large by assumption, Eqs. (25) can be approximated by Taking these approximations into Eq. (20), we find that the determinant of Eq. (21) is of order N 6 for large population size, i.e., For large population size the sign of the determinant of J is determined by a 3 /(8(1 − h) 3 ). Since v 0 > 0 by assumption, by Eq. (23) we have that 0 < h < 1 and a < 0. Thus a 3 /(8(1 − h) 3 ) is negative and non-zero, which leads to the non-zero determinant of J. Regarding J is derived by a series of elementary transformations of the Jacobian matrix of H(u, v) at (0, v 0 ), the Jacobian matrix of H(u, v) at (0, v 0 ) is also non-degenerate. By the inverse function theorem, there is a δ > 0 such that H(u, v) is invertible in the vicinity of (0, v 0 ), This completes the proof.

Lemma 3
Proof By the definition of u i,i+1 and v i,i+1 in Section 1, we have where i = 1, 2, 3.
By assumption N > 1, thus r(P ) = r(Q), i.e., the rank of P is the same as that of Q. On the one hand, Q ∈ R 6×9 , the rank of Q is no more than 6, i.e., r(Q) ≤ 6. On the other hand, we eliminate the first, the fifth and the ninth columns of Q, and arrive at a sub matrix of Q, i.e. a 6 × 6 matrix, Note the determinant of Q 1 is −N 3 which is nonzero, the rank of Q 1 is six, i.e., r(Q 1 ) = 6. Considering that Q 1 is obtained via eliminating the columns of Q, r(Q 1 ) ≤ r(Q). In summary, 6 = r(Q 1 ) ≤ r(Q) ≤ 6. This leads to r(Q) = r(P ) = 6. By elementary linear algebra, we have that all the solutions of P l = 0 forms a linear space, i.e., KerP , of dimension Dim(l) − r(P ) = 3, where Dim(l) is the dimension of vector l and r(P ) is the rank of matrix P . Furthermore, by assumption u i,i+1 and v i,i+1 are not all zero, q in Eq. (32) as a vector is non-zero, all the solutions of P l = q are presented as where l * is a special solution of P l = q. In other words, for a given N > 1, not-all-zero u i,i+1 and v i,i+1 , all the solutions of P l = q forms an affine space of dimension 3. Regarding that l = (a 11 , a 12 , a 13 , a 21 , a 22 , a 23 , a 31 , a 32 , a 33 ) T , it corresponds to a 3 × 3 matrix.
This completes the proof.
Notice that every step of the above proof is either "if and only if" or "be equivalent with", Eq. (40) is the necessary and sufficient condition of that the stationary distribution of M is uniform at β * . This completes the proof.
This completes the proof.