The Dynamics of Sex Ratio Evolution: From the Gene Perspective to Multilevel Selection

The new dynamical game theoretic model of sex ratio evolution emphasizes the role of males as passive carriers of sex ratio genes. This shows inconsistency between population genetic models of sex ratio evolution and classical strategic models. In this work a novel technique of change of coordinates will be applied to the new model. This will reveal new aspects of the modelled phenomenon which cannot be shown or proven in the original formulation. The underlying goal is to describe the dynamics of selection of particular genes in the entire population, instead of in the same sex subpopulation, as in the previous paper and earlier population genetics approaches. This allows for analytical derivation of the unbiased strategic model from the model with rigorous non-simplified genetics. In effect, an alternative system of replicator equations is derived. It contains two subsystems: the first describes changes in gene frequencies (this is an alternative unbiased formalization of the Fisher-Dusing argument), whereas the second describes changes in the sex ratios in subpopulations of carriers of genes for each strategy. An intriguing analytical result of this work is that the fitness of a gene depends on the current sex ratio in the subpopulation of its carriers, not on the encoded individual strategy. Thus, the argument of the gene fitness function is not constant but is determined by the trajectory of the sex ratio among carriers of that gene. This aspect of the modelled phenomenon cannot be revealed by the static analysis. Dynamics of the sex ratio among gene carriers is driven by a dynamic “tug of war” between female carriers expressing the encoded strategic trait value and random partners of male carriers expressing the average population strategy (a primary sex ratio). This mechanism can be called “double-level selection”. Therefore, gene interest perspective leads to multi-level selection.


Introduction
Sex ratio evolution is one of the basic examples of evolutionary mechanisms that are presented in every course on evolutionary biology. The first approach to this problem was presented by German biologist Carl Dusing [2]. Historically, it was the first application of mathematical modeling to evolutionary phenomena. Dusing argued that the fitness of females using different sex ratio strategies can be described by the number of their grandoffspring. A similar approach was applied by Fisher and Shaw and Mohler [3,4,5]. This is also an important example in evolutionary game theory, known as a sex ratio game [6,7,8,9,10,11]. The general prediction of this approach is that the sex ratio of 0.5 is evolutionarily stable. However, there is an alternative approach to the modeling of sex ratio evolution related to population genetics [5,12,13,14]. This approach is focused on tracing the genes encoding sex ratio strategies. Those models predict a stable structure of the population describing gene frequencies among males and females and a sex ratio as the effect of expression of those genes. Therefore, there is a major difference between the strategic phenotypic approach and genetic modeling [15,16,17]. The phenotypic approach describes the mean female strategy of 0.5 as evolutionarily stable, while genetic models show that the composition of the male population can also matter. To analyze this problem, in our previous paper [1], a new model of sex ratio evolution was developed. The new approach is an attempt to combine the genetic and phenotypic approach and to overcome the limitations of both of them. The goal was to solve the problem of different predictions and to obtain a coherent picture of the modeled phenomenon.
The new model focuses on the global dynamics of the system, and its structure resembles the genetic approach [5,12,13,14]. Whereas the classical Dusing-Fisher-Shaw-Mohler (DFSM) model is focused on the reproductive success of individual strategies carried by female strategic agents (as in Dusing's paper, see [2], or the sex ratio game) or some undescribed group of ''parents'' (as in [3,4], more on this topic in section 4.2). For a closer understanding of the relations between the classical and the new approach, the selection of individual strategies resulting from global dynamics must be analyzed, which is the subject of this paper.
In this paper a novel technique of change of coordinates will be applied to the model from [1]. This will reveal new aspects of the modelled phenomenon which cannot be shown or proven in the original formulation. Similarly the results from [1] will be hard to show in the new coordinates, thus the two papers complement each other. The underlying goal is to describe the dynamics of selection of particular genes in the entire population, instead of in the same sex subpopulation as in the previous paper and earlier population genetics approaches. In effect, an unbiased strategic model will be analytically derived from the non-simplified rigorous genetic model. Thus, the classical strategic approach analyzes the reproductive success of a female, while the genetic approach traces gene frequencies in the population. Therefore, what happens when we combine both perspectives and assume that the gene is the strategic agent?

Methods
Now we shall recall the structure of the new model (see Table 1 for the list of symbols). Section 1 can be skipped by readers familiar with paper [1].

Summary of basic formal details of the new model
There are u individual strategies described by P i [½0,1, the proportion of male offspring of a female playing strategy P i . There are x i females and y i male carriers of the strategy P i in the population. Therefore, the population consists of x~S i x i females and y~S i y i males. Thus, f~½f 1 ,:::, f u is the vector of frequencies of strategies of the female subpopulation, and m~½m 1 ,:::, m u is an analogous vector for the male subpopulation, where f i~x i x and m i~y i y : P~y yzx is the fraction of males in the population (the secondary sex ratio), and S j f j P j is the mean female strategy (the primary sex ratio). Assume that each female produces k offspring according to haploid inheritance. However, males are gene carriers too, and transfer those genes to their offspring with the probability 0.5. The influence of males can be described by the fitness exchange effect (i.e. daughters of male carriers contribute to the fitness of female carriers and sons of female carriers contribute to the fitness of male carriers). In [1] it was shown that  The Dynamics of Sex Ratio Evolution II PLOS ONE | www.plosone.org the female individual playing the strategy P i : Therefore, the following equations were obtained: -payoff function of the males carrying the strategy P i , -payoff function of the females playing the strategy P i : Now we have all elements needed to formulate multipopulation replicator dynamics (see appendix A in the File S1). In [1], this took the following form: Þ for i~(1,:::, u{1), Þ are the respective average payoff functions of the male, female and the whole population. This leads to the following system of equations: for i~(1,:::, u{1), for i~(1,:::, u{1), It was shown that, for biological reasons, we can limit the analysis of the model to values of primary and secondary sex ratios over the interval(0, 1):

Summary of predictions of the new model
An analysis of the behavior of this model shows that two phases of convergence can be distinguished. The first, rapid phase occurs when the secondary sex ratio P converges to the current value of the primary sex ratio S j f j P j , and the male subpopulation converges to the state termed the male subpopulation equilibrium (MSE), described by the condition f i P i~mi S j f j P j : During the second phase of convergence, the primary sex ratio converges to the value 0.5, and the value of the secondary sex ratio follows these changes to maintain equality. In addition, the state of the male subpopulation changes to maintain the MSE.

Reformulation of the model
In the previous paper [1], a change in the coordinates (described in appendix A in the File S1) was applied to the numerical solutions obtained to calculate the frequencies of all types of individuals (see Fig. 3c in [1] and section 3.2 there) and gene frequencies (see Fig. 6 in [1] and section 4 there). However, this method can be applied not only to numerical solutions, but also directly to replicator equations. In this way, we can reformulate the new model to focus on changes in gene frequencies. We have Pm i male carriers and 1{P ð Þf i female carriers of a strategy P i in the whole population. Thus, the frequency of carriers of a gene which encodes this strategy is equal to: The state of the population can be described by the vectorG~½G 1 ,:::, G u [D u , whereS j G j~1 : In this description, there is no information about the sex of the carriers of these genes. We can fill this gap by adding information about the sex ratio in the subpopulation of the carriers for every gene: -proportion of females among carriers of P i : Then, M~½M 1 ,:::, M u is the vector of subpopulation sex ratios. Therefore, this structure can be treated as a division of the entire population into u subgroups with one-dimensional subpopulation states. Then, according to the general notation from appendix A in the File S1, s i~M i and c j~Gj (see also [18]), the structure of the space of population states will take the form presented in Fig. 1. Note that in the previous formulation of the model, the space of population states was the product of two u{1 dimensional simplexes of the male and female subpopulation and a one-dimensional simplex of the proportion between these subpopulations (a secondary sex ratio); in general, the dimension of the whole space was 2u{1: In the new formulation, this space consists of one u{1 dimensional simplex of gene frequencies and u one-dimensional simplexes of subpopulation sex ratios, and the dimension of the whole space of population states is also 2u{1: Therefore, the dimension of the space of population states is invariant in response to the change of coordinates, which is consistent with the fact that we have a different parameterization of the same phase space. We can describe important population parameters in the new coordinates for parameters such as the mean female subpopulation strategy P P pr , i.e., the primary sex ratio and secondary sex ratio (among adult individuals) P : P P pr~Sj f j P j~1 1{P S j (1{M j )G j P j and P~S j G j M j : The average fitness functions from the previous paper (recalled in section 1.1) were: W W m (P, f , m)~k 1{P P S j f j P j -mean fitness of the male subpopulation, -mean fitness of the female subpopulation, shows the respective changes of sex ratios in carrier subpopulations M i : Note that sex ratios in carrier subpopulations rapidly converge to the values determined by the MSE phenomenon, and after that, they follow the changes of the primary sex ratio P P pr that slowly converges to 0.5. The sex ratio among carriers of male biased strategies change due to the dynamics of the primary sex ratio while among female biased strategies, it converges to the neighbourhood of the value encoded by the gene. doi:10.1371/journal.pone.0060405.g002 The Dynamics of Sex Ratio Evolution II PLOS ONE | www.plosone.org W W P, f , m ð Þk(1{P) -mean fitness of the whole population. Then, we can derive the mean payoff to the carrier of a gene for strategy P i (for a full derivation see appendix B in the File S1): which takes the form: where C~1 {P P is the number of females per single male individual. For the new coordinates we obtain the following replicator equations (for a detailed derivation, see appendix C in the File S1): -dynamics of sex ratios in carriers subpopulations,which take the form: for i~(1,:::, u{1), ð6Þ for i~(1,:::, u):

Behavior of trajectories of replicator equations
3.1 Trajectories of gene frequencies.
Here, we will examine the dynamics of gene frequencies. The product is responsible for the sign of the right side of equation (6). When both coefficients are negative or positive, then their product is positive (the frequency of gene P i increases), and when they have opposite signs, then their product will be negative (the frequency of gene P i decreases). The zero points of these coefficients,P~1 2 and P~M i , are stationary points of equation (6). Therefore, the dynamics of the gene frequencies can be described in the following way: G i is constant when G i~0 or M i~P or P~1 2 : Recall that P~S j G j M j , which means that the secondary sex ratio is equal to the average sex ratio in the carrier subpopulation over the entire population. Therefore, the frequency G i decreases when the sex ratio in the carrier subpopulation M i is shifted farther from 0.5 than the mean sex ratio in the carrier subpopulations for all strategies P: In the opposite case, G i will increase. This mechanism is illustrated in Fig. 2a. Therefore, the frequency of a gene that encodes the strategy 0.5 increases when the sex ratio in a subpopulation of its carriers is closer to 0.5 than the current value of the secondary sex ratio; this frequency decreases in the opposite case. A situation in which the secondary sex ratio is equal to 0.5 is the stationary state of the dynamics of gene frequencies (6). Therefore, this mechanism described by (8) is independent of individual strategies P i , but its dynamics are dependent on the trajectories of the sex ratios in the subpopulations of carriers of the strategies described by M i . Note that parameter M i also affects the secondary sex ratioP~S j G j M j , modifying the values of G j : However, sex ratios in carrier subpopulations M j are determined by mechanisms acting at the level of carrier subpopulations that are described in the next section.

Trajectories of sex ratios in subpopulations of
carriers. The dynamics of sex ratios in the carrier subpopulations are more sophisticated. The right side of equation (7) contains two coefficients: P P pr {M i À Á and P i {M i ð Þ , weighted by current values of M i C and 1{M i ð Þ : These coefficients are responsible for the direction of convergence. The coefficient P P pr {M i À Á induces attraction of M i to P P pr , and the coefficient Þcauses attraction of M i to P i : This is, in a sense, a tug of war between female partners of the male carriers (representing average strategy P P pr ) and female carriers of the same gene (representing encoded strategy P i ). As we can see in Fig. 2b, the shape of the trajectory of a 0.8 sex ratio strategy that produces mostly sons is almost parallel to the trajectory of parameter P, which is equal to P P pr in the slow phase of convergence (see [1]). On the other hand, the trajectory of a 0.2 sex ratio strategy that produces more daughters is closer to the constant function 0.2 than to the trajectory of P: Thus, the M i value of the strategies producing (and in effect carried by) mostly males resemble trajectories of the primary sex ratio, while female biased strategies have M i almost constant and equal to P i . This interesting aspect would be hard to show by static analysis. Below, we will characterize equilibrium in this ''tug of war''. Lemma 1 a) For every set of values of P, P P pr [(0, 1) and P i [(0,1, dynamics (7) has the unique stable conditional equilibrium M i that is contained in the interval limited by the values of P P pr and P i : b) For the strategy P i~0 , there is one stationary point, M i~0 , which is stable when unique. However, when P P pr w 1 C and For a proof, see Appendix D in the File S1. Lemma 1 indicates that, at every moment, there exists some attracting point for M i lying between the current value of the primary sex ratio P P pr (which also changes in time) and the value of individual strategy P i . By this dynamic equilibrium, the expression of individual strategies determines the parameter M i . The only exception is strategy P i~0 (production of female offspring only) for which the second stationary state may exist during the rapid phase of convergence. It was impossible to analytically derive the stable sex ratio in the carrier subpopulations, in the general case. This is possible only when the population is in the MSE state and will be presented in a subsequent paper devoted to the MSE. According to Lemma 1, we can numerically approximate this value because it is unique in these biologically significant cases.

The mechanism of ''double-level'' selection
Here, we will summarize the results we have obtained. The first intriguing analytical result of the reformulated model is that the fitness function of a gene (5) is independent of the individual strategy it encodes. Proliferation of a given gene depends on the current sex ratio in the subpopulation of its carriers M i : Note that the fitness function (5) is a good mathematical description of Fisher's idea, which is related to the reproductive value of carriers with different sexes according to the deviation of the secondary sex ratio P. It suggests that males are reproductively more efficient when they are in the minority (P,1/2), because each male can mate with several females (C.1). On the other hand, females are more efficient when they are in the minority (P.1/2), because each female will be expected to produce offspring, and there are not enough mates for all males (C,1). Therefore, parameter M i describes the proportion of carriers with the more reproductively efficient sex among all carriers of a gene. This fitness function explicitly considers male carriers from the mother's generation of unexpressed sex ratio genes. Function (5) can be transformed in the following way (recall that y i is the number of male carriers, and x i is the number of female carriers, of the strategy P i ): This is the per capita normalized sum (averaged over the carriers subpopulation) of the offspring produced by female partners of male carriers described by y i C k 2 and offspring of female carriers described by x i k 2 (where k 2 is the number of offspring of a single female multiplied by the probability of gene transfer from the focal parent). This is an explanation of the importance of male carriers of the unexpressed sex ratio genes, or rather their female partners. Their role is important, because each male carrier may have C partners, and the activity of their partners is an important component of gene fitness. Surprisingly, this function is independent of the value of a given strategy, P i , encoded by the carried gene. It depends only on C and M i : The phenomenon can be termed double level selection. The fitness of a gene that encodes an individual strategy is determined in some way by the current sex ratio in its carrier subpopulation and the secondary sex ratio in the population as a whole. Values of both parameters may be perturbed. However, the stable carrier subpopulation sex ratio should be determined in some way by the value of the encoded strategy (Fig. 3). This is a newly discovered mechanism. In general, the mechanism of double level selection can be regarded as an example of multi-level selection, which is the concept presented by [19,20,21,22,23]. The classical approach to the modeling of sex ratio evolution treats this phenomenon as single level selection, which means that the fitness is unambiguously determined by the values of individual strategy P i and a population state described by the secondary sex ratio (Fig. 3).
In the next subsection, a higher level of this process will be considered.

Dynamics of gene frequencies
The mechanism realized by gene frequency replicator equations (6), described by the rules (8) increases the frequency of a gene for which the value of a parameter M i is greater/smaller than the secondary sex ratio P (which is equal to the average M in the population) when P is smaller/greater than 0.5. Thus, it is profitable for the gene to be carried by that sex which is currently in the minority. There is an interesting relationship between the mechanism described by (8) and the replicator dynamics paradigm. In standard replicator equations, frequencies of strategies change according to the sign of the deviation of their fitness from average fitness (minus -decrease, plus -increase). If fitness depends linearly on a particular trait, then selection works according to deviations from the average trait value. Note that the payoff function (5) is linear with respect to the parameter (trait) M i , and the secondary sex ratio P is an average M i over the population. The difference between the mechanism in rules (8) and standard replicator dynamics is that parameter M i is not a description of a fixed individual strategy but of the current state of a subgroup of individuals (the subpopulation of carriers of strategy P i ). Dusing classically argued that female producing offspring of the sex that is currently in the minority will have more grand-offspring. This argument states that there are differences in fitness among females with different strategies, which is considered a proof of the existence of selection on individual strategies. However, our new model shows that a mechanism based on different reproductive values is independent of individual strategies P i , and it affects the primary sex ratio P P pr and the secondary sex ratio P (which is equal to the average sex ratio in the carrier subpopulation) by changing only gene frequencies G j : In [3], the following statement can be found: ''...it would follow that those parents, the innate tendencies of which caused them to produce males in excess, would for the same expenditure, produce a greater amount of reproductive value; and in consequence would be the progenitors of a larger fraction of future generations…''.
Therefore, Fisher in his original reasoning considered a group of individuals that adjusts the sex ratio among its members due to genetic mechanisms. However, the mechanisms for this adjustment were not explicitly explained. The perspective of a group adjusting the sex ratio among its members is also assumed by [4]. However, they also presented only a conjecture that the sex ratio is completely heritable within the group, without an explanation of how it is realized. Therefore, there is a difference between Fisher's reasoning that operated on the level of the subpopulation of all carriers of a gene and Dusing's approach related to the level of female individuals. The female perspective is not sufficient, especially for male-biased strategies, which will produce more male than female carriers. This means that the Fisherian argument about the different reproductive values of males and females is an important part of understanding sex ratio self-regulation. However, it is not enough for a full mechanistic explanation of this process. Therefore, we should investigate how the expression of individual strategies determines the sex ratio in the carrier subpopulation M i : This will allow us to overcome the limitations of Dusing's reasoning, which considers only female reproductive success and disregards the role of male gene carriers from the same generation.

Dynamics of sex ratios in carrier subpopulations: the ''tug of war'' mechanism
The sex ratio in carrier subpopulations is the effect of intrinsic dynamics that can be compared to a ''tug of war'' between P i and P P pr : It was proved in Lemma 1 that for every population state there exists a single unique attractor of M i dynamics contained in the interval that is limited by values of P P pr and P i : Let us describe the ''tug of war'' metaphor in a more formal way. The right-hand side of replicator equation (7) is proportional to The factor M i 1{P P that is the weight of P P pr {M i À Á can be written as y i C x i zy i and the proportion (1{M i ) that is the weight : Thus the right side of this equation is proportional to Since C is the number of females per single male, then y i C is also the number of female partners of male carriers of gene encoding the strategyP i : These females ''pull the rope'' toward the value of P P pr : On the other side, a team of x i female carriers of this gene ''pulls the rope'' toward the value P i : It is evident here that the expression of strategies of parental individuals determines the fate of their descendants, by the setting of the sex ratio among them.

An unresolved problem: the role of the male subpopulation equilibrium
Recall that, during the slow phase of the sex ratio dynamics, P P pr~P : Note that, if in rules (8) we substitute P P pr instead of P and f i instead of M i we obtain the following rules: f i increases when S j f j P j v 1 2 and S j f j P j vP i or S j f j P j w 1 2 and S j f j P j wP i , f i decreases when P i vS j f j P j v 1 2 or P i wS j f j P j w 1 2 , f i is stable when: f i~0 or f i~1 or P i~Sj f j P j : These describe the changes of a female subpopulation state when the MSE condition is satisfied (Lemma 1 from [1]). This leads to the problem of the role of the MSE phenomenon, which is responsible for the rapid phase of convergence and the dynamics of sex ratios in the carrier subpopulations. The first idea that comes to mind to explain this phenomenon is that the male subpopulation equilibrium is equivalent to some stable sex ratio in the carrier subpopulation (the equilibrium of the ''tug of war'' mechanism), which is conditional on current values of P, P P pr and P i : The rapid phase will then be equivalent to convergence to this stable value. When the subpopulation reaches a stable sex ratio, then it simply follows changes of the primary (and in effect the secondary) sex ratio, which are equivalent to the slow phase of convergence. Unfortunately, this idea is false. As shown in [1], when the MSE conditions are satisfied for all strategies, then all males in the population have the same fitness. If we assume that carrier subpopulations are in their stable states, then for all strategies females will have fitness equal to males. So, when all males have equal fitness, and all females have fitness equal to males, then all individuals in the population have equal fitness. In this case, the population would be in a global stationary state, which is not true. The nature and role of the male subpopulation equilibrium are the subjects of a subsequent paper.

Supporting Information
File S1 Appendices A-D. (DOC)