Runaway Competition: A Correction and Extension of Results for a Model of Competitive Helping

We investigate and generalize an existing model of competitive helping within a biological market, first introduced for a population of competing individuals in which one individual provides help to all others; the rest compete for the help available from this individual by providing help themselves. Our generalized model comprises two strategies in which each individual of a specific type provides the same amount of help as all other individuals of that type. Each individual’s fitness function is dependent on this level of help, the cost of providing the help, and the fact that help is proportionally reciprocated by other individuals. Competitive helping occurs when individuals actively try to help more than other individuals. To assess the emergence of equilibrium help strategies as adopted by proportions of the population, we examine the competition over available help within two settings: replicator dynamics and agent-based numerical simulations. To move one step further in our generalization, we use the agent-based model to study the N-person competitive helping game, where all individuals in the population are heterogeneous with respect to help provided. Our results show that helping does not increase indefinitely with the population size, as concluded previously, and while there are some instances of an increase in help provided as a result of competition, this competition can be detrimental to all individuals and in most cases, one type simply gives up (thus evolving to a “no help” strategy). The degree to which an individual’s help is reciprocated by the others in the population has strong implications in the long-term behaviour of equilibrium help levels of types of individuals (and of individuals themselves); these equilibrium help levels diverge from existing conjectures in current literature. Lastly, small amounts of passively provided (costless) help results in runaway competition among all individuals.


Introduction
Cooperation and the evolution of altruism between non-kin has been the focus of much research in evolutionary biology [1][2][3][4][5]. Natural selection results in inherent competition between individuals to survive and reproduce, and thus one would presume only selfish behaviour should be rewarded. Yet cooperation exists among and between many diverse species of animals including humans. Darwin noted it as a potential problem for his theory of natural selection [6].
Trivers first introduced the concept of reciprocal altruism as a mechanism to explain the evolution of cooperation [7]. Direct reciprocity relies on repeated interactions between two altruistic individuals in which one individual has a net cost less than the other's net benefit [4,8]. The net benefit to both individuals over a series of interactions is hence positive. Axelrod and Hamilton presented the Prisoner's Dilemma as a game-theoretic model that could provide insight into the strategies leading to serial cooperation [1,9]. Since then, researchers have presented alternative models that may explain a variety of other phenomena [5,8,10,11].
Since costly help is a limited resource, an individual receiving attention and help from another results in less available for other individuals within the biological market [12,13]. As a result, there is inherent competition to attract that attention, and by actively helping, individuals can perhaps increase their value in the eyes of others [13][14][15]. Competitive helping can occur when the best helpers receive more help than those helping less and when the benefit of receiving additional help outweighs the investment cost [13,16]. More specifically, it occurs when individuals actively try to help more than everyone else. This sort of competitive altruism can serve as a courtship display, especially by males [17]. The potlatch, a gift-giving feast practiced by First Nations people along the Northwest Coast of North America, is another example of competitive altruism that sometimes cascaded into economic collapse [18].
In this paper, we generalize and correct the analysis of a competitive helping model started by Barclay in [13]. In this model, individuals divide their attention among other individuals and dispense help, similar to food dispensers in foraging models. The help that they produce is available to be claimed by those they pay attention to, and thus helpers will tend to receive more attention than non-helpers. In particular, the model comprises a population of individuals within a biological market confronted with the appearance of one individual who provides a benefit (i.e., help) to the other members of the population, who aren't currently providing any help. The fitness function of this individual is dependent on the level of help it provides, the cost of providing this help, and the fact that its offer of help is proportionally reciprocated by the other individuals. Barclay concludes that the appearance of this individual will motivate the other individuals to offer some help as well and that competing over the help available in the population results in an increasing amount of help being provided [13].
In particular, we generalize the original model such that it explicitly comprises two types of individuals, each providing different levels of help. In order to assess the emergence of equilibrium help strategies as adopted by proportions of the population, we examine the competition over available help within two settings: replicator dynamics and agent-based numerical simulations. Replicator dynamics [19] let us determine equilibrium values for the proportions of both types whose help strategies we fix; while type-based numerical simulations reveal behaviour and equilibrium values of help provided by fixed fractions of the population allowed to evolve their help incrementally over time. Finally, we examine individual-based numerical simulations to gain insight into equilibrium help levels of individuals who are heterogenous in their levels of help provided.
We investigate the conditions under which the strategy of each type persists. We find that by correcting the analysis and generalizing both the theoretical and computational framework for the study of the proposed biological market with competition over help, the previously conjectured hypotheses do not hold, while other interesting implications emerge. To the former point, the original analysis in [13] leads to a series of conjectures: that equilibrium levels of help (as in, optimal levels that maximize the payoff of an individual) depend on the population size, N, and the degree of having the help reciprocated, z; and that offering help should be an invading strategy (over a market of individuals who do not offer any help). Our analysis shows that both of these conclusions are not supported by the full mathematical investigation of the model. Most interestingly, we find that competition can be detrimental to those involved and small amounts of passively provided help results in runaway competition among all individuals.

Model generalization
Outlined below is an overview and generalization of the model by Barclay [13] where the interaction of individuals in the population embodies a non-cooperative game in which individuals compete for available help. Within the biological market, a fixed number of competing individuals, N, is assumed and the population is partitioned into two types: those who produce help at a rate of h p and those who produce help at a rate of h m , which Barclay [13] called regular and mutant individuals respectively. The proportion of the population of the first type is denoted S p and the proportion of the latter type S m , with S p + S m = 1.
If we have a population in which only one individual of a certain type exists, then from their perspective the available help to be competed over is (N − 1)h p , and at any time, they are competing with N − 2 individuals over the attention of each of the N − 1 other individuals. As a result, Barclay [13] defines the total amount of help received, r, by a single individual in a population consisting of only individuals of the other type as: We generalize this for populations consisting of two types with proportions of any size. An individual receives a portion of each other individuals' help according to the proportion that they provide, as is the case in Barclay's model [13]. Given the fractions of each type in the population, total help produced is S p Nh p + S m Nh m . Thus, for an arbitrary individual i (of either type) producing help at a rate of h i , the amount of help they receive is denoted by r i and defined as the sum of help received from all other individuals: We note for clarification purposes that h i and h j will be equal to one of h p or h m depending on the type of individuals i and j, respectively (i.e., if individual i is of type p, then h i will be equal to h p ).
The help received, r p and r m , can then be rewritten in terms of S p and S m as: Eq (1), as well as the generalized r in Eq (3), assumes that each individual receives help in perfect proportion to the relative amount they provide. This is inspired by existing literature in operant conditioning on the matching law [20,21], which predicts that there is a quantitative relationship between the relative rates of response and relative rates of reinforcement. In this model, each individual is a source with an associated benefit conferred upon others for associating with them. To account for the fact that perfect proportional matching rarely occurs in nature, Barclay [13] introduces z ! 0 to represent "the degree to which one receives help in proportion to the relative amount of help one provides, " based on deviations called undermatching, overmatching, and bias described by Baum [22,23] and Wearden and Burgess [24]. The expectation is that when z = 0, individuals receive equal help regardless of how much they provide; when z = 1, perfect proportional matching occurs; and when z = 1, the most generous individual receives all of the help produced (i.e., winner-takes-all). We will show that the latter case does not completely hold as expected given the way the model was constructed.
The assumption that individuals can sometimes produce help at no cost to themselves is also added, with k representing the amount of such passive help provided. Barclay assumes that all individuals provide the same amount of passive help, for model tractability purposes, and we follow suit. The generalized equation for r i then becomes: with, The total fitness payoff, W, to a single individual is defined as the difference between the fitness benefits and the cost of producing help [13]. Barclay [13] uses a diminishing marginal returns function rather than a constant cost/benefit ratio. This can be applied to an individual of either type in a similar manner as above: where m is the maximum fitness benefit from receiving help and x determines the rate of diminishing marginal returns (see Table 1 for a list of all variables).    The conclusions in [13] are that: i) competitive helping increases with the population size, N, and ii) with the degree of matching, z; iii) equilibrium help levels are highest at low-intermediate values of x; iv) some passively-produced help, k, is necessary in the absence of any mutant helper for competitive helping to invade, but once it has, passive helping is no longer necessary.
Our results, stemming from our analyses below, show that the above conclusions do not hold as stated: i) helping does not increase indefinitely with the population size; ii) we find a different level of help that maximizes payoff in the case examined by [13], and in general, equilibrium help values scale with the maximum fitness benefit, m. The level of help provided in order to stay competitive increases as z increases, but the actual number of individuals competing decreases. iii) Moreover, equilibrium levels of help are highest at low-intermediate values of x when z 1, but as z increases, equilibrium help is highest at increasing values of x; iii) passive helping can force competition between individuals even when it is disadvantageous for everyone, but without it, many individuals will provide no help at all.

Replicator dynamics model
Evolutionary stable states are found among stationary points of a replicator dynamics model [19]. Therefore, in this section, we formulate and analyze such a game that reflects our extension of the competitive helping model introduced above.
With the generalized equations for r and W given in Eqs (5) and (6), replicator equations are defined below in order to describe the change in the proportions of the population as a function of each type's payoff over time.
Equating the replicator equations in Eq (7) to zero and solving for equilibria may provide insight into the existence and behaviour of steady states of the types and their help levels.
Recall that S p = 1 − S m , which gives 0 = S m (1 − S m )(W m − W p ). This implies that the population is in equilibrium when either all individuals are of one type with none of the other (i.e., S m = 0, S p = 1; or S m = 1, S p = 0), or the payoffs for individuals of both types are the same (W m = W p ). If the payoff to an individual is the same regardless of their strategy, there is no incentive to switch to the other strategy.
Lemma 1. Eq (9) has solutions of the form: Proof. Assume that the level of active help provided is the same for both types (h m = h p ). Then the help received by both types is also equal (r m = r p ), and thus h m = h p leads to a solution of Eq (9). Now, assume that one of h m or h p is 0. By direct computation, we obtain that the other is equal to m − mx. In this case, the type's fitness benefits and costs are the same and thus the payoff is zero for both types with no incentive to switch.
We can find the level of active help that maximizes the payoff in each of these cases by finding the derivative of W i with respect to h i , setting it to zero, and solving for h i .
Remark If the proportions of types in the population are not of equal size, h p is not equal to h m and cannot be assumed. Since the model in [13] is formulated with S p 6 ¼ S m , the equation for h Ã found in that paper is incorrect. Lemma 2. Given the case in which h p = h m , we first find that S m must equal S p . From here, the optimal help provided can be derived as: Remark We note that the limit of Eq (10) as N becomes sufficiently large does not, in fact, continue to depend on N and thus helping does not increase indefinitely with population size, which seems to contradict the title claim in [13].
Lemma 3. When S p = 1 or S m = 1, the help provided by the corresponding type (h p or h m respectively) which maximizes the payoff is simply h Ã ¼ ðÀ x þ ffiffi ffi x p Þm À k and does not depend on the degree of matching, z, or the population size, N.
As z increases, we also note the increasing value of x for which equilibrium help is highest (Fig 1).
Remark By evaluating the Jacobian of the replicator equations at the equilibrium points found above, we can determine the sign of the eigenvalues and thus the stability of these equilibria. For the equilibrium points in which one of S p or S m is equal to 1 while the other is 0 (and assuming k = 0), we find one eigenvalue of multiplicity 2: h Ã (h Ã + mx − m)/(h Ã + mx). This eigenvalue is negative (and thus asymptotically stable) when h Ã ¼ ðÀ x þ ffiffi ffi x p Þm < mð1 À xÞ, which holds true for all values of m and x. The third equilibrium results in the same eigenvalue above with multiplicity one as well as an eigenvalue of zero. This indicates that there should be a line of equilibria along S p = S m , however, we require that S p + S m = 1, and thus should be stable at S p = S m = 0.5 provided h Ã < m(1 − x). In this case, we must use h Ã from Eq (10) and we find that the eigenvalue is negative only when z < 2/x−1, which is also the threshold for nonnegative payoffs.
Further equilibria, assuming unknown values for both h p and h m , not necessarily equal, are investigated in the next section via simulations since the derivative of the payoff function is too complicated to extract meaning from when S p 6 ¼ S m . Additionally, we examine the equilibrium trajectories of the population over time while studying their sensitivity to various degrees of matching, z > 0.

Experimental design
The basic experiment uses a population comprising two types of fixed proportions; all individuals of the same type are considered homogeneous. The help provided by individuals evolves over time as each type attempts to maximize their payoff. At each time step, both types simultaneously mutate the current help they provide (incrementally) and calculate the expected payoff from making this change given their knowledge of the help provided by the other type during the previous time step. If the expected payoff is greater than their current payoff, they update the help provided to this new value, otherwise their help remains unchanged.
The simulation is repeated for constant values of S m from 0 to 1 at increments of 0.1, with S p = 1 − S m . The aggregate population size in these simulations is chosen to be 100000 with m = 1, x = 0.5, k = 0, and a mutation rate of 0.001 as default values. Since the proportions of each type are fixed over time in these simulations, we investigate the resulting equilibrium in terms of h m and h p and the associated payoffs, as well as the effect of the degree of matching, z. Further simulations allow for the proportion to change over time in addition to the help provided by each such type. At each time step, there is a chance that some percentage of individuals will realize they would likely receive a higher payoff by switching to the opposite type and thus they do so. The likelihood of switching is proportional to the difference in current payoffs between types.
Additional experiments were performed using a heterogeneous population of 200 individuals in which each agent controls the amount of help they personally provide. We also investigate the model's sensitivity to positive levels of passive help, k, provided by individuals in the population.

Results
The trajectory towards the equilibrium help values of each group of simulations is partially shown in Fig 2a-2d for values of z 2 {0, 1, 2, 5} and when S m = S p = 0.5. In particular, these plots show the evolution of help over time, illuminating the basins of attraction for the equilibria. We note that the fixed proportions have some effect on the boundaries of these basins of attraction, while the initial choice of h m and h p dictates the direction of evolution and the resulting observed equilibrium.
When z = 0, there is only one equilibrium point and it shifts from ðh Ã Þm À k; 0Þ to (0, 0) as S m increases from 0 to 0.5 (Fig 2a shows S m = 0.5). A further shift to ð0; ðÀ x þ ffiffi ffi x p Þm À kÞ would occur if S m increased to 1. For z = 1, the single equilibrium remains constant at (h Ã , h Ã ), unaffected by the proportions of each type in the population (Fig  2b). As z increases beyond 1, a second and third equilibrium emerge in which one type provides help at a rate of ðÀ x þ ffiffi ffi x p Þm À k while the other type provides no help (Fig 2c and 2d). These two points remain constant regardless of the values for S m and S p , though the basins of attraction shift slightly. The main competitive helping equilibrium which is characterized by (h Ã , h Ã ) at S m = S p = 0.5 clearly shifts towards the upper right of the plot as z increases, while the basin of attraction for this particular equilibrium shrinks. We can then say that as z increases so too does the amount of helping that each type provides in order to stay competitive, though the frequency of such competition decreases. The initial values of h m and h p must be close enough for competition to arise, otherwise one type will cease providing help altogether. Though not shown in these plots, we also note that competition results in an increasingly worse (and eventually negative) payoff.
The heatmaps in Fig 3a and 3b further illustrate that h p equals h m only when S p also equals S m . When the proportions of each type in the population are uneven we see that the larger proportion provides more help but receives a smaller payoff than the other. Individuals would have an incentive to switch types if this were allowed and we would arrive at an equilibrium of S p = S m = 0.5 even though the payoff is lower for everyone.
By allowing individuals to move from one type to the other over the course of the simulation, we can identify which, if any, of the proportional splits represent stable equilibria. We expect to see S m = 1 and S p = 1 (from the replicator dynamics model) and S m = S p = 0.5 (from the analysis of the heatmaps in Fig 3a and 3b) emerge as equilibrium points, depending on the value of z. The trajectories are distorted when the proportions, S m and S p , are also allowed to evolve, but the equilibrium help values remain the same (Fig 4a and 4b). We note that in these cases the equilibrium represented by h m = h p corresponds directly to an S m = S p result; while the equilibrium represented by h m = 0, h p = h Ã coincides with S m = 0, S p = 1 (and vice versa).
For heterogeneous populations, and particularly in the case of larger z, we found that competitive helping results in an increasing level of help provided at an increasingly smaller and eventually negative payoff (Fig 5). We also note that the act of competitive helping decreases as z increases, with a greater number of individuals providing no help instead.
However, when a small amount of passive help is provided, the equilibrium represented by competitive helping (h m = h p ) becomes the only one observed over any number of replicates of the simulation (Fig 6). The threshold of passive helping required for this equilibrium to dominate is very low for values of z near 1 and increases slowly as z increases. It is interesting to note that the payoff for competitive helping is still negative near and slightly above this threshold and doesn't become positive unless more passive help is provided.
Remark An evolutionary stable strategy (ESS) is such that if adopted by all individuals in the population it cannot be invaded by any alternative strategy through natural selection [19,25]. Let E(S, [T]) represent the payoff for playing strategy S against the rest of the population

Conclusion
In simulating the competitive helping model with two types, we are able to shed some light on the behaviour hinted at in the original analysis [13]. We highlighted the main conjectures of the original model above and below we summarize our own findings in terms of these conjectures.
Help provided does not increase indefinitely as populations become larger. For populations of sufficient size, which we found to be as little as 200 individuals, the resulting equilibrium does not depend on population size. For smaller populations, the individuals fall short of converging on the optimal help to provide that was calculated above.
Equilibrium help values are largest at low-intermediate values of x for z 1 only. As z increases, equilibrium help is higher for larger values of x (Fig 1).
Critical threshold of two for invasion. Helping can invade non-helping only if two or more individuals start helping during the same time step. If only one individual tries to help, their payoff will be negative instead of zero if they did not help. With the lag in information about the amount of others' help provided in the simulations above, it would be impossible for helping to invade non-helping without some sort of collusion between individuals. Passive help reduces active help at equilibrium. When individuals produce some amount of passive help, the amount of active help provided is reduced by this amount (see Eq (10)).
Bifurcation as a result of degree of matching, z. The value of the equilibrium represented by h m = h p increases as z increases, while the proportion of simulations resulting in this particular equilibrium decreases and we begin to see equilibria in which one type stops helping altogether. The type that doesn't give up provides help at the optimal value, h Ã ¼ ðÀ x þ ffiffi ffi x p Þm (Fig 2a-2d).
Competitive helping is disadvantageous to all individuals. As z increases, the value of the equilibrium represented by h m = h p increases beyond the optimal value and results in increasingly negative payoffs for both types (Fig 5). The idea of a 'winner-take-all' biological market as z approaches infinity is hence false in this model. Passive helping forces competitive helping when z > 1. Small amounts of passive help results in the equilibrium represented by competitive helping (h m = h p ) being the only one observed over any number of replicates of the simulation (Fig 6).
Finally, we note that when proportions of each type in the population are able to evolve alongside help, the two equilibria represented by one of the types ceasing to provide help are actually characterized by no individuals of that type remaining (S m = 0 or S p = 0, when h m = 0 or h p = 0, respectively). As a result, all individuals are in fact providing equal and optimal help.
The main purpose of this research was to provide an in-depth analysis of an existing model of competitive helping, presumably derived from a biological perspective. In particular, we were interested in the emergence of equilibria through simulations which could not be found mathematically, and furthermore, the influence of parameters on the observed equilibrium help values. In looking to more clearly analyze the effects of the degree of matching, we showed that the results do not seem to correspond with the desired objective in constructing of the model (i.e., winner does not take all). Most interestingly, we found that competition is increasingly detrimental to those involved and when passive help (e.g. byproduct benefits, mutualisms) is provided such competition is ingrained in everyone.  [26] is an exceptionally high-quality, fast random number generator; coded in Java by Sean Luke [27]. (JAVA)