Emergent coexistence is common in models with random pairwise interactions

We investigate the possibility of stable communities with EC in the GLV model (Eq. 2). We start from a pool of S species and random interaction coefficients A_ij defined by and , and study if the system reaches a stable state, in which species coexist with interaction coefficients A^*, where at least one pair of the coexisting species would not coexist outside of the community due to competitive exclusion. Species i and j are considered an excluding pair and would not coexist in isolation if either , or both. We call such a stable state that contains at least one exclusionary pair an EC state. Note that EC states require, by definition, coexistence between at least three species: a stable state with one survivor has no interactions, and a stable state with two survivors requires that their interaction is not exclusionary.

To explore the presence of EC states in the GLV model numerically, we define a range of possible mean () and heterogeneity () of interactions between species (Fig 2A). For each pair of and values, we sample 100 systems, each with a different interaction matrix A and random initial conditions x_i(t = 0). We integrate the GLV dynamics for these 100 systems, and we divide the number of stable states that contain EC against the number of stable states that contain at least three coexisting species, with or without EC (Fig 2A). This provides a proxy for how common or rare it is to find stable communities that are EC states: if we compare the number of EC states against all possible stable states without the three-species requirement, we observe the same results with a region where EC fractions are very low, indicating not that EC states are very rare, but that we are sampling values where coexistence of more than two species is very rare (Fig A in S1 Text). To focus on the role played by interspecies interactions, we present in the main text the results with homogeneous and normalized growth rates (r_i = r = 1) and carrying capacities (K_i = K = 1), and set network connectivity equal to 1 (C = 1). We describe in detail in S1 Text II.A,B,J how assuming heterogeneous growth rates, carrying capacities, and reduced connectivity yields qualitatively similar results to those presented here.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Emergent coexistence in models of pairwise interactions.

The GLV model has a rich parameter space with four dynamical regimes depending on the values of the mean interaction strength () and standard deviation () [33,39,53]. Note that in some GLV model studies [33], and are rescaled with pool diversity S, which implies a focus on the weak interactions domain. Other studies, such as here and [39,54], do not rescale interactions with diversity; this allows for stronger competition and leads to the full parameter space as presented here. (A) Simulations of the GLV model starting from a pool of S=80 species and interactions randomly sampled from a normal distribution (see Materials and Methods). For each, we simulate 100 systems, each with a different A matrix and different random initial conditions, and divide the number of final stable states that harbor at least one excluding pair (EC states) against the number of stable states that containt at least three coexisting species, with or without EC. Most stable communities harbor EC within an area delimited by two analytically estimated boundaries (, , red lines, see MM). For random and uncorrelated interactions, the EC regime is found inside the multistability regime, one of the four dynamical regimes of the GLV model (delimited by dashed lines, [54]). (B) Vertical slices of (A) at and and increasing , showing the transitions towards the regime where EC is prevalent. (C) For each community with EC found in (A), we measure the final diversity S^* and the fraction of excluding pairs. Simulated communities (purple circles) and empirical data ([4,11,12,24], colored squares) are consistent with the predicted minimum fraction (dark dashed line, one pair over all possible pairs) and maximum fraction of excluding pairs (red dashed line, obtained from predicting the maximum possible diversity and fraction of excluding interactions given , see MM and S1 Text II.B.2) for the mathematical details.

https://doi.org/10.1371/journal.pcbi.1014116.g002

In all cases, results reveal that EC is a common outcome in the GLV model, provided that pairwise interactions are competitive and sufficiently heterogeneous (Fig 2B). If heterogeneity is low (), EC is still possible provided that mean interaction strength is close to self-regulation (). In this large domain of the parameter space, a large fraction of stable communities harbor species pairs that cannot coexist in isolation (Fig 2A and 2B). We find analytical lower limits for the boundaries of this EC regime ( and in Fig 2A), by considering that, for EC to be observed in the final community, a necessary –albeit not sufficient– condition is that exclusionary interactions and coexisting pairs need to be present in the initial species pool (see MM as well as S1 Text II.A for a step-by-step derivation). Notably, most communities with EC in the assumption of random and uncorrelated interactions are found inside a regime of the GLV model characterized by multiple stable states [33,53]. If interactions are not uncorrelated, but purely anti-symmetrical, the GLV model does not harbor multistability, and EC can happen inside the unique fixed point regime [33]. If interactions are purely symmetrical, we show mathematically in S1 Text II.A that stable communities cannot harbor EC (see [21] for a similar discussion on interaction symmetry and growth-competition trade-offs). The presence of multistability implies that different initial conditions, yet with the same species pool S and interactions A, can lead to different subsets of surviving species [54]. This is consistent with empirical observations, that have found EC in systems where varying the initial species abundances leads to different stable species compositions [11,55]. This highlights a possible, yet under-explored, link between interaction heterogeneity, EC and community multistability.

By replicating the study of Fig 2A with different models, we find that EC is still a prevalent outcome when growth rates and carrying capacities are heterogeneous and when interactions are sparse (see MM and Fig D, Fig F and Fig S in S1 Text). Additionally, the same qualitative results hold beyond the assumption of fully random interactions, when A_ij have a predator-prey structure [56], are highly skewed [41,49,57], are row- and column-correlated due to heterogeneous carrying capacities [38] or have nested or single-resource structures (Fig D in S1 Text). Finally, we show that stable communities with EC are also present in non-GLV models with Allee effects, multilayer interactions, saturating responses or sublinear growth ([53,58], Fig E in S1 Text), while recent work has found EC across a family of models with growth-competition and competition-colonization trade-offs [21]. Taken together, these results suggest that EC is not particular to the GLV model or the random interaction assumption, but rather is a general outcome in species-rich models once many interactions are competitive and heterogeneous. Our analysis so far only detects states that have at least one excluding pair, treating EC as a binary property. Following recent experimental [11] and numerical [21] results, a follow up question is to study the fraction of excluding pairs and hence the extent of EC that a single community can sustain.

Analytical predictions match observations of the fraction of excluding pairs

A striking observation of EC experiments is that as many as of species pairs in a stable community do not coexist in co-culture [11,12]. Here, for each community found in the range of Fig 2A with at least one excluding pair, we measure the fraction of species pairs that interact via competitive exclusion and hence would not coexist if isolated (Fig 2C, MM). We find that communities can sustain a highly variable fraction of excluding pairs, with the maximum fraction of such pairs decaying with species diversity (Fig 2C). The decrease of strongly competitive pairs is consistent with classical results from random matrix theory and the GLV model, for which larger communities become unstable if competition strength overcomes given limits [33,46]. Interestingly, Fig 2C conveys the intuition that EC is intrinsically a phenomenon of intermediate diversity, that falls between the common modeling extremes of either very small or large-S limits and would be hard to capture by either limiting perspective [53,59].

To estimate the maximum competition, and hence the maximum fraction of excluding pairs that communities with EC can sustain before becoming unstable, we use recent corrections of May’s limit for communities of moderate size. This allows us to predict the maximum diversity and fraction of exclusionary interactions for a community given (Fig 2C red dashed line, see MM for an explanation of the method and S1 Text II.B.2 for the complete mathematical development, [54].) Additionally, the minimal fraction of excluding pairs in EC communities is simply one excluding pair over the total number of species pairs in a community of size S^*, (Fig 2C, dark dashed line). It is interesting to recall that these two bounds are qualitatively different: the lower bound reflects the presence of at least one exclusionary pair and is generically attained for all EC states. In comparison, the higher bound reflects the maximum fraction of excluding pairs that specific systems with random pairwise interactions and GLV dynamics can attain. In both cases, the bounds on the fraction of excluding pairs are consistent with experimental data [4,11,12], and confirm that the fraction of excluding pairs can be very large in communities of moderate size: a community of 10 species could sustain as many as 60% of species pairs that do not coexist in co-culture.

The qualitative trends observed in Fig 2C remain robust under heterogeneous carrying capacities or nested or hierarchical interactions (Fig F in S1 Text), as well as when incorporating growth rate heterogeneity or matrix sparsity, yet with small quantitative variations: increasing growth rate heterogeneity reduces the likelihood of observing large stable communities that sustain many excluding pairs. In contrast, greater network sparsity weakens overall competition, making diverse communities with many excluding pairs more likely (Fig G in S1 Text). Finally, recent work has found that larger fractions of excluding pairs can coexist if carrying capacities are sampled to ensure species coexistence [21]. In sum, we hypothesize that finding more excluding pairs in larger communities requires the presence of non-random interaction structures that can further enhance species coexistence. Given the results shown in Fig 2, a natural question to ask is how all these species can coexist if so many interactions are strongly competitive and there are no higher-order effects.

Multispecies coexistence decouples from pairwise interactions at high collectivity

Early work on three-species models showed that positive indirect effects of length two, as in “the enemy of my enemy is my friend”, could allow competitors to coexist [22,29]. Yet, in networks of many interacting species and long chains of indirect effects, it becomes hard to determine who is friend and who is enemy. Importantly, recent work has shed new light to our understanding of indirect effects in species-rich communities [32]. Given a pool of S species interacting through matrix A, equilibrium states of the GLV model will be those in which at least a subset of S^* species with interactions encoded in A^* coexist with positive abundances. This translates to finding a subset of species with positive abundances by solving:

(3)

Here the i,j element of the inverse matrix, , encodes the net effect of species j on the equilibrium abundance of species i [29–32] (see S1 Text I.B for a detailed step by step development). The link between direct and net effects can be better understood by using the Neumann series [32]:

(4)

which allows us to express the equilibrium abundance of a given species as

(5)

The abundance of a species is affected by direct interactions, but also by the cumulative effects of indirect interactions through all other species [32]. The inverse matrix therefore contains not only the effects of self-regulation and direct interactions explicitly written in Eqs. (1,2), but also all the indirect effects across increasingly longer chains of species, that together make up the net effects between species in a community [32,60,61]. The central question of EC is therefore to understand the relation between A^*, the direct interactions between species pairs, and , the net effects between species in the context of the full community.

A first fundamental result is that, even in a community where all interactions are competitive ( ∀ ), indirect effects involving an even number of species can be positive, so that some competitors positively impact the equilibrium abundance of others. For the communities with EC found in Fig 2A, we find that half of the elements of are positive, meaning that competitors are similarly exerting positive and negative net effects on others (Fig I in S1 Text). Given this result, the reductionist expectation is that, if we can estimate A_ij experimentally, we can assemble a community by selecting species pairs that can coexist [4], or, at least, by identifying the right “enemy of my enemy” motifs through the powers of A^*. Yet, to which extent is this possible, and how does the diversity of species and their interactions preclude our capacity to learn multispecies coexistence from A?

As presented in detail in [32] and [62], the series of Eqs. (4–5), for which we can write the abundance of a given species as a finite sum of chains of indirect effects across all species in the community, does not always converge. If interactions are consistently strong, each longer indirect effect might be more important than shorter ones, to the point where the series of Eq. (5) diverges. We hypothesize here that strong indirect effects provide an explanation to empirical observations of EC. More concretely, the series no longer converges when the collectivity metric , the spectral radius of the interaction matrix A^*, is larger than 1 ([32,63]). A fundamental consequence of high collectivity is that the direct, pairwise effect of species j on species i () becomes uncorrelated from the net effect of species j on i in the context of the full community (). Species j can therefore compete strongly with species i, but, through complex indirect effects across the community, may exert a positive net effect on it (Figs 3B, 3C and Fig F in S1 Text). The collectivity metric therefore quantifies the extent to which pairwise interactions correlate with (and hence inform about) the coexistence of given species within a species community (see S1 Text II.E and Fig J in S1 Text for a detailed analysis on the correlations between direct and net effects matrices).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Strong indirect effects decouple pairwise coexistence from community coexistence.

Collectivity (; A) and condition number (; D) for stable communities with (purple) and without (gray) EC, sampled within the range of Fig 2A (see MM). We also plot values of empirical matrices in [12], the only dataset from Fig 2A for which A^* matrices are estimated and and can be measured (green squares). (A) Most communities with EC have collectivity () close to or larger than 1 (dark dashed line), with analytical estimates predicting (red dashed lines, see MM). (B) For a community with low , the direct interactions between species (each off-diagonal element of the matrix of direct interactions , x-axis) and the net effects within the context of the full community (each off-diagonal element of the matrix of net species effects , y-axis) are highly correlated (Pearson correlation coefficient r and p-value p). (C) Instead, for a community with high , direct and net effects become uncorrelated: a species j can be a strong competitor when isolated against i, but have a positive impact on the abundance of i through interactions with the rest of the community (red dashed circle). (D) Most simulated and empirical [12] interaction matrices with EC have a condition much larger than 1, implying that small measurement errors of pairwise interactions can potentially induce large errors in the prediction of species coexistence. Condition number is plotted in logarithmic scale, and we show the lowest expectation for in communities with EC (red dashed line), which increases with diversity as (see MM).

https://doi.org/10.1371/journal.pcbi.1014116.g003

Again, we generate initial pools of S = 80 species and interaction matrices A sampled within the range of Fig 2A, we sample subcommunities from A that fulfill feasibility and stability conditions, and measure as the modulus of the largest eigenvalue of A^* for each stable community (see MM). Communities with no EC (Fig 3A, gray) can have , indicative of weak indirect effects, but also , indicating that indirect effects are strong. For example, many strong but not excluding competitors () can have high and no EC. In comparison, most communities with EC (Fig 3A, purple) have close to or larger than 1, with states becoming rarer for larger communities. Predictions from random matrix theory (red dashed lines) indicate that the smallest possible in these EC communities increases bilinearly with species diversity and interaction strength as (see MM). This implies that communities with EC will easily become highly collective and cross the threshold as diversity increases. Our results indicate that collectivity , even if is not a strict condition for EC, provides the simplest metric to quantify the complexity of indirect effects and, therefore, the extent to which pairwise observations provide information about multispecies community composition. As collectivity increases, the likelihood of correctly assembling communities based on pairwise coexistence decreases abruptly see S1 Text II.G and Fig L in S1 Text for simulated tests on assemblying communities from pairwise interactions with varying values).

If species coexistence (Eq. 3) cannot be directly inferred from adding chains of pairwise interactions (Eq. 4–5), an alternative approach could be to infer it by estimating experimentally and then inverting numerically. The feasibility of this method can be assessed using the condition number , which quantifies how small measurement errors in A^* are amplified when inverting the matrix, with being the best-case scenario where the magnitude of the errors is not amplified [64–66]. We study the same communities for which we previously measured , and we measure their condition number numerically using the singular value decomposition (see MM). Our analysis reveals that can remain low for stable communities with weak interactions and no EC, yet it rapidly increases with diversity for communities where strong indirect effects and EC are present (Fig 3D and Fig H in S1 Text). The condition number follows , where s_i are the largest (M) and smallest (m) singular values of the matrix A^*, and we use analytical estimates of s_i from random matrix theory to define a lower bound for in communities with EC (Fig 3E, red dashed line, see MM). The accuracy of inferring species abundances (and hence species coexistence) by inverting an imprecisely measured A^* matrix decays abruptly as increases: assuming for example an error of about 10% in measurements of A_ij and a system with a moderate , only about 50% of the predicted coexisting sets of species would truly coexist, while this precision drops to only 15% of successful predictions for (see S1 Text II.F and Fig L in S1 Text for simulated tests on predicting coexistence with measurement errors in A). This means that small errors in the measurements of pairwise interactions can lead to substantial errors in the inverse matrix and therefore a decreased predictability of species abundances at equilibrium.

These results align with evidence that reductionist assembly rules work well for small communities of three or four species [4,67], but can break down in more diverse communities where indirect effects can become pervasive [11]. While obtaining exact values of and for a given community would require precise and often inaccessible knowledge of the full interaction matrix A^* [43], random matrix theory provides a first approximation to estimate bounds for collectivity metrics only from the statistics of interactions and not the full matrix (Fig 3, red dashed lines, see [32] for a study of the validity of these estimates when having incomplete knowledge of A^*). An important open challenge is therefore to quantify how empirical interaction matrices deviate from random approximations, and how these deviations alter indirect effects and collectivity. By doing so, our results suggest that and will offer valuable metrics to infer collectivity and ecological predictability given the statistics of an empirical interaction matrix, as we demonstrate in two proposed experimental case studies in S1 Text II.G.

Balanced feedback loops maintain stability under strong competition

The results above describe the mechanisms by which strongly competing species can coexist due to indirect effects. Beyond coexistence, experimental communities with EC have also been found to be dynamically stable [4,11], and we study here the mechanisms that guarantee this stability. Mathematically, linear stability can be captured by the eigenvalues of the Jacobian matrix at equilibrium having negative real part [56].

Theoretical results pioneered by May predict that a community will become unstable once diversity, interaction strength and heterogeneity overcome a predictable threshold [33,46,56]. We measure the diversity S^* and interaction statistics of stable communities found with the same method of Fig 3A and 3D, and estimate their instability threshold, for which the community would be expected to become unstable if [33,54]. We find that communities with EC remain stable even if competition is stronger (more negative) than this theoretical limit (Fig M in S1 Text).

This apparent violation is because random matrix predictions only operate for very large communities, as numerically tested in [54], whereas EC experiments and simulations typically involve moderate diversity (Fig 2C). In communities of moderate size, not just the statistical properties of interactions, but their species-level organization influences stability. To understand how realistic communities between from the S = 2 and the limits can remain stable beyond May’s threshold for competition, we recall the Routh-Hurwitz criteria, necessary conditions for the stability of a Jacobian matrix of any given size [68]. These conditions require that interactions are structured so that negative, stabilizing loops control positive loops that would instead amplify perturbations (see MM, S1 Text II.H and [28,69] for detailed explanations on the Routh-Hurwitz criteria). We find that the stable communities with EC and moderate diversity, even when they violate predictions from random matrix theory, still fulfill the Routh-Hurwitz criteria as expected (Fig M and Fig N in S1 Text). Experimental observations of stable EC under strong competition could therefore be explained by the presence of specific loop architectures in the structure of pairwise interactions [63].

Emergent coexistence does not require intransitive competition

Coexistence under strong competition has often been linked to intransitivity in theoretical models [1,23,70,71]. Intransitivity can be defined by a lack of hierarchy in species competition, by which no species can become the most dominant of the community. This can be assessed by measuring the abundance of triplets forming a rock-paper-scissors (RPS) motif, or the likelihood of exclusions by species with a lower rank (Low Rank Exclusions, LRE) in the competitive hierarchy, among many possible metrics (see MM and S1 Text II.I for additional intransitivity metrics). Yet, in empirical communities with more than three species, the presence and role of these motifs remain unclear. While some communities appear highly intransitive [72], evidence indicates that intransitivity is in fact rare across species-rich systems [1,25,26]. Experimental communities with EC have been observed with strikingly low intransitivity percentages (0−0.3% of triplets are RPS, 1−3% of exclusions are LRE in [11–13]).

Here, we measure the fractions of exclusions that are LRE and triplets that are RPS for stable communities with EC sampled again from interaction matrices within the range of Fig 2A (see MM). We find that intransitivity metrics in communities with EC become lower as species diversity increases (Fig 4 and Fig L and Fig M in S1 Text). In small communities, specific intransitive motifs are common and necessary for coexistence (Fig 2C), consistent with classical RPS observations in small empirical systems [24]. In more complex and collective communities, many interaction chains can rescue species from extinction, and the presence of intransitive motifs tends towards the random expectation as diversity increases (Fig 4). Yet, in some experiments, rock-paper-scissors motifs are almost absent from diverse EC communities assembled in the lab [11,13], which is markedly below the random expectation of % of triplets being rock-paper-scissors (Fig 4C and 4D, see S1 Text II.I for a detailed analysis of the statistics of RPS triplets in random matrices).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Emergent coexistence does not require intransitive competition.

We sample stable communities with EC as done in Fig 3 for the GLV model with different interaction matrices, and measure their intransitivity properties (see MM). We compare intransitivity in communities with EC found under random interactions (A^*, purple, within the parameter range of Fig 2A), reshuffled A^* (gray, reshuffling the off-diagonal elements of each previously found A^* matrix), nested interactions (T^*, red), interactions correlated with carrying capacities (, gold) and single-resource competition with cross-feeding (, green). See MM for a description of how each interaction matrix is built. (A,B) Mean (dots) and standard deviation (shaded areas) for the fraction of exclusions where a lower-ranked species excludes a higher-ranked one (LRE) across stable communities with EC, where the rank is defined as the number of wins minus number of losses divided by the number of total interactions (see MM). (C,D) Mean (dots) and standard deviation (shaded areas) for the fraction of triplets that are rock-paper-scissors (RPS) across the same stable communities, where a triplet is a trio of species all connected by excluding interactions (see MM). For panels (B,D), Kernel Density Estimates (KDE, see S1 Text II.I.4 for the numerical method) allow us to visualize the expected probability distribution of intransitivity metrics in communities with species.

https://doi.org/10.1371/journal.pcbi.1014116.g004

One possible explanation by which EC could emerge with very low intransitivity is that competitive interactions are somehow hierarchical. This could be captured, for example, by a fully nested or triangular interaction matrix T, where the upper triangle of the matrix has random interactions as defined above, yet the lower triangle is filled with zeros (see MM). This implies a hierarchy in which species 1 is impacted by all others, species 2 by all except species 1, and so on. However, there is no a priori ordering by which the strengths of these interactions are themselves hierarchical: species 1 is impacted by all others, but the strength of these impacts is still sampled at random.

As recently proposed in [38], another mechanism by which interactions could be hierarchical without the need to impose a triangular structure is related to heterogeneity in carrying capacities. In the main text we have sampled A_ij from a random distribution, yet A_ij are originally defined as . If a is a random matrix, but K_i are heterogeneous, the resulting A_ij could have row and column correlations related to species carrying capacities (see MM), by which species with highest K_i are the less impacted by others, and the most impactful to others [38,59]. To differentiate from the random uncorrelated matrix A we have studied above, we here define as the matrix with elements , with a_ij sampled as A_ij above and K_i sampled from a uniform distribution (see MM).

Finally, hierarchical interactions could also emerge from specific properties of the studied microbial communities, in which species compete for a single resource and cooperate via cross-feeding [11,34,55]. Here we define a scenario in which competition is defined by a single trait (the capacity to consume the available resource, ), so that interactions become strictly hierarchical [48]. On top of this, we can incorporate other weak competition mechanisms [73], together with positive cross-feeding C_ij > 0 [34,55]. The new competition matrix B that implicitly describes resource competition and cross-feeding writes (see MM)

(6)

When two species compete, the species with the lowest perceives stronger competition, resulting in a hierarchical structure where the most efficient consumer dominates [17].

We study the EC and intransitivity properties of communities from the GLV model with a triangular matrix T, a row- and column-correlated matrix , and a competition and cross-feeding matrix B. We find that the EC regime and fraction of excluding pairs in these structured matrices are qualitatively similar to those of the random GLV model studied above (Fig D and Fig G in S1 Text), yet intransitivity is much lower than the random expectation due to different forms of competitive hierarchies (Fig 4). This result is also consistent with the absence of intransitivity in models with competition-colonization trade-offs recently shown in [21], and confirms that intransitive motifs are not required for EC in models with pairwise interactions. Our results indicate that experimental observations of common and pervasive EC without intransitivity could be related to communities governed by disordered yet hierarchical species interaction matrices with strong indirect effects [11,21].

Rescaling and parametrizing the GLV model

We study the GLV model of S interacting species [9,33,54], with species labeled as i = 1,2,...S and abundance dynamics following

(7)

To focus on the role played by interspecies interactions, we divide species abundances by carrying capacities () and rescale interaction strengths relative to self regulation (). We refer the reader to, e.g., [38,59] for a discussion on the role of different choices of the GLV model and how different rescalings relate to matrix randomizations. In any case, the normalization allows us to reach equation (2) where r_i and A_ij are the only controlling parameters of the dynamics. To focus on interspecies interactions, in the main text we apply the typical assumption (see, e.g., [32,54]) of homogeneous growth rates r_i = r and rescale time as , so that only interactions encoded in A_ij control the system dynamics. A_ij in the main text are sampled from a Gaussian distribution with and (Fig 2A, 2B and 2C), and we discuss below the impacts of considering heterogeneous carrying capacities. To study the role of network connectivity, we define the probability p = C at which a given interaction link A_ij is realized, yielding an Erdős-Rényi graph. We present results with C = 1 in the main text, and explore the effects of lowering C in S1 Text II.A.6. To study the role of growth rate heterogeneity, we sample r_i from a Gaussian distribution . We present in S1 Text II.J the results for fixed and for the case where , so that growth rates and species interactions have the same standard deviation. We also explore in S1 Text II.A.4 alternative parametrizations for the GLV model with nested, symmetric, antisymmetric, skewed and resource competition and cross-feeding matrices, and in S1 Text II.A.5 the study of EC in the GLV model with migration, the GLV model with saturating interactions, a model with multilayer interactions and a model with sublinear growth dynamics.

Simulating GLV dynamics

A simulation starts by defining a random matrix of S = 80 interacting species following the parameterizations and models proposed above. For all models, we set random initial conditions , integrate the system for timesteps and check if abundances are the same after additional timesteps to test for stationarity [53,54]. Once a final state is reached, we identify those species that have survived with positive abundance and capture their interactions in A^*. To draw Fig 2A, for each pair we sample a random interaction matrix A and random initial conditions, and measure the number of stable states that harbor at least one exclusionary interaction against the number of stable states that harbor at least 3 coexisting species, with or without exclusion interactions. A state has EC if , and there is at least one exclusionary interaction . The fraction of excluding pairs (Fig 2C) is measured as the number of pairs for which either or are smaller than -1, divided by the total number of pairs of surviving species, . The GLV equations are integrated numerically using scipy.integrate.solve_ivp with an adaptive Runge-Kutta (RK45) scheme; species whose abundance falls below a fixed extinction threshold are removed by setting their abundance and growth rate to zero during integration.

Sampling stable subsets of A^*

To find stable states from Eq. (1) without the need to run the complete dynamics, we use the following complementary technique from [54] that allows to efficiently generate Figs 3 and 4. Given a matrix of interactions A defined as above, we select a random subset of species and their interactions A^*. We first ask if this set of species is feasible by solving Eq. (3) and studying if all final abundances are positive (all species in the subset survive). We then obtain the Jacobian matrix for this species subset from , and check if all eigenvalues of the Jacobian are negative for linear stability. The subsets of A^* that fulfill these two conditions are stable and feasible solutions of equation (2) in the absence of migration [54]. We then use the same methods as above to evaluate if these subsets harbor EC. Previous work [54] analyzed the differences between stable states found by simulating the complete dynamics and those found by sampling subsets and evaluating their stability and feasibility. The latter method sometimes finds smaller and slightly more competitive communities, that could be invaded by weaker competitors from the pool when simulating the complete dynamics with all initial conditions positive. However, the two methods generate EC communities that fulfill feasibility and stability constraints and hence are valid solutions for Eq. (2) given certain initial conditions. To ensure that enough stable communities are found with different degrees of EC (Fig 3) and intransitivity (Fig 4) to observe accurate trends, we sample 10⁷ species subsets for each diversity and different values, and test them for feasibility, stability and EC.

The boundaries of the EC regime

We use simple statistical arguments to find a first estimate for the boundaries of the EC regime in the parameter space (Fig 2A and 2B, red lines). We propose two necessary, albeit not sufficient conditions, that provide good estimates for the EC regime in the case of large S and random and uncorrelated interactions presented in the main text. These conditions are based on the following heuristic argument: for EC to be present in the final surviving community (), it is necessary that at least one strong competition term was already present in the initial species pool (), and that at least two species coexist in a pair so that they can sustain the excluded one (A_kl,, see S1 Text II.A for a detailed analysis). We therefore search for the minimal combination for which at least one element within elements of A will be exclusionary (, S1 Text II.A). The probability of is , where is the cumulative distribution function (CDF) of the standard normal distribution. If S is large, this function predicts a sharp transition from P = 0 to P = 1 for given values ( in Fig 2). Using a similar argument for the case where most competitive terms are exclusionary and dynamics fall within the competitive exclusion regime, we assume that to find at least two species that coexist and can sustain a third one in the surviving interaction matrix A^*, we need at least a pair of coexisting species in the original pool (A_kl,). Using similar probabilistic methods, we therefore search for the maximal combination for which at least one species pair is not exclusionary (A_kl,) and can rescue a third excluded species. Note that these boundaries define necessary, but not sufficient conditions: for example, for the case of purely symmetric interactions, one finds that even if the initial pool can contain exclusionary elements (, ), this will not ensure that EC is present in the final community, as symmetric A_ij elements cannot yield EC (see S1 Text II.A for a mathematical proof of the absence of EC in symmetric interaction matrices).

The maximum fraction of excluding pairs

Given a stable community that harbors EC (at least one exclusionary element, ), we measure the fraction of excluding pairs as the fraction of pairs of species (i,j) in that community for which at least one of the two interactions or is exclusionary. In Fig 2C, we plot in purple these values for all communities harboring EC found in the simulations of Fig 2A. As described in full detail in S1 Text II.B, to obtain a first analytical estimate for the maximum fraction of pairs that a community of a given size can contain, we merge two results: given a pair of values that generate a matrix A, we want to know (i) the maximum possible species richness S^*, and (ii) the maximum fraction of elements of a stable community found for that interaction matrix. (i) Previous work on the GLV model provided an analytical estimate for the maximum species richness S^* that is possible to observe given inside the multistability domain [54]. This method is a few-species approximation of classical results in community ecology, for which a community becomes unstable beyond a given size S^* and interaction strength and heterogeneity [33,46]. Using this method, we provide a first estimate for the largest possible community S^* that can be stable given an original pool of interactions A sampled from a pair. (ii) We use probabilistic arguments to estimate the maximum fraction of elements given . Following the observation that most stable states in the multistability domain of the GLV model tend to harbor similar or lower competition than the original pool [54], we assume that the maximum fraction of elements is, at most, the fraction of elements in the pool, so that stable states that contain more competitive interactions than the original pool are assumed to be rare. The problem then is simplified to estimating the fraction of elements in a Gaussian sample of size . Taken together, for the whole range of Fig 2A, we provide a statistical bound for both the largest possible community (S^*, x-axis) and the maximum fraction of excluding pairs it can contain (y-axis) that together make the red dashed line of Fig 2C (see S1 Text II.B for a step-by-step development). Finally, the minimum fraction of excluding pairs in a community of S^* species is simply 1 over the number of pairs , because a community with zero excluding pairs will not harbor EC. The dark dashed line in Fig 2C is therefore frac.

Analytical bounds on collectivity

In Fig 3A we sample communities with and without EC from the GLV model within the parameter range of Fig 2A, and measure their diversity S^* and the spectral radius , the modulus of the largest eigenvalue of A^* obtained with numpy.linalg.eigvals. To obtain an estimate for the analytical minimum and maximum collectivity for each value of S^*, we use an estimate for the spectral radius of a random matrix, which for uncorrelated and fully connected matrices simplifies to [32,56]. This simplifies finding min and max (Fig 3A) to finding min and max of for each S^*, given a range as explained above and in S1 Text II.E and [54] in more detail.

Analytical bounds on condition number

In Fig 3D we follow the same sampling method and parameter range as for in Fig 3A described above, and we measure the diversity S^* and the condition number of the interaction matrix A^* of stable communities with (purple) and without (gray) EC. The condition number is estimated numerically in python using the numpy.linalg.cond method from numpy. To provide an analytical estimate for the smallest condition number of a given matrix A^*, we recall that , where s_i are the largest (M) and smallest (m) singular values of the matrix A^*. The smallest condition number is found from min , and the largest and smallest singular values have an analytical estimate for random matrices given and , as we describe in detail in S1 Text II.F and in [64].

Stability and feedback loops

An equilibrium community, defined by Eq. (3), is linearly stable if it recovers from infinitesimal perturbations, which requires that all eigenvalues of the Jacobian matrix evaluated at that equilibrium point are negative [56]. A classical result for large random matrices finds that a Jacobian with random interactions becomes unstable once S, or overcome a predictable threshold [33,46], yet we find that this threshold does not apply for communities with a moderate number of species and strong interactions. To understand what conditions ensure stability in these communities given that the classical May threshold is not fulfilled, we describe in detail in S1 Text II.H.3 the Routh-Hurwitz criteria, which provide strict necessary conditions for the stability of a Jacobian independently of its size, and allow for interesting ecological interpretations as better discussed in [28,69,84]. The Routh-Hurwitz criteria link linear stability to conditions on the coefficients C_i of the characteristic polynomial of the Jacobian matrix, found by solving det with (see S1 Text I.K for the step-by-step proof, [28,69]). For each community sampled from the GLV model in Fig 3A, we observe that it fulfills the Routh-Hurwitz criteria for stability by finding C_i numerically with numpy.poly and measuring the second Routh-Hurwitz stability condition, which requires as previously done in [84] (Fig M and Fig N in S1 Text).

Intransitive competition motifs

We measure intransitivity by the fraction of LRE and RPS in comparison with null expectations, and discuss other intransitivity metrics yielding similar results in S1 Text II.I.3. Following, e.g., [11,13], we define the rank of each species in a community as the number of species it excludes, minus the number of species it is excluded by. A low rank exclusion happens when a species with lower rank excludes a species with higher rank, which requires different motifs with at least 4 exclusions. We correct the LRE metric in Fig 4A for communities with at least four exclusionary interactions, to avoid underestimating LRE by studying many communities that have less than 4 excluding interactions (in Fig F in S1 Text we plot the uncorrected metric which yields very low LRE fractions for all values of S^*). We define a triplet as a set of three species connected by competitive exclusion (). A rock-paper-scissors triplet has a non-dominant architecture (A excludes B, B excludes C, C excludes A), and given a random triplet, 2 out of 8 configurations are RPS, making the null expectation for the fraction of triplets in a community RPS = 0.25. To visualize the probability distribution of rock-paper-scissors and low rank exclusion fractions (Fig 4), we sample stable communities with EC as described for Fig 3, measure the mean and standard deviation of observed intransitivity metrics across them and plot kernel density estimates (KDE) for these distributions as detailed in S1 Text II.I.4. We show similar results with other intransitivity metrics in S1 Text II.I.7 and Fig Q in S1 Text, by which intransitivity becomes rare in large communities with EC.

Hierarchy emerging from heterogeneous carrying capacities

In the main text, we study a model in which interactions A_ij are sampled following different randomization schemes. As recently proposed in [38], one possible mechanism by which interaction hierarchies can emerge is that carrying capacities are heterogeneous, and that this in turn impacts the structure of A. For example, one could assume that the original interactions a_ij are random, implying that the normalization (Eq. 3) is no longer random, but carries some row and column structure induced by the carrying capacities K_i. Following [38], we generate K_i uniformly from , a_ij from as typically done for A, and define a new correlated interaction matrix (to differentiate from A in the main text) as . If a_ij are random interactions, carry row and column correlations, by which the species with largest K_i is the most impactful to others and the less impacted by others, and so on. We refer the reader to [59] for an insightful discussion on whether a (the matrix for the interactions between abundances N_i) or A (the matrix for the interactions between relative yields x_i) are the matrices that one should consider as random within the perspective of disordered interactions in community ecology.

Nested and cross-feeding hierarchical matrices

To study if EC is still possible in other highly hierarchical (and hence, transitive) architectures, we build an idealized triangular matrix and a more realistic resource competition and cross-feeding matrix. The nested or triangular matrix T is generated from A within the same parameter range of the whole main text, but imposing that all elements in the lower triangle of A are zero ( if i < j, T_ij = 0 if i > j). This generates a hierarchy in the adjacency matrix (who impacts who), but not necessary in the strength of interactions, meaning that LRE could still be possible a priori. For B, positive values are sampled from and sorted by decreasing order to build a hierarchical matrix; are sampled from ; C_ij are sampled from (see S1 Text I.D for a detailed explanation). We do not have empirical support for these parameters and use them only as a proof of concept to explain how hierarchical competition and random cross-feeding can lead to transitive communities with EC. If the strength or heterogeneity of increases, the model becomes similar to the random interactions model. If the strength or heterogeneity of C_ij increases, cooperative interactions dominate and the model transitions towards the outgrowth regime of Fig 2A. Once these matrices are defined, we sample stable communities from the GLV model with different iterations of these matrices and S=80 as explained above.