Study system

We collected quantitative host-parasitoid interaction data from 16 training sites (8 from native forest and 8 from plantation) and 32 testing, i.e. validation, sites (16 from native forest and 16 from plantation) in the Nelson/Marlborough region of New Zealand (further details of sampling methods are provided in Frost et al. [21] and Peralta et al. [62]). Each training site consisted of 2 transects; an edge (10 m towards the forest interior from the edge between the two forest types, and running parallel to the edge) and interior (400–500 m from the forest edge). Each test site consisted of 1 edge transect (from its respective forest type). The test sites did not include interior transects; however, we chose to include both edge and interior transects in the training sites to use all available data. At each transect (50 m x 2 m), all foliage-dwelling Lepidoptera larvae (‘hosts’) were collected from plants and reared in the laboratory until they developed into adults or parasitoids emerged (Hymenoptera, Diptera and Nematoda; though we excluded Nematoda parasitoids from all our analyses as the traits we measured, see ‘Traits and phylogenies’ below, were not suited to these species). Parasitoids and hosts were then identified to species level where possible using available taxonomic information [63–66] and expert assistance (J. Dugdale, J. Berry, and R. Schnitzler), and otherwise to morphospecies. For parasitoids, morphospecies were validated as species using molecular methods. The training sites were sampled seven times over two summer seasons (season 1: December 2009, January, February 2010; season 2: October, November 2010, January, February 2011). To determine whether predictions of apparent competition were equally effective for large experimental and small natural changes in host abundances, host abundance was experimentally reduced in half of the plantation test sites by aerial spraying with a caterpillar-specific biopesticide in the middle of sampling. Test sites were sampled twice before this host reduction treatment (time t, October, November 2010) and twice after (time t+1, January, February 2011). Because Frost et al. [21] found that the predictive ability of apparent competition did not differ across the host-spray treatment, we did not include this variable in our analyses.

Traits, phylogenies, and abundance

We collected traits of host and parasitoid species that have been identified as being useful for predicting interactions: body size, trophic generality (measured as normalised number of food resource species, termed ‘normalised degree’ in network studies; ND), biogeographic status (native vs. exotic), and phenology. Note that ND was measured for parasitoids based on their interactions with hosts and for hosts based on their interactions with their plant hosts, rather than parasitoids. We chose these traits because body size is correlated with many important characteristics [67] and is often the best predictor of species (including host-parasitoid) interactions [45], while species generalism tends to be conserved across native and alien ranges [68], and recent evidence suggests that niche processes for species interactions (and therefore predictions based on traits and phylogenies) are weaker for generalists than specialists, and for exotic species than native species [59]. Additionally, phenology plays a role in determining interactions [56, 69]; we therefore calculated two aspects of phenology for both host and parasitoid species: abundance of each species in each month and phenological overlap between each host-parasitoid pair (see ‘Phenology’ in S1 Appendix). Because phylogenies can also help to explain interaction patterns [44, 46] by capturing variation in difficult to measure traits, we constructed host and parasitoid phylogenies to include along with the traits in the prediction of interactions (see ‘Traits and phylogenies’ in S1 Appendix). Additionally, abundance (a neutral process) can be important for determining interactions [55, 56]; we therefore summed abundance across sites and sampling dates for both host and parasitoid species, to be used in a later part of the analysis (see ‘Abundance’ in S1 Appendix).

Predicting direct effects

We compared two machine-learning techniques (random forest and KNN) for the prediction of quantitative host-parasitoid interactions (i.e. direct effects). Training site data were used to inform the models and test-site data (at time t) were used to test (i.e. validate) the models (Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Summary of methods.

(a) Lepidopteran larvae (‘hosts’) were collected from eight native forest sites and eight plantation forest sites (we call these ‘training sites’ as data from these sites were used to train the models). (b) Hosts were identified and reared to determine parasitism rates and identities. We created three quantitative training meta-networks (‘native’, ‘plantation’, ‘combined’) by pooling interaction data across native sites, plantation sites, and all sites, respectively. (c) We used machine-learning models (random forest and KNN), informed by training meta-networks, traits and phylogenies, to predict interaction frequencies at test sites (16 native forest and 16 plantation forest) at time t. We created a separate model for each of the three training meta-networks, for both machine-learning approaches (i.e., 6 models: random forest/KNN ‘native’/’plantation’/’combined’). (d, e) We then tested whether the ability of these machine-learning models to predict interaction frequencies depended on species generality or abundance, and whether predictive ability depended on the data (i.e. native, plantation, combined) used to inform the model. (f) We then calculated the potential for apparent competition, d_ij, between each pair of host species in the training data (using both native and plantation sites data). (g) We calculated expected parasitism rates (E) for host species at time = t+1 test sites using these d_ij values, along with predicted attack rates (α) calculated from our machine-learning predicted networks at time = t test sites, and abundance (n) from the test sites. We calculated expected parasitism rates (E) for both random forest and KNN. Additionally, we calculated expected parasitism rates using a data-based approach used by Frost et al. [21]. We tested whether expected parasitism rates could significantly predict observed parasitism rates at time = t+1 test sites (h, i).

https://doi.org/10.1371/journal.pone.0252448.g001

To predict interactions at new locations (or for new species such as biocontrol agents), the random-forest model learns (from the training data) which host and/or parasitoid trait (or phylogenetic) combinations are likely to correspond to an interaction occurrence. The algorithm then uses these rules, along with the traits (or phylogenetic position) of new species, to predict the probability of interaction occurrence between new combinations of host and parasitoid species pairs. These interaction probabilities could be used in risk assessment if it can be demonstrated that they correspond with higher observed interaction frequencies (which we test below). Random forest therefore assumes that the strength of trait (or phylogenetic) matching between species determines, in part, the likelihood of an interaction between them. In contrast to random forest, which incorporates matching of parasitoid traits (or phylogenetic position) to those of hosts, KNN uses the host-use preferences of parasitoid species (from the training data) with similar traits (or phylogenetic position) to recommend (i.e., predict) interaction partners and interaction frequencies for new parasitoid species. This is analogous to Netflix recommending films to users based on the preferences of similar users [47]. Thus, in KNN, parasitoid traits are used to determine similarity among parasitoid species, rather than to match with host traits as in random forest (see ‘Machine-learning techniques’ in S1 Appendix).

Predicting indirect effects

To predict indirect interactions (apparent competition), we used the quantitative interaction network (i.e. direct interactions at each test site at time t) predicted by each model (random-forest or KNN) trained on the ‘combined’ data (native forest and plantation data). This quantitative interaction network can be used to project a network of indirect linkages among hosts that share parasitoids. We can then predict how attack rates on each host should change as a function of changes in the abundance of host species that share parasitoids with that host (i.e. the indirect effect of ‘apparent competition’). Therefore, we predicted how patterns in attack would change if the indirect (parasitoid-sharing) links among hosts matched those predicted by the machine-learning methods. We then went on to validate how these predicted changes in attack rate matched real changes observed (between times t and t+1). Note that the machine-learning predicted networks are based only on species traits and phylogenies (i.e. information that captures niche processes), and abundance is included later into the predictions of indirect effects (see Eq 1 below). We used these ‘combined’ models because, in a biocontrol context, available data may not be specific to a target habitat, but rather aggregated from different sites and habitats.

Predicting indirect effects involved three steps. First, we used the ‘combined’ training meta-network (see above) to calculate a regional quantitative measure of shared parasitoids, also known as the potential for apparent competition, d_ij, using Equation 6 from Müller et al. [23]; see ‘Measure of shared parasitism’ in S1 Appendix. This d_ij value measures, for every pair of host species in a community, the proportion of parasitoids attacking host species i that recruited from host species j [23]. We used the PAC function from the bipartite v 2.15 R package [72] to calculate d_ij. Second, for each machine-learning approach (random forest and KNN), we used this measure of shared parasitism (d_ij), along with random-forest- (or KNN)-predicted initial attack rates (α in Eq 1 in S1 Appendix, see also ‘Calculating α from predicted networks’ in S1 Appendix) by each parasitoid on each host from time t at the test sites combined with known changes in host abundances (the change in abundance of each host species from sample t to t+1, in the site for which predictions are being made) to calculate the expected parasitism rate (E) at time t+1 of host species i for each test site. Parasitism rate is the number of parasitism events divided by the number of hosts sampled. We used Equation 3 from Frost et al. [21] to calculate E: (1) where n is host abundance, t is the first-time step of data (before experimental host reduction in half of the sites, with t+1 being after), α is the link strength (that is, number of attacks), i and j are a focal host species pair, H is the total number of host species, and l is all parasitoid species, from 1 to P (the total number of parasitoid species).

Finally, we tested whether this expected parasitism rate significantly predicted the observed parasitism rate of each host species i at time t+1 test sites (see equation 2 in S1 Appendix for how observed parasitism was calculated).

The logic of this approach is that a bipartite network of interactions, where links represent attack rates between hosts and parasitoids (the nodes), can be projected into a unipartite network with only hosts as nodes and links representing the proportional sharing of parasitoids (d_ij). Changes in abundance of hosts in this unipartite network should induce proportionate (and thus predictable) changes in the abundance of their parasitoids and cause a proportionate change in attack rates on any other hosts that share those parasitoid species (i.e. an indirect effect). Frost et al. [21] showed that these expected rates (E) significantly predicted observed parasitism rates, even when within-habitat intraspecific contributions (that is, delayed density dependent parasitism) were excluded from these expected rates, indicating that the indirect effects present could not simply be explained by within-habitat delayed density dependence. This method for predicting indirect effects has thus been experimentally validated using this dataset. Our aim here was to test whether the machine-learning-predicted networks (trained on data from different sites) could be used instead of observed networks to predict these indirect effects.

The number of host species for which we were able to calculate an expected parasitism rate at each site was much lower than the number of species collected at each site. This difference occurred because we could only calculate an expected parasitism rate for host species within a site that were collected at both time steps and had a non-zero predicted attack rate (α) at the first-time step (t, see Eq 1 above). However, for both machine-learning approaches, we used the potential for apparent competition (via the sharing of parasitoids) with every other host species that was collected in the site, to calculate the expected parasitism rates for these species. Therefore, predictions of expected parasitism rates for these species were based on data from the whole network. The numbers of species within sites for which it was possible to calculate an expected parasitism rate were 91 and 84 for the random-forest and KNN approaches, respectively.

Statistical analyses

Can random forest and KNN predict interaction frequencies and does this predictive ability vary with species’ generality and abundance, and whether the model was trained on data from the same vs. different habitat?

We tested whether machine-learning techniques (random forest and KNN) can predict host-parasitoid interaction frequencies. For the less complex set of models (including only random-forest-predicted probability or KNN-predicted frequency), we found that random forest was more successful than KNN at predicting interaction frequencies. Random-forest-predicted probability was significantly positively related to observed interaction frequency in all three models (‘native’, ‘plantation’, ‘combined’), and captured a reasonably high amount of variance (marginal R² > 0.338 for all three models) (Fig 2A; Tables 1 and 3 in S1 Appendix). In contrast, KNN-predicted interaction frequency was not retained in any of the three KNN models (Fig 2B; Tables 2 and 3 in S1 Appendix). There was a low probability of the predicted-probability fixed effect being significant in 3/6 models by chance (p<0.001, calculated using the Bernoulli process), indicating that the results are unlikely to be due entirely to type I error. For all three models (‘native’, ‘plantation’, ‘combined’), the random-forest model predicted observed interaction frequency better (i.e. lower AIC) than the respective KNN model (Table 3 in S1 Appendix). Additionally, for 2/3 models (‘native’ and ‘combined’), the random-forest model captured a greater amount of variation in observed frequency (higher conditional R² values) than the respective KNN models. All best-fitting models included parasitoid identity as a random factor, and a random slope for random-forest-predicted probability (or KNN-predicted frequency) (except for the random-forest ‘plantation’ and ‘combined’ models which only included an observation-level random effect, along with parasitoid identity as a random factor in the ‘combined’ model).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Predictions of observed interaction frequency.

(A) Random-forest-predicted probability significantly predicted observed interaction frequency for all three random-forest models (‘combined’, ‘native’, ‘plantation’), whereas (B) KNN-predicted frequency was not retained in any of the best-fitting models (tested using Poisson linear mixed models, and dashed lines represent non-significant relationships). Deviance residuals of the best-fitting model excluding the predictor on the x-axis are plotted, to better reflect the partial effect of the predictor. Each point represents a host-parasitoid species pair present within a site at the first-time step (t) test sites.

https://doi.org/10.1371/journal.pone.0252448.g002

For the second set of models, testing whether the ability to predict interaction frequency differed for generalists vs. specialists, whether predictive ability depended on species’ abundances, and whether predictions can be made equally within vs. across habitats, we found that the predictive ability of KNN, but not random forest, depended on the generality (ND) of the interacting partners (Fig 3; Tables 4 and 5 in S1 Appendix). We also found that predictive ability depended on species’ abundances, but not on the source of the training data, for both random forest and KNN.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Overall, machine-learning predictions of observed interaction frequency varied with the generality (measured as normalised degree; ND) of the interacting partners for KNN, but not random-forest models, and depended on the abundance of the interacting partners for both models.

https://doi.org/10.1371/journal.pone.0252448.g003

There was a significant positive relationship between random-forest-predicted probability and observed interaction frequency in all three models (i.e. higher predicted probabilities corresponded to higher observed frequencies), qualitatively the same as for the less complex models (see above). In contrast, predicted frequency was only significantly related to observed interaction frequency in the ‘native’ KNN model (for the less complex models, predicted frequency was not retained in any of the three KNN models).

The host ND fixed effect was included in all three KNN models, along with parasitoid ND in the ‘native’ and ‘combined’ KNN models; however, neither of these variables were included in any of the best-fitting random-forest models. There was a significant positive three-way interaction between predicted frequency, host ND, and parasitoid ND in the KNN ‘native’ model, such that when ND of both host and parasitoid were high, the slope of the relationship between predicted frequency and observed interaction frequency was more positive than expected based on additivity of the main effects alone. However, the presence of negative (and significant) two-way interactions between predicted frequency and ND (of both host and parasitoid), along with a negative host ND main effect, which outweighed the positive parasitoid ND main effect, meant that interactions involving a single generalist partner would tend to occur at lower frequency and have a weaker relationship between predicted and observed frequency, but these effects would be weakened when both partners were generalists. Although the predicted frequency x host ND x parasitoid ND interaction was retained in the KNN ‘combined’ model, it was not significant in this model. However, there was a significant negative two-way interaction between predicted frequency and host ND in this model, along with a negative host ND main effect, indicating that when host ND is high, the relationship between predicted frequency and observed interaction frequency was more negative than expected. In the KNN ‘plantation’ model, host ND did not interact with predicted frequency (and parasitoid ND was not retained in this model).

There was a significant negative three-way interaction between predicted probability (or predicted frequency), host abundance, and parasitoid abundance in the ‘plantation’ and ‘combined’ models for both random forest and KNN, meaning that the relationship between predicted probability (or predicted frequency) was more negative when the abundance of both interacting partners was high. However, the presence of two-way interactions involving predicted probability/frequency and abundance meant that high abundance of both interacting partners affected predictions differently to when the abundance of only one of the interacting partners was high. Because our data contain few species with very high abundance values, caution should be taken when interpreting these results.

There was a significant three-way interaction between predicted-frequency and generality of the interacting partners (host and parasitoid) in the KNN ‘native’ model. Additionally, there was a significant three-way interaction between predicted-frequency (or -probability) and abundance of the interacting partners for 4/6 models. Two are presented here to demonstrate the main patterns: (A) KNN ‘native’ model (which had a significant positive predicted frequency x host ND x parasitoid ND interaction), (B) random-forest ‘combined’ model, which had a significant host abundance x parasitoid abundance x predicted probability interaction (as did the ‘plantation’ random-forest model and the KNN ‘plantation’ and ‘combined’ models). However, the limited data between species with extreme abundance values, means this result should be treated with caution. Deviance residuals of the best-fitting model excluding the host and parasitoid ND (or abundance) variables are plotted as colour contours, to indicate whether observed interaction frequencies are higher (red) or lower (blue) than expected based on machine-learning predictions, showing how predictions vary with generality (or abundance) of the interacting partners. Each point represents a host-parasitoid species pair present within a site at the first-time step (t) test sites.

For all random-forest and KNN models, the relationship between interaction probability (or predicted frequency) and observed interaction frequency did not differ between the two forest types (i.e. the forest type fixed effect did not interact significantly with predicted-probability or -frequency in any of the 6 models), implying that predictions of interaction frequency (based on random-forest-predicted probabilities or KNN-predicted frequencies) are equally effective in both forest types and do not depend on the source of training data.

No random effects were retained in any of the best-fitting models for both random forest and KNN (except for the KNN ‘native’ model, which included the parasitoid identity random effect, which explained some variance; R_c² = 0.147). The fixed effects explained more variation in interaction frequency in all three random-forest models (R² > 0.23) than their respective KNN models (R² > 0.11).

The above results contrast with the analyses for predicting interaction occurrence (see ‘Tests on the prediction of interaction occurrence’ in S1 Appendix). For both random forest and KNN, all three models (‘native’, ‘plantation’, ‘combined’) predicted interaction frequencies better than binary interaction occurrences (models predicting frequencies had higher marginal and conditional R² values than their respective, i.e. ‘native’, ‘plantation’, ‘combined’, model predicting interaction occurrences). This suggests that raw predicted probabilities (or frequencies) are more useful for predicting interactions than assigning presence vs. absence of an interaction occurrence based on an arbitrary probability/frequency threshold.

Can we predict indirect effects using machine-learning (random forest and KNN) informed methods?

For both random-forest and KNN approaches, there was a significant positive relationship between expected parasitism rate and observed parasitism rate (random forest: z value = 2.72, p = 0.006, df = 85; KNN: z value = 2.80, p = 0.005, df = 77) (Fig 4; Table 7 in S1 Appendix). The probability of this relationship being significant in both models by chance was very low (p<0.001, calculated using the Bernoulli process). Although these models did not explain a large proportion of the variance (random forest marginal R² = 0.155, KNN-marginal R² = 0.0919), in nature we would expect many processes beyond just trait matching to determine field parasitism rates, though we take the significant effects of both RF- and KNN-predicted parasitism rates as evidence that these variables were informative (Table 8 in S1 Appendix). The site random effect was retained in both models and also explained a reasonable amount of variance (random forest: conditional R² = 0.403, KNN: conditional R² = 0.325).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Predictions of observed parasitism rate.

(A) Random-forest- and (B) KNN- expected parasitism rate both significantly predicted observed parasitism rate. Residuals of the best-fitting model excluding the predictor on the x axis are plotted. Each point represents a host species within a site in the second-time step (t+1) test sites. Three and four outliers, respectively, were removed from (A) and (B), though the results with the outliers included were qualitatively the same in both cases.

https://doi.org/10.1371/journal.pone.0252448.g004

Do predictions of indirect effects based on machine-learning-predicted networks differ from those based on observed network data?

The three methods (random forest, KNN and data-based, as used by Frost et al. [21]) did not differ in their ability to predict observed parasitism rates (difference in AICc<1) (Table 9 in S1 Appendix). In fact, with the reduced dataset (N = 20; data that could be used for all three analyses), expected parasitism rates did not even significantly predict observed parasitism rates for the random-forest and KNN approaches, contrasting with the results with the full dataset (see above), likely due to reduced statistical power (Table 10 in S1 Appendix). Similarly, expected parasitism rate was not retained in the best-fitting model for the data-based approach. The site random effect was retained in all best-fitting models and captured a reasonable amount of variance (conditional R² = 0.344 for all models).

Random forest predicts host-parasitoid interactions better than KNN

The better predictive ability of random forest, compared with KNN (Fig 2), may be explained by the different ‘philosophies’ of the two methods. While random forest incorporates trait and phylogenetic matching, which are important for determining, and therefore predicting, interactions [37–39, 44, 46], KNN assumes that similar parasitoid species interact with similar host species. However, parasitoid species with similar traits may be dissimilar in their host use, for at least two reasons. First, adaptation to different host plants can lead to insect speciation [85–88], and this may ‘cascade’ to higher trophic levels, leading to host-associated differentiation in parasitoids [89–92]. Similarly, host-shifts (including the formation of ‘host races’) can contribute to parasitoid speciation [93, 94], and this parasitoid speciation, by either adaptation or host shifts, may result in closely related parasitoid species that attack different sets of host species. In this way, parasitoid species with the most similar trait values (or that are most closely related) may be dissimilar in their host use, resulting in poor predictions of host use by KNN models. Second, it is possible that parasitoid species deemed as similar, based on the selection of traits we considered (see ‘Traits and phylogenies’ in Methods), differed in an important trait that we did not include (for example, counter-defences against immune responses of hosts [95]). However, this may be unlikely because we included parasitoid phylogenies, in addition to traits, which capture variation in a variety of measured and unmeasured traits. Additionally, random forest is a more complex algorithm than KNN. Because random forest is based on many trees (each with many branches), random forest is able to account for interactions between variables and non-linearities. This may also have contributed to the better predictive ability of random forest compared to KNN.

Overall, species’ generality and abundance impacts predictions of direct effects

The relative strength of the niche and neutral processes that generate plant-frugivore interactions has been shown to change with species generalism [59], and we similarly found for host-parasitoid interactions that the ability of KNN methods to predict observed interaction frequency depended on the generality (ND) of the interacting partners (Fig 3). Surprisingly, random-forest predictions did not depend on species generality; our random-forest models incorporated trait and phylogenetic matching (two indicators of niche processes), and a dependence of predictive ability on species generality would be congruent with the idea that the strength of niche processes varies with generality [59]. However, we found that predictions of direct effects, by both machine-learning methods, depended on species’ abundances, suggesting that strength of niche vs neutral processes may vary with species’ abundance.

Predicting networks within vs. across habitat types

Overall, predictions of host-parasitoid interaction frequencies (and occurrences; see ‘Tests on the prediction of interaction occurrence’ in S1 Appendix) using machine-learning techniques were equally effective in both habitat types (plantation and native forest). Furthermore, predictions of interaction frequencies (and occurrences) did not depend on the source of the training data, which contrasts with recent research showing that predicting networks across habitats (i.e. using data from one habitat to predict interactions at a different habitat) is difficult [51]. This difficulty is, in part, because interaction frequencies can differ markedly among habitat types (e.g., [96, 97]), arising from both changes in random encounters (due to changes in abundance) and interaction preferences based on niche processes [51]. Congruent with this, Staniczenko et al. [51] found that models that captured systematic changes in interaction preferences between different habitat types had the best predictive ability, while models based only on interaction data performed poorly. Interaction preferences are likely to differ more with increasing habitat dissimilarity (e.g., [96]). Network data from a more similar habitat (such as the two forest types we compared here) will therefore likely predict interactions better than the forest versus agricultural habitats compared by Staniczenko et al. [51], as differences in interaction preferences will likely be smaller. Additionally, similar habitats are more likely to contain species with similar values for traits, which would likely increase the predictive ability of trait-based models. Therefore, the similarity in physical structure (canopy height, etc) and high host and parasitoid species overlap between plantation and native forest habitat may explain why we found no difference in predictive ability of interaction frequencies between the two habitat types. Nevertheless, at face value, the ability to predict equally across habitats and with different training data are positive attributes for biocontrol risk assessment, and suggest that a common dataset (e.g., national databases of species interactions) could be used for risk assessment of interaction frequencies (a pre-requisite for predicting density-mediated indirect effects) in a range of habitats.

Predicting indirect effects

We found that, although machine-learning informed models significantly predicted indirect effects (i.e., for both random-forest and KNN approaches, expected parasitism rate was a significant predictor of observed parasitism rate) (Fig 4), the explanatory power of both these approaches was reasonably low (approximately 10% of variance). Surprisingly, the prediction of indirect effects was similar (i.e., similar marginal R² values) to the prediction of direct effects for KNN (Tables 3 and 8 in S1 Appendix), though the highest overall marginal R² value was observed for the model of direct effects with host and parasitoid generality (ND) and abundance included (Table 6 in S1 Appendix). Together, these results may suggest that the direct effects that were predicted well irrespective of generality (or abundance) were those that were most important for driving indirect effects. Previous work on pollination networks suggests that interactions with a strong signal of trait matching are more spatiotemporally constant and functionally important [98], so it is feasible that the interactions best predicted by machine-learning methods based on traits are those that are least variable and contribute most to indirect effects. These methods are therefore a long way from being able to make accurate predictions of the full strength of parasitoid-mediated indirect effects. However, they represent a promising starting point, and could potentially be used to rank hosts along a continuum from low to high risk of indirect effects. These machine-learning approaches, which use predicted networks rather than observed networks (at time t test sites) to calculate initial attack rates, α, (see ‘Predicting indirect effects’ in Methods), have the potential to capture interactions that are not recorded in the observed networks due to inadequate sampling.

Interestingly, the machine-learning approaches and the data-based approach used by Frost et al. [21] did not differ in their ability to predict indirect effects for the common dataset (N = 20) of host species, for which it was possible to calculate an expected parasitism rate with all three methods; this lack of difference was possibly due to reduced statistical power. Moreover, we found that the data-based approach did not significantly predict observed parasitism rate, contrasting the results of Frost et al. [21]. This is likely because we used slightly different data to Frost et al. [21]. In particular, Frost et al. [21] used only edge training sites and explicitly included host habitat (i.e., host species found in different habitats were treated as separate species) when calculating d_ij (a measure of shared parasitism), whereas we used both edge and interior training sites and did not explicitly consider host habitat. Species composition and interactions differ between edge and interior sites in this system [99, 100]; thus the inclusion of interior sites may have caused the different results. Additionally, we removed seven host species (for which we could not determine their host biogeographic status) from all our analyses, and we did not include nematodes because their traits were not comparable to those of insect parasitoids. Collectively, these differences likely explain the incongruence with the findings of Frost et al. [21].

Although we found that machine-learning-informed models could significantly predict indirect effects (discussed above), part of the field validation data arose from natural changes in abundance, rather than manipulative additions or removals of individuals. It is therefore possible that an unmeasured factor is correlated with both parasitoid sharing and changes in host abundance or attack rates. For example, habitat preferences of parasitoid species may co-vary with parasitoid sharing, such that the sharing of parasitoids (i.e., apparent competition) is not the cause of the model’s ability to predict observed parasitism rates, but rather a correlate. Additionally, we predicted expected parasitism rate, which captures changes in attack rates, but not subsequent changes in host population abundances; a prerequisite for demonstrating apparent competition between host species mediated by shared parasitoids. However, previous work on the same study system showed that accurate predictions of changes in host abundances followed from the ability to predict changes in attack rates, and were, in fact, better predicted than changes in attack rates [21].

Further caveats

Ecological networks are difficult to sample and the difficulty of distinguishing true non-interactions from unobserved interactions, which may simply result from inadequate sampling, is widely recognised [56, 101–103]. Although the ‘meta-networks’ used to train our machine-learning models (see ‘Training meta-networks’ in S1 Appendix) comprised the pooled data across training sites and sampling dates, to maximise the resolution of potential host-parasitoid links, we acknowledge that these meta-networks may not capture all realised interactions.

A further caveat to using these approaches in biocontrol is that predicting interactions involving exotic species (e.g., control agents) may be more difficult than predicting interactions among native species. The lack of co-evolutionary history between exotic and native species in the community suggests that trait or phylogenetic matching will be weaker for these interactions. In fact, Peralta et al. [59] found that niche processes were weaker, and Coux et al. [60] found that neutral processes were stronger, for plant-frugivore interactions involving exotic species than those among native species, suggesting that methods incorporating trait or phylogenetic matching will perform poorly for interactions involving exotic species compared to interactions between native species. For this reason, an approach that blends information on traits and abundances (as we did in the second linear modelling step) may help to alleviate the problems of predicting more neutrally-driven interactions. In addition, exotic species tend to interact with numerous species [50, 104, 105] and, as discussed above, interactions involving generalists may be more difficult to predict based on trait matching than those involving specialists [59]. However, this is less of a concern as biocontrol agents are typically chosen because they are specialists on the target pest.