The ecological impacts of alien species invasion are a major threat to global biodiversity. The increasing number of invasion events by alien species and the high cost and difficulty of eradicating invasive species once established require the development of new methods and tools for predicting the most susceptible areas to invasion. Invasive pollinators pose serious threats to biodiversity and human activity due to their close relationship with many plants (including crop species) and high potential competitiveness for resources with native pollinators. Although at an early stage of expansion, the bumblebee species Bombus terrestris is becoming a representative case of pollinator invasion at a global scale, particularly given its high velocity of invasive spread and the increasing number of reports of its impacts on native bees and crops in many countries. We present here a methodological framework of habitat suitability modeling that integrates new approaches for detecting habitats that are susceptible to Bombus terrestris invasion at a global scale. Our approach did not include reported invaded locations in the modeling procedure; instead, those locations were used exclusively to evaluate the accuracy of the models in predicting suitability over regions already invaded. Moreover, a new and more intuitive approach was developed to select the models and evaluate different algorithms based on their performance and predictive convergence. Finally, we present a comprehensive global map of susceptibility to Bombus terrestris invasion that highlights priority areas for monitoring.
Citation: Acosta AL, Giannini TC, Imperatriz-Fonseca VL, Saraiva AM (2016) Worldwide Alien Invasion: A Methodological Approach to Forecast the Potential Spread of a Highly Invasive Pollinator. PLoS ONE 11(2): e0148295. https://doi.org/10.1371/journal.pone.0148295
Editor: Adrian G. Dyer, Monash University, AUSTRALIA
Received: July 31, 2015; Accepted: January 15, 2016; Published: February 16, 2016
Copyright: © 2016 Acosta et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by São Paulo State Research Foundation (2011/12779-7) to ALA (http://www.fapesp.br/) and National Counsel of Technological and Scientific Development (472702/2013-0) to TCG (http://www.cnpq.br/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The ecological impacts of species invasion area major threat to global biodiversity [1, 2], with widespread effects on humanity [3, 4]. An alien invasive species is defined as a taxon introduced outside its native range, either deliberately or accidentally, presenting a high growth rate and fast range expansion, with noticeable environmental impacts [5–8].
The number of invasion events by alien speciesis rapidly increasing around the globe [1, 9]. Thus, the development and application of new methods and tools that allows predict the most susceptible areas to invasion are needed.
Species distribution modeling (SDM) has been applied to forecast the potential occupancy of a wide range of invasive species. Examples include plants (e.g., [10–12]), insects (e.g., [13–16]), mollusks (e.g., ) and amphibians (e.g., [18, 19]).
According to Lavergne et al.(2010) , SDMs have been largely influenced by Hutchinson’s concept of the ecological niche . Currently, this type of ecological modeling is also considered able to estimate habitat suitability, as it uses reported occurrences to identify other places with similar suitable conditions . In this work, we choose Habitat Suitability Modeling (HSM) as a standard denomination of an analytical procedure that encompasses methods and concepts described by other researchers and authors as Species Distribution Modeling (SDM) or Ecological Niche Modeling (ENM).
Abiotic factors are considered the initial environmental barrier that an invasive species must overcome when it is introduced or when it invades a non-native environment [23–27] sincemost individuals cannot survive on an environment where abiotic factors exceed their physiological limits [22, 27]. At large scale, climatic factors can be considered among the most influential types of abiotic factors that limit or, in some cases, promote the expansion process of an invasive species [11, 25, 28, 29].
Several HSM methodological approaches have been proposed to improve their application. For example, some studies have compared the performance of different algorithms (e.g., ) or sample sizes (e.g., ); or examined how to fit models (e.g., ); ensemble multiple projections (e.g., ); included biotic interactions (e.g., ); or evaluated model performance (e.g., ). Additionally, some studies have used HSMs to assess information on invasive species at many spatial scales (e.g., [11, 36–39]). However, as far as we know, there is no methodological approach for predicting areas that are susceptible to bee invasion on a worldwide scale.
The bumblebee Bombus terrestris (L.)(Hymenoptera: Apidae) may become one of the most representative cases of bee invasion at global scale , especially considering its velocity of spread (e.g., 200 Km per year in Chile and Argentina ).
Bombus terrestrisis a eusocial bee with a relatively large and hairy body of high thermoregulatory capacity. This species exhibits a generalist feeding habits, being able to explore a wide range of floral resources. The wild, native distributional range of Bombus terrestris covers almost the whole of Europe, mainly the temperate and Mediterranean zones, and encompasses surrounding areas of Asia and Africa [40, 42, 43].
Bombus terrestris (hereafter referred to as Bt) is able to perform buzz pollination [40, 44,45], which improves the pollination success of plants with poricidal anthers. Bt provides this important ecosystem service to wild plants and crop species with high economic value such as tomato, pumpkin, eggplant, potato and pepper [44, 46, 47]. Bt is well adapted to artificial conditions, and because of its ease of handling and breeding, colonies have been developed in captivity and commercialized for over 20 years to improve crop pollination, mainly in the greenhouse [40, 47–49].
These Bt´s commercial colonies are delivered to many countries, including some places located outside their natural range. This international trade is reported as the main cause of Bt invasion in Chile, China, Israel, Japan, Mexico, South Africa, South Korea and Taiwan [40–42, 48–58]. Bt is also invasive species in New Zealand and Tasmania (AUS). In New Zealand, Bt was introduced around 1884 for crop pollination purposes and currently is widely spread over the islands [59, 60]. There is no precise information on when Bt was first introduced in Tasmania. Semmens et al., (1993)  suggest that Bt individuals were brought to Tasmania from New Zealand in 1992.
Currently, the invasive distribution of Bt is increasing in addition to direct human intervention, by means of its own dispersal capabilities (e.g. [50, 62]). Invasive Bt was reported as presenting some negative effects on native bees; for instance, competing for nesting sites , floral resources , as a potential vector of exotic diseases and parasites [65, 66] and changing plant-pollinator interactions in non-native environments, impacting crops, native plants and pollinators [67, 68].
Therefore, we consider Bt as a representative case study of an incipient worldwide alien invasion, undergoing rapid expansion in many countries outside its native range.
We present here a methodological framework with new analytical and geospatial strategies to evaluate, select and ensemble the habitat suitability models based on a multi-algorithm approach, aiming to increase the overall predictive accuracy for invasive species studies.
We used the framework to detect susceptible areas to Bt invasion in a worldwide scale and to detect the potential range of invasive spread from already invaded areas. In this work we use exclusively global high-resolution topoclimatic variables in order to provide a method with wide applicability for invasive species and large-scale events of invasions; but the method is not limited to these variables and can be integrated with other bionomic data if necessary.
The global map of susceptibility to Bt invasion delineates areas that should be monitored to avoid new deliberated introduction of colonies and can be used to guide the development of precautionary measures and policies in order to avoid or mitigate future impacts on natural environments and human activities.
Materials and Methods
We obtained 20 layers of environmental topoclimatic data (19 bioclimatic and altitude) from Worldclim , with a spatial resolution of 5 minutes of arc (cell size approximately 10 km) over a global range (with the exception of southern latitudes greater than 60°). These layers present data on altitude and annual trends of seasonality, temperature extremes and average precipitation over the last 50 years.
To reduce co-linearity among predictors, we performed a Pearson’s pairwise correlation procedure using R v.3.0.3  and selected those layers with Pearson's correlation coefficients less than 0.75. When two layers were highly correlated, we chose the one least correlated, yielding a total of nine layers: Mean Temperature Diurnal Range, Maximum Temperature of the Warmest Month, Temperature Annual Range, Precipitation of the Wettest Month, Precipitation of the Driest Month, Precipitation Seasonality, Precipitation of the Warmest Quarter, Precipitation of the Coldest Quarter, and Altitude.
We did not include additional environmental variables in the modeling procedure due to the lack of available knowledge about the relationship of Bt with other abiotic and biotic variables in non-native environments. Rather, because climate and altitude have several similar conditions and combinations around the globe, we considered the extrapolation of ecological considerations based solely on topoclimatic variables to be more reliable.
Bombus terrestris data
We used presence records for Bt surveyed from two main sources (for source details, see S1 File): 1) presence data extracted from published literature and 2) presence data obtained from collections and museums and compiled in internet biodiversity databases (mainly from GBIF–Global Biodiversity Information Facility). For the published data, when Bt presence was georeferenced by city or other place name, the geographical coordinates were extracted using the toponyms from the Global Administrative Areas Database . If occurrences were exhibited on a map only, we plotted the map into ArcGIS 10  to estimate the geographical coordinates for each point using the Georeferencing Tool.
The complete dataset was divided into two subsets based on published data (see S1 File): 1) presence within the native Bt range, referred to as Native Presence and 2) reported alien invasive presence, referred as Invasive Presence (see S1 File). Although some publications have suggested the presence of invasive Bt in China, Israel, Mexico, South Africa, South Korea, and Taiwan, we have found no recorded locations. Thus, we only considered as invaded those places with reports of sightings of individuals or colonies of Bt. Both subsets were plotted in ArcGIS10 .
First, we used the sample function and visual inspections to detect unreliable or erroneous records. Any record located in a water body class (e.g., oceans, rivers, lakes) according to GlobCover 2009 land cover (cell size of approximately 0.0028° = ~300 m)  was excluded. Furthermore, we excluded presence records from outside the range of the environmental layers selected for modeling or from countries lacking previous reports of native or invasive occurrences.
To reduce statistical overfitting due to a large number of native presence records (over 10,000 raw data records) and to avoid redundancy [74–76], we built a fishnet composed of square cells with the same spatial extent and grid cell size of the environmental layers (~10x10 km). Subsequently, we joined the fishnet with the native Bt presence dataset in ArcGIS. To do so, we treated each respective location as a unique presence, using the centroid of fishnet cells as the geographic coordinate of species presence. This procedure transformed Bt presence into a binary variable; i.e., we attributed a value of 1 to cells with one or more geospatially coincident presence records and a value of 0 to cells without presence records. The same procedure was then conducted with invasive Bt presence records.
All the modeling algorithms used in this work required the input of pseudo-absences when true absences were not available (in the case of MAXENT, background points) [77–81]. Because we cannot identify the true absence data for Bt, we generated two types of pseudo-absence datasets, as follows.
The first type of pseudo-absence dataset was generated by surveying the GBIF data provider to obtain presence records of Bombus species other than Bt, inside countries with reported native Bt presence. This dataset is hereafter referred as BOPA (Bombus Other than Bt- Pseudo-Absences). Based on studies of foraging distance in Bt and the maximum distance a worker bee can travel before returning to its nest (e.g., [43, 82, 83]), we determined that Bt workers usually forage close to their nests, generally within 1 km . However, under extreme circumstances (e.g., a scarcity of resources), a worker bee can travel greater distances; the furthest distance reported to date is 9.8 km . We used the largest distance (10 km) as the maximum displacement distance of individual Bt from their nests, and we assumed that most Bt sightings could be positioned within this maximum range during the field surveys, which provided each record in our presence dataset.
Considering the previous assumption, we plotted both the BOPA and native Bt presence datasets into ArcGIS 10 , and using the function select by location, we detected and excluded those BOPA records located less than 10 km from a Bt presence record. This procedure removed any pseudo-absence record (BOPA) located within the maximum spatial range of each Bt record. The remaining BOPA locations, i.e., field surveys reporting no Bt but reporting other Bombus spp., were considered as reliable pseudo-absence locations. As performed previously with the native and invasive dataset, we used the fishnet to remove repeated BOPA locations per grid cell.
The second type of pseudo-absence dataset was randomly generated following two steps. First, in ArcGIS 10 , we created a spatial buffer using the geographic locations of both datasets: BOPA and the native presence dataset (not the invasive one). In addition, we considered the 10 km distance to define the buffer radius per record for both datasets, and the area jointly covered by all buffers was used as a restriction zone in the next step. Second, we used the function create random points of ArcGIS to generate a random pseudo-absence dataset with three spatial constraints. Random points were generated exclusively inside the extent of the climatic layers, outside the restriction zone defined in the previous step, and outside the water bodies class of land cover GlobCover 2009 . Additionally, we prevented the creation of points in repeated locations per grid cell, attributing a minimum distance of 15km between points. The total number of random points was calculated as follows:
The total number of Random Pseudo-Absence Points (RPA) generated (193,340) was randomly fractionated into five subsets without replacement (38,668 per subset), such that each subset held only exclusive locations, with no duplicates. Subsequently, for each of the five random pseudo-absence subsets, we added the BOPA records, totaling 42,090 records per subset. This yielded a total 10 times the number of native presence data points per subset (as recommended by Chefaoui and Lobo, 2008) .
Bt native presence was randomly partitioned such that 75% of the data was used for training the model and the remainder (25%) was used for mathematical evaluation using True Skill Statistic (TSS) . This random partitioning was repeated five times to obtain a robust estimate of the algorithms' performance .
To achieve comparability among algorithm results and considering the most frequently used parameters, we maintained the default settings in Biomod2 following the parameters recommended by the authors [80, 81]. The MAXENT algorithm was used in the same way, with default settings recommended by the authors , but with a different memory allocation size. These default parameters are described on S2 File.
Twenty-five models were fitted per algorithm (ALGO1 to ALGO10); this was accomplished by combining five native presence partitioning (RUN1 to RUN5) and five pseudo-absence datasets (PA1 to PA5). The same pairwise combination was repeated in each round with each algorithm [ALGO1&RUN1&PA1; ALGO1&RUN1&PA2 (…); ALGO1&RUN2&PA1 (…); ALGO2&RUN1&PA1 (…)]. We obtained 250 models, considering all possible combinations of ALGOs (10), RUNs (5) and PAs (5).
We developed a sequence of three evaluation criteria to select from the obtained models, as follows:
TSS ≥ 0.8: First, we used a TSS evaluation index (True Skill Statistics; ) greater than or equal to |0.8|. We chose TSS instead of AUC (Area Under the Curve of Receiver-Operating Characteristics) because a threshold-dependent measure was necessary to define the spatial cutting point before delineating each respective suitable (binary value = 1) and unsuitable area (binary value = 0) per model. TSS values range between -1 and 1; positive values near one indicate high predictive accuracy. Negative values and zero proximity indicate that the model performs no better than chance, consequently, the models are not useful for detecting habitat suitability [88, 89]. There are no precise threshold TSS scores defined for model evaluation; we chose a more conservative value (TSS ≥ 0.8) than the threshold values frequently used (TSS ≥ 0.75; e.g., [39, 90–92]).
Based on invasive presence hit rate: In the second stage, we evaluated the accuracy of each remaining model (those with TSS ≥ 0.8) to infer suitability predictions over areas other than the proximal native range of Bt. This method takes into account the rating of the geospatial coincidence of the suitable areas detected by each model (binary value = 1) with the known locations of invasive Bt presence. In this context, the invasive dataset acts as a “validation dataset” for evaluating model accuracy by quantifying the accuracy of suitability detection of each model over the known distribution of invasive Bt.
To calculate this, we used the sensitivity component of TSS metrics. Thus, we calculated the probability of detection or hit rate (HR) according to the formula:
Invasive Hit Rate (IHR) = Hits(binary one) / (Hits(binary one) + Misses(binary zero))
We assumed that the most accurate models would be those that predicted the records of the validation dataset with higher precision, i.e., the models with higher Invasive Hit Rate (IHR) values. The minimum accuracy threshold for selecting the most precise models was defined by the overall average IHR value, meaning that each individual model that yielded an IHR value equal to or higher than the average IHR of all models was selected to proceed to Stage 3.
The procedure of intersection was replicated using the native Bt presence data, building a dataset used exclusively to compare stage performance in the final evaluation. Each value of this dataset was considered a measure of the Native Hit Rate (NHR).
Convergence of suitability predictions: The third stage was applied aiming to filter models with statistical biases related to under- and over-fitting. These undesirable statistical effects have to be considered mainly when a massive number of presence data points and wide-extent, high-resolution environmental variables are used. The extensive amount of information incorporated into each environmental layer (worldwide extent: totaling 2,287,025 cells with information per layer) can jeopardizethe specificity component of the TSS evaluation metrics, even when a large number of pseudo-absences per modeling rounds are used. Another important advantage of this criterion is the detection and exclusion of high divergent predictions. Metaphorically, when facing multiple points of view provided by various experts about on a subject, it is usually desirable to consider the opinion shared by the majority rather than the divergent opinion(s) provided by a minority. Therefore, only the models with considerable differences were filtered at this stage, i.e., those strongly diverging in the shape and size of suitable areas detected and differing from a large number of other models generated. We emphasize that the small-scale differences among models are of great importance in a multi-modeling approach; these small differences increase the quality of the result and were not filtered by this procedure.
To evaluate the level of concordance among models, we calculated a similarity index using Pearson´s pairwise correlation coefficients in R , as highly correlated model pairs are presumably more similar in terms of habitat suitability and unsuitability prediction. For the models remaining after the previous selection stage, we estimated Pearson´s correlation coefficients between model pairs. For each model, we averaged the coefficients over all paired coefficient values with the other models (except with the model itself; value = 1). These averages (hereafter referred as PCCs—Pearson´s correlation coefficient) were used to rank the similarity of each model in relation to the others. Thus, a model with a high average PCC is more similar in its suitability predictions with the majority of models in the set than models with lower PCCs and vice versa (the lower the PCC average, the less similar).
We predefined an ascending sequence of minimum average PCC thresholds from 0.5 to 1 in increments of 0.01, totaling 51 PCC thresholds. We used this threshold sequence to generate sets of models with average PCC scores equal to or higher than each respective value. For each set generated, we projected an Ensemble Model termed the Overall Predictions Model (OPM; description provided in the next topic). Subsequently, we assessed the quality of each generated OPM to correctly predict suitability where the native (NHR) and invasive Bt presence (IHR) records were located. Moreover, we verified the ratio between the number of suitable grid cells predicted and the total number of grid cells per OPM (suitable + unsuitable cells) and termed this ratio the Suitable Cells Ratio (SCR), which was calculated as follows:
We also captured and calculated the minimum, maximum and average TSS values from the set of models that composed each OPM using the TSS values obtained from the Biomod2 output.
Ensemble forecast models are mathematical methods that combine multiple simulations (forecasts) of a complex system into a unique and more robust result. We developed our ensemble forecast model based on the committee averaging method of Biomod2, in which the probabilities of habitat suitability from different models are not averaged but are transformed into binary results [33, 39, 80]. We used each respective model threshold that maximizes both sensitivity and specificity to define the spatial cut-off, before converting each model in binary predictions. This threshold parameter is considered to produce the most accurate results [39, 88, 95]. Another advantage of the committee averaging method is the ease of comparing outputs (binary = 1 = presence; binary = 0 = absence) relative to the raw algorithm outputs (continuous probabilities) that do not necessarily have the same meaning or same range of variation (for further details, see  and ). Using binary models from each selection criterion (Stages 1, 2 and 3) and the Biomod2 output models, i.e., models without selection (hereinafter referred as Stage 0 models), we developed ensemble forecasts in two steps as follows:
Step 1) Agreement Level Ensemble Model (ALM). This model is based on the sum of binary values of each set of models, resulting in maps with geospatial classes ranging from zero (all models agree with the unsuitability of the area) to the total number of geospatial coincidences of suitable habitats detected by all models (a spatialized frequency histogram). Therefore, the class with value = 1 indicates that only one model in the set indicated suitability in the area, the class with value = 2 indicates that two models agreed, and so on. Note that this range is not related to probabilistic or suitability level but to the level of agreement among models in each set. We used ALMs twice. First, they were used to build the next ensemble model type. Second, they were used to build ensembles from each evaluation stage per algorithm, providing a way to compare and evaluate algorithm performance.
Step 2) Overall Prediction Ensemble Model (OPM). This model constitutes a binary map that considers every suitable habitat area predicted by the overall models of each set. Essentially, all classes exhibiting values equal to or higher than 1 in the Agreement Level Ensemble Model (previous step) were reclassified as a unique binary value = 1. In contrast, the unsuitable areas retained the binary value = 0. Each OPM was evaluated with respect to its predictive quality, and the selected OPMs were used to build the main ensemble model result, as described below.
Evaluating and selecting ensemble forecast models
To generate a global map of susceptibility to Bombus terrestris invasion, we first selected the most accurate set of models that were, subsequently, combined into a single model using techniques of ensemble forecast. Selection was performed by visual inspection of plotted curves and analysis of changes in four evaluation indices (TSS, Invasive and Native Hit Rates, and Suitable Cells Ratio).We emphasized Invasive Hit Rate (IHR), which compares the obtained model to known invaded areas, a distinctive method of our work.
Among all Overall Prediction Models (OPMs) generated across the predefined Pearson´s Correlation Coefficients (PCC) in Stage 3, we prioritized the selection of those positioned just before marked shifts in the evaluation indices, paying particular attention to IHR changes. When we detected a sequence of changes in the evaluation indices across the PCC range, showing a clear trend, we selected the OPM immediately preceding this sequence.
We also paid attention to the relationship between IHR and the SCR, as well as the relationship of IHR with shifts in the NHR and TSS values (minimum, mean and maximum) across the PCC range. We avoided choosing the last OPMs of the full PCC range because each of the previous OPMs in the range contains the suitable prediction of the next one, and the last OPMs tend to exhibit higher error than previous ones (as detected in the analysis). We selected the minimum possible number of OPMs, considering the highest representativeness of each selection to encompass the main changes across the PCC range.
Once the PCC thresholds were selected, the main ensemble model was produced by the sum of the respective OPMs using the raster package  in the R platform . We considered the highest class value resulting from the maximum geospatial coincidence of shared suitability prediction as the "Susceptible at Maximum". To the class value = 1 (only one OPM indicated suitability), we attributed the name "Susceptible". To the intermediate class values, we attributed denominations related to their levels of susceptibility. To the class value = 0 (all OPMs agree with unsuitability of the area), we attributed the denomination of “Very Low Susceptibility or Insusceptible”.
At the end of our filtering process, the final dataset was composed of 4,209 native locations, 547 invaded locations (Fig 1) and 3,422 other Bombus spp. (i.e., Bombus spp. other than Bt) pseudo-absences (BOPA).
The initial set of 250 models generated by 10 algorithms (Stage 0) was reduced in Stage 1 (TSS ≥ 0.8) to 195 models generated by 9 algorithms. All models from SRE were excluded, and there was a high reduction of FDA models, of which 23 were discarded (Fig 2 and Fig 3). Some small losses also occurred with ANN (loss of 3 models) and MARS (loss of 4).
Stage 0: the total number of models per algorithm from the Biomod2 output before selection. Stage 1: Models that yielded TSS ≥ 0.8. Stage 2: Models that yielded an Invasive Hit Rate (IHR) ≥ the total IHR average. Stage 3: Models that yielded a total average paired Pearson´s correlation coefficient (average of paired PCC values between itself and all others) above each predefined threshold.
In Stage 2 (Invasive Hit Rate), an extreme reduction of models occurred (loss of 64 models from Stage 1) compared to the loss in the previous stage (loss of 55 from Stage 0); thus, 131 models remained from the initial number. With the exception of FDA, GAM, GLM and MARS, all algorithms lost models: ANN lost 17; CTA, 16; GBM, 4; MAXENT, 2; and RF, all 25 models.
In Stage 3 (convergence of suitability predictions), the Pearson´s Correlation Coefficients (PCC) range had an upper bound of 0.79, as no model pair yielded values higher than this. Thus, 30 Overall Predictions Models (OPMs) were selected.
Throughout the entire evaluation range, i.e., from Stage 0 to the last PCC threshold in Stage 3, the minimum TSS increased at four points (Fig 2 and Fig 4). However, two increases were marked. The first occurred in Stage 1, when models with TSS ≥ 0.8 were excluded from the Biomod2 output models. The second occurred at the 0.62 PCC threshold (Stage 3) and was related to the marked reduction in the area of predicted suitable habitat from PCC 0.61 to 0.62 (minimum TSS from 0.800 to 0.858; Fig 2 and Fig 4) and the small SCR sequential reduction in the previous PCC thresholds, beginning beyond PCC 0.59 (Fig 4).
Each evaluation index was recalculated at each step, but the TSS (minimum, maximum, mean) values were obtained at each step from the evaluation results of the Biomod2 output.
Across the evaluation range, the maximum TSS had one marked decrease event in Stage 2 (falling from ~0.931 in Stage 1 to ~0.904 in Stage 2; Fig 2) due to the exclusion of some high-scored models that correctly detected the native distribution of Bt (based on TSS values). However, the same excluded models only weakly detected suitability in locations of reported invasive Bt presence (based on Invasive Hit Rate values). For example, all 25 RF models yielded high TSS values and passed through Stage 1 (TSS ≥ 0.8) without model losses (Stage 0 minimum and maximum RF TSS values of 0.907 and 0.931, respectively; Fig 2). However, after Stage 2, no RF models remained (Fig 2 and Fig 3). Subsequent to this decrease, the maximum TSS remained constant over almost the entire evaluation range until the last PCC threshold (Stage 3—PCC 0.79), where a small decrease occurred. The stabilization of the maximum TSS over almost the entire PCC range is due to the permanency of a single high-TSS evaluated model since Stage 2 (MAXENT, PA 4, RUN 3; S1 Fig) that was only excluded from the set after PCC 0.78.
As expected, the mean TSS was influenced by variation in its extreme values (minimum and maximum TSS) but provided an indication of the variation in TSS central tendency of models in each set across the evaluation range. The mid-range TSS variation became evident from the absent relationship between the observed interval of mean TSS variation and the changes in extreme TSS values. For example, from Stage 2 to 0.61 (PCC, Stage3) there were small but progressive increases in the mean TSS (Fig 2 and Fig 4) that were unrelated to maximum and minimum TSS variation but instead related to the exclusion of some lower-evaluated models in the TSS mid-range. This emphasizes the overall improvement in model accuracy across the evaluation range within the most central range of TSS values.
Decreases in the Native Hit Rate (NHR) were relatively minimal across the entire evaluation range (Fig 4 and Fig 2), the total decrease reaching only approximately 2.5%. This indicates that, from the initial number of 4,209 Bt native presence records used, 4,208 intersect suitable areas predicted by at least one model in Stage 0 (S2 Fig). From this value (Stage 0), only 104 hits were lost before the last PCC threshold (PCC 0.79, Stage 3; S2 Fig). Small NHR changes were observed at many points across the evaluation range (Fig 2), but only four were relatively marked: one in Stage 2 and three in the final PCCs of Stage 3 (PCCs 0.77, 0.78 and 0.79)(Fig 4 and Fig 2).
The Suitable Cells Ratio (SCR) exhibited the highest variability among all evaluation indices except in the range from PCC 0.62 to 0.72, where we observed a pattern of SCR stability (Fig 2). The SCR scale (Fig 4 and Fig 2) does not provide a clear representation of the spatial scale contraction; for example, from Stage 1 to 2, a decrease of approximately 7.5 million km2 of suitable area was estimated (grid cell size at equator)(S2 Fig).
We observed positive and negative influences in the evaluation indices related to the reduction of OPMs suitable areas (SCR) across the PCC range (Stage 3)(S3 Fig). The SCR variation exhibited a strong linear relationship (Pearson´s r) with IHR (r:~ +0.965; p = 6.9E-18) as well as strong, but negative, relationships with the mean TSS (r:~ -0.929; p = 1.1E-13) and the minimum TSS (r:~-0.884; p = 9.6E-11). The SCR exhibited a weak relationship with NHR (r: ~ +0.621; p = 0.00024) and a non-significant relationship with maximum TSS (r: ~ +0.299; p = 0.1).
Invasive Hit Rate (IHR) variation and its relationships with other indices are depicted in Fig 5, where each evaluation index value was subtracted from the previous one across the evaluation range (Valuei+1—Valuei).
This plot facilitated the identification of three marked changes across the PCC range (PCC: 0.53, 0.62, 0.79) and the sequence of changes from PCC 0.75 to 0.79 used to select the OPMs (PCC: 0.52; 0.61; 0.74) that built the final ensemble model.
Some decreases in IHR occurred across the evaluation range, mainly related to SCR decreases, but three extreme decrease events were apparent at PCC 0.53, 0.62 and 0.79. The latter PCC (0.79) is preceded by a sequence of decreases starting at PCC 0.75. We detected three PCC thresholds that represent the main changes that occurred across the Stage 3 evaluation range (Figs 4 and 5; Fig 2 and S2 Fig). The first one occurred at PCC 0.53, with a pronounced decrease in IHR relative to the previous PCC threshold, indicating a reduction of 42 invasive presences hitting suitable habitats (S2 Fig). Prior to this point, only 15 invasive hits were lost (Stage 0 to PCC 0.52). This event was also associated with a strong decrease in the Suitable Cells Ratio (SCR), an approximately 18.4% reduction in the global suitable area predicted from the previous PCC (SCR from ~0.14 to ~0.11; Fig 2). Based on this pronounced shift event in the evaluation indices, we chose the OPM positioned immediately preceding it to compose the final ensemble model (PCC 0.52). The second threshold occurred at PCC 0.62 (Figs 4 and 5), with an even more pronounced IHR decrease, resulting in a loss of 44 invasive hits from the previous PCC, which was also followed by a greater suitable area (SCR) reduction than the last event, approximately 22.3% from the previous PCC threshold (PCC 0.61; Fig 2). Thus, we also chose the PCC 0.61 OPM to compose the final ensemble model. In the third case, we choose the PCC 0.74 OPM, which is positioned in the evaluation range just before the beginning of successive events of Invasive and Native Hit Rates (IHR and NHR) decreases, initiated at PCC 0.75 (Fig 2; Figs 4 and 5). For each respective selected threshold in Stage 3 (PCC 0.52, 0.61 and 0.74), the OPMs were composed by 127 models generated by 8 algorithms; 113 by 6 algorithms; and 99 by 5 algorithms (Models per Algorithms in Fig 2 and Fig 3).
From the ten algorithms used in the modeling procedure, only four contributed at least 20 models (≥80% of total models per algorithm) to all selected models (OPMs): GAM, GLM, MAXENT and GBM (Fig 2). Considering also the algorithms that contributed more than 20 models for at least one selected OPM, we added MARS to this list, with 21 models only at PCC 0.52 (Fig 2 and Fig 3). Thus, we considered these five algorithms to be more reliable for extrapolating habitat suitability predictions over more distant areas from the native distribution range of Bt, particularly the first four: GAM, GLM, MAXENT and GBM.
From OPMs to Global Susceptibility Map to Bombus terrestris Invasion
We used the three selected OPMs to ensemble our main model (D, E and F in Fig 6), which was projected as a Global Susceptibility Map to Bombus terrestris Invasion (global view in Fig 7 and framed in the spatial range of native and invasive presences in Fig 8). The Agreement Level Ensemble Models (ALMs), which were used to build the OPMs, were also projected (see S4 Fig).
The OPM considers all suitability predictions of every model from each selected set per Stage.
As each OPM contains the susceptible area predicted by the set of models of each subsequent OPM, the OPM from PCC 0.52 models predicted the largest susceptible area to Bt invasion on a worldwide scale (Fig 6 D). This covered approximately 28 million km2 (cell size estimated at the Equator; S2 Fig), and its susceptible area intersects 4,188 native (approximately 99.5%) and 520 invasive presence records (approximately 95%) of the initial total (Fig 2 and S2 Fig). However, the susceptible area identified exclusively by this OPM (class value = 1 after the OPMs sum; green areas in Fig 7) is more spatially restricted and marginally distributed, covering approximately 9.9 million km2 and intersecting only 6 native (approximately 0.14%) and 47 invasive presence records (approximately 8.6%) of the initial total. We classified this area as Susceptible for the Global Susceptibility Map to Bt Invasion (Fig 7).
The second largest susceptible area to Bt invasion was provided by the OPM PCC 0.62 (Fig 6 E), covering approximately 18 million km2 and intersecting 4,182 native (approximately 99.3%) and 473 invasive presence records (approximately 86.4%) of the initial total (Fig 2 and S2 Fig). Additionally, in this case, the exclusive susceptible area identified by this OPM plus the susceptible area from the previous one (class value = 2; blue areas in Fig 7) are even more spatially restricted, covering approximately 4.4 million km2 and intersecting 4 native (approximately 0.09%) and 47 invasive presence records (approximately 8.6%) of the initial total. This area was classified as Highly Susceptible (Fig 7).
The susceptible area shared by all OPMs together, represented by the OPM PCC 0.74 prediction (class value = 3 after the OPMs sum; Fig 6 F; black areas in Fig 7), covers approximately 13.9 million km2 and contains the majority of the Bt presence records. This area exclusively intersects 4,178 native (approximately 99.2%) and 426 invasive Bt records (approximately 77.9%) and was classified as Susceptible at Maximum (Fig 7).
The largest shared area among all selected OPMs, where all models agreed with no suitability and consequently no or very low susceptibility to Bt invasion was classified as Very Low Susceptibility or Insusceptible to Bt invasion (white areas in Fig 7).
Our methodological approach based in the three-stage selection criteria reduced the presence of statistical artifacts and incoherent predictions, retaining models with high predictive accuracy and high extrapolative capacity, i.e., the best models for delineating the global map of areas susceptible to Bt invasion. Moreover, it also showed that four algorithms (GAM, GLM, MAXENT and GBM) provided the best results, yielding models with high predictive convergence among them and high predictive and extrapolative capacity.
Overall performance of the methodological framework
The three-stage procedure resulted in an overall improvement of TSS values, with minor decreases events in the maximum and mean TSS, as well some small losses in the native and invasive hit rates. However, even the lowest hit rate exhibited by the most conservative level of susceptibility can be considered a good score. In fact, the area classified as the maximum level of susceptibility contains the largest number of invasive and native Bt presence records. Moreover, it can be considered the most reliable representation of global susceptibility to Bt invasion, due to the highest prediction convergence among models and algorithms.
Stage 1 mainly excluded the models of lowest quality in terms of suitability prediction near the native Bt distribution. However, TSS did not detect models that were unable to extrapolate predictions to more distant areas nor did it detect some models with statistical artifacts and incoherent predictions (see S1 Text). Thus, Stage 2 was able to detect and filter out the models with low capacity to extrapolate suitability. However, we observed a small decrease in the Invasive Hit Rate (IHR) when compared with the previous stage, this being an undesirable but interesting result. This difference arises from the exclusion of some individual models in the Stage 2 due to their below-average IHR that nonetheless correctly predicted some specific fragments of suitable area covering invasive presence records. However, the selected models, i.e., those with scores greater than the total average IHR, were unable to predict suitability in those same specific fragments. Despite predicting some specific suitable areas that others did not, these models were correctly excluded at this stage due to their low overall capacity to extrapolate suitable areas. The main advantage of Stage 3 was the ability to detect and exclude models with highly divergent predictions when considering the majority of model predictions. These divergent predictions were mostly related to statistical artifacts and to under- and over-fitting. Therefore, the progressive reduction in these undesired statistical effects was directly related to the increase in the Pearson´s Correlation Coefficient (PCC) thresholds. However, a sequence of decreasing accuracy (IHR and NHR) is initiated above PCC 0.75, related to the extreme suitable area contraction (SCR). This suggests that in our case, PCC 0.74 is the last acceptable (accurate) threshold in the range.
Overall, our framework aggregated different mathematical logics (from ten algorithms) and variable input data (PAs and Bt presence partitioning) that yielded convergent results in the final sets of selected models. We consider the susceptibility levels of invasion obtained here to be more robust and accurate than the categorization (into classes) of continuous probabilistic approaches, even though we did not conduct a formal comparison. Additionally, we verify the performance of each particular algorithm across the evaluation criteria; this information can be consulted in S1 Text.
Global susceptibility to Bombus terrestris invasion
Areas susceptible to Bt invasion are almost entirely limited to the north and south temperate climate zones, suggesting the possible restriction of Bt invasion to tropical environments. If we consider exclusively the range of the native Bt zone, it is apparent that almost all suitable area is restricted to the Europe continent and western Russia, and the Susceptible at Maximum level covers almost exactly the entire native Bt distribution, emphasizing that Bt is predominantly a temperate species.
Large suitable areas were predicted in the easternmost region of the area of native Bt records used in the modeling, mainly from the northwest (Murmansk) to the southwest (Dagestan) of the Russian Federation. There was also a big suitable area covering the region of Moscow city. Nevertheless, we found no presence records there, despite the reporting of Russia as a native environment for Bt .
In the southeastern region, suitable areas were predicted surrounding the continental seas of Azov, Caspian and Marmara; the Black Sea; and areas in Georgia, Azerbaijan, Syria and Lebanon. We found no reports of native Bt presence in these countries. However, these areas are close to Turkey, a country with many records and reports of native Bt presence [40, 96, 97].
In the southern area of native Bt records, specifically south of the Mediterranean Sea, most of the relatively small suitable areas were identified in northern Algeria, northwest Morocco and the Western Sahara, along with small areas in Tunisia, the Gaza Strip, Libya, Saudi Arabia, Jordan and Egypt. Among these countries, Morocco, Tunisia, Saudi Arabia and Jordan are reported as non-native areas for Bt, with evidence of Bt invasion . Libya and Egypt have been reported as countries without Bt .
Only a very low susceptibility was detected for Israel, a country that uses Bt colonies for greenhouse pollination and where Bt is reported as invasive [40, 42, 49, 64, 98, 99]. Considering that the reported invasion of Bt in Israel is in the north of the country, mainly in the region of forest fires at Mt. Carmel , the resolution of variables may have been insufficient for detecting the particular topoclimatical conditions of this upland zone.
Reported occurrences of invasive Bt in South America were predominantly distributed in southern Chile and southwestern Argentina, but we detected large susceptible areas in eastern and southern Argentina, in central and eastern Uruguay, and in southeastern Brazil. These susceptible areas are almost entirely connected to invaded regions. Additionally, a large coastal corridor classified as Susceptible at Maximum connects the invaded regions to Uruguay and Brazil.
The first report of invasive Bt presence in Argentina  suggested that this species reached the zone of San Carlos of Bariloche (Argentina), crossing the Andean Mountains via low-altitude pathways from Chile, where Bt was first introduced in South America. Recently, it was reported that Bt expanded rapidly (200 Km/year) and massively their range in the south of this continent; remarkably, this species spread it invasion from the western–near the Pacific coast of Chile—to the easternmost of the continent—reaching the Atlantic coast of Argentina . This scenario demonstrates the well adaptation to the regional environment and the high dispersal capacity of Bt, which increases the probability of invasive expansion over these susceptible corridors detected. Furthermore, it is likely that colonies (or inseminated queens) could be carried and released into these susceptible areas, accidentally or deliberately, by humans.
There is some evidence of the spontaneous spread of Bt in Uruguay after introduction ; however, we found no more information about sightings of Bt specimens or colonies in this country. Recently (2013), a large survey conducted during the spring and bordering the frontier between Brazil and Uruguay was made by the author (ALA) and collaborators, aiming to find invasive Bt in the wild. We found a large number of native bee species (including other Bombus spp.), but we did not find any Bt specimens. Until now, there is no strong evidence of Bt presence in either Uruguay or Brazil, but there is a strong possibility that Bt could use this susceptible pathway to reach both countries, increasing its invasive distribution from Chile and Argentina.
Bt threat was also detected by our models in the temperate zone of southwestern Oceania; notably, a large susceptible area was identified in Australia, where there is no reported invasion. Although Bt invasion was reported in the islands of Tasmania (AUS), with approximately 200 km of oceanic barrier separating them from Australia, and Bt invasion reported in both main islands of New Zealand, with approximately 1700 km of ocean between them and Australia, the global model showed a large, connected susceptible area in Australia. Assuming trade and transportation among these islands, the probability of Bt invasion into Australia could be considered high, as any inseminated queen hitchhiking via a commercial ship could start an invasion in this country.
This possibility has been a concern for some time inside academic and governmental circles [99–101]. This concern could be aggravated due to planned Bt importation by private agricultural sectors for greenhouse pollination [40, 102] despite opposing initiatives (e.g., ). Regardless of whether Bt has yet to occur in Australia (we found no scientific reports of such), we strongly recommend monitoring and/or surveying in the susceptible area, particularly in the area of Susceptible at Maximum Level.
The susceptible areas detected in New Zealand and Tasmania precisely covered all reported invaded locations [13, 40, 42, 49, 51, 58, 99, 103, 104]. Recall that we did not include these invasive presence records in the modeling procedure; thus, in both countries, the predictions of susceptible areas can be considered very accurate. The susceptible areas in these islands cover almost all land except for the mountain ranges of highest altitude.
In Japan, the susceptible areas coincided with Bt invasions already reported on Hokkaido Island [15, 105, 106], mainly in the north and northeast. However, the model failed to detect the reported invasive presence on Honshu Island.
Companies that commercially produce Bt colonies for pollination in South Korea have been reported [45, 49], and there is evidence of invasive spread . However, the model detected low susceptibility to Bt invasion in this area.
A large susceptible area without reports of invasive Bt presence was detected in eastern Canada and the United States. This area has some environmental similarities with the native range of Bt, but almost all of the susceptible area detected was through a single model (from ANN); thus, we suggest that the classification for this area as suitable be considered with caution. However, we cannot disregard this prediction because other models in the selected set also identified some susceptible areas in these countries.
A long strip of susceptibility in the southeast of China and a wide contour band of Susceptible at Maximum in South Africa were detected; both countries are reported as threatened by Bt invasion [40, 42]. Bt has already been introduced into South Africa, but there is no evidence of invasive spread . Nevertheless, based on the model, spread is likely to occur, eventually reaching southern Namibia. Companies developing industrial pollination have been reported as commercializing Bt colonies in China  and according to our models, it is likely that invasive Bt will spread through the susceptible area in the temperate zone, including Taiwan.
Many other susceptible areas were detected in countries and regions of temperate climate, but we found no information on commercial Bt colonies or sightings of Bt individuals for these areas. Examples include a narrow strip of high susceptibility in the west of the Himalaya Mountain Range covered by the Kashmir Region in India and northeastern Pakistan; a large susceptible area covering Iceland, Greenland and Svalbard (Norway); small areas in Iran and in the easternmost zone of Russia (Kamchatka Krai) and various small islands. In contrast, in tropical zones, the potential for Bt invasion can be considered very low. However, we cannot discard the possibility for some areas, especially if Bt colonies are introduced by humans. For example, in the highlands region of Lake Eduard (Democratic Republic of Congo), where the temperature is mild and the precipitation is high, the models detected a maximum level of susceptibility. This also applies to South American areas and the tropical highlands of the Andes.
The framework developed here presents new insights into multi-modeling methodological approaches of habitat suitability, mainly suggesting new criteria for pseudo-absences generation and model evaluation and selection to build a unique ensemble model with improved prediction accuracy. This approach can be easily implemented in existing robust platforms of Habitat Suitability Modeling.
The global map of susceptible areas can aid the design of more effective action plans for monitoring Bt invasion. It is important to consider public campaigns involving local people, which could contribute to a broader campaign for monitoring invasion over a large area. For example, Australia, Brazil and Uruguay could use the map to develop monitoring and mitigation actions that prioritize the border regions of areas already invaded.
The modeling component of the framework is not limited to the use of topoclimaticvariables; others bionomics factors can be added if necessary. However, the method is not fitted to analyze rapid evolutionary process that species can potentially exhibit in the new invaded environments, which can be relevant to estimate the invasive process at medium to long term.
Finally, the framework proposed here can readily be adapted to other invasive species for predicting and monitoring spread. These actions could contribute to protect biodiversity and, in the case of Bombus terrestris, helping to reduce and avoid further threats to native bees, safeguarding their indispensable services for ecosystems and human food security.
S1 Fig. Biomod2: TSS evaluation output.
TSS values per algorithm (250 models = 5 pseudo-absences dataset x 5 training and test partitioning of the native presence records x 10 different algorithms).
S2 Fig. Presences hits and suitable habitat area.
Stages (y-axis) and the number of invasive and native presence records hitting suitable areas per each respective Stage OPM generated, as well as the raw number of suitable cells per model, the total suitable area (km2) and the difference in suitable area predicted from the current model minus the previous one (delta values).
S3 Fig. Pearson correlation between paired evaluation indices.
S1 File. Data providers and bibliographical sources surveyed for Bombus terrestris native and invasive occurrences.
We are grateful to Luisa Carvalheiro, Jean Paul Metzger and Jesus Aguirre-Gutierrez for their suggestions on earlier versions of this manuscript. We thank the São Paulo State Research Foundation (FAPESP) for the scholarship of Andre L. Acosta (2011/12779-7) and the National Counsel of Technological and Scientific Development (CNPq) for granting Tereza C. Giannini (472702/2013-0). We also thank the Research Center on Biodiversity and Computing (NAP-Biocomp, Universidade de São Paulo) for providing computing infrastructure.
Conceived and designed the experiments: ALA TCG. Performed the experiments: ALA. Analyzed the data: ALA TCG VLIF AMS. Contributed reagents/materials/analysis tools: VLIF AMS. Wrote the paper: ALA TCG. Provided computational resources to perform the analysis: AMS.
- 1. CDB—Secretariat of the Convention on Biological Diversity Assessment and Management of Alien Species that Threaten Ecosystems, Habitats and Species. Montreal: CBD Technical Series N1; 2001. Available from: http://www.cbd.int/doc/publications/cbd-ts-01.pdf
- 2. CDB—Convention on Biological Diversity, INVASIVE ALIEN SPECIES: a threat to biodiversity. 2009. Available from: http://www.cbd.int/doc/bioday/2009/idb-2009-booklet-en.pdf
- 3. Williamson M. Invasions. Ecography. 1999; 22:5–12.
- 4. Mack RN, Smith MC. Invasive plants as catalysts for the spread of human parasites. NeoBiota. 2011; 9: 13–29.
- 5. Alpert P, Bone E, Holzapfel C. Invasiveness, invasibility and the role of environmental stress in the spread of non-native plants. Perspectives in Plant Ecology. 2000; 3:52–66.
- 6. Richardson DM, Pysek P, Rejmanek M, Barbour MG, Panetta FD, West CJ. Naturalization and invasion of alien plants: concepts and definitions. Diversity and Distributions. 2000; 6:93–107.
- 7. Kenis M, Auger-Rozenberg MA, Timms L, Péré C, Cock JW, Settele J, et al. Ecological effects of invasive alien insects. Biological Invasions. 2008; 11: 21–45.
- 8. Roques A, Rabitsch W, Rasplus J-Y. Alien terrestrial invertebrates of Europe In: Nentwig WHP, Pysek P, Vila ME (Eds). Handbook of Alien Species in Europe. Springer-Verlag, Berlin. 2009.
- 9. Levine JM, D'Antonio CM. Forecasting biological invasions with increasing international trade. Conservation Biology. 2003; 17:322–326.
- 10. Peterson AT, Papes M, Kluza DA. Predicting the potential invasive distributions of four alien plant species in North America. Weed Science. 2003; 51: 863–868.
- 11. Broennimann O, Treier UA, Müller-Schärer H, Thuiller W, Peterson AT, Guisan A. Evidence of climatic niche shift during biological invasion. Ecology Letters. 2007; 10(8), 701–9. pmid:17594425
- 12. Evangelista PH, Kumar S, Stohlgren TJ, Jarnevich CS, Crall AW, Norman JB, etal.Modelling invasion for a habitat generalist and a specialist plant species. Diversity and Distributions. 2008; 14(5): 808–817.
- 13. Ward DF. Modelling the potential geographic distribution of invasive ant species in New Zealan. Biol Inv. 2007; 9: 723–735.
- 14. Peterson AT, Williams R, Chen G. Modeled global invasive potential of Asian gypsy moths, Lymantriadispar. EntomolExp Appl. 2007; 125: 39–44.
- 15. Kadoya T, Ishii HS, Kikuchi R, Suda S, Washitani I. Using monitoring data gathered by volunteers to predict the potential distribution of the invasive alien bumblebee Bombus terrestris. Biological Conservation. 2009; 142: 1011–1017.
- 16. Medley KA. Niche shifts during the global invasion of the Asian tiger mosquito, AedesalbopictusSkuse (Culicidae), revealed by reciprocal distribution models. Global Ecology and Biogeography. 2010; 19(1): 122–133.
- 17. Loo SE, MacNally R, Lake PS. Forecasting New Zealand mud snail invasion range: Model comparisons using native and invaded ranges. EcolApplic. 2007; 17:181–189.
- 18. Ficetola GF, Thuiller W, Miaud C. Prediction and validation of the potential global distribution of a problematic alien invasive species—the American bullfrog. Diversity and Distributions. 2007; 13(4): 476–485.
- 19. Giovanelli JGR, Haddad CFB, Alexandrino J. Predicting the potential distribution of the alien invasive American bullfrog (Lithobatescatesbeianus) in Brazil. Biological Invasions. 2008; 10: 585–590.
- 20. Lavergne S, Mouquet N, Thuiller W, Ronce O. Biodiversity and Climate Change: Integrating Evolutionary and Ecological Responses of Species and Communities. Annual Review of Ecology, Evolution, and Systematics. 2010; 41(1):321–350.
- 21. Hutchinson GE. Concluding remarks Cold Spring Harbor Symposia on Quantitative Biology. 1957; 22: 415–427.
- 22. Kearney M, Porter WP. Mechanistic niche modelling: combining physiological and spatial data to predict species' ranges. Ecology Letters. 2009; 12: 334–350. pmid:19292794
- 23. Pulliam HR. On the relationship between niche and distribution. Ecol Lett. 2000; 3:349–361.
- 24. Soberón J, Peterson AT. Interpretation of models of fundamental ecological niches and species distributional areas. Biodiversity Informatics. 2005; 2: 1–10.
- 25. Milbau A, Stout JC.,Graae BJ, Nijs I. A hierarchical framework for integrating invasibility experiments incorporating different factors and spatial scales. Biological Invasions. 2009; 11: 941–950.
- 26. Jiménez-Valverde A, Peterson AT, Soberón J, Overton JM, Aragón P, Lobo JM. Use of niche models in invasive species risk assessments. Biological Invasions. 2011; 13(12): 2785–2797.
- 27. Perkins LB, Leger EA, Nowak RS. Invasion triangle: an organizational framework for species invasion. Ecology and Evolution. 2011; 1(4): 610–25. pmid:22393528
- 28. Shi J, Luo Y-Q, Zhou F, He P. The relationship between invasive alien species and main climatic zones. Biodiversity and Conservation. 2010; 19(9): 2485–2500.
- 29. Nenzén HK, Araújo MB. Choice of threshold alters projections of species range shifts under climate change. Ecological Modelling. 2011; 222(18): 3346–3354.
- 30. Elith J, Graham CH, Anderson RP, Dudik M, Ferrier S, Guisan A, et al. Novel methods improve prediction of species' distributions from occurrence data. Ecography. 2006; 29:129–151.
- 31. Wisz MS, Hijmans RJ, Li J, Peterson AT, Graham CH, Guisan A. Effects of sample size on the performance of species distribution models. Divers Distrib. 2008; 14: 763–773.
- 32. Aguirre-Gutiérrez J, Carvalheiro LG, Polce C, Van Loon EE, Raes N, Reemer M, et al. Fit-for-Purpose: Species Distribution Model Performance Depends on Evaluation Criteria—Dutch Hoverflies as a Case Study. PloS one. 2013; 8 (5), e63708. pmid:23691089
- 33. Araújo MB, New M. Ensemble forecasting of species distributions. Trends in Ecology & Evolution. 2007; 22(1), 42–7.
- 34. Giannini TC, Chapman DS, Saraiva AM, Alves dos Santos I, Biesmeijer JC. Improving species distribution models using biotic interactions: a case study of parasites, pollinators and plants. Ecography. 2013; 36:520: 649–656.
- 35. Bahn V, McGill BJ. Testing the predictive performance of distribution models. Oikos. 2013; 122(3), 321–331.
- 36. Thuiller W, Richardson DM, Pyšek P, Midgley GF, Hughes GO, Rouget M. Niche-based modelling as a tool for predicting the risk of alien plant invasions at a global scale. Global Change Biology. 2005; 11: 2234–2250.
- 37. Broennimann O, Guisan A. Predicting current and future biological invasions: both native and invaded ranges matter. Biology Letters. 2008; 4, 585–589. pmid:18664415
- 38. Le Maitre DC, Thuiller W, Schonegevel L. Developing an approach to defining the potential distributions of invasive plant species: a case study of Hakea species in South Africa. Global Ecology and Biogeography. 2008; 17:569–584.
- 39. Gallien L, Douzet R, Pratte S, Zimmermann NE, Thuiller W. Invasive species distribution models—how violating the equilibrium assumption can create new insights. Global Ecology and Biogeography. 2012; 21(11):1126–1136.
- 40. Dafni A, Kevan P, Gross CL, Goka K. Bombus terrestris, pollinator, invasive and pest: An assessment of problems associated with its widespread introductions for commercial purposes. Applied Entomology and Zoology. 2010; 45(1), 101–113.
- 41. Schmid-Hempel R, Eckhardt M, Goulson D, Heinzmann D, Lange C, Plischuk S, etal.The invasion of southern South America by imported bumblebees and associated parasites. Journal of Animal Ecology. 2014; 83(4): 823–837. pmid:24256429
- 42. Winter K, Adams L, Thorp R, Inouye D, Day L, Ascher J,et al.Importation of non-native Bumble Bees into North America: potential consequences of using Bombus terrestris and other non-native Bumble Bees for greenhouse crop pollination in Canada, Mexico, and the United States. White Paper of the North American Pollinator Protection Campaign (NAPPC). 2006; p1-31. Available from: http://www.pollinator.org/Resources/BEEIMPORTATION_AUG2006.pdf
- 43. Walther-Hellwig K, Frankl R. Foraging habitats and foraging distances of bumblebees, Bombus spp. (Hym.,Apidae), in an agricultural landscape. Journal of Applied Entomology. 2000; 124:299–306.
- 44. Morandin LA, Laverty TM, Kevan PG, Effect of bumble bee (Hymenoptera: Apidae) pollination intensity on the quality of greenhouse tomatoes. J Econ Entomol. 2001a; 94: 172–179.
- 45. Goulson D. Bumblebees: Behaviour and Ecology. v2, Ed: Oxford, Oxford University Press. 2010.
- 46. McGregor SE. Insect pollination of cultivated crop plants United States Department of Agriculture, Washington, DC. 1976. Available from: http://www.ars.usda.gov/SP2UserFiles/Place/20220500/OnlinePollinationHandbook.pdf
- 47. Morandin LA, Laverty TM, Kevan PG. Bumble bee (Hymenoptera: Apidae) Activity and Pollination Levels in Commercial Tomato Greenhouses. Journal of Economic Entomology. 2001c; 94(2): 462–467.
- 48. Velthuis HHW. The historical background of the domestication of the bumble-bee, Bombus terrestris, and its introduction in agriculture IN: Kevan P, Imperatriz-Fonseca VL (eds) Pollinating Bees: the conservation link between agriculture and nature Ministry of Environment, Brazil. 2002; p177–184.
- 49. Velthuis HHW, Van Doorn A. A century of advances in Bumblebee domestication and the economic and environmental aspects of its commercialization for pollination. Apidologie. 2006; 37(4): 421–451.
- 50. Ruz L, Herrera R. Preliminary Observations on Foraging Activities of Bombusdahlbomii and Bombus terrestris (Hym: Apidae) on Native and Non-native Vegetation in Chile. ActaHorticulturae. 2001; 561: 165–169.
- 51. Hingston AB, Marsden-Smedley J, Driscoll DA, Corbett S, Fenton J, Anderson R, et al.Extent of invasion of Tasmanian native vegetation by the exotic bumblebee Bombus terrestris (Apoidea: Apidae). Austral Ecology. 2002; 27: 162–172.
- 52. Ruz L. Bee Pollinators Introduced to Chile: a Review In: Kevan PG, Imperatriz- Fonseca VL (Eds). Pollinating Bees—the Conservation Link between Agriculture and Nature. Brasília, Ministry of Environment. 2002; p155–167.
- 53. Goulson D. Effects of introduced bees on native ecosystems. Annual Review of Ecology Evolution and Systematics. 2003; 34: 1–26.
- 54. Goulson D, Hanley ME. Distribution and Forage Use of Exotic Bumblebees in South Island, New Zealand. New Zealand Journal of Ecology. 2004; 28(2): 225–232.
- 55. Griffiths D. A critical study on the introduction onto mainland Australia of the bumblebee Bombus terrestris for the commercial pollination of protected tomato and other crops. Practical Hydroponics and Greenhouses. 2004; 77, 42–59.
- 56. Matsumura C, Yokoyama J, Washitani I. Invasion status and potential ecological impacts of an invasive alien bumblebee, Bombus terrestris L (Hymenoptera: Apidae) naturalized in Southern Hokkaido, Japan. Global Environmental Research. 2004; 8(1): 51–66.
- 57. Inari N, Nagamitsu T, Kenta T, Goka K, Hiura T. Spatial and temporal pattern of introduced Bombus terrestris abundance in Hokkaido, Japan, and its potential impact on native bumblebees. Population Ecology. 2005; 47(1): 77–82.
- 58. Howlett B, Donovan B. A Review of New Zealand's Deliberately Introduced Bee Fauna: Current Status and Potential Impacts. New Zealand Entomologist. 2010; 33(1): 92–101.
- 59. Sladen FWL. The Humble-bee; its life history and how to domesticate it. Macmillan & Co., London, UK. 1912; 283 pp.
- 60. Hopkins I. History of the bumble bee in New Zealand: its introduction and results. New Zealand Department of Agriculture, Industries and Commerce. 1914; 46: 1–29.
- 61. Semmens TD; Turner E; Buttermore R. Bombus terrestris (L.) (Hymenoptera: Paidae) now established in Tasmania. Journal of the Australian Entomological Society. 1993; 32: 346.
- 62. Torretta JP, Medan D, Arahamovich AH. First Record of the Invasive Bumblebee Bombus terrestris (L) (Hymenoptera, Apidae) in Argentina. Transactions of the American Entomological Society. 2006; 132(3–4): 285–289.
- 63. Inoue MN, Yokoyama J, Washitani I. Displacement of Japanese Native Bumblebees by the Recently Introduced Bombus terrestris (L) (Hymenoptera: Apidae). Journal of Insect Conservation. 2008; 12(2): 135–146.
- 64. Dafni A, Shmida A. The possible Ecological Implications of the Invasion of Bombus terrestris (L) (Apidae) at Mt Carmel, Israel. In: Matheson A, Buchmann SL, O'toole C, Westrich P, Williams IH (Eds). London: The Conservation of Bees. Academic Press. 1996.
- 65. Arbetman MP, Meeus I, Morales CL, Aizen MA, Smagghe G. Alien parasite hitchhikes to Patagonia on invasive bumblebee. Biological Invasions. 2013; 15: 489–494.
- 66. Furst MA, McMahon DP, Osborne JL, Paxton RJ, Brown MJF. Disease associations between honeybees and bumblebees as a threat to wild pollinators. Nature. 2014;506:364–366. pmid:24553241
- 67. Aizen MA, Morales CL, Vázquez DP, Garibaldi LA, Sáez A, Harder LD. When mutualism goes bad: density-dependent impacts of introduced bees on plant reproduction. New Phytologist. 2014; 204: 322–328.
- 68. Morales CL, Arbetman MP, Cameron SA, Aizen MA. Rapid ecological replacement of a native bumble bee by invasive species. Frontiers in Ecology and the Environment. 2013; 11: 529–534.
- 69. Hijmans RJ, Cameron SE, Parra JL, Jones PG, Jarvis A. Very high resolution interpolated climate surfaces for global land areas International. Journal of Climatology 2005; 25(15):1965–1978.
- 70. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna: Austria. 2011. Available from: http://www.r-project.org.
- 71. GADM. Global Administrative Areas of Year 2012. 2012; v.20. Available from: http://www.gadm.org
- 72. ESRI—ArcGIS Desktop 10. Environmental Systems Research Institute. Redlands, CA. 2010; R10.
- 73. Bontemps S, Defourny P, Bogaert EV, Arino O, Kalogirou V, Perez JR. GlobCover: Product description and validation report UCLouvain and ESA GlobCover project. 2009. Available from: http://due.esrin.esa.int/globcover
- 74. Pearson RG, Raxworthy CJ, Nakamura M, Peterson AT. Predicting species distributions from small numbers of occurrence records: a test case using cryptic geckos in Madagascar. J Biogeogr. 2007; 34: 102–117.
- 75. Parolo G, Rossi G, Ferrarini A. Toward improved species niche modelling: Arnica montana in the Alps as a case study. Journal of Applied Ecology. 2008; 45(5): 1410–1418.
- 76. Merckx B, Steyaert M, Vanreusel A, Vincx M, Vanaverbeke J. Null models reveal preferential sampling, spatial autocorrelation and overfitting in habitat suitability modelling. Ecological Modelling; 2011;222(3),p588–597.
- 77. Phillips SJ, Dudík M, Elith J, Graham CH, Lehmann A, Leathwick J, et al.Sample selection bias and presence-only distribution models: implications for background and pseudo-absence data. Ecological Applications. 2009; 19: 181–197. pmid:19323182
- 78. Mateo RG, Croat TB, Felicísimo ÁM, Muñoz J. Profile or group discriminative techniques? Generating reliable species distribution models using pseudo-absences and target-group absences from natural history collections. Divers Distrib. 2010; 16: 84–94.
- 79. Barbet-Massin M, Jiguet F, Albert CH, Thuiller W. Selecting pseudo-absences for species distribution models: how, where and how many? Methods in Ecology and Evolution. 2012; 3: 327–338.
- 80. Thuiller W, Lafourcade B, Engler R, Araújo MB. BIOMOD—A platform for ensemble forecasting of species distributions. Ecography. 2009; 32: 369–373.
- 81. Thuiller W, Georges D, Engler R. Ensemble platform for species distribution modeling—Package "biomod2". 2013. Available from: http://cran.r-project.org/web/packages/biomod2/biomod2.pdf
- 82. Goulson D, Stout JC. Homing ability of the bumblebee, Bombus terrestris. Apidologie. 2001;32:105–112.
- 83. Wolf S, Moritz RFA. Foraging distance in Bombus terrestris L. (Hymenoptera: Apidae). Apidologie. 2008;39:419–427.
- 84. Chefaoui RM, Lobo JM. Assessing the effects of pseudo-absences on predictive distribution model performance. Ecological Modelling. 2008; V210, 4: 478–486.
- 85. Allouche O, Tsoar A, Kadmon R. Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS). Journal of Applied Ecology. 2006; 43(6), 1223–1232.
- 86. Franklin J. Mapping species distributions—spatial inference and prediction. Cambridge: Cambridge University Press. 2009.
- 87. Phillips SJ, Dudík M. Modeling of species distributions with Maxent: new extensions and a comprehensive evaluation. Ecography. 2008; 31(2): 161–175.
- 88. Jiménez-Valverde A, Lobo JM. Threshold criteria for conversion of probability of species presence to either-or presence-absence. ActaOecologica. 2007; 31(3): 361–369.
- 89. Jones CC, Acker SA, Halpern CB. Combining local- and large-scale models to predict the distributions of invasive plant species. Ecological Applications: Ecological Society of America. 2010; 20(2): 311–26.
- 90. Domisch S, Kuemmerlen M, Jähnig SC, Haase P. Choice of study area and predictors affect habitat suitability projections, but not the performance of species distribution models of stream biota. Ecological Modelling. 2013; V.257: 1–10.
- 91. Engler R, Randin CF, Thuiller W, Dullinger S, Zimmermann NE, Araújo MB, et al.21st century climate change threatens mountain flora unequally across Europe. Global Change Biology. 2011;17(7):2330–2341.
- 92. Mouton AM, De Baets B, Goethals PLM. Ecological relevance of performance criteria for species distribution models. Ecological Modelling. 2010; 221 (16): 1995–2002.
- 93. Hijmans RJ, van Etten J. Raster: Geographic analysis and modeling with raster data R package. 2012; v20-12. Available from: http://CRANR-projectorg/package=raster
- 94. Goshtasby AA. Image registration. Advances in Computer Vision and Pattern Recognition, Advances in Pattern Recognition.London: Springer London. 2012:7–66.
- 95. Liu CR, Berry PM, Dawson TP, Pearson RG. Selecting thresholds of occurrence in the prediction of species distributions. Ecography. 2005; 28: 385–393.
- 96. Özbek H. Decline in Bombus terrestris (L) populations in Turkey. Melissa. 1993; 6:7–8.
- 97. Aslan MM. Seasonal activity of Bombus terrestris in east Mediterranean region, Turkey. J Environ Biol. 2008; 29(2):151–4. pmid:18831364
- 98. Potts SG, Dafni A, Ne'eman G. Pollination of a core flowering shrub species in Mediterranean phrygana: variation in pollinator diversity, abundance and effectiveness in response to fire. Oikos. 2001; 92: 71–80.
- 99. Hingston AB, McQuillan PB. Does the recently introduced bumblebee Bombus terrestris (Apidae) threaten Australian ecosystems? Australian Journal of Ecology. 1998; 23: 39–549.
- 100. NSW—New South Wales Government, Australia Introduction of the large earth bumblebee, Bombus terrestris—key threatening process listing. NSW Scientific Committee. 2004; Available from: http://www.environment.nsw.gov.au/determinations/BombusTerrestrisKtpDeclaration.htm
- 101. McClay A. Climex: Models to Predict the Potential Naturalized Range of the European Bumblebee Bombus terrestris (L) in Mainland Australia. Report. IN: AHGA—Australian Hydroponics and Greenhouse Association. Proposal to Import Bombus terrestris into Mainland Australia for Crop Pollination Purposes. 2005.
- 102. AHGA—Australian Hydroponics and Greenhouse Association. Executive Summary: Proposal to import Bombus terrestris into mainland Australia for crop pollination purposes. 2005; Available from: http://www.protectedcroppingaustralia.com/documents/Final.pdf
- 103. Stout J, Goulson D. Bumblebees in Tasmania: their distribution and potential impact on Australian flora and fauna. Bee World. 2000; 81: 80–86.
- 104. Schmid-Hempel P, Schmid-Hempel R, Brunner PC, Seeman OD, Allen GR. Invasion success of the Bumblebee, Bombus terrestris, despite a drastic genetic bottleneck. Heredity. 2007; 99: 414–422. pmid:17551520
- 105. Kondo N, Yamanaka D, Kanbe Y, Kunitake Y, Yoneda M, Tsuchida K, et al. Reproductive disturbance of japanese bumblebees by the introduced european bumblebee Bombus terrestris. Naturwissenschaften. 2009; 96(4): 467–475. pmid:19089400
- 106. Kadoya T, Washitani I. Predicting the rate of range expansion of an invasive alien bumblebee (Bombus terrestris) using a stochastic spatio-temporal model. Biological Conservation. 2010; 143(5): 1228–1235.