Study population and outcomes

This study focused on Schistosoma mansoni (S. mansoni) infection hotspots in villages within the Lake Victoria basin, including 147 villages in Kenya and 148 villages in Tanzania (Fig 2). Schistosoma infection data in these villages were obtained based on a randomized controlled trial from 2011 to 2015 through the SCORE project [21]. In the SCORE project, the 295 study villages were randomly assigned to six arms, with each arm annually receiving one of three types of treatments (school-based treatment, community-wide treatment, or no treatment) through mass drug administration (MDA) with praziquantel from 2011 to 2015 (S1 Fig) [22]. Disease-related data were collected through annual epidemiological surveys, either shortly before or during MDA programs [14]. We considered two main outputs of the surveys: prevalence and intensity of infection among children aged 9 to 12 years (S1 Text). In addition, both outputs were used to define persistent hotspots (PHS) using the following two methods:

• PHS definition I: Included villages with a relative reduction in prevalence of less than 35% or a relative reduction in intensity of less than 50% from baseline to Year 5 [14,15,23];
• PHS definition II: Included villages with less than a 35% relative reduction in prevalence from baseline to Year 5 and a greater overall prevalence than 10% in Year 5 [9,17].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Map of hotspot areas around Lake Victoria, highlighted in red points, in Tanzania (top) and Kenya (bottom) using persistent hotspot definitions I–II.

The map layers were created using publicly available world map data from Natural Earth, accessed via the R package rnaturalearth [24].

https://doi.org/10.1371/journal.pntd.0013315.g002

Such PHSs, according to each definition, were presented in Fig 2, and the proportion of PHSs was provided in S1 Table. These PHSs were then used as study outcomes. Our aim was to develop methods for the early identification of PHSs in the first year (referred to as baseline methods). The advantage of the baseline method is that it does not require Schistosoma infection data after the baseline.

Exploring patterns of schistosomiasis, spatially weighted data fusion, and categorizing datasets

In this study, we employed empirical variograms, a standard geostatistical tool used to understand spatial structure and detect spatial patterns in infection prevalence and intensity, illustrating how infection data vary and correlate across space [25]. To account for spatial correlation at short distances while filtering out redundant noise from longer distances (Sect 3.1), a spatially truncated inverse distance weighting (spTIDW) approach was applied to each village to produce spatially weighted prevalence and intensity predictors [26,27]. For simplicity, we provided a category label with infection data around villages for the resulting predictors, which were subsequently used to explain whether the village was a hotspot or not.

Moreover, incorporating a broader range of schistosomiasis-related variables from different domains usually helps enhance the performance of prediction models. However, large number of predictors may increase the risk of overfitting, resulting in poor out-of-sample performance. Categorizing the variables by domain can help manage model complexity, provide deeper insights into hotspot formation, and improve the interpretability of predictive models. We used a knowledge-driven method to categorize these secondary predictors into five distinct categories: environment, agriculture, biology, society, and geography (S2 Text). For each category, we collected secondary datasets from publicly available sources based on prior studies [9,17,18,28,29]. Most of datasets are defined on spatial regular grids (S2 Table), and we applied spTIDW to summarize them from the grid level to the villages to fuse the baseline infection data. The resulting spatially weighted predictors were used to account for the hotspot status of the villages, similarly to those from infection data around villages. Some of these predictors have been shown to be closely associated with the status of schistosomiasis hotspots, although the strength and direction of their relationships may vary across the study villages [28].

Developing prediction models using different combinations of predictors

We investigated eight predictor configurations to develop prediction models: (C1) only baseline infection data, (C2) baseline data and infection data around villages, (C3) baseline data and environment predictors, (C4) baseline data and agriculture predictors, (C5) baseline data and biology predictors, (C6) baseline data and society predictors, (C7) baseline data and geography predictors, and (C8) all predictors. In light of this, relative improvements (RIs) of each predictor configuration from (C2) to (C8) can be assessed in terms of prediction accuracy by comparing them with the method developed using (C1) (see Sect 2.5 for the definition of RI). Therefore, the resulting RIs allow for quantifying each category’s importance in hotspot predictions.

From a modeling perspective, hotspot prediction is a classification problem with two classes. We tested fourteen classification models for each of the eight configurations, eight of which had previously been used to predict hotspots [14,17]. These included GBM (gradient boosting machine), RF (random forest), Tree (a single decision tree), Logit (elastic-net logistic regression), LASSO (logistic regression with the least absolute shrinkage and selection operator), LGT (traditional logistic regression), SVM (support vector machine), and an ensemble of these models implemented using the R package h2o [30]. To better account for the nonlinear nature of hotspot formation and the complex correlations of the disease, six additional models were investigated through this study. These models included LogitGPs (logistic regression with Gaussian processes [31]), sparseSVM (regularized SVM [32]), DyTrees (dynamic tree models implemented by particle learning [33]), regLogit (regularized logistic regression based on Gibbs sampling schemes [34]), Probit (probit regression [35]), and DNN (deep neural network model [36]).

Improving prediction models through random oversampling

In addition to considering different predictor configurations and developing more flexible prediction models, we also investigated the impact of the imbalance proportion between hotspots and non-hotspots on prediction accuracy. From an application perspective, an imbalanced proportion can heavily affect the development of prediction models, as such patterns often cause the models to learn features primarily from one side of either hotspots or non-hotspots, depending primarily on which has higher proportion, i.e., the imbalance learning problem [37]. This often results in low accuracy in hotspot predictions. To mitigate the influence of imbalanced hotspots on the accuracy of prediction models, a synthetic sampling-based method was applied for each of the eight predictor configurations mentioned earlier. This involved using current training data to generate new synthetic training data through random oversampling, where represent the predictors of the village i, the outcome , and here (non-hotspot) and (hotspot). This procedure includes three steps for each k:

1. (a) Randomly select with the probability for j = 0 or 1;
2. (b) Randomly select , such that , with the probability , where n_j is the number of within T_n; and
3. (c) Sample predictors from , with being a probability distribution, depending on the smoothing matrix ; see [37] for more details on setting and .

We implemented this method using the R package ROSE [38]. Following common practice, we set and chose a synthetic sample size of m = 300, which was comparable to the village size of n = 295. To further illustrate the benefits of addressing unbalanced problems for improving hotspot prediction, synthetic minority over-sampling (SMOT) was also implemented using the R package performanceEstimation [39]. To reduce the risk of overfitting, these data synthesis methods were applied only to the training set, after data splitting in cross-validation (CV; Sect 2.5) [40].

Cross-validation methods and evaluation criteria

Three scenarios were considered in model development and validation: within-country, combined-countries, and between-countries. CV was performed to validate the performance of the prediction models in each scenario. The within-country scenario included villages only from a single country in each CV setting. This scenario involved two CV settings. The first CV setting consisted of villages only from Kenya, while the second contained villages only from Tanzania. The combined-countries scenario involved a single CV setting, where the villages of both countries were completely incorporated. The scenario between countries involved two CV settings, labeled as Between I and Between II. In Between I, the training set consisted of villages only from Kenya, while the test set included villages only from Tanzania. In contrast, in Between II, the training set contained villages only from Tanzania, while the test set included villages only from Kenya.

In each CV setting of the within-country and combined-countries scenarios, 70% of the villages were randomly selected as the training set for model development, while the remaining 30% were reserved as the test set for model evaluation. In each CV setting of the between-countries scenario, 70% of the villages of the training set were randomly chosen to develop the models, while all the villages of the test set were used to evaluate the performance of the models in predicting the hotspots.

This work used the accuracy of the predictions on the test set as the evaluation metric. Accuracy in each simulation was measured as the proportion of correctly predicted hotspots and non-hotspots out of the total number of both. We conducted 200 simulations for each CV setting and calculated the average accuracy of the model across simulations. To evaluate the improvements of the proposed data fusion method with respect to other approaches, we further let , where Acc I and Acc II refer to the average accuracy of the reference approach and the proposed method for a specific model, respectively.

Developments of spatially weighted predictors

Empirical variograms showed that the spatial correlation ranges for prevalence in Kenya and Tanzania were approximately 12 km and 50 km, respectively, while for intensity, the spatial ranges were 20 km and 40 km in Kenya and Tanzania (S3 Fig). In addition, the disease exhibited spatially heterogeneous characteristics, with significant differences in disease prevalence and/or intensity between two villages located very close to each other, resulting in one being a hotspot and the other not (Fig 2). In this case, using commonly employed methods, such as nearest neighbor, to downscale secondary spatial grid data can result in the same value for the resulting predictor in two villages that are very close to each other. This is because these two villages may share the same closest spatial grid to derive predictor values. In contrast, spTIDW utilized multiple grids to construct spatially weighted predictors, typically resulting in different predictor values. This can be beneficial in identifying differences caused by small-scale variations. This work used a 20 km correlation range as the threshold in spTIDW to select neighboring villages to construct spatially weighted prevalence and intensity predictors in infection data around villages. For spatial grid data, a 50 km threshold was applied to select grids to generate all other spatially weighted predictors from the remaining five categories: environment, agriculture, geography, biology, and society (see S2 Text and S2 Table for more details of these predictors).

Spatially weighted data fusion methods for predicting persistent hotspots with definition I

For each model, we calculated RIs of the data fusion method for each scenario and predictor configuration, based on the prediction accuracy detailed in S3 Table. These RIs were then averaged across three CV scenarios for each predictor configuration, resulting in the average RIs (ARIs) presented in Table 1. Compared to models that used baseline infection data only, the fourteen models that employed the data fusion method in most cases achieved positive ARIs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Average relative improvements (ARIs, %) for each model in prediction accuracy on test sets from the proposed spatially weighted method using seven different predictor configurations C2–C8, compared to the method using the baseline infection data only (C1).

https://doi.org/10.1371/journal.pntd.0013315.t001

For each scenario and predictor configuration (S3 Table), we further calculated the RIs from the best of the 14 models, in terms of the highest prediction accuracy for each of both methods (i.e., the data fusion method and the method using infection data only) (Table 2). In general, the data fusion method tended to produce larger RIs in the combined-countries and between-countries scenarios compared to the within-country scenario. Based on the average RI across scenarios, combining baseline infection data with predictors from other categories usually resulted in positive improvements, with the exception of agricultural predictors (−0.92%). In general, the predictors of biology (8.64%), geography (7.04%), or society (4.89%) provided better improvements compared to the other two categories (infection data around villages (2.13%) and environment (0.47%)).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The relative improvements (RIs, %) for each scenario obtained using the spatially weighted data fusion method with different predictor configurations (C2–C8), compared to the method with configuration C1.

https://doi.org/10.1371/journal.pntd.0013315.t002

Furthermore, compared to the approach that uses baseline infection data only, the proposed data fusion method typically improved the lower bound of prediction accuracy in most cases based on the lowest accuracy of the 14 prediction models (S4 Table).

Imbalanced proportions of hotspots and improvement of predictions

This work highlighted the imbalance in the hotspot numbers or proportions (S1 Table). Specifically, the number of hotspots was much higher than that of non-hotspots in Tanzania (e.g., 72% versus 28% from PHS definition I), while in Kenya, the number of hotspots was much smaller than that of non-hotspots (e.g., 35% versus 65% from PHS definition I).

Like other areas [41], the performance of the prediction models was heavily affected by the imbalanced distribution of the hotspots. This impact was especially pronounced in the between-countries scenarios, as the proportion of hotspots was completely reversed between Tanzania and Kenya, as analyzed above. As an illustration, we compared the proposed method with the existing method using prediction accuracy presented in S6 Table, where the proposed method was developed using pre-processed training sets, while the reference method was developed using the original unbalanced training sets. For each CV scenario and predictor configuration, we used the highest accuracy to make comparisons by selecting the best model among the fourteen for each method. Based on the highest accuracy of each method, we calculated the RIs of the proposed method. These RIs, along with the highest accuracy, were averaged in the two scenarios between countries, resulting in the ARIs and the average of the highest accuracy for each predictor configuration (Fig 3). Among the seven different predictor configurations, including agricultural predictors contributed to the highest ARI of 37.9%. This was followed by environmental predictors (36%), infection data around villages (21.5%), all predictors (18.8%), societal predictors (15.4%), geographical predictors (10.5%), and biological predictors (6.5%). Detailed RIs achieved by the proposed method are provided in S7 Table, where RIs for a few models are negative, and RIs for most models are positive.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Under PHS definition I, the RIs, in prediction accuracy on unprocessed test sets, obtained from the models trained using pre-processed synthetic data based on the proposed different predictor configurations, compared to models using unprocessed original imbalanced training data, where the best model with the highest prediction accuracy was considered for each method (i.e., y-axis).

https://doi.org/10.1371/journal.pntd.0013315.g003

Importance of each predictor category and each predictor

In general, combining baseline infection data with other predictors improved overall prediction performance of the method that uses baseline infection data only (S4 Fig). Interestingly, including all predictors did not produce the best RI. More specifically, higher average RIs across scenarios were obtained by combining baseline infection data with biology predictors (10%) or geography predictors (8.6%), compared to using all predictors (7.2%).

Furthermore, the importance of predictors was assessed by evaluating their relative effects, quantified as the percentage of variance they explained. Based on CV, we summarized the percentage results along with their standard deviations in S5 Fig. Soil moisture, precipitation, and cropland predictors emerged as the top three most important among all predictors examined in this study. Several environmental predictors (e.g., dew point temperature and irradiation-related variables), socio-geographic and agricultural variables (e.g., buildings, child dependency ratio, population density, permanent water cover fraction), and infection data, also made important contributions to hotspot prediction.

Scalability of the proposed method for other definitions of persistent hotspots

PHS definition II was used to illustrate the scalability of the proposed method. This definition was considered in recent work [17], in which hotspot predictions in Kenya and Tanzania were also investigated by combining baseline infection data with other datasets. Based on the CV results in S8 Table, this section compared our method with two others based on RIs: (1) the approach that uses only the baseline infection data, and (2) the current results from [17].

The spatially weighted data fusion method achieved improvements in all CV scenarios when compared with the approach using baseline infection data only. For example, when considering the best models for each scenario (within Tanzania, combined-countries, and between-countries) the proposed method achieved a RI of at least 12.2%. Compared to the results provided in S5 Table of [17], the proposed method improved the current results by 2.8% in Kenya and 3.3% in Tanzania. In particular, the previous method for the between-countries scenarios typically produced low accuracy, with the highest accuracy being 31% and 46% in the between I and between II scenarios, respectively. In contrast, our method for these two scenarios achieved the highest accuracies of 63.1% and 70.2%, respectively, with RIs of 103.5% and 52.6%. However, these comparisons were made in a rough manner, as reproducing their results with the exact same settings is unlikely without their secondary datasets.

Highlighted models

Another contribution of ours to improving predictions stemmed from the investigation of broader models. In general, the best-performing model in each CV scenario was one of the six newly designed models rather than those from previous work, based on prediction accuracy on the test sets (S6 Fig). Specifically, in these three scenarios (within-country, combined countries, and between-countries), the highest median accuracies were achieved by DyTrees (73.65%), regLogit (72.2%), and LogitGPs (68.1%), respectively. Overall, LogitGPs and DyTrees, which ranked in the top two positions, achieved average accuracies of 70.3% and 70.2%, respectively, while the best among the prior models was the Ensemble model, with an average accuracy of 69.6%. The worst-performing model was a single Tree model, with an average prediction accuracy of 66.1%.