Ethics statement

The Ministry of Public Health, Guyana granted access to non-identifiable surveillance data obtained from the Malaria Program /Vector Control Services under IRB exemption. Further, IRB exemption was obtained from Harvard School of Public Health (US), Human Research Protection Program, Protocol number IRB18-1638.

We had access to daily case register data from 2006 to 2019 documented at health facilities in each of Guyana’s ten regions and compiled at the central office in Georgetown (region 4), which consists of demographic information, results from tests concerning the identity of the infecting parasite species, and the dates when the patient’s smear was taken, examined, and inputted in the central office. Delays are defined as the days elapsing from when a patient is registered as a case at a health facility and when the patient is documented at the central VCS database. We excluded the 5.7% of observations with implausible recorded date ranges, i.e. individuals whose smear was examined in a health facility following their documentation in the central office in Georgetown. A majority of the 28% of cases indicated as rechecks were noted to have been infected by P. vivax, or mixed infections with P. vivax (64%), as expected, given that P. vivax is the species largely implicated in relapses [25]. While we acknowledge that the inclusion of patients with recurrent infections may hinder a mechanistic analysis of potential transmission patterns in each region, the primary goal of this analysis is to learn from prior clinical reporting trends to arrive at more informed estimates of current case loads, which include all patients, in the face of reporting delays.

We assessed the potential influence of rainfall on reporting delays by analyzing precipitation data sourced from East Anglia University’s Climate Research Unit’s gridded dataset repository. Monthly total precipitation data was extracted for each region by year [26]. Locations of second-level administrative regions, or neighborhood democratic councils (NDCs) (n = 116) were identified through a shapefile of NDCs from GADM version 1 (accessed through DIVA-GIS) [27]. We defined NDCs, rather than regions, as our spatial unit of analysis to ensure adequate statistical power, and aggregated median delays across years by locality for each NDC. Connectivity was captured through measures of accessibility (defined as travel times to the “nearest urban centre”) for each region in 2015 estimated in Weiss et al [28]. A shapefile of mapped mines was drawn from the 2005 U.S. Geological Survey Mineral resources data system [29]. Finally, a shapefile of Amerindian settlements was obtained from the LandMark Global Platform of Indigenous and Community Lands through the GuyNode Spatial Data portal [30].

Descriptive analysis

We first assessed the stationarity of each region’s monthly delay distributions from 2006 to 2019 through an Augmented Dickey-Fuller test. We further examined potential synchronicities between regions’ monthly delay distributions, using the mean cross-correlation coefficient between the regional time series for monthly delays from 2006 to 2019.

For each region, we additionally estimated (a) the Pearson correlation between median reporting delays and regional connectivity in 2015, and (b) the Pearson correlation and cross-correlations between monthly total precipitation levels and median delays from 2006–2019. We focused our analysis on case register data from region 4, where many malaria patients come for treatment but there is no local transmission, and from malaria-endemic regions 1, 7, 8, and 9 (see Fig 1A for map).

We used a global Moran’s I index [31], to evaluate if delays across NDCs from 2017–2019 (the most recent three years of data) across all ten regions were spatially autocorrelated. This measure estimates whether, and to what extent, observations in the same neighborhood—defined using a range of possible criteria—share “features” which would indicate spatial autocorrelation [31,32]. We further used the Getis-ord statistic, which compares the sum of characteristics in each neighborhood to their overall mean [32,33], to identify local clusters of NDCs with higher reporting delays. To pinpoint the potential correlates of the observed spatial patterns in delays, we computed the local bivariate Moran’s I statistic for NDC-level delays and density of Amerindian settlements and for NDC-level delays and density of mines. We chose not to compute the local bivariate Moran’s I for NDC-level delays and connectivity, which we expect to exhibit a particularly strong spatial relationship with delays, given that we were restricted to data on connectivity at the scale of regions. For all spatial analyses, we omitted the 2.9% of patients whose localities could not be geocoded. Spatial weights were defined using the queen’s contiguity criterion of order = 1. All significance maps for local autocorrelation tests that we provide report FDR-adjusted p-values.

Nowcasting methods

We used the following two approaches to estimate revised (i.e. eventually reported) monthly malaria case counts in regions 1,4,7,8, and 9. In all of the proposed approaches, our real-time malaria case count estimations were produced via a data assimilation process, in that only information that would have been available at the time of estimation was used to train our models. The first ‘out-of-sample’ case count estimate for all regions was produced for January of 2007 using historical information available at the time (training set time period) that consisted of data from the previous 12 months, i.e. within 2006. Subsequent estimates were produced by dynamically training the models below, such that as more information became available the training set consisted of a new set of (12-month long) observations. This approach was chosen as a better way to capture the potential time-evolving nature of reporting delays. For all models, we centered the covariates to ensure standardization of inputs.

Synchronous data imputation model (DIM)

For each region i, we used an elastic net penalized regression, i.e. an ordinary least squares regression (OLS) subject to a combined L1- and L2- penalty [34] assuming a gaussian error distribution to estimate the revised case count for a given month t. The model was fit to data on the number of cases occurring in months t to t-12 that were known by the end of month t and takes the following form (1), where denotes the number of cases occurring in each month t* known by the end of month t. Note that the number of known cases for a given month t increases with the month of separation, as more information accumulates over time. For example, as we would expect to have more information about cases occurring in the prior month than cases occurring in the current month, would exceed . (1) where the h_i′s denote the estimated coefficients for each input variable

Data inputs and method of training and testing the simple data imputation model

A matrix of case counts from each of the twelve months prior to and including time t (the most up to date case count) that were known by the end of time t was used to dynamically train and test the data imputation model in order to predict the number of revised cases for t. Our models were assessed in their ability to accurately predict the revised case counts, only available at time t+m, where m was typically more than a month later, and were compared to the case counts available at time t. We computed a 3-fold cross validation within each twelve month window in order to identify the tuning parameters that resulted in the lowest mean square error and used these parameters for the subsequent (unseen) monthly prediction. In other words, training of our models was conducted using strictly data that would have been available at the time of prediction. Given the size of the training set was 12 observations, we chose a small number of folds, n = 3 for our cross-validation implementation. We experimented with different number of folds (3, 5, or 10) for cross-validation to explore whether this choice would have an impact on the quality of our predictions and found that these additional choices led to qualitatively similar results, so we report the results of the most cost-efficient approach in the manuscript.

Synchronous network models (NM)

For each region i, an elastic net regression estimating the revised case count for a given month t was calibrated to data on the number of cases occurring in months t to t-12 that were known by the end of month t, the number of cases occurring in months t-1 to t-12 that were known by the end of the month t in all other regions j≠i, and the total precipitation level in region i at month t, a hypothesized cofactor of reporting activity. We also implemented an elastic net regression which additionally takes as inputs the predicted number of cases in month t for all other regions j from the previous data imputation models.

Both network models consider the location where malaria activity is estimated as a node in a network potentially influenced by malaria activity happening in other (potentially neighboring) locations (nodes).

The first network model (NM1) takes the following form (2), where θ(t) denotes the total precipitation level for region i at time t.

(2)

The second network model (NM2) takes the following form (3), where y_j(t) denotes the predicted case counts for region j at time t, estimated from the data imputation model for region j (1) (3)

Please refer to S1 Diagram for a schematic visualizing the distinct and shared inputs of the two network models.

Data inputs and method of training and testing the two network models

A matrix of case counts from each of the twelve months prior to and including t that were known by the end of t for region i, case counts from each of the twelve months prior to t that were known by the end of t for all other regions j, and total precipitation levels for region i in month t, was used to dynamically train and test NM1 in order to predict the number of revised cases for t for region i.

A matrix of case counts from each of the twelve months prior to and including t that were known by the end of t for region i, case counts from each of the twelve months prior to t that were known by the end of t for all other regions j, DIM-predicted case counts in t for all other regions j, and total precipitation levels for region i in t was used to dynamically train and test NM2 in order to predict the number of revised cases for t for region i.

We replicate the same procedure as in the data imputation model to test and train the two network models.

We generated ensemble model estimates, consistent with the first ARGONet procedure described in Poirier et al³⁵ by assigning the point estimate for a given month to that of the DIM, NM, and NM2, depending on which model yielded the lowest rRMSE for the prior three months [24,35]. We assigned the mean across models as the point estimate for the first three months of observations.

To provide decision-makers with a prospective uncertainty quantification (error bars) of our estimates, we constructed 95% confidence intervals by considering the error (RMSE) of model predictions for the prior 24 months, starting with January 2009 (two years after the first out-of-sample prediction). Subsequently, following an approach outlined in Poirier et al [35] citing work by Yang et al [36] to capture changing uncertainty at different points in time, we generated the lower and upper limits of the confidence intervals by subtracting or adding the RMSE associated to a moving window of the last 24 errors, respectively, to the upcoming point estimate [35,36]. This confidence interval construction is well-justified as we found that the RMSEs broadly reflected the standard deviation (STD) of the distribution of residuals (predictions—eventually reported or “true” case counts) for each time point (S7 Fig) [35,36], and thus, assuming that our residuals are Gaussian distributed, 95% of residuals should be captured within the mean (the point estimate) plus and minus the STD of residuals.

Lastly, we compared our model to a modeling approach detailed by Bastos et al [17]. We chose to compare to the Bastos et al. model given that it 1) presents a Bayesian “chain ladder” alternative to our models that also considers spatial trends and 2) is implemented for a vector-borne disease in a distinct yet neighboring country context in Latin America (Brazil). For simplicity, we used all priors and hyperparameters as defined in their open access code provided in their manuscript [17].

See S1 Diagram for a diagram outlining key steps in this data processing to model implementation process.

All data pre-processing, a-spatial and nowcasting analyses were conducted in R version 4.0.3 [37]. All spatial analyses were conducted in ArcMap version 10.6.1 and GeoDa [38,39].

1. Descriptive analyses

Between 2006 and 2019, malaria cases were entered in the central database a median of 32 days after they were registered at local clinics, with a marked right skew in the distribution of delays, for all regions combined (sd = 52.7). Fig 1B illustrates how these delays occurred in different regions (see S2 Fig for full empirical density functions and S1 Fig for annual confirmed cases and median delays for each region), and highlights the considerable heterogeneity in the extent of reporting delays observed between them. Due to the pervasiveness of mobile populations, particularly in mining regions, population size estimates and consequently, measures of incidence, are neither meaningful nor tractably quantifiable, so we chose not to report population standardized case counts. Region 1 ranked lowest in the proportion of annually aggregated cases from 2006–2019 reported by the end of the month (median = 0.18, 95% CI = 0.11,0.25) and region 4, where the capital is located, ranked highest, reporting nearly all of its cases by the end of the month (median = 0.93, 95% CI = 0.87,0.97). Neighboring regions 7, 8 and 9 additionally captured a low fraction of cases by the end of the month (median = 0.49, 95% CI = 0.30,0.60, median = 0.24, 95% CI = 0.12,0.51; median = 0.30, 95% CI = 0.22,0.41, respectively).

Monthly total precipitation was not correlated with monthly median delays between 2006 and 2019 for any of the regions (p-value = 0.90, p-value = 0.80, p-value = 0.41, p-value = 0.62, p-value = 0.84, for regions 1,4,7,8 and 9, respectively). While we expect that the differences between region 4 and the other regions in reporting delays are likely due to the remoteness of regions 1, 7, 8, and 9, the estimated association between region-specific connectivity and median delays in 2015 was modest and non-significant (r = -0.13, p-value = 0.71). Total precipitation and monthly delays for region 4 reported a significantly (at a level = 0.05) negative cross-correlation of -0.18 at a lag of -1 months, indicating that increased precipitation precedes reduced delays by one month in region 4. For all other regions, we found no significant cross-correlations within a meaningful range in lags.

We observed significant spatial autocorrelation in median reporting delays by NDC for the most recent years of data (Global Moran’s I = 0.496,p = 0.001). NDCs characterized by long delays were found in close proximity to NDCs with similarly high delays and vice versa (S3 and S4 Figs). Significant clusters of NDCs marked by higher delays were most concentrated in regions 1 and 7, while clusters of NDCs marked by lower delays were most concentrated in regions 4 and 6 (S3 and S4 Figs). Delays were significantly positively spatially correlated with density of mines by NDC, with the bivariate analysis highlighting areas with NDCs characterized by elevated delays surrounded by a greater density of mines (largely concentrated in regions 1 and 7) (Figs 2A, 2B, S5A and S5B) (bivariate Moran’s I = 0.590, p = 0.001). We also found evidence for a significant positive bivariate spatial association between reporting delays and density of Amerindian settlements, revealing areas with NDCs characterized by higher delays surrounded by a greater density of Amerindian settlements, mainly concentrated in regions 1, 8 and 9 (Figs 2C, 2D, S5C and S5D) (bivariate Moran’s I = 0.587, p = 0.001). Both clustering maps signal NDCs within region 1 marked by higher reporting delays and a relatively increased presence of Amerindian settlements and mines. Note that NDCs showing the inverse of this relationship, i.e. lower delays surrounded by a higher density of Amerindian settlements or mines, were located in regions 6 and 10, where malaria is not endemic (S5A–S5D Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2.

A. Local bivariate Moran’s I clustering map of aggregated delays and density of mines, by NDC; B. Map of mining sites; C. Local bivariate Moran’s I clustering map of aggregated delays and density of Amerindian settlements, by NDC; D. Map of Amerindian areas. For visual purposes, we show only high-high clusters of areas reporting at higher delays with a greater density of mines or Amerindian settlements. Full bivariate cluster and significance maps can be found in S5 Fig. Base map sourced from DIVA-GIS.

https://doi.org/10.1371/journal.pcbi.1009570.g002

We found weak evidence for overall synchronicity in delay distributions across regions, with an estimated mean cross-correlation between the regional monthly delay distributions equal to 0.24 (95% CI = 0.20,0.27) and to 0.25 (95% CI = 0.18,0.41) when excluding region 9, due to low case numbers. Overall, these minimally correlated delay distributions may reflect varying seasonality in transmission between regions. Finally, there was significant evidence for stationary of the time series of monthly delay distributions for all regions (from 2006–2019) (p-value = 0.01 for all six regions). These findings support the use of data imputation models, which rely on previous trends in delays in a given region to inform ongoing predictions.

2. Nowcasting results

A. Data Imputation Models (DIM).

The model based on region 4, where the capital Georgetown is located, yielded the best estimate of monthly cases from 2007 to 2019 (rRMSE = 0.0781). DIM model predictions coincided with the two highest peaks in true cases and generally reflected trends throughout time, despite substantially underestimating a moderate peak in late 2012 and generally overshooting revised cases in late 2013 (Fig 3B). Given that the number of cases known by the end of each month in region 4 closely approximated the number of revised cases, this approach may be of limited practical utility. Due to the high accuracy of region 4 DIM, we chose not to implement network models for this region.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Results from Data Imputation Models.

(A) Regions 1 (B) Region 4 (C) Region 7 and (D) Region 8. Solid red lines indicate the number of cases estimated from the data imputation model, dashed red lines indicate the number of cases known by the end of the month and solid blue lines indicate the true number of eventually reported cases. Grey bands capture the moving 95% confidence intervals for each model.

https://doi.org/10.1371/journal.pcbi.1009570.g003

In contrast, the DIM models for regions 1, 7 and 8 significantly improved on revised case counts for each of these regions. Of these, the DIM model for region 8 reported the optimal performance (rRMSE = 0.151), exhibiting a slight lag in estimating revised cases, sharply undershooting cases in late 2011 and early 2013, while eclipsing cases in late 2013, but closely recapitulating trends from 2014 onwards (Fig 3D). The DIM models for regions 1 and 7 reported weaker model performance (rRMSE = 0.270 and 0.174, respectively), partially driven by the marked underestimation of revised cases in early 2011, late 2013, and late 2019, in the case of region 7, and the consistent underestimation of revised cases over the course of the study period, in the case of region 1 (Fig 3A and 3C). However, region 1 predictions broadly approximated converged case trends, although still of an attenuated magnitude, from 2017 onwards, and model predictions for region 7 still generally reflected the most recent upward trend in cases from 2016, with the exception of late 2019, improving on month-end reporting substantially (Fig 3A and 3C). Finally, the DIM model for region 9 failed to converge (identify a stable set of parameter values) due to data sparsity, largely driven by the exclusion of cases with implausible dates of documentation, as defined previously. Of note, the data imputation model estimates for all regions, particularly for region 1 and 7, were marked by wide confidence intervals from 2014–2016, a consequence of the poor model performance in the years just preceding. However, it is important to highlight that we observe consistently sharper confidence intervals, and thus an increased level of certainty in our estimates, for cases occurring in the more recent past, i.e. from 2016 onwards. Finally, for all regions, a slight majority of monthly converged case counts fall within our estimated confidence intervals, testifying the effectiveness of our instituted method for prospectively estimating confidence intervals.

B. Network Models (NMs).

Both network models exhibited improvements for all regions (Fig 4A–4D). The NMs for region 1 resulted in predictions that more closely tracked revised case counts over time (Fig 4A and 4B), showing reduced error rates for both models (rRMSE = 0.2475 [NM1] and 0.2490 [NM2]), compared to the data imputation models. However, both models demonstrated continued underestimation of case counts from 2011 to 2014. NM1 predictions for region 1 reported only marginally greater accuracy than the corresponding predictions from NM2. The NMs for region 7 revealed perceptible gains in accuracy, more closely reflecting true dynamics in case counts and importantly, for both models, attenuating the appreciable drop in estimated cases in late 2012 and correcting estimates to better mimic case trajectories from 2016 (Fig 4C and 4D), with NM1 resulting in the greatest improvement (rRMSE = 0.1706 [NM1] and rRMSE = 0.1718 [NM2]). Finally, NM1 and NM2 improved over the DIM model for region 8 (rRMSE = 0.1387 and 0.1359, respectively). Model predictions from late 2013 onwards better mirrored the magnitude and timing of current trends (Fig 4E and 4F). Unlike regions 1 and 7, model predictions were most improved for NM2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Results from Network models.

(A) Region 1 NM1 (B) Region 1 NM2 (C) Region 7 NM1 (D) Region 7 NM2 (E) Region 8 NM1 (F) Region 8 NM2.

https://doi.org/10.1371/journal.pcbi.1009570.g004

The data imputation and network models significantly improved over low known case counts for regions 1, 7, and 8, most strikingly for regions 1 and 8, with error rates reduced by a factor of 1.8, 1.6, and 2.1, respectively, for the regions’ best performing models (S1 Table). In all cases, when relying on predictions from the best performing models, the median fraction of cases captured within a month for all regions from 2007 to 2019 now ranges from 0.63 to 0.95.