Predictive models

Five supervised machine learning models were implemented in this study to predict COVID-19 cases from the community in a sewershed using SARS-COV-2 viral loads in wastewater and other sewershed/community-related features. Firstly, three widely used non-time series machine learning models were developed, including multiple/univariate linear regression (MLR/LR), gradient boosting decision tree (GBDT), and feed-forward deep neural network (DNN) [23, 24]. As COVID-19 case data can contain temporal information, two more popular time-series models were also examined: Facebook Prophet and long short-term memory (LSTM). Prophet is an open-source general additive model for time series data [25]. LSTM is an advanced, recurrent neural network for long sequential data, which can also address the gradient explosion/vanishing problem [26]. It is not expected for the COVID-19 case series to be stationary, so time-series models like autoregressive integrated moving average (ARIMA) are not considered. LSTM and Prophet are known to be good at handling the seasonality and trends in the data.

Wastewater samples and sewersheds

Six hundred twenty wastewater samples were obtained twice a week from nine sewersheds in central Ohio in the United States between September 2020 and June 2021. The sewersheds represent a diversity of communities in urban and rural areas, serving a population ranging from 14,000 to 900,000. More details about the sewersheds were explained in our previous study [14]. All sewersheds have independent sewer systems and wastewater treatment facilities. Daily confirmed COVID-19 case counts of the sewersheds were retrieved from the Ohio Coronavirus Wastewater Monitoring Network based on reported symptom onset time [27]. The sewersheds covered by this study are distinct in the scale of COVID-19 case incidences and SARS-CoV-2 viral loads in wastewater.

Feature selection and engineering

Time and seasonality are important drivers of COVID-cases, which are the key components of various non-WBE based COVID-19 trend predicting models [22]. This study aimed to mine predicting information from the wastewater samples and the associated sewersheds to provide a more accurate prediction for the population in the community covered by the wastewater. Around 30 domain-specific features were extracted from wastewater samples and the community in the sewersheds. These features can be assigned to three categories: (1) viral loads (SARS-CoV-2 gene concentrations) and other biochemical parameters of wastewater (RNA, fecal indicators, and total suspended solid concentration, pH, temperature, etc.); (2) geographical parameters of the sewersheds (population, precipitation, etc.); and (3) and socioeconomic parameters of the communities (social vulnerability indexes: socioeconomic status, household composition & disability, minority status & language, and housing type& transportation as well as number of testing center) [28] (Fig 1). The socio-economic features were selected based on our previous study [29]. These parameters were determined to be significantly correlated with reported COVID-19 incidences and SARS-CoV-2 wastewater concentrations. To handle the skewness in the continuous features, normalization and Box-Cox transformation were conducted on all models except GBDT. Categorical features such as community name, sewer type (combined or separate), and weekend or weekday are one-hot coded.

It is expected that the SARS-CoV-2 gene concentration is one of the most powerful features, while other features may enhance the performance of the model in handling uncertainties in the wastewater data. Despite that non-time series models cannot learn from the short/long-term dependency of sequential data, wastewater surveillance features can be fed to the non-time series models for COVID-19 case prediction in the community represented by the sewershed. These features can also be uptaken by the time-series model like LSTM and Prophet to provide a more comprehensive wastewater monitoring of COVID-19 in the sewersheds.

Model building and hyperparameter tuning

Data preprocessing was conducted using scikit-learn and scipy.stats package in Python 3. Models were further built with the Tensorflow framework (version 2.7.0). All models were trained and tested using a train/validate split ratio of 70% and 30%. A univariant linear regression model was built first to serve as a baseline using the SARS-CoV-2 viral loads only. Multivariate non-time series models were then trained for next-time-step COVID-19 case prediction in different communities. Data from all communities in different sewersheds are trained jointly. 5-fold cross-validation and grid search were performed for parameter tuning and model evaluation.

To prevent data leakage, for time series models, the COVID-19 case in time step A is predicted from data from the past 16 time steps (sliding window of 8 weeks) in different communities (Fig 2A). Features are added as regressors to the Prophet model for training, and COVID-19 cases are predicted with a 95% credible interval. For multivariate LSTM, the architecture of the model is summarized in Fig 2B. For each paralleled time series from the 9 sewersheds, input multivariate feature matrixes were firstly taken by a stacked LSTM with two hidden layers (64 and 32 memory cells). Then the hidden states of the 9 LSTM layers were merged using a concatenate layer, followed by two dense layers of 32 and 9 units (output shape). The concatenating layer combined the features of each sewershed, which gave a higher level of feature abstraction and helped learn the dependency among the sewersheds to some extent. In this way, the WBE data from all 9 sewersheds were trained jointly, despite the fact that the LSTM layers were learned separately. A dropout rate of 0.05 and L1 & L2 regularizer value of 0.025 was adopted for the LSTM layers. Adam optimizer was employed to adapt the learning rate.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. LSTM model flow.

a) Input type and sliding window; b) LSTM model architecture.

https://doi.org/10.1371/journal.pone.0277154.g002

Feature importance

The performance of the models was evaluated with the root mean square error (RMSE) metrics of the test set (Fig 3A). Compared to the baseline model (univariant linear regression model using the SARS-CoV-2 viral load feature only), the addition of other wastewater- and sewersheds-related features dramatically improved the accuracy of the MLR model. Permutation feature importance was measured to find the key features. Using the best performing model, LSTM, it is not surprising that SARS-CoV-2 viral loads, time, and sewershed identity are highly important. This finding supported the achievement of our overall goal, which is to build effective machine learning models to predict COVID-19 trends in multiple communities using the longitudinal data. Furthermore, although the significant contributing features slightly varied among different models, they covered all three feature categories. Among the wastewater parameters, total suspended solids, fecal indicator concentration, RNA concentration, flow rate, pH, and temperature are the key features besides viral loads. The accumulative number of COVID-19 testing sites, population, and precipitation is the main model contributors in the geographical parameters category. Within the socioeconomic parameters of the communities, social vulnerability indexes [28], and poverty are also important features.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Comparison of model performance.

a) RMSEs of five machine/deep learning models; b) Person’s correlation coefficient of the predicted (LSTM and Prophet model) vs. observed COVID-19 case numbers (15 days rolling average) for Athens (‘bad case’); c) LSTM model performance on the data from Athens sewershed in Ohio (Overlaid area plot of the predicted vs. observed COVID-19 case numbers. Potential undertesting was observed); and d) Prophet model on all sewersheds. The shaded area indicated the 95% credible interval of the model parameters. True observations are shown in solid dots.

https://doi.org/10.1371/journal.pone.0277154.g003

Model building and performance evaluation

Overall, time-series models outperform the non-time series models, with LSTM being the best. Prophet and LSTM are powerful models in jointly making long-term predictions for all communities/sewersheds. Compared to Prophet (training set R² = 0.83, test set R² = 0.63), LSTM model (training set R² = 0.94, test set R² = 0.81) derived higher R²s. Moreover, according to Pearson’s correlation coefficient between the predicted value and observed value, LSTM yielded higher predictive accuracy than Prophet for the Athens’ data, which is the ‘bad case’ among all sewersheds (Fig 3B). Our previous study about the correlation between the viral concentrations in wastewater and the daily COVID-19 confirmed cases identified as the only non-correlating sewershed among all 9 sites investigated (Ai et.al, 2021). Athens is a small college town with students as the major residence type. The confirmed case trend of Athens is different from other communities. Peak infection occurs in mid-September and early October during the student returning season, followed by a decline after that in November and December when the students were leaving campus for holidays. In addition, the discrepancy between the wastewater data and new case data can be observed, which might be explained by an underestimation of the cases as most students stay asymptomatic.

In other words, compared to Prophet, LSTM showed better generalizing capability among different communities, probably owing to the fully connected layer after concatenating the LSTM layers. This also indicates that the LSTM model is preferred when the communities are diverse in wastewater biochemical attributes, and geographical, and socioeconomic parameters, which is a typical case in real-world COVID-19 WBE projects. LSTM is known for its ability to handle high-dimensional data, while the capability of Prophet in that regard remained understudied. Owing to the bad case and the variations among the sewersheds, the performance of both models in the test set is not very satisfying. This performance can also result from underestimating the case number (Fig 3C), a relatively small dataset, and overfitting the model [18]. However, Prophet has its advantages over LSTM. As a Bayesian structural additive model, Prophet can provide seasonal trends and the credible interval of the prediction (Fig 3D) [25]. We also examined the potential lead time of WBE to the clinical case. In the LSTM model, the RMSE on the test set was improved by 10% after inducing a 5-day-lag time to the wastewater data, implying that the clinical reports can be backlogged.

Collectively, we recommend time-series machine learning models to deal with multi-site WBE data, especially LSTM. With proper feature selection, feature engineering, and hyperparameter tuning, we believe machine learning can be a powerful tool for predicting COVID-19 trends from wastewater surveillance.

Limitations and future research perspectives

The feasibility of applying machine learning in the COVID-19 wastewater surveillance system is discussed in this article. However, further attention is needed in many other realms. First, WBE systems or networks on a larger scale are preferred due to the lack of standardization in wastewater sampling and processing methods. The resulting dataset can cover communities of different types. As mentioned above, one of the challenges for multi-community prediction is overfitting. For LSTM, we found L1 and L2 regularization and dropout to be effective methods. Removing redundant features and correct feature engineering can further reduce the risk of overfitting. As a pilot study, the dataset in this study is relatively small. Larger datasets can enhance the generalization of the models on various communities.

Due to the small dataset size, this study only focused on the next-time-point prediction with a sliding window of 16-time points. Future research is welcomed for developing WBE-based machine learning models to forecast the COVID-19 trends in a longer future time period. A longer or shorter sliding window is also worth examining. Other LSTM variants, such as bidirectional LSTM and encoder and decoder LSTM models, have been successfully applied to forecast the spread of COVID-19 in India [30]. Application of these models to WBE data might also be promising. It is also worth mentioning that in the temporal LSTM model of this study, the correlation between the different communities/sewersheds was only addressed by the dense layer after concatenation. This might be sufficient for distant communities with independent sewer systems and when the interaction between the COVID-19 dynamics in the communities is poor. Spatiotemporal models like convolutional neural network (CNN)-LSTM can be powerful in demonstrating the correlation between the communities when needed [31, 32].

Moreover, deployed models and wastewater testing approaches need to be updated constantly using upcoming WBE data. For example, the lead time of WBE results to clinical reports might vary among the communities and change over time. The reported COVID-19 case number can become less reliable due to the increased self-testing and vaccination rate [18]. When new variants emerge, the target genes in wastewater surveillance may need to be adjusted accordingly. Therefore, based on the model’s performance on the new data gathered, hyperparameter tuning needs to be conducted occasionally. A WBE-machine learning system can be built to help improve the efficiency of model deployment and updating. Weekly wastewater data can be uploaded by the researchers to an online platform. The data will be used to update the model via automatic parameter tuning tools like Keras-tuner under the supervision of a human expert, followed by automated data visualization [33]. This system can also be coupled with an alarming outbreak system. Decisions can be made based on whether the trend predicted indicates an outbreak or not, which can also be achieved by machine learning algorithms. Eventually, hotspots or areas of concern can be accurately identified.