https://orcid.org/0000-0003-2604-9649
Figures
Abstract
Dengue is among the fastest-spreading vector-borne infectious disease, with outbreaks often overwhelm the health system and result in huge morbidity and mortality in its endemic populations in the absence of an efficient warning system. A large number of prediction models are currently in use globally. As such, this study aimed to systematically review the published literature that used quantitative models to predict dengue outbreaks and provide insights about the current practices. A systematic search was undertaken, using the Ovid MEDLINE, EMBASE, Scopus and Web of Science databases for published citations, without time or geographical restrictions. Study selection, data extraction and management process were devised in accordance with the ‘Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies’ (‘CHARMS’) framework. A total of 99 models were included in the review from 64 studies. Most models sourced climate (94.7%) and climate change (77.8%) data from agency reports and only 59.6% of the models adjusted for reporting time lag. All included models used climate predictors; 70.7% of them were built with only climate factors. Climate factors were used in combination with climate change factors (13.4%), both climate change and demographic factors (3.1%), vector factors (6.3%), and demographic factors (5.2%). Machine learning techniques were used for 39.4% of the models. Of these, random forest (15.4%), neural networks (23.1%) and ensemble models (10.3%) were notable. Among the statistical (60.6%) models, linear regression (18.3%), Poisson regression (18.3%), generalized additive models (16.7%) and time series/autoregressive models (26.7%) were notable. Around 20.2% of the models reported no validation at all and only 5.2% reported external validation. The reporting of methodology and model performance measures were inadequate in many of the existing prediction models. This review collates plausible predictors and methodological approaches, which will contribute to robust modelling in diverse settings and populations.
Author summary
Dengue is considered as a major public health challenge and a life-threatening disease affecting people worldwide. Over the past decades, numerous forecast models have been developed to predict dengue incidence using various factors based on different geographical locations. Dengue transmission appears to be highly sensitive to climate variability and change, however quantitative models used to assess the relationship between climate change and dengue often differ due to their distribution assumptions, the nature of the relationship and the spatial and/or temporal dynamics of the response. We performed a systematic review to examine current literature surrounding existing quantitative models based on development methodology, predictor variable used and model performance. Our analysis demonstrates several shortcomings in current modelling practice, and advocates for the use of real time primary predictor data, the incorporation of non-climatic parameters as predictors and more comprehensive reporting of model development techniques and validation.
This review collates methodological approaches adopted in the modelling practices in the field across current literature. This will provide an evidence-based framework for upgrading future modelling practice to develop more accurate predictive models with robust techniques. In turn, this also provided an opportunity for the effective distribution of limited public health resources to prepare for demand.
Citation: Leung XY, Islam RM, Adhami M, Ilic D, McDonald L, Palawaththa S, et al. (2023) A systematic review of dengue outbreak prediction models: Current scenario and future directions. PLoS Negl Trop Dis 17(2): e0010631. https://doi.org/10.1371/journal.pntd.0010631
Editor: Husain Poonawala, Tufts Medical Center, UNITED STATES
Received: July 5, 2022; Accepted: January 29, 2023; Published: February 13, 2023
Copyright: © 2023 Leung et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors declare that they have no conflicts of interest.
Introduction
Dengue fever is one of the fastest-spreading mosquitos-borne disease primarily of tropical and subtropical regions and is caused by various dengue virus strains [1,2]. In 2017 alone, over 100 million people were estimated to have acquired the infection, contributing to a globally increasing burden of disease [3]. Although most infections are mild, dengue shock syndrome and dengue haemorrhagic fever are severe forms of infections and can be fatal [4,5]. The case-fatality rate can be as high as 20% in the absence of prompt diagnosis and lack of specific antiviral drugs or vaccines [6,7], particularly in resource-limited settings. When an outbreak is particularly large, the influx of severe dengue cases can overwhelm the health system and prevent optimal care. Dengue also imposes an enormous societal and economic burden on many of the tropical countries where the disease is endemic [8]. An accurate prediction of the size of the outbreak and trends in disease incidence early enough can limit further transmission [5], and is likely to facilitate planning the allocation of healthcare resources to meet the demand during an outbreak.
Vector-borne pathogens characteristically demonstrate spatial heterogeneity—a result of spatial variation in vector habitat, climate patterns and subsequent human control actions [9–11]. The interplay of human, climate and mosquito dynamics give rise to a complex system that determines the pattern of dengue transmission, which in turn influences the potential for outbreak [12]. These relationships have been explored over the decades in the development of predictive models worldwide. Models vary widely in their purposes [13–15] and settings [16–21]. Many of these models excel at different tasks, however for a prediction model to be efficient, it requires a systematic, self-adaptive and generalizable framework capable of identifying weather and population susceptibility patterns across geographic regions. The scientific community has not yet agreed upon a model that provides the best prediction. The selection of predictors for the existing models is also quite heterogeneous. Some models rely solely on climate variables [16], some include vector characteristics [17,18] others use population characteristics [19–21]. A wide range of statistical techniques are used with varying degrees of accuracy and robustness among the existing models [16–21].
Clarity in the documentation of the model development processes and model performance are essential for ensuring the robustness of the prediction [22], which is scarce as many of the existing models have not yet been systematically appraised. Given the disparate approaches, a focused synthesis and appraisal of the existing models, along with their building techniques and factor catchments, is required. Carefully establishing these details will provide the foundation for updating and developing robust models in future. This study aimed to systematically review all published literature that reported quantitative models to predict dengue outbreaks, revealing several shortcomings in the usage of real time primary predictive data and non-climatic predictors in the development of models, as well as inadequate reporting of techniques, model and performance measure validation.
Methods
Search strategy and selection criteria
This systematic review’s aim, search strategy and study selection process were devised in accordance with the seven items in the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (‘CHARMS’) framework [23]. CHARMS framework is a systematic review tool, devised to facilitate and guide the methodological aspects the systematic review of prediction modelling studies, ranging from question development, appraisal of studies, and data extraction thereof. Detail of the CHARMS checklist can be found elsewhere [23]. The review followed the Preferred Reporting Items for Systematic Review and Meta-Analysis (‘PRISMA’) guidelines [24], and was registered in PROSPERO (CRD42018102100).
A literature search was conducted from inception until October 2022 using the electronic databases of Ovid MEDLINE, Embase, Scopus and Web of Science to obtain the information on the statistical models for predicting the number of dengue cases based on climatic factors. Google Scholar and the bibliography of included papers were also searched. The search strategies were developed under the guidance of an information specialist from Monash University Library. For the purposes of this study, dengue fever or dengue haemorrhagic fever or dengue shock syndrome were considered as a single entity “dengue”. Search strategy included Medical Subject Headings (‘MeSH’) and keyword terms including “dengue”, “severe dengue,” “weather,” “climate change,” “model,” “predict,” and “forecast.” The detailed search strategy and history are presented in S1 Table.
The review included studies focused on (1) prognostic prediction models which aim to review models predicting future events, (2) incidence of dengue fever or dengue haemorrhagic fever cases, (3) models to be used to predict the number of cases prior to an outbreaks, (4) models intended to inform public health divisions of future dengue outbreaks, (5) models with no restrictions on the time span of prediction and (6) prediction model development studies without external validation, or with external validation in independent data. Peer-reviewed original articles that presented a model and were available as full-text articles were considered eligible if they focused on predicting the number of dengue cases or an outbreak based on number of dengue incidence. Articles that focused on updating previously developed models were only included if they presented an updated version of the model. Articles which dealt exclusively with dengue in international travellers, or which only analyse the correlation between climate parameters and dengue cases without presenting a prediction model were excluded. Furthermore, articles which used models for predicting the population of dengue vectors (e.g., Aedes aegypti or Aedes albopictus) as well as articles which only offer susceptible-infected-recovered modelling stochastic or transmission rates modelling were excluded. Articles which presented a model only dealing with spatial or temporal components of dengue risk were considered ineligible. Conference proceedings, book chapters, abstracts or letters were also excluded. Titles and abstracts of the retrieved articles were screened independently by two reviewers (RMI, MMA). Two review team members (LM, XYL) then retrieved the full text of those potentially eligible studies and independently assessed their eligibility. Disagreements were resolved by a third reviewer (MNK). A detailed study selection process is illustrated in the PRISMA flow diagram (Fig 1) [24].
Data analysis
Based on the data extraction fields of the CHARMS framework [23], a standardised table was developed to extract data from the selected studies for assessment of quality and evidence synthesis. The data extraction table consists of eleven domains, each with a specific item, that extract data from the reports of the primary forecasting model. Key information extracted from the included articles were period and geographical region, sources of data, outcomes to be predicted, modelling covariates variables, sample size, statistical techniques, model performances, model evaluation, and key findings. Information regarding handling and/or reporting of missing data was also extracted. Each paper was independently reviewed by two reviewers (MMA, XYL) and discrepancies were resolved through discussion with each other or with a third reviewer (SP) where necessary.
Extracted data from the selected studies were summarised and the key information about the methodological characteristics of these models were tabulated. Descriptive statistics were generated based on model characteristics and comparative methodological features such as outcome types, target population, data sources and predictor selection techniques. All statistical analyses were performed using Stata (version 17.0).
Results
The initial search yielded 6553 studies. After duplicates were removed, 3244 studies were screened for titles and abstracts. This led to 153 studies for full text review, and 64 that strictly met the inclusion criteria (Fig 1), 16 of these studies reported multiple models. A total of 99 models from 64 selected studies were identified. Characteristics of the models including, year, country and source of data used, predictors and outcome of the models, overall model development technique and model performance related variables are summarised in Table 1 [14,15,17–21,25–81].
Table 2 presents the sources of data used for modelling. Most of the models (90.7%) sourced dengue incidence data from surveillance and 44.3% used registry data, while 34.0% also used hospital or laboratory data. While most (94.7%) of the models used climate data from government agency reports, only around 22.1% of the models used data from the meteorological stations in real-time. Climate change data was also sourced mostly (77.8%) from government agency reports, only 11.1% used international environmental agency data and 22.2% used local environmental agency report. Half (50.0%) of the models used vector data from entomological surveillance and 25.0% used vector data from laboratory sources. Around 83.8% of the models were built based on the sample from general population, 16.2% used only urban samples. Around 46.9% of the models used monthly aggerate data, over a third (29.3%) used weekly aggregate data and 23.2% used daily aggregated data of the predictors. The majority (59.6%) of the models incorporated reporting time lag adjustment. Although 17.2% of the models addressed the missing data, 30.3% did not address the issue, while the majority (52.5%) did not specifically report the missing value. Around 80.8% of the models were intended to predict the number of dengue cases and 19.2% focused on predicting dengue outbreaks, based on predetermined case number threshold.
Table 3 summarises the statistical methods adopted by the prediction models. Modelling techniques were broadly categorised under two genres, statistical models (60.6%) and machine learning (39.4%). The statistical models were broadly comprised of linear regression models (18.3%), time series/autoregressive models (26.7%), Poisson regression models (18.3%) and generalized additive models (16.7%). Neural networks models (23.1%), random forest models (15.4%), and ensemble models (10.3%) were types of machine learning models used.
All theoretically plausible predictors were considered as candidate predictors in 71.7% of models and pre-selection of predictors based on unadjusted association with the model outcome was considered in 28.3% of models. Reporting of essential modelling techniques was heterogeneous– 84.8% of models reported model performance, 58.6% reported model calibration and 47.5% reported model discrimination. Among the performance metrices, Root Mean Squared Error (‘RMSE’) (11.1%), Mean Squared Error (‘MSE’) (7.3%), Mean Absolute Percentage Error (‘MAPE’) (5.1%), and Receiver Operating Characteristic (‘ROC’) (5.1%) were notable. Of these models, most (75.8%) reported the internal validation alone, only 5.2% reported both internal and external validation and 20.2% reported no validation at all. The validation techniques included: split sample validation (development and validation) (20.3%), cross validation, which involves resampling of the derivation sample (40.5%) and performance metrics (29.1%).
Table 4 presents the factors used for prediction models. All of the models included in the review used climate predictors in their model. Among the climate predictors: humidity (77.4%), temperature (95.2%) and rainfall (81.0%), were used in most models. Windspeed and direction (27.4%), precipitation (15.5%) and sunshine (10.7%) were among other notable climate factors. Considering the similarity of the description of factors, climate change and environmental factors were collapsed in to one category under climate change. Overall, 18.2% of the models used climate change and/or environmental predictors. El Nino-Southern Oscillation (‘ENSO’), Southern Oscillation Index (‘SOI’), Oceanic Nino Index (‘ONI’), hydric balance and vegetation index were among the key climate change predictors. Vegetation Index and enhanced vegetation index were among the key environmental factors reported.
Vector-related predictors were included in 8.1% of models, and the key vector related predictors were container index, Breteau index, adult productivity index, breeding percentage and mosquito infection rate. Demographic predictors were included in 8.1% of models, and key demographic predictors were, population size, population density, area under the urban settlement, access to piped water, education coverage and GDP per capita (Table 4).
The combination of the predictors used in the model are depicted in Fig 2. While majority (70.7%) of the models were built solely on climate predictors, none of the models used the combination of all four (climate, climate change, vector and demography) categories of predictors. The combination of climate, climate change and demographic predictors was used in 3.1% of the models and the combination of climate and climate change predictors were used in 13.4% models. Among other notable combinations were, climate and vector predictors (6.3%) and climate and demographic predictor (5.2%).
Discussion
This systematic review evaluated 99 dengue outbreak prediction models from 64 studies, predicting the number of dengue cases or outbreaks from a variety of settings and populations. Our review identified, three major area of inadequacy in the current modelling practices. Firstly, use of secondary predictor data—acquired from reports—were quite prevalent among models. Secondly, as data for other non-climate variables were not included in the majority of the models, they failed to capture a holistic view of dengue development in the prediction process. Lastly, inadequacy in the reporting of methodology, model validation and performance measures were quite prevalent in the existing prediction models. One positive aspect seen in the current modelling practice is the shift toward robust modelling technique, such as use of machine learning algorithm and autoregressive time series techniques.
While effective treatments and prevention measures are still being developed, an early warning system for an epidemic has the potential to reduce the toll of severe disease on the health system and population [82]. Developing a clearer understanding of the factors affecting dengue transmission is an important step towards mitigating the impact of the disease on health systems and on communities at large. Early prediction of dengue incidences or alerts regarding impending outbreak may contribute to the health system preparedness through effective resource mobilization and creating public awareness. Such predictions also have policy implications, as epidemiological evidence generated through modelling feeds the policy making process and facilitates the prioritization of interventions, such as vector control and environmental modification particularly in regard to climate change [83]. Considering dengue is a mosquito-borne disease, the majority of outbreak prediction models focus on climate dependency of mosquito breeding and dengue transmission [4–7]. While many models have been successful in predicting relative cases of dengue in real settings, incorrect prediction results have been observed in several included studies. For example, a model by Adde et al. [26] was unable to forecast a dengue outbreak in 2001–2005 with the use of climate data from 1991–2000. One of the potential reasons for their inaccurate prediction is the geo-spatial variation of climate and environment within regions. In their study, the decision to include vastly heterogenous geographical areas led to variation in model prediction, which—be due to exclusion of non-climatic factors—may be the explanation of the poor performance in many of the earlier prediction models could. An increasing number of later models appears to incorporate a wide range of vector parameters as well as demographic parameters. Chang et al. pointed out that, entomological (vector) factor combined with other meteorological (climate) factors, have better prediction performance, and their prediction accuracy is often higher than that of climate predictors alone [21].
For dengue incidence data, the majority of the models relied on reports from government organizations based on notifiable data. Notification involves passive surveillance, where there is potential for systematic underreporting along with varying time lag. Modelling with data from active surveillance or real-time study may minimize such limitations. A considerable number of models did not consider the time lag affecting the prediction, which may be responsible for possible delays in weather affecting mosquito vectors and subsequently viruses. Due to the nature of dengue disease dynamics, failure to address time lag in model development is likely to affect prediction accuracy. Critical points in the natural history of disease timeline those may generate time lag may start with mosquito development, and subsequently also during acquisition and amplification virus in mosquitoes, mosquito host behaviour (i.e. biting and feeding pattern) and the incubation period of the virus in the human body [12,48]. Some studies have found a positive correlation between climate variables with time-lags at several points in the natural history of disease timeline [48,53]. Therefore, the adjustment for the time lags while predicting dengue is indispensable, especially when meteorological data is used [12].
The majority of included models were built on conventional regression techniques. According to recent literature, the time series technique is particularly considered effective in predicting the highly auto-correlated nature of dengue infection [84,85]. Machine learning techniques are employed in around 40% of the included models, and is particularly prevalent among the recently developed models. Batista et al. confirmed superiority with ML techniques demonstrating a lower error rate compared to the conventional statistics-based model in predicting dengue cases. In the age of big data, this technique can leverage data availability and in addition to being non-parametric in nature, can also provide some leeway in terms of strict assumption [86]. Random forest, neural networks, gradient boosting and support vector algorithms are notable subsets of machine learning algorithms, which have made significant contributions to several areas of public health, particularly in the forecasting of infectious diseases like malaria [87] and COVID-19 [88], and may have similar utility for making dengue outbreak predictions. Although machine learning in gaining popularity, future modelling in this area may benefit from using mechanistic models [89]. This modelling technique have played an essential role in shaping public health policy over the past decades [90]. Mechanistic models have the potential to provide additional insight regarding precise dynamics of the transmission and infection of dengue. As these models highlight underlying processes that drive the patterns. These models can particularly aid in the prediction through incorporating the observed trajectory of vectors.
In the modelling process, generating an algorithm or equation is only part of the process. It is not complete unless its performance has been assessed considering discrimination [91] and calibration [92], both internally and using the population outside of what it is developed from, respectively. Among the existing models examined, reporting of the discrimination and calibration is very low. Without knowledge of model performance through validation in both source populations and populations other than where it was developed, objective evaluation of models is difficult [93]. Predictive models can be of great value only if there is certainty of its accuracy, that is, how precisely the model can predict an outcome in real world [94]. In the majority of the published models, real-world validation has not been performed or reported. Generally, models are unlikely to predict as well in real-world samples as it would in the derivation sample; this validity shrinkage can often be quite substantial. Hence, future models should report a mechanism of estimating and reporting potential validity shrinkage as well as predictive validity in real world data [95, 96].
In a substantial proportion of the models that reported validation, the original dataset was randomly split into the development and validation subset. Although this approach is widely used in many model validation settings, there are some setbacks when using smaller operational sample sizes, as split-sample analyses give overly pessimistic estimates of model performance and are accompanied by large variability [97]. Bootstrapping is generally considered to be the preferred internal validation method in predictive models [98, 99]. Interestingly, bootstrapping was not used in any of the models in included studies, instead cross-validation technique was adopted in most of them. External validation, on the other hand, was used only in very few included studies. This is despite the fact that external validation is considered pivotal to model development and a key indicator of model performance through highlighting applicability to participants, centres, regions or environments [23]. The external validation is particularly essential for model redevelopment, where the original model is adjusted, updated, or recalibrated based on validation data to improve performance [100]. This update may include adjusting the baseline risk (interception or hazard) of the original model, adjusting the weight or regression coefficient of the predictor, adding new predictors, or removing existing predictors from the model.
This review has a number of strengths–specifically, the use of the CHARMS checklist [23], designed for the assessment of the applicability of the prediction models. In addition, inclusion and exclusion criteria were strictly followed, and database searches were conducted by an expert librarian. However, there are a few limitations of the review–the models in this review are not explicitly rated based on quality or performance due to the lack of accepted criteria for rating the quality of forecasting models. In addition, although calibration was reported in several studies, calibration measures lack clarification, which may impact the overall evaluation of the model performance. The model performance could not be compared across methodological approaches in quantitative synthesis because of a lack of model performance data, and those that did provide data are mostly generated from internal validation data which may result in overfitting.
Conclusion
In conclusion, failure to use of real time primary predictor data, failing to incorporate non-climatic parameters as predictor and insufficient reporting of model development techniques, model validation and performance measure were the major inadequacies identified in the current modelling practice. The paradigm shift towards robust modelling techniques, such as the use of machine learning algorithms and autoregressive time series, is a significant positive trend in contemporary model practices. The findings of this review have the potential to lay the groundwork for improved modelling practices in the future. These findings will contribute to robust modelling in different settings and populations and have important implications for the planning and decision-making process for early dengue intervention and prevention.
Supporting information
S1 Table. Search strategy for OVID Medline, as performed in October 2022.
https://doi.org/10.1371/journal.pntd.0010631.s001
(DOCX)
References
- 1. Simmons CP, Farrar JJ, Chau Nguyen vV, Wills B. Dengue. N Engl J Med. 2012;366: 1423–1432. pmid:22494122
- 2. Tatem AJ, Hay SI, Rogers DJ. Global traffic and disease vector dispersal. Proc Natl Acad Sci USA. 2006;103: 6242–6247. pmid:16606847
- 3. Zeng Z, Zhan J, Chen L, Chen H, Cheng S. Global, regional, and national dengue burden from 1990 to 2017: A systematic analysis based on the global burden of disease study 2017. EClinicalMedicine. 2021;32: 100712. pmid:33681736
- 4. Rigau-Pérez JG, Clark GG, Gubler DJ, Reiter P, Sanders EJ, Vorndam AV. Dengue and dengue haemorrhagic fever. Lancet. 1998;352: 971–977. pmid:9752834
- 5. Chen HL, Hsiao WH, Lee HC, Wu SC, Cheng JW. Selection and Characterization of DNA Aptamers Targeting All Four Serotypes of Dengue Viruses. PLoS One. 2015;10: e0131240. pmid:26110785
- 6. Capeding MR, Tran NH, Hadinegoro SRS, Ismail HI, Chopitayasunondh T, Chua MN, et al. Clinical efficacy and safety of a novel tetravalent dengue vaccine in healthy children in Asia: a phase 3, randomised, observer-masked, placebo-controlled trial. Lancet. 2014;384: 1358–1365. pmid:25018116
- 7. Halstead SB. Dengue vaccine development: a 75% solution? Lancet. 2012;380: 1535–1536. pmid:22975339
- 8. Halstead SB. Dengue. Curr Opin Infect Dis. 2002;15: 471–476. pmid:12686878
- 9.
World Health Organization. Dengue and severe dengue. 2022 Jan 10 [cited 2022 July 27]. In: WHO Fact Sheets [Internet]. Geneva: World Health Organization. Available from: who.int/news-room/fact-sheets/detail/dengue-and-severe-dengue.
- 10. Mbogo CM, Mwangangi JM, Nzovu J, Gu W, Yan G, Gunter JT, et al. Spatial and temporal heterogeneity of Anopheles mosquitoes and Plasmodium falciparum transmission along the Kenyan coast. Am J Trop Med Hyg. 2003;68: 734–742. pmid:12887036
- 11. Acevedo MA, Prosper O, Lopiano K, Ruktanonchai N, Caughlin TT, Martcheva M, et al. Spatial heterogeneity, host movement and mosquito-borne disease transmission. PLoS One. 2015;10: e0127552. pmid:26030769
- 12. Teurlai M, Menkès CE, Cavarero V, Degallier N, Descloux E, Grangeon JP, et al. Socio-economic and Climate Factors Associated with Dengue Fever Spatial Heterogeneity: A Worked Example in New Caledonia. PLoS Negl Trop Dis. 2015;9: e0004211. pmid:26624008
- 13. Medlock JM, Leach SA. Effect of climate change on vector-borne disease risk in the UK. Lancet Infect Dis. 2015.15: 721–730. pmid:25808458
- 14. Phung D, Talukder M, Rutherford S, Chu C. A climate-based prediction model in the high-risk clusters of the Mekong Delta region, Vietnam: Towards improving dengue prevention and control. Trop Med Int Health. 2016;21: 1324–1333. pmid:27404323
- 15. Karim MN, Munshi SU, Anwar N, Alam MS. Climatic factors influencing dengue cases in Dhaka city: a model for dengue prediction. Indian J Med Res. 2012;136: 32–39. pmid:22885261
- 16. Benedum CM, Seidahmed OME, Eltahir EAB, Markuzon N. Statistical modeling of the effect of rainfall flushing on dengue transmission in Singapore. PLoS Negl Trop Dis. 2018;12: e0006935. pmid:30521523
- 17. Aburas H, Cetiner B, Sari M. Dengue confirmed-cases prediction: A neural network model. Expert Syst Appl. 2010;37: 4256–4260.
- 18. Gharbi M, Quenel P, Gustave J, Cassadou S, Ruche GL, Girdary L, et al. Time series analysis of dengue incidence in Guadeloupe, French West Indies: forecasting models using climate variables as predictors. BMC Infect Dis. 2011;11: 166. pmid:21658238
- 19. Qureshi EMA, Tabinda AB, Vehra S. Predicting dengue outbreak in the metropolitan city Lahore, Pakistan, using dengue vector indices and selected climatological variables as predictors. J Pak Med Assoc. 2017;67: 416–421. pmid:28303992
- 20. Betanzos-Reyes AF, Rodríguez MH, Romero-Martínez M, Sesma-Medrano E, Rangel-Flores H, Santos-Luna R. Association of dengue fever with Aedes spp. abundance and climatological effects. Salud Publica Mex. 2018;60: 12–20. pmid:29689652
- 21. Chang F, Tseng Y, Hsu P, Chen C, Lian I, Chao D. Re-assess Vector Indices Threshold as an Early Warning Tool for Predicting Dengue Epidemic in a Dengue Non-endemic Country. PLoS Negl Trop Dis. 2015;9: e0004043. pmid:26366874
- 22. Steyerberg EW, Moons KGM, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research. PLoS Med. 2013;10: e1001381. pmid:23393430
- 23. Moons KGM, Groot AH, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist. PLoS Med. 2014;11: e1001744. pmid:25314315
- 24. Moher D, Liberati A, Etzlaff J, Altman D. Preferred reporting items for systematic reviews and Meta-analyses: The PRISMA statement. Ann Intern Med. 2009; 151: 264–269. pmid:19622511
- 25. Abualamah WA, Akbar NA, Banni HS, Bafail MA. Forecasting the morbidity and mortality of dengue fever in KSA: A time series analysis (2006–2016). Journal of Taibah University Medical Sciences. 2021;16(3): 448–55.
- 26. Adde A, Roucou P, Mangeas M, Ardillon V, Desenclos JC, Rousset D, et al. Predicting Dengue Fever Outbreaks in French Guiana Using Climate Indicators. PLoS Negl Trop Dis. 2016;10: e0004681. pmid:27128312
- 27. Anggraeni W, Nurmasari R, Riksakomara E, Samopa F, Wibowo RP, Pujiadi. Modified Regression Approach for Predicting Number of Dengue Fever Incidents in Malang Indonesia. Procedia Comput Sci. 2017;124: 142–150.
- 28. Bal S, Sodoudi S. Modeling and prediction of dengue occurrences in Kolkata, India, based on climate factors. Int J Biometeorol. 2020;64: 1379–1391. pmid:32328786
- 29. Banu S, Guo Y, Hu W, Dale P, Mackenzie JS, Mengersen K, et al. Impacts of El Niño Southern Oscillation and Indian Ocean Dipole on dengue incidence in Bangladesh. Sci Rep. 2015;5: 16105.
- 30. Baquero OS, Santana LMR, Chiaravalloti-Neto F. Dengue forecasting in São Paulo city with generalized additive models, artificial neural networks and seasonal autoregressive integrated moving average models. PLoS One. 2018;13(4): 1–12.
- 31. Bett B, Grace D, Lee HS, Lindahl J, Nguyen-Viet H, Phuc PD, et al. Spatiotemporal analysis of historical records (2001–2012) on dengue fever in Vietnam and development of a statistical model for forecasting risk. PloS One. 2019;14: e0224353. pmid:31774823
- 32. Bouzid M, Colón-González FJ, Lung T, Lake IR, Hunter PR. Climate change and the emergence of vector-borne diseases in Europe: Case study of dengue fever. BMC Public Health. 2014;14: 781. pmid:25149418
- 33. Buczak AL, Baugher B, Moniz LJ, Bagley T, Babin SM, Guven E. Ensemble method for dengue prediction. PLoS One. 2018;13: e0189988. pmid:29298320
- 34. Carvajal T, Viacrusis K, Hernandez L, Ho H, Amalin D, Watanabe K. Machine learning methods reveal the temporal pattern of dengue incidence using meteorological factors in metropolitan Manila, Philippines. BMC Infect Dis. 2018;18: 183. pmid:29665781
- 35. Chen J, Ding RL, Liu KK, Xiao H, Hu G, Xiao X, et al. Collaboration between meteorology and public health: Predicting the dengue epidemic in Guangzhou, China, by meteorological parameters. FRONTIERS IN CELLULAR AND INFECTION MICROBIOLOGY. 2022;12: 881745. pmid:36017372
- 36. Chen Y, Liu T, Yu X, Zeng Q, Cai Z, Wu H, et al. An ensemble forecast system for tracking dynamics of dengue outbreaks and its validation in China. PLoS Computational Biology. 2022;18(6): 1–22. pmid:35759513
- 37. Cheng YC, Lee FJ, Hsu YT, Slud EV, Hsiung CA, Chen CH, et al. Real-time dengue forecast for outbreak alerts in Southern Taiwan. PLoS Neglected Tropical Diseases. 2020;14(7): 1–18. pmid:32716983
- 38. Chuang T, Luis Fernando C, Chen P. Effects of local and regional climatic fluctuations on dengue outbreaks in southern Taiwan. PLoS One. 2017;12: e0178698. pmid:28575035
- 39. Colón-González FJ, Fezzi C, Lake IR, Hunter PR. The Effects of Weather and Climate Change on Dengue. PLoS Negl Trop Dis. 2013;7: e2503. pmid:24244765
- 40. Depradine C, Lovell E. Climatological variables and the incidence of Dengue fever in Barbados. Int J Environ Health Res. 2004;14: 429–441. pmid:15545038
- 41. Descloux E, Mangeas M, Menkes CE, Lengaigne M, Leroy A, Tehei T, et al. Climate-Based Models for Understanding and Forecasting Dengue Epidemics. PLoS Negl Trop Dis. 2012;6: e1470. pmid:22348154
- 42. Dey SK, Rahman MM, Howlader A, Siddiqi UR, Uddin KMM, Borhan R, et al. Prediction of dengue incidents using hospitalized patients, metrological and socio-economic data in Bangladesh: A machine learning approach. PLoS ONE. 2022;17(7): 1–17. pmid:35857776
- 43.
Dharmawardhana KGS, Lokuge JN, Dassanayake PSB, Sirisena ML, Fernando ML, Perera AS, et al. Predictive model for the dengue incidences in Sri Lanka using mobile network big data. In: 2017 IEEE International Conference on Industrial and Information Systems (ICIIS); 2017 Dec 15–16; Peradeniya, Sri Lanka. Piscataway, New Jersey: Institution of Electrical and Electronics Engineers; 2017. p. 1–6. https://doi.org/10.1109/ICIINFS.2017.8300381
- 44. Earnest A, Tan SB, Wilder-Smith A. Meteorological factors and El Nino Southern Oscillation are independently associated with dengue infections. Epidemiol Infect. 2012;140: 1244–1251.
- 45. Edussuriya C, Deegalla S, Gawarammana I. An accurate mathematical model predicting number of dengue cases in tropics. PLoS Neglected Tropical Diseases. 2021;15(11): 1–15. pmid:34748566
- 46. Guo P, Liu T, Zhang Q, Wang L, Xiao J, Zhang Q, et al. Developing a dengue forecast model using machine learning: A case study in China. PLoS Negl Trop Dis. 2017;11: e0005973. pmid:29036169
- 47. Jain R, Sontisirikit S, Iamsirithaworn S, Prendinger H. Prediction of dengue outbreaks based on disease surveillance, meteorological and socio-economic data. BMC Infect Dis. 2019;19: 272. pmid:30898092
- 48. Jayaraj V, Avoi R, Gopalakrishnan N, Raja D, Umasa Y. Developing a dengue prediction model based on climate in Tawau, Malaysia. Acta Trop. 2019;197: 105055. pmid:31185224
- 49. Lauer SA, Sakrejda K, Ray EL, Keegan LT, Bi Q, Suangtho P, et al. Prospective forecasts of annual dengue hemorrhagic fever incidence in Thailand, 2010–2014. Proc Natl Acad Sci USA. 2018;115: E2175–E2182. pmid:29463757
- 50. Li C, Wang X, Wu X, Liu J, Ji D, Du J. Modeling and projection of dengue fever cases in Guangzhou based on variation of weather factors. Sci Total Environ. 2017;605–606: 867–873. pmid:28683431
- 51. Li Z, Yang L, Dong J, Gurgel H, Xu L. Improving Dengue Forecasts by Using Geospatial Big Data Analysis in Google Earth Engine and the Historical Dengue Information-Aided Long Short Term Memory Modeling. Biology. 2022;11(2). pmid:35205036
- 52. Liu D, Guo S, Zou M, Chen C, Deng F, Xie Z, et al. A dengue fever predicting model based on Baidu search index data and climate data in South China. PLoS One. 2019;14: e0226841. pmid:31887118
- 53. Lowe R, Bailey T, Stephenson D, Jupp TE, Graham RJ, Barcellos C, et al. The development of an early warning system for climate-sensitive disease risk with a focus on dengue epidemics in Southeast Brazil. Stat Med. 2013;32: 864–883. pmid:22927252
- 54. Luz PM, Mendes BVM, Codeco CT, Struchiner CJ, Galvani AP. Time series analysis of dengue incidence in Rio de Janeiro, Brazil. Am J Trop Med Hyg. 2008;79: 933–939. pmid:19052308
- 55. McGough SF, Clemente L, Kutz JN, Santillana M. A dynamic, ensemble learning approach to forecast dengue fever epidemic years in Brazil using weather and population susceptibility cycles. J R Soc Interface. 2021;18: 20201006. pmid:34129785
- 56. Mincham G, Baldock KL, Rozilawati H, Williams CR. Development of a mechanistic dengue simulation model for Guangzhou. Epidemiol Infect. 2019;147: e125. pmid:30869038
- 57. Nakhapakorn K, Tripathi N. An information value-based analysis of physical and climatic factors affecting dengue fever and dengue haemorrhagic fever incidence. Int J Health Geogr. 2005;4: 13. pmid:15943863
- 58.
Nan J, Liao X, Chen J, Chen X, Chen J, Dong G, et al. Using Climate Factors to Predict the Outbreak of Dengue Fever. In: 2018 7th International Conference on Digital Home (ICDH); 2018 Nov 30—Dec 1; Guilin, China. International Conference on Digital Home (ICDH); 2018.p. 213–8. https://doi.org/10.1109/ICDH.2018.00045
- 59. Nguyen V, Tran TT, Mulhall J, Minh HV, Duong TQ, Chien NV, et al. Deep learning models for forecasting dengue fever based on climate data in Vietnam. PLoS Neglected Tropical Diseases. 2022;16(6): 1–22. pmid:35696432
- 60. Nuraini N, Fauzi IS, Fakhruddin M, Sopaheluwakan A, Soewono E. Climate-based dengue model in Semarang, Indonesia: Predictions and descriptive analysis. Infect Dis Model. 2021;6: 598–611. pmid:33869907
- 61. Olmoguez I, Catindig M, Fel M, Fatima G. Developing a Dengue Forecasting Model: A Case Study in Iligan City. Int J Adv Comput Sci Appl. 2019;10: 281–286.
- 62.
Pham DN, Aziz T, Kohan A, Nellis S, Jamil JBA, Khoo JJ, et al. How to efficiently predict dengue incidence in Kuala Lumpur. In: 2018 Fourth International Conference on Advances in Computing, Communication & Automation (ICACCA); 2018 Oct 26–28; Subang Jaya, Malaysia. Piscataway, New Jersey: Institution of Electrical and Electronics Engineers; 2018. p. 1–6. https://doi.org/10.1109/ICACCAF.2018.8776790
- 63. Pham NTT, Nguyen CT, Pineda-Cortel MRB. Time-series modelling of dengue incidence in the Mekong Delta region of Viet Nam using remote sensing data. Western Pacific Surveillance and Response. 2020;11(1): 13–21. pmid:32963887
- 64. Phung D, Huang C, Rutherford S, Chu C, Wang X, Nguyen M, et al. Identification of the prediction model for dengue incidence in Can Tho city, a Mekong Delta area in Vietnam. Acta Trop. 2015;141: 88–96. pmid:25447266
- 65. Pineda-Cortel MRB, Clemente BM, Pham NTT. Modeling and predicting dengue fever cases in key regions of the Philippines using remote sensing data. Asian Pacific Journal of Tropical Medicine. 2019;12(2): 60–6.
- 66. Pinto E, Coelho M, Oliver L, Massad E. The influence of climate variables on dengue in Singapore. Int J Environ Health Res. 2011;21: 415–426. pmid:21557124
- 67. Puengpreeda A, Yhusumrarn S, Sirikulvadhana S. Weekly Forecasting Model for Dengue Hemorrhagic Fever Outbreak in Thailand. Eng J. 2020;24: 71–87.
- 68. Ramadona AL, Lazuardi L, Hii YL, Holmner Å, Kusnanto H, Rocklöv J. Prediction of Dengue Outbreaks Based on Disease Surveillance and Meteorological Data. PLoS One. 2016;11: e0152688. pmid:27031524
- 69. Roster K, Connaughton C, Rodrigues FA. Predicting Dengue Fever in Brazilian Cities. bioRxiv:430949 [Preprint]. 2021 [cited 2022 June 27]. Available from: https://www.biorxiv.org/content/10.1101/2021.02.17.430949v1
- 70. Salim NAM, Wah YB, Reeves C, Smith M, Yaacob WFW, Mudin RN, et al. Prediction of dengue outbreak in Selangor Malaysia using machine learning techniques. Sci Rep. 2021;11: 939. pmid:33441678
- 71. Shi Y, Liu X, Kok S, Rajarethinam J, Liang S, Yap G, et al. Three-Month Real-Time Dengue Forecast Models: An Early Warning System for Outbreak Alerts and Policy Decision Support in Singapore. Environ Health Perspect. 2016;124: 1369–1375. pmid:26662617
- 72. Siriyasatien P, Phumee A, Ongruk P, Jampachaisri K, Kesorn K. Analysis of significant factors for dengue fever incidence prediction. BMC Bioinformatics. 2016;17: 166. pmid:27083696
- 73. Withanage GP, Hapugoda MD, Gunawardena YINS, Viswakula SD. A forecasting model for dengue incidence in the District of Gampaha, Sri Lanka. Parasit Vectors 2018;11: 262. pmid:29690906
- 74. Xu J, Xu K, Li Z, Meng F, Tu T, Xu L, et al. Forecast of Dengue Cases in 20 Chinese Cities Based on the Deep Learning Method. Int J Environ Res Public Health. 2020;17: 453. pmid:31936708
- 75. Yuan HY, Wen TH, Kung YH, Tsou HH, Chen CH, Chen LW, et al. Prediction of annual dengue incidence by hydro-climatic extremes for southern Taiwan. International Journal of Biometeorology. 2019;63(2):259–68. pmid:30680621
- 76. Yuan H, Liang J, Lin P, Sucipto K, Tsegaye MM, Wen TH, et al. The effects of seasonal climate variability on dengue annual incidence in Hong Kong: A modelling study. Sci Rep. 2020;10: 4297. pmid:32152334
- 77. Zafra B. Predicting dengue in the Philippines using artificial neural network. medRxiv:20209718 [Preprint]. 2021 [cited 2022 July 3]. Available from: https://www.medrxiv.org/content/10.1101/2020.10.08.20209718v3
- 78. Zambrano LI, Sevilla C, Reyes-Garcia SZ, Sierra M, Kafati R, Rodriguez-Morales AJ, et al. Potential impacts of climate variability on Dengue Hemorrhagic Fever in Honduras, 2010. Trop Biomed. 2012;29: 499–507. pmid:23202593
- 79. Zhang Y, Wang T, Liu K, Xia Y, Lu Y, Jing Q, et al. Developing a Time Series Predictive Model for Dengue in Zhongshan, China Based on Weather and Guangzhou Dengue Surveillance Data. PLoS Negl Trop Dis. 2016;10: e0004473. pmid:26894570
- 80. Zhao N, Charland K, Carabali M, Nsoesie EO, Maheu-Giroux M, Rees E, et al. Machine learning and dengue forecasting: Comparing random forests and artificial neural networks for predicting dengue burden at national and sub-national scales in Colombia. PLoS Negl Trop Dis. 2020;14: e0008056. pmid:32970674
- 81. Zhu B, Wang L, Wang H, Cao Z, Zha L, Li Z. Prediction model for dengue fever based on interactive effects between multiple meteorological factors in Guangdong, China (2008–2016). PLoS One. 2019;14: e0225811. pmid:31815950
- 82. Drake JM. Fundamental limits to the precision of early warning systems for epidemics of infectious diseases. PLoS Med. 2005;2: e144. pmid:15916474
- 83. Scott TW, Morrison AC, Lorenz LH, Clark GG, Strickman D, Kittayapong P. Longitudinal studies of Aedes aegypti (Diptera: Culicidae) in Thailand and Puerto Rico: population dynamics. J Med Entomol. 2000;14: 77–88. pmid:15218910
- 84. Johansson MA, Reich NG, Hota A, Brownstein JS, Santillana M. Evaluating the performance of infectious disease forecasts: A comparison of climate-driven and seasonal dengue forecasts for Mexico. Sci Rep. 2016;6: 33707. pmid:27665707
- 85. Riley P, Ben-Nun M, Turtle J, Bacon D, Riley S. SARIMA Forecasts of Dengue Incidence in Brazil, Mexico, Singapore, Sri Lanka, and Thailand: Model Performance and the Significance of Reporting Delays. medRxiv: 20141093 [Preprint]. 2021 [cited 2022 July 3]. Available from: https://www.medrxiv.org/content/10.1101/2020.06.26.20141093v1
- 86. Batista EDA, Araújo WC, Lira RV, Batista LIA. Predicting dengue cases through Machine Learning and Deep Learning: a systematic review. Res Soc Dev. 2021;10: e33101119347.
- 87. Zinszer K, Verma AD, Charland K, Brewer TF, Brownstein JS, Sun Z, et al. A scoping review of malaria forecasting: past work and future directions. BMJ Open. 2012;2: e001992. pmid:23180505
- 88. Bullock J, Luccioni A, Pham KH, Lam CSN, Luengo-Oroz M. Mapping the Landscape of Artificial Intelligence Applications against COVID-19. J Artif Intell Res. 2020;69: 807–845.
- 89. Lessler J, Cummings DA. Mechanistic Models of Infectious Disease and Their Impact on Public Health. Am J Epidemiol. 2016 Mar 1;183(5): 415–22. pmid:26893297
- 90. Baker RE, Peña J, Jayaratnam J, Jérusalem A. Mechanistic models versus machine learning, a fight worth fighting for the biological community? Biology Letters. 2018;14(5): 20170660. pmid:29769297
- 91. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21: 128–138. pmid:20010215
- 92. Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17: 230. pmid:31842878
- 93. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35: 1925–1931. pmid:24898551
- 94. Johansson MA, Apfeldorf KM, Dobson S, Devita J, Buczak AL, Baugher B, et al. An open challenge to advance probabilistic forecasting for dengue epidemics. Proceedings of the National Academy of Sciences of the United States of America. 2019 Nov 26;116(48): 24268–74. pmid:31712420
- 95. Ivanescu AE, Li P, George B, Brown AW, Keith SW, Raju D, et al. The importance of prediction model validation and assessment in obesity and nutrition research. Int J Obes (Lond). 2016 Jun;40(6):887–94. pmid:26449421
- 96. Steyerberg EW, Lingsma HF. Validating prediction models. BMJ. 2008;336: 789.
- 97. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. Journal of clinical epidemiology. 2001 Aug 1;54(8): 774–81. pmid:11470385
- 98. Efron B, Tibshirani R. Improvements on cross-validation: the 632+ bootstrap method. Journal of the American Statistical Association. 1997 Jun 1;92(438): 548–60.
- 99. Cavenaugh JS. Bootstrap Cross-Validation Improves Model Selection in Pharmacometrics. Stat Biopharm Res. 2020:10; 1–36.
- 100. Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012.98: 691–698. pmid:22397946