Early warning systems (EWSs) for chikungunya, dengue, malaria, yellow fever, and Zika outbreaks: What is the evidence? A scoping review

Background Early warning systems (EWSs) are of increasing importance in the context of outbreak-prone diseases such as chikungunya, dengue, malaria, yellow fever, and Zika. A scoping review has been undertaken for all 5 diseases to summarize existing evidence of EWS tools in terms of their structural and statistical designs, feasibility of integration and implementation into national surveillance programs, and the users’ perspective of their applications. Methods Data were extracted from Cochrane Database of Systematic Reviews (CDSR), Google Scholar, Latin American and Caribbean Health Sciences Literature (LILACS), PubMed, Web of Science, and WHO Library Database (WHOLIS) databases until August 2019. Included were studies reporting on (a) experiences with existing EWS, including implemented tools; and (b) the development or implementation of EWS in a particular setting. No restrictions were applied regarding year of publication, language or geographical area. Findings Through the first screening, 11,710 documents for dengue, 2,757 for Zika, 2,706 for chikungunya, 24,611 for malaria, and 4,963 for yellow fever were identified. After applying the selection criteria, a total of 37 studies were included in this review. Key findings were the following: (1) a large number of studies showed the quality performance of their prediction models but except for dengue outbreaks, only few presented statistical prediction validity of EWS; (2) while entomological, epidemiological, and social media alarm indicators are potentially useful for outbreak warning, almost all studies focus primarily or exclusively on meteorological indicators, which tends to limit the prediction capacity; (3) no assessment of the integration of the EWS into a routine surveillance system could be found, and only few studies addressed the users’ perspective of the tool; (4) almost all EWS tools require highly skilled users with advanced statistics; and (5) spatial prediction remains a limitation with no tool currently able to map high transmission areas at small spatial level. Conclusions In view of the escalating infectious diseases as global threats, gaps and challenges are significantly present within the EWS applications. While some advanced EWS showed high prediction abilities, the scarcity of tool assessments in terms of integration into existing national surveillance systems as well as of the feasibility of transforming model outputs into local vector control or action plans tends to limit in most cases the support of countries in controlling disease outbreaks.


Introduction
Epidemics of aAU : PleasenotethatasperPLOSstyle; italicsshouldnotbeusedforemphasis: rboviral diseases transmitted by Aedes mosquitoes-such as chikungunya, dengue, yellow fever, and Zika-have emerged or reemerged over the past 5 decades, overburdening already stretched health systems. Approximately 2.5 billion people live in risk areas of Aedes-borne diseases, and, collectively, an estimated 390 million infections occur annually in about 100 countries [1][2][3][4]. Unfortunately, risk forecasts indicate that these epidemics will intensify and reach new geographical areas throughout the 21st century [5]. This fact is largely driven by a combination of urbanization, poor living conditions, international travel and trade, changes in mosquito distribution and abundance, climate variability, and climate change [6][7][8].
Also, malaria, an Anopheles mosquito-transmitted disease in tropical and subtropical areas, has often shown its potential for large outbreaks. This may happen in the highly endemic areas of sub-Saharan Africa but also in areas of malaria elimination in Asia and Latin America where the fading herd immunity makes people more susceptible for infections and allows local outbreaks to occur [9,10].
Vector-borne diseases can mainly be controlled through effective vector control. Even after the advent of a vaccine, as available for yellow fever, vector management will continue to be important. Only for malaria, a number of therapeutic treatment options are available [11]. Due to the high vector capacity of Aedes mosquitoes, the required level of vector control interventions to prevent transmission is usually not being achieved, and outbreaks have become increasingly frequent [11]. Data are usually provided by the routine disease surveillance systems occasionally complemented by entomological data. The information often arrives too late or is of low quality or in the wrong format [12,13]. As a result, outbreaks are usually detected too late when infections have already spread.
Forecasting disease outbreaks is highly desirable to give time to the vector control services for preparing the response. For outbreak early warning systems (EWSs), countries need standard operational procedures (SOPs) to identify consistently through alarm signals an increased outbreak risk in time and space triggering an early response. Several EWSs have been developed for our target diseases, and most of these have commonalities in their structural design, functions, and analytical approach [14][15][16][17]. However, studies addressing the effectiveness of space and time prediction and how the EWS may improve coordination among the operators at national and district level are scarce. This scoping review summarizes and discusses the evidence of different EWSs, their performance, and abilities to predict outbreaks of our target diseases, providing an overview of the state of the art and recommendations for the future development of a practical outbreak prediction tool. This review will recapitulate (1) EWS prediction validity in terms of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of outbreak warning; (2) if studies addressed the operational feasibility of the EWS in terms of user-friendliness, cost, capacity of being easily implemented within countries' national control programs, coordination, and the requirements for increasing its efficiency; and (3) the scopes of study designs for assessing the EWS effectiveness in triggering adequate response activities.

Methods
MAU : PleasecheckwhethertheeditstothesentenceMethodsusedwerepredefinedinaprotocol:::arecorrect; a ethods used were predefined in a protocol based on the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement [18]. The inclusion criteria of articles were (1) primary research published in a peer-reviewed journal; (2) studies addressing any type of existing or developing prediction model; and (3) dealing with chikungunya, dengue, malaria, yellow fever, and Zika diseases or a combination of the diseases; presenting (a) experiences with existing EWS (stand-alone or integrated into the national surveillance systems); and (b) the development or implementation of EWS in a particular setting. EWS studies were excluded if merely investigating trends or correlations of particular alarm indicators. Studies were also excluded if they neither reported nor provided sufficient data to outline the type of mathematical model used or the temporal and spatial prediction of the tool. Further studies were excluded if they failed to demonstrate a developed or prototype of prediction model-i.e., merely presented and discussed candidate list of potential alarm indicators or studies with models constructed to elucidate transmission dynamics. No exclusion due to study design was made, except for conference abstracts, book chapters, or studies that are published by journals of a local institute.

Search strategy, databases, and search terms
Searches were conducted in English, assuming that most relevant studies are indexed in English, with title and abstract in English. No restrictions were applied for year of publication, geographical area, or language.
The terms "early warning," "forecasting," and "prediction" have been probed in a pilot search as Medical Subject Headings (MeSH) terms and as free-text terms. The terms "forecasting" and "prediction" have been discarded thereafter for the final search, as they did not show EWS in a public health sense predicting epidemics, but mostly clinical related studies predicting outcome of clinical disease.
The terms "chikungunya," "dengue," "malaria," "yellow fever," and "Zika" were used as MeSH terms and as free-text terms, depending on the function of the database. The searches were run for the combination of "early warning" AND the disease.
PubMed has been searched using MeSH terms for the diseases and free-text term for "early warning." For Google Scholar, LILACS, and WHOLIS, free-text terms for both categories have been use. For CIDG Specialised Register and CENTRAL, the disease and "early warning" were searched as MeSH terms.

Quality assessment and assurance
Searches were performed by the team of authors and double-checked until consistent results were found. One author (LH or EG) screened all hits for their relevance. As for Google Scholar, a large number of hits were derived (>10,000 for dengue), hits were sorted by relevance, and the first 120 hits were established as a suitable number, with no additional relevant hits encountered toward the end of the search.
Two data extractors (LH and SRR for chikungunya, dengue, yellow fever, and Zika; LH and EG for malaria) independently assessed the abstracts and full text of the preselected references for potential eligibility by applying all inclusion and all exclusion criteria.
After full agreement, data were entered into a predefined Excel data extraction form by author, title, journal, publication date, and study design as well as different outcome parameters as they emerged from the studies, following the Consolidated Standards of Reporting Trials (CONSORT) 2010 checklist [19]. Extracted parameters are listed below in detail.
Risk of bias of the included studies has not been assessed, as no systematic but a scoping review was performed, but the quality was assured by strict application of inclusion and exclusion criteria, considering several quality aspects as described above.

Data extraction and analysis
The above described data extraction form was developed based on the study objectives and relevant predefined quantitative and qualitative outcome variables. Further categories were also considered including (1) studies addressing surveillance as structured form of information on disease cases, meteorological, epidemiological, and entomological information (indicatorbased surveillance, IBS); and 2) studies on surveillance of unofficial or unstructured information (such as social media and community reports) or of social events (event-based surveillance, EBS). EWS using epidemiological, entomological, or climate data for outbreak prediction were termed "alarm-informed EWS." EWS based on increased case numbers in comparison with historical data were termed "case-informed EWS." Further characteristics of the studies taken into consideration were (a) methodological (case study, time series, and randomized control trials (RCT), etc.); (b) the models/tools that have been investigated (and their study scope); and (c) the disease(s) being investigated. Detailed information was extracted on specific outcomes as sensitivity, specificity, PPV/NPV outputs of the models, local coverage of the tool (central, district/province, or subdistrict level), integration (into national surveillance program), and the level of implementation (central, district, or subdistrict levels). FAU : PleaseconsiderrephrasingthesentenceFurthermore; theenduserofthetool:::forclarity: urthermore, the end user of the tool (e.g., public health officers, institutes, researchers, and others), the user-friendliness of the EWS tool, its temporal and spatial risk prediction and its integration feasibility including, resource needed to implement, prediction duration or timelag and, the duration of dataset were all captured in this review. The data extraction form has been further synthesized to evidence tables (Tables 1-3).

Evaluation criteria
Prediction models are typically evaluated according to their statistical and operational performance. Statistically, the sensitivity, specificity, PPV, or NPV measures are used to quantify the quality of prediction performance of the model. Based on common recommendations of appropriate cutoff for model performance, we attributed a statistical measure of >50% to high performance prediction models [20]. The spatial prediction is crucial in the context of vector control and action plan; spatial prediction alone is less useful and must be combined with the temporal model for effective public health response. Accordingly, we defined high temporal prediction by a prediction window of 1 to 12 weeks (allowing sufficient time between prediction and action taken). WAU : PleasecheckwhethertheeditstothesentenceWedefinedhighspatialpre e defined high spatial prediction by the prediction of hotspots within geographically defined areas of relatively small spatial scale such as villages, neighborhood, or household levels in contrast to a resolution within district scale.

Study characteristics
A total of 46,477 hits were obtained, with 11,710 studies focused on dengue, 2,757 on Zika, 2,706 on chikungunya, 24,611 on malaria, and 4,693 on yellow fever. After review by title and abstract as well as the application of a cutoff after 120 Google Scholar hits, sorted by relevance, 279 dengue, 22 chikungunya, 39 Zika, 89 malaria, and 13 yellow fever articles were further assessed. Based on the inclusion criteria and removing of duplicates, 151 articles were assessed as full texts, but exclusion criteria finally retained 37 articles including 28 for dengue, 2 for Zika, and 7 for malaria. Fig 1 summarizes the search process.
All of the 37 studies were published between 2001 and 2019, of which 33 between 2011 and 2018. Of the 28 dengue studies, 3 [12,21,22] were conducted across different countries (multicounty) and 16 in Asia (5 in China, 4 in Indonesia, 3 in Singapore, and 1 each in Cambodia, Sri Lanka, Taiwan, and Thailand). Another 8 studies were conducted in the Americas (4 in Brazil, 2 in Colombia, 1 in Cuba, and 1 in French Guiana) and 1 [23] in Europe. The 2 Zika studies were performed in the Americas [24,25].

Main study findings, including prediction quality (sensitivity + PPV)
Temporal and spatial risk prediction Prediction time lag

Study or model limitations Conclusions
M7 [46] Within early detection window (the past 6 weeks) and an early warning forecast window (the upcoming 4 weeks), the mean observed or forecasted incidence was classified as being above the mean outbreak threshold, between the mean threshold and the mean expected incidence, or below the mean expected incidence High temporal and spatial prediction  Of the 28 studies addressing EWS for dengue outbreaks, only 4 studies presented the user perspective on implementing EWS within national programs [12,21,26,27], and 24 studies showed EWS under development (i.e., model exercise); see Table 1. Only 2 dengue studies used a prospective study design [28,29], and one applied both prospective and retrospective approaches [30]; the other 23 were retrospective analyses of surveillance data (1 of those specified as cohort study [21], 1 with a cross-sectional design [31], and 1 as a time series analysis [30]), and 2 studies were exclusively relying on mathematical simulation models [32,33]. The 2 Zika studies reported on exercises of testing EWS [24,25]. The 7 EWS malaria studies included only one existing EWS [34], while 6 other studies reported on potential EWSs or under development. The studies used a large variety of statistical methods as summarized in Table 1.

EWS features per disease
Outbreak indicators of EWSs varied considerably across the selected studies, and, hence, we stratified the findings of the relevant outcomes by the 3 diseases (no studies were retrieved for chikungunya and yellow fewer) that were ultimately retained from the search. In line with the respective case definition used in the diverse reporting systems across national surveillance programs and between diseases, the outbreak indicators were grouped as hospitalized, laboratory-confirmed (including "reported," "confirmed," or "imported from neighboring districts"), and suspected cases (including "probable" cases as they were often not distinguished from "suspected") as well as case numbers resulting from mathematical modeling (no real cases but estimates based on other covariates). The alarm indicators were categorized as entomological (including an index of mosquito survival), meteorological (including the El Niño Southern Oscillation [ENSO] index and others), epidemiological (including sociodemographic information), and social media (such as "Google Trends").

Study results
Dengue EWS characteristics and features. All 28 dengue studies relied on IBS, of which 24 studies were alarm informed, i.e., of outbreak predictive nature-using at least 1 meteorological, entomological or epidemiological indicator-and 4 studies [28,31,32,35] were case-informed EWS, i.e., of early outbreak detection-relying solely on previous trends of cases for outbreaks. Almost all studies demanded routine access to data as well as advanced statistical and analytical skills, but 5 studies [12,21,26,28,31] reported prediction tools that are adapted for less skilled users. The outbreak indicators used were hospitalized cases (2 studies) [12,21], laboratory-confirmed cases (23 studies), or suspected cases (3 studies) [12,28,31]. One study [30] used mathematically simulated cases. All cases were reported by the surveillance system on either weekly or monthly basis, and, occasionally, data were obtained directly from the Ministry of Health (MOH).
With reference to the alarm indicators, 13 studies reported the use of data from local/ regional meteorological stations, whereas 3 studies referred to international sources such as the "World Meteorological Organization" [39], the "Global Historical Climate Network" [40], and the "National Oceanic and Atmospheric Administration" [13]. Furthermore, one study [23] used data from the regional Environment and Epidemiology Network ( Table 2). Summary findings and reported limitations. This section presents key findings extracted from EWS studies (see Table 3) following the evaluation criteria presented in the methodology and their reported limitations. Most of the studies presented sensitivity, specificity, and NPV/PPV as measurements of the validity of the corresponding models. A range between 40% and 100% for sensitivity and 12.5% to 100% for the PPV have been reported, depending primarily on the type of model and the alarm indicators used. Generally, meteorological indicators were the best performing predictors, but a combination of meteorological and other indicators outperformed the single indicator prediction-i.e., the prediction model improved when other nonclimatic alarm indicators were included to the model containing climate indicators. While several EWSs demonstrated reasonable predictive abilities, 5 models showed outstanding performance using the following alarm indicators: (1) the dynamic risk maps absolute shrinkage and selection operator (LASSO) processing multiple meteorological information (with 1 study adding on epidemiological and entomological alarm indicators) [26]; (2) autoregression integrated moving average (ARIMA) using meteorological information and Google Trends data [40]; (3) Shewhart moving average regression model (SMAR) maintaining a combination of meteorological, epidemiological, and entomological alarm indicators [12,21]; (4) seasonal autoregressive integrated moving average (SARIMA) [35]; and (5) the stochastic Bayesian maximum entropy (BME) model using a mix of meteorological and entomological alarm indicators [37]. A total of 20 EWS studies demonstrated high temporal prediction ability, but only 3 [27,28,38] showed high spatial prediction ability. Several studies reported dataor model-related limitations, with 10 studies indicating issues related to the outbreak information (such as poor or unreliable case reporting, small historical records, bias created by interventions or other confounders, and lack of geotagged weekly records). Three studies [22,41,42] referred to the lack of climate information and biased, inaccessible, or poor resolution data. Other limitations addressed the methodological approach with 3 studies [26,31,35] declaring method-related issues such as that methods (like LASSO) are not amenable to direct interpretation, requiring advanced statistical skills, or have a limited prediction robustness.
Concerning the EWS implementation, the following information has been provided: All studies described the EWS prediction coverage for national and district levels, but only 2 studies [27,38] showed predictions for the subdistrict level. Furthermore, half of the studies (14) featured the possibility of successfully implementing EWS, at national or district levels, but only 5 studies [21,26,28,36,43] showed the feasibility to be integrated into existing national surveillance programs. The users' perception of EWS was insufficiently assessed in several studies, but 18 indicated the possibility to be used by health managers at the MOH and district levels; 5 studies [32,39,40,44,45] reported the use of EWS being limited to research institutes. The user-friendliness of EWS was reported in only 2 studies [21,31], showing a general satisfaction by users.

Zika EWS characteristics and features.
Studies on Zika were primarily triggered by the 2015 to 2016 Zika virus (ZIKV) epidemic. Thus, only 2 studies were retrieved, which discussed the EWS for Zika outbreaks that matched our selection criteria. Both studies, of which one was an IBS-based and the second was an EBS-based study, applied new algorithms using alarminformed EWS. Both studies described the need for routinely accessing alarm information for running the tools. One study used both confirmed and suspected cases for early outbreak warning [24], while the other study referred only to suspected cases that reflects the diagnostic and reporting complexity associated with this disease. One study employed social media information (Google Trends search) [24], and another used meteorological information as predictors [25]; however, both studies used the national surveillance programs as sources for processing their data.
Summary findings and reported limitations. There was no validity testing presented by both studies, such as sensitivity or specificity. In one study [24], ARIMA was reported as a model to improve the prediction of Zika outbreaks when integrated with Google Trends data. The other study [25] showed the usefulness of humidity, rainfall, and maximum air temperature as alarm indicators for outbreak prediction. As concluded by Teng and colleagues, using dynamic data from Google Trends as indicators can significantly advance the prediction model mainly when more sophisticated statistical models like ARIMA are used [24]. Nevertheless, meteorological alarm indicators have generated high temporal predictions with up to 20 weeks ahead of an outbreak, but both studies failed the spatial prediction. No major limitations were reported, but one study showed a model-related limitation due to the complexity in handling temporal functions in relation to the spatial functions of the model [25].

Malaria EWS characteristics and features.
Out of the 7 EWS malaria studies, 6 used IBS and 1 [46] used an EBS-processing at least 1 meteorological, entomological, or epidemiological indicator. All studies demanded routine access to data, and all but 2 required advanced statistical and analytical skills for practicing or integrating the tool. The outbreak indicators used were hospitalized (1 study) [47] or laboratory-confirmed cases (3 studies) [34,48,49], while 2 studies [46,50] used clinically confirmed cases (hospitalized). Data collection was done on a weekly or monthly basis, and studies demonstrated a spectrum of indicators: All used meteorological and epidemiological information, some entomological (3 studies) [47,48,51], and one study demonstrated the cartographic indicator. None of the studies assessed the user experience with EWS, but one [34] described how the involvement of stakeholders improved the support and institutionalization of the tool; 3 studies [34,46,48] stated that the tool integration into existing epidemiological and climatic data hub created a better environment for further development and use of the tool. All studies used meteorological information as predictors. Surveillance data were obtained from local, regional, or national databases, and meteorological information was obtained from local stations or satellites.
Summary findings and reported limitations. Sensitivity and specificity were reported to be high in one study [48], mentioned to be high but not measured in 2 studies [47,49] and varied according to mosquito survival probability and temperature in a fourth study [51]. Several mathematical models were used to create predictions, with the additive and multiplicative models being the ones with high sensitivity and specificity. Several meteorological indicators were used as predictors of which temperature and rainfall were the most frequently used indicators. In general, environmental factors and climate variability correlated with malaria incidence, but no outbreak prediction model has been developed that includes different types of epidemiological, environmental, and meteorological alarm indicators. The alarm indicators have managed to generate a high temporal prediction with up to 6 months ahead of outbreak, particularly if unusual weather conditions like ENSO were concerned. Several limitations were noted: Some models were tested for specific settings (e.g., highlands in Africa), others include only remotely sensed environmental indicators-remote sensing is a useful approach in the context of climate predictions, including flood and earthquake disaster prediction, but tends to lack sufficient evidence for early warning of infectious diseases in general. Thus, when used solely in an early warning, they are viewed as a limitation in the model. Confounding factors that affect malaria risk such as land use/land cover, population mobility, local hydrology, socioeconomic factors, and public health interventions were not captured.

Discussion
When searching for high-level evidence in the area under investigation, we found 2 systematic reviews [52,53] reporting on predictive modeling tools particularly for dengue. However, we found no high-level evidence on tools applications outside their mathematical modeling or on potential alarm indicators to be used, although these were the prime focus of both reviews. The study by Racloz and colleagues has successfully summarized the benefits of combining various epidemiological tools focusing on the ability to incorporate climatic, environmental, epidemiological, and socioeconomic factors to create an EWS and has outlined optimal prediction models [52]. The second study by Louis and colleagues addressed the risk mapping-related issues and human mobility as promising alarm indicators and maintained a thorough review of their limitations [53]. Besides the structural and statistical features of the tool, our review has additionally addressed essential operational aspects (prediction quality, the implementation, integration, and user perspectives of EWS) and public health implications of EWS, independently for each retrieved disease, keeping in mind that for any outbreak prediction, a reliable surveillance system is essential. For instance, misclassifications, data arriving late, missing values, and human errors during the data entry may compromise the EWS.

Dengue
All 28 identified dengue studies maintained an IBS approach, and these demanded routine and timely access to surveillance information potentially impacting the effectiveness of early warning tools. Published studies show that consistent and frequent case reporting has stronger predictive capacity, which usually tends to be delayed by monthly reporting schedules [54]. Our results show that weekly reporting of surveillance data correlate with increased sensitivity and PPV; however, only one-third of all dengue studies presented weekly data, negatively impacting the effectiveness and potentially cost-effectiveness of the early warning process [12,21,[26][27][28]31,41,42,44].
A total of 27 dengue studies demonstrated outbreak forecasting, using multiple (9/27) and single alarm (18/27) indicators mostly meteorological ones. However, access to meteorological information on a regular time basis is challenging in several settings [55]. Three studies included regional or international data resources, which, however, demand highly skilled users and advanced digital systems for international data processing [23,33,56].
Furthermore, the mathematical model was identified to affect the predictive ability of EWS due to (1) the choice of the analytical approach employed; and (2) the range of lag times between independent variables and epidemic dengue transmission as demonstrated by Racloz and colleagues and others [52]. Among the retrieved dengue studies, almost all reported estimates on sensitivity, specificity, NPV, or PPV but their prediction performances varied substantially depending on the statistical model used and the data quality. Out of the 19 studies with reported high temporal prediction quality (Table 3), the Bayesian algorithms and generalized linear models were the most prevalent [29,31,36,39,45,57,58]. Additionally, 4 studies used LASSO [22,26,27,45], 3 with ARIMA and time series analysis [28,40,56] independently, and 2 used the Shewhart/endemic channel method [12,21]. Only 4 studies showed evidence of adaptation to less skilled users, which can be significant for public health use [12,21,26,27].
The interaction between changing climate and increasing human mobility as drivers for emerging diseases warrants novel frameworks for assessing the linkage between disease transmission, climate change, and public health intervention in order to reach effective EWS. The use of data mining techniques, such as social media or travel information, in combination with surveillance data has emerged as an alternative source of real-time high-resolution geospatial data on a large scale [59]. However, our review showed minimal evidence of studies exercising such potential alarm indicators, which limits their contribution to outbreak preparedness and response planning [59,60]. The LASSO model is one typical example of such a tool that can potentially contribute to this concept. However, LASSO or similar model concepts demand high-quality big data. As these resources are (a) typically scarce in many countries; and (b) having a tendency to complicate the interpretation of the prediction outputs [26,45], the application in data-constrained settings and by unskilled users is likely to be limited. While limited data accessibility and poor quality have been described by several studies [32,35,41,42,56], published reports highlight the benefits of combining temporal data for analysis of the temporal kinetics and spatial data for the identification of high risk areas [61]. Only 3 retrieved studies [27,28,38] demonstrated high spatial prediction abilities, all attributed to settings with advanced data and surveillance systems. Two of those showed high temporal and spatial prediction, allowing for identification of population at risk at smaller spatial units, which can significantly contribute to targeted vector control [27,38].

Zika
Since the Zika emergence in 2015, the number of PubMed references for ZIKV has risen from 181 to 516 in 2019, with a high proportion focusing on the consequences of ZIKV infection during pregnancy [62]. Only 2 studies have been identified in this review, with a focus on EWS for Zika outbreaks in the Americas. One study had explored the use of Google Trends data as a predictor [24], and the other used a set of meteorological information for generating outbreak predictions [25]. The use of "suspected cases" as outcome variables in both retrieved studies could possibly be explained by the complexity of the Zika diagnosis and the large number of mild cases [62]. For dengue, the prediction accuracy of EWS was superior when using hospitalized or laboratory-confirmed cases compared to suspected cases [12,21]. However, no measurements of sensitivity or PPV were used in both Zika studies, albeit both ARIMA and generalized linear models predicted up to 20 weeks ahead of the outbreak. Limitations of zika outbreak predictions are not discussed in the respective papers, but are likely to be similar as those shown for dengue predictions.

Malaria
Since the inception of Roll Back Malaria (RBM) in 1998, it was clear that the early detection, containment, and prevention of malaria epidemics were key elements of the Global Malaria Control Strategy and Malaria Early Warning Systems (MEWS) [63,64]. Some countries have developed epidemic risk monitoring using simple transmission risk indicators such as excess rainfall [65], but only Kenya has published the development and implementation of MEWS [34]. Since then, several studies and initiatives for EWS have been developed; however, the literature to address the tools' scope of its predictability, implementability, and users' perspectives are significantly scarce. The 7 studies identified in this review used different mathematical models and different combinations of indicators. Most of the mathematical models found in this review applied time series including additive and multiplicative models [34,[46][47][48], but lacked vulnerability indicators such as low immunity or drug resistance, which might be prevalent in these study settings. Meteorological indicators ranged from common indicators like rainfall and humidity to land surface temperature or vegetation indices obtained through more sophisticated satellite systems that are usually provided by projects or partnerships, of which their sustainability was never assessed. Like the case with dengue, this wide range of indicators is augmented by additional data sources sought from regional or international entities. The development of implementable and user-friendly malaria EWSs, as shown in the reviews by Githeko and colleagues and Merkord and colleagues [34,46], is a key factor for better disease preparedness and timely response activities.

Public health implications for EWS applications
The EWS tool is primarily aimed at supporting district health managers and national health planners to mitigate or prevent disease outbreaks, ideally using tools that are integrated in the national surveillance programs [66,67]. To further ensure effective functions, EWS should conceptually be perceived as an information system designed to support the decision-making of national-and local-level institutions but also enable vulnerable groups in the society to take actions to mitigate the impacts of an impending risk. As apparent from this review, users of current EWSs were mostly from the central (MOH) levels, with only few tools facilitating district-level applications. The integration of EWS into existing national surveillance program was marginal with only 5 studies [12,21,26,27,34], out of 37, demonstrating their experience of integration (Table 1). Furthermore, the majority of studies have not assessed the feasibility of implementing EWS into national programs, and a few studies have declared the need for highskilled users and resources-2 limiting factors that are unlikely to exist at small spatial levels in settings where disease outbreaks are public health burdens. Nevertheless, there is an observed trend toward applications of more advanced statistical models with higher predictive abilities that can further advance the prediction and control of disease outbreaks. Generally, early warning and response system that are capable of demonstrating evidence of prospective predictive ability and allows technical and practical adaptations of local public health responses while augmenting communications channels between users at central and district levels are tools that are more likely to be implemented into national surveillance programs. Advancing into frameworks that can facilitate at a low-cost IT maintenance and adapted to unskilled users are features of tools that are plausibly integrable into existing national systems.
As shown in the method section, the IBS and EBS are 2 main channels of information for a functioning EWS. Almost all studies reviewed in this paper maintained an IBS type of application to support EWS, with the exception of 1 Zika-and 1 malaria-related study using the EBS approach [24,46]. The combined use of both could potentially include other sources of information, such as sources from outside the health sector, which is the prime concept of the EBS. With the majority being of IBS-based EWS, the applications of the forecasting tools tend to be less efficient and deviate from the epidemic intelligence concept-the systematic collection, analysis, and communication of any information to detect, verify, assess, and investigate events and health risks with an early warning objective, as opposed to monitoring of disease trends or burdens [68]-which ideally combines both IBS and EBS for more robust outbreak detection.
Meteorological indicators are key predictors, but they are often inaccessible on a timely basis for health services managing the EWS. Benefiting from multiple assessments of users' perspectives while defining the tool end users, countries like Mexico and Brazil, for instance, have managed to recognize the essence of the availability of local meteorological stations and have therefore organized an improved access to meteorological information [21,29,36,58,69].
Our literature review has identified very limited information for chikungunya and yellow fever outbreaks prediction, none of them fulfilling the inclusion criteria. Studies of EWS for yellow fever outbreaks are limited probably due to the existence of vaccines for this disease. However, chikungunya has now expanded from Africa, Latin America, and Asia to the European region [70,71]. It is quite worrying that studies on EWS tools for diseases like chikungunya are scarce. Although many EWS rely on meteorological information, only 3 studies [28,29,47] performed a prospective type analysis of early warning performance, with no rigorous study could be found.

Limitations of our study
The authors decided to include EWS on malaria to broaden the scope and eventually learn mutually from applications of different fields of disease control (Aedes borne versus Anopheles borne), but also by including yellow fever (vaccine preventable) and other vector-borne disease (mainly relying on vector control interventions). By virtue of their variability in terminology and definitions as well as difficulties in synthesis, the format of a scoping review was applied as a form of review design. Nevertheless, this scoping review follows the PRISMA criteria for conducting a systematic review without performing rigorous critical appraisal of included studies due to time constraints. However, we think that by including only peer-reviewed papers focusing on implemented EWSs or those under development provides a certain guarantee for highquality papers, which will be an added value to the existing literature. The search strategy was limited to the target diseases not including search terms as "arboviruses" or "febrile illnesses" or "priority diseases" or others, which might be potentially relevant to the review, considering the broadness of the scoping review, Furthermore, stakeholders have been occasionally involved during the 14-month duration of this scoping process-during an Andean regional meeting in Bogota and in Colombia with an expert panel in charge of EWSs. Another limitation might be that we may have missed a few relevant studies; however, by including the Google Scholar search engine, we could overcome this issue, assuming that relevant papers could be identified, which were not indexed in other databases like PubMed. Due to the language barrier, we were unable to include publications in Chinese language, but several English papers from China were included, which could adequately represent the rich experience from the Chinese context.

Conclusions
This scoping review demonstrated gaps and challenges related to the structural, statistical, and operational designs of EWS, and these varied per disease and their corresponding settings. The country surveillance system is an integral part in the overall early warning process where the lack of accessibility to timely and quality data is crucial for establishing a reasonable EWS. Nevertheless, a substantial number of studies (except for dengue) failed to demonstrate any predictive power mainly that predictions based on complicated statistical models are difficult to carry out in low-and middle-income countries. This review has furthermore revealed a significant gap in effectively evaluating the role of EWS in the disease outbreak prediction and control given that the majority of EWS assessment studies have primarily been of retrospective designs. The lack of tool assessments regarding the implementation into existing routine surveillance as well as the feasibility of translating model outputs into local vector control and action plans will unlikely support the global health agenda for controlling disease outbreaks. Likewise, the missing user perspectives in the retrieved studies signals shows that most of the EWSs remain in the academic environment, and little effort has been spent on testing their effectiveness or cost-effectiveness in reducing disease outbreaks. Collectively, findings from this review claim the need for more pragmatic and context-adapted EWS tools, which address the user perspectives and its effectiveness in predicting outbreaks in local settings and trigger response activities.

Key learning points
• Only minimal studies have addressed the early warning system (EWS) users' perspectives with significant lack of implementation research assessments of EWS for chikungunya, dengue, malaria, yellow fever, and Zika outbreaks.
• While the majority of studies have focused on the development and applications of the temporal prediction of the EWS, the spatial analysis of the disease prediction is crucial for effective vector control and response but rarely discussed or assessed in the literature.
• The EWSs should be viewed as frameworks for improving the coordination of the overall disease outbreak control and response where full stakeholder involvement and assessment are warranted.