Dengue virus infection is the most common arthropod-borne disease of humans and its geographical range and infection rates are increasing. Health policy decisions require information about the disease burden, but surveillance systems usually underreport the total number of cases. These may be estimated by multiplying reported cases by an expansion factor (EF).
Methods and Findings
As a key step to estimate the economic and disease burden of dengue in Southeast Asia (SEA), we projected dengue cases from 2001 through 2010 using EFs. We conducted a systematic literature review (1995–2011) and identified 11 published articles reporting original, empirically derived EFs or the necessary data, and 11 additional relevant studies. To estimate EFs for total cases in countries where no empirical studies were available, we extrapolated data based on the statistically significant inverse relationship between an index of a country's health system quality and its observed reporting rate. We compiled an average 386,000 dengue episodes reported annually to surveillance systems in the region, and projected about 2.92 million dengue episodes. We conducted a probabilistic sensitivity analysis, simultaneously varying the most important parameters in 20,000 Monte Carlo simulations, and derived 95% certainty level of 2.73–3.38 million dengue episodes. We estimated an overall EF in SEA of 7.6 (95% certainty level: 7.0–8.8) dengue cases for every case reported, with an EF range of 3.8 for Malaysia to 19.0 in East Timor.
Studies that make no adjustment for underreporting would seriously understate the burden and cost of dengue in SEA and elsewhere. As the sites of the empirical studies we identified were not randomly chosen, the exact extent of underreporting remains uncertain. Nevertheless, the results reported here, based on a systematic analysis of the available literature, show general consistency and provide a reasonable empirical basis to adjust for underreporting.
Dengue is the most common disease transmitted by a mosquito, with about 100–200 million infections occurring each year in more than 100 tropical and subtropical countries. Policy-makers require accurate information about the number of symptomatic dengue episodes to make informed decisions concerning dengue control strategies. But dengue is usually underreported by national surveillance systems. Through a systematic literature review and analysis of empirical research, we estimated the rate of underreporting and the average annual dengue episodes by treatment (hospitalized and ambulatory) in 2001–2010, for 12 countries in Southeast Asia. We found an average reporting rate of 13.2% of the total symptomatic dengue episodes in the region, leading to an expansion factor of 7.6 for converting reported cases into estimated actual cases. While we focused in Southeast Asia, analogous principles apply to other regions of the world, and other diseases reported through surveillance systems. Estimating the total episodes of dengue is a critical step in studying the economic and disease burden of dengue fever, and the cost-effectiveness evaluation of dengue control and prevention strategies.
Citation: Undurraga EA, Halasa YA, Shepard DS (2013) Use of Expansion Factors to Estimate the Burden of Dengue in Southeast Asia: A Systematic Analysis. PLoS Negl Trop Dis 7(2): e2056. https://doi.org/10.1371/journal.pntd.0002056
Editor: Scott B. Halstead, Pediatric Dengue Vaccine Initiative, United States of America
Received: July 16, 2012; Accepted: December 26, 2012; Published: February 21, 2013
Copyright: © 2013 Undurraga et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This article is part of a study to estimate the economic burden of dengue that is funded by a contract from Sanofi Pasteur (http://www.sanofipasteur.com/) to Brandeis University. Sanofi Pasteur had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Dengue has become a major public health problem in many tropical and subtropical regions, with an estimate of 100–200 million dengue infections occurring each year in more than 100 countries, resulting in approximately 20,000 deaths . It represents a significant economic burden to communities and health services in endemic countries, with a thirtyfold incidence increase in the last fifty years . A variety of factors have created the ideal conditions for the expansion and distribution of dengue mosquito vector and viruses, including high rates of population growth, inadequate water, sewer, and waste management systems, rise in global commerce and tourism, global warming, changes in public health policy, and the development of hyperendimicity in urban areas, although it is difficult to estimate the contributions of each factor separately –.
Southeast Asia has the world's largest incidence of dengue, with cycles of epidemics of increasing magnitude occurring every three to five years , . The WHO regions of Southeast Asia and Western Pacific represent most of the current global burden of dengue , and account for most deaths . All four dengue serotypes have been found in most countries of SEA  which means that one person can get up to four dengue infections. The risk of developing a more severe manifestation of dengue illness (e.g., dengue hemorrhagic fever (DHF) and dengue shock syndrome, DSS) increases with subsequent infections , , which may explain the higher frequency of DHF and DSS in hyperendemic countries, such as those in SEA.
Having accurate information about the human and economic burden of dengue is essential to inform policy makers and international donors, set health policy priorities, and make informed decisions about disease-control technologies and resource allocation. Estimating the total cases of symptomatic dengue infection is a critical step in calculating its economic and disease burden, but dengue incidence data are heterogeneous and incomplete. Most dengue episodes are identified using clinical diagnosis and existing epidemiological information, and, less frequently, clinical laboratory tests . Passive surveillance systems require that identified episodes of dengue fever be reported, usually to the Ministry of Health (MoH), but reported episodes do not consistently indicate severity of dengue. While passive surveillance systems, which rely on clinicians' reports incidental to their providing treatment, are appropriate to help detect dengue outbreaks promptly and examine long-term trends, they are not designed to estimate the real disease burden and usually underreport the total number of symptomatic dengue cases , –. Limitations of passive surveillance systems include variations in both the system design and in the characteristics of surveillance implementation. Variations in system design include dengue case definition, clinical or lab-confirmed diagnosis, inpatient and outpatient reporting, reporting from sentinel or all hospitals and health services, public and private sector reporting, specific ages or dengue severity reported, and surveillance budget. The characteristics of surveillance implementation include reliance on health care professionals and laboratory staff, use of electronic or paper forms, and people's health-seeking behavior. Other sources of variation of dengue reporting rates include whether data is collected on epidemic or non-epidemic years, unrecognized or mild symptoms, the overall quality of the health care system, and the specific area of the country where incidence of dengue is measured (e.g., rural or urban) , , , –.
While Singapore has recently implemented more sophisticated techniques to report cases and promptly respond to dengue epidemics , , and Malaysia has some enhanced capacity , most countries in SEA have only passive surveillance systems , making underreporting of dengue episodes a significant challenge in estimating disease burden. Based on a systematic literature review and using available data from surveillance systems, we estimated the average annual episodes of dengue by type of treatment in 12 countries from SEA (2001–2010). We defined SEA for this study as consisting of the following 12 countries: Bhutan, Brunei, Cambodia, East-Timor, Indonesia, Laos, Malaysia, Myanmar, Philippines, Singapore, Thailand, and Viet Nam. We wanted to focus on a contiguous area with hyperendemic dengue and a reasonable amount of available data. We included all countries in the Association of Southeast Asian Nations, as in a previous cost-effectiveness study , and added the bordering countries Bhutan and East-Timor due to their geographic proximity . We obtained the incidence of symptomatic dengue by adjusting the reported episodes using an “expansion factor” (EF), which corrected for underreporting. While total dengue episodes remain an area of considerable uncertainty, we used the best empirical data available to provide as accurate estimates as possible.
Data sources, parameters, and strategy
We combined various data sources to obtain our best estimate of the average annual episodes of dengue in 2001–2010. The data included a systematic literature review of articles (1995–2012) that reported empirically derived EFs or the necessary data to estimate them, and available surveillance data on reported dengue episodes. An EF is the number by which reported cases need to be multiplied to obtain the most accurate estimate of the true number of episodes The goal was to obtain total episodes by identifying the following parameters: reported episodes, EFs for total (EFT: total episodes/reported episodes), hospitalized (EFH: total hospitalized episodes/reported hospitalized episodes), and ambulatory cases (EFA: total ambulatory episodes/reported ambulatory episodes) where possible, and last, the outpatient to inpatient ratio (OP: IP) to allow extrapolation from hospitalized episodes to total episodes, where necessary. Because the availability of country-specific data and the quality of published studies varied substantially, we used the best data available and extrapolated parameters based on a measure of health system quality and on assumptions about reporting of hospitalized episodes and OP∶IP (discussed below).
We obtained the reported cases of dengue from various sources, including surveillance data from country-specific MoH or department of statistics – and WHO regional offices , , . Cambodia was the only country considered in this study that only reported dengue episodes affecting patients of less than 15 years old , , . Because dengue is an infectious disease, the number of cases varied considerably among years. To generate more stable estimates of the total projected cases of dengue, we considered the average reported cases in the last decade of available data (2001–2010).
Estimates of expansion factors and total dengue episodes in Southeast Asia
An EF can be calculated as the analyst's best estimate of the total number of dengue episodes in a specified population divided by the episodes reported (whether or not they actually were laboratory-confirmed dengue). A recent study of the economic impact of dengue in the Americas identified five studies that permitted estimating EFs for the reported episodes of dengue . The EFs ranged from 1.6 inpatient dengue episodes reported for each hospitalized episode in Brazil (1996–2002)  to 28 episodes of dengue for each clinically diagnosed episode in Nicaragua (2005–2006) .
While there are considerable regional differences in the epidemiology of dengue between SEA and the Americas, in both regions dengue consists of the same four virus serotypes, is officially notifiable, and is considerably underreported . The attack rates of DHF and DSS in SEA are approximately 18 times that of the Americas, with infants and children most affected . Some authors have used EFs obtained from studies in the Americas to estimate the burden of disease in Asian countries –. Given the differences in epidemiology and surveillance systems, we think it is more appropriate to rely on studies from the same region.
To implement this approach, we conducted a systematic literature review of articles published in the Web of Science and MEDLINE databases using the keywords “dengue” and “surveillance”; “dengue” and “capture recapture”; or “dengue” and “sensitivity”. We included articles published between 1995 and 2012 in English, Spanish, French, or Portuguese, and obtained a total of 1,676 articles. We then reviewed the titles and abstracts of these articles and found 48 that contained information relevant to the study of EFs for dengue in SEA. We examined these 48 studies plus 14 related articles that we had collected from previous literature reviews in full text, checking references for any additional articles that we could have missed in our search (e.g., national publications not included in international indexes). Eight new articles resulted from reviewing the references.
We then filtered this literature, retaining studies that explicitly reported systematic data on EFs or included the necessary data to estimate them. The specific retention criteria were: (1) use of original, empirical data, (2) implementation of a scientifically valid approach, and (3) the external validity of the data gathered (plausible patterns among age groups, geographic regions, years, and study sites). We complemented the literature review with surveillance data to estimate corresponding EFs , .
We used original, empirically derived EFT and EFH, or the data needed to derive them, where available. Because studies that reported EF used various designs, sampling criteria, methods of analysis, study settings, time frames, among other aspects, for some countries we had to make assumptions about the rate of underreporting in hospitals and/or the share of ambulatory dengue episodes to derive an estimate of EF. We discuss these assumptions further in the next section.
For countries with no original, empirical data, we relied instead on extrapolation of EFT based on quality of health care of each specific country. Because we only had a few empirical observations of EFs, we created a Health Quality Index (HQI) using principal component analysis, including standardized measures of five country-level variables (Table 1) , : physicians density per 10,000 population (2005–2010), mortality rate for children <5 years (probability of dying by age 5 per 1,000 live births), neonatal mortality rate (per 1,000 live births), percentage of births attended by skilled health personnel, and total health expenditures per capita (US$). We chose these variables because they were readily available for most countries, and we hypothesized that they represented an underlying factor of healthcare quality. A similar method to address underreporting, based on a measure of accessibility to health care, was used by Murray et al.  in recent estimates for burden of disease for 291 diseases and injuries. With dengue being a primarily urban disease, we think that this measure of quality provides a more appropriate measure specifically for dengue. We then extrapolated values of EFT to other countries by running a linear regression with the reporting rates (RR = 1/EFT) as the dependent variable and the HQI as an independent variable. Because we only had five observations for EFT, we also included two additional empirically-derived estimates from Colombia (EFT = 9.0)  and Nicaragua (EFT = 20.3)  in the regression, and added a dummy variable to address regional differences. We used RR instead of EF to reduce heteroskedasticity, as suggested by visual inspection of residuals versus fitted values and a substantially lower chi-square in the Breusch-Pagan test. We used RR to predict EFT and confidence intervals from the regressions, and converted the final estimates into EFs for clarity. We only considered empirically-derived EFT and excluded estimates based on expert opinion (i.e., Malaysia) from the regression model.
While most underreporting occurs in ambulatory settings, hospitalized episodes are also underreported as indicated by the empirical evidence in the region and elsewhere . The main reason for underreporting in hospitals, even in well-funded health systems such as Singapore, appears to be under diagnosis, which may occur because of limited sensitivity of some diagnostic tests and cost constraints –, or because patients are not routinely tested. Other plausible causes include reliance on clinicians' reports , particularly in private settings, or limited technical expertise . To estimate EFH for countries where no empirically-derived data were available, we ran a regression with hospitalized dengue RR as the dependent variable and HQI as an independent variable. Because we found no significant correlation (see results for further discussion), we made two assumptions: (i) the rate of underreporting of hospitalized cases was, on average, the same as the average underreporting for countries in the region with empirical data, and (ii) OP∶IP of dengue episodes was, on average, the same as the weighted average of all available empirical studies in the region. While our assumption that EFH is constant for countries with no empirical data ignores idiosyncratic characteristics of health systems, we think that it is a reasonable approach given the relatively low variability we found on underreporting of hospitalized dengue episodes reported in empirical studies in SEA and elsewhere . Assumption (ii) was only necessary for the countries in which reported data came from outpatient and inpatient sources, i.e., Brunei, Laos, and the Philippines. Last, we estimated the annual average of dengue episodes by type of treatment (inpatient and outpatient) and the EFA for each country, where applicable.
Because the total cases of dengue remained an uncertainty, we conducted a probabilistic sensitivity analysis, simultaneously varying our parameter estimates based on available information. For countries with empirical data, (1) we estimated the range for EFT using a program evaluation and review technique (PERT) distribution with empirically derived EFT as the best estimate and either the range of empirically derived estimates as the lower and upper bounds where available (Cambodia and Thailand), or alternatively, the 95% prediction interval derived from the regression analysis as the lower and upper bounds (λ = 4 to approximate the shape of a Normal distribution). We used prediction intervals because we were predicting individual EFT for a specific country, and not the expected value of EFT for all subjects, and the standard errors differ in both cases. (2) For Cambodia and Thailand, we varied OP∶IP using a normal distribution based on the weighted average and standard deviation from country-specific studies in different years and/or sites , , . For Vietnam, Indonesia, Singapore, and Malaysia, for which we did not have enough country-specific OP∶IP observations, we varied EFH using a PERT (λ = 4) distribution with the country-specific empirical estimate (or expert-based estimate for Malaysia) as the best estimate, used 1.0 as the lower bound to be conservative (i.e. all hospitalized episodes of dengue were reported), and the maximum EFH (3.4) from empirical studies among the 12 countries as the upper bound.
For countries where no country-specific empirical data were available, we varied (1) EFT using a normal distribution with μ and σ based on predicted estimates from the regression analysis, (2) EFH using a PERT distribution (λ = 4) with 1 as the lower bound, the average empirical estimate from all 12 countries as the best estimate, and the highest empirical EFH estimate for all countries as the upper bound, and last, (3) OP∶IP using a normal distribution with μ = weighted average and σ = weighted standard deviation based on all available empirical results from the 12 countries. As an additional sensitivity analysis, we used triangular distributions instead of the PERT distributions to see how the results varied. We computed 20,000 Monte Carlo simulations for each parameter using RiskAMP, version 3.20 , which uses the Mersenne Twister random number generator. Iterations drew random values from the distribution of each input. We present results with 95% certainty level bounds.
Dengue incidence in selected countries
Figure 1 shows the total dengue episodes and associated deaths reported in SEA from 1988 through 2010.The three countries with the most reported episodes of dengue fever were Viet Nam, Thailand, and Indonesia with cumulative totals of 1.73, 1.54, and 1.43 million cases reported, respectively (1988–2010). Together they represent about 75% of the total reported dengue episodes in the region. In contrast, Bhutan, East Timor, and Brunei, which reported dengue only since 2002, summed 3,358 cases over the same time period. Figure 1 shows the combination of cycles of dengue epidemics in SEA, which peaked in 1998 and 2010 with 540,000 and 650,000 overall reported episodes, respectively, and an increasing trend of total reported episodes that reflects both a growing problem and better reporting. Total reported deaths peaked at 3,500 in 1998, and as expected, were significantly correlated to the total number of dengue episodes (r2 = 0.74, p<0.001).
We found considerable variation in surveillance systems in SEA, for example, in the type of dengue case reported by severity, age groups, or type of treatment. Table 1 shows the demographic, health quality, and surveillance system characteristics of the selected countries in SEA , , , , , , , –.
We identified 11 published articles that reported original, empirically derived EFs or the data needed to derive EFs –, , –, one study based on a systematic two-round Delphi process , three empirical studies on dengue burden , , , and seven studies that used EFs based on secondary analysis of published data or exclusively based on expert opinion , , , –. Table 2 shows the main results from the literature review for EFs, or necessary data, in SEA. We extracted data from the articles using a template similar to Table 2, with additional columns (e.g., date the article was reviewed, limitations). We did not consider secondary analysis of data. Although this study is an original research study and not a systematic review, we adapted relevant parts of the PRISMA check list and flowchart to our literature review (Figure S1, Table S1) .
Estimated EFs by country
As shown in Table 2, we found high quality data to estimate EFs for six countries: Cambodia, Thailand, Viet Nam, Indonesia, Singapore, and Malaysia. Our estimates for Thailand were based solely on cohort studies, and we combined a cohort and a capture-recapture study for Cambodia. We combined cohort studies and clinical surveillance studies to obtain our estimates for Indonesia and Viet Nam. The EFT for Singapore was based on blood samples from a national health survey and EFH was derived by combining these data with reported data from a recent multisite longitudinal study. We used data based on expert opinion to estimate appropriate EFs for Malaysia . While some important studies were not yet available when the expert workshop on dengue reporting in Malaysia took place (e.g., , ), the country's EFT and EFH were estimated through a rigorous two-round Delphi process (see Figure S2 for details) , . Table 3 shows a summary of the parameters used, sources, assumptions, and specific calculations for each country, which we discuss below.
Cambodia reports dengue episodes only among children <15 years old, but approximately 90% of the cases of dengue in Cambodia occur within this group , and about 80% occur among children <9 years old . Dengue case definitions are based on WHO guidelines, and do not require laboratory confirmation -only a sample undergo serological or virological testing . Our EF estimates for Cambodia were based on a cohort study  and a capture-recapture study  both in Kampong Cham province. Because both were carefully designed studies, we obtained EFH and EFT using a weighted average based on total dengue episodes by cohort by year. We also combined Vong et al.'s  and Wichmann et al.'s  cohort studies and obtained a weighted average of OP∶IP of 6.0∶1. Table 4 shows the estimated EFs and total cases by country. For the sensitivity analysis, we varied EFT using a PERT distribution based on the range of empirical estimates, as stated above, and OP∶IP using a normal distribution with μ = 6.0 and σ = 1.2 – the weighted average and standard deviation based on total dengue cases from both studies. Table 5 shows a summary of the distributions and parameters used in the sensitivity analysis for each country.
Thailand uses the WHO case definition to report patients of all ages, and laboratory testing is commonly applied to all hospitalized cases. Our estimates for EFs in Thailand were mostly based on Wichmann et al.'s study  but we refined their estimates using data from a previous cohort study (1998–2002) , . Wichmann et al. compared dengue incidence in the cohort to reporting data from the national surveillance dataset, stratifying data by type of management (inpatient and outpatient), year, and age group, and estimated an average EFH of 2.9, and OP∶IP of 2.5∶1. Using these estimates, the authors derived an EFT of 8.4. Another robust 1998–2002 cohort study in Kamphaeng Phet ,  provided an estimate of OP∶IP. A weighted average based on dengue episodes by year between these studies gave us an OP∶IP of 2.7∶1. Using detailed surveillance data on reported cases for years 2003–2009 which suggests that on average 79% of reported dengue corresponds to hospitalized cases , we adjusted Wichmann's  EFs. We derived an EFA = 29.8 and EFT = 8.5 (Table 4). For the sensitivity analysis, we varied OP∶IP using the weighted average and standard deviation based on total dengue episodes reported by Anderson et al.  and Wichmann et al. , and EFT using a PERT distribution based the range of empirically derived estimates  (Table 5).
We obtained EFT for Viet Nam based on a children cohort study with active surveillance in Lon Xuyen (2004–2007) . Tien et al. compared the average annual incidence rate of laboratory-confirmed dengue in the cohort with incidence data obtained from the national surveillance system for the same years, age groups, and region. Using the reported data, we estimated an average EFT = 5.8. Until 2005, only DHF and DSS were reported in Viet Nam; hence, we assumed that most reported episodes of dengue were hospitalized. Tien et al. reported an average rate of OP∶IP of suspected dengue of 0.8. While the OP∶IP ratio in Viet Nam is probably lower than in other countries because hospitalization is required for all children with suspected dengue , the very high proportion of cases that were hospitalized might have been an artifact of the study's procedures (a prospective children cohort adjacent to the provincial hospital). Instead, we used the weighted average OP∶IP for all studies in the region (4.4) and obtained an EFH = 1.2. Using active surveillance, Phuong et al.  found that there were 5.2 serologically confirmed dengue episodes for each patient diagnosed with dengue -although the accuracy of dengue diagnosis was less than 50%. This number provides an external validation of our EFT estimate for Viet Nam, since if all cases diagnosed were reported – which is not likely the case – we would expect about 5.2 laboratory-confirmed episodes of dengue for each reported episode.
The EFs for Indonesia were based on two empirical studies , . Reporting DHF episodes within 24 hours following diagnosis is required by law in Indonesia, and Chairulfatah et al.  found 3.3 hospitalized episodes of DHF for each reported episode, with 50% of cases >14 years. Because the accuracy of diagnosis increases with severity , we would expect DHF episodes to be reported more frequently than only acute dengue inpatient episodes, so we believe our estimate is rather conservative. Considering that Indonesia only reported inpatient dengue episodes, we combined this EFH with Porter et al.'s estimate of 2.3 episodes of dengue for every hospitalized episode , and obtained an EFT of 7.6.
We estimated EFT for Singapore mainly based on an empirical study by Yew et al. . Yew et al. found evidence suggesting that only one out of 23 dengue infections (including symptomatic and asymptomatic) were notified. Because Yew et al. did not provide information on the ratio of asymptomatic to symptomatic cases of dengue infection in Singapore, we obtained this ratio from a weighted average based on the total number of dengue infections by cohort-year from cohort studies in Indonesia, Thailand, and Viet Nam , , . On average, 18% of dengue infections were symptomatic, so we derived an EFT of 4.1. We estimated EFH using the OP∶IP (1.16∶1) ratio derived from data reported by Low et al. from the multicenter longitudinal Early Dengue Infection and Outcome Study (EDEN) in Singapore, 2005–2010 , . Last, we obtained from Carrasco et al.  that 56.5% of the total dengue episodes reported to the surveillance system were hospitalized patients. Carrasco et al. obtained this proportion using data from the Communicable Diseases Division of the MoH, the EDEN study, and the Adult Retrospective Dengue Study at Tan Tock Seng Hospital (ARDENT). From these estimates, we derived an EFH of 3.4, and an EFA of 5.0. Singapore has strict legal requirements and incentives for reporting dengue, and a high quality surveillance system. Thus, an EFH of 3.4 may be an overestimate of underreporting. To be conservative, we used instead an EFH of 2.5, equivalent to the average EFH from empirical studies as our best estimate and 3.4 as the upper bound in the sensitivity analysis. For the same reasons, we also used 1.0 as the lower bound for both EFH and EFT in the sensitivity analysis.
We obtained the EFs for Malaysia from a recent study by Shepard et al.  that combined multiple data sources to refine the estimates of underreporting of dengue cases, including data from the MoH, private laboratories, previous literature, and a two-round Delphi process (Figure S1). The first round of the Delphi process took place during a workshop in Malaysia, where evidence was discussed among experts from public and private sectors, and academia. The second round was conducted some weeks later among the same group after analyzing results from the workshop, updating evidence, and adjusting the results for internal consistency. The results from the Delphi process suggested an EFT = 3.8, EFH = 1.7, and an EFA = 65.6. The estimated EFs were conservative, since some important studies were not yet available for either round (e.g., , ). Shepard et al. obtained a distribution of dengue episodes by type of treatment using 2009 data, which was used to update their estimates using the average reported cases in 2001–2010. We think these EF estimates were as accurate as available evidence allowed at the time the Delphi panel took place. We varied EFT and EFH for Vietnam, Indonesia, Singapore, and Malaysia in the sensitivity analysis, as shown in Table 5.
EFs for hospitalized and total cases based on data extrapolation
We extrapolated EFT based on the country's quality of health care, defined by HQI, for countries where no country-specific empirical data were available. The five standardized country-level variables were internally consistent (Chronbach's alpha: 0.92; Kaiser-Meyer-Olkin measure >0.68 for all variables) and loaded to a single factor that accounted for most of the variability of the data (Eigenvalue: 3.8). Figure 2 shows the empirically derived reporting rates by country and the regression results with a 95% confidence interval (R2 = 0.93, HQI significant at p<0.01) for SEA. We predicted EFT for countries where no empirical data was available based on these regression results (Table 4). To check robustness, we also ran the regressions using estimates for countries in SEA only, and obtained similar results (R2 = 0.91, HQI significant at p = 0.01). Regression results were also similar using EFT as the dependent variable in the regression specification (all countries: R2 = 0.74, HQI significant at p = 0.08; only countries in SEA: R2 = 0.83, HQI significant at p = 0.03).
We found no significant correlation between HQI and EFH, which is possibly explained by the relatively low variability of underreporting of hospitalized dengue episodes and the few observations available. The average EFH for countries in SEA with empirical data was 2.5, which was within the range of EFH estimates obtained from systematic empirical studies in Puerto Rico ,  (all episodes 2.4; DHF: 2.9) and Brazil  (1.6). We also obtained a weighted OP∶IP average of 4.4∶1 , , , , , .
Based on assumption (i) EFH = 2.5, we directly derived the average distribution of dengue episodes in 2001–2010 for Bhutan, East Timor, and Myanmar. Considering both assumptions, (i) EFH = 2.5 and (ii) OP∶IP = 4.4∶1, we estimated EFA and the distribution of dengue episodes by treatment for the remaining three countries: Brunei, Laos, and Philippines. Table 4 shows a summary of the results: EFs by country and the average annual reported and estimated total dengue episodes by type of treatment (2001–2010).
Overall, there were on average 386,154 annual dengue episodes reported in SEA from 2001 through 2010. Using our expansion factors, we projected that a total of 2,917,368 symptomatic dengue episodes (95% certainty level: 2,722,270–3,378,463; interquartile range: 2,915,658–3,149,257) occurring each year on average, of which 815,636 were hospitalized (95% certainty level: 715,326–983,735) and 2,101,732 ambulatory (95% certainty level: 1,871,480–2,534,739) episodes. We obtained an overall EFT in the region of 7.6 (95% certainty level: 7.0–8.8) dengue episodes for every reported episode.
Last, as an additional sensitivity analysis, we did 20,000 Monte Carlo simulations using triangular distributions instead of PERT distributions, maintaining the same lower and upper bounds and best estimate for EFs. The results from were very similar. We obtained a 95% certainty level of 2,498,726–3,513,599 total dengue cases (interquartile range: 3,012,551–3,265,965), 676,098–1,023,528 hospitalized cases, and 1,930,568–2,668,726 ambulatory cases. The 95% certainty level of overall EFT was 7.2–9.1.
Obtaining an accurate estimate of the total number of episodes is a critical step in the study of the disease and economic burden of dengue. Our analysis suggested that there is substantial underreporting of symptomatic dengue illness in SEA, with an average of only about 13.2% (95% certainty level: 11.4%–14.3%) of all symptomatic dengue episodes reported to surveillance systems. Under-reporting is particularly a problem during inter-epidemic periods, while over-reporting (or substantially less under-reporting) might occur during epidemics. Undifferentiated fever due to dengue is indistinguishable for other viral fevers and even for DHF the differential diagnoses are very broad in the early febrile phase. But on balance, the overall effect is for all dengue to be underreported as evidenced by active surveillance studies of dengue in the region. We estimated a total of about 2.9 million annual dengue episodes occurring in 12 countries in SEA (2001–2010), with an average ratio of OP∶IP of 2.6 (95% certainty level: 2.0–3.3), which represent a serious burden to healthcare systems in the region.
The strengths of our approach include our systematic procedures to identifying high quality empirical studies on EFs in SEA, our systematic inclusion of all the studies that met these standards, and our adjustment for the most salient site-level characteristics. In implementing our study, we conducted a systematic literature review and filtered the studies based on specific criteria. The resulting empirical EFs reflect the behavior of patients, health professionals, and the laboratory and public health systems in diagnosing, treating and documenting dengue. Because EFs are ratios of two measures (projected and reported numbers of cases), they are more robust than raw numbers. High and low rates of dengue incidence tend to raise or lower both projected and reported numbers, without necessarily affecting the EF. The resulting empirical total EFs were relatively consistent, varying by a factor of only 3.1 from the lowest (4.1) to highest value (12.9) across all the countries. These similarities may reflect the health professionals shared experiences in training, professional conferences, publications, and guidance from the World Health Organization across SE Asia. Finally, while past studies have documented variations among dengue surveillance systems , , , , , partly summarized on Table 1, we were able to control for an important part of this variation. Because the completeness of dengue reporting was expected to reflect, in part, the quality of the health system overall, we controlled for this variation using a HQI. We expected higher quality health systems to have better reporting, and our regression results were consistent with these expectations. A relevant aspect of our estimates for total episodes of dengue is that they tend to smooth out geographic and time variation, since we used averages of reported cases in the last decade (2001–2010) and, in many cases, used weighted averages for parameters across studies and years. Even though our annual estimates do not reflect the actual idiosyncratic variability of total symptomatic dengue infections, they provide a more stable estimate of dengue burden in the region.
While we believe our methods provide a reasonable empirical basis to estimate EFs and the total symptomatic episodes of dengue in SEA, several limitations must be acknowledged. First, the number of published empirical studies in SEA for estimating the total expansion factor is limited -only 11. These studies varied in their methodologies (e.g., cohort studies, capture-recapture, hospital and health center surveillance, national surveys), age groups considered (e.g., cohorts of adult workers versus cohorts of children), types of study sites (e.g. major public hospitals, rural health posts), or severity of dengue infection reported (e.g., dengue fever, DHF, DSS). Similarly, our regression relating RR to HQI was also based only on our best estimates for five countries in SEA and two in the Americas. When there is little information on a subject, each new piece can make a considerable difference. For example, the new available evidence estimating EFT for Malaysia, based on expert opinion, seems to be conservative . As more empirical studies on underreporting become available, and surveillance systems improve their efficacy, EF estimates will be more accurate.
Second, the specific study locations and age groups were generally ones to which the researchers had access. They were generally not randomly selected and are not necessarily representative of the country as a whole. For example, the specific studies in some countries may suggest a lower proportion of ambulatory cases relative to hospitalized cases, as might be the case of Indonesia, were the OP∶IP of 1.3 may be explained because textile workers in Bandung  probably have higher income and better healthcare access than the average person in that country. We also found other empirical studies where this ratio may seem too high , .
Third, to the extent that site-to-site variation remains a factor, our ability to adjust for it was limited to our single variable, HQI. Also, the range we used for EFT in the sensitivity analysis considered only between-country and not within-country heterogeneity, which resulted in narrower ranges for our estimates of total episodes of dengue than if we had been able to include within-country heterogeneity.
Fourth, we found no empirical or high quality studies for six of the 12 countries included in our estimates of dengue episodes. We addressed this constraint by extrapolating data from countries with empirical studies, but could not control for other factors, such as characteristics of national healthcare systems, the level of dengue awareness, or various other relevant factors, such as virus serotypes, rainfall, or global commerce and tourism . As expected, the 95% certainty level for overall EFT in countries with empirical studies (6.8–8.1) was much narrower than for countries where we extrapolated EFT (6.4–13.6).
Fifth, some evidence suggests that the rate of underreporting varies by the severity of dengue symptoms, with reporting increasing for more severe dengue . This evidence may imply that the most modest cases would have the highest degree of underreporting. Due to data limitations, we were not able to categorize numbers of reported cases by dengue severity. Despite all these limitations, we believe that adjustment for underreporting of dengue is critical to estimate the true economic burden, and we sought to make the best adjustments possible with available data.
Estimating the rate of underreporting of dengue with accuracy is a very complex task, particularly distinguishing the factors that drive underreporting to surveillance systems. More accurate estimates of the rate of underreporting of dengue would require a better understanding of the epidemiology of dengue. We think that long-term nationally representative cohort studies that could factor in a wide range of variables related to healthcare systems (e.g., facility types, public and private sectors, number of physicians, specific lab tests used, dengue definition, diagnosis, healthcare access and coverage), geography (e.g., rural and urban population, altitude, latitude, rainfall), virus (e.g., dengue serotypes and genotypes), vectors (e.g., vector control activities, public awareness campaigns), and dengue sequelae (e.g., severity of disease, long-term symptoms, duration and intensity) would be the ideal source of data. However such studies require considerably more time and resources than alternative designs, such as regional or local cohort studies, capture-recapture studies, or Delphi panels.
By generating better estimates of EFs, this paper will contribute towards a better understanding of underreporting of dengue episodes and improving regional and country-specific estimates of the economic and disease burden of dengue in SEA . While this study was focused on dengue in SEA, analogous principles apply to other regions of the world and other diseases reported through health information and surveillance systems. Estimating EFs is an important middle step towards estimating the economic burden of disease, and the cost-effectiveness of vaccines and other preventive and curative approaches.
PRISMA 2009 Flow Diagram. Source: .
Description of a Delphi Process. Note: A Delphi process is designed to use expert knowledge systematically to help solve complex issues when there are insufficient data. It aims to get the most reliable expert opinion through several rounds of consultation with controlled opinion feedback. Since the Delphi method is based on experts with a range of field experience, discussion and opinions strengthen the grounding of the results which will be more robust with diverse contexts and settings. The Delphi method usually leads to independent thought and gradual formation of a considered opinion, as experts have the opportunity to revise individual views based on available evidence and other factors the expert might have overlooked on a previous round. Sources: , .
The authors thank Laurent Coudeville, PhD, and Germano Ferreira, PhD for helpful comments, and Clare L. Hurley, MMHS, for editorial assistance.
Conceived and designed the experiments: EAU YAH DSS. Analyzed the data: EAU DSS. Contributed reagents/materials/analysis tools: DSS YAH EAU. Wrote the paper: EAU DSS. Helped secure funding: DSS EAU YAH.
- 1. Gubler DJ (2011) Emerging vector-borne flavivirus diseases: are vaccines the solution? Expert Rev Vaccines 10: 563–565.
- 2. World Health Organization (2009) Dengue: Guidelines for Diagnosis, Treatment, Prevention and Control. New Edition. Available: http://whqlibdoc.who.int/publications/2009/9789241547871_eng.pdf. Accessed January 2012.
- 3. Guzman A, Isturiz RE (2010) Update on the global spread of dengue. Int J Antimicrob Agents 36: S40–S42.
- 4. Gubler DJ (1998) Dengue and dengue hemorrhagic fever. Clin Microbiol Rev 11: 480–496.
- 5. Jacobs M (2000) Dengue: emergence as a global public health problem and prospects for control. Trans Roy Soc Trop Med Hyg 94: 7–8.
- 6. Ooi EE, Gubler DJ (2008) Dengue in Southeast Asia: epidemiological characteristics and strategic challenges in disease prevention. Cad Saude Publica 25: S115–S124.
- 7. Patz JA, Epstein PR, Burke TA, Balbus JM (1996) Global climate change and emerging infectious diseases. JAMA 275: 217–223.
- 8. Gubler DJ (2004) The changing epidemiology of yellow fever and dengue, 1900 to 2003: full circle? Comp Immunol Microbiol Infect Dis 27: 319–330.
- 9. Simmons CP, Farrar JJ, Nguyen VVC, Wills B (2012) Current concepts: dengue. N Engl J Med 366: 1423–1432.
- 10. Gubler DJ, Meltzer M (1999) Impact of dengue/dengue hemorrhagic fever on the developing world. In: Advances in Virus Research, Vol 53. San Diego: Academic Press Inc. pp. 35–70.
- 11. World Health Organization - Regional Office for South East Asia (2010) Situation update of dengue in the SEA Region, 2010. Available: http://www.searo.who.int/LinkFiles/Dengue_Dengue_update_SEA_2010.pdf. Accessed 1 September 2011.
- 12. World Health Organization (2010) The Frst WHO Report on Neglected Tropical Diseases: Working to Overcome the Global Impact of Neglected Tropical Diseases. Geneva, Switzerland: WHO.
- 13. World Health Organization - Regional Office for South-East Asia (2011) Comprehensive Guidelines for Prevention and Control of Dengue and Dengue Haemorrhagic Fever. Revised and Expanded. Available: http://www.searo.who.int/entity/vector_borne_tropical_diseases/documents/SEAROTPS60/en/index.html. Accessed 10 January 2013.
- 14. Kliks SC, Nimmanitya S, Nisalak A, Burke DS (1988) Evidence that maternal dengue antibodies are important in the development of dengue hemorrhagic-fever in infants. Am J Trop Med Hyg 38: 411–419.
- 15. Kliks SC, Nisalak A, Brandt WE, Wahl L, Burke DS (1989) Antibody-dependent enhancement of dengue virus growth in human monocytes as a risk factor for dengue hemorrhagic-fever. Am J Trop Med Hyg 40: 444–451.
- 16. Shepard DS, Coudeville L, Halasa YA, Zambrano B, Dayan GH (2011) Economic impact of dengue illness in the Americas. Am J Trop Med Hyg 84: 200–207.
- 17. Vong S, Goyet S, Ly S, Ngan C, Huy R, et al. (2012) Under-recognition and reporting of dengue in Cambodia: a capture-recapture analysis of the National Dengue Surveillance System. Epidemiol Infect 140: 491–499.
- 18. Chairulfatah A, Setiabudi D, Agoes R, van Sprundel M, Colebunders R (2001) Hospital based clinical surveillance for dengue haemorrhagic fever in Bandung, Indonesia 1994–1995. Acta Trop 80: 111–115.
- 19. Wichmann O, Yoon IK, Vong S, Limkittikul K, Gibbons RV, et al. (2011) Dengue in Thailand and Cambodia: An assessment of the degree of underrecognized disease burden based on reported cases. PloS Negl Trop Dis 5: e996.
- 20. Vong S, Khieu V, Glass O, Ly S, Duong V, et al. (2010) dengue incidence in urban and rural Cambodia: Results from population-based active fever surveillance, 2006–2008. PloS Negl Trop Dis 4: e903.
- 21. Chantha N, Guyant P, Hoyer S (1998) Control of DHF outbreak in Cambodia, 1998. Dengue Bulletin 22: 69–74.
- 22. Ooi E, Gubler D, Nam V (2007) Dengue research needs related to surveillance and emergency response. Report to the Scientific Working Group Meeting on Dengue, Geneva, 1–5 October, 2006. Geneva: World Health Organization WHO. 124–133 p.
- 23. Deen JL, Harris E, Wills B, Balmaseda A, Hammond SN, et al. (2006) The WHO dengue classification and case definitions: time for a reassessment. Lancet 368: 170–173.
- 24. Gubler DJ (2002) How effectively is epidemiological surveillance used for dengue programme planning and epidemic response? Dengue Bulletin 26: 96–106.
- 25. King CC, Wen TH, Chao DY, Huang SYJ, Chuang SF, et al. (2009) Epidemiologic conditions associated with cases of dengue hemorrhagic fever and control efforts: applying Taiwan's experiences to global control. Am J Trop Med Hyg 81: 802.
- 26. Beatty ME, Stone A, Fitzsimons DW, Hanna JN, Lam SK, et al. (2010) Best practices in dengue surveillance: A report from the Asia-Pacific and Americas Dengue Prevention Boards. PloS Negl Trop Dis 4: e890.
- 27. Shepard DS, Suaya JA, Halstead SB, Nathan MB, Gubler DJ, et al. (2004) Cost-effectiveness of a pediatric dengue vaccine. Vaccine 22: 1275–1280.
- 28. Association of Southeast Asian Nations (ASEAN) (2011) The official website of the Association of Southeast Asian Nations. One vision, one identity, one community. Available: http://www.aseansec.org. Accessed June 13 2011.
- 29. Ministry of Health Indonesia (2012) Dengue Haemorrhagic Fever, 2011. Available: http://www.pppl.depkes.go.id/. Accessed 13 February 2012.
- 30. Ministry of Health Malaysia (2010) Vector borne disease control section. Available: http://www.dph.gov.my/vektor/eng/kes_dd_tahunan.html. Accessed 1 May 2010.
- 31. Ministry of Health Malaysia (2010) Notifiable Disease Database. Kuala Lumpur, Malaysia: Institute for Medical Research, Epidemiology and Biostatistics Unit.
- 32. Ministry of Health Singapore (2012) Weekly Infectious Diseases Bulletin. MOH weekly publication of statistics on local infectious disease situation. Available: http://www.moh.gov.sg/content/moh_web/home/statistics/infectiousDiseasesStatistics/weekly_infectiousdiseasesbulletin.html?year=2012. Accessed April 2012.
- 33. Bureau of Epidemiology Thailand (2011) Annual Surveillance Report. Available: http://22.214.171.124/Annual/. Accessed 22 November 2011.
- 34. Department of Statistics Singapore (2012) Statistics - Population. Available: http://www.singstat.gov.sg/pubn/reference/yos12/statsT-population.pdf. Accessed October 2012.
- 35. World Health Organization - Regional Office for South East Asia (2011) Reported Cases of DF/DHF in Selected Countries in SEA Region (1985–2005). Available: http://www.searo.who.int/en/Section10/Section332_1101.htm. Accessed 30 August 2011.
- 36. World Health Organization - Western Pacific Region (2011) Annual Dengue Data in the Western Pacific Region. Available: http://www.wpro.who.int/health_topics/dengue/data.htm. Accessed 30 August 2011.
- 37. World health Organization (2010) Data Query by Country - Global Health Atlas. Available: http://apps.who.int/globalatlas/dataQuery/default.asp. Accessed 11 January 2013.
- 38. World Health Organization - Western Pacific Region (2008) Asia-Pacific Dengue Program Managers Meeting. Singapore: World Health Organization (WHO).
- 39. Duarte HHP, Franca EB (2006) Data quality of dengue epidemiological surveillance in Belo Horizonte, Southeastern Brazil. Rev Saude Publica 40: 134–142.
- 40. Standish K, Kuan G, Aviles W, Balmaseda A, Harris E (2010) High dengue case capture rate in four years of a cohort study in Nicaragua compared to national surveillance data. PloS Negl Trop Dis 4: e633.
- 41. Halstead SB (2006) Dengue in the Americas and Southeast Asia: Do they differ? Rev Panam Salud Publica 20: 407–415.
- 42. Clark DV, Mammen MP, Nisalak A, Puthimethee V, Endy TP (2005) Economic impact of dengue fever/dengue hemorrhagic fever in Thailand at the family and population levels. Am J Trop Med Hyg 72: 786–791.
- 43. Carrasco LR, Lee LK, Lee VJ, Ooi EE, Shepard DS, et al. (2011) Economic impact of dengue iIlness and the cost-effectiveness of future vccination programs in Singapore. PloS Negl Trop Dis 5: e1426.
- 44. Garg P, Nagpal J, Khairnar P, Seneviratne SL (2008) Economic burden of dengue infections in India. Trans Roy Soc Trop Med Hyg 102: 570–577.
- 45. World Health Organization (2010) World Health Statistics 2010. Geneva, Switzerland: WHO, Department of Health Statistics and Informatics.
- 46. World Health Organization (2012) World Health Statistics 2012. Geneva, Switzerland: WHO, Global Health Observatory (GHO).
- 47. Murray CJL, Vos T, Lozano R, Naghavi M, Flaxman AD, et al. (2012) Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 380: 2197–2223.
- 48. Camacho T, de la Hoz F, Cardenas V, Sanchez C, de Calderon L, et al. (2004) Incomplete surveillance of a dengue-2 epidemic in Ibague, Colombia, 1995–1997. Biomedica 24: 174–182.
- 49. Guzman MG, Jaenisch T, Gaczkowski R, Vo TTH, Sekaran SD, et al. (2010) Multi-country evaluation of the sensitivity and specificity of two commercially-available NS1 ELISA assays for dengue dagnosis. PloS Negl Trop Dis 4: e811.
- 50. Dussart P, Labeau B, Lagathu G, Louis P, Nunes MRT, et al. (2006) Evaluation of an enzyme immunoassay for detection of dengue virus NS1 antigen in human serum. Clin Vaccine Immunol 13: 1185–1189.
- 51. Guzmán MaG, Kourí G (2004) Dengue diagnosis, advances and challenges. Int J Infect Dis 8: 69–80.
- 52. Anderson KB, Chunsuttiwat S, Nisalak A, Mammen MP, Libraty DH, et al. (2007) Burden of symptomatic dengue infection in children at primary school in Thailand: a prospective study. Lancet 369: 1452–1459.
- 53. Structured Data LLC (2007) RiskAMP User Guide. User Guide for the RiskAMP Monte Carlo Add-in. Available: www.riskamp.com. Accessed June 2012.
- 54. Beaute J, Vong S (2010) Cost and disease burden of dengue in Cambodia. BMC Public Health 10: 1–6.
- 55. International Monetary Fund (2011) World Economic Outlook Database. Available: http://www.imf.org/external/pubs/ft/weo/2011/02/weodata/index.aspx. Accessed January 2012.
- 56. Kalayanarooj S, Rimal HS, Andjaparidze A, Vatcharasaevee V, Nisalak A, et al. (2007) Short report: Clinical intervention and molecular characteristics of a dengue hemorrhagic fever outbreak in Timor leste, 2005. Am J Trop Med Hyg 77: 534–537.
- 57. Osman O, Fong MY, Devi S (2007) A preliminary study of dengue infection in Brunei. Jpn J Infect Dis 60: 205–208.
- 58. Prasittisuk C, Andjaparidze AG, Kumar V (1998) Current status of dengue/dengue haemorrhagic fever in WHO South-East Asia region. Dengue Bulletin 22.
- 59. Population and Housing Census of Bhutan (2005) Result highlights. Langjophakha, Thimphu, Bhutan: Office of the Census Commissioner.
- 60. United Nations (2011) World Urbanization Prospects: The 2007 Revision Population Database. Available: http://esa.un.org/unup/. Accessed 12 October 2011.
- 61. Vong S, Ngan C, Buchy P, Khieu V, Huy R, et al. (2007) Estimating the incidence of dengue fever in Cambodia: Results of a capture recapture analysis. Am J Trop Med Hyg 77: 26.
- 62. Phuong HL, de Vries PJ, Nga TTT, T Giao P, Hung LQ, et al. (2006) Dengue as a cause of acute undifferentiated fever in Vietnam. BMC Infect Dis 6.
- 63. Porter KR, Beckett CG, Kosasih H, Tan RI, Alisjahbana B, et al. (2005) Epidemiology of dengue and dengue hemorrhagic fever in a cohort of adults living in Bandung, West Java, Indonesia. Am J Trop Med Hyg 72: 60–66.
- 64. Tien NTK, Luxemburger C, Toan NT, Pollissard-Gadroy L, Houng VTQ, et al. (2010) A prospective cohort study of dengue infection in schoolchildren in Long Xuyen, Viet Nam. Trans Roy Soc Trop Med Hyg 104: 592–600.
- 65. Yew YW, Ye T, Ang LW, Ng LC, Yap G, et al. (2009) Seroepidemiology of dengue virus infection among adults in Singapore. Ann Acad Med Singap 38: 667–675.
- 66. Endy TP, Chunsuttiwat S, Nisalak A, Libraty DH, Green S, et al. (2002) Epidemiology of inapparent and symptomatic acute dengue virus infection: A prospective study of primary school children in Kamphaeng Phet, Thailand. Am J Epidemiol 156: 40–51.
- 67. Shepard DS, Undurraga EA, Lees RS, Halasa YA, Lum L, et al. (2012) Use of multiple data sources to estimate the economic cost of dengue illness in Malaysia. Am J Trop Med Hyg 87: 796–805.
- 68. Low JGH, Ong A, Tan LK, Chaterji S, Chow A, et al. (2011) The early clinical features of dengue in adults: challenges for early clinical diagnosis. PloS Negl Trop Dis 5: e1191.
- 69. Low JGH, Ooi EE, Tolfvenstam T, Leo YS, Hibberd ML, et al. (2006) Early dengue infection and outcome study (EDEN) - Study design and preliminary findings. Ann Acad Med Singap 35: 783–789.
- 70. Halstead SB, Suaya JA, Shepard DS (2007) The burden of dengue infection. Lancet 369: 1410–1411.
- 71. Burke DS, Nisalak A, Johnson DE, Scott RM (1988) A prospective study of dengue infections in Bangkok. Am J Trop Med Hyg 38: 172–180.
- 72. Suaya JA, Shepard DS, Chang MS, Caram M, Hoyer S, et al. (2007) Cost-effectiveness of annual targeted larviciding campaigns in Cambodia against the dengue vector Aedes aegypti. Trop Med Int Health 12: 1026–1036.
- 73. Kongsin S, Jiamton S, Suaya J, Vasanawathana S, Sirisuvan P, et al. (2010) Cost of dengue in Thailand. Dengue Bulletin 34: 77–88.
- 74. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097.
- 75. Dalkey N, Helmer O (1963) An experimental application of the Delphi method to the use of experts. Management Science 9: 458–467.
- 76. Delbecq AL, Van de Ven AH, Gustafson DH (1975) Group techniques for program planning: a guide to nominal group and Delphi processes. Glenview, IL: Scott, Foresman and Company.
- 77. Huy R, Buchy P, Conan A, Ngan C, Ong S, et al. (2010) National dengue surveillance in Cambodia 1980–2008: Epidemiological and virological trends and the impact of vector control. Bull World Health Organ 88: 650–657.
- 78. Dechant EJ, Rigau-Perez JG (1999) Hospitalizations for suspected dengue in Puerto Rico, 1991–1995: Estimation by capture-recapture methods. Am J Trop Med Hyg 61: 574–578.
- 79. Rigau-Perez JG (1999) Surveillance for an emerging disease: Dengue hemorrhagic fever in Puerto Rico, 1988–1997. Puerto Rico Association of Epidemiologists. P R Health Sci J 18: 337–345.
- 80. Pacific Rim Innovation and Management (2008) Regional: strengthening epidemiological surveillance and response for communicable diseases in Indonesia, Malaysia, and Philippines (TA No. 6305-REG). Asian Development Bank
- 81. Sabchareon A, Sirivichayakul C, Limkittikul K, Chanthavanich P, Suvannadabba S, et al. (2012) Dengue infection in children in Ratchaburi, Thailand: A cohort study. I. Epidemiology of symptomatic acute dengue infection in children, 2006–2009. PloS Negl Trop Dis 6: e1732.
- 82. Goeree R, Burke N, O'Reilly D, Manca A, Blackhouse G, et al. (2007) Transferability of economic evaluations: Approaches and factors to consider when using results from one geographic area for another. Curr Med Res Opin 23: 671–682.
- 83. Shepard DS, Undurraga EA, Halasa YA (2013) Economic and disease burden of dengue in Southeast Asia. PloS Negl Trop Dis 7: e2055.