A Systematic Review of Cost-Effectiveness Studies Reporting Cost-per-DALY Averted

Introduction Calculating the cost per disability-adjusted life years (DALYs) averted associated with interventions is an increasing popular means of assessing the cost-effectiveness of strategies to improve population health. However, there has been no systematic attempt to characterize the literature and its evolution. Methods We conducted a systematic review of cost-effectiveness studies reporting cost-per-DALY averted from 2000 through 2015. We developed the Global Health Cost-Effectiveness Analysis (GHCEA) Registry, a repository of English-language cost-per-DALY averted studies indexed in PubMed. To identify candidate studies, we searched PubMed for articles with titles or abstracts containing the phrases “disability-adjusted” or “DALY”. Two reviewers with training in health economics independently reviewed each article selected in our abstract review, gathering information using a standardized data collection form. We summarized descriptive characteristics on study methodology: e.g., intervention type, country of study, study funder, study perspective, along with methodological and reporting practices over two time periods: 2000–2009 and 2010–2015. We analyzed the types of costs included in analyses, the study quality on a scale from 1 (low) to 7 (high), and examined the correlation between diseases researched and the burden of disease in different world regions. Results We identified 479 cost-per-DALY averted studies published from 2000 through 2015. Studies from Sub-Saharan Africa comprised the largest portion of published studies. The disease areas most commonly studied were communicable, maternal, neonatal, and nutritional disorders (67%), followed by non-communicable diseases (28%). A high proportion of studies evaluated primary prevention strategies (59%). Pharmaceutical interventions were commonly assessed (32%) followed by immunizations (28%). Adherence to good practices for conducting and reporting cost-effectiveness analysis varied considerably. Studies mainly included formal healthcare sector costs. A large number of the studies in Sub-Saharan Africa addressed high-burden conditions such as HIV/AIDS, tuberculosis, neglected tropical diseases and malaria, and diarrhea, lower respiratory infections, meningitis, and other common infectious diseases. Conclusion The Global Health Cost-Effectiveness Analysis Registry reveals a growing and diverse field of cost-per-DALY averted studies. However, study methods and reporting practices have varied substantially.


Introduction
Cost-effectiveness analysis (CEA), which quantifies the value of interventions in terms of costs per unit of health achieved, can help inform resource allocation decisions. Cost-per-qualityadjusted life year (QALY) gained (also referred to as cost-per-QALY) and cost-per-disabilityadjusted life year (DALY) averted (also referred to as cost-per-DALY) studies are both approaches to cost-effectiveness analyses, in which the costs and effects of programs and at least one alternative are calculated and presented in a ratio of incremental cost to incremental effect [1]. QALYs and DALYs are metrics used to measure population health, which integrate morbidity and mortality, though there are some differences in how they are constructed [2]. Historically, QALYs have been applied predominantly in higher-income countries while DALYs have been used more extensively in global health analyses-e.g., the latter metric was favored by the WHO-CHOICE and Disease Control Priorities projects [3,4]. The World Bank and World Health Organization (WHO), along with academic researchers, developed the DALY in the 1990s as a generic measure of population health for assessing the burden imposed by a wide range of diseases and conditions [5,6]. DALYs capture both mortality in terms of years of life lost (YLL) and morbidity in terms of years of life with disability (YLD) [7]. The YLD component represents a duration that is scaled by disability severity weights that range from 0 (no adverse impact on quality of life) to 1 (burden equivalent in preference to being dead) [8,9]. The Global Burden of Disease project has developed standardized weights to facilitate comparisons across regions and countries [10]. The number of cost-per-DALY studies has grown in recent years, with many of these analyses focusing on health interventions in low-and middle-income countries. However, there has been no systematic attempt to characterize the literature and its evolution.
This article describes a systematic review of the cost-per-DALY literature as well as the creation of a new database that catalogs these studies, detailing the interventions, conditions, and target populations investigated, and the quality and consistency of the methods employed. The goals are to help policy makers and researchers understand the best opportunities to improve population health in their jurisdictions given resource constraints and to help standardize methods in the field, thus improving quality and comparability across cost-per-DALY studies. We also examine trends for the types of interventions, methods and reporting practices. Finally, we investigate the extent to which cost-per-DALY studies have targeted diseases imposing the largest population burden.

Development of the GHCEA registry
The Center for the Evaluation of Value and Risk in Health (CEVR) at Tufts Medical Center developed the Global Health Cost-Effectiveness Analysis (GHCEA) Registry, a repository of English-language cost-per-DALY studies indexed in PubMed. The Registry, supported by a grant from the Bill & Melinda Gates Foundation, is a free, online resource (http://healtheconomics. tuftsmedicalcenter.org/ghcearegistry/) for researchers, policy makers, and other parties. The current version of the website allows searches of various aspects of published cost-per-DALY studies, including titles of papers and cost-per-DALY ratios, stratified by characteristics such as disease type, region, country and subpopulation (e.g., children). Future versions of the website will provide further enhancements to expand the scope and improve usability.

Systematic search of the literature
The GHCEA Registry includes published English-language studies that report original costper-DALY assessments from 2000-2015 (regular updates are planned for the future). It builds on research protocols developed for the Tufts Medical Center CEA Registry, a database of published cost-per-QALY analyses (QALYs) [11,12]. To identify candidate studies, we systematically searched PubMed for articles with titles or abstracts containing the phrases "disabilityadjusted" or "DALY". We then screened candidate abstracts to determine if the referenced studies contain an original cost-per-DALY estimate. We excluded review, editorial, or methodological articles, and CEAs that do not quantify health impacts in terms of DALYs. Two reviewers with training in health economics independently reviewed each article selected in our abstract review, gathering information using a standardized data collection form. Reviewers extracted data relating to study design and methodology, interventions, and outcomes. The two reviewers then convened for a consensus audit to resolve potential discrepancies and to ensure the quality of the collected information.
For each article, we described the health intervention evaluated, the intervention to which it was compared (the "comparator"), and the target population. We also collected information for each study on: 1) intervention type (e.g., pharmaceutical, immunization, surgery); 2) country of focus; 3) funding source; 4) accuracy of the reported cost-per-DALY; 5) time horizon; 6) perspective (e.g., societal, health care sector or health care payer); 7) types of costs included (e.g., formal health care sector, informal health care sector, non-health care sector); 8) discount rate, if any; 9) age-weighting; 10) currency and currency year; 11) inclusion or exclusion of additional costs associated with greater longevity conferred by the treatment; 12) type of sensitivity analysis performed, and 13) the specified cost-per-DALY threshold value reported as a benchmark to assess cost-effectiveness favorability (ratios with values below the threshold indicate accrual of health benefits at a relatively low cost-per-DALY, hence suggesting the underlying intervention has "good value"). For each article, we also provided an overall quality score on a scale from 1 (low) to 7 (high), based on factors, such as whether articles present a correct computation of the ICERs, a complete characterization of uncertainty, and appropriate reporting of the intervention, comparator and target population.
We recorded the cost-effectiveness ratio value, and when available, incremental costs (the ratio's numerator) and health effects (DALYs-the ratio's denominator). We also identified the quadrant in the "cost-effectiveness plane" containing each ratio. As illustrated in Fig 2, the northeast and northwest quadrants contain ratios for interventions that increase costs. The northeast and southeast quadrants contain ratios for interventions that improve health (avert DALYs). So, the northeast quadrant contains the intersection of these two groups (ratios for interventions that increase costs but also improve health). The southeast quadrant contains ratios for interventions that both improve health and reduce costs. The northwest quadrant contains ratios for interventions that increase costs and make health worse.
To facilitate comparisons of ratios across countries, we converted costs to 2014 US dollars using Purchasing Power Parity (PPP) reported by the World Bank [13] and adjusted for inflation using the "All Urban Consumers" component of the US Bureau of Labor Statistics Consumer Price Index [14].
Finally, we characterized uncertainty information reported for each ratio. This information might include a cumulative distribution of plausible values (sometimes referred to as a costeffectiveness acceptability curve), a ratio confidence interval, or sensitivity analysis results (the range of cost-effectiveness ratio values associated with alternative assumptions). We recorded each health state and the disability weight value associated with it, as reported in the study. We also recorded the data source (primary or secondary) for the reported disability weight.

Analyses
We explored trends over time for various study characteristics for two time periods: 2000 to 2009 and 2010 to 2015. For disease, we used the International Statistical Classification of Diseases and Related Health Problems, tenth revision (ICD-10) and the Global Burden of Disease (GBD) initiative classification system [15,16]. For intervention location, we used the GBD classification of countries by region and super region [3]. These regions group countries by geography and by population characteristics (in particular, country income level). We described study and methodological characteristics, including presentation of cost-effectiveness acceptability curves, correct incremental cost-effectiveness calculation, discounting, presentation of sensitivity analyses, and proper identification of study perspective. Finally, we analyzed the association between the top 10 diseases by burden and the number of 2000-2015 cost-per-DALY studies targeting each disease area. We conducted this analysis for the entire literature and looked at associations within the three most studied regions (Sub-Saharan Africa; High-Income Countries; and Southeast Asia, East Asia, and Oceania). followed by studies in high-income countries (21%) and countries in Southeast Asia, East Asia, and Oceania (13%) ( Table 1). The most common GBD disease classifications were communicable, maternal, neonatal, and nutritional disorders (67%), followed by non-communicable diseases (28%) and injuries (2%). The most common ICD-10 classifications were infectious and parasitic diseases (56%), followed by endocrine, nutritional and metabolic diseases (8%), and mental and behavioral disorders (8%). The most commonly assessed interventions included pharmaceuticals (32%), immunizations (28%), and health education and behavior programs (22%).

Growth and characteristics of cost-per-DALY studies, 2000-2015
More than half of the studies (59%) evaluated primary prevention interventions (e.g. disease screening), followed by tertiary prevention interventions (treatments) (38%) and secondary prevention interventions (18%) (interventions targeting high-risk individuals who have not yet developed clinical disease). Government (47%) and foundations (34%) were the most common sources of funding. Pharmaceutical industry sponsors funded 4% of published studies.

Top journals publishing cost-per-DALY studies
The top journals publishing cost-per-DALY studies include Vaccine (47 studies), PLOS One (46), and WHO's Bulletin of the World Health Organization (18) ( Table 2). In total, 151 journals published at least one cost-per-DALY study from 2000-2015.

Methodological and reporting practices
Approximately 30% of studies presented results from a societal perspective (studies including costs related to formal healthcare and at least one category of informal healthcare costs, i.e.,  patient or informal caregiver time; transportation, productivity loss, income loss, or costs in non-healthcare sectors) ( Table 3). Most studies mentioned a WHO-recommended cost-effectiveness benchmark threshold of 1 times gross domestic product (GDP) per capita and 3 times GDP per capita (30%). Studies most often used a discount rate of 3 percent for costs (65%) and DALYs (73%). Most studies reported sensitivity analyses (92%). Adherence to good practices for conducting and reporting cost-effectiveness analysis improved for several items between the 2000s and the period 2010-15 (Table 3). For example, appropriate reporting of study perspective increased from 58% during the period 2000-2009 to 73% from 2010-2015; for these periods, presentation of acceptability curves increased from  Health care sector perspective includes costs related to medical treatment (formal healthcare) + out-of-pocket costs incurred by patients. Health care payer perspective includes costs incurred by a (typically 3rd party) health care payer.
The perspective noted here is that judged by our reviewers not by study authors (see note to this table below). ** Our reviewers of the cost-per-DALY studies sometimes disagreed with the study authors on the actual perspective taken in the analysis. For example, study authors may have claimed that they took a "societal perspective" when in fact they only accounted for costs in the health sector, thus following a "health care sector" perspective. * Global Burden of Disease (GBD) recommendations are as follows-GBD 1990-age-weighting, discount rate = 3; GBD 2001-2-no age-weighting, discount rate = 3; GBD 2004-age-weighting, discount rate = 3; GBD 2010-no age-weighting, discount rate = 3 Discount rates for DALYs or costs are considered not applicable if the time horizon reported in the study is less than or equal to 1 year. ++ Overall score for study quality ranges from 1 (low) to 7 (high). 8% to 20%; disclosure of funding increased from 78% to 86%; identification of a cost-effectiveness benchmark threshold increased from 67% to 83%. With the change in GBD methods away from age-weighting, the proportion of studies using age-weighting declined from 36% during the period 2000-2009 to 24% during the period 2010-2015. The quality score was roughly unchanged over the time period

Discussion
The Global Health Cost-Effectiveness Analysis Registry catalogs 479 cost-per-DALY studies published from 2000-2015 and reveals a growing and diverse field. Investigators have used the cost-per-DALY framework to examine a wide range of interventions, diseases, and settings, with a heavy focus on interventions targeting infectious diseases. Though there has been some improvement in adherence to good practices for conducting and reporting cost-effectiveness analysis, study methods have varied substantially. Notably, studies include different types of costs. Because these differences suggest the need for greater standardization, efforts like the Gates Reference Case and the report of the Second U.S. Panel on Cost-Effectiveness in Health and Medicine are welcome [2,17].
Most of the studies have focused on Sub-Saharan Africa and Asia and in general, have focused on infectious and parasitic diseases. This focus may reflect the priorities of funding agencies and disease burden, though our study also highlights certain diseases and conditions (e.g., neonatal disorders, cardiovascular disease, injuries) that are under-represented in the cost-effectiveness literature relative to the burden they impose. These disparities underscore the need for governments, foundations, and researchers to reexamine health economic research priorities.
These findings on the growth and variation in the cost-per-DALY literature are similar in many ways to reviews of the cost-per-QALY literature. Research has shown rapid growth in the number of published cost-per-QALY studies, with applications to diverse diseases and interventions, but also substantial variability in reporting practices as well disparities between certain conditions (e.g., injuries) with high burden but which have had few associated costeffectiveness analyses [18].
As our study shows, cost-per-DALY studies focus mostly on countries in Africa, and Asia. In contrast, cost-per-QALY studies focus primarily on the United States and Europe [18]. The differences seem to be a historical "accident," rather than reflecting a strong rationale or intention. QALYs were developed and nurtured in the US and Europe. On the other hand, DALYs were developed to measure global burden of disease and were popularized in the global health community. In future work, we will examine how the use of cost-per-DALY estimates and cost-per-QALY estimates might differentially influence prioritization of interventions. We will also investigate the extent to which estimates developed for one country or region can be applied generalized to other settings. Finally, we will consider expanding our work to include other databases beyond PubMed, and to include articles published in languages other than English.