Conceived and designed the experiments: EWB PA SC ME. Performed the experiments: EWB. Analyzed the data: EWB PA SC ME. Wrote the first draft of the manuscript: EWB. Contributed to the writing of the manuscript: EWB PA SC ME.
ME was responsible for developing the intervention that is the subject of the cost-effectiveness analysis and testing its implementation. All other authors have declared that no competing interests exist.
A cost-effective analysis conducted by Edwine Barasa and colleagues estimates that a complex intervention aimed at improving quality of pediatric care would be affordable and cost-effective in Kenya.
To improve care for children in district hospitals in Kenya, a multifaceted approach employing guidelines, training, supervision, feedback, and facilitation was developed, for brevity called the Emergency Triage and Treatment Plus (ETAT+) strategy. We assessed the cost effectiveness of the ETAT+ strategy, in Kenyan hospitals. Further, we estimate the costs of scaling up the intervention to Kenya nationally and potential cost effectiveness at scale.
Our cost-effectiveness analysis from the provider's perspective used data from a previously reported cluster randomized trial comparing the full ETAT+ strategy (
Improving quality of care at scale nationally with the full ETAT+ strategy may be affordable for low income countries such as Kenya. Resultant plausible reductions in hospital mortality suggest the intervention could be cost-effective when compared to incremental cost-effectiveness ratios of other priority child health interventions.
According to latest global estimates from UNICEF, 7.6 million children currently die every year before they reach five years of age. Half of these deaths occur in children in sub-Saharan Africa and tragically, most of these deaths are due to a few treatable and preventable diseases, such as pneumonia, malaria, and diarrhea, for which effective interventions are already available. In order to meet the target of the 4th Millennium Development Goal—which aims to reduce the under-five child mortality rate by two-thirds from 1990 levels by 2015—delivering these interventions is essential.
In Kenya, the under-five child mortality rate must be reduced by half from its 2008 level in order to meet the Millennium Development Goal (MDG) target and so improving the management of serious child illness might help achieve this goal. A study published last year in
In the study mentioned above, the researchers compared the implementation of various processes of care in intervention and control hospitals at baseline and 18 months later and found that performance improved more in the intervention hospitals than in the control hospitals. However, while this strategy was effective at improving the quality of health care, it is unclear whether scaling up the approach would be a good use of limited resources. So in this study, the same researchers performed a cost-effectiveness analysis (which they conducted alongside the original trial) of their quality improvement intervention and estimated the costs and effects of scaling up this approach to cover the entire population of Kenya.
In order to perform the cost part of the analysis, the researchers collected the relevant information on costs by using clinical and accounting record reviews and interviews with those involved in developing and implementing the intervention. The researchers evaluated the effectiveness part of the analysis by comparing the implementation of their improved quality of care strategy as delivered in the intervention hospitals with the partial intervention as delivered in the control hospitals by calculating the mean percentage improvement in the 14 process of care indicators at 18 months. Finally, the researchers calculated the costs of scaling up the intervention by applying their results to the whole of Kenya—121 hospital facilities with an estimated annual child admission rate of 2,000 per facility.
The researchers found that the quality of care (as measured by the process of care indicators) was 25% higher in intervention hospitals than in control hospitals, while the cost per child admission was US$50.74 in intervention hospitals compared to US$31.1 in control hospitals. The researchers calculated that each percentage improvement in the average quality of care was achieved at an additional cost of US$0.79 per admitted child. Extrapolating these results to all of Kenya, the estimated annual cost of scaling up the intervention nationally was US$3.6 million, about 0.6% of the annual child health budget in Kenya.
The findings of this cost-effectiveness analysis suggests that a comprehensive quality improvement intervention is effective at improving standards of care but at an additional cost, which may be relatively cost effective compared with basic care if the improvements observed are associated with decreases in child inpatient mortality. The absolute costs for scaling up are comparable to, or even lower than, costs of other, major child health interventions. As the international community is giving an increasing focus to strengthening health systems, these findings provide a strong case for scaling up this intervention, which improves quality of care and service provision for the major causes of child mortality, in rural hospitals throughout Kenya and other district hospitals in sub-Saharan Africa.
Please access these Web sites via the online version of this summary at
The researchers' original article appeared in
The
The World Health Organization (WHO) provides information on
UNICEF
An estimated 7.6 million children die globally every year before the age of five
This was a cost-effectiveness analysis alongside a cluster randomized controlled trial (cRCT). The time horizon selected was 18 mo (September 2006–April 2008), which was the period during which the intervention was implemented and evaluated. Costing took a provider's perspective. While this is often considered narrow
In this cRCT (described in full elsewhere
The intervention was a package of care intervention that was delivered in the form of evidence-based CPG dissemination
In intervention hospitals, the intervention was delivered over 18 mo as a combination of ETAT+ training for health care workers conducted over 5.5 d, dissemination of CPG booklets, job aids, and pediatric admission record (PAR) forms
In the control hospitals, a partial version of the intervention was delivered in the form of CPG booklet distribution, a 1.5-d seminar, and provision of written survey feedback based on written reports only. Control hospitals did not receive any follow-up supervisory support or local facilitation.
We used process indicators of quality of care to estimate the effectiveness of the intervention. In total 14 pre-specified indicators that span three broad areas were considered as primary outcomes
Costs were categorized as intervention development, intervention implementation, and inpatient pediatric treatment costs. The latter were included in order to capture any change in resource use associated with the implementation of best practice pediatric care. Costs were summed across all categories to obtain the total cost per hospital and per hospital admission in intervention and control hospitals. Each cost category is further described in the following sections. Costs were collected using clinical and accounting record reviews and interviews with those involved in developing and implementing the intervention.
Development costs included the staff costs incurred in the development of ETAT+ guidelines and training, the costs of course training materials, and costs of organizing and running meetings and workshops. Staff costs were calculated by interviewing key staff involved in guideline development in order to estimate the amount of time spent on these activities. The opportunity cost of this time was then assumed to be equivalent to the associated cost of employment. The costs of course training materials were assumed to be equivalent to the market prices of these items. Development costs were annualized over 4 y, which was assumed to be the useful life of the clinical guidelines.
Implementation costs included the costs of initial ETAT+ training of health workers, follow-up training, supervisory visits and phone calls, feedback meetings, and on-site local facilitator costs. The opportunity costs of resources used in these activities, e.g., staff time used in attending trainings, were evaluated by estimating the number of days spent at each training workshop and calculating the costs on the basis of the equivalent cost of employment. The costs of the initial training were considered to be capital costs as the effects of the training were expected to be realized over a period of more than 1 y. These costs were annualized over a useful life of 2.5 y, which was the length of time over which the practice change effects were seen to be sustained
Treatment costs were computed as the sum of “hotel,” medicine, and laboratory costs per admission. Resource use data for patient length of stay in hospital, medicines, and laboratory tests were collected from patient clinical records. Estimates of the utilization of these resources were then multiplied against the unit cost of each item. Per diem “hotel” unit costs were derived from the WHO, “Choosing Interventions that are Cost Effective” (WHO-CHOICE) estimates and recent work on the economic costs of inpatient care in Kenya
The cost-effectiveness analysis compared the implementation of the ETAT+ strategy as delivered in the intervention hospitals with the partial intervention as delivered in the control hospitals. The partial intervention was chosen as a comparator because it mirrors practice that would be considered a basic, standard approach to dissemination of guidelines that does not typically include active follow-up or supervision and for ethical reasons (withholding new national guidelines was deemed unreasonable). While “no intervention” is an alternative counterfactual, this assumes, somewhat unrealistically to us, that no national or international body will produce guidelines or disseminate them or make attempts to improve poor hospital services.
The summary measure of effect was the mean of the adjusted differences between control and intervention hospitals at 18 mo. This was calculated as the mean percentage improvement in the 14 process of care indicators in intervention compared to control hospitals (Equation 1), with 95% CIs obtained by bootstrapping with 2000 iterations.
The incremental cost-effectiveness ratio (ICER) was defined as the incremental cost per percentage gain in mean quality based on the 14 indicators. This is the ratio of the difference in the total admission cost per child between intervention and control hospitals, and the difference in mean quality improvement (Equation 2).
The cRCT was not designed to examine effects on health outcomes, therefore we explored the potential incremental cost per DALY averted on the basis of conservative assumptions of the effect of improving quality of care on inpatient childhood mortality. We assumed relative reductions in the mortality rate of between 1% and 10%, equivalent to absolute reductions of between 0.07% and 0.7% with median inpatient mortality, derived from the eight hospitals, equal to 7%. The proportion of lives saved from the respective diseases (malaria, pneumonia, and diarrhea) were assumed to be equivalent to the proportions of the contribution of each of these diseases to under-five childhood deaths in Kenya
Kenya has 121 hospital facilities with estimated median annual pediatric admissions of 2,000 per facility across this group, representing a total for pediatric admissions of 242,000 per annum (there are a larger number of smaller hospitals not considered in this analysis). We estimated the cost of scaling up this intervention with a number of assumptions: (1) Development costs do not vary with scale-up; given that they are only incurred once, they are not a function of the scale of the intervention; (2) That training, supervision, and follow-up costs (implementation) vary as a function of the number of hospitals; (3) That treatment costs vary as a function of the number of pediatric admissions; (4)That the intervention would reach all 121 hospitals when at scale. It is however difficult to estimate potential economies of scale and scope, for example for supervision, that might lessen costs or specific, higher, travel costs for hard to reach areas during scale up. Given the skewness of treatment costs, their scale-up component costs were calculated on the log-scale and then back transformed to the original scale.
Uncertainty was addressed by specifying distributions around cost and effectiveness parameters and conducting probabilistic sensitivity analysis using Crystal Ball software (Decisioneering). Triangular distributions with plausible ranges were fitted around the effectiveness estimate, development costs, salaries, medicines, and “hotel” components of costs (
Parameter (Costs per Child Admission) | Base Case (US$) | Range (US$) | Distribution |
Full intervention development costs | 8.11 | 0–8.11 | Triangular |
Partial intervention development costs | 4.95 | 0–4.95 | Triangular |
Full intervention salary costs | 12.46 | 11.42–12.46 | Triangular |
Full intervention hotel costs | 20.68 | 20.68–39.93 | Triangular |
Full intervention medicine costs | 2.30 | 0.66–8.06 | Triangular |
Partial Intervention salary costs | 3.65 | 1.67–3.65 | Triangular |
Partial intervention hotel costs | 20.15 | 20.15–38.89 | Triangular |
Partial intervention medicine costs | 1.74 | 0.50–6.09 | Triangular |
Intervention Effectiveness | 25.01 | 3.54–52.10 | Triangular |
The mean of the adjusted differences of the 14 process measures between control and intervention hospitals was 25.01% (95% CI 17.87%–32.18%). The findings of performance changes across all process measures in both control and intervention hospitals are presented in
Indicator of Quality of Care | Intervention | Control | Adjusted Difference between Groups at 18 mo (%) |
95% CI | |||
Survey 1 | Survey 4 | Survey 1 | Survey 4 | ||||
|
|||||||
|
59.30 | 84.50 | 21.00 | 63.20 | 22.80 | −4.05 | 49.70 |
|
11.90 | 71.90 | 25.10 | 46.60 | 26.50 | −4.49 | 57.50 |
|
24.00 | 94.00 | 32.00 | 65.00 | 29.00 | 5.00 | 54.00 |
|
9.29 | 95.10 | 14.70 | 57.00 | 38.57 | 9.87 | 67.30 |
|
1.85 | 89.20 | 3.54 | 74.40 | 17.05 | 8.04 | 26.10 |
|
24.90 | 2.16 | 23.40 | 8.99 | 6.77 | −11.90 | −1.59 |
3.78 | 6.25 | 7.15 | 9.82 | 3.54 | −11.10 | 4.00 | |
|
0.00 | 37.00 | 0.00 | 2.31 | 35.10 | 7.32 | 62.80 |
|
10.10 | 92.50 | 2.48 | 41.10 | 52.10 | 26.20 | 78.00 |
|
4.20 | 91.90 | 14.80 | 66.70 | 26.30 | −3.66 | 56.30 |
|
0.39 | 87.80 | 9.95 | 45.70 | 42.60 | 25.10 | 60.20 |
7.33 | 1.02 | 14.10 | 7.46 | 6.53 | −12.90 | −0.20 | |
|
52.40 | 98.30 | 60.50 | 84.80 | 14.40 | 4.27 | 24.60 |
|
7.32 | 67.20 | 15.00 | 40.60 | 29.90 | 10.90 | 48.90 |
|
13.79 | 58.98 | 15.15 | 39.24 |
|
17.87 | 32.18 |
Adjusted difference between intervention arms obtained from linear or logistic regression analysis of hospital summary data adjusting for child's sex and hospital factors (size, malaria endemicity, HIV prevalence, all cause mortality).
Total intervention costs and admission costs per child in intervention and control hospitals are presented in
Cost Items | Intervention Hospitals | Control Hospitals | As Percent of Total Intervention Costs | |||
Cost per Hospital US$ | Cost per Patient US$ |
As Percent of Total Intervention Costs | Cost per Hospital US$ | Cost per Patient US$ | ||
|
||||||
Development costs | 16,227.46 | 8.11 | 15.98 | 9,898.29 | 4.95 | 15.93 |
Training material costs | 692.92 | 0.35 | 0.69 | 0.00 | 0.00 | 0.00 |
|
16,920.39 | 8.46 | 16.67 | 9,898.29 | 4.95 | 15.93 |
|
||||||
Initial training | 8,069.32 | 4.03 | 7.94 | 2,017.33 | 1.01 | 3.25 |
|
||||||
Follow-up trainings | 4,348.05 | 2.17 | 4.28 | 0.00 | 0.00 | 0.00 |
Local facilitator costs | 5,697.87 | 2.85 | 5.62 | 0.00 | 0.00 | 0.00 |
Supervision costs | 10,135.50 | 5.07 | 9.99 | 0.00 | 0.00 | 0.00 |
|
20,181.42 | 10.09 | 19.89 | 0.00 | 0.00 | 0.00 |
|
28,250.73 | 14.13 | 27.85 | 2,017.33 | 1.01 | 3.25 |
|
45,171.11 | 22.59 | 44.52 | 11,915.62 | 5.96 | 19.19 |
|
||||||
|
45,080 | 22.54 | 44.42 | 41,800 | 20.90 | 67.28 |
|
5,080 | 2.51 | 4.95 | 3,600 | 1.80 | 5.80 |
|
11,260 | 5.63 | 11.10 | 6,660 | 3.33 | 10.72 |
|
56,304.79 | 28.15 | 55.48 | 50,202.30 | 25.10 | 80.81 |
|
101,475.90 | 50.74 | — | 62,117.92 | 31.06 | — |
Obtained by dividing the total cost per hospital by the estimated number of annual admissions for children under five per hospital (2,000).
An ordinary linear (OLS) regression of (log transformed) treatment costs revealed that costs did not significantly vary with child diagnosis, hospital, and time (i.e., across the four surveys) (unpublished data). We therefore pooled treatment costs across surveys and diagnoses within each study arm to increase sample sizes. The mean and median treatment costs were US$28.15 (95% CI 27.61–28.70) and US$22.47 (interquartile range [IQR] 14.33–32.78), respectively, in intervention hospitals and US$25.10 (95% CI 24.56–25.65) and US$19.25 (IQR 13.01–29.04) in control hospitals. “Hotel” costs were the key driver of treatment costs and contributed between 73.18% and 79.98% of treatment costs. Treatment costs disaggregated by category are presented in
The incremental cost per admission in intervention hospitals compared to control hospitals was US$19.68 (95% CI 5.31–31.92). The incremental cost per percentage improvement in quality of care was US$0.79 (95% CI 0.19–2.31) per child admission. These results are presented in
Strategy | Mean Admission Costs per Child US$ (95% CI) | Incremental Cost US$ (95% CI) | Incremental Effects (Percent Change in Quality of Care) (95% CI) | ICER (95% CI) |
|
31.06 (30.67–47.18) | — | — | — |
|
50.74 (49.26–67.06) | 19.68 (5.31–31.92) | 25.01% (17.87–32.18) | 0.79 (0.19–2.31) |
For an estimated coverage of 121 district hospitals and 242,000 annual under-five admissions, the estimated costs of scale-up were found to be US$3,559,328.78. This amount is estimated to be equivalent to 0.60% of the estimated 2010 annual budget for formal provision of care to children under five in Kenya (
Description | Full Intervention US$ | Partial Intervention US$ |
|
121 | 121 |
|
242,000 | 242,000 |
|
3559328.78 | 271386.32 |
|
572,000,000 | 572,000,000 |
|
0.60% | 0.06% |
Estimates of annual budget (2010) for provision of care to children under five derived from the Kenya national health sector strategic plan 2 (NHSSP II).
The mean baseline inpatient child mortality rate in the eight hospitals was 7%
Child Health Intervention | Incremental Cost per DALY Averted US$ |
Expanded immunization programme |
13.0–26.1 |
Hemophilus Influenzae vaccine (Hib) |
32.4–78.6 |
Provision of insecticide treated nets (ITNs) |
34.6–154.8 |
Improving inpatient care of very sick children (assuming between 10% and 1% reduction in baseline inpatient mortality rate) | 39.8–398.3 |
Integrated Management of Childhood Illnesses (IMCI) |
47.1–157.1 |
Pneumococcal conjugate vaccine |
71.1–230.7 |
Oral rehydration therapy (ORT) |
172.2–3352.0 |
Measles immunization |
335.2–5954.1 |
Breast feeding promotion programmes |
687.4–2609.9 |
ICER values adjusted to 2009 values using GDP deflators.
The incremental cost-effectiveness ratios were robust to changes in most of the variables included in the sensitivity analysis. Four factors (intervention effectiveness, hotel costs, medicine costs, and staff salaries) contributed 99% of the total variance in the ICER (
This analysis compares the costs and effects of a guideline-based intervention aimed at improving the quality of care of children admitted to district hospitals in Kenya. In analyzing the costs, we included the costs of developing clinical guidelines that are often left out in such analyses
There are many challenges in undertaking cost-effectiveness analyses of interventions targeting hospitals and multiple diseases. Basic challenges include the lack of high quality data on hotel costs. These represent between 73% and 80% of total treatment costs for children admitted with common diseases even though inpatient stays are typically 3 d or less. We used the WHO-CHOICE hotel cost estimates applicable to district hospitals in Kenya. In effect, the use of these data amounts to an assumption that there are no major differences in the intensity of staffing per patient. While this assumption was justified as we did not expect that our intervention would require different levels of health worker input within the different hospital settings, we acknowledge that poor primary data is a potential shortcoming and suggest that addressing this information gap is a priority.
Specifying summary measures that capture intervention effects is also a challenge
While acknowledging these limitations our findings suggest an additional cost of US$0.79 per child admitted to achieve a one percentage point improvement in this summary quality measure. The probabilistic sensitivity analysis reveals that hotel costs and intervention effectiveness contributed to more than 80% of the variation in this ICER. This finding has two implications: (1) The sensitivity analysis underlines the effects of poor data on hotel costs and methodological deficiencies in computing a summary measure of quality improvement; (2) Hotel costs (being key drivers of treatment costs) and intervention effectiveness have a significant impact on the cost effectiveness of this quality of care intervention.
Translating improvements in process measures into improved health status outcomes is problematic and the cluster randomized trial was not designed to measure effects on inpatient child mortality
Often the feasibility of scaling up is determined by likely costs. For the multifaceted intervention employed these were estimated to be US$3,559,328 if conducted by non-government personnel and 27% less if by government personnel. This amount can be compared with average annual projected expenditures by the Kenyan government on all care for children under five of US$572 million
Cost-effectiveness and affordability data are, however, not the only factors that should inform such allocative decisions. Other important considerations may include, equity, likely collateral benefits or adverse effects, and, of course, context and the politics of the day. Unfortunately methods to support and make transparent such complex prioritization decisions remain poorly developed. Advantages of scaling up such an integrated package of interventions encompassed in the ETAT+ strategy include potentially important externalities related to more general health system strengthening and introduction of a culture of improvement
Our work adds to a very small body of literature on economic evaluation of quality of care interventions
This analysis has shown that the improvement in quality of care attributed to the ETAT+ strategy (7) is associated with additional costs that are affordable to low-income countries like Kenya. The intervention may be relatively cost effective compared with standard care if the improvements observed are associated with reasonably conservative reductions in inpatient child mortality. The absolute costs for scaling up are comparable to or lower than costs of other, major child health interventions. As increasing focus is being given to strengthening health systems there would therefore appear to be a reasonably strong case for scaling up this intervention that improves service provision in rural hospitals for the major causes of child mortality in Kenya. This work also highlights the need for methodological developments in the economic analysis of complex, system-level interventions. These results are likely to be most usefully generalized to low-income countries beyond Kenya with similar facilities, burden of child mortality, and comparable or worse quality of pediatric care in hospitals.
Two-stage analysis plan for intervention effectiveness based on Hayes and Moulton
(TIF)
Treatment costs per admission.
(DOC)
Admission treatment costs per diagnosis.
(DOC)
The authors are grateful to the staff of all the hospitals included in the study and colleagues from the Ministry of Public Health and Sanitation, the Ministry of Medical Services, and the KEMRI/Wellcome Trust Programme for their assistance in the conduct of this study. This work is published with the permission of the Director of KEMRI.
clinical practice guideline
cluster randomized trial
disability adjusted life years
Emergency Triage and Treatment Plus
incremental cost-effectiveness ratio
World Health Organization