Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Cost analysis of an intrapartum quality improvement package for improving preterm survival and reinforcing best practices in Kenya and Uganda

  • Carolyn Smith Hughes ,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    carolyn.smithhughes@ucsf.edu

    Affiliation Institute for Global Health Sciences, University of California San Francisco, San Francisco, California, United States of America

  • Elizabeth Butrick,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Institute for Global Health Sciences, University of California San Francisco, San Francisco, California, United States of America

  • Juliana Namutundu,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Validation, Visualization, Writing – review & editing

    Affiliation School of Public Health, Makerere University, Kampala, Uganda

  • Easter Olwanda,

    Roles Data curation, Formal analysis, Methodology, Validation, Visualization, Writing – review & editing

    Affiliation Kenya Medical Research Institute, Nairobi, Kenya

  • Phelgona Otieno,

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Supervision, Validation, Writing – review & editing

    Affiliation Kenya Medical Research Institute, Nairobi, Kenya

  • Peter Waiswa,

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Supervision, Validation, Writing – review & editing

    Affiliation School of Public Health, Makerere University, Kampala, Uganda

  • Dilys Walker,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Institute for Global Health Sciences, University of California San Francisco, San Francisco, California, United States of America

  • James G. Kahn

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Institute for Global Health Sciences, University of California San Francisco, San Francisco, California, United States of America

Abstract

Introduction

Preterm birth is a leading cause of under-5 mortality, with the greatest burden in lower-resource settings. Strategies to improve preterm survival have been tested, but strategy costs are less understood. We estimate costs of a highly effective Preterm Birth Initiative (PTBi) intrapartum intervention package (data strengthening, WHO Safe Childbirth Checklist, simulation and team training, quality improvement collaboratives) and active control (data strengthening, Safe Childbirth Checklist).

Methods

In our analysis, we estimated costs incremental to current cost of intrapartum care (in 2020 $US) for the PTBi intervention package and active control in Kenya and Uganda. We costed the intervention package and control in two scenarios: 1) non-research implementation costs as observed in the PTBi study (Scenario 1, mix of public and private inputs), and 2) hypothetical costs for a model of implementation into Ministry of Health programming (Scenario 2, mostly public inputs). Using a healthcare system perspective, we employed micro-costing of personnel, supplies, physical space, and travel, including 3 sequential phases: program planning/adaptation (9 months); high-intensity implementation (15 months); lower-intensity maintenance (annual). One-way sensitivity analyses explored the effects of uncertainty in Scenario 2.

Results

Scenario 1 PTBi package total costs were $1.11M in Kenya ($48.13/birth) and $0.74M in Uganda ($17.19/birtth). Scenario 2 total costs were $0.86M in Kenya ($23.91/birth) and $0.28M in Uganda ($5.47/birth); annual maintenance phase costs per birth were $16.36 in Kenya and $3.47 in Uganda. In each scenario and country, personnel made up at least 72% of total PTBi package costs. Total Scenario 2 costs in Uganda were consistently one-third those of Kenya, largely driven by differences in facility delivery volume and personnel salaries.

Conclusions

If taken up and implemented, the PTBi package has the potential to save preterm lives, with potential steady-state (maintenance) costs that would be roughly 5–15% of total per-birth healthcare costs in Uganda and Kenya.

Introduction

Each year, about 15 million babies are born prematurely around the world [1], and 1 million infants and children will die due to prematurity-related complications [2]. In addition, more than 2 million stillbirths occur globally per year [3]. While stillbirth rates are generally declining, preterm birth rates are increasing, and the burden of preterm birth is highest in lower-resource settings, including sub-Saharan Africa [1, 3, 4]. Preterm birth and stillbirth lead to significant loss of life, lost productivity, and anguish among surviving family members; these conditions are also associated with substantial disability and healthcare costs [57]. Healthcare costs are generally inversely proportional to gestational age, not only during the initial days and weeks after birth, but also later in life [6].

Interventions to prevent stillbirths and improve preterm survival have been developed and tested in high- and low-resource settings alike [6, 8, 9], with lower-tech strategies demonstrating great promise and feasibility in low- and middle-income countries [9]. Given that the largest contribution to both stillbirths and early neonatal deaths occurs around the time of birth, this focus on intrapartum care is critical [10]. One such strategy to prevent stillbirth and improve preterm survival is the Preterm Birth Initiative East Africa (PTBi) package of interventions, which focuses on reinforcement of evidence-based practices to improve the quality of intrapartum and immediate newborn care provided to mothers and their infants [11].

The PTBi package of interventions (and other strategies to prevent preterm birth and promote neonatal survival) has been evaluated for effectiveness; however, data on the cost of quality improvement programs are limited [12, 13]. Understanding the start-up and longer-term costs and resources associated with integration of these interventions into current maternity and obstetric care practices may help inform resource allocation for these life-saving strategies.

Data regarding costs of maternity care are limited and estimates vary substantially by country, health facility type/delivery volume, and source/reference [14]. For example, one analysis estimated that health system costs per delivery/birth in sub-Saharan African countries range from $8-$73 USD for a vaginal delivery, $80-$562 per cesarean delivery [15]. However, a 2015 analysis found an average cost per birth of $106 USD in Kenya and $75 USD in Uganda [16]. The estimated cost of care for low-birthweight infants is estimated to be $514 per infant [15]. Understanding costs associated with the PTBi intervention in Kenya and Uganda may add to the knowledge base and support future programmatic and resource planning.

The goals of our study were 1) to calculate the implementation-level (ie, non-research) costs associated with the PTBi program in Kenya and Uganda, as observed in the PTBi study, and 2) to estimate costs associated with public-sector planning, implementation, and maintenance of the package in a non-study, hypothetical, Ministry of Health (MOH) model. Modeled costs in a non-study/MOH setting may be more relevant to decision-makers to inform resource allocation; as such, our analysis focuses primarily on estimates for the cost of public sector implementation. To our knowledge, this is among the first studies to assess the costs of a complex, multicomponent intrapartum quality improvement package in sub-Saharan Africa.

Methods

Overview

Through the PTBi study in Kenya and Uganda, we estimated the per-birth (term and preterm; live born and stillborn; in $2020 US) costs associated with the PTBi package of interventions compared to a more minimal active control. Using a healthcare system perspective, we analyzed incremental costs–costs that could be added to the current cost of obstetric care–associated with the PTBi package and the active control using an activity-based micro-costing approach. Our analysis includes incremental costs for all deliveries with an estimated gestational age of 28 weeks or higher, regardless of delivery outcome (ie, both live births and stillbirths). We calculated costs for both the package of interventions and active control through two scenarios: Implementation as observed in the PTBi study (excluding purely research costs) and a hypothetical model led by the Kenyan and Ugandan Ministries of Health.

Study setting

The PTBi study was a cluster-randomized controlled trial implemented in 17 health facilities in Migori County, Kenya and 6 health facilities in Busoga Region, Uganda [17] from 2016 to 2019. The study was approved by the Kenya Medical Research Institute, Makerere University School of Public Health, and the University of California, San Francisco Institutional Review Boards. Detailed descriptions of the study have been published previously [17, 18]. This cost analysis did not involve direct or indirect interaction with the patients (mothers or infants) or review of patient records/information from the effectiveness portion of the main PTBi study.

In the PTBi study, the comprehensive PTBi package of interventions was demonstrated to reduce the odds of stillbirth and early neonatal death among premature infants by 34% by improving intrapartum and immediate newborn care [11]. Thorough reviews of the implementation of the PTBi package found that the full package and its components were feasible, replicable, flexible in the face of challenges, and fostered relationships within and among health facilities [18, 19]. It was also found to have strong potential for “local ownership”, due in part to the process of intervention selection and implementation [18], as detailed in the following section.

PTBi intervention package and control

The PTBi package of interventions included 4 components: data strengthening, a modified WHO Safe Childbirth Checklist (mSCC), PRONTO simulation and team training [20, 21], and Quality Improvement (QI) Collaboratives [11, 17]. In the PTBi study, selection of the interventions was based on published literature reviews, robust assessments of health facilities in Kenya and Uganda, and several meetings with local stakeholders [11, 17, 18, 22]. Through this process, PTBi study investigators found that while global and local national guidelines were in place in Kenya and Uganda, actual care delivered in facilities was not consistently per these guidelines. Further, measuring and tracking of health data and quality indicators were deficient, and health workers often lacked the clinical skills and training to identify and manage obstetric emergencies and preterm birth. Based on this review, the 4 PTBi package components were identified as mutually-reinforcing quality improvement interventions that were aligned with WHO and local guidelines and could be tailored to address quality of clinical obstetric and neonatal care, with a focus on preterm birth [11, 17, 18]. While each of the intervention components had previously been evaluated individually for their effectiveness in other settings, some findings have been mixed, the 4 components had not been bundled, and none of the interventions were in place in Kenya or Uganda [17, 18].

From a clinical care perspective, this PTBi package was designed to reinforce evidence-based practices for maternal and newborn care around the time of birth, including provision of antenatal corticosteroids; immediate skin-to-skin contact and kangaroo mother care (KMC); newborn resuscitation; and appropriate feeding [11, 18]. It was also designed to strengthen provider skills in maternity and neonatal care, and to promote teamwork and communication among health workers [17]. Beyond direct clinical care, the package also aimed to promote data use for decision making–including gestational age estimation and tracking of key indicators–and other quality improvement strategies.

The active control was also selected through the above-mentioned process. Data strengthening and the WHO mSCC have been outlined by WHO and other local and national organizations as important interventions to better track and compare health outcomes, help improve care during childbirth, and promote the use of evidence-based practices [1, 9, 23, 24]. Intervention components are described further in Table 1, and detailed descriptions of the intervention components are available in study publications [11, 17, 18].

thumbnail
Table 1. Description of PTBi package and control components.

https://doi.org/10.1371/journal.pone.0287309.t001

In the PTBi study, this package of four interventions was compared to a more minimal active control that included data strengthening and mSCC [17]. Activities and resources for data strengthening and mSCC are the same between the package and control; however, because all four components of the PTBi package are complementary and mutually reinforcing, there is efficiency in delivering all 4 interventions in the package versus 2 in the control. For example, transportation related to monitoring of activities for all 4 interventions would be similar to/the same as that for data strengthening and mSCC alone. We chose to cost the active control for the following reasons: the PTBi package and active control were compared in the effectiveness study; the package and control were each implemented into, and as a supplement to, the existing current practice; and because the active control components are aligned to WHO and local recommendations.

As described above, the selection, design, and implementation of the PTBi package of interventions and control was a collaborative effort by researchers and clinicians in Kenya, Uganda, and the US, and was based on specific in-country needs [18]. Individual components of the package– DS, a mSCC, simulation and team training through PRONTO International, and QICs– were tailored to stakeholder-identified priorities and key clinical issues that impact preterm survival in each country [18]. Implementation in other regions, facilities, or settings and for other obstetric and neonatal needs may differ from the PTBi study.

Modeling implementation scenarios and framework

In our analysis, we costed implementation of the PTBi package and control under two scenarios: 1) implementation-level costs (non-research) as observed in the PTBi study (Scenario 1) and 2) costs for a hypothetical model of implementation by local Ministries of Health (Scenario 2). In Scenario 1, we took care to identify and exclude activities that were exclusively associated with research activities, such as data collection that would only be undertaken in a study setting. However, even after excluding activities and costs associated with research, clinical studies typically reflect prices and intensity levels that are not representative of costs associated with program implementation in the public sector. Because of this, we developed our second scenario to model hypothetical implementation (“ownership”) and management by local MOHs in a non-research setting, using mostly public-sector inputs.

In both Scenarios, we calculated costs incremental to the cost of providing intrapartum maternal and newborn care; as such, our findings represent the additional costs necessary to implement the PTBi package or control within the current level of care provided in our analysis setting. Further, costs in this analysis focus on those related to the direct planning, implementation, and maintenance of the PTBi package of interventions; costs do not include indirect or downstream changes in medical care. For example, we included costs for resources (paper, printing) and health worker time completing the mSCC for each delivery, but we did not include the costs associated with changes in the usage of medications or devices following PTBi package implementation.

To understand the costs for both scenarios, we created an “implementation framework” that documented the phases, activities, and key personnel involved in each PTBi package component for both scenarios. Within this framework, we also outlined activities that are shared between intervention components or are not easily distinguishable (eg, initial program planning and stakeholder meetings, and training sessions that encompass several interventions). Descriptions of both scenarios are outlined in Table 2.

thumbnail
Table 2. Description of costing scenarios and major inputs.

https://doi.org/10.1371/journal.pone.0287309.t002

In our framework, we outlined three critical and distinct implementation phases associated with both scenarios: 1) program design and planning (6–9 months), 2) high-intensity initial implementation (12–18 months) and 3) annual maintenance. Phases and timing of activities are depicted in Fig 1 and are based on discussions with PTBi study team members. More details on timing of activities for Scenario 2 is available in S1 Fig.

thumbnail
Fig 1. Implementation (non-research) phases and activities (Scenarios 1 and 2).

DS: data strengthening; mSCC: modified WHO safe childbirth checklist; QI: Quality improvement; TOT: training of trainers. *Includes activities related to DS, mSCC, PRONTO, and QI. Fig 1 describes the phases of implementation for the PTBi package of interventions. Each phase includes timing and critical activities to be undertaken throughout the phase. This applies to both Scenarios 1 and 2, though annual maintenance costs were not observed or included in Scenario 1.

https://doi.org/10.1371/journal.pone.0287309.g001

Activity-based micro-costing

For both scenarios, we used empirical bottom-up activity-based micro-costing to estimate incremental costs associated with activities within each phase of the PTBi package and control. Key activities included stakeholder meetings to discuss and adapt intervention components; component-specific training (initial training and refresher training) for program-based and facility-based staff and clinicians; program management by MOH MNCH staff; and time spent by program and facility staff “using” the interventions (eg, participating in PRONTO simulations in facilities, attending QIC meetings, and filling out the mSCC for each birth). Activities are further described in Table 1 and Figs 1 and S1.

Our main cost input categories included costs associated with personnel, consumable goods, capital equipment, travel, and program office/conference space. Input categories are outlined in Table 2 and are described in detail in the following sections. Scenario 2 select unit prices for each major input category are in Table 3; unit prices for Scenario 1 are available upon reasonable request.

thumbnail
Table 3. Select input prices and units for PTBi package of interventions (Scenario 2).

https://doi.org/10.1371/journal.pone.0287309.t003

We tabulated total costs per country, phase, intervention component, health/birth facility, and birth (regardless of gestational age, including live births and stillbirths) and report in $US 2020 (cost data were adjusted for inflation where appropriate for all inputs). When prices and costs were derived from PTBi study invoices and ledgers (eg, invoices for clinic supplies used for PRONTO simulations or office supplies for program implementation), in-country prices and costs were converted from Kenyan Shillings and Ugandan dollars into $US using the exchange rate for the month of the invoice/ledger. Where prices and costs were derived from annual statements or estimates (eg, MOH personnel salaries), costs were converted using the average exchange rate for the year of the estimate. All prices and costs were then converted to 2020 $US to account for inflation using annual within-country inflation rates.

Costs for each phase were tabulated for all activities within each phase; costs per facility were tabulated by dividing total country costs by the number of facilities in each country. Costs per birth were tabulated by dividing total country costs by the total number of births that occurred in facilities over the duration of implementation (monthly facility delivery volumes for all facilities, multiplied by the 27 of months of implementation [average 15 months of initial high-intensity implementation and 12 months maintenance]).

Key personnel.

We estimated Scenario 1 program-based personnel costs as observed in the PTBi study. Data sources included salary records, budget and expenditure records, timelines, and other project records for the PTBi study; such costs are representative of the private sector. For Scenario 1 facility-based personnel and all non-consultant Scenario 2 personnel, salary data were extracted from published pay scales for program-based, specialist, and facility-based workers that would be involved in the planning and implementation of the interventions in the public sector [25, 26]. We linked the pay scales for each worker cadre to job descriptions, required credentials, and roles and responsibilities of those cadres in recruitment, hiring, and employment guidelines [2730]. Select personnel salaries and units are in Table 3.

We quantified personnel effort for each facility-based cadre and program-based staff in both scenarios using study records and interviews with program experts. We estimated the amount of time spent on each intervention component and activity, taking care to exclude activities specifically associated with clinical research. Personnel time included time spent by study coordinators and program managers to lead, implement, and monitor intervention activities; clinical specialists and supervisors providing oversight and expertise on specific topics; PRONTO training specialists and Quality Improvement experts (specialist consultants); data specialists and health records clerks receiving training and tracking indicators; and health workers receiving training in facilities and participating in intervention activities. We estimated that, on average, between 8 and 15 health workers would receive at least some level of training or exposure to the PTBi intervention and control per facility [22].

In the PTBi study, new clinical providers were not added to the health facilities within the study; as such, we assumed that new clinical providers would not be hired specifically for the PTBi intervention or control in our cost analysis, and thus did not include costs associated with new clinical providers in either Scenario 1 or 2. We did, however, account for the need for additional training regarding PTBi intervention and control components for health workers who are hired to replace other staff as part of the natural hiring process (ie, hiring staff to replace others due to turnover, promotion, retirement, or other circumstances).

In Scenario 1, PTBi study staff undertook all program management activities; via interviews, we estimated the percentage of personnel time spent on general program activities and for each PTBi package component. In Scenario 2, we assumed that program management would be undertaken by local MOH personnel working on existing MNCH platforms/programs. However, we also estimated that two new roles would need to be created to implement PTBi activities: 1) a data specialist role that would be responsible for overseeing DS and mSCC activities, and a 2) QI specialist role that would be responsible for overseeing PRONTO and QI activities. We used MOH organograms to understand reporting, hierarchies, and appropriate pay scales, and supervision for these roles. We included onboarding and supervisory time in our personnel cost estimates for these roles, and we assumed that one specialist within each role would be needed for every 5–8 facilities, depending on facility delivery volume (eg, one person for 4–6 large facilities or 8–9 small facilities).

PRONTO Nurse Mentors are experts in maternity and neonatal care and are essential to the successful implementation of PRONTO activities [11, 18, 21, 31, 32]. In the PTBi study (Scenario 1), several mentors were hired in each country to undertake all PRONTO activities, including traveling to each study facility to lead simulations and team training activities, and provide bedside clinical mentoring [11, 17, 18]. In Scenario 2, we modeled the training of one nurse, midwife, or medical officer within each health facility to undertake facility-based PRONTO activities; this model was developed based on discussions with in-country experts with knowledge of sustainable practices in MOH training activities. Costs associated with training nurse mentors (Training of Trainers) was included in both scenarios.

In the PTBi study, the PRONTO and QI activities required the intensive involvement of private-sector experts and specialists for the design and implementation of simulation/team training and QI activities [18]. Scenario 1 costs reflect this involvement. Scenario 2 assumes minimal adaptation (not full development) of these existing interventions. Therefore, Scenario 2 costs reflect less intensive consultation with experts, their involvement in stakeholder meetings, their time spent adapting existing interventions (assuming minimal adjustments), and their role in training of trainers.

Consumable and capital goods.

Consumable goods in our scenarios included office and simulation supplies used for the package of interventions and control. Office supplies included printed training materials, checklists (mSCC), and goods used for QI activities (eg, paper, markers, and materials for QICs). Simulation supplies included syringes, gauze, dipsticks, and other clinical goods used for PRONTO training activities. Unit prices and quantities for all consumable goods for Scenario 1 were extracted from PTBi study budget and expenditure records; Scenario 2 unit prices were extracted from published price lists and order forms [33, 34], and quantities were based on those from the PTBi study. Select unit prices are in Table 3.

We identified the capital goods (including quantities and replacement frequencies) required to implement and maintain interventions through a review of study records and interviews with study staff. Capital goods in both scenarios included laptops and office furniture for program coordinators/managers, projectors for training and ongoing QICs, and the PRONTOPackTM simulation training kit and neonatal resuscitation mannequins for PRONTO activities. We assumed replacement laptops, projectors, and PRONTOPacksTM every 3–5 years. Because Scenario 2 assumes integration of the PTBi package into existing MNCH programming, we assumed that laptops and office furniture would only be purchased for the new roles described in the personnel section. The value of capital office goods for existing MNCH program staff was prorated based on the percent effort used to incorporate the package of interventions into the local MNCH platform. We also assumed that all PRONTOPackTM components in Scenario 2 would be purchased in-country. Unit prices and quantities in Scenario 1 were extracted from study budget and expenditure files; unit prices for Scenario 2 capital goods were extracted from MOH price lists for most items, with PTBi study prices for specialized goods that were not available in MOH price lists.

While the PTBi package of interventions does not include the use of clinical supplies or capital goods that are not already part of essential maternity and newborn care in-country, facility assessments in the PTBi study found that some facilities did not have all the necessary goods required to provide care per guidelines [22]. As such, for both scenarios, we included costs for some basic clinical goods, such as neonatal bag valve masks and tubing, prorated based on the percentage of PTBi study facilities that were lacking these goods at baseline.

Travel and other costs.

We estimated costs for transportation in each scenario, including vehicle costs for travel to and from facilities for regular intervention activities. We also estimated consultant travel costs, which included limited regional and international airfare, ground transportation, and lodging for multi-day activities, such as training of PRONTO mentors. Travel costs were calculated in Scenario 1 were using on observed prices and quantities in study documents and invoices. Scenario 2 costs were calculated using local price estimates collected in-country by local health economists in March 2020; travel units were based on our analysis of program activities that would require travel for consultants, program staff, and limited travel for select in-facility workers.

We estimated costs associated with program office space based on study office rent in the PTBi study (Scenario 1; prorated for the average amount of time personnel within those offices spent working on implementation activities) and hotel conference rooms rented for major meetings and training sessions, including food and beverages provided during offsite meetings. In Scenario 2, we estimated the value of program office space that would be used within existing MOH program offices for program-based staff and limited use of conference rooms for annual, offsite MNCH planning meetings, prorated for the percentage of the meeting that would be used to discuss PTBi activities. Scenario 2 assumes that all training activities, PRONTO sessions, and QICs would take place within existing space in health facilities; therefore we did not include health facility space costs for these activities.

Outcomes and assumptions

For Scenario 1 (costs observed in the PTBi study), we calculated total incremental costs associated with package implementation in 9 health facilities in Kenya and 4 in Uganda; we also calculated total costs for 8 control facilities in Kenya and 2 control facilities in Uganda. In our hypothetical MOH model (Scenario 2), we calculated total incremental costs associated with the package of interventions and active control in all 17 facilities in Kenya and all 6 facilities in Uganda. For both scenarios, we calculated total program costs, cost per facility, and cost per birth for each the package of interventions and control. We also tabulated the percent contributions of by input category and intervention component for the PTBi package.

Costs per facility were based on the number of facilities in each country. Cost per birth was based on number of facilities, birth volumes per month per facility, and duration of implementation. Kenyan facilities had volumes of 30–360 deliveries per month, with most facilities having fewer than 100 deliveries per month. Ugandan facilities had volumes of 90–585 deliveries per month, with most facilities having more than 150 deliveries per month. Total costs included 27 months of in-facility activities across phases 2 and 3.

For Scenario 2, we relied on several key assumptions, which were identified through interviews with in-country stakeholders. For example, we assumed that all intervention and control activities would be fully integrated within existing MOH MNCH activities. As such, we also assumed that majority of activities for each phase would be managed and implemented by MOH MNCH program leadership and staff within each geographic area. Other key assumptions and rationale are described in Table 4, and more detailed assumptions are in S2 Fig.

thumbnail
Table 4. Scenario 2 (Hypothetical MOH model) assumptions and rationale.

https://doi.org/10.1371/journal.pone.0287309.t004

Sensitivity analyses

We conducted sensitivity analyses to understand the impact of uncertainty in Scenario 2, including higher intensity of program planning activities and intervention adaptation; higher intensity of QI and PRONTO activities, including more frequent refresher training; hiring/use of “centralized” nurse mentors instead of facility-based nurse mentors; and differences in facility size and worker composition within facilities.

Results

Scenario 1

For Scenario 1, we calculated costs associated with package implementation in 9 health facilities in Kenya and 4 in Uganda, and 8 control facilities in Kenya and 2 control facilities in Uganda, as observed in the PTBi study. Scenario 1 total program costs by country, input type, intervention, facility, and birth are in Table 5. Total non-research implementation costs for the PTBi study was $2.23 million, including intervention and control activities and inputs; PTBi package costs were $1.85 million in total with a cost per birth of $48.13 per birth in Kenya, and cost per birth of $17.19 in Uganda. PTBi study control costs per birth were $15.22 in Kenya and $6.13 in Uganda.

thumbnail
Table 5. PTBi program incremental costs in Kenya and Uganda (2020 $US).

https://doi.org/10.1371/journal.pone.0287309.t005

Scenario 2

In the hypothetical MOH model (Scenario 2), we calculated costs associated with package intervention and control in all 17 facilities in Kenya and all 6 facilities in Uganda. Scenario 2 total program costs by country, input type, intervention, facility, and birth are in Table 5 above. In Scenario 2, total PTBi intervention costs were $0.86 million in Kenya ($50,497 per facility, $23.91 per birth) and $0.28 million in Uganda ($46,871 per facility, $5.47 per birth). Cost per birth in Phase 3 alone–representing steady-state costs after high-intensity implementation–was $16.36 in Kenya and $3.47 in Uganda.

Scenario 2 costs per birth were 50% lower in Kenya and 68% lower in Uganda compared to Scenario 1. In both scenarios, personnel time made up the majority of total program costs. In Scenario 2, 88% and 72% of costs were for personnel in Kenya and Uganda, respectively. Cost by input type, intervention component, and phase for Scenario 2 are in Fig 2 (Kenya) and Fig 3 (Uganda). Detailed costs by input type, phase, and package component are in S1 Table.

thumbnail
Fig 2. Scenario 2 costs by input type and PTBi intervention component in Kenya (2020 $US).

DS: data strengthening; mSCC: modified WHO safe childbirth checklist; PTBi: Preterm Birth Initiative; QI: Quality improvement. Fig 2 depicts Scenario 2 (Hypothetical MOH Model) costs by phase and intervention component in Kenya for the PTBi package of interventions. Phases include: 1) program design and planning prior to in-facility implementation; 2) high-intensity initial implementation and training within facilities; and 3) annual maintenance, including refresher training and program monitoring.

https://doi.org/10.1371/journal.pone.0287309.g002

thumbnail
Fig 3. Scenario 2 costs by input type and PTBi intervention component in Uganda (2020 $US).

DS: data strengthening; mSCC: modified WHO safe childbirth checklist; PTBi: Preterm Birth Initiative; QI: Quality improvement. Fig 3 depicts Scenario 2 (Hypothetical MOH Model) costs by phase and intervention component in Uganda for the PTBi package of interventions. Phases include: 1) program design and planning prior to in-facility implementation; 2) high-intensity initial implementation and training within facilities; and 3) annual maintenance, including refresher training and program monitoring.

https://doi.org/10.1371/journal.pone.0287309.g003

Among implementation phases in Scenario 2, Phase 1, 2, and 3 activities comprised 26%, 44%, and 30% of total costs in Kenya, respectively; in Uganda this was 38%, 37%, and 25%, respectively (S1 Table). Among intervention components, general/non-allocable costs were 24% of total costs in Kenya and 27% of total costs in Uganda; PRONTO costs were 44% of total costs in each Kenya and Uganda. Additional details on the relative contributions of each intervention component are in Table 6.

thumbnail
Table 6. Contributions of individual intervention components to total costs in Scenario 2.

https://doi.org/10.1371/journal.pone.0287309.t006

Sensitivity analyses

Sensitivity analyses focused on areas of uncertainty in Scenario 2, identified through discussions with PTBi study team members and experts in the local healthcare systems in Kenya and Uganda. Key sensitivity analyses included differences in the intensity of program activities, including a higher level of program adaptation during Phase 1, more frequent in-facility trainings in Phase 2, and a different “model” for PRONTO Nurse Mentors (hiring NMs to travel to each facility, versus having NMs embedded within each facility). We also analyzed the effect of differences in health facility volumes between Kenya and Uganda to understand the impact of PTBi study facility type/volume between the two countries. Finally, we estimated total costs when excluding costs associated with facility-based workers.

Results from our sensitivity analyses are outlined in Table 7, reported in cost per birth and percent different from base case. Differences in PRONTO Nurse Mentor models resulted in a 4% increase ($24.92 vs $23.91) in cost per birth in Kenya and a 10% increase ($6.02 vs $5.47) in Uganda. A higher intensity of intervention adaptation activities resulted in increases in per-birth costs that were 8% higher in Kenya ($25.82 vs $23.91) and 22% higher in Uganda ($6.68 vs $5.47). Excluding in-facility health worker costs (except for in-facility Nurse Mentor training) resulted in costs that were 55% lower ($10.76 vs $23.91) in Kenya and 18% lower in Uganda ($4.48 vs $5.47) (see S2 Table for costs by input type, phase, and component). Implementation in fewer, higher-volume facilities in Kenya decreased total cost by 50% versus the base case ($11.90 vs $23.91); implementation in a greater number of lower-volume facilities in Uganda increased total costs by 49% vs the base case ($8.11 vs $5.47).

thumbnail
Table 7. Scenario 2 sensitivity analyses–variation in cost per birth in Kenya and Uganda, testing several assumptions.

https://doi.org/10.1371/journal.pone.0287309.t007

Discussion

Using a combination of quantitative and qualitative methods, we estimated costs associated with the implementation of the PTBi package of interventions in two distinct scenarios: one that represents non-research costs observed in a study setting and one that represents hypothetical costs of an MOH-led model. We found that package costs were $48.13 per birth in Kenya in the PTBi Study and $17.19 per birth in Uganda (Scenario 1, all phases). In Scenario 2, costs per birth were $23.19 in Kenya and $5.47 in Uganda (all phases); steady-state costs for phase 3 alone were $16.36 per birth in Kenya and $3.47 in Uganda.

Our study adds to the limited evidence on the costs associated with interventions to improve preterm survival in lower-resource settings, where the burden of preterm birth is highest [1]. It is becoming increasingly accepted that complementary, multicomponent interventions—like the PTBi package—have more promise for preterm birth than interventions implemented individually [35]. Understanding the costs of intervention packages requires complex costing methods, including representation of shared costs for integrated programming and shared activities, understanding of potential synergies and efficiencies, and other contextual considerations, which we included in our study.

Due to limited data regarding the costs of interventions for preterm birth in lower-resource settings, differences in interventions, and differences in setting and results reporting, comparisons in costs between our analysis and other cost analysis in MNCH are limited. However, our input categories and unit costs are consistent with those in other cost analyses in similar settings for intrapartum care interventions [15, 36, 37], which may indicate that overall results are consistent with other studies and may be generalizable in Kenya and Uganda. Additionally, a 2015 analysis of facility-based health care costs found an average cost per birth of $106 USD in Kenya and $75 USD in Uganda [16]. After accounting for inflation, adding the PTBi package would increase the cost per delivery/birth by about 15% in Kenya and 5% in Uganda in Phase 3, unadjusted for offsetting and incurred costs of care for surviving preterm infants. This should be weighed against the high level of effectiveness of the program. Understanding the cost-effectiveness of the PTBi package of interventions—including downstream costs—and other similar multicomponent quality improvement packages, is an important area of future study.

Scenario 2 costs were substantially lower than Scenario 1 in both countries. This reflects differences in personnel pay, intensity of activities, and number of facilities in the two scenarios: Scenario 1 program-based staff were paid private-sector salaries/benefits; Scenario 1 includes time spent on full program and intervention development (higher intensity); and Scenario 2 includes more facilities (greater efficiency per facility and birth). The difference in Scenario 2 total costs and cost per birth are also due to differences in birth volumes in PTBi study facilities in Kenya and Uganda, which we used for estimating facility-level costs in both scenarios: In Kenya, most study facilities were smaller, lower-volume with a higher health-worker-to-birth ratio; in Uganda, most study facilities were larger, with higher birth volumes and a lower health-worker-to-birth ratios. As we found in sensitivity analyses, implementation in larger, high-volume facilities (vs lower volume) results in greater efficiencies and lower cost per birth. This may be helpful for resource allocation, as more complicated deliveries often are referred to larger facilities when possible. However, that does not warrant the exclusion of smaller facilities from intervention activities. Maximizing impact and efficiency is relevant for resource allocation, but decision-making should also be based on burden and other needs.

In Scenario 2, total costs in Uganda were 33% of those in Kenya. This reflects the differences in the Kenyan and Ugandan economies and health care systems—Kenya has transitioned to being a lower-middle income country (GDP per capita in 2020: $1,838 USD), while Uganda remains a low-income country (GDP per capita in 2020: $817 USD) [38, 39]. As we observed through in-country data collection, public-sector health program officers and health workers in Kenya receive salaries that are two- to five-times higher than those in Uganda.

In our study, personnel were the single largest cost input, which reflects the fact that the PTBi package uses training and monitoring to improve health worker skills, including the identification and management of preterm birth. The package does not require the purchasing or maintenance of new high-tech goods or the implementation of new clinical interventions, which helps account for the relatively low estimated costs for consumable and capital goods. Our analyses included time spent by consultants who provide specialized guidance and training in clinical care and quality improvement. In Scenario 2, the total cost for specialist consultants was relatively comparable between Kenya and Uganda; this is due to anticipated similar pay for specialist consultants between the two countries.

Because we aimed for a robust economic evaluation, we included in-facility health worker time spent in training and QI activities in our analysis. While this allows us to understand the total value of the PTBi package of interventions and the opportunity costs of implementing health interventions, it is important to note that findings from our cost analysis is neither a budget estimate nor a financial analysis and should not be used as such. Removing costs associated with health professional time, given they are salaried employees already working in facilities, resulted in 20–50% lower total costs. Because our analysis includes time spent by existing health workers—the total “new” costs incurred by the PTBi package of interventions is potentially lower than estimated in our analysis.

PRONTO had the single largest cost among intervention components. This is in part due to the limited local experience with highly realistic simulation and team training offered by PRONTO, including training of trainers, which is led by US-based experts in simulation and team-based training. While the costs associated with PRONTO are substantial compared to that of the mSCC, for example, they represent the intensive, skills-based, capacity-building nature of the PRONTO intervention. PRONTO has been demonstrated in several studies to significantly improve maternal and neonatal care [31, 40, 41] and has been met with much enthusiasm by health workers and local stakeholders in the PTBi study and other evaluations [21, 42]. However, success of the PRONTO intervention relies on continued use of simulations and team training activities—which we modeled in our analysis—and long-term investments in adding new curricular components and training of national-level cohorts of master trainers (not observed in the PTBi study or included in our analysis). Larger scale implementation of the PRONTO intervention is underway in India and other countries in sub-Saharan Africa; further analysis of PRONTO at scale and using different nurse mentor models is an important area of further study.

Our analysis has important limitations. First, since Scenario 2 is hypothetical, we did not directly observe costs for this scenario, and we relied on several assumptions, including heavy use of public sector personnel and minimal adaptation of package interventions. Actual in-country costs may differ, as outlined in our sensitivity analyses, and the effectiveness of the PTBi package of interventions and active control in Scenario 2 may differ from the PTBi study. We relied on publicly available information from in-country governments regarding public-sector health worker pay, especially in Scenario 2; any differences between publicly available salary information and actual health worker pay may affect our estimates. However, our assumptions were based on information from in-country researchers, health workers, and healthcare system experts. Changes in key model assumptions resulted increases in cost per birth of 3–22% in sensitivity analyses.

Second, our costs represent implementation in a county-specific context in Kenya and a region-specific context in Uganda over the course of 3 years. Our two scenarios and their associated costs are also based on information from the specific facilities included in the PTBi study. Scaling at a national level may result in modest efficiencies, though costs and resources required to plan, implement, and maintain the PTBi package may differ in other facilities. Third, the PRONTO implementation model in our study is focused on preterm birth and select obstetric emergencies. Broader PRONTO models covering more clinical situations would cost more.

Strengths of our study include its high level of detail, robust data collection, and in-depth analysis of costs by activity, input type, phase, and intervention component. Our modeling of scenarios provides insights into both private-sector and public-sector costs, and our analysis approach can be applied to other settings. While costs in our study are specific to local contexts in Kenya and Uganda, our methods may be of use for understanding costs in other geographic settings. Additionally, the analysis approach and our use of national data for health worker composition and costs may be applied to other clinical situations related to obstetric and neonatal care, using the same intervention components (DS, mSCC, PRONTO, and QI). Indeed, we found that costs for PTBi intervention maintenance were lower than for previous phases, which indicates that PTBi may be a suitable platform for the efficient inclusion additional training and curricula in other areas of emergency obstetric, intrapartum, and neonatal care. This is another important area that should be explored in future economic evaluations.

Conclusions

We found that the PTBi package may be implemented and maintained by the public sectors in Kenya and Uganda at a relatively low cost per birth. While our study is limited by its assumptions and specific setting and context, our findings add to the limited data regarding the total economic costs associated with interventions to improve preterm survival in lower-resource settings. Such findings may help inform future policy and implementation of this critical, life-saving package of interventions.

Supporting information

S1 Fig. Detailed PTBi phases and activities for Scenario 2 cost analysis.

S1 Fig provides detailed timing, phasing, and activities for each of the PTBi package intervention components through all phases of program planning, initial high-intensity implementation, and maintenance. All activities are based on activities for and findings from the PTBi study. All activities and durations were used for the Scenario 2 cost analysis. Control costs included only activities for data strengthening and the WHO modified safe childbirth checklist, along with general program activities that pertained to these two components. Intervention package costs included all activities for data strengthening, mSCC, PRONTO, Quality Improvement, and general program activities.

https://doi.org/10.1371/journal.pone.0287309.s001

(XLSX)

S2 Fig. Detailed PTBi costing assumptions for Scenario 2.

Detailed model assumptions for PTBi costing Scenario 2.

https://doi.org/10.1371/journal.pone.0287309.s002

(XLSX)

S1 Table. Scenario 2 total PTBi package costs by phase and intervention component.

https://doi.org/10.1371/journal.pone.0287309.s003

(XLSX)

S2 Table. Scenario 2 PTBi package costs excluding facility-based personnel.

https://doi.org/10.1371/journal.pone.0287309.s004

(XLSX)

Acknowledgments

We thank the Bill & Melinda Gates Foundation for their generous funding, and our program officers for their guidance, encouragement, and support. We would like to thank those who were critical to the PTBi study, including the mothers and infants who participated in the study; the healthcare providers and other staff working in study facilities; and the Ministry of Health officials and advisory boards who supported and guided implementation and research efforts. Furthermore, we thank the research teams and implementors at Makerere University, Kenya Medical Research Institute, UCSF, and PRONTO International who provided invaluable insights and input into this cost analysis specifically. We are grateful for the members of the Preterm Birth Initiative East Africa External Advisory Committee, our colleagues at the California Preterm Birth Initiative, and the members of our joint Strategic Advisory Board.

References

  1. 1. WHO | Born too soon: the global action report on preterm birth [Internet]. WHO. World Health Organization; [cited 2022 Feb 14]. Available from: https://www.who.int/reproductivehealth/publications/maternal_perinatal_health/9789241503433/en/
  2. 2. Levels and trends in child mortality [Internet]. UNICEF DATA. 2021 [cited 2022 Feb 14]. Available from: https://data.unicef.org/resources/levels-and-trends-in-child-mortality/
  3. 3. Stillbirths and stillbirth rates [Internet]. UNICEF DATA. [cited 2022 Feb 14]. Available from: https://data.unicef.org/topic/child-survival/stillbirths/
  4. 4. Blencowe H, Cousens S, Oestergaard MZ, Chou D, Moller AB, Narwal R, et al. National, regional, and worldwide estimates of preterm birth rates in the year 2010 with time trends since 1990 for selected countries: a systematic analysis and implications. Lancet Lond Engl. 2012 Jun 9;379(9832):2162–72. pmid:22682464
  5. 5. Global, regional, and national disability-adjusted life-years (DALYs) for 359 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet Lond Engl. 2018 Nov 10;392(10159):1859–922. pmid:30415748
  6. 6. Cheah IGS. Economic assessment of neonatal intensive care. Transl Pediatr. 2019 Jul;8(3):246–56. pmid:31413958
  7. 7. GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Lond Engl. 2017 Sep 16;390(10100):1151–210.
  8. 8. Newnham JP, Dickinson JE, Hart RJ, Pennell CE, Arrese CA, Keelan JA. Strategies to Prevent Preterm Birth. Front Immunol. 2014 Nov 19;5:584. pmid:25477878
  9. 9. WHO Recommendations on Interventions to Improve Preterm Birth Outcomes [Internet]. Geneva: World Health Organization; 2015 [cited 2022 Feb 14]. (WHO Guidelines Approved by the Guidelines Review Committee). Available from: http://www.ncbi.nlm.nih.gov/books/NBK321160/
  10. 10. Bhutta ZA, Das JK, Bahl R, Lawn JE, Salam RA, Paul VK, et al. Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost? The Lancet. 2014 Jul 26;384(9940):347–70. pmid:24853604
  11. 11. Walker D, Otieno P, Butrick E, Namazzi G, Achola K, Merai R, et al. Effect of a quality improvement package for intrapartum and immediate newborn care on fresh stillbirth and neonatal mortality among preterm and low-birthweight babies in Kenya and Uganda: a cluster-randomised facility-based trial. Lancet Glob Health. 2020 Aug 1;8(8):e1061–70. pmid:32710862
  12. 12. de la Perrelle L, Radisic G, Cations M, Kaambwa B, Barbery G, Laver K. Costs and economic evaluations of Quality Improvement Collaboratives in healthcare: a systematic review. BMC Health Serv Res. 2020 Mar 2;20(1):155. pmid:32122378
  13. 13. Tancred T, Manzi F, Schellenberg J, Marchant T. Facilitators and Barriers of Community-Level Quality Improvement for Maternal and Newborn Health in Tanzania. Qual Health Res. 2017 Apr;27(5):738–49. pmid:27022034
  14. 14. Banke-Thomas A, Abejirinde IOO, Ayomoh FI, Banke-Thomas O, Eboreime EA, Ameh CA. e-income countries from a provider’s perspective: a systematic review. BMJ Glob Health. 2020 Jun;5(6):e002371. pmid:32565428
  15. 15. Mori AT, Binyaruka P, Hangoma P, Robberstad B, Sandoy I. Patient and health system costs of managing pregnancy and birth-related complications in sub-Saharan Africa: a systematic review. Health Econ Rev. 2020 Aug 15;10(1):26. pmid:32803373
  16. 16. Ccess A, Ottlenecks B, Osts C, Quity E. Assessing Facility Capacity, Costs of Care, and Patient Perspectives. B OTTLENECKS.:78.
  17. 17. Otieno P, Waiswa P, Butrick E, Namazzi G, Achola K, Santos N, et al. Strengthening intrapartum and immediate newborn care to reduce morbidity and mortality of preterm infants born in health facilities in Migori County, Kenya and Busoga Region, Uganda: a study protocol for a randomized controlled trial. Trials. 2018 Jun 5;19(1):313. pmid:29871696
  18. 18. Namazzi G, Achola KA, Jenny A, Santos N, Butrick E, Otieno P, et al. Implementing an intrapartum package of interventions to improve quality of care to reduce the burden of preterm birth in Kenya and Uganda. Implement Sci Commun. 2021 Jan 28;2(1):10. pmid:33509293
  19. 19. Achola KA, Kajjo D, Santos N, Butrick E, Otare C, Mubiri P, et al. Implementing the WHO Safe Childbirth Checklist modified for preterm birth: lessons learned and experiences from Kenya and Uganda. BMC Health Serv Res. 2022 Mar 3;22(1):294. pmid:35241076
  20. 20. International P. WHAT WE DO [Internet]. PRONTO International. [cited 2022 Feb 14]. Available from: https://prontointernational.org/what-we-do/
  21. 21. Dettinger JC, Kamau S, Calkins K, Cohen SR, Cranmer J, Kibore M, et al. Measuring movement towards improved emergency obstetric care in rural Kenya with implementation of the PRONTO simulation and team training program. Matern Child Nutr. 2018 Feb;14:e12465. pmid:29493898
  22. 22. Morgan MC, Spindler H, Nambuya H, Nalwa GM, Namazzi G, Waiswa P, et al. Clinical cascades as a novel way to assess physical readiness of facilities for the care of small and sick neonates in Kenya and Uganda. PloS One. 2018;13(11):e0207156. pmid:30462671
  23. 23. Standards and tools to strengthen health services data—WHO [Internet]. [cited 2022 Oct 21]. Available from: https://www.who.int/data/data-collection-tools/standards-and-tools-to-strengthen-health-services-data
  24. 24. WHO safe childbirth checklist implementation guide: improving the quality of facility-based delivery for mothers and newborns [Internet]. [cited 2022 Oct 21]. Available from: https://www.who.int/publications-detail-redirect/9789241549455
  25. 25. Patrick M. Salary Structure FY 2018/2019 (Schedule 1–12) [Internet]. Ministry of Public Service. 2018 [cited 2022 Mar 3]. Available from: https://publicservice.go.ug/download/salary-structure-fy-2018-2019-schedule-1-12/
  26. 26. Salaries & Remuneration Commission. Phase IV Salary Review for County Government Employees. [Internet]. Government of Kenya; 2020 Jul. Available from: https://src.go.ke/wp-content/uploads/2020/07/Phase-IV-Salary-Review-for-County-Government-Employees-at-the-Executive.pdf
  27. 27. Health Service Commission. Guidelines for the Recruitment of Health Workers in Global Governments [Internet]. Recruitment Guidelines for LG Health Workers. 2020 [cited 2021 Mar 9]. Available from: https://www.hsc.go.ug/library/Recruitment%20Guidelines%20for%20LG%20Health%20Workers-A4%20Final%20(1).pdf
  28. 28. Patrick M. Establishment Notice No.1 of 2020 on Guidelines on Payroll, Wage Bill, Pension and Gratuity Management [Internet]. Ministry of Public Service. 2020 [cited 2022 Mar 3]. Available from: https://publicservice.go.ug/download/establishment-notice-no-1-of-2020-on-guidelines-on-payroll-wage-bill-pension-and-gratuity-management/
  29. 29. Guidelines for Governance and Management structures | Ministry of Health Knowledge Management Portal [Internet]. [cited 2022 Mar 3]. Available from: http://library.health.go.ug/publications/governance/guidelines-governance-and-management-structures
  30. 30. Directorate of Public Service Management. Revised Scheme of Service for Nursing Personnel [Internet]. Republic of Kenya; 2014 [cited 2021 Mar 9]. Available from: https://www.health.go.ke/wp-content/uploads/2015/09/SCHEME%20OF%20SERVICE%20FOR%20NURSING%20PERSONNEL.PDF
  31. 31. Vail B, Morgan MC, Spindler H, Christmas A, Cohen SR, Walker DM. The power of practice: simulation training improving the quality of neonatal resuscitation skills in Bihar, India. BMC Pediatr. 2018 Sep 3;18(1):291. pmid:30176831
  32. 32. Simulation and team training embedded nurse mentoring programme and improvement in intrapartum and newborn care in a low-resource setting in Bihar, India—PubMed [Internet]. [cited 2022 Feb 14]. Available from: https://pubmed.ncbi.nlm.nih.gov/33425334/
  33. 33. Uganda National Medical Stores [Internet]. FY2017/18 [cited 2021 Mar 9]. Available from: https://www.nms.go.ug/index.php/client-services/publications/category/16-order-forms
  34. 34. Sales Price List [Internet]. Kenya Medical Supplies Authority—KEMSA. 2017 [cited 2022 Mar 3]. Available from: https://www.kemsa.co.ke/salespricelist/
  35. 35. Frey HA, Klebanoff MA. The epidemiology, etiology, and costs of preterm birth. Semin Fetal Neonatal Med. 2016 Apr;21(2):68–73. pmid:26794420
  36. 36. Chaudhury S, Arlington L, Brenan S, Kairuki AK, Meda AR, Isangula KG, et al. Cost analysis of large-scale implementation of the ‘Helping Babies Breathe’ newborn resuscitation-training program in Tanzania. BMC Health Serv Res. 2016 Dec 1;16:681. pmid:27908286
  37. 37. Johns B, Hangoma P, Atuyambe L, Faye S, Tumwine M, Zulu C, et al. The Costs and Cost-Effectiveness of a District-Strengthening Strategy to Mitigate the 3 Delays to Quality Maternal Health Care: Results From Uganda and Zambia. Glob Health Sci Pract. 2019 Mar 11;7(Suppl 1):S104–22. pmid:30867212
  38. 38. GDP per capita (current US$)—Kenya | Data [Internet]. [cited 2022 Feb 14]. Available from: https://data.worldbank.org/indicator/NY.GDP.PCAP.CD?locations=KE
  39. 39. GDP per capita (current US$)—Uganda | Data [Internet]. [cited 2022 Feb 14]. Available from: https://data.worldbank.org/indicator/NY.GDP.PCAP.CD?locations=UG
  40. 40. Walker DM, Cohen SR, Fritz J, Olvera-García M, Zelek ST, Fahey JO, et al. Impact Evaluation of PRONTO Mexico: A Simulation-Based Program in Obstetric and Neonatal Emergencies and Team Training. Simul Healthc J Soc Simul Healthc. 2016 Feb;11(1):1–9. pmid:26312613
  41. 41. Ghosh R, Spindler H, Dyer J, Christmas A, Cohen SR, Das A, et al. Simulation and team training embedded nurse mentoring programme and improvement in intrapartum and newborn care in a low-resource setting in Bihar, India. J Glob Health. 2020 Dec;10(2):021010. pmid:33425334
  42. 42. Walker DM, Cohen SR, Estrada F, Monterroso ME, Jenny A, Fritz J, et al. PRONTO training for obstetric and neonatal emergencies in Mexico. Int J Gynaecol Obstet Off Organ Int Fed Gynaecol Obstet. 2012 Feb;116(2):128–33. pmid:22112786