https://orcid.org/0000-0002-9880-2591
Figures
Abstract
Availability of emergency obstetric and newborn care (EmONC) is a strong supply side measure of essential health system capacity that is closely and causally linked to maternal mortality reduction and fundamentally to achieving universal health coverage. The World Health Organization’s indicator “Availability of EmONC facilities” was prioritized as a core indicator to prevent maternal death. The indicator focuses on whether there are sufficient emergency care facilities to meet the population need, but not all facilities designated as providing EmONC function as such. This study seeks to validate “Availability of EmONC” by comparing the value of the indicator after accounting for key aspects of facility functionality and an alternative measure of geographic distribution. This study takes place in four subnational geographic areas in Argentina, Ghana, and India using a census of all birthing facilities. Performance of EmONC in the 90 days prior to data collection was assessed by examining facility records. Data were collected on facility operating hours, staffing, and availability of essential medications. Population estimates were generated using ArcGIS software using WorldPop to estimate the total population, and the number of women of reproductive age (WRA), pregnancies and births in the study areas. In addition, we estimated the population within two-hours travel time of an EmONC facility by incorporating data on terrain from Open Street Map. Using these data sources, we calculated and compared the value of the indicator after incorporating data on facility performance and functionality while varying the reference population used. Further, we compared its value to the proportion of the population within two-hours travel time of an EmONC facility. Included in our study were 34 birthing facilities in Argentina, 51 in Ghana, and 282 in India. Facility performance of basic EmONC (BEmONC) and comprehensive EmONC (CEmONC) signal functions varied considerably. One facility (4.8%) in Ghana and no facility in India designated as BEmONC had performed all seven BEmONC signal functions. In Argentina, three (8.8%) CEmONC-designated facilities performed all nine CEmONC signal functions, all located in Buenos Aires Region V. Four CEmONC-designated facilities in Ghana (57.1%) and the three CEmONC-designated facilities in India (23.1%) evidenced full CEmONC performance. No sub-national study area in Argentina or India reached the target of 5 BEmONC-level facilities per 20,000 births after incorporating facility functionality yet 100% did in Argentina and 50% did in India when considering only facility designation. Demographic differences also accounted for important variation in the indicator’s value. In Ghana, the total population in Tolon within 2 hours travel time of a designated EmONC facility was estimated at 99.6%; however, only 91.1% of women of reproductive age were within 2 hours travel time. Comparing the value of the indicator when calculated using different definitions reveals important inconsistencies, resulting in conflicting information about whether the threshold for sufficient coverage is met. This raises important questions related to the indicator’s validity. To provide a valid measure of effective coverage of EmONC, the construct for measurement should extend beyond the most narrow definition of availability and account for functionality and geographic accessibility.
Citation: Gausman J, Pingray V, Adanu R, Bandoh DAB, Berrueta M, Blossom J, et al. (2023) Validating indicators for monitoring availability and geographic distribution of emergency obstetric and newborn care (EmoNC) facilities: A study triangulating health system, facility, and geospatial data. PLoS ONE 18(9): e0287904. https://doi.org/10.1371/journal.pone.0287904
Editor: Kyaw Lwin Show, Ministry of Health, MYANMAR
Received: January 18, 2023; Accepted: June 15, 2023; Published: September 14, 2023
Copyright: © 2023 Gausman et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data have been anonymized to ensure compliance with human subject protections and study protocols. The anonymized data underlying the findings are deposited here: Jolivet, R Rima; Gausman, Jewel; Adanu, Richard; Bandoh, Delia; Beruetta, Mabel; Chakraborty, Suchandrima; Kenu, Ernest; Khan, Nizamuddin; Odikro, Magdalene; Pingray, Veronica; Ramesh, Sowmya; Vázquez, Paula; Williams, Caitlin; Langer, Ana, 2022, "Validation Data for "Maternal Death Review Coverage"", https://doi.org/10.7910/DVN/3UEY4I, Harvard Dataverse, V1, UNF:6:wh1AhrYk9dDrEMsQSRgELQ== [fileUNF].
Funding: This work was supported by the Bill and Melinda Gates Foundation through an award to RRJ and AL (Improving Maternal Health Measurement (IMHM) Capacity and Use, grant number OPP1169546). Funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
From a measurement perspective, the Sustainable Development Goal (SDG) period has seen a marked proliferation in the number of indicators put forward for monitoring health at all levels–clinical, health system, and policy–including in the area of maternal and newborn health. [1,2] Sustainable Development Goal (SDG) 3.1 calls for all countries to reduce their maternal mortality ratio (MMR) by at least two-thirds from their 2015 baseline by 2030 to achieve a global average maternal MMR of fewer than 70 maternal deaths per 100,000 live births. [3] This MMR target emanated from the “Strategies toward Ending Preventable Mortality (EPMM),” the global guidance report outlining targets and priority recommendations for maternal health and survival during the SDG period. [4] Ending preventable maternal mortality requires countries to effectively address all causes of maternal death by ensuring universal health coverage [5–7] that guarantees quality maternal and newborn care [8,9], and timely access to lifesaving interventions through availability of emergency obstetric and neonatal care (EmONC) [10,11].
Fundamental principles for effective monitoring of maternal health published in 2015 and updated in 2020 call for focus and fit, i.e., a minimum core set of tracer measures should be prioritized for monitoring, selected based on relevance to well-defined objectives that are useful to the end user. [12,13] Given the need to reduce measurement burden while optimizing the relevance and utility of core measures selected for monitoring, those with demonstrated validity—i.e. that show evidence that they reliably and accurately represent the construct of interest–should be prioritized for national and global monitoring. [14,15].
EmoNC availability is a strong supply side measure of essential health system capacity, closely and causally linked to maternal mortality reduction. [16] Monitoring the availability of EmoNC is often done by examining a succinct number of key clinical interventions designed to reflect health system capacity to treat the range of major obstetric complications consisting of parenteral antibiotics, anticonvulsants and uterotonics, manual removal of placenta, removal of retained products of conception, newborn resuscitation, assisted vaginal delivery, cesarean sections, and blood transfusions. Collectively, these interventions are known as the EmONC signal functions, which are classified at the basic-level (BEmONC), which includes the seven signal functions except cesarean section and blood transfusion, and at the comprehensive-level (CEmONC), which includes all nine signal functions. [17] Facilities are designated as BEmONC or CEmONC based on their performance of the signal functions [18]. While accurately tracking health system capacity to EmONC is inarguably a core construct of interest for ending preventable maternal mortality, how best to define and measure this construct is subject to uncertainty.
The WHO indicator “Availability of EmONC facilities” [17] was prioritized as a core EPMM indicator for its potential contribution toward achieving EPMM Key Theme #10: “Strengthen health systems to respond to the needs and priorities of women and girls” [19]. From its name, the indicator “Availability of EmONC” is often understood to focus solely on whether emergency care facilities exists in sufficient quantity to meet need (facility density) [20]. However, studies demonstrate that the number of facilities designated as providing EmONC is often not a true reflection of the emergency care services available to meet the needs of the surrounding population, because not all facilities designated as providing EmONC function as such [21,22]. To provide a meaningful measure of the construct intended for monitoring–that essential emergency interventions are available in facilities that demonstrate readiness to deliver them effectively and are distributed so that they are physically accessible to the population at risk for obstetric and neonatal emergencies in countries—the concept of availability in the indicator must be operationalized multidimensionally to capture aspects of facility functionality, readiness, and geographic accessibility.
The Office for the High Commissioner for Human Rights (OHCHR) declared the right to the highest attainable standard of health as an obligation of state duty-bearers in 2000 and stipulated that its realization is dependent on the following conditions: availability, accessibility, acceptability, and quality (known as the AAAQ framework). In the AAAQ framework, availability is defined as there being enough functioning facilities, goods, or services, with the enumeration and nature of these being dependent on factors at country level. Accessibility is defined to include both geographic access and socioeconomic dimensions of access. The definitions for “Acceptability” and “Quality” in the AAAQ framework reference respectful and culturally appropriate care, and scientifically sound and medically appropriate care, respectively, thereby aligning with the two principal dimensions of the WHO standards for improving quality of maternal newborn care in facilities. [23] The AAAQ framework provides a useful a lens through which to explore the multiple dimensions of the construct behind the indicator “Availability of EmONC.”
The global guidelines for monitoring emergency obstetric care developed by Columbia Averting Maternal Death and Disability, UNFPA, UNICEF, and WHO and released in 2009 are currently under revision [17]. Evidence is needed to assess the validity of the indicator “Availability of EmONC” to explore how well it captures the complex construct it seeks to measure overall, as well as to evaluate the individual components of the indicator’s definition and metadata.
To respond to this gap, this study seeks to validate several dimensions captured within the construct of “Availability of EmONC.” The first dimension is whether EmONC services are available within health facilities. To do this, we examined the demonstrated ability of a facility to deliver all EmONC signal functions, as well as staffing availability and whether services are offered 24 hours per day seven days per week. The second dimension is facility density, which measures whether there is a sufficient number of EmONC facilities to ensure coverage of essential emergency care. The measure of density of EmONC facilities is affected not only by the numerator (the number of functional emergency facilities) but also by the population reflected in the denominator (the population in need of emergency obstetric and neonatal care services). This study aims to examine these first two dimensions of availability on the impact on the value of the indicator if the numerator is redefined to account for facility performance and functionality, and also if the denominator is redefined to reflect different population groups that are considered to be in need (total population, women of reproductive age, and births).
Finally, the third dimension relates to facility distribution and geographic accessibility. This study aims to explore this dimension by estimating the proportion the population within two-hours travel time while taking into account facility performance and functionality. Travel times to EmONC greater than two hours have been associated with higher maternal mortality [24,25] and measuring the percentage women within two hours travel time of an EmONC facility has been recommended as a coverage target [22]. The inability of previous studies that have estimated travel time to EmONC to account for facility performance and functionality has been recognized as a limitation that has likely led to overestimates of access [26].
Materials and methods
This study utilized data from three different sources: 1) cross-sectional data derived from the health system, 2) primary data collected from facility records, and 3) geospatial population data to explore the underlying construct for measurement of the “Availability of EmONC,” as it relates to the indicator’s numerator and denominator, in a multi-step process.
Study setting
This study took place in four subnational geographic areas in Argentina, Ghana, and India. More details on the selection for each study area is available in the study protocol [27]. In Argentina, we included the provinces of Buenos Aires (Region V), Jujuy, La Pampa, and Salta. In Ghana, we included the districts of Bunkpurugu Yunyoo and Tolon in the Northern Regions and Techiman and Sunyani Municipal in the Brong-Ahafo Region. In India, we included the districts of Gonda and Meerut in the state of Uttar Pradesh and Krishnagiri and Thiruvallur in the state of Tamil Nadu.
Facility selection
In each study setting, we obtained an official list of all public and registered private facilities that provide birth care from the Ministry of Health. All facilities on the list were categorized according to their official government designation as being a BEmONC-designated facility, a CEmONC-designated facility, or as having no EmONC designation. All facilities identified by the Ministry of Health as providing birth care were included in the study. In total, 34 facilities were identified in Argentina, 51 facilities in Ghana, and 282 facilities in India. All eligible facilities participated in the study, leading to a 100% participation rate.
Facility data
Study staff visited each facility and reviewed the facility’s records to assess the performance of the EmONC signal functions. Facility records that were reviewed included maternity, delivery, general admissions, operating theater, female ward, discharge, drug inventory, and staffing registers.
From these records, researchers extracted data using a predesigned data collection form on the performance of each EmONC signal function. For performance of signal functions, we sought evidence that the facility had performed the following nine EmONC signal functions at least once in the 90 days preceding data collection: 1) administration of parenteral antibiotics, 2) administration of parenteral oxytocics, 3) administration of parenteral anticonvulsants, 4) manual removal of placenta, 5) removal of retained products of conception, 6) assisted vaginal delivery (vacuum or forceps), 7) newborn resuscitation with bag and mask, 8) cesarean delivery, and 9) blood transfusion was reviewed and documented. Signal functions 1–7 are BEmONC signal functions. CEmONC signal functions comprise all nine interventions.
The following data on EmONC facility readiness were also collected: 24/7 care, 24/7 staffing, and availability of essential drugs. We defined 24/7 care as whether the facility was open to provide care 24-hours per day/7 days per week. We defined 24/7 staffing as whether there were for obstetricians/gynecologists, midwives, auxiliary nurse midwives, or nurses available 24-hour/7 day per week. For essential drugs, researchers recorded whether the facility had a drug inventory register available and whether parenteral antibiotics, oxytocin/ergometrine, magnesium sulfate/diazepam, and misoprostol were in stock at the time of data collection.
Finally, data were collected from clinical records on the number and type of recorded cases of obstetric and neonatal emergencies.
Other covariates were also collected from each facility, including facility type (i.e. primary, secondary, or tertiary.), facility governance (public or private sector), and location (rural or urban). Further, we collected geographic information system (GIS) coordinates from each facility to document its precise geo-location. Facility data were collected and managed using REDCap (Research Electronic Data Capture) which is a secure web-based application designed to support data capture for research studies [28].
Population data
In each study setting, the total population, number of women of reproductive age, and number of births were estimated using geospatial methods. Population estimates were generated using ArcGIS software [29]. For India and Argentina we first obtained the GIS shapefile for each district/province’s administrative boundaries from the Database of Global Administrative Areas, version 2.0 [www.gadm.org]. For Ghana, we obtained a district boundary shapefile directly from the Ghana Ministry of Health. Then, we used WorldPop to estimate the specific populations of interest in each study setting. WorldPop is an open access GIS dataset that estimates human populations by drawing on census data, United Nations population estimates, and satellite imagery [30]. Total population estimates were obtained for the year 2020 and were available in raster format at a resolution of 100m while estimates of number of women of reproductive age and births were available only for the year 2015 and at a resolution of 1km. The ArcGIS “zonal statistics to table” function was applied to each population raster, specifying district border shapefiles as the input zones. This produced total population, number of women of reproductive age, and number of birth estimates for all districts. We used the WorldPop top-down constrained population modeling method to accurately identify rural areas, small settlements, and uninhabited areas [30,31].
Last, using the population estimates within each study setting, we estimated the total population, WRA, and births that occurred within two-hours travel time of a designated, fully or partially functioning BeMONC facility by incorporating data on roads, rivers, and lakes from Open Street Map, which is a mapping project involving both professional cartographers and citizens that map features such as roads, waterways, buildings, places of interest, and more, primarily using satellite imagery and GPS locations. The data were downloaded in GIS shapefile format from the Geofabrik OSM download server [32]. Data on elevation were obtained from the Shuttle Radar Topography Mission (SRTM), which is an international research effort that mapped elevation on a near-global scale at a ground resolution of 90 meters, and for select locations at 30m resolution [33].
Map background data source
For the maps in Figs 3, 4 and 6 the “Population density” data displayed in the background was accessed from WorldPop (www.worldpop.org) and is available for use under the Creative Commons Attribution 4.0 International License (Argentina: https://hub.worldpop.org/doi/10.5258/SOTON/WP00674, Ghana: https://hub.worldpop.org/doi/10.5258/SOTON/WP00674, India: https://hub.worldpop.org/doi/10.5258/SOTON/WP00674).
Ethics statement
The Institutional Review Board (IRB) of the Harvard T.H. Chan School of Public Health approved this study on 4 September 2019 (approval ID: IRB19-1086). The research is classified as Level 4 Data using Harvard’s Data Security Policy. The study also was approved in Argentina by the Comité de Ética de la Investigación de la Provincia de Jujuy (approval ID not applicable), Comisión Provincial de Investigaciones Biomédicas de la Provincia de Salta (approval ID: 321-284616/2019), Consejo Provincial de Bioética de la Provincia de La Pampa (approval ID not applicable), and Comité de Ética Central de la Provincia de Buenos Aires (approval ID: 2919-2056-2019); in Ghana by the Ghana Health Service Ethical Review Board (approval ID: GHS-ERC022/08/19); and in India by the national population council IRB (approval ID: 889) and local Sigma-IRB (approval ID: 10052/IRB/19-20).
After full board review, the need for informed consent was waived as it was determined our study did not collect any data on human subjects. Only anonymized and generalizable data were collected during retrospective medical chart review.
Inclusivity in global research
Additional information regarding the ethical, cultural, and scientific considerations specific to inclusivity in global research is included in the (S1 Checklist).
Analysis
In the first step of our analysis, we tabulated the performance of each of the nine EmONC signal functions in the 90 days prior to data collection among the health facilities included in the study according to their location, level, and their official EmONC designation. As signal function #6 (assisted vaginal delivery) is not routinely performed in some of the study settings, we created an additional category reflecting partial performance at the BEmONC and CEmONC level to identify facilities in which all B/CEmONC signal functions were performed, less signal function #6.
Next, we analyzed facility functionality. Our definition of a functional EmONC facility corresponds to the UNFPA definition which defines a fully functional EmONC facility as one that performed all seven BEmONC or nine CEmONC signal functions over the previous 90-day period as well as whether the facility is open 24/7 (24/7 care). As before, we also expanded our definition of partial functionality to include facilities with 24/7 care and in which all B/CEmONC signal functions, except #6, were performed. We descriptively analyzed the percentage of fully and partially performing BEmONC-designated facilities as well as the percentage of fully and partially functioning CEmONC-designated facilities at the both the CEmONC-level and BEmONC-level, given that CEmONC facilities are also by definition BEmONC facilities.
While not included in our definition of facility functionality, we also examined several other elements of facility readiness aside from 24/7 care to further explore the validity of the indicator, including 24/7 staffing, availability of essential medicines, and presences of a drug inventory register. We examined these elements of facility EmONC readiness versus facility EmONC designation by calculating the percentage of facilities that had 24/7 care, 24/7 staffing, and essential drugs in stock.
As a final examination of the numerator’s indicator, we used chi squared tests to explore whether facility performance of specific signal functions over the previous 90 days was associated with whether the facility had at least one case of an obstetric emergency that would warrant the performance of that specific signal function. As some clinics may not encounter the specific obstetric emergencies that may warrant performance of a given signal function, this enabled us to examine whether lack of performance of a signal function may be explained by case load, rather than a lack of ability to perform that signal function.
Then, we calculated the value of the indicator using different versions of the numerator and the denominator derived from the study results. We calculated the number of designated, fully functional, and partially functional BEmONC- and CEmONC- facilities per 500,000 population and per 20,000 births. We compared the results to the global reference standard that suggests a minimum number of five BEmONC facilities and one CEmONC facility per 500,000 population or 20,000 births, and explored the variation between these different measures across the study areas [16].
Finally, we calculated the percentage of total population, women of reproductive age, and births within 2 hours travel time of designated, fully functioning, and partially functioning B/CEmONC facilities and compare the results to global recommendations that 90% of the population denominator used be within 2 hours of a EmONC facility [22].
Analyses were performed using Stata v14 and graphical displays were created using the R Package ggplot [34].
Results
Table 1 shows the distribution of the facilities in the study areas across each country included in the study. In Argentina and Ghana, more that 40% of all the facilities providing birth care were located in one study area, with 47.0% in Buenos Aires Region V for Argentina and 43.1% in Sunyani district for Ghana. In India, facilities were more equally distributed across the four study areas. In Argentina, all facilities were either secondary- or tertiary-level facilities, while in Ghana and India, most facilities provided primary-level care. All facilities included in Argentina were urban, while 85.5% of facilities in India were in rural areas. In Argentina, all facilities were designated to provide care at the CEmONC level. In both Ghana and India, a larger percentage of facilities located in the study areas were designated at the BEmONC-level rather than at the CEmONC-level, though in both settings, facilities with no EmONC designation constituted the majority of all facilities (45.1% in Ghana and 71.6% in India).
Performance of EmONC signal functions
Facility performance of BEmONC and CEmONC signal functions varied considerably across and within study settings. Fig 1 shows performance of each signal function across all facilities in each country, regardless of a facility’s EmONC designation. In general, across all three countries, there was a high degree of variability in the performance of each of the nine signal functions in birthing sites. In all countries, administration of parental oxytocics was the most commonly performed signal function (100% in Argentina, 92.2% in Ghana, and 95.7% in Argentina), and signal function #6 (assisted vaginal delivery) was the least commonly performed (26.5% of facilities in Argentina, 13.7% of facilities in Ghana, and 1.8% of facilities in India). In Ghana, only 25.5% of facilities performed signal function #1 (administration of parenteral antibiotics), though it was among the most commonly performed signal functions in the other two study countries (100% in Argentina and 87.2% in India).
Performance of all B/CEmONC signal functions as well as 24/7 care, also varied considerably among facilities designated as B/CEmONC in each country. Table 2 shows the proportion of BEmONC-designated facilities in Ghana and India that had performed all seven BEmONC signal functions in the 90 days prior to data collection, all BEmONC signal functions less assisted vaginal delivery (considered as partial performance), and those that did not meet either standard. As shown in the table, the vast majority of BEmONC-designed facilities in Ghana and India were found to be non-performing. In Ghana, only one BEmONC-designated facility (reflecting 4.8% of all BEmONC-designated facilities), located in Bunkpurugu Yunyoo, had performed all seven BEmONC signal functions and no BEmONC-designated facilities evidenced partial performance. In India, no BEmONC-designated facilities had performed all seven BEmONC signal functions, and only 12 (17.9%) had evidenced partial performance. One district, Thiruvallur, did not have any BEmONC-designated facilities that evidenced either complete or partial performance of the seven BEmONC signal functions.
Performance of all seven basic signal functions and the two additional comprehensive signal functions was also fairly limited among CEmONC-designated facilities in the study countries. Table 3 shows the percentage of CEmONC-designated facilities that performed at either the BEmONC or CEmONC-level, and whether they exhibited full or partial performance. While performance of EmONC signal functions at CEmONC-designated facilities was better than among BEmONC-designated facilities, a substantial proportion of CEmONC-designated facilities did not perform all nine CEmONC signal functions or even all seven BEmONC signal functions in all three countries. In Argentina, only three (8.8%) of all CEmONC-designated facilities performed all nine CEmONC signal functions, and they were all located in Buenos Aires Region V. No additional CEmONC facilities in Argentina evidenced either partial or full performance at the BEmONC-level. Similarly, four CEmONC-designated facilities in Ghana (57.1%) and the three CEmONC-designated facilities in India (23.1%) evidenced full CEmONC performance. As in Argentina, no additional CEmONC-designated facilities evidenced full performance at the BEmONC-level. In all countries but Ghana, removing assisted vaginal delivery from the list of CEmONC functions resulted in an increased number of CEmONC-designated facilities that met the definition for partial performance. In Argentina, eleven CEmONC-designated facilities (32.4%) exhibited partial performance at the CEmONC-level. An additional three facilities (8.9%) exhibited partial performance at the BEmONC-level. In India, an additional four facilities (30.8%) evidenced partial CEmONC performance and one additional facility evidenced (7.7%) evidenced partial BEmONC performance. No CEmONC-designated facilities evidenced partial CEmONC or partial BEmONC performance in Ghana.
A comparison of a facility’s performance of a specific signal function during the 90 days prior to data collection to whether a facility had record of a corresponding obstetric emergency during that same time period suggests that in general, there is a significant association between case load and performance, as detailed in Table 4. For the most part, facilities that did not perform a specific signal function had no record of encountering an obstetric emergency that would require its performance. For example, among the facilities that failed to perform Signal Function #3 (administration of anticonvulsants), more than 80% did not have a confirmed case of severe pre-eclampsia in the facility register, which would be a clinical reason to administer anticonvulsants. There are some important exceptions to note. In India, a majority (61.1%) of facilities that did not perform Signal Function #9 (blood transfusion) had at least one case of maternal hemorrhage on file.
Facility readiness—24/7 care, 24/7 staffing and essential medicine availability by EmONC designation
In general, a majority of birthing facilities in all three countries were open to provide 24/7 emergency care, had at least one staff on call or on duty 24/7 to respond to an obstetric emergency, had all essential drugs in stock (except for non-EmONC designated facilities in Ghana), and had a drug inventory register present on site (Table 5). Further, more CEmONC-designated facilities provided 24/7 care and had greater availability of essential medicines than BEmONC-designated facilities or facilities without EmONC designation. Over 90% of B/CEmONC facilities in Argentina and India provided 24/7 obstetric and neonatal care (91.2% of CEmONC designated facilities in Argentina and 100.0% and 98.51% of CEmONC and BEmONC facilities in India, respectively). In Ghana, however, the pattern was different than that observed in the other countries, with 85.7% of CEmONC-designated facilities and 52.4% of BEmONC-designated providing obstetric and neonatal care 24/7, with a much higher percentage (91.3%) of non-EmONC designated facilities providing 24/7 obstetric and neonatal care. Nearly all B/CEmoNC-designated facilities had at least one staff member on call or duty 24/7, except for one CEmONC-designated facility in Ghana. Furthermore, nearly all CeMONC-designated facilities had all the essential drugs examined in stock (though two CEmONC facilities in Argentina and one in India did not have misoprostol in stock). In India, the vast majority of BEmONC-designated facilities had essential drugs in stock (ranging from 83.6% having misoprostol in stock to 98.5% having oxytocin/ergometrine in stock). In Ghana, drug availability was much lower in BEmONC-designated facilities. For example, only 33.3% of BEmONC-designated facilities had parenteral antibiotics in stock and 52.4% had oxytocin/ergometrine in stock. Availability of essential drugs was lowest in both India and Ghana among non-EmONC designated facilities.
Facility functionality (performance and 24/7 care)
Fig 2 presents the percentage of facilities in each study setting according to their highest level of validated EmONC functionality, which combines performance of the seven BEmONC and nine CEmONC signal functions, as well as being open 24/7 to provide obstetric and newborn care. Further, the results are disaggregated by facility location (rural/urban) and level (primary, secondary, and tertiary) to explore disparities by these characteristics. Across all countries, the vast majority of BEmONC-designated facilities, as well as facilities without EmONC designation, were non-functional. While in general, a larger proportion of CEmONC-designated displayed some degree of functionality, an important percentage were non-functional. In Ghana, only 4.8% of all BEmONC-designated facilities were fully functional, and three districts did not have either a fully or partially functional BEmONC facility. In India, 17.9% of all BEmONC-designated facilities were partially functional, and the percentage of facilities with partial functionality varied considerably across the study districts, ranging from 0% of facilities in Thiruvallur to 41.7% of facilities in Meerut being partially functional. Among CEmONC facilities, in all study countries, adding the requirement that a facility be open to provide 24/7 care to be considered fully or partially functional did not reduce the number of facilities operating at either level from those that performed the corresponding signal functions.
Facility density: Accounting for facility designation and functionality while varying the indicator’s reference population
After having examined the different dimensions that comprise the standard and alternative indicator definitions, we next calculated the number of facilities per 500,000 total population and 20,000 births, while accounting for facility designation and functionality (Table 6). First, when calculating the indicator to include all EmONC-designated facilities in the numerator regardless of their functionality, a greater number of study areas reached the target of 5 BEmONC facilities per 20,000 births than for 500,000 total population. The magnitude of the difference between indicator estimates obtained based on total population versus births was uneven across study areas within and across countries. Among the three countries in the study, there was the greatest variability across sub-national study areas within India. In Thiruvallur, results showed a 250% increase in the value of the indicator when calculating the indicator per 500,000 total population versus 20,000 births. In Krishnagiri, the magnitude of the difference between the two estimates was slightly lower than in Thiruvallur, yet still exhibited a 150% increase; however, in Gonda and Meerut, there was only a 63% and 73% increase in the value of the indicator, respectively. In Argentina, the difference in magnitude ranged from 100% in Salta to 155% in La Pampa. In Ghana, there was a much smaller magnitude of change in the value of the indicator as calculated based on total population versus births. In Sunyani, the indicator increased by about 170% when calculating it based on total population versus births; however, in Tolon and Techiman, the value of the indicator decreased by about 30%.
Next, we explore how the value of the indicator that was calculated using different population parameters is affected when facility functionality is considered (Table 6). Taking into account only fully or partially functioning facilities at the BEmONC-level, the value of the indicator changed dramatically in all countries and sub-national study areas. No sub-national study area in Argentina or India reached the target of 5 BEmONC-level facilities per 500,000 total population or 20,000 births when including only fully or partially functioning BEmONC-level facilities in the numerator. In Ghana, Bunkpurugu Yunyoo and Sunyani reached the target of 5 facilities per 500,000 total population and 20,000 births when partially functional BEmONC facilities were included in the numerator, but only Sunyani reached the target facility density for both total population and births when only fully functioning BEmONC facilities were counted.
At the CEmONC-level, there was less variation across sub-national study areas when changing the value of the numerator to incorporate facility functionality than at the BEmONC-level. All provinces in Argentina reach or exceed the target of one CEmONC facility per 500,000 total population or 20,000 births when considering designated facilities or partially functional facilities; however, only one province (Buenos Aires Region V) reached the target when considering only fully functional facilities. In Ghana, as all CEmONC-designated facilities are fully functional, the value of the indicator did not vary based on the changing definition of the numerator. In India, Krishnigiri, Meerut, and Thiruvallur reached the target of one CEmONC facility per 20,000 births, but not per 500,000 total population, when including CEmONC-designated facilities in the numerator, but Krishnigiri alone reached the target of one CEmONC facility per 20,000 births when only partially functional CEmONC facilities were included in the numerator. No district in India reached the target based on total population or births when only fully functional CEmONC facilities are included in the numerator. In Ghana, two districts did not have any CEmONC-designated facilities, and no facility of any other designation evidenced full or partial CEmONC performance.
Percentage of the reference population within two-hours travel time of a designated, partially-, or fully functional EmONC facility
The percentage of the total population, women of reproductive age, and births occurring within two hours travel time of a BEmONC-designated, partially functioning BEmONC, or fully functioning BEmONC facility is presented in Table 7. Changing the population parameter between total population, women of reproductive age, and births, produces little variation across sub-national study areas, and in most areas, does not affect whether the coverage target of 90% of the population within two-hours travel-time is met. However, there are some notable coverage reductions observed when changing the population parameters. For example, the percentage of the total population in Tolon within 2 hours travel time of a designated EmONC facility is estimated at 99.6%; however, only 91.1% of women of reproductive age are within 2 hours travel time–amounting to a reduction of almost 10 percentage points. When factoring in facility functionality, substantial reductions were observed in Argentina and Ghana in the percentage of the population within two-hours travel time from partially and fully functional EmONC facilities. Fig 3 graphically displays the catchment areas within two-hours travel time of a partially functional facility on district maps in relation to population density, and Fig 4 shows the same for fully functional facilities. According to the figures, the location of partially or fully functional facilities does not necessarily correlate with areas have the highest population density within a district, and further, in many districts, even if there is only one partially or fully functional EmONC facility, the majority of the population remains within two-hours travel time to the facility. For example, in Krishnagiri in India, there is only one fully functional EmONC facility (Fig 4), and four partially functional facilities (Fig 3); however, in both cases, 100% of the district is within two-hours travel time.
Fig 5 illustrates convergence of the value of indicator measuring the density and distribution of EmONC facilities using the two different estimation approaches: (1) the number of EmONC facilities per 500,000 total population and 20,000 births and (2) the percent of the total population and percent of births within two-hours travel time, taking into account facility designation and functionality using Salta Province in Argentina as an example. In general, changing the numerator or denominator leads to indicator estimates that exceed the suggested targets in one domain but not the other, thus leading to conflicting interpretations of adequacy. In Salta, the number of EmONC-designated facilities per 20,000 births exceeds the coverage target of five facilities per 20,000 births, and greater than 90% of births occur within two-hours travel time of an EmONC designated facility—again, exceeding the travel-time target. Conversely, there is discordance in whether the value of these two indicators represents sufficient coverage when calculated using the total population as the reference, instead of births. In Salta, there are only three EmONC designated facilities per 500,000 total population, which is far below the recommended number of five; however, at the same time, the percentage of the total population within two-hours travel time of an EmONC-designated facility exceeds the 90% target. When considering partially-functioning facilities with births as the reference population, neither method of calculating the indicator meets either target; however, the percentage of the total population within two-hours travel time of a partially functional facility is only slightly less than the 90% target, but the number of partially functional facilities (1.87) per 500,000 population is substantially less than the target of five.
Discussion
Our study posits that the indicator “Availability of EmONC,” despite the focus implied by its name, is intended to capture a broader construct for measurement that comprises multiple dimensions of availability: availability of all EmONC signal functions within designated EmONC facilities, facility readiness to deliver those essential interventions, coverage of sufficient EmONC facilities to meet the needs of the population, and appropriate geographic distribution of facilities to make those services available in places where they are accessible to those who need them; however, our results question the validity of the indicator as currently defined in incorporating these critical dimensions. Finally, we find that comparing the value of the indicators when calculated using different definitions reveals important inconsistencies, resulting in conflicting information about whether the threshold for sufficient coverage is met.
Following the AAAQ framework, the indicator explores the dimension of availability as it relates to facility density. Determining whether the quantity of emergency services is sufficient is the ultimate objective for measurement. Sufficiency is defined as the quantity that is “enough to meet the needs of a situation of a proposed end.” [35] Since achieving sufficiency is conditional on the best approximation of the need that must be filled, the population used as the denominator to calculate the value of the estimate is significant. While there is no consensus on the optimal parameter to estimate the population in need of EmONC, calculating the indicator using total population versus births reveals considerable differences in magnitude for the estimate of facility coverage, highlighting a lack of convergence between these two measures. Specifically, we find that study areas that achieve sufficiency when it is defined as 5 facilities per 20,000 births my not achieve sufficiency when it is defined as 5 facilities per 500,000 total population. Therefore, establishing a threshold for sufficient coverage is sensitive to demographic characteristics. For example, all the provinces in Argentina and two districts in the state of Tamil Nadu in India have relatively low birth rates. As a result, these areas all meet the threshold of sufficient EmONC density when defining it based on the number of facilities per births, while only one of these areas meets that threshold when defined as facilities per total population. Conversely, we see stronger convergence in the value of the indicator in areas where fertility is higher, such as in Ghana and Uttar Pradesh in India. To date, there is no clear evidence available that identifies the best denominator to use to measure whether there is a sufficient quantity of EmONC facilities.
The evidence base for the target values for sufficient coverage also needs strengthening, since the indicator for unmet obstetric need has not correlated reliably with maternal mortality related to absolute medical indications for emergency interventions. [26] Research has shown that targeting the number of facilities needed per 500,000 population does not correlate well with maternal mortality and instead, targeting the number of facilities needed per births in the population correlates better [16]; however, other plausible denominators could also be tested for their correlation with mortality reduction. The ideal measure of availability of EmONC would be grounded in strong evidence of the population need for emergency care based on the epidemiology of obstetric emergency conditions. Since the minimum thresholds of prevalence for absolute medical indications used to calculate the unmet obstetric need indicator that provides the epidemiologic rationale for the EmONC facility coverage targets may not be valid in all populations, lower rates of surgery for absolute medical indications do not necessarily signal higher maternal mortality from such conditions. [26] In turn, quality and appropriateness of care are potentially influenced by volume of care. In facilities that experience low volumes of emergencies, the risk arises of poorer quality of emergency responses and thus, poorer outcomes of emergency care [27]; however, when all low-risk women are funneled into emergency care settings, the risk of over-intervention increases [28].
While the definition of sufficiency focuses on there being enough supply, it also implies avoiding oversupply. More efficient population coverage of emergency care can be achieved by reducing the number of designated EmONC facilities and instead, focusing on functionality and facility readiness in a smaller number of hospitals [36]. Such an arrangement could have the added advantage of ensuring that the staff in facilities with higher volumes of high-risk cases retains the specialized skills necessary to perform emergency signal functions [37]. This raises questions about the validity of facility density as a measure of EmONC availability, if evidence suggests that more is not necessarily better. To wit, our findings showed that most facilities that perform specific signal functions had no evidence of corresponding complications in their facility records. The reasons for such non-performance are not elucidated by our research. While it is possible that facilities with poor record keeping failed to record both signal function performance and the corresponding cases of obstetric emergencies, even though both took place, it could also be related to a lack of need to perform those signal functions, which has been cited in other studies as a reason for nonperformance. [17].
Next, looking at “Accessibility,” the proportion of the population within two-hours travel time from an EmONC facility has been proposed as an improved measure of coverage over density, as it comprises the element of geographic accessibility. Defining the indicator in this way may be particularly important for determining optimal geographic distribution of facilities for sufficient population coverage, since accessibility is dependent on conditions on the ground, such as terrain, roadways, waterways and climactic factors. [38] In effect, our results demonstrate that greatly reducing the number of facilities by counting only those that are functional in our two-hour travel time calculations only minimally changes the value of coverage estimates in some settings. In a geographically small district such as Krishnagiri in India, for example, 100% of the population is covered within two-hours travel time of a fully functional emergency facility, even though only a small fraction of facilities meet that performance threshold. This supports the argument that effective coverage of emergency care could be achieved with fewer EmONC facilities, entailing more effective use of resources, as long as the functional facilities are distributed geographically so they are accessible to the population in need of them.
The two indicators included in the WHO Manual currently under revision, the first being EmONC facilities per population/births and the second being the proportion of the population with geographic access to EmoNC within two hours travel time, together are meant to fully capture the construct of sufficient supply of facility-based EmONC. Two-hour travel time is only a valid measure of coverage if all emergency facilities demonstrate the same level of functionality. In reality, women do not know which of the facilities within two hours of them are fully-functional, and they will likely seek services closest to them or use other information available to them on quality of care to influence their choice [39]. For health care decision makers concerned with the organization and delivery of obstetric and neonatal care, the best measure of EmONC facility availability must therefore account for both geographic accessibility as well as facility readiness and evidence of functionality. Moreover, the population in need may not be distributed evenly so that in population-dense areas, travel-time may not be the only relevant measure of accessibility. Even with a sufficient number of facilities, appropriately distributed, socioeconomic dimensions of accessibility are important to take into account.
Finally, the dimensions of both “Acceptability” and “Quality” focus on the provision of appropriate, evidence-based care. As Gabrysch et al. have pointed out, the term “EmONC” conflates emergency care interventions with the presence of emergency facilities [16], thus measurement of this construct is not meaningful without a measure of facility functionality. Numerous studies show a significant deficit in functionality among facilities designated as providing EmONC. [40–42] Removing those facilities that are not functioning well affects the density and distribution of facilities but may not affect adequate availability of emergency care. There is a need to review the effectiveness and indication for these interventions based on current best available evidence and to examine data on their coverage and quality. Studies of EmONC facility performance of all signal functions required for the level of EmONC designated (basic or comprehensive) within the previous 90 days, suggest that a few signal functions (e.g., manual removal of products of conception and assisted vaginal delivery) are seldom evidenced in facilities that may otherwise be considered functional. [43,44] We found that very few facilities designated as emergency care facilities evidenced performance of all or most BEmONC or CEmONC signal functions, which suggests that using facility designation to calculate the numerator would over-estimate actual availability of EmoNC. Further, evidence of specific elements of facility readiness to deliver all emergency services was also found to be lacking and, not surprisingly, was associated with a lack of EmONC signal function performance across the study sites. Facility readiness to deliver emergency interventions reliably and effectively, evidenced by the availability of essential drugs and core staff, is a core element of the construct of EmONC availability.
Systematically unpacking the various underlying components of the full construct for measurement reflecting EmONC sufficiency and exploring how they are reflected in various approaches to calculating the value of the estimate of Availability of EmONC in our study settings using multiple, innovative primary data and population data is a major strength of our study. Further, our facility sample consists of a census of all birthing sites in each study area, resulting in robust indicator estimates, which we believe is another strength. Our results are subject to some limitations as well. Given that the exact population in each of the study areas is unknown, we estimated the total population, population of women of reproductive age, and estimated births using publicly available data from WorldPop, using an analytical approach that has been published elsewhere [45]. Other limitations are primarily related to the scope of the research. For feasibility reasons, our study data are limited to specific subnational areas, not nationwide in study countries. Furthermore, we calculate travel time based on any mode of transportation; travel time estimates could be further examined by calculating travel time only by foot [26]. The study districts were selected to reflect a range of maternal health system performance, but characteristics of the study districts such as area and population density that are germane to the study question may vary across each country. Thus, our results may not be generalizable. Similarly, geographic calculation of two-hour travel time is limited to the population of the districts in our study sample and does not extend beyond district boundaries. Thus, a proportion of the population that lies outside the district may nevertheless be within the two-hour catchment area. To explore how changing these parameters would affect our results, we performed a sensitivity analysis of two districts in Ghana that border each other, allowing the facilities in Sunyani, a district that has several fully functioning EmONC facilities, to serve the population of Techiman, a district without any fully or partially functional EmONC facilities. Incorporating the facilities in Sunyani increases the percent of the total population in Techiman within two-hours travel time of a partially or fully functional EmONC facility from 0.0% to 95.6. Future research on indicators should consider how to handle accessibility across subnational borders, as accessibility across such borders may vary between settings. Finally, we had originally intended to include both pregnancies and births in our analysis; however, the differences in coverage estimates obtained using both denominators were so little as to not be meaningful. As a result, we only present data on births.
Future research should the explore the most effective array of evidence-based interventions based on epidemiology of risk that should constitute EmONC signal functions. More evidence is also needed to explore the optimal distribution of facilities and organization of care services to provide both effective routine care for physiologic pregnancy and childbirth and emergency care for complications to address all causes of maternal mortality, reproductive and maternal morbidity, and related disabilities, a key theme for ending preventable maternal mortality.
Conclusions
Our study highlights significant differences in the value of estimates of sufficient EmONC coverage derived from country data depending on the definition of the indicator and measurement approach used. The optimal definition and calculation of a core measure to capture this construct is subject to uncertainty and the global reference standard indicators are currently under revision. Our study applies primary data to generate evidence that can help inform the debate. To provide a valid measure of effective coverage of EmONC, future indicators, such as those included in global guidelines emitted by WHO, should go beyond the most narrow definition of availability of emergency facilities to include dimensions of AAAQ, including evidence of regular performance of emergency signal functions, facility readiness to do so reliably and effectively, and appropriate geographic distribution for accessibility to functional facilities by the best representation of the population in need.
Supporting information
S1 Checklist. Inclusivity in global research checklist.
https://doi.org/10.1371/journal.pone.0287904.s001
(DOCX)
Acknowledgments
The authors would like to thank the following people, without whose efforts the publication of this manuscript would not have been possible:
In Argentina, we gratefully acknowledge the support of the National Directorate of Maternal, Child and Adolescent Health and the Directorate of Sexual and Reproductive Health of the Ministry of Health of the Nation. We commend the commitment and dedication of the provincial teams, and the following members of the Maternal and Child Health Programs of the Provincial Ministries of Health: Dr. Adriana Martirena, Dr. Daniel Nowacky, Dr. Adriana Allones, Marta Ferrary, Dr. Claudia Castro, Ana Seimande, Antonio Tabarcachi, Noelia Coria, Cintia Jacobi, Laura Soto, Dr. Mara Bazán, Dr. Susana Velazco, Dr. Patricia Leal, and Marcela Tapia. Finally, we would like to express our deepest gratitude to all of the health workers who participated in the study as data collectors, working through the height of the COVID-19 pandemic in Argentina.
In Ghana, we gratefully acknowledge the support of the Ghana Health Service Family Health Division, The Director General–Ghana Health Service–Dr. Patrick Kuma-Aboagye; Dr. Ernest Konadu Asiedu, Ms. Roberta Asiedu, Dr. Margretta Chandi and Ms. Catherine Adu Asare; Dr. Benedicta Mensah, Ms. Keziah Dampare, and all regional and district health workers and field teams for their persistence in data collection despite the challenges.
In India, we gratefully acknowledge the support of Dr. Dinesh Baswal, Ex Deputy Commissioner at Maternal Health Division, Ministry of Health & Family Welfare, India; the Mission Directors, State Health Departments of Tamil Nadu and Uttar Pradesh, and the District health Officials of study districts. We also acknowledge the support of Dr. Manju Chhugani and Dr. Renu Kharb for their guidance in review of the secondary data on many indicators. Finally, we sincerely thank the district field teams for their untiring efforts and adaptation to new methodologies to collect good quality data, in the midst of COVID in India. We also thank all the health workers and facility staff who participated in the study despite their busy schedules due to COVID situation.
Finally, we acknowledge Jean-Pierre Monet at UNFPA for his consultation in the formative stages of methodology development.
References
- 1. Moller A-B, Newby H, Hanson C, Morgan A, El Arifeen S, Chou D, et al. Measures matter: a scoping review of maternal and newborn indicators. PLoS One. 2018;13(10):e0204763. pmid:30300361
- 2. Saturno-Hernández PJ, Martínez-Nicolás I, Moreno-Zegbe E, Fernández-Elorriaga M, Poblano-Verástegui O. Indicators for monitoring maternal and neonatal quality care: a systematic review. BMC pregnancy and childbirth. 2019;19(1):1–11.
- 3.
SDG Indicator Metadata [Internet]. 2021 [cited 17 December 2021]. Available from: https://unstats.un.org/sdgs/metadata/files/Metadata-03-01-01.pdf.
- 4. Organization WH. Strategies towards ending preventable maternal mortality (EPMM). 2015.
- 5. Quick J, Jay J, Langer A. Improving women’s health through universal health coverage. PLoS Medicine. 2014;11(1):e1001580. pmid:24399923
- 6. Neal S, Channon AA, Carter S, Falkingham J. Universal health care and equity: evidence of maternal health based on an analysis of demographic and household survey data. International Journal for Equity in Health. 2015;14(1):56.
- 7. Kruk ME, Kujawski S, Moyer CA, Adanu RM, Afsana K, Cohen J, et al. Next generation maternal health: external shocks and health-system innovations. The Lancet. 2016;388(10057):2296–306. pmid:27642020
- 8. Renfrew MJ, McFadden A, Bastos MH, Campbell J, Channon AA, Cheung NF, et al. Midwifery and quality care: findings from a new evidence-informed framework for maternal and newborn care. The Lancet. 2014;384(9948):1129–45. pmid:24965816
- 9. Homer CSE, Friberg IK, Dias MAB, ten Hoope-Bender P, Sandall J, Speciale AM, et al. The projected effect of scaling up midwifery. The Lancet. 2014;384(9948):1146–57. pmid:24965814
- 10. Freedman LP. Using human rights in maternal mortality programs: from analysis to strategy. International Journal of Gynecology & Obstetrics. 2001;75(1):51–60. pmid:11597619
- 11. Hofmeyr GJ, Haws RA, Bergström S, Lee AC, Okong P, Darmstadt GL, et al. Obstetric care in low-resource settings: what, who, and how to overcome challenges to scale up? International Journal of Gynecology & Obstetrics. 2009;107:S21–S45. pmid:19815204
- 12. Grove J, Claeson M, Bryce J, Amouzou A, Boerma T, Waiswa P, et al. Maternal, newborn, and child health and the Sustainable Development Goals—a call for sustained and improved measurement. The Lancet. 2015;386(10003):1511–4. pmid:26530604
- 13. Marchant T, Boerma T, Diaz T, Huicho L, Kyobutungi C, Mershon C-H, et al. Measurement and accountability for maternal, newborn and child health: fit for 2030? BMJ Global Health. 2020;5(7):e002697. pmid:32624502
- 14. Benova L, Moller A-B, Moran AC. “What gets measured better gets done better”: The landscape of validation of global maternal and newborn health indicators through key informant interviews. PLoS One. 2019;14(11):e0224746. pmid:31689328
- 15. Benova L, Moller A-B, Hill K, Vaz LM, Morgan A, Hanson C, et al. What is meant by validity in maternal and newborn health measurement? A conceptual framework for understanding indicator validation. PloS one. 2020;15(5):e0233969. pmid:32470019
- 16. Gabrysch S, Zanger P, Campbell OM. Emergency obstetric care availability: a critical assessment of the current indicator. Tropical medicine & international health. 2012;17(1):2–8. pmid:21831117
- 17. Bailey P, Lobis S, Maine D, Fortney JA. Monitoring emergency obstetric care: a handbook: World Health Organization; 2009.
- 18. Gabrysch S, Civitelli G, Edmond KM, Mathai M, Ali M, Bhutta ZA, et al. New signal functions to measure the ability of health facilities to provide routine and emergency newborn care. PLoS medicine. 2012;9(11):e1001340. pmid:23152724
- 19. Jolivet RR, Moran AC, O’Connor M, Chou D, Bhardwaj N, Newby H, et al. Ending preventable maternal mortality: phase II of a multi-step process to develop a monitoring framework, 2016–2030. BMC pregnancy and childbirth. 2018;18(1):1–13.
- 20. Tembo T, Chongwe G, Vwalika B, Sitali L. Signal functions for emergency obstetric care as an intervention for reducing maternal mortality: a survey of public and private health facilities in Lusaka District, Zambia. BMC Pregnancy and Childbirth. 2017;17(1):1–8.
- 21. Curtis A, Monet J-P, Brun M, Bindaoudou IA-K, Daoudou I, Schaaf M, et al. National optimisation of accessibility to emergency obstetrical and neonatal care in Togo: a geospatial analysis. BMJ open. 2021;11(7):e045891. pmid:34330852
- 22. Ebener S, Stenberg K, Brun M, Monet J-P, Ray N, Sobel HL, et al. Proposing standardised geographical indicators of physical access to emergency obstetric and newborn care in low-income and middle-income countries. BMJ global health. 2019;4(Suppl 5):e000778. pmid:31354979
- 23. World Health Organization. Standards for improving quality of maternal and newborn care in health facilities. 2016.
- 24. Pirkle CM, Fournier P, Tourigny C, Sangaré K, Haddad S. Emergency obstetrical complications in a rural African setting (Kayes, Mali): the link between travel time and in-hospital maternal mortality. Maternal and child health journal. 2011;15(7):1081–7. pmid:20697934
- 25. Gayet-Ageron A, Prieto-Merino D, Ker K, Shakur H, Ageron F-X, Roberts I, et al. Effect of treatment delay on the effectiveness and safety of antifibrinolytics in acute severe haemorrhage: a meta-analysis of individual patient-level data from 40 138 bleeding patients. The Lancet. 2018;391(10116):125–32.
- 26. Kim C, Tappis H, McDaniel P, Soroush MS, Fried B, Weinberger M, et al. National and subnational estimates of coverage and travel time to emergency obstetric care in Afghanistan: Modeling of spatial accessibility. Health & Place. 2020;66:102452. pmid:33011490
- 27. Jolivet RR, Gausman J, Adanu R, Bandoh D, Belizan M, Berrueta M, et al. Multisite, mixed methods study to validate 10 maternal health system and policy indicators in Argentina, Ghana and India: a research protocol. BMJ Open. 2022;12(1):e049685. pmid:35039284
- 28. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of biomedical informatics. 2009;42(2):377–81. pmid:18929686
- 29.
Esri. ArcGIS Pro 2.6 ed. Redlands, California:: Esri, Inc.; 2020.
- 30. WorldPop. Population Counts 2000–2020 UN-Adjusted Unconstrained 100m. 2020.
- 31. Stevens FR, Gaughan AE, Linard C, Tatem AJ. Disaggregating census data for population mapping using random forests with remotely-sensed and ancillary data. PloS one. 2015;10(2):e0107042. pmid:25689585
- 32. Ramm F. OpenStreetMap Data in Layered GIS Format Free Shapefiles. 2019 [Available from: https://download.geofabrik.de/osm-data-in-gis-formats-free.pdf.
- 33. Crosby C, Nandigam V, Baru C, Arrowsmith JR, editors. Opentopography: Enabling online access to high-resolution lidar topography data and processing tools. EGU general assembly conference abstracts; 2013.
- 34.
Wickham H. ggplot2: Elegant Graphics for Data Analysis. New York: Springer-Verlag; 2016.
- 35.
Merriam Webster. Dictionary 2022 [Available from: https://www.merriam-webster.com/dictionary/sufficient.
- 36. Brun M, Monet J-P, Moreira I, Agbigbi Y, Lysias J, Schaaf M, et al. Implementation manual for developing a national network of maternity units: Improving Emergency Obstetric and Newborn Care (EmONC). 2020.
- 37. Kruk ME, Leslie HH, Verguet S, Mbaruku GM, Adanu RM, Langer A. Quality of basic maternal care functions in health facilities of five African countries: an analysis of national health system surveys. The lancet global health. 2016;4(11):e845–e55. pmid:27670090
- 38. Chen YN, Schmitz MM, Serbanescu F, Dynes MM, Maro G, Kramer MR. Geographic Access Modeling of Emergency Obstetric and Neonatal Care in Kigoma Region, Tanzania: Transportation Schemes and Programmatic Implications. Global health science and practice. 2017;5(3):430–45. pmid:28839113
- 39. Kruk ME, Paczkowski M, Mbaruku G, De Pinho H, Galea S. Women’s preferences for place of delivery in rural Tanzania: a population-based discrete choice experiment. American journal of public health. 2009;99(9):1666–72. pmid:19608959
- 40. Echoka E, Kombe Y, Dubourg D, Makokha A, Evjen-Olsen B, Mwangi M, et al. Existence and functionality of emergency obstetric care services at district level in Kenya: theoretical coverage versus reality. BMC Health Services Research. 2013;13(1):113. pmid:23522087
- 41. Ameh C, Msuya S, Hofman J, Raven J, Mathai M, Van Den Broek N. Status of emergency obstetric care in six developing countries five years before the MDG targets for maternal and newborn health. PloS one. 2012;7(12):e49938. pmid:23236357
- 42. Gabrysch S, Simushi V, Campbell OMR. Availability and distribution of, and geographic access to emergency obstetric care in Zambia. International Journal of Gynecology & Obstetrics. 2011;114(2):174–9.
- 43. Bosomprah S, Tatem AJ, Dotse-Gborgbortsi W, Aboagye P, Matthews Z. Spatial distribution of emergency obstetric and newborn care services in Ghana: Using the evidence to plan interventions. International journal of gynecology and obstetrics. 2016;132(1):130–4. pmid:26725855
- 44. Pattinson RC, Makin JD, Pillay Y, Van den Broek N, Moodley J. Basic and comprehensive emergency obstetric and neonatal care in 12 South African health districts. South African medical journal. 2015;105(4):256–60. pmid:26294861
- 45. Tatem AJ, Campbell J, Guerra-Arias M, De Bernis L, Moran A, Matthews Z. Mapping for maternal and newborn health: the distributions of women of childbearing age, pregnancies and births. International journal of health geographics. 2014;13(1):1–11. pmid:24387010