Advertisement
  • Loading metrics

Facility-Based Delivery during the Ebola Virus Disease Epidemic in Rural Liberia: Analysis from a Cross-Sectional, Population-Based Household Survey

Facility-Based Delivery during the Ebola Virus Disease Epidemic in Rural Liberia: Analysis from a Cross-Sectional, Population-Based Household Survey

  • John Ly, 
  • Vidiya Sathananthan, 
  • Thomas Griffiths, 
  • Zahir Kanjee, 
  • Avi Kenny, 
  • Nicholas Gordon, 
  • Gaurab Basu, 
  • Dale Battistoli, 
  • Lorenzo Dorr, 
  • Breeanna Lorenzen
PLOS
x

Abstract

Background

The Ebola virus disease (EVD) epidemic has threatened access to basic health services through facility closures, resource diversion, and decreased demand due to community fear and distrust. While modeling studies have attempted to estimate the impact of these disruptions, no studies have yet utilized population-based survey data.

Methods and Findings

We conducted a two-stage, cluster-sample household survey in Rivercess County, Liberia, in March–April 2015, which included a maternal and reproductive health module. We constructed a retrospective cohort of births beginning 4 y before the first day of survey administration (beginning March 24, 2011). We then fit logistic regression models to estimate associations between our primary outcome, facility-based delivery (FBD), and time period, defined as the pre-EVD period (March 24, 2011–June 14, 2014) or EVD period (June 15, 2014–April 13, 2015). We fit both univariable and multivariable models, adjusted for known predictors of facility delivery, accounting for clustering using linearized standard errors. To strengthen causal inference, we also conducted stratified analyses to assess changes in FBD by whether respondents believed that health facility attendance was an EVD risk factor. A total of 1,298 women from 941 households completed the survey. Median age at the time of survey was 29 y, and over 80% had a primary education or less. There were 686 births reported in the pre-EVD period and 212 in the EVD period. The unadjusted odds ratio of facility-based delivery in the EVD period was 0.66 (95% confidence interval [CI] 0.48–0.90, p-value = 0.010). Adjustment for potential confounders did not change the observed association, either in the principal model (adjusted odds ratio [AOR] = 0.70, 95%CI 0.50–0.98, p = 0.037) or a fully adjusted model (AOR = 0.69, 95%CI 0.50–0.97, p = 0.033). The association was robust in sensitivity analyses. The reduction in FBD during the EVD period was observed among those reporting a belief that health facilities are or may be a source of Ebola transmission (AOR = 0.59, 95%CI 0.36–0.97, p = 0.038), but not those without such a belief (AOR = 0.90, 95%CI 0.59–1.37, p = 0.612). Limitations include the possibility of FBD secular trends coincident with the EVD period, recall errors, and social desirability bias.

Conclusions

We detected a 30% decreased odds of FBD after the start of EVD in a rural Liberian county with relatively few cases. Because health facilities never closed in Rivercess County, this estimate may under-approximate the effect seen in the most heavily affected areas. These are the first population-based survey data to show collateral disruptions to facility-based delivery caused by the West African EVD epidemic, and they reinforce the need to consider the full spectrum of implications caused by public health emergencies.

Author Summary

Why was this study done?

  • The 2014–2015 West African Ebola virus disease (EVD) epidemic killed approximately 11,300 people directly, but its effects are magnified by collateral health consequences.
  • Mathematical models and facility-based data provide evidence of substantial health service interruptions, but little population-based survey data has been published from the three principally affected countries.
  • Maternal health services, which are dependent on functioning health systems, are likely to be particularly susceptible to external shocks, such as the EVD epidemic.

What did the researchers do and find?

  • We used a cluster-sample survey to produce a representative sample of births in Rivercess County, Liberia, a part of the country with relatively limited Ebola transmission.
  • Controlling for potential confounders, we compared the odds of facility-based delivery among 686 births in the period before the EVD epidemic with 212 births during the epidemic.
  • We identified a 30% reduction in the odds of facility-based delivery during the EVD epidemic. The odds of facility-based delivery were 41% lower among women who reported a belief that Ebola was or may be transmitted in health facilities, but not significantly lower among women who reported believing that Ebola was not transmitted in health facilities.

What do these findings mean?

  • This study provides further evidence that the 2014–2015 West African EVD epidemic caused serious collateral harm to health services.
  • This study also underscores the need to maintain focus on long-term health system reconstruction after the EVD epidemic.

Introduction

The 2014–2015 Ebola virus disease (EVD) epidemic killed approximately 11,300 people in West Africa, with sporadic cases continuing to be reported [1]. While these numbers dwarf all prior hemorrhagic fever epidemics, EVD-related morbidity and mortality have been hypothesized to represent only a small fraction of the epidemic’s overall effect on health in the region. Long-term consequences of the epidemic will likely be exacerbated by the loss of over 500 healthcare workers who have died from EVD in the three most affected countries, all of which suffered from pre-existing healthcare worker shortages [2]. In the shorter-term, innumerable health facilities closed or interrupted access to preventive and therapeutic services for non-Ebola conditions [36]. Perhaps as disruptive to healthcare access was widely documented EVD-related stigma and fear of EVD transmission within health facilities, both of which have been hypothesized to decrease demand for services [79]

Recent work has sought to measure these indirect effects of the epidemic on health and health care access. [7,1022] Both modeling studies and analyses of health facility data have begun to shed light on the epidemic’s indirect health consequences, but are limited by assumptions and difficulties disentangling the epidemic’s effects on service utilization and routine data collection. A more precise examination of the collateral consequences to health caused by the Ebola epidemic will serve multiple purposes, including (1) identifying vulnerable points within health systems in resource-limited settings during public health emergencies, (2) improving our understanding of the scope of such shocks, and, perhaps most importantly, (3) signaling to the global community the need for persistent dedication to health systems rebuilding even after the last cases have been detected [2326]

Because effective maternal healthcare delivery is dependent on functioning health facilities and trained personnel, it represents an area of public health particularly susceptible to shocks. Prior to the Ebola epidemic, Liberia had made improving maternal health a national priority, which resulted in substantial gains in facility-based delivery (FBD) rates. Nationally, FBD rates increased from 38% to 56% between the 2007 and 2013 demographic and health surveys (DHS). The proportionate increase was even greater in rural areas, where rates improved from 26% to 46% [27,28]. In sub-Saharan Africa, FBD is a key component of the maternal health service cascade, associated with substantial improvements in maternal and neonatal outcomes when appropriate clinical functions are provided at adequate quality [2933]. Additionally, prior birth location tends to predict delivery location for future pregnancies when services are adequate [34,35,] and FBD is an important step in the continuum of care to post-natal and subsequent young child services [36]. As such, if the Ebola epidemic has reduced FBD, long-term ripple effects would be expected for affected mothers and children.

We analyzed data from a community-based household survey conducted in Rivercess County, Liberia, during March and April, 2015. The survey was designed as a baseline assessment prior to implementation of a community health worker (CHW) program. Survey questions included a complete birth history and location for all deliveries. We constructed a retrospective cohort of deliveries from the cross-sectional survey to estimate FBD rates before and during the Ebola epidemic. The primary aim of this study was to estimate the effect of Ebola on healthcare utilization by assessing changes in FBD in a rural Liberian setting before and during the 2014–2015 epidemic.

Methods

Ethics

This study was approved by the ethics review boards at the Liberian Institute for Biomedical Research, Georgetown University, and Partners Healthcare. All participants provided verbal informed consent prior to participation.

Setting, Sampling Approach, and Survey Administration

Rivercess County is a rural county located in south-central Liberia with approximately 71,500 residents as of the 2008 national census [37]. It had limited Ebola transmission, with 34 confirmed or probable cases reported to the World Health Organization, principally linked to a single cluster in October–November 2014 [3841]. Participants were sampled using a stratified, two-stage cluster-sample approach. The sample was stratified for purposes of the survey’s function as baseline for a stepped-wedge impact evaluation, with two strata corresponding to the intervention’s phased implementation. The third stratum included areas within 5 kilometers (km) of a health facility, where CHWs are not deployed by current Liberian national policy, and which were assessed to provide county-wide estimates for health officials.

Prior to sampling, we enumerated all households in each village in the county. At the first stage, we sampled villages with probability proportionate to size within each stratum using the standard DHS approach: listing clusters with a running cumulative number of households, determining the sampling interval necessary to take the correct number of clusters, randomly determining a starting value, and then selecting each subsequent cluster that corresponded to the sampling interval [4244]. At the second stage, 21 households were selected per cluster in compact segments by (1) spinning a laminated paper triangle on the ground in the village’s center, (2) using a random number generator to randomly select an initial house in the direction pointed between the center and margin of the village, and (3) continuing to the next closest dwelling until 21 households were sampled [45,46]. We surveyed all women aged 18 to 49 in each selected household.

The survey (see S1 Survey) included questions on household wealth, including asset ownership, water and toilet facilities, and housing materials, and maternal health, drawn principally from the 2013 Liberian Demographic and Health Survey [27]. A section on Ebola knowledge, attitudes, and practices was produced by the research team. All questions were translated from American English to Liberian vernacular English and back-translated by bilingual staff to ensure accuracy. Because some respondents were expected to speak only Bassa, a local language without a commonly used written form, bilingual enumerators administered the survey. Prior to survey administration, all enumerators attended a five-day training, which included practice administering the survey as well as training on informed consent, proper use of the mobile platform, survey skip logic, and techniques to reduce bias [47]. Field supervisors (with one supervisor per three-enumerator field teams) observed implementation of surveys daily and ensured quality assurance at the point of survey implementation, and one additional supervisor oversaw all field teams to ensure consistency. Data were entered using Commcare, an Android-based mobile platform, and maintained in a MySQL database, with basic data cleaning conducted prior to exportation for analysis.

Measures

Our primary outcome of interest, having a facility-based delivery, was recorded for all respondents with at least one prior pregnancy. Our primary predictor of interest, whether the delivery occurred during the EVD period, was generated from child dates of birth. June 15, 2014, was chosen to dichotomize the periods before and during the epidemic, because it was the approximate date on which Ebola re-emerged as a broadly perceived national threat and when the first media reports emerged that patients were avoiding health facilities [4854], and because, in limited published hospital record data from elsewhere in the country, June is the earliest month in which there appears a reduction in facility visits [15,16]. The pre-EVD comparison period began 4 y prior to the beginning of survey administration.

In our models, we included potential confounding variables that have been identified as determinants of FBD in prior studies [30,34,45]. These included whether the birth occurred in the rainy or dry season (rainy season is May to October [55]), maternal marital status, household language (Liberian English versus Bassa), birth order (categorized as first, second or third, and fourth or higher), and self-reported maternal education (categorized as none, primary school only, and any secondary school or higher). Maternal age at each birth was calculated by measuring elapsed time between date of birth and the mother’s reported age when surveyed. Because prior studies [30,34,45] have found no consistent relationship between maternal age and FBD to give us an a priori basis on how to include maternal age, we categorized it into quartiles. A household wealth index was constructed using the standard DHS approach of assessing household assets, housing quality, water source, and toilet facilities [56]. The index was constructed using principal components analysis and then assigning relative percentiles of wealth. Prior work has demonstrated that FBD increases consistently with greater wealth [27,28,57]; we investigated the association between FBD and wealth in the pre-Ebola period using locally weighted scatterplot smoothing (LOWESS), which confirmed that the relationship between wealth percentiles and FBD was logit-linear (see S1 Fig). As a result, we included wealth as a continuous variable, rescaling it by dividing by its own interquartile range for ease of interpretation. Road distance from the center of each cluster to the nearest health facility was measured by global positioning systems (GPS) devices (Garmin eTrex 10; Garmin Ltd.). Prior research has shown complex inverse relationships between distance and FBD [45,5861], so we examined the relationship using LOWESS plots, which suggested distance appeared to have a logit-linear, splined relationship with nodes at 10 and 21 km (see supplemental S1 Fig).

Finally, to assess the causality of a relationship between time period and FBD, we included a survey item about whether respondents believed health facilities to be a source of Ebola transmission. We dichotomized this variable by combining respondents who stated they believed health facilities posed a definite or uncertain risk versus those who stated facilities posed no Ebola risk.

Statistical Methods

We used standard summary statistical methods to describe respondents’ demographic and socioeconomic characteristics. We tested differences in respondents’ characteristics before versus during the Ebola period using design-corrected chi-squared analysis for categorical variables. Differences in the distribution of continuous variables, which were not normally distributed, were tested using Somers’ D, an analogue to the Mann-Whitney U test that can accommodate complex sample survey data [62,63].

We fit design-corrected logistic regression models to estimate associations between giving birth during the Ebola epidemic and FBD. While we had a priori bases to expect certain variables to be associated with FBD, we lacked a theoretical basis to expect particular variables to be associated with the EVD period and, therefore, to constitute potential confounders [64]. Therefore, we examined bivariate associations and constructed three multivariable models to assess for associations between the pre- and intra-EVD time period and FBD. The bivariate model assumes that Ebola is completely exogenous, so it presents only unadjusted relationships. In multivariable model 1, which serves as our primary model, we included only those predictors that were associated with both the EVD period and FBD during the pre-Ebola period at or below the p = 0.10 level [57]. Multivariable model 2 includes all variables associated with either Ebola or FBD at or below the p = 0.10 level. Multivariable model 3 includes all considered variables identified in the literature as potentially associated with FBD. Multivariable models 2 and 3 principally serve as robustness checks.

We plotted levels of FBD in the post-Ebola period compared to preceding years adjusted for the controls in multivariable model 1 using predictive margins with covariates held at their observed levels. As a robustness check, we also graphically depicted FBD as a function of continuous calendar time using local polynomial regression (see S1 Appendix).

To test whether observed associations between time period and FBD might be due to secular trends unrelated to Ebola, we conducted a stratified analysis that investigated FBD rates among (1) those who reported a belief that health facilities were a definite or uncertain risk for Ebola transmission and (2) those who believed that health facilities were not a risk for Ebola transmission. Because health facilities did not close in Rivercess County during the epidemic, we considered fear to be the greatest Ebola-related barrier to healthcare utilization in this area [65]. As such, we hypothesized that if Ebola were causally related to alterations in FBD, we would expect to identify a lower rate of FBD during the EVD period among those who perceived that health facilities posed a risk for transmission, and a lesser reduction among those who did not hold this belief. We assessed for differences in FBD rates in these stratified sub-analyses using survey-design-corrected logistic regression.

All analyses accounted for our survey’s stratified design and incorporated clustering at the village and household levels. Taylor linearization was used to adjust standard errors for clustering in all parametric analyses; jackknifed errors were used with Somers’ D because it is not amenable to linearized errors. Because sampling probabilities differed by stratum, sampling weights were incorporated as the inverse probability of selection and corrected for non-response at the stratum level [42,66]. We incorporated finite population corrections at both sampling levels. Based on estimates of FBD rates and survey design effects from prior household-based surveys in rural Liberia [27,45], we estimated that approximately 870 births would be required to detect a 10% reduction in FBD with 80% power, which equated to approximately 4 y of birth data. We used Stata version 14 for all analyses. Replication datasets (S1 Dataset) and statistical code (S1 Code) are provided as online supplements. This study is reported per STROBE guidelines (S1 Checklist). Details of the analysis and any changes to the analysis plan are included in S1 Text.

Sensitivity Analyses

We conducted several sensitivity analyses to assess for robustness to potential biases. First, because we did not survey currently hospitalized women, the survey risked underestimating FBD among recent births. We addressed this by fitting additional models that excluded women who gave birth within 2 wk before survey administration began. Second, we may have excluded births in the pre-EVD period to women who turned 50 before our survey was administered and therefore did not meet inclusion criteria. We addressed this with models restricted to births to women aged 45 or less so that the entire cohort met survey inclusion criteria. Third, we fit models excluding births from women who had moved since their last birth to avoid misallocation of household demographic variables. Fourth, we ran an analysis that combined the preceding three sensitivity analyses. Fifth, because the exact date when the perceived threat of EVD became salient is unknown (and likely varied between people), we varied our a priori definition of the start of the EVD epidemic. The three alternate dates chosen were: May 29, 2014, when Liberia’s second wave began; July 15, 2014, by which point Ebola transmission was widespread in the country; and August 6, 2014, when Liberia declared a national emergency. Sixth, because inclusion of more years in the control period may increase susceptibility to bias from secular trends, we ran analyses restricted to only 1 and 2 y of control data. Seventh, because date-of-birth heaping was observed for the first day of each month, we randomly redistributed these birth dates across each month, which was expected to have a mild effect on both inclusion and allocation between the pre-EVD and EVD periods. Finally, because variance estimation is sometimes sensitive to the approach chosen [67,68], we present findings with standard errors calculated by the jackknife method instead of linearization.

Results

Ninety-four percent of intended households were surveyed. A total of 1,298 women out of 1,319 eligible respondents (98%) from 941 households completed a birth history. Missing data were rare: <1% of observations for all variables except belief about health facility transmission risk (5%). Median age at the time of survey was 29 y, and 86.3% had less than secondary education. Among the 898 births during the study period, 686 occurred during the pre-Ebola period and 212 in the Ebola period. Respondent characteristics were similar between the two periods (Table 1), except that the median wealth index score was lower among households with births in the Ebola period (p < 0.001), and primary school attendance was higher (p = 0.002).

thumbnail
Table 1. Demographic and social characteristics, Rivercess, Liberia, by births that occurred before and during the Ebola period.

https://doi.org/10.1371/journal.pmed.1002096.t001

In the unadjusted analysis, birth during the Ebola period was associated with a 34% reduction in the odds of FBD (Table 2; OR = 0.66, 95% CI 0.48–0.90, p = 0.010). After adjustment for household wealth and maternal education, the odds of FBD were 30% lower during the Ebola period (AOR = 0.70, 95% CI 0.50–0.98, p = 0.037). This corresponds to a decrease in the adjusted FBD rate from 70.4% (95% CI 66.4–74.4%) in the pre-Ebola period to 62.9% (95% CI 54.9–70.8%) during the EVD epidemic (See Fig 1). These results did not change meaningfully after addition of other covariates (Table 2).

thumbnail
Table 2. Logistic regression models of facility-based delivery among women of child-bearing age in Rivercess County, Liberia, 2011–2015.

https://doi.org/10.1371/journal.pmed.1002096.t002

thumbnail
Fig 1. Proportion of women with facility-based delivery in Rivercess County, Liberia, during 2011–2014.

Years begin in June, such that the 2014 estimate corresponds to the generalized Ebola virus disease epidemic in the country (June 2014–April 2015). Error bars represent 95% confidence intervals.

https://doi.org/10.1371/journal.pmed.1002096.g001

Belief that health facilities posed a risk of EVD transmission was reported by 56.0% (95% CI 51.5–60.4) of respondents. Adjusting for the same potential confounders as in multivariable model 1, the odds of FBD were 41% lower (AOR = 0.59, 95% CI 0.36–0.97, p = 0.038) during the EVD epidemic then prior among women who reported believing that health facilities are or may be a source of Ebola transmission. In contrast, no significant difference was detected between time periods among those who did not report such a belief (AOR = 0.90, 95% CI 0.59–1.37, p = 0.612). Unadjusted and adjusted results are presented in S1 and S2 Tables.

The observed association between delivery during the Ebola period and FBD varied little in the sensitivity analyses (S2S14 Tables). Observed odds ratio point estimates remained between 0.69 and 0.79 and significant or marginally significant in all analyses but one. The one exception was an analysis that selected August 6 as the start of the Ebola period (S13 Table), for which the relationship lost statistical significance (AOR = 0.79, 95% CI 0.56–1.12, p = 0.186). In this same conceptualization, birth during the Ebola period remained significantly associated with lower FBD among those who believed health facilities to be an Ebola transmission source (AOR = 0.60, 95% CI 0.36–0.99, p = 0.044) but not among those who did not report this belief (OR = 1.03, 95% CI 0.63–1.70, p = 0.902).

Discussion

This study is the first to our knowledge to leverage population-based survey data to examine relationships between the West African Ebola epidemic and facility-based delivery. Importantly, our survey was conducted in a county of Liberia with relatively few confirmed Ebola cases and in which health facilities never officially closed. We estimate a 30% reduction in the odds of facility-based delivery during the Ebola period in this area of the country. This corresponds to an approximately 8-percentage-point reduction in FBD, which is substantial in a country where FBD had only increased by 20 percentage points since the first post-war survey in 2007 [27,28]. If the same reduction in FBD we identified were extrapolated to all of rural Liberia, we would estimate approximately 5,900 deliveries would occur outside health facilities because of fear or other Ebola-related barriers to facility-based delivery (see S2 Appendix). However, this figure might substantially underestimate the impact in certain high-burden areas, where the Ebola epidemic resulted in facility closures. We observed the reduction in FBD to only be significant among those who perceived health facilities to be a possible or definite risk for EVD transmission, suggesting that Ebola-related fear was a major factor in the decrease in health utilization.

Our results are in keeping with prior assessments of the collateral effects of the West African Ebola epidemic on non-Ebola health outcomes. One modeling study by Takahashi et al. estimated that vaccine disruptions may be sufficiently large to undermine population immunity against measles and other childhood illnesses [10]. Another model by Parpia and colleagues suggests that excess deaths from interruptions to HIV/AIDS, tuberculosis, and malaria programs may rival direct Ebola deaths in the three most affected countries [22]. Walker et al.’s model suggested that treatment interruptions would increase the number of untreated malaria cases by 3.5 million in 2014 [11].

Studies based on health management information systems (HMIS) data have also suggested deleterious health consequences. Two studies of maternal health visits in heavily affected Liberian counties where facilities closed or non-essential services were restricted found reductions of approximately 80% in recorded services [7,20]. Hospital record studies have shown similar reductions in maternal health utilization in Guinea [12] and lesser but substantial reductions in Sierra Leone [21]. Others found substantial reductions in surgical services and outpatient services [13,19]. A Guinean study found a 42% increase in HIV treatment default at the country’s largest treatment site [14], and a Liberian study similarly found a 41% reduction in HIV clinic visits from May to June at a Monrovia hospital in which a healthcare worker was infected in June [15]. Another Liberian study found significant degradation in the total number of visits, number of new patients, and delays for follow-up among HIV patients at two hospitals in Monrovia [16]. Guinean reviews of clinic records found comparable reductions in HIV testing and enrollment, tuberculosis diagnoses, and outpatient care in one study [17] and substantial reductions in malaria treatment in another [18].

Our study is unique from prior reports in the use of population-based survey data, which provides two important methodological benefits. First, our study design, using community-based, population-representative sampling, enables population-wide estimation and reduces selection bias from clinic-based samples. Second, our analysis avoids confounding, which might occur in health records and HMIS-based studies, in which timely, accurate, and/or complete recordkeeping interruptions might also be impacted by public health shocks.

Because our study is observational, causality can be inferred but not proven [69,70]. Causal inference in our study is supported by two factors. First, the Ebola epidemic was exogenously introduced and therefore unrelated to secular trends in the Liberian health system. Second, a stratified analysis demonstrated decreased FBD during the Ebola period, principally among women reporting a belief that health facilities were potential EVD transmission sites. The principal threat to causality is secular changes coincident with the Ebola epidemic, which independently reduced FBD. However, this risk is mitigated by several aspects of our study, including Ebola’s exogeneity. Furthermore, FBD increased in Liberia in the years preceding the Ebola epidemic [27,28], so secular trends would be expected to bias our results toward the null. Finally, our findings are robust to sensitivity analyses, including variations in the EVD start date and definitions of inclusion criteria.

Our study is also subject to the standard limitations of population-based surveys, including potential selection and non-response biases, recall errors, and response biases [71,72]. We mitigated selection bias by enumerating villages immediately prior to sampling, and our response rates were within customary limits for demographic and health surveys [73]. We limited response biases by utilizing broadly accepted demographic and health survey items. Recall errors cannot be excluded, but the principal threat to our findings would be if women who delivered at home erroneously reported delivering in a health facility during the pre-Ebola period but not during the Ebola period, and if countervailing error did not occur among women who delivered in a facility. Finally, we cannot exclude the possibility of social desirability bias, which could affect FBD reporting in either direction: over the long term, it most likely causes over-reporting, as Liberia has prioritized FBD, but there is a chance of underreporting if facility visits during the epidemic were stigmatized. This risk should be mitigated, because virtually all women reported deliveries older than Ebola’s incubation period (and, therefore, possibility of infection).

Our findings should be generalizable to similar regions of the three principally affected countries. In Liberia, most of the southern and eastern parts of the country had sporadic cases, as was seen in the catchment area studied here. This was also the case in many parts of rural Guinea and, to a lesser degree, Sierra Leone [1]. Several risk perception surveys and qualitative analyses have documented substantial fears about acquiring Ebola from health facilities in Sierra Leone and Liberia [65,74,75]. In contrast, our data are less likely to represent effects in regions with differing epidemic burdens. For example, one would expect greater FBD reductions in areas with a greater EVD caseload, areas with facility closures, and/or areas with decreased availability of healthcare workers or supplies, such as in Monrovia, where routine health services largely ceased during the epidemic’s peak. In such areas, our results likely underestimate the total collateral effects of the epidemic. Future work to document health disruptions through population-based surveys in a wider variety of locations would be valuable, as would additional modeling studies that incorporate greater subnational heterogeneity in epidemic intensity.

Our findings have implications for countries with ongoing Ebola epidemics and for post-Ebola public health initiatives. Though health systems were most drastically affected in locations where Ebola transmission directly affected healthcare workers and facilities, our results suggest that significant collateral health effects also occurred in relatively lightly affected regions. This finding reinforces a need to strengthen and maintain basic services amidst public health emergencies, and also that risk communication strategies to prevent and curtail stigma are vital to maintaining demand for health services. Our results also add support to recent calls from within West Africa and the greater global health community to invest in health systems strengthening in the region [26,76,77]. Both our results and those of prior studies reinforce that the need will extend beyond epidemiologic surveillance and EVD vaccination to include investments in basic healthcare delivery, even in areas distant from the epicenters of the epidemic.

Indeed, efforts to maintain basic, primary health care should be considered an essential part of the response to outbreaks and other emergencies. Anecdotal reports from affected locations include the interruption of a wide range of essential services—from family planning outreach to fever care and community vaccination campaigns—even in communities where facility-based services continued. While building and staffing Ebola treatment units was a critical part of the public health response, in future epidemics it is important to incorporate efforts to maintain staffing and services to address routine health needs. Additionally, delegating services to community health workers or other community-based providers may enable continuity of routine services while also providing inroads into communities that can strengthen epidemic surveillance and response [7882]. Finally, though our analysis examines collateral healthcare consequences, the epidemic also appears to have exacerbated social vulnerability, including worsened poverty, the loss of a year of education, increased child labor, and broad psychosocial consequences [83]. Increasing resiliency to epidemics and other emergencies should be a top global priority [84].

In the immediate term, health ministries in Ebola-affected locations and their partners should redouble efforts to restore FBD and other health service utilization to pre-Ebola levels and to continue positive trajectories achieved before the epidemic. For instance, the Liberian Ministry of Health’s Post-Ebola Investment Plan calls for not only strengthening epidemic surveillance and control efforts but also achieving universal health coverage—including the creation of a National Health Workforce Program that will lead to the deployment of thousands of new primary health care workers, including rural nurses, midwives, and community health workers. Such strategies will require a restoration of trust and amelioration of fear of health facilities [65]. Active engagement of community leaders; effective, culturally appropriate, and evidence-based communication strategies; and involvement of trusted local health and lay figures will likely be required as affected countries rebuild their health sectors [4,85].

Supporting Information

S1 Appendix. Smoothed graph of facility-based delivery as a function of a continuous date of birth variable.

https://doi.org/10.1371/journal.pmed.1002096.s001

(DOC)

S2 Appendix. Estimate of the reduction in rural Liberian births occurring in health facilities (not including facility closures).

https://doi.org/10.1371/journal.pmed.1002096.s002

(XLSX)

S1 Code. Replication statistical code (Stata format).

https://doi.org/10.1371/journal.pmed.1002096.s004

(DO)

S1 Dataset. Replication dataset in Stata format.

https://doi.org/10.1371/journal.pmed.1002096.s005

(DTA)

S2 Dataset. Replication dataset in Excel format.

https://doi.org/10.1371/journal.pmed.1002096.s006

(XLSX)

S1 Fig. LOWESS curves of facility-based delivery by distance and wealth.

https://doi.org/10.1371/journal.pmed.1002096.s007

(EPS)

S1 Table. Analysis restricted only to those respondents who reported a belief that clinics did or may pose an Ebola transmission risk.

https://doi.org/10.1371/journal.pmed.1002096.s009

(DOCX)

S2 Table. Analysis restricted only to those respondents who reported a belief that clinics did not pose an Ebola transmission risk.

https://doi.org/10.1371/journal.pmed.1002096.s010

(DOCX)

S3 Table. Sensitivity analysis: restricted to only households who lived in current village at the time of the delivery.

https://doi.org/10.1371/journal.pmed.1002096.s011

(DOC)

S4 Table. Sensitivity analysis: excludes births within 2 wk before the survey began.

https://doi.org/10.1371/journal.pmed.1002096.s012

(DOC)

S5 Table. Sensitivity analysis: excludes women over age 45 at time of birth to avoid possible bias from attrition.

https://doi.org/10.1371/journal.pmed.1002096.s013

(DOC)

S6 Table. Sensitivity analysis: excludes births within 2 wk of the survey, women over age 45 at the time of delivery, and women who lived in a different village at the time of delivery.

https://doi.org/10.1371/journal.pmed.1002096.s014

(DOC)

S7 Table. Sensitivity analysis: includes observations from 2012 to 2015.

https://doi.org/10.1371/journal.pmed.1002096.s015

(DOC)

S8 Table. Sensitivity analysis: includes observations from 2013 to 2015.

https://doi.org/10.1371/journal.pmed.1002096.s016

(DOC)

S9 Table. Sensitivity analysis: includes observations from 2010 to 2015.

https://doi.org/10.1371/journal.pmed.1002096.s017

(DOC)

S10 Table. Sensitivity analysis: jackknifed standard errors instead of Taylor linearization.

https://doi.org/10.1371/journal.pmed.1002096.s018

(DOC)

S11 Table. Sensitivity analysis: Ebola period begins on May 29, 2014, when Liberia’s second wave began.

https://doi.org/10.1371/journal.pmed.1002096.s019

(DOC)

S12 Table. Sensitivity analysis: Ebola period begins on July 15, 2014, by which point Ebola was a national epidemic.

https://doi.org/10.1371/journal.pmed.1002096.s020

(DOC)

S13 Table. Sensitivity analysis: Ebola period begins on August 6, 2014, when Liberia’s president declared a national emergency.

https://doi.org/10.1371/journal.pmed.1002096.s021

(DOC)

S14 Table. Sensitivity analysis: random redistribution of heaped birth dates from first day of month.

https://doi.org/10.1371/journal.pmed.1002096.s022

(DOC)

S1 Text. Analysis history for the analysis reported in this study.

https://doi.org/10.1371/journal.pmed.1002096.s023

(DOCX)

Author Contributions

Conceived and designed the experiments: VS ZK AK GB DRT MJS JDK. Performed the experiments: VS TG NG AK DB LD BL. Analyzed the data: JDK MJS. Wrote the first draft of the manuscript: JDK. Contributed to the writing of the manuscript: JL VS TG ZK AK NG GB DB LD BL DRT AW UGM RR WLS MJS. Agree with the manuscript’s results and conclusions: JL VS TG ZK AK NG GB DB LD BL DRT AW UGM RR WLS MJS JDK. Interpreted Data: JL VS TG ZK AK NG GB DB LD BL DRT AW UGM RR WLS MJS JDK. All authors have read, and confirm that they meet, ICMJE criteria for authorship.

References

  1. 1. World Health Organization. Ebola Situation Report—16 March 2016. 2016; Available at: http://apps.who.int/ebola/current-situation/ebola-situation-report-16-march-2016. Accessed March 20, 2016.
  2. 2. Evans DK, Goldstein M, Popova A. Health-care worker mortality and the legacy of the Ebola epidemic. The Lancet Global Health 2015 2015/07.
  3. 3. World Health Organization. Ebola Situation Report—5 August 2015. 2015. http://apps.who.int/ebola/current-situation/ebola-situation-report-5-august-2015. Accessed August 6, 2015.
  4. 4. Abramowitz SA, McLean KE, McKune SL, Bardosh KL, Fallah M, Monger J, et al. Community-centered responses to Ebola in urban Liberia: the view from below. PLoS Negl Trop Dis 2015 Apr 9;9(4):e0003706. pmid:25856072
  5. 5. Hamel MJ, Slutsker L. Ebola: the hidden toll. Lancet Infect Dis 2015 Apr 23.
  6. 6. WHO Ebola Response Team. Ebola virus disease in West Africa—the first 9 months of the epidemic and forward projections. N Engl J Med 2014 Oct 16;371(16):1481–1495. pmid:25244186
  7. 7. Iyengar P, Kerber K, Howe CJ, Dahn B. Services for mothers and newborns during the ebola outbreak in liberia: the need for improvement in emergencies. PLoS Curr 2015 Apr 16;7:
  8. 8. Helleringer S, Noymer A. Assessing the direct effects of the ebola outbreak on life expectancy in liberia, sierra leone and Guinea. PLoS Curr 2015 Feb 19;7:
  9. 9. De Roo A, Ado B, Rose B, Guimard Y, Fonck K, Colebunders R. Survey among survivors of the 1995 Ebola epidemic in Kikwit, Democratic Republic of Congo: their feelings and experiences. Trop Med Int Health 1998 Nov;3(11):883–885. pmid:9855400
  10. 10. Takahashi S, Metcalf CJ, Ferrari MJ, Moss WJ, Truelove SA, Tatem AJ, et al. Reduced vaccination and the risk of measles and other childhood infections post-Ebola. Science 2015 Mar 13;347(6227):1240–1242. pmid:25766232
  11. 11. Walker PG, White MT, Griffin JT, Reynolds A, Ferguson NM, Ghani AC. Malaria morbidity and mortality in Ebola-affected countries caused by decreased health-care capacity, and the potential effect of mitigation strategies: a modelling analysis. Lancet Infect Dis 2015 Apr 23.
  12. 12. Delamou A, Hammonds RM, Caluwaerts S, Utz B, Delvaux T. Ebola in Africa: beyond epidemics, reproductive health in crisis. Lancet 2014 Dec 13;384(9960):2105-6736(14)62364-3.
  13. 13. Bolkan HA, Bash-Taqi DA, Samai M, Gerdin M, von Schreeb J. Ebola and indirect effects on health service function in sierra leone. PLoS Curr 2014 Dec 19;6:
  14. 14. Ndawinz JD, Cisse M, Diallo MS, Sidibe CT, D'Ortenzio E. Prevention of HIV spread during the Ebola outbreak in Guinea. Lancet 2015 Apr 11;385(9976):1393-6736(15)60713-9. pmid:25890415
  15. 15. Tattevin P, Baysah MK, Raguin G, Toomey J, Chapplain JM, Taylor ME, et al. Retention in care for HIV-infected patients in the eye of the Ebola storm: lessons from Monrovia, Liberia. AIDS 2015 Mar 27;29(6):N1–2. pmid:25849843
  16. 16. Loubet P, Mabileau G, Baysah M, Nuta C, Taylor M, Jusu H, et al. Likely effect of the 2014 Ebola epidemic on HIV care in Monrovia, Liberia. AIDS 2015;Publish Ahead of Print.
  17. 17. Leuenberger D, Hebelamou J, Strahm S, De Rekeneire N, Balestre E, Wandeler G, et al. Impact of the ebola epidemic on general and HIV care in Macenta, forest guinea, 2014. AIDS 2015;Publish Ahead of Print.
  18. 18. Plucinski MM, Guilavogui T, Sidikiba S, Diakite N, Diakite S, Dioubate M, et al. Effect of the Ebola-virus-disease epidemic on malaria case management in Guinea, 2014: a cross-sectional survey of health facilities. Lancet Infect Dis 2015 Jun 23.
  19. 19. Barden-O'Fallon J, Barry MA, Brodish P, Hazerjian J. Rapid Assessment of Ebola-Related Implications for Reproductive, Maternal, Newborn and Child Health Service Delivery and Utilization in Guinea. PLoS Curr 2015 Aug 4;7:
  20. 20. Lori JR, Rominski SD, Perosky JE, Munro ML, Williams G, Bell SA, et al. A case series study on the effect of Ebola on facility-based deliveries in rural Liberia. BMC Pregnancy Childbirth 2015 Oct 12;15:254-015-0694-x.
  21. 21. Brolin Ribacke KJ, van Duinen AJ, Nordenstedt H, Hoijer J, Molnes R, Froseth TW, et al. The Impact of the West Africa Ebola Outbreak on Obstetric Health Care in Sierra Leone. PLoS ONE 2016 Feb 24;11(2):e0150080. pmid:26910462
  22. 22. Parpia AS, Ndeffo-Mbah ML, Wenzel NS, Galvani AP. Effects of Response to 2014–2015 Ebola Outbreak on Deaths from Malaria, HIV/AIDS, and Tuberculosis, West Africa. Emerg Infect Dis 2016 Mar;22(3):433–441. pmid:26886846
  23. 23. Kruk ME, Myers M, Varpilah ST, Dahn BT. What is a resilient health system? Lessons from Ebola. Lancet 2015 May 9;385(9980):1910–1912. pmid:25987159
  24. 24. Siedner MJ, Gostin LO, Cranmer HH, Kraemer JD. Strengthening the Detection of and Early Response to Public Health Emergencies: Lessons from the West African Ebola Epidemic. PLoS Med 2015;12(3):e1001804. pmid:25803303
  25. 25. Gostin LO, Friedman EA. A retrospective and prospective analysis of the west African Ebola virus disease epidemic: robust national health systems at the foundation and an empowered WHO at the apex. Lancet 2015 May 9;385(9980):1902–1909. pmid:25987158
  26. 26. Heymann DL, Chen L, Takemi K, Fidler DP, Tappero JW, Thomas MJ, et al. Global health security: the wider lessons from the west African Ebola virus disease epidemic. Lancet 2015 May 9;385(9980):1884–1901. pmid:25987157
  27. 27. Liberia Institute of Statistics and Geo-Information Services (Monrovia). Liberia Demographic and Health Survey 2013. Liberia Institute of Statistics and Geo-Information Services; 2014.
  28. 28. Liberia Institute of Statistics and Geo-Information Services (Monrovia). Liberia Demographic and Health Survey 2007. Liberia Institute of Statistics and Geo-Information Services; 2008.
  29. 29. Campbell OM, Graham WJ, Lancet Maternal Survival Series steering group. Strategies for reducing maternal mortality: getting on with what works. Lancet 2006 Oct 7;368(9543):1284–1299. pmid:17027735
  30. 30. Moyer CA, Mustafa A. Drivers and deterrents of facility delivery in sub-Saharan Africa: a systematic review. Reprod Health 2013 Aug 20;10:40-4755-10-40.
  31. 31. Moyer CA, Adanu RM, Engmann CM. The relationship between facility-based delivery and maternal and neonatal mortality in Sub-Saharan Africa. Int J Gynaecol Obstet 2013 Sep;122(3):263–265. pmid:23809657
  32. 32. Darmstadt GL, Bhutta ZA, Cousens S, Adam T, Walker N, de Bernis L, et al. Evidence-based, cost-effective interventions: how many newborn babies can we save? Lancet 2005 Mar 12–18;365(9463):977–988. pmid:15767001
  33. 33. Koblinsky M, Matthews Z, Hussein J, Mavalankar D, Mridha MK, Anwar I, et al. Going to scale with professional skilled care. Lancet 2006 Oct 14;368(9544):1377–1386. pmid:17046470
  34. 34. Gabrysch S, Campbell OM. Still too far to walk: literature review of the determinants of delivery service use. BMC Pregnancy Childbirth 2009 Aug 11;9:34-2393-9-34.
  35. 35. Wanjira C, Mwangi M, Mathenge E, Mbugua G, Ng'ang'a Z. Delivery practices and associated factors among mothers seeking child welfare services in selected health facilities in Nyandarua South District, Kenya. BMC Public Health 2011 May 21;11:360-2458-11-360.
  36. 36. Kerber KJ, de Graft-Johnson JE, Bhutta ZA, Okong P, Starrs A, Lawn JE. Continuum of care for maternal, newborn, and child health: from slogan to service delivery. Lancet 2007 Oct 13;370(9595):1358–1369. pmid:17933651
  37. 37. Liberia Institute of Statistics and Geo-Information Services. 2008 National Population and Housing Census: Final Results. 2009.
  38. 38. Liberia Ministry of Health and Social Welfare. Liberia Ebola Daily Sitrep no. 350 for 30th April 2015. 2015.
  39. 39. Summers A, Nyenswah TG, Montgomery JM, Neatherlin J, Tappero JW, T N, et al. Challenges in responding to the ebola epidemic—four rural counties, Liberia, August-November 2014. MMWR Morb Mortal Wkly Rep 2014 Dec 19;63(50):1202–1204. pmid:25522089
  40. 40. Hagan JE, Smith W, Pillai SK, Yeoman K, Gupta S, Neatherlin J, et al. Implementation of Ebola case-finding using a village chieftaincy taskforce in a remote outbreak—Liberia, 2014. MMWR Morb Mortal Wkly Rep 2015 Feb 27;64(7):183–185. pmid:25719680
  41. 41. Kateh F, Nagbe T, Kieta A, Barskey A, Gasasira AN, Driscoll A, et al. Rapid response to Ebola outbreaks in remote areas—Liberia, July-November 2014. MMWR Morb Mortal Wkly Rep 2015 Feb 27;64(7):188–192. pmid:25719682
  42. 42. ICF International. Sampling and Household Listing Manual: Demographic and Health Surveys Methodology. 2012 September 2012.
  43. 43. Turner AG. Sampling Strategies. Expert Group Meeting to Review the Draft Handbook on Designing of Household Sample Surveys 2003 November 3, 2003;ESA/STAT/AC.93/2.
  44. 44. United Nations. Designing household survey samples: practical guidelines. 2008;Series F, No. 98.
  45. 45. Kenny A, Basu G, Ballard M, Griffiths T, Kentoffio K, Niyonzima JB, et al. Remoteness and maternal and child health service utilization in rural Liberia: A population-based survey. J Glob Health 2015 Dec;5(2):020401. pmid:26207180
  46. 46. Milligan P, Njie A, Bennett S. Comparison of two cluster sampling methods for health surveys in developing countries. Int J Epidemiol 2004 Jun;33(3):469–476. pmid:15020569
  47. 47. ICF Macro. Training Field Staff for DHS Surveys. 2009.
  48. 48. Williams WCL. Ebola Deaths Turn Redemption Hospital into Ghost Town. 2014. http://frontpageafricaonline.com/index.php/health-sci/1987-ebola-deaths-turn-liberia-s-redemption-hospital-into-ghost-town. Accessed April 30, 2015, 2015.
  49. 49. Front Page Africa. Deadly Ebola Bounces Back in Liberia—10 Unconfirmed Deaths. 2014. http://allafrica.com/stories/201406160729.html. Accessed April 30, 2015.
  50. 50. Gbelewala K. Report—One Dies of Ebola in Voinjama. 2014. http://allafrica.com/stories/201406190955.html. Accessed April 30, 2015.
  51. 51. StarAfrica. Liberia: Fresh Ebola Outbreak in Monrovia. 2014; Available at: http://en.starafrica.com/news/liberia-fresh-ebola-outbreak-in-monrovia.html. Accessed April 30, 2015.
  52. 52. Reuters. Sierra Leone Shuts Borders, Closes Schools to Fight Ebola. 2014. http://www.nytimes.com/reuters/2014/06/11/world/africa/11reuters-health-ebola-leone.html. Accessed April 30, 2015.
  53. 53. Press Associated. Doctors Without Borders: Ebola 'Out of Control'. 2014. http://www.nytimes.com/aponline/2014/06/20/world/africa/ap-af-west-africa-ebola.html. Accessed April 30, 2015.
  54. 54. Press Associated. Liberian Official: 7 More Deaths Linked to Ebola. 2014. http://www.nytimes.com/aponline/2014/06/17/world/africa/ap-af-west-africa-ebola.html. Accessed April 30, 2015.
  55. 55. The World Bank Group. Average Monthly Temperature and Rainfall for Liberia from 1990–2009. 2015. http://sdwebx.worldbank.org/climateportal/index.cfm?page=country_historical_climate&ThisRegion=Africa&ThisCCode=LBR. Accessed April 30, 2015.
  56. 56. Rutstein SO. The DHS Wealth Index: Approaches for Rural and Urban Areas. 2008;No. 60.
  57. 57. Kruk ME, Hermosilla S, Larson E, Vail D, Chen Q, Mazuguni F, et al. Who is left behind on the road to universal facility delivery? A cross-sectional multilevel analysis in rural Tanzania. Trop Med Int Health 2015 Apr 15.
  58. 58. Lohela TJ, Campbell OM, Gabrysch S. Distance to care, facility delivery and early neonatal mortality in Malawi and Zambia. PLoS ONE 2012;7(12):e52110. pmid:23300599
  59. 59. Gabrysch S, Cousens S, Cox J, Campbell OM. The influence of distance and level of care on delivery place in rural Zambia: a study of linked national data in a geographic information system. PLoS Med 2011 Jan 25;8(1):e1000394. pmid:21283606
  60. 60. Mwaliko E, Downing R, O'Meara W, Chelagat D, Obala A, Downing T, et al. "Not too far to walk": the influence of distance on place of delivery in a western Kenya health demographic surveillance system. BMC Health Serv Res 2014 May 10;14:212-6963-14-212.
  61. 61. Gartland MG, Taryor VD, Norman AM, Vermund SH. Access to facility delivery and caesarean section in north-central Liberia: a cross-sectional community-based study. BMJ Open 2012 Oct 31;2(6): Print 2012.
  62. 62. Newson R. Parameters behind nonparametric statistics: Kendall's tau, Somers' D and median differences. Stata Journal 2002;2(1):45–64(20).
  63. 63. Newson R. Confidence intervals for rank statistics: Somers' D and extensions. Stata Journal 2006;6(3):309–334(26).
  64. 64. Weinberg CR. Toward a clearer definition of confounding. Am J Epidemiol 1993 Jan 1;137(1):1–8. pmid:8434568
  65. 65. Dynes MM, Miller L, Sam T, Vandi MA, Tomczyk B, Centers for Disease Control and Prevention (CDC). Perceptions of the risk for Ebola and health facility use among health workers and pregnant and lactating women—Kenema District, Sierra Leone, September 2014. MMWR Morb Mortal Wkly Rep 2015 Jan 2;63(51):1226–1227. pmid:25551595
  66. 66. Yansaneh IS. Construction and Use of Sampling Weights. Expert Group Meeting to Review the Draft Handbook on Designing of Household Sample Surveys 2003 November 3, 2003;ESA/STAT/AC.93/5.
  67. 67. Porto Alves MC, Silva NN. Variance estimation methods in samples from household surveys. Rev Saude Publica 2007 Dec;41(6):938–946. pmid:17992355
  68. 68. Kish L, Frankel MR. Inference from complex samples. Journal of the Royal Statistical Society, Series B 1974;36:1–37.
  69. 69. Hill AB. The Environment and Disease: Association or Causation? Proc R Soc Med 1965 05;58(5):295–300.
  70. 70. Savitz DA. Interpreting epidemiologic evidence: strategies for study design and analysis. Oxford; New York: Oxford University Press; 2003.
  71. 71. Groves Robert M., Fowler Floyd J., Couper Mick., Lepkowski, James M., Singer, Eleanor.,Tourangeau, Roger.,. Survey methodology. Hoboken, N.J.: Wiley; 2009.
  72. 72. Banda JP. Nonsamping Errors in Surveys. Expert Group Meeting to Review the Draft Handbook on Designing of Household Sample Surveys 2003 November 3, 2003;ESA/STAT/AC.93/7.
  73. 73. Vaessen M, Thiam M, Le T. The Demographic and Health Surveys. In: United Nations Statistics Division, editor. Household Sample Surveys in Developing and Transition Countries New York: United Nations; 2005. p. 495–522.
  74. 74. Kobayashi M, Beer KD, Bjork A, Chatham-Stephens K, Cherry CC, Arzoaquoi S, et al. Community Knowledge, Attitudes, and Practices Regarding Ebola Virus Disease—Five Counties, Liberia, September-October, 2014. MMWR Morb Mortal Wkly Rep 2015 Jul 10;64(26):714–718. pmid:26158352
  75. 75. Liberia Ministry of Health and Social Welfare. National Knowledge, Attitudes, and Practices (KAP) Study on Ebola Virus Disease in Liberia. 2015:1–98.
  76. 76. Barbiero VK. It's not Ebola … it's the systems. Global Health: Science and Practice 2014 December 01;2(4):374–375.
  77. 77. Wilkinson A, Leach M. BRIEFING: EBOLA—MYTHS, REALITIES, AND STRUCTURAL VIOLENCE. African Affairs 2014 December 04.
  78. 78. Kamal-Yanni M. Building Resilient Health Systems and Learning from the Ebola Crisis. 2015;Oxfam Briefing Paper 203.
  79. 79. Oliphant NP, Muniz M, Guenther T, Diaz T, Lainez YB, Counihan H, et al. Multi-country analysis of routine data from integrated community case management (iCCM) programs in sub-Saharan Africa. J Glob Health 2014 Dec;4(2):020408. pmid:25520798
  80. 80. Lassi ZS, Haider BA, Bhutta ZA. Community-based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes. Cochrane Database Syst Rev 2010 Nov 10;(11):CD007754. doi(11):CD007754. pmid:21069697
  81. 81. Amouzou A, Morris S, Moulton LH, Mukanga D. Assessing the impact of integrated community case management (iCCM) programs on child mortality: Review of early results and lessons learned in sub-Saharan Africa. J Glob Health 2014 Dec;4(2):020411. pmid:25520801
  82. 82. Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X, van Wyk BE, et al. Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev 2010 Mar 17;(3):CD004015. doi(3):CD004015. pmid:20238326
  83. 83. Risso-Gill I, Finnegan L. Children's Ebola Recovery Assessment: Sierra Leone. 2015.
  84. 84. Seifman R, Kornblet S, Standley C, Sorrell E, Fischer J, Katz R. Think big, World Bank: time for a public health safeguard. Lancet Glob Health 2015 Apr;3(4):e186–7. pmid:25678307
  85. 85. American Anthropological Association. Strengthening West African Health Care Systems to Stop Ebola. 2014.