http://orcid.org/0000-0002-5842-9518
Figures
Abstract
Introduction
We implemented a participatory quality improvement strategy in eight primary health care units of Ethiopia to improve use and quality of maternal and newborn health services.
Methods
We evaluated the effects of this strategy using mixed-methods research. We used before-and-after (March 2016 and November 2017) cross-sectional surveys of women who had children 0–11 months to compare changes in maternal and newborn health care indicators in the 39 communities that received the intervention and the 148 communities that did not. We used propensity scores to match the intervention with the comparison communities at baseline and difference-in-difference analyses to estimate intervention effects. The qualitative method included 51 in-depth interviews of community volunteers, health extension workers, health center directors and staff, and project specialists.
Results
The difference-in-difference analyses indicated that 7.9 percentage points (95% confidence interval [CI]: 1.8–13.9%) increase in receiving skilled delivery care between baseline and follow-up surveys in the intervention area that is attributable to the strategy. The intervention effect on postnatal care in 48 hours of the mother was 15.3% (95% CI: 7.4–23.2). However, there was no evidence that the strategy affected the seven other maternal and newborn health care indicators considered. Interview participants said that the participatory design and implementation strategy helped them to realize gaps, identify real problems, and design appropriate solutions, and created a sense of ownership and shared responsibility for implementing interventions.
Conclusions
Community participation in planning and monitoring maternal and newborn health service delivery improves use of some high-impact maternal and newborn health services. The study supports the notion that participatory community strategies should be considered to foster community-responsive health systems.
Citation: Tiruneh GT, Zemichael NF, Betemariam WA, Karim AM (2020) Effectiveness of participatory community solutions strategy on improving household and provider health care behaviors and practices: A mixed-method evaluation. PLoS ONE 15(2): e0228137. https://doi.org/10.1371/journal.pone.0228137
Editor: Emily A. Hurley, Children's Mercy Hospitals and Clinics Department of Pathology and Laboratory Medicine, UNITED STATES
Received: May 28, 2019; Accepted: January 8, 2020; Published: February 5, 2020
Copyright: © 2020 Tiruneh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The article write-up and publication fee was supported by the Bill & Melinda Gates Foundation, Grant Number OPP1131042. JSI Research & Training Institute, Inc. has provided us support in the form of salaries for authors [GT, AK, NZ, WB]. However, any of the funders did not have role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”
Competing interests: The authors declare that they have no competing interests. The authors have been working for JSI Research & Training Institute, Inc., a commercial company. We declared that this commercial affiliation does not alter our adherence to PLOS ONE policies on sharing data and materials. The senior author of this manuscript has been working for JSI Research and Training Institute Inc. during the design, field work, and report-up of this manuscript. However, he recently left JSI and joined Gates Foundation. We would like to declare that we do not have any conflict of interest with the Foundation-paid staff in preparing this manuscript.
List of abbreviations: ANC, antenatal care; BEmONC, basic emergency obstetric and newborn care; CBDDM, community-based data for decision making; EmONC, emergency obstetric and newborn care; FMOH, Federal Ministry of Health; HEP, Health Extension Program; HEW, health extension worker; HSTP, Health Sector Transformation Plan; IDI, in-depth interview; KMC, kangaroo mother care; L10K, Last Ten Kilometers 2020 project; MgSO4, magnesium sulfate; MNH, maternal and newborn health; PHCU, primary health care unit; PNC, postnatal care; PROM, prolonged premature rupture of membrane; PSBI, possible serious bacterial infection; QI, quality improvement; RMNCH, reproductive, maternal, newborn, and child health; SDG, Sustainable Development Goal; SNNP, Southern Nations, Nationalities and Peoples; WDA, women's development army
Introduction
There is a wealth of evidence on a range of essential interventions to prevent maternal and newborn deaths [1–3]. More than half of all newborn deaths could be averted by providing care during the postpartum period: 30% can be averted with care of small and ill newborns; 12% with care of healthy newborns; and 10% with immediate newborn care [2]. Evidence also suggests that high coverage and quality of essential packages for maternal and newborn health (MNH) services, basic and emergency obstetric care, and postnatal care across the continuum of care could avert about two-thirds of newborn and child deaths [4]. A recent systematic review also shows the continuity of care from antepartum to postpartum periods may reduce the risk of combined newborn, perinatal, and maternal mortality by 15% [5] and reduce neonatal and perinatal mortality risk by 21% and 16%, respectively [6].
However, such life-saving care has not been implemented adequately in low-and-middle-income countries [6–9]. The Ethiopian situation is not different. Despite a three-fold increase in institutional delivery in Ethiopia, from 10% in 2011 to 28% in 2016, there are high dropout rates across the continuum of MNH care [10, 11]. The Ethiopian Demographic and Health Survey 2016 reports that while 62% of pregnant women received skilled antenatal care (ANC) and 28% delivered their babies under the guidance of a skilled person, only 17% received postnatal care (PNC) within 48 hours [10, 11]. Moreover, studies report that some key components are not provided at ANC, delivery, and PNC stages [12–14], even if women visit health facilities. The proportion of women with obstetric complications treated in health facilities (i.e., who met need for emergency obstetric and newborn care [EmONC]) remains very low, though it did increase from 6% in 2008 to 18% in 2016 [15].
Evidence shows that use of MNH services is influenced by myriad socio-demographic, health service-related, and cultural factors [16–26]. Multiple studies find maternal age, parity, lack of time, education, marital status, women’s economic status, residence, and distance to health facility are significant predictors of use of maternity care services [16, 18, 27, 28]. Accordingly, intervention strategies targeting improved use of MNH services should address both supply- and demand-side factors.
Community participation has been promoted as a critical component of a human rights-based approach to primary health care to build a resilient health system [29, 30]. It has been presumed to improve health outcomes, access, equity, acceptability, service quality, and responsiveness [31]. Community participation in MNH program planning and implementation as well as in quality improvement (QI) processes for MNH services is a compelling strategy for improving service quality [30, 32]. However, its impact on QI depends on its proper design of strategy and implementation.
Evidence from developing countries shows that community participation is effective in decreasing newborn mortality [33–35]. Studies also indicate that community participation in health care QI strategies lowers newborn mortality rates. In Malawi, a randomized controlled trial of combined participatory women’s groups and QI at health centers demonstrates a reduction in newborn mortality [36]. Another observational study in Ethiopia indicated that QI approaches increase use of maternal care services [37]. There is, however, no sufficient evidence that community participation is associated with better maternal health outcomes, such as improving service access, use, quality, or responsiveness [33, 34]. Particularly, the effect of community-participation on the uptake of skilled care and immediate postnatal care is not well documented [33, 37–39].
We implemented a participatory community and facility QI intervention, the Participatory Community Solutions (PC-Solutions) strategy to remove supply- and demand-side barriers to desirable MNH care behaviors and practices. This study used mixed-methods research to evaluate the effect of the PC-Solutions strategy on improving MNH care behaviors and practices in selected rural areas of Ethiopia.
In the first phase, the quantitative surveys were conducted to evaluate the effectiveness of the PC-Solution strategy on the use of MNH behaviors and practices. In the second phase, qualitative research was conducted to explain the findings of the quantitative study and facility surveys; document what happened during the PC-Solutions implementation process; understand the complex participatory community QI process; and assess the scalability of the intervention. The qualitative component was to illuminate how the intervention affected outcomes.
Methods
Study settings
Ethiopian health system.
Ethiopia has a three-tiered health system designed to deliver services. The first level is primary health care, which serves an administrative district (woreda) with an average population of about 100K. The second level is a general hospital that serves the catchment population of approximately 10 woredas (about 1 million people), and the third level is a specialized hospital that serves the catchment population of five general hospitals (about 5 million people). In rural woredas, primary care comprises a primary hospital and a primary health care unit (PHCU) of one health center and five satellite health posts for every 25K people in the woreda [40].
Within the PHCU is Ethiopia’s flagship Health Extension Program (HEP), which establishes one health post and deploys two health extension workers (HEWs) in each kebele (a community of about 5,000 people) in the country to provide basic promotive, preventive, and curative health services [41]. To extend the reach of the HEP and mobilize the community and households, the Federal Ministry of Health (FMOH) established a network of women’s development army (WDA) members. Each WDA member is assigned to five households and encourages families to adopt and practice healthy behaviors [42].
The FMOH is committed to achieving the health-related Sustainable Development Goals (SDGs). The targets for improving reproductive, maternal, newborn, and child health (RMNCH) outcomes of its Health Sector Transformation Plan (HSTP) are aligned with the SDGs of reducing maternal and newborn mortality rates [43, 44]. To achieve these objectives, the HSTP aims to strengthen health systems to provide universal access to high-quality promotive, preventive, curative, and rehabilitative services. One of the key aspects of ensuring universal access to health services is engaging communities in the governance and accountability of their health system by monitoring program performance and ensuring service quality [45].
Project description.
JSI Research & Training Institute, Inc. (JSI), with funding from the Bill & Melinda Gates Foundation, implements the Last Ten Kilometers (L10K) 2020 project strategies to involve communities in improving high-impact RMNCH care behavior and practices in 115 woredas in four of the most populous regions of the country (Amhara, Oromia, Southern Nations, Nationalities and Peoples [SNNP], and Tigray), covering about 19 million people, to help meet HSTP MNH targets. Its strategies include community-based data for decision-making (CBDDM), family conversation, and birth notification [46].
CBDDM is used to identify pregnant women and ensure they receive ANC, intrapartum care, and PNC for themselves and their infants [46]. L10K 2020 also introduced a birth notification strategy to promote postnatal care. Family conversation is a forum conducted at the home of a pregnant woman with her family members and relatives who are encouraged to support her during pregnancy, labor, delivery, and the postpartum period to promote birth preparedness and essential newborn care [47].
Intervention description.
The project implemented the PC-Solutions strategy in eight of the 115 L10K 2020 Platform woredas (two in each region) between March 2016 and October 2017. This integrated intervention joined communities (including health posts) and facilities (health centers) and built on two previous JSI interventions; Participatory Community Quality Improvement and Early Care-Seeking and Referral Solutions [37, 48, 49]. PC-Solutions emphasized quality of care throughout the primary health care level by including communities, along with providers and managers at the primary health care level and the woreda health office, as a critical constituency.
The PC-Solutions strategy is a four-step QI process (plan-do-study-act), for MNH services provided at PHCUs (Fig 1). In the first step, a joint situational analysis was conducted in the PHCUs using workflow mapping, client exit interviews, document review, and focus group discussions with mothers and WDA members. Following the assessment, a meeting with community members, HEWs, health center staff, woreda health office staff, and referral hospital staff was held to discuss assessment findings, consolidate points, and identify priority problems and their solutions.
Challenges identified during the assessment included delayed ANC booking, suboptimal use of PNC, and poor quality ANC, obstetric care, and PNC. The main problems were: 1) delayed identification of pregnant women and linking them to health center and health post/HEW; 2) inability of health workers to use partograph; and 3) delayed notification of recently delivered women to HEWs for PNC. Community members also mentioned cultural taboos on disclosing pregnancy before a certain amount of time (i.e., people believe that the pregnancy might end in miscarriage if mothers disclosed earlier); lack of awareness (even among WDA members) of the importance of early ANC booking; suboptimal use of PNC; and late notification of births to the HEWs hindering ability to initiate PNC immediately following birth.
Encouraging early ANC booking, increasing use of PNC, and improving quality of ANC, obstetric care, and PNC were PC-Solutions’ priorities. Its interventions included early identification and notification system of pregnancy and postpartum mothers; introduction of ANC defaulter tracing mechanism through the WDA members; using mentors and peer learning for onsite partograph training for health care providers; introduction of automated monitoring tools at PHCUs to use data for decision making; and establishing health facility QI teams that included community members. The strategy also included monthly follow up and coaching visits from L10K 2020; monthly QI meetings at health centers and in communities for health post staff, HEWs, and WDA members to review data and progress; quarterly learning sessions; and QI refresher training for facilitators (Table 1).
In the first plan of the PC-Solution strategy, early ANC and PNC, continuity of ANC visits, and partograph use were prioritized and implemented over two years. Intrapartum and newborn care quality were introduced after a reassessment at the end of fiscal year 2017.
Quality improvement teams were formed at the health center and community levels and included health center service providers, HEWs, WDA members, and local administrators. The QI team collated and triangulated administrative data from health centers and posts to inform QI cycle plan and study fora.
The overall QI approach of PC-Solutions included joint “planning and acting” and independent kebele- and the health center level “do and study” fora. The kebele-level do and study cycles were facilitated by health center staff with participation of the kebele-level QI team. Community members identified bottlenecks and solutions and helped implement and monitor the process to improve facility quality and performance.
Study design
We evaluated the effects of PC-Solutions strategy using mixed-methods research. Four rounds of cross-sectional surveys of all eight health centers were conducted in March 2016, October 2016, April 2017, and October 2017.
A pre-/post-test nonequivalent group study design was nested within the household surveys of women with children 0–11 months conducted in March 2016 and November 2017 to monitor MNH care behaviors and practices in the 115 L10K 2020 intervention areas [50]. To evaluate the effectiveness of the PC-Solution strategy, changes in household MNH care behavior and practices between the two surveys were compared between L10K 2020 Platform areas with and without the PC-Solutions strategy.
Researchers choose a programmatic qualitative research design to answer the qualitative objectives. The interview technique was face-to-face in-depth interviews (IDIs) of WDA members, HEWs, health center directors, health center staff, and L10K 2020 QI specialists from four PHCUs. The qualitative study was conducted in September 2018.
Sample size and data collection
All eight health centers were visited for the facility survey. Data were collected through interviews with providers and a review of patient records and service statistics.
For the household surveys, the sample size was powered to detect 10 percentage-points difference between two survey periods for an indicator with alpha error set at 0.05; beta error set at 0.20; and cluster survey design-effect set at 2.0 for the comparison area and 1.0 for the intervention area. The point estimates for an indicator at baseline and follow-up were assumed to be 45% and 55%, respectively, to yield the largest sample size to detect the desired change. Accordingly, the sample size for the intervention area was determined to be 400 women with children ages 0–11 months; for the comparison area, the sample size was 800 women with children ages 0–11 months.
The household surveys employed a two-stage cluster sampling method stratified by program domain and region. Within each stratum, kebeles were selected as primary sampling units with the probability of selection being proportionate to population size (first stage); at the second stage, the sampling strategy described by Lemeshow and Robinson (1985) [51] was used to select households with target respondents and interview them. To do so, a kebele was sub-divided into three equal segments (sub-kebeles) and four respondents from each segment were interviewed. To identify the first respondent, the interviewers went to the population center of the segment (the point in the segment where the population is about equally distributed on all sides), spun a pen on the ground, and chose the first household in the direction that the pen pointed after it stopped spinning. Consecutive households were visited until the desired sample size was achieved, moving away from the middle of the segment. If the household had women with children 0–11 months and they consented, they were interviewed.
Kebeles visited for data collection during the baseline survey were revisited during the follow-up survey.
A total of 39 kebeles in the intervention area were selected for the study. The strategy estimated that about 80 kebeles from the comparison area would be available to obtain the required sample size. However, as the L10K 2020 innovations were not scaled-up to other woredas as initially planned, the domain of the comparison area included more woredas. Therefore, the number of kebeles representing the comparison area was 148, and the power of the sample was greater than 80% to detect a minimum of 10 percentage points intervention effect. The final sample size of respondents at baseline and follow-up were 2,268 (473 interventions and 1,795 comparison) and 2,244 (468 intervention and 1,776 comparison), respectively.
Data were collected using a structured interview questionnaire (S2 Appendix) designed into an Android mobile application SurveyCTO collect [52].
An interview guide with open-ended questions was used to capture qualitative information from informants (S3 Appendix). We recruited four research assistants, (one per region), who spoke the local language; who have a health background; and who have experiences in qualitative research, to do the interview. We oriented research assistants to maintain neutral so as to not influence the participants’ responses. Moreover, we thoroughly discussed with the research assistants on interview techniques to engage participants throughout the interview and to get a truthful and honest answer.
Stratified purposive sampling schemes were used to obtain in-depth information. First, we selected one intervention PHCU from each region to gather detail and contextually relevant data. Then, in each PHCU, staff from the health center and all health posts and selected active WDA members in catchment kebeles were recruited for the study. The research team interviewed the PHCU director, PHCU staff who were actively participating in the implementation of the PC-Solutions strategy, and those who facilitated the community-level QI cycle. We also interviewed the L10K 2020 technical specialists for the PC-Solutions strategy in each region. Through the facilitation of the PHCU director and regional staff, the research team approached HEWs and WDA members and invited them to participate in IDIs at the health post and their home, respectively.
Theoretical sampling technique was used to collect rich information from community health workers until saturation of categories with data is achieved. In-depth interviews of HEWs and WDAs were conducted until saturation of information is reached. All health workers who were actively participating in the implementation of the PC-Solutions strategy and L10K 2020 QI specialists were included. Accordingly, 51 IDIs were conducted with WDAs, HEWs, health center directors, health center staff and L10K 2020 QI specialists (Table 2). All recruited participants participated, no one refused to participate in the study.
Measurements
The dependent variables of interest were the household and provider MNH care behaviors and practices that were expected to be affected by the intervention measured by the household survey.
MNH care indicators and facility readiness and performance definitions are in Table 3.
The independent variables that were considered as potential confounders were the individual-, household-, and kebele-level sample characteristics and administrative regions. The individual-level characteristics considered were age, education, marital status, parity, and religion; the household-level characteristics were wealth and distance of the respondents’ household from the nearest health facility; and the kebele-level characteristic considered was region.
The wealth index score was constructed for each household with the principal component analysis of the household possessions (electricity, watch, radio, television, mobile phone, telephone, refrigerator, table, chair, bed, electric stove, and kerosene lamp), and household characteristics (type of latrine and water source). The households were ranked according to the wealth score and then divided into five quintiles [53].
Analysis
Stata 15.1 was used for the statistical analysis conducted for this study [54]. The health facility indicators were presented by survey periods.
The background characteristics of household respondents were compared between study arms at baseline and at follow-up survey periods using Pearson’s chi-squared statistics. Similarly, the unadjusted MNH indicators were compared.
To estimate the adjusted intervention effects, the propensity scores were first estimated for each kebele using a logit model that predicted the kebeles in the intervention area at baseline. The covariates of the logit model were kebele averages of individual and household characteristics at baseline, kebele averages of MNH care behavior and practice indicators at baseline, kebele characteristics at baseline, and administrative regions. Covariates that had less than 0.2 p-value in the logit model were dropped using stepwise-backward selection [55, 56]. The final logit model from the stepwise procedure included the following covariates: education, religion, administrative region, first ANC, ANC in the first trimester, complete ANC, ANC experience score, ANC in the first and last trimester, PNC within 48 hours at home and at facility for the mother, and home birth notification. Intervention and comparison kebeles with similar propensity scores at baseline were coded so that they could be identified as similar.
To assess the adequacy of the matching, t-tests were performed to ensure that the covariates of the final logit model were not statistically significantly (p>0.1) between the intervention and the control kebeles, after accounting for the matched kebeles.
Finally, intervention effects (difference-in-difference) was estimated from kebele-level random effect models predicting the outcome of interest with indicator variables for study arm, survey period, the interaction term between study arm and survey period, and for the kebeles that matched between the intervention and comparison areas (dummy variables) as the predictors. Stata’s ‘margins’ command was used to obtain adjusted estimates of the outcomes of interest according to study arm and survey period and the difference-in-difference (DiD) (i.e. intervention effects).
For the qualitative component of the study, audio records from IDIs were transcribed verbatim. The data were analyzed thematically. The transcript texts were manually coded. Then, themes were derived from the data coded. The codes, categories and the concepts emerged from an interview group were verified by linking the emerging categories with the data received from another group of informants to improve the trustworthiness of the qualitative data analysis. These categories were also linked to quotes from the research informants to ensure the reliability of the study. During reporting, participant quotations are presented to illustrate the themes/findings.
Ethics approval and consent to participate
Ethical clearances for the surveys were obtained from the ethical review boards of Amhara, Oromia, SNNP, and Tigray Regional Health Bureaus, and JSI. All participants were informed about the purpose of the study; benefits and hazards of the study were explained to all study participants, and each participant was notified of his/her right to opt out when responding to questions. Verbal consent was sought and documented before conducting any interviews. If the respondent was younger than 18 years old, consent was sought from her husband or guardian. Because the majority of the respondents were not expected to be able to read or write; written consent was not sought. If the respondent agreed to be interviewed after listening to the consent statement, the interviewer marked the questionnaire as consent given below the consent statement and signed below that. The interviewer continued with the interview only after receiving and documenting consent. The survey protocol submitted to the ethical review committee included the study questionnaire with the statement that described the consent-obtaining procedure. Moreover, the information obtained from the research participants was kept private (codes were used during reporting of the IDI quotes).
Results
The evaluation results are presented as follows; first, the MNH indicators and BEmONC functions from the facility surveys; followed by, the quasi-experimental study findings; and finally, the qualitative study findings.
The MNH indicators and the BEmONC functions from the facility survey
Performance of BEmONC signal functions.
Based on the definition of functional BEmONC facility [57], about half of the health centers were ready to provide the seven functions, but only 13% were doing so in the last three months. Likewise, while 69% of them were ready to provide six BEmONC functions, only about 13% of health centers were doing so, with no significant change over time (Table 4).
A significant number of health centers lacked provision of parenteral anticonvulsants and manual removal of placenta. Moreover, most lacked readiness and/or provision of most newborn signal functions including KMC, alternate feeding for non-breastfeeding babies, antibiotics for preterm premature rupture of membrane, corticosteroids for preterm labor, and antibiotics for sepsis.
Coverage and quality of the facility-level MNH interventions.
Integrated maternal and newborn health care of mothers who gave birth in the last month at the surveyed health centers were retrieved for review. In each period, about 36 delivery records (mothers’ chart) per health center (a total of 1,014) were reviewed to assess the active management of third-stage labor and use of partograph. Seventy-six percent (774) of them used partograph and 86% (870) mothers were given prophylactic uterotonics for AMTSL to prevent PPH. Fig 2 shows that trends in partograph use and administration of prophylactic uterotonics increased over the survey period.
During each survey period, data on number of deliveries, delivery outcomes, obstetric complications managed, low-birth-weight babies, and newborn sepsis and asphyxia were collected retrospectively for six months from surveyed health centers to understand the service use. The syphilis-testing rate increased from 26% in March 2016 to 38% in October 2017. The facility delivery rate increased from 57% in March 2016 to 64% in October 2017. The fulfilled need for BEmONC stalled at about 18% on average. Also, the percent of expected possible serious bacterial infection (PSBI) cases managed at health centers declined over the survey period.
The proportion of asphyxiated newborns managed with Ambu-bag and mask increased from 29% to 93% during the last two years. The proportion of preterm/low-birth-weight babies initiated KMC increased from 13% to 92% in the same period.
Sample characteristics of the household survey respondents
As depicted in Table 5, respondents were significantly different in their education distribution between the study arms. During both survey periods, intervention arm respondents were more likely to have higher education. However, during both survey periods, respondents of the two study arms were similar in terms of age, administrative region, and household wealth.
Unadjusted estimates of MNH indicators
At baseline, the intervention arm had significantly higher coverage of MNH indicators including first ANC, ANC 4, complete ANC, ANC experiences mean score, birth notification, and use of motor vehicle transport than the comparison arm. On the other hand, the intervention arm had significantly lower coverage of ANC counseling mean score and perceived knowledge of providers. Likewise, at follow-up period, the intervention arm had higher coverage of MNH indicators except for ANC 4+, complete ANC, ANC counseling score, women’s satisfaction with delivery care, disrespect and abuse, and use of motor vehicle transport (Table 6).
Intervention effects in MNH care practices
The intervention effects were adjusted for the baseline differences in the kebele-level MNH indicators and socio-demographic characteristics between the study arms (Tables 6 and 7, respectively), using the propensity score matching (PSM) technique. The assessment of the balance of the sample characteristics and MNH indicators at baseline was found to be adequate. Between the baseline and follow-up surveys, ANC in the first trimester increased by 5.6 percentage points (from 26.5% to 32.1%) in the comparison area, which was statistically significant (p<0.05). ANC in the first trimester increased by 13.3 percentage points (from 25.6% at baseline to 38.9% at follow-up) in the intervention area, which was also statistically significant (p < .01). The increase in the coverage of that indicator was 7.6 percentage points higher in the intervention area than in the comparison area, which is attributable to the PC-Solutions strategy. In other words, the intervention effect (or the DID) was 7.6 percentage points. But the two-sided p-value of the statistical significance of the intervention effect was just above 0.05 (0.051), indicating it was not statistically significant. Nonetheless, the p-value for testing one-sided hypothesis (i.e., the increase in the coverage of the indicator in the intervention arm was higher than that in the comparison arm) was statistically significant (p = 0.026).
We observed statistically significant intervention effects on skilled delivery; PNC (at home and health facility) of the mother in 48 hours; PNC (at home and health facility) of the newborn in 48 hours; and stayed in facility for 24 hours or more, were 7.9% (1.8–13.9); 15.3% (7.4–23.2); 17.0% (9.1–24.8); and 13.5% (5.7–21.4), respectively.
However, the DiDs were not statistically significant (p>.05) for first ANC; complete ANC; four and more ANC visits; perceived knowledge of providers; women’s satisfaction with delivery care score; birth notification; motor vehicle use for transport; and experience of any form of disrespect and abuse during childbirth.
Perceived effect of the intervention
The following perceived effects of the strategy were identified: 1) increased service use and improved quality of care; 2) enhanced knowledge and skill of health workers and provision of standardized care; 3) enhanced community involvement; 4) strengthened linkages between communities and the formal health care system; and 5) helped to measure and evaluate quality.
Improved service use and quality of care.
WDA informants mentioned that the project brought tremendous changes in their village as a result of implementing this strategy. As WDA in SNNP said, “Changes are untold compared to the past.”
According to the informants, this strategy increased care-seeking behavior of pregnant mothers who participated in the project. The community realized how important beginning first ANC at three months for avoiding pregnancy-related risks, and women began disclosing their pregnancy to network/WDA members and getting ANC in the first trimester. According to the accounts of the interviews, another result of the strategy was that dropout rates across the continuum of MCH care decreased.
Enhanced the knowledge and skills of health workers.
Informants perceived enhanced health worker capacity in and community engagement in MNH services. The project’s participatory design and continuous learning process heightened WDA members’ awareness of the quality of MNH services. This knowledge was transferred to WDA members to mothers subsequently.
“The good legacy of the strategy is it showed us what components of MNH services we should provide mothers and newborns.” -Health center informant in Amhara.
Participants observed that training and technical assistance filled skill gaps and standardized practice among health workers.
Strengthened linkages between communities and the formal health care system.
Interview participants also reflected that the PC-Solution strategy helped to enhance linkages between HEWs and WDAs. It also strengthened the linkage between health centers and posts and connected communities to the health system. Respondents said that a strong link has created among WDAs, HEWs, and community. “As positive consequences …we established a strong relationship among community, WDAs, HEWs, and health center staff.”–Health center informant, SNNP.
Introduced quality of care indicators with the routine monitoring and evaluation systems of the PHCUs.
Respondents said that they used to evaluate coverage of MNH services; since PC-Solutions, they are measuring and evaluating MNH services in the view of quality. Participants mentioned that they learned how to conduct formative assessment, design solutions, and how to measure performance.
“From PC-Solution, I learned how to review my work with evidence and generate change ideas.”–Health center informant, Amhara.
Facilitators of and barriers to implementation
Facilitators.
Participants said that full stakeholder participation in all stages of the project, strong coordination, robust support, continuous performance review, and staff commitment facilitated the implementation of the PC-Solutions strategy.
Another key to the strategy’s success was high community engagement in QI planning, implementation, and monitoring. Informants mentioned that having shared responsibilities at all levels of the woreda health system and a detailed micro-plan, indicating who is responsible for what and when was also helpful. They also indicated that the development of coordinated activities, especially communication between the community, WDAs, HEWs and health center staff was another success factor.
“The project brought a new idea. It was participatory in that everyone who was supposed to be stakeholder was participating in the project and focused on continuous assessment of the problems and identifying potential solutions, plan accordingly and continues like this to get better results every time the team meets”, health center participant in Tigray.
Early PNC adoption was attributed to HEWs, who visited newly delivered mothers at home traveled long distances and often difficult roads and terrain, and to health center staff, who encouraged mothers to stay at facilities least 24 hours post-delivery for early PNC.
Barriers.
The implementation of this strategy was accompanied by several challenges, including staff turnover at the health centers, the workload of the health workers and HEWs, competing priorities of the health service providers and the WDAs, and magnesium sulfate (MgSO4) and vacuum extractor shortages.
High staff turnover compromised quality improvement because it took 2–3 months to train people in QI. Frequent community leadership change also hindered QI committee implementation. As HEW in SNNP said, “The kebele was not stable for the last two years; leadership was constantly changing. Due to this, QI committee was not working regularly and I can say it wasn’t functional. Workload, coupled with lack of HEW, hindered the QI project; I am the only HEW in the kebele and have remote villages that are difficult to reach.”
High client caseloads at health center thwarted PC-Solutions implementation. It also overloaded birth attendants, who were unable to attend women in delivery adequately and which caused mothers to give low delivery satisfaction scores. Unmanageable workloads lead staff to provide contact-focused rather than content-focused care, which led to insignificant intervention effects for specific ANC indicators. Respondents reported poor counseling and care quality, which likely contributed to the low use of ANC services. As study participants noted, if ANC counseling is not well organized and pregnant women are not convinced to visit the health center again, it is unlikely that they will come for subsequent appointments.
Due to competing priorities such as campaigns, QI events, in particular meetings, were held irregularly. And when staff were working on other assignments, MNH services were compromised.
Shortages of inputs such as MgSO4 and vacuum extractor sets, which were not available at the market, also contributed to poor performance of BEMONC signal functions at health centers.
Discussion
This evaluation demonstrates that participatory QI improved use of MNH care services including early ANC care-seeking, skilled delivery, and provision of home- and facility-based PNC for both the mother and the newborn. Interview respondents also perceived that the PC-Solution strategy resulted in a number of changes in the use and quality of MNH services. Participants said that links between communities and health systems and between the health center, health post, and community improved. Communication between WDA and community members improved as WDA member knowledge did.
These findings are in line with a systematic review of participatory learning and action cycles with women’s groups reporting improved clean home-delivery practices and uptake of any ANC [35], and with other studies on participatory community QI approaches that reported improved use of maternal health services [33, 37].
In this study, skilled delivery coverage was higher in the intervention than in the control areas. Mixed findings were reported from women’s group trials. It is in line with a previous quasi-experimental study in Ethiopia that engaged communities in identifying barriers to access and quality of services and reported an 11 percentage-point increase in average treatment effect in institutional deliveries [37]. But it is inconsistent with other studies elsewhere. For instance, studies from India, Bangladesh, and Malawi reported no effect on increasing health facility deliveries [36, 38, 39, 58–60]. And a systematic review of community-based interventions packages and a quasi-experimental study in Tanzania and Uganda also did not show significant improvement in skilled attendance at birth [61, 62].
Unlike other studies conducted in Ethiopia, Bangladesh, and India [37–39], this study indicated that community participation had a positive effect on PNC coverage. This might be due to improving HEW home visits regularity, which is supported by previous studies showing that community participation improved the accountability of health care providers [33, 63], and improved birth notification systems and practices of keeping women at facilities for at least 24 hours after delivery for PNC.
Contrary to respondents’ opinion that early care-seeking behavior of pregnant mothers improved following the intervention, early ANC booking, and quality of maternal services did not significantly change over time. Complete ANC (ANC 4 and more visits), ANC counseling score, women’s satisfaction with delivery care, and experience of any form of disrespect and abuse during childbirth did not show significant improvement in this study. This is in line with a cluster-randomized controlled evaluation of a community participation intervention in Malawi that reported no significant difference in perceived quality of delivery care and sufficient ANC [33]. Although disrespect and abuse during childbirth were not prevalent in this study, the reliability and validity of disrespect and abuse measurements are not well known [64]. As mentioned by respondents, a possible reason for non-significant intervention effects observed for specific ANC indicators is the practice of contact-focused rather than content-focused care. The other possible reason for no improvement in complete ANC, ANC 4 and more visits, and women’s satisfaction with delivery care is the short intervention period. These thematic areas were introduced later in 2017 and only had about one year of implementation, so these outcomes took place before the intervention strategies were matured. Moreover, the quality of care measure to events spread over the past 12 months preceding the follow-up survey. As such, many respondents in the intervention area were not exposed to aspects that were introduced during the second part of 2017.
About two-thirds of the health centers were ready to provide the seven BEmONC functions, but only 13% of these did so in the past three months preceding the survey. Though this is much higher than the recent national EmONC survey findings [15], a significant number of health centers did not provide parenteral anticonvulsants or remove placenta manually. Moreover, most health centers lacked readiness and/or provision of most newborn signal functions including KMC, alternate feeding for not breastfeeding babies, antibiotics for preterm premature rupture of membrane, corticosteroids for preterm labor, and antibiotics for sepsis. As such, availability of critical MNH equipment and drugs needs to be improved and facilities need to upgrade their provision of EmONC functions.
Percent of deliveries used partograph and provision of prophylactic uterotonics for AMTSL to prevent PPH increased over the survey periods. The percent of health centers using paragraph was higher than the national EmONC survey [15]. Moreover, access to and use of most maternal and newborn critical interventions, including syphilis testing, facility delivery, asphyxiated newborns managed with Ambu-bag and mask, and preterm/low birth weight babies initiated on KMC improved over time. However, the percent of expected PSBI cases managed at health centers declined over the survey periods. This could be because HEWs began managing PSBI cases through a community-based newborn care program.
This study examined the complex participatory community QI interventions in maternal and newborn health by interviewing those who were involved in the design and implementation of PC-Solutions strategy. Researchers explored the extent of community engagement in the health system and motivating and demotivating factors for sustained engagement in the health system. As such, the findings should be relevant to similar QI projects and stakeholders who intend to scale similar interventions.
The support system, which means facilitating review forums, conducting supervisions and mentoring from partners and/or woreda health office, was critical to the implementation of this participatory intervention. Woreda-level joint reviews were used to evaluate PHCU activities and identify service and knowledge gaps, and launch interventions between communities. They also motivated and capacitate stakeholder knowledge for further engagement and keeping their momentum. Continuous refresher trainings and supportive supervision visits to monitor and coach were helped deliver expected outcomes. Regular performance review and continual support help ensure provision of high-quality health care services.
We did multiple analyses for multiple different endpoint outcomes. Adjusting p-values is often recommended when conducting multiple hypotheses tests `simultaneously’ [65–68]. However, it is not always necessary [69–71]. As such, we did not adjust the p-value in our case as it would have increased the probability of making type II error, i.e., concluding that the intervention was not effective when it is true [69, 70, 72, 73].
Although combining PSM with DiD methods can help resolve the problem of time-invariant unmeasured confounders, the presence of time-varying unobserved confounders would bias the observed treatment effects. For instance, other programs or other developmental inputs (road, health facility construction, etc.) that influence MNH could be kebele-level time-varying or time-invariant confounders. If the government or any other development agency were implementing QI interventions for MNH services in the PC-Solutions areas, then the intervention effects estimated would be biased. However, there is no compelling reason to believe that the government or other development partners were systematically providing inputs in the intervention areas but not the comparison areas. The other major confounder is the Hawthorne effect because the intervention area was not masked. The providers knew that they were under study, which may have led them to perform better than they normally do. Moreover, study results may have been subjected to recall and social desirability biases because the survey used a 12-month recall of self-reported behavior.
We analyzed and presented the consistency between the findings with the quantitative study and the existing body of knowledge. However, one limitation could be that feedback from participants on the findings was not sought.
Conclusions
The PC-Solutions strategy suggests community engagement in the design and implementation of QI would improve MNH outcomes. Engaging communities in the design of the intervention would yield local solutions to local problems. As such, scale-up of QI initiatives would benefit from the engagement of all relevant local stakeholders throughout the design and implementation. Using community wisdom to implement interventions increases sustainability. Moreover, a strong support system was critical to the implementation of this participatory intervention. Further research is needed in this area before concluding that disrespect and abuse was low.
Supporting information
S1 Appendix. Survey dataset.
This is survey data with variables and their values we used for the analysis.
https://doi.org/10.1371/journal.pone.0228137.s001
(XLS)
S2 Appendix. Survey questionnaire.
Survey questionnaire we used to collect information from study participants. The first sheet contains variable definitions (data dictionary) in English and other local languages (Amharic, Oromiffa, and Tigregna), and the second sheet contains variable answer choices.
https://doi.org/10.1371/journal.pone.0228137.s002
(XLSX)
S3 Appendix. In-depth interview guide.
This is an in-depth interview guide we used to interview the participants in our study.
https://doi.org/10.1371/journal.pone.0228137.s003
(DOCX)
S1 File. Consolidated criteria for reporting qualitative studies (COREQ): 32-item checklist.
https://doi.org/10.1371/journal.pone.0228137.s004
(DOCX)
Acknowledgments
We thank the Federal Ministry of Health and regional health bureaus of Amhara, Oromia, the Southern National, Nationalities, and Peoples’ Region, and Tigray regions, for their support in implementing these surveys. We take this opportunity to extend our gratitude to all the study participants for the time they gave to respond to the survey questionnaires and provide us with valuable information. We would like to acknowledge our colleagues at The Last Ten Kilometers 2020 Project of JSI Research & Training Institute, Inc. for their contributions at all stages of implementing this work. Finally, researchers would like to acknowledge our colleagues Adey Abebe and Julie Ray for editing this manuscript.
References
- 1. Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, Mathai M, et al. Cost effectiveness analysis of strategies for maternal and neonatal health in developing countries. Bmj. 2005;331(7525):1107. pmid:16282407
- 2. Bhutta ZA, Das JK, Bahl R, Lawn JE, Salam RA, Paul VK, et al. Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost? The Lancet. 2014;384(9940):347–70.
- 3. Darmstadt GL, Walker N, Lawn JE, Bhutta ZA, Haws RA, Cousens S. Saving newborn lives in Asia and Africa: cost and impact of phased scale-up of interventions within the continuum of care. Health policy and planning. 2008;23(2):101–17. pmid:18267961
- 4. Lawn JE, Cousens S, Zupan J, Team LNSS. 4 million neonatal deaths: when? Where? Why? The lancet. 2005;365(9462):891–900.
- 5. Kikuchi K, Ansah EK, Okawa S, Enuameh Y, Yasuoka J, Nanishi K, et al. Effective Linkages of Continuum of Care for Improving Neonatal, Perinatal, and Maternal Mortality: A Systematic Review and Meta-Analysis. PLOS ONE. 2015;10(9):e0139288. pmid:26422685
- 6. Kikuchi K, Okawa S, Zamawe COF, Shibanuma A, Nanishi K, Iwamoto A, et al. Effectiveness of Continuum of Care—Linking Pre-Pregnancy Care and Pregnancy Care to Improve Neonatal and Perinatal Mortality: A Systematic Review and Meta-Analysis. PLOS ONE. 2016;11(10):e0164965. pmid:27788176
- 7. Lassi ZS, Mallick D, Das JK, Mal L, Salam RA, Bhutta ZA. Essential interventions for child health. Reproductive health. 2014;11(1):S4.
- 8.
WHO, UNICEF, UNFPA, World Bank Group, United Nations Population Division. Trends in Maternal Mortality: 1990 to 2015 Population and Development Review. Geneva: World Health Organization2015.
- 9.
Holmes W, Kennedy E. Reaching emergency obstetric care: overcoming the ‘second delay’. Melbourne: Burnet Institute on behalf of Compass. 2010.
- 10.
Central Statistical Agency (CSA) [Ethiopia], ICF International. Ethiopia Demographic and Health Survey 2016 Addis Ababa, Ethiopia and Rockville, Maryland, USA: CSA and ICF,; 2016.
- 11.
Central Statistical Agency (CSA) [Ethiopia], ICF International. Ethiopia demographic and Health Survey 2011. Addis Ababa, Ethiopia and Calverton, Maryland, USA: CSA and ICF; 2012.
- 12. Kinney MV, Kerber KJ, Black RE, Cohen B, Nkrumah F, Coovadia H, et al. Sub-Saharan Africa's mothers, newborns, and children: where and why do they die? PLoS medicine. 2010;7(6):e1000294. pmid:20574524
- 13. Nesbitt RC, Lohela TJ, Manu A, Vesel L, Okyere E, Edmond K, et al. Quality along the Continuum: A Health Facility Assessment of Intrapartum and Postnatal Care in Ghana. PLOS ONE. 2013;8(11):e81089. pmid:24312265
- 14. Hodgins S, D'Agostino A. The quality–coverage gap in antenatal care: toward better measurement of effective coverage. Global Health: Science and Practice. 2014;2(2):173–81.
- 15.
EPHI, FMoH, AMDD. Ethiopian Emergency Obstetric and Newborn Care (EmONC) Assessment 2016: Final Report. Ethiopian Public Health Institute, Addis Ababa, Ethiopia; Federal Ministry of Health, Addis Ababa, Ethiopia; and Averting Maternal Death and Disability (AMDD), Columbia University, New York, USA; 2017.
- 16. Neupane S, Doku D. Utilization of postnatal care among Nepalese women. Maternal and child health journal. 2013;17(10):1922–30. pmid:23292803
- 17. Tarekegn SM, Lieberman LS, Giedraitis V. Determinants of maternal health service utilization in Ethiopia: analysis of the 2011 Ethiopian Demographic and Health Survey. BMC pregnancy and childbirth. 2014;14(1):161.
- 18. Worku AG, Yalew AW, Afework MF. Factors affecting utilization of skilled maternal care in Northwest Ethiopia: a multilevel analysis. BMC international health and human rights. 2013;13(1):20.
- 19. Darega B, Dida N, Tafese F, Ololo S. Institutional delivery and postnatal care services utilizations in Abuna Gindeberet District, West Shewa, Oromiya Region, Central Ethiopia: A Community-based cross sectional study. BMC pregnancy and childbirth. 2016;16(1):149.
- 20. Sepehri A, Sarma S, Simpson W, Moshiri S. How important are individual, household and commune characteristics in explaining utilization of maternal health services in Vietnam? Social Science & Medicine. 2008;67(6):1009–17.
- 21. Babalola S, Fatusi A. Determinants of use of maternal health services in Nigeria-looking beyond individual and household factors. BMC pregnancy and childbirth. 2009;9(1):43.
- 22. Parkhurst JO, Penn-Kekana L, Blaauw D, Balabanova D, Danishevski K, Rahman SA, et al. Health systems factors influencing maternal health services: a four-country comparison. Health Policy. 2005;73(2):127–38. pmid:15978956
- 23. Kruk ME, Galea S, Prescott M, Freedman LP. Health care financing and utilization of maternal health services in developing countries. Health policy and planning. 2007;22(5):303–10. pmid:17681975
- 24. Perkins J, Capello C, Vilgrain C, Groth L, Billoir H, Santarelli C. Determinants of Low Maternal and Newborn Health Service Utilization in Haiti: A Community-Based Cross-Sectional Study. Journal of Womens Health, Issues and Care. 2017;2017.
- 25. Kaba M, Taye G, Gizaw M, Mitiku I. Maternal health service utilization in urban slums of selected towns in Ethiopia: Qualitative study. The Ethiopian Journal of Health Development (EJHD). 2017;31(2).
- 26. Agunwa CC, Obi IE, Ndu AC, Omotowo IB, Idoko CA, Umeobieri AK, et al. Determinants of patterns of maternal and child health service utilization in a rural community in south eastern Nigeria. BMC Health Services Research. 2017;17(1):715. pmid:29132329
- 27. Singh K, Brodish P, Chowdhury ME, Biswas TK, Kim ET, Godwin C, et al. Postnatal care for newborns in Bangladesh: The importance of health-related factors and location. J Glob Health. 2017;7(2):020507. pmid:29423184
- 28. Ononokpono DN, Odimegwu CO, Imasiku EN, Adedini SA. Does it really matter where women live? A multilevel analysis of the determinants of postnatal care in Nigeria. Maternal and child health journal. 2014;18(4):950–9. pmid:23812800
- 29.
Van Lerberghe W. The world health report 2008: primary health care: now more than ever: World Health Organization; 2008.
- 30. Marston C, Hinton R, Kean S, Baral S, Ahuja A, Costello A, et al. Community participation for transformative action on women’s, children’s and adolescents’ health. Bulletin of the World Health Organization. 2016;94(5):376. pmid:27152056
- 31.
WHO. Community participation in local health and sustainable development: Approaches and techniques. World Health Organization; Geneva; 2002.
- 32.
WHO. WHO recommendations on health promotion interventions for maternal and newborn health 2015: World Health Organization; 2015.
- 33. Marston C, Renedo A, McGowan C, Portela A. Effects of community participation on improving uptake of skilled care for maternal and newborn health: a systematic review. PloS one. 2013;8(2):e55012. pmid:23390509
- 34. Bath J, Wakerman J. Impact of community participation in primary health care: what is the evidence? Australian Journal of Primary Health. 2015;21(1):2–8. pmid:24176202
- 35. Prost A, Colbourn T, Seward N, Azad K, Coomarasamy A, Copas A, et al. Women's groups practising participatory learning and action to improve maternal and newborn health in low-resource settings: a systematic review and meta-analysis. The Lancet. 2013;381(9879):1736–46.
- 36. Colbourn T, Nambiar B, Bondo A, Makwenda C, Tsetekani E, Makonda-Ridley A, et al. Effects of quality improvement in health facilities and community mobilization through women's groups on maternal, neonatal and perinatal mortality in three districts of Malawi: MaiKhanda, a cluster randomized controlled effectiveness trial. International health. 2013;5(3):180–95. pmid:24030269
- 37. Wereta T, Karim A, Betemariam W, Fesseha N, Dagnew S, Workneh A, et al. Effectiveness of participatory community quality improvement strategy on improving maternal and newborn health care behavior and practices: A propensity score analysis. BMC Pregnacy and Childbirth. 2018.
- 38. Tripathy P, Nair N, Barnett S, Mahapatra R, Borghi J, Rath S, et al. Effect of a participatory intervention with women's groups on birth outcomes and maternal depression in Jharkhand and Orissa, India: a cluster-randomised controlled trial. The Lancet. 2010;375(9721):1182–92.
- 39. Azad K, Barnett S, Banerjee B, Shaha S, Khan K, Rego AR, et al. Effect of scaling up women's groups on birth outcomes in three rural districts in Bangladesh: a cluster-randomised controlled trial. The Lancet. 2010;375(9721):1193–202.
- 40.
FMoH. Health sector Development Program IV. Annual Performance Report 2012/2013. Available at www.moh.gov.et/resources. 2012.
- 41. Wakabi W. Extension workers drive Ethiopia's primary health care. The Lancet. 2008;372(9642):880.
- 42. Karim AM, Admassu K, Schellenberg J, Alemu H, Getachew N, Ameha A, et al. Effect of Ethiopia’s health extension program on maternal and newborn health care practices in 101 rural districts: a dose-response study. PLoS One. 2013;8(6):e65160. pmid:23750240
- 43.
WHO. Strategies toward ending preventable maternal mortality (EPMM). Geneva, Switzerland: World Health Organization2015.
- 44. Chou D, Daelmans B, Jolivet RR, Kinney M, Say L. Ending preventable maternal and newborn mortality and stillbirths. BMJ. 2015;351:h4255. pmid:26371222
- 45.
FMOH. Health Sector Transformation Plan, 2016–2020. Federal Ministry of Health: Addis Ababa, Ethiopia2015.
- 46. Karim AM, Fesseha N, Shigute T, Emaway D DS, Solomon F, Hailu M, et al. Effects of community-based data for decision-making intervention on maternal and newborn health care practices in Ethiopia: A dose-response study. BMC Pregnancy and Childbirth. 2018;18(1).
- 47. Emaway D, Karim AM, Betemariam W, Fesseha N, Shigute T, Pauline S. Effects of family conversation on health care practices in Ethiopia: a propensity score matched analysis. BMC Pregnacy and Childbirth. 2018;18(1).
- 48.
Betemariam W, Karim A. Effect of Participatory Community Quality Improvement on Maternal and Newborn Health Care Practices: A Quasi-Experimental Study. Global Maternal and Newborn Health Conference; Mexico City, Mexico2015.
- 49.
Tiruneh G, Karim A, Yihun B, Betemariam W, Fesseha N, Wereta T. Effective Referral System for the Utilization of Critical Maternal and Newborn Health at Rural Health Centers of Ethiopa. American Public Health Association Annual Meeting Chicago, USA2015.
- 50.
L10K. Maternal and Newborn Health Care Behaviors and Practices in 115 L10K Woredas of Ethiopia: Baseline and Follow-up Surveys 2016–2017. Addis Ababa, Ethiopia: JSI Research and Training Institute Inc./The Last Ten Kilometers Project; 2018.
- 51. Lemeshow S, Robinson D. Surveys to measure programme coverage and impact: a review of the methodology used by the expanded programme on immunization. World Health Stat Q. 1985;38(1):65–75. pmid:4002731
- 52.
Dobility Inc. SurveyCTO [Available from: http://www.surveycto.com.
- 53. Filmer D, Pritchett LH. Estimating wealth effects without expenditure data—or tears: an application to educational enrollments in states of India. Demography. 2001;38(1):115–32. pmid:11227840
- 54.
StataCorp. Stata: Release 15. Statistical Software. College Station, Texas: Stata Press; 2017.
- 55. Austin PC. Optimal caliper widths for propensity‐score matching when estimating differences in means and differences in proportions in observational studies. Pharmaceutical statistics. 2011;10(2):150–61. pmid:20925139
- 56. Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples. Statistics in medicine. 2009;28(25):3083–107. pmid:19757444
- 57.
WHO, UNICEF, UNFPA, AMDD. Monitoring emergency obstetric care: A Handbook. Geneva: World Health Organization; 2009.
- 58. Lewycka S, Mwansambo C, Rosato M, Kazembe P, Phiri T, Mganga A, et al. Effect of women's groups and volunteer peer counselling on rates of mortality, morbidity, and health behaviours in mothers and children in rural Malawi (MaiMwana): a factorial, cluster-randomised controlled trial. The Lancet. 2013;381(9879):1721–35.
- 59. Fottrell E, Azad K, Kuddus A, Younes L, Shaha S, Nahar T, et al. The effect of increased coverage of participatory women’s groups on neonatal mortality in Bangladesh: A cluster randomized trial. JAMA pediatrics. 2013;167(9):816–25. pmid:23689475
- 60. More NS, Bapat U, Das S, Alcock G, Patil S, Porel M, et al. Community mobilization in Mumbai slums to improve perinatal care and outcomes: a cluster randomized controlled trial. PLoS medicine. 2012;9(7):e1001257. pmid:22802737
- 61.
Lassi ZS, Bhutta ZA. Community‐based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes. The Cochrane Library. 2015.
- 62. Waiswa P, Manzi F, Mbaruku G, Rowe A, Marx M, Tomson G, et al. Effects of the EQUIP quasi-experimental study testing a collaborative quality improvement approach for maternal and newborn health care in Tanzania and Uganda. Implementation Science. 2017;12(1):89. pmid:28720114
- 63. Tancred T, Mandu R, Hanson C, Okuga M, Manzi F, Peterson S, et al. How people-centred health systems can reach the grassroots: experiences implementing community-level quality improvement in rural Tanzania and Uganda. Health policy and planning. 2014;33(1):e1–e13.
- 64. Sando D, Abuya T, Asefa A, Banks KP, Freedman LP, Kujawski S, et al. Methods used in prevalence studies of disrespect and abuse during facility based childbirth: lessons learned. Reproductive health. 2017;14(1):127. pmid:29020966
- 65. Tukey JW. Some thoughts on clinical trials, especially problems of multiplicity. Science. 1977;198(4318):679–84. pmid:333584
- 66. Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. Bmj. 1995;310(6973):170. pmid:7833759
- 67. Greenhalgh T. How to read a paper: Statistics for the non-statistician. I: Different types of data need different statistical tests. Bmj. 1997;315(7104):364–6. pmid:9270463
- 68. Ludbrook J. Multiple comparison procedures updated. Clinical and Experimental Pharmacology and Physiology. 1998;25(12):1032–7. pmid:9888002
- 69. Feise RJ. Do multiple outcome measures require p-value adjustment? BMC medical research methodology. 2002;2(1):8.
- 70. Perneger TV. What's wrong with Bonferroni adjustments. Bmj. 1998;316(7139):1236–8. pmid:9553006
- 71. Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology. 1990:43–6. pmid:2081237
- 72.
Cole P. The evolving case-control study. The Case-Control Study Consensus and Controversy: Elsevier; 1979. p. 15–27.
- 73. Thomas D, Siemiatycki J, Dewar R, Robins J, Goldberg M, Armstrong B. The problem of multiple inference in studies designed to generate hypotheses. Am J Epidemiol. 1985;122(6):1080–95. pmid:4061442