Impact of program transfer from a non-governmental organization to a district health office: Evaluation of a program integrating water treatment and hygiene kits into reproductive health and HIV services, Machinga District, Malawi, 2010-2012

Background In September 2009, the Machinga Integrated Antenatal Water Hygiene Kit Program began addressing problems of unsafe water, high infant mortality, and low antenatal care (ANC) attendance in Machinga District, Malawi. In March 2011, the supporting international non-governmental organization transitioned management of the program to the Machinga District Health Office (DHO). We evaluated maternal and HIV service use before and after program transition to the DHO. Methods We compared pre- and post-transition periods by examining data recorded in ANC and maternal registries in 15 healthcare facilities (HCFs) by proportion z-tests. We classified HCFs by size, using the median monthly patient volumes as the split for large or small facilities. We used logistic regression to evaluate changes in the use of ANC, maternal, and HIV services and their interactions with HCF size. Results The percentage of women attending their first ANC visit during the first trimester was similar in the pre-and post-transition periods (9.3% vs 10.2%). Although the percentage of women with ≥4 ANC visits was similar from pre- to post-transition (26.0% vs 24.8%), the odds increased among women in small facilities (OR: 1.37, 95% CI: 1.24–1.51), and decreased among women in large facilities (OR: 0.80, 95% CI: 0.75–0.85). Although a similar percentages of pregnant women were diagnosed with HIV in all HCFs in the pre- and post-transitions periods (6.4% vs 4.8%), a substantially larger proportion of women were not tested for HIV in large HCFs (OR: 6.34, 95% CI: 5.88–6.84). A larger proportion of women gave birth at both small (OR: 1.30, 95% CI: 1.16–1.45) and large HCFs (OR: 1.55, 95% CI: 1.43–1.67) in the post-transition vs. the pre-transition period. Conclusions The evaluation results suggest that many positive aspects of this donor-supported program continued following transition of program management from a non-governmental organization to a DHO.


Conclusions
The evaluation results suggest that many positive aspects of this donor-supported program continued following transition of program management from a non-governmental organization to a DHO.

Background
Malawi has a high burden of maternal (675 per 100,000 live births) and infant (66 per 1,000 live births) mortality [1]. While a majority of pregnant women attend at least one antenatal care (ANC) visit, only 24% have their first ANC visit during the first trimester of pregnancy [2]. In 2015, only about half of all pregnant women completed the recommended four or more ANC (ANC4+) visits [2].
Inadequate use of ANC services throughout a pregnancy is associated with adverse health outcomes, such as maternal and perinatal mortality, low birth weight, and premature delivery [3][4]. In recent years, the government of Malawi and supporting international non-governmental agencies have prioritized efforts to increase the number of ANC clinic visits and healthcare facility (HCF) deliveries. As a result, the proportion of pregnant women in Malawi that received ANC from a skilled provider rose from 90% in 1992 to 95% in 2015 [2].
In 2010, only 66% of pregnant women attending antenatal clinics were tested for human immunodeficiency virus (HIV) [5], despite high HIV prevalence among women of reproductive age. Coverage of pregnant women in prevention of mother-to-child transmission (PMTCT) programs was 53% in 2010, well below the goal for 2013 of 80% set by the Malawi Ministry of Health (MOH) [6].
Improved maternal and child health, as well as comprehensive HIV services, were set as priority targets toward the achievement of Millennium Development Goals 4, 5, and 6 by the United Nations from 1990 to 2015 [7]. To address these goals, from September 2009 -March 2011, the Malawi MOH and Clinton Health Access Initiative (CHAI), an international nongovernmental organization, developed a package of incentives to increase use of ANC, delivery, and HIV services. The incentives, called "water hygiene kits"-consisting of water storage container with a lid and tap, sodium hypochlorite water treatment solution, oral rehydration salts, and soap-were offered to rural-dwelling pregnant women at their first ANC visit, and refills of water treatment solution and soap were provided at subsequent ANC visits and delivery. At the time of this program, PMTCT Option B+ (provision of lifelong antiretroviral therapy to HIV-infected pregnant women), was implemented in Malawi. Because success of this program was heavily dependent on male partner participation [8][9], CHAI required partners' presence for women to receive the initial water hygiene kit. During the 18-month program, CHAI distributed over 25,000 water hygiene kits as incentives to promote male involvement and family-centered ANC at 15 healthcare facilities in Machinga District.
At the time of the first ANC visit, HIV counseling and testing (HCT) services were offered to participating couples. Trained community volunteers living with HIV/AIDS provided counseling to individuals testing HIV-positive and assisted them to referral locations able to stage their infections and provide recommended PMTCT services including antiretroviral therapy (ART). All women were eligible to receive up to four free refills of sodium hypochlorite solution and soap during subsequent ANC visits, at delivery, and during postnatal checkups. An evaluation conducted 1 year into the project found increased ANC attendance, delivery at HCFs, household water treatment with sodium hypochlorite, safe storage of drinking water, and the ability of mothers to demonstrate proper handwashing technique [10]. The program exceeded expected levels of HCT participation among pregnant women and their partners.
Program transition, defined as the formal handing over of a program to local partners or the host government, is vital in sustaining key aspects of programs over the long-term [11]. While a number of studies based in high-income countries have documented the sustainability of program transition [12][13], few studies have examined the sustainability of program transition in low and middle-income nations [11]. In March 2011, CHAI transitioned the water hygiene kit program management to the Machinga District Health Office (DHO) over a 9-month period. During this period, all but one member of the CHAI team left the project, leaving one staff member to provide program guidance to DHO personnel. The transition consisted of handing over warehousing and management of water hygiene kit distribution to DHO personnel, providing project records to the DHO, ensuring that DHO staff continued scheduling quarterly HCF visits, and orienting the DHO statistician to management of ANC and delivery registry data, which included providing quarterly aggregated district reports and healthcare facility-specific reports to the DHO and district HCFs. In January 2012, we evaluated whether levels of ANC attendance, hospital delivery, and HIV service use were maintained following program transition.

Study site
The population of Machinga District in the Southern Region of Malawi has low educational levels and socioeconomic status, with only 5.6% of women completing primary school [1]. The district also has limited access to safe water and sanitation, with 77.8% of households using an improved water source, 16.8% of households using an effective water treatment method, and 10.1% of households having an improved, not shared, sanitation facility [1].
In addition, the district has a low percentage of women delivering in a healthcare facility (74.4%), and high rates of infant mortality (77 per 1,000 live births) compared to the country average (66 per 1,000 live births) [1]. In 2010, the prevalence of HIV in Malawi was 13.0 (11.9-13.5) for women 15-49 years old [14].

Evaluation design
In March 2011, CHAI initiated transition of the water hygiene kit program to the Machinga DHO. One CHAI project officer remained in Machinga to support the DHO during the transition (Fig 1). Water hygiene kits had been previously procured by CHAI and were stored in a hospital warehouse until they were ready to be distributed. To determine whether the program remained successful in motivating ANC and delivery service use, we compared post-transition antenatal and maternity registry data in 15 healthcare facilities from January-June 2012 to pre-transition registry data from January-June 2010, when a full CHAI implementation team was involved in all aspects of the project.

Registry data collection
Data from antenatal and maternity registries were recorded at 15 participating HCFs during the pre-transition (January-June 2010) and post-transition periods (January-June 2012). For the cohorts of women whose first antenatal visit occurred during the January to June periods in 2010 (pre-transition cohort) and in 2012 (post-transition cohort), the antenatal registries showed aggregate data for outcome variables of interest. These variables included the number of first ANC visits during each of the two periods, the total number of ANC visits made during each of these periods, and HIV status determined during each of these periods. Maternity registry data were collected from mothers when they presented at a HCF at the time of delivery or within 72 hours of giving birth. Registry data included location of delivery and ART use during delivery.

Statistical analysis
Data were analyzed using SAS version 9.4 (Cary, NC). We used 2-sample z-test to compare ANC and maternity registry outcomes between pre-transition and post-transition periods at all HCFs. The 15 facilities were grouped into "small" and "large" HCFs using the median 6-month aggregated pre-transition period patient volumes (post-transition patient volumes did not change sufficiently in any of the HCFs to change these groupings). We applied logistic regression models to test the interaction between changes in proportion of women using ANC and delivery services during the two periods and HCF size. Significant interactions were found in several outcome variables, implying the changes from pre-to post-transition periods depended on facility size. Then we evaluated the changes in ANC and delivery service use by HCF size (large vs. small) under logistic models. Women who had previously tested positive for HIV were excluded from the HIV analyses.

Ethical considerations
The Human Subjects Advisor at the National Center for Emerging and Zoonotic Infectious Diseases at CDC determined that, because this activity consisted of an evaluation of a proven public health practice, did not use maternal interviews, and obtained no personal identifying data, it was exempt from human subjects research review (protocol #6082). The Malawi Ministry of Health authorized the evaluation and provided CHAI with a letter of approval for the evaluation of the comprehensive and integrated PMTCT pilot program in Machinga District.

Results
ANC registries in the 15 HCFs included 14,935 women with at least 1 visit during the pre-transition period (January-June 2010), and 15,848 with at least 1 visit during the post-transition

First antenatal care (ANC1) visit during the first trimester
Overall, the percentage of women attending ANC1 during the first trimester was similar in the pre and post-transition periods (9.3% vs 10.2%) ( Table 1). Comparing the pre-transition to post-transition period, there was an increase in the odds of pregnant women receiving their

Four or more ANC visits (ANC4+)
Among 15 HCFs, the percentage of women with ANC4+ visits changed little from the pretransition to post-transition period (26.0% vs 24.8%) ( Table 1), although the amount of change varied by facility size. In small HCFs, comparing ANC coverage before and after transition, there was an increase in the odds of pregnant women with ANC4+ visits after transition (OR: 1.37, 95% CI: 1.24-1.51), but in large HCFs there was a decrease in women with ANC4+ after transition (OR: 0.80, 95% CI: 0.75-0.85) ( Table 2).

HIV testing
From the pre-intervention to post-intervention period, there were decreases in the odds of women who were tested for HIV. This was primarily observed in large HCFs (OR: 6.34, 95% CI: 5.88-6.84), with only a slight increase in small HCFs (OR: 1.14, 95% CI: 0.98-1.31) ( Table 2). Among those tested, the percentage of pregnant women newly diagnosed with HIV decreased modestly from the pre-transition to post-transition period (6.4% vs 4.8%) ( Table 1). Comparing pre-to post-transition data, the odds of women newly diagnosed with HIV decreased slightly in both small (OR: 0.74, 95% CI: 0.61-0.89) and large (OR: 0.69, 95% CI: 0.60-0.80) HCFs (Table 2). Among 15 HCFs, the percentage of women seen in a HCF within 72 hours of a delivery increased from 87.7% to 90.6% from pre-to post-transition period (Table 3). This increase was observed in both small (OR: 1.30, 95% CI: 1.16-1.45) and large facilities (OR: 1.55, 95% CI: 1.43-1.67) ( Table 4).

Discussion
To our knowledge, this evaluation is the first to examine whether ANC, maternity, and HIV service delivery levels could be maintained following the transition from an incentives program administered by an NGO to local management by a Ministry of Health District Office. Although we found some positive and negative changes in service use for several outcomes in HCFs, the magnitude of change was less than 10% for most outcomes. The variations in magnitude and direction for several indicators merit examination.

ANC1 visit in the first trimester
The proportion of women attending their first ANC visit in the first trimester increased in more than two thirds of HCFs from the pre-transition to post-transition periods. A possible explanation for this finding is that water hygiene kits motivated women to attend ANC visits earlier than normal. A previous evaluation of the CHAI program revealed that the improved water storage container, which was distributed in the water hygiene kit at the time of the first ANC visit, was particularly desirable [10]. It is possible that women were motivated to make their ANC1 visit earlier than the usual practice to ensure that they received their container. These findings suggest that the program was sufficiently maintained by the DHO during the post-transition period to sustain achievement of the objective of early initiation of antenatal care. To our knowledge, no other programs designed to increase ANC service were in progress in Machinga District during this project.

ANC4+
Among the 15 facilities included in our analysis, we found a small decrease (26.0% to 24.8%) in the proportion of women with four or more ANC visits. However, disaggregating results by size of HCF identified an increase in the percentage of ANC4+ visits within small HCFs, whereas large HCFs had a small decrease. One possible explanation for this divergence in findings by size of clinic could be a result of changes in the ART program to incorporate Option B + which required monthly clinic visits and decentralization of the program from large to smaller HCFs. Alternatively, it is possible that pregnant women attending HCFs with large patient volumes experienced longer waiting times or poor treatment by overburdened health workers-both of which have been documented in previous evaluations [15][16][17]. Other studies have suggested that economic incentives were insufficient to motivate women to increase their number of ANC visits [18]. An evaluation of this CHAI program was completed in 2011, and found that pregnant women and their partners valued the water hygiene kits, refills of water treatment solution, and soap; however, this evaluation did not examine how facility size or quality of care influenced these opinions [10]. The interaction of incentives and antenatal service use is complex and merits further research.

HIV testing and treatment
In large HCFs, we found that the proportion of women that were not tested for HIV during their current pregnancy significantly increased from the pre-to post-transition periods. The increase could have resulted from large stock outs of HIV testing kits that occurred in early 2012 [9,19]. Although the reasons for stock outs were not clear, a previous study examining the transition of a large scale HIV/AIDS prevention program from a donor-run program to local stakeholders similarly found substantial commodity stock outs post-transition as a result of changes to supply source and schedules [11]. The reality of commodity stock outs in a program's post-transition period justifies an extended transition period to ensure the establishment of a functioning commodity forecasting and procurement system [11,20]. Alternatively, it is possible that the proportion of women not tested for HIV increased as a result of a reduction in the number of staff testing women for HIV as the NGO transitioned control of the program over to the DHO. However, staffing records from the time of transition were not available to further investigate this theory.

Delivery in HCFs
Our evaluation results suggest that maternity services in both small and large facilities were maintained from pre-to post-transition periods. However, this result could be the result of a MOH policy promoting delivery in HCFs [2]. The percentage of women in Malawi that delivered in an HCF increased from 57% in 2004, to 73% in 2010, to 91% in 2015-16 [1][2]21]. These increases parallel similar findings in other countries that have instituted policies designed to increase healthcare facility delivery [18,22], although increased patient volumes can also have the effect of decreasing service use [23]. The paradoxical decrease in ANC4+ visits and increase in deliveries in a HCF could also be explained, in part, by a common perception among pregnant women that labor and delivery pose greater health risks to mother and child than pregnancy, as noted in a previous study [24].

ART during labor
In contrast to HIV testing data, the proportion of women who received ART during labor significantly increased in both small and large facilities from pre-to post-transition periods. These findings may be influenced by the implementation of Option B+, a program initiated by the MOH in 2011 in which all HIV-infected pregnant and breastfeeding women became eligible for lifelong ART [9]. Implementation of Option B+ preceded the transition period, which increased the need for a functioning procurement system [25].

Limitations
This evaluation, which used programmatic data collected at the healthcare facility level, had several important limitations. First, because we examined surveillance data at the healthcare facility level, we were unable to directly link increases in, or maintenance of, ANC utilization, HCF deliveries, or HIV services with successful transition of the program from CHAI to the District Health Office. Other environmental changes occurring during the study interval (e.g., introduction of Option B+) may have accounted for some of the findings. Second, this evaluation had no true control group because financial and logistical constraints limited our analysis to registry data from Machinga District (and precluded data collection from other districts not participating in the program). Consequently, we were unable to definitively attribute evaluation results to the water hygiene kit intervention. Third, CHAI assigned one project officer to remain in the district to support the DHO during the post-transition period, who ensured water hygiene kits were procured and prepositioned and continuously available in the hospital warehouse. These activities were not sustainable long-term and may have exaggerated the actual impact of the program on service use. Fourth, because program implementation was limited to Machinga District, the results are not generalizable to other districts in Malawi or elsewhere. Fifth, this evaluation collected data for only 1 year after transition from CHAI to the Malawi District Health Office, limiting our ability to assess longer-term impact of the program's management, as well as the potential sustainability of a program requiring external funding. Finally, we received anecdotal information that commodity stock outs of water hygiene kits, HIV testing kits, and other supplies were common throughout the evaluation period, which may have affected program impact on service use through loss of incentives.

Conclusion
Low and middle-income countries are often faced with the challenge of maintaining health programs after external funding support is withdrawn, NGOs transition out of a district or country, or both. Results of this study suggest that program outcomes can be maintained following transition from intensive NGO support to full local management, particularly when supported by local health policy. Access to commodities is an important determinant of programmatic success; however, prepositioned commodities and the continued presence of one NGO staff member during this evaluation may have positively influenced these findings. In a world with limited development resources, it is important to continuously examine the determinants of program sustainability and consider how to maintain similar programs as they transition from NGO support to local management.
Supporting information S1 Data. Minimal data set. (ZIP)