Impact of Increasing Capacity for Generating and Using Research on Maternal and Perinatal Health Practices in South East Asia (SEA-ORCHID Project)

Background Maternal and neonatal mortality and morbidity remain unacceptably high in many low and middle income countries. SEA-ORCHID was a five year international collaborative project in South East Asia which aimed to determine whether health care and health outcomes for mothers and babies could be improved by developing capacity for research generation, synthesis and use. Methods Nine hospitals in Indonesia, Malaysia, the Philippines and Thailand participated in SEA-ORCHID. These hospitals were supported by researchers from three Australian centres. Health care practices and outcomes were assessed for 1000 women at each hospital both before and after the intervention. The capacity development intervention was tailored to the needs and context of each hospital and delivered over an 18 month period. Main outcomes included adherence to forms of care likely to be beneficial and avoidance of forms of care likely to be ineffective or harmful. Results We observed substantial variation in clinical practice change between sites. The capacity development intervention had a positive impact on some care practices across all countries, including increased family support during labour and decreased perineal shaving before birth, but in some areas there was no significant change in practice and a few beneficial practices were followed less often. Conclusion The results of SEA-ORCHID demonstrate that investing in developing capacity for research use, synthesis and generation can lead to improvements in maternal and neonatal health practice and highlight the difficulty of implementing evidence-based practice change.


Introduction
Context Improving maternal mortality is the Millennium Development Goal towards which the least progress has been made globally [1]. Despite a 20% reduction in maternal deaths in Asia since 1990, it is estimated that over 240,000 women still die in childbirth each year in the region compared to fewer than 1000 in developed regions [2]. The main causes of maternal mortality directly related to pregnancy and childbirth are bleeding, infection, hypertension (including eclampsia) obstructed labour and unsafe abortions [3]. Serious acute and chronic maternal morbidity has been estimated to occur in one in four women [4]. Poor maternal health and pregnancy care leads to high rates of perinatal death and also to low birth weight, perinatal asphyxia and infection [5]. Many of these outcomes could be avoided through provision of simple, effective, evidence-based care before, during and after birth [1].
To improve the quality of health care, low and middle income countries need to develop local capacity to use, synthesise and generate research relevant to their health needs [6]. This capacity is vital to ensure that relevant research is undertaken, identified, disseminated and implemented, so ensuring that optimal care is provided.
'Capacity' refers to appropriate resources, skills, commitment and structure [7]. Capacity development therefore requires more than just provision of technical assistance, but rather a multifaceted approach that includes: N access to research resources N training and skills development in doing, finding, understanding and using research N development of sustainable networks within and between organisations and countries to support a culture of research, research synthesis and research use N structures that enable each of the above.
We aimed to build on existing links with the Cochrane Collaboration and International Clinical Epidemiology Network to implement and evaluate a targeted capacity development intervention in South East Asia.
We hypothesised that a targeted intervention to build capacity for generating, synthesising and implementing relevant evidence would lead to improved adherence to appropriate clinical practices leading to better health outcomes for women and babies in South East Asia.
This paper describes the changes in clinical practices and outcomes in the participating hospitals following the intervention phase of the SEA-ORCHID (South East Asia -Optimising Reproductive and Child Health in Developing Countries) Project.

Objective
The SEA-ORCHID project aimed to assess whether the health of mothers and babies in Thailand, Indonesia, the Philippines and Malaysia could be improved by increasing capacity for the synthesis of relevant research, implementation of effective interventions, and identification of gaps in knowledge needing further research [8].
To achieve this aim, the objectives of the SEA-ORCHID project were: 1. to build capacity in research synthesis 2. to increase capacity and skills for evidence-based practice 3. to demonstrate effective implementation strategies for evidence-based practice change 4. to improve access to quality healthcare information 5. to increase locally derived and relevant research activity 6. to increase evidence-based policy making 7. to influence the broader socio-economic and health policy environment [9].
Design SEA-ORCHID was a before-after study, using action research to design, tailor and implement a capacity development intervention which included skills development (training and support), network building (training fellowships and project meetings), mentoring (between-country exchanges) and resourcing (subscription to evidence based materials, travel to project meetings and relevant conferences, IT infrastructure and support).
The SEA-ORCHID project protocol, methods and design, and data on clinical practices at baseline have been published previously [8,9,10].

Setting
The SEA-ORCHID project was conducted between 2004 and 2009 (see Figure 1) in nine hospitals across Indonesia, Malaysia, The Philippines and Thailand with support from three universities in Australia. Seven of the hospitals were tertiary referral institutions with regional referrals of women with a high risk pregnancy. Two hospitals were provincial or district institutions. In all hospitals, vaginal births were conducted by doctors, obstetric specialists and/or midwives (including nurses with midwifery qualifications) with caesarean section facilities available.

Population
Data were collected on at least 1000 women at each site pre and post-intervention. This sample size was based on the maximum possible recruitment at the smaller participating hospitals. The five smaller hospitals collected data on consecutive women until at least 1000 women were included. The four larger hospitals sampled women using appropriate ratios to ensure there was at least a threemonth data collection period and avoid biases associated with short data collection periods. At pre-intervention, 9550 women (9665 infants including 111 twins and two sets of triplets) presenting to one of the labour wards at each site were enrolled in the study and their medical records audited by trained researchers to determine compliance with the practices of interest. At post-intervention, the medical records of 9263 women (9351 infants including 84 twins and two sets of triplets) were audited using the same methods. The time taken to enrol 1000 women per site varied according to the number of births per month. Due to this, in the post-intervention phase, enrolment at the two Indonesian sites began in August, 2007, several months before the other sites.

Intervention
The intervention was implemented for 24 months (January 2006-December 2007). Tailored intervention strategies aiming to increase capacity in generating, synthesising and using research were implemented at each hospital based on local need and evidence for their effectiveness in leading to change in the maternal and child healthcare practices. The intervention was iteratively refined using action research methods [9].
The intervention included: i. Training based around a fellowships program (see below), teaching tours of SE Asian nodes and project meetings. Training focused on activities for three target groups:

Main outcome measures
The primary outcomes were changes in maternal and perinatal care practices during the time of the SEA-ORCHID project intervention. These included adherence to forms of care likely to be beneficial and avoidance of forms of care likely to be ineffective or harmful drawn from The Cochrane Library and the World Health Organization Reproductive Health Library No. 7 (see Table 1).
The secondary outcomes were measures of maternal and perinatal morbidity and mortality, including: N Maternal outcomes # maternal death, eclampsia, perineal trauma, caesarean section rate, postpartum haemorrhage and postpartum pyrexia N Perinatal outcomes # perinatal death, preterm birth (gestational age ,37 weeks), low birth weight (,2500 grams), small for gestational age (SGA) and perinatal asphyxia (Apgar score ,7 at 5 minutes).

Data collection
Data were collected during the following two intervals: preintervention (baseline) during January-December, 2005 and postintervention (endpoint) during January-June, 2008. Manually completed data extraction forms were sent to the country project office for entry into a secure web-based data management system.

Statistical analyses
We used descriptive analyses to describe maternal and infant characteristics for the pre-and post-intervention cohorts across countries and hospitals. Frequencies were used to describe categorical data variables such as nulliparity, preterm birth and caesarean section, and means and standard deviations to describe the continuous data variables of maternal age, gestational age at birth and birth weight. Differences in beneficial and harmful practices, and health outcomes between pre-and post-intervention were presented using risk differences and 95% confidence intervals (CI).
We categorised maternal age to be ,20, 20-34 and $35 years and gestational age to be preterm (,37 weeks) and term ($37 weeks). Potential confounding factors, including maternal age, gestational age, parity and caesarean rates were evaluated for their association with each health outcome in hospitals using chisquare test at the 5% level of significance. All health outcome results were adjusted for gestational age and parity. For each health outcome, risk differences and confidence intervals (RD, 95% CI) were adjusted for any other significant associated factors in the individual country data set using Mantel-Haenzel method. Analyses were done at the Thai site using Stata (version 10.0).

Results
Mean maternal age and nulliparous rate were similar between pre-and post-intervention periods, as were preterm birth and low birth weight. Caesarean section rates were higher in the postintervention period in the participating hospitals in all four countries (Table 2).
Some care practices were largely in line with the evidence at baseline. Following the capacity development intervention, alignment between clinical practices and best evidence improved in many cases. However, in some cases there was no significant change in practice and there were decreases in a few potentially beneficial practices.

Forms of care likely to be beneficial
Practices during the antenatal period (Table 3). There was moderate to high use of antibiotics for women with preterm prelabour rupture of the membranes (pPROM) both at baseline (range 62-92%) and post-intervention (range 75-96%) in line with recommended practice. There was however, a significant decrease in the use of antibiotics for pPROM in the participating hospitals in the Philippines from 92 to 79% (RD 213, 95% CI 222 to 24).
The use of magnesium sulphate for eclampsia was 100% both pre-and post-intervention in the participating hospitals in Indonesia, Malaysia and Thailand. At baseline the use of magnesium sulphate for women with pre-eclampsia varied (range 24-100%). There was a significant increase in the use of magnesium sulphate for pre-eclampsia in the Malaysian sites (24 to 65%, RD 41, 95% CI 24-57) but a significant decrease in the sites in Indonesia (100 to 90%, RD 210, 95% CI 216 to 25) and the Philippines (86 to 60%, RD 225, 95% CI 237 to 214).
Antenatal corticosteroids for women who gave birth before 34 weeks gestation were infrequently used in sites in Indonesia and the Philippines (12% and 15% respectively) at baseline. Postintervention there were significant increases in the proportion of women given antenatal corticosteroids for this indication in the Indonesian (12 to 37%, RD 25, 95% CI 11-39) and Malaysian (68 to 91%, RD 23, 95% CI 7-40) hospitals.  Table 3. Percentage of women receiving forms of care likely to be beneficial during the antenatal period.
Appropriate use of prophylactic antibiotics for caesarean section was close to zero in both the pre-and post-intervention periods in the Indonesian and Malaysian hospitals. A significant increase was observed post-intervention in the hospitals in the Philippines (5 to 38%, RD 33, 95% CI 29-38), but there was a significant decrease in Thailand (49 to 40%, RD 29, 95% CI 213 to 25).
Early cord clamping and cutting was almost universally practiced in pre-and post-intervention periods in the participating hospitals in Indonesia and Malaysia. Significantly lower levels of practice were observed post-intervention in the Philippines (96 to 59%, RD 237 95% CI 240 to 235) and Thailand (100 to 71%, RD 29 95% CI 231 to 227), in keeping with newer recommendations.
Controlled cord traction to deliver the placenta was almost universally practiced in pre and post intervention periods in the Indonesian and Malaysian hospitals. Post-intervention practice significantly decreased in the Philippines (98 to 93%, RD 25 95% CI 27 to 24) and significantly increased in the Thai hospitals (56 to 90%, RD 34 95% CI 31-37).
Practices during postpartum period. There were significant increases in immunisation for hepatitis B in the hospitals in Indonesia (2 to 38%, RD 36, 95% CI 34-38) and the Philippines (2 to 64%, RD 63, 95% CI 60-65). Pre and postintervention immunisation rates for hepatitis B were high in the hospitals in Thailand (100%) and Malaysia (99%).
Forms of care likely to be harmful (Table 5) Perineal hair shaving before birth was significantly less likely post-intervention across all four countries. The largest decrease was observed in the Thai hospitals (74 to 42%, RD 232 95% CI 234 to 229).
Enema use significantly decreased post-intervention in the hospitals in Indonesia, the Philippines and Thailand, which showed the biggest decrease (30 to 6%, RD 224 95%CI 226, 222).
Routine episiotomy for vaginal births significantly decreased post-intervention in the hospitals in Indonesia (54 to 50%, RD 24 95% CI 28 to 20.2), Malaysia (46 to 41%, RD 25 95% CI 29 to 22) and Thailand (91 to 84%, RD 28 95% CI 210 to 26). Despite this, the practice was still frequent in the post-intervention period across hospitals in all countries, especially in Thailand.
Maternal and perinatal outcomes (Table 6) There were no significant differences in the rates of maternal death, severe postpartum haemorrhage or postpartum pyrexia after vaginal birth between pre-or post-intervention. Caesarean rates significantly increased across the hospitals in all four countries (see Table 2). There were small statistically significant variations in postpartum haemorrhage after vaginal birth, intact perineum and pyrexia after caesarean section at some sites, but no consistent pattern.
There were no significant differences in low birth weight, small for gestational age, birth asphyxia and perinatal death detected between pre-and post-intervention periods.
A summary of all the changes in clinical practice is provided in Figure 2.

Discussion
SEA-ORCHID was a large, complex, multi-site project which involved several countries and a substantial sample of women. The outcomes were assessed at the level of the patient, with clinical practices and health outcomes evaluated through file audit by trained staff. The participating sites included provincial, regional and university hospitals. The project was truly collaborative with the centre of gravity of the research in the South East Asian countries, led by teams of local clinicians and researchers supported from Australia. The project built on the existing networks developed through the Cochrane Collaboration and International Clinical Epidemiology Network. The intervention aimed to develop sustainable capacity for individuals rather than simply provide technical support and was tailored to meet local needs and be appropriate to the local context. The intervention was multidisciplinary; targeting doctors, nurses, midwives and non-clinical staff including librarians and biostatisticians. Importantly, SEA-ORCHID was resourced to deliver a complex, multifaceted capacity development project over five years.
The focus of the project was examining whether an intervention to develop capacity for research generation, synthesis and use could result in improvements in healthcare practice. We viewed capacity development in terms of knowledge and skills (training, mentoring, fellowships), resources, leadership and commitment. We believe our intervention had a substantial impact on improving knowledge and skills related to using, synthesising and generating research and resulted in a supportive network of clinical leaders with expertise and commitment to evidence-based practice; this in turn led to some changes in clinical practice. The project was not designed to overcome more structural/systemic barriers to change such as limited availability of drugs or suture materials, shortages of skilled staff, and this is evidenced in our limited impact on practices that were dependent on these factors.
Our study demonstrates that investment in capacity development for research can lead to changes in clinical practice in a broad range of clinical environments. The nature of the SEA-ORCHID intervention which focused on capacity development Table 4. Percentage of women receiving forms of care likely to be beneficial during the intrapartum and postpartum period.
Defined as administration of oxytocin or syntocin at anterior shoulder or after birth (denominator is total vaginal birth), (2) Defined as the proportion of husbands, mothers, sisters, other family members or friends giving either ''some/little'' or ''all/most'' support. For post-intervention the time period is first stage of labour, Defined as given a single dose of ampicillin or cephalosporin after cord clamped (denominator is total caesarean), (4) Vacuum/vacuum + forceps, Rate of use polyglycolic acid suture material (where perineum sutured), (6) Rate of continuous skin closure (where perineum sutured), Excluded stillbirths. doi:10.1371/journal.pone.0023994.t004 and relied on investment in knowledge and skills rather than infrastructure or equipment, makes it likely that similar approaches could be effective in other clinical areas and others settings, both low and high resource. The design of the project, which encouraged tailoring of the intervention to address local needs and barriers, also increases its generalisability. SEA-ORCHID had some limitations. Randomised controlled trials (RCTs) are the most reliable design for assessing the impact of an intervention as they minimise selection bias and confounding. A cluster-RCT design could have been used for this project, with hospitals as clusters. However, the number of clusters included would have been severely limited by the project resources and timelines, leading to substantial risk of baseline imbalance, particularly as there is wide variation in clinical practice and other baseline variables between hospitals. In light of this, a before-andafter design was chosen, a design which was optimal for the project purpose, and the timeframe and funding available.
We were powered to demonstrate an effect on healthcare practices and not maternal or neonatal health outcomes as the resources and time available did not make it feasible to recruit the sample size required to demonstrate an impact on health outcomes. However the healthcare practices examined have been demonstrated to impact on health outcomes. Increased rates of family support have been shown to decrease duration of labour, increase rates of normal vaginal birth and decrease need for analgesia [12]. Similarly our reported decreased rates of episiotomy have the potential to improve health outcomes by reducing perineal trauma and preventing complications [13].
The impact of SEA-ORCHID on other clinical practices varied widely between participating countries and hospitals, reflecting the ways in which these sites differed in terms of culture, infrastructure, and resources. It is likely that the impact of SEA-ORCHID on many of these practices may also be seen over a longer timeframe, reflecting the time required for improvements in knowledge to translate into changes in practice. The barriers to and enablers of each of these changes is being explored through a qualitative evaluation.
Despite being well resourced and comprising a large and experienced research team with excellent local and international connections, the observed improvements to clinical practice were limited. This reflects both the difficulties involved in implementing evidence-based practice changes, and the emphasis of this project on developing research capacity rather than clinical skills. The results of the forthcoming qualitative analysis of barriers and enablers of practice change at the participating hospitals may provide some insights into the reasons for variation between hospitals as well as highlight opportunities for further improvements in practice. In a similar study of four hospitals in China that investigated the impact of national and local initiatives to promote evidence-based obstetric care, the influence of other factors, such as hospital directors and legislation, were deemed to be especially important in changing provider behaviour [14]. A recent study in three hospitals in Jordan highlighted that the need to identify effective methods to support evidence-based maternal healthcare practice remains an imperative [15] and the results of SEA-ORCHID also suggest that there is still much to be learnt about effective knowledge translation strategies in this context.
Over the course of the SEA-ORCHID project there was an increase in the overall caesarean section rate. We believe this change is reflective of a global increase in caesarean section rate [16]. A change from secondary to tertiary status at some of the participating hospitals during the project and subsequent increase in their high risk patient profile may also have influenced this. Table 5. Percentage of women receiving forms of care likely to be harmful.  Table 6. Maternal and perinatal outcomes.   In addition to the clinical practice and health outcomes reported in this paper we have also assessed a range of intermediate outcomes at the participating sites. These outcomes include randomised controlled trials, systematic review and clinical practice guideline development, and awareness and understanding of evidence-based practice (for examples see [17,18]). The impact of SEA-ORCHID on these outcomes will be reported in detail elsewhere, but the findings are consistent with the hypothesis that increasing capacity in these areas can lead to improved clinical practice.
The capacity built through SEA-ORCHID will enable practice at participating hospitals to keep pace with research discoveries and facilitate a research-informed culture. We observed this during the project when the evidence about the best practice in management of third stage of labour changed. Pleasingly, several centres modified their practice in line with this new evidence, demonstrating their capacity to understand and apply evidence appropriately. It is also likely that the impact of the SEA-ORCHID project will extend beyond maternal and perinatal health care and into practice in other clinical areas.
While we no longer have resources to invest in the training or to support travel and fellowships, we are hopeful that the network developed through the project will be maintained and will lead to ongoing relationships, mentoring, and research collaborations. We are exploring opportunities for shared projects and are encouraging involvement in the Cochrane Collaboration as a vehicle to sustain the relationships developed.
If we are to reach the Millennium Development Goals related to health we need to substantially invest in capacity development for generating, synthesising and using research in LMICs, in order to improve healthcare practice. This paper has demonstrated that such an investment can have an impact on improving healthcare practice and contribute to this goal.