Citation:George A, Young M, Bang A, Chan KY, Rudan I, et al. (2011) Setting Implementation Research Priorities to Reduce Preterm Births and Stillbirths at the Community Level. PLoS Med 8(1): e1000380. doi:10.1371/journal.pmed.1000380
Published: January 4, 2011
Copyright: © 2011 George et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding:IR and CGV received support as consultants from the Child Health and Nutrition Research Initiative (CHNRI) and the Global Alliance to Prevent Prematurity and Stillbirths (GAPPS). GAPPS, with support from the Bill & Melinda Gates Foundation, March of Dimes, PATH, Save the Children, UNICEF, and WHO, also funded the meeting in which the research prioritization process was initiated. There were no other sources of funding and all authors (except IR and CGV) volunteered their time to conduct this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: While this paper was being developed IR was a consultant for the Child Health and Nutrition Initiative (CHNRI) of the Global Forum for Health Research. Because the authors all work in this research area, it is conceivable that they could benefit from any increased attention and financing regarding implementation research in preventing prematurity and stillbirth.
Abbreviations: AEA, average expert agreement;CHNRI, Child Health Nutrition Research Initiative;GAPPS, Global Alliance to Prevent Prematurity and Stillbirth;MDG, Millennium Development Goal;RPS, research priority score;WHO, World Health Organization
Provenance: Not commissioned; externally peer reviewed.
- Preterm birth complications are the leading cause of neonatal mortality, contributing 1 million deaths annually. Stillbirths account for another 3.2 million deaths. Both causes of perinatal mortality are inextricably linked to maternal health and to conditions at birth.
- While some community-based interventions have proved effective in controlled settings and specific contexts, the implementation research challenge is to understand how to sustain these interventions at scale in different contexts.
- A systematic process based on the Child Health and Nutrition Research Initiative (CHNRI) methodology was used to score and rank implementation research questions regarding community-based maternal–newborn interventions that address prematurity and stillbirths in different contexts at scale.
- Of the 55 research questions that were reviewed in this way, the top five addressed equity (e.g., reaching the poor and marginalized, reducing financial barriers), behavioral practices and skills (e.g., engaging with social norms, identifying prematurity), and quality of care provided by community health workers. The top 15 questions encompassed issues pertaining to behavioral interventions, community health workers, referral, and managing health systems.
It is estimated that 3.2 million stillbirths occur each year globally, 1 million of which happen during birth . In addition, complications from preterm birth (before 37 completed weeks of gestation) are the leading cause of death for newborns, contributing an additional 1 million or 12% of child deaths ,. In 2009, more than 200 stakeholders attended the International Conference on Prematurity and Stillbirth convened by the Global Alliance to Prevent Prematurity and Stillbirth (GAPPS, http://www.gapps.org/). The community expert group at the conference included 15 members drawn from technical and funding organizations in addition to program implementers and researchers from around the world (see Acknowledgments section for specific names). In their discussions, the group framed efforts to address preterm and stillbirths within the broader context of maternal–newborn interventions. As most of the evidence supporting these interventions emanates from research projects in controlled settings in specific contexts, the group identified the main challenge being implementing interventions at scale in different contexts. Based on these discussions, the group began a research prioritization exercise for implementation research on community-based maternal-newborn interventions that address prematurity and stillbirths at scale in different contexts. In this paper, we present the results of this exercise.
A number of research prioritization efforts have recently been applied to various health topics and health system themes –. The GAPPS community expert group chose the methodology proposed by the Child Health and Nutrition Initiative (CHNRI) to systematically list and score research questions. The CHNRI methodology was selected because its conceptual framework – has been used in numerous areas by different national and international organizations – (further information on CHNRI methodology, validity, and potential limitations are discussed in Table S1). The group followed three main stages to derive research priorities (detailed in Box 1). Briefly, guided by the CHNRI methodology the group evaluated 55 research questions against five main criteria:
- Is the research question answerable in an ethical way?
- Does the research question have the potential to reduce the disease burden (due to prematurity and stillbirths)?
- Is it likely that the proposed research would address obstacles to scaling up?
- Would the proposed research attract funding support and national policy attention?
- Would the research results be owned by local actors, including political authorities and elected representatives, health workers, district managers, and communities?
Box 1. CHNRI Process
Stage 1: Defining the research context, questions, and criteria for priority setting
When: May–September 2009
How: Group discussions and subsequent e-mails
- Consensus on research context defined by space (developing countries), time (the next 5–10 y), the population of interest (children under five years of age), and disease burden of interest (preterm and stillbirths). Respondents were also asked to keep in mind that all research questions started with the following introduction: “When implementing a community based maternal newborn intervention package that addresses prematurity and stillbirths in different contexts at scale…”
- Consensus around 55 implementation research questions grouped according to the following research domains: community engagement, behavioral skills and practices, community health workers, rational drug use, management health systems, and referral.
- Consensus on the five criteria used to rank the research questions: ethical answerability, disease burden reduction, ability to support scale-up, likelihood to attract financial and policy support, ownership by local actors.
Stage 2: Enlisting experts to systematically score the research questions
When: October 2009 – March 2010
How: Preliminary e-mails sent to 85 leading experts on community based approaches and maternal-newborn health in developing countries identified through a literature search and through snowballing of program managers. The spreadsheet was also translated into French and Spanish in order to ensure the participation of colleagues from Francophone Africa and Latin America.
- 42 experts agreed to participate
- 31 experts were able to complete the spreadsheets, independently scoring the 55 research questions by each of the five criteria by answering “Yes” (1 point), “No” (0 points), undecided (0.5 points), or insufficiently informed to answer the question (missing input).
Stage 3: Computing and writing up results
When: March–August 2010
How: An intermediate score was calculated for each of the five criteria and the RPS computed as the mean of all five intermediate priority scores – (Table S3). AEA scores were computed for each research question as the average proportion of scorers that agreed on the 55 questions asked (Table S1).
- 29 correctly completed spreadsheets analyzed with all 55 research questions systematically scored and ranked in order of priority and agreement.
- Draft circulated to all participants for feedback before being finalized.
Respondents were 39% women and diverse in terms of regional representation (26% sub-Saharan Africa, 16% Asia, 16% Latin America, 10% Europe, 32% North America). While a substantial number of respondents were based in North America, they all work full-time in developing country contexts. Half of the respondents were based in research institutions, whereas the other half were in charge of implementing programs whether through nongovernmental organizations, UNICEF country offices, or USAID headquarters. Nonrespondents were not significantly different from respondents (Table S2).
The research question that was highlighted as the most important out of all 55 reviewed was “Evaluate ways to reduce the financial barriers to facility births at the community level—e.g., user fee exemptions, emergency loans, conditional cash transfers, transportation vouchers, etc.” Other research questions among the top five prioritized also addressed equity issues (reaching the poor and marginalized), but also behavioral practices and skills (engaging with social norms, identifying prematurity) and service delivery (measuring and maintaining quality of care provided by community health workers). The remaining top ten research questions (Table 1) include other behavioral skills and practices (thermal care and feeding for preterm babies, birth planning), concerns about how to best motivate and compensate community health workers and their supervisors, and different dimensions of making referrals more effective. Congruent with the priority need to measure and maintain quality of care by community health workers, rational drug use by community health workers and community engagement with regard to audits was also listed among the top 25 research questions that received an overall research priority score (RPS) of 0.75 or greater (Table 2).
Table 3 shows the ten research questions that were assigned the lowest RPSs. Several broad policy questions (human resource planning, gender profiles, budget flows, accountability, and monitoring systems) are listed here, along with some questions related to the sequencing of community interventions and one specific question regarding private provider practice (delayed cord clamping). Questions from almost all research avenues were found among the bottom ten research questions, suggesting that no one area was completely discriminated against by the scoring. Furthermore, even these lower-ranked research questions received relatively high RPSs compared to those arising from other CHNRI exercises. The RPS for all 55 questions ranged from 0.86 to 0.56, in contrast to other CHNRI exercises, which have generated RPS ranges from 0.90 to 0.25 –. This suggests that respondents collectively considered all implementation research questions as fairly important.
Research questions did vary in specificity. For example, broad questions such as “evaluate community-based strategies to reach the poor and marginalized” were scored alongside very specific questions like “evaluate ways to provide thermal care and feeding the preterm baby.” Both broad and specific questions were ranked in the top and bottom ten implementation research questions, suggesting that no bias existed against the kind of question asked.
The CHNRI methodology evaluates certain dimensions of each research question according to defined criteria. For example, “Evaluate methods and levels of accountability that can be ensured” was not considered to affect disease burden and “Evaluate ways to ensure delayed cord clamping in deliveries assisted by private providers” was not scored as likely to attract funding support or national policy attention. Among the five criteria, the most discriminative was the one related to disease burden reduction, while the least discriminative was the one regarding ethical answerability.
As mentioned, the relatively high mean scores assigned to questions across all criteria (apart from disease burden reduction) indicate that most of the respondents were fairly optimistic about the value of implementation research questions. Average expert agreement (AEA) ranged from 0.82 to 0.49. Similar to other CHNRI exercises, AEA showed a direct positive association with RPSs, indicating that there was more agreement among experts about what were the priority research questions. This is a property that is inherent to the way AEA is measured: very high or very low RPS scores require high levels of expert agreement, while substantial disagreement among experts will lead to RPS moving closer to a mean value -.
To determine whether any systematic bias existed against certain questions due to the profile of the respondent, we analyzed scores for researchers and implementers separately. We found at least a 10% difference in the scoring assigned for 20% of the research questions (Table 4). The 11 questions for which there was a significant difference between researchers and implementers are spread across each research avenue, suggesting no one particular research area was affected by this difference of opinion. In ten out of these 11 questions, implementers ranked the implementation research question as being of higher value than researchers.
The top 25 research questions that have been prioritized span a broad range of issues (Table 2). These implementation research priorities address fostering and sustaining specific behavioral skills and practices at the community level, engaging communities in monitoring service delivery through audits, and improving referral. With regard to service delivery, a host of implementation research questions about the management of community health workers, along with the health system supports they require to function, were stressed. Finally, issues of equity, financing, and referral were highlighted, reflecting how community-based approaches cannot be dealt with in isolation from broader health system concerns.
While many of the implementation research priorities identified can be generalized across community-based maternal, newborn, and child health areas, a few distinctions may be particular to this specific exercise. Issues related to referral were present three times within the top 25 research questions. There is little implementation research on linking families from homes to facilities, or referral more broadly, in low-income countries –. While important gains have been made with task-shifting, effective and equitable referral remains vital, because the most serious cases of prematurity and other birth complications cannot be handled at the community level.
Implementation research questions related to community engagement and some other broader policy concerns central to managing health systems, such as human resource planning and monitoring systems, were overall not given high priority by respondents. Nonetheless, even the bottom ten research questions received high RPSs relative to other CHNRI exercises. This could be because the other exercises had more discriminatory criteria or because previous exercises compared different kinds of research (basic science versus implementation research). It may be easier for experts to discern between very different research areas (basic science versus implementation research) than to discern between areas of implementation research, which they may consider to be of relatively similar importance.
In addition, many of the implementation research questions do not by themselves contribute to improved maternal newborn outcomes. Their value comes forth when combined with other implementation issues that together make a more comprehensive and coherent community-based response with linkages to primary health care service delivery. It might therefore be difficult for respondents to think about specific implementation research questions in isolation from their broader social and health systems contexts.
The partiality toward some areas of implementation research could reflect the profile of respondents. A comparison of scoring by implementers and researchers did show some differences—not across any particular kind of research question, but in the direction of the bias, with implementers ranking implementation research questions higher than did researchers. The reasons for this difference among 20% of the questions are not known, but seem to indicate that implementers perceive the results of implementation research to be more powerful if effectively implemented.
While the CHNRI methodology provides a systematic and transparent way to rank research questions that purposefully avoids biases introduced by group dynamics dominated by powerful individuals, it still is a very lengthy process to undertake. Respondents must score 55 research questions according to five criteria that have three subcomponents each, resulting in 825 dimensions to respond to in the spreadsheet. This makes it a complex spreadsheet and likely does not help response rates. Eliciting participation via e-mail alone was not successful—only 42 out of 85 experts responded to the preliminary e-mail. The 42 experts that did express interest reflected a group that was more familiar with the GAPPS conference and had current working relationships with the lead authors who managed the exercise.
Despite these drawbacks, this exercise represents an important collaboration between researchers and program implementers to jointly identify the key implementation research questions vital to improving community-based maternal and newborn interventions that address preterm and stillbirths. The exercise also developed new criteria deemed more appropriate to implementation research, which require further testing and refinement to improve their discriminatory power.
Success in reducing stillbirth and prematurity rates, and in increasing the survival of preterm infants in low-income countries, is strongly dependent on achieving high and equitable coverage with existing cost-effective interventions ,. Yet coverage of such interventions remains unacceptably low in most countries. For example, across 68 countries with the highest mortality, only 54% of women deliver with a skilled birth attendant and 38% receive a postnatal visit . Furthermore, coverage levels are particularly low among poor and rural families in these countries. Community-based interventions are therefore essential to reach population subgroups whose current access to health facilities is severely limited. The effect of expanding coverage of family and community care to 90% can by itself lead to a 15%–32% reduction in neonatal mortality . Nonetheless, the knowledge gaps around how to sustain these programs at scale in different contexts remain significant.
While important reviews – have helped to spur attention to community-based maternal newborn issues, with intriguing results for specific interventions ,, the implementation research priorities identified in this article will, we hope, help to secure further research attention and financing for this important area. Priority research areas identified include equity concerns (such as removal of financial barriers and responsiveness to the poor and marginalized), specific behavioral skills and practices, and the management of community health workers including referral care. The challenge is now raised; will communities, governments, donors, research institutions, and international organizations respond?
The CHNRI methodology for setting priorities in health research investments.
(0.04 MB DOC)
Profile of respondents and non-respondents.
(0.04 MB DOC)
All 55 implementation research questions scored and ranked.
(0.15 MB DOC)
The authors thank the CHNRI team who helped to develop these methods, particularly Robert Black, Shams El Arifeen, and Harry Campbell. We thank Paola Canahuati, the UN translation service, Mariame Sylla, and Jose Rolando Figueroa for translating the CHNRI scoring spreadsheet from English into French and Spanish. We are grateful to Zulfiqar Bhutta, Neal Brandes, and Luc de Bernis for their expert review of an earlier draft of this manuscript.
The GAPPS expert group on community-based strategies and constraints includes those who attended the Seattle GAPPS meeting and those who participated in the research prioritization exercise: Kaosar Afsana (Bangladesh Rural Advancement Committee, Dhaka, Bangladesh), Fernando Althabe (Institute for Clinical Effectiveness and Health Policy, Buenos Aires, Argentina), Shams Arifeen (International Centre for Diarrheal Disease Research, Bangladesh, Dhaka, Bangladesh), Abhay Bang (Society for Education, Action and Research in Community Health, Gadchiroli, India), Maria Bardolet (UNICEF, Niamey, Niger), Fernando Barros, Cesar G. Victora (University of Pelotas, Pelotas, Brazil), Neal Brandes, Troy Jacobs (USAID Washington, DC, USA), Guy Clarysse (UNICEF, Dakar, Senegal), Joseph de Graft-Johnson (Saving Newborn Lives, Washington, DC, USA), Alfredo Fort (Macro International, Calverton, MD, USA), Hernando Gaitan (Institute of Clinical Research, National University of Columbia, Bogota, Columbia), Asha George, Mark Young (UNICEF New York, USA), Kate Gilroy, Vishwajeet Kumar, Henry Perry (John Hopkins Bloomberg School of Public Health, Baltimore, USA), Zelee Hill (University College London, London, UK), Hakeem Jokhio (Aga Khan University, Karachi, Pakistan), Nilda Lambo (UNICEF, Lusaka, Zambia), Tippawan Liabsuetrakul (Faculty of Medicine, Prince of Songkla University, Hat Yai, Songkla, Thailand), Stephanie Martin (Program for Appropriate Technology in Health, Seattle, WA, USA), Grace Mlava (UNICEF, Lilongwe, Malawi), David Marsh, Salim Saddrudin (Save the Children, Washington, DC, USA), Indira Narayan (BASICS/USAID Washington, DC, USA), Pius Okong (St. Francis Nsambya Hospital, Kampala, Uganda ), Adama Ouedrago (UNICEF, Niamey, Niger), Luwei Pearson (UNICEF, Addis Ababa, Ethiopia), Stefan Peterson (Karolinska Institute, Stockholm, Sweden), Shamim Qazi (WHO, Geneva, Switzerland), M. G. Sall (University of Dakar, Dakar, Senegal), Sharad Sharma (Department of Health Services, Kathmandu, Nepal), Judith Standley (Independent Program Consultant, Israel), Aboubacry Thiam (BASICS/USAID, Dakar, Senegal), Celestin Traore (UNICEF Bujumbura, Burundi).
ICMJE criteria for authorship read and met: AG MY AB KYC IR CGV MC CR. Agree with the manuscript's results and conclusions: AG MY AB KYC IR CGV MC CR. Designed the experiments/the study: AG MY IR CGV MC. Analyzed the data: AG MY KYC IR MC. Collected data/did experiments for the study: AG CGV. Enrolled patients: AG. Wrote the first draft of the paper: AG MY. Contributed to the writing of the paper: AG MY KYC IR CGV MC CR. Participated in designing the study and contributed to writing the manuscript by way of comments: AB. Led the development of the CHNRI methodology with assistance of the large multidisciplinary team of experts: IR. Suggested the original idea: CGV. Reviewed the data analysis: CR.
- 1. Lawn JE,Lee ACC,Kinney M,Sibley L,Carlo WA,et al. (2009) Two million intrapartum-stillbirths and neonatal deaths: Where, why and what can be done? Int J Gynaecol Obstet 107: S5–S19.
- 2. Rubens CE,Victora CG,Gravett MG,Nunes TM, editors. (2010) Global report on preterm birth & stillbirth: The foundation for innovative solutions and improved outcomes. BMC Pregnancy Childbirth 10(Suppl 1).
- 3. Black RE,Cousens S,Johnson HL,Lawn JE,Rudan I,et al. (2010) Child Health Epidemiology Reference Group of WHO and UNICEF. Global, regional, and national causes of child mortality in 2008: a systematic analysis. Lancet 375: 1969–1987.
- 4. Sharan P,Gallo C,Gureje O,Lamberte E,Mari JJ,et al. (2009) World Health Organization-Global Forum for Health Research - Mental Health Research Mapping Project Group. Mental health research priorities in low- and middle-income countries of Africa, Asia, Latin America and the Caribbean. Br J Psychiatry 195: 354–363.
- 5. Ranson K,Law TJ,Bennett S (2010) Establishing health systems financing research priorities in developing countries using a participatory methodology. Soc Sci Med 70: 1933–1942.
- 6. Ranson K,Chopra M,Atkins S,Dal Poz MR,Bennett S (2010) Priorities for research into human resources for health in low- and middle-income countries. Bull World Health Organ 88: 435–443.
- 7. Robotin MC,Jones SC,Biankin AV,Waters L,Iverson D,et al. (2010) Defining research priorities for pancreatic cancer in Australia: Results of a consensus development process. Cancer Causes Control 21: 729–736.
- 8. Rudan I,Gibson J,Kapiriri L,Lansang MA,Hyder AA,et al. (2007) Setting priorities in global child health research investments: Assessment of principles and practice. Croat Med J 48: 595–604.
- 9. Rudan I,Chopra M,Kapiriri L,Gibson J,Ann Lansang M,et al. (2008) Setting priorities in global child health research investments: Universal challenges and conceptual framework. Croat Med J 49: 307–317.
- 10. Rudan I,Gibson JL,Ameratunga S,El Arifeen S,Bhutta ZA,et al. (2008) Setting priorities in global child health research investments: Guidelines for implementation of the CHNRI method. Croat Med J 49: 720–733.
- 11. Rudan I,El Arifeen S,Black RE,Campbell H (2007) Childhood pneumonia and diarrhoea: setting our priorities right. Lancet Infect Dis 7: 56–61.
- 12. Tomlinson M,Chopra M,Sanders D,Bradshaw D,et al. (2007) Setting priorities in child health research investments for South Africa. PLoS Med 4: e259. doi:10.1371/journal.pmed.0040259.
- 13. Tomlinson M,Swartz L,Officer A,Chan KY,Rudan I,et al. (2009) Research priorities for health of people with disabilities: An expert opinion exercise. Lancet 374: 1857–1862.
- 14. Tomlinson M,Rudan I,Saxena S,Swartz L,Tsai AC,et al. (2009) Setting investment priorities for research in global mental health. Bull World Health Organ 87: 438–446.
- 15. Bahl R,Martines J,Ali N,Bhan MK,Carlo W,et al. (2009) Research priorities to reduce global mortality from newborn infections by 2015. Pediatr Inf Dis J 28(Suppl 1): S43–S48.
- 16. Fontaine O,Kosek M,Bhatnagar S,Boschi-Pinto C,Chan KY,et al. (2009) Setting research priorities to reduce global mortality from childhood diarrhoea by 2015. PLoS Med 6: e41. doi:10.1371/journal.pmed.1000041.
- 17. Macintyre K,Hotchkiss DR (1999) Referral revisted: Community financing schemes and emergency transport in rural Africa. Soc Sci Med 49: 1473–1487.
- 18. Murray SF,Pearson SC (2006) Maternity referral systems in developing countries: current knowledge and future research needs. Soc Sci Med 62: 2205–2215.
- 19. Lee AC,Lawn JE,Cousens S,Kumar V,Osrin D,et al. (2009) Linking families and facilities for care at birth: What works to avert intrapartum-related deaths? Int J Gynaecol Obstet 107(Suppl 1): S65–S88.
- 20. Darmstadt GL,Walker N,Lawn JE,Bhutta ZA,Haws RA,et al. (2008) Saving newborn lives in Asia and Africa: cost and impact of phased scale-up of interventions within the continuum of care. Health Policy Plan 23: 101–117.
- 21. Adam T,Lim SS,Mehta S,Bhutta ZA,Fogstad H,et al. (2005) BMJ 331: 1107.
- 22. Darmstadt GL,Bhutta ZA,Cousens S,Adam T,Walker N,et al. (2005) Evidence-based, cost-effective interventions: How many newborn babies can we save? Lancet 365: 977–988.
- 23. Bhutta ZA,Darmstadt GL,Hasan BS,Haws RA (2005) Community-based interventions for improving perinatal and neonatal health outcomes in developing countries: A review of the evidence. Pediatrics 115(Suppl 2): 519–617.
- 24. Bhutta ZA,Ali S,Cousens S,Ali TM,Haider BA,et al. (2008) Alma-Ata: Rebirth and Revision 6 Interventions to address maternal, newborn, and child survival: What difference can integrated primary health care strategies make? Lancet 372: 972–989.
- 25. Darmstadt GL,Lee AC,Cousens S,Sibley L,Bhutta ZA,et al. (2009) 60 Million non-facility births: Who can deliver in community settings to reduce intrapartum-related deaths? Int J Gynaecol Obstet 107(Suppl 1): S89–S112.
- 26. Lewin S,Munabi-Babigumira S,Glenton C,Daniels K,Bosch-Capblanch X,et al. (2010) Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev 3: CD004015. doi:10.1002/14651858.CD004015.pub3.
- 27. Nair N,Tripathy P,Prost A,Costello A,Osrin D (2010) Improving newborn survival in low-income countries: Community-based approaches and lessons from South Asia. PLoS Med 7: e1000246. doi:10.1371/journal.pmed.1000246.
- 28. Bang AT,Bang RA,Reddy HM (2005) Home-based neonatal care: Summary and applications of the field trial in rural Gadchiroli, India (1993–2003). J Perinatol 25: S108–S122.
- 29. Azad K,Barnett S,Banerjee B,Shaha S,Khan K,et al. (2010) Effect of scaling up women's groups on birth outcomes in three rural districts in Bangladesh: A cluster-randomised controlled trial. Lancet 375: 1193–1202.
- 30. Tripathy P,Nair N,Barnett S,Mahapatra R,Borghi J,et al. (2010) Effect of a participatory intervention with women's groups on birth outcomes and maternal depression in Jharkhand and Orissa, India: A cluster-randomised controlled trial. Lancet 375: 1182–1192.