https://orcid.org/0000-0002-6392-0475
Figures
Abstract
In 2017, WHO and global partners launched ‘The Network for Improving Quality of Care for Maternal, Newborn and Child Health’ (QCN) seeking to reduce in-facility maternal and newborn deaths and stillbirth by 50% in health facilities by 2022. We explored how the QCN theory of change guided what actually happened over 2018–2022 in order to understand what worked well, what did not, and to ultimately describe the consequences of QCN activities. We applied theory of change analysis criteria to investigate how well-defined, plausible, coherent and measurable the results were, how well-defined, coherent, justifiable, realistic, sustainable and measurable the assumptions were, and how independent and sufficient the causal links were. We found that the QCN theory of change was not used in the same way across implementing countries. While the theory stipulated Leadership, Action, Learning and Accountability as the principle to guide network activity implementation other principles and varying quality improvement methods have also been used; key conditions were missing at service integration and process levels in the global theory of change for the network. Conditions such as lack of physical resources were frequently reported to be preventing adequate care, or harm patient satisfaction. Key partners and implementers were not introduced to the network theory of change early enough for them to raise critical questions about their roles and the need for, and nature of, quality of care interventions. Whilst the theory of change was created at the outset of QCN it is not clear how much it guided actual activities or any monitoring and evaluation as things progressed. Enabling countries to develop their theory of change, perhaps guided by the global framework, could improve stakeholder engagement, allow local evaluation of assumptions and addressing of challenges, and better target QCN work toward achieving its goals.
Citation: Dube A, Mwandira K, Akter K, khatun F, Lemma S, Seruwagi G, et al. (2024) Evaluating theory of change to improve the functioning of the network for improving quality of care for maternal, newborn and child health. PLOS Glob Public Health 4(8): e0003532. https://doi.org/10.1371/journal.pgph.0003532
Editor: Julia Robinson, PLOS: Public Library of Science, UNITED STATES OF AMERICA
Received: December 4, 2023; Accepted: July 3, 2024; Published: August 1, 2024
Copyright: © 2024 Dube et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The original qualitative data are not made available since some responses contain potentially identifiable information at multiple levels including for individuals, health facilities, and organizations; the Institutional Review Board conditions stipulated that participants will not be at risk of identification through their responses; participant consent forms did not include obtaining explicit consent regarding the possible use of anonymized data in the public domain via a data repository. Non-author contact information for enquiries about data access: please email dataprotection@ucl.ac.uk.
Funding: This work was funded by the Medical Research Council (MRC) Health Systems Research Initiative 5th call via grant MR/S013466/1 to TC at UCL Institute for Global Health, United Kingdom, YS and JS at Johns Hopkins University, United States of America, KA and AK at Diabetic Association of Bangladesh Perinatal Care Project, Bangladesh, CM at Parent and Child Health Initiative, Malawi, GS at Makerere University School of Public Health, Uganda, and ME at University of Oxford, United Kingdom; and by the Bill & Melinda Gates foundation via grant INV-007644 to TM at LSHTM, United Kingdom. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Far too many women and babies continue to die from complications in pregnancy and childbirth due to poor quality of and access to peripartum care, especially in low and middle income countries. In 2020, over 2.4 million newborn babies and an estimated 287 000 women died due to lack of quality of care at or immediately after child birth and in the first days of newborn life [1–6]. In 2017, WHO and global partners launched ‘The Network for Improving Quality of Care for Maternal, Newborn and Child Health’ (QCN) [7] seeking to reduce in-facility maternal and newborn deaths and stillbirth by 50% in health facilities by 2022, initially in nine countries: Bangladesh, Cote d’Ivoire, Ethiopia, Ghana, India, Malawi, Nigeria, Tanzania and Uganda. Their aim was to build a cross-country platform for joint-learning around quality improvement implementation approaches based on a shared theory of change and shared health outcome goals. The QCN operated at global, national and local levels, facilitating implementation of a range of quality improvement efforts for different aspects of maternal, newborn and child healthcare, starting in selected ‘learning facilities’ in ‘learning districts’ and scaling up to other health facilities and districts [7].
Although there is an emerging body of work on networks as a potential change strategy in both low- and middle-income countries and high-income countries [8–11], interventions to improve quality of care for maternal, newborn and child health are complex and require comprehensive and detailed theories of change that contribute nuanced understanding of how to address evidence-to-practice gaps and ultimately contribute to better health outcomes [12]. Investigating assumptions and causal linkages set out in theories of change through evaluation can also contribute to better design and implementation of programmes, and refined theories of change.
Theories of change can play a critical role during the formulation and design of programmes and can as well enable monitoring and evaluation of programmes that is responsive to stakeholder needs [13]. Theories of change have been used as frameworks for designing simple and complex interventions. Project and programme stakeholders have utilised theories of change as strategic plans [14–16]. Theories of change have also been at the centre of public health intervention evaluation. For example, programme managers and evaluation teams have used theories of change in the development of process as well as outcome indicators for monitoring of implementation [17], formulation of evaluation questions, testing assumptions [16, 18, 19] and exploration of the influence of programme context on performance [16]. Theories of change can also be used in the identification of implementation or programme failures, side effects, effectiveness and causal explanations [20].
In 2016–7, WHO, governments and key international development partners developed a cross-country theory of change for the quality of care network for maternal and newborns. The theory of change was developed by global level stakeholders of the QCN (led by the WHO and UNICEF core group) from September 2016 to February 2017 and shared with 340 national stakeholders and partners at the launch of the network in Lilongwe, Malawi in February 2017 [21]. The theory highlights intervention themes and processes that are necessary for the attainment of clinical and behavioural outcomes that lead to improved quality and outcomes of care for women and newborns. The theory of change further presents socioeconomic status, gender and resilience and sustainability as cross-cutting infleunces (Fig 1). The theory of change was used by QCN stakeholders primarily as a logical framework to guide areas of implementation, and to guide monitoring and evaluation of the network including via comparison of progress between countries [7, 21].
The aim of this paper is to explore how the quality of care network theory of change guided what actually happened over 2018–2022 in order to understand what worked well, what did not, and to ultimately help explain the consequences of QCN activities [22].
Methods
We conducted an evaluation of the global quality of care network focusing on four implementing countries as case studies: Malawi, Bangladesh, Ethiopia and Uganda [23]. Our main research questions were around the emergence [24], legitimacy [25], and effectiveness of the network [26]. We also conducted a social network analysis of the quality of care network [27], evaluated the influence of contextual factors at individual, organisational and institutional levels on the functioning of the network [28], looked at the learning and innovation aspects of the network [29], and its sustainability [23] (also see S1 Text for an overview of QCN evaluation work). The main findings of our evaluation were that global and national leadership elements of QCN have been most effective to date; action, learning and accountability have been more challenging, partner or donor dependent, remaining to be scaled-up, and pandemic-disrupted. We draw on all of these studies as well as bespoke analyses for this paper specifically investigating how and why aspects of the network worked, or did not. Iterative rounds of interviews were conducted with key stakeholders such as QCN coordinators, UNICEF, WHO, UNFPA, and GIZ national representatives, Ministry of Health monitoring and evaluation officers, directors of health and social services at each learning facility, and quality improvement officers in each country at both national and local level. We reviewed accessible published and unpublished documents and communications relating to the quality of care network at global level and at national and local levels in each of the four countries. These included strategic plans and management documents, operational plans, directives, formal minutes, and reports. We conducted non-participant observations of multi-country, national-level and district level meetings in case-study countries. Activities at district level were also observed; including visits to two better and two least performing ‘learning facilities’ in each of the four countries in several iterative rounds. Table 1 provides an overview of the data we collected.
We adapted a psychometrically validated tool developed for evaluating clinical networks to evaluate the network at national and local levels in Bangladesh, Ethiopia, Malawi and Uganda (Appendix 5 in S2 Text). We conducted several rounds of the survey in each country between October 2019 and March 2022 and in each round a wide variety of network member cadres (clinicians, managers, and advisors) were surveyed. A detailed report on the methodology of the whole quality of care network evaluation is presented in S2 Text, and we provide reflections on our experiences of evaluating the quality of care network in another paper in our QCN Evaluation collection by Gloria Seruwagi and colleagues [30].
Theory of change analysis
We applied theory of change analysis criteria developed by John Mayne [31]. We did an ex-post analysis in order to appraise the whole theory of change. We looked at results, assumptions and causal pathways. We investigated how well-defined, plausible, coherent and measurable the results were, how well-defined, coherent, justifiable, realistic, sustainable and measurable the assumptions were, and how independent and sufficient the causal links were. Findings from this analysis were then used to explore how the quality of care network theory of change guided what actually happened over 2018–2022 in order to understand what worked well, and what did not, in each country, and globally [31]. A data extraction tool was used to pull together data from quality of care network survey reports, qualitative findings, facility observation reports, meeting observation reports, strategic documents and our other seven papers evaluating the quality of care network [23–29]. This data extraction tool, Table 2, was refined from Daruwalla 2019 [32] to mine resources and stakeholders, quality of care network activities, changes at community level, changes at facility level, and outcomes at national level, documentation of pre-conditions, assumptions and challenges.
Ethics
Ethical approval was received from University College London Research Ethics Committee (ref: 3433/003); BADAS Ethical Review Committee (ref: BADAS-ERC/EC/19/00274), Ethiopian Public Health Institute Institutional Review Board (ref: EPHI-IRB-240-2020), National Health Sciences Research Committee in Malawi (ref: 19/03/2264) and Makerere University Institutional Review Board (ref: Protocol 869). The conduct of the evaluation was based on clear ethical standards which assured confidentiality, privacy, anonymity and informed consent. All respondents provided verbal or written informed consent. All respondents were informed of: (i) the purpose of the evaluation; (ii) their right to refuse to participate; and (iii) that their possible decision not to participate would not be held against them or affect their status in the network.
Results
In this section, we present the results of our analysis of the global theory of change for a quality of care network for improving maternal, newborn and child health in low and middle income countries focusing on four countries: Malawi, Uganda, Bangladesh and Ethiopia. We first present the overall understanding of the network. This is followed by theory of change intervention themes, processes, and clinical and behaviour outcomes. This section ends with a presentation of aspects that we found to be missing in the theory of change even though we found them to be key in the achievement of QCN results in specific countries.
Awareness, understanding and owning the network
We found that governments and national level stakeholders in Bangladesh, Ethiopia, Malawi and Uganda were aware of the network history, mission and its organisation structure while awareness was limited amongst implementers at sub-national and facility level. Several respondents reported:
"The other problem is a national problem in Ethiopia in general, particularly in our region, it (QCN) was considered as a campaign work. I thought it is just one or two months’ work. I understood the details later on. It’s after reviewing different documents about the network, that I understood it is a global initiative" (Ethiopia-05-facility staff)
“Of course, they [Quality of care network activities] were there but doing it unknowingly, you would find that you are doing work very nicely but you are not aware, you are not appreciating yourself like you have done this one.” (Uganda-02-facility staff)
“We still need to do capacity building to the leaders. I am saying this because the leaders or management team of the health bureau don’t have sufficient awareness about quality of care. When I join the health bureau as a quality officer, I have clearly observed this and it has been a serious challenge.” (Ethiopia-07-national partner)
It was also found however that awareness at learning facilities and sub-nationally increased over time.
We found that, much as the network was a government initiative within each country, funding and technical support for QCN activities was provided to national level QCN implementers by different national level NGO implementing partners. This resulted in some sub-national level actors perceiving QCN activities of the network as partner’s work rather than looking at the QCN as a government-driven initiative. For instance:
…these significant national-level partners, which could be best classified as “change agents” within the Bangladesh system, are Save the Children-IHI with their MaMoni project, and UNICEF, with its EMEN project…A further “change agent” within the system is QIS/HEU, which holds operational planning control over many of the country’s quality projects (Bangladesh-02-National partner).
Similarly, another respondent noted:
… But you see USAID is everywhere, in almost all these regions of the country. We have Mbale, Lira, Gulu, South West Mbarara has like 16 districts, and in fact UNFPA is doing some good work in the Karamoja region with around 9 districts. Then UNICEF has supported 3 JPHIEGO about 13 districts in the west Nile region and we have USAID in Lira has like 5 districts, Gulu has like 7 districts. But this is something we are telling the Ministry of Health, can they now start to have a map for these activities so that we can tell where these activities are happening? (Uganda-03-National partner).
This perception was however transformed progressively, though not entirely, through meetings, workshops, and training organised by implementing partners, and when health workers at facilities started to comprehend the benefits. Some participants perceived the network vision, objectives and standards as being more of the higher up agenda which rarely takes into consideration all local resource environments and realities as noted here:
…. The standards are great but the variation is in the resource envelope. ….Uganda’s playground is bushy and there are no resources to clear and remove all. So, sometimes that’s the challenge with global goals. They are high up there and the ground is not flat but we always travel without coming up with excuses to aim at achieving the same. (Uganda-03- National partner)
QCN theory of change: Intervention themes
Integrated quality improvement by participatory teams.
The Network for Improving Quality of Care for Maternal, Newborn and Child Health (QCN) was designed to take an integrated approach to quality improvements by participatory teams by working through learning, action, leadership and accountability (LALA) in the context of the facility management, organisation culture and political commitment. We found variations in the approaches that counties took to implement QCN activities. For instance, Plan, Do, Check and Action (PDCA) and 5S (Sort, Straighten, Shine, Standardise and Sustain). These were used in Bangladesh and Malawi respectively.
“Currently, most of the Quality improvement teams where these WITs [Work Improvement Teams] are attached to, have done the five S (sort, set in order, shine, standardize, sustain the cycle) activities”. (Malawi-01-National partner)
In addition to different implementation approaches adopted by implementing countries, actors working at different levels of the health system reported poor documentation practices; poor quality of data collected; lack of data collection tools; lack of knowledge on how to use them; and lack of a culture of data use for decision making, including parallel reporting systems for QNC related indicators.
"The other thing is the parallel reporting system, which is our biggest challenge. This is not included in DHIS 2. Extracting these 15 common core indicators from the chart is a big challenge…most of all; the parallel reporting system is our biggest challenge" (Ethiopia 04-National partner).
These differences in approaches are suggestive of the fact that leadership, learning, action and accountability assumptions are not similar across implementing countries. We found that these assumptions are not explicitly put in the theory of change. Such approaches were linked to different implementation methodologies of key stakeholders already implementing quality improvement activities in health facilities.
Intervention processes: Structural, financial, social and human capital.
The theory of change for the network presents four pillars of process level changes that are necessary to effect clinical and community level outcomes. We found that countries were at different levels in terms of the status of these pillars. QCN initiated trainings were perceived to have improved the availability of skilled professionals in learning facilities however not all skilled staff provided care, facilities experienced high levels of staff attrition and a demotivated workforce due to high demand for services. One respondent noted
"The second challenge is also the continuous dropping out of trained workers to be replaced by new workers. The new employees are not trained and take additional time to fill the skill gap." (Bangladesh-05 sub-national partner)
One of the assumptions of the network was that knowledge and learning would be shared among network member health professionals, and periodic learning forums were organised for this purpose in order to strengthen the social capital in the provision of quality care. While, in Ethiopia and Malawi, the learning forums were considered very motivating for health workers, it was observed that some staff attending these forums competed over allowances, which affected attendance in some knowledge sharing meetings in Uganda.
We found that lack of physical resources–infrastructure, basic amenities, and equipment—were frequently mentioned that prevented adequate care or reduced patient satisfaction. Providers reported that this problem had tied their hands when exercising skills and knowledge they acquired through various quality of care trainings.
“…Most donors will say that is a responsibility of government and will focus largely on software but if the facility has a small maternity room where there is no privacy, no light, no water and government has no capacity to provide this, how can we even start to talk about experience of care?…” (Uganda 02-Sub national partner)
Clinical and behavioural change outcomes
Clinical and community behaviour change outcomes (yellow boxes in Fig 1) were expected to result from the successful intervention processes. We found limited reports on achievements of results at this level in the causal pathways. Table 3 presents key changes that were reported to be related to QCN activities in the participating countries. In countries where some results at outcomes and impact levels were reported, these relate to initiatives that predated the launch of quality of care network activities.
Quality of care network implementation coverage, timelines and other omissions
Quality of care network activities were set to be implemented in learning sites and by 2019 all four countries had identified initial learning sites. With a total of 255 public hospitals and 5054 private hospitals, Bangladesh scaled up QCN activities to over 298 facilities by early 2022. During the same period with a total of 6937 health facilities, Uganda moved from 18 facilities to 88 facilities while no scale up was reported in Ethiopia which has a total of 353 public hospitals and 3706 public health centres and Malawi which has a total of 1724 facilities. Ethiopia’s plan to scale up was disrupted by the Covid-19 pandemic. We also found differences in the acceleration of network activities and different levels of implementation across implementing countries.
Fig 2 presents key challenges experienced during the implementation of the quality of care network in the four study countries.
These challenges manifested differently in different countries and affected the network activities, learning and accountability. One of the aspects of the network that was affected by these challenges was the speed at which implementation could take place. In the period between 2017 and 2022, Bangladesh developed and popularised its operational plans and standard operating procedures while Ethiopia, Malawi and Uganda developed operational plans more slowly and by 2019 few sub-national and facility level actors were aware of these plans. In Uganda however, non-awareness of QCN plans was due to the fact that the ministry of health mainstreamed QCN activities into maternal newborn and child health and general quality improvement activities and as a result it was challenging for implementers to isolate QCN focused activities.
We found that infrastructure and its contribution to improvement in quality of care for women, newborn and children is missing in the theory of change. We also found that all countries reported challenges relating to structural, social and human capital. Reports on how QCN activities have contributed to improved ability to cover costs of care and improved access to health care were not common. It is not clear whether non-reporting on this pillar was due to a lack of clarity on who might cover costs or a lack of specificity as to whether the focus is on access to general health care or access to quality maternal, newborn and child health care.
Discussion
The aim of this paper was to explore how the quality of care network theory of change–that primarily took the form of a high-level logic model–guided what actually happened over 2018–2022 in order to understand what worked well, and what did not. Studies have proposed a more comprehensive application of theory of change to the micro-and meso-levels of individuals, households and communities, where the results of impact investments matter most [33–35]. We found that the QCN theory of change was not specifically adapted for use at country level and was not used in the same way across implementing countries. While the theory stipulated Leadership, Action, Learning and Accountability as the the principles to guide network activity implementation, other principles and quality improvement methods, such as the 5 S and PDCA, have also been used. In addition to difference in implementation approaches we also found differences in reporting systems. While the majority of the countries integrated reporting on Network indicators through DHIS2, one country developed a parallel reporting system. Our findings are suggestive of the fact that while it might be useful to develop an overall theory of change for QCN it would then make more sense to use this as the starting point for country level theory of change development, that involves all relevant local stakeholders, to create a specific and more elaborate theory of change for the country context [13, 36].
We found strong consensus and partner engagement at global and national level. This was weaker among key stakeholders and network implementers at sub-national and facility levels to the extent that some were not aware of the network and its objectives, while others perceived the network as implementing partner activities rather than a ministry of health national initiative championed by national governments. The network and its theory of change were developed following a top-down approach leading to issues of ownership, adoption and accountability at local level. Key partners and sub-national and facility level implementers were not introduced to the network theory of change early enough for them to raise critical questions about their roles and the need for, and nature of, quality of care interventions [37].
It has been argued that many of the assumptions about how the world works are based on implicit theories of change, based on our worldview, developed through our education and upbringing [38]. This means that notwithstanding differences in worldviews, education levels and upbringing, differences in approaches to solving social challenges also prevail. These differences are external and outside the direct control of project management and implementers, but nevertheless should be addressed when developing programme theories of change [39]. We found that key conditions were missing at service integration and process levels in the global theory of change for the network. These missing elements suggest that national level differences were either remote to the developers of the theory of change or these country level differences could not all be used to inform its development. Our findings raise questions about how a high-level, global theory of change should inform, or can inform, national or sub-regional programmes. For example, conditions such as lack of physical resources that are locality dependent were frequently reported to be preventing adequate care, or harm patient satisfaction. Lack of physical resources tied the hands of trained service providers from exercising skills and knowledge they acquired through various quality of care training sessions. We also found that some pillars of the QCN theory of change, such as community empowerment, were rarely being worked on or at least not reported on, suggesting that such pillars were not seen as a priority in the local context.
The main strength of this study is that we were able to use data from multiple sources, over four years. This enabled us to triangulate and validate results from more than one source. The main limitation of our study is that we were not able to investigate all aspects of the theory of change in all countries. Although our analysis drew on a lot of primarily qualitative data (interviews, document review, a survey, observations), we were ultimately limited by the data we had and were not able to explore all assumptions and potential causal pathways. Our investigation was also limited by the original theory of change not explicating assumptions and mechanisms that might have informed our evalution, and by the QCN not having any in-built evaluations at country level that themselves exposed and examined assumptions and mechanisms. This left some gaps in both our understanding, and the QCN team’s understanding, of the programme. For example, in relation to whether, how, and how much, the QCN was able to improve access to care for marginalised groups, improve dignity and satisfaction of patients, and reduce mortality and morbidity in network learning facilities or more broadly.
In conclusion, our paper has provided an assessment of the global theory of change for a network on quality of care for maternal and newborn health. We found different levels of awareness of the network and its objectives, different approaches being used in different countries, absence of key conditions as reported by different implementing countries, and the presence of pillars in the global theory of change that are rarely reported on. Our review of implementation and coordination meeting reports, analysis of learning facilities observation reports, analysis of transcripts from in-depth interviews with health workers, QCN district and national level coordinators as well as international stakeholders indicated non-reporting of activities on certain pillars meant such activities were not being carried out. It appeared that the theory of change was created at the outset of QCN but it is not that clear how much it guided actual activities or any monitoring and evaluation as things progressed. Enabling countries to develop their theory of change, perhaps guided by the global framework, could improve stakeholder engagement, progress to identify and work through assumptions relevant locally (e.g. regarding available infrastructure and data), and better target QCN work toward achieving its goals. Such work should be part of efforts to build learning health systems [39] that are able to create and sustain improved quality of care.
We propose that future efforts be directed toward the development of country-specific theories of change that capture the processes, causal pathways and assumptions in more detail and that those theories of change should be evaluated empirically.
Supporting information
S1 Text. PLOS GLOBAL HEALTH QCN evaluation collection 2-page summary.
https://doi.org/10.1371/journal.pgph.0003532.s001
(DOCX)
Acknowledgments
We thank all respondents and stakeholders for their time and contributions toward making this work possible. The QCN Evaluation Group is: Nehla Djellouli, Kasonde Mwaba, Callie Daniels-Howell, Tim Colbourn (UCL Institute for Global Health, UK), Kohenour Akter, Fatama Khatun, Mithun Sarker, Abdul Kuddus, Kishwar Azad (BADAS-PCP Bangladesh), Kondwani Mwandira, Albert Dube, Gladson Monjeza, Rachel Magaleta, Zabvuta Moffolo, Charles Makwenda (Parent and Child Health Initiative, Malawi), Mary Kinney, Fidele Mukinda (independent researchers, South Africa), Mike English (Oxford University), Yusra Shawar, Will Payne, Jeremy Shiffman (Johns Hopkins University, USA), Kathy Lubowa, Agnes Kyamulabi, Hilda Namakula, Gloria Seruwagi (Makerere University, Uganda), Anene Tesfa, Asebe Amenu, Theodros Getachew, Geremew Gonfa (Ethiopia Public Health Institute, Ethiopia), Seble Abreham, Tanya Marchant (LSHTM), UK
References
- 1. World Health Organization. Trends in maternal mortality 2000 to 2017: estimates by WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division. 2019.
- 2. Mallick L, Yourkavitch J, Allen C. Trends, determinants, and newborn mortality related to thermal care and umbilical cord care practices in South Asia. BMC pediatrics. 2019;19(1):1–16.
- 3. Saaka M, Ali F, Vuu F. Prevalence and determinants of essential newborn care practices in the Lawra District of Ghana. BMC Pediatrics. 2018;18(1):173. pmid:29793543
- 4. Khan SM, Kim ET, Singh K, Amouzou A, Carvajal-Aguirre L. Thermal care of newborns: drying and bathing practices in Malawi and Bangladesh. J Glob Health. 2018;8(1):010901. Epub 2018/06/05. pmid:29862028; PubMed Central PMCID: PMC5943716 form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare no competing interests.
- 5. Chou D, Daelmans B, Jolivet RR, Kinney M, Say L. Ending preventable maternal and newborn mortality and stillbirths. Bmj. 2015;351. pmid:26371222
- 6. World Health Organization. Ending preventable maternal mortality (EPMM): A renewed focus for improving maternal and newborn health and well-being. 2021.
- 7.
World Health Organization. The Network For Improving Quality of Care for Maternal, Newborn, and Child Health. Evolution, Implementation and Progress. 2017–2020 Report. Geneva: WHO, 2021.
- 8.
Sheaff R, Schofield J. Inter-Organizational Networks in Health Care: Program Networks, Care Networks, and Integrated Care. In: Ferlie E, Montgomery K, Pedersen AR, editors. The Oxford Handbook of Health Care Management. Oxford, UK: Oxford University Press; 2016.
- 9. Britto MT, Fuller SC, Kaplan HC, Kotagal U, Lannon C, Margolis PA, et al. Using a network organisational architecture to support the development of Learning Healthcare Systems. BMJ Quality & Safety. 2018:Epub ahead of print: [5th Feb 2018]. pmid:29438072
- 10. Carmone AE, Kalaris K, Leydon N, Sirivansanti N, Smith JM, Storey A, et al. Developing a Common Understanding of Networks of Care through a Scoping Study. Health Systems & Reform. 2020;6(2):e1810921. pmid:33021881
- 11. Kalaris K, English M, Wong G. Developing an understanding of networks with a focus on LMIC health systems: How and why clinical and programmatic networks form and function to be able to change practices: A realist review. SSM—Health Systems. 2023;1:100001. pmid:38144421
- 12. Lynch EA, Mudge A, Knowles S, Kitson AL, Hunter SC, Harvey G. “There is nothing so practical as a good theory”: a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Services Research. 2018;18(1):857. pmid:30428882
- 13. De Buck E, Hannes K, Cargo M, Van Remoortel H, Vande Veegaete A, Mosler H-J, et al. Engagement of stakeholders in the development of a Theory of Change for handwashing and sanitation behaviour change. International journal of environmental health research. 2018;28(1):8–22. pmid:29260884
- 14. Weitzman BC, Silver D, Dillman K-N. Integrating a comparison group design into a theory of change evaluation: The case of the Urban Health Initiative. American Journal of Evaluation. 2002;23(4):371–85.
- 15. Hernandez M, Hodges S. Applying a theory of change approach to interagency planning in child mental health. American journal of community psychology. 2006;38:165–73. pmid:17001524
- 16. Mookherji S, LaFond A. Strategies to maximize generalization from multiple case studies: Lessons from the Africa Routine Immunization System Essentials (ARISE) project. Evaluation. 2013;19(3):284–303.
- 17. Clarke DJ, Godfrey M, Hawkins R, Sadler E, Harding G, Forster A, et al. Implementing a training intervention to support caregivers after stroke: a process evaluation examining the initiation and embedding of programme change. Implementation Science. 2013;8(1):1–15. pmid:23972027
- 18. Breuer E, Lee L, De Silva M, Lund C. Using theory of change to design and evaluate public health interventions: a systematic review. Implementation Science. 2016;11(1):63. pmid:27153985
- 19. Bourey C, Williams W, Bernstein EE, Stephenson R. Systematic review of structural interventions for intimate partner violence in low- and middle-income countries: organizing evidence for prevention. BMC Public Health. 2015;15(1):1165. pmid:26597715
- 20. Coryn CL, Noakes LA, Westine CD, Schröter DC. A systematic review of theory-driven evaluation practice from 1990 to 2009. American journal of Evaluation. 2011;32(2):199–226.
- 21.
The Network for Improving Quality of Care for Maternal Newborn and Child Health. Quality, equity, dignity: the network to improve quality of care for maternal, newborn and child health–strategic objectives (https://apps.who.int/iris/bitstream/handle/10665/272612/9789241513951-eng.pdf accessed 20 Mar 2024). Geneva: World Health Organization, 2018.
- 22. Segal L, Sara Opie R, Dalziel K. Theory! The missing link in understanding the performance of neonate/infant home‐visiting programs to prevent child maltreatment: A systematic review. The Milbank Quarterly. 2012;90(1):47–106.
- 23. Lemma S, Daniels-Howell C, Tufa AA, Sarker M, Akter K, Nakidde C, et al. Opportunities to sustain a multi-country quality of care network: Lessons on the actions of four countries Bangladesh, Ethiopia, Malawi, and Uganda. PLOS Global Public Health. 2023;3(9):e0001672. pmid:37698985
- 24. Shawar Y, Djellouli N, Shiffman J, Akter K, Kinney MV, Mwaba K, et al. Factors Shaping Network Emergence: A Cross-Country Comparison of Quality of Care Networks in Bangladesh, Ethiopia, Malawi & Uganda. PLOS Global Public Health. 2024;(in press).
- 25. Akter K, Shawar Y, Tesfa A, Daniels-Howell C, Seruwagi G, Kyambulani A, et al. Influences on policy-formulation, decision-making, organisation and management for maternal, newborn and child health in Bangladesh, Ethiopia, Malawi and Uganda: the roles and legitimacy of a multi-country network PLOS Global Public Health. 2023;3(11):e0001742. pmid:37988328
- 26. Djellouli N, Shawar YR, Mwaba K, Akter K, Seruwagi G, Tufa AA, et al. Effectiveness of a multi-country implementation-focused network on quality of care: delivery of interventions and processes for improved maternal, newborn and child health outcomes PLOS Global Public Health. 2024;4(3):e0001751. pmid:38437217
- 27. Mukinda FK, Djellouli N, Akter K, Sarker M, Tufa AA, Mwandira K, et al. Individual interactions in a multi-country implementation-focused quality of care network for maternal, newborn and child health: A social network analysis. PLOS Global Public Health. 2023;3(9):e0001769. pmid:37733733
- 28. Tesfa A, Nakidde C, Akter K, Khatun F, Mwandira K, Lemma S, et al. Individual, organizational and system circumstances, and the functioning of a multi-country implementation-focused network for maternal, newborn and child health: Bangladesh, Ethiopia, Malawi, and Uganda. PLOS Glob Public Health. 2023;3(7):e0002115. Epub 2023/07/10. pmid:37428713; PubMed Central PMCID: PMC10332621.
- 29. Mwandira K, Lemma S, Akter K, Tufa AA, Kyambulani A, Nakkide C, et al. Do Networks facilitate innovation, learning and sharing? An evaluation of the Quality-of-Care Network (QCN) in Bangladesh, Ethiopia, Malawi, and Uganda. medRxiv. 2023;https://www.medrxiv.org/content/10.1101/2023.12.05.23299487v1.
- 30. Seruwagi G, Mwaba K, Tesfa A, Dube A, Mwandira K, Akter K, et al. How to evaluate a multi-country implementation-focused network: lessons from the Quality of Care Network (QCN) evaluation PLOS Global Public Health. 2024;in press.
- 31. Mayne J. Theory of change analysis: Building robust theories of change. Canadian Journal of Program Evaluation. 2017;32(2).
- 32. Daruwalla N, Jaswal S, Fernandes P, Pinto P, Hate K, Ambavkar G, et al. A theory of change for community interventions to prevent domestic violence against women and girls in Mumbai, India. Wellcome open research. 2019;4. pmid:31489380
- 33. Jackson ET. Interrogating the theory of change: evaluating impact investing where it matters most. Journal of Sustainable Finance & Investment. 2013;3(2):95–110.
- 34. Connell JP, Kubisch AC. Applying a theory of change approach to the evaluation of comprehensive community initiatives: progress, prospects, and problems. New approaches to evaluating community initiatives. 1998;2(15–44):1–16.
- 35.
Rogers P. Theory of change: methodological briefs-impact evaluation No. 2. 2014.
- 36. Hudson JL, Moon Z, Hughes LD, Moss-Morris R. Engagement of stakeholders in the design, evaluation, and implementation of complex interventions. The handbook of behavior change. 2020:349–60.
- 37. Valters C. Theories of change in international development: Communication, learning, or accountability. JSRP Paper. 2014;17:1–29.
- 38. Stein D, Valters C. Understanding theory of change in international development. 2012.
- 39. Archibald T, Sharrock G, Buckley J, Cook N. Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development. Evaluation and Program Planning. 2016;59:119–27. pmid:27324286