Realtime assessment of WASH coordination in three humanitarian emergencies

To understand how water, sanitation, and hygiene (WASH) coordination leads to humanitarian response outcomes, we conducted a nine-month mixed-method evaluation in three humanitarian contexts (Cox’s Bazar Bangladesh, Democratic Republic of Congo, and Yemen) varying in terms of humanitarian and WASH response and coordination. We completed 93 key informant interviews, 157 online surveys, and monthly activity reports with National Coordination Platform (NCP) staff and partners. We identified a key set of NCP tools, activities, and products that assisted partners in consistently reaching humanitarian WASH outcomes of “making strategic decisions” and “identifying/reducing gaps in the response”, and partially reaching “defining metrics for Monitoring and Evaluation”, “complet-ing Monitoring and Evaluation and evaluating quality”, and “obtaining funding”. Key contextual factors enabling outcomes were stable staff; cooperation with partners; context-specific tools; and, incorporating the humanitarian-development nexus. One activity, three outcomes (“building capacity”, “strengthening relationships”, and “long-term planning”), and two enabling conditions were added into a pre-existing Theory of Change. This revised Theory of Change is being used to improve NCP strategies, including the Minimum Requirements, and ultimately, benefit affected populations.


Introduction
A key component of effective humanitarian response is coordination, which "establishes a coherent and complementary approach that identifies ways to work together for better collective results" [1]. To coordinate relevant stakeholders, the Cluster approach was developed by the Inter-Agency Standing Committee (IASC) in 2005 [2] for key sectors, including water, sanitation, and hygiene (WASH).
At the global level, the Cluster approach aims to strengthen system-wide preparedness and technical capacity to respond to humanitarian emergencies by designating Cluster leads and ensuring predictable leadership and accountability [3]. The global Clusters have three key areas of responsibility: standards and policy-setting; building response capacity; and, operational support. UNICEF is the designated Cluster Lead Agency (CLA) for WASH, and the Global WASH Cluster (GWC) is comprised of partner agencies that enable the core coordination function of supporting "the effective and predictable delivery of appropriate WASH for those affected by an emergency" [4]. At the humanitarian context level, the Cluster approach aims to ensure a coherent and effective response by completing the 6+1 core functions of: supporting service delivery; informing strategic decision-making; planning and implementing cluster strategies; monitoring and evaluating performance; building national capacity; and, supporting robust advocacy; all while remaining accountable to affected populations (AAP) (IASC, 2015). National Coordination Platforms (NCPs) (termed "WASH Sector" or "WASH Cluster" when the Cluster approach is formally activated) aim to coordinate activities and organizations to ensure adequate WASH coverage, reduce potential for overlap, ensure capacity and quality, and meet response needs [5].
NCPs operate as a partnership of national and international response organizations, host government(s), donors, relevant stakeholders, and other humanitarian coordination platforms. Partners commit to participating in Cluster mechanisms, making data available for response monitoring, and participating in improving Accountability to Affected Populations [6]. While each NCP adapts to the local context, an NCP is usually facilitated with two positions, mostly supported and accountable to UNICEF, including: a Cluster Coordinator (CC) who facilitates improved coordination and equal partnership between actors involved in responding to WASH sector needs; and, an Information Manager (IM) who generates and shares information to assist coordination efforts and reporting [7]. Other positions (at national-or sub-national levels) may be added to assist with specific geographic scope, workload, and/or local partnerships. Additionally, the Strategic Advisory Group (SAG) and Technical Working Groups (TWiGs), made of smaller groups of key actors, are established to formulate strategic orientations and support decision making, and guide technical norms and practices, respectively [6]. Please note NCPs are intended as temporary mechanisms, with coordination expected to transition to local authorities and stakeholders.
As there was a need to document results from the Cluster approach, previous research summarized the outcomes and impacts of WASH coordination in humanitarian response by collating information from the literature, UNICEF documents, and key informant interviews [8]. Overall, consistent themes were identified, including: the Cluster approach evolved into a costeffective 'best-fit' model; NCP staffs require technical and coordination skills; context is related to Cluster performance; and, there are trade-offs of Cluster participation for partners. This research culminated with the development of a Theory of Change of activities (inputs), outputs, outcomes, and impacts of humanitarian WASH coordination (Fig 1). However, previous research was largely internal to United Nations respondents and did not take into account all WASH coordination partners' views. Therefore, it was recommended to consider the perspective of direct stakeholders of WASH Coordination-the NCP partners themselves-to validate the Theory of Change and ensure the full range of Cluster outcomes and impacts are included.
The goal of this research is to strengthen the previously developed Theory of Change [8] that will be used to improve global NCP strategies and, ultimately, intend to benefit affected populations. More specifically, this study aims to add NCP staff and partners feedbacks into the Theory of Change by gathering information from different humanitarian and WASH coordination contexts on the activities (inputs), outputs, outcomes, and impacts of humanitarian WASH coordination.

Methods
To complete the research, three different contexts were selected, and a mixed-methods protocol was conducted over a 9-month period, including: 1) key informant interviews with NCP staff and collection of a monthly activity report; 2) key informant interviews with partners incontext; and, 3) online surveys with partners conducted globally.

Context selection
In collaboration with the GWC, three contexts were selected, including Cox's Bazar in Bangladesh (Cox's Bazar), Democratic Republic of the Congo (DRC), and Yemen (Table 1). These contexts varied in terms of: 1) humanitarian situation and WASH response, with Yemen and DRC having the largest number of people in need and funding needs, and being (or having been) designated as Level 3 emergencies requiring system-wide mobilization; and, 2) coordination landscape, including activation at the Cox's Bazar refugee response level in Bangladesh and national activation in DRC and Yemen; a relatively higher number of NCP staff positions in Cox's Bazar and Yemen; and a higher total number of partners involved in the DRC and Yemen Clusters (Table 1).

Ethics statement
Oral (for key informant interviews) and written (for online surveys) formal consent was obtained from each respondent willing to participate in the study. The work was approved by the Social, Behavioral, and Educational Research Institutional Review Board (SBER IRB) at Tufts University (#1910029); local ethics approval was not required as all human subject interactions were conducted remotely. Additional information regarding the ethical, cultural, and scientific considerations specific to inclusivity in global research is included in the Supporting Information (S1 Checklist).

Key informant interviews with NCP staff and activity reporting
To collect information from NCP staff, a 35-question key informant interview guide was developed and administered to explore the: humanitarian and WASH response context; NCP staff responsibility and relationship with relevant stakeholders; NCP tools, activities, and products; associated outcomes; support received; successes and areas for improvement; evolution and greatest impact of coordination; and, suggestions for enhancing NCP effectiveness. Additionally, an activity report template was developed to collect information on NCP functionality with detailed tracking of NCP staff availability, meetings, information sharing, Strategic Advisory Group and Technical Working Groups activity, and sub-national coordination. Interviews were conducted with available NCP staff three times over nine months, and activity reporting was completed monthly. Interviews were conducted in English or French by Zoom (San Jose, California, US) or Skype (Palo Alto, California, US), and recorded.

Key informant interviews with partners
To collect information from partners, a 24-question interview guide was developed and administered to explore the: humanitarian and WASH response context; NCP tools, activities, and products; associated outcomes; barriers to coordination; successes and areas for improvement; meeting expectations, evolution and greatest impact of the coordination; and, suggestions for enhancing NCP effectiveness. To select informants, the NCP list was provided to Tufts, and each contact was categorized by role by the NCP and Tufts. To interview a representative sample of the NCP, informants were randomly selected with a population proportionate to size methodology, aiming for: 50% of informants from international nongovernmental organizations (INGOs), and 10% each from national NGOs (NNGOs), host governments, donors, Strategic Advisory Group members, and non-WASH stakeholders. The goal was to interview 10-20 informants at each time point in each context (total minimum 90, maximum 180). Each selected potential informant was contacted up to two times to schedule an interview. For the second and third data collection timepoints, informants previously interviewed were removed from the selection process. Interviews were conducted in English or French by WebEx (San Jose, California, US), Zoom (San Jose, California, US), Skype (Palo Alto, California, US), or phone, and recorded. Informants could also respond in writing (e.g., email) upon request.

Online survey
A survey was developed in Qualtrics (Provo, Utah, US) consisting of six questions to collect a wider range of respondents and explore: NCP tools, activities, and products; successes and areas for improvement; and suggestions for enhancing NCP effectiveness. The survey link was emailed to the NCP contact, GWC mailing, and USAID emergency lists, and anyone who selfdefined as involved in the response was eligible. Survey links were emailed three times over the 9-month study, in conjunction with each key informant interview round, in English and locally appropriate languages (French, Arabic, and Bengali).

Data analysis
Interviews were transcribed using Temi software (San Francisco, California, US) or by hand. Online surveys were downloaded and translated if needed. Interviews and online surveys were uploaded to NVivo (Burlington, Massachusetts, US) for coding. The monthly activity report data were encoded and analyzed in Microsoft Excel 2016 (Redmond, VA, US). Data were triangulated across the three time points and contexts to assess whether NCP tools, activities, and products enable response outcomes. A list of NCP tools, activities, and products discussed by informants and respondents across the study were collated. Results were analyzed by emergent themes, stratified by informants' role and contextual similarities/ differences, and discussed herein when mentioned by NCP staff informants or by >10% of partner informants/respondents. As NCP staff were interviewed several times over the study, quantitative analysis was not completed for NCP staff informants. For partner informants/ respondents, thresholds of 25%, 50%, and 75% were selected to facilitate analysis of the described existing NCP tools, activities, and products and coordination outcomes.
This consistent use of the same mixed-method evaluation design across time and contexts was utilized to enable rich context-specific details while generating global data comparable between contexts.

Summary statistics
From February to October 2020, we: 1) conducted 17 key informant interviews with NCP staff and collected monthly activity reports; 2) conducted 76 key informant interviews with partners; and 3) collected 157 online surveys (Table 2). Fewer interviews and surveys were   conducted in Cox's Bazar because only two rounds were conducted due to local logistical considerations, and the Cox's Bazar NCP had completed a similar online survey in February 2020 (results of which were incorporated into this study). Nine Cluster Coordinators, co-Cluster Coordinators, and Information Managers were interviewed across the three contexts, all with experience in coordination. NCP staff turnover was higher in DRC and Cox's Bazar, with three of four informants in Cox's Bazar and six of seven in DRC in their role for <1 year. In Yemen, informants were in their role for 2 and 5 years. Monthly activity reports were filled out by the NCP staff or by the research team with information gathered from individual NCP information sharing platforms.

Respondents Time Spent in Context
The proportion of partner informants from International NGOs and National NGOs was similar across contexts but differed slightly from planned methodology. Informants in government roles were sufficiently represented in DRC, but under-represented in Cox's Bazar and Yemen, while donors were sufficiently represented in Yemen, but under-represented in DRC and Cox's Bazar. Other notable differences across contexts included: 1) a higher proportion of informants who spent more time in DRC and in Cox's Bazar; and 2) a higher proportion of informants who participated at the multi-country level or national level in Yemen.
Online survey respondents were primarily from International NGOs and National NGOs, with donors and government representatives again under-represented. Other differences across contexts included: 1) a higher proportion of respondents who spent more time in DRC and Cox's Bazar, consistent with interviews with partners; and 2) a higher proportion of respondents who participated at the multi-country level or national level, especially in DRC and Cox's Bazar.
These noted differences in participant characteristics are considered during further analysis.
Nine themes emerged in data collection across methods and contexts, including: 1) meeting expectations and what could change if there was no NCP; 2) existing NCP tools, activities, and products; 3) outcomes of WASH coordination; 4) successes of WASH coordination; 5) areas for improvement of WASH coordination; 6) barriers to WASH coordination; 7) evolution of NCPs; 8) emergence of COVID-19; and, 9) recommendations. Each theme is further described below, stratified by informant type (NCP staff or partners) and similarities/differences across contexts. Results from partner informants in key informant interviews and partner respondents in the survey are combined, and all data sources are combined to present recommendations.

Meeting expectations and what could change if there was no NCP
Meeting expectations was discussed with partner informants only. Overall, the number of partner informants who said the NCP met their expectations as a coordinating body ranged, across context, from 59-86%. When asked what would change if there was no NCP, informants discussed similar elements across contexts, including there would be: 1) overlap and uncovered areas; and 2) no harmonization and coordination between organizations. Other elements (including wasted resources and more conflicts) were also discussed by partners in Cox's Bazar and DRC (S1 Table).
[without the NCP] "It will be chaos. We won't be able to reach the most vulnerable groups. We won't be able to identify the priorities. We won't be able to do anything. We would be just implementing activities randomly." -Yemen NNGO

Existing NCP tools, activities, and products
The tools, activities, and products reported by NCP staff during the study timeframe were similar across contexts, including: coordination meetings, context-specific strategic and technical documents, activity reporting form and "Monitoring and Evaluation" (M&E) tools, mapping products, and building capacity (S2 Table). However, each NCP had also adapted to the local context, and there were variations in tool formats, frequency of activities and meetings, and production and dissemination of specific resources and analysis. For example, in Cox's Bazar and Yemen, the NCP used online tools, provided a higher number of mapping products to partners, developed mechanisms to monitor response quality and Accountability to Affected Populations, and undertook a risk assessment and monitoring. In DRC and Yemen, coordination meetings were more decentralized, field visits were organized less frequently, and the NCP was not currently planning a long-term transition plan.
Similarly, data collected from partners showed that >50% of informants and >25% of respondents discussed a consistent set of tools, activities, and products across contexts, including (S2 Table): coordination meetings, provision of technical documents, and collection of activity reporting forms (e.g., 4W, 5W). Additionally, information sharing and the Technical Working Groups were discussed by >25% of informants and respondents. Similar comments across contexts on these tools included the: high value of coordination meetings; frequent "information sharing" through emails or via the information-sharing platform; and the complexity of the activity reporting tools. Additionally, an activity discussed in all three contexts was the importance of bilateral (informal or formal) meetings between NCP staff and partners to provide specific advice or to collect data.
Overall, partner informants mentioned they participated, used, or provided data for all the tools, activities, and products they cited as known across contexts. When they were specifically asked which NCP tools, activities, and products they did not participate in or use, 11-26% of informants mentioned they did not attend all the coordination meetings and 18-62% discussed they did not use all the tools. Reasons were because of: difficulties in attending due to schedule or access; lack of relevance to their work; or, they had pre-existing tools that served the same purpose. Additionally, 2-3 informants per context stated they did not always fill out activity reporting forms because it was burdensome to complete monthly.
Differences were also observed within partners' knowledge and use of the tools across contexts (S2 Table) and included: a higher occurrence of mention of "Monitoring and Evaluation" and mapping products in Cox's Bazar and Yemen; a higher occurrence of mention of needs assessment tools in DRC and Yemen; and, a higher proportion of partners submitted monthly activity reporting data in Cox's Bazar and attended national-level coordination meetings than in DRC and Yemen. Therefore, results suggest that: there is alignment between the tools, activities, and products described as provided by NCP staff and received by partner informants and respondents; there is a core set of tools, activities, and products used by partners; and there are differences in tools, activities, and products, and in knowledge and use of them, across contexts.

Outcomes of WASH coordination
Information was collected on whether NCP staff and partner informants thought NCP tools, activities, and products helped organizations to reach the five specific response outcomes detailed in the Theory of Change (Fig 1), including: "making strategic decisions", "identifying/ reducing gaps in the response", "defining metrics for Monitoring and Evaluation", "completing Monitoring and Evaluation and evaluating quality", and "obtaining funding".
NCP staff informants stated in all contexts that NCP tools, activities, and products helped partners to reach four outcomes (S3 Table): "making strategic decisions", "identifying/reducing gaps in the response", "defining metrics for Monitoring and Evaluation", and "obtaining funding". In Cox's Bazar, NCP staff informants described the remaining outcome ("completing Monitoring and Evaluation and evaluating quality") as generally well done (although described potential for improvement and mentioned some monitoring tools were not well understood or perceived as extra workload by partners). In Yemen, NCP staff mentioned lack of comprehensive and systematized tools to evaluate response quality, and in DRC informants stated there were no practical tools to complete Monitoring and Evaluation and evaluate quality.
"As soon as I share this analysis [. . .], it will help them. So I try to synthesize and make sure that one person spends time reading these analyses, understanding and interpreting them, and sharing them to others." -Cox's Bazar NCP staff Across contexts, >50% of partner informants stated NCP tools, activities, and products helped organizations to reach all outcomes. Other similarities across contexts included that: 1) >75% of partner informants described "making strategic decisions" and "identifying/reducing gaps in the response" outcomes as met; and, 2) <75% of informants described "completing Monitoring and Evaluation and evaluating quality" as met because they perceived the activity reporting form negated qualitative aspects and therefore the opportunity for descriptive feedback, field visits were infrequent, and the NCP did not provide tools to assess intervention quality (especially in Yemen and DRC).
For the two other outcomes, partner informants' perceptions were different across contexts (S3 Table). The outcome "obtaining funding" was met according to <75% of partner informants in Cox's Bazar and DRC, and >75% in Yemen. Informants discussed the NCP did not connect (enough) donors with partners or discussed too restrictive eligibility requirements or biased funding selection. Conversely, the outcome "defining metrics for Monitoring and Evaluation" was perceived to be met by <75% of partner informants in Yemen, and >75% in Cox's Bazar and DRC. Informants in Yemen stated partners developed their own Monitoring and Evaluation metrics based on organization or donor policy, standards, and recommendations.

"We have our own department for M&E. The Cluster doesn't really get involved in project M&E from what I've observed." -Yemen INGO
In DRC, analysis suggested that the perception of outcomes met (or not) varied by partner informant role, as: 1) fewer International NGOs informants felt NCP tools helped "defining metrics for Monitoring and Evaluation" (80% of International NGOs informants; 100% from National NGOs); 2) fewer informants from National NGOs stated the NCP helped "obtaining funding" (66% of National NGOs informants; 79% from International NGOs); and, 3) more informants from National NGOs highlighted the need for new outcomes of "building capacity" and "strengthening relationships" between relevant stakeholders (respectively 67% and 50% of NNGO informants; 10% and 35% from International NGOs).
Lastly, although >50% of partner informants stated NCP tools, activities, and products helped partners to reach outcomes, most attributed this to NCP work in general, without specifying a specific tool, activity, or product. However, across contexts, informants discussed the activity reporting form, need assessment tools, and mapping products as specifically helping organizations in "making strategic decisions" and "identifying/reducing gaps in the response", especially when information flowed through meetings, bilateral communications, emails, and information-sharing platforms. The Humanitarian/Joint Response Plan, the WASH strategic documents, and the technical documents and standards were also mentioned as helping to reach these two outcomes. But in DRC and Yemen, the response-tracking mapping product was perceived as not updated or shared regularly enough, and in Cox's Bazar, two informants described the lack of an agreed-upon monitoring system and differences in decision-making between the NCP and the site management and government, making "making strategic decisions" difficult. "Obtaining funding" was facilitated by NCP templates, and by identifying gaps in the response. In DRC and Yemen, partner informants discussed enablers from the NCP including: most donors required response organizations to be an NCP partner; and, the NCP was involved in pooled fund distribution and/or in close contact with donors. "Defining metrics for Monitoring and Evaluation" was facilitated by technical documents and standards, while "Completing Monitoring and Evaluation and evaluating quality" was facilitated by existing monitoring systems. A few informants discussed the need for unified monitoring systems, especially in Cox's Bazar and DRC.

"The WASH Cluster gives guidance in terms of activity strategies to best respond to the issues, but also in terms of geographical priorities." -DRC INGO "All donors understand that all funds should be coordinated with the Cluster. If the Cluster does not agree, this project should not be implemented." -Yemen NNGO
In addition to the outcomes presented in the original Theory of Change (Fig 1), "building capacity" and "strengthening relationships" between WASH partners, and nuancing "making strategic decisions" to include "long-term planning" were additional outcomes considered critical that emerged across the three contexts by NCP staff and partner informants.

"All the partners have the same goal when they come to the Cluster: building partnerships and see if they can access funding. There is this enthusiasm. There is this desire" -DRC NNGO
The results herein highlighted that NCP staff and partners were overall aligned in their perspectives of the outcomes met (and not met) by WASH coordination, although differences were noted between NCP staff and partners, across contexts, and in different partner roles. Additionally, new outcomes not currently in the Theory of Change were also described (Fig 1).

Successes of WASH coordination
Across contexts, NCP staff informants discussed good communication and information sharing as a main success. Another success, mentioned in Cox's Bazar and Yemen, by NCP staff informants was Information Management products (S4 Table).

"The staff is generally available to talk with the partners [. . .], the door is always open. This availability is appreciated." -DRC NCP staff
Similarly, the main success described by 44-81% of partner informants in all contexts was the NCP frequently sharing and communicating information, which helped partners to obtain real-time information (S4 Table). Additionally, meetings were described as a venue for productive discussions with NCP staff and other stakeholders. Between 37-59% of informants also described the success of the NCP to improve relationships between partners by: reducing inequalities and giving everyone a voice; uniting partners through shared goals; and, serving as an effective mediator between partners and government.

Areas for improvement of WASH coordination
"Building capacity" and "strengthening relationships" with partners and the host government were described by NCP staff informants as the main areas for improvement in the three contexts. Other areas for improvement included the need for better advocacy in Cox's Bazar and DRC, and for Monitoring and Evaluation in DRC and Yemen (S5 Table).
The main area for improvement, described by 11-21% of partner informants and respondents in all contexts, was advocacy. Informants/respondents described there were not sufficiently strong relationships between donors and WASH partners, and that advocacy was not as effective as they wanted. Additionally, "strengthening relationships" was described as an area for improvement by 13-18% of partner informants and respondents (in addition to being a success, as above), as the NCP was described as not successfully navigating differences in partner activities or schedules, maintaining partner participation, and having relationships with other coordination platforms, the government, sub-coordination platforms, and/or development partners. Partner informants also mentioned other areas for improvement in DRC and Yemen, including the need for building capacity, improved Monitoring and Evaluation, and increased frequency of field visits (S5 Table).
"We have a challenge contacting with donors, securing fund to our projects. We still feel that priority still goes to international organizations [. . .] We still need that kind of support." -Yemen NNGO

"The re-motivation of partners, their interest to participate, to share experiences from the field is a work that the lead [CC] and co-lead really need to improve" -DRC Non-WASH
These results highlighted NCP staff overall understood partners' expectations and future work to complete in each context. The analysis also showed the need to emphasize the outcomes of "strengthening relationships" and "building capacity" (and associated activity) not already present in the Theory of Change (Fig 1).

Barriers to the WASH coordination
Barriers discussed by NCP staff informants across contexts included: coordination with host government due to multiple jurisdictions, lack of participation and communications, restrictions, and/or different priorities; lack of funding; NCP staff issues caused by frequent turnover and non-dedicated or unfilled positions; and partners' diversity of different needs and expectations (S6 Table). Other barriers, not consistently discussed in the three contexts, included: the coordination landscape framework and coordination with donors in DRC and Yemen; and, in Cox's Bazar and DRC, the lack of some important stakeholders understanding the mission of the NCP which made WASH coordination more difficult, and the lack of harmonization of methodologies and tools between the NCP and other relevant stakeholders.
"Turnover is killing the Sector. There's no other word for it, it's one of the big weaknesses." -Cox's Bazar NCP staff Similarly, barriers reported by partner informants across the three contexts were the: host government relationship; coordinating landscape framework; and lack of funding for the response and coordination (e.g. to conduct field visits for Monitoring and Evaluation) (S6 Table). Other barriers discussed included: NCP staff turnover in Cox's Bazar and DRC; and, in DRC and Yemen, the partners' diversity and lack of NCP mission understanding, the inaccessibility to reach some affected populations or operational partners, and the insecurity (thus lack of access) of the operating environment.

.], you can still be prevented to implement if you don't get your travel permits on a daily or weekly basis." -Yemen INGO "The WASH Cluster is within UNICEF, but UNICEF is also an important stakeholder in WASH. And sometimes it's confusing." -DRC INGO
Again, similarities in barriers were noted between NCP staff and partner informants and some differences were observed between contexts. While many of these barriers were outside of the control of the individual NCP, some of these barriers severely impacted the NCP's ability to effectively coordinate and support partners. Among them, the barrier of "acknowledging and understanding the NCP mission among all relevant stakeholders at any level" was not previously included in the Theory of Change (Fig 1).

Evolution of the NCP over time
NCP staff informants discussed the NCP has positively evolved (S7 Table), but one informant in DRC highlighted that the NCP performance evolved rapidly (positively or not), depending on the presence of staff, the context, specific humanitarian crisis, and pre-existing coordination mechanisms. This evolution was echoed by partners. More than 64% of informants in the three contexts stated the NCP has positively evolved, reporting there were more tools and better coordination among partners over time.
"Before there wasn't really a very good coordination, it was just meetings. There were decisions that were made without the partners being able to know what was going on. But now, there is a good coordination, a good evolution." -DRC INGO However, 4-29% of partner informants (with a higher proportion of informants in Cox's Bazar and DRC) did not see any evolution of the NCP; they observed the same humanitarian needs, WASH strategies, and coordination meeting design. Additionally, 4-23% of informants (with higher proportion in Cox's Bazar and DRC) discussed how the NCP negatively evolved. In Cox's Bazar, informants discussed how organizations had pulled out from the response altogether, and in DRC, they felt there was less funding for WASH activities and coordination, and that partners participated less in meetings and information sharing.
"Certain organizations have pulled out of the response for various reasons, funding, those kinds of stuff."-Cox's Bazar NNGO

Emergence of COVID-19
This project was planned knowing there would likely be an emergent humanitarian crisis during data collection. SARS-CoV-2 emerged and, at the end of Round 1 of data collection, measures were taken in all three contexts to prevent SARS-CoV-2 transmission (e.g., border closures, curfews, assembly bans). Therefore, the NCP addressed this new emergency and adapted coordination as needed.
According to the NCP staff informants, this emergent crisis (and associated restrictions) created new barriers to coordination (including new impact on travel restrictions, field visits cancellations, working relationships weakened due to the lockdown, extra workload), but also some enablers (virtual meetings allowed more partners to attend, and NCP staff felt some partners were more comfortable communicating through typed messages in Yemen).
Between 18-50% of partner informants and respondents also mentioned the emergence of COVID-19 and discussed additional barriers (decrease of operations and participation and confusion for camps responsibilities in Cox's Bazar, difficulty to attend meetings for partners with poor internet access in DRC, and extra workload and responsibility for incountry staff in Yemen). Enablers identified by partner informants and respondents included: virtual meetings allowed more partners to attend; and partners felt more freedom to communicate candidly away from government buildings where coordination meetings were held in Yemen.

Recommendations
Throughout data collection, recommendations on how to improve NCPs were discussed by NCP staff and partners. Between 58-70% of NCP staff and partner informants and respondents across contexts addressed recommendations to improve or create tools, activities, or products, and 17-25% discussed the need to invest resources in strengthening (sub-)NCP coordination and staffing. Conversely, 2-3 informants per context also highlighted there were too many tools' updates, especially with the arrival of new NCP staff, organizations, or consultants, and recommended to control the frequency rate of these updates.
The main similar recommendations across the three contexts were stratified by category, including to: 1) improve the strategy (e.g. reduce NCP staff turnover, improve coordination between levels, harmonize strategies and tools with other coordination platforms and relevant stakeholders, improve the activity reporting form, provide tools in appropriate languages, and develop long-term planning); 2) enhance participation (enhance partner participation and engagement into the NCP, strengthen relationships with the government when possible, and include relevant stakeholders into WASH coordination); 3) improve building capacity for partners and authorities; and, 4) improve accountability (ensure response quality and accountability, emphasize a strategic focus on gender and inclusion).
In DRC and Yemen, additional recommendations were to: connect partners with donors (and share funding opportunities in DRC); update some technical documents and standards; and conduct more field visits (or support focal points in conducting field visits).

Discussion
To gain a deeper understanding of the views of NCP staff and partners on outcomes enabled by NCP activities, tools, and products in humanitarian response, we conducted evaluations in three contexts (Cox's Bazar in Bangladesh, DRC, and Yemen). From February to October 2020, we conducted 93 key informant interviews, and collected 157 online surveys with NCP staff and partners. Nine themes emerged from the data collection, which can be summarized into three results: 1) partners have a positive perception of NCPs, although there is the need to reinforce coordination, improve strategies, and overcome barriers; 2) NCP staff and partners are overall aligned in perceptions, with contextual factors influencing WASH coordination and perceptions; and, 3) while results are consistent with previous work and validate the previously-developed Theory of Change, new elements should be incorporated into the Theory of Change.
Firstly, and overall, NCPs met partners' expectations and partners considered NCPs to have positively evolved. A set of tools, activities, and products, strikingly consistent across contexts, was key for partners and successfully improved communication, strengthened relationships among partners, and assisted stakeholders to reach the humanitarian WASH outcomes of "making strategic decisions" and "identifying/reducing gaps in the response. Additional response outcomes considered critical for NCP partners were: "long-term planning", "building capacity" (especially for implementing national agencies and government), and "strengthening relationships" between all relevant stakeholders in the WASH response.
Although these outcomes were facilitated by the NCP in the contexts as expected [15,16], partners overall felt that tools, activities, and products to meet these outcomes should be reinforced and improved in the future. Partners also discussed the need to address questions on response quality, accountability to affected populations, and transitioning out of the emergency. Please note the GWC is currently developing Accountability Quality Assurance (AQA) Initiatives to address concerns on response quality. Additionally, barriers were identified across contexts that may hinder NCP's work, including the: lack of funding; complex coordinating landscape framework; and host government relationships. Despite being mostly outside individual NCP control, these were identified as consistent barriers to coordination. NCP staff also mentioned high staff turnover, partner diversity or role, and the NCP mission understanding among all relevant stakeholders as common barriers. The emergence of COVID-19 over the time period of the study allowed for assessment of the adaptation of individual NCP and additional barriers and enablers occurring during this time.
Overall, these results were consistent with previous evaluations of the Cluster approach that have found positive progress of coordination, added value in needs assessment/gap analysis and targeting, and preventing duplication [17][18][19]. Those studies also highlighted similar challenges related to coordination (e.g. rotating staff, coordination landscape, participation) and shortage of activities (accountability, transition/exit strategies) for the WASH and others Clusters [20][21][22]. The novelty of our research is that, to our knowledge, this is the first time our work is inclusive of Cluster partner perspectives.
Please note there is extensive literature on emergency response in high-income contexts documenting a top-down approach to emergency response backed by government-sponsored agencies where "Governments must be ready to direct and control emergency operations" [23]. However, in low-and middle-income countries, Cluster Coordination is a soft-power system dependent on collaboration; thus, these lessons from top-down responses are not comparable. Recent research has found a mix of top-down and bottom-up approaches is ideal for disaster resilience [24].
Secondly, results highlighted that the NCP staff and partners were overall aligned by contexts in: 1) their perspectives of the existing tools, activities, and products; 2) the outcomes met (and not met) by the NCP; and, 3) the main successes and barriers to WASH coordination. However, differences in existing NCP tools, outcomes met, successes and areas for improvement, and barriers were found across the contexts and highlighted the importance of contextual factors in WASH coordination strategies. Differences were seen in whether large and stable NCP staffing was present, the size of the response and needs, differing partner capacity and participation, and inaccessibility and insecurity of some areas. With these expected differences between contexts [8,25], we feel we have captured variations sufficient to validate the general Theory of Change of Coordination across contexts.
Thirdly, these results were consistent with previous work that included a literature review and interviews with NCP-experienced personnel (but not interviews or surveys from partners) [8]. The main similarities from the two datasets included the: 1) importance of the NCP staff; 2) challenges of coordinating with different relevant WASH stakeholders (including subnational coordination, inter-coordination, donors, and governments), with different perspectives, needs, and tools, thus the importance of effective communication between all stakeholders; 3) need to include contextual factors to coordination; and, 4) a sense of 'what is next' for the NCP approach, with the need to address questions on quality of response, accountability to affected populations, and transition out of the emergency. These elements were also discussed in other studies exploring overall Humanitarian Coordination [26][27][28][29]. Additionally, NCP stakeholders corroborated the activities, outputs, and outcomes to reach an 'inclusive, strategic, and effective response that addresses gaps, builds partnerships, and enables a predictable response by NCP partners that respects core humanitarian standards' (Fig 1). Therefore, this work validated the previous work and the Theory of Change for the WASH coordination.
NCP stakeholders in the field, however, provided additional elements to describe and gain a deeper understanding of the overall WASH coordination. This led to the expansion of the Theory of Change that now includes the: 1) activity of "building capacity" (including specific WASH technical aspects, use of specific NCP tools, NCP mission); 2) outcome of "building capacity" to better participate in NCP activities and WASH response; 3) outcome of "strengthening relationships" to improve overall coordination; 4) outcome of "long-term planning" focusing on an exit strategy to hand over coordination and develop the humanitarian-development nexus; 5) enabling condition of "acknowledging and understanding the NCP mission" among all relevant stakeholders at any level; and, 6) enabling condition of "using appropriate and inclusive language(s)" in the coordination. With these additions, we believe the Theory of Change now includes the perspective of national and international organizations, national governments, UNICEF, the GWC, and the larger humanitarian response structure, and better represents the complex relationship and elements needed for an effective humanitarian WASH coordination.
Limitations of this work included: 1) participants were randomly selected from NCP contact lists, thus we did not select only active participants who might have been more aware of NCP activities, tools, and products; 2) this led to a response rate of 98 interviews for 457 emails which might lead to selection bias and a lower representativity of government, donors, and other non-WASH stakeholders; and, 3) the total number of key informant interviews and online surveys collected over the study differed across contexts, thus the weight given to each informant and respondent varied between contexts. Overall, we feel these limitations did not impact the results of this research. Please note that incorporation of affected population response was not assessed herein, and future research on the NCP impact from the perspective of the affected population should be considered [8].
Overall, the project strengthened the previously developed Theory of Change by gathering and incorporating feedback from direct stakeholders of the WASH coordination from different humanitarian contexts and led to a deeper understanding of the overall value of the NCP to the humanitarian response. The results of this work and the associated Theory of Change for WASH coordination will be used to review and improve NCP strategies (including the Minimum Requirements, the practical framework for monitoring the NCP based on the core functions), and ultimately, benefit affected populations. Additionally, this study highlights that the main key factors for achieving successful WASH NCP outcomes in humanitarian contexts were: having stable and dedicated staff with both technical and leadership strengths in the Cluster Coordinator and Information Manager positions; developing and maintaining strong cooperation with partners, sub-coordination platforms, UNICEF, the local government, and other relevant stakeholders at the national and sub-national levels, catalyzed by a high degree of perceived NCP value; using context-specific (understandable on how to use and harmonized) tools consistently to improve program delivery and evaluate quality; and making a continuous effort towards building a resilient and sustainable humanitarian development nexus. Thus, we specifically recommend focusing on: how the Cluster Coordinator and Information Manager positions can become more robust and less dependent on individual excellence and turnover, how partners' participation and NCP mission understanding can be enhanced as they have an active role and responsibilities in the WASH coordination mechanisms; and lastly, how we can build stronger in-country leadership to handover coordination to. We wish that those recommendations, alongside with the Theory of Change expanded in this research, will benefit the WASH sector, and provide new tools and tracks to monitor and improve the humanitarian WASH coordination.
Supporting information S1 Checklist. Inclusivity in global research. (DOCX) S1 like to thank the Global WASH Cluster (GWC) Cluster Advocacy and Support Team (CAST) for their helpful supervision and revisions, and Julia Debertin, Sarhin Jahan, Eddy Wanyonyi, Marine Ricau, and Elizabeth Mitchell for data entry and analysis assistance.