Introduction

Access to sufficient safe water for human consumption and hygiene is crucial for health and dignity. Despite 2.1 billion people gaining access to safely managed water globally since 2000, around two billion people still lack access to safely managed water sources [1]. The rural poor in low- and middle-income countries (LMICs) are among the least likely to have access to safely managed drinking water [2–4]. As a result, promoting sustainable water access for remote and rural populations is key to the achievement of the Sustainable Development Goal (SDG) 6 of “clean water and sanitation for all” [5].

Historically, the response to water access issues has focused on the construction of new infrastructure. This is reflected in the topline goals of programs ranging from the first Water Decade of the 1980s, which sought to provide access to a public tap or protected source within 500 m of the home [6], to the current Jal Jeevan Mission, which aims to provide rural Indian households with a functional piped water connection [7]. In alignment with global goals, Latin America and the Caribbean made consistent progress towards universal infrastructure access both during earlier decades and through the Millennium Development Goals. Regional coverage of piped on-premises service expanded from 46% in 1980 to 73% in 1990 and 89% in 2015 [8,9]. However, coupled with this slowing advancement in piped water access, coverage under SDG 6.1.1’s more demanding definition of “safely managed drinking water” has stagnated, reaching 75% of the region’s population in 2015 and a projected 77% by 2030. [10–13]

One reason for this stagnation has been that Latin American governments face multiple, systematic challenges in ensuring the sustainability and quality of rural water services needed to maintain safely managed drinking water [14]. Rural infrastructure tends to be delivered to communities in discrete ‘projects’ rather than as long-term investments that are accountable to local governments [15]. As a result, once constructed, logistical, political, economic, and environmental barriers often impede the maintenance and repair of these systems. Rural communities often have low population density, limited local financial capacity, and limited geographic access to skilled external support [16]. Challenges to water governance include decentralized and semi-voluntary management models for water services, and ossified and underfunded regulatory and support structures [14, 17, 18]. Climate change compounds these issues, as small water systems are disproportionately constrained by altered rainfall patterns and intensifying extreme weather events such as prolonged droughts and floods [19,20].

An additional challenge to the delivery of safe and affordable drinking water for all is that, while the current SDGs encompass nuanced indicators of water service to account for the complexity of quality water services, the data gathered to support their reporting still relies heavily on the infrastructure-centric framework of prior efforts. Sources are often classified as safely-managed without evaluating the consistency of their service or the quality of water delivered [21,22]. This lack of clear data on how systems are actually functioning has immediate impacts on public policy: as national census-level data lags in the uptake of improved service quality indicators, the prioritization of investment in politically expedient areas or those with unimproved water access may appear more attractive. Some of these data gaps are the result of technical or methodological limitations, as tracking water quality and service uptime is challenging, and complementary indicators of service quality including affordability are often entirely absent from evaluations due to a lack of consistent methodologies [23]. However, for rural communities and the often smaller organizations that support rural water services, these gaps are further compounded by limited financial support for monitoring and evaluation, making it hard to determine when failures in service reach a point where health is adversely impacted, or to identify high-impact sites for additional investment [24].

Calls to better capture indicators of service quality have been accompanied by efforts to more fully capture user experiences. Unreliable water systems push families towards a range of coping behaviors [25], including household water storage and the use of alternative sources [26] that can increase the risk of diarrhea [27] and vector-borne diseases [28]. Beyond these relatively well-studied outcomes, experiences of water insecurity (WI), defined as ‘the inability to access and benefit from affordable, adequate, reliable and safe water for wellbeing and a healthy life’ [29] are a key indicator of water related disruption [30]. WI is associated with a wide range of consequences for human health and wellbeing, including poorer mental health, poorer infant feeding practices, and exacerbation of pre-existing chronic conditions [31–34].

While water insecurity can be evaluated in several ways, the research community has increasing adopted the Household Water Insecurity Experiences (HWISE) Scale - a recently developed, validated measure that captures experience of water inaccess at the household level -to evaluate water insecurity in LMIC settings [35,36]. The HWISE Scale is cross-culturally validated (including in multiple Latin American contexts) [37], complements other traditional indicators of water access, such as the JMP-ladder definitions of drinking water source and measures of water quality. Both a 12-item and an abbreviated 4-item version of the scale are available [36], offering implementers a choice between higher resolution capable of identifying a broader range of WI or a more rapid screening tools [38]. The HWISE scale is an experimental, parsimonious, survey-based indicator designed to function across a range of water access and enumerator conditions. This may provide a path to cut the Gordian knot of high costs, technological dependency, and time limits associated with service quality monitoring, illuminate the variety of experiences currently gathered under “safely managed water” to guide high-impact public policy. While few studies, to date, have evaluated the utility of the HWISE Scale specifically for monitoring and evaluation, a recent report of case studies from Nepal and Sierra Leone demonstrated that the HWISE scales were sensitive to water service interventions, thereby complementing traditional resource-based indicators [39]

Ecuador faces challenges to rural water access similar to those across much of the Latin American region. Although rural access to piped water is still gradually increasing in the country, the percentage of the population with safely-managed water supply remains highly heterogeneous, with 48.5% of rural households having access to safely managed water services compared to 76.9% of households in urban areas [40]. Coverage has decreased slightly over the last decade in some subnational regions as infrastructure fails [40,41]. In many rural areas, such as the northern coastal and Amazon regions, rural populations rely on combinations of unimproved water sources including springs, rivers, streams, rainwater, and private unprotected wells [42]. Services are atomized and inefficient: of the 5000 + community-managed systems in Ecuador, 61% service a population of less than 100 households, and only 6% are able to fully cover operating costs from tariff collection [43]. At the same time, community-managed systems are five times more likely than urban systems to deliver unsafe water to consumer’s taps [44]. While community-managed water systems are culturally ingrained, and perhaps the only feasible strategy for providing rural household access in Ecuador, government arrangements to support these systems are largely absent, and legal barriers prevent communities from directly accessing public or financial-sector funding for system maintenance or improvement [45,46]. Community-led water systems also come with challenges related to obtaining community payments for service and access to skilled expertise, which is often not only costly but located far from communities, creating a barrier to the construction, maintenance, and repair of rural water supplies [17]. In addition, external pressures from extractive industries, such as oil production and mining further threaten water management in rural Ecuador.

Given this background, the goal of this project was to assess the real-world performance of the HWISE scale for monitoring and evaluation of geographically and ethnically diverse rural community-managed waters systems in Ecuador. Ecuador was not included in the original HWISE validation study [35,37], although the HWISE Scale has now been used in other parts of the country, including the Galapagos [47]. Evidence in support of the field usability of the HWISE Scale – which we define here as the extent to which the HWISE Scales can be used ‘out of the box’ by implementers outside of the context of a research study - would suggest usefulness for future policy and programmatic decision-making, and usefulness in monitoring and evaluation efforts in similar contexts globally. Our specific aims were: i) to evaluate the field usability of the HWISE Scale by examining internal consistency when implemented by local non-governmental actors across diverse, rural Ecuadorian communities; ii) to evaluate associations between other indicators of water accessibility, availability, and acceptability and the 4-item HWISE-4 and 12-item HWISE-12 Scales, and iii) to test whether HWISE-4 scores were reduced after the implementation of water supply improvement projects, including one project where the improvement was imperceptible to users.

Methods

Green Empowerment is a US-based non-governmental organization (NGO) that works with a network of local NGOs across Ecuador (and other Latin America countries) to deliver water systems to primarily small, rural communities (Fig 1). Green Empowerment provides technical support to the local partners, while local partners lead intervention implementation and data collection associated with monitoring and evaluation. There is considerable ethnic, linguistic, and geographic diversity in Ecuador. Communities served by Green Empowerment and its partners include Afro-Ecuadorian and Indigenous (Cha’palaa-speaking) communities on the coast, and Mestizo and Indigenous (Kichwa-speaking) communities in the highlands and the Amazon. These communities also face growing challenges due to climate change, which is altering water availability, increasing extreme weather events, and affecting the reliability of local water sources. Local water access conditions also vary considerably. Riverine communities on the coast have constant surface water access tempered by consistent low quality, seasonal flooding, and the drudgery of hauling water. Highland communities depend on piped and often untreated water which benefit from protected watersheds but still suffer quality issues during heavy rains due to increased turbidity, and scarcity during dry periods. Amazonian populations utilize community-specific combinations of rainwater, surface sources, unprotected wells and untreated piped water which are vulnerable to both excessive and insufficient rainfall. While many households report the use of secondary sources, the use of bottled water is uncommon throughout the communities due to both financial and logistical constraints.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Regions of Ecuador Represented in this study.

Communities contributing to this analysis are displayed above. A single Colombian community, ‘Acueducto Espino Sur’, located on the Ecuadorian border, is included as intervention that resulted in a change to the water system’s energy source did not impact service delivery. Base layers for this map (i.e., country and regional boundaries) are made publicly available by the Ecuadorian National Institute of Statistic and the Census (INEC) at: https://www.ecuadorencifras.gob.ec/documentos/web-inec/Geografia_Estadistica/Micrositio_geoportal/index.html.

https://doi.org/10.1371/journal.pwat.0000471.g001

Study design and Data Collection

As part of an effort to standardize a survey instrument for monitoring and evaluation across Green Empowerment partner NGOs and the communities they support, we evaluated WI by implementing the HWISE Scales in two phases:

In Phase 1 (02/03/2023 - 23/04/2024), the HWISE-12 was implemented in 15 Ecuadorian communities representing different regions (coast, highlands, and Amazon), languages, and water issues. This data were used to conduct preliminary checks of internal consistency and to determine whether the HWISE-4 would be an acceptable substitute for the HWISE-12.

In Phase 2 (24/04/2024–31/10/2024), a combination of the HWISE-12 or HWISE-4, depending on community partner preferences related to survey length, was collected in four new Ecuadorian communities for baseline evaluation, and repeated in four Ecuadorian communities for endline evaluation of water system infrastructure construction or repair projects. Details of these interventions are included in Table 1. In addition, we also collected baseline and endline data to evaluate the HWISE Scale in one Colombian community located 6km from the Ecuadorian border, where an improvement to the water system, the addition of PV solar to reduce electricity costs for pumping, was made. This intervention did not produce any change in water quality, availability or the cost of service for users. This community was used to evaluate whether surveyor bias, or the visible presence of an implementing NGO in the community, would influence HWISE scores at endline. In total, therefore, 20 communities (19 Ecuadorian and 1 Colombian) were included in the final analysis, including baseline data only for 15 communities, and paired baseline and endline data for 5 communities (4 Ecuadorian and 1 Colombian).

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Infrastructure and Intervention Details for Five Communities.

https://doi.org/10.1371/journal.pwat.0000471.t001

In each phase, the recall period of the HWISE scales was ‘in the last four weeks’ and response options were, ‘Never’ (0 times), ‘Rarely’ (1–2 times), ‘Sometimes’ (3–10 times), ‘Often’ (11–20 times), Always (more than 20 times) ‘I don’t know’, and ‘Prefer not to answer’. Results were coded as 0, 1, 2, 3, or missing (NA) for ‘never’, ‘rarely’, ‘sometimes’, ‘often/always’, and don’t know/no answer, respectively. HWISE-4 and HWISE-12 scales were calculated by summing the responses to all questions (maximum of 12 for the HWISE-4 and maximum of 36 for the HWISE-12). If the participant responded ‘don’t know’ or ‘prefer not to answer/not applicable’ to any question, the HWISE score was not calculated [38].

Other WASH information collected as part of the standardized survey included: the household’s primary drinking water source, the distance and time to the primary drinking and domestic water sources, water usage (either read at meter and/or estimated by the household respondent); reported use of secondary drinking and domestic water sources, reported household water treatment, and household water storage (directly observed by the surveyor). Among households with a piped water connection, we also asked about the duration of outages, and whether the household felt that outages were predictable or unpredictable.

To complement these household survey data, we also gathered feedback from implementing partner organizations through informal debriefing of 2–3 individuals per organization, via phone or in-person discussion, during which they were questioned about their experiences administering the HWISE Scales, both in terms of the time required to complete the HWISE-12 and the HWISE-4, and the perceived acceptability of the survey to surveyors and to respondents.

As these activities were conducted within the context of routine programmatic monitoring and evaluation efforts, formal sample size calculations were not undertaken. Surveys targeted 100% of households in each community, omitting those not present after repeated visit attempts, except in 4 communities where a smaller proportion of households were randomly selected for surveys based on logistical constraints. The proportion of households successfully surveyed in each community is noted in Table A in S1 Text). Adults who self-identified as knowledgeable about household water practices were accepted as respondents. For all analyses, data were assumed to be missing at random.

Results

Data from N = 412 households were collected in Phase 1 (all baseline data), and data from N = 836 households were collected in Phase 2 (N = 553 at baseline and N = 283 at endline). Of these, HWISE-12 data was available for N = 963 households (N = 880 at baseline). 278 households lacked HWISE-12 data because the community partner preferred to implement the shorter HWISE-4 only, while 7 households provided one or more ‘don’t know’ or ‘not applicable’ responses and therefore lacked an HWISE-12 score. Descriptive characteristics of each community are presented in Table 2. Communities ranged in size from 14-210 households (Table A in S1 Text). Most coastal households reported rainwater (32.5%) or surface water (50.6%) as their primary drinking water source, while most Amazonian households relied on well water (31.3%) or piped water (41.3%). In contrast, most highland households reported piped water as their primary source (99.2%), although these systems were frequently unreliable, with occasional, unpredictable service interruptions year-round, high turbidity peaks during the rainy season, and water quantity limitations during the dry season (Table 2).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Household and respondent characteristics at baseline.

https://doi.org/10.1371/journal.pwat.0000471.t002

Internal Consistency, Floor and Ceiling effects were calculated for baseline data only. Among N = 880 households with complete HWISE-12 data at baseline, 330 were administered in Spanish (6 communities), 304 were administered in a combination of Cha’palaa and Spanish (9 communities), and 246 were administered in Kichwa with some Spanish (4 communities). Floor effects are reported in Table B in S1 Text: by implementation partner, between 11.1%% and 52% of households had the lowest possible HWISE-4 score, and between 9.2-19.7% had the lowest possible HWISE-12 scores. Very few households reported the highest possible score for either scale. The overall internal consistency of the HWISE-12 was high (Cronbach’s α = 0.92), suggesting good internal reliability relative to a commonly used cut-off of 0.90 [50]. When stratifying on the language of administration, α = 0.88 when the survey was administered in Spanish, α = 0.83 when administered in Spanish/Cha’palaa, and α = 0.95 when administered in Spanish/Kichwa. The internal consistency of the HWISE-4 was somewhat lower (Cronbach’s α = 0.79 overall and 0.83, 0.70, 0.87 for Spanish-speaking, Cha’palaa speaking, and Kichwa speaking communities, respectively). At endline, HWISE-4 results were somewhat lower (Cronbach’s α = 0.69 overall, α = 0.62 for Spanish-speaking communities only, and α = 0.74 for Kichwa-speaking communities). Every household in the single Cha’palaa speaking community surveyed at endline (post-intervention) reported ‘none’ to all survey items. This will be discussed further below.

When comparing water insecurity by the HWISE-12 Scale cutoff of ≥12 versus the HWISE-4 Scale cutoff of ≥4, there was substantial agreement overall (84.8% agreement, Cohen’s Kappa of 0.69, 53.1% classified as water insecure by the HWISE-12 versus 62.8% by the HWISE-4) Agreement was strongest in Spanish and Kichwa speaking households (96.7% agreement, Cohen’s Kappa of 0.88 and 95.9% agreement, 0.92 respectively, with 72.6% and 67.2% classified as water insecure by the HWISE-12 and the HWISE-4, respectively), but much lower in Cha’palaa speaking households (62.8% agreement, Kappa = 0.29), with a lower proportion of households classified as water-insecure by the HWISE-12 (16.1%) versus the HWISE-4 (53.3%) providing ‘fair’ agreement according to commonly used cutoff criteria for Cohen’s Kappa [51]. There was no evidence that inconsistency was driven any specific item, as item-level response distributions suggested that multiple items included in the HWISE-12 but not the HWISE-4 had relatively lower positive response rates (Table C in S1 Text).

Internal consistency checks were also performed based on the community partner that conducted the survey, which also corresponds to the geographic region where the partner operates, linguistic diversity, and enumerators (each organization deployed at most three enumerators). (Table B in S1 Text).

Feedback from partners implementing the HWISE Scales suggested that, although the longer HWISE-12 was quick to administer and generally well understood, some HWISE items, particularly those related to anger or shame, were uncomfortable to administer across communities. Anecdotally, one HWISE-12 item, ‘In the last 4 weeks, how frequently have you or anyone in your household had to change schedules or plans due to problems with your water situation?’, was especially confusing or problematic for Kichwa-speaking residents when enumerated in Spanish, and may require further analysis of face validity for these populations.

Phase 2. Baseline data: comparison across communities & associations between the HWISE Scales and WASH behaviors/perceptions/coping strategies. We observed a wide range of HWISE-4 scores, both within and between communities. While the lowest HWISE-4 scores were reported in communities with piped water access (Gualchan, Bellavista, and Acueducto Espino Sur), we also observed communities with piped water that had a high prevalence of WI compared to those communities reliant on rainwater or surface water (Inchillaki and Tierras Orientales) (Fig 2). The distribution (variability) in HWISE-4 scores within individual communities was also very high. Water source was not significantly associated with HWISE-4 scores, but the use of rainwater or surface water as a drinking source was associated with higher odds of WI when measured using the HWISE-12 (OR:2.24, 95%CI 1.01, 4.98, p-value = 0.047 and OR:2.02, 95%CI 1.01, 4.98, p-value = 0.047, respectively) (Table 3). The need to carry drinking water versus access to a source within or near the home (OR:5.02, 95%CI 2.65, 9.53, p-value<0.001); the consumption of water that was perceived to be unsafe (OR:2.28, 95%CI 1.69, 3.07, p-value<0.001), and lower volumes of water stored in the household and observed at the time of the interview (OR:2.09, 95%CI 1.08, 4.03, p-value = 0.028 comparing households with 5-50L per capita stored versus>50L per capita stored), were also associated with a greater odds of WI. Among households with an on-premises piped water connection (regardless of whether that connection was functioning at the time of the assessment or not), greater intermittency (OR:3.94, 95%CI 1.98, 7.82, p-value<0.001) and less predictability in water outages (OR:1.77, 95%CI 1.01, 3.12 p-value = 0.046) were associated with greater water insecurity (Table 4 and Table D in S1 Text). Among households with a piped water connection, 46.5% were water insecure according to the HWISE-4 cut-off, while only 5 (2.2%) of N = 223 households with a piped water connection and no reported outages were classified as water insecure. Almost all (N = 222 of 223) households that reported no intermittency were from post-intervention community assessments.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Associations between primary drinking water source and water insecurity.

https://doi.org/10.1371/journal.pwat.0000471.t003

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Associations between characteristics of piped water systems and reported water insecurity based on binary HWISE-4 and ordinal HWISE-12 categories.

https://doi.org/10.1371/journal.pwat.0000471.t004

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. HWISE-4 results by community.

Boxplots demonstrating the median and interquartile range of HWISE-4 scores reported by each community. Reported water sources by household are underlaid. Communities with highly unreliable piped water (e.g., El Paraiso, Tierras Orientales, and Inchillaki) reported mean HWISE-4 scores similar to communities reliant on surface water (e.g., Estero Vicente, Calle Larga).

https://doi.org/10.1371/journal.pwat.0000471.g002

Comparison of HWISE-4 in pre-/post-intervention communities: Mean HWISE-4 scores were lower in all four Ecuadorian communities post-intervention. 3.73 versus 0.43 for Ardilla Urku (t = 5.63, p < 0.001); 3.75 versus 0.89 for Inchillaki (t = 5.65 p < 0.001); 3.18 versus 0.37 for Maldonado (t = 6.31, p < 0.001); and 1.63 versus 0.00 for Pichiyaku Grande (t = 9.34, p < 0.001) (Fig 3A–3D). HWISE-4 scores were also statistically significantly lower in the Colombian community of Acueducto Espino Sur (1.16 versus 0.34, t = 3.75, p-value: 0.0003) (Fig 3E). When categorized as water secure versus insecure, this corresponded to a change in prevalence of 11.7% of households categorized as water insecure at baseline to 0.0% at endline, which was also statistically significant (t = 2.70, p = 0.0078). When evaluated by item, differences in HWISE-4 scores between baseline and endline in Acueducto Espino Sur were most strongly driven by the ‘worry’ item (Fig 3 and Table E in S1 Text).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Spider plots showing changes in mean HWISE-4 responses from pre- to post-intervention in 5 communities.

Mean responses to each item making up the HWISE-4 for each community pre- (blue) and post- (orange) intervention.

https://doi.org/10.1371/journal.pwat.0000471.g003

Discussion

We found the HWISE Scales to be a useful addition to monitoring and evaluation data collection efforts for small, rural Ecuadorian communities. Our results suggest a burden of water insecurity comparable, or in some cases higher, than what has previously been reported in Latin American settings [48]. Estimates of none-to-marginal, low, medium and high water insecurity among Highland communities was similar to what has been reported in urban Mexico [52], while water-rich coastal and Amazonian communities reported higher rates of water insecurity than previously noted in other water-rich but still water-insecure settings [53].

The small number of households in these communities implies that efforts to monitor the impact of water system improvements are often underpowered to demonstrate significant improvements in household health and well-being outcomes. This obstacle can deter government and donor investment in such projects [24]. We found that water system interventions were associated with statistically significant reductions in reported water insecurity. Although we cannot conclusively demonstrate that the interventions were the cause of these reductions, the ability to observe changes within communities of small population size is in and of itself encouraging as it suggests that measurable improvements in experiences of water access could be monitored by the community (several enumerators were local health promoters), local partner organizations, or conceivably local governments. The scales were also readily implemented, with minimal training, by field staff of local NGOs, in multiple languages, with high internal consistency.

We also identified some challenges with the HWISE Scales. Although partners generally found HWISE-12 quick to administer and generally well understood, they also reported that some HWISE items were uncomfortable to administer, particularly for monolingual Spanish-language enumerators working in bilingual Kichwa/Spanish-speaking communities. While these enumerators reported good rapport with respondents, and mutual understanding of other survey items, the question regarding water-induced changes to plans proved difficult to understand even following multiple prompts. Prior Spanish-language piloting of the surveys with key informants and formal tests of internal consistency among Kichwa communities did not highlight issues with this question, but this may be due to respondents selecting responses in alignment with those of the prior HWISE questions. Lower internal consistency, and poorer agreement between the HWISE-4 and the HWISE-12, was observed in Cha’palaa-speaking communities. Among the 5 communities with post-intervention data, notably, the single Cha’palaa-speaking community reported ‘0’ to every HWISE-4 question (perfect scores of 0 across the entire community) which may suggest that the water system was highly successful, or potentially that community members felt motivated to provide positive feedback to the organization that had implemented the system improvement. A Spanish-speaking community surveyed by the same organization did not show the same trend.

A similar trend was noted in the single Colombian community where an intervention, which consisted of a modification to the water system that did not affect water access, quality, reliability, costs, or other elements of water service perceivable to the community, nevertheless resulted in improved HWISE-4 scores. It is possible the presence of the implementing NGO in the community, which was visible in organizing training workshops and other activities in the community, may have introduced bias when NGO employees conducted endline surveys and led to these changes in HWISE scores. However, unlike other communities where positive responses to all four HWISE-4 questions decreased following intervention, in this community, the only statistically significant decrease was in the ‘worry’ item. Therefore, it is possible that this item may reflect worries about the water system per se (such as concerns or frustrations over the administration and management of this system), rather than worries related to water access directly. The ‘Colombian community also had relatively good water access at baseline, so reported changes in HWISE-4 scores were small, although still statistically significant. Most households fell below the threshold for water insecurity at both baseline and endline (11.7% at baseline versus 0.0% at endline, suggesting improvements in perceived security).

We also found that communities with intermittent piped water access, especially when unpredictable and/or unreliable [54], reported mean HWISE scores similar to those of communities reliant on surface water. Interruptions to service that were predictable and of limited duration were protective against reported water insecurity. This aligns with recent studies which reported that frequent interruptions and unpredictability are associated with greater disruptions to consumers, as well as greater emotional distress and stressful behaviors and higher HWISE scores [34,52,54] However, we were limited in our ability to fully assess the relationship between continuous (i.e., non-intermittent) water and HWISE scores, as almost all data from households with continuous access was collected post-intervention. Households who perceived their drinking water to be unsafe also reported greater water insecurity. Both findings suggest the need to further evaluate the added value of water insecurity experiences alongside standard JMP water ladder indicators of availability, access, and quality. Given that water insecurity has been associated with a range of health and well-being outcomes [55–57], data beyond the presence of a piped water connection on premise are likely needed to characterize the expected health benefits of water service construction and maintenance activities.

There is growing interest in the household use of multiple water sources and technology types in response to variations in rainfall, water availability and quality, and source performance, to meet an array of needs, including drinking, cooking, personal hygiene and care of livestock [58,59]. Access to a ‘portfolio’ of water sources has been described as a source of climate resilience [58]. However, while nearly half of respondents reported using a secondary drinking water source over the last two weeks, we found that use of these secondary sources was associated with greater water insecurity, rather than decreased water insecurity. Although this association was not statistically significant, one possible explanation is that households experiencing water insecurity may be more likely to use multiple drinking water sources as a coping strategy, which has been reported previously in northwestern Ecuador [60] and other locations [61,62]. A limitation of our work, however, is that we cannot distinguish households who had access to a secondary source that they did not need to use, from those who did not have access to a secondary source although they would have benefited from such a source. An additional limitation is our lack of longitudinal data: households’ portfolio of water sources, and their satisfaction with each source, may vary significantly based on rainfall patterns, particularly in areas with significant seasonal variations. As a result, observed changes in water insecurity following intervention may be a result of seasonal climatic variability. Our study design also did not evaluate test-retest reliability by collecting repeated HWISE measures in a short-time frame. However, such data would be important for establishing the HWISE Scales as broader monitoring and evaluation tools.

Our study had several additional limitations. First, we evaluated the HWISE Scales, which documents water insecurity at the household level, even though water insecurity is often experienced differently within households, and is often more severe for women [63,64]. In Cha’palaa-speaking communities, enumerators reported that male respondents were reticent to report “negative” emotions experienced by female household members, potentially impacting the results of the HWISE questions related to “shame” and “anger” and highlighting one of the motivations for studies focusing on individual, rather than household experiences. The HWISE Scales were more feasible for implementation partners, as they could be asked of any member of the household who was available at the time of the survey. Enumerators viewed the additional time required for multiple visits to survey specific household members, for example primary cooks, as a significant barrier to achieving the desired sample size in small communities, and the reduced time requirement of the HWISE-4 was preferred by implementers.

We also noted markedly higher rates of water insecurity when using the HWISE-4 versus the HWISE-12 in coastal communities. Coastal communities tend to be ‘water rich’ sites where abundant surface water limits water for drinking more than for hygiene or laundry, which may explain why items present in both the HWISE-4 and the HWISE-12 tended to be affirmed more frequently than items present in the HWISE-12 only. As coastal communities generally depended on unimproved sources and often reported per-capita consumption below international standards, this may indicate that the more detailed HWISE-12 survey underrepresents water insecurity in water-rich sites, where some water-related activities are significantly more impacted than others, thus shifting the communities into lower ordinal categories based on the HWISE-12. Secondly, we lack longitudinal indicators of water quality, although the interventions installed in communities with endline data were fully operational during follow-up visits, and capable of delivering safe water if properly operated. Finally, as an evaluation of the field usability of the HWISE Scales by non-researcher implementation partners, we did not evaluate the validity of the HWISE items with local populations, although our results do suggest potential differences in how items may have been interpreted by the Cha’palaa-speaking groups, in particular. Although we solicited feedback from interviewers about their experiences throughout the process, we did not collect formal qualitative evaluative data, which limits our ability to draw inference based on their experiences.

In light of these limitations, further work is needed to understand how gratitude bias may impact HWISE scores when implementing organizations are also interviewers, to evaluate the feasibility and added value of the individual IWISE Scales versus the HWISE Scales, and to collect repeated data over time to better evaluate reliability and seasonal variability, so that the WISE Scales can also be interpreted in the context of water service improvement and sustainability. However, our experience suggests that the WISE Scales have a place within a larger effort of standardized measures to better capture indicators of system performance and user experiences. In conclusion, the HWISE Scales were readily implemented by multiple Ecuadorian non-governmental organizations (NGOs) engaged in rural water delivery. This enabled a better understanding of the effectiveness of improvements in water access at a community level, rather than at an aggregated level, making it a useful complement to other WASH indicators.

Supporting information