https://orcid.org/0000-0002-3045-8851
Figures
Abstract
The Water Insecurity Experiences (WISE) Scales have been validated to comparably measure water insecurity globally. The scales consist of 12 items that can be administered in approximately 3 minutes. There is interest in developing more rapid versions of the tools for when time is limited. One alternative is to use a subset of 4 items, which has been validated, but has some drawbacks. Here we investigate another alternative: dichotomous (yes/no) response options instead of the original four levels of frequency-based (polytomous) responses. We used nationally representative data from 39 countries to simulate dichotomized responses by collapsing the four levels of frequency (never, rarely, sometimes, often/always) into yes/no. We first explored if “rarely” is meaningful in the gradation of water insecurity, as experiences that occur “rarely” may not be affirmed with dichotomous response options. We tested item-by-item if “rarely” responses predicted dissatisfaction with water quality using logistic regression and found that they were associated with higher odds of dissatisfaction with water quality. As such, some meaningful nuance may be lost if “rare” experiences are not affirmed as “yes”. We then compared the predictive accuracy of WISE scores using simulated dichotomous responses compared to those calculated using polytomous responses. Based on receiver-operator-characteristic (ROC) curves and regression models, scores calculated using dichotomized responses had good predictive accuracy. Scores calculated using the abbreviated 4-item version were similarly accurate. Finally, we examined whether levels of water insecurity, as calculated from the original responses, could be classified using dichotomized responses. Using ROC curves, we found that this approach was effective, offering an advantage over the 4-item scales. While polytomous response options provide more detailed information, dichotomous responses offer the potential advantage of a quicker alternative for measuring water insecurity.
Citation: Bose I, Miller JD, Bethancourt HJ, García OP, Melgar-Quiñonez HR, Salles-Costa R, et al. (2025) Investigating the suitability of dichotomous responses for the Water Insecurity Experiences (WISE) Scales using nationally representative data from 39 countries. PLOS Water 4(5): e0000338. https://doi.org/10.1371/journal.pwat.0000338
Editor: Tafadzwanashe Mabhaudhi, London School of Hygiene and Tropical Medicine / University of KwaZulu-Natal School of Agricultural Earth and Environmental Sciences, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Received: January 6, 2025; Accepted: April 10, 2025; Published: May 20, 2025
Copyright: © 2025 Bose et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: This study was based on survey data collected by the Gallup World Poll (GWP) and ENSANUT 2021. Data from ENSANUT are publicly available on the website of the National Institute of Public Health (Mexico). Nationally aggregated WISE data can be accessed from https://doi.org/10.21985/n2-avk4-9932, additional data can be requested by emailing WISE_scales@northwestern.edu.
Funding: This work was supported by the National Institute of Mental Health of the US National Institutes of Health (R03 MH133200), a Leverhulme Trust Visiting Professorship (Recipient: SLY), and the Innovative Methods and Metrics for Agriculture and Nutrition Action (IMMANA) programme, led by the London School of Hygiene & Tropical Medicine, in partnership with Tufts University and the University of Sheffield. IMMANA is co-funded with UK International Development from the UK government and by the Gates Foundation INV-002962 / OPP1211308. The conclusions and opinions expressed in this work are those of the authors alone and shall not be attributed to the Foundation. Under the grant conditions of the Foundation, a Creative Commons Attribution 4.0 Generic License has already been assigned to the accepted manuscript. The authors also acknowledge the support of the Carnegie Corporation and Northwestern University, including from the Northwestern Institute on Complex Systems, the Global Poverty Research Lab, Weinberg College, the Office for Research, Leslie and Mac McQuown via the Center for Engineering Sustainability and Resilience, the Buffett Institute for Global Affairs, the Center for Water Research, and the Paula M. Trienens Institute for Sustainability and Energy, for data collection. The funders had no role in study design, data analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Water insecurity, the inability to reliably access sufficient water for basic domestic needs, is an increasing global concern given its negative impacts on health and well-being. Accurate and reliable measurement of water insecurity is critical for identifying affected and vulnerable populations, developing policies and programs to reduce its burden, and advancing progress toward Sustainable Development Goal 6, ensuring access to safe water for all [1–4].
The Water Insecurity Experiences (WISE) Scales comparably measure experiences with issues with water access and use across countries [5]. The Scales comprise 12 questions that can be used for assessments at the household (the HWISE Scale) [3] or individual level (the IWISE Scale) [6]; they take approximately 3 minutes to administer [5]. The WISE Scales complement existing “supply-side” water indicators (e.g., access to safely managed drinking water services using WHO/UNICEF’s Joint Monitoring Programme’s criteria [7]) by capturing the state of people’s water insecurity [8]. That is, they offer “user-side” perspectives on experiences with accessing and using water for consumption, hygiene, and other activities. In recognition of the added value of these scales to existing measures, they are increasingly being adopted by governments and development organizations to inform policy and practice, and to monitor and evaluate the impact of water-related programs [5,9–11].
In the original versions of the HWISE and IWISE Scales, item responses not only measured if each of the 12 water-related issues were experienced, but also the frequency (“never”, “rarely”, “sometimes”, or “often/aways”) with which they were experienced in a specified recall period (Fig 1). The recall period can vary depending on the survey objectives. For example, “rarely” refers to an experience occurring on 1–2 days over a 4-week recall period or in 1–2 months over a 12-month recall period (Fig 1). Each response is scored 0–3; the sum of the item responses can then be used to classify individuals or households as experiencing no-to-marginal, low, moderate, or high water insecurity [12].
The 12 items take approximately 3 minutes to administer. The 4 items on the top row are those that comprise the abbreviated versions and take one minute to administer. Full phrasing is in Table A in S1 File. Figure reproduced from a prior publication [13]. Note: Polytomous response options are “Never”, “Rarely”, “Sometimes”, “Often/Always”; dichotomized response options are “No” or “Yes”.
Academics, as well as those in governmental and non-governmental organizations, have expressed interest in developing versions of the WISE Scales that are quicker to implement and reduce participant burden [8,14]. To that end, abbreviated versions of the WISE Scales – the HWISE-4 [15] and IWISE-4 [13] Scales – which are composed of a subset of four items and take only one minute to administer, have been developed and validated for use in low- and middle-income countries [13,15] (Fig 1). In March 2024, the IWISE-4 Scale was recommended by WHO/UNICEF’s Joint Monitoring Programme for generating gender-disaggregated indicators to monitor progress toward Sustainable Development Goal 6 [16].
Although the 4-item WISE Scales have practical benefits, they are limited in two important ways. First, because they only query four experiences, they may not capture key experiences of water insecurity in some settings. For instance, water interruptions were the most commonly reported item by respondents to the 2022 Gallup World Poll in Australia and the United States [17], but “interruptions” is not an item included in the abbreviated scales. Second, these abbreviated scales do not generate enough information to assess multiple categories of water insecurity. That is, the 4-item WISE Scales can only classify individuals or households as experiencing moderate-to-high water insecurity or not, and are not well-suited to capture varying degrees of water insecurity. Given the global heterogeneity in experiences of water insecurity and the effects that even low levels of water insecurity can have on health and well-being, the ability to classify individuals into multiple categories of water insecurity using the 12 WISE items has practical utility [12].
Dichotomous response options: An alternative for rapid assessment
Given the limitations of the 4-item WISE Scales, there is interest in investigating other strategies for efficiently measuring water insecurity experiences while maintaining validity for situations in situations where rapid data collection and limited resources are key concerns [8,14]. One such strategy is using dichotomous response options, whereby respondents report whether they ever experienced any of the issues (i.e., “yes” or “no”) within a particular recall period. Dichotomous responses with experience-based food insecurity scales have been used at the national, regional, and global level for decades [18–20]. In Brazil, for example, the Brazilian Scale of Food Insecurity has been useful for identifying categories of food insecurity, enabling policymakers and program designers to address the distinct causes associated with each severity category [21].
Scales with dichotomous response options offer several potential advantages over those with polytomous levels (e.g., never, rarely, sometimes, often, always). They may be quicker to administer and therefore less costly, making them more feasible when budgets and time are limited. For example, a study in Mexico found that use of dichotomous response options, compared to polytomous options, for the HWISE Scale reduced the time to administer the scale from 3 to 2 minutes [22]. The reason for this may be that dichotomous response options reduce the cognitive burden on respondents and enumerators; respondents are not required to recall the exact frequency of each experience and prompting by enumerators can be reduced. Furthermore, dichotomous responses can be more stable than polytomous responses when assessing a scale’s psychometric properties and equating it across countries using Rasch models [23], which is the current modeling approach used with the Food Insecurity Experience Scale (FIES) to track progress toward Sustainable Development Goal target 2.1 [24,25]. For instance, if responses of “rarely” are less frequent than responses of “sometimes”, this can lead to disordered thresholds (i.e., severity levels that do not increase monotonically across response categories) in a Rasch rating (or partial credit) model, hindering the interpretation of the construct frequency [26,27]. One solution to this is to collapse response categories, but it is preferrable to present response options to participants that avoid these issues altogether.
Despite these potential advantages, the use of dichotomous response options has been questioned in other contexts, such as when measuring anxiety related to health and in scales measuring attitudes and opinions [28,29]. There are concerns that scales calculated using dichotomous responses may discriminate less well between degrees of the construct of interest (e.g., varying severity and frequency of experiencing anxiety). Furthermore, it is unclear how respondents will answer when they have fewer response options, especially if they only occasionally experience the condition under question. When offered only dichotomous options, some might not consider an experience that occurred infrequently to merit affirmation (i.e., they might respond “no/never” when the experience only occurred once or twice). Therefore, despite dichotomous response options potentially reducing complexity and respondent burden, they may compromise the scale’s sensitivity to varying frequencies of water insecurity experiences. This trade-off must be critically evaluated in diverse settings.
Considering these potential advantages and disadvantages, it is uncertain how accurately the WISE Scales with dichotomous responses would capture experiences of water insecurity. Therefore, we sought to understand the potential consequences of using dichotomous responses rather than polytomous response options for the WISE Scales. Because the WISE Scales have thus far primarily been administered using polytomous response options, we addressed this goal by simulating dichotomous responses. We used two dichotomization scenarios to account for any potential uncertainty around how people who responded “rarely” to any of these experiences may respond when given a dichotomous option. We used nationally representative datasets from 39 countries in which the 12-item WISE Scales had been administered to answer four questions:
- Are “rarely” responses affirmed frequently enough to be meaningful in the calculation of water insecurity experience scores?
- Can 12-item WISE scores calculated using simulated dichotomous responses accurately predict scores calculated using (the original) polytomous responses?
- How does the predictive accuracy of the 12-item WISE Scales using simulated dichotomous responses compare to that of the 4-item WISE Scales using polytomous responses?
- Is it possible to create four ordinal categories of water insecurity using simulated dichotomous responses to the 12-item WISE scales that distinguish between levels of water insecurity severity as well as the categories made using the original scoring?
Methods
Ethics statement
This study using secondary, deidentified data was determined to not constitute human subjects research by the Institutional Review Board at the University of North Carolina at Chapel Hill. Gallup World Poll survey procedures were approved by governing bodies as required in each country. Gallup obtained informed consent from participants. We received deidentified data from Gallup for our analyses. All procedures for the Mexican National Health and Nutrition Survey (ENSANUT) were reviewed and approved by the Research, Biosecurity, and Ethics Committees of the National Institute of Public Health, Mexico. Each respondent to the household survey provided his or her written informed consent (Project ID: 1750).
Study design, population, and key variables
This analysis used nationally representative datasets from two sources: IWISE data from the 2020 and 2022 Gallup World Poll (GWP) [30,31] and HWISE data from the 2021 Mexican National Health and Nutrition Survey (ENSANUT 2021) [32]. GWP implemented the IWISE Scale with a 12-month recall period among individuals aged 15 years and older across 31 countries between September 2020 and February 2021 [31], and a further 7 countries in 2022 (n = 50,768). The methodology for data collection and obtaining informed consent from participants followed Gallup’s established protocols, which have been detailed in prior publications [6,30], and were approved by governing bodies as required in each country. ENSANUT 2021 surveyed 12,463 households in Mexico, with a recall period of 4 weeks. Full details on the ENSANUT survey methodology and nationally representative sampling strategy have been described in a prior publication [32]. ENSANUT participants provided written informed consent.
The WISE Scales ask about 12 experiences related to problems with water, including modified or limited behaviours (e.g., unable to wash hands), psychosocial impacts (e.g., worry about water), and supply interruptions (Fig 1; full phrasing in Table A in S1 File). Respondents were asked to report how frequently they (when the IWISE Scale was used) or anyone in their household (when the HWISE Scale was used) experienced these issues. Data were collected with response options “never” (scored as 0), “rarely” (1), “sometimes” (2), and “often” or “always” (3).
GWP surveys measured dissatisfaction with water quality, an indicator previously used to assess the construct validity of the IWISE-12 scale [6], with the item “In your city or area where you live, are you satisfied or dissatisfied with the quality of water?”. Participants responded either “satisfied” or “dissatisfied”.
Calculating WISE scores and categories using the 12-item scales with polytomous response options
WISE Scale responses are summed to create a score with a possible range of 0–36, with higher scores indicating greater water insecurity. Cut-points have been established to classify individuals and households as experiencing four levels of water insecurity: “no-to-marginal” (scores of 0–2), “low” (3–11), “moderate” (12–23), or “high” water insecurity (24–36) [6,12].
Calculating WISE scores using the 12-item scales with dichotomized response options: Two scenarios
Using the data from the polytomous responses described above, we simulated dichotomised responses (i.e., participants affirming whether the experience occurred during the given recall period or not). We simulated two potential scenarios. In the first scenario, termed “Any Affirmation,” any affirmative response (“rarely,” “sometimes,” “often”, or “always”) was recoded as “yes” (1), and “never” was recoded as “no” (0). Given the aforementioned uncertainty about how individuals who responded “rarely” to an experience may answer questions with dichotomous responses, we simulated a second scenario. In the second scenario, termed “Sometimes-to-Always Affirmation”, “sometimes”, “often”, and “always” were recoded as “yes” (1), whereas “rarely” and “never” were recoded as “no” (0). The resulting summed scores for both versions ranged from 0 to 12.
Calculating WISE scores and categories using the 4-item scales with polytomous responses
The abbreviated 4-item versions of the IWISE and HWISE Scales, referred to as IWISE-4 [13] and HWISE-4 [15], respectively, include a subset of four of the 12 experiences asked about in the full versions: worrying about not having enough water, not being able to wash hands after dirty activities due to problems with water, not having enough water to drink, and having to change plans due to problems with water (Fig 1). These responses are summed to create overall water insecurity scores; these can range from 0-12. A score of ≥4 has been used as a cut-point to categorize individuals or households as experiencing water insecurity [13,15].
Statistical analysis using IWISE data
For our first question (if “rarely” responses are affirmed sufficiently frequently to be meaningful in the gradation of water insecurity), we plotted the response frequency for each experience across all countries in the Gallup World Poll. To identify potential differences in the frequency of affirming “rarely” by national water insecurity burden, we also plotted the frequency among countries with low (i.e., United States, Australia) and high (i.e., Cameroon, Zambia) national prevalence of water insecurity. We then used multiple logistic regression models, adjusted for country, to test whether responding “rarely” to an experience was associated with self-reported dissatisfaction with one’s water quality.
For our second question (the predictive accuracy of simulated dichotomous WISE responses to the 12-item scales, relative to those with polytomous responses), we conducted four sets of analyses. First, we used linear regression models to regress the scores from polytomous response options on the scores from dichotomised response options. In these models, we estimated root mean squared errors (RMSE) to quantify the magnitude of error due to dichotomising responses. These models were estimated separately for each country, and the average and ranges across countries were calculated.
Second, we constructed receiver operator characteristic (ROC) curves to evaluate the sensitivity and specificity of different cut-points for scores calculated using dichotomized responses in relation to moderate-to-high water insecurity, as classified using scores generated from the original scale with polytomous responses. The ROC curve displays on the y-axis the sensitivity (true positive rate) and one minus the specificity (false positive rate) on the x-axis of each potential cut-point being able to identify whether an individual is experiencing moderate-to-high water insecurity [33]. Examination of these curves can help identify the cut-point that provides the best sensitivity and specificity and thus will result in the highest proportion of individuals being correctly classified [33]. We then examined the areas under the curve (AUC) to understand the accuracy of the scores calculated using the two dichotomous scenarios (“Any Affirmation” and “Sometimes-to-Always Affirmation”). The AUC is a measure of the overall accuracy of, in this case, the dichotomized scores to discriminate whether someone or not would be classified as experiencing moderate-to-high water insecurity using scores generated from the original scale with polytomous responses, with an AUC of 0.5 showing chance and 1.0 perfect discrimination [33].
Third, we calculated the simulated weighted prevalence of moderate-to-high water insecurity using the optimal cut-points identified by the ROC curves (the thresholds that were found to have the highest sensitivity and specificity) and compared these to the estimated prevalence when using scores generated from polytomous responses. For each country, we estimated the absolute percentage-point differences in prevalence estimates, the percentage of people correctly classified, and the AUC.
Finally, we used logistic regression to compare how scores generated from dichotomized and polytomous responses predicted water quality dissatisfaction. We compared a 3-point difference in the scores from polytomous to a 1-point difference in the dichotomized responses versions. We examined the AUC to compare the accuracy of the polytomous compared to the dichotomized versions. In this case, the AUC allows us to examine the ability of the different scores to discriminate between people that are dissatisfied or not with their water quality. We also ran logistic regressions using the optimal cut-points identified by the ROC curves for dichotomized versions, and a cut-point of ≥12 for the polytomous version to examine how these definitions of moderate-to-high water insecurity were associated with the odds of water quality dissatisfaction.
For our third question (about the predictive accuracy of the WISE Scales using dichotomized responses compared to that of the 4-item WISE Scales using polytomous responses), we ran linear regressions to test the association between scores from the 12-item scales (using polytomous and dichotomous responses) and scores from the 4-item scale with polytomous responses. We calculated RMSE and residuals to quantify prediction errors. Additionally, we calculated the weighted prevalence of moderate-to-high water insecurity using a cut-point of ≥4 for the 4-item scale [13] and compared it to that estimated using the original 12-item scale. We then used logistic regression to understand if water insecurity, as classified using the four different versions of the tool, was associated with water quality dissatisfaction, which was used to assess the construct validity of the IWISE-12 scale [6].
For our fourth question (if it is possible to calculate 4 levels of water insecurity using scores generated from dichotomized response options), we used ROC curves to determine whether cut-points in scores from the dichotomous responses could be identified. Specifically, we tried to determine if we could identify cut-points with high sensitivity and specificity for each water insecurity category (low, moderate and high), as calculated using scores from the full scale with polytomous responses.
Analyses were weighted, where appropriate, using the post-stratification within-country sampling weights constructed by GWP, which account for the probability of selection depending on the sampling frame for each country, non-response and to match the country’s population characteristics. Analyses were weighted using complex survey commands in Stata, which were used to specify sampling strata and post-stratification sampling weights from GWP.
Sensitivity analyses using HWISE data
To assess whether results differed when household, rather than individual, water insecurity was the outcome, we repeated all analyses using data from ENSANUT 2021 – the only nationally representative survey that, to our knowledge, includes the HWISE Scale. Of the 12,619 individuals interviewed, 156 were missing responses to one or more WISE experiences and excluded from this analysis, resulting in a final analytic sample of 12,463 individuals with complete household water insecurity data.
For our first question, we examined response frequencies for each household water insecurity experience. For our second question, we used linear regressions to estimate associations between HWISE scores calculated using dichotomized compared to polytomous responses. We also used ROC curves to explore cut-points in the scores generated from dichotomized responses that maximized sensitivity and specificity for classifying moderate-to-high water insecurity, as defined by a HWISE score of 12 or greater using polytomous responses [3]. We compared unweighted prevalence estimates of moderate-to-high household water insecurity using scores derived from dichotomous responses to those derived from polytomous responses. We calculated the absolute percentage-point differences in prevalence estimates, the percentage of people correctly classified, and the AUC. A question about water quality satisfaction was not asked in this survey.
For our third question, we ran linear regressions to test the association between scores from the 12-item scales (using polytomous and dichotomous responses) and scores from the 4-item scale with polytomous responses. We calculated RMSE and residuals to quantify prediction errors. For our fourth question, we used ROC curves to determine if cut-points with high sensitivity and specificity for each water insecurity category (low, moderate, and high), relative to those used for scores calculated using polytomous responses, could be identified.
Results
The IWISE Scale was administered to 52,560 individuals in 38 countries through the Gallup World Poll. Of these, 1,792 (3.4%) were missing values to one or more experiences and excluded from the analysis using the full scores, resulting in a final sample of 50,768 individuals.
The contribution of “rarely” to WISE scores (Question 1)
“Rarely” was a common response to each experience (Fig 2A). For each item, between 32% and 37% of respondents who affirmed an experience reported it as occurring “rarely” (Table B in S1 File). In countries with a low prevalence of moderate-to-severe water insecurity, such as Australia (A, 0.973%) and the US (U, 3.67%), “rarely” accounted for most of the affirmations (Fig 2B). In Australia, between 50% to 93% of respondents who affirmed an experience reported it as occurring “rarely”, compared to between 58% and 79% in the United States (Table B in S1 File). Conversely, for countries experiencing a high national prevalence of moderate-to-high water insecurity, such as Cameroon (C, 63.9%) and Zambia (Z, 48.1%), “rarely” was affirmed less frequently (Fig 2C). In Cameroon, between 19% to 27% of respondents affirming each experience reported it as occurring “rarely”, compared to 23% to 29% in Zambia (Table B in S1 File).
= 50,768, Gallup World Poll 2020, 2022). A: Aggregated across countries. B: Distribution of “rarely” responses in countries with a relatively low prevalence of moderate-to-high water insecurity (Australia (A) 0.973%; USA (U) 3.67%). C: Distribution of “rarely” responses in countries with a relatively high prevalence of moderate-to-high water insecurity (Cameroon (C) 63.9%; Zambia (Z) 48.1%).
In logistic regression for each WISE item, the odds of reporting water quality dissatisfaction increased monotonically across polytomous options (Table 1). The odds of reporting water quality dissatisfaction were higher among those who responded “rarely” compared to those who responded “never” experiencing a given issue. These results suggest that rare occurrences of water issues can meaningfully predict other constructs related to water insecurity.
The predictive accuracy of scores calculated using dichotomized responses (Questions 2 & 3)
Both versions of the dichotomised-response scores – “Any Affirmation” and “Sometimes-to-Always Affirmation” – accurately predicted scores calculated using the 12-item IWISE Scale with polytomous responses (Table 2; see Tables C & D in S1 File for country-specific results).
The RMSE (i.e., the standard deviation of the residuals from the regression model) was lower for the “Sometimes-to-Always Affirmation” version compared to the “Any Affirmation” version, indicating better overall predictive accuracy of the former. There was, however, greater variability of the residuals at higher values of the “Sometimes-to-Always Affirmation” version, whereas the variability of the residuals appeared to be even across values of the “Any Affirmation” version (Fig A in S1 File); both exhibited heteroskedasticity. The residual pattern and RMSE of the 4-item IWISE Scale were similar to that of the “ Sometimes-to-Always Affirmation” version (Table 2; see Table E in S1 File for country-level results). The mean beta coefficient from the models regressing the polytomous score on the dichotomized scores was highest for the IWISE-4 Scale (2.65, range: 2.24-2.85) and lowest for the “Any Affirmation” version (1.95, range: 1.39-2.26). The “Sometimes-to-Always Affirmation” version showed a slightly higher mean beta value of 2.35, ranging from 2.13 to 2.60 across countries. The IWISE-4 scale had the highest mean beta value of 2.68, ranging from 2.24 to 2.84 across countries. Despite these differences, the R-squared values, correlation coefficients, and standard errors were similar across both dichotomized versions and IWISE-4. In short, both dichotomized versions and IWISE-4 had similar predictive accuracy.
The AUC for scores calculated using both versions of dichotomization showed high accuracy, with values close to 0.98 (Fig B in S1 File). This indicates that both versions were accurate at predicting moderate-to-high water insecurity, as defined as scores ≥12 in the original 12-item scale with polytomous responses.
For the “Sometimes-to-Always Affirmation” dichotomised version, cut-points of ≥4 and ≥5 resulted in the highest overall correct classification of moderate-to-high water insecurity, at 94% and 95%, respectively (Tables F & G in S1 File). For the “Any Affirmation” dichotomised version, greater accuracy was achieved with higher cut-points (≥6 and ≥7), although a high percentage were correctly classified when using a cut-point of ≥4 (86%) and ≥5 (90%) (see Tables H & I in S1 File for country-level results).
We identified different cut-points for estimating water insecurity prevalence using the two dichotomized versions (Fig 3). A cut-point of ≥4 in the “Sometimes-to-Always Affirmation” version provided an estimate of water insecurity prevalence that was 2-percentage-points higher than the estimate from polytomous responses, whereas that same cut-point in the “Any Affirmation” version resulted, on average, in a 16-percentage-point over-estimation (Fig 3). In contrast, a cut-point of ≥7 for “Any Affirmation” resulted in a 1-percentage-point over-estimation of water insecurity prevalence but a 12-percentage-point under-estimation of water insecurity prevalence using the “Sometimes-to-Always Affirmation” version (Tables J & K in S1 File show the weighted prevalence estimates per cut-point for each country). By comparison, using a cut-point of ≥4 for IWISE-4 (which has been previously validated), resulted in an average 3-percentage-point overestimation (Table L in S1 File). Therefore, while similar prevalences can be estimated using both dichotomized versions, the cut-points will differ depending on whether response patterns with dichotomous options align more closely with the simulated “Any Affirmation” version or “Sometimes-to-Always Affirmation” version.
= 50,768, Gallup World Poll 2020, 2022).
The associations between dichotomised-response IWISE scores and odds of reporting water quality dissatisfaction (Fig 4) were similar for both the “Any Affirmation” (red line) and “Sometimes-to-Always Affirmation” (green line) scenarios compared to that observed when using polytomous responses (blue line). The 4-item IWISE Scale (yellow line) had comparable associations, demonstrating the consistency of these results across different scoring methods. Similar AUC values suggests that both dichotomized versions and IWISE-4 had comparable accuracy to the original 12-item scale with polytomous responses in predicting dissatisfaction with water quality. In other words, the abbreviated scales behaved similarly to the full scale in predicting another construct related to water insecurity.
Score options include the full IWISE Scale using polytomous responses (at 3-point intervals), the full IWISE Scale using dichotomized responses (“Any Affirmation” and “Sometimes-to-Always Affirmation” versions), and the 4-item IWISE Scale using polytomous responses, based on nationally representative data from 38 countries (n = 50,768, Gallup World Poll 2020, 2022).* * All models were adjusted for country and weighted by survey weights. For the full IWISE12 score with polytomous items, the range was 0 to 36 (i.e., each point as labeled on the x-axis corresponds to 3 points on the full IWISE12 polytomous item score).
When using various cut-points (≥4, ≥ 5, ≥ 6, and ≥7) to define moderate-to-high water insecurity, both dichotomized versions yielded odds of reporting water quality dissatisfaction that were similar to those obtained when using a cut-point of ≥12 for the full scale with polytomous responses (Table M in S1 File). For example, for individuals experiencing moderate-to-high water insecurity, as classified using the full scale with polytomous responses, the odds of water quality dissatisfaction were 4.50 times higher (95% CI: 4.18-4.85) compared to those experiencing no-to-low water insecurity. In comparison, the estimated odds for the “Any Affirmation” version at a cut-point of ≥5 was 4.28 (95% CI: 3.99-4.58) and 4.11 (95% CI: 3.81-4.44) for the same cut-point using the “Sometimes-to-Always Affirmation” version. At higher cut-points, similar associations were observed, although the strength of associations tended to decrease as the cut-points increased. The IWISE-4 scale, using a cut-point of ≥4, had a comparable association with water quality dissatisfaction (OR: 4.29, 95% CI: 3.99-4.61). These results indicate that all scoring versions exhibited similar construct validity.
Creating water insecurity categories with dichotomized responses (Question 4)
Using ROC curves, we identified cut-points that enabled the categorization of water insecurity for both dichotomized versions (“Any Affirmation” and “Sometimes-to-Always Affirmation”). These cut-points resulted in similar distributions of individuals across water insecurity categories (no-to-marginal, low, moderate, and high) when compared to the 12-item scale with polytomous responses (Fig 5). Thus, it was possible to estimate ordinal water insecurity categories using both dichotomized versions. Tables N-Q in S1 File provide details on the proportion of individuals correctly classified at each cut-point, as well as the AUC values for countries with low and high overall water insecurity.
= 50,768, Gallup World Poll 2020, 2022).
These results demonstrate the feasibility of categorizing water insecurity using scores calculated with dichotomous response options. We cannot propose definitive cut-offs, however, because the current analysis relies on simulated data. Establishing appropriate cut-offs requires empirical data that capture how individuals respond when the items are explicitly presented with dichotomous response options.
Sensitivity analyses using HWISE data
We observed similar relationships using HWISE data collected in the ENSANUT survey in Mexico (Text 1, Tables R-U, & Fig C-E in S1 File).
Discussion
We evaluated whether experiencing issues with water access and use only rarely was associated with other water problems (e.g., dissatisfaction with water quality) and simulated the potential consequences of administering WISE Scales with dichotomous instead of polytomous responses using data from nationally representative surveys in 39 countries (Table 3). First, even a rare experience of any of the 12 WISE items was strongly associated with higher odds of also reporting dissatisfaction with water quality. Second, two scenarios for dichotomizing polytomous items (either considering rare experiences as an affirmation or not) were simulated. Both dichotomization scenarios accurately predicted the 12-item scale score calculated using polytomous responses. Third, the predictive accuracies of the 12-item scale with dichotomized responses were similar to the predictive accuracy of the abbreviated IWISE-4. Fourth, estimation of low, moderate, and high water-insecurity categories was reasonable with both versions of the dichotomized WISE scales. Taken together, these findings suggest that administering the WISE Scales with dichotomous instead of polytomous response options may be a useful strategy in some situations, but it may come at the cost of some lost information, as discussed below.
Given the importance of even “rare” experiences in the gradation of water insecurity experiences, the interpretation of water insecurity may be affected if rare occurrences are not adequately captured when items are administered with dichotomous response options (i.e., if respondents choose not to affirm an experience that occurred only once or twice). Thus, when administering the WISE Scales with dichotomous response options, efforts should be made to encourage respondents to carefully consider if these experiences have occurred even once over the recall period, and if so, to affirm these experiences, so as not to miss people who are experiencing infrequent water insecurity. That is, instructions must clarify that any occurrence should be considered an affirmation.
Dichotomous response options may limit the ability to understand water insecurity dynamics in some situations. For example, an intervention may cause people to shift from experiencing an issue often to rarely, as has been found for food insecurity [34]. This difference is meaningful for understanding the impact of an intervention but would be lost if only dichotomous response options were provided. We therefore recommend polytomous response options that capture the frequency of experience for program evaluations (Table 4). Similarly, understanding the frequency with which these experiences occur might be important for designing effective targeted strategies (Table 4), as those experiencing issues more frequently may require a different level of intervention. Understanding frequency of food insecurity coping strategies has been shown to be useful to inform targeting of food security interventions [35].
Both versions of simulated dichotomized responses (i.e., whether rare experiences were considered an affirmation or not) accurately predicted IWISE and HWISE scores calculated using the polytomous response options. The scores from dichotomized responses also had similar accuracy in predicting a related construct of water insecurity (dissatisfaction with water quality). The WISE-4 has similar predictive accuracy to the dichotomized scores, both in terms of predicting scores calculated from WISE-12 polytomous responses and predicting a related water insecurity construct. While both abbreviated versions of the scales (WISE-4 and WISE-12 with dichotomous responses) may offer viable alternatives to the full scale when time and resources are limited, the 12-item WISE Scales with dichotomous responses will better capture the full array of ways in which water insecurity can manifest and interrupt life (Table 4).
The prevalence of no-to-marginal, low, moderate, and high water insecurity could be estimated using WISE scores with dichotomized responses, providing a clear advantage over the abbreviated IWISE-4 and HWISE-4 Scales (Table 4). What those cut-points are, however, will depend on how participants respond when offered dichotomous response options. For IWISE, if all respondents who responded “rarely” were considered to have affirmed the experience (the “Any Affirmation” scenario), then a cut-point of ≥7 has the best specificity and a cut-point of ≥6 has the best sensitivity for classifying moderate-to high water insecurity. If, however, those who responded “rarely” were not to have not affirmed the experience (the “Sometimes-to-Always Affirmation” scenario), then a cut-point of ≥5 has the best specificity and a cut-point of ≥4 has the best sensitivity for classifying moderate-to high water insecurity. Ultimately, the establishment of appropriate thresholds for defining different levels of water insecurity with WISE Scales scores with dichotomized responses must be based on data from WISE surveys administered with this response format from the start.
A challenge in knowing which of the versions of simulated dichotomized responses best illustrates the amount of information lost when administering WISE surveys with dichotomized responses is that we do not know how people would have responded if they had been presented with a dichotomous response option. Research is needed in which dichotomous responses are presented to the respondent, ideally in direct comparison using a split sample to the presentation of polytomous responses. One such split-sample study has been conducted in Mexico with HWISE-12; this study found that the prevalences of water insecurity estimated in the sub-sample that was administered the survey with dichotomous response options was comparable to the prevalence of the sub-sample that was administered the survey with polytomous response options [22].
Further research is required to understand how affirmations might change depending on how many response options are presented and whether prompts are provided to encourage respondents to consider rare occurrences as an affirmation. Further research is also required to understand how respondents’ affirmation of items that occur only rarely might change when presented with dichotomous (experienced the issue or not) instead of a polytomous (frequency of experiencing the issue) response options. It will also be important to assess if losing this nuance is worth the practical, logistical, or cost advantages. Specifically, comparing findings from administering the scales with dichotomous and polytomous response options in similar populations will permit informed decisions about which format of responses best capture the information that is most important to organizations, researchers, and policymakers. It will also be valuable to validate dichotomous response options in diverse contexts, particularly high-income countries and areas with low water insecurity prevalence, to determine their robustness across settings.
Whilst the sample used in our study contained data from low-, middle-, and high- income countries, there are only two countries that were classified as high-income. Further research is required in high-income settings to understand how use of dichotomous response options in WISE surveys may affect measurement of water insecurity in contexts where it is less prevalent.
Conclusion
Polytomous response options provide more information, but dichotomous response options hold promise for measuring water insecurity when there is need for a more rapid but still comprehensive survey. For program evaluation, WISE Scales with polytomous responses are likely more suitable because they offer greater nuance in understanding both if an experience occurred as well as its frequency. WISE Scale items with dichotomous response options have the potential to provide a time-saving, valid alternative to polytomous response options for measuring occurrence of water insecurity experiences and estimating prevalence of no-to-marginal, low, moderate, and high water insecurity. Additional data collection using dichotomously phrased responses is needed to fully understand all that might be gained and lost with the dichotomization of WISE response options.
Supporting information
S1 File. File containing all supplementary tables (Tables A-U), figures (Fig A-E) and text (Text 1).
https://doi.org/10.1371/journal.pwat.0000338.s001
(DOCX)
References
- 1. Jepson WE, Wutich A, Colllins SM, Boateng GO, Young SL. Progress in household water insecurity metrics: a cross-disciplinary approach. WIREs Water. 2017;4(3).
- 2. Rosinger AY, Young SL. The toll of household water insecurity on health and human biology: current understandings and future directions. WIREs Water. 2020;7(6).
- 3. Young SL, Boateng GO, Jamaluddine Z, Miller JD, Frongillo EA, Neilands TB, et al. The Household Water InSecurity Experiences (HWISE) Scale: development and validation of a household water insecurity measure for low-income and middle-income countries. BMJ Glob Health. 2019;4(5):e001750. pmid:31637027
- 4. Young SL, Frongillo EA, Jamaluddine Z, Melgar-Quiñonez H, Pérez-Escamilla R, Ringler C, et al. Perspective: the importance of water security for ensuring food security, good nutrition, and well-being. Adv Nutr. 2021;12(4):1058–73. pmid:33601407
- 5.
Young SL, Miller JD, Bose I. Measuring human experiences to advance safe water for all. Evanston, Illinois: Institute for Policy Research, Northwestern University; 2024.
- 6. Young SL, Bethancourt HJ, Ritter ZR, Frongillo EA. The Individual Water Insecurity Experiences (IWISE) Scale: reliability, equivalence and validity of an individual-level measure of water security. BMJ Glob Health. 2021;6(10):e006460. pmid:34615660
- 7.
Slaymaker T, Johnston R, Young SL, Miller J, Staddon C. Detailed review of a recent publication: an innovative measure of the experience of water insecurity can help identify vulnerable households and evaluate interventions. UNC Water Institute- WaSH Policy Research Digest; 2020.
- 8.
Young SL, Bose I, Miller JD, Pascavis K, Alzarez K, Barstow C, et al. The Water Insecurity Experiences (WISE) Scales: a manual for implementation and analysis of people’s experiences with water. Evanston, Illinois Institute for Policy Research, Northwestern University; 2024.
- 9. Editorial N. Water crisis: how local technologies can help solve a global problem. Nature. 2023;620(7972):7.
- 10. Mundo-Rosas V, Shamah-Levy T, Muñoz-Espinosa A, Hernández-Palafox C, Vizuet-Vega NI, Torres-Valencia M de LÁ, et al. Inseguridad alimentaria y del agua. Salud Publica Mex. 2024;66(4, jul-ago):581–8. pmid:39977093
- 11. Melgar-Quiñonez H, Gaitán-Rossi P, Pérez-Escamilla R, Shamah-Levy T, Teruel-Belismelis G, Young SL, et al. A declaration on the value of experiential measures of food and water insecurity to improve science and policies in Latin America and the Caribbean. Int J Equity Health. 2023;22(1):184. pmid:37670356
- 12. Frongillo E, Bethancourt H, Miller J, Young S. Identifying ordinal categories for the water insecurity experiences scales. J Water Sanit Hyg Dev. 2024.
- 13. Bethancourt HJ, Frongillo EA, Young SL. Validity of an abbreviated Individual Water Insecurity Experiences (IWISE-4) Scale for measuring the prevalence of water insecurity in low- and middle-income countries. J Water Sanitation Hyg Develop. 2022;12(9):647–58.
- 14. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health. 2018;6:149. pmid:29942800
- 15. Young SL, Miller JD, Frongillo EA, Boateng GO, Jamaluddine Z, Neilands TB, et al. Validity of a four-item household water insecurity experiences scale for assessing water issues related to health and well-being. Am J Trop Med Hyg. 2021;104(1):391–4. pmid:33124535
- 16.
Caruso BA, Chipungu J, Hennegan J, Motivans A, Pandolfelli L, Patrick M, et al. Priority gender-specific indicators for WASH Monitoring under SDG Targets 6.1 and 6.2: recommendations for National and Global Monitoring. New York: United Nations Children’s Fund (UNICEF) and World Health Organization (WHO); 2024.
- 17.
Young S, Bethancourt H, Frongillo E, Bose I. Water insecurity experiences (WISE) country profiles. Institute for Policy Research Northwestern University; 2024.
- 18. Pérez-Escamilla R, Salles-Costa R, Segall-Corrêa AM. Food insecurity experience-based scales and food security governance: a case study from Brazil. Global Food Security. 2024;41:100766.
- 19. Pérez-Escamilla R. Food and nutrition security definitions, constructs, frameworks, measurements, and applications: global lessons. Front Public Health. 2024;12:1340149. pmid:38550323
- 20. Salles-Costa R, Segall-Corrêa AM, Alexandre-Weiss VP, Pasquim EM, Paula NM de, Lignani J de B, et al. Rise and fall of household food security in Brazil, 2004 to 2022. Cad Saude Publica. 2023;39(1):e00191122. pmid:36753092
- 21. Pérez-Escamilla R, Vilar-Compte M, Gaitan-Rossi P. Why identifying households by degree of food insecurity matters for policymaking. Global Food Security. 2020;26:100459.
- 22. García O, Parás P, Melgar-Quiñonez H. Validity of a binary household water insecurity scale for assessing water-related issues and its relationship with public health indicators. Under review. 2024.
- 23.
Engelhard Jr G, Wang J. Rasch models for solving measurement problems: invariant measurement in the social sciences. Sage Publications; 2021.
- 24.
Cafiero C, Nord M, Viviani S, Del Grossi ME, Ballard T, Kepple A, et al. Methods for estimating comparable prevalence rates of food insecurity experienced by adults throughout the world. Rome: Food and Agriculture Organization of the United Nations (FAO), 2016; 2016. Report No.
- 25. Cafiero C, Viviani S, Nord M. Food security measurement in a global context: the food insecurity experience scale. Measurement. 2018;116:146–52.
- 26.
DeAyala RJ. The theory and practice of item response theory. Guilford Publications; 2013.
- 27.
Bond TG, Fox CM. Applying the Rasch model: fundamental measurement in the human sciences. Psychology Press; 2013.
- 28. Welch PG, Carleton RN, Asmundson GJG. Measuring health anxiety: moving past the dichotomous response option of the original Whiteley Index. J Anxiety Disord. 2009;23(7):1002–7. pmid:19560314
- 29. Preston CC, Colman AM. Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychol (Amst). 2000;104(1):1–15. pmid:10769936
- 30.
Gallup. Worldwide research methodology and codebook. Washington, DC; 2020.
- 31. Young SL, Bethancourt HJ, Ritter ZR, Frongillo EA. Estimating national, demographic, and socioeconomic disparities in water insecurity experiences in low-income and middle-income countries in 2020-21: a cross-sectional, observational study using nationally representative survey data. Lancet Planet Health. 2022;6(11):e880–91. pmid:36370726
- 32. Romero-Martínez M, Barrientos-Gutiérrez T, Cuevas-Nasu L, Bautista-Arredondo S, Colchero MA, Gaona-Pineda EB, et al. National Health and Nutrition Survey 2021 methodology. Salud Pública de México. 2021;63(6):813–8.
- 33. Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240(4857):1285–93. pmid:3287615
- 34. Cohen CR, Weke E, Frongillo EA, Sheira LA, Burger R, Mocello AR, et al. Effect of a multisectoral agricultural intervention on HIV health outcomes among adults in Kenya: a cluster randomized clinical trial. JAMA Netw Open. 2022;5(12):e2246158. pmid:36508217
- 35.
Maxwell D, Watkins B, Wheeler R, Collins G. The coping strategies index: a tool for rapidly measuring food security and the impact of food aid programs in emergencies. Nairobi: CARE Eastern and Central Africa Regional Management Unit and the World Food Programme Vulnerability Assessment and Mapping Unit; 2003.