https://orcid.org/0000-0003-2026-3687
Figures
Abstract
Many people in rural Sub-Saharan Africa use a communal handpump as their primary drinking water source. Handpump performance is assessed as “functionality,” typically measured using a binary indicator of whether a pump can produce water. This indicator does not reflect service delivery goals defined by the United Nations. We used a cross-sectional dataset of 1682 handpumps from 10 Sub-Saharan African countries (Ethiopia, Ghana, Kenya, Malawi, Mali, Mozambique, Niger, Uganda, Zambia, and Zimbabwe) to explore alternative functionality indicators and potential performance standards for those indicators. Alternative functionality indicators studied here included whether a breakdown occurred in the past year and, if so, in the two weeks preceding the survey, downtime during the most recent breakdown, strokes required to produce water, the water produced per subsequent stroke, and service continuity. We conducted a factor analysis to determine the uniqueness of each of these indicators and a threshold analysis to determine the sensitivity of functionality rates (such as “X% of handpumps are functional”) to performance standards. We found that the studied indicators are unique, not redundant, and cannot be combined without an unacceptable loss of performance information. Using Monte Carlo analysis, we found that a functionality assessment using these indicators would be highly sensitive to the thresholds used to evaluate performance. This work demonstrates the importance of selecting functionality indicators to measure progress and thresholds to assess progress. The indicators explored here cannot be reduced or combined and should not be conflated, and selected thresholds may greatly impact how rural handpumps are evaluated. Handpump functionality indicators and thresholds should be selected to reflect performance goals for communal water supply. Once goals are established, indicators and contextualized thresholds can be selected to reflect those goals. Policy decisions related to handpump management should be based on assessments using indicators that reflect the intended policy outcomes.
Citation: McManus C, Lane K, Diarra S, Cronk R (2025) Alternative indicators of handpump functionality: Toward consistent performance-oriented monitoring. PLOS Water 4(5): e0000271. https://doi.org/10.1371/journal.pwat.0000271
Editor: Sara Marks, Eawag, SWITZERLAND
Received: May 17, 2024; Accepted: April 6, 2025; Published: May 19, 2025
Copyright: © 2025 McManus et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data may be found at https://doi.org/10.15139/S3/J3IVUG.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Two hundred million people in Sub-Saharan Africa use approximately 700,000 handpumps as their primary drinking water source [1]. This is more common in rural than urban areas; an estimated 26% of the rural population of Sub-Saharan Africa uses a handpump as their primary source of drinking water, compared to an estimated 7% in urban areas [1]. Despite calls from practitioners and policymakers to shift water service provision from handpumps to piped supplies and on-plot water service [2–4], millions will continue to rely on community handpumps in the 2020s, the 2030s, and beyond [5].
Performance (of any organization or system, not just handpumps) is often measured using key performance indicators (KPIs). KPIs are used to measure progress in achieving goals set for a system [6], which are visionary and can be subdivided into objectives focusing on a specific component of that vision. Thresholds are then used to evaluate performance according to KPIs – thresholds specify performance standards such as good, fair, or poor. For example, Sustainable Development Goal (SDG) 6 sets a goal of “ensuring availability and sustainable management of water and sanitation for all” with eight targets. Target 6.1 is to “by 2030, achieve universal and equitable access to safe and affordable drinking water for all.” Target 6.1 is measured using Indicator 6.1.1, “proportion of the population using safely managed drinking water services.” [7].
A safely managed drinking water service is provided by an improved source accessible on premises, available when needed, and free from fecal and chemical contamination [7]. Communal handpumps installed in drilled boreholes (wells) are considered an improved source but are, by definition, not an on-premise water supply. The service these handpumps provide (including an implicit assessment of whether that service is available when needed) is assessed using functionality evaluations.
“Functionality” was defined by the World Health Organization (WHO) in 1983, although no definition of functionality has been formalized or standardized – or distinguished from a definition of “service delivery.” In the decades since 1983, functionality indicators have changed from the WHO’s original approach of monitoring water quantity, quality, handpump reliability, and convenience [8] – an indicator set reminiscent of the definition of safely managed drinking water service. A 2018 study found that functionality is typically measured using a binary indicator reporting if a pump produced any water on the day of an inspection [9]. This indicator reports statistics such as “approximately one in four handpumps in sub-Saharan Africa are non-functional at any point in time.” [10]. In this study, we distinguish the binary functionality indicator from the service delivery goal of functionality by referring to it as “operationality” because it can only be used to report if a pump is operating.
There are several problems with this operationality indicator. Because it is a “yes/no” indicator, it cannot be used to assess pumps along a spectrum of performance; a high-yield, well-maintained handpump is considered the same as a poorly maintained pump that barely produces water [11]. It only reflects the state of handpumps at a single point in time [11], and data are not collected regularly [10] but are used to make claims about sustainability [12,13]. Snapshot assessments, especially those that conflate operationality with sustainability, cannot be used to predict performance over time and do not consider how seasonality affects aquifer and handpump yields [11,14]. They may not consider how the siting, drilling, and design of the borehole and installation of the pump affect performance, including the season of drilling and initial installation [15–17]. A 2018 study in Malawi found that problems with handpumps were more likely to be due to inappropriate initial implementation than due to inadequate ongoing maintenance [18]. However, functionality assessments are often used to evaluate and indict the handpump’s management and maintenance system, despite the many factors that can affect the performance of a handpump. In addition to the quality of initial installation and seasonality, the functionality of a handpump may be affected by the quality of spare parts available in supply chains [19], competing priorities of committee members who volunteer their time [20,21], and water quality [22].
The operationality indicator treats all non-working pumps as equivalently failing; a pump that broke down two hours before a survey is considered the same as a pump that has been broken and unusable for two weeks or months. The important distinction in evaluating these pumps would be the difference in “downtime,” or time that the pump was non-operational due to a breakdown or water resource issue. Pumps were never infallible; the Afridev pump was designed to prioritize simple, swift repairs [23]. The goal is not to avoid breakdowns but to respond to them quickly, and using the operationality indicator does not allow handpump monitors to consider that occasional breakdowns are inevitable.
In response, academic and practitioner groups have developed and are using new indicators and thresholds to measure the performance of rural handpumps. The UPGro Research Consortium (Unlocking the Potential of Groundwater for the Poor) proposed a definition of functionality consisting of two indicators. UPGro assesses handpumps based on “yield” (with a flow rate threshold of 10 liters per minute) and “reliability” (with a threshold of 30 days of cumulative downtime over the preceding year) [9].
Downtime, a measure of how much time a handpump does not produce water due to breakdown or water source supply issues, is increasingly used to monitor handpumps. However, it is used two ways: counting days of downtime experienced cumulatively over a year (as in the UPGro definition) or counting downtime during each breakdown. Uptime, a provider of results-based funding to handpump maintenance organizations, requires that breakdowns be repaired in approximately three days or fewer for an organization to receive financial subsidies. However, they describe this as a cumulative downtime measure (which can be no more than 4% of a quarter) [24]. UNICEF recommends reporting the number of breakdowns per year and the average downtime per breakdown, although this is part of a sustainability check rather than strict “functionality;” UNICEF considers functionality and reliability components of sustainability [25].
Reliability, while an agreed-upon goal for rural water service provision, is not measured using consistent indicators. Some organizations (such as UPGro and Uptime) use downtime as an indicator of “reliability.” In contrast, others use downtime as an indicator of “continuity” and measure reliability using an indicator measuring the frequency (not duration) of breakdowns [26].
Ghana’s Community Water and Sanitation Agency (CWSA) considers reliability a component of service level, not functionality. Once a pump is deemed functional, its service level is assessed – including its reliability, which CWSA assesses using a cumulative downtime threshold of 5% (interpreted as 20 days of downtime over a year) [27]. Before assessing service level, CWSA measures handpump functionality using an indicator reporting how many strokes are required to produce the first drop of water; a pump producing water in five or fewer strokes is considered fully functional [12,27]. This same indicator and threshold (five strokes or fewer to produce water) is used to assess leakage in the rising main or foot valves by Area Mechanics in Malawi [28].
The rationale behind these organizations’ functionality indicators and thresholds is not always reported. Indicators such as efficient water production and breakdown response should be selected to reflect water service or handpump performance goals set by international, national, or local monitoring agencies. Units of these indicators should reflect the monitored service – for example, using units of stroke counts versus flow rates when measuring productivity. While UPGro specified their testing protocol (40 strokes/minute) [29], other tests found that surveyors are inconsistent in stroke rates (the number of strokes per minute) [30]. Indicators based on counting strokes (e.g., number of strokes to produce 20 L of water) were more reliable (across surveyors) than indicators based on controlling or assuming stroke rates to calculate flow rates (in L/min) [30].
Rural water supply delivery chain actors may use other indicators for various purposes. For example, a mechanic may use an indicator selected to monitor productivity (such as volume per stroke or strokes to produce the first drop of water) to identify the need for repairs; if the volume of water produced per stroke declines over time, it may indicate that there is a leak [28,31]. A District Water Officer may use a different indicator (such as days of downtime after a breakdown is reported) to assess the responsiveness of Area Mechanics to breakdowns.
Once indicators are defined, thresholds should be selected carefully, especially if the indicators measure continuous phenomena. Categorizing or “binning” performance (e.g., into “good” and “bad” levels of performance) can be problematic if the indicator used to measure performance is incremental. These problems could be further compounded if the consequences of decreased performance are also incremental (i.e., if no threshold exists at which negative impacts appear). For example, a quantitative microbial risk assessment estimated that each additional day of reliance on raw water will increase a person’s annual probability of E. coli, Cryptosporidium, and Rotavirus infection from drinking water [32]. However, another study in Kenya found that children in households whose pump was repaired within 24 hours had a lower incidence of diarrhea. These benefits were absent for children whose pump was repaired in more than 24 hours [33]. Threshold-setting exercises should consider how thresholds reflect service delivery objectives and the implications of categorical evaluation.
This work explores the applicability and use of seven performance indicators for monitoring and assessing rural handpump functionality. This study aimed to demonstrate how indicators can be evaluated with two objectives: maximizing information collected while minimizing monitoring burden. Using data from ten Sub-Saharan African countries, we 1) conducted factor analysis to determine the uniqueness of each indicator and 2) conducted threshold analysis (including Monte Carlo Analysis) to determine how numerical standards for these indicators could affect calculated functionality rates.
Methods
Data collection
Water point survey data were collected in ten Sub-Saharan African countries (Ethiopia, Ghana, Kenya, Malawi, Mali, Mozambique, Niger, Uganda, Zambia, and Zimbabwe) as part of a broader survey of World Vision (WV) WASH (water, sanitation, and hygiene) activities. Data were also collected in four other countries: Honduras, India, Rwanda, and Tanzania. Data from Honduras and India were excluded from this study to maintain a geographical focus on Sub-Saharan Africa, and data from Rwanda and Tanzania were excluded due to the low number of surveys conducted at handpumps. Surveys were conducted at four handpumps in Rwanda and 19 handpumps in Tanzania. Data collection occurred in 2017 [34].
Water points were selected from within a broader household survey sampling frame. Sub-national sampling units (including districts) were identified within each country; units where WV had conducted WASH interventions were matched with comparison units where WV had not operated. If a sampling unit included more than 200 households, it was subdivided into units with 200 households or less; one of these secondary sampling units was randomly selected to represent the original sampling unit. 56 WV units and 56 comparison units were randomly selected in each country. After the sampling units were identified, 25 randomly selected households in each sampling unit were surveyed. All water points within the sampling unit were surveyed and investigated.
The water point survey instrument included interview questions about the use and operation of the water point. Respondents were community leaders or committee members responsible for managing the water point. The survey also included direct observations, including a visual inspection of the pump and a test of pump operations. Five water points in each sampling unit were randomly selected for water quality testing. The survey instrument was translated into local languages for each country and verified by research consultants or WV staff in each country. Surveys were programmed using mWater, a mobile survey tool, which included skip patterns, automatic recording of GPS points, minimum and maximum allowable values to prevent enumerators from entering implausible values, and other quality assurance measures.
The survey instrument was translated into each country’s local language and verified by research consultants or WV staff in each country. Data was collected in the local language of the respondent.
This analysis was restricted to handpumps installed on boreholes. Eight survey questions were used as the seven indicators of handpump functionality used in this study (Table 1). Three indicators were binary, one was ordinal categorical (with three levels), and three were continuous counts.
Factor analysis
The seven functionality indicators in the dataset were used to explore and evaluate the performance of rural handpumps. Exploratory factor analysis was used to determine whether these seven indicators could be reduced in number or combined to decrease the data collection burden.
Factor analysis is used to identify unobserved (latent) variables using observed measured variables [35]. For example, factor analysis may determine that the latent variable or concept of “athletic ability” can be specified using the measured variables of “time to run five kilometers,” “strength to weight ratio,” and “time to complete agility test.” Here, factor analysis was used to identify the latent variables related to handpump performance.
The null hypothesis of a factor analysis is that there are no latent variables, and all measured variables are equal. The hypothesis tested was that latent variables represent handpump reliability and yield. The reliability latent variable would be observed using indicators related to breakdowns: the occurrence of a breakdown in the last two weeks and year and the downtime at the most recent breakdown. The yield latent variable was observed using indicators of the handpump’s water production: the number of strokes to produce water and the volumetric production of subsequent strokes.
Factor analysis identifies sets of indicators (known as factors) that are highly internally correlated (highly correlated with each other) and minimally externally correlated (minimally correlated with indicators outside the set) [35]. Eigenvalues state the variance of each factor relative to the individual indicators. A factor with an eigenvalue greater than one has more variance than any other indicator [36]. The uniqueness of each indicator is calculated. Uniqueness is the proportion of an indicator’s variance that is not shared with other indicators (is unique to that indicator). Uniqueness is equivalent to the commonality of an indicator subtracted from 1 (1 – communality).
Threshold analysis
Agencies and organizations monitoring handpumps and setting performance standards may set numerical thresholds for individual functionality indicators (such as Uptime’s 3-day threshold for downtime). These thresholds, which could be based on performance goals and the feasibility of achieving those goals, could be used to identify high- and low-performing handpumps for targeted interventions such as training or technical support or to provide performance-based funding. Threshold analysis was conducted on the three continuous indicators measured in this dataset (strokes to first drop of water, volume of water produced per stroke, and days of downtime at the last breakdown). The threshold analysis determined what proportion of handpumps would meet varying thresholds for each indicator.
A range of thresholds was simulated for each of the three continuous indicators, and the proportion of handpumps meeting each threshold within the range was calculated. Ranges for the three indicators were:
- Number of strokes to produce water: 1–20 strokes (1-stroke interval)
- Volume of water per stroke: 0.1-0.5 L/stroke (0.05-L/stroke interval)
- Downtime of most recent breakdown: 1–31 days (1-day interval)
The proportion of handpumps that met each threshold within each range was calculated and plotted on a curve.
Monte Carlo analysis
The previous threshold analysis explored how functionality rates would differ if handpumps were assessed separately on the three continuous indicators. However, some monitoring programs may use a composite indicator for functionality comprising those three indicators. Each indicator would have a threshold, and a handpump must perform adequately against all three indicator thresholds to be considered functional (Fig 1).
Monte Carlo Analysis is a tool that can explore how changing multiple independent values (known as assumptions) can affect a single value (known as a forecast) calculated using those independent values. Assumption ranges are specified, and random values are sampled from within the ranges over many trial simulations. The simulated value of the forecast is calculated for each trial, and results are reported as a distribution of forecast values [37]. In this study, the thresholds for the three individual indicators were the assumptions. The forecast value was the functionality rate; the proportion of surveyed handpumps met the simulated threshold for all three continuous indicators.
The Monte Carlo Analysis was used to conduct 10,000 trials, randomly sampling potential thresholds for strokes to water, stroke volume, and downtime at recent breakdown. The thresholds were sampled from specified uniform ranges:
- Downtime of most recent breakdown: 1–5 days
- Number of strokes to produce water: 3–10 strokes
- Volumetric production of strokes: 0.25-0.5 L/stroke
The range for the downtime at the most recent breakdown was based on the Uptime downtime limit (3 days) [24]. The range of strokes to produce water was established on the CWSA threshold (5 strokes) [27]. The range for the volume produced per stroke was based on the specifications for the Afridev (0.4125 L/stroke) [38] and India Mark (0.375 L/stroke) [39] handpumps.
Results
Descriptive statistics
Pump type varied by country (Table 2). Most countries used Afridev and India Mark (II and III) pumps, except for Niger and Zimbabwe. In Niger, Vergnet pumps were the most common (45% of surveyed pumps). In Zimbabwe, 97% of surveyed pumps were recorded as “other” type; it should be noted that the Bush pump is common in Zimbabwe.[40]
The operationality of pumps varied by country; the overall operationality rate was 85%, ranging from 72% in Mali to 98% in Niger (Table 3). Afridev pumps had a higher operationality rate (90%) than India Marks (81%). 75% of pumps were continuously available (24 hours per day) – even if they were non-operational.
There was no statistically significant difference in the average downtime during the last breakdown for Afridevs and India Marks. However, Afridevs broke down more in the preceding year than India Marks (60% versus 52%). Afridevs produced water within fewer strokes than India Marks (4.8 versus 7.5 strokes) and then produced more water per stroke (0.57 versus 0.31 L/stroke).
Data related to one of the functionality indicators were missing for 733 (44%) handpumps in the 10 Sub-Saharan African countries. Most of this missingness was due to a need for more response about the number of strokes required to fill a 20 L bucket at operational handpumps.
Factor analysis
No identified factor had an eigenvalue greater than 1 for the entire dataset (Table 4); in other words, no factor explained more of the total variance than any indicator [36]. This is likely due to the high uniqueness values of each indicator (Table 5). One factor had an eigenvalue greater than 1 for the Afridev subset, but none had an eigenvalue greater than 1 for the India Mark subset.
The uniqueness of each indicator is the proportion of its variance that is unique to that indicator and not shared with the other indicators. Uniqueness values can range from 0 to 1. The average uniqueness of the seven indicators was 0.80 for the entire dataset, ranging from 0.68 (number of pump strokes required to produce water; least unique) to 0.99 (volume of water produced per stroke; almost unique) (Table 5). The uniqueness values of indicators in the Afridev subset were slightly less than the whole dataset (average uniqueness: 0.75). The uniqueness values of indicators in the India Mark subset were slightly higher than the entire dataset (average uniqueness: 0.85).
Threshold analysis
The three continuous indicators measured in this study are not used sector-wide and do not have universal thresholds. The indicator-specific threshold analysis determined the proportion of surveyed handpumps that would meet each value within a range of potential thresholds for these indicators (Fig 2).
a) Duration of most recent breakdown (days); b) Strokes to produce water (strokes); c) Volumetric production per stroke (L/stroke).
Pumps that did not experience a breakdown in the year preceding the survey (41% of all pumps, 40% of Afridevs, and 48% of India Marks) were considered to have a downtime of zero days at the most recent breakdown. 31% of recent breakdowns lasted five days or fewer, and more than 25% lasted longer than one month (Fig 2a). 26% of all breakdowns would meet Uptime’s repair threshold within three days. Breakdowns of Afridev pumps had less downtime than those of India Mark pumps.
78% of operational handpumps (those that could produce water and were tested) would meet the Ghana CWSA’s standard for pump strokes to produce the first drop of water (Fig 2b). Afridevs produced water in fewer strokes than India Marks.
The potential threshold for volumetric production per stroke has the highest sensitivity of the three continuous indicators, with a sharp drop between 0.4 and 0.5 L/stroke (Fig 2c). 90% of operational handpumps provided at least 0.4 L/stroke, while 34% provided 0.5 L/stroke. 99% of operational handpumps provided at least 0.2 L/stroke.
Monte Carlo analysis
The indicator-specific threshold analysis explored how functionality rates would change if a threshold for a single indicator were changed. Monte Carlo Analysis was used to analyze how a functionality rate based on a composite of three continuous indicators (Fig 1) would change with changes to the thresholds of those indicators.
The “base” functionality case was based on assumed thresholds of three days of downtime at the most recent breakdown (based on Uptime’s standard), five strokes to produce water (based on CWSA’s standard), and 0.35 L/stroke of water production (based on design specifications). Using these thresholds, 42% of all pumps (52% of operational pumps) would be considered functional.
Over 10,000 trials, the thresholds of these three continuous indicators were simulated and sampled randomly from specified ranges (1–5 days of downtime, 3–10 strokes to produce water, and 0.25-0.5 L/stroke). We used a uniform distribution across the ranges because we had no information about the possible distribution of potential thresholds within these ranges. Over the 10,000 trials, the functionality rate ranged from 11-41% of all surveyed handpumps (Fig 3). 6–47% of Afridevs and 8–41% of India Mark pumps would be considered functional if thresholds were selected from within the simulated ranges. 15% of surveyed handpumps could not produce water on the day of the survey, so they could not be tested. They were considered to have failed to meet the thresholds for strokes to water and volume per stroke. Between 15 and 58% of handpumps that could produce water on the day of the survey would be considered functional if thresholds were selected from within the simulated ranges. Between 8% and 60% of operational Afridevs and 12–65% of operational India Marks would be considered functional.
a) All handpumps; b) Afridevs; and c) India Marks.
The modeled functionality rates were most sensitive to the threshold for the volume of water produced per stroke (Fig 4). This held for the entire dataset and the Afridev and India Mark subsets. The functionality rates of all pumps and the Afridev subset were more sensitive to the downtime threshold than the threshold for strokes to water. In contrast, the India Mark functionality rate was more sensitive to the threshold for strokes to water than downtime.
a) All handpumps; b) Afridevs; and c) India Marks.
Discussion
This study explored a dataset that included 1682 handpumps, about which we collected data for a set of alternative functionality indicators. First, we conducted a factor analysis to test if the set of indicators can be reduced in number. The binary operationality indicator is simple to monitor, and a new functionality indicator (or set of indicators) should seek to maximize information while minimizing the monitoring burden. However, the factor analysis results indicated that the indicators tested here were unique and not redundant. Reducing the individual indicators to a set of factors (composite indicators) would result in an unacceptable loss of information about the performance of rural handpumps.
Next, we explored the potential thresholds of those indicators – individually and in a composite. Using threshold analysis (including Monte Carlo Analysis), we found that functionality would be sensitive to thresholds of the three continuous indicators measured. While 84% of handpumps were operational at the time of the survey, only 37% would have met potential thresholds for downtime during the last breakdown (3 days), strokes to produce water (5 strokes), and volume of water per stroke (0.35 L/stroke). When these thresholds were varied (1–5 days downtime, 3–10 strokes to produce water, and 0.25-0.5 L/stroke), that proportion ranged from 11-41%. Thresholds of indicators used to assess functionality may greatly affect subsequent determinations of calculated “functionality rates.”
The proportion of handpumps meeting increasing thresholds of three continuous indicators did not decrease linearly. Where impacts of incremental changes in performance may also be incremental if not linear (such as health impacts associated with longer periods of downtime) [32], thresholds should be selected to reflect service delivery goals, balanced with the feasibility of certain levels of service delivery (e.g., breakdown response time). The factor analysis results indicated that the indicators should not be used in a composite. Still, national and international monitoring programs may need to determine a comparable functionality rate. In this case, thresholds should be selected even more carefully.
Future functionality monitoring indicators
The rural water supply sector needs a new definition of functionality that reflects rural water supply goals [9,11]. This goal-setting process should consider how functionality contributes to the pursuit of universal, safely managed drinking water, and if and how functionality is to be assessed separately from the assessment of service provision (as in Ghana and Rwanda) [27,41], and/or separately from/as part of the assessment of sustainability and reliability of rural water supply (as by UNICEF) [25]. Evaluation of functionality may focus on the hardware (handpump productivity and breakdown frequency). In contrast, service provision evaluation focuses on managing the handpump (downtime at each breakdown and over the year).
Once these goals are agreed upon, this functionality definition should consist of two components: the indicators used to measure functionality and the thresholds of those indicators used to evaluate the functionality of monitored handpumps. Indicators reflect the questions asked about the service provided by handpumps, and thresholds reflect the level of acceptability of answers to those questions.
The first component of a new definition of handpump functionality will be the indicators selected to measure performance. A complete set of functionality indicators (including those not studied here) should be evaluated and selected to reflect the agreed-upon handpump performance goals. The second component is the set of thresholds used to assess the performance of handpumps according to each indicator.
In response to known issues with the binary operationality indicator, organizations that monitor rural water supply systems (including governments and non-governmental organizations) have developed alternative indicators of functionality and rural handpump performance in general [9,24,25,27,41,42]. These indicators measure the yield and reliability of rural handpumps in different ways, including cumulative or breakdown-specific downtime, flow rates, and stroke counts. The design of these indicators and the selection of thresholds used to assess performance on those indicators is not always straightforward. Water sector actors should select standard indicators of functionality, but each evaluating entity should set thresholds for those indicators. This will allow for the prioritization of different performance objectives (e.g., reliability vs. yield) and for consideration of local context (e.g., disjointed supply chains, which make speedy repairs challenging).
For example, Ministries in Rwanda and Ghana include a cumulative downtime indicator in their assessment of service delivery at rural handpumps. However, the threshold in Rwanda is 30 days of downtime over a year [41], and in Ghana, the threshold is 20 days of downtime over a year [27]. The different thresholds may be due to the perceived feasibility of minimizing downtime. Monitoring agencies that set contextualized thresholds can update their regulations in response to changes in handpump managers’ capacities to meet those thresholds.
Similarly, different handpump service delivery chain actors may use monitoring data differently. A District Water Officer may use downtime monitoring to assess the performance of mechanics repairing handpumps. A handpump operator may use longitudinal data about stroke-based indicators (such as volume produced per stroke) to identify the need for repairs. A national Director of Water Resources may aggregate that same volumetric productivity data, or data about the continuity of water availability, to assess the nationwide health of groundwater resources.
Contextualization of functionality assessments
Functionality assessments and studies identifying functionality determinants should consider more than a handpump’s current conditions. Indicators that evaluate the performance of a handpump or its maintenance provider can only tell so much of the handpump’s story [11]. This is especially true if a handpump is not performing adequately. If a well was drilled too shallow or in poor soils [43], the pump was installed incorrectly, or supply chains only include low-quality replacement parts, the management committee will be limited in their effectiveness.
Much of the academic literature exploring “determinants” of handpump functionality explore the functionality impacts of community management systems [12,15,26,44,45]. However, the functionality of a handpump also depends on the quality of its initial siting, borehole drilling, and installation – over which the current water committee may have had little or no control. Longitudinal collection of functionality data would allow for relative comparisons of a single water point’s performance over time, hopefully reducing the undue responsibility for handpump “failure” assigned to volunteer water committees.
Study limitations
Several limitations may affect the validity of this study’s results. Data were initially collected for an impact evaluation, and households were sampled using a matching methodology. Findings may not be generalizable within surveyed countries.
Substantial nonresponse contributed to a large degree of missingness in the dataset. While we believe this missingness to be random, we do not know if or how the missingness may have affected the findings.
Other work has shown the impacts of seasonality on handpump performance [13,46], but data were collected during both rainy and dry seasons in only one country (Ethiopia).
This analysis explored how a set of alternative functionality indicators might affect functionality assessments, but the set of alternative indicators was non-exhaustive. Further research should examine a complete set of possible indicators, including the frequency of breakdowns. It should also compare cumulative and breakdown-specific downtime indicators.
Because of these limitations, we do not propose any indicators or thresholds.
Conclusions
Handpumps provide water service to millions in Sub-Saharan Africa and worldwide and will continue to play a role in rural water service provision throughout the coming decades. There is growing recognition that handpump performance should be monitored using an expanded set of indicators beyond “working/not working,” but different organizations and agencies in the rural water supply sector are developing inconsistent definitions of functionality. This work calls for an overhaul of these alternative definitions and demonstrates a new functionality monitoring definition could be developed.
The rural water supply sector needs to collectively determine goals for water supply systems such as handpumps and then identify specific indicators to measure progress in achieving those goals. This goal-setting exercise should consider how functionality, sustainability, and service provision differ, and indicators should be selected to reflect those distinctions. Indicator selection should also consider who (e.g., users, mechanics, government officials) needs what information about specific aspects of handpump performance.
Then, different agencies or organizations that monitor functionality can select thresholds to use to evaluate the progress (measured by indicators) in achieving established goals. The indicators should be universal, but thresholds can be contextualized. Because thresholds define success and failure (and ultimately, calculated “functionality” rates), they should also be carefully selected to reflect feasibility of achieving targets given the hydrogeological and sociotechnical capacity where handpumps are operating.
References
- 1.
Danert K. Handpump reliance, functionality and technical failure. 2022. Report No.: 1.
- 2.
Hutchings P. Reflecting on the “Future perspectives for rural water supply” debate. 2014. Available: https://www.ircwash.org/news/future-perspectives-rural-water-supply-“-death-handpump”
- 3.
Wedgwood A. The Age of the Handpump Should Be Over (But Isn’t). 2021. Available: https://www.linkedin.com/pulse/age-handpump-should-over-isnt-alison-wedgwood/.
- 4.
Fonseca C. The death of the communal handpump? Rural water and sanitation household costs in lower-income countries. Cranfield University. 2014.
- 5.
Carter R. Long Live the Humble Handpump. International Association of Hydrogeologists. 2015.
- 6. Bauer K. KPIs – The Metrics That Drive Performance Management. Information Management. 2004;14:63.
- 7.
UN. Goal 6: Targets and Indicators. 2015 [cited 5 Feb 2024]. Available: https://sdgs.un.org/goals/goal6#targets_and_indicators
- 8.
WHO. Minimum evaluation procedure (MEP) for water supply and sanitation projects. 1983.
- 9. Bonsor H, MacDonald A, Casey V, Carter R, Wilson P. The need for a standard approach to assessing the functionality of rural community water supplies. Hydrogeol J. 2018;26(2):367–70. pmid:31983889
- 10. Foster T, Furey S, Banks B, Willetts J. Functionality of handpump water supplies: a review of data from sub-Saharan Africa and the Asia-Pacific region. International Journal of Water Resources Development. 2019;36(5):855–69.
- 11. Richard C. Carter RCC, Ian Ross IR. Beyond ‘functionality’ of handpump-supplied rural water services in developing countries. Waterlines. 2016;35(1):94–110.
- 12. Fisher MB, Shields KF, Chan TU, Christenson E, Cronk RD, Leker H, et al. Understanding handpump sustainability: Determinants of rural water source functionality in the Greater Afram Plains region of Ghana. Water Resour Res. 2015;51(10):8431–49. pmid:27667863
- 13. Foster T. Predictors of sustainability for community-managed handpumps in sub-Saharan Africa: evidence from Liberia, Sierra Leone, and Uganda. Environ Sci Technol. 2013;47(21):12037–46. pmid:24111486
- 14. MacAllister DJ, MacDonald AM, Kebede S, Godfrey S, Calow R. Comparative performance of rural water supplies during drought. Nat Commun. 2020;11(1):1099. pmid:32132535
- 15. Foster T, Willetts J, Lane M, Thomson P, Katuva J, Hope R. Risk factors associated with rural water supply failure: A 30-year retrospective study of handpumps on the south coast of Kenya. Sci Total Environ. 2018;626:156–64. pmid:29335170
- 16.
Liddle ES, Fenner RA. The impact of procurement and siting procedures on handpump functionality in Uganda. 41st WEDC International Conference: Transformation Towards Sustainable and Resilient WASH Services. 2018.
- 17.
Harvey P. Borehole sustainability in rural Africa: an analysis of routine field data. 30th WEDC International Conference: People-Centered Approaches to Water and Environmental Sanitation. 2004.
- 18.
Mannix N, Norrie J, Paterson F, Nhlema M, Mleta P, Nkhata M, et al. Making the case for improved planning, construction and testing of water supply infrastructure in Malawi. 41st WEDC International Conference: Transformation Towards Sustainable and Resilient WASH Services. 2018.
- 19.
Danert K. Handpump standards, quality and supply chains with Zambia case study. 2022. Report No.: 3. doi:10.13140/RG.2.2.18986.70089
- 20. Kativhu T, Mazvimavi D, Tevera D, Nhapi I. Implementation of Community Based Management (CBM) in Zimbabwe: The dichotomy of theory and practice and its influence on sustainability of rural water supply systems. Physics and Chemistry of the Earth, Parts A/B/C. 2018;106:73–82.
- 21.
Lane M. Community-based management of handpumps in rural Ghana: A quantitative analysis of what needs to change. 2018.
- 22.
Danert K. Rapid corrosion of handpumps. 2022. Report No.: 2. doi:10.13140/RG.2.2.34086.19528
- 23.
Baumann E, Furey S. How Three Handpumps Revolutionised Rural Water Supplies: A brief history of the India Mark II/III, Afridev and the Zimbabwe Bush Pump. 2013. Report No.: 2013–01.
- 24.
McNicholl D, Hope R, Money A. Results-based contracts for rural water services. 2020.
- 25.
Scharp C, Saleh A, Gnilo ME, Jiménez A, Ledeunff H. Sustainability Checks: Guidance to Design and Implement Sustainability Monitoring in WASH. 2017.
- 26. Morita T, Bain R, Mommen B, Trelles J, Muianga A, Nhacume A, et al. Determinants of the operational performance of community-managed handpumps in rural Mozambique: An analysis of five years of cross-sectional sustainability assessments. Groundwater for Sustainable Development. 2024;25:101149.
- 27.
CWSA. Framework For Assessing And Monitoring Rural And Small Town Water Supply Services In Ghana. 2014.
- 28.
MoAIWD. Training Manual for Area Mechanics. Rural Water Supply Operation and Maintenance Series. Lilongwe; 2015. Report No.: 4.
- 29.
Fallas H, MacDonald A, Casey V, Kebede S, Owor M, Mwathunga E. Measuring functionality and performance levels. 2019.
- 30. Smith DW, Wind S, Markov IA, Ongom SA, Davis J. Inter- and intra-rater reliability of handpump functionality field tests. Sci Total Environ. 2023;869:161616. pmid:36646225
- 31. Adank M. On strokes and leaks, or how to assess a handpump. In: IRC [Internet]. 2013. Available: https://www.ircwash.org/blog/strokes-and-leaks-or-how-assess-functionality-and-service-levels-handpumps
- 32. Hunter PR, Zmirou-Navier D, Hartemann P. Estimating the impact on health of poor reliability of drinking water interventions in developing countries. Sci Total Environ. 2009;407(8):2621–4. pmid:19193396
- 33. Thomson P, Stoler J, Byford M, Bradley DJ. The Impact of Rapid Handpump Repairs on Diarrhea Morbidity in Children: Cross-Sectional Study in Kwale County, Kenya. JMIR Public Health Surveill. 2024;10:e42462. pmid:38227359
- 34.
UNC Water Institute. The World Vision 14-Country WaSH Evaluation: Final Report. 2020.
- 35.
Gorsuch RL. Factor Analysis. 2nd ed. New York: Routledge; 2015.
- 36. Kaiser HF. The Application of Electronic Computers to Factor Analysis. Educ Psychol Meas. 1960;XX:141–51.
- 37. Oracle. One-Minute Spotlight: Oracle’s Crystal Ball. 2008. Available: https://www.oracle.com/docs/tech/middleware/crystal-ball.pdf
- 38. Baumann E, Keen J. Afridev Handpump Specification. 2007. Available: https://rural-water-supply.net/en/resources/318
- 39. Erpf K. India Mark Handpump Specifications. 2007. Available: http://www.rural-water-supply.net/en/resources/details/327
- 40.
MacArthur J. Handpump standardisation in sub-saharan africa: Seeking a champion. 2015.
- 41.
Republic of Rwanda Ministry of Infrastructure. National Guidelines for Sustainable Rural Water Supply Services. 2019.
- 42.
Twyman B, Boulenouar J, Motte E, Harris B, Mambulu J. Professionalising the management of Malawi’s rural water supply services: situation analysis and proposed management models. 2023.
- 43. MacAllister DJ, Nedaw D, Kebede S, Mkandawire T, Makuluni P, Shaba C, et al. Contribution of physical factors to handpump borehole functionality in Africa. Sci Total Environ. 2022;851(Pt 2):158343. pmid:36041625
- 44. Foster T, Hope R. Evaluating waterpoint sustainability and access implications of revenue collection approaches in ruralKenya. Water Resources Research. 2017;53(2):1473–90.
- 45. Behnke NL, Klug T, Cronk R, Shields KF, Lee K, Kelly ER, et al. Resource mobilization for community-managed rural water systems: Evidence from Ghana, Kenya, and Zambia. Journal of Cleaner Production. 2017;156:437–44.
- 46. Klug T, Cronk R, Shields KF, Bartram J. A categorization of water system breakdowns: Evidence from Liberia, Nigeria, Tanzania, and Uganda. Sci Total Environ. 2018;619–620:1126–32. pmid:29734591