Measuring people’s views on health system performance: Design and development of the People’s Voice Survey

Todd Lewis and co-authors discuss development and use of the People’s Voice Survey for health system assessment.


Background and introduction to the people's voice survey
Estimates based on 2016 excess death data show that approximately 5 million people die each year in low-income and middle-income countries (LMICs) from treatable conditions despite seeking health care, pointing to a global crisis in health system quality [1].The 2018 Lancet Global Health Commission on High Quality Health Systems in the Sustainable Development Goal Era (HQSS Commission) noted that large-scale health system transformations are needed to improve quality of care [2][3][4].The HQSS Commission defined a high-quality health system as one that "optimizes health care in a given context by consistently delivering care that improves or maintains health outcomes, by being valued and trusted by all people, and by responding to changing population needs."Thus, locating people at the center of health systems is of utmost importance and this requires obtaining their feedback on health system performance.This feedback can improve patient experience which is both intrinsically valuable to users and instrumentally valuable for patient safety and clinical effectiveness, including adherence to recommended clinical practice and use of preventive care [5].
There are few cross-nationally comparable instruments for tracking people's perspectives of health system performance [6,7].Available survey instruments, such as the Demographic and Health Surveys and Service Provision Assessments, capture a useful but limited set of concepts regarding people's experiences of health care (Table 1).In LMICs in particular, many existing measures focus on supply-side factors (e.g., spending, provider numbers) instead of the processes and outcomes that matter most to people, such as experience of respectful care and trust in the system [8].Further, the population's assessment of the health system is a necessary complement to measures of health system-amenable outcomes currently in wide use [6].
Recent tools, especially in high-income countries, have begun to include a limited set of questions on these topics, though these surveys focus exclusively on recent users (e.g., patient surveys) or specific disease groups (e.g., noncommunicable diseases).Many patient experience surveys rely on in-person interviews that are expensive and cumbersome to implement [9,10].Meanwhile, people's opinions can shift rapidly in response to political, social, economic, and population health need changes making repeat assessment essential.
The People's Voice Survey (PVS) is a new tool designed by the Quality Evidence for Health System Transformation (QuEST) Network, an initiative focused on measuring and improving • The People's Voice Survey (PVS) aims to fill this gap.The PVS is a novel multicountry survey of people's perspective on health system performance.It measures a wide range of domains, including health status, health system utilization patterns, ratings of care quality, and confidence and trust in the health system.
• The survey allows for a flexible, mixed mode design that uses telephone, online, and in-person data collection approaches to achieve a nationally representative sample of adults.Critically, the survey includes both health system users and non-users.It can be adapted for use in high-, middle-, and low-income countries.
• We describe the motivation for this new instrument, the multistep collaborative development and validation process, and policy use cases for 19 countries participating in the first wave of data collection.
• Findings from the survey can be used to integrate people's voices into health system policymaking and guide strategic investments towards higher quality health systems.health system quality through multicountry partnerships.The PVS is a rapid population-representative survey that aims to inform action toward more effective and people-centered health systems and promote health system accountability to populations.It can assess public sentiment on factors like confidence and trust in the health system among the full adult population in each country, including recent users of health services as in patient experience surveys, but also past and future users.This allows policymakers to understand how well the health system is serving the entire population in addition to specific age or disease groups.
The PVS was collaboratively developed by researchers in the first wave of implementing countries, policymakers, and key regional health system stakeholders.It builds upon existing tools by measuring a wider range of domains of health system performance among full populations in a rapid, cross-nationally comparable way.The findings can inform national and multinational policy decisions and evaluation of health system programs to improve health care access and quality.Through the QuEST Network, the survey methods and findings will become freely accessible to promote policy change by governments and data uptake by health system researchers.

Content
The PVS is based on the definition of a high-quality health system proposed by the HQSS Commission in 2018.The Commission emphasized that high-quality health systems need to work with people not only to improve health outcomes, but to generate trust and economic benefit for all people.The PVS focuses on the elements of system performance that are most apparent to and valued by the population, including positive user experience and confidence in the health system [2,11,12].Notes: People care about outcomes beyond good health, which include trusting that the system can meet their needs, confidence that they can afford services, and endorsement of health system performance.These perceptions are informed by processes of care, including system competence (e.g., whether the health system provides coordinated, easy-to-use care integrated across platforms), care competence (e.g., provision of high-quality care from knowledgeable, high-skilled providers), and user experience (e.g., good customer service and respect).These processes and outcomes are underpinned by the foundations of the health system, including health status, demographic characteristics, patient activation, and expectations of care.https://doi.org/10.1371/journal.pmed.1004294.g001 The PVS tool focuses on questions for which people, rather than providers, clinical observations, or facility records, are the gold standard source of information.It assesses perceptions at multiple levels of the health system, including the micro level (e.g., the point of care), the meso level (e.g., use of local facilities), and the macro level (e.g., overall health system assessment) as outlined in Box 1.Where possible, we used questions that have been validated across multiple contexts.
In addition to standard demographic questions such as age, location, and health status, we include 2 items that measure patient activation, defined as patients' willingness and ability to take independent actions to manage their health and care, to understand people's autonomy and decision-making power [13].To better understand health care use patterns, we ask about frequency and heterogeneity of facility visits over the last 12 months and how users rate the quality of those visits, including for telemedicine.We also include questions on perceived medical mistakes and whether the respondent did not obtain needed care.
For those who have used care in the last 12 months, we ask them to rate multiple domains of quality such as time spent with the provider and perceived provider knowledge and skills.We include both objective and subjective rankings to gauge what is acceptable to users in different contexts.We ask whether respondents would recommend the recently used facility to a friend or family member to calculate a net promoter score, an approach to assessing whether users are "promoters" or "detractors" of health care facilities [14].
The final section of the survey assesses overall confidence in the health system, including publicly provided primary care and respondent confidence in their ability to obtain and afford high-quality care if needed.Other items include assessment of the health system trajectory (i.e., is the health system getting better or worse?) and the need for reform of the system (i.e., does the health system need major or minor changes?).As user satisfaction is deeply intertwined with people's expectations for care, we include 2 vignettes of care to help us adjust other quality ratings in the survey for local expectations and provide more accurate crossnational comparisons.

Implementation
The PVS is designed to be used in any country regardless of region, income level, or health system structure.Implementation features are described in Table 2.The target population is all individuals aged 18 years and older; in the future, the survey could be adapted to younger

Survey feature Description
Survey mode • The PVS was designed as a brief instrument to be delivered via telephone.Wave 1 implementation also included in-person and web-based delivery in some countries.Telephone-based data collection is faster and less costly than face-to-face surveys, which will promote repeated use of PVS over time.
• The PVS can be delivered through other modes, including in person, online probability panels, online self-administration, and hybrid methods.

Sampling and sample size
• A minimum nationally representative sample of 1,000 respondents is recommended; a sample size of 2,000 is recommended to permit some stratified analysis.
• Telephone surveys represent the adult population well in countries with minimum population telephone ownership of 80% (all but Ethiopia and Kenya among Wave 1 countries; supplemental in-person samples were added in these countries).
• Random-digit dialing (RDD), known-list sampling, or sampling from an online probability panel was used.Details of this design are in Text A in S1 Appendix.
Data stewardship • Data were cleaned and recoded in a comparable way across countries such that datasets could be easily appended for multicountry analyses and cross-national comparisons.
• Code, including construction of a standard set of indicators for PVS survey items, will be available for public use.
• Aggregate national data will be made available for use by policymakers, researchers, and other stakeholders as soon as available; deidentified, individual-level data will be made publicly available after a one-year embargo period following completion of data collection, analyses, and reporting by PVS collaborators.individuals.The survey includes both users and non-users of the health system because health systems should be equipped to serve everyone and, especially in settings where services are supported through public funds, all people should derive value from the health system.Further, non-users may be more likely than users to have issues with the current health system and may provide insight as to how the system might best serve them as potential future consumers of health care.The Wave 1 survey has thus far included 19 countries and was available in 34 languages (Fig 2 and Table A in S1 Appendix).Details on participating countries, ethics, and funding are in Text A in S1 Appendix.
The PVS was primarily designed for telephone but is adaptable to mixed mode delivery, including in-person and web-based delivery, as required to reach the population of each country.When using telephone, we conducted interviews with the full sample of participants required in each country.Where telephone ownership was less than 80% of the population, as in Ethiopia and Kenya, we conducted an additional sample of in-person interviews to augment the full telephone sample.We also used computer-assisted web-interviewing (CAWI) when this was the most effective way to reach the population, as in the United States, United Kingdom, and South Korea.
The PVS aims to obtain population sentiment about performance of the health system by estimating population proportions agreeing with a range of statements.A survey of 1,000 individuals selected at random will produce an estimate that is within a 3% margin of error of the population proportion 95% of the time.This is the case when the prevalence is 50%; smaller numbers are needed when prevalence is higher or lower.Thus, we used a minimum sample of 1,000 in all countries.Several of our samples are larger than this to permit some stratified analysis (e.g., by urban/rural).
In each setting, detailed metadata were captured to describe the context and events surrounding data collection, such as elections, COVID-19 spikes, and major health system reforms.These data can be used to account for significant events or circumstances that may influence these cross-sectional data in dissemination efforts and to contextualize opinion data that may be perceived as unstable.Findings will be disseminated through short fact sheets and summary briefs for policymakers and through longer survey reports that provide an in-depth look at methods and results from each survey wave.A publicly available metadata registry will contain contextual data for each wave of the survey in each country.Finally, aggregate data from surveys across countries will be available via dashboards on the QuEST Network website (www.questnetwork.org),where datasets will also be downloadable by the public.

Development and validation
QuEST Network researchers conducted a collaborative, multistage process to develop and validate the PVS (Fig 3).To guide this process, we assembled a diverse Global Development Group (GDG) comprised of 30 health system experts from 18 high-, middle-, and low-income countries.Development group researchers conducted a broad scoping review of recent survey literature and relevant survey tools to identify questions and response options used to measure key survey domains.The GDG generated survey aims and priorities, critically appraised survey domains and items for value, clarity, and relevance, and assessed survey construction.The group met regularly over the course of 18 months in an iterative process to help ensure content validity of the survey and co-produce a "draft zero" instrument.
The GDG sought external consultation on both methodology and content of the survey.This process included peer review by survey experts from the University of Michigan and content and measurement experts from academic and multilateral organizations, including the World Health Organization (WHO) and World Bank.Reviewers assessed for best practices in survey design and delivery, such as question formulation and flow, and completeness and relevance of survey content.This feedback informed subsequent drafts and bolstered content validity of the instrument.
In Wave 1 countries, the survey instrument underwent a collaborative adaptation by local researchers and/or policymakers to ensure the instrument would be locally applicable, interpretable, and comprehensive.This helped to maintain content equivalence by ensuring crossnational comparability, clarity, and local relevance of questions in each setting.As part of this process, we conducted cognitive interviews with eligible respondents in 11 countries.Results highlighted multiple areas for improvement, including cutting lengthy items and simplifying questions with high cognitive burden.
PVS implementers in each setting identified the minimum set of survey languages necessary to reach most of the national population based on recent, nationally representative surveys.When possible, we used a team translation approach; at minimum, translation and back-translation were performed by qualified translators and closely reviewed by QuEST researchers to ensure content equivalence across languages.
To test the survey in practice, we conducted a pre-test of the full survey among a 200 person sample of respondents from an online panel in the United States.We also conducted a pilot test in one or more languages in every country.Pre-testing and pilot surveys allowed us to assess factors such as survey length, coherence, local interpretability, and quality of the response data.Additional details on steps taken to validate and adapt the PVS for participating countries are available in Text B in S1 Appendix.

Pathways to policy
Data from the PVS will be used to answer research and policy questions to inform government practices and strengthen health system performance.Key indicators measured in the PVS are provided in Box 2. The survey will provide data that can spur health systems research, including on novel items that can inform best practices for future health system measurement, such as whether adjusting for population expectations is essential for accurately gauging health care quality ratings.PVS data are owned by collaborating researchers in the QuEST Network.As the PVS instrument and data are intended to be global public goods, they will be freely accessible to the public after a one-year embargo period to address national or regional research questions and deepen future inquiry into health systems performance.Ideally, the PVS would be integrated as a routine component of health system planning.It can be incorporated into ongoing population surveys as an additional module or run in parallel to existing consumer surveys to provide complementary data.As a relatively low-cost, rapid survey, the PVS can be used for both ad hoc assessment at critical moments of health system evolution and longitudinal measurement of population sentiment.National governments can also use the PVS to plan for and monitor implementation of universal health coverage initiatives and inform health system design to better meet people's needs and preferences.Multinational organizations and global partners can use the tool to monitor effectiveness of investments.Advocacy groups can measure community demand for health system � Mental health service use � Respondents with chronic disease who have usual source of care � Respondents who needed but did not use health services by health status � Reasons respondents did not use care

PLOS MEDICINE
improvements to better hold governments accountable.We have identified several specific policy uses for PVS in Wave 1 countries in Box 3.
The PVS will be available for use by national governments, multilateral organizations, and other institutions to bring a social voice to health system planning.While the survey is designed to work in any country regardless of income-level, location, or health system structure, implementation will require (1) adaptation of a limited number of questions to the local health system context; (2) translation to local languages; (3) obtaining relevant ethical Box 3. From data to policy: Using the People's Voice Survey for health system improvement 1.The PVS can be embedded in routine health system evaluation to facilitate crossnational comparison and spur action by governments.
� In Kenya, the PVS can be integrated into routine service readiness and provision surveys to capture the user perspective on quality.
� In countries with heterogenous health system performance by state, such as India and the United States, PVS data can enable routine subnational comparisons.
� The PVS can be conducted in parallel with existing consumer surveys, such as the Primary heAlth Care quAlity Cohort ChinA (ACACIA), to expand domains measured and bolster ongoing initiatives.
2. The PVS can help governments evaluate large-scale health system reforms, capturing population perspective before and after implementation.
� Romania aims to improve overall quality of care and target areas such as midwifery care in light of the COVID-19 pandemic.In Greece, the government has been implementing a large-scale primary health care reform that requires patients to identify a personal doctor as a first point of care.PVS data can signal endorsement of or opposition to these changes.
� In Kenya and other countries moving towards universal health coverage, the PVS can provide baseline data to motivate action and target investments.
� Data can inform feasibility assessments for redesign of existing service delivery models, an approach currently underway in the Province of Mendoza, Argentina.
3. Public access to PVS data can help researchers and policymakers understand critically undermeasured areas of the health system.
� Data from the PVS can be shared through regional repositories, such as the integrated African Health Observatory, and national repositories, such as the Kenya Health and Research Observatory and the Ethiopian National Data Management Center for health.
� Fact sheets, summary briefs, and online dashboards can facilitate engagement with government, media, and civil society.approvals where applicable; (4) establishing in-house data collection or contracting a data collection partner and tailoring data collection strategies to the national context; and (5) data sharing with PVS partners.The QuEST Network will update and maintain the PVS based on previous survey waves and support continued expansion of the PVS network of collaborators.We aim for the survey to be an integral part of routine assessment of health system performance in multiple countries to help policymakers make smart investments that are responsive to people's needs.
Fig 1 shows the main components of the PVS framework.

Fig 1 .
Fig 1. People's Voice Survey framework.Notes:People care about outcomes beyond good health, which include trusting that the system can meet their needs, confidence that they can afford services, and endorsement of health system performance.These perceptions are informed by processes of care, including system competence (e.g., whether the health system provides coordinated, easy-to-use care integrated across platforms), care competence (e.g., provision of high-quality care from knowledgeable, high-skilled providers), and user experience (e.g., good customer service and respect).These processes and outcomes are underpinned by the foundations of the health system, including health status, demographic characteristics, patient activation, and expectations of care.

Fig 2 .
Fig 2. Participating countries planned for the first wave of the People's Voice Survey.*In Argentina, the People's Voice Survey was conducted in the Province of Mendoza only.Map shapefile available from the World Bank at: https://datacatalog.worldbank.org/search/dataset/0038272.https://doi.org/10.1371/journal.pmed.1004294.g002

Box 2 .
Key indicators from the People's Voice Survey • Health • Health status � Mental health status � Chronic illness status • Patient activation/empowerment � Level of activation � % of population who can bring up concerns to provider � % of population with low expectations • Health care use � % of population with a type of insurance � % of population with usual source of care by facility ownership and level � Reasons for selecting usual source of care � Total visits, visits by facility, and type of visit (covid, virtual, home, inpatient) � Number of facilities used in past 12 months • Care people need

Table 1 .
(Continued) Notes:The table includes multicountry surveys in current use that are purpose-built for assessing health system performance.It excludes opinion polls, ad hoc surveys and event-based polling, single-country surveys, and surveys of one aspect of health system functioning, such as National Health Accounts.https://doi.org/10.1371/journal.pmed.1004294.t001