Background and introduction to the people’s voice survey

Estimates based on 2016 excess death data show that approximately 5 million people die each year in low-income and middle-income countries (LMICs) from treatable conditions despite seeking health care, pointing to a global crisis in health system quality [1]. The 2018 Lancet Global Health Commission on High Quality Health Systems in the Sustainable Development Goal Era (HQSS Commission) noted that large-scale health system transformations are needed to improve quality of care [2–4]. The HQSS Commission defined a high-quality health system as one that “optimizes health care in a given context by consistently delivering care that improves or maintains health outcomes, by being valued and trusted by all people, and by responding to changing population needs.” Thus, locating people at the center of health systems is of utmost importance and this requires obtaining their feedback on health system performance. This feedback can improve patient experience which is both intrinsically valuable to users and instrumentally valuable for patient safety and clinical effectiveness, including adherence to recommended clinical practice and use of preventive care [5].

There are few cross-nationally comparable instruments for tracking people’s perspectives of health system performance [6,7]. Available survey instruments, such as the Demographic and Health Surveys and Service Provision Assessments, capture a useful but limited set of concepts regarding people’s experiences of health care (Table 1). In LMICs in particular, many existing measures focus on supply-side factors (e.g., spending, provider numbers) instead of the processes and outcomes that matter most to people, such as experience of respectful care and trust in the system [8]. Further, the population’s assessment of the health system is a necessary complement to measures of health system-amenable outcomes currently in wide use [6].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Select multicountry tools measuring population views on the health system in current use.

https://doi.org/10.1371/journal.pmed.1004294.t001

Recent tools, especially in high-income countries, have begun to include a limited set of questions on these topics, though these surveys focus exclusively on recent users (e.g., patient surveys) or specific disease groups (e.g., noncommunicable diseases). Many patient experience surveys rely on in-person interviews that are expensive and cumbersome to implement [9,10]. Meanwhile, people’s opinions can shift rapidly in response to political, social, economic, and population health need changes making repeat assessment essential.

The People’s Voice Survey (PVS) is a new tool designed by the Quality Evidence for Health System Transformation (QuEST) Network, an initiative focused on measuring and improving health system quality through multicountry partnerships. The PVS is a rapid population-representative survey that aims to inform action toward more effective and people-centered health systems and promote health system accountability to populations. It can assess public sentiment on factors like confidence and trust in the health system among the full adult population in each country, including recent users of health services as in patient experience surveys, but also past and future users. This allows policymakers to understand how well the health system is serving the entire population in addition to specific age or disease groups.

The PVS was collaboratively developed by researchers in the first wave of implementing countries, policymakers, and key regional health system stakeholders. It builds upon existing tools by measuring a wider range of domains of health system performance among full populations in a rapid, cross-nationally comparable way. The findings can inform national and multinational policy decisions and evaluation of health system programs to improve health care access and quality. Through the QuEST Network, the survey methods and findings will become freely accessible to promote policy change by governments and data uptake by health system researchers.

Survey features: Content, implementation, and validation

Content

The PVS is based on the definition of a high-quality health system proposed by the HQSS Commission in 2018. The Commission emphasized that high-quality health systems need to work with people not only to improve health outcomes, but to generate trust and economic benefit for all people. The PVS focuses on the elements of system performance that are most apparent to and valued by the population, including positive user experience and confidence in the health system [2,11,12]. Fig 1 shows the main components of the PVS framework.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. People’s Voice Survey framework.

Notes: People care about outcomes beyond good health, which include trusting that the system can meet their needs, confidence that they can afford services, and endorsement of health system performance. These perceptions are informed by processes of care, including system competence (e.g., whether the health system provides coordinated, easy-to-use care integrated across platforms), care competence (e.g., provision of high-quality care from knowledgeable, high-skilled providers), and user experience (e.g., good customer service and respect). These processes and outcomes are underpinned by the foundations of the health system, including health status, demographic characteristics, patient activation, and expectations of care.

https://doi.org/10.1371/journal.pmed.1004294.g001

The PVS tool focuses on questions for which people, rather than providers, clinical observations, or facility records, are the gold standard source of information. It assesses perceptions at multiple levels of the health system, including the micro level (e.g., the point of care), the meso level (e.g., use of local facilities), and the macro level (e.g., overall health system assessment) as outlined in Box 1. Where possible, we used questions that have been validated across multiple contexts.

Box 1. Domains of the People’s Voice Survey

1. Health and demographics
   1. 1.1. Demographic information
   2. 1.2. Health status
   3. 1.3. Patient activation
2. Utilization of care and system competence
   1. 2.1. Usual source of healthcare
   2. 2.2. Health service utilization patterns
   3. 2.3. Health system competence in population health
   4. 2.4. Non-use of healthcare
3. Care experience
   1. 3.1. User experience and care competence
   2. 3.2. Respondent endorsement of clinic
4. Health system confidence
   1. 4.1. Assessment of public primary care
   2. 4.2. Overall health system assessment
   3. 4.3. Expectations for health system quality
5. Optional targeted modules on priority issues (e.g., COVID-19)

In addition to standard demographic questions such as age, location, and health status, we include 2 items that measure patient activation, defined as patients’ willingness and ability to take independent actions to manage their health and care, to understand people’s autonomy and decision-making power [13]. To better understand health care use patterns, we ask about frequency and heterogeneity of facility visits over the last 12 months and how users rate the quality of those visits, including for telemedicine. We also include questions on perceived medical mistakes and whether the respondent did not obtain needed care.

For those who have used care in the last 12 months, we ask them to rate multiple domains of quality such as time spent with the provider and perceived provider knowledge and skills. We include both objective and subjective rankings to gauge what is acceptable to users in different contexts. We ask whether respondents would recommend the recently used facility to a friend or family member to calculate a net promoter score, an approach to assessing whether users are “promoters” or “detractors” of health care facilities [14].

The final section of the survey assesses overall confidence in the health system, including publicly provided primary care and respondent confidence in their ability to obtain and afford high-quality care if needed. Other items include assessment of the health system trajectory (i.e., is the health system getting better or worse?) and the need for reform of the system (i.e., does the health system need major or minor changes?). As user satisfaction is deeply intertwined with people’s expectations for care, we include 2 vignettes of care to help us adjust other quality ratings in the survey for local expectations and provide more accurate cross-national comparisons.

Implementation

The PVS is designed to be used in any country regardless of region, income level, or health system structure. Implementation features are described in Table 2. The target population is all individuals aged 18 years and older; in the future, the survey could be adapted to younger individuals. The survey includes both users and non-users of the health system because health systems should be equipped to serve everyone and, especially in settings where services are supported through public funds, all people should derive value from the health system. Further, non-users may be more likely than users to have issues with the current health system and may provide insight as to how the system might best serve them as potential future consumers of health care. The Wave 1 survey has thus far included 19 countries and was available in 34 languages (Fig 2 and Table A in S1 Appendix). Details on participating countries, ethics, and funding are in Text A in S1 Appendix.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Participating countries planned for the first wave of the People’s Voice Survey.

*In Argentina, the People’s Voice Survey was conducted in the Province of Mendoza only. Map shapefile available from the World Bank at: https://datacatalog.worldbank.org/search/dataset/0038272.

https://doi.org/10.1371/journal.pmed.1004294.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Implementation features of the People’s Voice Survey.

https://doi.org/10.1371/journal.pmed.1004294.t002

The PVS was primarily designed for telephone but is adaptable to mixed mode delivery, including in-person and web-based delivery, as required to reach the population of each country. When using telephone, we conducted interviews with the full sample of participants required in each country. Where telephone ownership was less than 80% of the population, as in Ethiopia and Kenya, we conducted an additional sample of in-person interviews to augment the full telephone sample. We also used computer-assisted web-interviewing (CAWI) when this was the most effective way to reach the population, as in the United States, United Kingdom, and South Korea.

The PVS aims to obtain population sentiment about performance of the health system by estimating population proportions agreeing with a range of statements. A survey of 1,000 individuals selected at random will produce an estimate that is within a 3% margin of error of the population proportion 95% of the time. This is the case when the prevalence is 50%; smaller numbers are needed when prevalence is higher or lower. Thus, we used a minimum sample of 1,000 in all countries. Several of our samples are larger than this to permit some stratified analysis (e.g., by urban/rural).

In each setting, detailed metadata were captured to describe the context and events surrounding data collection, such as elections, COVID-19 spikes, and major health system reforms. These data can be used to account for significant events or circumstances that may influence these cross-sectional data in dissemination efforts and to contextualize opinion data that may be perceived as unstable.

Findings will be disseminated through short fact sheets and summary briefs for policymakers and through longer survey reports that provide an in-depth look at methods and results from each survey wave. A publicly available metadata registry will contain contextual data for each wave of the survey in each country. Finally, aggregate data from surveys across countries will be available via dashboards on the QuEST Network website (www.questnetwork.org), where datasets will also be downloadable by the public.

Development and validation

QuEST Network researchers conducted a collaborative, multistage process to develop and validate the PVS (Fig 3). To guide this process, we assembled a diverse Global Development Group (GDG) comprised of 30 health system experts from 18 high-, middle-, and low-income countries. Development group researchers conducted a broad scoping review of recent survey literature and relevant survey tools to identify questions and response options used to measure key survey domains. The GDG generated survey aims and priorities, critically appraised survey domains and items for value, clarity, and relevance, and assessed survey construction. The group met regularly over the course of 18 months in an iterative process to help ensure content validity of the survey and co-produce a “draft zero” instrument.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Development and validation steps of the People’s Voice Survey.

https://doi.org/10.1371/journal.pmed.1004294.g003

The GDG sought external consultation on both methodology and content of the survey. This process included peer review by survey experts from the University of Michigan and content and measurement experts from academic and multilateral organizations, including the World Health Organization (WHO) and World Bank. Reviewers assessed for best practices in survey design and delivery, such as question formulation and flow, and completeness and relevance of survey content. This feedback informed subsequent drafts and bolstered content validity of the instrument.

In Wave 1 countries, the survey instrument underwent a collaborative adaptation by local researchers and/or policymakers to ensure the instrument would be locally applicable, interpretable, and comprehensive. This helped to maintain content equivalence by ensuring cross-national comparability, clarity, and local relevance of questions in each setting. As part of this process, we conducted cognitive interviews with eligible respondents in 11 countries. Results highlighted multiple areas for improvement, including cutting lengthy items and simplifying questions with high cognitive burden.

PVS implementers in each setting identified the minimum set of survey languages necessary to reach most of the national population based on recent, nationally representative surveys. When possible, we used a team translation approach; at minimum, translation and back-translation were performed by qualified translators and closely reviewed by QuEST researchers to ensure content equivalence across languages.

To test the survey in practice, we conducted a pre-test of the full survey among a 200 person sample of respondents from an online panel in the United States. We also conducted a pilot test in one or more languages in every country. Pre-testing and pilot surveys allowed us to assess factors such as survey length, coherence, local interpretability, and quality of the response data. Additional details on steps taken to validate and adapt the PVS for participating countries are available in Text B in S1 Appendix.

Pathways to policy

Data from the PVS will be used to answer research and policy questions to inform government practices and strengthen health system performance. Key indicators measured in the PVS are provided in Box 2. The survey will provide data that can spur health systems research, including on novel items that can inform best practices for future health system measurement, such as whether adjusting for population expectations is essential for accurately gauging health care quality ratings. PVS data are owned by collaborating researchers in the QuEST Network. As the PVS instrument and data are intended to be global public goods, they will be freely accessible to the public after a one-year embargo period to address national or regional research questions and deepen future inquiry into health systems performance.

Ideally, the PVS would be integrated as a routine component of health system planning. It can be incorporated into ongoing population surveys as an additional module or run in parallel to existing consumer surveys to provide complementary data. As a relatively low-cost, rapid survey, the PVS can be used for both ad hoc assessment at critical moments of health system evolution and longitudinal measurement of population sentiment. National governments can also use the PVS to plan for and monitor implementation of universal health coverage initiatives and inform health system design to better meet people’s needs and preferences. Multinational organizations and global partners can use the tool to monitor effectiveness of investments. Advocacy groups can measure community demand for health system improvements to better hold governments accountable. We have identified several specific policy uses for PVS in Wave 1 countries in Box 3.

The PVS will be available for use by national governments, multilateral organizations, and other institutions to bring a social voice to health system planning. While the survey is designed to work in any country regardless of income-level, location, or health system structure, implementation will require (1) adaptation of a limited number of questions to the local health system context; (2) translation to local languages; (3) obtaining relevant ethical approvals where applicable; (4) establishing in-house data collection or contracting a data collection partner and tailoring data collection strategies to the national context; and (5) data sharing with PVS partners.

The QuEST Network will update and maintain the PVS based on previous survey waves and support continued expansion of the PVS network of collaborators. We aim for the survey to be an integral part of routine assessment of health system performance in multiple countries to help policymakers make smart investments that are responsive to people’s needs.

Supporting information

Acknowledgments

The authors would like to sincerely thank the participants of the People’s Voice Survey Global Development Group: Patricia Akweongo, Brittany Blouin, Adama Faye, Georgiana Gordon-Strachan, Kaushalendra Kumar, Hwa-Young Lee, Borwornsom Leerapan, Suresh Mehata, Odet Sarabia, Abhishek Singh, Marius Ungureanu, and Susan (Luwen) Zhang. We also thank the many individuals who helped to test, adapt, translate, or implement the survey, including Juan Alonso, Patrizio Armeni, Catherine Arsenault, Wondimu Ayele, Amadou Diallo, Svetlana Doubova, Damen Haile Mariam, Devraj Jindal, Noel Joseph, Janette Karimi, Chaeun Kim, Meng Lou, Londiwe Mthethwa, Emily Odipo, Tarylee Reddy, Bibian Robert, Mercy Rop, Yonela Scina, Rosanna Tarricone, Jean Augustin Diégane Tine, Bhoikutso Tlou, and Hongmei Zhao.