Introduction

After the World Health Organization declared COVID-19 a global pandemic on March 11, 2020, those involved in pandemic response needed timely measurement of all-cause mortality (i.e., a count of all incident deaths, regardless of cause) as a broad measure of pandemic impact and trajectory [1,2]. Many countries used diagnostic testing, case reports, and counts of deaths among diagnosed cases, but these measures were not universally available, nor did they provide a full picture.

Many low- and middle-income countries had limited diagnostic testing early in the pandemic. When coupled with challenges consistently identifying confirmed and suspected cases, this left few options aside from attempting to measure total and excess mortality to assess the pandemic’s impact. “Total mortality” encompasses all deaths, from all causes, in a specified period (by age, sex, place of usual residence, and place of death), including deaths directly and indirectly related to the pandemic and unrelated deaths. For example, it includes deaths potentially attributable to pandemic-related disruptions in healthcare or behavior changes. Early in the pandemic, it became apparent that counting total mortality based on existing civil registration and vital statistics (CRVS) systems would be difficult for many countries. Data were sometimes not analyzed or produced at the desired frequency and most deaths were not registered in a timely way, risking loss to follow-up and registration.

In response, from 2020 through 2021, the Bloomberg Philanthropies Data for Health Initiative and Resolve to Save Lives (“the initiatives”) supported 13 countries in Latin America, Africa, and Asia with differing levels of CRVS digitalization and completeness to implement rapid mortality surveillance (RMS) (See Table 1 for details). This support included technical assistance to improve how countries rapidly record, retrieve, and analyze total mortality data using existing CRVS systems or other data sources when needed [1,3]. Assistance included a) guidance on measuring total mortality by age, sex, place of usual residence, and place of death, b) recommendations for governments to build on existing surveillance and CRVS systems, and c) a tool for calculating excess deaths associated with COVID-19 [4]. These components enabled standardized, streamlined mortality reporting to be shared daily, weekly, or monthly with authorities such as emergency operations centers.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Description of Rapid Mortality Surveillance by Country. Below is a table describing the characteristics of the RMS system in each country that was interviewed. “Level of Digitalization” refers to the extent that a data collection system is digitalized, ranging from a “Low” level where only a single individual component may be digitalized, to a “High” level characterized by a nearly end-to-end digital system. The last column includes the number of deaths recorded by national governments through systems built for rapid mortality surveillance of all-cause mortality with the characteristics outlined in this table.

https://doi.org/10.1371/journal.pone.0333157.t001

This internal process evaluation of RMS implementation sought to identify:

• Major issues of implementation and how challenges were addressed.
• Barriers and facilitators to establishing a sustainable RMS system.
• Degree of success in producing timely and useful data.

We are not aware of another multi-country process evaluation of establishing mortality surveillance systems under pandemic conditions, particularly in settings of varying CRVS system completeness and digitalization. While some countries were more successful than others in generating and using this data for pandemic response, our goal was to identify barriers and facilitators of success, with the hypothesis that when successfully established, RMS can provide timely and useful counts of deaths, even in settings that lack strong CRVS systems and with the competing demands of the pandemic. We sought to validate a hypothesis that, given key enablers, the timely collection and analysis of total mortality data would lead to informed public health action. This logic model is presented in Fig 1 along with the most common barriers encountered.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. RMS Logic Model.

https://doi.org/10.1371/journal.pone.0333157.g001

Methods

Our qualitative process evaluation used semi-structured interviews with key people to learn about their RMS implementation experiences in 13 countries: Bangladesh, Brazil, Burkina Faso, Colombia, Ethiopia, Ghana, Liberia, Malawi, Peru, Rwanda, Senegal, Sierra Leone, and Togo. We sought participation from government focal points in all supported countries. This resulted in 16 participants:

• 13 men and three women.
• 11 government officials, primarily representing CRVS and statistics agencies, who were involved in the day-to-day process of setting up and implementing RMS.
• Five Vital Strategies in-country coordinators or consultants seconded to government departments for RMS work.

Interviews were conducted August–October 2021 with: two people in each of three countries (n = 6) and one person in each of the other 10 countries (n = 10). For each country, clearances and approvals were obtained from appropriate national research ethics bodies. No remuneration or inducement was offered, and participants were informed they could stop the interviews at any time. All interview transcripts were de-identified prior to coding and stored in a password secured database.

Our evaluation encompassed data reporting, capture, and use, focusing on the RMS experience in countries where CRVS systems were not routinely capable of generating the data with the desired completeness and timeliness. Double-blind coding and qualitative analysis of interview transcripts using Dedoose (v 9.0.46) [5] surfaced themes that were categorized into key barriers and facilitators of RMS implementation.

It was not necessary or appropriate to have patients or the public involved in this research. To evaluate RMS implementation and achieve the desired goals of the research, it was necessary only to engage the staff who worked on this as part of their day-to-day responsibilities with their respective governments or public health agencies. The research was therefore also determined to be non-HSR based on the following “The proposed evaluation is not human subjects research because all information collected from respondents relates to the rapid mortality surveillance system in their country”..

Results

Five themes surfaced. To preserve the anonymity of sources, we refer only to the region of the respondent in citing quotations to demonstrate key points.

Discussion

This evaluation highlighted several themes.

• COVID-19’s Impact on Mortality Surveillance: After WHO declared COVID-19 a pandemic, timely mortality data became essential for understanding its impact. In countries with limited diagnostic capacity, total and excess mortality were crucial metrics. Yet not all countries were able to produce timely, representative measures of total or excess mortality, especially those launching surveillance during the pandemic.
• Government Buy-In: Successful RMS implementation depended on strong government ownership and commitment to using mortality data to guide pandemic response.
• Data Collection and Digitalization: In countries with well-established CRVS systems, RMS was integrated smoothly, while countries with less experience faced data gaps and delays.
• Interagency Collaboration: RMS relied on strong collaboration among government agencies. Data-sharing agreements and cross-checking between systems, such as health and CRVS, improved data accuracy and utility.
• Data Analysis and Use: Skilled personnel were crucial for data analysis, especially in triangulating data from multiple sources. RMS data informed public health measures such as lockdowns and vaccination strategies.
• Challenges and Improvements: Despite the successes, challenges persisted, including data incompleteness, reporting delays, and lack of coordination. This highlighted the need for sustained investments in CRVS systems and better integration with pandemic response.

This process evaluation, to our knowledge, represents the first comprehensive assessment of rapid mortality surveillance implementation during a pandemic, across countries, each with varying levels of CRVS systems. Our findings underscore the critical role of government ownership, digital infrastructure, and interagency collaboration to establish RMS systems. Countries with more advanced digitalization and data-sharing were more successful, underscoring the importance of technology as a driver of success.

Countries with robust, well-integrated CRVS systems were better equipped to use RMS to produce timely mortality data. Strong integration between digitalized CRVS systems and national health surveillance frameworks was notably linked to supportive government leadership, clear roles, and defined responsibilities. And governments used RMS data to inform public health decision-making, revealing these systems’ value, particularly in pandemic response.

We also uncovered significant gaps in national systems, particularly in CRVS frameworks, for recording incident deaths. Barriers to effective RMS implementation included delays in data collection and issues with data completeness, compounded by data-sharing challenges, especially in countries with less-developed CRVS systems. These findings highlight the urgent need for sustained investment in CRVS systems and emphasize the importance of establishing robust governance structures to support RMS.

Strengths and limitations

This evaluation has shown that countries appreciate the value of regular, routine analysis of death data to identify excess mortality early. Lockdowns in some countries disrupted vital statistics registration, rendering the collection of community-based notification and registration data impossible. Countries are keen to continue building and strengthening their mortality surveillance systems, integrating them into broader CRVS frameworks. Future government policies, public health strategies, and research should actively support these efforts.

This assessment also had some limitations. First, it may suffer from self-selection and self-reporting biases. Those who chose to participate in the interviews might have had strong views on RMS—either positive or negative—not fully representative of the governments’ perspectives. In terms of self-reporting, social desirability bias may have been at play. Second, our purposive sampling yielded a strong gender imbalance in the sample with 13 men and only 3 women. Lastly, interviews were conducted more than a year after the initial wave of COVID-19 during which many of the initial country RMS efforts were launched.

Conclusion

The evidence supporting our hypothesis that RMS can provide effective mortality counts in low- and middle-income countries, even those with weak CRVS systems, was mixed. There was a broad appreciation of the value of mortality surveillance, and countries with fairly complete and digitalized CRVS systems were able to provide RMS data to government and to the public. However, many countries, particularly in Africa, struggled to produce facility data in a timely, complete, representative or scaled manner.

Given this mixed picture, the experience of implementing RMS during the COVID-19 pandemic offers crucial lessons for future public health surveillance efforts. A well-established, government-owned and -led system is essential for strengthening long-term capacity, spotlighting health disparities, making evidence-based decisions on resource allocation, and predicting future needs based on current data, historical trends, and patterns. We hope that these insights will guide the development of more resilient and responsive systems to produce measures of total and excess mortality.

Acknowledgments

The authors thank the interviewees in the 13 countries for their time and insights; Victor Hugo Alvarez Castaño, Benjamin Clapham, Valens Nndagijimana Kagisha, Fatima Marinho, Raven Meade, Emmanuel Ngwakongnwi, Elizabeth Ortiz, and Moyeen Uddin, for assistance in study design and data collection and interpretation; Ellen Kuwana for assistance in drafting the manuscript; and Karen Schmidt for editorial support.