Introduction

Pharmacovigilance (PV) is also called the safety of medicines and helps to avoid further catastrophes [1]. The World Health Organization (WHO) recommends its member countries to devise an effective pharmacovigilance system to deal with the identification, reporting and monitoring of adverse drugs reactions (ADRs) [2]. The pharmacovigilance system is aimed to promote and protect public health by ensuring the availability of essential medicines in the market and reducing the burden of ADRs [3]. A national government is responsible for providing safe, effective, and quality medicines to its citizens. To monitor the safety of medicines, an adverse drug reaction (ADR) reporting system is required to be established [4]. Pharmacovigilance should also be promoted as a priority by the drug regulatory bodies [5]. The assessment of a pharmacovigilance system identifies its weaknesses and assists in remediating and improving the quality and quantity of ADR reports as well as other potentials for the improvement of PV systems [6].

In 2008, WHO and the University of Washington entered into a joint project for the development of a global pharmacovigilance strategy. As a first step of the project, the Uppsala Monitoring Center (UMC) was tasked to conduct the study of pharmacovigilance activities of public sector organizations. A questionnaire designed by the UMC in various languages including English, Spanish and French was administered to key informants in more than 100 countries. Almost 50% of the study participants responded to the questionnaire. The purpose of the project was to document the current pharmacovigilance situation of non-ICH countries and to submit a proposal to the Melinda and Bill Gates Foundation to fund global PV [7]. Wilbur, K. et al conducted a survey of 13 Arabic-speaking countries by utilizing the UMC assessment of Country PV Situation questionnaire. The results showed that most of the country’s PV systems are government-funded and 67% of systems facilitated spontaneous ADR submission [8].

WHO and partners also conducted a consultative process. This included face-to-face meetings of stakeholders and discussion on the proposed draft minimum requirements by the WHO advisory committee. The document outlined the minimum requirements for any national PV system. As a minimum, these requirements ensure that a PV system exists and is functioning properly [9].

To assess the structure, process and impact of PV systems there are three available globally recognized assessment tools, i.e., indicator-based pharmacovigilance assessment tool (IPAT) [10], WHO pharmacovigilance indicators [11] and WHO global benchmarking tool (GBT) vigilance module [4]. More than 50 countries have successfully implemented the IPAT to date [12–24]. WHO pharmacovigilance indicators have been used for the assessment of the pharmacovigilance systems of many countries [6,25–30]. By using the GBT, as of December 2024, WHO has listed 15 regulatory authorities at maturity level-3 and three at maturity level-4 [31]. Twenty-two authorities have been listed as transitional NRAs[32]. Korea’s Ministry of Food and Drug Safety (MFDS) [32], Singapore’s HAS and Swissmedic [33] have achieved the WLA status by maintaining a well-established vigilance function.

Each of the three tools serves to evaluate the functionality of national regulatory authorities (NRAs) in the context of national pharmacovigilance systems [4,10,11]. However, determining which tool is superior for NRA assessment is challenging. Although the quantity of mandatory indicators across all tools remains relatively consistent, distinctions arise in the acquisition of data through different core and complementary/supplementary indicators in the first two tools.

The present study aims to contribute to the advancement of pharmacovigilance systems through a systematic analysis and comparison of existing assessment tools. The primary objective of this study is to identify, evaluate and compare the characteristics and functionalities of different pharmacovigilance assessment tools. These tools are developed to assess the performance of pharmacovigilance systems worldwide.

Methodology

To achieve the objectives of the study, document analysis was conducted on the identified three tools, i.e., indicator based pharmacovigilance assessment tool (IPAT) [10], WHO pharmacovigilance indicators [11] and the vigilance module of the WHO global benchmarking tool (GBT) [4]. Document analysis was selected as it facilitates answering research questions by providing valuable insights and information that can shape the understanding of a topic or issue [34]. A similar approach was used in other studies as well [35–38]. The findings of the document analysis were summarized and organized to facilitate comparison and evaluation. The document analysis provided valuable insights into the purpose, objectives, and specific aspects of pharmacovigilance that the IPAT, WHO pv indicators and vigilance module of the WHO global benchmarking tool.

USAID’s indicator-based pharmacovigilance assessment tool, published in 2009, is commonly used to assess the function, capacity, and gaps of the current PV system. The assessment includes a document review, structured interviews using assessment questions, key informant interviews, and additional data from respondents. IPAT contains 43 indicators, out of which 26 are core indicators and 17 are supplementary. These indicators address five pharmacovigilance and medicine safety system components namely policy, law, and -regulation; systems, structures, and stakeholder coordination; signal generation and data management; risk assessment and evaluation; and risk management and communication. The indicators are further classified by “structure,” “process,” or “outcome” according to the product or result they measure. National regulatory authorities, public health programs (PHPs), hospitals and health facilities, the pharmaceutical industry and other concerned can use IPAT [10]. IPAT was the first to identify the need for active surveillance as part of risk evaluation in low- and middle-income countries.

The WHO published an indicator-based manual for the assessment of the PV system in 2015. The WHO has set up minimum requirements for a functional PV center and proposed indicators are based on these requirements. The manual does not replace the World Health Organization’s global benchmarking tool for NRAs. The manual provides indicators for the assessment of national regulatory authority and public health programs. There is a total of 63 indicators. Out of that 27 are core PV indicators including 10 structural, 9 processes, and 8 outcome or impact indicators, and 36 complimentary indicators comprise 11 structural, 13 processes, and 12 outcome or impact indicators. There are nine separate indicators for public health programs [11].

WHO uses a global benchmarking tool for the evaluation of the national regulatory system of medicines. A mapping of internal and external benchmarking tools led to the development of the unified WHO GBT in 2013. Vigilance is one of the modules of the GBT [4]. It contains 6 main indicators and 26 sub-indicators. There is no grouping of core or complementary/supplementary indicators. NRA’s assessment tool includes a subset of indicators from the WHO manual of pharmacovigilance indicators to assist in assessing pharmacovigilance as a delivery item of the NRA [4].

Pre-existing text and data from the three selected pharmacovigilance (PV) assessment tools were systematically collected, reviewed, and analyzed. These tools were examined with the aim of extracting, synthesizing, and comparing key indicators relevant to national PV systems’ performance and maturity. A comprehensive list of core and supplementary/complementary indicators was compiled and organized into thematic categories (see S1 Table in supporting information). This list was not limited solely to the indicators explicitly mentioned in the selected tools. Rather, it was informed by broader literature, including a recent systematic review that identified 77 health service indicators diverging from the WHO’s standard PV indicators [39]. A qualitative content analysis approach was employed to examine the structure, intent, and thematic focus of each tool. The comparative analysis criteria were developed iteratively through an initial exploratory review of the tools, followed by expert consultation and thematic coding. The extracted data were tabulated and analyzed using Microsoft Word.

All documents were in digital (PDF) format and were reviewed in their most recent available versions as of the time of analysis. The primary author conducted the initial data extraction from the full-text versions of each tool. The senior team members (co-author with regulatory experience in pharmacovigilance system strengthening) independently reviewed the extracted content for accuracy and completeness. The synthesized and categorized data were reviewed and approved by all co-authors, with final responsibility resting with the corresponding author.

Results

The results of the study revealed a diverse range of pharmacovigilance tools designed to assess different aspects of drug safety (S1 Table in supporting information). These tools ranged from simple reporting platforms to advanced studies on medicine safety. The PV and drug safety system of a country must meet all core indicators in order to be considered fully functioning, according to IPAT. WHO also considers the core indicators as most relevant, important and useful. Accordingly, Table 1 presents the number of core indicators for each component within each tool.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Number of core indicators for each component within each tool.

https://doi.org/10.1371/journal.pone.0327363.t001

According to the results, IPAT and WHO pharmacovigilance indicators both require a national pharmacovigilance center. GBT’s vigilance module does not contain a standalone indicator of the requirement for an NPC, but it does require an NPC as part of the national vigilance system. It is crucial for all frameworks to have legal provisions, regulations, and guidelines with varying amounts of core indicators and sub-indicators. Compared with WHO indicators, IPAT and GBT vigilance frameworks tend to cover these aspects more thoroughly. One framework recognizes NRAs as essential, namely WHO indicators. The funding of PV activities is recognized across all frameworks as a key indicator of commitment to drug safety.

Across all frameworks, adequate human resources and training are emphasized, with GBT vigilance presenting additional indicators. According to the WHO framework, PV should be incorporated into educational curricula. IPAT considers it supplementary, while GBT neither consider it core nor supplementary. Adverse drug reactions (ADRs) reporting forms and procedures for collecting and assessing these reactions are widely accepted, though specific indicators and subsets differ.

A national pharmacovigilance advisory committee, formed by NRAs, is required for each tool. Each framework includes a comprehensive risk assessment and evaluation as core indicators, although the number and type of sub-indicators may differ. Data management and signal detection are key components of robust pharmacovigilance practices, with all frameworks documenting core indicators. Drug information Centre is recognized by IPAT, while standardized operating procedures (SOPs) are recognized by IPAT and GBT. A core value of IPAT is the sampling of medicines for testing. Newsletters and other forms of communication are recognized as effective communication strategies by IPAT and GBT frameworks, but specifics differ while WHO indicators only specify the availability of newsletters.

A comparative analysis of pharmacovigilance system assessment tools includes the key components (Table 2).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. A comparative analysis of pharmacovigilance system assessment tools.

https://doi.org/10.1371/journal.pone.0327363.t002

Supplementary or complementary indicators for IPAT and WHO indicators are described in S2 Table in supporting information, while indicators for PHPs are detailed in S3 Table in supporting information.

Discussion

The three commonly used tools to evaluate NRAs in the context of national PV systems share a common goal to assess NRA functionality. The choice of tool depends on the specific needs and objectives of the NRA evaluation, as well as the context and available data. Various studies in low-and middle-income countries identified that medicines safety issues are linked to weak ADR reporting systems [40,41]. An absent or dysfunctional PV system causes regulatory actions to fail and medicine safety to be compromised [30].

A National Pharmacovigilance Center (NPC) is required by both IPAT and WHO pharmacovigilance indicators. GBT vigilance module does not separately provide an indicator of the requirement of an NPC, but it does require that the national vigilance system includes an NPC. An NPC of a member country of the WHO Programme for International Drug Monitoring (PIDM) is responsible for sharing reports with VigiBase [42]. It is the basis for national PV systems since every country is recommended to establish an NPC by the WHO [43].

Embedding pharmacovigilance into the curriculum of health professionals plays a critical role in preparing them for career challenges on issues of medicine safety [44], which is a core indicator of the WHO indicators. On the IPAT, pharmacovigilance a part of the curriculum of HCPs is considered a supplementary indicator, and forms a part of the assessment of the national pharmacovigilance system. It is worth mentioning that the vigilance module of GBT does not ask questions on the inclusion of PV into HCPs curriculum while evaluating the NRA. Healthcare providers have a role to play in increasing vaccine awareness in the population, according to a study [45]. A study argued that developing countries lack professionals who are capable of evaluating drug safety and managing risks [46]. Future pharmacists and healthcare professionals should be trained in PV and ADR reporting in order to instil a culture and practices associated with ADR reporting during their professional careers [46–48]. Some studies, by using the WHO PV indicators, evaluated the national PV systems by incorporating questions on PV in the HCPs curriculum [6,25,29,30].

The WHO pharmacovigilance indicators provide eight outcomes or impact indicators that measure the effectiveness of pharmacovigilance systems in different countries. These indicators require quantitative data. However, in low and middle-income countries (LMICs), the collection of data from hospitals can be challenging, hence it becomes difficult to collect specific data on the number of adverse drug reactions (ADRs)-related hospital admissions and deaths [49,50]. A study in Nigeria concluded that it was difficult to compute the process and outcomes indicators because of poor record keeping in all facilities [27] on LMICs utilizing the WHO indicators for pharmacovigilance may not have collected information for these indicators due to the difficulty of obtaining accurate and reliable data. On the other hand, it is relatively easy to collect information about how many signals have been sent and what regulatory actions have been taken by the NRA. This may explain why epidemiological data is less frequently required by WHO indicators. IPAT, on the other hand, does not require similar data, which makes it simpler to use.

One of the major challenges in calculating the costs of treatment of medicines-related illnesses, costs of hospitalization, and extension in hospital stay in LMICs is the lack of robust monitoring systems and mechanism to capture relevant data. A combination of advanced methodologies such as machine learning and the availability of large amounts of electronic healthcare data offers the potential for optimizing drug benefit-risk profiles in real-world environments [51]. A systematic review found that PV system performance was low in terms of both ‘process’ and ‘outcome’ indicators, which reflected the incapacity to collect and use local data to identify signs of drug-related problems [29].

Despite not being recognized by other tools, IPAT recognizes the drug information centre as an independent entity. By providing education on medicines and supporting pharmaceutical services, medicine information services can also help minimize medication errors [52]. The IPAT indicator assesses the quality of medicines in circulation in a particular country. Both WHO indicators and GBT lack this indicator. A sampling and testing program allow the government to confirm that the products on the market are compliant with its specifications [53].

The Erice declaration emphasizes the importance of communication on drug safety [54]. In order to ensure medicine safety, it is important to communicate risks related to real and potential safety issues [55]. Only one core indicator is provided by the WHO indicators in terms of core communication requirements, and that is the existence of a newsletter. In addition, a complementary indicator asks whether the pharmacovigilance center has communication facilities. The same indicator on IPAT is categorized as a core requirement.

It is the complementary indicators (S2 Table in supporting information) of WHO PV indicators that make the real difference with IPAT. For many LMICs, collecting data on complementary indicators is difficult because most of the studies only utilized WHO core indicators [6,26,27,30,56]. Data on medicines consumption and prescription, HCP’s knowledge of ADRs, congenital malformations caused by medicines, and average work or school days lost due to drug problems are complementary indicators. The cost savings associated with pharmacovigilance activities, the health budget impact associated with pharmacovigilance activities, the knowledge of patients about ADRs and the rationale for the use of medicines are all specific to WHO indicators. [27].

WHO PV indicators provide only nine pharmacovigilance indicators for assessing public health programs (PHPs), whereas IPAT provides 31. There are seven indicators on both tools which completely or partially seek similar information, while the others ask for different details. Researchers can use both tools separately to evaluate PHPs and obtain better insights by combining them. Currently, one study has combined the WHO indicators for PV and IPAT to assess the PHPs. Barry et al conducted a comparative assessment of the PV systems within the neglected tropical diseases programs in four East African countries including Tanzania, Kenya, Ethiopia, and. Rwanda. The study shows that the pharmacovigilance system in public health programs is neglected in EAST African countries. The authors utilized “the East African community harmonized PV indicators tool for PHPs” to interview the staff of the national NTD programs. This tool was derived from the WHO pharmacovigilance indicators and the IPAT [56]. Another study [57] also utilized the WHO-based indicators to assess the pharmacovigilance activities in the national HIV/AIDS, malaria, and tuberculosis control programs. However, the majority of the studies that evaluated the PHPs as part of national PV systems have utilized the IPAT [12–15,23,58].

In particular, two indicators on IPAT, namely, “number of medicine utilization reviews conducted in the last year and number of active surveillance activities currently ongoing or carried out in the last five years”, and two indicators on WHO manual, namely, “Number of medicine-related hospital admissions per 1000 individuals exposed to medicines in the public health programme in the previous year and Number of medicine-related deaths per 1000 individuals exposed to medicines in the public health programme in the previous year”, require studies that are cost and time-consuming. However, these studies are vital for introducing priority interventions to improve the PV system in PHPs.

Like IPAT, WHO’s PV indicators manual does not provide a separate module for the assessment of hospitals. However, several indicators in WHO’s tool can be selected for the assessment of hospitals. Opadeyi et al studied the status of PV in tertiary care hospitals in the South-South Zone of Nigeria by administering the modified WHO indicator-based tool [27].

Conclusion

This study highlights the diversity of pharmacovigilance tools available and the need for customization when selecting the tools to suit the specific requirements of a specific pharmacovigilance program. Pharmacovigilance is an evolving scientific field. It evolves as treatment modalities and medical practice evolves. A joint working group comprising experts mainly from WHO, UMC and regulators from high and low-income countries can be established to reach a harmonized PV tool for assessment of the national pharmacovigilance system. This collaboration aims to have a common PV tool for the assessment of the national PV system and to improve the safety and efficacy of medicine worldwide by sharing information, expertise, and best practices across different regions and economic settings.

Supporting information