Citation: Caillet C, Vickers S, Zambrzycki S, Luangasanatip N, Vidhamaly V, Boutsamay K, et al. (2021) Multiphase evaluation of portable medicines quality screening devices. PLoS Negl Trop Dis 15(9): e0009287. https://doi.org/10.1371/journal.pntd.0009287
Editor: Paul J. Brindley, George Washington University School of Medicine and Health Sciences, UNITED STATES
Published: September 30, 2021
Copyright: © 2021 Caillet et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work has been co-funded by the Regional Malaria and other Communicable Disease Threats Trust Fund, which has been co-financed by the Government of Australia (Department of Foreign Affairs and Trade); the Government of Canada (Department of Foreign Affairs, Trade and Development); and the Government of the United Kingdom (Department for International Development). The grant (RETA 8763) was managed by the Asian Development Bank (https://www.adb.org/) and awarded to PNN. Additional support was provided by Wellcome Trust Grant N°202935/Z/16/Z (https://wellcome.org/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
Competing interests: The authors have declared that no competing interests exist.
Substandard and falsified (SF) medicines have important but neglected consequences including increased morbidity and mortality, economic losses, and diminished public confidence in health systems. SF antimicrobials, particularly those containing reduced quantities of active pharmaceutical ingredients (APIs), may also be key but overlooked drivers of antimicrobial resistance . Substandard medicines result from negligence and errors made during the manufacturing process by authorized manufacturers or degradation in supply chains. Falsified medicines are the result of criminal activity. Falsified medicines purport to be real, authorized medicines but are deliberately and fraudulently mislabeled with respect to their identity and/or source . Falsified medicines usually have packaging that are copies of a genuine product and may contain the APIs, although often at the incorrect amount, or, more commonly, they contain other API(s) or none at all. The term “falsified medicines,” adopted by the World Health Assembly in May 2017, references the public health issues of poor quality medicines rather than the term “counterfeit” that refers to trademark infringement. As countermeasures vary according to the type of “defect,” understanding the differences between the types of poor quality medicines is essential from a public health and regulatory perspective.
According to a recent report from the World Health Organization (WHO), approximately 10% of medicines circulating in low- and middle-income countries (LMICs) are either substandard or falsified . The issue appears to be of greater magnitude in LMICs than in wealthier countries [4–6].
The Coronavirus Disease 2019 (COVID-19) pandemic has heightened the risk for SF medicines reaching patients by disrupting pharmaceutical production and supply chains, impeding regulatory inspection, and causing great economic hardship, making it difficult to afford genuine medicines. A surge of SF medical product cases has been reported, especially in settings with already vulnerable supply chains [7–10]. In early April 2020, soon after chloroquine and its derivatives were widely publicized for their potential efficacy for COVID-19 treatment and/or prevention, WHO alerted about falsified versions of chloroquine in 5 countries, of which 4 were African LMICs . The Medicine Quality Monitoring Globe identified Google News reports of 2 English language newspaper articles about COVID-19 medical products quality issues in January 2020, rising to 576 in February 2021 [9,12]. As new medical products are developed for COVID-19 treatments and prevention, it is highly likely that the incidence of SF products will increase in our stressed pharmaceutical world.
Medicines Regulatory Authorities (MRAs) are the keystones for many of the interventions to prevent, detect, and respond to SF medicines. However, national MRA, international procurement agencies, and wholesaler/distributor medicine inspectors performing post-marketing surveillance (PMS) have to largely rely only on their own senses and knowledge to detect circulating SF medicines . Suspicious samples may be sent to formal chemical analysis laboratories for further advanced chromatographic assays such as high-performance liquid chromatography (HPLC) or liquid chromatography-mass spectrometry (LC-MS), but these assays are expensive, time-consuming, and not readily available in many countries. These complex assays and centralized laboratory testing lead to significant delay between collection of the suspicious medicine and confirmation of its poor quality, with its harm spreading unchecked in the interim. Rapid detection of SF medicines in the field is key to inform timely actions to prevent unsafe poor quality medicines from reaching patients. In 2018, the Member State Mechanism on SF medical products raised the improvement of detection as an important aspect in the fight against SF medicines at the World Health Assembly .
As an example of regulatory practice, in the Lao People’s Democratic Republic (Lao PDR, Laos), where much of the data were collected for the multiphase study described in this PLOS Neglected Tropical Diseases Collection, typically, inspection of medicines quality is conducted by medicine inspectors from the Bureau of Food and Drug Inspection (BFDI) within the Ministry of Health. Inspectors undertake routine inspection of pharmacies bi-annually, focusing on adherence to legislation (e.g., appropriate paperwork completion) and drug registration. Convenience sampling of certain medicines, especially antimalarials and antiretrovirals, is undertaken as part of specific vertical program projects supported by donors. In convenience sampling within a province, medicines are purchased from a selection of pharmacies in each district and brought back to a central location where they undergo initial screening using the Minilab . All samples that fail Minilab screening conducted in regional offices and a further 10% of those that pass are then sent to the national Food and Drug Quality Control Center (FDQCC) for pharmacopeial testing.
Over the last 2 decades, a diversity of portable devices have been developed to better equip medicine inspectors to detect suspect medicines, offering the potential for more objective analysis of medicines in the “field” [16,17]. These devices have a great breadth in underlying technology, outputs, limitations, cost, training requirements, and ease of use. They range from sophisticated and expensive handheld Raman and near-infrared devices to a portable thin-layer chromatography kit to single-use paper cards and cassettes. This plethora of devices holds great hope for empowering medicine inspectors, making their work more actionable and cost-effective and improving MRA capacity to protect patients from harmful SF medicines. The need for such innovation is emphasized by the fragile character of pharmaceutical supply systems, as highlighted by the consequences of the COVID-19 pandemic. However, there are enormous key gaps in the scientific evidence to inform national MRA of the optimal, cost-effective choice of device to detect and respond to SF medicines [13,17]. This includes the lack of independent comparative evaluation of the majority of devices particularly in field settings, the amount and nature of training required for accurate use, an assessment of where in the supply chain different devices are best employed, and the cost-effectiveness of introducing devices within PMS systems. These gaps of knowledge impede decisions on how to best use these portable devices [13,17].
In this PLOS Collection, “A multiphase evaluation of portable screening devices to assess medicines quality for national Medicines Regulatory Authorities,” we describe the results of the multiphase collaborative study conducted between 2016 and 2018 that was part of the Results for Malaria Elimination and Communicable Diseases Control (RECAP) under the Regional Malaria and Communicable Disease Trust Fund (RMTF) at the Asian Development Bank. The study aimed to evaluate the accuracy, utility, usability, and cost-effectiveness of different portable devices to identify SF medicines across a variety of essential anti-infective medicines commonly used in the Greater Mekong Sub-region (GMS) to treat malaria and bacterial infections. This project was conducted in parallel with the United States Pharmacopeia Technology Review program [13,18,19].
The Collection is composed of 5 articles. This Editorial represents the first of these, introducing the background and rationale for the Collection. Twelve devices were first evaluated in a laboratory setting (“laboratory evaluation”) to provide information on their performances to identify SF medicines and to select the most field-suitable devices for further preliminary evaluation of their utility/usability (“field evaluation” phase) by medicines inspectors (Fig 1). Using part of the data gathered in the laboratory and the field evaluations, a cost-effectiveness evaluation [“cost-effectiveness analysis” (CEA)] of the implementation of the 6 devices tested in the field evaluation for PMS in Laos was conducted. The fifth article discusses the results presented in the series, highlights the evidence gaps, and provides recommendations on the key aspects to consider in the implementation of portable devices and their main advantages/limitations.
$ The CoDI (developed by the United States CDC)—a device that uses laser absorption and fluorescence—could not be assessed because of intellectual property issues. A misunderstanding of the operational procedures of the CD3+—a device using UV-Vis-IR light to reveal differences in the packaging as compared to its genuine counterpart (developed by the US FDA)—led to potential significant bias in the performance results of the device during its evaluation. Those are thus not presented in this article. CDC, Centers for Disease Control and Prevention; CoDI, Counterfeit Detection Indicator; FDA, Food and Drug Administration.
The laboratory evaluation included 12 devices selected from a literature review and expert advice . Each device was evaluated on its ability to detect good and poor quality medicines, including field-collected samples (mainly from the GMS), and simulated SF samples produced in the laboratory. The devices’ performance, consumable needs, training requirements, and setup are reported. These results were utilized to select the most field-suitable devices for the field study.
In the field study, an evaluation pharmacy was constructed to resemble a Lao type 2 pharmacy to evaluate the utility and usability of devices. Inspectors were observed by members of the medicine quality research group while doing simulated pharmacy inspections. This allowed the recording of time taken to conduct evaluation pharmacy inspections with the devices and the observation of potential user errors when using the device in a real-life pharmacy setting. Inspectors were also observed while testing a predefined sample set of medicines (SSM) with the devices in an office setting, allowing more scrutiny by the observers. The GPHF-Minilab was also tested by the chemists in the laboratory evaluation, and by FDQCC technicians, already trained in Minilab use, at their laboratory, in line with its current routine use in Laos.
Using results of the above studies, the fourth article presents a CEA of implementing the 6 devices selected for the field phase experiments. Key criteria for each device such as cost per sample analysis, the sensitivity and specificity of each device, and experiment time were utilized to estimate the cost-effectiveness of using the devices under 2 scenarios of high and low prevalence of SF antimalarials in circulation. The results include a variety of device sampling strategies and multiway head-to-head comparisons.
In the fifth article, these 4 components are synthesized and discussed, and recommendations are made. This is based on the discussion in a multi-stakeholders meeting held in April 2018 in Vientiane. The meeting included 53 participants from MRAs from 7 Asian and African countries and international health and funding organizations. Hands-on sessions with devices and group discussions about the promises and pitfalls of the devices were held during this 2-day meeting. The advantages/disadvantages, cost-effectiveness, and optimal use of medicine quality screening devices in the medicine supply chains were discussed. Based on the results from this multiphase study, we list policy recommendations for MRAs and other institutions who wish to implement screening technologies, as well as gaps of scientific knowledge to be filled.
As far as we are aware, this multiphase study is the first collaborative independent investigation listing the advantages and disadvantages from diverse chemical, economical, and regulatory points of view. The overall study assesses and compares device accuracy, describes potential barriers, and evaluates the costs versus the benefits of the implementation of a wide diversity of portable medicine quality screening devices in a public health perspective. The series also highlights the difficulties and barriers to perform screening devices research and the important remaining gaps of scientific evidence.
This work has been a large multicountry effort, larger than we anticipated, and we are extremely grateful to the many who have made it possible. We are very grateful to the Government of the Lao PDR for their support, especially the Bureau of Food and Drug Inspection, the Food and Drug Department, the Food and Drug Quality Control Centre, and the University of Health Sciences. We are very grateful to the Directors and staff of Mahosot Hospital for allowing us to install the Evaluation Pharmacy in the hospital grounds and to Assoc. Prof Mayfong Mayxay for his advice. We are grateful for the support of the Infectious Diseases Data Observatory and the Centre for Tropical Medicine & Global Health of Oxford University who invested in the administration of the project and to the members of the Substandard and Falsified Medical Products of the World Health Organization, Geneva. We are very grateful for the discussions with the manufacturers and developers of the devices and to the members of the United States Pharmacopeial Convention Expert Panel on “Review of Surveillance and Screening Technologies for the Quality Assurance of Medicines.” We are very grateful for the technical assistance of Douglas Ball, Sonalini Khetrapal, and Susann Roth.
- 1. Newton PN, Caillet C. Guerin PJ. A link between poor quality antimalarials and malaria drug resistance?. 2016;14(6):531–3. pmid:27187060
- 2. SF Medical Products Group, Essential Medicines and Health Products WHO. WHO Member State Mechanism on Substandard/Spurious/Falsely-Labelled/Falsified/Counterfeit (SSFFC) Medical Products. In: Seventieth World Health Assembly [Internet]. Geneva, Switzerland; 2017. p. A70/23: 33–36. Available from: http://www.who.int/medicines/regulation/ssffc/A70_23-en1.pdf?ua=1.
- 3. World Health Organization. WHO Global Surveillance and Monitoring System for substandard and falsified medical products: executive summary. [Internet]. Geneva, Switzerland; 2017. Available from: https://www.who.int/medicines/regulation/ssffc/publications/gsms-report-sf/en/.
- 4. Tivura M, Asante I, van Wyk A, Gyaase S, Malik N, Mahama E, et al. Quality of Artemisinin-based Combination Therapy for malaria found in Ghanaian markets and public health implications of their use. BMC Pharmacol Toxicol. 2016;17(1):48. pmid:27788677
- 5. Wafula F, Dolinger A, Daniels B, Mwaura N, Bedoya G, Rogo K, et al. Examining the Quality of Medicines at Kenyan Healthcare Facilities: A Validation of an Alternative Post-Market Surveillance Model That Uses Standardized Patients. Drugs Real World Outcomes. 2016;4(1):53–63.
- 6. Kaur H, Clarke S, Lalani M, Phanouvong S, Guérin P, Mc Loughlin A, et al. Fake anti-malarials: start with the facts. Malar J. 2016;15(1):86. pmid:26873700
- 7. INTERPOL. Global operation sees a rise in fake medical products related to COVID-19. 2020 [cited 2020 Dec 10]. Available from: https://www.interpol.int/News-and-Events/News/2020/Global-operation-sees-a-rise-in-fake-medical-products-related-to-COVID-19.
- 8. Newton PN, Bond KC, Adeyeye M, Antignac M, Ashenef A, Awab GR, et al. COVID-19 and risks to the supply and quality of tests, drugs, and vaccines. Lancet Glob Health. 2020:e754–5.
- 9. MORU Tropical Health Network. Medicine quality [Internet]. [cited 2020 Aug 26]. Available from: https://www.tropmedres.ac/research-areas/medicine-quality.
- 10. Gnegel G, Hauk C, Neci R, Mutombo G, Nyaah F, Wistuba D, et al. Identification of falsified chloroquine tablets in Africa at the time of the COVID-19 pandemic. Am J Trop Med Hyg. 2020;103(1):73–6. pmid:32400349
- 11. World Health Organization. Falsified chloroquine products circulating in the WHO region of Africa [Internet]. 2020 [cited 2020 Aug 26]. Available from: https://www.who.int/news-room/detail/09-04-2020-medical-product-alert-n4-2020.
- 12. Infectious Diseases Data Observatory. Medicine Quality Monitoring Globe [Internet]. [cited 2020 Aug 26]. Available from: https://www.iddo.org/medicine-quality-monitoring-globe.
- 13. Roth L, Nalim A, Turesson B, Krech L. Global landscape assessment of screening technologies for medicine quality assurance: stakeholder perceptions and practices from ten countries. Glob Health. 2018;14(1):43. pmid:29695278
- 14. World Health Organization. Report of the Sixth Meeting of the Member State Mechanism on Substandard and Falsified Medical Products. Geneva; 2018.
- 15. Global Pharma Health Fund E.V. GPHF-Minilab—Fact Sheet [Internet]. [cited 2017 Jun 30]. p. Global Pharma Health Fund E.V. (n.d.). GPHF-Minilab. Available from: https://www.gphf.org/en/minilab/factsheet.htm.
- 16. Kovacs S, Hawes SE, Maley SN, Mosites E, Wong L, Stergachis A, et al. Technologies for detecting falsified and substandard drugs in low and middle-income countries. PLoS ONE. 2014;9(3):e90601/1-e90601/11, 11 pp. pmid:24671033
- 17. Vickers S, Bernier M, Zambrzycki S, Fernandez FM, Newton PN, Caillet C. Field detection devices for screening the quality of medicines: a systematic review. BMJ Glob Heal. 2018;3(4):e000725. pmid:30233826
- 18. Roth L, Biggs KB, Bempong DK. Substandard and falsified medicine screening technologies. AAPS Open. 2019;5(1):2.
- 19. United States Pharmacopeial Convention. Technology Review Program | USP [Internet]. Available from: https://www.usp.org/global-public-health/technology-review-program.