Rapid Diagnostic Tests for Neglected Infectious Diseases: Case Study Highlights Need for Customer Awareness and Postmarket Surveillance

Accurate diagnosis of infectious diseases is essential for appropriate targeting of treatment and disease control. Rapid diagnostic tests (RDTs) are quick and easy to perform, they give results during one clinic visit, and they can be used in settings with little infrastructure or trained personnel. RDTs are promising tools to improve diagnosis in remote or low-resource settings. In the field of neglected infectious diseases, new manufacturers, RDTs, and users are coming onto the scene [1, 2]. As access to RDTs improves, the need for quality assurance and postmarket surveillance increases. 
 
The International Medical Device Regulators Forum has formulated guidelines about quality assurance of medical devices, including RDTs, which have been adopted as regulatory standards in Australia, Canada, the European Union, Japan, and the United States [3]. Specific quality standards for in vitro diagnostic tests (IVDs) (ISO 13485:2003) and medical laboratories (ISO 15189:2012) have been published by the International Organisation for Standardisation [4, 5]. In less-regulated settings, the World Health Organisation (WHO) has stepped in to promote IVD quality [3, 6, 7]. 
 
Participation from various stakeholders is required to assure RDT quality. Manufacturers must ensure that their products are ready for the market—i.e., that product design, development, testing, manufacturing, packaging, and labelling meet the required standards of safety and performance. The role of RDT users is, among other responsibilities, to know indications, contraindications, and operating procedures of the devices. Most regulatory authorities recognise that efficient communication between manufacturers and users is key to postmarket surveillance [3–7]. In low-resource settings and in the field of neglected infectious diseases, this communication between manufacturers and users may be suboptimal, as well as the pre- and postmarketing oversight of national regulatory authorities. 
 
The Neglected Infectious Diseases dIAGnosis (NIDIAG) consortium aims to improve diagnostic approaches for different clinical syndromes in low-resource settings where neglected infectious diseases are prevalent. In this case study, we assessed several quality aspects of RDTs used in the NIDIAG study about persistent fever: we focused on RDT labelling and instructions for use (IFU) and on product-related incidents, including communication with manufacturers about these incidents.

infectious diseases are prevalent. In this case study, we assessed several quality aspects of RDTs used in the NIDIAG study about persistent fever: we focused on RDT labelling and instructions for use (IFU) and on product-related incidents, including communication with manufacturers about these incidents.

Methods
The NIDIAG study about persistent (!7 days) fever recruited 1,926 patients in four countries (Sudan, the Democratic Republic of the Congo, Nepal, and Cambodia). The study evaluated 11 RDTs from seven manufacturers targeting human African trypanosomiasis, leptospirosis, malaria, typhoid fever, and visceral leishmaniasis ( Table 1). These RDTs were selected by an expert committee based on their potential to improve prognosis of patients with persistent fever in the study areas. To be included as index tests, the RDTs had to target severe and treatable infectious diseases that cause persistent fever and they had to be ready for evaluation in phase III or phase IV diagnostic studies.
The assessment of RDT labelling and IFU was done by one trained person using a checklist that had been developed previously based on international regulatory documents, a review of the literature, field observations, and comments from manufacturers and implementers. This checklist is given as supporting information (S1 Table) [8]. The product-related incidents were reported by the laboratory technicians in the study sites. They had received on-the-job training to correctly manipulate and interpret the RDTs and to follow standard operating procedures in Rapid diagnostic test device consisting of only the nitrocellulose strip that is placed on or in a platform (tube, backing paper, etc.). c Test device consisting of a nitrocellulose strip that is dipped into a plastic housing with buffer and specimen wells. line with good clinical laboratory practice guidelines. The diagnostic accuracy (sensitivity and specificity) of the RDTs is not included in this report.
The following definitions were used. Test results were invalid if there was no control line. Migration was considered to be delayed if the sample/buffer mixture did not start to migrate immediately (this usually takes only a few seconds). Poor background clearing was defined as the presence of a red-coloured background in the result window of the test strip at the time of result interpretation.

Results
The results for adherence to recommended RDT labelling and IFU are shown in Table 2. Five of the 11 RDT products had a meaningful name (i.e., mentioning at least the target condition and the type of test). Ten products displayed internationally recognized symbols, but none of them provided a complete symbol key. Four products displayed all essential information on the box. The cassettes from three products and buffer bottles from five products were correctly labelled. Fig 1 shows cassettes and buffer bottles for leptospirosis and typhoid fever tests from the same manufacturer, which are indistinguishable from each other. Four RDT products included appropriately labelled lancets. The IFU of eight RDT products included all required information but were not sufficiently readable (font size too small, closed letter type, and/or too difficult phrasing precluding easy reading and understanding). In summary, none of the 11 RDTs complied with all the specifications.
We registered 15 quality-related incidents for five RDT products: one related to supply (insufficient buffer volume), three to shipment (lack of temperature monitoring or shipment at too high temperature), and one to storage (elevated temperature). The remaining 10 incidents were product related, seven of which were technical: change in buffer consistency (n = 1), delayed migration (n = 2) (Fig 2), high invalid test rate (n = 1), poor background clearing (n = 2), and dust in cassette packaging (n = 1). The three remaining incidents concerned changes in IFU: two changes related to sample volume and one to reading time. None of the concerned IFU displayed track changes or a revision history, which would allow quick and accurate identification of the changes. Only one IFU displayed a new version number, and only two IFU showed a different date of issue (Fig 3). The changes had not been communicated to the user by writing or by an alert on the box.
For six of the incidents, we contacted the manufacturers (n = 3). For four incidents, the manufacturers' reply was swift and correct: they provided information on changes in the IFU (n = 1) and on temperature stability (n = 1), and they performed corrective actions (n = 2, replacement of affected RDT kits, provision of additional buffer vials). For two incidents, the reply was inadequate. One reply was correct but delayed for 12 days (ascribed to summer holidays). For the second incident, the manufacturer replied that the validation of the new reading time should be done by us or that, alternatively, we could simply disregard the new IFU.
Overall, at least one quality problem was detected for each of the 11 RDTs. There was no clear overlap between different types of incidents: problems occurred with all RDT types and all manufacturers. To our knowledge, the observed shortcomings did not cause serious adverse events for participants (most of these RDTs were used as index tests and not to guide patient care), nor did they affect study data reliability.

Discussion
This case study raises the issue of quality and postmarket surveillance of RDTs for neglected infectious diseases, irrespective of their diagnostic performance. When RDTs are put under scrutiny, shortcomings appear to be common. Some of the problems also occur with CE- Table 2. Assessment of labelling and instructions for use of 11 rapid diagnostic tests used in the NIDIAG persistent fever study.

Criteria a Number of RDT products complying with the criteria Comment
The product has a meaningful name (mentioning at least the target condition and the type of test)

Five products from three manufacturers
For none of the RDT products, the items included in the kit were consistently labelled (different product name on RDT box, device packaging, and buffer bottle).
If internationally recognized symbols are used, a complete symbol key b is provided Zero products from zero manufacturers Internationally recognized symbols were displayed for ten products from six manufacturers. A symbol key was provided for four products from three manufacturers, but this was incomplete.
Labels and prints are durable (humidity-proof and wellfixed) Eight products from five manufacturers This criterion was not evaluated for one product because it did not display any labelling on the outer packaging.
The list of components is displayed on the box Four products from three manufacturers This criterion was not evaluated for one product because it did not display any labelling on the outer packaging.
Essential product information is printed on the box (product name, reference number, lot number, expiration date, storage temperature, number of tests included)

Six products from four manufacturers
This criterion was not evaluated for one product because it did not display any labelling on the outer packaging.One product displayed a different product name on the box than the one printed in the IFU. The remaining nine products displayed the same product name on the box as in the IFU.
The cassette is appropriately labelled (target condition and single printed reading legend) and has enough place to write patient identity information Three products from two manufacturers One RDT with dipstick format was not taken into account for this criterion.Seven products did not comply with the criterion:five products from four manufacturers did not display the target condition on the cassette (three of which did not display any information);-one had two reading legends instead of one-one did not have enough place to write patient identity information.
The buffer bottle is appropriately labelled (product name, lot number, and expiration date)

Five products from two manufacturers
In addition to product name, lot number, and expiration date, four products from one manufacturer also displayed manufacturer name, storage conditions, and buffer volume (which is correct practice).

If included, transfer devices are of the appropriate type Eight products from four manufacturers
Transfer devices were not included in two RDT kits.A glass capillary tube was included for one product, which is not recommended because of risk of cuts. One of the two malaria RDT products included an inverted cup, which is one of the recommended transfer devices for malaria RDTs [9]. The remaining RDT products included plastic pipettes and straws with an indication line (six products from four manufacturers) and loops (one product).For one product, the packaging displayed the volume of the transfer device and "single use." If included, lancets are appropriately labelled (lot number, expiration date, and a symbol and/or word for sterile product and single use)

Four products from two manufacturers
Lancets were not included in five RDT kits.
If included, alcohol swabs are appropriately labelled (lot number and expiration date) Two products from two manufacturers Five RDT kits did not include alcohol swabs.The remaining six RDT kits included alcohol swabs, which were easy to open (without need for scissors), and mentioned on the packaging the type and concentration of the alcohol used. Two of these also mentioned lot number and expiration date.
IFU are complete (full contents of the kit, the test procedure, correct interpretation of the test results, version number, and date of issue)

Eight products from six manufacturers
Instructions for use were not provided in paper format but through email for one RDT product, which is not practical, certainly not in low-resource settings. The criteria are based on a review of international regulatory documents and scientific literature, complemented with field observations and comments from manufacturers and implementers (S1 Table)  marked RDTs. This is not surprising because in the current European Union regulation, IVDs for neglected infectious diseases are not considered to be high risk, and consequently, CE marks can be obtained by so-called self-certification by the manufacturer [10]. Of note, only small investments would be required from the manufacturers to correct the shortcomings here described.
The most worrying shortcomings are the changes in reading time and sample volume without notification to the user. Such changes are categorised as substantial because they may influence RDT performance [11]. One of these incidents illustrates the failure of the manufacturer to correctly handle a customer complaint: the attempt to shift responsibility for the validation of the changed reading time towards the user and the recommendation to disregard the IFU are unacceptable. At the users' end, quality assurance systems should clearly include proactive monitoring of IFU changes. In line with this, the ISO 15189 guideline that has been developed for medical laboratories (end users of diagnostic devises) recommends "document control"i.e., the prospective monitoring, updating, and archiving of all documents that may vary in versions or time, including IFU [4].
Limitations of this case study include the relatively small number of RDTs evaluated and the fact that not all elements of RDT quality were assessed. Likewise, one might debate the relevance of the findings, as the reported shortcomings did not cause any apparent serious adverse events. However, one should note that the possible effects of these shortcomings actually were mitigated by the study-related standard operating procedures in case of the shortcomings in labelling and IFU and by quick and appropriate action of the study team in case of RDT-related incidents. In support of its relevance, this study assessed real-life RDT products, and the product-related shortcomings reported here were observed thanks to the quality system of the clinical study. We expect that in the daily practice of a remote or low-resource setting, these shortcomings would not have been detected nor reported and may affect RDT performance and impact patient outcome.
Similar quality problems have been reported for RDTs targeting other infectious diseases. For example, in a study of 42 different malaria RDTs, shortcomings were found at the level of devices, buffers, packaging, labelling, and IFU [12]. For malaria and other frequent infections such as HIV and hepatitis B and C, WHO has taken a leading role in quality assurance of IVDs through prequalification and guidance on postmarket surveillance [6,7,11]. In the case of RDTs for neglected infectious diseases, such formal and international quality assurance mechanisms are less strong [13]. Moreover, stakeholders in this field may be less coordinated and less informed than those in the fields of malaria or HIV, for which national control programmes, funding agencies, nongovernmental organizations, and product experts constitute influential stakeholders. In the context of RDTs for neglected infectious diseases, production volumes are low, commercial interests are poor, and manufacturers may have limited ISO competence. At the other end of the supply chain, users of such RDTs tend to be dispersed and poorly trained [14]. At the same time, the requirements for these RDTs are particularly high: they should be user-friendly, withstand extreme environmental conditions, and be valid in heterogeneous patient populations [10].
Based on this experience, we recommend proactive interest from IVD manufacturers in line with ISO 13485 guidelines. Likewise, users even in remote settings should be motivated and trained to take up their role as emancipated customers. This can be achieved through biomedical curricula, national health authorities, and regional professional and scientific organisations such as the African Society for Laboratory Medicine. In addition, stakeholders involved in selection, procurement, and implementation of IVDs should be aware about these quality issues. Finally, regulatory authorities could subject RDTs for moderate-risk infectious diseases to a more rigorous premarketing scrutiny [13].
Supporting Information S1