The authors have declared that no competing interests exist.
Conceived and designed the experiments: MHD MG FM CLC. Performed the experiments: MHD. Analyzed the data: MHD MG. Contributed reagents/materials/analysis tools: MG FM CLC. Wrote the paper: MHD MG. Review and editing of manuscript: MHD MG FM CLC.
Cholera, an ancient scourge, continues to inflict high rates of mortality today. The rising incidence of epidemics in areas of poor sanitation and crowding highlight the need for better epidemic prevention and early response. Such interventions require the availability of rapid and accurate diagnostic techniques to trigger timely response and mitigate the scale of the outbreak. The current gold standard of bacterial culture is inadequate for rapid diagnosis, highlighting the overarching neglect of field diagnostic needs. This paper was written to support the World Health Organisation's Global Task Force on Cholera Control mandated Cholera and diarrhoeal disease laboratory Network (CholdiNet) in devising a protocol for the validation of Rapid Diagnostic Tests (RDTs) for
Review of post-1990 peer-reviewed and grey literature on rapid diagnostic tests for
Since 1990, twenty four diagnostic tests have been developed for the detection of
Rising prevalence of cholera outbreaks highlights the need for accurate detection tools. Diagnosing cholera early at the onset of an epidemic, at field level, should allow for a more timely response and a quick containment of the spread and thus a diminished case load. Currently the gold standard to identify the bacteria, Vibrio cholerae, from patient samples remains reliant on lengthy bacterial cultures and an array of biochemical tests. Furthermore, the need for highly-skilled operators and numerous laboratory resources underline the inadequacy of sophisticated tests for use in remote locations. Research to develop more appropriate tools has largely focused on rapid diagnostic tests and attempts to simplify existing technologies. This is yet to deliver evidence-based appropriate tools to address the burden-of-disease cholera inflicts. In light of this neglect we have taken the first step, assessing developments in commercialised diagnostic tools, reviewing previous evaluations undertaken in the literature since 1990. In doing so, we highlight evaluation study parameters that could benefit from stringent standardisation, and identify five tests that show promise for use in field conditions. The authors recommend an indipendent assessment of these products, including technical improvements as required and production to trigger early detection of cholera epidemics.
Cholera is an infectious disease caused by the bacteria
Cholera inflicts a heavy economic burden in its endemic settings due to its rapid spread and ability to cause large epidemics
Recent trends suggest that the number of outbreaks of cholera will continue to increase in vulnerable areas in the future. As populations of poor countries continue to coalesce in mega-cities with poor sanitation and people move rapidly around the globe, new and more virulent strains of
To combat this threat, attention in clinical and public health circles has expanded to encompass efforts to improve sanitation and utilization of new vaccines to improving epidemic response through early detection. By detecting cholera outbreaks as early in their course as possible additional resources can be more rapidly brought to bear to mitigate the size, scope and duration of the outbreak and subsequent spread of the illness. In addition, using a cheap rapid diagnostic test on watery stool samples, under field conditions with limited resources holds great potential to aid cholera control efforts, not because empirical treatment or later treatment is less effective but rather because decreasing the numbers of cholera infections and aiding in epidemic-preventing surveillance will free up resources that should be directed towards the excessively complex task of fixing the underlying socio-economic and environmental factors that propagate the spread of cholera. This shifts the focus from response measures, which often arrive too late to halt the course of an epidemic.
With this approach in mind, the current gold standard for laboratory diagnosis of cholera becomes evidently inadequate due to lengthy culturing on selective growth media. Preliminary identification based on colony appearance on Thiosulfate Citrate Bile Salts Sucrose Agar (TCBS) is traditionally confirmed using an array of biochemical tests, taking a few days to confirm a case of cholera and requiring numerous laboratory resources
At the time of writing there is no consensus as to the most efficacious cholera diagnostic for early case detection. Very few diagnostic tests have been well described and throroughly evaluated
Systematic searches were conducted using PubMed, SCOPUS, EMBASE, LILACS, ScienceDirect, GoogleScholar, Medline Plus, and ResearchGATE. The following search strategy was used: All published English literature since 1990 focusing on diagnostic tools for human clinical samples of cholera, and the evaluation of diagnostic tests, using the initial terms “Cholera”[Mesh] OR “
At the analysis stage, the diagnostic technologies were described in terms of: their detection limit, their diagnostic target, the method used (microscopy, agglutination, ELISA, immunochromatography or PCR), turn around time, intended use and settings of use. The sensitivity and specificity of the tests were reported from laboratory evaluation, highlighting field studies (when performed). These field studies were then filtered for having a sample size greater than 100 stool samples and reporting Positive Predictive Value (PPV) and Negative Predictive Value (NPV) and presented in tabulated form according to the aformentioned values. Following this, a review of the evidence on the validity and reproducibility underpinning the rapid diagnostic tests (RDTs) was performed.
In January 2010, twenty four commercialised diagnostic tests were identified (
Since 1990 there have been eighteen laboratory and field evaluations of commercial diagnostic tools for the detection of
Rank | Product name | PPV |
1[18] (Arya) | COAT | 100 |
2[15] (Nato) | IP cholera dipstick | 95.6 |
3[15] (Nato) | IP cholera dipstick | 94.8 |
4[15] (Nato) | IP cholera dipstick | 86 |
5[14] (Kalluri) | SMART | 84 |
6[14] (Kalluri) | IP dipstick | 83 |
7[14] (Kalluri) | Medicos | 71 |
Rank | Product name | NPV |
1[15] (Nato) | IP cholera dipstick | 100 |
2[15] (Nato) | IP cholera dipstick | 98.6 |
3[18] (Arya) | COAT | 95 |
4[15] (Nato) | IP cholera dipstick | 93.3 |
5[14] (Kalluri) | Medicos | 90 |
6[14] (kalluri) | IP dipstick | 88 |
7[14] (kalluri) | SMART | 84 |
Rank | Product name | Value |
1[21] Colwell | Cholera Screen | 100 |
2[20] Hasan | Cholera SMART | 100 |
3[16] Hasan | Bengal Screen | 100 |
4[17] Islam | Cholera SMART | 100 |
5[19] Hasan | DFA COLTA | 100 |
6[18] Arya | COAT | 100 |
7[20] Hasan | Cholera SMART | 100 |
8[15] Nato | IP cholera dipstick | 96 |
9[14] Kalluri | SMART | 95 |
10[15] Nato | IP cholera dipstick | 92.5 |
11[16] Hassan | Bengal DFA | 89 |
12[13] Wang | IP O1 cholera dipstick | 89 |
13[15] Nato | IP cholera dipstick | 84 |
14[14] Kalluri | Medicos | 79 |
15[21] Colwell | Cholera Screen | 77.8 |
16[12] Harris | Crystal VC | 71–76 |
17[14] Kalluri | IP dipstick | 67 |
18[21] Colwell | Cholera Screen | 60 |
Rank | Product name | Value |
1[15] Nato | IP cholera dipstick | 100 |
2[16] Hassan | Bengal DFA | 100 |
3[21] Colwell | Cholera Screen | 100 |
4[20] Hasan | Cholera SMART | 100 |
5[19] Hasan | DFA COLTA | 100 |
6[17] Islam | Cholera SMART | 100 |
7[15] Nato | IP cholera dipstick | 98.5 |
8[21] Colwell | Cholera Screen | 98 |
9[12] Harris | Crystal VC | 97 |
10[20] Hasan | Cholera SMART | 95.6 |
11[13] Wang | IP O1 cholera dipstick | 95 |
12[16] Hasan | Bengal Screen | 95 |
13[15] Nato | IP cholera dipstick | 94.2 |
14[18] Arya | COAT | 92 |
15[21] Colwell | Cholera Screen | 85.7 |
16[14] Kalluri | Medicos | 84 |
17[14] Kalluri | IP dipstick | 83 |
18[14] Kalluri | SMART | 58 |
Alongside other diarrheal diseases, cholera is a major cause of preventable childhood and adult deaths in developing countries
On the background of such neglect, the authors found twenty four commercialised diagnostic tools for the detection of
PROBLEMS | PROPOSED SOLUTIONS | |
STUDY DESIGN | ||
1 | Small sample size• Limits validity and reproducibility | Larger studies are feasible |
2 | Variable sample type• Fresh vs. archives | Controversial |
3 | Gold standard• Uncertain standard case definition• Newer tests outperforming old | Recommend reviewing gold standard according to accuracy, cognisant of different uses for different contexts |
4 | Testing stools vs. purified cultures• Affects specificity | Stool samples should be used |
5 | Not approximating field conditions• Population descriptors are essential for PPV, NPV | Population descriptors should be reported for every population studied |
6 | Specificity• Limited data against common gut organisms | Specificity testing against confounding gut microflora should be considered prior to commercialisation |
TEST DESCRIPTORS | ||
7 | Little data available on stability• Importance of climate variations i.e. temperature, humidity on shelf life and accuracy | Manufacturers should perform stability studies (real time preferably) |
8 | Reporting of reproducibility• Between different users• As an inherent quality of the tools• Assessment of manufacturing and inter-lot variation | User reproducibility reported between at least three users and with sufficient number of samples |
SCIENTIFIC RIGOR | ||
9 | Conflation sensitivity with lowest detectable dilution | Scientific rigor should be a criteria for further inclusion and re-evaluation of test |
One implication of this critique of present evaluations is that even though it would appear at first that a particular test has the best PPV, NPV, specificity or sensitivity, the limitations of the evaluations means that no one diagnostic can be recommended without further improvement and standardized evaluation.
It is important to note that the scientific community remains divided regarding the intended use of rapid cholera tests, failing to provide any clear consensus about the requirements for such a rapid diagnostic test. In trying to formulate such requirements one must recognise the stark difference between individual diagnostic utility and early detection of epidemic or other public health roles. This preliminary step is critical as the characteristics of the tests are expected to differ in each scenario. In the case of PCR technologies, they currently have an important role in individual diagnostic confirmation but also play a role in outbreak detection in combination with RDTs through epidemiology and surveillance. PCR methods have shown to be advantageous at being particularly sensitive to small amounts of infectious bacteria and having the added value of characterising the strains of the epidemic during the same diagnostic investigation. The tests in
As stipulated in the DEEP guidelines, the study design is the first parameter to consider to ensure that an evaluation study will answer the intended question, be it a lab evaluation or field testing. NPV and PPV can only be derived from field testing when disease prevalence is known. It is therefore critical that studies analysing cholera diagnostic tests mimic the conditions the tests are designed to be used in. Among the rapid diagnostic test evaluations, only a limited number of studies were conducted in field conditions on fresh stool samples with provision of population descriptors. Such demographics may ensure that tests are really applicable to those who suffer from cholera. Therefore it is not unreasonable to suggest that current tests may encounter unexpected results when taken to the field. In the studies analyzed, recurrent conflation of the term ‘sensitivity’ to describe the lowest detectable dilution in the stools made it more difficult to carry out an accurate comparison of the tests. However, overall, some of the new rapid tests (Smart Q – 105 CFU/mL) are performing to the same detection standards as the PCR based tests (BAX - 104 CFU/mL).
A key study design parameter is the sample size of the investigation. A small sample size limits the validity of the evaluation and the reproducibility of the results. This was the case in most of the evaluations examined, suggesting a direct impact on their specificity and sensitivity measurements. Indeed many of the studies were found to have 100% sensitivity or specificity using approximately 100 participants. In spite of this caveat, it is important to recognize that due to the challenge of long-term preservation of cholera organisms in stools specimen sample archives are hard to come by and conducting the study on larger sample size of fresh faecal samples in a resource-poor setting is challenging. However, in the broader literature covered during the course of this analysis, four larger cohorts were found, each reviewing tests that have not yet to our knowledge become commercialised (hence being excluded from the final analysis)
Another point to consider is whether sample archives are ideal for a disease like cholera, and whether greater benefit may be gained from the acquisition and analysis of fresh samples. For example, if fecal material from children was under-represented in the study population, health workers may find that the tests are not as accurate or more prone to diagnostic errors. Such errors relate to our ability to infer test efficacy from the field data. Since positive predictive values and negative predictive values are dependent on the prevalence of the disease in the population, a factor that can differ between different age groups, the utility of these tests may differ for children. As children tend to experience greater mortality in association with severe cholera, it is paramount to diagnose the onset rapidly.
Studies lacked a consistent comparison to any one gold standard, many studies neglecting to undertake detection by both test and reference methods across the sample range to confirm the validity of the result. The variability in the use of sample panels and study populations underlined the lack of a standardised case definition. The absence of a clear benchmark for assessing the tests is made worse by the newer tests performing better than the gold standard, leaving no adequate comparator. Furthermore, some of the diagnostic tests were tested against purified cultures of cholera. This fails to assess how they would fare when used with real patient stool samples packed with gut micro flora and likely to have many other contaminating interactions. Therefore, if used, the results of this investigation are only valid for the use of the diagnostic after culturing, hence defeating the purpose of making these into rapid diagnostic tests.
Following the same principle, many of the tests appear to be rather specific on first glance, however only few were thoroughly tested against confounding microorganisms commonly found in stool samples which could lead to a similar case presentation of watery stools. Some studies went so far as to not utilise a negative control. Testing performance against a well characterised challenge panels is required.
Data showing robustness of the test design are important to support deployment in the areas of highest need. Test descriptors such as a test's reproducibility were markedly underreported; lacking rigorous replication important for assessing the inherent quality of the diagnostic tool, the reliability of different production lots and the ease of replication by different users. This also reflected the lack of important mention of test heat stability. A test's intended use is directly linked to analytical parameters such as heat stability. In the case of cholera much of its endemic settings are affected by harsh temperatures and humidity, factors which must be taken into account for the shelf life and accuracy of the test.
It is important to note that this is not a true systematic analysis, nor did the analysis review the entire literature, the authors selected to review only articles written in English from the year 1990 onwards. In spite of the aforementioned limitations of these studies (see
Caveats in the available information make it harder to make overarching comparisons between the different diagnostic tools, and so does the redundancy in product re-packaging under different names. The neglected nature of cholera in terms of the ratio of its burden to the relative attention it gets (funding, awareness, research) means that most of the diagnostic tools developed thus far have never progressed to mass production and can only be found in small batch numbers for a short period of time. This makes any future evaluation of these tools fraught with difficulties. Much of the information about the respective diagnostic tools has been provided by manufacturers in media releases and product information and is therefore often lacking in reliable information regarding the testing protocol. These are aspects that a future evaluation could address by carrying out independent testing of the products, facilitating an evidence-based revision of the diagnostic strategies for cholera. The authors recommend a prioritization of research and development agenda, and would like to see the five promising field evaluated products highlighted in this article (
Despite the limitations of this review, it is the first time a comprehensive picture of the market of diagnostic tests for cholera has been revealed. In doing so, many of the criteria for more thorough investigation have been elucidated and will likely be incorporated into future standard formulations of evaluation guidelines. The authors hope that the availability and use of the resulting reliable and user friendly rapid diagnostic tests will allow for the triggering of timely response to outbreaks and thus limit spread and burden of cholera.
24 Cholera diagnostics tests have been developed since 1990, 14 of which have been reviewed in 18 peer-reviewed analytical papers.
The majority of the tests were RDTs and PCR technologies, others include agglutination and direct fluorescence antibodies.
Overall the quality of peer-reviewed evaluations of diagnostic tests for cholera is problematic with issues raised in regards to sample size, sample types, gold standard, context suitability for the testing of rapid diagnostic tests and insufficient information provision for test descriptors.
DEEP guidelines propose a roadmap for standardised evaluation studies of diagnostic technologies and ought to be used to further evaluate the five RDTs that have shown promise under field conditions; COAT, IP cholera dipstick, SMART, IP dipstick and Medicos.
The limited research and funding in this field highlight the neglect of cholera and calls for increased coordination and prioritisation in the domain of R&D for cholera diagnostic tests to promote early epidemic prevention.
Banoo S, Bell D, Bossuyt P, Herring A, Mabey D and Poole F et al (2010). Evaluation of diagnostic tests for infectious diseases: general principles. Nature Reviews Microbiology; 8(12): S17–S29.
Bhuiyan NA, Qadri F, Faruque AS, Malek MA, Salam MA, et al. (2003). Use of dipsticks for rapid diagnosis of cholera caused by
Harris JR, Cavallaro EC, de Nobrega AA, Dos S Barrado JC, Bopp C, et al. (2009). Field evaluation of Crystal VC Rapid Dipstick test for cholera during a cholera outbreak in Guinea-Bissau. Tropical Medicine & International Health; 14(9): 1117–21.
Kalluri P, Naheed A, Rahman S, Ansaruzzaman M, Faruque AS, et al. (2006). Evaluation of three rapid diagnostic tests for cholera: does the skill level of the technician matter? Tropical Medicine & International Health; 11: 49–55.
Nato F, Boutonnier A, Rajerison M, Grosjean P, Dartevelle S, et al. (2003). One-step immunochromatographic dipstick tests for rapid detection of
(DOC)
(DOC)
(DOC)
For creating a fostering environment and closely facilitating this work, many thanks to the diagnostics group at the TDR, WHO, including; Sanne, Jane, Véronique, Jia Si, Izabela, Andy and Luis. We would also like to express our gratitude to A Prof Christine Phillips of the Australian National University (ANU), Academic Unit of General Practice and Community Health, for her valuable advice and help in reviewing this manuscript.