Figures
Abstract
Background: Leptospirosis is a growing public health concern in many tropical and subtropical countries. However, its diagnosis is difficult because of non-specific symptoms and concurrent other endemic febrile diseases. In many regions, the laboratory diagnosis is not available due to a lack of preparedness and simple diagnostic assay or difficult access to reference laboratories. Yet, an early antibiotic treatment is decisive to the outcome. The need for Rapid Diagnostic Tests (RDTs) for bedside diagnosis of leptospirosis has been recognized. We developed a vertical flow immunochromatography strip RDT detecting anti-Leptospira human IgM and evaluated it in patients from New Caledonia, France, and French West Indies.
Methodology/Principal Findings: Whole killed Leptospira fainei cells were used as antigen for the test line and purified human IgM as the control line. The mobile phase was made of gold particles conjugated with goat anti-human IgM. Standards for Reporting of Diagnostic Accuracy criteria were used to assess the performance of this RDT. The Microscopic Agglutination Test (MAT) was used as the gold standard with a cut-off titer of ≥400. The sensitivity was 89.8% and the specificity 93.7%. Positive and negative Likelihood Ratios of 14.18 and 0.108 respectively, and a Diagnostic Odds Ratio of 130.737 confirmed its usefulness. This RDT had satisfactory reproducibility, repeatability, thermal tolerance and shelf-life. The comparison with MAT evidenced the earliness of the RDT to detect seroconversion. When compared with other RDT, the Vertical Flow RDT developed displayed good diagnostic performances.
Conclusions/Significance
This RDT might be used as a point of care diagnostic tool in limited resources countries. An evaluation in field conditions and in other epidemiological contexts should be considered to assess its validity over a wider range of serogroups or when facing different endemic pathogens. It might prove useful in endemic contexts or outbreak situations.
Author Summary
The major burden of leptospirosis happens in low-income populations from tropical or subtropical regions. Because of nonspecific symptoms in human leptospirosis, the biological confirmation is needed to ascertain the disease. However, this biological diagnosis relies on sophisticated and time-consuming techniques that are most often hardly (if ever) available to clinicians in peripheral health centers. Yet, the outcome of leptospirosis in humans largely depends on an early antibiotic treatment. Taken together, these factors highlight the need of rapid simple diagnostic tests for leptospirosis that could be used directly on the bedside even in remote health centers. In this study, we developed and evaluated a prototype point of care strip test for the serological diagnosis of human leptospirosis in New Caledonia, mainland France, and the French West Indies. The sensitivity was 89.8% [95% CI, 84.7–93.4] and the specificity 93.7% [95% CI, 89.65–96.2]. This easy, early and portable diagnostic test will be evaluated in other epidemiological conditions and under field conditions.
Citation: Goarant C, Bourhy P, D'Ortenzio E, Dartevelle S, Mauron C, Soupé-Gilbert M-E, et al. (2013) Sensitivity and Specificity of a New Vertical Flow Rapid Diagnostic Test for the Serodiagnosis of Human Leptospirosis. PLoS Negl Trop Dis 7(6): e2289. https://doi.org/10.1371/journal.pntd.0002289
Editor: Philippe Büscher, Institute of Tropical Medicine, Belgium
Received: October 29, 2012; Accepted: April 11, 2013; Published: June 27, 2013
Copyright: © 2013 Goarant et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study has been supported by the Pasteur International Network, the French Ministries for Research, the Institut de Veille Sanitaire (InVS), and for Overseas Territories, the Government of New-Caledonia. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal's policy and have the following conflicts: A patent application related to the subject of the present article has been filed and includes CG, PB, SD, MP, FN and SC.
Introduction
Leptospirosis is a bacterial disease of high incidence in many tropical and sub-tropical areas [1], [2]. Its clinical presentation is both highly variable and most often characterized by non-specific signs and symptoms; the complete triad of Weil's disease (hepatic failure, renal failure and hemorrhage) is recognized to account for less than one third of human cases [3], [4]. Most of the early signs and symptoms point to the so-called “acute fever of unknown origin” (FUO), a major diagnostic challenge in tropical and subtropical areas. Beside the epidemiological context and patient exposure history, to quickly diagnose and implement an appropriate treatment, an etiological investigation is necessary, especially in malaria, hantavirus, or viral hepatitis endemic regions or during influenza, chikungunya or dengue outbreaks. Leptospirosis is also reported to be an emerging or re-emerging disease in industrialized countries, with probable increasing impacts due to global warming and increasing travel-related cases [5], [6]. Unfortunately, the biological confirmation of leptospirosis is both tedious and rarely available in a timely manner. It notably requires sophisticated techniques that are most frequently available only in central reference laboratories [1]. These techniques are of prime importance in disease surveillance and epidemiological investigations but are inappropriate for early clinical care in peripheral health centers that support a major part of the leptospirosis burden. Additionally, an early and proper antibiotic treatment is known to be a key to rapid recovery and a major determinant of outcome in leptospirosis [3], [7], [8]. The need for portable rapid diagnostic tests (RDT) is striking and largely recognized to improve clinical management of leptospirosis patients, notably in remote dispensaries of tropical and subtropical regions. As an example, the major part of the leptospirosis burden in New Caledonia occurs away from the city and the central hospital of Noumea [9], [10].
In this study, we developed a vertical flow immunochromatographic RDT for the early diagnosis of leptospirosis to detect Anti-Leptospira human IgM and evaluated its sensitivity, specificity, reproducibility, repeatability, temperature stability and simulated shelf-life in the context of leptospirosis endemic (New-Caledonia and French West Indies) or non-endemic (mainland France) countries using clearly defined case definitions and the reference Microscopic Agglutination Test (MAT) results as the gold standard [4].
Methods
Antigen preparation
The antigen was prepared at the Institut Pasteur, National Reference Center for Leptospirosis (Paris, France) as follow: a culture of L. fainei serovar Hurstbridge strain BUT 6T at OD = 0.5 at 420 nm was killed by 0.2% formalin for 3 hours at ambient temperature, boiled for 45 minutes and its pH adjusted to 9.6. This preparation was kept at 4°C until use as an antigen for the Vertical Flow RDT and was also used for an in-house IgM ELISA at the National Reference Center [11](Bourhy et al., manuscript in revision).
Development and production of RDTs
The positive control line was made of purified human IgM (MP Biomedicals) at 2 mg/mL. Both control and test lines were sprayed as lines onto nitrocellulose membranes. Gold particles labelled with goat anti-human IgM (BBI International BA.GAHM40/X) adjusted to the concentration of OD520 nm = 3 were used as the capture mobile phase to construct our one-step vertical flow immunochromatography RDT, as previously described [12].
Two batches of RDT were produced at the Institut Pasteur in Paris (platform 5) and used in this study: batch numbers 110,211 and 120,511.
Case definitions and gold standard
Leptospirosis cases were defined as confirmed when a clinical and epidemiological suspicion was complemented by either a positive specific PCR evidencing genomes of pathogenic Leptospira spp. in the blood or urine of the patient, or when the MAT on acute and convalescent sera showed a seroconversion (from nil to a titer ≥400) or a significant seroascencion (at least a fourfold raise in titers) [13]. Because most of the serum specimens originated from leptospirosis endemic regions where a high MAT titer threshold is usually used, we adopted this ≥400 threshold throughout the study. Probable cases were defined as clinical and epidemiological suspicion together with a unique serum with a MAT titer ≥400. The panels of strains used for MAT were adapted to the local epidemiology of New Caledonia, mainland France and French West Indies [9], [14] and are provided in details in Table S1. High rates of agglutination of the serum with one particular strain are used to identify the presumptive serogroup of the infecting bacterium as described elsewhere [4].
Study area and selection of sera for the evaluation
All sera used in this study were addressed to the Institut Pasteur in Nouméa or Paris for diagnostic purpose and originated from patients from New Caledonia, mainland France or the French West Indies (Martinique and Guadeloupe). The workflow is summarized as a flowchart in Figure S1. These laboratories are territorial (New Caledonia) and national (France and West Indies) reference centers for leptospirosis and receive all (New Caledonia) or more than 75% (France and French West Indies) of leptospirosis diagnosis requests. Sera were stored at −20°C, selected according to case definitions, and then tested blindly with RDTs. To assess the sensitivity of the RDT, only MAT-positive sera from confirmed cases were used: we tested 187 confirmed leptospirosis cases sera with a MAT titer ≥400 (120 from New-Caledonia, 38 from mainland France, 29 from French West Indies). The specificity was assessed using 221 sera (142 from New-Caledonia, 79 from mainland France): 12 anti-Chikungunya virus IgM positive sera, 58 anti-dengue virus IgM positive sera from all 4 serotypes, 6 anti-hepatitis A virus total Ig positive sera, 7 rheumatoid factor positive sera, 25 syphilis (TPHA and VDRL) positive sera, one acute malaria serum and 112 sera from healthy blood donors (Platform ICAReB, Institut Pasteur). These 221 sera were then confirmed to be MAT-negative (titer<100) within the same week.
Evaluation of the test
Preliminary experiments determined dilution of sera between 1/100 and 1/800 in Phosphate Buffer Saline (PBS, pH 7.4) as suitable for testing with RDT. Because rapid tests are mostly to be used in endemic regions and similarly to the high MAT titer threshold, a 1/400 dilution of sera was used throughout the study. Briefly, Vertical Flow RDT strips were introduced into 200 µL diluted serum in 5 mL polystyrene tubes, for 15 minutes. The strips were then removed and placed on absorbent paper and read within 5 minutes. All results were recorded using a grading scale from 0 (no visible trace on test band) to 3+ (intensity of the test band equal to the intensity of the control band). The grading included a “weak” value for low but visible traces on the test band. Weak, 1+, 2+ and 3+ were then considered positive for further analysis and 0 was considered as negative.
Sensitivity and specificity
All analyses were run blindly: any person involved in one particular analysis had no access to the results of the other tests results from the same serum.
To assess the sensitivity of the RDT, only the first MAT-positive serum from each confirmed case was used. For specificity evaluation, all 221 negative control sera have been tested using MAT and were all negative (titer<100) (see Table 1).
Possible false negative results due to high levels of anti-Leptospira IgM (prozone phenomenon) was controlled using two positive sera with 25,600 and 51,200 MAT titers: serial two-fold dilutions of the sera (1/400 to 1/6,400) were tested and test band intensity of the Vertical Flow RDT were compared.
Temperature stability and accelerated aging method for shelf life
The strips were stored at 4°C in sealed aluminium foils with a silica gel bag to avoid exposition to humid conditions, and the test was performed at laboratory room temperature (20°–25°C in New Caledonia and Paris).
The predictive shelf life of the RDT was assessed by testing serial dilutions of a MAT-positive serum (MAT titer = 800, pointing to serogroup Icterohaemorrhagiae) twice per week over a period of 3 weeks exposure of the strips at 60°C. During this period, the positive control serum was kept at 4°C to avoid repeated freeze-thaw cycles. This accelerated stability method is equivalent to two years of actual storage time at 25°C [15], [16].
Reproducibility and repeatability
Several experiments were performed in New Caledonia to evaluate the the Vertical Flow RDT. For reproducibility and repeatability, four sera (3 positives and one negative) were tested blindly by three different operators on three different days; one serum was tested blindly 14 times by integrating 14 aliquots of the same specimen within 4 independent experiments and using two different batches of RDTs by the same operator; 177 sera from both confirmed and probable cases (28 negative and 149 positive samples) were read independently by two technicians, including 157 sera (16 negative and 141 positive samples) by three technicians. We additionally simulated tropical field conditions by performing the tests in parallel at 37°C in an incubator (simulating tropical conditions) and at laboratory temperature (standard condition)
Comparative kinetics of seroconversion using MAT and RDT
The earliness of IgM seroconversion using MAT or RDT was assessed on serial sera (day 2 to day 18 after the onset of symptoms) from 17 confirmed cases, based on the date of onset as declared by the patients.
The RDT was also tested on early sera from 99 patients who were tested positive by PCR but negative by MAT (titer = 0, 100 or 200).
Concordance between RDT and MAT on unique MAT positive sera (probable cases of leptospirosis)
One hundred and fifty MAT-positive sera from probable cases, including 124 sera from the IPNC collection and 26 from the French National Reference Centre, were tested using RDT.
Comparison with three commercial diagnostic assays
To compare the newly developed RDT with currently available techniques, we compared its performance on identical sera from New Caledonia. To assess the sensitivity, 72 MAT-positive sera from confirmed cases were randomly selected from the 118 New Caledonian control sera. For the specificity, 72 negative controls were randomly-selected, corresponding to 10 anti-Chikungunya virus IgM positive sera, 30 healthy blood donors, 11 anti-dengue virus IgM positive sera from all 4 serotypes, 6 anti-hepatitis A virus total Ig positive sera, 7 rheumatoid factor positive sera, 7 syphilis (TPHA and VDRL) positive sera, one acute malaria serum. The results using our RDT were compared with those obtained using two Elisa assays (Leptospira IgM ELISA, Panbio, Inverness Medical, QLD Australia, and SERION ELISA classic Leptospira IgG/IgM, Institut Virion/Serion GmbH, Germany) and one lateral flow IgM immunochromatography assay (Leptocheck, Zephyr Biomedicals, India). The Serion ELISA test was used together with the Rheumatoid Factor Absorbent as recommended by the manufacturer. All tests were made within a 5 day period. For calculations, the “uncertain” results of ELISA were considered as positive.
Statistical analysis
The evaluation of our RDT for the serodiagnosis of leptospirosis was performed according to the WHO Tropical Diseases Research Diagnostics Evaluation Expert Panel for the evaluation of diagnostic tests for infectious diseases [17].
Data were captured into Excel 2007 (Microsoft Corporation, Redmond, United States of America).
We calculated sensitivity (Se), specificity (Sp), positive and negative predictive values (PPV and NPV, respectively) of the RDT, using the reference MAT serology as the gold standard. The 95% confidence intervals (95% CI) were calculated using the Wilson's method. The variations of the PPV and NPV according to the prevalence of the disease were also plotted.
We also calculated likelihood ratios (LR). The positive LR (LR+ = Se/[1 - Sp]) indicates how many times a positive result is more likely to be observed in specimens with the target disorder than in those without the target disorder. The negative LR (LR− = [1 - Se]/Sp) indicates how many times a negative result is more likely to be observed in specimens with the target disorder than in those without the target disorder. The more accurate the test is, the more LR differs from 1. LR+ above 10 and LR− below 0.1 are considered convincing diagnostic evidence [18]. The 95% CIs were calculated for LR+ and LR− [19].
The diagnostic odds ratio (DOR) measures the test performance by combining the strengths of sensitivity and specificity, with the advantage of representing a single indicator of accuracy. These characteristics make the DOR particularly useful for comparing tests whenever the balance between false negative and false positive rates is not of immediate importance [20]. The DOR is defined as the ratio of the odds of positive test results in specimens with the target disorder relative to the odds of positive test results in specimens without the target disorder. It was calculated as follows:The DOR does not depend on prevalence and its value ranges from 0 to infinity, with higher values indicating better discriminatory test performance. The 95% CIs for DOR values were also calculated [21].
Samples, ethics statement
The IPNC and the French NRC are reference diagnostic laboratories for leptospirosis. In New Caledonia, leptospirosis is a notifiable disease. The serum samples used in this study were selected from the 2008–2001 IPNC and 2009–2011 French NRC collections of sera issued from routine diagnostic activities and as part of public health surveillance. This biobank of sera was declared to the French Ministry of Research (DC-2010-1222, Collections number 1 and 2). This study was part of a protocol approved by the Institut Pasteur (protocol # RBM2008-16) and the French Ministry for Education & Research (protocol # AC-2007-44). All sera were tested as anonymous samples. Negative sera from mainland France were provided by Platform ICAReB (Investigation Clinique et Acces aux Ressources Biologiques). The STARDT checklist is provided as Table S2.
Results
Evaluation of the test
The sera used in this study were from patients or donors of both sex and all age classes, being selected among leptospirosis suspicions (positive sera), other pathologies or blood donors (negative sera). All along the study and whatever the batch used, we observed no invalid test: all RDT displayed an intensely marked control line and very little to no background coloration. Out of the 187 gold standard positive sera tested, 168 were RDT positive, including 15 RDT with a test line intensity graded as “weak” (8.9% of positives). The putative serogroups of the 19 RDT negative sera were: Icterohaemorrhagiae (n = 12), Pyrogenes (n = 3), Australis (n = 2), Panama (n = 1) and one could not be determined due to co-agglutination of multiple serogroups.
Out of the 221 MAT-negative sera tested, 207 were RDT negative. All 14 RDT positive sera were graded “weak” and originated from 9 healthy blood donors and five patients positive for anti-dengue virus IgM.
The sensitivity and specificity of the RDT were therefore, respectively, Se = 89.8% [95% CI, 84.7–93.4] and Sp = 93.7% [95% CI, 89.65–96.2]. The Likelihood Ratios (LR) were therefore LR+ = 14.18 [95% CI, 8.52–23.56] and LR− = 0.11 [95%; 0.01–0.17]; and the Diagnostic Odds Ratio DOR of 130.74 [95% CI, 63,65–268,52].
The results are summarized in Table 1, positive and negative predictive values of our RDT according to prevalence are presented in Figure 1.
Values calculated using 187 positive and 221 negative serum specimens.
The absence of false negative due to prozone phenomenon was demonstrated using dilutions of sera with highest MAT titers: serial two-fold dilutions actually yielded test lines of decreasing intensities..
Temperature stability and accelerated aging method for shelf life
To simulate tropical conditions, the RDT results of 10 MAT-positive sera run at 37°C were compared and proved identical to those run at 25°C. For accelerated shelf-life evaluation, serial two-fold dilutions (from 1/400 to 1/12,800) of one MAT positive serum (titer 800) were tested twice a week for three weeks with RDT stored at 60°C. At day 1, the RDT was positive at a 6,400 dilution, and remained the same until day 17. At day 21, the 3,200 dilution was the last giving a positive test line (a one dilution decrease of the serum).
Reproducibility and repeatability
One serum tested 14 times with strips from the two different batches gave 14 similar results, including the grade of the test line.
Inter-readers variability was assessed by two independent operators on 177 sera (28 negative and 149 positive) of which 157 sera were read by three independent operators. These readings provided an excellent inter-operator agreement (>99%) in all cases but one: one weakly positive RDT from a probable case was rated “weak” by two operators but negative by the third one.
Inter-operator variability was also assessed using 4 sera (RDT graded from negative to 3+) blindly and independently tested on three different days by three different operators. Two operators provided perfectly concordant grading results on all three tests, the third one graded “weak” a negative serum once out of the three tests.
Comparative kinetics of MAT and RDT
Of 17 confirmed cases analysed (see Table 2), one patient (number 1) seroconverted for MAT at day 6 (pointing to Icterohaemorrhagiae) but remained negative for RDT; oppositely, 5 PCR-confirmed patients (numbers 2–6) were MAT negative whereas they were RDT-positive. For one of these patients (number 6), PCR and RDT tests were both positive at day 4 after onset of symptoms. Five patients (numbers 7–11) were positive for MAT and RDT on the same day (days 5–11 after the onset of symptoms); lastly, for 6 patients (numbers 12–17), the RDT was positive earlier than the MAT (day 3 to day 7). Out of these 6, four (numbers 12, 15, 16 and 17) had a positive blood PCR and RDT results on the same day (on days 5, 4, 7 and 3 respectively).
Similarly, in 16 out of 99 early sera from confirmed patients from New Caledonia, the RDT was positive whereas the MAT was still negative (6 out of 62 MAT negative) or displayed low titers (4 out of 21 with a MAT titer of 100; and 6 out of 16 with a MAT titer of 200).
Concordance between RDT and MAT in patients diagnosed as “probable cases”
Of 150 sera from probable cases of leptospirosis (unique sera with a MAT≥400), 109 gave a positive result using the RDT, corresponding to a concordance of 72.7% [65–79.1].
Out of these, 108 had a MAT>400, from which 81 (75% [66.1–82.2]) were RDT-positive, while 63 had a MAT titer >800, from which 53 (84.1% [73.2–91.1]) were RDT- positive.
Comparison of our RDT with three commercial diagnostic assays
The use of 72 gold standard positive (MAT≥400 from confirmed cases) and 72 negative (MAT<100) serum specimens selected randomly allowed a comparison of our RDT with three commercially available tests: two ELISA tests (from Serion (using the recommended RF absorbant) and from Panbio) and one IgM lateral flow immunochromatographic assay (Leptocheck). The results of these tests are detailed in Table 3. The IgM ELISA from Panbio had 100% specificity on these particular specimens together with the lowest sensitivity (75%). This 100% specificity does not allow the calculation of a Diagnostic Odds Ratio (DOR) that would however be very high. The ELISA test from Serion had both a good sensitivity (91.7%) and a good specificity (81.9%), therefore showing a good DOR of 49.9. Another rapid diagnostic test, namely Leptocheck (from Zephyr) had a very good sensitivity (91.2%) but a quite low specificity (52.8%), giving a DOR of 39.1. The Vertical Flow RDT we developed displayed a very good specificity (95.8%) and a good sensitivity (81.9%) and had therefore a very good DOR of 104.4. The corresponding curves of predictive values according to the prevalence of the two IgM rapid tests on these specimens are compared in the figure 2.
Positive and negative predictive values for the diagnosis of leptospirosis using the IgM Rapid Diagnostic Test evaluated in this study and a commercial lateral flow IgM assay were calculated using 72 positive and 72 negative serum specimens randomly selected from New Caledonian specimens.
Discussion
Serum samples from leptospirosis patients contain antibodies that become detectable approximately one week after the onset of symptoms [4]. MAT is a long used gold-standard method for the serological diagnosis of leptospirosis. This method relies on the detection of agglutinating antibodies (both IgM and IgG) against antigens that are live Leptospira strains corresponding to representative serogroups of epidemiological significance. However, many laboratories and hospitals do not have the facilities and expertise required to perform the MAT. Therefore, there is a growing use of qPCR for early diagnosis [22], providing the opportunity to rapidly confirm leptospirosis suspicions in acute phase sera. More simple and rapid serological diagnostic tests including ELISA-based assays detecting antibodies are also used. However, none of these techniques are prone to be implemented in health centers of endemic regions where the highest burden of leptospirosis occurs.
Pathogenic Leptospira strains are classified into 9 species and more than 200 serovars reflecting the structural heterogeneity in the carbohydrate component of the lipopolysaccharide. ELISA-based assays using crude whole-cell lysates of Leptospira strain (usually the saprophyte L. biflexa serovar Patoc strain Patoc 1) as antigens may not recognize the diversity of circulating strains and the sensitivity of these tests are generally poor [3], [4]. A major challenge is to discover antigens that are conserved across the major leptospiral strains. In this study, we tested a one-step vertical flow immunochromatography RDT coated with heat-killed L. fainei serovar Hurstbridge as an antigen. L. fainei belongs to the intermediate group of Leptospira [23] and, as such, may share common antigenic features with saprophytes and pathogens which constitute the two other phylogenetic groups in the genus Leptospira. In addition, a previous study has suggested that L. fainei serovar Hurstbridge may cross-react with different pathogenic serovars [24].
Rapid diagnostic tests should ideally be accurate, simple to use, relatively inexpensive, easy to interpret, stable under extreme conditions, with little or no processing, and give the results within less than 2 hours.
A proper evaluation of a diagnostic test has to face two major challenges: first, the samples used for validation must have a very well-defined status with regard to the diagnostic target of interest; second, the results of the test under evaluation must be compared with the results of the same samples characterized using a validated reference test defined as the “gold standard”. Our study evaluated the sensitivity and specificity of a new Vertical Flow RDT for the serological diagnosis of leptospirosis in endemic (New Caledonia and French West Indies) and non-endemic (mainland France) countries. We only used sera from confirmed leptospirosis cases [13] for this evaluation. Therefore, the positive samples for the evaluation of sensitivity were both gold standard positive (a MAT titer of at least 400) and from confirmed leptospirosis cases (either a positive PCR or a seroconversion from nil to ≥400 or a ≥4-fold rise in MAT titer in paired sera). This high MAT threshold was chosen because most of the specimens (149 out of 187) originated from endemic regions, where a similar MAT threshold is used for diagnosis and surveillance [4], [9]. Additionally, negative sera for specificity evaluation were tested blindly using the reference MAT and were only considered as true negatives if the MAT titer was below 100. These latter originated from both healthy volunteers and a selection of patients with pathologic conditions of relevance in endemic countries. Using this clearly defined case definition, sensitivity and specificity were assessed using a high dilution of sera (1/400) reflecting the high MAT threshold titer. The sensitivity and specificity of the new Vertical Flow RDT in these conditions were 89.8% and 93.7% respectively. These results compare and are slightly better than the ones reported by Smits et al. who reported a 85.8% sensitivity and a 93.6% specificity with another Vertical Flow RDT [25].
The performance of ELISA tests vary widely in terms of sensitivity and specificity. For example, a commercial IgM ELISA (Panbio) gave a sensitivity and specificity of 76% and 82% in northeast Thailand [26], 35% and 98% in Hawai [27], and 61% and 66% in Laos [28], respectively. Reported variations in diagnostic assay performance may reflect population-related differences such as past exposure to leptospirosis or environmental leptospires. This can also be attributable to differences in the choice of the case definition; MAT is usually used as the reference test [29].
To increase the statistical power of our evaluation, we included as many serum samples as possible, only including the earliest positive serum when serial samples were available. Because the patient population recruited through our laboratories represents all (New Caledonia) or around 80% (France and West Indies) of leptospirosis suspicions, our collection can be regarded as representative of the total patient population in these regions. These included sera from New Caledonia older than 3 years, stored frozen at −20°C. It is well recognized that the long term storage of serum specimens at −20°C and their freeze/thawings may result in a drop of IgM titers. Actually, the sensitivity was higher in sera stored for less than two years than in sera stored for more than two years (90.6% versus 81.5%). This may have resulted in a slight under-estimation of the sensitivity of this Vertical Flow RDT.
The RDT we developed reacts with IgM to at least serogroups Australis, Autumnalis, Ballum, Bataviae, Canicola, Cynopteri, Grippotyphosa, Hebdomadis Icterohaemorrhagiae, Panama, Pomona, Pyrogenes, Sejroe and Tarassovi, indicating that the assay reacts broadly with antibodies mounted against Leptospira strains circulating worldwide.
Probable leptospirosis cases were defined as cases with a leptospirosis-compatible clinical presentation but a unique serology with a MAT titer ≥400. We tested our Vertical Flow RDT using these unique sera from such cases. Interestingly, the proportion of Vertical Flow RDT -positive sera was significantly lower than the sensitivity of the test as determined using confirmed cases (69.4% versus 89.8%, χ2 = 9.08, p<0.01), suggesting a poorer positive predictive value of the RDT for patients classified as probable cases. Two main reasons might contribute in explaining this difference. First, our Vertical Flow RDT only detects IgMs whereas the MAT is known to detect both IgMs and IgGs. Because IgM titers are known to decline faster than IgG, some positive MAT results may reveal IgGs remaining from previous exposure to leptospires. MAT could therefore be less specific than IgM-specific assays to detect acute and recent leptospirosis. It is also well known that direct visual methods such as MAT (agglutination of bacteria using microscopy) are less sensitive than indirect amplified techniques such as ELISA or colloidal gold particles immunochromatography assays. The possibility of false positive MAT results was already observed in other contexts [30]. Actually, though MAT is the recognized reference technique for the serological diagnosis of leptospirosis, it also suffers some drawbacks and weaknesses. Some concerns about both its sensitivity and its specificity have been raised and discussed [30]–[36]. More recently, a mathematical modeling study again demonstrated the limitations of MAT as a gold standard [29]. Since no diagnostic assay is adequately sensitive and specific enough to diagnose all acute cases of leptospirosis, results should be confirmed by another method. Regarding sensitivity, the most widely recognized weakness of MAT is its low sensitivity in early acute phase sera. The ability to detect anti-Leptospira antibodies earlier in the course of the disease with specific IgM detection tests than with MAT has already been largely recognized [30], [33], [34], [37]–[41]. We also observed an earlier positivity of our IgM Vertical Flow RDT when compared to MAT in serial sera. In our study, 11 out of 17 leptospirosis seroconversions could be diagnosed earlier with the Vertical Flow RDT than with the MAT (Table 2). Similarly, in 16 out of 99 early sera from confirmed cases, the Vertical Flow RDT was positive before the seroascension (10/37) or seroconversion (6/62) could be evidenced with the MAT. However, for strictness reasons, our strategy was to only use gold standard-positive specimens (MAT titers ≥400) for sensitivity assessment. Therefore, the sensitivity evaluated here might not reflect the conditions of rapid tests use in routine medical conditions, where patients may be seen and tested before seroconversion.
Previous studies have generally found that ELISA-based assays detect anti-Leptospira antibodies earlier in the course of the disease than with MAT [4], [30], [33], [37], [38]. Anti-leptospiral IgM cannot be detectable before 4–5 days after onset of symptoms, before the appearance of IgG and agglutinating antibodies [42].
Because an early diagnosis is of prime importance in the clinical management of leptospirosis [43], the possibility to ascertain the disease earlier in the course of the infection should be regarded as a real asset.
When considering the need of RDT for bedside diagnosis, the comparison of our Vertical Flow RDT with a RDT that is commercially available shows that our test has a lower sensitivity (81.9% versus 97.2%) but a much higher specificity (95.8% versus 52.8%) and therefore a better Diagnostic Odds Ratio (104.4 versus 39.1). This better performance is also shown by the comparison of the curves of their predictive values according to prevalence (Figure 2). There may be various reasons for these differences, including a different antigen used for IgM detection. However, the most probable cause is a much lower dilution of serum specimens in the Leptocheck lateral flow IgM assay (ca. 1/20) when compared with the dilution used for our Vertical Flow RDT (1/400). Because the MAT positive threshold titers may vary depending on the region [44], endemic countries usually using a higher threshold, it might be worth also considering using a different serum dilution for RDT according to the local epidemiology of leptospirosis. Because leptospirosis is endemic in New Caledonia and French West Indies, we decided to consider both a MAT titer of 400 as a gold standard and a 1/400 dilution of the serum for the Vertical Flow RDT evaluation. It is highly probable that the use of a lower dilution (1/200 or 1/100) would result in both an increased sensitivity and a decreased specificity.
Our results demonstrate that this new rapid diagnostic test could prove useful in endemic contexts, especially in low and middle-income countries. Actually, most of the leptospirosis burden occurs in the back-country with delayed access to the reference laboratories. In epidemics situations, especially during post-disaster periods like in the Philippines in 2009 [45], reference diagnostic tests are seldom if ever available. Therefore, a RDT with good diagnostic performances would also be particularly useful [1]. For easier use, further development of the technique could allow its use with capillary blood.
However, because an early initiation of antibiotherapy is a major contributor to a rapid recovery, the recommendation of treating the patient on the sole basis of a clinical and epidemiological suspicion should be maintained. The use of this Vertical Flow RDT as an initial screen for leptospiral infections would still allow facilitating the difficult differential diagnosis of leptospirosis [46]. Lastly, because the MAT provides important epidemiological information at the population level, it should still be recommended that sera be sent to the reference laboratory for subsequent confirmation by MAT, as suggested by other authors [41].
Supporting Information
Table S1.
MAT Panels used at Institut Pasteur in New Caledonia and at French National Reference Center at Institut Pasteur in Paris.
https://doi.org/10.1371/journal.pntd.0002289.s002
(DOCX)
Acknowledgments
The authors would like to thank Catherine Werts for sharing purified leptospiral LPS for preliminary trials. Thanks are also due to Stéphanie Laffont, Sophie Andruet, Lionel Lazzarotto, Sylvie Brémont, Farida Zinini and Annie Landier for MAT analyses.
Author Contributions
Conceived and designed the experiments: CG PB FN SC SD. Performed the experiments: PB SD CM MESG LBO FN. Analyzed the data: CG PB EDO MP FN SC. Contributed reagents/materials/analysis tools: ACG. Wrote the paper: CG EDO MP SC. Additional manuscript editing and corrections: PB MESG FN.
References
- 1. Hartskeerl RA, Collares-Pereira M, Ellis WA (2011) Emergence, control and re-emerging leptospirosis: dynamics of infection in the changing world. Clin Microbiol Infect 17: 494–501.
- 2. Adler B, de la Pena Moctezuma A (2009) Leptospira and leptospirosis. Vet Microbiol 140: 287–296.
- 3. McBride AJ, Athanazio DA, Reis MG, Ko AI (2005) Leptospirosis. Curr Opin Infect Dis 18: 376–386.
- 4. Levett PN (2001) Leptospirosis. Clinical Microbiology Reviews 14: 296–326.
- 5. Lau C, Smythe L, Weinstein P (2010) Leptospirosis: an emerging disease in travellers. Travel Med Infect Dis 8: 33–39.
- 6. Leshem E, Segal G, Barnea A, Yitzhaki S, Ostfeld I, et al. (2010) Travel-related leptospirosis in Israel: a nationwide study. Am J Trop Med Hyg 82: 459–463.
- 7. Suputtamongkol Y, Pongtavornpinyo W, Lubell Y, Suttinont C, Hoontrakul S, et al. (2010) Strategies for Diagnosis and Treatment of Suspected Leptospirosis: A Cost-Benefit Analysis. PLoS Negl Trop Dis 4: e610.
- 8.
World Health Organization WHO, International Leptospirosis Society ILS (2003) Human Leptospirosis: guidance for diagnosis, surveillance and control; Society WHO-IL, editor: World Health Organization. 122 p.
- 9. Berlioz-Arthaud A, Kiedrzynski T, Singh N, Yvon JF, Roualen G, et al. (2007) Multicentre survey of incidence and public health impact of leptospirosis in the Western Pacific. Trans R Soc Trop Med Hyg 101: 714–721.
- 10. Goarant C, Laumond-Barny S, Perez J, Vernel-Pauillac F, Chanteau S, et al. (2009) Outbreak of leptospirosis in New Caledonia: diagnosis issues and burden of disease. Tropical Medicine and International Health 14: 926–929.
- 11.
Bourhy P, Nato F, Chanteau S, Goarant C, Dartevelle S, et al.. (2012) European Patent EP123062150: Use of Leptospira fainei serovar Hurstridge bacteria for diagnosing leptospirosis. In: European Patent Office, editor.
- 12. Chanteau S, Dartevelle S, Mahamane AE, Djibo S, Boisier P, et al. (2006) New rapid diagnostic tests for Neisseria meningitidis serogroups A, W135, C, and Y. PLoS Med 3: e337.
- 13.
World Health Organization WHO (1999) WHO recommended standards and strategies for surveillance prevention and control of communicable diseases: Leptospirosis: World Health Organization, W.H.O.
- 14. Hochedez P, Rosine J, Theodose R, Abel S, Bourhy P, et al. (2011) Outbreak of leptospirosis after a race in the tropical forest of martinique. Am J Trop Med Hyg 84: 621–626.
- 15. Kirkwood TB (1977) Predicting the stability of biological standards and products. Biometrics 33: 736–742.
- 16. Paek SH, Lee SH, Cho JH, Kim YS (2000) Development of rapid one-step immunochromatographic assay. Methods 22: 53–60.
- 17. Banoo S, Bell D, Bossuyt P, Herring A, Mabey D, et al. (2010) Evaluation of diagnostic tests for infectious diseases: general principles. Nat Rev Microbiol 8: S17–29.
- 18. Jaeschke R, Guyatt GH, Sackett DL (1994) Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 271: 703–707.
- 19. Simel DL, Samsa GP, Matchar DB (1991) Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol 44: 763–770.
- 20. Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM (2003) The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol 56: 1129–1135.
- 21.
Armitage P, Berry G (1994) Statistical methods in medical research. 3rd edition. Boston: Blackwell Scientific Publications.
- 22. Perez J, Goarant C (2010) Rapid Leptospira identification by direct sequencing of the diagnostic PCR products in New Caledonia. BMC Microbiol 10: 325.
- 23. Perolat P, Chappel RJ, Adler B, Baranton G, Bulach DM, et al. (1998) Leptospira fainei sp. nov., isolated from pigs in Australia. International Journal of Systematic Bacteriology 48: 851–858.
- 24. Chappel RJ, Khalik DA, Adler B, Bulach DM, Faine S, et al. (1998) Serological titres to Leptospira fainei serovar hurstbridge in human sera in Australia. Epidemiol Infect 121: 473–475.
- 25. Smits HL, Eapen CK, Sugathan S, Kuriakose M, Gasem MH, et al. (2001) Lateral-flow assay for rapid serodiagnosis of human leptospirosis. Clin Diagn Lab Immunol 8: 166–169.
- 26. Desakorn V, Wuthiekanun V, Thanachartwet V, Sahassananda D, Chierakul W, et al. (2012) Accuracy of a Commercial IgM ELISA for the Diagnosis of Human Leptospirosis in Thailand. Am J Trop Med Hyg 86: 524–527.
- 27. Effler PV, Bogard AK, Domen HY, Katz AR, Higa HY, et al. (2002) Evaluation of eight rapid screening tests for acute leptospirosis in Hawaii. J Clin Microbiol 40: 1464–1469.
- 28. Blacksell SD, Smythe L, Phetsouvanh R, Dohnt M, Hartskeerl R, et al. (2006) Limited diagnostic capacities of two commercial assays for the detection of Leptospira immunoglobulin M antibodies in Laos. Clin Vaccine Immunol 13: 1166–1169.
- 29. Limmathurotsakul D, Turner EL, Wuthiekanun V, Thaipadungpanit J, Suputtamongkol Y, et al. (2012) Fool's gold: why imperfect reference tests are undermining the evaluation of novel diagnostics. A re-evaluation of five diagnostic tests for leptospirosis. Clin Infect Dis 55: 322–31.
- 30. Bajani MD, Ashford DA, Bragg SL, Woods CW, Aye T, et al. (2003) Evaluation of four commercially available rapid serologic tests for diagnosis of leptospirosis. J Clin Microbiol 41: 803–809.
- 31. Levett PN (2003) Usefulness of serologic analysis as a predictor of the infecting serovar in patients with severe leptospirosis. Clin Infect Dis 36: 447–452.
- 32. Murray CK, Gray MR, Mende K, Parker TM, Samir A, et al. (2011) Use of patient-specific Leptospira isolates in the diagnosis of leptospirosis employing microscopic agglutination testing (MAT). Trans R Soc Trop Med Hyg 105: 209–213.
- 33. Cumberland P, Everard CO, Levett PN (1999) Assessment of the efficacy of an IgM-elisa and microscopic agglutination test (MAT) in the diagnosis of acute leptospirosis. Am J Trop Med Hyg 61: 731–734.
- 34. Brandao AP, Camargo ED, da Silva ED, Silva MV, Abrao RV (1998) Macroscopic agglutination test for rapid diagnosis of human leptospirosis. J Clin Microbiol 36: 3138–3142.
- 35. Goris MGA, Leeflang MMG, Boer KR, Goeijenbier M, Van Gorp ECM, et al. (2012) Establishment of valid laboratory case definition for human leptospirosis. J Bacteriol Parasitol 3: 132.
- 36. Smythe LD, Wuthiekanun V, Chierakul W, Suputtamongkol Y, Tiengrim S, et al. (2009) The microscopic agglutination test (MAT) is an unreliable predictor of infecting Leptospira serovar in Thailand. Am J Trop Med Hyg 81: 695–697.
- 37. Aviat F, Rochereau-Roulet S, Branger C, Estavoyer JM, Chatrenet B, et al. (2010) Synthetic peptide issued from Hap1/LipL32 for new early serodiagnosis of human leptospirosis. Comp Immunol Microbiol Infect Dis 33: 375–387.
- 38. Doungchawee G, Kositanont U, Niwetpathomwat A, Inwisai T, Sagarasaeranee P, et al. (2008) Early Diagnosis of Leptospirosis by IgM Immunoblot Testing. Clin Vaccine Immunol 15: 492–498.
- 39. Chirathaworn C, Kaewopas Y, Poovorawan Y, Suwancharoen D (2007) Comparison of a slide agglutination test, LeptoTek Dri-Dot, and IgM-ELISA with microscopic agglutination test for Leptospira antibody detection. Southeast Asian J Trop Med Public Health 38: 1111–1114.
- 40. Levett PN, Branch SL, Whittington CU, Edwards CN, Paxton H (2001) Two methods for rapid serological diagnosis of acute leptospirosis. Clin Diagn Lab Immunol 8: 349–351.
- 41. Winslow WE, Merry DJ, Pirc ML, Devine PL (1997) Evaluation of a commercial enzyme-linked immunosorbent assay for detection of immunoglobulin M antibody in diagnosis of human leptospiral infection. J Clin Microbiol 35: 1938–1942.
- 42. Silva MV, Camargo ED, Batista L, Vaz AJ, Brandao AP, et al. (1995) Behaviour of specific IgM, IgG and IgA class antibodies in human leptospirosis during the acute phase of the disease and during convalescence. J Trop Med Hyg 98: 268–272.
- 43. Amilasan AT, Ujiie M, Suzuki M, Salva E, Belo MCP, et al. (2012) Outbreak of Leptospirosis after Flood, the Philippines, 2009. Emerging Infectious Diseases 18: 91–94.
- 44. Smits HL, Ananyina YV, Chereshsky A, Dancel L, Lai AFRF, et al. (1999) International multicenter evaluation of the clinical utility of a dipstick assay for detection of Leptospira-specific immunoglobulin M antibodies in human serum specimens. J Clin Microbiol 37: 2904–2909.
- 45. Truong KN, Coburn J (2012) The emergence of severe pulmonary hemorrhagic leptospirosis: questions to consider. Front Cell Infect Microbiol 1: e24.
- 46. Ellis T, Imrie A, Katz AR, Effler PV (2008) Underrecognition of leptospirosis during a dengue fever outbreak in Hawaii, 2001–2002. Vector Borne Zoonotic Dis 8: 541–547.