Advertisement

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Accuracy of rapid lateral flow immunoassays for human leptospirosis diagnosis: A systematic review and meta-analysis

  • Teerapat Nualnoi ,

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Validation, Visualization, Writing – original draft, Writing – review & editing

    teerapat.n@psu.ac.th (TN); supawadee.n@psu.ac.th (SN)

    Affiliations Department of Pharmaceutical Technology, Faculty of Pharmaceutical Sciences, Prince of Songkla University, Hat Yai, Songkhla, Thailand, Drug Delivery System Excellence Center (DDSEC), Faculty of Pharmaceutical Sciences, Prince of Songkla University, Hat Yai, Songkhla, Thailand

    ⨯
  • Luelak Lomlim,

    Roles Investigation, Methodology, Writing – review & editing

    Affiliations Department of Pharmaceutical Chemistry, Faculty of Pharmaceutical Sciences, Prince of Songkla University, Hat Yai, Songkhla, Thailand, Phytomedicine and Pharmaceutical Biotechnology Excellence Center (PPBEC), Faculty of Pharmaceutical Sciences, Prince of Songkla University, Hat Yai, Songkhla, Thailand

    ⨯
  • Supawadee Naorungroj

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing

    teerapat.n@psu.ac.th (TN); supawadee.n@psu.ac.th (SN)

    Affiliation Department of Conservative Dentistry, Faculty of Dentistry, Prince of Songkla University, Hat Yai, Songkhla, Thailand

    ⨯

Abstract

Background

In the last two decades, several rapid lateral flow immunoassays (LFIs) for the diagnosis of human leptospirosis were developed and commercialized. However, the accuracy and reliability of these LFIs are not well understood. In this study, we aimed to evaluate the accuracy of leptospirosis LFIs as well as the factors affecting the test efficiency using systematic review and meta-analysis.

Methods and results

Original articles reporting the accuracy of human leptospirosis LFIs against microagglutination tests (MAT) or immunofluorescent assays (IFA) were searched from PubMed, Embase, and Scopus, and selected as per pre-set inclusion and exclusion criteria. A total of 49 data entries extracted from 24 eligible records published between 2003 and 2023 were included for meta-analysis. A meta-analysis was performed using STATA. The quality of the included studies was assessed according to the revised QUADAS-2. Only nine studies (32.1%) were considered to have a low risk of bias and no concern for applicability. Pooled sensitivity and specificity were calculated to be 68% (95% confidence interval, CI: 57–78) and 93% (95% CI: 90–95), respectively. However, the ranges of sensitivity (3.6 – 100%) and specificity (53.5 – 100%) of individual entries are dramatically broad, possibly due to the heterogeneity found in both study designs and LFIs themselves. Subgroup analysis demonstrated that IgM detection has better sensitivity than detection of IgG alone. Moreover, the test performance seems to be unaffected by samples from different phases of infection.

Conclusions

The pooled specificity of LFIs observed is somewhat acceptable, but the pooled sensitivity is low. These results, however, must be interpreted with caution because of substantial heterogeneity. Further evaluations of the LFIs with well-standardized design and reference test will be needed for a greater understanding of the test performance. Additionally, IgM detection type should be employed when leptospirosis LFIs are developed in the future.

Author summary

Several rapid lateral flow immunoassays (LFIs) for the diagnosis of leptospirosis, the most common bacterial infection transmitted from animals to humans, have been developed during the last two decades. The test accuracy, however, seems inconsistent among studies, raising questions about their reliability and applicability. Systematic review and meta-analysis were exploited to answer these questions. A total of 28 studies evaluating the human leptospirosis LFIs were included in this review. Major findings from our analysis include: i) the overall specificity of the LFIs is likely acceptable (93%), but the sensitivity is significantly low (68%); ii) substantial variation among studies was observed, alerting the reliability of this meta-analysis results; iii) the accuracy of the LFIs seems to be unaffected by samples collected from different phases of infection (acute vs. convalescent); and iv) IgM detection LFIs exhibit higher sensitivity as compared to IgG detection type. From our findings, we suggest that IgM detection LFIs should be focused on for future development. Also, in order to understand the performance of the LFIs better, other evaluation studies are still needed.

Citation: Nualnoi T, Lomlim L, Naorungroj S (2024) Accuracy of rapid lateral flow immunoassays for human leptospirosis diagnosis: A systematic review and meta-analysis. PLoS Negl Trop Dis 18(5): e0012174. https://doi.org/10.1371/journal.pntd.0012174

Editor: Manisha Biswal, Postgraduate Institute of Medical Education and Research, INDIA

Received: February 2, 2024; Accepted: April 29, 2024; Published: May 15, 2024

Copyright: © 2024 Nualnoi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files. The raw data used to generate plots/graphs presenting the pooled sensitivity and specificity values are mainly presented in Table 3. Additionally, the calculated values have been reported along with their 95% confidence interval.

Funding: TN was supported by a Prince of Songkla University Research Grant (grant number: PHA6202066S) to conduct this research. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Leptospirosis is the most prevalent zoonotic infections worldwide, with the highest incidence occurring in tropical resource-poor nations [1]. It was estimated to be responsible for 1.03 million cases and 58,900 deaths annually [2]. Humans may acquire the infection from direct contact with infected animals’ fluids or from water or soil contaminated with those through cuts, abrasive skin, or mucosal contact [3]. Leptospirosis cases are usually asymptomatic, but some patients may develop symptoms. The most common symptom observed among leptospirosis patients is an acute febrile illness. The infection can be treated by using common antibiotics such as doxycycline, and azithromycin. However, if the appropriate treatment is not given in a timely manner, the disease may progress to more severe clinical manifestations, which could potentially lead to multiple organ failure and death [1]. Early and accurate diagnosis, therefore, plays a key role in the management of the infection. Unfortunately, clinical presentations of leptospirosis are unspecific, making the disease difficult to differentiate from other infections, e.g., malaria and dengue.

Leptospirosis is caused by spirochete bacteria belonging to the genus Leptospira. More than 300 serovars of Leptospira spp. have been reported to be associated with the infection [4]. The current gold standard diagnosis of leptospirosis is the microagglutination test (MAT) [5]. The test usually requires live leptospires (approximately 20 serovars), expert personnel, and specific equipment, limiting its availability to only central or reference laboratories [6]. To overcome these challenges, several diagnostics such as enzyme-linked immunosorbent assays (ELISA), polymerase chain reactions (PCR), and rapid lateral flow immunoassays (LFIs) have been developed [6,7].

In the last two decades, several LFIs have been created for the diagnosis of human leptospirosis and many of them have been commercialized [7]. The use of these LFIs in clinical settings, however, remains unpopular, and they are only applied as a screening test [7]. This is possibly due to the sensitivity and specificity of the tests which are inconsistent among studies, raising questions about the reliability and applicability of their use [7,8]. Thus, the primary objective of this study is to determine the overall sensitivity and specificity of the currently available human leptospirosis LFIs using systemic review and meta-analysis. Additionally, a secondary objective is set to investigate factors that may affect the assay accuracy or heterogeneity of test results. The investigated factors include brand, detection target, and phase of infection. We expected that our findings would be helpful for not only healthcare providers but also researchers or inventors developing LFIs for the diagnosis of human leptospirosis.

Methods

This systematic review was conducted in accordance with the Preferred Reporting of Systematic Review Meta-analyses (PRISMA) guidelines [9]. The protocol was registered with the International Prospective of Systematic Reviews (PROSPERO registration number CRD42022371788).

Search strategy

We searched the following databases for relevant literature: PubMed, Embase, and Scopus. The search was first conducted on December 1st, 2022 and latest updated on January 28th, 2024. Restrictions were applied exclusively to English literature, but not for publication date or country of study. A search strategy was constructed with three key concepts to identify studies that report the sensitivity and specificity of different lateral flow assays for leptospirosis diagnosis (S1 Appendix). An additional screening from the reference lists of included studies and published systematic reviews was also performed to identify relevant studies.

Eligibility criteria

Original research articles either prospective or retrospective studies that met the following conditions were included in the meta-analysis: i) included patients intended to be diagnosed with leptospirosis; ii) examined the diagnostic accuracy of LFI; iii) used MAT or immunofluorescent assay (IFA) as a reference test; and iv) reported true positive (TP), false positive (FP), true negative (TN), and false negative (FN) values, allowing for calculation of sensitivity and specificity. We decided to include studies using the IFA as a reference test because the technique is based on the same principle as MAT and has been routinely used in some countries [10–12]. Studies were excluded if one or more of the following criteria were met: i) published in a language other than English; ii) evaluations of assays other than LFI; iii) evaluations performed in animals; and iv) insufficient data for the calculation of sensitivity and specificity.

Study selection and data extraction

All retrieved articles were imported to Rayyan, and duplicated studies were removed [13]. Two authors (TN and LL) screened and selected the articles independently based on title and abstract, followed by a full-text review. Any disagreements in selection were solved through discussion among all three researchers (TN, LL, and SN).

After piloting and standardization, the developed Microsoft Access form (Microsoft Corp., WA, USA) was used to extract data from the eligible studies. Data extraction was performed in duplicate by two authors (TN and LL) independently, and the collected data were cross-verified. Additionally, the relevant data from each selected article was summarized in a descriptive format (S2 Appendix).

Quality assessment

The quality assessment of the included articles was carried out using the criteria modified from Maze et. al. [7]. The criteria developed are in accordance with the revised Quality Assessment of Diagnostic Accuracy Studies criteria (QUADAS-2) [14]. For each selected study, the risk of bias was assessed in four domains: patient selection, index test, reference test, and flow and timing, using criteria listed in S1 Table. The study applicability was graded in the domains of patient selection, index test, and reference test, using the criteria listed in the S2 Table. The studies were scored as “low risk or no concern, +” if the criteria were met; “high risk or concern, -” if one of the criteria was not met; or “unclear, ?” if the information reported in the studies was not sufficient for assessment.

Statistical analysis

Estimates and plots.

The meta-analysis for this study was carried out using STATA version 16.1 (College Station, TX, USA) with all relevant packages (metandi, midas, and mylabels) installed. The extracted or calculated TP, FP, FN, and TN values from each study were included in the dataset. The primary measures are pooled sensitivity and specificity. We also plotted sensitivity and specificity data in forest plots calculated using a random effect bivariate model, and graphed study-specific estimates of sensitivity and specificity with 95% confidence intervals (CIs) in the summary receiver operating characteristic (SROC) curves, and the area under curve (AUC). To identify influential studies, the spike plot with Cook’s distance was used. Outlier detection was performed by interpreting the standardized level 2 residuals for sensitivity and specificity data of the studies included in the quantitative analysis.

Heterogeneity and subgroup analysis.

We used a bivariate box plot to evaluate the spread and skewness of the data. We also examined heterogeneity across studies by visually inspecting the forest plots. Subgroup meta-analysis was performed for the selected covariates: brand, detection target, and phases of infection.

Publication bias.

We assessed publication bias for diagnostic tests using Deeks’ funnel plot asymmetry test: a scatter plot of the inverse of the square root of the effective sample size [1/root (ESS)] versus the diagnostic log odds ratio (lnDOR). A symmetrical funnel shape with a p-value> 0.05 is considered to have no substantial publication bias.

Results

A total of 459 records were identified from an electronic search. After removing duplicates, 323 records remained for screening titles and abstracts. Of these, 42 reports were considered for full-text evaluation. After the application of inclusion and exclusion criteria, 18 reports were excluded, giving a total of 24 reports eligible for the review and meta-analysis (Fig 1) [11,15–37]. The records that were excluded and reasons for exclusion are listed in S3 Table [38–55].

thumbnail
Download:
Fig 1. The PRISMA flow diagram.

https://doi.org/10.1371/journal.pntd.0012174.g001

Characteristics of eligible studies

The articles or reports included in this review are listed in Table 1. These reports were published from 2003 through 2023. There are 28 studies extracted from 24 reports. This is because the reports published by Alia et. al., Lee et. al., and Chang et. al. presented the results from two, three, and two studies, respectively [18,23,27]. Studies were conducted among patients from Argentina, Brazil, Bulgaria, Cambodia, Congo, India, Korea, Laos, Malaysia, Micronesia, Nepal, Netherlands, Philippines, Slovenia, Sri Lanka, Sudan, and Thailand. Among all 28 studies, 12 studies (42.8%) are retrospective studies in which the evaluations were performed using archived samples. The number of patients or samples varied from 50 to approximately 2,800. All the studies used MAT (solely or along with other tests) as a reference test, except one study published by Silpasakorn et. al., in which only IFA was used to define leptospirosis cases [11]. In addition, variety in case definition was observed among the studies (S4 Table). In all 24 reports eligible for this meta-analysis, 16 different commercial and 4 different in-house LFIs were evaluated. However, it should be noted that, in this study, the articles published in non-English languages were excluded. Thus, the total number of 20 LFIs included in this review may not correspond to the number of all developed LFIs.

thumbnail
Download:
Table 1. List of the articles included in this review.

https://doi.org/10.1371/journal.pntd.0012174.t001

Study quality

The results of the quality assessment were summarized in Table 2. The assessment was carried out using the criteria presented in the S1 and S2 Tables. The results showed that only nine studies (32.1%) were considered “low risk” of bias and “no concern” for applicability. Thirteen studies (46.4%) were considered “low risk” of bias for all four domains. Eleven studies (39.3%) were classified as “high risk” of bias for patient selection because healthy participants were used as controls; ten of them are retrospective case-control selection studies. Three studies (10.7%) were rated “high risk” of bias for reference tests since MAT was performed only on acute-phase sera. Seven studies (25.0%) were graded “unclear” for flow and timing domain because it is unclear whether samples in each study were subjected to the same reference tests. In terms of applicability, all 28 studies were classified as “no concern” on patient selection and index test domains. However, four studies (14.3%) were rated “unclear” and four studies (14.3%) were rated “concern” in the reference test domain. This is because the serovar panel used in MAT test in those studies was not listed or did not cover serovars circulating in the areas, respectively.

thumbnail
Download:
Table 2. Quality assessment of the studies included in this review.

https://doi.org/10.1371/journal.pntd.0012174.t002

Descriptive analysis

All LFIs that were evaluated in this review are listed in the S5 Table and the evaluation results are summarized in Table 3. Among the 20 LFIs evaluated, 18 were designed to detect Leptospira-specific antibodies in blood or serum samples, whereas the other two assays are antigen detection targeting leptospiral lipopolysaccharide (LPS) in patient urines. These antigen detection assays have a sensitivity of 80–89% and a specificity of 74–87% [30]. Most antibody detection LFIs (11/18) aim to detect specific IgM. All these IgM detection assays were designed to have one reading window with a single control line and a single test line, except the ImmuneMed Leptospira IgM Duo Rapid Test that was developed to contain two reading windows [20]. This design allows samples to be tested at two different cutoff titers (1:50 and 1:200). The second most common design (5/18) for the antibody detection test strips is IgM/IgG detection. Of these, only the ImmuneMed Leptospira Rapid Test, was designed to contain two separate reading windows for IgM and IgG, while the remaining were constructed with three lines (IgM, IgG, and control lines) on the same strip [23]. The variety of designs of leptospirosis LFIs also includes ImmuneMed AFI rapid Test that was developed in a multiplex format to detect leptospirosis simultaneously with scrub typhus and hemorrhagic fever [11]. Additionally, the Dual Path Platform from Chembio Diagnostic Systems, USA, was designed in such a way that the sample and chase buffer were flown perpendicularly [21,31]. The accuracy of these LFIs were summarized in Tables 3 and S6.

In addition to the variety of LFI configurations described above, we also looked at the leptospiral antigens that were selected to be incorporated into the LFIs. Among the 18 antibody detection LFIs evaluated in this review, only 11 LFIs have disclosed the antigen they used. L. interrogans heat extract was found to be the most common antigens employed in the LFIs, followed by polysaccharide from nonpathogenic L. biflexa serovar Patoc. Of these, recombinant protein has been used in only one LFI [21,31].

thumbnail
Download:
Table 3. Summary of diagnostic accuracy of LFIs evaluated in this review.

https://doi.org/10.1371/journal.pntd.0012174.t003

Meta-analysis

Of these 28 studies extracted from 24 reports, several studies evaluated LFIs using samples collected from different phases of infection i.e., acute phase sera (mainly collected within 10 days after onset) and convalescent phase sera (mainly collected within two weeks after acute phase). In addition, the study conducted by Nabity et. al. evaluated the accuracy of the Dual Path Platform from Chembio Diagnostic Systems using samples collected by three different means i.e., finger stick blood (FSB), vein whole blood (VWB), and serum [21]. Thus, multiple sensitivity and specificity values were reported from these studies. As a result, a total of 49 data entries were employed for meta-analysis (Table 3).

Pooled sensitivity and specificity.

We found that the sensitivity of individual entries varied widely, from 3.6% to 100.0%. The range of specificity for each entry was narrower, ranging from 53.5% to 100.0%. Firstly, the estimation of the pooled sensitivity and specificity was calculated by a random effect bivariate model with all 49 data entries included. The entry-specific results were displayed in the forest plot (Fig 2). The estimated pooled sensitivity and specificity were 68% (95% confidence interval, CI: 57–78) and 93% (95% CI: 90–95), respectively. The I2 values for sensitivity and specificity were 95.76% and 96.35%, respectively, indicating that the data entries included might be too heterogeneous to achieve accurate meta-analysis results. The AUC from the SROC curve was 0.92 (Fig 3). The Deeks’ funnel plot asymmetry showed no potential publication bias (Fig 4). The residual-based goodness-of-fit and bivariate normality plots suggested that the random effect model used was applicable. The influence and outlier detection analyses, however, identified 3 common outliers (S1 Fig). Next, we performed sensitivity and specificity analysis by removing the outlier that was graded high risk during the study quality assessment (entry ID 11) and found that the pooled sensitivity and specificity did not change significantly (S2 Fig). Thirdly, we calculated the combined sensitivity and specificity values using only one data entry per study. In this case, sensitivity and specificity obtained from acute phase samples were chosen as representative for the studies reporting multiple data entries. The combined sensitivity and specificity were 66% (95% CI: 51–78) and 92% (95% CI: 88–94), respectively, which were comparable to those obtained from all data entries (S3 Fig).

thumbnail
Download:
Fig 2. Forest plot showing entry-specific and mean sensitivity and specificity with corresponding heterogeneity statistics.

Forest plots of the analysis about the prediction value of LFI kits for leptospirosis in terms of sensitivity and specificity with the data of all 49 entries. Square symbols represent the sensitivity or specificity of each study according to the entry ID shown on the y-axis, while the short lines cutting through represent the relative 95% CI. The diamond symbols refer to the combined sensitivity or specificity, which was automatically calculated and displayed by Stata software. A “COMBINED” label corresponding to the diamond symbol is shown on the y-axis underneath all entry IDs.

https://doi.org/10.1371/journal.pntd.0012174.g002

thumbnail
Download:
Fig 3. Summary ROC curve with confidence and prediction regions around mean operating sensitivity and specificity points.

https://doi.org/10.1371/journal.pntd.0012174.g003

thumbnail
Download:
Fig 4. Funnel plot with superimposed regression line (publication bias).

STATA command: midas tp fp fn tn, pubbias.

https://doi.org/10.1371/journal.pntd.0012174.g004

Subgroup analysis by LFI brands.

The sensitivity and specificity of each brand of LFI were calculated using subgroup analysis and summarized in the S6 Table. This analysis, however, was limited to LFIs that have four or more data entries available. The results showed that the Dual Path Platform had the highest sensitivity (90%; 95% CI: 82–94), followed by Leptocheck WB (75%; 95% CI: 66–82), and LeptoTek Lateral Flow (65%; 95% CI: 52–76). The highest specificity was of ImmuneMed AFI rapid (99%; 95% CI: 97–100), followed by Dual Path Platform (89%; 95% CI: 81–94), and LeptoTek Lateral Flow (89%; 95% CI: 76–95), respectively.

Subgroup analysis by detection targets.

We also performed subgroup analysis to investigate the influence of the detection targets (S7 Table). The LFIs that provided the greatest sensitivity are the IgM detection type (69%; 95% CI: 58–78), followed by the IgM/IgG detection type (62%; 95% CI: 26–88). IgG detection LFIs demonstrated the lowest sensitivity of 23% (95% CI: 3–73). On the contrary, the greatest specificity is of IgG detection LFIs (100%; 95% CI: 94–100), followed by IgM/IgG (97%; 95% CI: 94–98), and IgM detection (90%; 95% CI: 86–94), respectively.

Subgroup analysis by infection phases.

The pooled sensitivity and specificity as grouped by infection phases were also determined (S8 Table). When samples collected from acute phase, convalescent phase, and mixed samples were used to evaluate the LFIs, the pooled sensitivities of the LFIs were 67% (95% CI: 50–80), 64% (95% CI: 39–83), and 69% (95% CI: 56–80), and the pooled specificities were 93% (95% CI: 88–96), 94% (95% CI: 89–97), and 94% (95% CI: 85–97), respectively.

Discussion

In this study, original articles evaluating the human leptospirosis diagnostic LFIs were systematically recruited for review and meta-analysis. The recruited articles were restricted to those that used MAT as a reference test because MAT is considered a gold standard diagnosis for leptospirosis and has been used globally [3,6]. We also included studies with IFA as a reference test because the technique is based on the same antigen-antibody reaction as MAT and has been used routinely in some endemic areas [10–12].

Meta-analysis was initially performed with all 49 data entries (Table 3) extracted from 28 eligible studies (Table 1). We found that the estimated pooled sensitivity and specificity were 68% and 93%, respectively (Fig 2). However, we thought that samples from the acute phase of infection were more relevant to real-world settings, as patients tend to visit hospitals a few days after onset. The pooled sensitivity and specificity were recalculated regarding this matter, and the results are not significantly different compared to the initial analysis. We noted that the pooled specificity of LFIs is somewhat acceptable, but the pooled sensitivity is low [56]. Nevertheless, it is important to emphasize that the estimated pooled sensitivity and specificity should be interpreted with caution due to the substantial heterogeneity observed among the included studies, despite the adoption of a random effect model. Furthermore, the forest plot’s I2 values indicate that the included studies were considerably heterogeneous to produce dependable meta-analysis results. Several potential factors contributing to heterogeneity were spotted during data extraction.

First, we identified that the study designs could be a potential cause of the data heterogeneity. Two types of study designs were observed among the eligible studies: a case-control type and a cohort type cross-sectional study [57]. We found that 17 studies (61%) were cohort type cross-sectional studies. These were considered low risk for patient selection bias. The others 11 studies (39%), instead, were case-control type and were graded as high risk of bias in patient selections.

Second, reference tests and case definitions varied dramatically across the studies (S4 Table). For examples, some studies used MAT or IFA solely as a reference (54%) while others (46%) used MAT together with other methods. Also, the Leptospira serovar panel used in MAT might not cover all the predominant serovars in some studies. While several studies defined leptospirosis cases based on a four-fold increase in MAT antibody titers together with single MAT antibody titers, some evaluations used either single titers or paired titers only to define the cases. We noted that the investigators might define the single MAT titer cutoffs differently based on the prevalence of leptospirosis in the areas, and these cutoffs could be distinct from the WHO recommended cutoff of 1:400 [7,58]. However, various single MAT titer cutoffs within the same settings were detected (S4 Table). We determined that 10 studies (36%) were at risk (high risk and unclear) of bias and applicability for the reference test domain.

Third, we found that the samples themselves could also cause the heterogeneity of the evaluation results. While some investigators used samples collected from the acute phase of infection to evaluate the LFIs, others might use convalescent or mixed samples. Principally, acute phase samples contain a lower level of Leptospira-specific antibodies, and they are mainly IgM. Convalescent phase samples, on the contrary, have a higher titer of Leptospira-specific IgG [6]. Thus, even the same assays could have different sensitivity and specificity if different phases of samples were used for the evaluation.

Fourth, we also found that detection targets varied among the investigated LFIs. These variations include LPS detection LFIs, IgM detection LFIs, IgG detection LFIs, and combined IgM/IgG detection LFIs. Since the presence of IgM and IgG in the samples is determined by the phase of infection, evaluations of IgG detection LFIs with acute phase samples, for example, could give an expected lower sensitivity as compared to IgM detection LFIs [6]. This demonstrates that the detection target of LFI, among other factors, impacts the test performance.

Fifth, in addition to detection target, differences between each brand of the investigated LFIs also include: i) test designs (single-plex, multiplex, or two reading-windows formats); ii) flow patterns (regular lateral flow or perpendicular flow); iii) types of antigens incorporated (heat extract of the bacteria, purified LPS, recombinant proteins, etc.); and iv) miscellaneous factors such as types of membranes, amounts of antigen or antibody on the test and control lines, particle size of conjugated gold, etc. As a result, different brands or models of LFIs could perform differently due to the variations in these elements.

A question that is often asked in clinical settings (especially when there are several tests available, but test performances are heterogeneous among studies) is which brand of LFIs has the highest accuracy. To answer this question, we investigated further using multivariate meta-regression and subgroup analysis. However, this analysis is limited to the Dual Path Platform, Leptocheck WB, LeptoTek Lateral Flow, and ImmuneMed AFI Rapid only due to the availability of the data. The analysis results illustrated that the sensitivity and specificity are statistically comparable between each brand of LFIs (S6 Table).

Additionally, we conducted another subgroup analysis to understand how the LFIs performed with samples from different phases of infection. The results demonstrated that the sensitivity and specificity are practically unchanged, suggesting that the LFIs could be used with samples collected from either phase (S8 Table). Initially, this finding surprised us because acute phase samples are considerably not suitable for the IgG detection LFIs in principle. This is because the specific IgG has not been raised yet during the acute phase of the infection. However, when we examined the data closely, we found that the majority of the investigated LFIs are IgM and IgM/IgG detection types. Therefore, in this analysis, the results from IgM-targeted LFIs probably masked those from IgG detection types and gave the overall results as reported. We also found it interesting that the sensitivity of IgM detection LFIs remained unaffected when convalescent samples were used. Theoretically, the level of specific IgM declines during the convalescent phase of infections, causing IgM-targeted diagnostics to become less sensitive with convalescent samples. However, it has been reported that the specific IgM may stay in leptospirosis patients for months, which means that IgM-targeted LFIs will react with the convalescent samples [59].

In this study, subgroup analysis on detection targets was also carried out. As expected, IgM detection LFIs demonstrated a higher sensitivity over IgG detection types whereas specificity seems to be unaffected by this covariate (S7 Table). This finding would suggest that the IgM detection type should be chosen for further developments of the leptospirosis diagnostic LFIs.

In summary, during the past two decades, at least 20 LFIs for the diagnosis of human leptospirosis have been developed and marketed. Several studies were conducted to evaluate those LFIs; however, the results were markedly varied. In this study, we attempted to gain a better understanding of the accuracy of the available LFIs using a systematic review and meta-analysis. We found that the available data are heterogeneous; thus, the estimated accuracy of the LFIs derived from this study may not be at most reliable, especially the pooled sensitivity of the assays. This finding is in agreement with the systematic review and meta-analysis studies published previously [7,8]. The sources of heterogeneity in the data potentially came from both LFIs themselves and evaluation procedures. Variations in evaluation procedures seem to contribute to both assay sensitivity and accuracy. On the basis of our findings, it is difficult to ascertain whether the LFIs are clinically useful. Sequentially, we recommend that further evaluations of LFIs are needed, as also suggested by the previous reviews [7,8]. Further evaluations should be conducted as per standard guidelines, such as the Standard for Reporting Diagnostic Accuracy Studies (STARD) [60]. Additionally, they should use reference tests with sensitivity and specificity close to 100% or statistical tools that account for imperfect reference test accuracy, e.g., Bayesian latent class model [10]. Diversity in LFI designs, on the other hand, is likely to primarily affect the sensitivity of the assays. Our finding suggests that IgM or combined IgM/IgG detection types of the LFIs would be more suitable for the diagnosis of leptospirosis as compared to IgG detection types. However, in this current study, the comparison between antigen detection and antibody detection LFIs could not be done because the data from the antigen detection type is not sufficient.

Supporting information

S1 Checklist. PRISMA-DTA.

https://doi.org/10.1371/journal.pntd.0012174.s001

(DOCX)

S2 Checklist. PRISMA DTA for abstract.

https://doi.org/10.1371/journal.pntd.0012174.s002

(DOCX)

S1 Table. Criteria for assessing risk of bias of studies included in this review.

https://doi.org/10.1371/journal.pntd.0012174.s003

(DOCX)

S2 Table. Criteria for assessing the applicability of the studies included in this review.

https://doi.org/10.1371/journal.pntd.0012174.s004

(DOCX)

S3 Table. Reasons for exclusion.

https://doi.org/10.1371/journal.pntd.0012174.s005

(DOCX)

S4 Table. List of reference tests and case definitions used in the selected studies.

https://doi.org/10.1371/journal.pntd.0012174.s006

(DOCX)

S5 Table. List of LFI evaluated by the studies included in this review.

https://doi.org/10.1371/journal.pntd.0012174.s007

(DOCX)

S6 Table. Subgroup analysis by brand of LFI.

https://doi.org/10.1371/journal.pntd.0012174.s008

(DOCX)

S7 Table. Subgroup analysis by detection targets.

https://doi.org/10.1371/journal.pntd.0012174.s009

(DOCX)

S8 Table. Subgroup analysis by phases of infection.

https://doi.org/10.1371/journal.pntd.0012174.s010

(DOCX)

S1 Fig.

Graphical depiction of influence (A) and outlier detection (B) analyses.

https://doi.org/10.1371/journal.pntd.0012174.s011

(TIFF)

S2 Fig. Forest plot showing mean sensitivity, specificity, and corresponding heterogeneity statistics with all data entries except the high-risk outlier (entry ID 11).

https://doi.org/10.1371/journal.pntd.0012174.s012

(TIFF)

S3 Fig. Forest plot showing mean sensitivity, specificity, and corresponding heterogeneity statistics with data limited to one data entry per study.

https://doi.org/10.1371/journal.pntd.0012174.s013

(TIFF)

S1 Appendix. Search strategy.

https://doi.org/10.1371/journal.pntd.0012174.s014

(DOCX)

S2 Appendix. Summary of individual reports included in the review.

https://doi.org/10.1371/journal.pntd.0012174.s015

(DOCX)

References

  1. 1. Bradley EA, Lockaby G. Leptospirosis and the environment: A review and future directions. Pathogens. 2023;12: 1167. pmid:37764975
  2. 2. Costa F, Hagan JEJE, Calcagno J, Kane M, Torgerson P Martinez-Silveira MS, et al. Global morbidity and mortality of leptospirosis: A systematic review. PLoS Negl Trop Dis. 2015;9: e0003898. pmid:26379143
  3. 3. Karpagam KB, Ganesh B. Leptospirosis: a neglected tropical zoonotic infection of public health importance—an updated review. Eur J Clin Microbiol Infect Dis. 2020;39: 835–846. pmid:31898795
  4. 4. Sykes JE, Haake DA, Gamage CD, Mills WZ, Nally JE. A global one health perspective on leptospirosis in humans and animals. J Am Vet Med Assoc. 2022;260: 1589–1596. pmid:35895801
  5. 5. Chacko CS, Lakshmi S S, Jayakumar A, Binu SL, Pant RD, Giri A, et al. A short review on leptospirosis: Clinical manifestations, diagnosis and treatment. Clin Epidemiol Glob Health. 2021;11: 100741.
  6. 6. Sykes JE, Reagan KL, Nally JE, Galloway RL, Haake DA. Role of Diagnostics in Epidemiology, Management, Surveillance, and Control of Leptospirosis. Pathogens. 2022;11. pmid:35456070
  7. 7. Maze MJ, Sharples KJ, Allan KJ, Rubach MP, Crump JA. Diagnostic accuracy of leptospirosis whole-cell lateral flow assays: a systematic review and meta-analysis. Clin Microbiol Infect. 2019;25: 437–444. pmid:30472422
  8. 8. Clemente BM, Pineda-Cortel MR, Villaflores O. Evaluating immunochromatographic test kits for diagnosis of acute human leptospirosis: A systematic review. Heliyon. 2022;8: e11829. pmid:36458294
  9. 9. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339: 332–336. pmid:19622551
  10. 10. Limmathurotsakul D, Turner EL, Wuthiekanun V, Thaipadungpanit J, Suputtamongkol Y, Chierakul W, et al. Fool’s gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis. Clin Infect Dis. 2012;55: 322–31. pmid:22523263
  11. 11. Silpasakorn S, Kim YW, Suputtamongkol Y. Evaluation of combined rapid immunoglobulin M and immunoglobulin G lateral flow assays for the diagnosis of leptospirosis, scrub typhus, and hantavirus infection. Siriraj Med J. 2020;72: 253–258.
  12. 12. Wilairatana P, Mala W, Rattaprasert P, Kotepui KU, Kotepui M. Prevalence of malaria and leptospirosis co-infection among febrile patients: A systematic review and meta-analysis. Trop Med Infect Dis. 2021;6: 122. pmid:34287366
  13. 13. Ouzzani M, Hossam H, Zbys F, Ahmed E. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5: 210. pmid:27919275
  14. 14. Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. Quadas-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155: 529–536. pmid:22007046
  15. 15. Bottieau E, Van Duffel L, El Safi S, Koirala KD, Khanal B, Rijal S, et al. Etiological spectrum of persistent fever in the tropics and predictors of ubiquitous infections: a prospective four-country study with pooled analysis. BMC Med. 2022;20: 1–11. pmid:35491421
  16. 16. Dinhuzen J, Limothai U, Tachaboon S, Krairojananan P, Laosatiankit B, Boonprasong S, et al. A prospective study to evaluate the accuracy of rapid diagnostic tests for diagnosis of human leptospirosis: Result from THAI-LEPTO AKI study. PLoS Negl Trop Dis. 2021;15: e0009159. pmid:33606698
  17. 17. Dawson P, Marfel M, Galloway R, Tareg A, Paz-Bailey G, Muñoz-Jordán JL, et al. Notes from the field: Interpretation of rapid diagnostic tests for leptospirosis during a dengue outbreak — Yap State, Federated States of Micronesia, 2019. MMWR Morb Mortal Wkly Rep. 2020;69: 1832–1833. pmid:33270615
  18. 18. Alia SN, Joseph N, Philip N, Azhari NN, Garba B, Masri SN, et al. Diagnostic accuracy of rapid diagnostic tests for the early detection of leptospirosis. J Infect Public Health. 2019;12: 263–269. pmid:30502041
  19. 19. Rao M, Amran F, Aqilla N. Evaluation of a Rapid Kit for Detection of IgM against Leptospira in Human. Can J Infect Dis Med Microbiol. 2019;2019: 5763595. pmid:30881530
  20. 20. Amran F, Liow YL, Halim NAN. Evaluation of a commercial immuno-chromatographic assay kit for rapid detection of IgM antibodies against Leptospira antigen in human serum. J Korean Med Sci. 2018;33: e131. pmid:29686599
  21. 21. Nabity SA, Hagan JE, Araújo G, Damião AO, Cruz JS, Nery N, et al. Prospective evaluation of accuracy and clinical utility of the Dual Path Platform (DPP) assay for the point-of-care diagnosis of leptospirosis in hospitalized patients. PLoS Negl Trop Dis. 2018;12: e0006285. pmid:29462146
  22. 22. Doungchawee G, Sutdan D, Niwatayakul K, Inwisai T, Sitthipunya A, Boonsathorn N, et al. Development and evaluation of an immunochromatographic assay to detect serum anti-leptospiral lipopolysaccharide IgM in acute leptospirosis. Sci Rep. 2017;7: 1–9. pmid:28536483
  23. 23. Lee J-W, Park S, Kim SH, Christova I, Jacob P, Vanasco NB, et al. Clinical evaluation of rapid diagnostic test kit using the polysaccharide as a genus-Specific diagnostic antigen for leptospirosis in Korea, Bulgaria, and Argentina. J Korean Med Sci. 2016;31: 183–189. pmid:26839470
  24. 24. Eugene EJ, Handunnetti SM, Wickramasinghe SA, Kalugalage TL, Rodrigo C, Wickremesinghe H, et al. Evaluation of two immunodiagnostic tests for early rapid diagnosis of leptospirosis in Sri Lanka: A preliminary study. BMC Infect Dis. 2015;15: 1–5. pmid:26259957
  25. 25. Niloofa R, Fernando N, de Silva NL, Karunanayake L, Wickramasinghe H, Dikmadugoda N, et al. Diagnosis of Leptospirosis: Comparison between Microscopic Agglutination Test, IgM-ELISA and IgM Rapid Immunochromatography Test. PLoS One. 2015;10: e0129236. pmid:26086800
  26. 26. Podgoršek D, Cerar T, Logar M, Lešničar G, Remec T, Baklan Z, et al. Evaluation of the immunochromatographic (Leptocheck) test for detection of specific antibodies against leptospires. Wien Klin Wochenschr. 2015;127: 948–953. pmid:26381276
  27. 27. Chang CH, Riazi M, Yunus MH, Osman S, Noordin R. Limited diagnostic value of two commercial rapid tests for acute leptospirosis detection in Malaysia. Diagn Microbiol Infect Dis. 2014;80: 278–281. pmid:25241641
  28. 28. Colt S, Pavlin BI, Kool JL, Johnson E, McCool JP, Woodward AJ. Human leptospirosis in the federated states of micronesia: A hospital-based febrile illness survey. BMC Infect Dis. 2014;14: 1–6. pmid:24708723
  29. 29. Goris MGA, Leeflang MMG, Loden M, Wagenaar JFP, Klatser PR, Hartskeerl RA, et al. Prospective evaluation of three rapid diagnostic tests for diagnosis of human leptospirosis. PLoS Negl Trop Dis. 2013;7: e2290. pmid:23875034
  30. 30. Widiyanti D, Koizumi N, Fukui T, Muslich LT, Segawa T, Villanueva SYAM, et al. Development of immunochromatography-based methods for detection of leptospiral lipopolysaccharide antigen in urine. Clin Vaccine Immunol. 2013;20: 683–690. pmid:23467776
  31. 31. Nabity SA, Ribeiro GS, Lessa Aquino C, Takahashi D, Damião AO, Gonçalves AHO, et al. Accuracy of a Dual Path Platform (DPP) assay for the rapid point-of-care diagnosis of human leptospirosis. PLoS Negl Trop Dis. 2012;6: e1878. pmid:23133686
  32. 32. Silpasakorn S, Waywa D, Hoontrakul S, Suttinont C, Losuwanaluk K, Suputtamongkol Y. Performance of Leptospira immunoglobulin M ELISA and rapid immunoglobulin G immunochromatographic assays for the diagnosis of leptospirosis. J Med Assoc Thai. 2011;94: S203–S206. pmid:21721448
  33. 33. Cohen AL, Dowell SF, Nisalak A, Mammen MP, Petkanchanapong W, Fisk TL. Rapid diagnostic tests for dengue and leptospirosis: antibody detection is insensitive at presentation. Trop Med Int Health. 2007;12: 47–51. pmid:17207147
  34. 34. Blacksell SD, Smythe L, Phetsouvanh R, Dohnt M, Hartskeerl R, Symonds M, et al. Limited diagnostic capacities of two commercial assays for the detection of Leptospira immunoglobulin M antibodies in Laos. Clin Vaccine Immunol. 2006;13: 1166–1169. pmid:17028219
  35. 35. Sehgal SC, Vijayachari P, Sugunan AP, Umapathi T. Field application of Lepto lateral flow for rapid diagnosis of leptospirosis. J Med Microbiol. 2003;52: 897–901. pmid:12972585
  36. 36. Jirawannaporn S, Limothai U, Tachaboon S, Dinhuzen J, Kiatamornrak P, Chaisuriyong W, et al. The combination of RPA-CRISPR/Cas12a and Leptospira IgM RDT enhances the early detection of leptospirosis. PLoS Negl Trop Dis. 2023;17: e0011596. pmid:37624847
  37. 37. Campos H de S, da Silva ED, de Lima GS, de O Resende R, Burth P. Leptospira interrogans insoluble fraction as a potential antigen source for lateral flow immunochromatography. Mem Inst Oswaldo Cruz. 2023;118. pmid:37194802
  38. 38. Sapna K, Ashaiba A, Kumar TNR, Shashidhar V, Arun AB, Prasad KS. Evaluation of anti-LipL32 carbon nanotube immunofluorescence probe (carbo-lip) and comparison with MAT, IgM ELISA, IgM spot test and culture for early detection of leptospirosis at local hospital. J Microbiol Methods. 2022;195: 106448. pmid:35283263
  39. 39. Arafah S, Blacksell S, Mayo M, Currie B, Mace A, Ongarello S, et al. Performance evaluation of a novel multiplexed lateral flow assay to identify common causes of fever in Asia and inform treatment decisions. Am J Trop Med Hyg. 2019. pp. 482–483.
  40. 40. Poonati R, Mallepaddi PC, Punati RD, Maity SN, Alamuri A, Manchikalapudi S, et al. Development and validation of point of care diagnostics for the rapid detection of multiple species of Leptospira at resource-limited areas. Curr Trends Biotechnol Pharm. 2019;13: 270–282.
  41. 41. Iwasaki H, Chagan-Yasutan H, Leano PSA, Koizumi N, Nakajima C, Taurustiati D, et al. Combined antibody and DNA detection for early diagnosis of leptospirosis after a disaster. Diagn Microbiol Infect Dis. 2016;84: 287–291. pmid:26860351
  42. 42. Panwala T, Rajdev S, Mulla S. To evaluate the different rapid screening tests for diagnosis of leptospirosis. J Clin Diagn Res. 2015;9: DC21. pmid:25859456
  43. 43. Vanithamani S, Shanmughapriya S, Narayanan R, Raja V, Kanagavel M, Sivasankari K, et al. Lipopolysaccharide specific immunochromatography based lateral flow assay for serogroup specific diagnosis of leptospirosis in India. PLoS One. 2015;10: e0137130. pmid:26340095
  44. 44. Valverde J M de los A, León B, Taylor L, Visona K. Development of a Lepto-IgM EIACR test to diagnose leptospirosis disease in Costa Rican patient samples. Invest Clin. 2007;48: 295–304. pmid:17853789
  45. 45. Boonyod D, Poovorawan Y, Bhattarakosol P, Chirathaworn C. LipL32, an outer membrane protein of Leptospira, as an antigen in a dipstick assay for diagnosis of leptospirosis. Asian Pac J Allergy Immunol. 2005;23: 133. pmid:16252844
  46. 46. Kemapunmanus M, Sretrirutchai S, Khuntikij P, Pradutkanchana S, Pradutkanchana J. A prospective evaluation of four immunodiagnotic assays for human leptospirosis. Southeast Asian J Trop Med Public Health. 2004;35: 863–867. pmid:15916082
  47. 47. Bajani MD, Ashford DA, Bragg SL, Woods CW, Aye T, Spiegel RA, et al. Evaluation of four commercially available rapid serologic tests for diagnosis of leptospirosis. J Clin Microbiol. 2003;41: 803–809. pmid:12574287
  48. 48. Effler P V., Bogard AK, Domen HY, Katz AR, Higa HY, Sasaki DM. Evaluation of eight rapid screening tests for acute leptospirosis in Hawaii. J Clin Microbiol. 2002;40: 1464–1469. pmid:11923374
  49. 49. Levett PN, Branch SL, Whittington CU, Edwards CN, Paxton H. Two methods for rapid serological diagnosis of acute leptospirosis. Clinical Diagnostic Laboratory Immunology. 2001;8: 349–351. pmid:11238220
  50. 50. Smits HL, Eapen CK, Sugathan S, Kuriakose M, Gasem MH, Yersin C, et al. Lateral-flow assay for rapid serodiagnosis of human leptospirosis. Clin Diagn Lab Immunol. 2001;8: 166–169. pmid:11139212
  51. 51. Hatta M, Smits HL, Gussenhoven GC, Gooskens J. Introduction of a rapid dipstick assay for the detection of Leptospira-specific immunoglobulin m antibodies in the laboratory diagnosis of leptospirosis in a hospital in Makassar, Indonesia. Southeast Asian J Trop Med Public Health. 2000;31: 515–520. pmid:11289012
  52. 52. Smits HL, Hartskeerl RA, Terpstra WJ. International multi-centre evaluation of a dipstick assay for human leptospirosis. Trop Med Int Health. 2000;5: 124–128. pmid:10747272
  53. 53. Sehgal SC, Vijayachari P, Sharma S, Sugunan AP. LEPTO Dipstick: a rapid and simple method for serodiagnosis of acute leptospirosis. Trans R Soc Trop Med Hyg. 1999;93: 161–164. pmid:10450439
  54. 54. Smits HL, Ananyina Y V., Chereshsky A, Dancel L, Lai-A-Fat RFM, Chee HD, et al. International Multicenter Evaluation of the Clinical Utility of a Dipstick Assay for Detection ofLeptospira-Specific Immunoglobulin M Antibodies in Human Serum Specimens. J Clin Microbiol. 1999;37: 2904–2909. pmid:10449473
  55. 55. Yersin C, Bovet P, Smits HL, Perolat P. Field evaluation of a one-step dipstick assay for the diagnosis of human leptospirosis in the Seychelles. Trop Med Int Health. 1999;4: 38–45. pmid:10203172
  56. 56. Bujang MA, Adnan TH. Requirements for Minimum Sample Size for Sensitivity and Specificity Analysis. J Clin Diagn Res. 2016;10: YE01–YE06. pmid:27891446
  57. 57. Mathes T, Pieper D. An algorithm for the classification of study designs to assess diagnostic, prognostic and predictive test accuracy in systematic reviews. Syst Rev. 2019;8: 1–8. pmid:31481098
  58. 58. Goris M, Leeflang MMG, Boer KR, Goeijenbier M, Gorp ECM, Wagenaar JFP, et al. Establishment of valid laboratory case definition for human leptospirosis. J Bacteriol Parasitol. 2011;3: 1–8.
  59. 59. Cumberland P, Everard COR, Wheeler JG, Levett PN. Persistence of anti-leptospiral IgM, IgG and agglutinating antibodies in patients presenting with acute febrile illness in Barbados 1979-1989. Eur J Epidemiol. 2001;17: 601–608. pmid:12086073
  60. 60. Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6. pmid:28137831