https://orcid.org/0000-0001-5464-2381
Figures
Abstract
American tegumentary leishmaniasis (ATL) diagnosis is an open question, and the search for a solution is urgent. The available tests that detect the etiological agent of the infection are specific for ATL diagnosis. However, they present disadvantages, such as low sensitivity and the need for invasive procedures to obtain the samples. Immunological methods (leishmanin skin test and search for anti-Leishmania antibodies) are good alternatives to the etiological diagnosis of ATL. Presently, we face problems with disease confirmation due to the discontinuity in the production of leishmanin skin test antigen, particularly in resource-poor settings. Aiming to diagnose ATL, we validated rLb6H-ELISA for IgG antibodies using 1,091 samples from leishmaniasis patients and healthy controls, divided into four panels, living in 19 Brazilian endemic and non-endemic states. The rLb6H-ELISA showed a sensitivity of 98.6% and a specificity of 100.0%, with the reference panel comprising 70 ATL patient samples and 70 healthy controls. The reproducibility evaluation showed a coefficient of variation of positive samples ≤ 8.20% for repeatability, ≤ 17,97% for reproducibility, and ≤ 8.12% for homogeneity. The plates sensitized with rLb6H were stable at 4°C and -20°C for 180 days and 37°C for seven days, indicating 12 months of validity. In samples of ATL patients from five research and healthcare centers in endemic and non-endemic areas, rLb6H-ELISA showed a sensitivity of 84.0%; no significant statistical difference was observed among the five centers (chi-square test, p = 0.13). In samples of healthy controls from four areas with different endemicity, a specificity of 92.4% was obtained; lower specificity was obtained in a visceral leishmaniasis high endemicity locality (chi-square test, p<0.001). Cross-reactivity was assessed in 166 other disease samples with a positivity of 13.9%. Based on the good diagnostic performance and the reproducibility and stability of the antigen, we suggest using ELISA-rLb6H to diagnose ATL.
Citation: Valencia-Portillo RT, Lindoso JA, Celeste BJ, Bittencourt AA, de Brito MEF, Duthie MS, et al. (2024) ELISA with recombinant antigen Lb6H validated for the diagnosis of American tegumentary leishmaniasis. PLoS ONE 19(6): e0304268. https://doi.org/10.1371/journal.pone.0304268
Editor: Mariana Lourenço Freire, Oswaldo Cruz Foundation Rene Rachou Institute: Fundacao Oswaldo Cruz Instituto Rene Rachou, BRAZIL
Received: December 21, 2023; Accepted: May 8, 2024; Published: June 5, 2024
Copyright: © 2024 Valencia-Portillo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting information files.
Funding: MCAS received grant #2021/12535-2 São Paulo Research Foundation (FAPESP) (https://fapesp.br/). RTVP received a scholarship from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance code 88882.376665/2019-01 and 88887.689454/2022-00 (https://www.gov.br/capes/pt-br). HG received a research fellowship from Conselho Nacional de Desenvolvimento Científico e Tecnológico – Brasil (CNPq) - n°: 302940/2019-7, (https://www.gov.br/cnpq/pt-br). This study was financed in part by Laboratório de Investigação Médica (LIM 38) Hospital das Clínicas da Faculdade de Medicina da Universidade de Sao Paulo (https://limhc.fm.usp.br/portal/). The funders had no role in study design, data collection, analysis, publication decision, or manuscript preparation.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Leishmaniases are neglected diseases caused by protozoa of the genus Leishmania prevalent in tropical and subtropical areas and determining the visceral (VL) and tegumentary (TL) forms. The American tegumentary leishmaniasis (ATL) is mainly caused by Leishmania (Viannia) braziliensis, L. (V.) guyanensis, or L. (Leishmania) amazonensis. ATL comprises a broad spectrum of clinical manifestations that depend on the Leishmania species, the parasite load, and the host response [1]. Based on the different clinical presentations, it is classified as cutaneous leishmaniasis (CL) in its various forms and mucosal leishmaniasis (ML) [2,3]. World Health Organization estimates that between 600,000 and one million new cases of TL occur annually worldwide, and CL is the most prevalent form [4]. Brazil is endemic for both CL and ML; in 2022, 12,878 cases of ATL were reported [5].
The diagnosis of ATL is based on clinical, epidemiological, and laboratory aspects [3]. Etiological diagnostic methods (based on the detection of parasites or their genetic material) are considered the gold standard; nevertheless, the sensitivity is not high but may increase when the tests are combined [6]. Regarding molecular methods, laboratory standardization is underway [7]. Nevertheless, implementing molecular methods is difficult in settings with limited resources [8].
The immunodiagnosis can be a potential tool to increase access and improve the ATL diagnosis [9–11]. The leishmanin skin test was an alternative used in clinical routines to detect current or past infections [2,3,12]. However, the discontinuity in the production of this antigen has caused problems in diagnosing ATL, mainly in cases of chronic evolution constituting a threat to the patient’s health. Without a test for the establishment of the diagnosis, they are often treated based on diagnostic assumptions. Further, the drugs used to treat ATL may cause severe side effects [1,13].
Conversely, serological tests may have advantages for diagnosing ATL, as they use a less invasive sample than skin biopsy and can be automated. They are quantitative and may be easily used at the point of care [14,15]. Assays using recombinant antigens are already integrated into the routine diagnosis of visceral leishmaniasis, and rapid immunochromatographic tests are available for use in patient care settings [16]. However, there is no consensus on the diagnosis of cutaneous leishmaniasis, and serological methods are not incorporated. Since patients with CL produce low levels of anti-Leishmania antibodies [17], there is an interest in developing serological tests that have high sensitivities.
The diagnosis of ATL is an open question in our midst, and the search for a solution is aligned with WHO’s goals for leishmaniasis detection, reporting, and treatment [18,19]. In this scenario, a serological test in ELISA format using the recombinant Lb6H to search for anti-IgG antibodies was standardized and validated to diagnose ATL. Furthermore, the option for the rLb6H antigen was based on its performance in a phase 1 study showing 100.0% sensitivity in ATL samples and 98.5% specificity in healthy controls [20]. Therefore, the importance of this study is evident, as it will fill the lack of a complementary diagnostic test for ATL, and in any case, the more tests available, the faster and more efficient the diagnosis and, consequently, the treatment will be.
Materials and methods
Ethics statement
The Ethics Committees that approved this research are 1) Faculdade de Medicina da Universidade de Sao Paulo (FMUSP), Sao Paulo state, approval number 2.530.363. 2) Instituto de Infectologia Emilio Ribas (IIER), Sao Paulo, Sao Paulo state, approval number 3.255.860). The prospective participants signed a written informed consent form. 3) Centro de Pesquisas Aggeu Magalhaes (CPqAM), FIOCRUZ, Recife, Pernambuco state approval number 2.638.758. The participants were obtained retrospectively, and the samples were coded to prevent their identification during or after data collection. 4) Fundacao de Medicina Tropical Dr. Heitor Vieira Dourado (FMT-HVD), Manaus, Amazonas state, approval number 2.584.959. The participants were obtained retrospectively, and the samples were coded to prevent their identification during or after data collection. 5) Hospital das Clinicas da Faculdade de Medicina da Universidade de Sao Paulo (HCFMUSP), approval number 4.717.914. Samples were selected from the SORO-IMT-FM-USP Biorepository. All the samples were coded to prevent their identification during or after data collection.
Study design
This diagnostic method study employed 1,091 serum samples from ATL patients, healthy asymptomatic individuals, and patients with potentially cross-reactive diseases collected in Brazil’s endemic and non-endemic leishmaniasis areas. At Instituto de Infectologia Emilio Ribas (IIER), Sao Paulo, Sao Paulo state, 37 ATL patients were prospectively recruited between 04/21/2019 and 10/28/2021. Centro de Pesquisas Aggeu Magalhaes (CPqAM), FIOCRUZ, Recife, Pernambuco state sent samples from 101 ATL patients retrospectively obtained. The data were accessed on 08/05/2021. Fundacao de Medicina Tropical Dr. Heitor Vieira Dourado (FMT-HVD), Manaus, Amazonas state, sent samples from 67 ATL patients and 32 healthy individuals retrospectively obtained. The data were accessed on 05/06/2021. Eight hundred fifty-four samples were selected from SORO-IMT-FM-USP Biorepository: 258 samples were obtained from ATL patients, 430 from healthy controls, and 166 from patients with other diseases. The data were accessed on 05/20/2021. The experiments were conducted at Instituto de Medicina Tropical da Faculdade de Medicina, Universidade de Sao Paulo, between 2018 and 2021.
Study subjects
Our cohort consisted of four panels comprising randomized and relabeled samples using study codes to preserve the participants’ anonymity and prevent bias. Panel 1 (S1 Fig) consisted of 70 samples of patients with ATL and 70 sera from healthy blood donors selected according to clinical-epidemiological criteria and routine blood bank tests. Panel 2 consisted of 393 samples from patients with ATL, with at least one positive laboratory test at the time of collection, success in the leishmaniasis therapeutic test, or clinical-epidemiological criterion (S2 Fig). ATL samples were collected in five collection centers from three states: Manaus, Amazonas state (N = 63), Recife, Pernambuco state (N = 80), and Sao Paulo, Sao Paulo state (N = 250). The collection centers are reference hospitals that treat patients from different regions. Panel 3 was composed of 392 samples of healthy asymptomatic individuals living in areas of different endemicities (S3 Fig). The samples of healthy individuals were collected in Manaus, Amazonas state, an endemic area for ATL (N = 32), Tres Lagoas, Mato Grosso do Sul state, an area of VL transmission (N = 90), and two districts in Sao Paulo, Sao Paulo state, one is non-endemic (N = 185), and the other, Engenheiro Marsilac, is a transmission area for CL (N = 85). Panel 3 samples were assayed for Chagas disease and rheumatoid factor, and the 369 samples, negative by both tests, were maintained for the specificity study. For the cross-reactivity study, panel 4 was used (S4 Fig). It was composed of 141 samples from nine different diseases: autoimmune disease (N = 20), Chagas disease (N = 23), histoplasmosis (N = 5), malaria (N = 11), infectious mononucleosis (N = 9), paracoccidioidomycosis (N = 30), syphilis (N = 13), toxoplasmosis (N = 20) and tuberculosis (N = 10), and 25 samples positive to rheumatoid factor.
Sample size calculation
The calculation of the minimum number of cases and controls to be included in the validation study followed the formula of Banoo et al. [21] and Smith et al. [22], considering sensitivity and specificity of 90%, a lower margin of 95% confidence interval of 85% and a test power of 90%. In this case, a sample size of at least 377 cases and 377 controls was required. In our previous study [20], we obtained a sensitivity of 100.0% (95% CI, 98.3–100.0) in patients with ATL, a specificity of 98.5% (95% CI 92.1–100.0), in healthy individuals, 92.0% (95% CI 87.5–95.3), in patients with other infections, and 93.6% (95% CI 90.1–96.2), in healthy individuals and patients with other infections.
Leishmania recombinant protein
Infectious Disease Research Institute, Seattle, USA (IDRI) gently provided the rLb6H used in this work. This antigen is a heat shock protein identified in 1995 while screening a genomic expression library of L. (V.) braziliensis with serum from a patient infected by L. braziliensis with mucosal leishmaniasis and constructed in the ZAP II bacteriophage [23]. In 2017, the protein gene was inserted into the vector pET17b, and for protein expression, the plasmid pET17b-His6-Lb6H was transformed into Escherichia coli BL-21 host cells [20]. The rLb6H protein was purified from the insoluble inclusion bodies by affinity chromatography using the nickel-nitrilotriacetic acid (Ni-NTA) protein purification system[20]. The purified protein was evaluated by dodecyl sulfate-polyacrylamide gel electrophoresis (SDS-PAGE) [24], quantified by BCA protein assay [25], and the reactivity against positive and negative sera was analyzed by Immunoblotting [26], at a concentration of 3 μg/mL (18 ng/well).
Enzyme-linked immunosorbent assay—ELISA
ELISA was performed according to Sato et al. [20] with some modifications. Briefly, 96-well half-area plates (Corning Costar, New York, USA) were sensitized with 50 μL/well rLb6H antigen diluted to 1 μg/mL in 0.06 M carbonate-bicarbonate buffer, pH 9.6. After overnight incubation in a moist chamber at 4°C, the plates were washed three times with phosphate-buffered saline containing 0.05% Tween 20 (Polyoxyethylene-sorbitan monolaurate, Sigma-Aldrich, St. Louis, EUA) (PBST). Next, the plates were blocked with 125 μL/well of PBST with 5% skimmed milk (PBSTM), incubated in a moist chamber for 2 hours at 37°C, and washed three times with PBST. After dilution at 1:100 in PBSTM, samples were applied in duplicate (50 μL/well) and incubated at 37°C in a moist chamber for 30 minutes, followed by five washes with PBST. Next, anti-human IgG-peroxidase conjugate (Merck 401445, USA), diluted at 1:30,000 in PBSTM (50 μL/well), was added, and the plates were incubated at 37ºC in a moist chamber for 30 minutes, followed by another wash cycle. For reaction development, 50 μL/well of TMB/H2O2 (tetramethylbenzidine/H2O2) chromogen (Novex-Life Technologies, Carlsbad, CA, USA) was added, and the plates were incubated for 10 minutes, protected from light and at 25°C. The reaction was stopped by adding 25 μL/well of 2N H2SO4. Absorbances were measured in the ELISA reader (Multiskan GO, Thermo Scientific, Finland) at 450 nm. Each plate had positive and negative controls and blank (PBSTL). Sera absorbance values were normalized based on the absorbance of the positive standard serum tested in duplicate on all the rLb6H-ELISA test plates. For each sample, the percentage of the absorbance of the positive standard (ABS%) was calculated:
Diagnostic performance of rLb6H-ELISA
The ROC (receiver operating characteristic) curves were constructed using the ABS% values of the samples from panel 1, obtained in rLb6H-ELISA. These ROC curves determined the test’s cut-off, sensitivity, specificity, and 95% confidence intervals.
The reactivity index (RI) was calculated by dividing the values of the "ABS%" of each sample by the cut-off obtained in the ROC curve, considering the test reagent when RI ≥ 1. The following equation was used:
Reproducibility, repeatability, homogeneity, and stability of sensitized plates
Five operators performed ten determinations of one positive sample for ATL and one negative on five consecutive days for the reproducibility study. For repeatability, 30 determinations of positive and negative controls were performed on the same plate. The plate’s homogeneity was evaluated by applying the positive and negative controls in 15 different plate wells. The coefficient of variation (CV% = standard deviation/ mean of absorbance x 100) was calculated for each parameter.
The stability of sensitized plates was studied by employing 14 plates sensitized with rLb6H (day 0) and storing them at -20ºC (five plates), 4ºC (five plates), and 37ºC (four plates). At -20ºC and 4ºC, the experiments were carried out on days 1, 7, 30, 90, and 180. At 37ºC, the tests were carried out on days 1, 3, 5 and 7. In addition, control plates were sensitized one day before each experiment [27]. Negative and positive controls were assayed in triplicate for the stability test, and a panel of 12 samples from ATL patients with different titers and nine samples from healthy individuals were assayed in duplicate. The stability of the plates at each storage temperature was evaluated by constructing linear regression graphs for each sample tested using the values ABS% obtained at each storage time, both for the test plates and the control plates.
Validation of rLb6H-ELISA with samples from different regions of Brazil
For validation, panels 2 (ATL patients), 3 (healthy controls), and 4 (other disease patients) were assayed by rLb6H-ELISA.
Statistical analysis
The results were evaluated using statistical software. GraphPad Prism, version 9.3.1 for Windows (GraphPad Software Inc., San Diego, CA, USA), was used to construct ROC curve and linear regressions; to calculate sensitivity, specificity, positive and negative likelihood ratios with 95% confidence intervals (95% CI); to compare the ABS% obtained with positive and negative samples from Reference Panel using Mann-Whitney Rank Sum Test. R version 4.3.0 for Windows and RStudio version 2023.02.3 [28] for Windows were used to test for the normal distribution of the values (Shapiro-Wilk test) and the homogeneity of variances (Levene test), to construct Venn diagrams, the map, pie charts, and bar graphs. GraphPad/ Quickcalcs [29] was used to calculate the confidence interval of proportions, compare two proportions with Fisher’s exact test, and calculate the agreement between diagnostic tests and rLb6H-ELISA using the Kappa index [30,31]. MedCalc [32] was used to compare the sensitivity of diagnostic tests and rLb6H-ELISA using the McNemar chi-square test and compare three or more proportions with the chi-square test. A significance level of 0.05 was considered (p < 0.05).
Results
American tegumentary leishmaniasis patient’s characterization
Table 1 shows the demographic and clinical data from the ATL patients evaluated in this study.
Of the ATL samples from Panel 1, 64.3% (45/70) were identified by species (Table 2), and 100% had a known clinical presentation. The infecting species was identified through different methods, according to the collection center. At Instituto Evandro Chagas (IEC), Belem, Para state (N = 14), the species was identified by monoclonal antibodies; at Fundacao de Medicina Tropical Dr. Heitor Vieira Dourado (FMT-HVD), Manaus, Amazonas state (N = 4), by PCR–RFLP Hsp70; at Centro de Pesquisas Aggeu Magalhaes (CPqAM), FIOCRUZ, Recife, Pernambuco state (N = 21), by schizodemas; at Instituto de Infectologia Emilio Ribas (IIER), Sao Paulo, Sao Paulo state (N = 6), by restriction enzymes–Hae III (Table 2). At the IIER and the Hospital das Clinicas da Faculdade de Medicina da Universidade de Sao Paulo (HCFMUSP), Sao Paulo, SP, 25 patients were selected by clinical-epidemiological criteria complemented by laboratory diagnosis. ATL patients proceeded from 19 Brazilian states and three neighboring countries (Fig 1).
ATL–American tegumentary leishmaniasis. The map was generated through RStudio, version 4.3.2, using the geobr package from rstudio.com [33].
Lb6H recombinant antigen reactivity evaluation
The electrophoretic profile in SDS-PAGE is seen in Fig 2A, where the presence of the band corresponding to the rLb6H is seen between the molecular masses of 97 kDa and 66 kDa, with the expected mass of approximately 79 kDa. Fig 2B shows the results obtained in the Immunoblotting; in strip two, a strong band is observed between the high titer leishmaniasis positive control and the band of interest (approximately 79 kDa). In strip three, the reaction with the low titer leishmaniasis positive control was less intense. No band was observed in strip four, which was tested with the negative control, evidencing the absence of reaction.
A. Electrophoretic profile of rLb6H protein on SDS-PAGE, applying 18 ng of protein per well. B. Immunoblotting reactivity profile of rLb6H. 1- Molecular weight control stained with Ponceau S. 2—High titer leishmaniasis positive control. 3 –Low titer leishmaniasis positive control. 4—Negative control.
rLb6H-ELISA presented high accuracy for ATL diagnosis
The ABS% obtained with samples from the reference panel (Panel 1) showed that rLb6H-ELISA significantly discriminated between healthy controls and ATL patients’ samples (p < 0.0001; Mann–Whitney test) (Fig 3A) and demonstrated great accuracy (area under the curve = 1) (Fig 3B). Table 3 shows the diagnostic performance for the rLb6H ELISA, with a sensitivity of 98.6% (95% CI: 92.3–99.9) and a specificity of 100.0% (95% CI: 94.8–100.0) at the cut-off point of 2.8. Considering the ATL patients’ clinical form, 74.3% had the cutaneous form, and 25.7% had the mucosal form. One patient with ML caused by L. braziliensis was not detected.
A—Percentage of absorbance of the positive standard (ABS%) obtained in the rLb6H-ELISA; *p < 0.0001 compared to the control, Mann–Whitney test. B—ROC (receiver operating characteristic) curve constructed with ABS% values; ** p-value < 0.0001 means the test significantly discriminates between American tegumentary leishmaniasis patients and controls.
The rLb6H-ELISA meets the requirements for a serological test
The reproducibility, repeatability, and homogeneity were evaluated by calculating the coefficient of variation (CV%) of the absorbances obtained by testing replicates of a positive and a negative serum from panel 1 (S1 Fig), as described above. In the positive samples, the rLb6H-ELISA was reproducible, with a CV% of 18.0% in reproducibility (S5A Fig), 8.2% in repeatability (S5B Fig), and 4.2% in homogeneity (S5C Fig) (S1 Table). These values are within the recommended range (< 25%, < 20%, and < 20%, respectively), confirming the accuracy of the measurements. Regarding the negative samples, CV% values were > 25% in reproducibility and > 20% in homogeneity, which may be justified by the increase of imprecision (error) of measurement with the decrease in the concentration of the analyte, leading to a higher coefficient of variation in the repetitions [34]. Despite this, in repeatability, a CV% of 13.9% was obtained.
In assessing the stability of the recombinant antigen on the plate over time, the sensitized plates were stable for 180 days when stored at -20°C and 4°C. Stability at 4°C is advantageous for storing and transporting sensitized plates. In the accelerated stability study, the plates remained stable, without loss of reactivity, for up to seven days of storage at 37°C, indicating the validity for 12 months [27].
Validation of the rLb6H-ELISA for the Diagnosis of ATL with samples from different regions of Brazil
After an initial evaluation with the reference panel, the rLb6H-ELISA was validated using panels 2, 3, and 4 (S2–S4 Figs). In the 393 samples from patients with ATL from different regions of Brazil (panel 2), the sensitivity was 84.0% (95% CI: 80,0–87,3). Although the samples came from different endemic areas, similar sensitivity was observed at different collection centers (chi-square test: p = 0.1259) (Table 4).
The samples were distributed in four groups, according to the laboratory tests done at the collection centers: immunological diagnosis (IFAT, ELISA, LST) (S6 Fig), etiological diagnosis (direct examination, culture, immunohistochemistry, PCR) (S7 Fig), success in the leishmaniasis therapeutic test and clinical-epidemiological criterion. In summary, 59.0% of the 393 patients had at least one positive etiological test; 68.4% had been positive by at least one immunological test; 2.3% had a positive therapeutic test, and the clinical-epidemiological criterion classified 2.5%.
Regarding the clinical form, 249 (63.4%) patients had cutaneous leishmaniasis, 86 (21.9%) mucosal leishmaniasis, and 58 (14.7%) were classified as tegumentary leishmaniasis. The rLb6H-ELISA showed a significant difference in the percentage of detection of the CL (86.4%; 95% CI: 81.5–90.1) and ML (76.7%; 95% CI: 66.7–84.5) samples, being more sensitive in the detection of antibodies in the cutaneous form (p = 0.0421, Fisher’s exact test).
In the analysis of panel 3, the 369 samples that were negative for the presence of rheumatoid factor and anti-Trypanosoma cruzi IgG antibodies were considered, and the specificity obtained was 92,4% (95% CI: 89,2–94,7). Across the different collection centers, a significant difference was observed (p < 0.0001, chi-square test), with high specificity in samples from Sao Paulo, Engenheiro Marsilac, Sao Paulo state, and Manaus, Amazonas state (Table 5).
The 166 samples from panel 4 were used in the interference analysis to evaluate the rLb6H-ELISA cross-reactivity, giving 13.9% (95% CI: 9.3–20.0) of positive reactions that exhibited lower-level responses (RI varying from 1.0 to 3.1, and median of 1.7). The diseases that showed the highest proportion of positive reactions were tuberculosis (median positive RI = 2.5), malaria (median positive RI = 1.4), and paracoccidioidomycosis (median positive RI = 1.6) (p = 0.0106, Chi-square test) (Table 6 and Fig 4).
The black dotted line represents the cut-off.
Evaluation of rLb6H-ELISA as a complementary test for ATL diagnosis
From the 393 ATL patient samples tested by rLb6H-ELISA, 153 were tested by direct exam, 104 by culture, 74 by leishmanin skin test, 51 by immunohistochemistry, 237 by L. major-like-ELISA, 282 by immunofluorescent antibody test, and 167 by PCR in the collection centers. Comparing the results of the tests mentioned above with the ones obtained by rLb6H-ELISA (S8 Fig), no significant difference was obtained with L. major-like-ELISA (p = 1.000, McNemar test) and PCR (p = 0.1093, McNemar test). The more significant difference was obtained with culture (p = 0.0001, McNemar test), immunohistochemistry (p = 0.0001, McNemar test), immunofluorescent antibody test (p = 0.0010, McNemar test), and leishmanin skin test (p = 0.0371, McNemar test) with a higher positivity by rLb6H-ELISA (S8 Fig).
Comparing the results, we observed that many positive samples in the rLb6H-ELISA were not detected by the diagnostic tests used in the collection centers. Fig 5 shows the gain in diagnosing the ATL patients with rLb6H-ELISA that varied according to the reference test. Higher gain values were obtained with immunohistochemistry (43%), immunofluorescent antibody test (39%), culture (39%), and direct exam (22%). On the other hand, lower gain values were found with L. major-like-ELISA (12%), leishmanin skin test (8%), and PCR (8%). Notably, the high sensitivity of PCR was due to the use of more than one target or different sample types in approximately 50% of patients.
Blue bars show the number of samples that were positive by the test at the collection centers. Dark pink bars show the number of samples that were negative by the test at the collection centers but positive by rLb6H-ELISA. The percentages of positivity in the collection centers appear between the colored bars. The percentages at the top of the bars refer to the sum of positive samples at the collection centers plus positive samples in the rLb6H-ELISA that were negative by the previous test. IFAT**–immunofluorescent antibody test with L. major-like (Sao Paulo) and L. braziliensis (Manaus); IHC–immunohistochemistry; ELISA*–enzyme-linked immunosorbent assay with L. major-like; PCR–polymerase chain reaction; LST–leishmanin skin test.
Discussion
This study evaluated the performance of the recombinant antigen Lb6H derived from the gene sequence of L. (V.) braziliensis in the ELISA platform for potential application in clinical practice to aid in diagnosing ATL.
The rLb6H-ELISA diagnostic performance obtained in this study was consistent with Sato et al. [20], who used another batch of the same antigen in the phase 1 study. The 95% CI of the positive likelihood ratio (8.7–∞) indicated strong evidence for disease in ATL patients, and the low negative likelihood ratio value (0.01–95% CI: 0.001–0.1) indicated no association with the presence of the disease in healthy individuals [35]. It is worth emphasizing the importance of the likelihood ratio in clinical practice, as it allows the clinician to quantify the probability of disease for any patient. Furthermore, their values do not depend on the prevalence of the disease, as occurs with the positive and negative predictive values, and, therefore, do not vary in different populations [36].
After an initial evaluation with the reference panel, the rLb6H-ELISA was validated using a cohort of samples from cases and controls from 19 Brazilian states and three neighboring countries. The large number of samples from patients with ATL and controls used in the present study represents a significant advantage compared with most studies in Brazil, where a small number of samples are analyzed, mainly from patients with ATL, as described in a recent systematic review [15]. Another important aspect of this study was the detailed characterization of most patients and samples, providing greater robustness to the results. We also selected cases representing the entire spectrum of disease presentation and healthy controls from endemic and non-endemic areas of Brazil to reduce the bias, increasing the sensitivity and specificity of the test [37].
In the diagnosis of cutaneous leishmaniasis, the gold standard tests are specific but lack a high sensitivity, and that is why the collection centers carry out more than one laboratory test to increase the detection of suspected cases. Most ATL-positive samples were collected retrospectively, and it was not feasible to standardize the collection and diagnostic tests performed. Composite reference standards (CRS) increase sensitivity at the expense of specificity unless all component tests have perfect specificity. [38] ATL cases were based on clinical, epidemiological, and laboratory data to reduce bias in using CRS. Although the ATL samples came from different endemic areas, similar test performance was observed regarding sensitivity at different collection centers (p = 0.1259). Regarding clinical forms, rLb6H-ELISA was more sensitive in the detection of antibodies in the cutaneous form (86.4%) compared with ML (76.7%) (p = 0.042). The good performance of the rLb6H-ELISA in patients with CL should be highlighted since these patients produce low levels of anti-Leishmania antibodies [17], resulting in variable or low sensitivity of the serological tests [2,39]. Although with a few samples, similar results were obtained in a study with Hsp70 from L. (V.) braziliensis in 30 patients with CL and 20 with ML, with a sensitivity of 83.3% and 85.0%, respectively [40].
In analyzing the 369 samples from healthy controls residing in areas of different endemicities (panel 3), a 92.4% specificity was achieved. Differently from what was obtained with sensitivity, there was a significant difference in test specificity with samples from different centers (p < 0.0001, chi-square test). Lower levels of specificity were found in Tres Lagoas, MS (80.9%), which is an endemic municipality for VL [41], and it is known that rLb6H may also detect antibodies in the visceral form [20]. Although all participants were asymptomatic, there may be cases of spontaneous cure and subclinical infections, which would be detected as false positives. Although previous studies showed that rLb6H may react with VL samples with a positivity of 89.4% (16), we used another batch of the Lb6H antigen, obtaining 66.7% positivity (Fisher’s exact test, p = 0.0025).VL samples (S1 Dataset).
Previous reports have suggested variable sensitivity and specificity using variable recombinant and total antigens for antibody search in ATL. Using L. braziliensis Hsp70-ELISA in 50 patients with cutaneous and mucosal forms, a 14% to 84% sensitivity was reported in samples from Cuzco, Peru, and 92% to 100% specificity in 20 sera from residents of Spain and Peru [40]. In Southeastern Brazil, 63% to 94% sensitivity was obtained with L. braziliensis Hsp83.1-ELISA in 65 patients with cutaneous and mucosal forms caused by L. braziliensis from Belo Horizonte, MG, and 90% to 96% specificity in 50 samples from the non-endemic area of Belo Horizonte [42]. Using total antigen from L. braziliensis and L. amazonensis promastigotes, ELISA sensitivity ranged from 19% to 81% in 58 samples from Belo Horizonte, MG and Vitória, ES, and specificity, from 58 to 71%, in 49 samples from Uberlandia, MG [43]. With L. braziliensis amastigotes, the sensitivity was 98% in samples from endemic areas of Argentina, and the specificity was 98.4% in samples from Japan [44]. The hypothetical protein LiHypA in an ELISA gave 100% sensitivity in 57 ATL leishmaniasis patients and 98.2% specificity in 40 healthy controls and 15 Chagas disease patients [45]. Another hypothetical protein, rLbHyM, was evaluated by ELISA in 45 ATL patients, 50 healthy controls, and 10 Chagas disease patients, with 100% sensitivity and 98% specificity [46]. As stated in the systematic review of Freire et al., among the 38 studies included in the synthesis, most were proof-of-concept and phase I studies [15], with a small number of samples.
The evaluation of potential cross-reactivity of the rLb6H-ELISA showed 13.9% (23/166) of positive reactions with a RI median of 1.7. More positive reactions were obtained in malaria, tuberculosis, and paracoccidioidomycosis patients, corroborating other studies using crude and recombinant Leishmania antigens [3,11]. rLb6H-ELISA provided low (8.7%, 2/23) cross-reactivity with Chagas disease, similar to that achieved with rHsp70 (10%), which was the most specific recombinant among those tested by Souza [11]. Conversely, around 100% cross-reactivity with Chagas disease was observed with L. (V.) braziliensis amastigotes antigens [44]; 63.0% with antigens of L. infantum chagasi promastigotes [11]; 75.8% with L. major-like promastigotes [20]. Concerning the cross-reactivity observed with malaria and Chagas disease, the possibility of a latent infection with Leishmania cannot be excluded due to the overlapping of endemic areas. Due to the high homology between the sequences of Lb6H and T. cruzi heat-shock proteins, one could expect a higher cross-reactivity than obtained in sera from patients with Chagas disease here and by other reports [11,47,48]. However, the possible discontinuity of the similar sequences in the T. cruzi protein may give rise to different immunodominant epitopes that elicit antibodies that do not cross-react with rLb6H.
The results of this study, using samples from different regions of Brazil, allow us to state that, in the absence of the LST, the rLb6H-ELISA may be a better alternative for diagnosing ATL since in endemic areas, LST positivity may vary between 20 and 30% in the absence of an active lesion or scar since the test can be positive in cured, asymptomatic individuals, in individuals allergic to the diluent, and with other diseases (Chagas disease, sporotrichosis, multibacillary leprosy, tuberculosis, chromomycosis, among others) [38]. Also, Brazilian studies applying the LST test reported 77,0%-89,1% sensitivity and 60,0%-71,4% specificity [49] similar to or lower than this study’s.
One limitation was the diagnostic tests used to characterize ATL patients obtained in different regions of Brazil (panel 2), which were different according to the collection center. Another limitation is the cross-reactivity study (panel 4) due to the lack of samples of cancer patients with skin or oral/nasal mucosa ulcers and the small number of samples of some diseases. The lack of healthy controls from Recife was also a limitation.
The good diagnostic performance, reproducibility characteristics, antigen stability, and validation with samples from endemic areas qualify this rLb6H-ELISA as a good complementary test in diagnosing ATL.
In conclusion, in the scenario where new tests to complement the diagnosis of ATL are needed [6], the rLb6H-ELISA emerges as an option, increasing access availability and establishing a timely and more accurate ATL diagnosis.
Supporting information
S1 Fig. Positive and Negative serum samples used to construct the ROC (receiver operating characteristic) curve (Panel 1).
* Diagnosis based on clinical-laboratory criteria. **Diagnosis based on positive parasite culture and Leishmania species identification. n = number of samples.
https://doi.org/10.1371/journal.pone.0304268.s002
(TIF)
S2 Fig. Serum samples from American Tegumentary Leishmaniasis patients (Panel 2) obtained in different regions of Brazil.
FMTHVD—Fundacao de Medicina Tropical Dr. Heitor Vieira Dourado, Manaus, AM; CPqAM -Centro de Pesquisas Aggeu Magalhaes, FIOCRUZ, Recife, PE; HCFMUSP—Hospital das Clínicas da Faculdade de Medicina da Universidade de Sao Paulo, Sao Paulo, SP; IIER—Instituto de Infectologia Emilio Ribas, Sao Paulo, SP; SCMSP–Santa Casa de Misericórdia de Sao Paulo, Sao Paulo, SP. n = number of samples.
https://doi.org/10.1371/journal.pone.0304268.s003
(TIF)
S3 Fig. Serum samples from negative control individuals (Panel 3) obtained in different regions of Brazil.
n = number of samples.
https://doi.org/10.1371/journal.pone.0304268.s004
(TIF)
S4 Fig. Serum samples from patients with other diseases (Panel 4).
n = number of samples.
https://doi.org/10.1371/journal.pone.0304268.s005
(TIF)
S5 Fig. Distribution of the coefficients of variations obtained in reproducibility, repeatability, and homogeneity studies.
A–reproducibility, B—repeatability, C–homogeneity. ABS%—Percentage of absorbance of the positive standard.
https://doi.org/10.1371/journal.pone.0304268.s006
(TIF)
S6 Fig. Venn diagrams of samples with a positive immunological diagnosis (N = 269).
IFAT–Indirect Immunofluorescent antibody test; ELISA–enzyme-linked immunosorbent assay; LST–leishmanin skin test.
https://doi.org/10.1371/journal.pone.0304268.s007
(TIF)
S7 Fig. Venn diagram of samples with a positive etiologic diagnosis (N = 223).
IHC–Immunohistochemistry; PCR–polymerase chain reaction.
https://doi.org/10.1371/journal.pone.0304268.s008
(TIF)
S8 Fig. Pie charts show the distribution of positivity obtained by the tests used in each collection center compared with rLb6H-ELISA.
IFAT–immunofluorescent antibody test with L. major-like (Sao Paulo) and L. braziliensis (Manaus); IHC–immunohistochemistry; ELISA–enzyme-linked immunosorbent assay with L. major-like; PCR–polymerase chain reaction; LST–leishmanin skin test.
https://doi.org/10.1371/journal.pone.0304268.s009
(TIF)
S1 Table. Coefficients of variation obtained in reproducibility, repeatability, and homogeneity studies.
1 –Mean optical densities (OD); 2—SD Standard deviation; 3—CV Coefficient of variation; 4 –P samples from patients with American Tegumentary Leishmaniasis (N = 1); 5 –N samples of healthy individuals (N = 1).
https://doi.org/10.1371/journal.pone.0304268.s010
(PDF)
S1 Dataset. Raw data obtained by rLb6H-ELISA applied to the various groups of samples studied.
ID–identification number; CL–cutaneous leishmaniasis; ML–mucosal leishmaniasis; MCL–mucocutaneous leishmaniasis; ATL–American tegumentary leishmaniasis; Pos–positive; Neg–negative; M–male; F–female; MI–missing information; ND–not done; Age–in years; ELISA binary result: 1– positive; 0 –negative; Evolution time–in months; IFAT–immunofluorescent antibody test; RF–rheumatoid factor; Kalazar Detect and IT-Leish rK39-based RDT: 0—negative; 1, 2, 3 –positive; DAT–direct agglutination test; RI–reactivity index; VL–visceral leishmaniasis.
https://doi.org/10.1371/journal.pone.0304268.s011
(XLSX)
Acknowledgments
The authors thank Taiana Cunha Ribeiro MD, Ícaro Santos Oliveira MD, and Lívia Vieira de Almeida MD for their support in obtaining clinical-epidemiological-laboratory patient data.
References
- 1. Burza S, Croft SL, Boelaert M. Leishmaniasis. The Lancet. 2018;392: 951–970. pmid:30126638
- 2. Goto H, Lindoso JAL. Current diagnosis and treatment of cutaneous and mucocutaneous leishmaniasis. Expert Rev Anti Infect Ther. 2010;8: 419–433. pmid:20377337
- 3. Brasil. Ministério da Saúde. Manual De Vigilância Da Leishmaniose Tegumentar. Manual de vigilância da leishmaniose tegumentar. 2017: 189. Available: http://pesquisa.bvsalud.org/bvsms/resource/pt/mis-38798#.WmSchsQMGLw.mendeley.
- 4. World Health organization. Leishmaniasis. In: World Health organization [Internet]. 20 May 2022. Available: https://www.who.int/news-room/fact-sheets/detail/leishmaniasis.
- 5.
Ministerio de saúde—Saúde da A—Z. Casos de leishmaniose tegumentar. Brasil, Grandes Regiões e Unidades Federadas. 1990 a 2021. 2023 [cited 7 Feb 2023]. https://www.gov.br/saude/pt-br/assuntos/saude-de-a-a-z/l/lt/situacao-epidemiologica/lt-casos.pdf.
- 6. Gonzalez K, Calzada JE, Díaz R, Paz H, García V, Miranda A, et al. Performance of immunohistochemistry as a useful tool for the diagnosis of cutaneous leishmaniasis in Panama, Central America. Parasitol Int. 2019;71: 46–52. pmid:30885795
- 7. Menezes Maíra. Fiocruz and PAHO standardize diagnosis for cutaneous leishmaniasis in the Americas. In: https://portal.fiocruz.br/en/news. 30 Nov 2023.
- 8. de Vries HJC, Schallig HD. Cutaneous Leishmaniasis: A 2022 Updated Narrative Review into Diagnosis and Management Developments. Am J Clin Dermatol. 2022;23: 823–840. pmid:36103050
- 9. Reed SG, Badaró R, Carvalho EM, Lisboa A, Teixeira R, Johnson WD, et al. Selection of a Skin Test Antigen for American Visceral Leishmaniasis *. Am J Trop Med Hyg. 1986;35: 79–85. pmid:3946739
- 10. Veland N, Miranda-Verastegui C, Ramos AP, Valencia BM, Llanos-Cuentas A, Low DE, et al. Clinical and Demographic Stratification of Test Performance: A Pooled Analysis of Five Laboratory Diagnostic Methods for American Cutaneous Leishmaniasis. Am J Trop Med Hyg. 2010;83: 345–350. pmid:20682880
- 11. Souza AP, Soto M, Costa JML, Boaventura VS, de Oliveira CI, Cristal JR, et al. Towards a More Precise Serological Diagnosis of Human Tegumentary Leishmaniasis Using Leishmania Recombinant Proteins. Zamboni DS, editor. PLoS One. 2013;8: e66110. pmid:23776617
- 12. Reithinger R, Dujardin J-C, Louzir H, Pirmez C, Alexander B, Brooker S. Cutaneous leishmaniasis. Lancet Infect Dis. 2007;7: 581–596. pmid:17714672
- 13. de Menezes JPB, Guedes CES, Petersen AL de OA, Fraga DBM, Veras PST. Advances in Development of New Treatment for Leishmaniasis. Biomed Res Int. 2015;2015: 815023. pmid:26078965
- 14. Tavares CAP, Fernandes AP, Melo MN. Molecular diagnosis of leishmaniasis. Expert Rev Mol Diagn. 2003;3: 657–667. pmid:14510185
- 15. Freire ML, Rêgo FD, Cota G, Pascoal-Xavier MA, Oliveira E. Potential antigenic targets used in immunological tests for diagnosis of tegumentary leishmaniasis: A systematic review. Schriefer A, editor. PLoS One. 2021;16: e0251956. pmid:34043671
- 16. Cunningham J, Hasker E, Das P, El Safi S, Goto H, Mondal D, et al. A Global Comparative Evaluation of Commercial Immunochromatographic Rapid Diagnostic Tests for Visceral Leishmaniasis. Clinical Infectious Diseases. 2012;55: 1312–1319. pmid:22942208
- 17. Torres-Guerrero E, Quintanilla-Cedillo MR, Ruiz-Esmenjaud J, Arenas R. Leishmaniasis: a review. F1000Res. 2017;6: 750. pmid:28649370
- 18. World Health Organization. TARGET PRODUCT PROFILE for a point-of-care diagnostic test for dermal leishmaniases. 2022. Available: https://iris.who.int/bitstream/handle/10665/353980/9789240045224-eng.pdf?sequence=1.
- 19. World Health Organization. Ending the neglect to attain the Sustainable Development Goals A road map for neglected tropical diseases 2021–2030. 2020. Available: http://apps.who.int/bookorders.
- 20. Sato CM, Sanchez MCA, Celeste BJ, Duthie MS, Guderian J, Reed SG, et al. Use of Recombinant Antigens for Sensitive Serodiagnosis of American Tegumentary Leishmaniasis Caused by Different Leishmania Species. Gilligan P, editor. J Clin Microbiol. 2017;55: 495–503. pmid:27927927
- 21. Banoo S, Bell D, Bossuyt P, Herring A, Mabey D, Poole F, et al. Evaluation of diagnostic tests for infectious diseases: general principles. Nat Rev Microbiol. 2006;4: S20–S32. pmid:17366684
- 22.
Smith Peter G., Morrow Richard H., Ross David A. Field Trials of Health Interventions. Smith PG, Morrow RH, Ross DA, editors. Oxford University Press; 2015. https://doi.org/10.1093/med/9780198732860.001.0001 pmid:26225404
- 23. Skeiky YA, Benson DR, Guderian JA, Whittle JA, Bacelar O, Carvalho EM, et al. Immune responses of leishmaniasis patients to heat shock proteins of Leishmania species and humans. Infect Immun. 1995;63: 4105–4114. pmid:7558326
- 24. Laemmli UK. Cleavage of structural proteins during the assembly of the head of bacteriophage T4. Nature. 1970;227: 680–5. pmid:5432063
- 25. Fisher Scientific T. Pierce™ BCA Protein Assay Kit. 2020. Available: https://www.thermofisher.com/document-connect/document-connect.html?url=https://assets.thermofisher.com/TFS-Assets%2FLSG%2Fmanuals%2FMAN0011430_Pierce_BCA_Protein_Asy_UG.pdf.
- 26. Towbin H, Staehelin T, Gordon J. Electrophoretic transfer of proteins from polyacrylamide gels to nitrocellulose sheets: procedure and some applications. Proceedings of the National Academy of Sciences. 1979;76: 4350–4354. pmid:388439
- 27.
Meneghisse CS. Boas práticas de fabricação e o processo de validação no desenvolvimento e produção de kit imunodiagnóstico. Universidade de São Paulo. 2007. https://doi.org/10.11606/T.9.2007.tde-07052008-170634
- 28.
R: The R Project for Statistical Computing. [cited 30 May 2023]. https://www.r-project.org/.
- 29.
GraphPad Software. [cited 30 May 2023]. https://www.graphpad.com/quickcalcs/.
- 30.
Fleiss JL, Levin B, Paik MC. Statistical Methods for Rates and Proportions. Wiley; 2003. https://doi.org/10.1002/0471445428
- 31. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33: 159–74. Available: http://www.ncbi.nlm.nih.gov/pubmed/843571. pmid:843571
- 32.
MedCalc’s Free statistical calculators. [cited 30 May 2023]. https://www.medcalc.org/calc/.
- 33. Pereira R, Goncalves C, Pereira P, Carvalho G, Arruda R, Nascimento I, et al. geobr: Download Official Spatial Data Sets of Brazil. In: https://ipeagit.github.io/geobr/ [Internet]. 2024 [cited 25 Mar 2024].
- 34.
Crowther JR. ELISA Guidebook, The. New Jersey: Humana Press; 2000.
- 35. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ. 2004;329: 168–169. pmid:15258077
- 36. Attia J. Moving beyond sensitivity and specificity: using likelihood ratios to help interpret diagnostic tests. Aust Prescr. 2003/10/01. 2003;26: 111–113.
- 37. Delgado-Rodriguez M. Bias. J Epidemiol Community Health (1978). 2004;58: 635–641. pmid:15252064
- 38. Schiller I, van Smeden M, Hadgu A, Libman M, Reitsma JB, Dendukuri N. Bias due to composite reference standards in diagnostic accuracy studies. Stat Med. 2016;35: 1454–1470. pmid:26555849
- 39.
Interativo Atlas. Atlas interativo de leishmaniose nas Américas: aspectos clínicos e diagnósticos diferenciais. Atlas interativo de leishmaniose nas Américas: aspectos clínicos e diagnósticos diferenciais. Pan American Health Organization; 2021. https://doi.org/10.37774/9789275721902
- 40. Zurita AI, Rodríguez J, Piñero JE, Pacheco R, Carmelo E, Del Castillo A, et al. Cloning and characterization of the Leishmania (viannia) braziliensis HSP70 gene. Diagnostic use of the C-terminal fragment rlb70(513–663). Journal of Parasitology. 2003;89: 372–378. pmid:12760657
- 41. Lopez LFB, Carneiro LEP, Lope ALK, Brighenti KB, Flores EF, Zuque MA da S, et al. EPIDEMIOLOGIA DA LEISHMANIOSE VISCERAL NO MUNICIPIO DE TRÊS LAGOAS ‐ MATO GROSSO DO SUL, UMA NOVA REGIÃO ENDÊMICA NO BRASIL. The Brazilian Journal of Infectious Diseases. 2021;25: 101213.
- 42. Menezes-Souza D, De Mendes TAO, De Gomes MS, Reis-Cunha JL, Nagem RAP, Carneiro CM, et al. Epitope mapping of the HSP83.1 protein of Leishmania braziliensis discloses novel targets for immunodiagnosis of tegumentary and visceral clinical forms of leishmaniasis. Clinical and Vaccine Immunology. 2014;21: 949–959. pmid:24807053
- 43. Gomes-Silva A, Souza MA, Afonso-Cardoso SR, Andrade LR, Dietze R, Lemos E, et al. Serological reactivity of different antigenic preparations of Leishmania (Leishmania) amazonensis and the Leishmania braziliensis complex. Rev Soc Bras Med Trop. 2008;41: 135–141. pmid:18545832
- 44. Bracamonte ME, Álvarez AM, Sosa AM, Hoyos CL, Lauthier JJ, Cajal SP, et al. High performance of an enzyme linked immunosorbent assay for American tegumentary leishmaniasis diagnosis with Leishmania (Viannia) braziliensis amastigotes membrane crude antigens. PLoS One. 2020;15: 1–12. pmid:32379842
- 45. Carvalho AMRS, Costa LE, Salles BCS, Santos TTO, Ramos FF, Lima MP, et al. An ELISA immunoassay employing a conserved Leishmania hypothetical protein for the serodiagnosis of visceral and tegumentary leishmaniasis in dogs and humans. Cell Immunol. 2017;318: 42–48. pmid:28602279
- 46. Lima MP, Costa LE, Duarte MC, Menezes-Souza D, Salles BCS, de Oliveira Santos TT, et al. Evaluation of a hypothetical protein for serodiagnosis and as a potential marker for post-treatment serological evaluation of tegumentary leishmaniasis patients. Parasitol Res. 2017;116: 1197–1206. pmid:28150041
- 47. Celeste BJ, Angel SO, Castro LGM, Gidlund M, Goto H. Leishmania infantum heat shock protein 83 for the serodiagnosis of tegumentary leishmaniasis. Brazilian Journal of Medical and Biological Research. 2004;37: 1591–1593. pmid:15517072
- 48. Magalhães FB, Castro Neto AL, Nascimento MB, Santos WJT, Medeiros ZM, Lima Neto AS, et al. Evaluation of a new set of recombinant antigens for the serological diagnosis of human and canine visceral leishmaniasis. PLoS One. 2017;12. pmid:28957332
- 49. Espir TT, Guerreiro TS, Naiff M de F, Figueira L de P, Soares FV, da Silva SS, et al. Evaluation of different diagnostic methods of American Cutaneous Leishmaniasis in the Brazilian Amazon. Exp Parasitol. 2016;167: 1–6. pmid:27119738