Protocol and registration

This systematic review and meta-analysis was carried out according to the PRISMA-DTA guidelines (Preferred Reporting Items for Systematic Reviews and Meta-analysis of Diagnostic Test Accuracy Studies -The PRISMA-DTA Statement) [15] for the abstract and the body of the manuscript (S1 and S2 Checklists). The protocol was registered in the PROSPERO database (International Prospective Register of Systematic Reviews) with number CRD42020186588.

Eligibility criteria

The search included studies that estimated sensitivity and specificity of ELISA or RDT index tests for chronic CD, with participants over five years old, patients with chronic CD, and patients without this disease; studies conducted in endemic and non-endemic areas for CD, that described the reference standards used, studies with a cross-sectional design and a case-control type; written in English, Spanish and Portuguese, published between 2010 and 2020; with research done with volunteers and with samples that included humans. Studies indicating that patients were receiving treatment for CD, those that were related exclusively to acute infection or in newborns, and those with mixed data on patients with acute and chronic infection were excluded.

Data sources

The databases used for the search, which was carried out from May to August 2020, were: Pubmed/Medline, Scopus; ISIWeb/Web of Science, and LILACS. The corresponding authors of articles included were contacted by email to inquire about missing data or request clarification on studies.

Study search and selection

The standard search strategy described in The Joanna Briggs Institute Reviewers’ Manual 2015 [16] was used. Thus, there was an initial limited search to identify relevant keywords and indexing terms, followed by a comprehensive search in the databases included with strategies for each of the search engines (S1 Database). Two reviewers (SHSC-LXRL) assessed article titles and abstracts in an independent and blinded manner. Disagreements in the inclusion of studies were resolved by consensus, taking into account that the abstracts should meet the proposed eligibility criteria. Subsequently, the articles were reviewed in full text.

Data collection process

Two authors (SHSC-LXRL) extracted the following data independently: author(s), year of publication, type of participants, study area, index test, reference test, study period, country of implementation, number of patients and healthy subjects, total number of participants, sensitivity and specificity, risk of bias and applicability.

Definitions for data extraction

The subjects included in the different studies were classified into: patients who had lived or resided in an endemic area for CD and patients who reside in a non-endemic area.

The study area was considered endemic if CD occurred in this geographic area; and as a non-endemic area, otherwise. The index tests were considered commercial when they were part of a brand of laboratory diagnostic reagents, validated by medical device regulatory agencies and those available on the market; and considered in-house tests when studies indicated that immunoadsorption assays had been designed with different peptides or proteins with the application of non-standard “internal” methods. RDTs are those immunochromatographic assays that throw qualitative results and can be read at first sight.

Reference tests met the standard if they included a combination of serological tests with different antigens detecting antibodies against T. cruzi, and an additional test to reach a definitive diagnosis if the results were inconclusive.

The study design was considered clinical-comparative or case-control type if a group of participants diagnosed with chronic CD and a group without this diagnosis had been included; and it was considered non-comparative if a consecutive and representative series of patients with suspected CD had taken the test to be evaluated, as well as the reference test.

Risk of bias and applicability

Three authors (SHSC-LXLR-CSC) assessed the methodological quality and risk of bias of the studies included, in a blinded and independent manner, using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool, which comprises four domains: patient screening, index test, reference test, and flow and time [17]. Each domain was assessed for risk of bias, and the first three domains were also assessed for applicability.

The QUADAS-2 tool was adjusted to the needs of this review, as follows: the risk of bias in patient screening was considered high if a consecutive or random sample of patients had not been used; and unclear if patient recruitment was not specified. The risk of bias related to the index test was considered unclear if there was no specification that the results of the index tests were interpreted without knowing the results of the reference test. The risk of bias related to reference tests was considered high if these tests were interpreted knowing the results of the index test, or if a single reference test had been used (taking into account that the WHO establishes that serological diagnosis in the chronic phase of CD should be based on positive results in two tests that are based on different immunological principles and, in case of inconsistency, on a third test).

Diagnostic accuracy measures

The reported measures were sensitivity and specificity for each of the index tests assessed for diagnosing chronic CD. When the studies did not have these two measures, they were calculated based on the number of true positives and negatives, as well as on the number of false positives and negatives and the total number of patients.

Summary of results

Sensitivity and specificity were modeled bivariately with binomial-normal random effects, with a gold standard (GS) assumption, but also with an imperfect gold standard (IGS) model. The GS models were fitted with a Bayesian and classical approach; and the IGS model with a Bayesian approach only. Models were selected with the deviance information criterion (DIC) for the Bayesian models, and with the likelihood ratio test for the classical models. Six possible models for the GS were evaluated according to the type of distribution that followed the random effects (normal or mixed normal) and the type of connection (logit, cloglog and probit), and the best model was selected according to the smallest DIC with at least two points difference. The specification of the model with the best fit (in bamdit metadiag) was reproduced in the rest of the packages (meta4diag: Binomial-normal with probit, and metandi and IGS: Binomial-normal with logit) to facilitate comparisons.

The bivariate random effects model was used to estimate the overall sensitivity and specificity and their respective 95 % confidence intervals (CI). The results were plotted in forest-plots and ROC space (R DTAplots program), and heterogeneity between studies was assessed visually. R 1.3 software (DTAplots, bamdit::plotcompare and meta4diag::meta-regression) [18], Stata 15 (metandi) [19], midas and JAGS were used to conduct the meta-analysis.

Additional analyses

Meta-regressions were carried out with potential modifiers of diagnostic validity (bamdit plotcompare and meta4diag meta-regression). The variables of interest were study design (clinical comparative or non-comparative), study area (endemic or non-endemic), study risk (low or high risk of bias), sample type (serum, whole blood or not applicable) for the RDTs, and type of test (commercial or in-house) for the ELISA tests but not for the RDTs because of the low number of studies, which made it impossible to estimate them.

All variables were categorized at two levels in both the ELISA and RDT assays to facilitate the comparison of predictive regions and validity estimates. A QUADAS-2 assessment was applied in each study in order to analyze by subgroups. The three levels of the QUADAS-2 became two: low risk and high risk (which included the high risk and unclear categories). Of the 7 items of the tool, item 1 (patient screening) and item 3 (reference standard) were considered since they were the only ones with a sufficient number of studies with a high risk of bias. In the rest of items, most studies were low risk.

A sensitivity analysis was carried out excluding influential outliers. Influential studies were reviewed based on the assumption that the subsequent interval distribution of study weight should include one. The publication bias was assessed using Deeks’ asymmetry test, which was considered statistically significant with a value of p < 0.1 [20].

Study selection

As shown in Fig 1, 897 publications were initially identified, of which 739 were eliminated due to duplication in the databases. Of the remaining 158 publications, 75 did not meet the selection criteria in the review by title and abstract. Of the remaining 83 articles, 40 were excluded for the following reasons: 17 due to inadequate study design, 9 did not meet the diagnostic reference test criteria suggested by the WHO, 8 due to non-concordance between the index test and the tests that were to analyze the present investigation (ELISA or RDT), 4 used a population that was in the acute phase of the disease or were studies that analyzed subjects in the acute and chronic phase and 2 included patients with previous treatment for CD. Finally, 43 full-text articles were used for qualitative and quantitative analysis [9,10,21–61].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Study selection.

https://doi.org/10.1371/journal.pntd.0010860.g001

Study features

The 43 selected studies were published between 2010 and 2019, with between 3 and 6 studies being published per year, except for 2015, a year in which there were no publications. 27 articles [9,21,22,24,25,27–35,38,41,42,44–46,48,51,53,55,57–59] evaluated ELISA tests; 14 used RDT [10,23,26,32,37,39,40,43,47,50,52,54,60,61] and 2 [36,56] evaluated both tests (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Qualitative analysis of studies selected (ELISA and RDT assays).

https://doi.org/10.1371/journal.pntd.0010860.t001

In 33 articles (76 for ELISA tests and 39 for RDT), several substudies included aspects such as tests with different peptides, populations from different countries, participation of several reference laboratories or evaluation of different index tests. This data is presented and analyzed by separated in the present investigation. Six articles did not present sensitivity and specificity data [22,24,25,37,38,51] but were included considering that they indicated true and false positive and negative data. A total of 30,356 participants were reported in the 43 articles (from 56 to 10,284 subjects per study).

According to the nature of the sample, 76.7 % of the investigations were carried out with serum [9,21,22,24–30–36,39,41,42,44,45,48–51,53–60], 7% with venous blood [40,46,61], 7% with venous blood and serum [23,43,52], 4.7% with capillary blood [10,37], 2.3% with serum, plasma, venous blood and capillary [47]; and 2.3 % did not report the type of sample used [39].

Of those studies where ELISA tests were assessed, 17 were carried out with in-house tests [21,24,26,28,31,35,38,40,44,46,48,52,53,55,60–62]; 12 were commercial trials [9,22,24,25,30,34,36,42,45,48,55,56], and one article studied an in-house assay and a commercial assay [26]. The brands assessed that were present in more than one study were Architect Chagas Abbott [9,30,34,36,48,55], ELISA Chagas III BIOSChile [9,36,41], and Chagatest Wiener [9,25]. 11 different RDTs were assessed; the most used commercial brands were Chagas Detect Plus InBios [10,40,43,54,56], Chagas Stat-Pak assay ChemBio [10,40,50,52,60], and WL Check Chagas Wiener [23,32,60]. Of the 16 studies that analyzed RDTs, 8 did not report who read the results [23,26,36,37,40,54,60,61], 4 did not specify whether or not they had training in reading this type of test [10,47,50 and 56] and 4 studies indicated that the reading was done by qualified personnel [32,39,43,52].

62.7 % of the studies were carried out in countries with endemic regions and native population such as Brazil, Argentina, Bolivia, Venezuela, Paraguay, Mexico, Colombia, Guatemala, Panama, Ecuador, Peru, Nicaragua, Honduras and Costa Rica. 27.9 % of the studies were carried out in non-endemic regions, with a migrant population from countries such as Spain, the United States, Italy, Switzerland and the United Kingdom.

51.2 % of the studies had a non-comparative observational design (cohort or cross-sectional) [10,23,27,32,36–41,43,44,46,48–53,57–59]; 46.5 % were comparative clinical design studies (case-control) [9,21,22,24–26,28–31,33–35,42,45,47,54–56,60] and 2.3 % corresponded to mixed studies (comparative and non-comparative) [61].

Of the 43 studies analyzed, 18 of them [21,22,27,31,33,38,42,44–47,49,51,53,55,57–59] included individuals with other diseases to assess cross-reaction. These diseases were infectious parasitic diseases such as cutaneous, mucocutaneous, visceral and tegumentary leishmaniasis, malaria, toxoplasmosis, schistosomiasis, strongyloidiasis, filariasis, neurocysticercosis, taeniasis, ascariasis, trichuriasis, amoebic liver abscess, visceral larva migrans, and patients positive for Trypanosoma rangeli; bacterial, such as syphilis, rheumatic fever, leptospirosis, tuberculosis, brucellosis, and Lyme disease; viral infections such as hepatitis B and C, HIV, rubella, measles, dengue, Zika, parvovirus, HTLV, Epstein-Barr, or cytomegalovirus; fungal infections, such as South American blastomycosis; and other non-infectious diseases, such as lymphoblastogenesis, juvenile diabetes, lupus erythematosus, and idiopathic megaesophagus. However, only 13 of these [21,26,27,29,31,38,42,45–47,55,58,59] reported cross-reaction percentages, which ranged between 0% and 62.5%, and whose main reaction was related to Leishmania spp.

Risk of bias and applicability

The quality assessment of the studies included in the analysis of ELISA tests is shown in S1 and S2 Figs; and those of RDTs are shown in S3 and S4 Figs. The risk of bias was assessed in the four domains:

1. Patient selection was assessed at high risk of bias in 19 articles for ELISA tests and in 6 for RDT because a consecutive or random sample of patients was not used, since it was unclear in 5 articles for ELISA tests, and because it did not specify patient recruitment; and it was assessed as low risk in 9 studies for RDTs.
2. The risk of bias related to the index test was assessed as unclear in 19 studies for ELISA and in 7 for RDTs because they did not state clearly whether the index test results were interpreted without knowledge of the reference standard results.
3. The bias related to the reference test had 19 studies for ELISA tests and 7 for RDTs, which were classified as high risk because the result of the reference test was interpreted knowing the results of the index test, or a single diagnostic test was used as a reference standard ([62] [63],).
4. Flow and time were assessed as high risk of bias in four studies for ELISA tests, as not all patients received the same reference standard; while all included studies for RDTs were found to be at low risk of bias on this dimension.

Regarding the applicability in the first three domains, 100% of the articles that assessed both ELISA and RDTs tests were classified as low risk because they coincided with the review question.

Synthesis of results

Sensitivity and specificity of ELISA tests.

The ELISA tests had an overall sensitivity of 99% (95% CI: 98–99) and an overall specificity of 98% (95% CI: 97–99) (Fig 2). Some studies presented outliers in sensitivity [33] and specificity [25]. In the predictive region, greater variability is observed for specificity than for sensitivity (Fig 3), therefore, it is observed that there is greater heterogeneity in specificity than in sensitivity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Sensitivity and specificity of ELISA tests in the studies included in the meta-analysis.

TP: true positives; FP: false positives; FN: false negatives; TN: true negatives.

https://doi.org/10.1371/journal.pntd.0010860.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Predictive region in the ROC space, with all the studies for ELISA tests.

https://doi.org/10.1371/journal.pntd.0010860.g003

Sensitivity and specificity of Rapid Diagnostic Tests.

The RDTs had an overall sensitivity of 95% (95% CI: 94–97), and an overall specificity of 97% (95% CI: 96–98). There were studies with atypical values or outliers [10,54] (Those that did not follow the patron of most of the studies, which means that they were strayed of the tendency) (Fig 4). The predictive region (Fig 5) is more symmetric, and a slightly higher variability is observed for sensitivity. Therefore, it is identified that there is a slightly greater heterogeneity for sensitivity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Sensitivity and specificity of Rapid Diagnostic Tests in the studies included in the meta-analysis.

TP: true positives; FP: false positives; FN: false negatives; TN: true negatives.

https://doi.org/10.1371/journal.pntd.0010860.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Predictive region in the ROC space, with all the studies for RDTs.

https://doi.org/10.1371/journal.pntd.0010860.g005

Summary of evidence

The combined sensitivity and specificity of ELISA tests were 99% (95% CI: 98–99) and 98% (95% CI: 97–99); whereas the ones for RDTs were 95% (95% CI: 94–97) and 97% (95% CI: 96–98). The overall sensitivity of RDTs was lower than that of ELISA tests. According to the results obtained in this meta-analysis, the sensitivity and specificity of ELISA tests were higher than those reported in another meta-analysis, in which ELISA tests were compared with RDTs [64] and in which the sensitivity was 97.7. % (96.7%-98.5%) and specificity 96.3% (94.6%-97.6%). The sensitivity of ELISA tests was also higher than in another meta-analysis, where different types of tests were analyzed for diagnosing CD and in which they obtained a sensitivity result of 90% (89%-91%) and a specificity of 98%. % (98 %-98 %) [65], as in this study. Regarding RDTs, the sensitivity and specificity data obtained in this study were lower than those documented in another meta-analysis, where they included clinical trials with recruitment of cohorts of individuals at risk of exposure to T. cruzi, which were 96.6 % (CI95%: 91.3–98.7) and 99.3 % (CI95 %: 98.4–99.7), respectively [66].

Regarding heterogeneity, in the ELISA tests it was identified that this measure was higher in specificity than in sensitivity, which is similar to that reported by Brazil [64], while in the RTDs, the heterogeneity was limited, being slightly higher for sensitivity, coinciding with what was described in the systematic review carried out by Angheben [39], results that differ from those reported by Afonso [65].

This meta-analysis showed greater specificity for ELISA tests in non-endemic areas, and greater sensitivity for endemic areas. The diagnostic performance of RDTs was the same for endemic and non-endemic areas, unlike what Angheben et al. [66] reported, where performance was higher in endemic than non-endemic areas.

No differences in sensitivity were observed between the in-house and commercial ELISA tests. The same showed to be true for specificity in the two types of tests. This differs from what was stated in the meta-analysis by Afonso et al. [65], where the commercial tests were more sensitive than the in-house ones; however, the specificity was similar between the two, as it was observed in this meta-analysis.

All ELISA test assays were made on serum, with the exception of the study by Cervantes-Landín et al. [46], who used venous blood impregnated on filter paper before its coagulation. However, RDTs were made on all types of samples, mostly serum [23,26,32,36,50,54,56,60], but also venous blood and serum [23,43,52]; venous blood [40,61]; capillary blood [10,37]; serum, venous blood, and capillary blood [47], and one study did not specify the sample type [39]. These RDTs are frequently used in Point-of-Care tests (immediate diagnostic analyses), which are performed outside the laboratory, closer to the patient, with easily transportable material and equipment, and which results are available in minutes or in less than one hour. Its application is greater in developing countries [67] since the use of samples such as capillary blood -easy to collect without requiring collection tubes or centrifugation-, would be the best choice for RDTs. The results obtained in this study, depending on the type of sample, allow us to infer that the diagnostic performance of the RDTs was good, regardless of the type of sample.

The trypanosomatids that affect men in America belong to the Leishmania and Trypanosoma genera. ELISA tests have been valuable for diagnosing these two agents, but their specificity may be low due to the cross-reactivity between the two species of parasites; thus, it is important to take this into account, especially when you want to know the prevalence of these two diseases in endemic areas [68]. Furthermore, most of the areas where T. cruzi is found are co-endemic for Leishmania sp. and T. rangeli, which complicates the diagnosis of CD [69]. This coincides with what was documented in 13 of the studies analyzed, according to which the percentage of cross-reaction is between 0% and 62.5% with various diseases, among which leishmaniasis stands out.

Limitations

The items in which a risk of bias was detected in more articles were patient screening and reference standard for the two types of tests assessed; furthermore, the risk of bias associated with the index test was unclear in more than 50% of the studies for ELISA tests. On the other hand, in the applicability items, the studies selected did not present a risk, similar to that described in the meta-analysis carried out by Angheben et al. [66], in relation to the quality of the RDT tests. For the ELISA tests, the sensitivity in the studies with low or high risk of bias was very similar, different from that described by Afonso et al. [65], who reported higher sensitivity in low-risk studies than in high-risk ones. As for specificity, it was higher in the articles that presented a high risk of bias, different from what was reported in this same meta-analysis, where the studies with low risk showed greater specificity.

46.5% of the studies were found to be comparative, and 2.3% mixed studies (comparative and non-comparative). This influences the quality of the studies included because the best design to assess the validity of diagnostic tests is the non-comparative one (cohort or cross-sectional), where a consecutive and representative series of patients with suspected disease are given the test to be evaluated and the reference test in a blind and independent manner and interpreted in the absence of any additional clinical information, which will not be available when the test is used in the practice, either.

In the literature found on this topic, most studies evaluating diagnostic tests take into account the blinding of the reference test results, but few of them have used this non-comparative design [70]. In this meta-analysis, lower sensitivity and specificity values ​​have been found in RDT studies with a comparative clinical design, where a group of patients diagnosed with the disease and a group without this diagnosis were included. This is an unexpected result since studies with a clinical comparative or case-control design tend to overestimate sensitivity and specificity; however, it is possible that this effect only occurs when severe cases are included in the case group [71]. Additionally, a high risk of publication bias was identified for ELISA tests, which coincides with the study by Afonso et al. [65].

The meta-analysis could have been affected by the search made in only four databases because not all studies related to the topic might have been included. Regarding the studies included, ELISA and RDT tests were not compared because only one study [36] had done so; the same was true for the in-house and commercial RDT subgroups which could not be assessed since only one study [26] included in-house RDTs.

The quality of the systematic review was also influenced by the results of the selected articles and their design (some of which were clinical-comparative); that is, some variables could not be explored in the subgroup analysis because they were not reported in many studies, for instance, study period, cross reactions, distribution by rural and urban area, generation or version of diagnostic tests, estimation of cut-off points, geographic areas of the strains used as sources of antigens, and the type of antigen (multiepitope or combination of recombinant proteins).

Subsequent studies should follow the instructions given by the WHO and carry out two tests in parallel using different antigens due to the immunogenic diversity of the different strains of the parasite, the immune response of the patients, and the existence of cross-reactions with other trypanosomatids that coexist in endemic areas [72]. Likewise, a sufficient number of samples should be included to evaluate the cross-reactions between chronic CD and Leishmania infection, considering that patients with either of the two infections, or with mixed infection, may be misdiagnosed given the crossed serological reactions when combinations of uncharacterized antigens are used [22].

Regarding the current status of the implementation of RDTs for diagnosing chronic CD in endemic areas of Latin America, these are used as tests of choice in screening programs for its detection, early treatment and control, and they represent a first approach at point of care for the rapid diagnosis of CD in endemic countries [73]. In addition, since RDTs are easier to use than ELISA, it would be feasible to use them more often in screening programs, which would facilitate the detection of CD cases without ignoring the current recommendations to confirm positive results through conventional methods.