Accuracy of Xpert Clostridium difficile assay for the diagnosis of Clostridium difficile infection: A meta analysis

Background There is an urgent need for rapid and accurate microbiological diagnostic assay for detection of Clostridium difficile infection (CDI). We assessed the diagnostic accuracy of the Xpert Clostridium difficile assay (Xpert CD) for the diagnosis of CDI. Methods We searched PubMed, EMBASE, and Cochrane Library databases to identify studies according to predetermined criteria. STATA 13.0 software was used to analyze the tests for sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, and area under the summary receiver operating characteristic curves (AUC). QUADAS-2 was used to assess the quality of included studies with RevMan 5.2. Heterogeneity in accuracy measures was tested with Spearman correlation coefficient and Chi-square. Results A total of 22 studies were included in the meta-analysis. The pooled sensitivity (95% confidence intervals [CI]) was 0.97 (0.95–0.99) and specificity was 0.95 (0.94–0.96). The AUC was 0.99 (0.97–0.99). Significant heterogeneity was observed when we pooled most of the accuracy measures of selected studies. Conclusions The Xpert CD assay is a useful diagnostic tool with high sensitivity and specificity in diagnosing toxigenic CDI, and this method has excellent usability due to its rapidity and simplicity.


Introduction Methods
We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines in our study.

Literature search
Original articles published in English up to the end of July 2017 were searched in PubMed, EMBASE and Cochrane Library databases by two investigators (Y. Bai and J. Li). The search terms used were as follows: Clostridium difficile AND (Xpert C. difficile OR molecular diagnostic techniques). Reference lists from included studies were also searched.

Study criteria
We systematically searched the literature using the following predetermined inclusion criteria. Studies evaluating Xpert CD as a diagnostic test for CDI were eligible for inclusion if the studies 1) described original research; 2) performed stool samples analyses from human patients, either children or adults; 3) compared Xpert CD to a reference method-either CCNA or anaerobic TC; 4) had extractable data to fill the 4 cells of a 2 × 2 table for diagnostic tests (true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN)).
Relevant publications were excluded if they were duplicated articles, letters without original data, animal studies, case reports, editorials, and reviews. Studies with fewer than 20 samples were also excluded to reduce selection bias. Articles that contain data from infants were excluded because infants rarely develop clinical infection.

Data extraction
Two investigators (Y. Bai and J. Li) extracted data from full text of the included studies independently. Disagreements were resolved by consensus. Information was extracted on the first author, publication year, country where the study was conducted, sample size, reference tests the diagnosis used, the number of TP, the number of FP, the number of FN, and the number of TN. These were summarized as sensitivity, TP/(TP+FN); specificity, TN/(TN + FP);and prevalence, (TP+FN)/(TP+FN+TN+FN).

Quality of study reports
We applied the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) to assess the quality of included studies (http://www.bris.ac.uk/quadas/), an updated version of the original software [12]. QUADAS-2 is used in systematic reviews to evaluate the risk of bias and applicability of diagnostic accuracy studies, and consists of four key domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed for risk of bias and the first three are also evaluated for applicability. Signaling questions were included to assist in judgments about the risk of bias [13]. If the answers to all signaling questions for a domain were "yes," the risk of bias is judged as "low;" if any signaling question in a domain was "no," risk of bias is judged as "high." The unclear bias should only be used if insufficient information was supplied [13]. Applicability was judged as low, high, or unclear with the similar criteria.

Statistical analysis
Accuracy estimates. Meta-analyses were performed using two software programs: STATA 13.0 (Stata Corporation, Texas, USA) and Cochrane RevMan 5.2. Sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR), forest plots and summary receiver operating characteristic (SROC) curves were analyzed with the 'midas' module for STATA 13.0, based on the random model effect. Quality of studies was assessed with RevMan 5.2.
Heterogeneity. We used chi-square test and I 2 (p < 0.05 and I 2 > 50% indicated significant heterogeneity) to identify heterogeneity. The methods to evaluate the heterogeneity were described detailedly in our previous published study [14]. The further reasons for heterogeneity of the data were addressed by performing subgroup analyses on prespecified variable: the calculated prevalence of C. difficile (<15% and !15%) and the sample size (< median size 246 and !246).
Fagan's nomogram. The method to depict visual Fagan's nomogram was described in detail as previously [15].

Characteristics of selected studies
A flow chart of the study selection process is shown in Fig 1. A total of 193 potentially relevant citations were identified from all searches. Finally, according to the inclusion and exclusion criteria, 20 eligible articles fulfilled the inclusion criteria and were included in the meta-analysis. Because diagnostic tests performed with different reference methods occurred in the same article, 22 independent studies (including 9352 samples) were defined in the meta-analysis. Table 1 shows the characteristics of these included studies [16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35]. The prevalence of CDI across all studies ranged from 10% to 47.9%. Two studies used CCNA as a reference test [16,19].In one study, the investigators reported the diagnostic accuracy separately for both the reference standards [19]. In another study, the investigators reported the diagnostic accuracy separately for TC and Enriched TC [17]. Most of the studies were prospective in design.

Quality assessment
A quality assessment of all of the included articles is illustrated in Fig 2. In conclusion, patient selection was the most high-risk or unclear risk bias and high risk applicability concerns. More than half of the included articles were at either high risk or unclear risk bias in "patient selection" and "flow and timing" domains of QUARDAS-2 due to lack of detail regarding timing, inconsecutive, or nonrandom patient selection and blinding. A total of 9 (45%) articles were at low risk, 7 articles (35%) were of unclear risk, and 4 articles (20%) were at high risk for patient selection bias. A total of 12 articles (60%) were at high risk for flow and timing bias, because of the fact that not all selected patients were included in the diagnostic analysis. Most of the articles were at either low or unclear risk for index test and reference standard bias. Regarding applicability, half of the articles were at high risk for patient selection; however, all selected articles (n = 20, 100%) were at low risk of index test and the reference standard. In conclusion, patient selection was the most high-risk or unclear risk bias and high risk applicability concerns.

Diagnostic accuracy
Results are given as values (95% CI). Using a random-effects model, the results were as follows:  The relationship between pretest probability and posttest probability was depicted by visual Fagan's nomogram. For patients with a pretest probability of 20%, the posttest probability of positive results was 84%, and posttest probability of negative results was 1% (Fig 4).
Heterogeneity. There was substantial heterogeneity for all the statistical measures. The heterogeneity test results of sensitivity and specificity are illustrated in the forest plots (Fig 3). The Spearman correlation coefficient between the logit of sensitivity and logit of 1-specificity was used to assess the threshold/cut-off effect. The Spearman correlation coefficient (p value) in diagnostic of CDI was 0.270 (p = 0.201). This indicated that the heterogeneity might not be due to threshold/cut-off effect. To assess for causes of variations other than threshold, we performed subgroup analyses in terms of CDI prevalence and sample size.  The theoretical values of PPV and NPV were calculated using the pooled sensitivity (0.97) and specificity (0.95) values and plotted against increasing CDI prevalence. The PPV performance is variable and correlated positively with increasing CDI prevalence, whereas NPV remained almost quite high (Fig 5).

Discussion
In recent years, NAATs for the direct detection of C. difficile toxin genes in stool samples have garnered strong research interest worldwide and are a highly sensitive alternative to the EIAs, the time-consuming TC and CCNA. To that end, we focused on the Xpert CD assay which has been cleared by FDA to rapidly diagnose patients with CDI. The Xpert CD assay is now implemented in many countries due to its shorter turnaround time, thus a more effective procedure. The most significant advantage of the Xpert CD assay is its rapidity and simplicity. As described previously [9,28], besides detecting toxigenic C. difficile, the Xpert CD assay reports presumptive identification of 'hypervirulent' 027/NAP1/BI (positive for cdt and tcdC Δ117). The 14-day mortality was very high for 027 /NAP1/BI (20%), compared to an overall mortality of 13% (p < 0.0001) [36]. A previously published study reported that the agreement between the Xpert CD assay and PCR-ribotyping was 93% [37]. And another study found "very good" agreement at 97.9% between this assay and multilocus sequence typing (MLST) for identification of C. difficile NAP1 [38]. Recently, studies focusing on the diagnostic accuracy of the Xpert CD assay were conducted in many settings, but with inconsistent results. To provide much more evidence-based results for utility of this assay in routine testing, we conducted this meta-analysis to evaluate the diagnostic accuracy of the Xpert CD assay for direct detection of CDI compared with conventional reference tests. In the literature there are two meta-analyses in which the Xpert CD assay has been assessed [39,40]. The first analysis, performed in 2012, was limited by only including four studies that could not fully assess the clinical application of this assay [39]. The second analysis, as a part of a review published in 2013, focused on three popular commercial NAATs, including the Xpert CD assay, only evaluated the assay compared to TC and only reported pooled sensitivity [40]. As we know, to evaluate the diagnostic accuracy of this assay, other measures such as specificity, DOR, PLR and NLR also should be reported. Our meta-analysis identified several additional studies since the publication of the two reviews. To the best of our knowledge, the present meta-analysis, with 22 studies included, is the first study that has comprehensively evaluated the overall diagnostic accuracy of the Xpert CD assay in detecting toxigenic CDI. In our meta-analysis, the Xpert CD assay showed high pooled sensitivity (97%) and specificity (95%) for detection of CDI, with lower specificity than sensitivity which was one of the common characteristics of NAATs. While NAATs have highly sensitivity, there are some important issues that have been raised regarding their practical application. The main issue regarding the clinical utility of NAATs, especially the PPV and specificity, is linked to the detection of genes versus actual toxins. Someone concerned that NAATs can be positive in both disease states and colonization [9]. Detection of toxin genes does not necessarily correlate with expression of toxin, nor disease. Another limitation of the NAATs is that nearly all assays focus on tcdB detection, but if a strain did not produce toxin B or was tcdB-variant, it would led to a false-negative result. Some researches have reported that inappropriate test ordering can also impact the clinical specificity of tests [21].
Given a pretest probability of 20%, the posttest probability for a positive test results was 84%, and posttest probability of negative test result was 1%. The PPV performance correlated positively with increasing CDI prevalence.This meta-analysis showed that Xpert CD assay had high mean DOR and large AUC values, indicating a high value of overall accuracy for the detection of CDI. A quite high PLR and a very low NLR for the detection CDI in our study indicated an excellent ability to both confirm and exclude CDI. The better diagnostic accuracy found in our study may provide more powerful evidence for routine clinical application of Xpert CD assay.
We found significant heterogeneity for diagnostic parameters among the studies analyzed. The Spearman correlation coefficient between the logit of sensitivity and logit of 1-specificity was not significant, indicating that the heterogeneity was not caused by threshold/cut-off effect. Thus, subgroup analyses based on CDI prevalence and sample size were performed to test for causes of variations other than threshold effect. There were no significant heterogeneity for PLR (I 2 = 25.39%, p = 0.01) when CDI prevalence of studies greater than 15% were pooled and that for sensitivity (I 2 = 39.1%, p = 0.09) and NLR (I 2 = 15.31%, p = 0.30) when sample size of studies less than median sample size were pooled. The results suggested that the CDI prevalence and sample size could partly explain the heterogeneity. Even so, the considerable heterogeneity in the results remained unexplained, which may be caused by the various baseline criteria for accepting stool samples for testing.
Our meta-analysis had several methodological strengths, such as standard protocol and rigorous statistical methods. However, our meta-analysis also had several limitations. First, the studies included in this analysis have various baseline criteria for accepting stool samples, different patient populations and institutional characteristics. Most studies did not specify the exact criteria used to submit patient stool samples to the laboratory for testing, which made it difficult to avoid detecting asymptomatic carriers. Second, the present authors only included studies published in English, and some studies missing data to calculate sensitivity and specificity were excluded since the authors could not be contacted. Moreover, we did not show the detailed data about analyzing studies that used TC or CCNA or enriched TC as the reference standard separately, because of only 2 studies used CCNA and 3 studies used enriched TC as reference tests. It did not reduce potential heterogeneity when studies with TC as reference method were pooled (data not shown).
In general, although the Xpert CD assay showed good accuracy for CDI detection in this meta-analysis, some important issues remain to be addressed. Like the problems about NAATs we mentioned above, one of the more important questions concern the clinical utility of this assay is that it specifically detects the tcdB gene encoding the toxin and not an actual toxin. Therefore, asymptomatic carriers can be misdiagnosed as disease state if inappropriate testing is performed. The asymptomatic colonization rate is about 2% in healthy adults and can be as high as 51% for residents of long term care facilities [41]. To avoid over diagnosis and overtreatment of toxigenic CDI by using the Xpert CD assay, it must be strictly limited to diarrheal stool specimen in patients without laxatives. The clinician should be mindful of the limits of this rapid molecular assay and clinical assessment is necessary to detect true infection. Also, the assay can remain positive for weeks after the resolution of clinical symptoms and should not be repeated for monitoring treatment.
Another important issue of cost-effectiveness of the molecular methods, including the Xpert CD assay, has been raised by laboratorians and administrators. The molecular assays cost up to 2-to 3-fold more than EIAs. To lessen the cost of NAATs, a two-step algorithm based on glutamate dehydrogenase (GDH) detection and NAAT in case of a positive result constitute an excellent alternative to the exclusive use of NAATs. Culbreath et al. reported that this algorithm was 56% cheaper than applying the Xpert CD systematically in their institution (US$70,633 vs. US$159,877 per 1000 tests) [42]. Given the rapidity and simplicity of the Xpert CD assay that are helpful for timely implementation of appropriate therapy and contact precaution, this two-step algorithms may not offer an advantage but may be used for cost savings. On the other hand, the higher cost of NAATs may be counterbalanced by a decrease in healthcareassociated infections costs. Babady et al. reported that performing the Xpert CD test alone was less expensive than two-step algorithms when labor costs (accessioning, test performance, and reporting of results) were considered [43]. In the future, it is needed to assess the overall costeffectiveness of Xpert CD for the diagnosis of C. difficile disease, including comprehensive laboratory costs and overall hospital costs.
In conclusion, the present meta-analysis showed that the Xpert CD assay had good accuracy for detecting CDI, suggesting that it has good utility as a rapid screening molecular tool. In the future, studies are needed to focus on the prediction of the disease severity. While several biomarkers that correlates with active CDI have been evaluated (such as fecal lactoferrin, calprotectin, interleukin-8), combining the Xpert CD assay with biomarkers to diagnosis active CDI will likely be an area of investigation in the coming years. This assay will probably be considered as one of the standard diagnostic tests for CDI, either as a standalone test or included in a multistep algorithm.