GenoType MTBDRplus Assay for Rapid Detection of Multidrug Resistance in Mycobacterium tuberculosis: A Meta-Analysis

Background There is an urgent demand for rapid and accurate drug-susceptibility testing for the detection of multidrug-resistant tuberculosis. The GenoType MTBDRplus assay is a promising molecular kit designed for rapid identification of resistance to first-line anti-tuberculosis drugs, isoniazid and rifampicin. The aim of this meta-analysis was to evaluate the diagnostic accuracy of GenoType MTBDRplus in detecting drug resistance to isoniazid and rifampicin in comparison with the conventional drug susceptibility tests. Methods We searched PubMed, EMBASE, and Cochrane Library databases to identify studies according to predetermined criteria. A total of 40 studies were included in the meta-analysis. QUADAS-2 was used to assess the quality of included studies with RevMan 5.2. STATA 13.0 software was used to analyze the tests for sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, and area under the summary receiver operating characteristic curves. Heterogeneity in accuracy measures was tested with Spearman correlation coefficient and Chi-square. Results Patient selection bias was observed in most studies. The pooled sensitivity (95% confidence intervals were 0.91 (0.88–0.94) for isoniazid, 0.96 (0.95–0.97) for rifampicin, and 0.91(0.86–0.94) for multidrug-resistance. The pooled specificity (95% CI) was 0.99 (0.98–0.99) for isoniazid, 0.98 (0.97–0.99) for rifampicin and 0.99 (0.99–1.00) for multidrug-resistance, respectively. The area under the summary receiver operating characteristic curves ranged from 0.99 to 1.00. Conclusion This meta-analysis determined that GenoType MTBDRplus had good accuracy for rapid detection of drug resistance to isoniazid and/or rifampicin of M. tuberculosis. MTBDRplus method might be a good alternative to conventional drug susceptibility tests in clinical practice.


Introduction Methods
We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines in our study. We registered the review in PROSPERO (crd.york.ac.uk CRD42015027271).

Literature Search
Original articles published in English up to the end of July 2015 were searched in PubMed, EMBASE, and Cochrane Library databases by two investigators (Y. Bai and Y. Jin). The search terms used were as follows: (Tuberculosis OR Mycobacterium tuberculosis) AND (Hain Life Science OR line probe assay OR GenoType MTBDR OR molecular diagnostic techniques). Conference abstracts were included when sufficient data were reported. Reference lists from included studies were also searched.

Study Criteria
We included studies that evaluated GenoType MTBDRplus for detection of drug resistance of M. tuberculosis to rifampicin (RIF) and/or isoniazid (INH). Included studies should have compared the GenoType MTBDRplus with one or more reference standard methods that were recommended by the WHO (including L-J PM, Middlebrook 7H10/7H11 agar, BACTEC 460, and BACTEC MGIT 960). The study report must have had extractable data to fill the 4 cells of a 2 × 2 table for diagnostic tests (true resistant-TR, false resistant-FR, false susceptible-FS, and true susceptible-TS).
Relevant publications were excluded if they were duplicated articles, letters without original data, case reports, editorials, and reviews. Studies with fewer than 10 samples were also excluded to reduce selection bias.

Data Extraction
The final set of articles was independently assessed by two investigators (Y. Bai and Y. Jin). The full-text of each study was carefully read according to the inclusion criteria to assess whether it should be included. Disagreements were resolved by consensus. Information was extracted on the first author, publication year, country where the study was conducted, specimen type, sample size, gold standard DST used, the number of TR, the number of FR, the number of FS, and the number of TS to each drug. Sensitivity was defined as the proportion of isolates correctly determined as resistant by use of the GenoType MTBDRplus compared with gold standard. Specificity was defined as the proportion of isolates correctly determined susceptible by use of the GenoType MTBDRplus compared with gold standard.

Quality of Study Reports
We applied the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) to assess the quality of included studies (http://www.bris.ac.uk/quadas/), an updated version of the original software. QUADAS-2 is used in systematic reviews to evaluate the risk of bias and applicability of diagnostic accuracy studies, and consists of four key domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed for risk of bias and the first three are also evaluated for applicability. Signaling questions were included to assist in judgments about the risk of bias [14]. If the answers to all signaling questions for a domain were "yes," the risk of bias is judged as "low;" if any signaling question in a domain was "no," risk of bias is judged as "high." The unclear bias should only be used if insufficient information was supplied [14]. Applicability was judged as low, high, or unclear with the similar criteria.

Statistical Analysis
Accuracy Estimates. Meta-analyses were performed using two software programs: STATA 13.0 (Stata Corporation, Texas, USA) and Cochrane RevMan 5.2. Sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR), forest plots and summary receiver operating characteristic (SROC) curves were analyzed with the STATA 13.0 software, based on the random model effect. Quality of studies was assessed with RevMan 5.2. The SROC curve was used to evaluate the effect of the assay. The area under the curve (AUC) displayed the overall diagnostic accuracy and range between 0 and 1, with higher values indicating better test performance [15].
Heterogeneity. Heterogeneity refers to a high degree of variability in accuracy estimates across studies and is often concerned in meta-analyses. We used chi-square test and I 2 (P < 0.05 and I 2 > 50% indicated significant heterogeneity) to identify heterogeneity [16]. The Spearman correlation coefficient between the logit of sensitivity and logit of 1-specificity was used to assess the threshold/cut off effect, which is a possible cause of variations in sensitivity and specificity among the included studies [15]. Heterogeneity due to factors other than threshold/cut-off effect was tested by visual inspection of the forest plots. The further reasons for heterogeneity of the data were addressed by performing subgroup analyses with the Geno-Type MTBDRplus performed directly on clinical specimens or indirectly on clinical isolates, in either solid or liquid medium.

Characteristics of Selected Studies
A flow chart of the study selection process is shown in Fig 1. A total of 1282 potentially relevant citations were identified from all searches. Finally, according to the inclusion and exclusion criteria, 33 eligible articles fulfilled the inclusion criteria and were included in the meta-analysis. The 20 full-text excluded articles were listed in S1 Table with the reasons for exclusion. Because diagnostic tests were performed in different sample types or acid fast bacillus (AFB) smear status occurred in the same article, 40 independent studies (including 7913 samples) were defined in the meta-analysis. Table 1 shows the characteristics of these included studies .

Quality Assessment
A quality assessment of all of the included studies is illustrated in Fig 2. Most of the included studies were at either high risk or unclear risk bias in "patient selection" and "flow and timing" domains of QUARDAS-2 due to lack of detail regarding timing, inconsecutive, or nonrandom patient selection and blinding. A total of 13 (32.5%) studies were at low risk, 7 studies (17.5%) were of unclear risk, and 20 studies (50%) were at high risk for patient selection bias. A total of 24 studies (60%) were at high risk for flow and timing bias, resulting from the fact that not all selected patients were included in the diagnostic analysis and the patients did not receive the same gold standard DST. Most of the studies were at either low or unclear risk for index test and reference standard bias. Regarding applicability, half of the studies were at high risk for patient selection; however, all selected studies (n = 40, 100%) were at low risk of index test and the reference standard. In summary, patient selection was the most high-risk bias and highrisk applicability concerns.

Heterogeneity
Significant heterogeneity was observed when we pooled sensitivity, specificity, PLR, NLR, and DOR of selected studies. The heterogeneity test results of sensitivity and specificity are illustrated in the forest plots (Figs 3, 4 and 5). The Spearman correlation coefficient between the logit of sensitivity and logit of 1-specificity was used to assess the threshold/cut-off effect. The Spearman correlation coefficient (p value) in detecting resistance to INH, RIF and MDR was 0.153 (p = 0.345), 0.017 (p = 0.915), -0.227 (p = 0.298), respectively. This indicated that the heterogeneity might not be due to threshold/cut-off effect. To assess for causes of variations other than threshold, we performed subgroup analysis with the GenoType MTBDRplus assay performed directly on clinical samples or indirectly on clinical isolates, in either solid or liquid medium.

Subgroup Analyses
According to the type of specimen as well as medium, 40 studies were included in the subgroup analyses. Pooled sensitivity, specificity, PLR, NLR, and DOR for INH, RIF, and MDR are presented in Tables 3 and 4. We found significant heterogeneity for most of these measures, except for only clinical isolates were pooled when using GenoType MTBDRplus to detect specificity of MDR (I 2 = 45.5%, p = 0.06).

Discussion
Molecular drug susceptibility testing for M. tuberculosis has garnered strong research interest worldwide. To that end, we focused on the GenoType MTBDRplus assay which has been recommended by the WHO to rapidly screen patients at risk of MDR-TB [10]. MTBDRplus assay is now used routinely in many countries due to its shorter turnaround time, thus a more effective procedure. The direct use of the assay on clinical specimens is another key advantage, as this precludes waiting for cultures to grow. Different from other rapid molecular tests such as INNO-LiPA and GeneXpert, MTBDRplus assay not only detects RIF resistance, but also INH resistance. Although RIF resistance may be regarded as a surrogate for MDR to some extent, there are still some RIF-monoresistant TB strains that are not MDR. Thus, the inclusion of testing mutations that cause INH resistance is highly desirable, especially in settings with relatively   GenoType MTBDRplus for Multidrug-Resistant Tuberculosis low MDR-TB prevalence [50]. Furthermore, the MTBDRplus assay has been the most costeffective rapid test for Asian populations in current practice [13], and its implementation to detect MDR-TB can improve clinical outcomes significantly in some settings [51]. Recently, studies focusing on the diagnostic accuracy of GenoType MTBDRplus were conducted in many settings, but with inconsistent results. The aim of this meta-analysis was to evaluate the diagnostic accuracy of GenoType MTBDRplus for direct detection of resistance to RIF and INH compared with conventional reference methods.
In the literature there are three meta-analyses in which the GenoType MTBDRplus assay has been assessed. The first analysis, performed in 2008, evaluated the performance of both the old GenoType MTBDR and GenoType MTBDRplus, with analysis of only five MTBDRplus  [12]. The second analysis, published in 2009, evaluated the performance of four direct-testing methods, including GenoType MTBDRplus, also with analysis of only five studies for the determination of MDR [50]. The recently reported systematic review, published in 2015, focused on four main molecular diagnostic tests for antibiotic resistance in M. tuberculosis, including GenoType MTBDRplus, and only evaluated the assay on clinical specimens and could not perform subgroup analysis to investigate the potential causes of heterogeneity due to the small number included studies [13].
To the best of our knowledge, the present meta-analysis, with 40 studies included, is the first study that has comprehensively evaluated the overall diagnostic accuracy of the GenoType MTBDRplus assay in detecting drug resistance of RIF, INH, and MDR. In our meta-analysis, GenoType MTBDRplus showed excellent pooled sensitivity and specificity for detection of resistance to INH (91%, 99%), RIF (96%, 98%), and MDR (91%, 99%), with lower and more inconsistent sensitivity than specificity. While specificity did not vary across subgroups, sensitivity was slightly higher when only DST of studies based on liquid medium was pooled (INH 92%, RIF 98%, MDR 94%). When compared with the previously published meta-analyses, the pooled sensitivity was also found to be more variable and lower than specificity, which varied from 84% to 96% for INH and 96% to 99% for RIF [12,13,52]. This may be partially attributed to the limitations of molecular methods for the detection of first line drug resistance, that 5% of RIF-resistant M. tuberculosis strains and 10-25% of lowlevel INH-resistant strains have no known resistance mutations [53,54].
The DOR is defined as the ratio of the odds of the test being positive for a patient with or without disease [55], and is an indicator of diagnostic accuracy that combines the data from sensitivity and specificity into a single variate. The value of a DOR ranges from 0 to infinity, with higher values indicating higher accuracy. This meta-analysis showed that GenoType MTBDRplus had very high mean DOR and large AUC values, indicating a high value of overall accuracy for the detection of MDR. Because of the limitations of SROC and DOR in clinical practice, the likelihood ratios (LRs) are of more clinical significance [56]. A very high PLR and a very low NLR for the detection resistance of INH, RIF, and MDR in our study indicated an excellent ability to both confirm and exclude the presence of drug resistance. Although in the present analysis, indices such as AUC, DOR, PLR, and NLR showed good diagnostic accuracy of GenoType MTBDRplus assay, the confidence intervals for the PLR and the DOR were wide for all included studies due to high sample variation and there was significant heterogeneity in the measures.
The purpose of a meta-analysis is not only to compute a single summary measure, but also to explore the reasons for heterogeneity [57]. We found significant heterogeneity for sensitivity, specificity, PLR, NLR, and DOR among the studies analyzed, except for only clinical isolates were pooled when using GenoType MTBDRplus to detect specificity of MDR (I 2 = 45.5%, p = 0.06). The Spearman correlation coefficient between the logit of sensitivity and logit of 1-specificity was not significant, indicating that the heterogeneity was not caused by threshold/ cut-off effect. Thus, subgroup analyses were performed to test for causes of variations other than threshold effect. The results suggested that the sample type could partly explain the heterogeneity. Even so, the considerable heterogeneity in the results remained unexplained, which may be caused by variations in the study, patient selection, sample collection method (consecutive or random collection of samples), and/or geographic and genetic variations in the distribution of drug-resistant strains of M. tuberculosis [58,59].
Our meta-analysis had several strengths. First, we performed a standard protocol to carry out the meta-analysis, including a comprehensive search strategy [60]. Second, two reviewers independently carried out various stages of the process, including article selection, data extraction, and quality assessment, and disagreements were resolved by consensus. Third, we used rigorous statistical methods for data analysis, including SROC analyses, quality assessment relying on QUADAS-2, as methods for exploring heterogeneity. Moreover, the present metaanalysis updates previous estimates on the performance of the MTBDRplus test for identifying resistance of first-line anti-TB drugs. Compared with the recently published comprehensive systematic review [13], our study showed similar pooled specificity, but higher pooled sensitivity for detecting both RIF and INH resistance (97% versus 94.6%; and 90% versus 83.4%, respectively) directly on clinical specimens. The DOR, as an indicator of diagnostic accuracy, was also much higher in the current study than previously shown for detecting RIF resistance (1105.23 versus 666). The better diagnostic accuracy found in our study may provide more powerful evidence for routine clinical application of GenoType MTBDRplus assay.
However, our meta-analysis also had several limitations. First, sampling methods, blinding strategies and population (e.g. severity of disease or treatment status) were unclear in most of the included studies. Inappropriate sampling methods can generate selection bias which may result in high levels of sample variation and wide confidence intervals. The lack of blinding when interpreting index and reference test results may result in overestimating accuracy [61]. Second, an obvious limitation was the lack of data on cost-effectiveness, feasibility, patient management and treatment outcomes, and how much value they contributed to existing diagnostic and treatment regimens beyond conventional DST methods. Third, the present authors only included studies published in English, and some studies missing data in 2 by 2 tables were excluded since the authors could not be contacted. As currently available statistical approaches for publication bias are not recommended for diagnostic meta-analysis, we did not use funnel plots and regression tests to assess publication bias [62], and it is therefore difficult to rule out potential publication bias in our meta-analysis.
Furthermore, there were not enough studies in the literature for us to acquire adequate data to stratify by smear status, as smear-negative patients would be most likely to benefit from using molecular methods. Until now, it seems there is still a great challenge to rapidly and reliably identify M. tuberculosis in smear-negative samples, especially in human immunodeficiency virus (HIV)-infected patients. M. tuberculosis is the most prevalent opportunistic infection and cause of the death for HIV-infected patients, whose smear-positivity of M. tuberculosis can be as low as 20% [63]. To overcome this limitation, the revised version 2.0 of MTBDRplus was released in 2011 with reported improved diagnostic accuracy in detecting M. tuberculosis and their resistance status against RIF and INH in AFB-negative specimens [31,64], further supporting the ability to use this assay in smear-negative samples.
In general, although GenoType MTBDRplus test showed good accuracy for INH, RIF, and MDR drug resistance detection in this meta-analysis, some important issues remain to be addressed. In recent years, several studies showed that RIF resistance can be regarded as a proxy for MDR in different settings [65,66]. Arentz et al. performed a systematic review to evaluate six different WHO-endorsed rapid tests for RIF resistance detection [67], and determined that these tests for RIF resistance can accurately predict MDR-TB in areas with high prevalence, but not in areas with low prevalence of RIF resistance. Compared with other tests, GenoType MTBDRplus had the lowest PPV at prevalence rates of 15% and 3% for RIF resistance which meant the higher false positive rates for detecting RIF resistance and MDR-TB. However, these results relied on an assumption that RIF resistance was strongly correlated with MDR. In fact, this correlation may vary in different settings [50]. Future studies should focus on the diagnostic accuracy of rapid tests in areas with different prevalence rates of RIF resistance in order to determine the threshold that constitutes RIF resistance is as a sufficient marker for MDR-TB.
In addition to rapid detection of MDR-TB, there is also an urgent need for rapid and accurate tests for extensively drug-resistant tuberculosis (XDR-TB). As a serious threat to public health, XDR-TB is caused by strains of M. tuberculosis that are resistant to INH, RIF, and any of the fluoroquinolones (FLQs) and at least one second-line injectable agent (SLIDs; i.e. amikacin, kanamycin or capreomycin) [68]. XDR-TB has now been detected in more than 90 countries and nearly 10% of MDR-TB cases are also XDR-TB cases [2]. A recently published systematic review found GenoType MTBDRsl, the only commercially-available molecular routine test to detect second-line anti-TB drug resistance, had good accuracy for detecting drug resistance to FLQs, amikacin and capreomycin, but may not be an appropriate choice for kanamycin and ethambutol due to poor sensitivity [69]. Future studies that test the accuracy of the MTBDRsl in different laboratory settings are necessary. Furthermore, differences should be accounted for geographical regions, special patient populations (for example, pediatric or HIV/ TB co-infected patients), and should also assess the effect of MTBDRsl implementation on cost-effectiveness and clinical outcomes. Future molecular tests for XDR-TB should have additional genetic targets beyond gyrA, rrs and embB. Rapid and accurate detection of MDR-TB and XDR-TB is important in improving patient care and decreasing transmission.
In conclusion, the present meta-analysis showed that GenoType MTBDRplus assay had good accuracy for detecting drug resistance to INH, RIF, and MDR of M. tuberculosis, suggesting that it has good utility as a rapid screening molecular tool. Further studies are needed to compare the accuracy of the MTBDRplus assay in smear-positive versus smear-negative specimens and pulmonary versus extra-pulmonary cases, and to evaluate the utility of this assay in HIV/TB co-infection. MTBDRplus assay might be a good alternative to conventional drug susceptibility tests in clinical practice.