Accuracy of Immunodiagnostic Tests for Active Tuberculosis Using Single and Combined Results: A Multicenter TBNET-Study

Background The clinical application of IFN-γ release assays (IGRAs) has recently improved the diagnosis of latent tuberculosis infection. In a multicenter study of the Tuberculosis Network European Trialsgroup (TBNET) we aimed to ascertain in routine clinical practice the accuracy of a novel assay using selected peptides encoded in the mycobacterial genomic region of difference (RD) 1 for the diagnosis of active tuberculosis in comparison with tuberculin skin test (TST), QuantiFERON-TB GOLD In-Tube (Cellestis Ltd., Carnegie, Australia) and T-SPOT.TB (Oxfordimmunotec, Abingdon, UK). Principal Findings 425 individuals from 6 different European centres were prospectively enrolled. We found that sensitivity of the novel test, TST, QuantiFERON-TB GOLD In-Tube and T-SPOT.TB was respectively 73.1%, 85.3%, 78.1%, and 85.2%; specificity was respectively 70.6%, 48.0%, 61.9% and 44.3%; positive likelihood ratios were respectively 2.48, 1.64, 2.05, and 1.53; negative likelihood ratios were respectively 0.38, 0.31, 0.35, 0.33. Sensitivity of TST combined with the novel test, QuantiFERON-TB GOLD In-Tube and T-SPOT.TB increased up to 92.4%, 97.7% and 97.1%, respectively. The likelihood ratios of combined negative results of TST with, respectively, the novel test, QuantiFERON-TB GOLD In-Tube and T-SPOT.TB were 0.19, 0.07 and 0.10. Conclusions The assay based on RD1 selected peptides has similar accuracy for active tuberculosis compared with TST and commercial IGRAs. Then, independently of the spectrum of antigens used in the assays to elicit mycobacterial specific immune responses, the novel test, IGRAs, and the TST do not allow an accurate identification of active tuberculosis in clinical practice. However, the combined use of the novel assay or commercial IGRAs with TST may allow exclusion of tuberculosis.


Introduction
Tuberculosis control is based on the consequent use of preventive chemotherapy in individuals with latent tuberculosis infection (LTBI) who are at risk of developing active disease and on the rapid diagnosis and effective treatment of infectious cases [1][2][3]. While the identification of patients with active tuberculosis can rapidly be established by the detection of alcohol acid fast bacilli (AFB) on sputum smears, early diagnosis of infectious cases by sputum microscopy is only possible in approximately 50% of cases [4]. The sub-optimal performances of existing diagnostic tools [4], in terms of both speed and sensitivity, delayed diagnosis and, consequently, treatment of active tuberculosis.
The recent introduction of T-cell-based interferon (IFN)-c release assays (IGRAs), using antigens belonging to M. tuberculosis region of difference (RD) 1 (including early secreted antigenic target [ESAT]-6 and culture filtrate protein 10 [CFP]-10) represents a significant step towards improved LTBI diagnosis [5][6][7][8][9]. There is growing evidence that in low incidence settings both the commercial IGRAs currently available, the Quantiferon-GOLD In-tube assay (Cellestis Ltd., Carnegie, Australia) and the T-SPOT.TB assay (Oxfordimmunotec, Abingdon, UK) are less affected by bacillus Calmette-Guerin (BCG) vaccination than the tuberculin skin test (TST) and that they are more specific and correlate better with exposure to an infected index case [10][11][12][13]. Although these commercial assays provide an accurate diagnosis of M. tuberculosis infection and detect active tuberculosis disease, they cannot discriminate between active tuberculosis and LTBI. Thus, further clinical workup is required to rule out active tuberculosis after a positive response to these tests.
Recently the design of a novel in vitro immune diagnostic enzyme-linked immunospot (ELISPOT) and whole blood ELISA (WBE) for IFN-c using multiepitopic peptides that are selected by computational analysis from CFP-10 and ESAT-6 as stimulating antigens has been reported [14]. It has been shown that the response to RD1 selected peptides can be detected in subjects with ongoing M. tuberculosis replication, such as during active tuberculosis and/or recent infection [15][16][17]. This response is mediated by CD4 + T effector cells, shown to undergo clonal expansion during M. tuberculosis replication, followed by a contraction phase after efficacious therapy culminating in the generation of CD4 + memory T-cells [18,19]. These studies were conducted at one center in Italy, a country with a low tuberculosis incidence of ,10/100.000 [20]. The aims of this multicenter study were: i) to evaluate whether this assay based on RD1 selected peptides may help in providing evidence of active tuberculosis; ii) to compare the response to this novel assay with TST and the commercially available RD1 tests, individually and in combination for the diagnostic work-up of active tuberculosis [21].

Study design
Following obtaining of informed consent, patients with a clinical suspicion of tuberculosis (abnormal chest radiograph suggestive of tuberculosis and/or other signs and symptoms such as persistent cough, haemoptysis, weight loss, fever) were prospectively recruited at participating centers of the Patients underwent clinical and microbiological examinations including chest radiographs to confirm or exclude the diagnosis of tuberculosis. Briefly, 3 sequential respiratory expectorated or 2 induced sputum smears over the first 7 days following clinical evaluation were collected. AFB smear and culture (on both, Lowenstein-Jensen and Bactec MGIT, BD Biosciences Division, Sparks, Maryland, USA) were performed on each specimen. Additionally M. tuberculosis-specific RNA amplification was performed on specimens from patients with a high likelihood for tuberculosis in which examinations for AFB were negative [(Gen-ProbeH Amplified TM Mycobacterium Tuberculosis Direct (MTD) Test, San Diego, CA, USA)]. TSTs were administered by the Mantoux method with bioequivalent 5 Tuberculin Units (Biocine, Chiron, Siena, Italy) or 2 Tuberculin Units (RT23, Statens Serum Institute, Copenhagen, Denmark) [22,23] or 5 Units of PPD Tuberculin Mammalian (BulBio-NCIPD, Sofia, Bulgaria). Indurations were measured 48-72 hours following tuberculin administration by the ballpoint technique. Individuals with an induration $10 mm [24] or in Bulgaria $15 mm [25][26] for those with past BCG vaccination were classified as TST-positive [22].
For extra-pulmonary tuberculosis, M. tuberculosis-specific RNA amplification (MTD Test) and/or nucleic acid amplification test (NAT) for M. tuberculosis-specific DNA based on a commercial test (BD ProbeTec ET system; BD Diagnostic Systems, Sparks, MD) or based on a homemade version developed from the literature [27] was performed on biopsy specimens and/or biological fluids; moreover histology and AFB staining were performed on biopsies.
Enrolled patients were classified as ''confirmed tuberculosis'' if the diagnosis was based: i) in those with pulmonary tuberculosis by a positive culture for M. tuberculosis; ii) in those with extrapulmonary tuberculosis by a) positive M. tuberculosis-specific RNA amplification and/or M. tuberculosis-specific NAT from biological specimens or b) by histological pathological finding consistent with tuberculosis and presence of AFB in a tissue sample or c) by positive culture for M. tuberculosis in clinical samples (pleural fluid and abscesses). Conversely, patients were classified as ''clinical tuberculosis'' if the diagnosis was based on clinical and radiologic criteria (having excluded other disease) including appropriate response to anti-tuberculosis therapy.
We defined patients without tuberculosis as those admitted with a suspicion of active tuberculosis, who subsequently showed negative sputum for AFB smear and culture for M. tuberculosis with either a resolution of clinical symptoms and radiographic abnormalities after an antibiotic therapy not involving M. tuberculosis active drugs, or presenting a confirmed alternative diagnosis (e.g.: lung cancer).
Following admission, a 10-20 ml (depending on the center) heparin venous blood sample was drawn from all enrolled individuals. ELISPOT or WBE based on RD1 selected peptides was performed. In a subgroup of patients the test was done in parallel with the commercially available immune assays for tuberculosis. Clinicians were blinded to the results of in vitro assays and laboratory personnel were blinded to the status of the patient. The study was approved by the ethics committee at all the institutions in which the study was performed.

RD1 selected peptides and stimuli used for cell cultures
The selection of Human Leukocytes Antigens (HLA)-class II restricted epitopes of ESAT-6 and CFP-10 M. tuberculosis proteins was performed by quantitative implemented HLA peptide-binding motifs analysis as previously described for ESAT-6 [14,15]. Peptides were synthesized as free amino acid termini using Fmoc chemistry (ABI, Bergamo, Italy). Lyophilized peptides were diluted in DMSO at stock concentrations of 10 mg/mL for each peptide and stored at 280uC. RD1 selected peptides were used as follows: a pool of the two ESAT-6 peptides (at 10 mg/mL each), a pool of the three CFP-10 peptides (at 2 mg/mL each). DMSO was used as negative control at 10 mg/mL. As positive control we used Phytohemagglutinin (PHA) (Sigma, St Louis, MO, USA) at 5 mg/mL. RD1 selected peptides from the same batch were provided to the all centers with a detailed protocol. Four out of the 5 external centers received personal training from INMIs' laboratory personnel for at least 2days. Inter-site communication was present all over the performance of the study to solve any potential problem.
ELISPOT. 2.5610 5 peripheral blood mononuclear cells (PBMC) were separated, washed twice and plated in the T-SPOT.TB plates stimulated with or without RD1 selected peptides and PHA, as described above and previously [14,15]. Cell cultures were incubated overnight at 37uC, with 5% CO2. On the next morning, the cells were washed off, and the ELISPOT was developed following the manufacturer's instructions (Oxford Immunotec, UK). Spots were then counted by an automated ELISA-Spot assay video analysis system (AELVIS, Hannover, Germany). Evaluated spots had a size .15 U (1 U = 50 mm2). Indeterminate results were defined by values in the PHAstimulated samples below 34 spot-forming cells per million PBMC. The RD1 selected peptide responses were scored as positive if above 34 spot-forming cells/million PBMC. This cutoff value was determined by constructing a receiver operator characteristic (ROC) curve by means of LABROC-1 software. To obtain the absolute value, the number of spot-forming cells in the negative controls was subtracted from the number of spot-forming cells in the stimulated cultures. Clinicians were blinded to the laboratory test results and laboratory personnel were blinded to the status of the patients.
WBE. Briefly, aliquots of 0.5 ml per well of heparinized blood were seeded in a 48-well plate and stimulated with or without RD1 selected peptides and PHA, as described above. Samples were then incubated for 24 hours at 37uC in presence of 5% CO 2 when an amount of 100 ml of plasma was harvested. IFN-c levels in culture supernatants were assessed by a commercially available kit (QuantiFERON-CMI kit, Cellestis). For the results scoring, a cut-off value of 0.7 IU/mL was chosen for all stimuli by constructing a ROC curve. Indeterminate results were defined by values in the PHA-stimulated samples below 0.7 IU/mL.

Commercially available assays
T-SPOT.TB and QuantiFERON-TB GOLD In-Tube assays were performed and their results were scored as indicated by the manufacturers.
Statistical Analysis. The tests performance was evaluated by using categories of confirmed tuberculosis, clinical tuberculosis, and no active tuberculosis. Cases with indeterminate responses to in vitro assays were not included in the analysis. Sensitivity, specificity and likelihood ratios with their 95% Confidence Interval (CI), were computed for each test overall and according to the diagnostic categories and tuberculosis localization. Proportions were compared by using Fisher exact test and, for paired data, McNemar chi-square test. Sensitivities of 2 tests used in combination were obtained assuming that a positive result is given by a positive response to at least one assay.
Furthermore the accuracy of two tests used in combination was analyzed by computing the likelihood ratios, together with the distribution of subjects with and without active tuberculosis, according to the responses to the tests. Sensitivities and specificities of diagnostic tests were compared by using a logistic regression model with robust standard errors to account for the correlation between observations. Two-tailed P values are reported.

Results
We consecutively enrolled 425 consenting adult patients from 6 different centres in Europe. Complete data were unavailable from 1 patient. Results from 11 (2.5%) subjects were found to be indeterminate by in vitro assays based on RD1 selected peptides and/or QuantiFERON-TB GOLD In-Tube ( Figure 1). Among them, 4 had active tuberculosis and 7 were without active tuberculosis. These patients were similar to those without indeterminate results in terms of age, sex, ethnicity, immune suppressive therapy intake and presence of comorbidity conditions (data not shown). Hereafter, the data analysis was performed only in the 413 samples with valid in vitro test results. Demographic characteristics of these subjects are shown in Table 1. Among them we classified 173 patients (41.9%) as having confirmed tuberculosis and 43 (10.4%) as having clinical tuberculosis. We excluded active tuberculosis in 197 patients (47.7%). Based on localization site, 146 (67.6%) were classified as having pulmonary tuberculosis, 56 (25.9%) extrapulmonary tuberculosis and 14 (6.5%) had both pulmonary and extrapulmonary localization ( Table 2).
Response to RD1 selected peptides assay and comparison with the other tests WBE and ELISPOT readouts significantly correlate for the detection of the responses to RD1 selected peptides. Evaluation of the response to RD1 selected peptides was performed by 2 different readouts, the ELISPOT and the WBE that we previously demonstrated to significantly correlate with each other [15]. Also in this study, 138 samples were run in parallel with a significant correlation (percentage of agreement: 80.4%; p = 0.0001). Moreover no differences were found in terms of detection of positive results in those with active tuberculosis among the patients from the different centers (p.0.5). Given the concordance of the results, the data were pooled together and analyzed as a whole. We also evaluated the sensitivities of the different tests based on tuberculosis localization. In Table 2, sensitivities for pulmonary, extra-pulmonary and disseminated tuberculosis (pulmonary and extra-pulmonary tuberculosis) are shown. Considering each test per se, no significant difference in proportion of positive results was observed in patients with active tuberculosis according to tuberculosis localization [with the exception of TST for which the highest proportion of positive results was recorded among patients with pulmonary tuberculosis (p = 0.027)].
Of the 132 patients with culture confirmed pulmonary tuberculosis, 28 had a negative sputum smear. Among these patients, sensitivity results were 64.3% ( Table 2). The specificity for active tuberculosis was significantly higher for the assay based on RD1 selected peptides compared with TST (p,0.001) and T-SPOT.TB (p,0.001).
Then we assessed whether the pair-wise combination of the tests could lead to a better evaluation of active tuberculosis diagnosis and we calculated the probabilities for the potential outcomes (double positive, double negative and discordant results) given the disease status. The probability of a positive result of at least one of the two tests considered was higher than sensitivities of each test (  (Table 4).
Negative results on combined tests were associated with lower negative likelihood ratios compared to that obtained by single assay especially when a blood test was associated with TST. In particular the negative likelihood ratio of the combination of RD1 selected peptides test with TST was 0.19 (CI, 0.11-0.33), with QuantiFERON-TB GOLD In-Tube was 0.25 (CI, 0.16-0.38) and with T-SPOT.TB was 0.27 (CI, 0.14-0.50) ( Table 4). To note that better negative likelihood ratios were obtained by the combination of commercial tests with TST (Table 4).

Discussion
We present the results of a prospective multicenter trial of the TBNET that was designed to investigate the performance of a novel blood test based on RD1 selected peptides for the immunodiagnosis of active tuberculosis.
The novel assay had a higher specificity for active tuberculosis than the TST and commercial IGRAs, but it had a lower sensitivity. Although the novel assay had a higher likelihood ratio, none of the tests evaluated was accurate enough to discriminate patients with active tuberculosis from those without, probably because of the high levels of LTBI in the population studied. Combined use of TST with either the RD1 selected peptides test or with the other commercial IGRAs improved the diagnostic accuracy for active disease, especially when considering the combination of negative results, contributing to rapid exclusion of tuberculosis. However, M. tuberculosis culture remains the diagnostic gold standard for active tuberculosis and is required for identifying drug resistance. Consequently, active tuberculosis should not be ruled out in a high-risk individual without a thorough microbiological work-up for tuberculosis disease.
The specificity of the assay based on RD1 selected peptides was lower in this multicenter trial compared with earlier, smaller studies of more limited patient groups [15,16]. Nevertheless, also in the present study the test based on RD1 selected peptides maintains the higher specificity compared to commercial IGRAs and TST which is not unexpected. In fact, the commercial IGRAs and TST use a greater variety of epitopes to elicit M. tuberculosis immune responses by effector memory T-cells [28][29] being TST a crude preparation of several mycobacterial antigens, and commercially IGRAs based on pools of overlapping peptides spanning the whole length of CFP-10 and ESAT-6 proteins [5]. Conversely the selective approach of the design of the test based on RD1 selected peptides reduces false positive test results at the cost of a loss of diagnostic sensitivity [14][15][16]. Which would be more acceptable between false positive test results that may lead to overtreatment or false negative test results that potentially lead to missing of cases with active tuberculosis to be treated, is a matter of debate and is largely dependent upon the prevalence of M. tuberculosis infection and the pre-test probability of tuberculosis in a community.
Specificity of commercial IGRAs is considerably lower in the present study compared to what was reported in a recent updated meta-analysis [30]. This may be due to the fact that this report involve the enrollment of patients with a suspicion of active tuberculosis that could by affected also by LTBI, while the literature reported in the meta-analyses [30] enclosed low-risk subjects with no known tuberculosis exposure in low incidence settings. Conversely sensitivity results were similar to those recently reported in the literature because based on patients with active disease [5,16,30].
In terms of parameters used to evaluate the accuracy of diagnostic tests it is important to consider that while sensitivity and specificity are easy and straightforward measures, they are limited and must be considered as surrogates for patient-important outcomes. There is still lack of adequate data on important outcomes such as accuracy of diagnostic algorithms (rather than single tests), incremental or added value of IGRAs, impact of IGRAs on clinical decisionmaking and therapeutic choices, and the prognostic ability of IGRAs to accurately identify individuals with LTBI who are at the highest risk for progressing to active tuberculosis and therefore are most likely to benefit from preventive therapy. These issues need to be evaluated for further studies.
New tools for a rapid diagnosis of active tuberculosis are needed especially for the smear negative and extra-pulmonary cases. In the smear negative tuberculosis group, the overall response to RD1 selected peptides was 64.3% and, among the tests used for comparison, a significantly higher sensitivity was found only for the QuantiFERON-TB GOLD In-Tube. Conversely the sensitivity for extra-pulmonary tuberculosis was 71.4% for the RD1 selected peptides and this value did not significantly differ from the results obtained by the other tests, with the exception of TST. However, in general, the size of these sub-groups of patients was small and no definitive conclusion can be drawn.
The rate of indeterminate results in our study was similar to the 3% to 4% rates observed in other studies [31][32][33]. Moreover, the proportion of patients with tuberculosis was no higher among those with indeterminate results. As expected, false-negative results to any of the immune assays considered were associated with factors known to cause anergy such as disseminated disease. Taken together, our results suggest that none of the tests considered is accurate enough to be used in clinical practice to diagnose active tuberculosis, and new approaches should be considered. Recently, it has been shown that the discrimination of active tuberculosis from LTBI may be ameliorated by documenting recruitment of M. tuberculosis-specific lymphocytes to the site of the infection by RD1-specific ELISPOT assays [34][35][36] which may open a new strategy for the distinction of the two different status of tuberculosis.
Another recent study suggests that the combination of different immunodiagnostic tests may improve their diagnostic accuracy [21]. In fact it has been shown that, T-SPOT.TB or a new ELISPOT assay incorporating the Rv3879c to RD1 antigens, when used in combination with TST have an increased positive and negative likelihood ratio compared with single tests suggesting that this approach can be used to exclude active tuberculosis in patients with moderate to high pre-test probability of disease [21]. The results of the present study confirm and extend these findings. In fact, we confirmed the diagnostic performance of the commercial version of T-SPOT.TB when combined to TST, and additionally, we substantiated the data analyzing the results obtained by the assay based on RD1 selected peptides and the QuantiFERON-TB GOLD In-Tube. The likelihood ratio of a negative test result that was 0.38 with the assay based on RD1 selected peptides, became 0.19 when combined with a negative TST. Similarly, the combination of a negative result of the QuantiFERON-TB GOLD In-Tube assay with a negative TST scoring reached the lowest negative likelihood ratio of 0.07. This means that while a negative result to RD1 selected peptides test reduces 2.6-fold the odds of tuberculosis, a negative result to both tests, RD1 selected peptides and TST, would reduce the odds of tuberculosis 5.3-fold and using the combination of TST and QuantiFERON-TB GOLD In-Tube by 14.2-fold which may currently be the best available option to rapidly exclude tuberculosis by immunodiagnostic tests. Conversely, a positive result of the assay based on RD1 selected peptides with a corresponding likelihood ratio of 2.48 is of limited value and it is not significantly modified in those with positive results from either the assays based on RD1 selected peptides or the TST or QuantiFERON-TB GOLD In-Tube (likelihood ratio goes up to 2.92 and 3.21 respectively). Similarly, results on other tests combinations improve the positive likelihood ratio of the single test per se, but do not increase significantly that obtained by the combination of RD1 selected peptides with TST or Quanti-FERON-TB GOLD In-Tube.
The higher sensitivity of combined use of the novel assay or commercial IGRAs with TST reflects the fact that patients who had a false-negative result with one test were distinct from those who had a false-negative result with the other. This implies that distinct immunologic processes underlie failure of these different, yet complementary, immune-based tests.
The study has some limitations. Not all individuals were tested by all the assays in parallel, as not all techniques for the different tests were established in the participating centers. In addition, the restricted number of immunocompromised patients does not allow a generalization of the results to this patients group. However, the prospective and multicenter design of the study, the high consistency of data across the different centers and the large number of patients enrolled to evaluate the diagnostic accuracy of different immune based tests in clinical practice render the results robust.
In conclusion, current approaches to elicit M. tuberculosis-specific immune responses in PBMC or in the skin by using either a broad or narrow spectrum of epitopes of RD1 mycobacterial antigens have a limited value for the diagnosis of active tuberculosis, as these tests do not reliably distinguish patients with active tuberculosis from those without. This is important to be considered in populations with a high pre-test probability of M. tuberculosis infection. However, the combined use of negative test results obtained by IGRAs or the test based on RD1 selected peptides with TST may enable rapid exclusion of tuberculosis.