Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A pre-clinical validation plan to evaluate analytical sensitivities of molecular diagnostics such as BD MAX MDR-TB, Xpert MTB/Rif Ultra and FluoroType MTB

  • Markus Beutler ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Visualization, Writing – original draft

    m.beutler@imlred.de

    Affiliation Institute of Microbiology and Laboratory Medicine, Department IML Red GmbH, WHO-Supranational Reference Laboratory of Tuberculosis, Munich-Gauting, Germany

  • Sara Plesnik,

    Roles Investigation

    Affiliation Institute of Microbiology and Laboratory Medicine, Department IML Red GmbH, WHO-Supranational Reference Laboratory of Tuberculosis, Munich-Gauting, Germany

  • Marina Mihalic,

    Roles Investigation

    Affiliation Institute of Microbiology and Laboratory Medicine, Department IML Red GmbH, WHO-Supranational Reference Laboratory of Tuberculosis, Munich-Gauting, Germany

  • Laura Olbrich,

    Roles Resources, Writing – review & editing

    Affiliations Division of Infectious Diseases and Tropical Medicine, LMU University Hospital, Munich, Germany, German Centre of Research on Infectious Diseases (DZIF), Partner Site Munich, Germany

  • Norbert Heinrich,

    Roles Resources, Writing – review & editing

    Affiliations Division of Infectious Diseases and Tropical Medicine, LMU University Hospital, Munich, Germany, German Centre of Research on Infectious Diseases (DZIF), Partner Site Munich, Germany

  • Samuel Schumacher,

    Roles Conceptualization

    Affiliation Foundation for Innovative New Diagnostics, Geneva, Switzerland

  • Michael Lindner,

    Roles Resources

    Affiliation Asklepios Fachkliniken München-Gauting, Munich, Germany

  • Ina Koch,

    Roles Resources, Writing – review & editing

    Affiliation Asklepios Fachkliniken München-Gauting, Munich, Germany

  • Wolfgang Grasse,

    Roles Methodology, Resources

    Affiliation gerbion GmbH & Co. KG, Kornwestheim, Germany

  • Christoph Metzger-Boddien,

    Roles Resources

    Affiliation gerbion GmbH & Co. KG, Kornwestheim, Germany

  • Sabine Hofmann-Thiel,

    Roles Resources, Writing – review & editing

    Affiliations Institute of Microbiology and Laboratory Medicine, Department IML Red GmbH, WHO-Supranational Reference Laboratory of Tuberculosis, Munich-Gauting, Germany, SYNLAB Gauting, SYNLAB Human Genetics Munich, Munich-Gauting, Germany

  • Harald Hoffmann

    Roles Conceptualization, Funding acquisition, Investigation, Project administration, Supervision, Validation, Writing – review & editing

    Affiliations Institute of Microbiology and Laboratory Medicine, Department IML Red GmbH, WHO-Supranational Reference Laboratory of Tuberculosis, Munich-Gauting, Germany, SYNLAB Gauting, SYNLAB Human Genetics Munich, Munich-Gauting, Germany

A pre-clinical validation plan to evaluate analytical sensitivities of molecular diagnostics such as BD MAX MDR-TB, Xpert MTB/Rif Ultra and FluoroType MTB

  • Markus Beutler, 
  • Sara Plesnik, 
  • Marina Mihalic, 
  • Laura Olbrich, 
  • Norbert Heinrich, 
  • Samuel Schumacher, 
  • Michael Lindner, 
  • Ina Koch, 
  • Wolfgang Grasse, 
  • Christoph Metzger-Boddien
PLOS
x

Abstract

Rapid diagnosis of tuberculosis (TB) and antibiotic resistances are imperative to initiate effective treatment and to stop transmission of the disease. A new generation of more sensitive, automated molecular TB diagnostic tests has been recently launched giving microbiologists more choice between several assays with the potential to detect resistance markers for rifampicin and isoniazid. In this study, we determined analytical sensitivities as 95% limits of detection (LoD95) for Xpert MTB/Rif Ultra (XP-Ultra) and BD-MAX MDR-TB (BD-MAX) as two representatives of the new test generation, in comparison to the conventional FluoroType MTB (FT-MTB). Test matrices used were physiological saline solution, human and a mucin-based artificial sputum (MUCAS) each spiked with Mycobacterium tuberculosis in declining culture- and qPCR-controlled concentrations. With BD-MAX, XP-Ultra, and FT-MTB, we measured LoD95TB values of 2.1 cfu/ml (CI95%: 0.9–23.3), 3.1 cfu/ml (CI95%: 1.2–88.9), and 52.1 cfu/ml (CI95%: 16.7–664.4) in human sputum; of 6.3 cfu/ml (CI95%: 2.9–31.8), 1.5 cfu/ml (CI95%: 0.7–5.0), and 30.4 cfu/ml (CI95%: 17.4–60.7) in MUCAS; and of 2.3 cfu/ml (CI95%: 1.1–12.0), 11.5 cfu/ml (CI95%: 5.6–47.3), and 129.1 cfu/ml (CI95%: 82.8–273.8) in saline solution, respectively. LoD95 of resistance markers were 9 to 48 times higher compared to LoD95TB. BD-MAX and XP-Ultra have an equal and significantly increased analytical sensitivity compared to conventional tests. MUCAS resembled human sputum, while both yielded significantly different results than normal saline. MUCAS proved to be suitable for quality control of PCR assays for TB diagnostics.

Introduction

With 10 million new and 1.6 million mortal cases in 2017, tuberculosis (TB) still remains the world’s leading cause of death among single pathogen infections [1]. Anti-TB drug resistance (DR), especially multi-drug-resistance (MDR), pose major challenges to end TB [24]. Multi-drug-resistance is defined as resistance towards the two most powerful first line drugs rifampicin (Rif) and isoniazid (Inh). In previous decades, DR-TB mainly resulted from insufficient treatment; however, MDR-TB transmission is long promoting its own pandemic with an estimated 558,000 cases annually [5]. Rapid diagnosis is key to early initiation of effective treatment and to interrupt further spread of resistant TB, but in reality less than a third of MDR-TB cases are detected and therefore not treated [6,7].

Nucleic acid amplification tests (NAAT) are the fastest tools to accurately diagnose TB on the day of sample collection. While early generations of CE marked commercial NAATs like ProbeTec ET DTB (DTB) (Becton-Dickinson, USA) and COBAS TaqMan MTB (Roche, Switzerland) had relatively low sensitivities, more recent generations like FluoroType MTB (FT-MTB, Hain Lifescience, Germany) and Xpert MTB/RIF (XP-MTB; Cepheid, USA) have improved in this respect, but still markedly lag behind culture [810]. The cartridge-based XP-MTB additionally identifies Rif-resistance as a marker of MDR-TB [11] and has significantly ameliorated notification of TB and MDR-TB worldwide [12,13]. Now, with the launch of Abbott RealTime MTB and MTB INH/RIF (Abbott, USA), BD MAX MDR-TB (BD-MAX, Beckton Dickinson, USA), and Xpert MTB/Rif Ultra (XP-Ultra, Cepheid, USA) (Rif only), a new generation of closed and virtually fully automated qPCR assays detecting DNA of TB bacteria as well as Rif and/or Inh resistance markers are available which, according to the manufacturers’, have sensitivities nearing that of culture [1417].

The analytical sensitivity expressed as 95% percentile of the limit of detection (LoD95TB) is a key parameter for the evaluation of the performance of NAATs [14]. Several factors impact the LoD95TB, most importantly the matrix in which the bacteria are dispersed. Deionized water, normal saline or buffer are frequently experimentally used, but do not reflect the reality of diagnostics with clinical samples. Human sputum is the most frequently collected clinical specimen in TB diagnostics, which strongly differs in its complexity, composition, and consistency from water or saline and even from sample to sample. Therefore, LoD95TB values provided by authors from different studies are neither comparable among one another nor do they necessarily reflect the situation of diagnostics. In order to standardize a sputum matrix, either tremendous volumes of human sputum need to be pooled and extensively validated for the absence of Mycobacterium tuberculosis complex (MTBC)-DNA, or artificial sputum (AS) can be used which is produced under controlled and standardized conditions. Three major types of AS have been developed based either on polyacrylamide, on methylcellulose, or on watery dispersions [1820]. Unfortunately, none of these resembles natural human sputum in consistency, chemical properties or viscosity; therefore, they do not simulate a clinical diagnostic sample. In order overcome this challenge, we have recently developed a novel and infinitely reproducible mucin-based artificial sputum (MUCAS), which very closely resembles human sputum in all these aspects.

In this study, we aimed to measure and head-to-head compare the analytical sensitivity (LoD95TB) of XP-Ultra, BD-MAX and FT-MTB using physiological saline, human sputum and our novel MUCAS as test matrices using a standardized validation plan.

Methods

Preparation of test matrices

Pooled human sputum, normal saline and mucin based artificial sputum (MUCAS) were used as test matrices for the determination of LoD95TB. MUCAS consisted of a mucin-based matrix enriched with bacteria of the normal respiratory flora (Neisseria lactamica, Streptococcus rubneri and Moraxella lincolnii) and human mononuclear cells (hMNC). All test matrices were spiked with M. tuberculosis (MTB) bacteria.

Mycobacterium tuberculosis

MTB reference strain H37Rv (ATCC 35829) and a pre-characterized clinical MDR-MTB isolate T837 were cultured on Loewenstein-Jensen (LJ) medium (Enclit, Germany) for 2–4 weeks at 36±1°C. Bacterial suspensions were prepared in 0.85% saline solution containing 2.5 g of 1 mm glass beads (Neolab, Germany) and vortexed for 1 min. After two sedimentation steps of 10 min each, the final supernatant was adjusted to a turbidity of McF 4.0 measured in a BD phoenix nephelometer by adding 0.85% saline solution. To remove bulks of bacteria, this suspension was filtered through a 5 μm PVDF filter (Merck-Millipore, USA). The final turbidity was adjusted to McF 0.5 and serial dilutions produced with 0.85% saline.

Mucin based artificial sputum

MUCAS was produced following the procedures specified in the MUCAS patent (patent application pending in the name of IML red GmbH; application number: EP19165015.9). The main ingredients of MUCAS were: mucin from porcine stomach type II (Sigma-Aldrich, USA), Middlebrook 7H9 powder (BD, USA), salmon DNA (Sigma-Aldrich, USA) and glycerine (VWR Chemicals, USA). Each MUCAS LOT was controlled for the absence of MTB DNA by performing XP-Ultra tests from 1 ml MUCAS from three different aliquots following the instructions of the manufacturer.

Human sputum

Around 500 fully anonymized non TB, left-over sputum samples from regular patient care were mixed and stored at -20°C until usage. It was impossible to identify individual patients. No clinical or personal data were collected, stored or used for any purposes. Patients consented to use their samples for bacteriological research on TB. The study was reviewed and approved by the ethics committee of the Ludwig-Maximilians-University Munich (Germany). 7 x 200 ml sputum samples were pooled and controlled for the absence of MTB DNA by performing one XP-Ultra test per pool. Four MTB-negative pools were united to one master pool, homogenized by vigorous shaking and aliquoted in 50 ml aliquots. MTBC negativity was verified for five representative aliquots by inoculation of LJ slants after decontamination following the NALC-NaOH standard protocol [21], and incubating the cultures for 8 weeks at 36±1°C. Aliquots were stored at -20°C until usage.

LoD95TB and LoD95HR (H: isoniazid-, R: rifampicin-resistance)

LoD95 values were determined in two phases (Fig 1). In phase 1 the approximate LoD was determined for each target assay. 10x pre-dilutions of H37Rv bacteria were produced in 0.85% saline and spiked into the test matrices to reach final 10-fold dilution series ranging from 1x104 to 1x100 cfu/ml. In phase 2 the precise LoD95 values were measured by the use of 1:2 titer dilutions of suspensions of H37Rv bacteria spiked in the test matrices to reach six bacterial concentrations around the approximate LoD of each assay determined in phase 1. In the same way, dilutions of MDR-TB strain T837 were prepared around the target ranges of the assays to determine the LoD95HR of the assays to detect Inh (H) and/or Rif (R) resistance markers. Control plating on 7H11 agar plates (incubation at 36±1°C for 3 weeks) were performed for each dilution series and the LoD95 values corrected according to the counted numbers of cfu. The LoD95, i.e. the bacterial concentration in the respective matrix that yielded positive results in 95% of test runs, was calculated using GraphPad Prism 8.0.2 (GraphPad Software, USA).

thumbnail
Fig 1. Schematic overview of LoD95 experiments.

10x pre-dilutions of H37Rv and an MDR MTBC isolate were prepared and spiked in human sputum, MUCAS or saline solution. In phase 1 three 10-fold dilution series (104 to 0 cfu/ml) were produced in all test matrices. The orienting LoD was determined for each matrix/test combination (in the example, 100 negative & 101 positive = > LoD 10 cfu/ml). In phase 2 the LoD95TB / HR were determined using at least six 2-fold dilutions per matrix/test around the respective orienting LoDs. Calculated LoD values were corrected according to colony counts of plates and interpolated using GraphPad Prism 8.0.2 to yield the 95% LoD and the 95%-CIs (GraphPad Software, USA).

https://doi.org/10.1371/journal.pone.0227215.g001

Determination of cfu’s

Dilutions started with McF 0.5 corresponding to approximately 2.5 x 106 cfu/ml of viable bacteria when processed according to our protocol, which is comparable to a previous study [22]. The exact number of cfu’s of each dilution series was determined by plating 100 μl of the suspensions with expected concentrations of 100 and 1000 cfu/ml on 7H11 agar plates and counting colonies after 3 weeks of incubation at 36±1°C. 900 μl aliquots of each suspension plated for colony counts were heat inactivated (30 min at 80°C) within 2 hours after production and stored at -20°C for maximum six months until quantitative IS6110 PCR was performed as described below.

BD-MAX, XP-Ultra, FT-MTB

All commercial PCR assays were CE marked and performed following to the manufacturer’s instructions:

BD-MAX and XP-Ultra.

2 ml of either sample treatment reagent (BD-MAX) or sample reagent (XP-Ultra) were added to 1 ml spiked test matrix and inverted 10–20 times. After 5 min (BD-MAX) or 10 min (XP-Ultra) incubation, the samples were again inverted 10–20 times and further incubated for 25 min (BD-MAX) or 5 min (XP-Ultra) at room temperature. Samples were either transferred to BD-MAX tubes and sealed with a septum-cap or to XP-Ultra cartridges. The pre-treated samples together with the respective cartridges were loaded on the BD-MAX and GeneXpert machines and the programs were started following the algorithms of the respective platforms.

FT-MTB.

1.0 ml of spiked test matrix was decontaminated with NALC-NaOH following the WHO standard protocol [21]. DNA extraction from 500 μl of decontaminated sample was performed using the FluoroLyse (Hain Lifescience, Germany). FT-MTB PCR mixes were prepared from AM-A and AM-B solutions and 6 μl of DNA was added. Positive (6 μl of C+ FT MTB) and negative controls (6 μl of FluoroLyse master-mix without spiked sample) were included in each run. PCRs were performed in FluoroCycler 12 instruments (Hain Lifescience, Germany) using the FluoroType Analysis software.

Quantitative IS6110 PCR and LoD95IS6110

The CE marked diarella MTB/NTM/MAC Kit (gerbion, Germany) and a synthetic IS6110 DNA standard (gerbion, Germany) were used for qPCR to absolutely quantify numbers of IS6110 insertion elements in test samples. A standard curve with tenfold dilutions ranging from 106 to 100 IS6110 insertion elements/ml was produced and run in duplicates to determine PCR efficiency (result = 109%) and the dynamic range (down to 100 copies) of the assay. PCR reactions were executed on a Roche LightCycler 480 following the manufacturers’ instructions. Intensities of FAM (MTBC specific) and HEX (internal control) fluorescence were measured and recorded after each 60°C step. Colour compensation was performed using the alphaCube LC480 Colour Compensation kit (Mikrogen Diagnostik, Germany). Numbers of IS6110 insertion elements accessible for PCR were quantified by reproducing the initial standard curve produced with a single synthetic IS6110 standard using the LightCycler software (Roche, Germany). For the determination of the LoD95IS6110, IS6110 copies/ml were quantified for each H37Rv and T837 suspension with an expected bacterial concentration of 103 cfu/ml. Individual cfu’s at the LoD in cfu/ml of each standard curve were determined and multiplied with the average IS6110 copies/cfu. Furthermore, colony numbers counted on control plates were correlated to qPCR results. The resulting linear regression curve is presented in S1 Fig.

Statistics and data analysis

Data were graphed and statistically analysed using GraphPad Prism 8.0.2. LoD95 values were interpolated and plotted with 95% confidence intervals. Differences between multiple groups were compared using one-way ANOVA with Tukey’s multiple comparison test. A p-value ≤ 0.05 was considered statistically significant. Numbers IS6110 insertion per cfu were expressed as the geometric mean with 95% confidence interval.

Results

BD-MAX and XP-Ultra yield comparable analytical sensitivities

To comparatively investigate the analytical sensitivities of BD-MAX, XP-Ultra and FT-MTB, serial dilutions of H37Rv suspensions with controlled bacterial concentrations (in cfu/ml and IS6110 copies/ml) were spiked in three test matrices: human sputum, MUCAS and 0.85% saline. The LoD95 of each assay was determined independently for each test matrix.

The analytical sensitivities of BD-MAX and XP-Ultra were comparable in all three matrices and five to 56 times higher than those of FT-MTB (Fig 2). Values were interpolated from semi-log transformed curves. In both human sputum and MUCAS, BD-MAX (2.1 cfu/ml, CI95%: 0.9–23.3; and 6.3 cfu/ml, CI95%: 2.9–31.8; respectively) and XP-Ultra (3.1 cfu/ml, CI95%: 1.2–88.9; and 1.5 cfu/ml, CI95%: 0.7–5.0; respectively) yielded LoD95TB values within similar though markedly lower ranges than FT-MTB (52.1 cfu/ml, CI95%: 16.7–664.4; and 30.4 cfu/ml, CI95%: 17.4–60.7; respectively). This difference became even more distinct in normal saline with which FT-MTB (129.1 cfu/ml, CI95%: 82.8–273.8) yielded a 11 to 56 times higher LoD95TB than BD-MAX or XP-Ultra (2.3 cfu/ml, CI95%: 1.1–12.0; and 11.5 cfu/ml, CI95%: 5.6–47.3; respectively).

thumbnail
Fig 2.

LoD95TB of BD-MAX MDR-TB (BD-MAX), Xpert MTB/Rif Ultra (XP-Ultra) and FluoroType MTB (FT-MTB) in human sputum (A), MUCAS (B) and physiological saline solution (C). Red arrow: calculated LoD95TB, solid black line: predicted positive assay, blue dashed line: 95% probability of a positive assay, black dashed lines: upper and lower CI95 intervals.

https://doi.org/10.1371/journal.pone.0227215.g002

LoD95IS6110.

While LoD95TB was referenced to the number of bacteria determined as cfu/ml in quantitative cultures, LoD95IS6110 was referenced to copies of IS6110 DNA in the solution. IS6110 copies of H37Rv and MDR-TB strain T837 suspensions were determined using the CE marked commercial assay diarellaMTB/NTM/MAC Kit and the associated synthetic IS6110 DNA standard (gerbion, Germany) which yielded an average of 11,777 IS6110 copies/cfu of H37Rv (IC95: 5,743 to 24,150) and 2,900 IS6110 copies/cfu of T837 (IC95: 339 to 24,815), suggesting that under in vitro conditions of spiked test matrices a large quantity of MTBC DNA occurs either as free molecules or bound to inanimate bacterial bodies.

The LoD95IS6110 was defined as the limit of detection at which the concentration of MTB DNA in the respective matrix corresponded to the reported numbers of IS6110 copies and the assay under investigation yielded 95% positive results. The distribution of LoD95IS6110 was determined in a total of 74 test series of two-fold dilutions of H37Rv bacteria in the respective test matrices, which are displayed in Fig 3 as Whisker min/max boxes around the respective medians. While LoD95IS6110 distributions of BD-MAX and XP-Ultra largely overlapped, LoD95IS6110 values of FT-MTB were approximately two logs higher in all three test media (p≤0.05 to p≤0.0001; one-way ANOVA). Investigating the influence of the test media on the LoD95IS6110 of the test assays, saline yielded significantly higher values with all assays than human sputum and MUCAS (p≤0.01), while MUCAS yielded only with BD-MAX slightly higher LoD95IS6110 values in MUCAS than in human sputum (p≤0.05).

thumbnail
Fig 3. Whiskers Min to Max of IS6110 copies at the limit of detection of BD-MAX, XP Ultra and FluoroType in different test matrices.

Assays were performed in human sputum (Sputum), artificial sputum (MUCAS) and 0.85% saline solution (Saline). The limit of detection (LoD) was determined using two-fold dilutions of H37Rv spiked in test matrix. Number of test series of BD-MAX, XP-Ultra, FT-MTB in human sputum, MUCAS and saline were 8, 10, 6, and 8, 9, 9 and 9, 6, 9, respectively. **** p≤0.0001, *** p≤0.001, ** p≤0.01, * p≤0.05 (one-way ANOVA with Tukey’s multiple comparison test).

https://doi.org/10.1371/journal.pone.0227215.g003

Sensitivity of Rif and Inh resistance marker detection is decreased

The analytical sensitivities of BD-MAX and XP-Ultra with regards to the detection of genetic resistance markers (reported as LoD95HR; H = Inh, R = Rif) were lower than the detection of MTB (Table 1, S2 and S3 Figs). With H37Rv, XP-Ultra and BD-MAX validly indicated Rif (and Inh) susceptibility at 1.8 to 9.2 times and 4.9 to 48.4 times, respectively, higher bacterial concentrations than needed to validly detect MTBC. Both, LoD95Rif and LoD95Inh were lower in saline and higher in MUCAS than in human sputum.

thumbnail
Table 1. Analytical sensitivity for the detecting genetic resistance markers of Inh (H) and Rif (R) reported as LoD95HR of BD-MAX and XP-Ultra determined by mathematical interpolation.

https://doi.org/10.1371/journal.pone.0227215.t001

In order to verify that the LoD95HR were reproducible for resistant strains, we twice tested MDR-TB strain T837 which harbors the most frequent resistance mutations rpoB S531L and katG S315T. In human sputum, LoD95HR values of BD-MAX and XP-Ultra were within the same ranges with both strains, H37Rv and T837 (Table 1, S2 and S3 Figs). T837 yielded high LoD95HR values for Rif with XP-Ultra in all matrices and with BD-MAX in saline and MUCAS, respectively; H37Rv yielded high LoD95HR values for Inh with BD-MAX in MUCAS, however, 95% confidence intervals were wide.

Discussion

Recently launched PCR assays including BD-MAX, XP-Ultra and FluoroType MTBDR allow microbiologists to choose from a larger variety of improved TB diagnostics. Besides framework conditions like spectrum of analysis, investment, running costs and practicability, the sensitivity is one of the major criteria driving the decision for a specific test. However, sensitivity rates reported by manufacturers and authors of evaluation studies are rarely comparable because test conditions, particularly test matrices and bacterial concentrations of test samples, can strongly deviate from one another. With our study we provide not only a direct head-to-head comparison of analytical sensitivities of two prominent representatives of the new generation of TB assays, we also propose a standardized evaluation plan of analytic sensitivity using an artificial sputum with precisely defined bacterial concentrations.

When using FT-MTB as a well-established representative of the recent TB-PCR generation, both BD-MAX and XP-Ultra showed significantly higher analytical sensitivity for MTBC. Only FT-MTB requires NALC-NaOH decontamination before DNA release; therefore, the loss of bacteria and / or free DNA before amplification might account for the lower sensitivity as it has been observed that NALC-NaOH decontamination reduces colony counts [23]. When we correlated the numbers of IS6110 copies with the colony counts in this study, the presence of free DNA or dead bacteria became obvious: one cfu of H37Rv corresponded to more than eleven thousand IS6110 copies in bacterial suspensions before spiking the test matrices. As H37Rv contains sixteen IS6110 elements [24], we observed on average 736 MTB genomes per cfu. Since we have filtered the bacterial suspensions through 5 μm pores before plating for cfu counts, we hypothesize that 1 cfu originated only from one to very few viable bacterial cells. Free DNA might have partially resulted from osmotic cell rupture in saline solutions or from NaOH induced denaturation [25]. During decontamination, free DNA would be washed off with the effect that total DNA content introduced in the FT-MTB assay would be significantly lower than in XP-Ultra and BD-MAX resulting in potentially biased LoD95 values of FT-MTB in such type of in vitro studies. Whether decreased sensitivity due to DNA loss during sample decontamination has a clinical impact needs to be further investigated. One should be also aware that free DNA in spiked samples might artificially lower the limit of detection. In a previous study, the LoD95TB value of XP-Ultra was reported to be 15.6 cfu/ml [14], higher than our measured LoD95TB of 1.5 to 11.5 cfu/ml (depending on the matrix). However, free DNA has equally lowered LoD95 values of both XP-Ultra and BD-MAX in our system; therefore, a direct comparison of analytical sensitivities can be made. To overcome the bias of free DNA, future investigations of the diagnostic sensitivity should include sputum samples of relevant TB-patient groups.

Although BD-MAX and XP-Ultra yield comparable analytical sensitives, slight inter-test variations were observed using different test matrices. For BD-MAX, MTB detection increased 3-fold in MUCAS compared to human sputum and declined two-fold by XP-Ultra. In addition, the LoD95 for MTB detection in saline was almost identical as in human sputum by BD-MAX, but increased nearly 4-fold by XP-Ultra. The difference in lysis chemistries and DNA extraction procedures applied by both assays might account for this observation.

Rapid detection of antibiotic resistances is essential for early and appropriate treatment initiation. With the upgrade from XP MTB/RIF to XP-Ultra, Cepheid has markedly increased the assay’s sensitivity which is mainly noticeable at very low bacterial concentrations identified by the assay as ‘trace call’ [26]. Additionally, at such low concentrations XP-Ultra flags Rif resistance markers as ‘indeterminate’. In our recent evaluation of the ABBOTT RealTime MTB INH/RIF, we observed a high rate of indeterminate resistance markers for both Rif and Inh, when smear microscopy was scanty or negative [17]. Zimmermann et al observed 8.9% of indeterminate Rif/Inh results with BD-MAX in 518 clinical samples [15] which is in line with the before mentioned publications on Xpert MTB/RIF [14] and Abbott RealTime MTB RIF/INH [17]. This confirms that molecular resistance profiling is less sensitive compared to MTBC detection and coincides with our findings that XP-Ultra’s and BD-MAX’s LoD95HR were higher than its LoD95TB, especially when saline or MUCAS were used albeit not being approved by the manufacturer. MUCAS was inferior to human sputum when antibiotic resistance was tested using BD-MAX, which might indicate a more limited use for monitoring resistance. However, only a slightly decreased analytical sensitivity for Rif resistance screening was observed in MUCAS by XP-Ultra. This is in line with a recent study using TB/MDR-TB sputum samples showing that indeterminate results are obtained more frequently with BD-MAX than XP-Ultra for the detection of drug resistance [27]. Nevertheless, BD-MAX detected MTBC as well as antibiotic resistance with high sensitivity and specificity comparable to Xpert MTB/RIF in a clinical multicenter study in South Africa, Uganda, India and Peru [28].

With this study, we have also introduced a novel fully standardized mucin based artificial sputum for the evaluation and inter-assay comparison of the analytical sensitivity of diagnostic assays. We demonstrated that NAAT-based diagnostic assays performed mostly comparable with spiked MUCAS and pooled human sputum, but differently with saline. Since human sputum is not always obtainable and highly heterogeneous [2932], test results and inter-laboratory and inter-study comparability of results are impossible. MUCAS is chemically very similar to and behaves like human sputum in molecular assays. It is easily manufacturable and always reproducible in required quantities and can serve as an alternative test matrix for any kind of evaluation, validation or verification of molecular TB assays applied to respiratory specimens. By adhering to the manufacturers’ instructions, using identical test matrices which are produced and spiked following the same protocol, and calculating LoD95 values using the same algorithms, analytical sensitivity values will become fully comparable even when determined at different times and places. This would be tremendously helpful for laboratory experts to make the right decision for or against an assay which might otherwise fulfil all framework conditions.

It was beyond the scope of this study to include more clinical isolates with relevant mutations causing Inh and Rif resistance [33]; however this should be considered in future research. Diagnostic sensitivities using sputum from patients were also not investigated. This paper provides a pre-clinical evaluation plan for analytical sensitivities of molecular diagnostics.

Supporting information

S1 Fig. Correlation of IS6110 insertion elements with cfu’s.

https://doi.org/10.1371/journal.pone.0227215.s001

(TIF)

S2 Fig.

LoD95HR (detection of Inh and/or Rif resistance markers) of BD-MAX, XP-Ultra, and FT-MTB in human sputum (A), MUCAS (B), and physiological saline solution (C) using MTB reference strain H37Rv.

https://doi.org/10.1371/journal.pone.0227215.s002

(TIF)

S3 Fig.

LoD95HR (detection of Inh and/or Rif resistance markers) of BD-MAX, XP-Ultra, and FT-MTB in human sputum (A), MUCAS (B), and physiological saline solution (C) using clinical MDR strain T837.

https://doi.org/10.1371/journal.pone.0227215.s003

(TIF)

Acknowledgments

We deeply thank SYNLAB Gauting for the provision of the diagnostic MDR-MTB isolate T837 and for providing access to the BSL-3 laboratory for work performed for this study. Special thanks go to the colleagues of the TB-Tube and TB-SeqDisK consortia for the excellent and fruitful collaboration which has motivated us to this research initiative. We also thank Dr. Caroline Corbett for critically proofreading the manuscript.

References

  1. 1. WHO. Global tuberculosis report 2018. In: WHO [Internet]. 2018 [cited 22 Feb 2019]. Available: http://www.who.int/tb/publications/global_report/en/
  2. 2. Franco-Sotomayor G, Garzon-Chavez D, Leon-Benitez M, de Waard JH, Garcia-Bereguiain MA. A First Insight into the katG and rpoB Gene Mutations of Multidrug-Resistant Mycobacterium tuberculosis Strains from Ecuador. Microb Drug Resist Larchmt N. 2018. pmid:30513056
  3. 3. Orenstein EW, Basu S, Shah NS, Andrews JR, Friedland GH, Moll AP, et al. Treatment outcomes among patients with multidrug-resistant tuberculosis: systematic review and meta-analysis. Lancet Infect Dis. 2009;9: 153–161. pmid:19246019
  4. 4. Tang S, Tan S, Yao L, Li F, Li L, Guo X, et al. Risk Factors for Poor Treatment Outcomes in Patients with MDR-TB and XDR-TB in China: Retrospective Multi-Center Investigation. PLOS ONE. 2013;8: e82943. pmid:24349402
  5. 5. Shah NS, Wright A, Bai G-H, Barrera L, Boulahbal F, Martín-Casabona N, et al. Worldwide Emergence of Extensively Drug-resistant Tuberculosis. Emerg Infect Dis. 2007;13: 380–387. pmid:17552090
  6. 6. Nathanson E, Nunn P, Uplekar M, Floyd K, Jaramillo E, Lönnroth K, et al. MDR Tuberculosis—Critical Steps for Prevention and Control. N Engl J Med. 2010;363: 1050–1058. pmid:20825317
  7. 7. Nema V. Tuberculosis diagnostics: Challenges and opportunities. Lung India Off Organ Indian Chest Soc. 2012;29: 259–266. pmid:22919166
  8. 8. Antonenka U, Hofmann-Thiel S, Turaev L, Esenalieva A, Abdulloeva M, Sahalchyk E, et al. Comparison of Xpert MTB/RIF with ProbeTec ET DTB and COBAS TaqMan MTB for direct detection of M. tuberculosis complex in respiratory specimens. BMC Infect Dis. 2013;13: 280. pmid:23786563
  9. 9. Hofmann-Thiel S, Hoffmann H. Evaluation of Fluorotype MTB for detection of Mycobacterium tuberculosis complex DNA in clinical specimens from a low-incidence country. BMC Infect Dis. 2014;14: 59. pmid:24498967
  10. 10. Steingart KR, Schiller I, Horne DJ, Pai M, Boehme CC, Dendukuri N. Xpert® MTB/RIF assay for pulmonary tuberculosis and rifampicin resistance in adults. Cochrane Database Syst Rev. 2014 [cited 16 Jan 2019]. pmid:24448973
  11. 11. Boehme CC, Nabeta P, Hillemann D, Nicol MP, Shenai S, Krapp F, et al. Rapid Molecular Detection of Tuberculosis and Rifampin Resistance. N Engl J Med. 2010;363: 1005–1015. pmid:20825313
  12. 12. Albert H, Nathavitharana RR, Isaacs C, Pai M, Denkinger CM, Boehme CC. Development, roll-out and impact of Xpert MTB/RIF for tuberculosis: what lessons have we learnt and how can we do better? Eur Respir J. 2016;48: 516–525. pmid:27418550
  13. 13. Calligaro GL, Zijenah LS, Peter JG, Theron G, Buser V, McNerney R, et al. Effect of new tuberculosis diagnostic technologies on community-based intensified case finding: a multicentre randomised controlled trial. Lancet Infect Dis. 2017;17: 441–450. pmid:28063795
  14. 14. Chakravorty S, Simmons AM, Rowneki M, Parmar H, Cao Y, Ryan J, et al. The New Xpert MTB/RIF Ultra: Improving Detection of Mycobacterium tuberculosis and Resistance to Rifampin in an Assay Suitable for Point-of-Care Testing. Nacy CA, editor. mBio. 2017;8. pmid:28851844
  15. 15. Zimmermann S, Dalpke A, Murray P, Paradis S, Cooper C. Pre-validation of the BD MAX MDR-TB* assay for the rapid detection of MTBc DNA and mutations associated with rifampin and isoniazid resistance Introduction Results. In: ResearchGate [Internet]. 4 Apr 2018 [cited 22 Feb 2019]. Available: https://www.researchgate.net/publication/324804012_Pre-validation_of_the_BD_MAX_MDR-TB_assay_for_the_rapid_detection_of_MTBc_DNA_and_mutations_associated_with_rifampin_and_isoniazid_resistance_Introduction_Results
  16. 16. Hofmann-Thiel S, Molodtsov N, Antonenka U, Hoffmann H. Evaluation of the Abbott RealTi m e MTB and RealTi m e MTB INH/RIF Assays for Direct Detection of Mycobacterium tuberculosis Complex and Resistance Markers in Respiratory and Extrapulmonary Specimens. Land GA, editor. J Clin Microbiol. 2016;54: 3022–3027. pmid:27733630
  17. 17. Hofmann-Thiel S, Molodtsov N, Duffner C, Kadyrov A, Kalmambetova G, Kabirov O, et al. Capacity of Abbott RealTime MTB RIF/INH to detect rifampicin- and isoniazid-resistant tuberculosis. 1 Apr 2019 [cited 20 May 2019]. pmid:31064625
  18. 18. Diraviam Dinesh S. Artificial Sputum Medium. Protoc Exch. 2010 [cited 22 Feb 2019].
  19. 19. Yamada H, Mitarai S, Wahyunitisari MR, Mertaniasih NM, Sugamoto T, Chikamatsu K, et al. Improved Polyacrylamide-Based Artificial Sputum with Formalin-Fixed Tubercle Bacilli for Training of Tuberculosis Microscopists. J Clin Microbiol. 2011;49: 3604–3609. pmid:21813720
  20. 20. Rogers JV, Choi YW. Preliminary Evaluation of Mycobacterium tuberculosis Detection in Culture and Artificial Sputum Using a BioNanoPore Membrane and Real-time PCR. J Microb Biochem Technol. 2012;4: 147–151.
  21. 21. Kent PT, Kubica GP. Public Health Mycobacteriology: A Guide for the Level III Laboratory. | National Technical Reports Library—NTIS. 1985 [cited 8 May 2019]. Available: https://ntrl.ntis.gov/NTRL/dashboard/searchResults/titleDetail/PB86216546.xhtml
  22. 22. Peñuelas-Urquides K, Villarreal-Treviño L, Silva-Ramírez B, Rivadeneyra-Espinoza L, Said-Fernández S, León MB de. Measuring of Mycobacterium tuberculosis growth: a correlation of the optical measurements with colony forming units. Braz J Microbiol. 2013;44: 287–290. pmid:24159318
  23. 23. Burdz TVN, Wolfe J, Kabani A. Evaluation of sputum decontamination methods for Mycobacterium tuberculosis using viable colony counts and flow cytometry. Diagn Microbiol Infect Dis. 2003;47: 503–509. pmid:14596969
  24. 24. Zheng H, Lu L, Wang B, Pu S, Zhang X, Zhu G, et al. Genetic Basis of Virulence Attenuation Revealed by Comparative Genomic Analysis of Mycobacterium tuberculosis Strain H37Ra versus H37Rv. PLOS ONE. 2008;3: e2375. pmid:18584054
  25. 25. Luo Y, Roux B. Simulation of Osmotic Pressure in Concentrated Aqueous Salt Solutions. J Phys Chem Lett. 2010;1: 183–189.
  26. 26. WHO. WHO meeting report of a technical expert consultation: non-inferiority analysis of Xpert MTB/RIF Ultra compared to Xpert MTB/RIF. World Health Organization; 2017.
  27. 27. Mokaddas EM, Ahmad S, Eldeen HS. GeneXpert MTB/RIF Is Superior to BBD Max MDR-TB for Diagnosis of Tuberculosis (TB) in a Country with Low Incidence of Multidrug-Resistant TB (MDR-TB). J Clin Microbiol. 2019;57: e00537–19. pmid:30971464
  28. 28. Shah M, Paradis S, Betz J, Beylis N, Bharadwaj R, Caceres T, et al. Multicenter Study of the Accuracy of the BD MAXTM MDR-TB Assay for Detection of Mycobacterium tuberculosis Complex and Mutations Associated with Resistance to Rifampin and Isoniazid. Clin Infect Dis Off Publ Infect Dis Soc Am. 2019. pmid:31560049
  29. 29. Andrews RH, Radhakrishna S. A Comparison of Two Methods of Sputum Collection in the Diagnosis of Pulmonary Tuberculosis. Tubercle. 1959;40: 155–62. Available: https://www.cabdirect.org/cabdirect/abstract/19602700098 pmid:13793613
  30. 30. Boat TF, Cheng PW, Iyer RN, Carlson DM, Polony I. Human respiratory tract secretions: Mucous glycoproteins of nonpurulent tracheobronchial secretions, and sputum of patients with bronchitis and cystic fibrosis. Arch Biochem Biophys. 1976;177: 95–104. pmid:999298
  31. 31. Stockley RA, Bayley D, Hill SL, Hill AT, Crooks S, Campbell EJ. Assessment of airway neutrophils by sputum colour: correlation with airways inflammation. Thorax. 2001;56: 366–372. pmid:11312405
  32. 32. Zhao J, Li J, Schloss PD, Kalikin LM, Raymond TA, Petrosino JF, et al. Effect of Sample Storage Conditions on Culture-Independent Bacterial Community Measures in Cystic Fibrosis Sputum Specimens. J Clin Microbiol. 2011;49: 3717–3718. pmid:21865433
  33. 33. Miotto P, Tessema B, Tagliani E, Chindelevitch L, Starks AM, Emerson C, et al. A standardised method for interpreting the association between mutations and phenotypic drug resistance in Mycobacterium tuberculosis. Eur Respir J. 2017;50: 1701354. pmid:29284687