Figures
Abstract
Acute Respiratory Infections (ARIs) are responsible for considerable morbidity and mortality worldwide. Documentation of respiratory specimens can help for an appropriate clinical management with a significant effect on the disease progress in patient, the antimicrobial therapy used and the risk of secondary spread of infection. Here, we compared the performances of four commercial multiplex kits used in French University Hospital diagnostic microbiology laboratories for the detection of ARI pathogens (i.e., the xTAG Respiratory Viral Panel Fast, RespiFinder SMART 22, CLART PneumoVir and Fast Track Diagnostics Respiratory Pathogen 33 kits). We used a standardised nucleic acids extraction protocol and a comprehensive comparative approach that mixed reference to well established real-time PCR detection techniques and analysis of convergent positive results. We tested 166 respiratory clinical samples and identified a global high degree of correlation for at least three of the techniques (xTAG, RespiFinder and FTD33). For these techniques, the highest Youden’s index (YI), positive predictive (PPV) and specificity (Sp) values were observed for Core tests (e.g., influenza A [YI:0.86–1.00; PPV:78.95–100.00; Sp:97.32–100.00] & B [YI:0.44–1.00; PPV:100.00; Sp:100.00], hRSV [YI:0.50–0.99; PPV:85.71–100.00; Sp:99.38–100.00], hMPV [YI:0.71–1.00; PPV:83.33–100.00; Sp:99.37–100.00], EV/hRV [YI:0.62–0.82; PPV:93.33–100.00; Sp:94.48–100.00], AdV [YI:1.00; PPV:100.00; Sp:100.00] and hBoV [YI:0.20–0.80; PPV:57.14–100.00; Sp:98.14–100.00]). The present study completed an overview of the multiplex techniques available for the diagnosis of acute respiratory infections.
Citation: Salez N, Vabret A, Leruez-Ville M, Andreoletti L, Carrat F, Renois F, et al. (2015) Evaluation of Four Commercial Multiplex Molecular Tests for the Diagnosis of Acute Respiratory Infections. PLoS ONE 10(6): e0130378. https://doi.org/10.1371/journal.pone.0130378
Editor: Kwok H. Chan, University of Hong Kong, HONG KONG
Received: October 20, 2014; Accepted: May 20, 2015; Published: June 24, 2015
Copyright: © 2015 Salez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Acute Respiratory Infections (ARIs) are responsible for considerable morbidity and mortality worldwide [1]. The precise and rapid documentation of respiratory specimens can lead to a significant effect on the disease progress in patients [2] when identification of the causative agents leads to providing an appropriate therapy administration, but it is also useful for decreasing the use of unnecessary antimicrobial therapy [3,4] and limiting the risk of secondary spread of infection [5]. However, ARIs are most often associated with poorly specific clinical presentations [6] and therefore, their aetiological diagnosis mainly relies on laboratory testing.
Conventional laboratory diagnostic methods have several limitations for routine detection of respiratory pathogens. For example, culture isolation of viruses of bacteria is poorly sensitive, time consuming and usually too slow to provide a biological result at the acute phase of the disease [7]; direct immunofluorescence assays and immunochromatographic antigen testing can provide results very rapidly but represent time- and labor-consuming tasks for the former, and are poorly sensitive for the latter [8,9,10]. Molecular biology testing allows the detection of a variety of viral and bacterial pathogens within hours, with excellent sensitivity and specificity and may represent a credible alternative to the aforementioned biological assays. However, given the number of aetiological agents potentially implicated in ARIs, conventional monoplex PCR assays are tedious, expensive and require large amounts of biological samples [10,11]. Consequently, multiplex molecular detection tests have been increasingly developed in recent years and a number of commercial kits have been proposed to diagnostic microbiology laboratories, which allow the detection of 12 to 33 different pathogens. This includes the xTAG Respiratory Viral Panel, Fast or v1, assays (Luminex Molecular Diagnostics, Toronto, Canada); the FilmArray respiratory viral panel assay (BioFire Diagnostics, Salt Lake City, UT); the Fast Track Diagnostics Respiratory Pathogen, 21 or 33, assays (Fast Track Diagnostics, Luxembourg); the CLART PneumoVir (Genomica, Coslada, Spain); the RespiFinder, 19 or SMART 22, assay (Pathofinder, Maastricht, Netherlands); the eSensor Respiratory Viral Panel (GenMark Dx, Carlsbad, CA); the MultiCode-PLx (EraGen Biosciences); the Easyplex respiratory pathogen 12 kit (Ausdiagnostics, Sydney, Australia); the Seeplex RV15 OneStep ACE Detection and Pneumobacter ACE Detection (Seegene Inc., Seoul, South Korea); the Magicplex RV Panel Real-time Test (Seegene Inc); and the ResPlex II Panel v2.0 (Qiagen, Hilden, Germany). Precisely evaluating such multiplex kits is difficult in the absence of undisputable golden standard reference techniques for a large number of the pathogens tested. A number of studies have compared the detection of the molecular multiplex platforms with in-house conventional molecular methods [11,12,13,14] but since the reference methods vary from study to study [15], estimating the actual performance of the assays remains difficult. More recently, direct comparison of commercialised multiplex kits was undertaken, with reference to in-house multiplex real-time PCR systems [16], or to commercial duplex PCR tests [17].
Here, as a follow-up to these pioneer studies, we compared the performances of four commercial multiplex kits used in French University Hospital diagnostic microbiology laboratories for the detection of ARI pathogens, i.e., the xTAG Respiratory Viral Panel Fast, RespiFinder SMART 22, CLART PneumoVir and Fast Track Diagnostics Respiratory Pathogen 33 kits. Our analysis mainly focussed on viral pathogens that represent the common target of all kits, the number of bacteria potentially detected varying from 4 (RespiFinder SMART 22) to 12 (Fast Track Diagnostics Respiratory Pathogen 33), making any sound comparison impossible. We combined normalised extraction of nucleic acids, "reference" monoplex real-time PCR detection and a consensus interpretation algorithm to estimate the intrinsic capability of each kit to detection of the various pathogens tested.
Materials and Methods
Nasopharyngeal swabs
Out of the 1,465 patients enrolled in the CoPanFlu-France cohort study [18], 166 presented with ARI during the 2010 and 2011 influenza epidemic seasons. Nasopharyngeal samples were available for all these patients and were selected for this study.
The patients' age ranged from 1 month to 79 years (median: 31.2yo).
Broad range multiplex Kits
Four commercial kits for Multiplex detection of respiratory viruses were tested:
- -. The CLART PneumoVir (PneumoVir) assay is based on PCR-DNA micro-array detection. This assay allows the detection of 17 respiratory viruses.
- -. The Fast Track Diagnostics Respiratory Pathogen 33 (FTD-RP33) version 3 assay is based on multiplex one-step reverse transcription polymerase chain reactions (RT-PCR) with probes for detecting 33 respiratory pathogens (21 viruses and 12 bacteria). It was used with the Bio-Rad CFX96 thermocycler and The SuperScript III Platinum One-Step Quantitative RT-PCR System without ROX (Invitrogen).
- -. The xTAG Respiratory Viral Panel Fast (xTagRVP-F) is based on a multiplex RT-PCR reaction in which the target-specific primers are chimeric, including a terminal Universal Tag sequence. This allows the detection of 19 different respiratory viruses [4,19] by sorting on a Luminex xMAP instrument [20].
- -. The RespiFinder SMART 22 (RespiFinder) assay is based on the multiplex ligation-dependent probe amplification (MLPA) technology, preceded by a pre-amplification step [20]. Results are analysed by capillary electrophoresis [9]. This assay differentiates 18 respiratory viruses and 4 bacteria in one reaction by melt curve analysis.
The different protocols were performed according to manufacturers’ instructions but the extraction of nucleic acids was standardised to use a similar molecular substrate in each assay. One extraction dedicated to each test (each multiplex assay and for the “confirmation tests”) was performed from one of the identical 100μL aliquots prepared from nasopharyngeal swabs. The internal controls of each test were included in the dedicated extraction, and their final concentration adapted to manufacturers' recommendations. Nucleic acids were extracted using the EZ1 Virus Mini Kit v2 on the EZ1 advanced XL Biorobot workstation (Qiagen). Elution was performed in a 90μL volume and nucleic acids were subsequently stored at -80°C before analysis.
The combination of all systems potentially allowed the detection of 41 respiratory pathogens: Influenzavirus A (Inf A) (H3N2, H1N1, H5N1), Influenzavirus B (Inf B), Influenzavirus C (Inf C), Parainfluenzaviruses (PIV) (1, 2, 3 and 4a/b), human Respiratory Syncytial Virus (hRSV) (A and B), Enteroviruses (EV), Rhinoviruses (hRV), Adenoviruses (AdV), Parechoviruses, human Bocavirus (hBoV), human Coronaviruses (hCoV) (229E, NL63, OC43, HKU1), human Metapneumovirus (hMPV) (A and B), Cytomegalovirus (CMV), Pneumocystis jirovecii (PCP), Mycoplasma pneumoniae (Mpneu), Chlamydia pneumoniae (Cpneu), Streptococcus pneumoniae (Spneu), Haemophilus influenzae species (Haeinf), Haemophilus influenzae type B (HIB), Staphylococcus aureus (Saur), Moraxella catarrhalis (Morax), Bordetella pertusis (Bord), Klebsiella pneumoniae (Kpneu), Legionella species (Legio) and Salmonella species (Salm).
The spectrum of detection of the different kits is reported in Table 1. Tests allowing the detection of respiratory pathogens were classified in two different classes: the "Core tests" and the "Extra tests", the "Core tests" being defined as those corresponding to pathogens detected by all four kits.
Processing of samples and interpretation of results
Firstly, all samples were tested with the four broad range multiplex kits (Fig 1). In addition, they were systematically tested for Inf A and Inf B using in-house standard real-time RT-PCR assays [21], and for Core tests (Inf A, Inf B, hRSV, hMPV, EV + hRV, AdV and hBoV) using the Argene (Respiratory Multi Well System r-gene) low range kit, consisting of 4 Taqman-based duplex PCR or RT-PCR reactions (Inf A+B, hRSV+hMPV, hRV&EV+cellular control, AdV+hBoV).
(Inf A: Influenza A, Inf B: Influenza B, hRSV: human Respiratory Syncytial Virus, hMPV: human Metapneumovirus, EV + hRV:; Enterovirus + human Rhinovirus, AdV: Adenovirus and hBoV: human Bocavirus).
Samples testing negative for a given pathogen with all detection methods were proposed negative for this pathogen. Samples testing positive for a pathogen with at least 2 different techniques were proposed positive. Samples with discrepant results (positive for one pathogen with only one detection method) were further processed with "confirmation" assays, consisting of one or several real-time or nested amplification protocols (Table A in S1 File) and subsequently declared positive or negative according to the result obtained.
Positive predictive value (PPV), negative predictive value (NPV), Sensitivity, Specificity and agreement (as evaluated by the Youden’s index) values were calculated for the Core tests of the kits tested (see below).
Secondly, all samples were analysed according to the results obtained by the four broad range multiplex kits for Extra tests (S1 Fig): Influenza A H1N1pdm 2009; "seasonal" Influenza A H1N1; Flu A H5N1; Flu A H3N2; hRSV A/B, hMPV A/B, PIV 1/2/3/4a/4b, PCP and bacteria (Mpneu, Cpneu, Spneu, Haeinf, HIB, Saur, Morax, Bord, Kpneu, Legio and Salm).
Only results relating to viral pathogens were analysed in depth since a majority of bacterial pathogens were detected only by the FTD-RP33 test.
A decision tree similar to that used for Core tests was designed and confirmation techniques were used to confirm or invalidate discrepant results (Table B in S1 File). PPV, Sensitivity, NPV, and Specificity values were calculated for Extra tests (see below).
Performances of the kits
To compare the performances of the kits, Sensitivity, Specificity and Positive and Negative Predictive values were calculated. In accordance with the interpretation rules decreed in the previous section, calculations relied on the following definitions: (i) True Positives were Positive results confirmed by another kit or a confirmation test; (ii) False Positives were Positive results that were not confirmed by another kit or a confirmation test; (iii) False Negatives were Negative results invalidated by at least 2 other kits or 1 kit and 1 confirmation test; (iv) True Negatives were Negative results that were not invalidated by at least 2 other kits or 1 kit and 1 confirmation test.
Virus positive controls and limits of detection
Culture supernatants were obtained from the European virus archive collection (EVA, www.europeanvirusarchive.com) for Influenza A H1N1 (A/Marseille/9410380/2009), Influenza A H3N2 (A/Marseille/4781598/2009), Influenza B (B/Marseille/4461097/2008), Influenza C (C/Leningrad:232/83), Respiratory Syncytial virus type A (Long) and B (Gb2), Rhinovirus (MAR2007 3813047), Enterovirus (Human Coxsackievirus B3 clinical strain B32679), human Parechovirus type 3 (UNK), Adenovirus type B (MAR2007 7416184 clinical strain). In the case of the human Metapneumovirus type A, a sample testing positive using both molecular detection and direct immunofluorescence was used as a positive control.
Nucleic acids were extracted as described above but using an initial sample volume of 400μL and an elution volume of 120μL. Tenfold serial dilutions of nucleic acids were performed and tested using the relevant in-house "confirmation" real time PCR or RT-PCR assays (S1 File). The last three dilutions allowing unambiguous detection (i.e., the lowest dilution had a Ct at 33 and the highest dilution had a Ct between 37 and 40) were used to create a panel, which was subsequently used to evaluate the limit of detection of the 4 broad range multiplex kits.
Ethics statement
The CoPanFlu-France cohort protocol was approved by the French ethics committee (“Comité de Protection des Personnes Ile-de-France I”) on 8 September 2009. All study participants provided written informed consent [18].
Results
The analysis of 166 Nasopharyngeal samples using a combination of four commercial broad range multiplex kits and of confirmation tests provided confirmed aetiological viral diagnosis for 87 samples (52%), including 8 samples (4.8%) with confirmed dual infection (see details in Table 2).
The distribution of confirmed positive detections provided the following results: Enterovirus/Rhinovirus: 34 (39.1%); Influenza A H1N1 2009: 17 (19.5%); Coronavirus: 13 (14.9%); Influenza B: 9 (10%); Metapneumovirus: 7 (8.1%); Respiratory Syncytial Virus: 6 (6.9%); Bocavirus: 5 (5.7%); Parainfluenzaviruses: 3 (3.5%); Adenovirus: 1 (1.1%).
Performances of the four commercial broad range multiplex kits
The performances of the kits for the detection of specific pathogens are presented in Table 3. Regarding the detection of Influenza A viruse(s), the sensitivity and PP values ranged from 88.2% and 79% respectively for the PneumoVir kit to 100% for the FTD-RP33 kit. Regarding Influenza B, the best sensitivity was observed for the RespiFinder kit (100%). It was under 80% for the PneumoVir and FTD-RP33 kits and under 50% for the xTagRVP-F kit. PP values were at 100% for all kits except the PneumoVir kit (87.5%).
Globally, the performances of the PneumoVir kit were hindered by the existence of false positive results for a proportion of the pathogens tested, but also of false negative results for the most common pathogens and in particular human Rhinoviruses/Enteroviruses. The performances of the xTagRVP-F kit were hampered by its low sensitivity for the detection of Influenza B, human bocaviruses and human coronaviruses, but the kit was associated with high PP values (very few false positives). This profile is somewhat similar to that of the FTD-RP33 kit that exhibited high PP values and perfectible sensitivity for the detection of Influenza B, Parainfluenzaviruses, human Respiratory Syncytial virus and human Rhinoviruses/ Enteroviruses. The RespiFinder Kit displayed the highest overall sensitivity values, associated with high PP values. Its lowest performances were for human Metapneumoviruses (sensitivity: 71.4%) and Parainfluenzaviruses (66.6%, similar with other kits).
Regarding the global performances of the kits, the Positive Predictive values were >90% for three of the four kits (xTagRVP-F, FTD-RP33 and RespiFinder) (Table 4). The highest overall specificity was observed for the FTD-RP33 kit (94.4%), with a 100% score for five common pathogens (Influenzaviruses A and B, Parainfluenzaviruses, human Respiratory Syncytial viruses, human Rhinoviruses/Enteroviruses). The most important differences between the kits tested were identified for the overall sensitivity (ranging from 63.9% for the PneumoVir Kit to 88.4% for the RespiFinder Kit) and for the Negative Predictive Value (ranging from 70% for the PneumoVir Kit to 85.3% for the RespiFinder Kit).
The Youden’s index, which summarise the performance of a diagnostic test, was globally strong (> 0.60) except for the PneumoVir kit (0.48).
Limits of detection of the four commercial broad range multiplex kits
Results are presented in Fig 2 and show that, at the highest dilution proposed, the RespiFinder kit detected 8 pathogens (out of the 9 tested), whilst the PneumoVir kits detected 6 (out of 10 tested), the xTagRVP-F kits detected 5 (out of 9 tested) and the FTD-RP33 detected 4 (out of 11 tested).
Inf A H1N1: Influenza A H1N1 (A/Marseille/9410380/2009), Inf A H3N2: Influenza A H3N2 (A/Marseille/4781598/2009), Inf B: Influenza B (B/Marseille/4461097/2008), Inf C: Influenza C (C/Leningrad:232/83), hRSV A and B: human Respiratory Syncytial virus type A (Long) and B (Gb2), hRV: Rhinovirus (MAR2007 3813047), EV Cox B3: Enterovirus (Human Coxsackievirus B3 clinical strain B32679), hPeV: human Parechovirus type 3 (UNK), AdV B: Adenovirus type B (MAR2007 7416184 clinical strain), and hMPV A: human Metapneumovirus type A.
Discussion
We have compared 4 commercialised multiplex PCR techniques for the detection of respiratory pathogens using standardised nucleic acid extracts from French patients presenting with ARIs in 2010 and 2011. Our study mainly focussed on viral pathogens.
There is previous significant information relating to the comparison of the performances of the kits studied with conventional diagnostic technique. This includes: (i) direct immunofluorescence, viral culture, and home-brew PCR in the case of the CLART PneumoVir assay [22]; (ii) home-brew PCR, direct immunofluorescence, and viral culture in the case of the xTAG Respiratory Viral Panel Fast assay [7,19,23–25]; (iii) home-brew PCR, and in-house multiplex real-time PCR in the case of the Fast Track Diagnostics Respiratory Pathogen assay [13,16]; (iv) home-brew PCR and viral culture in the case of the RespiFinder SMART 22 assay [26,27]. In addition, previous studies included comparison with other multiplex commercial kits: xTAG Respiratory Viral Panel Fast assay versus FilmArray respiratory viral panel, ResPlex II Panel v2.0, MultiCode-PLx, Seeplex RV15 OneStep ACE Detection and Pneumobacter ACE Detection and xTAG Respiratory Viral Panel v1 kits [7,23,25]; RespiFinder SMART 22 assay versus Seeplex RV15 OneStep ACE Detection and Pneumobacter ACE Detection, ResPlex II Panel v2.0, RespiFinder 19 kits [2,9,26,27]; Fast Track Diagnostics Respiratory Pathogen 21 versus xTAG Respiratory Viral Panel fast and Easyplex respiratory pathogen 12 kits [16]. The most complete study compared six commercialised techniques, i.e., the RespiFinder SMART 22, Seeplex RV15 OneStep ACE Detection and Pneumobacter ACE Detection, Magicplex RV Panel Real-time Test, CLART PneumoVir, xTAG Respiratory Viral Panel fast and ResPlex II Panel v2.0 kits [17].
Globally, the literature indicates that multiplex molecular techniques compare favourably with traditional diagnostic technique. They can be (at least partially) automated, are highly reproducible and are generally more sensitive.
However, a first difficult issue for comparing respiratory pathogens multiplex techniques is the absence of gold standard for the detection of respiratory pathogens. If we refer only to the ability of a technique to detect a pathogen or some of its molecular components in biological samples, some newly developed molecular techniques may be more sensitive than the reference techniques, (i.e., some "false positives", referring to the reference technique, may be true positives that have not been detected by this reference method). This situation may be met when the new technique is able to detect viral variants that could not be detected by previous methods and, in that case, corresponds to an actual improvement of the technique if the variants detected are true pathogens that can be implicated in ARIs. However, it can also be met when the new technique is able to detect a lower viral load. The actual clinical impact of an improved detection of low viral loads in respiratory samples has not been accurately evaluated and its medical significance at the acute phase of ARIs obviously requires to be further documented. Altogether, this indicates that the interpretation of evaluation scores should remain careful and open-minded. Accordingly, we have tried to build a comprehensive approach of the comparison of diagnostic kits that mixed reference to well established–and clinically evaluated- real-time PCR detection techniques (this included in the current study the Argene Respiratory Multi Well System r-gene low range kit for Core tests, which has been used for years for routine detection in medical virology laboratories; and a series of published home-brewed techniques) and analysis of convergent positive results. For the latter, we considered that, given (i) the intrinsic high specificity of molecular amplification techniques, and (ii) the fact that the techniques compared used different technologies and amplification primers, the probability of obtaining false positive results with two different techniques for the same sample was negligible for an evaluation purpose. Consequently, samples in which a given pathogen was detected independently by two different techniques were declared positive for this pathogen, otherwise it was declared negative. We cannot exclude that a few samples were wrongly assigned a negative result (if a specific technique performed better than all others for a given pathogen), but the high degree of convergence between the different techniques suggests that the number of "false negative" results that could have been potentially generated is limited.
A second difficult issue for comparing respiratory pathogens multiplex techniques is the choice of the panel of samples tested. The nature and frequency of pathogens present in the samples tested are highly dependent on the epidemiological circumstances and on the populations tested, and have an obvious influence on the results of the analysis, e.g., the low number or absence of positive samples for some viruses represents in most of the studies an obvious obstacle for accurate comparisons. In the current study, we tested patients with ARIs from all age groups recruited in a household cohort that aimed at providing a picture of the French general population. This means that, in comparison with previous hospital-based studies, the paediatric population was under-represented and the clinical presentations were milder. In addition, the specific period studied (2010–2011) was associated with the predominant circulation of the pandemic H1N1 variant of the influenza A virus. Accordingly, the low number or absence of parainfluenza virus, human coronavirus type 229E, H3N2 influenza A virus, influenza C virus, human adenovirus and human parechovirus infections did not allow a significant comparison of kits for these pathogens. This is important to notice since in previous studies the low number of positive samples for some pathogens may have concealed the poor performances of some kits for detecting these pathogens (e.g., the case of Adenoviruses for the xTAG Respiratory Viral Panel fast assay [9,17,25]).
A third difficult issue for comparing respiratory pathogens multiplex techniques relies in the different spectra of detection and precision of identification offered by the various commercialised kits. To take this aspect into account, we have identified "Core tests", i.e., a series of common pathogens detected by all kits (Inf A, Inf B, hRSV, hMPV, EV + hRV, AdV and hBoV). We have also limited our analysis to viral pathogens since bacterium detection was absent or very limited in 3 of the 4 kits tested. The simultaneous detection of bacteria and viruses represents a theoretical great advantage in terms of clinical management (e.g., for determining the appropriateness of starting an antibiotic treatment). However, it is noticeable that, in the current study, bacteria detection provided a large number of positive results, potentially shifting the rate of aetiological elucidation from 52% to >77% in the panel studied. Obviously, this deserves further in depth investigations, which should include samples from both ARIs and paired asymptomatic individuals. Determining the actual imputability of bacterial detection in the occurrence of ARIs may be difficult to achieve from qualitative results and clinically relevant interpretation algorithms remain to be established. As noted above, the detection of low pathogen loads may be an issue for viruses, but the case of bacteria is specifically difficult due to the frequent asymptomatic carriage of potentially pathogenic bacteria.
For Core tests, we identified an important degree of correlation between the performance of the xTAG Respiratory Viral Panel fast, the Fast Track Diagnostics Respiratory Pathogen 33 and the RespiFinder SMART 22 assays (Youden’s index: 0.67; 0.73 and 0.83, respectively) (Table 5). The CLART PneumoVir assay exhibited the most divergent performance (global Core tests Youden’s index value: 0.51; global Core tests sensitivity and specificity values: 63.3% and 87.4%, respectively), despite previous favourable performance by comparison with home-brew PCR techniques [22]. When examined in more details (Table 3), the sensitivity values for Core tests compared with those of the other kits, except for human Rhinoviruses/Enteroviruses (sensitivity: 41.2%) and human Bocaviruses (40%). There was also a trend for the existence of false positive results for Influenza A and Metapneumovirus tests, which altered the global positive predictive value of the kit (81.9% vs >90% for all other kits). False positive results may hypothetically be related to the fact that the technique includes an open-tube post amplification step, potentially associated with an increased risk of PCR contamination. In serial dilution tests, 4 (Inf A H3N2, Inf B, Inf C and hMPV A) out of the 10 pathogens detected in the most concentrated dilution tested negative at dilutions 10−1 and 10−2 (Fig 2).
Our study globally confirms the previously observed good performances of the xTAG Respiratory Viral Panel Fast [2,9,7,15,17,23–25,28]. For Core tests, the global sensitivity and specificity values were 73.4% and 93.1%, respectively (Table 5). The poorest results were observed for influenza A positive predictive value (78.9%) and influenza B and human bocavirus sensitivity values (44.4% and 20%, respectively). In serial dilution tests, out of the 9 pathogens detected in the most concentrated dilution, one (hMPV A) tested negative at dilution 10−1 and 4 (Inf B, hRSV A, hRSV B and hMPV A) at dilution 10−2.
The Fast Track Diagnostics Respiratory pathogens 33 kit performed well for Core tests (global sensitivity and specificity values were 75.9% and 96.5%) (Table 5). The excellent specificity values (the highest in the panel tested) may be related in part to the simple and safe real-time PCR format of the technique, which does not require any post-PCR manipulation. Amongst Core tests, and in agreement with previous findings [13], the lowest sensitivity values were observed for human Respiratory Syncytial Virus and Enterovirus/Rhinovirus (50% and 61.8%, respectively). In serial dilution tests, out of the 11 pathogens detected in the most concentrated dilution, 3 (Inf B and C, hRSV B) tested negative at dilution 10−1 and 7 (Inf B and C, hRSV B, EV Cox B3, hPeV, hAdV B and hMPV A) at dilution 10−2, in coherence with previous observations [13].
The RespiFinder SMART 22 assay, provided for Core tests both the highest Youden’s index performance values (global Core tests Youden’s index value: 0.83) and the best global sensitivity (87.3%). It also was associated with an excellent specificity (95.4%), in agreement with previous evaluations [2,9,26,27]. Amongst Core tests, the lowest sensitivity value was observed for human Metapneumovirus (71.4%). The kit also provided the best results in serial dilution tests: of the 9 pathogens detected in the most concentrated dilution, all tested positive at dilution 10−1 and 1 (hMPV A) tested negative at dilution 10−2.
Conclusion
The present study identified a high degree of correlation for at least three of the techniques and completed an overview of the multiplex techniques available for the diagnosis of acute respiratory infections. All of the kits evaluated performed properly for Core tests and Youden’s index, positive predictive and specificity values were systematically higher for Core tests than for Extra tests. Significant differences in the ability of the kits to detect low viral loads were identified, but the medical implications of this observation remain to be evaluated. The range of detected pathogens, the technology used for PCR product revelation, the laboratory organization and other issues, such as sample through-put, staffing levels and staff expertise, clinicians’ expectations and overall funding structures appear as determinant features for the selection of a kit.
Supporting Information
S1 File. PCR confirmation assays for Core tests (Table A) and extra-tests (Table B).
https://doi.org/10.1371/journal.pone.0130378.s001
(DOCX)
S1 Table. Bacterial etiologies identified, co-infection with viruses and viral results.
https://doi.org/10.1371/journal.pone.0130378.s002
(DOCX)
S1 Fig. Decision tree (algorithm used for extra-tests).
https://doi.org/10.1371/journal.pone.0130378.s003
(TIFF)
Acknowledgments
The authors are grateful to Drs. Nathanael Lapidus and Rosemary Delabre (UMR-S 707 INSERM, Paris, France) for management of the CoPanFlu programme.
Author Contributions
Conceived and designed the experiments: XdL FC. Performed the experiments: NS FR AV MLV. Analyzed the data: NS XdL. Contributed reagents/materials/analysis tools: AV MLV LA. Wrote the paper: NS XdL.
References
- 1. Verani JR, McCracken J, Arvelo W, Estevez A, Lopez MR, Reyes L, et al. Surveillance for hospitalized acute respiratory infection in Guatemala. PLoS One. 2013;8(12):e83600. Epub 2014/01/07. PONE-D-13-24234 [pii]. pmid:24391792 PubMed Central PMCID: PMC3877070.
- 2. Raymaekers M, de Rijke B, Pauli I, Van den Abeele AM, Cartuyvels R. Timely diagnosis of respiratory tract infections: evaluation of the performance of the Respifinder assay compared to the xTAG respiratory viral panel assay. J Clin Virol. 2011;52(4):314–6. Epub 2011/09/20. doi: S1386-6532(11)00344-1 [pii] pmid:21925937.
- 3. Gonzales R, Malone DC, Maselli JH, Sande MA. Excessive antibiotic use for acute respiratory infections in the United States. Clin Infect Dis. 2001;33(6):757–62. Epub 2001/08/21. CID001457 [pii] pmid:11512079.
- 4. Fox JD. Respiratory virus surveillance and outbreak investigation. J Clin Virol. 2007;40 Suppl 1:S24–30. Epub 2008/01/31. doi: S1386-6532(07)70006-9 [pii] pmid:18162251.
- 5. Loeffelholz MJ, Pong DL, Pyles RB, Xiong Y, Miller AL, Bufton KK, et al. Comparison of the FilmArray Respiratory Panel and Prodesse real-time PCR assays for detection of respiratory pathogens. J Clin Microbiol. 2011;49(12):4083–8. Epub 2011/10/15. doi: JCM.05010-11 [pii] pmid:21998418; PubMed Central PMCID: PMC3232999.
- 6. Bulla A, Hitze KL. Acute respiratory infections: a review. Bull World Health Organ. 1978;56(3):481–98. Epub 1978/01/01. pmid:308414; PubMed Central PMCID: PMC2395579.
- 7. Babady NE, Mead P, Stiles J, Brennan C, Li H, Shuptar S, et al. Comparison of the Luminex xTAG RVP Fast assay and the Idaho Technology FilmArray RP assay for detection of respiratory viruses in pediatric patients at a cancer hospital. J Clin Microbiol. 2012;50(7):2282–8. Epub 2012/04/21. doi: JCM.06186-11 [pii] pmid:22518855; PubMed Central PMCID: PMC3405573.
- 8. Welch DF, Ginocchio CC. Role of rapid immunochromatographic antigen testing in diagnosis of influenza A virus 2009 H1N1 infection. J Clin Microbiol. 2010;48(1):22–5. Epub 2009/12/17. pmid:20007399; PubMed Central PMCID: PMC2812308.
- 9. Dabisch-Ruthe M, Vollmer T, Adams O, Knabbe C, Dreier J. Comparison of three multiplex PCR assays for the detection of respiratory viral infections: evaluation of xTAG respiratory virus panel fast assay, RespiFinder 19 assay and RespiFinder SMART 22 assay. BMC Infect Dis. 2012;12:163. Epub 2012/07/26. doi: 1471-2334-12-163 [pii] pmid:22828244; PubMed Central PMCID: PMC3519643.
- 10. Perez-Ruiz M, Pedrosa-Corral I, Sanbonmatsu-Gamez S, Navarro-Mari M. Laboratory detection of respiratory viruses by automated techniques. Open Virol J. 2012;6:151–9. Epub 2012/12/19. TOVJ-6-151 [pii]. pmid:23248735; PubMed Central PMCID: PMC3522051.
- 11. Renaud C, Crowley J, Jerome KR, Kuypers J. Comparison of FilmArray Respiratory Panel and laboratory-developed real-time reverse transcription-polymerase chain reaction assays for respiratory virus detection. Diagn Microbiol Infect Dis. 2012;74(4):379–83. Epub 2012/09/18. doi: S0732-8893(12)00332-X [pii] pmid:22981482.
- 12. Fox JD. Nucleic acid amplification tests for detection of respiratory viruses. J Clin Virol. 2007;40 Suppl 1:S15–23. Epub 2008/01/31. doi: S1386-6532(07)70005-7 [pii] pmid:18162249.
- 13. Sakthivel SK, Whitaker B, Lu X, Oliveira DB, Stockman LJ, Kamili S, et al. Comparison of fast-track diagnostics respiratory pathogens multiplex real-time RT-PCR assay with in-house singleplex assays for comprehensive detection of human respiratory viruses. J Virol Methods. 2012;185(2):259–66. Epub 2012/07/17. doi: S0166-0934(12)00250-9 [pii] pmid:22796035.
- 14. Gadsby NJ, Hardie A, Claas EC, Templeton KE. Comparison of the Luminex Respiratory Virus Panel fast assay with in-house real-time PCR for respiratory viral infection diagnosis. J Clin Microbiol. 2010;48(6):2213–6. Epub 2010/04/02. doi: JCM.02446-09 [pii] pmid:20357215; PubMed Central PMCID: PMC2884497.
- 15. Popowitch EB, O'Neill SS, Miller MB. Comparison of the Biofire FilmArray RP, Genmark eSensor RVP, Luminex xTAG RVPv1, and Luminex xTAG RVP fast multiplex assays for detection of respiratory viruses. J Clin Microbiol. 2013;51(5):1528–33. Epub 2013/03/15. doi: JCM.03368-12 [pii] pmid:23486707; PubMed Central PMCID: PMC3647947.
- 16. Anderson TP, Werno AM, Barratt K, Mahagamasekera P, Murdoch DR, Jennings LC. Comparison of four multiplex PCR assays for the detection of viral pathogens in respiratory specimens. J Virol Methods. 2013;191(2):118–21. Epub 2013/04/16. doi: S0166-0934(13)00109-2 [pii] pmid:23583489.
- 17. Pillet S, Lardeux M, Dina J, Grattard F, Verhoeven P, Le Goff J, et al. Comparative evaluation of six commercialized multiplex PCR kits for the diagnosis of respiratory infections. PLoS One. 2013;8(8):e72174. Epub 2013/09/24. pmid:24058410; PubMed Central PMCID: PMC3751960.
- 18. Lapidus N, de Lamballerie X, Salez N, Setbon M, Ferrari P, Delabre RM, et al. Integrative study of pandemic A/H1N1 influenza infections: design and methods of the CoPanFlu-France cohort. BMC Public Health. 2012;12:417. Epub 2012/06/09. pmid:22676272; PubMed Central PMCID: PMC3461458.
- 19. Pabbaraju K, Wong S, Tokaryk KL, Fonseca K, Drews SJ. Comparison of the Luminex xTAG respiratory viral panel with xTAG respiratory viral panel fast for diagnosis of respiratory virus infections. J Clin Microbiol. 2011;49(5):1738–44. Epub 2011/03/18. doi: JCM.02090-10 [pii] pmid:21411570; PubMed Central PMCID: PMC3122679.
- 20. Merante F, Yaghoubian S, Janeczko R. Principles of the xTAG respiratory viral panel assay (RVP Assay). J Clin Virol. 2007;40 Suppl 1:S31–5. Epub 2008/01/31. doi: S1386-6532(07)70007-0 [pii] pmid:18162252.
- 21. van Elden LJ, Nijhuis M, Schipper P, Schuurman R, van Loon AM. Simultaneous detection of influenza viruses A and B using real-time quantitative PCR. J Clin Microbiol. 2001;39(1):196–200. Epub 2001/01/04. pmid:11136770; PubMed Central PMCID: PMC87701.
- 22. Huguenin A, Moutte L, Renois F, Leveque N, Talmud D, Abely M, et al. Broad respiratory virus detection in infants hospitalized for bronchiolitis by use of a multiplex RT-PCR DNA microarray system. J Med Virol. 2012;84(6):979–85. Epub 2012/04/14. pmid:22499022.
- 23. Jokela P, Piiparinen H, Mannonen L, Auvinen E, Lappalainen M. Performance of the Luminex xTAG Respiratory Viral Panel Fast in a clinical laboratory setting. J Virol Methods. 2012;182(1–2):82–6. Epub 2012/04/03. doi: S0166-0934(12)00083-3 [pii] pmid:22465255.
- 24. Balada-Llasat JM, LaRue H, Kelly C, Rigali L, Pancholi P. Evaluation of commercial ResPlex II v2.0, MultiCode-PLx, and xTAG respiratory viral panels for the diagnosis of respiratory viral infections in adults. J Clin Virol. 2011;50(1):42–5. Epub 2010/11/06. doi: S1386-6532(10)00406-3 [pii] pmid:21050809.
- 25. Gharabaghi F, Hawan A, Drews SJ, Richardson SE. Evaluation of multiple commercial molecular and conventional diagnostic assays for the detection of respiratory viruses in children. Clin Microbiol Infect. 2011;17(12):1900–6. Epub 2011/06/29. pmid:21707834.
- 26. Bruijnesteijn van Coppenraet LE, Swanink CM, van Zwet AA, Nijhuis RH, Schirm J, Wallinga JA, et al. Comparison of two commercial molecular assays for simultaneous detection of respiratory viruses in clinical samples using two automatic electrophoresis detection systems. J Virol Methods. 2010;169(1):188–92. Epub 2010/08/10. doi: S0166-0934(10)00274-0 [pii] pmid:20691735.
- 27. Loens K, van Loon AM, Coenjaerts F, van Aarle Y, Goossens H, Wallace P, et al. Performance of different mono- and multiplex nucleic acid amplification tests on a multipathogen external quality assessment panel. J Clin Microbiol. 2012;50(3):977–87. Epub 2011/12/16. doi: JCM.00200-11 [pii] pmid:22170925; PubMed Central PMCID: PMC3295136.
- 28. Rand KH, Rampersaud H, Houck HJ. Comparison of two multiplex methods for detection of respiratory viruses: FilmArray RP and xTAG RVP. J Clin Microbiol. 2011;49(7):2449–53. Epub 2011/04/22. doi: JCM.02582-10 [pii] pmid:21508156; PubMed Central PMCID: PMC3147849.