Multiplex PCR System for Rapid Detection of Pathogens in Patients with Presumed Sepsis – A Systemic Review and Meta-Analysis

Background Blood culture is viewed as the golden standard for the diagnosis of sepsis but suffers from low sensitivity and long turnaround time. LightCycler SeptiFast (LC-SF) is a real-time multiplex polymerase chain reaction test able to detect 25 common pathogens responsible for bloodstream infections within hours. We aim to assess the accuracy of LC-SF by systematically reviewing the published studies. Method Related literature on Medline, Embase, and Cochrane databases was searched up to October 2012 for studies utilizing LC-SF to diagnose suspected sepsis and that provided sufficient data to construct two-by-two tables. Results A total of 34 studies enrolling 6012 patients of suspected sepsis were included. The overall sensitivity and specificity for LC-SF to detect bacteremia or fungemia was 0·75 (95% CI: 0·65–0·83) and 0·92 (95%CI:0·90–0·95), respectively. LC-SF had a high positive likelihood ratio (10·10) and a moderate negative likelihood ratio (0·27). Specifically, LC-SF had a sensitivity of 0·80 (95%CI: 0·70–0·88) and a specificity of 0·95(95%CI: 0·93–0·97) for the bacteremia outcome, and a sensitivity of 0·61 (95%CI: 0·48–0·72) and a specificity of 0·99 (95%CI: 0·99–0·99) for the fungemia outcome. High heterogeneity was found in the bacteremia outcome subgroup but not in the fungemia outcome subgroup. Conclusion LC-SF is of high rule-in value for early detection of septic patients. In a population with low pretest probability, LC-SF test can still provide valuable information for ruling out bacteremia or fungemia.


Introduction
The burden of sepsis is increasing globally. A survey conducted in USA in 2000 revealed that there were more than 650 thousand of cases of sepsis annually, with an average mortality rate of 18% [1]. Another U.S. report showed that the incidence of hospitalized patients with septicemia or sepsis had increased more than two folds in the last decade [2].
Aside from early optimization of hemodynamics [3,4], timely adequate empirical antibiotics are a cornerstone of the sepsis treatment [3,5]. Empirical therapy is then adjusted by the blood culture results, which provide information on causative microorganisms and in vitro sensitivity of antibiotics. Although blood culture has long been viewed as the gold standard test for the diagnosis of sepsis, it suffers from low sensitivity, prolonged turnaround time (.48 hours), and liability for contamination [6]. Efforts have been made to improve timeliness and accuracy of sepsis diagnosis. Recent advances include the development of novel clinical biomarkers [7,8], refined clinical criteria [9], intricate algorithms [10], and molecular diagnostic methods [11].
The LightCycler SeptiFast Test (Roche Diagnostics, Mannheim, Germany) is a commercial diagnostic test utilizing real-time multiplex polymerase chain reaction (PCR). The diagnostic probes for PCR target the internal transcribed sequences situated between 16S and 23S bacterial ribosomal RNA as well as between 18S and 5?6S fungal ribosomal RNA [12][13][14]. Once the DNA of the pathogen is extracted from the blood and amplified by the LightCycler machine, a positive detection is recorded if the fluorescent signal emitted by internal hybridization probes reaches the threshold. Subsequently, a melting curve analysis is proceeded to identify the species. Overall, LightCycler SeptiFast Test is designed to detect 25 common pathogens ( Table 1). The analytical sensitivity reported by the manufacturer is 100 CFU/mL for Candida glabrata, Streptococcus spp., and coagulase-negative Staphylococcus spp., and 30 CFU/mL for the others. With its broad range of detection, short turnaround time, and manufacturer-reported high sensitivity and specificity, such a molecular method might be a promising alternative to blood culture.
Since its debut, LightCycler SeptiFast has been intensively studied. Nevertheless, the results are inconsistent. Taken individually, the sensitivity and specificity are dotted in a wide range, yet potentially worthwhile accuracy and benefits of LightCycler SeptiFast. Therefore, we aim to quantitatively synthesize current literatures by critiquing literatures, extracting data, and pooling with meta-analysis statistical methods to determine the diagnostic implication and significance of this method.

Methods
Our systemic review and meta-analysis conformed to the methods and procedures recommended by Cochrane Collaboration on the meta-analysis of the diagnostic tests and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [15,16].

Search Strategy
We performed a comprehensive search of literatures on the MEDLINE, EMBASE, and Cochrane databases to identify studies related to clinical utilization of LightCycler SeptiFast test for patients with suspected sepsis. We combined several search keywords to be ''(multiplex PCR OR multiplex polymerase chain reaction OR septifast OR sepsitest OR vyoo) AND (sepsis OR bloodstream infection OR bacteremia OR septicemia)'' from inception to June 2011. No language, study type or any other filter was set. We also searched bibliographies of retrieved full-text articles and latest reviews to include more related studies. We also searched bibliographies of retrieved articles and latest review and updated our search to October 2012 before the deploying of statistical analysis.

Study Selection
We systematically included studies using predetermined inclusion criteria, which included: a) evaluation of the LightCycler SeptiFast test on blood specimens for diagnosing sepsis; and b) comparison of the LightCycler SeptiFast test results with reference standards, and c) sufficient information to calculate sensitivity and specificity. We excluded reviews, case reports, comments, and studies using the same dataset. Two authors independently assessed all the titles and abstracts to identify studies matching the inclusion criteria. Discrepancies on inclusion and exclusion were resolved by consensus meeting where additional reviewers were enrolled.

Data Extraction
We piloted a data extraction from a few eligible studies and developed a comprehensive standardized data extraction form for subsequent use. Extracted data included characteristics of study design, characteristics of study patients, diagnostic method, and reference standard. More than one reference standard were used in many studies. We defined those using clinical criteria to diagnose infection as clinically-documented Infection (CDI), those using microbiological data from other specimens with or without blood culture as laboratory-documented infection (LDI), and those using blood culture alone as BC.

Assessment of study quality
We assessed the quality of studies using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) instrument [17].

Data Preparation and Statistical Analysis
We used the bivariate model for diagnostic meta-analysis to obtain weighted overall estimates of the sensitivity and specificity [18]. The bivariate approach models the logit-transformed sensitivity and specificity and adjusts for the negative correlation between the sensitivity and specificity of the index test that may arise from different thresholds used in different studies. A hierarchical summary receiver operating characteristic (HSROC) curve was constructed as a way to summarize the true-and falsepositive rates from different diagnostic studies [19]. The area under the HSROC curve measures the overall accuracy of diagnostic tests. We also performed diagnostic odds ratio (DOR) meta-analysis. The DOR combines both positive and negative likelihood ratios and is a global measure of test performance. We quantify the extent of between-study heterogeneity by calculating the I 2 statistics [20]. To explore the source of heterogeneity, we defined potential relevant covariates a priori and tested these covariates one at a time in the meta-regression model. We used Egger's test for funnel plot asymmetry to test possible publication bias. Statistical analyses were conducted using STATA 11?0 (Stata Corp, College Station, TX). All statistical tests were two-sided, and statistical significance was defined as a P value less than 0?05.

Identification of Studies
Our initial search yielded 248 citations (Appendix S1). After two rounds of inclusion and exclusion, a total of 34 primary studies including 6,012 patients (8,438 episodes) were eligible for analysis, of which 1,920 episodes (22?8%) were confirmed bacterial or fungal infection. Appendix S1 displays the literature selection process.

Quality of the Included Studies
The studies varied in quality. Most of the study populations were representative of the target population. The diagnostic tests were deployed independently of the reference standards. We did not find differential verification of outcomes in the included studies. Because there was no unanimous standard to confirm clinically significant systemic infection, various definitions of reference standards were used and outcome misclassification was likely. Furthermore, few studies clearly mention the blinded interpretations between the LightCycler SeptiFast results and the clinical diagnosis; therefore, incorporation bias is likely. Results of risk of bias evaluation by QUADAS instrument were summarized in Appendix S2.

Study Characteristics and Patient Populations
Details of the individual studies characteristics were summarized in Table 2. Most included studies prospectively enrolled patients with suspected sepsis from intensive care unit (ICU), emergency department (ED), and hematology and oncology unit. Studies by Casalta JP specifically targeted at patients with infectious endocarditis. Most of the included studies study on adult patients, except five studies included both children and adults and two included neonates or children. Eighteen of the 34 included studies reported accuracy data on bacteremia and fungemia separately. Various criteria were used as the reference standards, which can be grouped as three main broad categories. Ten (52?6%) studies used the preferred combined clinical and laboratory criteria. Seven studies (36?8%) chose to stick to the blood culture results. The remaining two (10?5%) used other laboratory specimens along with blood culture as the reference standard.

Diagnostic Accuracy of the LightCycler SeptiFast Test for fungemia
The performance data for the LC-SF test in detecting fungemia were available in 18 studies. Compared with the performance of the LC-SF test in detecting bacteremia, the LC-SF test had a poor sensitivity (0?61; 95% CI: 0?48-0?72) but a nearly perfect specificity (0?99; 95%: 0?99-0?99) when detecting fungemia. Results from the nineteen studies showed a similar trend with a nearly perfect heterogeneity measure (I2 = 0). The pooled LR+ was high (LR+: 66?8, 95% CI: 39?8-112), while the pooled LRwas unacceptably poor (LR-:0?40, 95% CI: 0?29-0?54). The results suggested the LC-SF test was only good for ruling in fungemia. Figure 1 shows the HSROC curves for three different outcomes and figure 2 shows the DOR from all studies for three different outcomes in forest plots.

Publication Bias and meta-regression analysis
We performed meta-regression analysis to explore source of heterogeneity and to help explain the variation after subgroup analysis (Table 4). Meta-regression analysis yielded a relative DOR for each pre-specified covariate in the model. We did not find the effect estimate significantly changed by the reference standard definition, design characteristics, study setting, and

Discussion
Our study was designed to assess the diagnostic accuracy of the LC-SF test for detecting bacterial and fungal infection among patients suspected of infection. Our meta-analysis, which included 34 studies comprising a total of 6,012 patients, provided an overall summary of the diagnostic accuracy of the PCR methods. Overall, SeptiFast had a high specificity with a modest and highly variable sensitivity. For the clinicians, this means the rule-in value is higher than the rule-out value. In the presence of a positive SeptiFast result in a patient with suspected bacterial or fugal sepsis, a clinician can confidently diagnose bacteremia or fungemia and begin appropriate antimicrobial therapy, while forgoing unnecessary additional diagnostic testing. However, a negative SeptiFast result has a reasonable likelihood of being false-negative and should be confirmed by other clinical or laboratory diagnostic tests if the result is likely to affect patient management.
On the basis of our study, the pooled LR+ of the LC-SF test to diagnose bacterial sepsis was 15?9 (95% CI: 10?4-24?3); and the pooled LR-was 0?21 (95% CI: 0?13-0?33), which could translate into a positive post-test probability of 80% and a negative post-test probability of 5% in a virtual population with the prevalence of bacterial sepsis as 0?20 (the actual prevalence of this study was 0?19). As far as fungal sepsis was concerned, the LC-SF test had a LR+ of 66?8 (95% CI: 39?8-112), and a LR-of 0?40 (95% CI: 0?29-0?54), which could derive a positive post-test probability of 66?8% and a negative post-test probability of 1% in a virtual population with the prevalence of fungal sepsis as 0?02 (the actual prevalence of this study was 0?019). These figures help us gain further insight in their use in the clinical practice. Although the value of the LC-SF test in ruling out either systemic bacterial or fungal infection was not as good as that in ruling them in, the low background prevalence of both diseases makes these test still provide valuable rule-out information. A post-test probability as low as 5% for bacterial sepsis may justify withholding antibiotics treatment in selected cases whose LC-SF test is negative and clinical manifestation and other ancillary laboratory tests do not strongly suggest a severe infection. Likewise, although the LR-for the LC-SF to diagnose systemic fungal infection is only 0?44, the extremely low pretest probability of fungemia in most clinical setting allows the negative results of LC-SF test to remain as useful information for clinical decision. The 1% post-test probability in patients with a negative LC-SF test for fungal infection also justifies withholding anti-fungal therapy and searching for other causes of clinical deterioration and repeating the microbiological workup. If a post-test probability of negative LC-SF test of 10% is a clinically tolerable threshold for withholding antimicrobial treatment, the diagnostic value of LC-SF test would lose its reference value once the pretest probability rise to 35% for bacterial infection and 22% for fungal infection.
From the technical viewpoint, the lack of sensitivity in the LC-SF test may be attributable to insufficient concentration of bacteria and limited sets of primers in the diagnostic kit. Although it seems logical to include more primers in a diagnostic kit or to draw more blood from a patient, the blood volume allowed in a PCR machine is limited, and drawing large amount of blood from a patient may not be feasible, especially for pediatric or hematological patients. Therefore, certain modification has been suggested. Pä ivi T et al. [21] raised the number of bacteria or fungi in the blood by culturing the blood specimens 48 hours before deploying hybridization assay. Such a combination method was shown to effectively raise the sensitivity of a multiplex PCR-based diagnostic array to 0?95 (95% CI: 0?94-0?96) and a specificity of 0?99 (95% CI: 0?98-0?99). The cost of this strategy is the delayed turnaround time as an additional 24 to 48 hours are required for the direct LC-SF test. Another new technology that may address this problem may be the broad-range PCR amplification of conserved bacterial DNA sequences, such as the 16S ribosomal RNA (rRNA), 23S rRNA, and 16S-23S rRNA interspace regions. Numerous studies [22] have demonstrated that broad-range PCR of the conserved bacterial DNA sequences generates valuable information that complements results of time-consuming and subjective phenotypic tests for detecting bacterial infections. When real-time PCR and high-resolution melting analysis are adopted, broad-range amplification of bacterial DNA offers additional benefits including minimal labor, rapid turnaround time and a reduced risk of PCR carryover contamination.
There are three previous meta-analyses addressing the accuracy of multiplex PCR-based microbiological diagnostic methods. Carlo Mengoli et al. [23] reviewed literatures studying the diagnostic accuracy of several in-house PCR methods on patients with invasive aspergillosis and reported a pooled sensitivity of 0?88 (95% CI: 0?75-0?94) and a pooled specificity of 0?75 (95% CI: We did not have a sufficient number of pediatric studies to perform subgroup analysis, but excluding several studies with mixed pediatric and adult population showed raised sensitivity in detecting bacteremia. Unless there is a head-to-head parallel comparative study, we cannot conclude whether the accuracy of PCR-based microbiological diagnosis varies among age groups.
Our study has strengths and limitations. This is the first systemic review that focuses on the accuracy of commercial real-time-PCRbased system LC-SF. Previous meta-analysis included studies using various kinds of in-house multiplex PCR kits and the results could not be readily generalized to current practice. Another major strength of our study is that we extracted, analyzed, and reported the accuracy of data on bacterial and fungal infection separately. It turned out the accuracy profile of LC-SF test in bacterial and fungal sepsis detection was drastically different. There are also several limitations in our study. First, currently, there is no evidence that LC-SF improves patient-important outcomes. Second, the higher false-negative rate of the LC-SF test still carries a potential adverse impact on patient safety. It is therefore recommended that these tests should be interpreted in the context of pre-test probability. Third, by pooling studies dealing with a variety of sample types, clinical settings, and study populations, we may have introduced heterogeneity. No major controllable factor was found to explain the heterogeneity. Lastly, at present, there is no formal cost-effectiveness analysis for the LC-SF test. If the use  of LC-SF can lead to reduction of use of broad spectrum antibiotics at the early course of sepsis treatment, the additional cost may prove worthwhile.

Conclusion
Based on the published studies, we conclude that the LC-SF test has higher rule-in than rule-out diagnostic value. In populations in which the prevalence of systemic bacterial or fungal infection is low, the negative LC-SF test still offer useful information for clinical decision. The major limitation of the LC-SF test is its suboptimal sensitivity. Before newer technology is available, we recommend clinicians combine biomarkers, clinical findings, and the LC-SF test to enhance the diagnostic accuracy.

Supporting Information
Appendix S1 The flow chart shows the procedure used by the current systematic review to identify studies using the LightCycler SeptiFast molecular diagnostic method to detect bacterial or fungal infection.