Score performance of SAPS 2 and SAPS 3 in combination with biomarkers IL-6, PCT or CRP

Objective We aimed to evaluate the effects of combining the Simplified-Acute-Physiology-Score (SAPS) 2 or the SAPS 3 with Interleukin-6 (IL-6) or Procalcitonin (PCT) or C-Reactive Protein (CRP) concentrations for predicting in-hospital mortality. Material and methods This retrospective study was conducted in an interdisciplinary 22-bed intensive care unit (ICU) at a German university hospital. Within an 18-month period, SAPS 2 and SAPS 3 were calculated for 514 critically ill patients that were admitted to the internal medicine department. To evaluate discrimination performance, the area under the receiver operating characteristic curves (AUROCs) and the 95% confidence intervals (95% CIs) were calculated for each score, exclusively or in combination with IL-6 or PCT or CRP. DeLong test was used to compare different AUROCs. Results The SAPS 2 exhibited a better discrimination performance than SAPS 3 with AUROCs of 0.81 (95% CI, 0.76–0.86) and 0.72 (95% CI, 0.66–0.78), respectively. Overall, combination of the SAPS 2 with IL-6 showed the best discrimination performance (AUROC 0.82; 95% CI, 0.77–0.87), albeit not significantly different from SAPS2. IL-6 performed better than PCT and CRP with AUROCs of 0.75 (95% CI, 0.69–0.81), 0.72 (95% CI, 0.66–0.77) and 0.65 (95% CI, 0.59–0.72), respectively. Performance of the SAPS 3 improved significantly when combined with IL-6 (AUROC 0.76; 95% CI, 0.69–0.81) or PCT (AUROC 0.73; 95% CI, 0.67–0.78). Conclusions Our analysis provided evidence that the risk stratification performance of the SAPS 3 and, to a lesser degree, also of the SAPS 2 can increase when combined with IL-6. A more accurate detection of aberrant or dysregulated systemic immunological responses (by IL-6) may explain the higher performance achieved by SAPS 3 + IL-6 vs. SAPS 3. Thus, implementation of IL-6 in critical care scores can improve prediction outcomes, especially in patients experiencing acute inflammatory conditions; however, statistical results may vary across hospital types and/or patient populations with different case mix.

reached in cases of severe bacterial, fungal, or parasitic infection. However, high serum levels of PCT are also detectable after trauma and/or following major surgery; overall, viral infection or inflammation of a noninfectious origin have been associated with lower PCT serum levels on average [20,21].

Ethics statement
This non-interventional study protocol was approved by the local institutional review board of the University Hospital Essen (IRB: Ethik-Kommission am Universitätsklinikum Essen). Because of the observational design of this cohort study, the institutional review board waived the requirement for patients' informed consent.

Design, setting, patients
The study was conducted at an interdisciplinary 22-bed ICU at the University Hospital Essen, Germany, an academic clinical institution with a nearly 1300-bed capacity. Ten ICU beds were managed by the clinic of neurology, twelve ICU beds were covered by five specialized departments for internal medicine (cardiology, gastroenterology/hepatology, nephrology, hematology, endocrinology, angiology). According to the proposed classification of the World Federation of Societies of Intensive and Critical Care Medicine (WFSICCM) [37], this ICU meets all level-3-criteria except for a formal ICU follow-up program and a nurse-to-patient ratio (NPR) of 1:1 or 1:2; the NPR of this ICU was 1:3. The ICU medical team involved six physicians (1 or 2 specialists, 4 or 5 attending hospitalists) who worked in 8-to 12-hour shifts as critical care physicians with 24-hour in house coverage.
Within an 18-month period, we recorded each patients´characteristics, medical history, reasons for admission as well as the worst clinical conditions and laboratory values within the first hour after ICU-admission for the SAPS 3 [11] or within the first 24 hours after ICUadmission for the SAPS 2 [10]. The SAPS 2 includes 15 variables, i.e., 12 physiology variables, age, type of admission, and one variable related to underlying disease [10]. The SAPS 3 utilizes 20 variables, i.e., 5 variables regarding patient characteristics prior to admission, 5 variables regarding the circumstances of the admission, and 10 physiology variables [11]. Under "supporting information", the authors provide a side-by-side overview of both scoring systems along with a detailed description of parameters included in each score (S1 Table). For SAPS 2, we calculated the predicted mortality using the according general equation [10], while for SAPS 3, we chose the North European Logit out of the available customized formulas for calculation [11].
During the study period from June 2006 until January 2008, 603 patients were admitted to the ICU. As already stated in a previous publication using the same study population [6], we excluded patients with one of the following criteria: younger than 18 years (n = 1), arteriovenous coronary bypass surgery within 2 weeks before admission (n = 9), less than 24 hours stay at the ICU (n = 22), readmission in the study period (n = 23). Besides the aforementioned criteria, we also had to exclude further 34 patients due to missing biomarkers IL-6, PCT and CRP. Finally, a sample size of 514 patients remained and was subjected to further statistical analysis. In-hospital mortality was the endpoint of this study. Furthermore, we revised the definition of the acute conditions acute kidney injury and respiratory failure. While the previous publication [6] only registered these conditions if they were originally leading to the ICUadmission, we retrospectively applied the KDIGO-criteria [38] to define an acute kidney injury and the Berlin definition to define respiratory failure [39].

Measurement of IL-6, PCT, CRP
Throughout the study period (2006)(2007)(2008), IL-6, PCT and CRP plasma levels were routinely measured in blood samples that were collected within the first hours after ICU-admission. After collection at room temperature, samples were directly transferred to the laboratory of the hospital, where they were processed on a 24-hour basis. IL-6 serum concentrations were measured using a solid phase, enzyme-labelled chemiluminescent immunometric assay (IMMULITE1 2000 XPi; Siemens healthcare GmbH, Erlangen, Germany). For PCT measurement, a 2-site sandwich immunoassay with direct chemiluminescent technology was used (ADVIA Centaur1 XPT Immunoassay-System, Siemens healthcare GmbH, Erlangen, Germany). CRP concentrations were measured in a serum sample, using a turbidimetric immunoassay test (ADVIA1 1800 Clinical Chemistry System, Siemens Healthcare Diagnostic, Erlangen, Germany). The reference ranges for the biomarkers were as follows: IL-6<15 pg/ml, PCT 0-0.5 ng/ml and CRP<0.5 mg/dl.

Statistical analysis
SPSS (version 21.0; SPSS Inc, Chicago, IL) and SAS software (version 9.4; SAS Institute Inc., Cary, NC) were used to perform statistical analysis. For descriptive statistics, absolute and relative frequencies were calculated for categorical parameters, whereas continuous parameters were characterized using the median (MD) as well as the first and third quartile (Q1, Q3). Inferential statistics to compare deceased with non-deceased patients included Fisher's Exact Test for categorical variables and the Mann-Whitney U test for continuous variables. Results were considered statistically significant when p�0.05.
We calculated AUROCs and their respective 95% CIs, to describe the discrimination of the SAPS 2 and SAPS 3 and their corresponding extended versions when combined with IL-6, PCT or CRP [40]. The extension of the SAPS scores with the biomarkers was conducted on the base of a binomial logistic regression model for in-hospital mortality using the respective SAPS and biomarker data as explanatory variables. In a further step, the ROC analysis was performed considering the predicted probabilities obtained from this model. In order to compare the discrimination performance of either the SAPS 2 or SAPS 3 against their extended versions, the difference between their AUROCs was calculated; then, the DeLong test was applied and considered statistically significant if p�0.05.
When combined with the aforementioned biomarkers, improvements in discrimination performance were observed for both, the SAPS 2 and 3 (Table 3). However, statistically Table 2

PLOS ONE
Mortality estimation after combining SAPS 2 and SAPS 3 with Biomarkers IL-6, PCT or CRP significant improvements of the AUROCs were only achieved when PCT or IL-6 were added to the SAPS 3, whereby these "hybrid" versions of the SAPS 3 still performed worse than the original SAPS 2 (Tables 2 and 3 and Fig 2). In detail, the combination of the SAPS 2 with IL-6 delivered the best discrimination performance with an AUROC of 0.82 (95% CI, 0.77-0.87),

PLOS ONE
followed by an almost equal performance of SAPS 2 on its own or in combination with PCT and CRP, respectively (Tables 2 and 3 and Fig 3).

Discussion
A recent research by Keegan et al. including more than 2500 critical care patients suggested that scoring systems with more predictor variables likely achieve better overall performances relative to those with fewer variables [42]. These results are in contrast to the higher accuracy in mortality prediction that was achieved by the SAPS 2 compared to the SAPS 3 in the present study. Even though SAPS 3 employs more variables than SAPS 2, the SAPS 2-related results of the present work are still in line with the findings of a prospective observational study with 3661 patients from more than 100 Italian intensive care units [4]. Nevertheless, alike the postulations by Keegan et al., we observed an improved discrimination performance of the SAPS 3 when combined by the additional variables IL-6, PCT or CRP and also of the SAPS 2, albeit only in combination with IL-6 ( Table 2).
To the authors´best knowledge, this study represents the first scientific effort to compare the discrimination performance between IL-6, PCT and CRP with regard to mortality prediction in a mixed population of critically ill medical patients. To date, the use of PCT and CRP in predicting sepsis or sepsis-related mortality or other adverse outcomes has been evaluated
In contrast to PCT and CRP, the predictive capacity of IL-6 has been studied in diverse intensive care contexts, showing overall good performances regarding prediction of mortality or other adverse outcomes [24][25][26][27][28][29][30][31][32][33][34][35][36]. Thus, IL-6 seems to be a rather reliable marker of illness severity and mortality in association with acute inflammatory responses [49]. The present data attest to the superior accuracy of IL-6 serum or plasma levels in predicting critical illnessrelated mortality. In fact, IL-6-as a marker-exceeded the predicitve perfomance of CRP or PCT ( Table 2 and Fig 1) and also the discrimination performance of SAPS 3 (Table 2) in our cohort of predominantly non-septic patients.
However, these results need further external validation, since application of single parameters must be considered susceptible to errors. For example, IL-6 is prone to be influenced by specific patient characteristics. In more detail, gender, obesity, alcohol abuse and recent exercise or training seem to influence IL-6 serum levels [50][51][52][53]. Moreover, the impact of comorbidities or medications on IL-6-levels has not yet been sufficiently examined.
As proper processing of blood samples is a crucial pre-analytical step for the integrity of biomarker-related results, the techniques of specimen collection and laboratory processing in the present study deserves further discussion. First, blood samples were transferred uncooled;

PLOS ONE
therefore, they were processed in less than 6 hours from blood draw [54]. Otherwise, laboratory tests would become susceptible for biased measurements due to ongoing IL-6 production through activated leukocytes [55] or proteolytic degradation and temperature lability [56]. Second, the herein considered biomarkers were only measured during the routinely obtained blood samples at ICU-admission. This procedure fits the requirements of the SAPS 3 in terms of parameters to be collected only within the first hour before or after ICU admission. However, SAPS 2 requires the worst parameters within the first 24 hours after initiation of ICU treatment. As half-life period and peak time of the biomarkers strongly vary, a more frequent monitoring of biomarkers throughout the first 24 hours could differently affect the discriminative performance of these biomarkers. This limitation should be addressed in future research.
Finally, characteristics that may have influenced score performances in our single-center study are the predominant fraction of patients with critical cardiovascular events, the low proportion of septic patients and the rather unfavorable nurse-to-patient ratio of 3:1. Since the validation of intensive care scores always dependent on specific case mixes, different admission and discharge criteria, diversity in hospital care and heterogeneous staff-or shift-work patterns, we hope that our research encourages clinicians to re-evaluate and further validate the herein presented findings in other cohorts and intensive care settings.

Conclusions
In the present study, the SAPS 2 exerted a superior discrimination performance relative to the SAPS 3. Among the analyzed markers, IL-6 achieved the highest discrimination performance over PCT and CRP. Discrimination performance of SAPS 3 improved significantly when combined with IL-6 and PCT. The combined SAPS 2/IL-6 model showed the best overall discrimination performance, albeit not significantly different in comparison to the SAPS 2.
Taken together, our data stress out the beneficial effect of IL-6 on the SAPS 2 and SAPS 3 predictive performance. The low proportion of septic patients at admission in our nonselected cohort of medical critical care patients even indicate the broad range of applicability of supplementing IL-6 to risk stratification tools in intensive care medicine. Even though immunological processes during critical illness are not completely elucidated yet, it is assumable that systemic inflammatory response syndromes (SIRS) and counter regulatory response syndromes (CARS), which may facilitate early organ failure, are not sufficiently assessed by body temperature, white blood cell count and heart rate or other parameters employed by scores such as the SAPS2 or the SAPS3. Since intensive care scores like the SAPS 2 or SAPS 3 are prone to be outdated due to changes in ICU-populations, evolution of diagnostic, therapeutic, technological and economic aspects, the implementation of IL-6 in intensive care scores may be a valuable contribution towards a modern and precise risk stratification method among heterogeneous critical care patients and settings.
Supporting information S1