Performance on the APACHE II, SAPS II, SOFA and the OHCA score of post-cardiac arrest patients treated with therapeutic hypothermia

Objective This study assessed the ability of the Acute Physiologic and Chronic Health Evaluation (APACHE) II score, Simplified Acute Physiology Score (SAPS) II, Sequential Organ Failure Assessment (SOFA) score, and out-of-hospital cardiac arrest (OHCA) score to predict the outcome of OHCA patients who underwent therapeutic hypothermia (TH). Methods This study included OHCA patients treated with TH between January 2010 and December 2013. The APACHE II score, SAPS II, and SOFA score were calculated at the time of admission and 24 h and 48 h after intensive care unit admission. The OHCA score was calculated at the time of admission. The area under the curve (AUC) of the receiver operating characteristic curve and logistic regression analysis were used to evaluate outcome predictability. Results Data from a total of 173 patients were included in the analysis. The APACHE II score at 0 h and 48 h, SAPS II at 48 h, and OHCA score had moderate discrimination for mortality (AUC: 0.715, 0.750, 0.720, 0.740). For neurologic outcomes, the APACHE II score at 0 h and 48 h, SAPS II at 0 h and 48 h, and OHCA score showed moderate discrimination (AUC: 0.752, 0.738, 0.771, 0.771, 0.764). The APACHE II score, SAPS II and SOFA score at various time points, in addition to the OHCA score, were independent predictors of mortality and a poor neurologic outcome. Conclusions The APACHE II score, SAPS II, SOFA score, and OHCA score have different capabilities in discriminating and estimating hospital mortality and neurologic outcomes. The OHCA score, APACHE II score and SAPS II at time zero and 48 h offer moderate predictive accuracy. Other scores at 0 h and 48 h, except for the SOFA score, are independently associated with 30-day mortality and poor cerebral performance.

Introduction year. The study was conducted in the ED and ICU from January 1, 2010 to December 31, 2013.

Population
The study population included all adult (age > 18 years) patients who were admitted to the ICU after OHCA, with successful return of spontaneous resuscitation (ROSC) and underwent TH. CA patients with a primary traumatic etiology and in-hospital cardiac arrest (IHCA) patients were excluded.

Post-resuscitation care
Post-resuscitation care was based on the recommendations and guidelines of the International Liaison Committee on Resuscitation (ILCOR) [4]. The decision on whether to initiate TH was made by the treating physician according to the TH inclusion criteria: 1) successful ROSC from CA; 2) comatose patient (Glasgow coma scale [GCS] of less than or equal to 8); 3) mean arterial pressure of greater than or equal to 60 mmHg with or without the use of vasopressor agents. If the patient did not satisfy the TH indications, TH was not performed. In addition, patients were excluded if the guardian refused treatment because of malignancy with multiple metastases or old age. All patients were given mechanical ventilation, sedatives, analgesics and muscle relaxants during TH. The target temperature of 33˚C was induced by either an external or endovascular cooling device in addition to ice packs, 4˚C normal saline infusion, and bladder irrigation. External cooling was performed using a water blanket (Blanketrol1 II, Cincinnati Sub-Zero Products, OH, USA) or a pad (ArcticGelTM Pads & Arctic Sun1 2000, Medivance, CO, USA). Endovascular cooling was performed using a catheter (Cool Line1 Catheter & CoolGard 30001, ZOLL, MA, USA). After 24 hours of the maintenance phase, rewarming began with a rate of 0.2~0.3˚C/h up to 36.0˚C, and normothermia was maintained for 72 hours.

Data collection and definitions
Data collected included patient demographic information, comorbid conditions, variables that are necessary for calculating severity scores, mortality and neurologic status at 30 days. CA event data was recorded in the Utstein style [22]. The APACHE II score and SAPS II were originally measured during the first 24 hours after ICU admission but in this study, the APACHE II score, SAPS II, and SOFA score were calculated at the time of admission (0 h), from the admission to the first 24 h (24 h), and from the 24 h to the 48 h (48 h) based on the worst vitals and laboratory findings in each 24-h interval. Initial physiologic values used at 0 h were not included in the calculation at 24 h. If there were any missing data points, they were calculated according to the methodology described by Vincent et al [23]. The GCS, which is included in calculation of the score, used the last value measured before sedation for TH [24]. The OHCA score was calculated based on the initial recorded rhythm, no-flow interval, lowflow interval, serum creatinine, and arterial lactate [19]. The neurologic outcome was measured using cerebral performance category (CPC) score [25]. The CPC score at 30 days after ICU admission was assessed by an ICU doctor via direct examination when the patient stayed in our hospital. The other patients were evaluated by a patient's caregiver or doctor in the transferred hospital via telephone to define the CPC score at 30 days after CA. Patients were divided into two groups by CPC score. A CPC score of 1 or 2 is considered a good neurologic outcome, while a CPC score of 3, 4, or 5 is considered a poor neurologic outcome [5,6]. The study was approved by the Institutional Review Board at Gachon University Gil Medical Center (IRB No. GCIRB2015-100).

Statistical analysis
Continuous variables were presented as the mean with standard deviation (SD) or the median with inter-quartile range (IQR) according to data distribution. Categorical variables were presented as frequency with percentage. The t-test was used to compare differences between groups when the dependent variable was continuous. A chi-square test was used to examine the association between categorical variables. The discrimination of these scores was assessed using area under the curve (AUC) of the receiver operator characteristic (ROC) curve. An AUC value of 0.9-1.0 indicates excellent, 0.8-0.89 good, 0.7-0.79 moderate, 0.6-0.69 poor and 0.5-0.59 not useful. Multivariate analysis was performed to identify the predictors for mortality and poor neurologic outcome at 30 days. Variables had a p-value of 0.2 or less on univariate analysis entered in the backward stepwise elimination. A p-value less than 0.05 was considered statistically significant. Statistical analysis was performed using SPSS version 18.0 (SPSS, Chicago, IL, USA) and MedCalc program version 12.7.7.0 (MedCalc Software, Mariakerke, Belgium).

Results
Of the 237 patients, 173 were included, and 64 patients with the following conditions were excluded (Fig 1, S1 File). The mean age was 53 years (SD: ±14.8), and 118 patients (68.2%) were male. Baseline characteristics and cardiac arrest data are summarized in Table 1. Hospital mortality within 30 days was 39.9%. Good neurologic outcome at 30 days after ICU admission was recorded in 53 (30.6%) patients. Internal cooling was performed in 74 of 173 patients and external cooling was performed in 99 patients. There was no difference between the survivors/ non-survivors (p = 0.301) and good/poor CPC groups (p = 0.226). The median time between The serial mean values of each scoring system according to mortality and neurologic outcome are shown in Table 2. When each score was measured over time, there was a significant difference between the survivor and non-survivor groups and between the good CPC and poor CPC groups.
Time-dependent AUC values for each scoring system to predict mortality and poor neurologic outcome at 30 days are shown in Table 3. The APACHE II score at 0-h (AUC: 0.715) and 48-h (AUC: 0.750), SAPS II at 48-h (AUC: 0.720), and OHCA score (AUC: 0.740) had a moderate ability to discriminate survivors from non-survivors. The APACHE II score at 0-h (AUC: 0.752) and 48-h (AUC: 0.738), SAPS II at 0-h (AUC: 0.771) and 48-h (AUC: 0.771), and OHCA score (AUC: 0.764) showed moderate discrimination for poor neurologic outcomes. The SOFA score showed poor discrimination of both mortality and neurologic outcomes. All scores measured at 24-h had poor discrimination. In addition, we analyzed whether mortality and neurologic outcome could be predicted by GCS alone. The initial GCS measured before sedation was not useful for discrimination for mortality (AUC: 0.486, 95% CI: 0.393-0.580, P = 0.773) and poor neurologic outcome (AUC: 0.518, 95% CI: 0.429-0.607, P = 0.688).
The ROC curves for the APACHE II score, SAPS II, SOFA score, and OHCA score are shown in Fig 2. In predicting mortality, there was no statistical difference of the AUC values between scorings for each time period. In the neurologic outcome prediction, the AUC value of SAPS II measured at 0-h was statistically higher than the AUC value of SOFA score (P = 0.039). The AUC values for the SAPS II and APACHE II score measured at 48-h were statistically higher than the AUC value for the SOFA score (P = 0.001, 0.022). Severity scores and OHCA score association at each time point are shown in Table 4. Multivariate logistic regression was performed for each severity scoring system and time interval  164) were also independently associated with poor neurologic outcomes. The OHCA score was a significant predictor of death and poor CPC.

Discussion
To the best of our knowledge, this is the first study to simultaneously evaluate the performance of different scoring systems, such as the APACHE II score, SAPS II, SOFA score, and OHCA score, in a homogeneous group of OHCA patients who underwent TH. In this investigation, we found that the APACHE II score, SAPS II, and OHCA score had a moderate ability to discriminate outcomes following CA. When examining various severity scores by time, we were able to identify that the scores at certain time points were independently associated with mortality and poor neurologic outcome. It should be emphasized, however, that the scoring systems are intended to predict disease severity and mortality, not to determine patient's medical management. In clinical settings, this result should not influence the decision to discontinue treatments.
The SAPS II at 48-h and the APACHE II score at 0-h and 48-h time points and the OHCA score were moderate predictors of 30-day mortality. The SAPS II and APACHE II score at 0-h and 48-h time points and the OHCA score were moderate predictors of the 30-day neurologic outcome. The SAPS II and APACHE II score at 24-h and the SOFA score at all time-points were poor predictors. The largest AUC was obtained by the SAPS II at 0 h (AUC: 0.771) and 48 h (AUC: 0.771) for neurologic outcome and the APACHE II score at 48 h (AUC: 0.750) for mortality. In multivariate logistic regression analysis, the SAPS II and APACHE II score at the 0-h and 48-h time points were associated with 30-day mortality and poor neurologic outcome. We also analyzed whether any of the variables except the severity scores were related to mortality or poor CPC. Variables such as cause of CA (non-cardiac origin), interval form collapse to ROSC, initial non-shockable rhythm, and initial lactate level were significant predictors of both mortality and poor neurologic outcome. However, these factors are not included in the calculation of the APACHE II score, SAPS II, and SOFA score. As noted in previous studies [11][12][13][14], severity scoring systems should be considered to not be specifically developed for CA patients' evaluation. Pre-and intra-arrest conditions and factors contribute substantially to the severity of the post-cardiac arrest syndrome and on outcomes [26]. If the severity scores, such as the APACHE II score and SAPS II, are increasingly combined with related factors, they may improve predictability of outcome in OHCA patients.
In a study by Donnino et al. [11], the APACHE II score measured at 0 h (AUC: 0.58) was not useful to predict neurologic outcome in OHCA patients. The performance score increased incrementally over the next 24-h (AUC: 0.74), 48-h (AUC: 0.79), and 72-h (AUC: 0.90) time points. However, in our series, the APACHE II score at 0 h (AUC: 0.752) showed moderate discrimination. These differences are explained by the following reasons. First, the GCS score included in the calculation of severity scoring systems in our study was the last measured GCS before sedation for TH. In the study by Livingstan et al. [24], using the GCS recorded before sedation increased the discrimination of APACHE II score. Second, other studies include a heterogeneous group with TH implementation. On the other hand, our study only included patients undergoing TH that may have more effect on results. The differences in the patient groups included in the analysis, the outcome predictability of scoring systems could differ.
The OHCA score incorporates variables available early after ROSC, such as no-flow and low-flow intervals, serum lactate and creatinine levels at admission, and initial cardiac rhythm. Previous validation studies of OHCA score showed a similar good performance in terms of discrimination, which supports the generalizability of the score despite the differences in age, no-flow time, initial rhythm, and the rate of TH. In our study, OHCA score achieved moderate discrimination (AUC: 0.740 for mortality, 0.764 for neurologic outcome) for the 30-day outcome, but it was lower compared with that of previous studies [18][19][20]. In addition to the OHCA score, the Pittsburgh Cardiac Arrest Category (PCAC) was showed AUC of 0.79-0.82 for mortality and functional outcome [27]. This can be due to differences in patient baseline characteristics, emergency medical system, and TH application rate. Specifically, previous studies with high discriminatory performance have only a small percentage (11% and 34%) of TH application compared with 100% in our study. This suggests that TH affects the performance of the OHCA score.
The time-dependent predictive performances of the APACHE II score and SAPS II were moderate at 0 h and 48 h, but the predictability of all scoring systems declined at 24 h. Regarding adjusted OR, the scores at 24 h were not predictors of outcome except for SAPS II at 24 h for mortality. This tendency seems to be due to the effect of TH. Induction of TH usually starts at the time of admission. Parameters needed to calculate scores at 24 h were obtained during the first 24 hours after ICU admission. Parameters obtained in this period may reflect the effect of TH because the target temperature is maintained for 24 hours after induction of TH. TH affects various clinical and laboratory parameters [28]. Mean arterial pressure increases slightly (±10mmHg), and cardiac output decreases due to a decrease in heart rate. In blood samples from hypothermic patients, PO2 and PCO2 are overestimated, while pH is underestimated, and thus metabolic acidosis looks more severe. White blood cells and platelets decrease and hematocrit mildly increases. These physiologic changes generally do not require treatment and also do not affect prognosis. However, the severity scores were changed because these parameters are used to calculate severity scores. It is thought that outcome predictability is reduced when the score is calculated using measured parameters during 24 hour hypothermic treatment.
In our study, survival was 104/173 patients but good CPC occurred in only 53. This is because of many patients with CPC 4 are in the survival group compared with previous studies. The end-of-life decision such as the withdrawal of life-sustaining therapy (WLST) is a very sensitive and difficult problem due to medical, ethical, and economical reasons. In Korea, WLST is not permitted unless the patient is brain death and donate organ. Therefore, comatose resuscitated patients receive intensive care support for prolonged periods compared with Europe and may lead to different outcome of early stage. As a result, discrimination for mortality decreases compared with that for poor neurologic outcome.
This study has several limitations because it was a single-center study and because a sample size calculation was not performed. In addition, the severity scoring was performed within the first 48 hours after ICU admission. As discussed, the effect of hypothermia including rewarming is not resolved within this time. In the case of the APACHE score and SAPS, the temperature variable is included in calculations. In our study, no correction was made for the temperature effect, thus the results may look worse. The TH also affects other variables such as vital signs and laboratory tests' results, especially the 24-h outcome. The clearance of drugs such as sedatives and muscle relaxants may not have been metabolized within 48 hours due to the influence of TH. If these score calculations were extended to 72 hours after ICU admission, facilitation of the identification of changes in scores over time and clearer determination of the effect of TH would be possible. Finally, although this study is meaningful to analyze only patients who underwent TH, patients who were too unstable initially to be treated with TH were excluded, thus possibly affecting the outcome due to bias.

Conclusions
The OHCA score and most severity scores of different time points, except scores measured at 24 h, are significantly associated with mortality and poor neurologic outcome. The OHCA score, 0-h and 48-h scores of the APACHE II score and SAPS II offer moderate prediction accuracy ability in OHCA patients who underwent TH; however, the SOFA score remained poor. To improve the outcome predictability of CA patients, development of a new scoring system including cardiac arrest-related variables is needed.