Mortality Prediction in Patients Undergoing Non-Invasive Ventilation in Intermediate Care

Background Intermediate Care Units (ImCU) have become an alternative scenario to perform Non-Invasive Ventilation (NIV). The limited number of prognostic studies in this population support the need of mortality prediction evaluation in this context. Objective The objective of this study is to analyze the performance of Simplified Acute Physiology Score (SAPS) II and 3 in patients undergoing NIV in an ImCU. Additionally, we searched for new variables that could be useful to customize these scores, in order to improve mortality prediction. Design Cohort study with prospectively collected data from all patients admitted to a single center ImCU who received NIV. The SAPS II and 3 scores with their respective predicted mortality rates were calculated. Discrimination and calibration were evaluated by calculating the area under the receiver operating characteristic curve (AUC) and with the Hosmer-Lemeshow goodness of fit test for the models, respectively. Binary logistic regression was used to identify new variables to customize the scores for mortality prediction in this setting. Patients The study included 241 patients consecutively admitted to an ImCU staffed by hospitalists from April 2006 to December 2013. Key Results The observed in-hospital mortality was 32.4% resulting in a Standardized Mortality Ratio (SMR) of 1.35 for SAPS II and 0.68 for SAPS 3. Mortality discrimination based on the AUC was 0.73 for SAPS II and 0.69 for SAPS 3. Customized models including immunosuppression, chronic obstructive pulmonary disease (COPD), acute pulmonary edema (APE), lactic acid, pCO2 and haemoglobin levels showed better discrimination than old scores with similar calibration power. Conclusions These results suggest that SAPS II and 3 should be customized with additional patient-risk factors to improve mortality prediction in patients undergoing NIV in intermediate care.


Objective
The objective of this study is to analyze the performance of Simplified Acute Physiology Score (SAPS) II and 3 in patients undergoing NIV in an ImCU. Additionally, we searched for new variables that could be useful to customize these scores, in order to improve mortality prediction.

Design
Cohort study with prospectively collected data from all patients admitted to a single center ImCU who received NIV. The SAPS II and 3 scores with their respective predicted mortality rates were calculated. Discrimination and calibration were evaluated by calculating the area under the receiver operating characteristic curve (AUC) and with the Hosmer-Lemeshow goodness of fit test for the models, respectively. Binary logistic regression was used to identify new variables to customize the scores for mortality prediction in this setting.

Patients
The study included 241 patients consecutively admitted to an ImCU staffed by hospitalists from April 2006 to December 2013.

Introduction
Non-Invasive Ventilation (NIV) reduces in-hospital mortality in selected patients with acute respiratory failure (ARF) [1][2][3][4]. Currently, indications and success of this ventilation mode depend on ARF etiology, the evolution in the first hours of the PaO2/FiO2 ratio, pH, pCO2 and the respiratory rate [5][6][7]. For this reason, initiation of NIV in ARF should be done under continuous surveillance [5]. In this context, Intermediate Care Units (ImCU) may offer a suitable alternative to the intensive care unit (ICU), due to the availability of non-invasive monitoring and specialized staff and nursing dedication [8].
Statistically significant differences in Simplified Acute Physiology Score (SAPS) II and Acute Physiology and Chronic Health Evaluation (APACHE) II scores have been found between survivors and non-survivors who underwent NIV due to acute lung injury (ALI) [9], acute respiratory distress syndrome (ARDS) [10], COPD exacerbation [11], acute pulmonary edema (APE) [12], interstitial lung disease [13] and acute myasthenia gravis [14]. Regarding the performance of SAPS 3, a revised version of SAPS II, which includes more comorbidities and physiological values within the first hour of admission [15], the information is even more limited. Metnitz et al., [16] described a cohort of 157 patients who underwent Pressure Support Ventilation (PSV) from a multicenter multinational study of the SAPS 3 data base. The study was descriptive, without information regarding the calibration and discrimination power of the score.
In this context, conventional scores show high dependence on case-mix for accurate prediction [17][18][19] and do not include important respiratory variables, such as ARF etiology, hemoglobin and pCO2 levels [20][21][22][23][24]. Therefore, the objectives of this study were: (i) to describe the outcomes of NIV population in an ImCU, (ii) to evaluate the performance of SAPS II and 3 in mortality prediction of patients undergoing NIV in this area and (iii) to customize the scores with new variables in order to improve their performance.

Materials and Methods
The study was performed at the Clínica Universidad de Navarra, an academic medical center in Pamplona, Spain. Our ImCU is a 9-bed multi-purpose area staffed by hospitalists and shared with coronary and stroke units while being independent from the mixed ICU. Each bed is equipped with continuous telemetry, pulse-oxymetry, noninvasive arterial blood pressure, central venous pressure monitoring, and non-invasive pressure support ventilation with BiPAP.
The signals are relayed to a central monitoring station and the nurse-patient ratio is 1:3. Demographics, past medical history, reasons for admission, physiological parameters at the time of admission and during the first 24 hours of ImCU stay, laboratory variables and survival to hospital discharge were prospectively recorded by the authors. Admission and discharge criteria of the ImCU were set according to previous guidelines defined by The American College of Critical Care Medicine [8].
Every consecutive patient undergoing NIV from April 2006 to December 2013 was evaluated and included in the study. Exclusion criteria from ImCU admission were: age less than 18 years, severe respiratory failure at imminent risk of intubation, status epilepticus and catastrophic brain illness. ImCU readmissions were also excluded from the final analysis. In-hospital mortality was the end point of the study.
Indication of NIV was established according to previously published guidelines [5]. Patients were divided in four groups of ARF etiology: COPD exacerbation, acute pulmonary edema, hypoventilation due to neuromuscular disease and hypoxemic ARF. NIV failure was defined as the discharge of patients to the ICU to be intubated or the in-hospital death of subjects due to ARF when do-not resuscitate (DNR) orders were set.

Ethical Standards
The study protocol was approved by the Institutional Review Board (IRB) at the Clínica Universidad de Navarra (ref. 129/2010). The IRB waived the need for informed consent, because it did not interfere with decisions related to patients care. The study has been performed in accordance with the ethical of the 1964 Declaration of Helsinki and its later amendments.

Statistical Analysis
Data were prospectively collected after patient admission, into a computer database by the authors and based on the electronic medical record system of the Clínica Universidad de Navarra. Continuous variables were reported as mean and standard deviation, and as median and interquartile range (IQR) when appropriate. These variables were analyzed by t-Student, ANOVA, U-Mann Whitney and Kruskal Wallis tests. Categorical variables were summarized as absolute frequencies and percentages, and analyzed by chi-square test. Performance of the prognostic scores was assessed by standard procedures to measure discrimination and calibration. Discrimination was evaluated by calculating the area under receiver operating characteristic (ROC) curve (AUC) [25] and calibration was assessed using the Hosmer-Lemeshow goodness-of-fit test, in which a high p value (> 0.05) was interpreted as a good fit for the models [26,27]. In the rest of the analyses, statistically significant results were considered when p <0.05. Standardized mortality ratios (SMR) (ratio of observed to expected deaths) were calculated for each model and ARF etiology subgroup. Customized models were developed using binary logistic regression. The variables included in the model were: hemoglobin, pCO2 and lactic acid levels, SAPS II score, COPD diagnosis and APE, previously described in NIV mortality prediction [5,[21][22][23][24]. Immunosuppression, DNR orders and the SAPS 3 score, were also included in the analysis, due to their potential clinical application in predicting NIV mortality.
Statistical analyses was performed, using SPSS for Windows, version 20.0 (SPSS Inc, Chicago, IL).

Model Validation
The customized models were validated internally by bootstrapping. The bootstrap method randomly draws multiple samples with replacement from the original cohort. The model is developed again in each bootstrap sample yielding a different AUC (c-statistic) to each bootstrap model [28]. An average of these c-indexes was calculated. The bootstrap procedure was performed with 1000 draws with replacement in the study cohort.

Results
From April 2006 to December 2013, a total of 241 patients were non-invasively ventilated in the ImCU of our hospital and constituted the study group. Demographic characteristics, comorbidities, location prior to admission, discharge location, variables at admission and length of hospital stay are described in Table 1. The population had a mean age of 68 years, and a Mortality prediction based on SAPS II showed conflicting results, considering the NIV indication. A mismatch was observed between the SMR of the hypoxemic respiratory failure and the neuromuscular disease groups (1.63 vs 0.55, respectively). However, the prediction power was acceptable for the groups of COPD (1.10) and APE (0.85). Interestingly, SAPS 3 tended to overestimate mortality in all the groups according to SMR, being more accurate for hypoxemic ARF (0.86) ( Table 2).
Performance of SAPS II and 3 is described in Table 3. Discrimination according to AUC was of 0.73 (CI 95% 0.69-0.80) for SAPS II and 0.69 (CI 95% 0.63-0.77) for SAPS 3. Results Univariate analysis was done for mortality prediction, including SAPS II, SAPS 3 and other important variables in the ARF context. Statistically significant differences were found for NIV indications, immunosuppression, patients with do-not-resuscitate orders (DNR), previous diagnosis of COPD, pCO2 (mmHg), lactic acid (mg/dl), hemoglobin (g/dl), SAPS II and SAPS 3 (p<0.05 for all) ( Table 4). Other variables such as hypertension, coronary artery disease and white blood cell count were found to be not statistically significant in this analysis.
All these variables were converted to dichotomous after sensitivity analysis and included together with SAPS II and 3 in independent logistic regression models. Then, two customized models (SAPS II-CNIV and SAPS 3-CNIV) were developed from logistic regression results (Tables 5 and 6).
In the SAPS II-CNIV model, immunosuppression and lactic acid over 2 mg/dl were found to be risk factors, while previous diagnosis of COPD, acute pulmonary edema and haemoglobin levels over 10.7 g/dl were found to be protective factors (p<0.05). Carbon dioxide pressure (pCO2) > 55 mmHg was protective and near to statistical significance (p = 0.09). Interestingly, DNR orders were not statistically significant in this model (p = 0.20), probably due to a colinearity phenomenon (comorbidities, immunosuppression and metastatic cancer). Each variable was given a prognostic value to be included in the customized score, which was established as the weight of each variable in mortality prediction in comparison to a SAPS II single point ( Table 5).
The SAPS 3-CNIV model showed statistical significance of DNR orders and lactic acid > 2 mg/dl as risk factors and previous diagnosis of COPD, APE and haemoglobin levels > 10.7 g/ dl as protective factors (p<0.05). Meanwhile, pCO2 > 55 mmHg was a protective factor near to statistical significance (p = 0.09). Immunosuppression was not included in the analysis to avoid co-linearity phenomenon, as this variable is part of the original SAPS 3. Prognostic values were assigned to each variable in the same way than in SAPS II-CNIV (Table 6).
In an effort to simplify the application of the customized scores, sensitivity analysis was performed on SAPS II-CNIV and 3-CNIV to find critical values which could classify patients in statistically different mortality groups. After this procedure, SAPS II-CNIV was divided in 4 mortality groups and SAPS 3-CNIV in 3. Cut-off values and death probability for each group are shown in Tables 7 and 8.

Discussion
The results of the current study highlight the importance of customizing SAPS II and 3 with patient-risk factors to improve mortality prediction in patients undergoing NIV in Intermediate Care.
To our knowledge, no previous studies described a complete evaluation of the discrimination and calibration power of SAPS II and 3 in non-invasively ventilated patients. In this study, SAPS II and 3 showed decreased prediction accuracy in terms of discrimination and calibration as compared to original cohort results of both scores in critically ill patients [15,29] and also in ImCU population [30][31][32].
This fact could be explained by the importance of patient-risk factors in the NIV mortality prediction. Mortality in ARF is highly dependent on respiratory variables at the beginning of the technique [5]. In this context pO2, pCO2, lactic acid and hemoglobin levels at admission have demonstrated to influence mortality prediction in this subgroup of patients [20][21][22][23][24].
Differences in mortality depending on ARF etiology have been extensively described [33][34][35]. The results in the etiology subgroups of our study are in agreement with previous descriptions;  being lower in COPD exacerbation, neuromuscular disease and APE (10-20%), and higher in hypoxemic respiratory failure (50-80%) [5,36,37]. Nevertheless, neither ARF etiology nor previously described physiological variables are included in the original SAPS models. For this reason, an evaluation combining the use of severity scores and patient-risk factors, could improve the accuracy of mortality prediction in NIV users. In our study, the two customized SAPS II and 3 models appeared to be a synthesis of both concepts, with an acceptable discrimination and calibration power. Additionally, internal validation was assessed by bootstrapping method.
The higher overall mortality of our cohort, in comparison to previously published series [16,36], could be due to differences in case-mix population. The contribution of advanced oncologic (24.5%) and DNR-order patients (29.9%), of our cohort could, at least, partially explain these results. Furthermore, the results of this study support previous descriptions regarding the efficacy of Intermediate Care for NIV application.
Previous studies described the potential of Intermediate Care in terms of cost-containment, rational ICU utilization and triage flexibility for acute patients [38,39]. Recently, Capuzzo et al. [40] described the positive impact of the presence of an ImCU in terms of adjusted hospital mortality for patients admitted in intensive care. Nonetheless, we need more, larger and controlled trials to definitively establish the indications and limitations of the ImCU.
The patients with higher scores and significant mortality risk should be followed carefully, in order to avoid unnecessary delays in transfer to ICU or intubation. However, we need external validation studies to establish the cut-off points for ICU admission.
The most relevant studies for mortality prediction in critical care were multicentric and with extensive number of patients [15,29]. However, the number of patients in other NIV studies is similar to the present sample [1][2][3][4][11][12][13][14]36,37]. The cohort of Metnitz et al. [16] derived from a multicenter multinational study of the SAPS 3 database, included 157 patients who underwent PSV. The study was descriptive, without information regarding the calibration and discrimination power of the score and with an observed to expected mortality ratio of 0.79, compared with our SMR of 0.69. These differences could be explained by the overfitting bias of the SAPS 3 score when the validation case mix cohort is substantially different from the original study. These limitations emphasize the need to find specific scores for mortality prediction in NIV. We need more studies, with validation cohorts of the SAPS II-CNIV and SAPS 3-CNIV scores, before its routine application in intensive care.

Conclusions
SAPS II and 3 models, combined with complementary patient-risk factors, could improve mortality prediction in patients undergoing NIV in the ImCU.