Establishment and Validation of GV-SAPS II Scoring System for Non-Diabetic Critically Ill Patients

Background and Aims Recently, glucose variability (GV) has been reported as an independent risk factor for mortality in non-diabetic critically ill patients. However, GV is not incorporated in any severity scoring system for critically ill patients currently. The aim of this study was to establish and validate a modified Simplified Acute Physiology Score II scoring system (SAPS II), integrated with GV parameters and named GV-SAPS II, specifically for non-diabetic critically ill patients to predict short-term and long-term mortality. Methods Training and validation cohorts were exacted from the Multiparameter Intelligent Monitoring in Intensive Care database III version 1.3 (MIMIC-III v1.3). The GV-SAPS II score was constructed by Cox proportional hazard regression analysis and compared with the original SAPS II, Sepsis-related Organ Failure Assessment Score (SOFA) and Elixhauser scoring systems using area under the curve of the receiver operator characteristic (auROC) curve. Results 4,895 and 5,048 eligible individuals were included in the training and validation cohorts, respectively. The GV-SAPS II score was established with four independent risk factors, including hyperglycemia, hypoglycemia, standard deviation of blood glucose levels (GluSD), and SAPS II score. In the validation cohort, the auROC values of the new scoring system were 0.824 (95% CI: 0.813–0.834, P< 0.001) and 0.738 (95% CI: 0.725–0.750, P< 0.001), respectively for 30 days and 9 months, which were significantly higher than other models used in our study (all P < 0.001). Moreover, Kaplan-Meier plots demonstrated significantly worse outcomes in higher GV-SAPS II score groups both for 30-day and 9-month mortality endpoints (all P< 0.001). Conclusions We established and validated a modified prognostic scoring system that integrated glucose variability for non-diabetic critically ill patients, named GV-SAPS II. It demonstrated a superior prognostic capability and may be an optimal scoring system for prognostic evaluation in this patient group.


Introduction
Critical care medicine is a multi-disciplinary specialty concerned with the management of lifethreatening conditions in critically ill patients. These patients account for 11.3% of hospital mortality and even a high mortality rate in the six months after discharge [1,2]. Over the last three decades, several scoring systems for critical illness have been proposed for assisting physicians to quantify severity of disease and assess the prognosis. The Simplified Acute Physiology Score (SAPS) is one of the most widely used scoring systems at intensive care unit (ICU), which was first constructed in 1984 as an improvement of the Acute Physiology And Chronic Health Evaluation (APACHE) scoring system. The second generation SAPS score (SAPS II), was further validated in several studies and proved to be applicable in other cohorts [3,4,5,6].
Blood glucose levels are a crucial physiological variable for patients admitted to an ICU department with infection, sepsis and other critical conditions [7,8,9]. Of note, acute hyperglycemia and hypoglycemia were reported as independent detrimental factors for hospital mortality [10]. In scoring systems for critical illness, however, serum glucose levels have shown no significant association after adjusting for other parameters [6,11]. In recent years, glucose variability has been increasingly recognized as an independent risk factor for mortality in non-diabetic patients at ICU rather than blood glucose level [12,13,14]. Therefore, glucose variability can be considered as a novel parameter in scoring system for non-diabetic subjects at ICU.
The aim of this study was to construct and validate a modified SAPS II scoring system with additional glucose variability parameters. The system is designed to be specific for non-diabetic patients from ICU and was tested on a patient cohort from the Beth Israel Deaconess Medical Center to determine its effectiveness in predicting the accuracy of SAPS II for the risk of shortterm and long-term mortality. Furthermore, the prognostic ability of the novel scoring system was compared with other standard scoring systems.

The database
The Multi-parameter Intelligent Monitoring in Intensive Care III version 1.3 (MIMIC-III v1.3) is a publicly and freely available database comprising de-identified health-related data associated with over forty thousand patients who come from a variety critical care units of the Beth Israel Deaconess Medical Center between 2001 and 2012 [15]. In order to apply for permission to access the database, researchers are mandated to complete the NIH web-based training course named "Protecting Human Research Participants" (Our certification number: 1605699).

Study design
In this study, we by extracted data from MIMIC III database, established and validated a modified SAPS II scoring system incorporating glucose variability and named the new scoring system Glucose Variability associated SAPS II Scoring System (GV-SAPS II). The training cohort consisted of individuals admitted to ICU in the Beth Israel Deaconess Medical Center from 2001 to 2008 and the validation cohort comprised of patients admitted during 2009 to 2012 from the same database.
The start date for follow-up was the date of patient's admission. The date of death for patients was obtained from Social Security Death Records from the US government. All the patients were followed up for at least 9 months.

Population selection and definitions
A total of 58,976 ICU admissions were recorded in the MIMIC III database. Patients with diabetes were excluded from our study, and this was determined by medical history, diagnosis at admission (International Classification of Disease 9 code: 250.xx) or admission HbA1c value of ! 6.5% (recommended for the diagnosis from the American Diabetes Association [16]). For non-diabetic patients, hyperglycemia was defined as any serum glucose level ! 11.1 mmol/l, and hypoglycemia was defined as any glucose measurement 3.9 mmol/l [17].

Date extraction
Patient data was exacted from MIMIC III using structure query language (SQL) with Mysql tools (version 5.6.24), including patient identifiers, demographic parameters, clinical parameters, laboratory parameters and scoring systems. According to the patient identifier system, we can obtain the hospital records of a particular patient from 2001 to 2012 at Beth Israel Deaconess Medical Center. Records of baseline characteristics were exacted in the first 24 hours after patient admission.
Physiological information (heart rate, respiratory rate, systolic blood pressure and diastolic blood pressure) was measured by bedside monitors. Age, gender, the length of stay in hospital, readmission records were also recorded in the database.
Laboratory measurements included white blood cell (WBC) and platelet count, urea nitrogen (BUN), serum potassium, serum sodium, partial pressure of oxygen (PO 2 ), fraction of inspiration O 2 , bicarbonate, serum glucose, creatinine, and bilirubin. The mean interval of glucose records were 20 hours. Hyperglycemia and hypoglycemia were mapped to classes according to the following thresholds: 0: non-hyperglycemia, non-hypoglycemia; 1: hyperglycemia, hypoglycemia.
Three other standard scoring systems were evaluated enabling a comparison with our GV-SAPS II (original SAPS II, Sepsis-related Organ Failure Assessment Score (SOFA) and Elixhauser comorbidity score). Scores were all calculated using physiological measurements and clinical information according to published recommendations and accepted formulae [6,18,19].

Construction of the GV-SAPS II Score
In this study, three parameters were defined as glucose variability components and are: hyperglycemia, hypoglycemia and standard deviation of blood glucose levels (Glu SD ). For the training cohort, glucose variability components and SAPS II score were selected for Cox proportional hazard regression analysis for determining the association with prognosis and survival time. The hazard or instantaneous risk of death h(t) at time t after randomization for a patient with variables x l ,. . .,x n has the form h(t) = h 0 (t) exp(b 1 x 1 + b 2 x 2 + . . . b n x n ). According to the coefficients, a prognostic index (PI = b 1 x 1 + b 2 x 2 + . . . + b n x n ) can be calculated for each patient on the basis of the final mode. Higher values of index signify a worse prognosis, and lower signify a better prognosis [20]. Therefore, the PI can be used as a novel prognostic scoring system, named GV-SAPS II score, based on four parameters (hyperglycemia, hypoglycemia, Glu SD and SAPS II score). For ease of use, we defined GV-SAPS II score as a ten-fold PI.
To compare the 30-day and 9-month prognostic ability of GV-SAPS II score with other models, the area under the curve of the receiver operator characteristic (auROC) curve was determined, which is a measure of discrimination. In addition, the standard index of validity, such as the Youden index, sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value, and negative predictive value, were calculated according to the ROC results.

Statistical analysis
We categorized Glu SD into three groups using optimal binning strategies: G1: 0.7 mmol/l, G2: 0.7 to 2.1 mmol/l, G3: ! 2.1 mmol/l. In the training cohort, the hazard ratios (HRs) and 95% confidence intervals (CIs) of scoring system parameters were calculated using Cox proportional hazard regression. In addition, Kaplan-Meier survival curves were calculated to describe the incidence of outcomes after 30 days and 9 months and stratified by different risk levels of the GV-SAPS II.
The Kolmogorov-Smirnov test was used to determine whether sample data were likely to be derived from a normal distribution population. Continuous variables were summarized as mean ± standard deviation (SD) or median (inter-quartile range (IQR)), respectively. The categorical variables were displayed as counts or percentages (%). The characteristics of the study population in two cohorts were compared using Student's t test or non-parametric Wilcoxon test for continuous variables and χ 2 -test for categorical variables. All P-values were two-sided and a P value of < 0.05 was considered statistically significant. Analyses were performed in SPSS version 20.0 (SPSS, Chicago, IL, USA), MedCalc version 12.7 (MedCalc Software, Ostend, Belgium).

Characteristics of the study sample
A total of 58,976 admission records were extracted and enrolled in our cohort. After exclusion of those who did not meet the inclusion criteria, 4,895 and 5,048 eligible individuals were finally included in the training and validation cohorts, respectively (Fig 1). Table 1 summarizes the patient characteristics and glucose indices for the two cohorts. In the training cohort, median and IQR of Glu SD was 1.1 mmol/l (0.7 to 1.8 mmol/l), of which the proportion of hyperglycemia patients was 14.2% and hypoglycemia was 6.6%. In the validation cohort, median and IQR of GluSD was 1.0 mmol/l (0.7 to 1.6 mmol/l), of which the proportion of hyperglycemia patients was 10.6% and hypoglycemia was 6.0%. The scoring systems of training subjects showed the scores were 4.0 (2.0 to 6.0), 31.0 (22.0 to 45.0), 0.0 (-1.0 to 4.0) in SOFA, SAPS II, Elixhauser score, respectively. In the validation cohort, the prognostic scores were 3.0 (1.0-5.0), 29.0 (21.0-43.0), 0.0 (-1.0-6.0) in SOFA, SAPS II and Elixhauser score. Hospital characteristics and clinical outcomes showed that the 30-day mortality of subjects was 12.5% and 9.7% in training and validation cohorts, respectively. Furthermore, the mortality of the two cohorts were 18.7% and 15.9% at the end of 9 months.

Construction of the GV-SAPS II
Glucose variability was defined as the Glu SD per patient in our study and this has been widely used in previous studies [21]. Measures of hyperglycemia and hypoglycemia as outcomes of serious glucose fluctuations, were included in the glucose variability components as well.
The performance of GV-SAPS II to predict 30-day and 9-month outcomes in the training cohort is presented in Fig 2A and 2B  Moreover, we used an optimal cut-off value of 28 and 26 for 30-day and 9-month prediction respectively. The sensitivities were 75.94% and 71.33% respectively, the specificities were 73.23% and 67.55% respectively (Table 3).
In the validation cohort, the novel scoring system also presented an improved capability to predict 30-day and 9-month mortality. As shown in Fig 2C and 2D significantly lower than GV-SAPS II score (all P < 0.001). Using the best cutoff values of 26 and 24 for 30 days and 9 months, the sensitivities were 77.91% and 70.61% respectively, the specificities were 71.11% and 63.25% respectively (Table 3).

Survival distributions in different risk levels of the GV-SAPS II
To understand the survival distributions in different risk levels of the novel scoring system, we classified GV-SAPS II score into quartiles as follows: group 1 (< 22); group 2 (22 to 34); group 3 (34 to 41); group 4 (> 41). As shown in Fig 3, Kaplan-Meier curves indicate significantly worse outcomes in patients in higher score groups for both 30-day and 9-month mortality (all P< 0.001).

Discussion
SAPS is one of many ICU scoring systems, which has been available since 1984 and designed to measure and predict the severity and prognosis of disease. The SAPS II score is calculated from 12 physiological measurements including age, heart rate, systolic blood pressure, temperature, GCS, mechanical ventilation or CPAP, PaO 2 , FiO 2 , urine output, blood urea nitrogen, sodium, potassium, bicarbonate, bilirubin, white blood cell, chronic diseases, type of admission. In this study, we constructed a modified SAPS II scoring system by adding glucose variability parameters (hyperglycemia, hypoglycemia, SD of blood glucose levels), named GV-SAPS II, for nondiabetic critically ill subjects. Although it was based on 30-day outcomes, we have demonstrated a prognostic value both in short-term and long-term mortality measurements using ROC analysis. In comparison with other standard scoring systems, GV-SAPS II performed significantly better with a higher auROC in both training and validation cohorts. Moreover, Kaplan-Meier survival curves showed that higher GV-SAPS II score groups were associated with a higher risk for death at 30 days and 9 months.
To our knowledge, this is the first modified prognostic scoring system that integrates glucose variability for non-diabetic critically ill patients. In previous studies, various physiological parameters were considered to build scoring systems, including serum glucose concentration. After adjusting for other parameters, the glucose level showed no significance prognostic capability [6,11]. In contrast, abnormal glucose levels have been demonstrated to represent an increased risk of mortality in several studies of critically ill patients [22,23,24,25]. The conclusion for these contradictory observations suggests that glucose variability, rather than serum glucose concentration has a crucial role in the mortality of critically ill patients [26,27,28,29]. From a clinical perspective, a single point serum glucose measurement can be easily influenced by a wide range of confounders, such as drug, diet, inflammation and physiological stress state. Therefore, it may not adequately reflect metabolic state in patients with critical illness. On the contrary, glucose variability may reveal dynamic changes of glucose levels, assessing the control of blood glucose. Moreover, the underlying mechanism for glucose variability has been reported to be associated with oxidative stress, neuronal damage, and blood coagulation activity [13,21]. In a vitro study, it has been demonstrated that acute fluctuations of glucose may be more detrimental to endothelial cell function than a constant abnormal level of glucose. This may contribute to increasing cardiovascular risk and reducing the homeostatic potential of the vasculature to accommodate perturbations in stress [30]. Thus it is feasible that glucose variability may play an important role in pathological processes associated with patients who are critically ill, suggesting that glucose variability should be considered as a part of the future development of prognostic models predicting patient mortality. In this study, patients with diabetes were excluded from our study. Although glucose variability has been shown to be an independent risk factor in several mixed cohorts, previous studies have reported nonsignificant association between glucose variability and mortality of patients with diabetes [29,31]. In addition, it is a subject of debate as to whether hyperglycemia is an independent risk factor for patients with diabetes in ICU [29,32,33]. It has been proposed that these patients may become desensitized to rapid fluctuation of glucose levels, however, firm evidence is lacking and future research needs to establish specific models which may be applicable to patients with diabetes.
There are four main limitations of the present study. First, this is a single center cohort study and different conclusions may be reached using patient records from other centers, suggesting that a multicenter and prospective study are needed. Secondly, in order to ensure the accuracy of glucose variability, enough times of blood glucose test are needed. Thirdly, although SAPS II is still most widely used model, SAPS III has been available since 2005 [34] and a lack of surgery site information in the patient data base precluded a comparison of our model with SAPS III. Fourthly, due to the inconsistent pattern for glycemic metabolism, the diabetes patients have been excluded in our study. This may limit the scope of this scoring tool. Additionally, patients with impaired fasting glucose or impaired fasting glucose may be included in our cohort, which may have an impact on our prognostic system.

Conclusions
We have constructed a modified prognostic scoring system that integrates glucose variability for non-diabetic patients who are critically ill. The GV-SAPS II scoring system was shown to have superior prognostic capability in study cohorts and may have utility as a scoring system for medical decision making and prognostic evaluation.