Prediction of 28-day mortality in critically ill patients with COVID-19: Development and internal validation of a clinical prediction model

Background COVID-19 pandemic has rapidly required a high demand of hospitalization and an increased number of intensive care units (ICUs) admission. Therefore, it became mandatory to develop prognostic models to evaluate critical COVID-19 patients. Materials and methods We retrospectively evaluate a cohort of consecutive COVID-19 critically ill patients admitted to ICU with a confirmed diagnosis of SARS-CoV-2 pneumonia. A multivariable Cox regression model including demographic, clinical and laboratory findings was developed to assess the predictive value of these variables. Internal validation was performed using the bootstrap resampling technique. The model’s discriminatory ability was assessed with Harrell’s C-statistic and the goodness-of-fit was evaluated with calibration plot. Results 242 patients were included [median age, 64 years (56–71 IQR), 196 (81%) males]. Hypertension was the most common comorbidity (46.7%), followed by diabetes (15.3%) and heart disease (14.5%). Eighty-five patients (35.1%) died within 28 days after ICU admission and the median time from ICU admission to death was 11 days (IQR 6–18). In multivariable model after internal validation, age, obesity, procaltitonin, SOFA score and PaO2/FiO2 resulted as independent predictors of 28-day mortality. The C-statistic of the model showed a very good discriminatory capacity (0.82). Conclusions We present the results of a multivariable prediction model for mortality of critically ill COVID-19 patients admitted to ICU. After adjustment for other factors, age, obesity, procalcitonin, SOFA and PaO2/FiO2 were independently associated with 28-day mortality in critically ill COVID-19 patients. The calibration plot revealed good agreements between the observed and expected probability of death.


Introduction
Coronavirus disease 2019 (COVID-19) is a primarily respiratory tract infection caused by a newly recognized betacoronavirus named SARS-CoV-2, firstly diagnosed in China (Wuhan), in December 2019 [1]. Since then, the outbreak of this infection has spread rapidly across the globe. As of March 15, 2021, 87.994 critically ill patients and 2.668.036 deaths had been reported [2]. The clinical spectrum of COVID-19 ranges from asymptomatic infection to severe respiratory failure [3].
Piacenza is a small city of Northern Italy very close to Codogno, the first city where a COVID-19 patient was identified in Italy. Consequently, the local hospital was quickly changed in a "COVID-19 hospital" [4] to manage a sudden increase in COVID-19 patients requiring hospital admission.
The knowledge of COVID-19 patient characteristics and risk factors associated with intensive care unit (ICU) admission and mortality is still limited. Older age, male sex, comorbidities, lower ratio of arterial partial pressure of oxygen/fraction of inspired oxygen (PaO 2 /FiO 2 ) and higher SOFA score (Sequential Organ Failure Assessment) are independently associated with worse outcome in those admitted to the ICU [5][6][7].
However, only few studies analyzed the clinical characteristics and predictors of mortality in COVID-19 patients admitted to ICU in Italy [8,9]. The definition of risk factors for mortality are mandatory to guide ICU capacity and resource allocation.
The aim of this study was to develop a clinical prediction model for 28-day mortality in critically ill COVID-19 patients admitted to ICU.

Population
This study was approved by the Local Ethics Committee and was conducted at Guglielmo da Saliceto Hospital of Piacenza. We retrospectively analysed a cohort of consecutive critically ill patients admitted to our ICU from Feb 22, 2020 to Apr 3, 2020 diagnosed with SARS-CoV-2 pneumonia, according to WHO interim guidance [10]. All the data were fully anonymised before the access and a random alphanumeric code was used to identify each patient in the database.
COVID-19 infection was diagnosed by a positive result of real-time reverse transcriptasepolymerase chain reaction (RT-PCR) assay of nasal and pharyngeal swabs.
Critically ill patients were defined as those admitted to ICU who required mechanical ventilation or had a fraction of inspired oxygen (FiO 2 ) of at least 60% or more [11]. Pregnant women, children (those younger than 18 years of age) and patients with two negative RT-PCR assay were excluded from the study. Informed consent was collected in only a small amount of patients due to the rapidly worsening of their clinical conditions. The ethics committee allowed this conduct since the early COVID-19pandemic phase has seriously hampered the ability to achieve a traditional informed consent before study enrolment.

Data collection
We reviewed all the electronic medical records to collect demographic, clinical, laboratory and radiologic data from the hospital management software within the first 24 hours of ICU admission. The following laboratory variables were considered: complete blood cells count, C-reactive protein (CRP), creatinine, glucose, total bilirubin and procalcitonin.
Data from high-resolution chest computed tomography (CT) performed within 2 days of ICU admission were collected. CT lung pattern were defined as Patchy Ground-Glass Opacities (GGO), diffuse GGO, mixed consolidation + GGO or consolidation. The presence of bilateral lungs involvement, the visual assessment of lung involved percentage [12] and the presence of lung consolidation were also considered. Visual quantification was used to classify patients as the percentage of lung parenchyma affected by COVID-19 lesions. A radiologist with five years of experience performed the evaluation of each lung CT.
Patients' clinical history including demographic data, medical comorbidities, Covid-19 symptoms duration before hospitalization were also collected. Lung protective ventilatory strategies were adopted and patients were treated according to current guidelines [13].

Statistical analysis
A descriptive statistics was carried out. Continuous variables are reported as median and interquartile range while categorical data as relative number and percentage. Shapiro-Wilk test was used to test normality of distribution. We used the Mann-Whitney U test, χ2 test, or Fisher's exact test to compare differences between survivors and non-survivors.
Potential predictors variables of 28-day mortality were firstly chosen based on their ease measurement during the ICU admission or for their previously showed role as mortality predictor [5,14]. Due to the high clinical and radiological homogeneity of critically ill COVID-19 patients admitted to our ICU during the study period, we decided to use a Cox model to consider time-dependent covariates. A Kaplan-Meier survival estimates were used to evaluate the 28-day survival. The association of risk factors with 28 day-mortality was assessed in univariable and multivariable Cox proportional hazards regression models. The proportional hazard assumption was tested by plotting the Nelson-Aalen cumulative hazard function and Schoenfeld residuals test. A forward regression analysis was used to select variables accepted in the multivariable model. Factors for which p values were less than 0.1 in univariable analysis were used as candidate variables for multivariable approach. The Akaike information criterion was used to compare different regression models and to select the most parsimonious model.
Model performance was assessed via discrimination and calibration measures. To assess for discrimination, the C statistic was used. A calibration curve was implemented by comparing the predicted probabilities and the actually observed proportions, using the Stata module "pmcalplot" [15].
The TRIPOD (transparent reporting of a multivariable model for individual prognosis or diagnosis) guidance was used to conduct this study and to report the results of the prediction model [16].
For internal validation of the model, a non-parametric bootstrap (1000 replications) of the original model was run. The bootstrapped samples were created by drawing random samples with replacement from the development database. The prediction model was fitted on each of bootstrap samples. To adjust for optimism after model development, estimates of a uniform shrinkage factor (the average calibration slope from each of the bootstrap samples) were obtained and multiplied by the original β coefficients to obtain optimism adjusted hazard ratios for each variable [17].
Results are expressed as hazard ratio with 95% confidence intervals (95%CI) and p values. Statistical significance was set at a two tailed P value <0.05. STATA MP, version 16.0 (STATA Corp., Texas, USA) was used for the analysis.

Results
242 patients with a confirmed SARS-CoV-2 infection were admitted to our ICU during the study period and represent the studied population. Two other patients were excluded due to negative RT-PCR findings for SARSCoV-2. The median age of the patients was 64 years (56-71 IQR) and 196 (81%) were male. Almost one comorbidity was present in 147 patients (61%) of which hypertension was the most common (46.7%), followed by diabetes (15.3%) and heart disease (14.5%), ( Table 1). The most common findings at hospital admission were respiratory symptoms and fever (97.5% and 92.1% respectively), followed by gastrointestinal manifestations, mainly vomiting and diarrhea in 18.6%. Eighty-five patients (35.1%) died within 28 days after ICU admission and the median time from ICU admission to death was 11 days (IQR 6-18), (Fig 1).
The comparison of patients characteristics showed a higher prevalence of obesity (defined as BMI of at least 30 kg/m 2 ) in the non-survivors compared to survivors (18.8% vs 9.6%, p = 0.03, respectively). Non-survivor patients were older than survivors, with a median age of 66 (60-73 IQR) years in non-survivors and 62 (55-69) years in survivors (p = 0.0002) ( Table 1). The median time from respiratory symptoms onset to hospital admission was not different in the two groups (7 days, 6-10 IQR in non-survivors vs 7 days, IQR 7-10 days in survivors, p = 0.67) as well as the length of hospital stay prior to ICU admission (4 days, 1-7 IQR in nonsurvivors vs 4 days, 1-6 IQR in survivors, p = 0.69).
At ICU admission, 26.9% of the patients were treated with C-PAP for almost 1 day (IQR 1-3) while 177 (73.1%) required immediate mechanical ventilation and 80 patients (35%) required the use of prone position ventilation. All the patients received dexamethasone 6mg once daily for 7-10 day while only five patients received compassionate-use remdesivir.

3
). The calibration plot revealed good agreements between the observed and expected probability of death (Fig 2).

Discussion
The COVID-19 pandemic is a worldwide novel challenge for critical care systems since it has strongly proved ICU capacities.
In the present study, multivariable Cox proportional hazards regression identified several prognostic markers for 28-day mortality. After adjustment for other factors, age, obesity, procalcitonin, SOFA and PaO 2 /FiO 2 were independently associated with 28-day mortality in critically ill COVID-19 patients.
The majority of our patients (73%) were admitted to the ICU because of acute hypoxemic respiratory failure that required mechanical ventilation. The need for mechanical ventilation among COVID-19 patients admitted to ICUs ranges from 29.1% in one Chinese study [18] to 89.9% in a U.S. study [19] and 88% in an Italian study [8]. In this context, several studies have investigated the factors associated with death or ICU admission but limited information exist in Italian population for the prognostic factors associated with mortality in critically ill COVID-19 patients.
The ICU worldwide mortality for COVID-19 respiratory failure is 25.7% [20]. In Italy, a rate of 25.6% [8] was initially reported but a few months later the same authors reported a mortality rate of 48.7% in a subsequent study [9]. In the current study, the 28-day mortality rate of COVID-19 critically ill patients was 35.1%. This data is close to the average of the two previously mentioned mortality rates and it is similar to what was reported for ARDS [21]. However, as Quah outlined it in a letter to editor, almost a half of patients were still in ICU when previous studies of ICU mortality were published [20]. On the contrary, at the time of writing this paper, no patients were still in ICU as they were deceased or discharged. This element increase the meaning of our data since all the patients concluded the 28-day follow-up. In our study, the patients were mainly middle-aged men with hypertension and the most common initial symptoms were fever and respiratory symptoms such as cough and dyspnea. These findings are in concordance with previously published studies [7,8,22,23]. In fact, the age was independently associated with an increased risk of 28-day mortality in the multivariable analysis. This is not unexpected since this variable was extensively associated with adverse outcome [9,24,25]. Obesity was another important predictor of 28-day mortality (p = 0.04). This finding seems related to the chronic inflammatory state in obese patients [26] that can increase the excessive cytokine response to viral infection leading to adverse outcome [27,28]. As we previously reported, obesity and morbid obesity were risk factors for death in critically ill COVID-19 patients [29].
An increase in procalcitonin levels was reported as an indicator of disease severity in COVID-19 patients [30] and as a risk factor for mortality [31]. Our findings seems in line with these evidences since an increase in procalcitonin levels is associated with increased 28-day mortality rates at multivariable analysis (p = 0.04).
Interestingly, in our patients the presence of diabetes was not associated with an increased mortality risk. This is in contrast with many case series reporting that people with diabetes are at higher risk of COVID-19-related mortality [32] and ICU admission with poor outcome than people without diabetes [33]. However, some factors rarely considered such as type of diabetes, length of the disease and related complications, type of treatment and glycaemic controls during the infection could have affected the outcome.
The lymphocytopenia observed in our patients confirms that they were critically ill COVID-19 patients since a low lymphocyte count is related to the severity of disease [34].
Higher Sequential Organ Failure Assessment (SOFA) score on admission was independently associated with an increased 28-day mortality risk (HR 1.37, 95% CI 1.01-1.05, p = 0.04). In fact, as it was previously published, this score is a highly sensitive marker of inhospital mortality in COVID-19 patients and can be considered as a risk-stratification tool for critical COVID-19 patients [35]. Moreover, a low PaO 2 /FiO 2 ratio at ICU admission was an independent risk factor associated with 28-day mortality. This is not unexpected since acute respiratory failure is the leading feature in critical COVID-19 patients.
Even if male are at higher risk for mortality in the overall population, any difference in mortality rate could not be demonstrated between male and female patients once admitted to the ICU as previously reported by Nachtigall in a large group of critically ill COVID-19 patients hospitalized in Germany [36].
As it was previously published, a lung involvement >50% is associated with ICU admission in hospitalized patients [37]. On the other hand, patients with well-aerated lung parenchyma less than 73% are at increased risk for ICU admission or death [38]. Interestingly, in our patients, the radiological findings at chest CT were approximately the same in survivors and non-survivors and no predictive ability was found. Since the median time from symptom onset to the ICU admission as well as the length of hospital stay prior to ICU admission were similar in survivors and in non-survivors (p = 0.67) it is possible to argue that the observed lung abnormalities are related to the disease time course. Consequently, in our study it is not possible to verify the rule of radiological data to predict mortality.
A high percentage of patients required mechanical ventilation since ICU admission and the principal reason for ICU admission was related to the severity of respiratory failure. We can suppose that disease severity was similar in survivors and in non-survivors. Therefore, due to the characteristics of our sample a validation of the clinical prediction model is needed.
Many prognostication model for patients with COVID-19 were recently developed but a possible risk of bias was outlined due to the limited use of validation techniques [39]. For these reasons, we decided to perform an internal validation to prevent model over-fitting and to obtain a reduction of potential false positive prediction estimates. The bootstrap technique was used since it provides stable estimates with low bias of predictors [40] and it is considered as a central technique to correct overfitting and to quantify optimism in model performance [41].
To our knowledge, this is the first model of 28-day mortality in critically ill COVID-19 patients that was internal validated and it is the third study about critically ill COVID-19 patients requiring ICU admission in Italy. Moreover, our patients were admitted to ICU in a geographic area that is one of the first place where the COVID-19 outbreak spread across Italy.
Our results were obtained from the first phase of the COVID-19 pandemic in Italy and helped us to improve our healthcare organization for the second pandemic phase. However, this model should be confirmed in the subsequent phases of this pandemic.
Nevertheless, this study has several limitations. First, it is a retrospective study during a pandemic that overwhelmed the medical resources. Therefore, it is possible that some data present inaccuracies that might introduce some bias in the study results. Second, we did not collect data about complications during ICU length of stay such as secondary infection, organ failure or thrombosis. Thrombosis and thromboembolism have been reported as a relevant topic in COVID-19 critically ill patients with a prevalence up to 30% but these data were not clearly known at the time we admitted our patients to ICU. The same problem can be applied to Ddimer and IL-6 that we did not collect in every patient. Third, we did not collect the cause of death in our patients with COVID-19 but it is possible to suppose that hypoxemia was the leading cause of death. Forth, no data were collected about type of mechanical ventilation used along with pulmonary compliance and resistance even if two possible respiratory patterns were described [42] and a possible role in mortality rate could be supposed. Another major limit is the absence of an external validation for our model that we hope to perform in a subsequent cohort of COVID-19 critically ill patients.

Conclusions
In this study we developed an internal validated prediction model for 28-day mortality of critically ill COVID-19 patients admitted to ICU with the use of simple and easy to collect clinical variables. Age, obesity, procalcitonin, SOFA score and PaO 2 /FiO 2 ratio emerged as independent predictors for mortality and should be carefully evaluated in these patients. We hope that these data might help a better organization of ICUs for the treatment of COVID-19 patients. Future studies with increased patients number and longer follow-up are needed to confirm our findings.