Development of a multivariate prediction model of intensive care unit transfer or death: A French prospective cohort study of hospitalized COVID-19 patients

Prognostic factors of coronavirus disease 2019 (COVID-19) patients among European population are lacking. Our objective was to identify early prognostic factors upon admission to optimize the management of COVID-19 patients hospitalized in a medical ward. This French single-center prospective cohort study evaluated 152 patients with positive severe acute respiratory syndrome coronavirus 2 real-time reverse transcriptase–polymerase chain reaction assay, hospitalized in the Internal Medicine and Clinical Immunology Department, at Pitié-Salpêtrière’s Hospital, in Paris, France, a tertiary care university hospital. Predictive factors of intensive care unit (ICU) transfer or death at day 14 (D14), of being discharge alive and severe status at D14 (remaining with ventilation, or death) were evaluated in multivariable logistic regression models; models’ performances, including discrimination and calibration, were assessed (C-index, calibration curve, R2, Brier score). A validation was performed on an external sample of 132 patients hospitalized in a French hospital close to Paris, in Aulnay-sous-Bois, Île-de-France. The probability of ICU transfer or death was 32% (47/147) (95% CI 25–40). Older age (OR 2.61, 95% CI 0.96–7.10), poorer respiratory presentation (OR 4.04 per 1-point increment on World Health Organization (WHO) clinical scale, 95% CI 1.76–9.25), higher CRP-level (OR 1.63 per 100mg/L increment, 95% CI 0.98–2.71) and lower lymphocytes count (OR 0.36 per 1000/mm3 increment, 95% CI 0.13–0.99) were associated with an increased risk of ICU requirement or death. A 9-point ordinal scale scoring system defined low (score 0–2), moderate (score 3–5), and high (score 6–8) risk patients, with predicted respectively 2%, 25% and 81% risk of ICU transfer or death at D14. Therefore, in this prospective cohort study of laboratory-confirmed COVID-19 patients hospitalized in a medical ward in France, a simplified scoring system at admission predicted the outcome at D14.


Introduction
In January 2020, the World Health Organization (WHO) declared the outbreak of coronavirus disease 2019  to be a Public Health Emergency of International Concern [1]. This outbreak started in China (Wuhan), from where most of the data is available to now. Clinical presentation varies widely among individuals. Although population-based data are lacking, up to one third of patients might be asymptomatic [2,3]. Among the symptomatic ones, more than 80% develop a mild disease, while only a minority presents the severe form of severe acute respiratory syndrome coronavirus 2 (SARS-CoV2) infection [4]. Intensive care unit (ICU) admissions range from 5% to 16%, depending on characteristics of the studied population [5,6]. Also, Chinese retrospective studies reported an inpatient mortality rate of 17.6-28.2%, with median time to death between 15 and 18.5 days [7,8]. Different prognostic factors emerge in this context, such as age and comorbidities [9,10]. After Asia, Europe was quickly and severely affected by the epidemic. First in Italy then in France, the outbreak rapidly overwhelmed the public health system and ICUs were filled. As of May 12th 2020, France had already confirmed 177.547 cases with 26.646 deaths [11].
Currently, there are no validated treatments for COVID-19 and huge efforts have allowed designing and implementing very rapidly randomized controlled trials. Also, predictive prognostic factors are critical to improve management of high-risk COVID-19 patients. It is crucial to early identify those at risk of worsening for (i) an optimized management of patients' flow and to (ii) to define the population to treat, ensuring healthcare quality [12]. At this time, very limited prospective data is available on outcome and prognostic factors of COVID-19 patients among European population. Our objective through this French single-center prospective cohort study of 152 COVID-19 patients was to develop and validate multivariable predictive models for the patient status at day 14, i.e. (i) major clinical worsening (death or ICU transfer by day 14), (ii) severe status at day 14 (remaining with non-invasive or mechanical ventilation, or death, at day 14), and (iii) favorable hospital outcome (discharge alive by day 14), in adult patients requiring initial hospitalization in a medical ward.

Study population
This is a prospective single-center observational cohort study of 152 COVID-19 adult patients admitted from March 16th 2020 till the 4th of April in the Internal Medicine and Clinical Immunology Department, at Pitié-Salpêtrière's Hospital, in Paris, France, a tertiary care university hospital. Included patients were those older than 18 years with initial requirement for hospitalization in medical ward, and diagnosed with COVID-19, defined as positive SARS-CoV-2 real-time reverse transcriptase-polymerase chain reaction (RT-PCR) assay from nasal swabs. Hospitalization criteria in medical ward was either the need for oxygen support (oxygen mask or non-invasive ventilation, but not mechanical ventilation) with hemodynamic stability, or a high-risk comorbidity profile that would need close follow-up according to emergency room judgement.
All patients benefitted from current standard COVID-19 care at the time. The study followed the Strengthening Reporting of Observational Studies in Epidemiology (STROBE) and the TRIPOD reporting guideline for cohort studies [12] We received local ethical committee approval (Comité d'éthique de la recherche Sorbonne University, CER-2020- 14), and our study is registered as (NCT04320017).
All data were prospectively collected in a standardized form from the medical files of the patients. At baseline (i.e., hospital admission), we assessed demography and epidemiology features, comorbidity profile, previous treatments, clinical presentation along with the laboratory, chest computed tomography (CT) scan and echocardiogram data. Routine blood examinations included full blood count, glycaemia, renal and liver function tests, creatine kinase, lactate dehydrogenase, C-reactive protein (CRP), procalcitonin, fibrinogen, D-dimer, troponin, ferritin and interleukin-6 (IL-6). CT scan imaging results were reported according to the predominant pattern of lesions and the extent of the lesions. The first administered treatments and clinical course during hospitalization were recorded.
Patients were categorized using the WHO clinical improvement Scale [13] on day 1 (D1) and day 14 (D14). This 9-point ordinal scale measures illness severity over time as follows: 0, uninfected; 1, ambulatory, no limitation of activities; 2, ambulatory, limitation of activities; 3, hospitalized, no oxygen therapy; 4, hospitalized, oxygen by mask or nasal prongs; 5, hospitalized, oxygen by non-invasive ventilation or high-flow; 6, intubation and mechanical ventilation; 7, ventilation with additional organ support (i.e., vasopressors, dialysis, extracorporeal membrane oxygenation); 8, death. All data were collected and reviewed by three physicians (AH, GM and MV). Patients discharged from hospital before D14 were contacted by phone to assess their status at that time point.
Eligibility criteria for the validation cohort was the same used for the development cohort, being carried out in another hospital close to Paris, in Aulnay-sous-Bois, Île-de-France. The outcome was defined and assessed in a similar way to that of development cohort. Data were collected from medical hospitalization records, which included the date of admission and, as appropriate, date of hospital discharge, date of ICU transfer, date of ICU discharge, date of invasive ventilation initiation and withdrawal, date of death. From those dates, outcomes at day 14 of admission were derived, as defined for the analyses.
Non opposition to participate was obtained from each participant, and a dated non opposition form was collected and included in their medical hospitalization records, following French legislation for observational studies on standard of care data.

Definitions of study endpoints
The study endpoints were defined as the occurrence of ICU transfer or death within 14 days of admission (main endpoint), the need for non invasive or mechanical ventilation, or death, at day 14 after hospital admission, and being discharged alive within 14 days of admission.

Statistical analysis
The sample size (number of individuals, n = 152) consisted in all consecutive eligible patients hospitalized at the study center, during the first weeks of the 2020 SARS-CoV2 outbreak in Paris, France. For descriptive analyses, categorical variables are reported with counts (percent) and quantitative variables with median [interquartile range]. The association between groups and variables was evaluated using Fisher's exact test for categorical variables, and with Wilcoxon's rank sum test for quantitative variables. Categorical variables were compared using Fisher's exact test and quantitative variable with Wilcoxon's rank sum test. Analyses were performed on complete cases. Quantitative predictors were considered as continuous variables (except for age) and qualitative as binary or dummy variables, for model development. A set of predictors was defined after checking for redundancy among candidate predictors based on clinical expertise, as well as and multicollinearity, and accounting for an acceptable number of degrees of freedom given the limiting number of events. We considered predictors that would be available in most medical wards, in routine practice, representing patients status at baseline, both clinically and biologically. The predictor variables used were age, CRP level, lymphocyte count, and respiratory presentation presented as WHO score. These data are measured at the initial presentation of the patient. Poor respiratory presentation is defined as WHO score equal or superior to 5, oxygen by NVI or high flow oxygen (more than 6 L/min). No statisticalbased variable selection was performed. The multivariable models of the endpoints of interest were evaluated using logistic regression models, with maximum likelihood. Validation was performed in two stages. Internal validation of the models was first performed using 1000 bootstrap resamples [14]; we estimated models performances, corrected for over-optimism (see S1 File). The models were further evaluated on an external validation sample from another French hospital close to Paris, in Aulnay-sous-Bois, Île-de-France (see S1 Table in S1 File). We defined a tentative simplified scoring system, for the main endpoint (ICU transfer or death within 14 days of admission); to that aim, continuous variables were to be dichotomized (for simplified field risk-assessment) and a unit coefficient was allocated to each of the model variables (see S1 File). The simplified score was validated internally using a resampling approach by bootstrap (number of bootstrap sample, N = 1000), and on the external cohort. For each variable, missing data was described with count. For model development, we used routinely obtained predictors (no missing data). All statistical tests were two-sided at a 5%-significance level. Analyses were performed on R statistical platform, version 3.5.3.

Results
A total of 152 consecutive eligible patients were hospitalized in the ward and included in the study. The main baseline features are presented in Table 1. Median age was 77 years [60-83], male sex and Caucasian origin were predominant, and 80.9% of the patients had comorbidities. By the time of arrival, 28 (18.4%) patients reported angiotensin-converting-enzyme inhibitors as continuous-use medication, while 16 (10.5%) had taken nonsteroidal antiinflammatory drugs. Dyspnea was the most frequently symptom, followed by fever and dry cough. On admission, 44 patients (28.9%) had a WHO score of 3, 89 patients (58.6%) had a WHO score of 4, and 19 patients (12.5%) had a score of 5. Half of the patients presented with lymphopenia, with values below 800 cells/mm 3 . Chest CT scan showed that ground glass opacities were the most frequent lesions with an extent greater than 50% of the parenchyma evidenced in 24.7% of patients. IL-6 level was 31.8 pg/mL [14.8-56.0] and higher levels (161.1 pg/ mL [32.7-237.8]) were observed in patients with extensive lung opacities (> 50%) as compared to those with a non-extensive lung involvement (31.7 pg/mL [15.4-51.6], p = 0.022). At admission, 129 (84.9%) patients received antibiotics, 68 (45%) hydroxychloroquine and 6 (3.9%) tocilizumab.
In univariable analysis, age at admission, chronic respiratory failure, respiratory rate � 24 breaths per minute, peripheral capillary oxygen saturation (SpO2) on room air, oxygen therapy on admission, SpO2 on oxygen, dyspnea, myalgia, WHO clinical scale, neutrophilia, eosinopenia, lymphopenia, CRP level, IL-6 level, procalcitonin, fibrinogen, serum ferritin, highsensitivity cardiac troponin T, lactate dehydrogenase (LDH), D-dimer, and chest CT scan were associated with ICU transfer and/or death within 14 days (Table 1). For adjusted model development, the limiting number of events was 47 patients with ICU transfer or death within 14 days in the original sample. The multivariable model included age (� or > 60 years), respiratory baseline presentation (assessed by WHO scale levels from 3 to 5), CRP level and lymphocytes count. Older age (OR 2.61, 95% CI 0.96-7.10), poorer respiratory presentation (OR 4.04 per 1-point increment on WHO scale, 95% CI 1.76-9.25) and higher CRP level (OR 1.63 per 100mg/L increment, 95% CI 0.98-2.71) were associated with an increased risk of ICU requirement or death, while lymphocytes count were associated with better outcome (OR 0.36 per 1000/mm 3 increment, 95% CI 0.13-0.99) (Fig 2, S2 Table in S1 File). Fig 2 shows a forest plot of the multivariable models of COVID-19 patient's outcomes. Internal and external validation of the model was performed: the C-index (equivalent to AUC) was 0.80, 0.78 after correction for over-optimism by resampling, and 0.78 on the external cohort (see S1 File for further details and S1 Table in S1 File for description of the external cohort).

PLOS ONE
A tentative simplified scoring system was defined for the main endpoint (ICU transfer or death within 14 days of admission), for routine clinical field practice. To that aim, based on the linear predictor and the coefficients of the multivariable model, in an additive manner, 1 point was allocated for age above 60 years old; 1 point for oxygen therapy by nasal prongs or mask (WHO scale level 4); 3 points for high flow oxygen or NIV (WHO scale level 5); 1 point if 10 � CRP plasma level � 75 mg/L, 2 points if 75 � CRP � 150 mg/L, 3 points if CRP � 150 mg/L; 1 point if lymphocytes count below 800/mm3 (See S1 File). Fig 3 displays stratified risk according to each score. Therefore, we defined three risk groups: low (score 0-2), moderate (score 3-5), and high (score 6-8). Cumulative incidence for each of these groups is shown in Fig 4. Overall, the estimated sensitivity of a score greater than 2 (moderate and severe risk groups) was 97% (95% CI 94-100), and the specificity of a score lower than 6 (low and moderate risk groups) was 94% (95% CI 89-98) for the main outcome. The positive predictive value for a high-risk score was 76% (95% CI 61-91), while the negative predictive value for a low risk score was 94% (95% CI 82-100).
At day 14, a total of 40 patients were still treated with NIV (n = 1) or MV (n = 7) ventilation, or had died (n = 32), out of 146 evaluable patients. In univariable analysis, age at admission, weight, chronic respiratory failure, respiratory rate � 24, SpO2 on room air, Oxygen therapy on admission, SpO2 on oxygen, dyspnea, myalgia, WHO clinical scale, neutrophils, eosinophils, lymphocytes, platelets, CRP level, IL-6 level, procalcitonin, serum ferritin, high-sensitivity cardiac troponin T, D-dimer, and chest CT-scan were associated with WHO scale � 5 within day 14. Multivariable analysis is represented in Fig 2. Eighty-four patients had been discharged by day 14, out of 146 evaluable patients. In univariable analysis, age at admission, respiratory rate < 24, SpO2 on room air, Oxygen therapy on admission, ageusia, dyspnea, WHO clinical scale, neutrophils, eosinophils, lymphocytes, platelets, CRP level, IL-6 level, procalcitonin, fibrinogen, serum ferritin, high-sensitivity cardiac troponin T, LDH, D-dimer, and chest CT scan were associated with discharge alive within 14 days. Multivariable analysis is represented in Fig 2.

Discussion
The natural history and outcome of the COVID-19 patients initially hospitalized in a medical ward remain unpredictable. Currently, the main existing medical information stem from China and prognostic factors of COVID-19 among European population are lacking. The most striking conclusions drawn by this study are (i) up to 35% of the COVID-19 patients hospitalized in a medical ward were transferred to ICU or died at day 14, (ii) we defined high-risk group of ICU transfer or death using a simplified scoring system from the multivariable models including age, CRP level, lymphocytes count and WHO scale and (iii) we highlighted correlation between IL-6 level and extensive lesions in CT scan.
A clear and strong age gradient in death risk has been identified, increasing dramatically after 60 years [15]. Besides older age, comorbidities are also highlighted as key factors associated with death [7,8,16]. Compared to the present study, retrospective Chinese cohorts population were younger (from 51 to 56 years) and had less comorbidities (up to 48%) [7,16]. Even with a median age of 77 years and more than 80% of comorbidity, our reported 21.9% mortality rate lies within the 17.6-28.2% range extracted from other cohorts [7,8]. In contrast, the median time from symptoms onset to death in our population (11 days) is shorter than the 18.5 days previously reported [7], which can be ultimately the consequence of the higher risk profile of patients in the present study. Additionally, our ICU transfer rate (11.6%) was lower than the 26% described in Chinese cohorts [7,16]. In this regard, we must underline that our patients presented with less severe infection at baseline [7,16]. In addition, they were less eligible to ICU admission, due to age and comorbidities. Beyond demographic and clinical characteristics, several laboratory features have been linked to a higher mortality. Studies identified a positive correlation with mortality for neutrophilia, lymphopenia, troponin, LDH and Ddimer levels [7,16]. Additionally, high levels of serum CRP, procalcitonin, and ferritin have also occasionally been associated with mortality [16,17]. In our cohort, two simple biomarkers from routine practice, lymphocytes count and CRP level, are independently associated with a worse prognosis. CRP level higher than 75 mg/L and lymphopenia below 800/mm 3 increased by two fold the odds of being transfer in ICU or death.
Herein, we provided for the first time a simplified scoring system which allows stratifying COVID-19 patients initially hospitalized in a medical ward, at low, intermediate, or high risk of ICU transfer or death. The score was validated with calibration evaluated both with an internal resampling approach and by external validation on a cohort sample from a different hospital. Based on the linear predictor of the multivariate model, age above 60 years, WHO scale, CRP level (10-75, 75-150, or > 150 mg/L), and lymphocytes count below 800/mm3 were included in the scoring system. A score equal or greater than 6 at baseline had a predicted probability of more than 60% to be transferred to ICU or dead by D14. In our regard, this high-risk patient profile should be monitored more closely and eventually considered for more aggressive treatment protocols than a patient with a score of less than 3. In a systematic review of the prediction models for diagnosis and prognosis of COVID 19 patients, Wynants et al identified ten prognostic models proposed by different Chinese teams [12]. By the time of this article writing, all these models were only available in pre-print and had not been peerreviewed. They were exclusively based on small retrospectives cohorts, with most of them lacking an external validation cohort, or presenting a non-comparable small validation cohort. Nguyen et al [18] developed a 7-point score based on a retrospective analysis of 279 hospitalized patients but without external validation. The strengths of the score presented here are its prospective nature and its external validation. In addition, its readily accessible variables make it easily reproducible in clinical practice.
Apart from CRP level and lymphocyte count, other significant findings from our study could be further used to refine the score. Chest CT scan is a useful diagnostic tool, especially for RT-PCR negative patients, but its role as a prognostic instrument is still unclear [19]. Herein, we pointed out that parenchymal involvement greater than 50% on chest CT scan at admission was associated with ICU transfer or death in 41% of cases. In parallel, high levels of serum IL-6 have been reported in moderate to severe cases of COVID-19 pneumonia [7,17]. IL-6 may result in increased alveolar-capillary blood-gas exchange dysfunction, especially impaired oxygen diffusion, and lead to pulmonary fibrosis and organ failure [20]. We were able to establish for the first time the correlation between IL-6 level and extensive parenchymal involvement on chest CT scan for ICU transfer or death.
Our study has some limitations. We presented models with both internal and external validation. Discrimination of the model and of the simplified score for the main endpoint was consistent in the external cohort. Calibration assessment showed a slightly overestimated risk of event in the external cohort for those with higher scores. The external sample consisted of patients from a regional non-university hospital, which could explain the differences on catchment area and patient recruitment. In the acute context of the first SARS-CoV-2 epidemic wave in France, we relied on a sample prospectively defined by consecutive eligible patients in the study center. Overall, the limited sample sizes of both development and validation samples require caution in interpreting results. Ideally, a sample size calculation at planning stage of the study should ensure sufficient collected data for predictive model development and validation; approaches have been proposed to that aim [21,22]. Further external validation on larger prospective cohorts with planned sample sizes will be useful.
To our knowledge, this is the first prospective European cohort of COVID-19 non-critical inpatients and one of the largest standardized studies describing short term patients outcome. We provided a very simple and easily accessible score to estimate the risk of ICU transfer or death by day 14. In the context of the pandemic, this tool can help the management of patient flow, and also clinical trial design and therapeutic management.