https://orcid.org/0000-0003-0059-7735
Figures
Abstract
As SARS-CoV-2 infections continue to cause hospital admissions around the world, there is a continued need to accurately assess those at highest risk of death to guide resource use and clinical management. The ISARIC 4C mortality score provides mortality risk prediction at admission to hospital based on demographic and physiological parameters. Here we evaluate dynamic use of the 4C score at different points following admission. Score components were extracted for 6,373 patients admitted to Barts Health NHS Trust hospitals between 1st August 2020 and 19th July 2021 and total score calculated every 48 hours for 28 days. Area under the receiver operating characteristic (AUC) statistics were used to evaluate discrimination of the score at admission and subsequent inpatient days. Patients who were still in hospital at day 6 were more likely to die if they had a higher score at day 6 than others also still in hospital who had the same score at admission. Discrimination of dynamic scoring in those still in hospital was superior with the area under the curve 0.71 (95% CI 0.69–0.74) at admission and 0.82 (0.80–0.85) by day 8. Clinically useful changes in the dynamic parts of the score are unlikely to be associated with subject-level measurements. Dynamic use of the ISARIC 4C score is likely to provide accurate and timely information on mortality risk during a patient’s hospital admission.
Citation: Crocker-Buque T, Myles J, Brentnall A, Gabe R, Duffy S, Williams S, et al. (2022) Using ISARIC 4C mortality score to predict dynamic changes in mortality risk in COVID-19 patients during hospital admission. PLoS ONE 17(10): e0274158. https://doi.org/10.1371/journal.pone.0274158
Editor: Jordi Camps, Hospital Universitat de Sant Joan, Universitat Rovira I Virgili, SPAIN
Received: May 11, 2022; Accepted: August 23, 2022; Published: October 12, 2022
Copyright: © 2022 Crocker-Buque et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because they are health services management data. Data are available from the Barts Health NHS Trust Institutional Data Access (Data Warehouse). The data for this are held within the Barts Health NHS Trust Data Warehouse and were analysed by a team at Queen Mary University of London using an existing data sharing agreement between the two institutions. The data are contained within the Trusts wider COVID-19 patient database that is used for analysis and evaluation of service delivery of hospital services. We do not have permission from either Barts Health or the Queen Mary University of London Joint Research Management Office to place the data in a public repository, however the data can be made available to researchers looking to further evaluate and validate our findings by request to the Joint Research Management Office at research.governance@qmul.ac.uk.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The global pandemic caused by severe acute respiratory coronavirus 2 (SARS-CoV-2) infection and resulting COVID-19 disease continues, including in the UK, where an average of around 1,000 patients per day were admitted with COVID-19 during August and September 2021, with many more seen and discharged from emergency departments [1]. The mean length of admission resulting for a COVID-19 infection in the UK National Health Service (NHS) is 8–9 days (compared to 1 day for an acute medical admission), during which time patients may deteriorate to require critical care, resulting in a longer admission (15–16 days), while others will improve and be discharged [2]. This combination is likely to put increasing pressure on health services through winter months particularly as population based public health measures are reduced [3].
The ISARIC 4C mortality score provides an indication of mortality risk at admission based on demographic and physiological parameters, derived from a national level population cohort study in the UK [4,5]. Components and score weights are shown in Table 1. Total scores were grouped into low risk (score 0–3, mortality rate 1.2%), intermediate risk (score 4–8, 9.9% mortality), high risk (score 9–14, 31.4% mortality), and very high risk (score ≥15, mortality 61.5%).
The score provides useful, early information enabling clinicians to make decisions around treatment requirements and safe discharge. Barts Health National Health Service Trust (BHNHST) provides health services through 5 hospital sites across east London to a demographically and socio-economically diverse population of more than 2.5 million people, and we have previously evaluated the score in our local population [6]. However, although the score provides useful information on admission, there is a need to better understand how a patient’s risk evolves during a hospital stay to support health services planning. We also sought to identify whether there was a point during patients with long admissions where repeating the scoring would be clinically useful, as currently it is only validated for use at admission to hospital. Therefore, our aim was to study the dynamic use of the ISARIC 4C mortality score at different points following a patient’s admission with COVID-19 to evaluate whether additional information on mortality risk can be established through admission.
Methods
Details of the construction of the dataset have previously been published [6]. In summary, a range of demographic, prognostic and clinical factors were extracted from the BHNHST Electronic Health Record System (EHR) for all patients with a laboratory confirmed reverse transcription polymerase chain reaction (RT-PCR) positive swab result for SARS-CoV-2 from any anatomical site. All patients aged 18 years or more admitted to three hospital sites from BHNHST and with a positive test recorded up to 7 days before, or 7 days after their first admission were included. The resulting cohort for this study included 6,373 patients admitted between 1st August 2020 and 19th July 2021. The primary outcome measure was death, defined as all-cause mortality within 28-days of admission and all patients were followed-up for at least 28 days.
Statistical methods
Characteristics included in the ISARIC 4C mortality score (4C) were extracted for each included patient for the duration of their admission as described in the paper by Knight et al. [4]. We calculated a dynamic version of 4C denoted (4CD) every 48 hours for 28 days by applying the same score at different time points using the following method: for a patient alive and not in ICU at a time (t) hours after admission, we considered all measurements of the time-dependent variables in 4CD (respiratory rate, oxygen saturation (%), Glasgow Coma Scale (GCS) score, urea (mmol/L), C-Reactive Protein (mmol/L; CRP), lymphocyte count (n x 10^9/L) taken during the interval (t-48,t hours). If no measurements of a given variable were taken during this interval, we found the most recent 48-hour interval during which measurements on that variable were taken and used that interval. If there was no recorded measurement of a score components, we assumed no increased risk for that component. Where multiple recordings of a variable were made, we calculated and used the mean value. These values were incorporated into the 4CD score calculator along with the non-time dependent variables (age, sex, number of comorbidities) for each patient at each time-point. Receiver operating characteristic (ROC) curves were plotted to compare sensitivity and specificity of the 4C at admission and 4CD at relevant time points. AUC statistics were based on these with 95% DeLong confidence interval. Furthermore, in all patients still alive, in hospital, and not in ICU at day 8 we analysed the extent to which modifiable parameters (respiratory rate, oxygen saturation, GCS, urea and CRP) contributed to the change in score. The score change was calculated by the difference in 4CD between day 8 and entry alongside the mean change in 5 time-dependent variables (respiratory rate, urea, oxygen saturations, GCS and CRP). 95% CIs for mean changes in these were estimated a non-parametric bootstrap (5000 resamples). The distribution of 4CD factors at admission and day 8 was tabulated based on risk groups at day 8. A sensitivity analysis was used to do the same analysis only including those with complete data for all components at both time points. Cutpoints for continuous variables were based on the 4C definitions. Cutpoints based on 4C scores were based on recommendations from ISARIC. We have presented examples of score change over time for the individual score with the most number of patients within the intermediate, high and very high score groups.
Linear regression and Wald tests were used to investigate the association between age, sex and any comorbidities on change from baseline to day 8 (in those still in hospital) in inflammation components of 4CD (score due to GCS, Urea, and CRP), or respiratory components (score due to oxygen saturation and respiratory rate). All statistical analyses were undertaken in R version 4.1.1.
Ethical approval
The study proposal was submitted the Joint Research Management Office at Queen Mary, University of London, who reviewed the methods and determined that as it is an evaluation of routinely collected hospital data it did not require ethics committee review and approval. The study was registered as a health services evaluation with the Barts Health NHS Trust Clinical Effectiveness Unit (reference 11121). Individual patient consent was not required or sought.
Results
Baseline characteristics of the 6,373 included patients are shown in Table 2. Unlike many cohorts of patients in hospital with SARS-CoV2 infection the cohort had good representation from all major ethnic groups.
To explore the change in score between survivors and decedents we created graphics for score changes during admission for each total score value. Here we present figures for the the score with the largest number of patients in the intermediate (score = 5), high (score = 10) and very high (score = 15) score groups to illustrate how the underlying score total changes during admission in patients with different risk profiles. Fig 1 considers the distribution of 4CD in patients with a score of 10 (n = 1,937) at admission and still in hospital over time, by subsequent mortality status. Equivalent figures for scores of 5 (n = 1,079) and 15 (n = 438) are presented in the (S1 and S2 Figs). These demonstrate that it is possible to better assess prognosis in groups of patients with identical scores at entry, by dynamically updating 4CD.
S1 Table gives statistics on baseline characteristics of these sub-cohorts. The main factors driving the higher scores were older age, being male, having a greater number of comorbidities and being more likely to be confused (GCS<15). The highest baseline score group showed much greater concentrations of urea and were more likely to have high CRP levels. However, there was very little variation in respiratory rate or oxygen saturation between the groups.
We undertook further analysis in patients at still in hospital 8 days after admission. Fig 2 shows distinct patterns in survival for patients classified at day 8 using 4CD into low (0–3), intermediate (4–8), high (9–14) and very high (15+) risk groups, with high mortality in the very high-risk group, and moderate mortality over a longer period in the high group. The ROC obtained by classifying patients using rules based on the 4CD at day 8 and the 4C score at admission is shown in the (S3 Fig). The AUC obtained with the 4CD is 0.823 (95% CI 0.800–0.845) and that with the 4C score at admission is 0.713 (95% CI (0.686–0.740), S3 Fig is the equivalent plot with 4CD calculated at day 16, again showing a much higher AUC than using the admission score alone.
This pattern of superior risk assessment using up to date information and dynamic application of 4CD was also observed at all time points after admission (Table 3).
In further analysis to identify which time-dependent components contributed most to the change in score changes in Urea and CRP showed evidence to have been larger contributors to change, with a lesser contribution from respiratory rate (Table 4).
A comparison of the distribution of 4C components at day 8 and entry by score band at day 8 is shown in Table 5. Those in the highest risk group at day 8 were predominantly aged 80 years or more, and male. As a group they had seen a worsening in their prognosis associated with Urea and CRP, but less change in respiratory rate, and a generally improved profile for oxygen saturation (which is likely partly due to receiving oxygen in hospital). In contrast, the lower risk groups had all a better profile at day 8 in most of the 4CD components, except oxygen saturation. Findings were unchanged in a complete case analysis (S2 Table).
Risk groups: Low (0–3), intermediate (4–8), high (9–14) and very high (15+).
Finally, we did not find any evidence that clinically useful changes in the dynamic parts of the score were associated with subject-level measurements. Specifically, age, sex and number of co-morbidities were not associated with changes to CRP, urea or GCS. We observed a slight association of age with respiratory score change (p<0.001), however, this was too small to be of clinical utility (increase in age by 10 years associated with a respiratory score change of approximately 0.09).
Discussion
Due to the continuing pressure COVID-19 is putting on health services there remains a need for accurate clinical risk scoring to enable appropriate use of resources, as well as to identify and intervene early in deteriorating patients [7,8]. The ISARIC 4C score has been shown to provide accurate mortality risk predictions in COVID-19 patients at admission, but here we demonstrate that repeated, dynamic application of the score increases accurate mortality prediction as risk changes during hospital admission. Repeating the ISARIC 4C score at any further day after admission increases the sensitivity and specificity of the score in identifying those with the highest risk of dying. This supports previous evidence that dynamic scoring during hospital admission involving biochemical parameters can increase accuracy in predicting mortality [9]. Repeated scores may be exploited to change management of the individual patient in response to change or failure to change in the score. However, the area under the curve using dynamic scoring remains 0.8, which is adequate for a test of this kind, but leaves residual risk of false positive or false negative categorisation. The risk of falsely scoring a patient as being lower risk could result in false reassurance. Therefore, this score should only be used as one tool within a full clinical risk assessment of an admitted patient.
We have demonstrated that the change in score is being driven by increasing CRP and urea, with associated worsening respiratory and neurological function. This supports the wider evidence that increasing systemic inflammatory response is associated with worse outcomes for patients [10]. Urea also forms a component of the CURB-65 clinical risk core for community acquired pneumonia and is known to be associated with worsening fluid status and renal dysfunction [11]. Existing inpatient systems to monitor patients for deterioration, such as the National Early Warning Score 2 (NEWS2) only include vital sign measurements, which cover oxygen saturation, respiratory rate, and GCS, along with oxygen use, pulse and blood pressure [12]. However, NEWS2 doesn’t include any biochemical measurements, nor age or sex, which are known to be significant contributors to COVID-19 mortality [13]. Other studies have added age to NEWS parameters to improve accuracy, but these are lower than the dynamic application of ISARIC 4C described here [14].
These results have been derived from a large, ethnically, and socio-economically diverse population of hospital in-patients, with a wide range of underlying co-morbidities. The window for included patients covers the introduction of core therapeutic interventions, such the use of the anti-viral remdesivir, systemic steroids (dexamethasone), interleukin-6 receptor blockade (tocilizumab/sarilumab), REGEN-COV (casirivimab and imdevimab) amongst others as identified through trials such as the RECOVERY trial [15]. It also includes patients infected with both alpha and delta SARS-CoV-2 variant viruses [16]. It includes a cohort of patients prior to widespread vaccination being introduced in the UK (from December 2020), those partially and completely vaccinated between December and July 2021 [17]. Vaccinated patients are less likely to suffer from severe disease, therefore, it is likely that ISARIC 4C will remain of most relevance in patients who are unvaccinated and more likely to have high levels of viral replication and subsequent inflammatory response [18]. The ISARIC 4C score already features as a recommended tool for risk assessment when prescribing Remdesivir, 4C score ≥4 (those with a lower score 0–3 being likely to recover without treatment), however there is potential future scope for the ISARIC 4C score to assist in guiding therapeutic decisions for other treatments [19].
Limitations
Limitations include lack of inclusion of prescribing and therapeutic data within the dataset, which cannot be factored into the appropriateness of using the score. We only included parameters within the ISARIC 4C model and did not consider other factors which may contribute to risk. Included patients were from a defined geographic area and findings may not be generalisable to other contexts. We did not have information on the vaccine status of patients included in this study.
Supporting information
S1 Fig. Showing mean and range of ISARIC 4C scores at 48-hour intervals amongst survivors (turquoise) and decedents (red) with a score of 5 at admission.
https://doi.org/10.1371/journal.pone.0274158.s001
(DOCX)
S2 Fig. Showing mean and range of ISARIC 4C scores at 48-hour intervals amongst survivors (turquoise) and decedents (red) with a score of 15 at admission.
https://doi.org/10.1371/journal.pone.0274158.s002
(DOCX)
S3 Fig. Receiver operating characteristic curves comparing sensitivity and specificity of mortality risk using 4C score at admission (green) and at day 16 (blue).
https://doi.org/10.1371/journal.pone.0274158.s003
(DOCX)
S1 Table. Baseline characteristics on score components at admission in patient grouped by admission 4C scores of 5, 10 and 15.
https://doi.org/10.1371/journal.pone.0274158.s004
(DOCX)
S2 Table. 4CD components (n, %) in patients with low (0–3), intermediate (4–8), high (9–14) and very high (15+) scores at day 8, for those with complete data at both time points in all measurement.
https://doi.org/10.1371/journal.pone.0274158.s005
(DOCX)
References
- 1.
GOV.UK. Coronavirus in the UK. Accessed 08/02/2022. Available from: https://coronavirus.data.gov.uk/details/healthcare.
- 2. Vekaria B, Overton C, Wiśniowski A, Ahmad S, Aparicio-Castro A, Curran-Sebastian J, et al. Hospital length of stay for COVID-19 patients: Data-driven methods for forward planning. BMC Infect Dis. 2021;21(1):700. pmid:34294037
- 3. Mahase E. Covid-19: Experts condemn UK “freedom day” as dangerous and unethical. BMJ. 2021;374: n1829. pmid:34281906
- 4. Knight SR, Ho A, Pius R, Buchan I, Carson G, Drake TM, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339. pmid:32907855
- 5. Gupta RK, Harrison EM, Ho A, Docherty AB, Knight SR, van Smeden M, et al. Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study. Lancet Respir Med. 2021;9(4): 349–359. pmid:33444539
- 6. Crocker-Buque T, Williams S, Brentnal AR, Gabe R, Duffy S, Prowle JR, et al. The Barts Health NHS Trust COVID-19 cohort: Characteristics, outcomes and risk scoring of patients in East London. Int J Tuberc Lung Dis. 2021;25(5):358–366. pmid:33977903
- 7. Coughlan C, Rahman S, Honeyford K, Costelloe CE. Developing useful early warning and prognostic scores for COVID-19. Postgraduate Medical Journal. 2021;97:477–480. pmid:34049992
- 8. Wynants L, Sotgiu G. Improving clinical management of COVID-19: the role of prediction models. The Lancet Respiratory Medicine. 2021;9:20–321. pmid:33444541
- 9. Berzuini C, Hannan C, King A, Vail A, O’Leary C, Brough D, et al. Value of dynamic clinical and biomarker data for mortality risk prediction in COVID-19: a multicentre retrospective cohort study. BMJ Open. 2020;10(9):e041983. pmid:32967887
- 10. Ji P, Zhu J, Zhong Z, Li H, Pang J, Li B, et al. Association of elevated inflammatory markers and severe COVID-19: A meta-analysis. Medicine. 2020;99(47):e23315. pmid:33217868
- 11. Capelastegui A, España PP, Quintana JM, Areitio I, Gorordo I, Egurrola M, et al. Validation of a predictive rule for the management of community-acquired pneumonia. Eur Respir J. 2006;27(1):151–157. pmid:16387948
- 12.
RCP London. National Early Warning Score (NEWS) 2. Accessed 20/09/2021. Available: https://www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news-2.
- 13. Williamson EJ, Walker AJ, Bhaskaran K, Bacon S, Bates C, Morton CE, et al. OpenSAFELY: factors associated with COVID-19 death in 17 million patients. Nature. 2020;584(7821):430–436.
- 14. Maves RC, Richard SA, Lindholm DA, Epsi N, Larson DT, Conlon C, et al. Predictive Value of an Age-Based Modification of the National Early Warning System in Hospitalized Patients with COVID-19. Open Forum Infect Dis. 2020;ofoab421.
- 15.
RECOVERY Trial. Results. Accessed 20/09/2021. Available from: https://www.recoverytrial.net/results.
- 16.
GOV.UK. COVID-19 variants: genomically confirmed case numbers. Accessed 20/09/2021. Available from: https://www.gov.uk/government/publications/covid-19-variants-genomically-confirmed-case-numbers.
- 17.
GOV.UK. COVID-19: the green book, chapter 14a. Accessed 20/09/2021. Available from: https://www.gov.uk/government/publications/covid-19-the-green-book-chapter-14a.
- 18.
Public Health England. Direct and indirect impact of the vaccination programme on COVID-19 infections and mortality. Accessed 08/02/2021. Available from: https://www.gov.uk/government/publications/phe-monitoring-of-the-effectiveness-of-covid-19-vaccination.
- 19.
Department of Health and Socal Care. Interim Clinical Commissioning Policy: Remdesivir for patients hospitalised with COVID-19 (adults and children 12 years and older). Accessed 08/02/2022. Available: https://www.england.nhs.uk/coronavirus/wp-content/uploads/sites/52/2020/07/C0654-uk-interim-clinical-commissioning-policy-remdesivir-for-patients-hospitalised-with-covid-19-adults-and-c.pdf.