http://orcid.org/0000-0002-0241-4974
Figures
Abstract
This study aimed to determine the accuracy of the quick Sequential Organ Failure Assessment (qSOFA) score in predicting mortality among prehospital patients with and without infection. This single-center, retrospective, cross-sectional study was conducted among patients who arrived via the emergency medical services (EMS). We calculated the qSOFA score and Modified Early Warning Score (MEWS) from prehospital records. We identified patients as infected if they received intravenous antibiotics at the emergency department or within the first 24 hours. Receiver operating characteristic analysis was used to evaluate and compare the performance of the qSOFA score, each physiological parameter, and the MEWS in predicting admission and in-hospital mortality in patients with and without infection. Multivariate analysis was used to evaluate the qSOFA score and other risk factors. Out of 1574 prehospital patients, 47.1% were admitted and 3.2% died in the hospital. The performance of the qSOFA score in predicting in-hospital mortality in noninfected patients was 0.70, higher than for each parameter and the MEWS. The areas under the curve for the qSOFA+ model vs. the qSOFA- model was 0.77 vs. 0.68 for noninfected patients (p <0.05) and 0.71 vs. 0.68 for infected patients (p = 0.41). The likelihood ratio test comparing the qSOFA- and qSOFA+ groups demonstrated significant improvement for noninfected patients (p <0.01). Multivariate regression analysis for in-hospital mortality demonstrated that the qSOFA score is an independent prognosticator for in-hospital mortality, especially among noninfected patients (odds ratio, 3.60; p <0.01). In conclusion, the prehospital qSOFA score was associated with in-hospital mortality in noninfected patients and may be a beneficial tool for identifying deteriorating patients in the prehospital setting.
Citation: Kitahara O, Nishiyama K, Yamamoto B, Inoue S, Inokuchi S (2018) The prehospital quick SOFA score is associated with in-hospital mortality in noninfected patients: A retrospective, cross-sectional study. PLoS ONE 13(8): e0202111. https://doi.org/10.1371/journal.pone.0202111
Editor: Chiara Lazzeri, Azienda Ospedaliero Universitaria Careggi, ITALY
Received: February 13, 2018; Accepted: July 28, 2018; Published: August 16, 2018
Copyright: © 2018 Kitahara et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Physiological parameters are indicators of the patient’s health condition and are routinely used by emergency medical service (EMS) providers. In the prehospital setting, early identification of high-risk patients is essential to transfer them appropriately and possibly to allow early intervention of emergency department (ED) staff. Scoring systems for vital signs such as the Modified Early Warning Score (MEWS) and National Early Warning Score (NEWS) have been developed for the hospital setting and are currently used in the prehospital setting to identify patients who require intensive care unit (ICU) admission, have adverse in-hospital events, or mortality [1–3].
The quick Sequential Organ Failure Assessment (qSOFA) score is a new tool for identifying critically-ill infected patients outside the ICU [4]. The qSOFA score is based on only three parameters: respiratory rate, systolic blood pressure, and any alteration in mental status. Recently, the qSOFA score was reported to be useful in predicting mortality in ED patients with and without infection [5]. However, few studies have evaluated the utility of the qSOFA score in the prehospital setting.
This study aimed to investigate the accuracy of the qSOFA score in predicting the risk of admission and mortality in prehospital patients with and without infection.
Materials and methods
The study protocol was approved by the institutional review board of Tokai University School of Medicine (#17R-107) and patient consent was exempted because of the retrospective nature. All data were fully anonymized before we analyzed them in this retrospective study.
Study design and participants
This is a single-center, retrospective, cross-sectional study of patients presenting to the ED of Tokai University Hachioji Hospital (a 500-bed, general hospital obtained the stroke and ST-elevated myocardial infarction receiving center) in Tokyo, Japan. The ED serves a population of 0.55 million. All patients who arrived at the ED in ambulances from January 1 to June 30, 2016, were enrolled. We gathered data from prehospital records that documented initial vital signs and hospital electronic medical records. Patients with young age (<18 years), cardiopulmonary arrest, and missing information were excluded. We identified the patients as infected if they received antibiotics for suspicion of infection at the ED or within the first 24 hours.
Measurements
We calculated the qSOFA score and MEWS from prehospital records of blood pressure, heart rate, body temperature, oxygen saturation, and respiratory rate. The components of qSOFA were blood pressure ≤100 mmHg, respiratory rate ≥22 breaths/min, and altered mental status (S1 Table). A Japan Coma Scale score (range: 0–300 points) of <1 (equivalent to a Glasgow Coma Scale score of <15 points) was used in estimating the qSOFA score (Table 1) [6]. The MEWS is based on five basic physiological parameters (the AVPU [alert, voice, pain, unresponsive] score, respiratory rate, heart rate, systolic blood pressure, and body temperature) (S2 Table) [7].
Outcomes
The primary outcome was in-hospital mortality, which was defined as death during the hospital stay as documented in the medical record. The secondary outcome was hospital admission.
Statistical analysis
Baseline characteristics were presented as medians and interquartile ranges for continuous variables and as number of patients for categorical variables. The Kruskal-Wallis test was performed to compare four score groups (qSOFA: 0, 1, 2, and 3) for continuous variables, and the chi-square test was used for categorical variables. To predict in-hospital mortality, each qSOFA score point and the MEWS were assessed using sensitivity, specificity, predictive values, and likelihood ratios. Receiver operating characteristic (ROC) analysis was employed to evaluate and compare the performance of the qSOFA score, each physiological parameter, and the MEWS in predicting the admission and in-hospital mortality of patients with and without infection. Logistic regression models were used to evaluate risk factors (age, sex, qSOFA score) with multivariate analyses. We used two multivariate analysis models and performed the likelihood ratio test and ROC analysis to evaluate these models in the prediction of mortality. The model that included age, sex, and qSOFA score (qSOFA+) as independent variables was compared with the other model that consisted of age and sex without qSOFA score (qSOFA-). A two-tailed p value of <0.05 was considered statistically significant. All statistical analyses were performed using R version 3.4.1 (The R Development Core Team, Vienna, Austria).
Results
Characteristics of study subjects
A total of 1870 patients presented to the ED via the EMS during the study period. Patients who were younger than 18 years (n = 235), had cardiopulmonary arrest (n = 4), and/or had missing values required for calculating the qSOFA score and MEWS (n = 57) were excluded. Thus, 1574 patients were included in the analysis (Fig 1). Among them, 88.6% were patients who were not infected. The median age was 72 years (interquartile range, 55–81 years), and 54.3% were male. For all patients, infected patients, and noninfected patients, the admission rates were 47.1% (n = 741), 92.8% (n = 167) and 41.2% (n = 574), and the in-hospital mortality rates were 3.2% (n = 51), 11.1% (n = 20) and 2.2% (n = 31), respectively. Among all the patients, 821 (52.2%), 625 (39.7%), 113 (7.2%), and 15 (1%) had qSOFA scores of 0, 1, 2, and 3, respectively. The baseline characteristics and primary outcome are summarized in Table 2.
Main results
Table 3 summarizes the sensitivity, specificity, predictive values, and likelihood ratios of predicting mortality across the qSOFA scores and MEWS. The positive likelihood ratio was higher for a qSOFA score of 3 (26.13; 95% confidence interval [CI], 9.85–69.29) than for other qSOFA scores and MEWS.
The results of the ROC analysis for admission in all patients, infected patients, and noninfected patients are shown in S1 Fig. The performance of each score and parameter in all three groups was low (Range of the areas under the curve [AUCs], 0.47–0.60). ROC curves for predicting in-hospital mortality were also evaluated (Fig 2). In all patients, AUCs for qSOFA score vs. MEWS were 0.71 (95% CI, 0.64–0.78) vs. 0.65 (95% CI, 0.57–0.73) (p = 0.09). The respective AUCs were 0.65 (95% CI, 0.52–0.77) vs. 0.56 (95% CI, 0.42–0.69) in infected patients (p = 0.22), and 0.70 (95% CI, 0.61–0.79) vs. 0.62 (95% CI, 0.52–0.72) in noninfected patients (p = 0.06). The AUCs for qSOFA score in the three groups, especially in all patients and noninfected patients, were higher than for each parameter and the MEWS.
(A) qSOFA score vs. each parameter in all patients, noninfected patients, and infected patients; (B) qSOFA score vs. MEWS in the three groups. AUC, the area under the curve; BT, body temperature; HR, heart rate; MEWS, Modified Early Warning Score; qSOFA, quick Sequential Organ Failure Assessment; RR, respiratory rate; SBP, systolic blood pressure.
The AUCs for the qSOFA+ model vs. the qSOFA- model were 0.79 (95% CI, 0.73–0.85) vs. 0.70 (95% CI, 0.64–0.77) in all patients, 0.77 (95% CI, 0.69–0.85) vs. 0.68 (95% CI, 0.60–0.76) in noninfected patients, and 0.71 (95% CI, 0.61–0.82) vs. 0.68 (95% CI, 0.56–0.81) in infected patients (Fig 3). In all patients and noninfected patients, both models were statistically significant (p <0.01 and p <0.05, respectively), suggesting that the qSOFA score increases the AUC, especially in noninfected patients. In the likelihood ratio test comparing qSOFA- and qSOFA+ in the three groups, while the qSOFA+ model did not show significant improvement in infected patients, it indicated a significant improvement in all and noninfected patients (both p <0.01) (Table 4).
The qSOFA+ model was based on the risk factors of age, sex, and qSOFA score. The qSOFA- model was based on age and sex only. AUC, the area under the curve; qSOFA, quick Sequential Organ Failure Assessment.
Table 5 shows the multivariate regression analysis for mortality in the three groups. The odds ratios of qSOFA score were 3.13 (95% CI, 2.19–4.48, p <0.01) in all patients, 3.60 (95% CI, 2.26–5.72, p <0.01) in noninfected patients, and 1.76 (95% CI, 0.99–3.13, p <0.01) in infected patients. These results showed that qSOFA score is an independent prognostic factor for in-hospital mortality, especially in noninfected patients.
Discussion
In this study, we found that the prehospital qSOFA score was associated with in-hospital mortality in noninfected patients compared to the MEWS and physiological parameters.
Although the qSOFA score is a tool for identifying infected patients with high-risk outcomes outside the ICU, few studies have investigated its utility in the prehospital setting. In recent studies, the performance of the qSOFA score in predicting complications was evaluated only in prehospital patients with infection [8–10]. To our knowledge, this study is the first to investigate the prediction of mortality in prehospital patients regardless of the presence of infection.
Here, the performance of the qSOFA score in association with in-hospital mortality was useful in noninfected patients compared with infected patients. In a retrospective study conducted in the ED setting, Singer et al. found that qSOFA scores were associated with in-hospital mortality in patients with and without infection [5]. However, the average age and mortality of the patients in their study were lower than those in our study. Furthermore, they excluded fast-track care, dental, psychiatric, and labor and delivery patients. Similar to our study, it was reported that the predictive ability of the qSOFA score in predicting complications in prehospital patients with infection was not satisfactory [10]. Moreover, in the ED setting, the performance of the qSOFA score had low accuracy in predicting mortality among critically ill septic patients [11]. Considering such findings, the qSOFA score is therefore not sufficient for predicting mortality in infected patients in the prehospital setting.
Early warning scores (EWS) such as the MEWS and NEWS are also tools for identifying patients at risk for critical illness outside the ICU. The MEWS is based on five physiological parameters and used in the United States and Europe [7]. Meanwhile, the NEWS is based on seven components (six vital signs and supplemental oxygen) and used in the United Kingdom [12]. Several studies have evaluated the prediction of mortality in prehospital patients. In a retrospective study by Fullerton et al., the authors showed that the MEWS in the prehospital setting was associated with adverse outcomes [1]. The AUC was 0.799 (95% CI, 0.738–0.856). Silcock et al. also showed that the NEWS in the prehospital setting was associated with mortality [2]. They established that the AUC for 30-day mortality was 0.740 (95% CI, 0.661–0.819). These results suggest that EWS are useful in predicting mortality. However, it may be difficult and complex for EMS staff to calculate EWS in a busy and stressful environment. A previous study showed that errors occurred by calculating EWS manually [13]. In contrast, the qSOFA score is a simple tool that can be quickly calculated without tables and laboratory tests because it comprises only three parameters (systolic blood pressure, alteration in mental status, and respiratory rate). We also conducted an additional multivariate analysis to evaluate risk factors by the addition of other parameters (heart rate and oxygen saturation). The results were consistent with our finding that qSOFA score is an independent prognosticator of in-hospital mortality in all patients and noninfected patients (data not shown).
This study has some limitations. First, it was a single-center, retrospective study, so the results cannot be generalized. Also, our hospital is not a tertiary care hospital; thus, it was not possible to transfer severe sepsis or septic shock patients to our hospital. The mortality rate in a tertiary care hospital in Japan is very high (9.1%) comprising 56% trauma patients; however, it includes only 3% infected patients [14]. Our study population is similar to that of Singer et al., which comprised 18% infected patients with 1.6% in-hospital mortality. Large multicenter, prospective studies are thus required to confirm our results. Second, the AVPU score and MEWS were calculated retrospectively from prehospital records by three ED physicians. In the original study, the qSOFA score was evaluated according to the Glasgow Coma Scale [4]. However, in Japan, EMS staff use the Japan Coma Scale score to evaluate the patient’s level of consciousness. Lastly, we defined patients as infected based on the administration of intravenous antibiotics only, while one study included patients who received both oral and intravenous antibiotics as suspected infected patients [4]. Thus, we might have underestimated the number of infected patients. To refine the identification of infected patients, we distinguished between proven infection (culture positive and/or clinically obvious infection) and suspected infection among 180 infected patients. The number of proven infected patients who were culture “positive” and/or “clinically obvious infected cases” was 120 (66.7%) (data not shown). We also conducted subgroup analysis comprising both patients with proven infection and patients with suspected infection; however, the results were consistent with our main findings (data not shown).
Conclusion
We found that this tool was not sufficient for predicting mortality in infected patients. Nevertheless, the prehospital qSOFA score was more accurate than the MEWS and physiological parameters in predicting in-hospital mortality in noninfected patients. Further multicenter, prospective studies may be required to achieve more accurate results.
Supporting information
S1 Table. Quick Sequential Organ Failure Assessment (qSOFA) score.
https://doi.org/10.1371/journal.pone.0202111.s001
(PDF)
S2 Table. Modified Early Warning Score (MEWS).
https://doi.org/10.1371/journal.pone.0202111.s002
(PDF)
S1 Fig. Receiver operating characteristic curves for admission.
(A) qSOFA score vs. each physiologic parameter in all patients, noninfected patients, and infected patients; (B) qSOFA score vs. MEWS in the three groups. AUC, the area under the curve; BT, body temperature; HR, heart rate; MEWS, Modified Early Warning Score.
https://doi.org/10.1371/journal.pone.0202111.s003
(PDF)
Acknowledgments
Mr. Shogo Morita, Ryota Takanashi, and Keita Suzuki had registered all of the data in the study.
References
- 1. Fullerton JN, Price CL, Silvey NE, Brace SJ, Perkins GD. Is the Modified Early Warning Score (MEWS) superior to clinician judgement in detecting critical illness in the pre-hospital environment? Resuscitation. 2012;83(5):557–62. Epub 2012/01/18. pmid:22248688.
- 2. Silcock DJ, Corfield AR, Gowens PA, Rooney KD. Validation of the National Early Warning Score in the prehospital setting. Resuscitation. 2015;89:31–5. Epub 2015/01/15. pmid:25583148.
- 3. Shaw J, Fothergill RT, Clark S, Moore F. Can the prehospital National Early Warning Score identify patients most at risk from subsequent deterioration? Emergency medicine journal: EMJ. 2017;34(8):533–7. Epub 2017/05/16. pmid:28501815.
- 4. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). Jama. 2016;315(8):762–74. Epub 2016/02/24. pmid:26903335; PubMed Central PMCID: PMCPMC5433435.
- 5. Singer AJ, Ng J, Thode HC Jr., Spiegel R, Weingart S. Quick SOFA Scores Predict Mortality in Adult Emergency Department Patients With and Without Suspected Infection. Annals of emergency medicine. 2017;69(4):475–9. Epub 2017/01/24. pmid:28110990.
- 6. Ohta T, Waga S, Handa W, Saito I, Takeuchi K. [New grading of level of disordered consiousness (author's transl)]. No shinkei geka Neurological surgery. 1974;2(9):623–7. Epub 1974/09/01. pmid:4477641.
- 7. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM: monthly journal of the Association of Physicians. 2001;94(10):521–6. Epub 2001/10/06. pmid:11588210.
- 8. Jouffroy R, Saade A, Carpentier A, Ellouze S, Philippe P, Idialisoa R, et al. Triage of Septic Patients Using qSOFA Criteria at the SAMU Regulation: A Retrospective Analysis. Prehospital emergency care: official journal of the National Association of EMS Physicians and the National Association of State EMS Directors. 2017:1–7. Epub 2017/08/10. pmid:28792256.
- 9. Jouffroy R, Saade A, Ellouze S, Carpentier A, Michaloux M, Carli P, et al. Prehospital triage of septic patients at the SAMU regulation: Comparison of qSOFA, MRST, MEWS and PRESEP scores. The American journal of emergency medicine. 2017. Epub 2017/10/24. pmid:29056391.
- 10. Tusgul S, Carron PN, Yersin B, Calandra T, Dami F. Low sensitivity of qSOFA, SIRS criteria and sepsis definition to identify infected patients at risk of complication in the prehospital setting and at the emergency department triage. Scandinavian journal of trauma, resuscitation and emergency medicine. 2017;25(1):108. Epub 2017/11/05. pmid:29100549.
- 11. Hwang SY, Jo IJ, Lee SU, Lee TR, Yoon H, Cha WC, et al. Low Accuracy of Positive qSOFA Criteria for Predicting 28-Day Mortality in Critically Ill Septic Patients During the Early Period After Emergency Department Presentation. Annals of emergency medicine. 2017. Epub 2017/07/04. pmid:28669551.
- 12.
Royal College of Physicians. National Early Warning Score (NEWS)—Standardising the assessment of acute-illness severity in the NHS. London: RCP; 2012.
- 13. Prytherch DR, Smith GB, Schmidt P, Featherstone PI, Stewart K, Knight D, et al. Calculating early warning scores—a classroom comparison of pen and paper and hand-held computer methods. Resuscitation. 2006;70(2):173–8. Epub 2006/06/30. pmid:16806641.
- 14. Miyamoto K, Shibata N, Nakashima T, Kato S. Prehospital quick sequential organ failure assessment as a tool to predict in-hospital mortality. Am J Emerg Med. 2018 10. (18)30129–3. pmid:29463437.