https://orcid.org/0000-0002-3478-6967
Figures
Abstract
We evaluated the performance of cardiac arrest-specific prognostication scores developed for outcome prediction in the early hours after out-of-hospital cardiac arrest (OHCA) in predicting long-term outcomes using independent data. The following scores were calculated for 1,163 OHCA patients who were treated with targeted temperature management (TTM) at 21 hospitals in South Korea: OHCA, cardiac arrest hospital prognosis (CAHP), C-GRApH (named on the basis of its variables), TTM risk, 5-R, NULL-PLEASE (named on the basis of its variables), Serbian quality of life long-term (SR-QOLl), cardiac arrest survival, revised post-cardiac arrest syndrome for therapeutic hypothermia (rCAST), Polish hypothermia registry (PHR) risk, and PROgnostication using LOGistic regression model for Unselected adult cardiac arrest patients in the Early stages (PROLOGUE) scores and prediction score by Aschauer et al. Their accuracies in predicting poor outcome at 6 months after OHCA were determined using the area under the receiver operating characteristic curve (AUC) and calibration belt. In the complete-case analyses, the PROLOGUE score showed the highest AUC (0.923; 95% confidence interval [CI], 0.904–0.941), whereas the SR-QOLl score had the lowest AUC (0.749; 95% CI, 0.711–0.786). The discrimination performances were similar in the analyses after multiple imputation. The PROLOGUE, TTM risk, CAHP, NULL-PLEASE, 5-R, and cardiac arrest survival scores were well calibrated. The rCAST and PHR risk scores showed acceptable overall calibration, although they showed miscalibration under the 80% CI level at extreme prediction values. The OHCA score, C-GRApH score, prediction score by Aschauer et al., and SR-QOLl score showed significant miscalibration in both complete-case (P = 0.026, 0.013, 0.005, and < 0.001, respectively) and multiple-imputation analyses (P = 0.007, 0.018, < 0.001, and < 0.001, respectively). In conclusion, the discrimination performances of the prognostication scores were all acceptable, but some showed significant miscalibration.
Citation: Heo WY, Jung YH, Lee HY, Jeung KW, Lee BK, Youn CS, et al. (2022) External validation of cardiac arrest-specific prognostication scores developed for early prognosis estimation after out-of-hospital cardiac arrest in a Korean multicenter cohort. PLoS ONE 17(4): e0265275. https://doi.org/10.1371/journal.pone.0265275
Editor: Muhammad Tarek Abdel Ghafar, Tanta University Faculty of Medicine, EGYPT
Received: December 9, 2021; Accepted: February 25, 2022; Published: April 1, 2022
Copyright: © 2022 Heo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: This study was supported by a grant (BCRI21040) from the Chonnam National University Hospital Biomedical Research Institute (Recipient: KWJ). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Out-of-hospital cardiac arrest (OHCA) remains the leading cause of mortality and disability worldwide [1, 2]. Most of the patients resuscitated from OHCA eventually die in hospital or develop severe neurologic sequelae; only 10%–30% survive with good neurologic outcome [2, 3]. Current guidelines recommend delaying neurologic prognosis estimation in comatose cardiac arrest patients until at least 72 h after return of spontaneous circulation (ROSC) [4]. However, there is a need for an accurate prognostic tool useful during the early hours after OHCA. In the case of comatose OHCA patients, families desire precise information on the neurologic prognoses as early as possible. Treating physicians often have to make critical decisions regarding the use of costly and resource-intensive therapies, such as extracorporeal membrane oxygenation (ECMO), in the early stages of post-cardiac arrest care, when the patients’ neurologic prognoses are uncertain.
Several cardiac arrest-specific prognostication scores for use in the early hours after OHCA have been developed from retrospective or prospective analyses of OHCA data [5–16]. These scores have several limitations that must be addressed to render them useful in clinical practice. A risk prediction score derived from one study population may not be accurate in other populations. Thus, external validations in various patient populations are required to enable widespread reliance on a risk prediction score, but few such scores have undergone any external validation using independent data; where this has been done, it was usually limited to retrospective analyses of discrimination performance [7, 9, 17–22]. Most of the scores are intended to predict short-term outcomes, such as survival to hospital discharge or neurologic outcome at hospital discharge [5–8, 10–12, 14–16], and have not been evaluated as a means to predict long-term outcomes. Targeted temperature management (TTM) is now the standard treatment for comatose OHCA patients. However, several scores were developed before the widespread use of TTM or derived from studies that included OHCA patients irrespective of whether they had undergone TTM [5–7, 10, 12–14].
To address these limitations, we sought to evaluate the performance of cardiac arrest-specific prognostication scores developed for outcome prediction in the early hours after OHCA in predicting long-term outcomes, using independent data from a multicenter registry of comatose OHCA patients who underwent TTM. We hypothesized that the scores would accurately predict long-term outcomes in an independent cohort of OHCA patients who underwent TTM.
Materials and methods
Study design and setting
This study conformed to the principles outlined in the Declaration of Helsinki. It was a retrospective analysis of data from the Korean Hypothermia Network prospective (KORHN-pro) registry, which enrolled adult OHCA patients treated with TTM at 22 teaching hospitals in the Republic of Korea [3]. In brief, a principal investigator at each participating hospital reviewed the medical records of patients who were eligible for registry enrollment and collected their demographic, prehospital resuscitation, in-hospital treatment, and outcomes data in an anonymous fashion using a web-based case report form based on the Utstein Resuscitation Registry Templates [23]. Data quality was assured by five clinical research associates who queried any concerns with the investigators, and a data manager with final responsibility for determining data acceptability. The study design and registry protocol were approved by the institutional review board of all participating hospitals, including the Chonnam National University Hospital Institutional Review Board (CNUH-2015-164) and registered at the International Clinical Trials Registry Platform (ClinicalTrials.gov identifier: NCT02827422). Written informed consent was obtained from the legal surrogates of all patients enrolled in the registry.
Study population
The KORHN-pro registry included all adult (≥ 18 years) unconscious (Glasgow Coma Scale [GCS] score < 8) OHCA survivors treated with TTM at participating hospitals between October 2015 and December 2018, except those with the following conditions: OHCA associated with hemorrhagic or ischemic stroke; poor pre-arrest neurologic status (cerebral performance category [CPC] of 3 or 4); body temperature < 30°C on admission; pre-arrest do-not-resuscitate order; or known terminal illness leading to life expectancy < 6 months. One of the scores included in this study (PROLOGUE [PROgnostication using LOGistic regression model for Unselected adult cardiac arrest patients in the Early stages]) is developed using data from one of the participating hospitals [7]. Thus, patients enrolled from this center were excluded from this study, as were patients without data on outcomes at 6 months. The patients included in the registry were managed according to the treatment protocols of each hospital.
Variables
Data on the following variables were obtained for each patient: age, sex, hospital, pre-existing chronic diseases (coronary artery disease, heart failure, arrhythmia, cerebrovascular accident [CVA], neurologic disease other than CVA, diabetes, hypertension, pulmonary disease, chronic kidney disease, liver cirrhosis, and malignancy), patient location at the time of cardiac arrest, presence of a witness to the collapse, bystander cardiopulmonary resuscitation (CPR), first monitored rhythm, no-flow duration, low-flow duration, time to ROSC, dose of epinephrine given during CPR, etiology of cardiac arrest, circulatory status on emergency department arrival (prehospital ROSC), GCS motor score and pupillary light reflex obtained before intensive care unit (ICU) admission, initial laboratory parameters after ROSC (lactate, arterial pH, partial pressure of arterial oxygen [PaO2], partial pressure of arterial carbon dioxide [PaCO2], potassium, phosphate, creatinine, glucose, and hemoglobin), duration and target temperature of TTM, Sequential Organ Failure Assessment score on the first day after hospital admission, occurrence of rearrest before ICU admission, critical care interventions implemented during hospitalization (coronary angiography and ECMO), length of hospital stay, and CPC at 6 months after OHCA. No-flow and low-flow durations were defined as the time interval from collapse to first CPR attempt and the time interval from first CPR attempt to ROSC, respectively. Time to ROSC was defined as the time interval from collapse to ROSC. CPC at 6 months after OHCA was evaluated through in-person or telephone interviews conducted by medical staff at each center who were blinded to patient data. A CPC of 1 or 2 was defined as a good outcome and a CPC of 3–5 as a poor one (primary outcome). After literature review, the following cardiac arrest-specific prognostication scores were selected based on availability of the data required for score calculation and were calculated using the formulas presented in the original publications. The scores were as follows: OHCA [5]; cardiac arrest hospital prognosis (CAHP) [6]; PROLOGUE [7]; C-GRApH [8], named on the basis of its variables; TTM risk [9]; prediction score by Aschauer et al. [10]; 5-R [11]; NULL-PLEASE [12], named on the basis of its variables; Serbian quality of life long-term (SR-QOLl) [13]; cardiac arrest survival [14]; revised post-cardiac arrest syndrome for therapeutic hypothermia (rCAST) [15]; and Polish hypothermia registry (PHR) risk [16]. The characteristics of these scores are summarized in Table 1. A greater risk of poor outcome is indicated by lower scores for the 5-R and SR-QOLl scores, but otherwise by higher scores.
Statistical analysis
Data analysis and reporting were performed in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement [24]. The sample size of this study far exceeded the suggested minimum sample size for external validation studies of multivariable prediction models [25, 26]. Statistical analyses were conducted using T&F programme version 3.0 (YooJin BioSoft, Goyang, Republic of Korea) and R language version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria). Continuous variables are presented by medians with interquartile ranges, unless otherwise specified. Categorical variables are expressed as numbers of cases with percentages. Comparisons between two independent groups were performed using the Mann–Whitney U test for continuous variables and the chi-square test with continuity correction for categorical variables. To determine the association of each prognostication score with the primary outcome, binary logistic regression analyses were performed after dividing the patients into two groups according to the optimal cut-off for each score. The discrimination abilities of the prognostication scores were assessed using receiver operating characteristic (ROC) analysis, and quantified with area under the ROC curve (AUC). The AUC values were compared in a pairwise manner using the method of DeLong et al. [27]. For each score, sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated for the optimal cut-off, determined using the Youden index. The calibration performances of the prognostication scores were assessed using the calibration belt [28, 29]. To allow for comparisons between scores, score performances were initially evaluated for patients for whom all 12 score values were calculable. To evaluate the robustness of the results, missing values of the variables required for the calculation of the prognostication scores were imputed using the MICE package in R, and the performances of the prognostication scores were reassessed. Statistical significance was indicated by a two-sided P-value of < 0.05.
Results
A total of 1,373 adult OHCA patients treated with TTM were enrolled in the KORHN-pro registry. Among these, 187 who were enrolled from the hospital involved in the development of PROLOGUE and 23 without data on CPC at 6 months after OHCA were excluded from this study, leaving 1,163 included patients (Fig 1). These were mostly male (70.9%), with a median age of 58.3 years old (46.8–69.9). The majority of the patients had a witnessed cardiac arrest (71.6%), received bystander CPR (63.1%), and presented with a non-shockable initial cardiac arrest rhythm (63.2%). The no-flow duration, low-flow duration, and time to ROSC were 1.0 (0.0–6.0), 25.0 (14.0–38.0), and 30.0 (18.0–43.0) min, respectively. The cardiac arrest was cardiac in origin in 714 (61.4%) patients. Four hundred (34.4%) patients underwent coronary angiography, and 57 (4.9%) received ECMO during hospitalization. Of the included patients, 357 (30.7%) had a good outcome 6 months after OHCA, while the remaining 806 (69.3%) patients had a poor outcome. The clinical and laboratory characteristics of patients, stratified by outcomes at 6 months after OHCA, are summarized in Table 2. As shown in Table 2, all 12 of the prognostication scores in the present study were significantly associated with the primary outcome (all P < 0.001).
OHCA, out-of-hospital cardiac arrest; TTM, targeted temperature management; PROLOGUE, PROgnostication using LOGistic regression model for Unselected adult cardiac arrest patients in the Early stages; CPC, cerebral performance category.
Prognostic performances of the scores
There was a total of 804 patients for whom all 12 prognostication scores were calculable, of whom 241 (30.0%) had a good outcome and 563 (70.0%) had a poor outcome. In binary logistic regression analyses examining the association between scores above the optimal cut-off (below the optimal cut-off for the 5-R and SR-QOLl scores) and the risk of poor outcome at 6 months after OHCA for each score (Fig 2), the odds ratios ranged from 6.813 (C-GRApH score) to 32.143 (PROLOGUE). The discrimination abilities of the prognostication scores in these patients are shown in Fig 3 and Table 3. All scores could predict poor outcome at 6 months after OHCA with statistical significance (all P < 0.001). PROLOGUE showed the highest AUC (0.923; 95% confidence interval [CI], 0.904–0.941), whereas the SR-QOLl score had the lowest AUC (0.749; 95% CI, 0.711–0.786). All scores showed similar AUC in the analyses after multiple imputation (Table 4). Table 5 shows sensitivity, specificity, positive and negative predictive values, and accuracy for different cut-offs. The results of pairwise comparisons of the ROC curves are summarized in Table 6.
This analysis only included the 804 patients for whom all of the 12 prognostic scores were available. The scores are displayed in descending order of odds ratio. CI, confidence interval; PROLOGUE, PROgnostication using LOGistic regression model for Unselected adult cardiac arrest patients in the Early stages; CAHP, cardiac arrest hospital prognosis; TTM, targeted temperature management; rCAST, revised post-cardiac arrest syndrome for therapeutic hypothermia; PHR, Polish hypothermia registry; OHCA, out-of-hospital cardiac arrest; SR-QOLl, Serbian quality of life long-term.
This analysis only included the 804 patients for whom all of the 12 prognostication scores were available. PROLOGUE, PROgnostication using LOGistic regression model for Unselected adult cardiac arrest patients in the Early stages; TTM, targeted temperature management; CAHP, cardiac arrest hospital prognosis; rCAST, revised post-cardiac arrest syndrome for therapeutic hypothermia; PHR, Polish hypothermia registry; OHCA, out-of-hospital cardiac arrest; SR-QOLl, Serbian quality of life long-term.
The calibration performances of the prognostication scores in the 804 patients are shown in Fig 4. Calibration belts for the PROLOGUE, TTM risk, CAHP, NULL-PLEASE, 5-R, and cardiac arrest survival scores contained bisecting lines (representing perfect calibration) across the entire range of predictions. For the rCAST and PHR risk scores, the 80% CI boundaries of the calibration belt did not contain bisecting lines at extreme predicted probability values, although such lines were present in the 95% CI boundaries of calibration belts across the entire range of predictions (P = 0.060 and 0.114, respectively). Calibration belts for the prediction score by Aschauer et al. (P = 0.005), OHCA score (P = 0.026), C-GRApH score (P = 0.013), and SR-QOLl score (P < 0.001) significantly deviated from the bisecting line. This was also true in the analyses following inclusion of imputed data (prediction score by Aschauer et al., P < 0.001; OHCA score, P = 0.007; C-GRApH score, P = 0.018; and SR-QOLl score, P < 0.001).
(A) PROLOGUE, (B) TTM risk score, (C) CAHP score, (D) prediction score by Aschauer et al., (E) NULL-PLEASE score, (F) 5-R score, (G) rCAST score, (H) PHR risk score, (I) OHCA score, (J) cardiac arrest survival score, (K) C-GRApH score, (L) SR-QOLl score. The bisecting lines correspond to perfect agreement between observed outcomes and predicted outcomes (perfect calibration). The light and dark shaded areas represent 80% and 95% confidence intervals, respectively. This analysis only included the 804 patients for whom all of the 12 prognostication scores were available. PROLOGUE, PROgnostication using LOGistic regression model for Unselected adult cardiac arrest patients in the Early stages; TTM, targeted temperature management; CAHP, cardiac arrest hospital prognosis; rCAST, revised post-cardiac arrest syndrome for therapeutic hypothermia; PHR, Polish hypothermia registry; OHCA, out-of-hospital cardiac arrest; SR-QOLl, Serbian quality of life long-term.
Discussion
We evaluated the performances of 12 existing prediction scores developed for early prognosis estimation after OHCA in predicting poor outcome at 6 months after cardiac arrest using independent data from a multicenter registry of comatose OHCA patients who underwent TTM. In this study, the discrimination performances of the scores were all acceptable, some even being excellent. However, some scores (prediction score by Aschauer et al., OHCA score, C-GRApH score, and SR-QOLl score) showed significant miscalibration. To the best of our knowledge, this is the largest study to evaluate the performances of multiple cardiac arrest-specific prognostication scores in an East Asian population.
Our study population differed in many aspects from the original study populations used to develop the scores included in this study. Most of the scores were derived from studies conducted in European countries or Unites States [5, 6, 8–10, 12–14, 16], where the prehospital and in-hospital care processes are quite different from Korean practice. In the patient populations used to derive the C-GRApH, TTM risk, 5-R, and PHR risk scores [8, 9, 11, 16], the proportion with initial shockable rhythm was over 85%; in contrast, this proportion was only 36.8% in our study. The proportion of witnessed arrest was 71.6% in our study population, whereas it was higher than 85% in the study populations for the CAHP score, C-GRApH score, TTM risk score, 5-R score, and prediction score by Aschauer et al. [6, 8–11]. In contrast to our study population, only 11% and 51.7% of patients were treated with TTM in the studies generating the OHCA and PROLOGUE scores, respectively [5, 7]. In addition, the primary outcome of our study was poor outcome at 6 months after OHCA, whereas most of the scores were developed for prediction of outcomes at hospital discharge [5–8, 11, 12, 14, 16]. Despite these differences, the PROLOGUE, TTM risk, CAHP, NULL-PLEASE, 5-R, and cardiac arrest survival scores demonstrated satisfactory discrimination and calibration performances for predicting poor outcome at 6 months after OHCA. Although the calibration performance was not perfect, the rCAST and PHR risk scores also showed acceptable overall calibration and decent discrimination performances. These results not only support the robustness and generalizability of these scores, but also extend their applicability to the prediction of long-term outcomes.
Prognostication scores commonly estimate outcomes using combination of predictor variables selected through logistic regression. However, the studied scores vary greatly in terms of complexity. The prediction score by Aschauer et al. is composed of only four variables, whereas PROLOGUE is composed of 12 variables. Some scores are simply calculated as the sum of points awarded for each of the variables that are present [8–14], whereas others are calculated using complex formulas or nomograms [5–7, 15, 16]. Among those in this study, the PROLOGUE, TTM risk, and CAHP scores showed outstanding predictive performance (median AUC values > 0.9), but these scores require elaborate calculations, as they use a relatively complex nomogram or multi-point scoring system with a different weight for each parameter. Although these scores are relatively complex, this would not hinder practicality for clinical use if they could be calculated electronically using a desktop calculator or mobile device.
In this study, the prediction score by Aschauer et al., OHCA score, C-GRApH score, and SR-QOLl score showed acceptable discrimination but significant miscalibration. The prediction score by Aschauer et al. and C-GRApH score overestimated the actual risk of poor outcome at extreme predicted probability values, whereas the OHCA score and SR-QOLl score underestimated it. Although the calibration performances of the prediction score by Aschauer et al., C-GRApH score, and SR-QOLl score, to the best of our knowledge, have not been evaluated in separate studies, the low calibration capacity of the OHCA score for predicting poor outcome (CPC 3–5) at 6 months after OHCA has also been reported by other researchers [9, 19]. Our study suggests that these scores need to be updated for use in settings similar to ours.
These scores would allow treating physicians to provide a patient’s likely long-term outcome in a more objective manner in the early hours after OHCA. Although the prognostication scores in the present study could predict poor outcome with statistical significance, they were not specific enough to be used for important therapeutic decision-making (e.g., withholding or withdrawing life-saving treatment). These scores can be used as an adjunct to guide therapeutic decision-making. However, given the insufficient specificities observed in this study, important therapeutic decisions should not be made based on these prognostication scores alone.
Our study has several limitations. First, it was a retrospective analysis of data collected from teaching hospitals in the Republic of Korea. The performances of prognostication scores may be different in other healthcare or country settings. Second, we evaluated the performances of prognostication scores, but could not assess their clinical usefulness. Further studies are required to evaluate this. Third, we could not evaluate several cardiac arrest-specific prognostication scores that required variables unavailable from our registry data [19, 30–32]. Lastly, the treating physicians were not blinded to the constituent results of the prognostication scores, thereby introducing the potential for self-fulfilling prophecy bias.
Conclusions
We evaluated the performances of 12 existing cardiac arrest-specific prognostication scores in predicting poor outcome at 6 months after OHCA using data from a multicenter registry of comatose OHCA patients who underwent TTM. The PROLOGUE, TTM risk, CAHP, NULL-PLEASE, 5-R, and cardiac arrest survival scores showed satisfactory discrimination and calibration performances. Although the calibration performance was not perfect, the rCAST and PHR risk scores also showed acceptable overall calibration and good discrimination performances. The prediction score by Aschauer et al., OHCA score, C-GRApH score, and SR-QOLl score showed acceptable discrimination but significant miscalibration. None of the prognostication scores in this study were specific enough to be used alone in important therapeutic decision-making. These study findings may improve our understanding of these prognostication scores and thereby aid in the interpretations of the prediction results.
Acknowledgments
We would like to thank Editage (www.editage.co.kr) for English language editing. The following investigators participated in the Korean Hypothermia Network. Chair: Kyung Woon Jeung (Chonnam National University Hospital, E-mail: neoneti@hanmail.net). Principal investigators of each hospital: Kyu Nam Park (The Catholic University of Korea, Seoul St. Mary’s Hospital); Minjung Kathy Chae (Ajou University Medical Center); Won Young Kim (Asan Medical Center); Byung Kook Lee (Chonnam National University Hospital); Dong Hoon Lee (Chung-Ang University Hospital); Tae Chang Jang (Daegu Catholic University Medical Center); Jae Hoon Lee (Dong-A University Hospital); Yoon Hee Choi (Ewha Womans University Mokdong Hospital); Je Sung You (Gangnam Severance Hospital); Young Hwan Lee (Hallym University Sacred Heart Hospital); In Soo Cho (Hanil General Hospital); Su Jin Kim (Korea University Anam Hospital); Jong-Seok Lee (Kyung Hee University Medical Center); Yong Hwan Kim (Samsung Changwon Hospital); Min Seob Sim (Samsung Medical Center); Jonghwan Shin (Seoul Metropolitan Government Seoul National University Boramae Medical Center); Yoo Seok Park (Severance Hospital); Hyung Jun Moon (Soonchunhyang University Hospital Cheonan); Won Jung Jeong (The Catholic University of Korea, St. Vincent’s Hospital); Joo Suk Oh (The Catholic University of Korea, Uijeongbu St. Mary’s Hospital); Seung Pill Choi (The Catholic University of Korea, Yeouido St. Mary’s Hospital); Kyoung-Chul Cha (Wonju Severance Christian Hospital).
References
- 1. Gräsner JT, Lefering R, Koster RW, Masterson S, Böttiger BW, Herlitz J, et al. EuReCa ONE-27 Nations, ONE Europe, ONE Registry: A prospective one month analysis of out-of-hospital cardiac arrest outcomes in 27 countries in Europe. Resuscitation. 2016;105: 188–195. pmid:27321577
- 2. Ong ME, Shin SD, De Souza NN, Tanaka H, Nishiuchi T, Song KJ, et al. Outcomes for out-of-hospital cardiac arrests across 7 countries in Asia: The Pan Asian Resuscitation Outcomes Study (PAROS). Resuscitation. 2015;96: 100–108. pmid:26234891
- 3. Kim SH, Park KN, Youn CS, Chae MK, Kim WY, Lee BK, et al. Outcome and status of postcardiac arrest care in Korea: results from the Korean Hypothermia Network prospective registry. Clin Exp Emerg Med. 2020;7: 250–258. pmid:33440102
- 4. Nolan JP, Sandroni C, Böttiger BW, Cariou A, Cronberg T, Friberg H, et al. European Resuscitation Council and European Society of Intensive Care Medicine Guidelines 2021: Post-resuscitation care. Resuscitation. 2021;161: 220–269. pmid:33773827
- 5. Adrie C, Cariou A, Mourvillier B, Laurent I, Dabbane H, Hantala F, et al. Predicting survival with good neurological recovery at hospital admission after successful resuscitation of out-of-hospital cardiac arrest: the OHCA score. Eur Heart J. 2006;27: 2840–2845. pmid:17082207
- 6. Maupain C, Bougouin W, Lamhaut L, Deye N, Diehl JL, Geri G, et al. The CAHP (Cardiac Arrest Hospital Prognosis) score: a tool for risk stratification after out-of-hospital cardiac arrest. Eur Heart J. 2016;37: 3222–3228. pmid:26497161
- 7. Bae DH, Lee HY, Jung YH, Jeung KW, Lee BK, Youn CS, et al. PROLOGUE (PROgnostication using LOGistic regression model for Unselected adult cardiac arrest patients in the Early stages): Development and validation of a scoring system for early prognostication in unselected adult cardiac arrest patients. Resuscitation. 2021;159: 60–68. pmid:33388366
- 8. Kiehl EL, Parker AM, Matar RM, Gottbrecht MF, Johansen MC, Adams MP, et al. C-GRApH: A Validated Scoring System for Early Stratification of Neurologic Outcome After Out-of-Hospital Cardiac Arrest Treated With Targeted Temperature Management. J Am Heart Assoc. 2017;6: e003821. pmid:28528323
- 9. Martinell L, Nielsen N, Herlitz J, Karlsson T, Horn J, Wise MP, et al. Early predictors of poor outcome after out-of-hospital cardiac arrest. Crit Care. 2017;21: 96. pmid:28410590
- 10. Aschauer S, Dorffner G, Sterz F, Erdogmus A, Laggner A. A prediction tool for initial out-of-hospital cardiac arrest survivors. Resuscitation. 2014;85: 1225–1231. pmid:24960427
- 11. Okada K, Ohde S, Otani N, Sera T, Mochizuki T, Aoki M, et al. Prediction protocol for neurological outcome for survivors of out-of-hospital cardiac arrest treated with targeted temperature management. Resuscitation. 2012;83: 734–739. pmid:22281226
- 12. Ahmad R, Lumley S, Lau YC. NULL-PLEASE: a new ‘Futility score’ in the management of survivors of out-of-hospital cardiac arrest. Resuscitation. 2016;106: e83.
- 13. Sladjana A. A prediction survival model for out-of-hospital cardiopulmonary resuscitations. J Crit Care. 2011;26: 223. e11–e18. pmid:20655699
- 14. Balan P, Hsi B, Thangam M, Zhao Y, Monlezun D, Arain S, et al. The cardiac arrest survival score: A predictive algorithm for in-hospital mortality after out-of-hospital cardiac arrest. Resuscitation. 2019;144: 46–53. pmid:31539610
- 15. Nishikimi M, Ogura T, Nishida K, Takahashi K, Nakamura M, Matsui S, et al. External validation of a risk classification at the emergency department of post-cardiac arrest syndrome patients undergoing targeted temperature management. Resuscitation. 2019;140: 135–141. pmid:31153943
- 16. Kołtowski Ł, Średniawa B, Tycińska A, Czajkowska M, Niedziela M, Puchalski W, et al. Predicting survival in out-of-hospital cardiac arrest patients undergoing targeted temperature management: The Polish Hypothermia Registry Risk Score. Cardiol J. 2021;28: 95–100. pmid:30994183
- 17. Gue YX, Sayers M, Whitby BT, Kanji R, Adatia K, Smith R, et al. Usefulness of the NULL-PLEASE Score to Predict Survival in Out-of-Hospital Cardiac Arrest. Am J Med. 2020;133: 1328–1335. pmid:32387318
- 18. Luescher T, Mueller J, Isenschmid C, Kalt J, Rasiah R, Tondorf T, et al. Neuron-specific enolase (NSE) improves clinical risk scores for prediction of neurological outcome and death in cardiac arrest patients: Results from a prospective trial. Resuscitation. 2019;142: 50–60. pmid:31306716
- 19. Pareek N, Kordis P, Beckley-Hoelscher N, Pimenta D, Kocjancic ST, Jazbec A, et al. A practical risk score for early prediction of neurological outcome after out-of-hospital cardiac arrest: MIRACLE2. Eur Heart J. 2020;41: 4508–4517. pmid:32731260
- 20. Song HG, Park JS, You Y, Ahn HJ, Yoo I, Kim SW, et al. Using Out-of-Hospital Cardiac Arrest (OHCA) and Cardiac Arrest Hospital Prognosis (CAHP) Scores with Modified Objective Data to Improve Neurological Prognostic Performance for Out-of-Hospital Cardiac Arrest Survivors. J Clin Med. 2021;10: 1825. pmid:33922191
- 21. Kim HS, Park KN, Kim SH, Lee BK, Oh SH, Jeung KW, et al. Prognostic value of OHCA, C-GRApH and CAHP scores with initial neurologic examinations to predict neurologic outcomes in cardiac arrest patients treated with targeted temperature management. PLoS One. 2020;15: e0232227. pmid:32330180
- 22. Hunziker S, Bivens MJ, Cocchi MN, Miller J, Salciccioli J, Howell MD, et al. International validation of the out-of-hospital cardiac arrest score in the United States. Crit Care Med. 2011;39: 1670–1674. pmid:21494106
- 23. Perkins GD, Jacobs IG, Nadkarni VM, Berg RA, Bhanji F, Biarent D, et al. Cardiac Arrest and Cardiopulmonary Resuscitation Outcome Reports: Update of the Utstein Resuscitation Registry Templates for Out-of-Hospital Cardiac Arrest: A Statement for Healthcare Professionals From a Task Force of the International Liaison Committee on Resuscitation (American Heart Association, European Resuscitation Council, Australian and New Zealand Council on Resuscitation, Heart and Stroke Foundation of Canada, InterAmerican Heart Foundation, Resuscitation Council of Southern Africa, Resuscitation Council of Asia); and the American Heart Association Emergency Cardiovascular Care Committee and the Council on Cardiopulmonary, Critical Care, Perioperative and Resuscitation. Resuscitation. 2015;96: 328–340. pmid:25438254
- 24. Collins GS, Reitsma JB, Altman DG, Moons KG; TRIPOD Group. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. The TRIPOD Group. Circulation. 2015;131: 211–219. pmid:25561516
- 25. Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005;58: 475–483. pmid:15845334
- 26. Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med. 2016;35: 214–226. pmid:26553135
- 27. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44: 837–845. pmid:3203132
- 28. Nattino G, Finazzi S, Bertolini G. A new test and graphical tool to assess the goodness of fit of logistic regression models. Stat Med. 2016;35: 709–720. pmid:26439593
- 29. Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G. Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLoS One. 2011;6: e16110. pmid:21373178
- 30. Eertmans W, Tran TMP, Genbrugge C, Peene L, Mesotten D, Dens J, et al. A prediction model for good neurological outcome in successfully resuscitated out-of-hospital cardiac arrest patients. Scand J Trauma Resusc Emerg Med. 2018;26: 93. pmid:30413210
- 31. Kumagai K, Oda Y, Oshima C, Kaneko T, Kaneda K, Kawamura Y, et al. Development of a prompt model for predicting neurological outcomes in patients with return of spontaneous circulation from out-of-hospital cardiac arrest. Acute Med Surg. 2014;2: 176–182. pmid:29123717
- 32. Hayakawa K, Tasaki O, Hamasaki T, Sakai T, Shiozaki T, Nakagawa Y, et al. Prognostic indicators and outcome prediction model for patients with return of spontaneous circulation from cardiopulmonary arrest: the Utstein Osaka Project. Resuscitation. 2011;82: 874–880. pmid:21482007