https://orcid.org/0000-0001-6330-9820
Figures
Abstract
Background
Due to an aging population and the increasing proportion of patients with various comorbidities, the number of patients with acute ischemic heart disease (AIHD) who present to the emergency department (ED) with atypical chest pain is increasing. The aim of this study was to develop and validate a prediction model for AIHD in patients with atypical chest pain.
Methods and results
A chest pain workup registry, ED administrative database, and clinical data warehouse database were analyzed and integrated by using nonidentifiable key factors to create a comprehensive clinical dataset in a single academic ED from 2014 to 2018. Demographic findings, vital signs, and routine laboratory test results were assessed for their ability to predict AIHD. An extreme gradient boosting (XGB) model was developed and evaluated, and its performance was compared to that of a single-variable model and logistic regression model. The area under the receiver operating characteristic curve (AUROC) was calculated to assess discrimination. A calibration plot and partial dependence plots were also used in the analyses. Overall, 4,978 patients were analyzed. Of the 3,833 patients in the training cohort, 453 (11.8%) had AIHD; of the 1,145 patients in the validation cohort, 166 (14.5%) had AIHD. XGB, troponin (single-variable), and logistic regression models showed similar discrimination power (AUROC [95% confidence interval]: XGB model, 0.75 [0.71–0.79]; troponin model, 0.73 [0.69–0.77]; logistic regression model, 0.73 [0.70–0.79]). Most patients were classified as non-AIHD; calibration was good in patients with a low predicted probability of AIHD in all prediction models. Unlike in the logistic regression model, a nonlinear relationship-like threshold and U-shaped relationship between variables and the probability of AIHD were revealed in the XGB model.
Citation: Kim KH, Park JH, Ro YS, Hong KJ, Song KJ, Shin SD (2020) Emergency department routine data and the diagnosis of acute ischemic heart disease in patients with atypical chest pain. PLoS ONE 15(11): e0241920. https://doi.org/10.1371/journal.pone.0241920
Editor: Giulia Bivona, University of Palermo, ITALY
Received: August 25, 2020; Accepted: October 22, 2020; Published: November 5, 2020
Copyright: © 2020 Kim et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This study was financially supported by Seoul National University Hospital. (Grant No. 04-2018-0750)
Competing interests: The authors have declared that no competing interests exist.
Introduction
Acute ischemic heart disease (AIHD) is a major public health concern worldwide [1, 2]. One case of AIHD occurs approximately every 40 seconds in the United States [3]. Chest pain is the most common presenting complaint of AIHD [4, 5]. Various studies have shown that prompt identification of AIHD is important [6], and delayed treatment of AIHD leads to poor outcomes [7, 8]. However, making a precise and early diagnosis for chest pain is difficult [9–11]. Approximately 2% of patients with AIHD are missed on initial presentation to the emergency department (ED) [12].
Due to an aging population and the increasing proportion of patients with various comorbidities, the number of patients with AIHD who present to the ED with atypical chest pain is increasing [1, 13]. Although 12-lead electrocardiography (ECG) and cardiac biomarkers are key diagnostic tools for assessing AIHD [14–16] they may show nondiagnostic results in the initial ED period. High-sensitive cardiac troponin assays can also be used as diagnostic tests, but these assays have limitations, including variability and a lack of universal protocols to guide the reference interval [17, 18]. Several novel cardiac biomarkers, such as galectin-3 or heart-type fatty acid binding protein, have been investigated [19–21]. Approximately half of patients with AIHD have no diagnostic ECG findings, and only one-third of patients have increased cardiac biomarker levels at the time of ED presentation [22, 23]. In addition, because of the atypical presentation of symptoms, physicians may not order those tests in the early period of diagnosis. The likelihood of this happening may be increased when atypical chest pain is comorbid with various symptoms. In this scenario, AIHD may be missed, or its diagnosis may be delayed.
Demographic data, such as age and vital signs, are routinely collected during the initial examination, and they provide important information for screening for AIHD [5, 24]. Many patients may receive a complete blood count (CBC) and comprehensive metabolic panel (CMP) regardless of their characteristics when presenting in many EDs. In previous studies, components of the CBC were evaluated for their role in the pathogenesis of atherosclerosis and as risk predictors for ischemic heart disease (IHD) [25, 26]. Glucose, serum blood urea nitrogen, and creatinine have also been evaluated as indicators of metabolic disturbance in IHD progression [27]. However, the diagnostic performance of routinely collected information for patients with atypical chest pain has not been widely evaluated in previous studies.
Knowing the diagnostic performance of routinely collected data for the diagnosis of AIHD has clinical implications. First, such information can be used to screen for patients with possible AIHD among those who were not initially thought to have AIHD and therefore lack ECG and cardiac biomarker results. Second, because most of this information can be easily collected, it has many possible applications, especially for situations in which only limited testing can be performed. In low-level EDs or outpatient clinics, this information could be supportive and aid in screening patients with AIHD. Furthermore, this information could help inform health care providers’ decision making when requesting additional cardiac workups, such as serial ECG, cardiac biomarker follow-up, and coronary computed tomography angiography (CCTA).
Traditional models usually assume that each predictor is associated with outcomes in a linear fashion [28]. However, the association between routinely collected data and outcomes in atypical chest pain is uncertain. For example, components of routinely collected data, e.g., aspartate transaminase (GOT) and white blood cell count (WBC), may exhibit different patterns in relation to AIHD. There is also a possibility of high-order interaction between each predictor. A machine learning-based prediction model can be beneficial in this scenario because of its potential for use in evaluating complex data, including nonlinear data and high-order interactions [29].
The aim of this study was to develop and validate a machine learning-based prediction model for AIHD using routinely collected data (excluding 12-lead ECG and cardiac biomarker test results) in patients with atypical chest pain who visited the ED. We also evaluated the predictor variable importance of the model and how each predictor affected the probability of AIHD according to its value.
Materials and methods
Study design and ethical statements
This study was a single-center retrospective study in the ED of a large, urban, academic teaching hospital that receives ~60,000 ED visits annually.
This study was approved by the Institutional Review Board, and the requirement for informed consent was waived (IRB No. 1808-001-962). This study complied with the Declaration of Helsinki, and we adhered to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis statement on reporting predictive models [30].
Study setting
The ED had approximately 10–12 emergency physicians, 9–11 emergency residents, 5 specialty board staff, 50–60 emergency nursing staff, and 15–17 emergency medical technicians from 2014 to 2018. A structured protocol for triage based on patients’ vital signs, brief history, and chief complaint was used. When patients visited the ED, dedicated ED nurses triaged patients based on a five-level scale (level 1: immediate, level 2: very urgent, level 3: urgent, level 4: less urgent, and level 5: not urgent) [31, 32]. Based on this scale, the most unstable patients, such as those with cardiac arrest or definite shock status, were classified as level 1. If a triage nurse found no evidence of severe shock or desaturation, all patients with chest pain were assigned to on-duty physicians in the cardiovascular section, which assists with efficient ED operation and sensitive diagnosis of cardiovascular disease. Patients with a chief complaint other than chest pain were included if they also had chest pain. After interviews and physical examinations, the primary physician completed the structured chest pain workup registry. They assigned all eligible patients into categories consisting of typical angina and other angina, as per the guidelines [33]. Typical angina was defined as having all three conditions: substernal squeezing chest pain, pain subsiding with rest or nitroglycerine administration, and pain aggravated by exercise [34, 35]. The registry was reviewed every other day by attending physicians for quality control of the management of patients with chest pain. If patients were suspected of having AIHD, consultation with a cardiology specialist was conducted for further management and admission.
Selection of participants
We included adult (≥18 years old) nontrauma patients with chest pain who visited the ED from January 2014 to December 2018. Patients who presented with typical characteristics of angina were excluded because most needed to undergo further cardiac testing regardless of their characteristics. Patients who did not visit the ED until 1 week after symptom onset, received interhospital transfer, or presented with cardiac arrest or unstable vital signs and were classified as level 1 by a triage nurse, were excluded.
The study population was divided into a training cohort from which each of the machine learning prediction models was derived and a validation cohort in whom the prediction models were applied and tested. The training cohort was derived from data collected from January 2014 to December 2017; the validation cohort comprised data collected from January 2018 to December 2018.
Data sources
We obtained data from three independent databases, including the ED administrative database, clinical data warehouse (CDW) database [36], and chest pain workup registry. Data were obtained from January 2014 to December 2018. The ED administrative database contains patients’ demographic characteristics, route of visit, time of visit, and diagnosis and disposition. The CDW database includes laboratory study results and imaging study results. The chest pain workup registry includes chest pain characteristics. We integrated the three databases using a common deidentified key to produce a comprehensive clinical dataset that contained sufficient information. If patients visited the ED multiple times within 7 days, only the data from the index visit were analyzed.
Data description and preprocessing
Because our primary purpose of prediction was diagnosis of AIHD using routinely collected information aside from 12-lead ECG findings and cardiac biomarker measurements in the early ED period, only data available at the initial ED visit were used as prediction variables. For laboratory tests, we chose the initially retrieved CBC and CMP, which have been frequently used in most EDs. We selected 25 predictors according to eligibility, and a detailed description of the variables is presented in S1 Table. Among them, 23 predictors were continuous variables (age, vital signs, and blood laboratory test results), and there was a range of proportions of missing data (2.5% to 16.8%, S1 Table). Median imputation, which is a common method used to deal with missing values in machine learning models, was conducted [37]. Extreme value imputation was conducted for outlier replacement of continuous variables except for age. Using a training cohort, the 1st percentile value of each continuous variable and 99th percentile value of each continuous variable were defined as cutoff values. Values smaller than the 1st percentile cutoff or larger than the 99th percentile cutoff were defined as extreme values and replaced in both the training and validation cohorts. This method was used to develop a model that is less sensitive to extreme values, in order to reduce the effect of outliers [38].
Outcome variable
The diagnosis of AIHD, which was extracted from the ED administrative database and CDW database, was used as the outcome. We defined patients as having AIHD if both of the following conditions were satisfied. First, among patients who visited the ED, the diagnostic code according to the International Statistical Classification of Diseases and Related Health Problems (ICD-10) needed to be between I-20 and I-25, which indicates IHD. The ED administrative database has two types of primary diagnostic codes: the final diagnostic codes at ED discharge and at hospital discharge. We defined the diagnostic code as positive for ischemic heart disease if a confirmative diagnostic code was found in any level of the discharge record. Next, a diagnosis of AIHD was accepted when coronary angiography (CAG) was performed during the patient’s hospital stay. We defined patients who were discharged without CAG results as nondiagnosed.
Model development
We developed prediction models for the diagnosis of AIHD using extreme gradient boosting (XGB). The XGB algorithm operates by refitting a classifier iteratively to residuals of models [39]. The XGB algorithm was tuned using five-fold cross-validation. Grid and manual searches on the hyperparameters within the training cohort were conducted. The hyperparameters investigated in our models were as follows: nrounds: 50, 100, 150, 300, 350, 400, 450, and 500; max_depth: 2, 3, 4, 5, and 6; eta: 0.01, 0.02, 0.03, 0.04, 0.05, 0.1, and 0.3; gamma: 0, 0.05, 0.1. 0.5, 0.7, 0.9, and 1.0; colsample_bytree: 0.4, 0.6. 0.8, and 1.0; min_child_weight: 1, 2, and 3; and subsample: 0.5, 0.75, and 1.0. After tuning with cross-validations, we selected hyperparameters in the modeling as follows: nrounds: 450, max_depth = 2, eta = 0.02, gamma = 0.7, colsample_by_tree = 1, min_child_weight = 1, and subsample = 0.5.
Statistical analysis
Sample size estimation was not conducted since this was designed as a hypothesis-generating epidemiological study, and all eligible patients were included to maximize the statistical power.
Characteristics including baseline characteristics, vital signs, laboratory test results, and the study outcome were compared between the training cohort and validation cohort using the t-test or Wilcoxon rank-sum test for continuous variables and the chi-square test or Fisher exact test for categorical variables, as appropriate [40]. Cardiac tests during hospital stays were also compared between the groups according to the study outcome.
A machine-learning model using the XGB algorithm was developed using 25 predictor variables. The XGB model has been widely used in the development of prediction models in the clinical field, and it has demonstrated good performance [41, 42]. The performance of the predictive model was evaluated by the area under the receiving operating characteristic curve (AUROC) as a primary measure. We assessed calibration power using the scaled Brier score, Hosmer-Lemeshow test, and a calibration plot in the validation cohort. The test characteristics of each model in the validation cohort, including the sensitivity, specificity, and positive and negative predictive values with 95% confidence intervals, were reported. The optimal cutoff probability for evaluation of the test characteristics was calculated using the Youden index. The variable importance of the XGB model was also reported. The XGB models were ranked by variable importance on the gain, which implies the relative contribution of the corresponding variable to the model calculated by taking each variable’s contribution for each tree in the model [43]. In addition, partial dependence plots were used to determine the marginal effect of features on the predicted outcome in the XGB model.
Two baseline models were developed to compare with the XGB model and traditional model. First, because troponin is a cardiac biomarker that is most commonly used in the diagnosis of AIHD, a single-variable logistic regression model using troponin was developed to assess the usability of the XGB model in the clinical setting. Second, to compare the performance of the machine learning model and the traditional model, a logistic regression model of all predictors was developed. The variable importance of the logistic regression model was also reported. The logistic regression model was ranked by variable importance using z-statistics (the beta estimate divided by the standard error of beta). Partial dependence plots for the logistic regression model were also evaluated. Comparison of the AUROC between the XGB model and the two baseline models was performed using the De-Long test [44].
A p-value of 0.05 was considered statistically significant. All analyses were performed using R, version 3.5 (R Foundation for Statistical Computing, Vienna, Austria) with packages including caret and xgboost for the analysis of the machine learning algorithms.
Results
Characteristics of study subjects
Altogether, 10,217 patients were screened from the comprehensive dataset. After excluding patients who were transferred from other hospitals (N = 1,085), transferred to another hospital (N = 39), had symptom onset more than 7 days before their ED visit (N = 1,298), or had the highest triage level at presentation (N = 17), 5,415 patients remained. Among them, 437 patients with typical chest pain were excluded; therefore, 4,978 patients were included in the final analysis (Fig 1). The distribution of chest pain characteristics and proportion of patients with AIHD among the 5,415 patients are presented in S1 Fig.
Among the 4,978 patients, 619 (12.4%) patients were diagnosed with AIHD. The study population was divided into 2 cohorts: a training cohort of 3,833 patients and a validation cohort of 1,145 patients. The baseline characteristics of the training cohort and validation cohort are shown in Table 1. Compared to patients in the training cohort, those in the validation cohort were more likely to visit the ED earlier, to call emergency medical services (EMS), to have a higher blood pressure and heart rate, and to be diagnosed as having AIHD (11.8% vs. 14.5%). Several laboratory test results, such as WBC, total protein level, albumin level, and electrolyte level, were significantly different between the two groups, and mortality was low in both the training and validation groups (0.5% and 0.1%, respectively; p = 0.071) (Table 1).
Main results
The characteristics of the study patients according to the study outcome are presented in Table 2.
Compared to patients without AIHD, those with AIHD were more likely to be older men, to call EMS, and to have a higher systolic blood pressure (SBP) and lower heart rate. Among the 17 laboratory test results, 11 were significantly different between patients with and without AIHD (Table 3).
Among the 4,978 patients, most underwent 12-lead ECG (99.0%) and cardiac biomarker analysis (97.1%). Troponin I was elevated in 51.1% of AIHD patients and 12.3% of non-AIHD patients (p < 0.001). CCTA was conducted in 21.3% of non-AIHD patients and 12.0% of AIHD patients (p < 0.001). Seventeen non-AIHD patients underwent CAG, and their main diagnoses were as follows: heart failure, 2; arrhythmia, 2; pericardial disease, 2; cardiomyopathy, 2; and other, 2.
Classification results of the machine learning models in the validation cohort are presented in Table 4. There was no significant difference in AUROC between the XGB model and the baseline models (Table 4). The test characteristics of the prediction models are also shown in Table 4. The accuracy and F1 scores of the logistic regression model and XGB model were similar (logistic regression model: 0.66 and 0.36; XGB model: 0.67 and 0.36, respectively). Calibration metrics are presented in Fig 2. Calibration was poor in patients with a high predicted probability of AIHD in all prediction models.
The observed probability of acute ischemic heart disease (with a 95% confidence interval) is plotted against predicted good neurological recovery by 10% intervals of the predicted probability. Point size indicates the relative number of observations in a group. AIHD, acute ischemic heart disease.
Variable importance and partial dependence plot
Variable importance was calculated for the logistic regression model and XGB model (S2 Table). Age and glucose ranked first and second in both models, and there was variability in variable importance between the models. The relationship between the probability of AIHD and all features in the models was demonstrated according to importance, as shown in Fig 3. A notable nonlinear trend was observed in several features in the XGB model (Fig 3).
A. Partial dependence plot for the A) logistic regression model and B) extreme gradient boosting model. The orders of each plot are according to the variable importance of each model. GOT, aspartate transaminase; WBC, white blood cell count; GPT, alanine aminotransferase; ALP, alkaline phosphatase; BUN, blood urea nitrogen; DBP, diastolic blood pressure; CO2, carbon dioxide; SBP, systolic blood pressure; EMS, emergency medical service.
Discussion
We applied a machine-learning algorithm to patients with atypical chest pain who visited the ED in order to generate a predictive model of AIHD. We found that routinely collected data showed considerable predictive power, comparable to that of cardiac biomarkers. We also found that there were differences in variable importance between the XGB model and logistic regression model. Unlike the logistic regression model, many predictors showed nonlinear associations with the study outcome in the XGB model (Fig 3).
We found that discrimination power was comparable between the XGB model and troponin model. Cardiac biomarkers are important for the diagnosis of AIHD in patients with chest pain. Our findings suggest that a machine learning model with a combination of less relevant predictors could achieve equivalent performance to biomarkers with biological relevance. Because the XGB model showed good calibration in patients with a low probability of AIHD and identified most patients as having a low probability of AIHD (Fig 2), it can be helpful when deciding whether to conduct further cardiac tests, such as CCTA, in patients with a low risk of AIHD even when ECG and troponin findings are not available. In our study, one-fifth of non-AIHD patients underwent CCTA (Table 3). Decreasing the proportion of patients receiving CCTA may reduce radiological hazards and length of stay for those patients and preserve ED resources [45].
We also found that the discrimination power was comparable between our model and the logistic regression model. However, we found that the variable importance was markedly different between the XGB model and logistic regression model (S2 Table). The inherent linear relationship between a feature and the outcome of AIHD could contribute to different variable importance between the two models. SBP was the 24th most important variable in the logistic regression model but the 9th most important variable in the XGB model. Because the relationship between SBP and AIHD was U-shaped in the XGB model, the logistic regression model could not detect an important relationship between SBP and AIHD. A U-shaped relationship between SBP and the outcome of AIHD was also reported in a previous study [46]. GOT was among the top 5 most important variables in both models. GOT was proportionate to the risk of AIHD in the logistic regression model, but in the XGB model, that risk did not increase for GOT >75 IU/L. GOT is one of the oldest known biomarkers for AIHD [47]. However, because GOT originates from skeletal muscles or the liver rather than the heart, a high GOT level usually reflects diseases in those organs and not AIHD [48]. Therefore, the nonlinear relationship derived from the XGB model may be more compatible with biological relevance. A similar finding was also shown for glucose and WBC. Hyperglycemia and leukocytosis were associated with a high risk of AIHD in previous studies [49, 50]. However, severe hyperglycemia and leukocytosis are usually associated with an endocrine problem or an infection acquired in the ED, respectively.
The similarity of discrimination power between the XGB model and logistic regression model may be due to the preprocessing method used in our study. We replaced the extreme values of each feature in preprocessing to develop a less sensitive prediction model for outliers [38]. Because of the linear relationship between the predictor and the outcome in the logistic regression model, patients with extreme values of predictors, such as glucose, WBC, or GOT, may be classified as having AIHD in the logistic regression model. This result could diminish the discrimination power of the logistic regression model because those laboratory results could be caused by other diseases, such as an endocrine problem, infection, or hepatitis, rather than AIHD. Additionally, the inclusion of all predictors in both the XGB and logistic regression models also contributed to the similar predictive power between the models. Because the variable importance and internal processing of predictors are different between the two models, the performance of each model may be different when limited information can be used in constructing it.
We evaluated the predictive performance of routinely collected information for AIHD in patients with atypical chest pain. Atypical chest pain has been reported to have a high prevalence in AIHD patients, especially in the elderly and in women [51–53]. Because the diagnosis of AIHD in atypical chest pain is often challenging, decision support tools for the accurate diagnosis of these patients could result in decreased misdiagnosis, inappropriate discharge, and in-hospital mortality [51]. As chest pain characteristics are not routinely collected data in many administrative databases or CDW, this group of patients is not easily identified. In this study, merging various databases allowed us an opportunity to evaluate hypotheses that could not solely be addressed by one database. We found that chest pain characteristics critically affected the probability of AIHD. Patients with three typical characteristics showed a 9.5 times higher probability of AIHD compared to patients without typical characteristics. Even in patients with one typical characteristic, the probability of AIHD doubled compared to patients without typical characteristics (S1 Fig). Because uncertainty in AIHD diagnosis increases with lower numbers of typical characteristics, the utility of prediction models is greater in patients with few characteristics.
ECG and serum cardiac biomarkers are well-known predictors for diagnosing AIHD [54, 55]. Both variables can achieve a high level of performance, and prediction models and stratification tools tend to utilize these variables [10, 56]. However, we did not use these variables because we considered that each variable would show dominant significant predictive power in the model by itself, which would make it difficult to evaluate other clinically important variables. Moreover, we expected that prediction models for AIHD that do not utilize those test results would have their own clinical implications. Our model can be applied in various settings without high-level laboratory facilities or specialists who interpret ECG findings. Even in facilities with many resources, nurses can undertriage, physicians may overlook diagnoses, or ED overcrowding can create issues [57, 58]. Our model could also be applied to patients who visit the ED after their routine laboratory test was performed in an outpatient clinic or another hospital. Because our model can be applied in situations where some laboratory findings are missing, the potential coverage of our model is extensive.
This study has several limitations. First, the final diagnosis of the patient was defined based on the diagnostic code and procedure result recorded in his/her electronic medical record. This definition did not include whether the culprit lesion was observed or whether further intervention, such as ballooning or stenting, was performed. We focused on CAG rather than percutaneous coronary intervention (PCI) since this population should not be overlooked in the ED, even if they do not ultimately undergo PCI. Second, this study was conducted based on a population that visited one tertiary ED; thus, further external validation is required for data generalization. Third, only patients with chest pain who visited the ED were enrolled, and the chief complaint was determined by a triage nurse. Patients with dyspnea, syncope, or palpitation but no chest pain at the time of their ED visit were not included. In addition, patients with altered mental state who could not verbalize their complaint were not enrolled.
Conclusion
In summary, we used the XGB algorithm to develop and validate prediction models for AIHD in patients with atypical chest pain who visited the ED. Our prediction model showed similar performance to the troponin and logistic regression models for detecting AIHD. However, we identified a notable nonlinear relationship between predictors and the study outcome and a different variable importance pattern by using the XGB model. Further prospective validation of our results is warranted, and a response protocol based on our model should be evaluated. Because we developed our prediction model using routinely collected data, a rapid response system based on our model may be applied more broadly to critical patients in the emergency setting. An automatic screening process that uses basic important variables and routine testing should be considered.
Supporting information
S1 Fig. Number of patients with certain chest pain characteristics and proportion of patients with acute ischemic heart disease in each population.
AIHD, acute ischemic heart disease.
https://doi.org/10.1371/journal.pone.0241920.s001
(DOCX)
S1 Table. List of analyzed variables.
*Values of median, 1st percentile and 99th percentile of continuous variables were extracted from the training cohort. All continuous variables were preprocessed for extreme values by replacing the 1st and 99th percentiles. ED, emergency department; N/A, not applicable; CBC, complete blood count.
https://doi.org/10.1371/journal.pone.0241920.s002
(DOCX)
S2 Table. Full ranking of important variables for the prediction models.
Variable importance was determined based on the z value (logistic regression) and gain (XGB) obtained from the training cohort. XGB, extreme gradient boosting; EMS, emergency medical service.
https://doi.org/10.1371/journal.pone.0241920.s003
(DOCX)
References
- 1. Ahmed S, Khan A, Ali SI, Saad M, Jawaid H, Islam M, et al. Differences in symptoms and presentation delay times in myocardial infarction patients with and without diabetes: A cross-sectional study in Pakistan. Indian Heart J. 2018;70(2):241–5. pmid:29716701
- 2. Sharp AL, Baecker AS, Shen E, Redberg R, Lee MS, Ferencik M, et al. Effect of a HEART Care Pathway on Chest Pain Management Within an Integrated Health System. Ann Emerg Med. 2019;74(2):171–80. pmid:30797573
- 3. Benjamin EJ, Muntner P, Alonso A, Bittencourt MS, Callaway CW, Carson AP, et al. Heart Disease and Stroke Statistics-2019 Update: A Report From the American Heart Association. Circulation. 2019;139(10):e56–e528. pmid:30700139
- 4. Canto JG, Shlipak MG, Rogers WJ, Malmgren JA, Frederick PD, Lambrew CT, et al. Prevalence, clinical characteristics, and mortality among patients with myocardial infarction presenting without chest pain. JAMA. 2000;283(24):3223–9. pmid:10866870
- 5. Hess EP, Brison RJ, Perry JJ, Calder LA, Thiruganasambandamoorthy V, Agarwal D, et al. Development of a clinical prediction rule for 30-day cardiac events in emergency department patients with chest pain and possible acute coronary syndrome. Ann Emerg Med. 2012;59(2):115–25 e1. pmid:21885156
- 6. Barrabes JA, Bardaji A, Jimenez-Candil J, Bodi V, Freixa R, Vazquez R, et al. Characteristics and Outcomes of Patients Hospitalized With Suspected Acute Coronary Syndrome in Whom the Diagnosis is not Confirmed. Am J Cardiol. 2018;122(10):1604–9. pmid:30213384
- 7. Bugiardini R, Ricci B, Cenko E, Vasiljevic Z, Kedev S, Davidovic G, et al. Delayed Care and Mortality Among Women and Men With Myocardial Infarction. J Am Heart Assoc. 2017;6(8).
- 8. Fakhri Y, Schoos MM, Sejersten M, Ersboll M, Valeur N, Kober L, et al. Prehospital electrocardiographic acuteness score of ischemia is inversely associated with neurohormonal activation in STEMI patients with severe ischemia. J Electrocardiol. 2017;50(1):90–6.
- 9. Body R, Mueller C, Giannitsis E, Christ M, Ordonez-Llanos J, de Filippi CR, et al. The Use of Very Low Concentrations of High-sensitivity Troponin T to Rule Out Acute Myocardial Infarction Using a Single Blood Test. Acad Emerg Med. 2016;23(9):1004–13. pmid:27178492
- 10. Hess EP, Agarwal D, Chandra S, Murad MH, Erwin PJ, Hollander JE, et al. Diagnostic accuracy of the TIMI risk score in patients with chest pain in the emergency department: a meta-analysis. CMAJ. 2010;182(10):1039–44. pmid:20530163
- 11. Mahler SA, Miller CD, Hollander JE, Nagurney JT, Birkhahn R, Singer AJ, et al. Identifying patients for early discharge: performance of decision rules among patients with acute chest pain. Int J Cardiol. 2013;168(2):795–802. pmid:23117012
- 12. Pope JH, Aufderheide TP, Ruthazer R, Woolard RH, Feldman JA, Beshansky JR, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med. 2000;342(16):1163–70. pmid:10770981
- 13. Calle P, Jordaens L, De Buyzere M, Rubbens L, Lambrecht B, Clement DL. Age-related differences in presentation, treatment and outcome of acute myocardial infarction. Cardiology. 1994;85(2):111–20. pmid:7954562
- 14. Amsterdam EA, Wenger NK, Brindis RG, Casey DE Jr., Ganiats TG, Holmes DR Jr., et al. 2014 AHA/ACC Guideline for the Management of Patients with Non-ST-Elevation Acute Coronary Syndromes: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;64(24):e139–e228. pmid:25260718
- 15. Ibanez B, James S, Agewall S, Antunes MJ, Bucciarelli-Ducci C, Bueno H, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J. 2018;39(2):119–77.
- 16. Nilsson T, Lundberg G, Larsson D, Mokhtari A, Ekelund U. Emergency Department Chest Pain Patients With or Without Ongoing Pain: Characteristics, Outcome, and Diagnostic Value of the Electrocardiogram. J Emerg Med. 2020;58(6):874–81.
- 17. Laugaudin G, Kuster N, Petiton A, Leclercq F, Gervasoni R, Macia JC, et al. Kinetics of high-sensitivity cardiac troponin T and I differ in patients with ST-segment elevation myocardial infarction treated by primary coronary intervention. Eur Heart J Acute Cardiovasc Care. 2016;5(4):354–63. pmid:25943557
- 18. Agnello L, Bellia C, Scazzone C, Bivona G, Iacolino G, Gambino CM, et al. Establishing the 99(th) percentile for high sensitivity cardiac troponin I in healthy blood donors from Southern Italy. Biochem Med (Zagreb). 2019;29(2):020901.
- 19. Bivona G, Agnello L, Bellia C, Lo Sasso B, Ciaccio M. Diagnostic and prognostic value of H-FABP in acute coronary syndrome: Still evidence to bring. Clin Biochem. 2018;58:1–4. pmid:29698621
- 20. Bivona G, Bellia C, Lo Sasso B, Agnello L, Scazzone C, Novo G, et al. Short-term Changes in Gal 3 Circulating Levels After Acute Myocardial Infarction. Arch Med Res. 2016;47(7):521–5. pmid:28262193
- 21. Agnello L, Bellia C, Lo Sasso B, Pivetti A, Muratore M, Scazzone C, et al. Establishing the upper reference limit of Galectin-3 in healthy blood donors. Biochem Med (Zagreb). 2017;27(3):030709. pmid:29180917
- 22. Zarling EJ, Sexton H, Milnor P Jr., Failure to diagnose acute myocardial infarction. The clinicopathologic experience at a large community hospital. JAMA. 1983;250(9):1177–81. pmid:6876356
- 23. Gibler WB, Runyon JP, Levy RC, Sayre MR, Kacich R, Hattemer CR, et al. A rapid diagnostic and treatment center for patients with chest pain in the emergency department. Ann Emerg Med. 1995;25(1):1–8. pmid:7802357
- 24. Backus BE, Six AJ, Kelder JC, Bosschaert MA, Mast EG, Mosterd A, et al. A prospective validation of the HEART score for chest pain patients at the emergency department. Int J Cardiol. 2013;168(3):2153–8. pmid:23465250
- 25. Madjid M, Awan I, Willerson JT, Casscells SW. Leukocyte count and coronary heart disease: implications for risk assessment. J Am Coll Cardiol. 2004;44(10):1945–56. pmid:15542275
- 26. Madjid M, Fatemi O. Components of the complete blood count as risk predictors for coronary heart disease: in-depth review and update. Tex Heart Inst J. 2013;40(1):17–29. pmid:23467296
- 27. Fan Y, Li Y, Chen Y, Zhao YJ, Liu LW, Li J, et al. Comprehensive Metabolomic Characterization of Coronary Artery Diseases. J Am Coll Cardiol. 2016;68(12):1281–93. pmid:27634119
- 28. Obermeyer Z, Emanuel EJ. Predicting the Future—Big Data, Machine Learning, and Clinical Medicine. N Engl J Med. 2016;375(13):1216–9. pmid:27682033
- 29. Zhang Z, Zhao Y, Canes A, Steinberg D, Lyashevska O, written on behalf of AMEB-DCTCG. Predictive analytics with gradient boosting in clinical medicine. Ann Transl Med. 2019;7(7):152. pmid:31157273
- 30. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD). Ann Intern Med. 2015;162(10):735–6.
- 31. Kwon H, Kim YJ, Jo YH, Lee JH, Lee JH, Kim J, et al. The Korean Triage and Acuity Scale: associations with admission, disposition, mortality and length of stay in the emergency department. Int J Qual Health Care. 2018.
- 32. Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of scores on The Emergency Severity Index version 3. Acad Emerg Med. 2004;11(1):59–65. pmid:14709429
- 33. Gibbons RJ, Balady GJ, Bricker JT, Chaitman BR, Fletcher GF, Froelicher VF, et al. ACC/AHA 2002 guideline update for exercise testing: summary article: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Update the 1997 Exercise Testing Guidelines). Circulation. 2002;106(14):1883–92. pmid:12356646
- 34.
Godara H. The Washington manual of medical therapeutics: Lippincott Williams & Wilkins; 2013.
- 35. Diamond GA. A clinically relevant classification of chest discomfort. J Am Coll Cardiol. 1983;1(2 Pt 1):574–5. pmid:6826969
- 36. Kang J, Kim JH, Lee KH, Lee WS, Chang HW, Kim JS, et al. Risk Factor Analysis of Extended Opioid Use after Coronary Artery Bypass Grafting: A Clinical Data Warehouse-Based Study. Healthc Inform Res. 2019;25(2):124–30. pmid:31131147
- 37. Batista GEAPA, Monard MC. An analysis of four missing data treatment methods for supervised learning. Appl Artif Intell. 2003;17(5–6):519–33.
- 38. Andersson JLR, Graham MS, Zsoldos E, Sotiropoulos SN. Incorporating outlier detection and replacement into a non-parametric framework for movement and distortion correction of diffusion MR images. Neuroimage. 2016;141:556–72. pmid:27393418
- 39. Chen T, Guestrin C. XGBoost. 2016:785–94.
- 40. Zhang Z, Gayle AA, Wang J, Zhang H, Cardinal-Fernandez P. Comparing baseline characteristics between groups: an introduction to the CBCgrps package. Ann Transl Med. 2017;5(24):484.
- 41. Frizzell JD, Liang L, Schulte PJ, Yancy CW, Heidenreich PA, Hernandez AF, et al. Prediction of 30-Day All-Cause Readmissions in Patients Hospitalized for Heart Failure: Comparison of Machine Learning and Other Statistical Approaches. JAMA Cardiol. 2017;2(2):204–9.
- 42. Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care. 2019;23(1):112.
- 43. Shi X, Wong YD, Li MZ, Palanisamy C, Chai C. A feature learning approach based on XGBoost for driving assessment and risk prediction. Accid Anal Prev. 2019;129:170–9. pmid:31154284
- 44. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45. pmid:3203132
- 45. Hoffmann U, Truong QA, Schoenfeld DA, Chou ET, Woodard PK, Nagurney JT, et al. Coronary CT angiography versus standard evaluation in acute chest pain. N Engl J Med. 2012;367(4):299–308. pmid:22830462
- 46. Shiraishi J, Kohno Y, Sawada T, Hashimoto S, Ito D, Kimura M, et al. Prognostic impact of systolic blood pressure at admission on in-hospital outcome after primary percutaneous coronary intervention for acute myocardial infarction. J Cardiol. 2012;60(2):139–44. pmid:22521431
- 47. Nissen NI, Ranlov P, Weis-Fogh J. Evaluation of Four Different Serum Enzymes in the Diagnosis of Acute Myocardial Infarction. Br Heart J. 1965;27:520–6. pmid:14324110
- 48. Johnson RD O'Connor ML, Kerr RM. Extreme serum elevations of aspartate aminotransferase. Am J Gastroenterol. 1995;90(8):1244–5. pmid:7639223
- 49. Undas A, Wiek I, Stepien E, Zmudka K, Tracz W. Hyperglycemia is associated with enhanced thrombin formation, platelet activation, and fibrin clot resistance to lysis in patients with acute coronary syndrome. Diabetes Care. 2008;31(8):1590–5. pmid:18487475
- 50. Nunez J, Nunez E, Sanchis J, Bodi V, Llacer A. Prognostic value of leukocytosis in acute coronary syndromes: the cinderella of the inflammatory markers. Curr Med Chem. 2006;13(18):2113–8. pmid:16918341
- 51. Ricci B, Cenko E, Varotti E, Puddu PE, Manfrini O. Atypical Chest Pain in ACS: A Trap Especially for Women. Curr Pharm Des. 2016;22(25):3877–84. pmid:26956231
- 52. Nowak KA. Atypical chest pain in the elderly. Nurse Pract. 1997;22(5):11, 4. pmid:9172231
- 53. Summers RL, Cooper GJ, Carlton FB, Andrews ME, Kolb JC. Prevalence of atypical chest pain descriptions in a population from the southern United States. Am J Med Sci. 1999;318(3):142–5. pmid:10487403
- 54. Kim JH, Roh YH, Park YS, Park JM, Joung BY, Park IC, et al. Risk score to predict false-positive ST-segment elevation myocardial infarction in the emergency department: a retrospective analysis. Scand J Trauma Resusc Emerg Med. 2017;25(1):61. pmid:28666458
- 55. Wu CC, Hsu WD, Islam MM, Poly TN, Yang HC, Nguyen PA, et al. An artificial intelligence approach to early predict non-ST-elevation myocardial infarction patients with chest pain. Comput Methods Programs Biomed. 2019;173:109–17. pmid:31046985
- 56. Body R. Acute coronary syndromes diagnosis, version 2.0: Tomorrow's approach to diagnosing acute coronary syndromes? Turk J Emerg Med. 2018;18(3):94–9. pmid:30191187
- 57. Schull MJ, Vermeulen M, Slaughter G, Morrison L, Daly P. Emergency department crowding and thrombolysis delays in acute myocardial infarction. Ann Emerg Med. 2004;44(6):577–85. pmid:15573032
- 58. Kuhn L, Worrall-Carter L, Ward J, Page K. Factors associated with delayed treatment onset for acute myocardial infarction in Victorian emergency departments: a regression tree analysis. Australas Emerg Nurs J. 2013;16(4):160–9. pmid:24199901