Artificial intelligence (AI) techniques are increasingly applied to cardiovascular (CV) medicine in arenas ranging from genomics to cardiac imaging analysis. Cardiac Phase Space Tomography Analysis (cPSTA), employing machine-learned linear models from an elastic net method optimized by a genetic algorithm, analyzes thoracic phase signals to identify unique mathematical and tomographic features associated with the presence of flow-limiting coronary artery disease (CAD). This novel approach does not require radiation, contrast media, exercise, or pharmacological stress. The objective of this trial was to determine the diagnostic performance of cPSTA in assessing CAD in patients presenting with chest pain who had been referred by their physician for coronary angiography.
This prospective, multicenter, non-significant risk study was designed to: 1) develop machine-learned algorithms to assess the presence of CAD (defined as one or more ≥ 70% stenosis, or fractional flow reserve ≤ 0.80) and 2) test the accuracy of these algorithms prospectively in a naïve verification cohort. This report is an analysis of phase signals acquired from 606 subjects at rest just prior to angiography. From the collective phase signal data, features were extracted and paired with the known angiographic results. A development set, consisting of signals from 512 subjects, was used for machine learning to determine an algorithm that correlated with significant CAD. Verification testing of the algorithm was performed utilizing previously untested phase signals from 94 subjects.
The machine-learned algorithm had a sensitivity of 92% (95% CI: 74%-100%) and specificity of 62% (95% CI: 51%-74%) on blind testing in the verification cohort. The negative predictive value (NPV) was 96% (95% CI: 85%-100%).
Citation: Stuckey TD, Gammon RS, Goswami R, Depta JP, Steuter JA, Meine FJ III, et al. (2018) Cardiac Phase Space Tomography: A novel method of assessing coronary artery disease utilizing machine learning. PLoS ONE 13(8): e0198603. https://doi.org/10.1371/journal.pone.0198603
Editor: Dalin Tang, Worcester Polytechnic Institute, UNITED STATES
Received: December 8, 2017; Accepted: May 22, 2018; Published: August 8, 2018
Copyright: © 2018 Stuckey et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: A4L, Inc. (Morrisville, NC) was the sole funder of this research. Independent investigators at each study site of this multi-center prospective trial collected the data. Scientists and clinical staff of A4L assisted in the analysis of the data and the preparation of the manuscript. The funder provided support in the form of salaries only for authors who were employees of A4L, Inc. (Morrisville, NC) or Analytics For Life, Inc. (Toronto, ON) [SR, TB, PG, AK, IS, WES]. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: Authors SR, TB, PG, AK, IS, WES are employees of A4L or Analytics For Life (https://www.analytics4life.com/) and receive salary for that employment. This does not alter our adherence to all the PLOS ONE policies on sharing data and materials. There are no patents, products in development or marketed products to declare. All other authors have no competing interests to declare.
The broad application of artificial intelligence (AI) techniques to all facets of medicine is rapidly changing practice economics as well as the available clinical tools for treatment and diagnosis [1–4]. In cardiovascular medicine, machine learning (ML) has been applied in arenas ranging from genomics to cardiac imaging analysis [1–2]. Recent refinements in the methods of machine learning, including deep learning/neural networks and cognitive computing, now permit the expansion of AI analysis beyond the traditional realm of “big data” outcomes to that of novel diagnostic tests [2, 5]. ML provides a pathway to improve the accuracy and reliability of diagnostic modalities, and has potential to significantly contribute, on multiple levels, to achieving the goal of precision medicine in the cardiovascular area.
Cardiac Phase Space Tomography Analysis (cPSTA) is a novel method to assess the presence of significant CAD in major coronary arteries, defined as ≥ 70% stenosis by angiography or ≤ 0.80 fraction flow reserve (FFR). A hand-held device collects a patient’s resting phase signals without the use of ionizing radiation, contrast agents, exercise, or pharmacologic stressors. The acquired 10 million data points are transferred to the cloud and evaluated by an analytic engine (CAD AE) employing machine-learned algorithms/predictors. The results are subsequently displayed as a phase space tomography model, which is accessible via a web portal.
We now report the development of machine-learned algorithms for assessing CAD and the performance of the final predictors in a blindly tested cohort compared to angiography results. These findings are an analysis of the initial algorithm development and validation stage (Stage I) of the ongoing Coronary Artery Disease Learning & Algorithm Development (CADLAD) trial .
Stage I of the CADLAD trial entailed two distinct steps. First, a prospective, non-randomized development step was conducted to generate machine-learned algorithms for assessment of the presence of significant coronary artery disease (CAD) using paired phase signals with clinical outcomes data. Second, a blinded, paired comparison was performed to verify the performance of the machine-learned algorithm from step 1 with regard to the assessment of significant CAD as determined by coronary angiography, which represents the “gold standard” for evaluation of CAD. A patient was considered CAD positive for development purposes if at least one of the two following criteria were fulfilled: 1) one lesion had a stenosis of ≥ 70% or 2) if at least one lesion has a reduced fractional flow reserve (FFR) of ≤ 0.80. This is consistent with the accepted definition of clinically significant diameter narrowing of a coronary artery [7–8]. Based on the American College of Cardiology (ACC) guidelines . FFR evaluation was the final determinate of CAD in vessels undergoing flow wire assessment.
In order to accurately machine learn, the results of a known standard must be paired with a signal to develop a solution that correctly reflects actual physiologic status. In this trial, all subjects (N = 606) had been referred for coronary angiography, and all subjects underwent the planned catheterization procedure with dye injections of both right and left coronary arteries. The CADLAD trial was designed to utilize the results of coronary angiograms as the “standard” and paired these catheterization findings with phase signals for the purpose of machine learning. Thereby, the machine-learning platform developed solutions to assess the presence of CAD. This method required enrollment of only subjects already referred by their physician for coronary angiography.
An objective of the protocol was that the results should reflect a broad cross section of clinical practice, and this goal prompted selection of hospitals and clinics that mirrored the diverse array of facilities providing care to patients with heart disease. The aim was attainment of machine-learned solutions that could be generalized. The physicians participating in the protocol followed angiogram interpretation consistent with the ACC guidelines for determining CAD including the use of flow wire when recommended . Employing this well-established categorization of lesion significance replicates the current general cardiology practice with reference to interpretation.
Written informed consent was obtained from all participants in this trial. The individuals in this manuscript have given written informed consent (as outlined in PLOS consent form) to publish these case details. The non-significant risk protocol for the CADLAD trial was approved by a central institutional review board (Western IRB) and performed at 12 enrolling centers in the United States. All study sites were community-based hospitals with expertise in interventional cardiology.
All participants had suspected but not known CAD and had been referred by their physicians for nonurgent angiography. Additional inclusion criteria included age > 21 years old and the ability to understand protocol requirements as well as provide informed consent. Exclusion criteria were prior documented myocardial infarction (MI) or previous percutaneous coronary interventions (PCI), prior coronary artery bypass grafting (CABG), indication for invasive coronary angiography other than to assess for obstructive CAD (e.g. arrhythmia, cardiomyopathy, valvular abnormality), previous heart valve replacement, previous sustained or paroxysmal atrial or ventricular arrhythmia, infiltrative myocardial disease (amyloid, sarcoid, right ventricular dysplasia), presence of cardiac implantable electronic device, including implantable cardioverter defibrillator, pacemaker, implantable loop recorders and other monitors, implantable neuro-stimulators, congenital heart disease, breast implants, pregnancy, breast feeding, currently taking any Type IA, IC or III antiarrhythmic, any history of amiodarone therapy, clinically significant chest deformity (e.g., pectus excavatum or pectus carinatum), and neuromuscular disease if the condition resulted in tremor or muscle fasciculations.
System and device description
The cPSTA System is a medical device system that uses tomography to analyze phase signals and assess the presence of significant coronary artery disease in the major coronary arteries (Fig 1). The first component is the Phase Signal Acquisition (PSAQ) System which includes: 1) the phase signal recorder (PSR), a hand-held instrument that acquires and transmits resting phase signals along with ancillary patient-specific information (gender, age, etc.); and 2) a cloud-based phase signal data repository (PSDR) that accepts, stores, and allows retrieval of the signals as well as ancillary patient-specific information. The second component is a coronary artery disease analytical engine (CAD AE) that processes and evaluates the phase signals to assess the presence and significance of coronary artery disease using the machine-learned algorithms. The final component is Cloud Services (CS), including the health care provider (HCP) Web Portal that the clinician utilizes to interpret images, review results, and generate a report. The report can be saved as a record for inclusion in the patient’s medical record.
Phase signal data are collected and transferred to cloud. The generated models and analysis are available for physician assessment. cPSTA System = Cardiac Phase Space Tomography Analysis System, CAD = coronary artery disease. Reprinted from presentation materials of A4L under a license, with permission from A4L and W20, original production 2016.
After providing informed consent, participants underwent phase signal acquisition within seven days prior to angiography. Signals were captured utilizing the hand-held (PSR) device via seven sensors positioned on the chest and back. Phase signal data was collected for approximately 3 minutes and the data package was then transmitted wirelessly to the cloud based PSDR. An analytic engine, consisting of software based on the machine-learned algorithms, analyzed the acquired data and generated tomography images. The results can be made available through a secure web portal (Fig 2); however, in this trial, the outcomes from cPSTA were not provided to the physician caring for the participant, and no clinical decision-making was based on the cPSTA findings. All phase signals were procured by trained staff at each enrolling center. Angiography was performed by local physicians who made all clinical decisions regarding appropriate medical therapy. If the interventional cardiologist observed a lesion of questionable significance, use of a fractional flow wire was at his/her discretion. The participant’s involvement in the study was complete following acquisition of the phase signals and performance of the angiography. There was no required follow-up.
The learning phase pairs “gold standard” results with phase signals for machine learning to develop algorithms. The verification phase tests the performance of the final algorithms on naïve signal data. cPSTA System = Cardiac Phase Space Tomography Analysis System. Reprinted from presentation materials of A4L under a license, with permission from A4L and W20, original production 2016.
Statistical analysis of the machine learned algorithm performance was done by direct calculation of the sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) using standard formulas. Bootstrap confidence intervals were used to assess the significance of these scores. Each subject was categorized into true negative (TN), true positive (TP), false negative (FN) and false positive (FP) by using a fixed, experimentally-determined threshold on the continuous output of the predictor, with an output below this threshold indicating a prediction of CAD negative; otherwise, the predictor is indicating CAD positive.
The 95% confidence intervals on these statistics were computed with the use of bootci, a MatLab R2016b (MathWorks; Natick, MA) function that returns the intervals employing bootstrap evaluation . The receiver-operator characteristic curve (ROC) and its corresponding area under the curve (AUC) were computed with the use of the MatLab R2016b function perfcurve .
Machine learning and algorithm development
The ML method employed was elastic net, which is a linear regression method with L1 and L2 regularization penalties . The development set of signals (N = 512) was divided into separate training (N = 339) and validation (N = 173) sets. Elastic net was chosen because the combination of L1 and L2 penalties helps the approach overcome issues surrounding high-dimensional data with small training sets. In addition, it has been shown that linear regression techniques only need two independent observations per variable to avoid overfitting . The elastic net received as input a set of features that represent tomographic and dynamical properties of the signal, along with the subject’s age, gender and heart rate. The features used in the machine-learned algorithms were extracted from each signal, meaning the signal is a sample and the features are components of the sample. A total of 405 features were used, with correlation-based feature reduction being applied before the linear model was learned (see below). When a high number of features are used for development of the model, dimensionality problems may arise with machine learning. High regularization methods can be used effectively to avoid overfitting and ameliorate the difficulties related to this issue . Elastic net employed in our study automatically selects and regularizes the features to overcome the dimensionality issues. Elastic net has been shown to effectively train predictive models in high dimensional datasets with small sample sizes .
Additionally, the quality of subject’s signal was evaluated with a noise score (“noise volume”) designed to evaluate the impact of noise in the signal through changes in three-dimensional phase space by environmental factors such as complex powerline harmonics, medical imaging devices, and other electrical equipment in the proximity of the acquisition.
The elastic net algorithm learns a weight (wj) for each extracted feature (fj) with the final prediction for a given participant being calculated as where b is the bias term. The L1 and L2 regularization methods penalized large weight values during learning, which helps prevent overfitting on the training set. Elastic net optimizes the above summation using the following loss function, incorporating both the fit of the output from the model as well as the regularization terms.
The parameters α and λ control the L1 and L2 penalties. Training was performed on a modified (continuous) Gensini score as the target , which applied severity and location scores for each lesion in the coronary tree and summed over the lesions to produce the score. This model only took into account the subject’s worst-case lesion (that with the maximal product of severity and lesion scores) and applied a logarithm such that it was more tractable for a linear model. A dampening factor of 0.25 was applied to the worst-case Gensini score when collaterals were present in order to increase the value of that modified Gensini score in predicting the subject’s true (binary) CAD label.
As with all machine learning algorithms, elastic net has several hyper-parameters, which modify the optimization process (e.g., α and λ). The implementation employed supported the upweighting of data points in the training set based on a threshold on the modified Gensini score, allowing for the adjustment of the importance of various subject categories during learning. An additional supported parameter is the reduction of the feature set by removing features that are correlated over a specific threshold. For the model reported here, a correlation threshold of 0.907 was used, reducing the feature set from 405 features to 315.
To automatically tune these various hyper-parameters, composed of weight, Gensini threshold for weighting, correlation reduction threshold, and two elastic net-specific hyper-parameters α and λ, a genetic algorithm (GA) was utilized [16,17]. At each generation of the GA, the performance of every genome (set of parameter values) in the current iteration was evaluated by inputting its values into elastic net, running elastic net, and measuring the final model’s performance on both the training and validation sets. The model’s continuous output of the modified Gensini score was treated as a predictor of the subjects’ binary CAD status. The model’s performance on these datasets was measured using the area under the receiver operator characteristic curve, so that all possible thresholds on the continuous output were considered. Since CAD is a continuous disease, with a large variety of lesion locations and sizes within the disease-positive and disease-negative cohorts, the modified Gensini score allowed the linear model to predict the disease along the natural continuum, and binning was only considered when assessing the fitness of the hyperparameters. Additionally, the fitness function allowed higher quality data, as measured by noise volume, to receive a higher weighting in the performance assessment. The fitness function is summarized in equation form below where T and V denote training and validation sets respectively and i denotes the noise subset as identified by thresholds on the noise volume. Five overlapping noise subsets are used as corresponding to four noise thresholds, where all data is contained in the largest noise set, but data above the first threshold was excluded from the second noise subset, and so on. Defining the fitness function to optimize the AUC on 10 different overlapping noise subsets from the development set has the advantage of avoiding overfitting any single set as well as optimizing the performance on all the sets.
Genomes whose parameters enabled elastic net to find good models relative to other genomes in the population were selected for reproduction to create new genomes. During reproduction, each parameter in a genome underwent “mutation” with a probability of 0.33, with mutations either perturbing the parameter by a random value or replacing it with a new random value. Half of all new genomes also underwent single-point crossover, where a random position on the genome was selected, with the parameters before this point coming from one high-performing genome from the previous generation, and the second set of parameters coming from a different, high-performing genome.
These new genomes become the population for the next generation, and the whole process repeated until no further improvements in fitness scores were observed for 10 consecutive generations of the GA. The resultant linear model that adequately assessed significant CAD was embedded in the analytic software and verified against the naïve cohort.
Enrollment began in May 2016 and was completed in June 2017. There were 606 participants in the study, with 159 (31%) having protocol defined obstructive coronary lesions. The algorithm development cohort (training [N = 339] and validation [N = 173] sets) consisted of signals from 512 participants with an additional 94 participants’ signals serving as a naïve verification cohort (not utilized for machine learning purposes). The verification cohort was used for the blind testing of the machine-learned predictor. Demographics for the studied population are shown in Table 1. There was no significant difference between the development and verification cohorts except with regard to age of the subjects (p = 0.04). The entire population, both development and verification cohorts, included nearly 40% women. However, in this population presenting with chest pain and referred for angiography, only one third of participants had significant obstructive lesions as defined by the protocol at coronary angiography.
Performance of the machine-learned predictor
The final output of ML, the fixed algorithms incorporating features of the signal, were embedded within the CAD AE. Testing of the analytic engine of the cPSTA System blindly in the naïve verification cohort showed the final predictors had a sensitivity of 92% (95% CI: 74%-100%) and specificity of 62% (95% CI: 51%-74%) for the assessment of obstructive coronary artery disease . There was an automatic selection of a clinically-relevant threshold from the ROC curve. The negative predictive value (NPV) was 96% (95% CI: 85%-100%) and the PPV = 46% (95% CI: 33%-62%).
Chest pain is ubiquitous, with studies showing a lifetime prevalence of 20–40% in the general population . In those patients with chest pain that have been determined to have a pretest intermediate or high of coronary artery disease (CAD) by careful physical exam and history, functional (stress) diagnostic testing or computed tomographic angiography (CTA) is recommended . Positive results from any of these tests presumably define the highest risk population for which coronary angiography may be required to determine if intervention is needed. In the United States, millions of stress tests are performed each year to evaluate patients presenting with complaints of stable chest pain [20–21]. Although simple treadmill testing with ECG evaluation may provide evidence of ischemia, cardiac stress testing with imaging comprises over 85% of the evaluations acquired, and the majority of tests are nuclear myocardial perfusion imaging (MPI) studies with single photon emission computed tomography (SPECT) . This test typically entails exercising on a treadmill, injection of a radionuclide tracer, and two approximately 30-minute gamma camera scans, which occur 4 hours apart. Over seven million SPECT MPI studies are performed annually in the United States  at an average cost of $1000 . Yet, in a recent study examining over 10,000 pretest intermediate risk patients, only 10–12% undergoing functional testing or CTA had positive results, and when those with post-test high probability of disease patients subsequently underwent angiography, less than half were shown to have significant obstructive CAD . In large registry trials of patients without known heart disease who underwent angiography, less than 42% showed obstructive CAD . The currently available screening imaging tests for CAD typically require physical or pharmacologic stress, commonly involve radiation exposure, and uniformly incur substantial cost. Functional testing and CTA are utilized in a gatekeeping fashion designed to risk stratify patients with chest pain and elucidate those individuals that would benefit from angiography [24–25]. Little has changed with regard to the accuracy of these technologies in the last decade, and better tools for screening CAD are needed.
In a community-based population presenting with chest pain and deemed by their physician to warrant angiography, we examined the utility of a novel form of tomography focusing on machine-learned predictors for the assessment of obstructive coronary arteries. The principle findings of this evaluation using data from Stage I of the CADLAD trial are 1) features extracted from phase signals can be employed in ML to develop final mathematical predictors that assesses the presence of significant CAD with an ROC-AUC = 0.80 (95% CI: 0.70, 0.88); 2) performance of the cPSTA is comparable to the most commonly employed functional test of MPI (Table 2); and 3) acquisition of phase signals is extremely safe and requires minimal patient time.
Artificial intelligence is rapidly evolving and its dramatic impact on physicians practice and the tools they utilize is beginning to take shape. In cardiovascular medicine during the last 10 years, ML techniques have been applied with success to disease diagnosis and prediction . We report the development of machine-learned algorithms designed to assess the most commonly encountered cardiovascular disorder, coronary artery disease. The cPSTA, in this cohort, had a 92% sensitivity with a negative predictive value of 96%. The system was explicitly optimized (threshold chosen using the AUC-ROC curve) to maximize safety and, therefore sensitivity. The specificity of 62% remains comparable to other functional tests. Signals are collected with the patient at rest without radiation or stress, physical or otherwise.
Both MPI and CTA subject the patient to substantial radiation exposure ranging 8–12 mSv and 12–25 mSV for CTA and MPI, respectively . Functional testing requires stress, physical or pharmacologic, which carries some risks including that of allergic reaction. Performance of CTA necessitates the use of contrast media, with potential deleterious renal effects. SPECT MPI has been performed for decades and remains the predominate imaging procedure utilized prior to cardiac catheterization. Practice patterns for evaluating chest pain that employ imaging studies have changed minimally over the years due to a lack of alternative technologies with equivalent or higher accuracy and enhanced safety.
ML is now an essential component for solving complex problems in most sciences and these same methods provide immense possibilities in medicine . ML techniques can accommodate various “configurations of data, assign context weighting, and calculate the predictive power of every combinations of variables available” to evaluate the diagnostic elements . We chose elastic net to learn linear models, as the L1 and L2 penalties help prevent overfitting, while linear regression requires only two independent observations per variable . The main drawback with this approach is that it cannot learn potentially important nonlinear relationships between variables. As precision cardiovascular medicine evolves, machine-learned methods will increasingly provide diagnostic tools that permit better physician assessment of the individual patient’s condition, thereby augmenting safety and illuminating best care pathways. Machine learned solutions are being rapidly applied in cardiovascular medicine. Recently, lesion-specific ischemia, as assessed by invasive fractional flow reserve, was predicted by an integrated machine learning ischemia risk score based on quantitative plaque measures from CTA . Wearable technologies that record cardiac function with embedded machine learned algorithms have demonstrated the capability to distinguished compensated from decompensated heart failure . In the near future, new diagnostic tools based on machine learning will become widely available. Our analysis of Cardiac Phase Space Tomography suggests that machine-learned algorithms may offer a valuable new method for the assessment of patients with coronary artery disease.
The development of machine learned algorithms and their subsequent incorporation into diagnostic tools presents multiple challenges. First, in this pilot trial the models were driven by a relatively small sample size. When machine learning with reduced sample sizes, overfitting must be avoided, and we utilized methods to overcome this potential source of error. Second, the algorithm was trained and evaluated in a specific population, patients with pretest intermediate to high risk of coronary artery disease. In order to generalize the framework, a larger and more heterogeneous data set would be required. Finally, the effects of environmental noise on the unfiltered signals and thus, the resultant predictors require further investigation in order to attain optimal models.
The cPSTA System is a novel noninvasive tomographic imaging method for the detection of clinically significant CAD, which is based on machine learning. cPSTA exhibits comparable diagnostic performance to existing functional and anatomical modalities without the requirement of cardiac stress (exercise or pharmacological) and without exposure of the patient to radioactivity. This technology may provide a new and efficient technique for assessing the presence of obstructive coronary lesions in patients presenting with chest pain suspected to be of cardiac etiology.
- 1. Narula S, Shameer K, Mabrouk SO, Dudley JT, Sengupta PP. Machine-learned algorithms to automate morphological and functional assessments in 2D echocardiography. J Am Coll Cardiol 2016;68:2287–95. pmid:27884247
- 2. Krittanawong C, Zhang H, Wang Z, Aydar M, Kitai T. Artificial intelligence in precision cardiovascular medicine. J Am Coll Cardiol 2017;69:2657–64. pmid:28545640
- 3. Oakden-Rayner L, Carneiro G, Bessen T, Nascimento JC, Bradley AP, Palmer LJ. Precision radiology: predicting longevity using feature engineering and deep learning methods in a radiomics framework. Sci Rep 7,1648; (2017). pmid:28490744
- 4. Collins F, and Varmus H. A new initiative in precision medicine. N Engl J Med 2015;372:793–95. pmid:25635347
- 5. Forssen H, Patel R, Fitzpatrick N, Hingorani A, Timmis A., Hemingway H, et al. Evaluation of Machine Learning Methods to Predict Coronary Artery Disease Using Metabolomic Data. Stud Health Technol Inform 2017;235:111–115. pmid:28423765
- 6. Analytics For Life. Cardiac Phase Space Tomography Analysis (cPSTA) System: Coronary Artery Disease Learning and Formula Development Study. https://clinicaltrials.gov/ct2/show/NCT02784197?term=NCT02784197&rank=1. NLM identifier: NCT02784197. Accessed October 6, 2017.
- 7. Fihn SD, Gardin JM. 2012 ACCF/AHA/ACP/AATS/PCNA/SCAI/STS Guidelines for the diagnosis and management of patients with stable ischemic heart disease. J Am Coll Cardiol 2012;60:e44–e164. pmid:23182125
- 8. Levine GN, Bates ER, Blankenship JC, Bailey SR, Bittl JA, Cercek B, et al. 2011 ACCF/AHA/SCAI Guidelines for Percutaneous Coronary Intervention. J Am Coll Cardiol 2011;58:e44–122. pmid:22070834
- 9. MatLab Mathworks: Bootci. https://www.mathworks.com/help/stats/bootci.html. Accessed October 6, 2017.
- 10. MatLab MathWorks: Perfcurve. https://www.mathworks.com/help/stats/perfcurve.html. Accessed October 6, 2017.
- 11. Zou H, Hastie T. Regularization and variable selection via the elastic net. J. R. Statist. Soc. B. 2005;67(Part2):301–20.
- 12. Austin PC and Steyerberg EW. The number of subjects per variable required in linear regression analyses. J Clin Epidemiol. 2015;68(6): 627–636. pmid:25704724
- 13. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. Springer; 2009. pp. 694–93.
- 14. Zou H, Hastie T. Regularization and variable selection via the elastic net. J. R. Statist. Soc. B. 2005;67(Part2):301–20.
- 15. Gensini GG. A more meaningful scoring system for determining the severity of coronary heart disease. Am. J. Cardiol. 1983;51(3):606. pmid:6823874
- 16. Holland JH. Adaptation in Natural and Artificial Systems. University of Michigan Press; 1975.
- 17. De Jong KA. Evolutionary computation: a unified approach. MIT press; 2006.
- 18. Platt RW, Hanley JA, Yang H. Bootstrap confidence intervals for the sensitivity of a quantitative diagnostic test. Stat Med. 2000;19:313–322. pmid:10649298
- 19. Ruigómez A, Rodríguez LAG, Wallander M-A, Johansson S, Jones R. Chest pain in general practice: incidence, comorbidity and mortality. Family Practice 2006;23:167–74. pmid:16461444
- 20. Ladapo JA, Blecker S, and Douglas PS. Physician decision making and trends in the use of cardiac stress testing in the United States. Ann Intern Med 2014;161:482–90. pmid:25285541
- 21. Chen J. The frontiers of nuclear cardiology research. J Biomed Res 2013;27(6):437–38. pmid:24285941
- 22. Mark DR, Federspiel JJ, Cowper PA, Anstrom KJ, Hoffmann U, Patel MR, et al. Economic outcomes with anatomical versus functional diagnostic testing for coronary artery disease. Ann Intern Med 2016;165:94–102. pmid:27214597
- 23. Douglas PS, Hoffmann U, Manesh RP, Mark DB, Al-Khalidi HR, Cavanaugh B, et al. Outcomes of anatomical versus function testing for coronary artery disease. N Engl J Med 2015;372:1291–1300. pmid:25773919
- 24. Patel MR, Peterson ED, Dai D et.al. Low diagnostic yield of elective coronary angiography. N Engl J Med 2010;362:886–895. pmid:20220183
- 25. Patel MR, Dai D, Hernandez AF, Douglas PS, Messenger J, Garratt KN, Maddox TM et al. Prevalence and predictors of nonobstructive coronary artery disease identified with coronary angiography in contemporary clinical practice. Am Heart J 2014;167:846–852. pmid:24890534
- 26. Kwok Y, Kim C, Grady D, Seqal M, Redberg R. Meta-analysis of exercise testing to detect coronary artery disease in women. Am J Cardiol. 1999;83:660–6. pmid:10080415
- 27. Duke Radiology: Benefits of cardiac CTA. https://radiology.duke.edu/patient-care/specialized-services/cardiac-cta/. Accessed October 6,2017.
- 28. Obermeyer Z, Emanuel E. Predicting the future-big data, machine learning, and clinical medicine. N Engl J Med 2016;375(13):1216–1219. pmid:27682033
- 29. Darcy AM, Louie AK, Roberts LW. Machine learning and the profession of Medicine. JAMA 2016;315(6):551–52. pmid:26864406
- 30. Dey D, Gaur S, Ovrehus KA, Slomka PJ, Betancur J, Goeller M, et al. Integrated prediction of lesion-specific ischemia from quantitative coronary CT angiography using machine learning: a multicentre study. Eur Radiol. 2018 Jan 19. pmid:29352380
- 31. Inan OT, Baran Pouyan M, Javaid AQ, Dowling S, Etemadi M, Dorier A, et al. Novel Wearable Seismocardiography and Machine Learning Algorithms Can Assess Clinical Status of Heart Failure Patients. Circ Heart Fail. 2018;11(1):e004313. pmid:29330154