Machine learning assessment of myocardial ischemia using angiography: Development and retrospective validation

Background Invasive fractional flow reserve (FFR) is a standard tool for identifying ischemia-producing coronary stenosis. However, in clinical practice, over 70% of treatment decisions still rely on visual estimation of angiographic stenosis, which has limited accuracy (about 60%–65%) for the prediction of FFR < 0.80. One of the reasons for the visual–functional mismatch is that myocardial ischemia can be affected by the supplied myocardial size, which is not always evident by coronary angiography. The aims of this study were to develop an angiography-based machine learning (ML) algorithm for predicting the supplied myocardial volume for a stenosis, as measured using coronary computed tomography angiography (CCTA), and then to build an angiography-based classifier for the lesions with an FFR < 0.80 versus ≥ 0.80. Methods and findings A retrospective study was conducted using data from 1,132 stable and unstable angina patients with 1,132 intermediate lesions who underwent invasive coronary angiography, FFR, and CCTA at the Asan Medical Center, Seoul, Korea, between 1 May 2012 and 30 November 2015. The mean age was 63 ± 10 years, 76% were men, and 72% of the patients presented with stable angina. Of these, 932 patients (assessed before 31 January 2015) constituted the training set for the algorithm, and 200 patients (assessed after 1 February 2015) served as a test cohort to validate its diagnostic performance. Additionally, external validation with 79 patients from two centers (CHA University, Seongnam, Korea, and Ajou University, Suwon, Korea) was conducted. After automatic contour calibration using the caliber of guiding catheter, quantitative coronary angiography was performed using the edge-detection algorithms (CAAS-5, Pie-Medical). Clinical information was provided by the Asan BiomedicaL Research Environment (ABLE) system. The CCTA-based myocardial segmentation (CAMS)-derived myocardial volume supplied by each vessel (right coronary artery [RCA], left anterior descending [LAD], left circumflex [LCX]) and the myocardial volume subtended to a stenotic segment (CAMS-%Vsub) were measured for labeling. The ML for (1) predicting vessel territories (CAMS-%LAD, CAMS-%LCX, and CAMS-%RCA) and CAMS-%Vsub and (2) identifying the lesions with an FFR < 0.80 was constructed. Angiography-based ML, employing a light gradient boosting machine (GBM), showed mean absolute errors (MAEs) of 5.42%, 8.57%, and 4.54% for predicting CAMS-%LAD, CAMS-%LCX, and CAMS-%RCA, respectively. The percent myocardial volumes predicted by ML were used to predict the CAMS-%Vsub. With 5-fold cross validation, the MAEs between ML-predicted percent myocardial volume subtended to a stenotic segment (ML-%Vsub) and CAMS-%Vsub were minimized by the elastic net (6.26% ± 0.55% for LAD, 5.79% ± 0.68% for LCX, and 2.95% ± 0.14% for RCA lesions). Using all attributes (age, sex, involved vessel segment, and angiographic features affecting the myocardial territory and stenosis degree), the ML classifiers (L2 penalized logistic regression, support vector machine, and random forest) predicted an FFR < 0.80 with an accuracy of approximately 80% (area under the curve [AUC] = 0.84–0.87, 95% confidence intervals 0.71–0.94) in the test set, which was greater than that of diameter stenosis (DS) > 53% (66%, AUC = 0.71, 95% confidence intervals 0.65–0.78). The external validation showed 84% accuracy (AUC = 0.89, 95% confidence intervals 0.83–0.95). The retrospective design, single ethnicity, and the lack of clinical outcomes may limit this prediction model’s generalized application. Conclusion We found that angiography-based ML is useful to predict subtended myocardial territories and ischemia-producing lesions by mitigating the visual–functional mismatch between angiographic and FFR. Assessment of clinical utility requires further validation in a large, prospective cohort study.


Conclusion
We found that angiography-based ML is useful to predict subtended myocardial territories and ischemia-producing lesions by mitigating the visual-functional mismatch between angiographic and FFR. Assessment of clinical utility requires further validation in a large, prospective cohort study.

Author summary
Why was this study done?
• Invasive fractional flow reserve (FFR, defined as the ratio of maximum flow in a diseased artery to the proximal normal maximum flow) has been a standard tool to detect ischemia-producing lesions with FFR < 0.80.
• Although the current guidelines recommend the routine use of FFR for identifying ischemia-producing lesions, the majority of treatment decisions still rely on visual assessment of the degree of angiographic stenosis because of the time and expense associated with FFR-guided decision-making.
• Conventional angiographic parameters cannot predict the presence of ischemia in cases in which this is affected by the size of subtended myocardium.
• Integration and optimization of information about both myocardial territory and stenosis degree are expected to improve the performance of angiographic prediction of low FFR (FFR < 0.80).
What did the researchers do and find?
• A retrospective study was conducted using data from 1,132 angina patients with 1,132 intermediate coronary lesions (932 in the training dataset and 200 in the internal test dataset), who underwent coronary angiography, coronary computed tomography angiography (CCTA), and FFR to evaluate the lesion morphology, subtended myocardial territories, and inducible ischemia, respectively.

Introduction
Stratification of cardiovascular risk in patients with stable coronary artery disease is a key to identify high-risk patients who will benefit from percutaneous coronary intervention (PCI). The appropriateness of revascularization has been determined by the presence and extent of myocardial ischemia. A myocardial perfusion imaging study previously suggested that revascularization has a greater survival benefit in patients with a moderate to large degree of ischemic myocardium (�10% of the total myocardium) [1]. Invasive fractional flow reserve (FFR, defined as the ratio of maximum flow in a diseased artery to the proximal normal maximum flow) has been a standard tool for lesion-specific hemodynamic assessment and treatment decision-making [2][3][4]. With abundant clinical evidence showing a significant reduction in major adverse cardiac events using FFR-guided PCI (versus angiography-guided PCI), current guidelines recommend FFR measurement when assessing intermediate coronary stenosis.
However, in clinical practice, over 70% of treatment decisions still rely on a visual estimation of angiographic stenosis. This may be due to the prolonged procedure time and high shortterm costs associated with FFR-guided diagnosis, as well as the need for adenosine-induced hyperemia and the fact that reimbursement systems do not favor this approach [5,6]. Although invasive coronary angiography and intravascular ultrasound (IVUS) are commonly utilized for evaluating coronary anatomy and optimizing PCI, the subjective nature of visual estimation limits the accurate estimation of stenosis severity [7]. In addition, the integration of morphologic and physiologic parameters and the identification of clinically relevant coronary lesions remain challenging [8][9][10]. In previous studies, the overall diagnostic accuracy of quantitative angiography for predicting FFR < 0.80 was shown to be only 60%-65% [10,11]. One of the reasons for the visual-functional mismatch is that myocardial ischemia is primarily determined by the variable size of the supplied myocardium at risk, as well as by the degree of stenosis [11]. Our previous data suggested that the application of coronary computed tomography angiography (CCTA)-based myocardial segmentation (CAMS)-derived percent myocardial volume subtended to a stenotic segment (CAMS-%V sub ) improves the diagnostic accuracy of angiographic indices used to identify ischemia-producing lesions [12,13]. Nonetheless, the necessity of concurrently performing noninvasive CCTA and invasive angiography limited the clinical utility of the mathematical model.
Machine learning (ML) techniques have emerged as highly effective computer algorithms for the identification of patterns in large datasets with a multitude of variables, facilitating the construction of models for data-driven prediction or classification [14][15][16][17]. The aims of this study were to develop an angiography-based supervised ML algorithm for predicting the CAMS-%V sub and to build an angiography-based supervised ML model to classify lesions into those with an FFR < 0.80 and those � 0.80.

Study population
Between 1 May 2012 and 31 January 2015, 5,378 consecutive patients with stable or unstable angina underwent invasive coronary angiography at the Asan Medical Center, Seoul, Korea. Preprocedural FFR and CCTA data for assessing an intermediate coronary lesion (defined as an angiographic stenosis diameter of 30%-80% on visual estimation) were available for 1,143 patients. Among them, 10 patients with tandem lesions, 10 with stented lesions, 17 with instent restenosis, 22 with chronic total occlusion, 10 with side branch evaluation, 145 with significant left main coronary artery stenosis, and 5 with scarred myocardium and regional wall motion abnormality on echocardiography were excluded. When FFR was measured in multiple lesions, the lesion with the lowest FFR value was selected. Following exclusions, 932 patients (932 lesions) were used for model training (the training sample). In addition, data from a nonoverlapping population of 200 stable and unstable angina patients (200 lesions) who underwent preprocedural angiography, IVUS, and FFR in a different phase (between 1 February 2015 and 30 November 2015) were used as a test sample to validate the diagnostic performance of the ML models for the prediction of FFR < 0.80 (Table 1). De-identified clinical information, including patient age and sex, was supported by the Asan BiomedicaL Research Environment (ABLE) system. All patients provided written informed consent for the procedures. The protocol of retrospective data analysis (1 January 2017 to approximately 30 November 2017) was approved by the institutional review board of the Asan Medical Center (S1 file), and a waiver for informed consent was granted. This study is reported as per the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines (S1 Checklist).
The external validation of the ML models was conducted in 79 angina patients (64 patients from CHA University, Seongnam, Korea, and 15 patients from Ajou University, Suwon, Korea) who underwent invasive coronary angiography and FFR to assess an intermediate coronary lesion.

Computed tomography imaging and CAMS analysis
Computed tomography imaging, including CCTA, was performed using first-or second-generation dual-source computed tomography (Definition or Definition Flash, Siemens, Germany). The CCTA data with the fewest motion artifacts and clearest demarcation of the coronary artery were transferred to customized software for CAMS analysis (A-View Cardiac, Asan Medical Center, Korea). After extracting the centerline of each coronary artery and the left ventricular myocardium on the computed tomographic images, the 3D Voronoi algorithm was used to assign the myocardial territories of the major epicardial coronary arteries, including the left anterior descending artery (LAD), left circumflex artery (LCX), and right coronary artery (RCA). In brief, the Voronoi algorithm is a mathematical algorithm that divides the area or space between predetermined points or lines according to the shortest distances from those points or lines [18][19][20]. The left ventricular myocardial volume was divided into three major epicardial coronary artery territories based on the shortest distance from the coronary artery. The CAMS-%RCA, CAMS-%LCX, and CAMS-%LAD were defined as the percentage ratios of the myocardial volumes supplied by the RCA, LCX, and LAD to the total left ventricular myocardial volume. CAMS-%V sub was defined as the percentage ratio of the myocardial volume subtended to a stenotic coronary segment to the total left ventricular myocardial volume. Fig 1 shows an example of the CAMS analysis.

Angiographic measurements
Quantitative coronary angiography was performed using standard techniques with automated edge-detection algorithms (CAAS-5, Pie-Medical, the Netherlands). After automatic contour calibration by using the known caliber of guiding catheter, angiographic diameter stenosis (DS), minimal lumen diameter (MLD), lesion length, and the proximal and distal reference lumen diameters (RLDs) were measured. Definitions of the angiographic features used for ML training are summarized in Table 2.

FFR measurement
FFR is defined as the ratio of the mean distal coronary pressure (Pd, measured with the pressure wire) to the mean aortic pressure (Pa, measured simultaneously with the guiding catheter) at maximum hyperemia. First, "Equalizing" was performed with the guidewire sensor positioned at the guiding catheter tip. A 0.014-inch FFR pressure guidewire (Radi, St. Jude Medical, Uppsala, Sweden) was then advanced distal to the stenosis. The FFR was measured at the maximum hyperemia induced by an intravenous infusion of adenosine administered through a central vein at 140 μg/kg/min increasing to 200 μg/kg/min, to enhance detection of hemodynamically relevant stenoses. Hyperemic pressure pullback recordings were performed. A stenosis was considered functionally significant when the FFR was <0.80 [3,4].

IVUS analysis
After intracoronary administration of 0.2 mg nitroglycerin, IVUS imaging was routinely performed using motorized transducer pullback (0.5 mm/s) and a commercial scanner (Boston Scientific Scimed, Minneapolis, MN, United States) with a rotating 40-MHz transducer within a 3.2-French imaging sheath. For the 630 patients for whom preprocedural IVUS data were available,  the IVUS-derived minimum lumen area (IVUS-MLA) within a stenotic segment was obtained using computerized software (EchoPlaque 3.0, Indec Systems, Mountain View, CA, USA).

ML model for predicting CAMS-V sub
The overall flow of the supervised ML models using the angiographic features is shown in Fig  2. A theoretical overview and summary of the ML algorithms and technical details are described in the supporting information (S1 Text). First, a light gradient boosting machine (GBM) with leave-one-out cross-validation was applied to predict the CAMS-derived percent myocardial volume supplied by each coronary artery (CAMS-%RCA, CAMS-%LCX, and CAMS-%LAD). The angiographic attributes affecting each vessel territory are summarized in Table 2. The variables estimated in our pilot data on the basis of the lumen diameters of LAD, RCA, and LCX (calculated %RCA, %LCX, and % LAD) were also included as attributes (see method in S1 Text). Then, the percent myocardial volumes supplied by each coronary artery (ML-%RCA, ML-%LCX, and ML-%LAD), as predicted by the algorithm, were used for the next step.
The second step was to build a model to predict the CAMS-%V sub values. The ML algorithms evaluated were ordinary least squares (OLSs), ridge and lasso regressions, elastic net, random forests, extra trees, GBM, light GBM, CatBoost, and multilayer perceptrons. A 5-fold cross-validation scheme divided the training sample into five nonoverlapping partitions (method described in S1 Text). Each partition was rotated to be the validation set, with the remaining partitions being used as the training set (S1 Fig). As attributes, the ML-%RCA, ML-%LCX, and ML-%LAD were added to the angiographic features affecting the CAMS-%V sub ( Table 2). The mean absolute errors (MAEs) and mean squared errors (MSEs) were the metrics used to evaluate the performance of the models for predicting the CAMS-%V sub values.

ML model for predicting FFR < 0.80
To develop the binary classifiers to separate the lesions with an FFR < 0.80 from those �0.80, 43 clinical and angiographic features including age, sex, involved segment (proximal LAD, mid LAD, distal LAD, proximal RCA, mid RCA, distal RCA, proximal LCX, distal LCX, first and second obtus marginalis), and the angiographic features affecting vessel territories, CAMS-%V sub , and lesion severity were used and summarized in S2 Table. The evaluated algorithms were K-nearest neighbor, binary class L2 penalized logistic regression, support vector machine, random forest, extra tree, AdaBoost, light GBM, CatBoost, Gaussian naïve Bayes, and multilayer perceptron (S1 Text). The receiver operating curve (ROC), which was based on the relative performances considering the whole range of possible probability thresholds (from 0 to 1), has an area that ranges from 0.5 for classifiers without any prediction capability to 1 for perfectly classifying algorithms. Analyses based on precision-recall curves were also conducted. Using a 5-fold cross-validation scheme (S1 Fig), the accuracy was calculated by averaging the accuracies over the five tests performed in the multiple rounds of cross-validation. For a nonbiased assessment of the performance for identifying lesions with an FFR of <0.80, the classifiers that had been previously built on the training samples were applied to a completely independent test set of 200 lesions enrolled in the different phase.
In the training set, the algorithms were independently trained on the 200 train-validation random splits with a 3:1 ratio by bootstrap, and the average performances and 95% confidence intervals were calculated. In the 200 bootstrap replicates obtained by random sampling of 50 out of the 200 test samples, the average performance and bootstrap confidence intervals were also calculated.

Statistical analysis
The statistical analyses for evaluating patient and lesion characteristics at baseline were performed using SPSS (version 10.0, SPSS, Chicago, IL, USA). All values are expressed as means ± 1 standard deviation (continuous variables) or as counts and percentages (categorical variables). Continuous variables were compared using unpaired t tests; categorical variables were compared using χ 2 statistics. A p-value < 0.05 was considered statistically significant. ROCs were analyzed using Med-Calc Software (Mariakerke, Belgium) to assess the best cutoff for angiographic DS or IVUSmeasured lumen area to predict FFR < 0.80 with maximal accuracy.

Clinical and lesion characteristics
The clinical characteristics and angiographic data of the patients in the training and test sets are summarized in Table 1. The mean age was 63 ± 10 years, and 76% were men. The evaluated vessels were LAD in 63%. The overall CAMS-%RCA, CAMS-%LAD, and CAMS-%LCX were 27.2% ± 9.3%, 42.4% ± 7.2%, and 27.7% ± 9.6%, respectively. The CAMS-%V sub was 31.1% ± 10.2%. FFR < 0.80 was shown in 41.6% of the lesions.

Angiographic prediction of vessel territory
By applying the light GBM to the training sample, the feature importance metrics for determining the CAMS-derived percent myocardial volume subtended to each coronary artery were determined, with the values being summarized in S1 Table. The estimated percent myocardial volumes (calculated %RCA, %LCX, and %LAD) based on the proximal vessel diameters were the most important features for predicting the CAMS-%RCA, CAMS-%LCX, and CAMS-%LAD, respectively. Additionally, the presence of ramus intermedius, a diminutive RCA, and an apical LAD curve affected the vessel territories. With the light GBM and leaveone-out cross-validation, the MAEs and MSEs were 5.42% and 7.10%, respectively, for predicting CAMS-%LAD, 8.57% and 11.28% for predicting CAMS-%LCX, and 4.54% and 6.51% for predicting CAMS-%RCA. Table 3 summarizes the diagnostic performances of the various ML models used to predict CAMS-%V sub . Among the models, the MAEs between the ML-predicted percent myocardial volume subtended to a stenotic segment (ML-%V sub ) and the CAMS-%V sub were minimal with the use of the elastic net algorithm (6.26% ± 0.55% for LAD lesions, 5.79% ± 0.68% for LCX lesions, and 2.95% ± 0.14% for RCA lesions). When the elastic net algorithm was applied to all cases, the overall MAE was 5.39%. Table 4 shows the feature importance metrics by elastic net for the prediction of CAMS-%V sub .

Prediction of FFR < 0.80 in the training sample
To classify the lesions into those with an FFR < 0.80 versus � 0.80, 43 clinical and angiographic features affecting vessel territories, CAMS-%V sub , and lesion severity were used for ML (S2 Table). Based on the feature importance metrics by CatBoost, the top-12 features for determining the FFR were identified (Table 4). The ROC-based diagnostic performances of the ML algorithms are shown in Table 5, S3 Table, and Fig 3. In addition, the diagnostic performances based on the precision-recall curves are shown in S4 Table. In the subgroup that included the 630 patients with available preprocedural IVUS data, the IVUS-MLA was 2.77 ± 1.32 mm 2 . When the IVUS-MLA was added as an attribute, the classifiers using L2 penalized logistic regression, random forest, and support vector machine showed an overall accuracy of 78%-80% (area under the curve [AUC] = 0.87) to predict an FFR < 0.80 that is used as a hemodynamic index requiring revascularization (Fig 3 and S5 Table).

Performance in test sample
The test samples including the 200 lesions that were not utilized during the training showed no significant differences in clinical and lesion characteristics in comparison with the training sample (Table 1). In the identification of lesions with an FFR < 0.80, angiographic DS > 53% as the cutoff derived from an ROC analysis showed a sensitivity of 74%, a specificity of 61%, and an overall accuracy of 66% (AUC = 0.71). In addition, an IVUS-MLA < 2.34 mm 2 had a sensitivity of 53%, a specificity of 79%, and an overall accuracy of 67% (AUC = 0.72).
Using clinical and angiographic features, the overall diagnostic accuracies of the ML classifiers (L2 penalized logistic regression, support vector machine, and random forest) in the test set were approximately 80% for predicting an FFR < 0.80 (AUC = 0.84-0.87, Table 5 and S6  Table). Table 6 summarizes the performances with bootstrap confidence intervals in the 200 bootstrap replicates for each of the training and test sets.
By adding the IVUS-MLA, the classifiers using L2 penalized logistic regression and support vector machine achieved an overall accuracy of 78%-79% in the test set (AUC = 0.86-0.87, Fig  4 and S5 Table).

External validation
In the external validation cohort including 79 patients, the age was 59.6 ± 9.0 years, and 58 (73.4%) were men. An FFR < 0.8 was seen in 25 (31.6%) lesions. The angiographic DS and MLD were 48.3% ± 8.0% and 1.64 ± 0.39 mm, respectively. The performances of the ML models for the prediction of FFR < 0.8 were shown in Table 5.

Discussion
This study demonstrated that (1) angiography-based ML predicted the CAMS-%V sub with an MAE of 6.26%, 5.79%, and 2.95% for LAD, LCX, and RCA lesions, respectively, and (2) for the identification of ischemia-producing lesions with a FFR < 0.80, the ML classifiers (L2 penalized logistic regression, support vector machine, and random forest) using the angiographic features showed an overall diagnostic accuracy of 80% (maximal AUC = 0.87), which was greater than that of angiographic DS criterion (66%, AUC = 0.71) or even that of the IVUS-MLA threshold (67%, AUC = 0.72). Assessment of the myocardial mass at risk is of great importance because the presence and extent of ischemic myocardium determines the clinical relevance of revascularization [21][22]. A recent meta-analysis suggested that, in comparison with medical therapy, PCI significantly reduces mortality in patients with objective ischemia documented by functional tests [21]. Moreover, myocardial perfusion imaging suggests that revascularization has a greater survival benefit in patients with a moderate to large degree of ischemic myocardium [1]. These data  have provided an insight into a higher-risk population that may benefit from an approach that incorporates ischemia-guided revascularization.
In daily practice, lesion-specific FFR is used to identify ischemia-producing lesions and to decide whether or not to treat it [2][3][4]. Although coronary angiography and IVUS have been commonly utilized to assess lesion severity, the diagnostic accuracy for predicting an FFR < 0.80 by angiographic DS or IVUS-MLA alone is <60%-70%, which restricts their clinical utility in treatment decision-making [8][9][10]. Similarly, the current study showed poor diagnostic accuracies for the detection of FFR < 0.80 using angiographic DS > 53% and IVUS-MLA < 2.34 mm 2 based on the ROC analysis (66% and 67%, respectively). One of the reasons for the visual-functional mismatch is that myocardial ischemia is also determined by the variable size of the supplied myocardium, as well as the degree of stenosis [11].
In our previous study, the use of CAMS-V sub improved the diagnostic performance of angiographic MLD and/or IVUS-MLA for the prediction of FFR < 0.80 [12,13]. Although a mathematical model using V sub /MLD 4 > 6.26 increased the accuracy to 82%, it could be applied only when the patient underwent noninvasive CCTA prior to catheterization. The current angiography-based ML model showed an overall MAE of 5.39% for predicting the CCTA-measured %V sub . During the procedure, the angiographic prediction of the amount of supplied myocardium supports clinicians by confirming the clinical relevance of revascularization treatment in lesions with a large area of myocardium at risk and by precisely identifying the ischemia-producing lesions by reducing the discrepancy between anatomical and functional severity. Several approaches for an FFR approximation of FFR using angiography-based models have recently been introduced [23][24][25][26]. A virtual functional assessment index and quantitative flow ratio based on computational fluid dynamics have shown the overall accuracies of 80%-86% in predicting an FFR < 0.80. Those approaches require a 3D reconstruction of at least two angiographic projections without foreshortening or overlapping vessels and the subsequent computational analyses. Using the clinical and 2D angiographic features affecting the subtended myocardial mass and degree of stenosis, our current ML classifiers predicted an FFR < 0.80 with an overall diagnostic accuracy of 80%. Therefore, the ML models not only reduce procedural expense by avoiding FFR testing but also provide information on the subtended myocardial territory that cannot be predicted by the FFR value. Ultimately, this datadriven approach extends the role of angiography in decision-making for the management of intermediate coronary stenosis. Machine learning prediction of myocardial ischemia Although traditional statistical methods validate the association between specific features and an endpoint, the development of a prediction model remains challenging, particularly in the setting of a nonlinear relationship between a factor and an outcome, interactions among variables, and the presence of many predictor variables. ML, an application of artificial intelligence, provides the ability to automatically learn a task without being explicitly programmed [14][15][16][17]. The algorithms attempt to balance two competing interests, "bias and variance," which are summarized by loss functions to optimize a prediction model. Using angiographic features, both regression and decision tree models showed good performance in the prediction of CAMS-%V sub , which led to greater detection of ischemia-producing lesions with reduced FFR.
The current study demonstrated the impact of the individual variables according to metrics (feature importance). For the prediction of CAMS-%V sub , the important features were seen to be proximal segment involvement, RLD, ML-predicted territory of each vessel, the sum of the distal branch diameters, and the distance between the ostium and the narrowest site. Moreover, the key features for predicting FFR < 0.80 were MLD; %DS; age, proximal vessel size of LAD, LCX, and RCA; lesion length; distance between the ostium and MLD site; and the involvement of the proximal LAD, which suggested the importance of the impact of the angiographic determinant for stenosis degree and vascular territory on the FFR value. Although the rank in each algorithm is specific to the ML model, the approach may be hypothesis generating, suggesting which features are valuable for inclusion in future studies.

Limitations
This study may be subject to selection bias. As the analysis included the single ethnicity and excluded significant left main disease, side branch, and diffuse and tandem lesions, the model cannot be applied generally. Although the developed models were validated in the historical test set and the external validation cohort, the possibility of overfitting cannot be completely excluded. This model did not include the computational fluid dynamics for estimating the anatomical severity. A large prospective trial is required to validate whether the models allow clinicians to dispense with FFR measurement and therefore change the current clinical practice. Finally, prespecified angiographic features were used for ML; an image-based deep learning strategy using big data is worthy of investigation to achieve optimal diagnostic performance for clinical use.

Conclusion
Angiography-based ML models were useful for the prediction of CAMS-%V sub and for improving the detection of ischemia-producing lesions. The data-driven approach may support clinicians in the identification of clinically relevant coronary lesions and in treatment decision-making.
Supporting information S1 Checklist. TRIPOD statement. TRIPOD, transparent reporting of a multivariable prediction model for individual prognosis or diagnosis. (PDF) S1 Text. Definition of angina, descriptions of pilot study, 5-fold cross-validation, CAMS measurements, and ML algorithms. CAMS, coronary computed tomography angiographybased myocardial segmentation; ML, machine learning. (DOC) S1