http://orcid.org/0000-0002-2357-3923
Figures
Abstract
Background
The National Lung Screening Trial (NLST) demonstrated that annual screening with low dose CT in high-risk population was associated with reduction in lung cancer mortality. Nonetheless, the leading cause of mortality in the study was from cardiovascular diseases.
Purpose
To determine whether the used machine learning automatic algorithms assessing coronary calcium score (CCS), level of liver steatosis and emphysema percentage in the lungs are good predictors of cardiovascular disease (CVD) mortality and incidence when applied on low dose CT scans.
Materials and methods
Three fully automated machine learning algorithms were used to assess CCS, level of liver steatosis and emphysema percentage in the lung. The algorithms were used on low-dose computed tomography scans acquired from 12,332 participants in NLST.
Results
In a multivariate analysis, association between the three algorithm scores and CVD mortality have shown an OR of 1.72 (p = 0.003), 2.62 (p < 0.0001) for CCS scores of 101–400 and above 400 respectively, and an OR of 1.12 (p = 0.044) for level of liver steatosis. Similar results were shown for the incidence of CVD, OR of 1.96 (p < 0.0001), 4.94 (p < 0.0001) for CCS scores of 101–400 and above 400 respectively. Also, emphysema percentage demonstrated an OR of 0.89 (p < 0.0001). Similar results are shown for univariate analyses of the algorithms.
Citation: Stemmer A, Shadmi R, Bregman-Amitai O, Chettrit D, Blagev D, Orlovsky M, et al. (2020) Using machine learning algorithms to review computed tomography scans and assess risk for cardiovascular disease: Retrospective analysis from the National Lung Screening Trial (NLST). PLoS ONE 15(8): e0236021. https://doi.org/10.1371/journal.pone.0236021
Editor: Christophe Leroyer, Universite de Bretagne Occidentale, FRANCE
Received: December 21, 2019; Accepted: June 26, 2020; Published: August 3, 2020
Copyright: © 2020 Stemmer et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: This paper uses data from the NLST. The data is fully available upon request in the following site: https://cdas.cancer.gov/learn/nlst/instructions/.
Funding: EE, RS, DC, OBA, and MA are employees of Zebra Medical Vision and have stock options in the company. The funder provided support in the form of salaries for authors EE, RS, DC, OBA, MA and LD, but did not have any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: EE, RS, DC, OBA, and MO are employed by Zebra Medical Vision, a commercial entity. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
The National Lung Screening Trial (NLST) was conducted to determine whether annual screening with low-dose computed tomography (LDCT) scans could reduce mortality from lung cancer in high-risk individuals [1]. In the NLST, more people died of cardiovascular disease (24.8%) than lung cancer (24.1%) (1). This result is consistent with the repeated observations that cardiovascular disease (CVD) is the leading cause of death in the 21st century [2] and the leading cause of death of smokers [3].
Routine measurement of the Coronary Calcium Score (CCS), a well-established risk factor for CVD quantified by the Agatston score [4–6], has been shown to detect CVD in LDCT scans. This type of CVD screening has been shown to lead to finding 84 patients with a CCS above 1000 out of 1000 patients screened, who could benefit from starting secondary preventive medical treatment of CVD such as antihypertensive or lipid lowering drugs or both, that they didn’t receive before [7]. In addition, LDCT could also identify fatty liver, which is independently associated with increased CVD risk [8–10]. Finally, a specific diagnosis of emphysema can also be made from the screening LDCT, which is by itself a risk factor for CVD [11], this screening could also lead to other disease specific interventions for pulmonary disease as well as CVD.
Although early diagnostics based on CT screening is very effective, the current approach, which relies primarily on manual evaluation of medical imaging is limited due to varied interobserver reliability [12–14] and limited time for scan [15]. With recent advances in artificial intelligence (AI) and machine learning algorithms, using automation to immediately and accurately analyze each CT scan and create query-able, consistent interpretations is closer than ever. Such algorithms, if implemented appropriately, could assist physicians and lead to early diagnosis and improved patient care.
In this study, we aimed to apply three automated machine learning algorithms that provide CCS, liver steatosis and emphysema percentage in the lungs to low-dose CT scans, in order to assess the utility of these tools as predictors of CVD incidence and mortality.
Materials and methods
Data acquisition
The study was submitted to review by the NCI (National Cancer institute) and was approved as NLST-246.
Data from 15,000 participants of the NLST trial were acquired. For each participant three LDCT screenings (T0, T1, T2) were requested, each screening was made at 1-year intervals. These participants were 55–74 years old with a history of cigarette smoking of at least 30 pack years, and, if former smokers, had quit within the previous 15 years [1]. Participants who met the NLST exclusion criteria [1], were screened with an annual Chest X-ray rather than LDCT or did not complete the NLST study, were not selected. A sample of 15,000 participants was received from the NIH, representing a dataset enriched for subjects who had self-reported cardiovascular events and those who died of cardiovascular disease during the NLST interval. Subject data was excluded (Fig 1) if it contained only a single LDCT time point or in the event that algorithmic assessment of either Coronary Calcium, Liver Density or Quantitative Emphysema was technically unsuccessful. Median duration of follow up was 6.5 years with a maximum duration of 7.4 years.
Algorithms used
In this study we used three automated machine learning algorithms which were designed to assess biomarkers on non-contrast chest CT, specifically the percent emphysema (Emphy-Alg), liver density (LD-Alg) and quantity of coronary calcium (CCS-Alg).
CCS-Alg is an algorithm designed to assist radiologists in evaluating coronary artery calcification (CAC) by providing CCS and its quantification the Agatston score from CT scans as was previously published [16]. CCS is measured as a continuous score while Agatston is reported in categories.
The Emphy-Alg is an algorithm for analyzing images from CT thoracic scans. This application performs automatic segmentation and volume calculation of the pulmonary tissue from non-contrast thoracic CT scans and quantifies the volume of low-attenuation air in the lungs (low-attenuation volume is defined as volume with attenuation lower than -950 Hounsfield units (HU) [17]. The application provides the low-attenuation volume as a percentage of the overall lung volume, this allows us to assess the percentage of emphysema.
LD-Alg is an algorithm which automatically analyzes pre-existing chest and abdomen CT scans (with or without a contrast agent) that include part of the liver and assesses liver attenuation. This algorithm accepts CT scans (DICOM files) and provides an averaged HU value of the liver. Steatosis will cause a reduced HU value [18] and LD-Alg score. The LD-Alg score is provided as a continuous score.
Statistical methods
Statistical analyses were performed in SAS v9.4 (SAS institute, Cary, NC, USA) and in R v3.5.1.
The first valid measurement of each of the scores (CCS-Alg, Emphy-Alg, and LD-Alg) was used in the analysis. Logistic regression was used to determine whether each of the 3 scores is associated with the outcomes of interest. The 3 scores were modeled individually and in multivariate logistic regression models together with age, gender and pack years. The CCS-Alg score was modeled both as a 5-level categorical variable similar to the Agatston score scale and as a continuous score. The Emphy-Alg and LD-Alg scores were modeled as continuous scores with odds ratios (OR) reported for 10% or 10-unit increments of each respectively. The area under the ROC curve (AUC) was used as a measure of the discriminatory power of the models. Nested models were compared by the difference between AUC as well as the correlated 95% confidence interval [19]. Model parameters with level of significance are presented. ORs and 95% confidence intervals (CI) for both univariate and multivariate models are also presented. A P value < 0.05 was considered statistically significant, nominal p-values are presented.
Results
Demographics
The final analysis included 12,332 participants (Fig 1). The majority of participants were male (n = 7308, 59.3%) mean (SD) age was 61.9 (5.1), females (n = 5024, 40.7%) mean (SD) age was 61.5 (5), and overall, the mean (SD) age was 61.8 (5.1) years (range, 55–74 years) (Table 1). All patients have a history of cigarette smoking (of at least 30 pack years or had stopped within 15 years before the initiation of the NLST study), with mean (SD) of cigarette packs per year of 56.3 (24.1) and range of 30–295 cigarette pack years (Table 1).
The clinical outcomes of interest in this population were CVD incidence (n = 2855, 23.2%) and CVD mortality (n = 358, 2.9%) (Table 1). CCS-Alg Agatston categories stayed the same throughout the 3 time points in 52.7% (n = 6505) of the participants. Of the remaining, 21% (n = 2592) changed 1 group over and 26.2% (n = 3235) changed more than one group. Among those who had a change, the net movement from T0 to T2 was increased in 25.9% (n = 3196) and decreased in 21.3% (n = 2631). The biggest shift in categories was between Agatston category 3 in T1 to Agatston category 4 in T3, 5.8% (n = 724).
Data from 15,000 participants was gathered under NIH protocol NLST 246. 1856 (12%) participants were excluded because data only existed for a single time point and thus could not be analyzed for temporal dynamics. An additional 812 (5%) were excluded because of technical algorithmic failure pertaining to any one of the three automatic assessments of Coronary Calcium, Liver Density or Quantitative Emphysema.
CVD mortality
Univariate analyses.
A statistically significant association between CCS-Alg categories and CVD mortality was observed, with approximately 1.5-fold, 2-fold and 3.5-fold increased risk of CVD mortality in categories 11–100 (n = 2742), 101–400 (n = 2374), and >400 (n = 2780) vs 0 (n = 3451), respectively, and when treated as a continuous variable CCS-Alg demonstrated a 1.05-fold (P < 0.0001) increased risk for CVD mortality for every 100-unit increase in score (Table 2). A statistically significant association was also found between Emphy-Alg score and CVD mortality, with 1.14-fold (95% CI, 1.03–1.26) increased risk of CVD mortality for each 10% increment in the Emphy-Alg score (P = .0146 S1 Table in S1 Appendix), and between LD-Alg score and CVD mortality, with 1.16-fold (95% CI, 1.04–1.29) increased risk of CVD mortality for each 10 HU reduction in the LD-Alg score (P = .0077 S1 Table in S1 Appendix).
Multivariate models.
The relationship between CVD mortality and CCS-Alg, Emphy-Alg, and LD-Alg was further explored via multivariate logistic regression together with age, gender and pack years which are known risk factors (Table 3). The AUC for this model was 0.6724 (95% CI, 0.64–0.7), suggesting low-to-moderate discrimination ability (Fig 1). The analysis demonstrated an increased CVD mortality risk of approximately 1.71-fold and 2.6-fold in CCS-Alg categories 101–400 and >400 vs 0, respectively. There was no increased risk in the 1–10 or the 11–100 category vs 0. The analysis also demonstrated approximately 1.12-fold increased risk of CVD mortality per 10 HU units decrease in LD-Alg scores (Table 3).
A multivariate logistic regression consisting of CCS-Alg, LD-Alg, Emphy-Alg was performed and resulted in an AUC of 0.648 (95% CI, 0.62–0.68) (S1 Fig in S1 Appendix).
The relationship between CVD mortality and age, gender and pack years was also assessed with a multivariate logistic regression, this model resulted in an AUC of 0.639 (95% CI, 0.61–0.67. AUC difference between the combined model and this one is 0.032 (95% CI, 0.03–0.033), AUC difference between the CCS-Alg, LD-Alg, Emphy-Alg model and this model is 0.009 (95% CI, 0.01–0.008)). this analysis demonstrated an increased CVD mortality risk of 1.94-fold (P < .0001) for every 10-year increase in age, and increased risk of 1.44-fold (P = 0.0019) for males and an increased risk of 1.073-fold (P = 0.0001) for every 10-year increase in pack years.
CVD incidence
Univariate analyses.
A statistically significant association between CCS-Alg categories and CVD incidence was found, with approximately 2.2-fold, and 5.9-fold increased risk of CVD incidence in categories 101–400, and >400 vs 0, respectively. When treating CCS-Alg as a continuous variable an increased 1.12-fold risk for CVD incidence was found for every 100 increment in the CCS-Alg score (Table 2). No statistically significant association was observed between Emphy-Alg score and CVD incidence (P = .176 S1 Table in S1 Appendix). However, a statistically significant inverse association was found with the LD-Alg score (1.1-fold difference [95% CI, 1.05–1.15] for each 10 HU decrements in the LD-Alg score; P < .0001 S1 Table in S1 Appendix).
Multivariate models.
The relationship between CVD incidence and CCS-Alg, Emphy-Alg, and LD-Alg was explored via a multivariate logistic regression (Table 4). The AUC for the overall model was 0.7159 (95% CI, 0.71–0.73), suggested moderate discrimination ability (Fig 2B). The analysis demonstrated an increased CVD incidence risk of approximately 1.96-fold, and 4.9-fold in CCS-Alg categories 101–400 and >400 vs 0. There was no increased risk in the 1–10 or 11–100 category vs 0. The analysis also demonstrated reduction of CVD incidence risk (0.89-fold difference) per 10% increments in Emphy-Alg score (Table 4).
ROC curves with AUC for CCS-Alg, Emphy-Alg, LD-Alg univariate analyses and a multivariate model consisting of all the above together with age, gender and pack years depicting the discriminatory power of the scores for CVD mortality (A) and incidence (B). AUC difference comparing the Model AUC and CCS-Alg is 0.036 (95% CI, 0.037–0.035) for mortality (A) and 0.025 (95% CI, 0.024–0.026) for incidence (B).
A multivariate logistic regression consisting of CCS-Alg, LD-Alg, Emphy-Alg was performed and resulted in an AUC of 0.699 (95% CI, 0.68–0.7) (S2 Fig in S1 Appendix).
The relationship between CVD incidence and age, gender and pack years was also assessed with a multivariate logistic regression, this model resulted in an AUC of 0.6288 (95% CI, 0.62–0.64, AUC difference between the combined model and this one is 0.08 (95% CI, 0.086–0.087), AUC difference between the CCS-Alg, LD-Alg, Emphy-Alg model and this model is 0.071 (95% CI, 0.07–0.071)). This analysis demonstrated an increased CVD incidence risk of 1.67-fold (P < .0001) for every 10-year increase in age, and increased risk of 1.81-fold (P < .0001) for males and an increased risk of 1.06-fold (P = 0.0001) for every 10-year increase in pack years.
Discussion
The current analysis demonstrated that the automatic CT scan evaluation tools CCS-Alg, Emphy-Alg, and LD-Alg are statistically significant predictors for CVD mortality, with CCS-Alg providing the highest predictive value of these 3 tools. As for CVD incidence, our findings suggest that CCS-Alg and LD-Alg (but not Emphy-Alg) are statistically significant predictors.
Our findings show that CCS-Alg, Emphy-Alg, and LD-Alg are statistically significantly better predictors for CVD incidence than age, gender or pack year, which are by themselves well known predictors of both CVD mortality and or CVD incidence [20–23]. It can be seen that when adding all of the above predictors into one multivariate model we achieve an improved model with a statistically significant and higher AUC both for CVD mortality and for CVD incidence.
Our results with the CCS-Alg are similar in their prediction trend to results obtained using manual Agatston score categories. For example, Oudkerk et al. [24] have shown a rise in relative risk (RR) of 1.9 in the 1–100 category to 7.2 RR in 400–1000 while we show a rise in OR from 1.1 to 5.9 respectively. A study of the American College of Radiology Imaging Network (ACRIN) arm of the NLST which randomly selected 1575 cases and manually assessed CCS, showing that for Agatston scores of 1–100, 101–1000 and >1000 the hazard ratios were 1.27, 3.57 and 6.63, respectively. Our results show similar numbers while increasing the population and automating the process [25].
The advantage of CCS-Alg over manual processing of scans for quantifying calcifications in the coronary arteries is that this post-processing tool is fully automated utilizing machine learning and AI approaches. Multiple studies have shown the benefits of machine learning and AI to the medical field with respect to prediction of diagnosis [26], time saving [27] and better results in finding a certain pathology [28]. Furthermore, using automated tools such as CCS-Alg could help address a wider range of predictors of various diseases, even in cases where these diseases are not the focus/motivation of the screening. For example, in the case of NLST, patients were screened for lung cancer; however, the scans could also be used for identifying CVD risk. The automated process also enables the addition of these scores into a database in a reliable and efficient way. This will essentially help encode radiology reports into data that will allow a researcher or clinician to query the desired information that today is only available through manually looking at each report and hand picking the wanted data.
Our finding that LD-Alg and Emphy-Alg are statistically significant predictors of CVD mortality is consistent with prior studies showing that emphysema [29] and fatty liver [8–10] (on which Emphy-Alg and LD-Alg are based, respectively) are predictors of CVD mortality. However, surprisingly, for each 10% increment in the Emphy-Alg score, we observed reduced risk of CVD incidence.
All three of our algorithms are fully automated and provide additional and critical information to the physician. They all provide continuous exact measurements about the severity of the disease, as opposed to simply a category. This information can support the physician in better assessing CVD severity and consequently in better treatment decisions. For example, a CCS-Alg score of 401 is in the same Agatston category as a score of 802; however the probability of having a CVD incidence with a CCS of 802 is higher by approximately 25% during a median follow-up period of 3.8 years than that of a CCS of 401 [4].
Our study is limited by the lack of manual Agatston score on the NLST data. Therefore, the automatic tools could not be directly compared to the manual Agatston score procedure. However, we were able to show that the algorithms (particularly CCS-Alg) can predict CVD incidence and mortality even without a manual Agatston score. Our approach has a few limitations as well, by automating the process we increase the risk of overdiagnosis. While is still isn’t clear that the added information will influence management. However, it might also give the opportunity to approach a patient not by a single disease at a time but identify multiple risk factors with a single test.
In conclusion, our study demonstrated that automatic algorithms such as CCS-Alg, on its own or in combination with Emphy-Alg, LD-Alg, age, gender and pack years can be used in clinical practice as predictors of CVD incidence and mortality.
References
- 1. Team TNLSTR. Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening. N Engl J Med [Internet]. 2011 Aug 4 [cited 2018 Nov 20];365(5):395–409. Available from: http://www.nejm.org/doi/10.1056/NEJMoa1102873 pmid:21714641
- 2.
The top 10 causes of death [Internet]. [cited 2018 Nov 27]. http://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death
- 3. Doll R, Peto R, Boreham J, Sutherland I. Mortality in relation to smoking: 50 Years’ observations on male British doctors. Br Med J. 2004 Jun 26;328(7455):1519–28.
- 4. Detrano R, Guerci AD, Carr JJ, Bild DE, Burke G, Folsom AR, et al. Coronary Calcium as a Predictor of Coronary Events in Four Racial or Ethnic Groups. N Engl J Med [Internet]. 2008 Mar 27 [cited 2018 Nov 21];358(13):1336–45. Available from: http://www.nejm.org/doi/abs/10.1056/NEJMoa072100 pmid:18367736
- 5. Shaw LJ, Raggi P, Schisterman E, Berman DS, Callister TQ. Prognostic Value of Cardiac Risk Factors and Coronary Artery Calcium Screening for All-Cause Mortality. Radiology [Internet]. 2003 Sep [cited 2018 Nov 30];228(3):826–33. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12869688
- 6. Agatston AS, Janowitz WR, Hildner FJ, Zusmer NR, Viamonte M, Detrano R. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol [Internet]. 1990 Mar 15 [cited 2018 Nov 21];15(4):827–32. Available from: http://www.ncbi.nlm.nih.gov/pubmed/2407762
- 7. Jacobs PC, Gondrie MJA, van der Graaf Y, de Koning HJ, Isgum I, van Ginneken B, et al. Coronary Artery Calcium Can Predict All-Cause Mortality and Cardiovascular Events on Low-Dose CT Screening for Lung Cancer. Am J Roentgenol [Internet]. 2012 Mar 23 [cited 2018 Nov 27];198(3):505–11. Available from: http://www.ajronline.org/doi/10.2214/AJR.10.5577
- 8. Osawa K, Miyoshi T, Yamauchi K, Koyama Y, Nakamura K, Sato S, et al. Nonalcoholic Hepatic Steatosis Is a Strong Predictor of High-Risk Coronary-Artery Plaques as Determined by Multidetector CT. Targher G, editor. PLoS One [Internet]. 2015 Jun 30 [cited 2018 Nov 27];10(6):e0131138. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26125952
- 9. Puchner SB, Lu MT, Mayrhofer T, Liu T, Pursnani A, Ghoshhajra BB, et al. High-Risk Coronary Plaque at Coronary CT Angiography Is Associated with Nonalcoholic Fatty Liver Disease, Independent of Coronary Plaque and Stenosis Burden: Results from the ROMICAT II Trial. Radiology [Internet]. 2015 Mar [cited 2018 Nov 27];274(3):693–701. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25369449
- 10. Pickhardt PJ, Hahn L, del Rio AM, Park SH, Reeder SB, Said A. Natural History of Hepatic Steatosis: Observed Outcomes for Subsequent Liver and Cardiovascular Complications. Am J Roentgenol [Internet]. 2014 Apr [cited 2018 Nov 27];202(4):752–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24660702
- 11. Morgan AD, Zakeri R, Quint JK. Defining the relationship between COPD and CVD: what are the implications for clinical practice? Ther Adv Respir Dis. 2018 Jan 19;12.
- 12. Gierada DS, Pilgram TK, Ford M, Fagerstrom RM, Church TR, Nath H, et al. Lung Cancer: Interobserver Agreement on Interpretation of Pulmonary Findings at Low-Dose CT Screening. Radiology [Internet]. 2008 Jan [cited 2019 Feb 18];246(1):265–72. Available from: http://www.ncbi.nlm.nih.gov/pubmed/18024436
- 13. Williams MC, Golay SK, Hunter A, Weir-McCall JR, Mlynska L, Dweck MR, et al. Observer variability in the assessment of CT coronary angiography and coronary artery calcium score: substudy of the Scottish COmputed Tomography of the HEART (SCOT-HEART) trial. Open Hear. 2015 May 1;2(1):e000234.
- 14. Abdel Razek AAK, Elrakhawy MM, Yossof MM, Nageb HM. Inter-observer agreement of the coronary artery disease reporting and data system (CAD-RADSTM) in patients with stable chest pain. Polish J Radiol. 2018;83:e151–9.
- 15. McDonald RJ, Schwartz KM, Eckel LJ, Diehn FE, Hunt CH, Bartholmai BJ, et al. The Effects of Changes in Utilization and Technological Advancements of Cross-Sectional Imaging on Radiologist Workload. Acad Radiol [Internet]. 2015 Sep [cited 2019 Feb 18];22(9):1191–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26210525
- 16.
Shadmi R, Mazo V, Bregman-Amitai O, Elnekave E. Fully-convolutional deep-learning based system for coronary calcium score prediction from non-contrast chest CT. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) [Internet]. IEEE; 2018 [cited 2019 Apr 7]. p. 24–8. https://ieeexplore.ieee.org/document/8363515/
- 17. Thurlbeck WM, Müller NL. Emphysema: definition, imaging, and quantification. AJR Am J Roentgenol [Internet]. 1994 Nov 19 [cited 2019 Feb 17];163(5):1017–25. Available from: http://www.ncbi.nlm.nih.gov/pubmed/7976869 pmid:7976869
- 18. Stern C, Castera L. Non-invasive diagnosis of hepatic steatosis. Hepatol Int [Internet]. 2017 Jan 25 [cited 2019 Feb 15];11(1):70–8. Available from: http://link.springer.com/10.1007/s12072-016-9772-z pmid:27783208
- 19. Demler O V., Pencina MJ, D’Agostino RB. Misuse of DeLong test to compare AUCs for nested models. Stat Med. 2012 Oct 15;31(23):2577–87. pmid:22415937
- 20. Mosca L, Barrett-Connor E, Kass Wenger N. Sex/gender differences in cardiovascular disease prevention: What a difference a decade makes. Circulation. 2011;
- 21. Lubin JH, Couper D, Lutsey PL, Woodward M, Yatsuya H, Huxley RR. Risk of cardiovascular disease from cumulative cigarette use and the impact of smoking intensity. Epidemiology. 2016;
- 22. Dhingra R, Vasan RS. Age As a Risk Factor. Medical Clinics of North America. 2012.
- 23. Vasan RS, Cobain M, Kannel WB, Wolf PA, D’Agostino RB, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;
- 24. Oudkerk M, Stillman AE, Halliburton SS, Kalender WA, Möhlenkamp S, McCollough CH, et al. Coronary artery calcium screening: current status and recommendations from the European Society of Cardiac Radiology and North American Society for Cardiovascular Imaging. Int J Cardiovasc Imaging [Internet]. 2008 Aug [cited 2018 Dec 16];24(6):645–71. Available from: http://www.ncbi.nlm.nih.gov/pubmed/18504647
- 25. Chiles C, Duan F, Gladish GW, Ravenel JG, Baginski SG, Snyder BS, et al. Association of Coronary Artery Calcification and Mortality in the National Lung Screening Trial: A Comparison of Three Scoring Methods. Radiology [Internet]. 2015 Jul [cited 2019 Feb 18];276(1):82–90. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25759972
- 26. Ding Y, Sohn JH, Kawczynski MG, Trivedi H, Harnish R, Jenkins NW, et al. A Deep Learning Model to Predict a Diagnosis of Alzheimer Disease by Using 18 F-FDG PET of the Brain. Radiology [Internet]. 2018 Nov 6 [cited 2018 Dec 1];180958. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2018180958
- 27. Wang R, Meinel FG, Schoepf UJ, Canstein C, Spearman J V., De Cecco CN. Performance of Automated Software in the Assessment of Segmental Left Ventricular Function in Cardiac CT: Comparison with Cardiac Magnetic Resonance. Eur Radiol [Internet]. 2015 Dec 30 [cited 2018 Dec 1];25(12):3560–6. Available from: http://link.springer.com/10.1007/s00330-015-3767-3 pmid:25925355
- 28. Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, et al. CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning. 2017 Nov 14 [cited 2018 Dec 1]; Available from: https://arxiv.org/abs/1711.05225
- 29. Johannessen A, Skorge TD, Bottai M, Grydeland TB, Nilsen RM, Coxson H, et al. Mortality by Level of Emphysema and Airway Wall Thickness. Am J Respir Crit Care Med [Internet]. 2013 Mar 15 [cited 2018 Dec 1];187(6):602–8. Available from: http://www.atsjournals.org/doi/abs/10.1164/rccm.201209-1722OC pmid:23328525