https://orcid.org/0000-0001-7772-7513
Figures
Abstract
Background and objective
Detecting patients at high risk of occurrence of an Invasive Disease Event after a first diagnosis of breast cancer, such as recurrence, distant metastasis, contralateral tumor and second tumor, could support clinical decision-making processes in the treatment of this malignancy. Though several machine learning models analyzing both clinical and histopathological information have been developed in literature to address this task, these approaches turned out to be unsuitable for describing this problem.
Methods
In this study, we designed a novel artificial intelligence-based approach which converts clinical information into an image-form to be analyzed through Convolutional Neural Networks. Specifically, we predicted the occurrence of an Invasive Disease Event at both 5-year and 10-year follow-ups of 696 female patients with a first invasive breast cancer diagnosis enrolled at IRCCS “Giovanni Paolo II” in Bari, Italy. After transforming each patient, represented by a vector of clinical information, to an image form, we extracted low-level quantitative imaging features by means of a pre-trained Convolutional Neural Network, namely, AlexNET. Then, we classified breast cancer patients in the two classes, namely, Invasive Disease Event and non-Invasive Disease Event, via a Support Vector Machine classifier trained on a subset of significative features previously identified.
Results
Both 5-year and 10-year models resulted particularly accurate in predicting breast cancer recurrence event, achieving an AUC value of 92.07% and 92.84%, an accuracy of 88.71% and 88.82%, a sensitivity of 86.83% and 88.06%, a specificity of 89.55% and 89.3%, a precision of 71.93% and 84.82%, respectively.
Conclusions
This is the first study proposing an approach which converts clinical information into an image-form to develop a decision support system for identifying patients at high risk of occurrence of an Invasive Disease Event, and then defining personalized oncological therapeutic treatments for breast cancer patients.
Citation: Fanizzi A, Bove S, Comes MC, Di Benedetto EF, Latorre A, Giotta F, et al. (2024) Prediction of breast cancer Invasive Disease Events using transfer learning on clinical data as image-form. PLoS ONE 19(11): e0312036. https://doi.org/10.1371/journal.pone.0312036
Editor: Daniele Ugo Tari, Local Health Authority Caserta: Azienda Sanitaria Locale Caserta, ITALY
Received: January 17, 2024; Accepted: September 30, 2024; Published: November 21, 2024
Copyright: © 2024 Fanizzi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data are not publicly available because are the property of Istituto Tumori ‘Giovanni Paolo II,’ Bari, Italy and contain potentially identifying patient information. The data presented in this study are available on request from the Scientific Direction of Istituto Tumori ‘Giovanni Paolo II,’ Bari, Italy (dirscientifica@oncologico.bari.it)
Funding: This work was supported by funding from the Italian Ministry of Health “Ricerca Corrente 2024”. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors declare no competing interests.
Introduction
Although breast cancer death rates have steadily decreased in recent years, breast cancer is still a leading cause of cancer death in women worldwide [1]. Recently, significant progress has been made to improve the therapeutic outcome with the introduction of novel therapies [2]; however, the possibility of accurately predicting the occurrence of an Invasive Disease Event (IDE) after a first diagnosis of breast cancer, such as recurrence, distant metastasis, contralateral tumor and second tumor, remains an unmet clinical need.
So far, in the state-of-the-art, several studies have demonstrated the central role of breast cancer subtypes on both cancer prognosis and treatment effectiveness [3–6]. According to their molecular subtype, patients are assigned to a specific adjuvant treatment after surgery, thus saving low-risk patients from unnecessary and/or potentially toxic pharmacological treatments. Besides, several prognostic online tools, such as Predict and Adjuvant, are currently available for estimating the survival rate of patients depending on the selected therapeutic option. Though these tools are usually adopted in clinical practice, different validation studies have demonstrated their accuracy is not reliable on all subsets of patients [7–9].
Within this scenario, predictive models able to achieve a well-balanced compromise between reliable therapy response predictions and cost-effectiveness are urgent. Accurate tools based on both molecular and protein markers have been already proposed, but they are expensive and not all centers are equipped with laboratories capable of performing this analysis [10]. On the other hand, the great impact that artificial intelligence has had in the biomedical field [11–13] has encouraged the design and development of machine learning models to support clinical decision-making processes in the treatment of breast cancer [14–18]. These automated models are trained on both clinical and histological data commonly collected in clinical practice, and do not require additional expansive information. However, even though several studies have proposed models achieving encouraging results, these are not yet suitable in clinical practice due to either not great performances or the absence of a proper validation on consistent datasets [19–25]. Hence, the prediction of an invasive disease event after a first breast cancer diagnosis is still a challenging task [26–29].
Recently, we proposed a novel ensemble machine learning classification model able to predict the occurrence of IDEs after a primary breast tumor, such as recurrence, metastasis, contralateral and second tumors at both 5- and 10-years follow-ups [30]. IDE prediction, despite being less investigated in literature, is of great interest in the adjuvant clinical trial setting due to treatment-related causes may play a role in the occurrence of second tumors or contralateral breast cancers [31, 32]. Though the proposed approaches [30, 33] were the result of a complex classification technique based on the concept of voting among multiple models and was validated on a substantial sample, performances were not acceptable for the application in clinical practice. Performances achieved by these models are affected by the presence of the so-called confounding patients having the same clinical and histological characteristics as other patients but responding differently to the adjuvant treatment [33].
According to other models proposed in literature [25], even the most cutting-edge machine learning algorithms turned out to be unsuitable for modelling the complexity of this problem.
With the aim of overcoming these limits, in this study we propose a model which relies on pre-trained Convolutional Neural Networks (CNNs) to analyze images obtained transforming each patient, represented by a vector of clinical information, to an image form. Particularly, the image transformation procedure reproduces an approach employed until now to analyze only genomic data [34], thus this is the first time this procedure is applied on clinical and histological characteristics and with the purpose of predicting an IDE after a breast cancer diagnosis. Actually, to the best of our knowledge, this is the first study which propose an approach converting clinical information into an image-form, allowing to exploit pre-trained CNNs capabilities for extracting latent information enclosed in the structured data. Pre-trained CNNs refer to a transfer learning approach which lets to extract quantitative imaging features from images according to which the networks have previously learned during training on a very huge (millions) number of images of different nature [35, 36]. Thus, the knowledge acquired from the network during this training phase has been then transferred and applied on images associated to our patients, with the purpose of predicting the risk of occurrence of an IDE after a first breast cancer diagnosis within both 5-year and 10-year after diagnosis.
Material and methods
Experimental dataset
This retrospective study was approved by the Ethics Committee of the Istituto Tumori “Giovanni Paolo II” of Bari, Italy, and involves 696 female patients registered for a first invasive breast cancer diagnosis in the period 1992–2021. A written ‘informed consent’ for publication was collected for all the patients involved in the study, except for patients who are dead or not reachable, as it is a retrospective study, and the data were accessed for research purposes on February 7th, 2023. Afterwards, a data de-identification process was performed in order to not allow to identify individual participants during or after data collection. Patients who underwent neoadjuvant chemotherapy and/or had carcinoma in situ and/or had metastasis ab initio were not included in this study. Besides, only patients who had an IDE within 5 years from the first tumor diagnosis or patients who had not an IDE after the first tumor and with at least 5-years of follow-up were involved in this study, for a total of 168 recurrence IDE cases and 528 control cases (non-IDE cases). Then, with the aim of also predicting the possibility for a patient to have a relapse within 10 years from the first breast cancer diagnosis, we defined a sub-sample of the experimental dataset which counts 626 patients having either an IDE within 10 years from the first tumor diagnosis or having not an IDE and with a follow-up of at least 10 years, for a total of 251 IDE cases and 375 control cases. The difference between the two sample sizes is due to the absence of a 10-years follow-up for some patients belonging to the first dataset. Breast cancer clinical and histopathological data, as well as chemotherapy information, were collected from the patients’ medical records. For each patient, a total of 28 features were compiled, comprising age at diagnosis, previous tumors (values: absent/present), breast site of the tumor (abbr. breast, values: right/left), histological type (values: ductal, lobular, others), intraductal component (values: absent, not typed, G1, G2, G3), multifocality (values: absent/present), angioinvasion (values: absent, focal, extensive, not typed), estrogen receptor expression (abbr. ER, % value), progesterone receptor expression (abbr. PgR, % value), cellular marker for proliferation (abbr. ki67, % value), human epidermal growth factor receptor-2 score (abbr. HER2/neu, values: 0, 1, 2, 3), human epidermal growth factor receptor-2 (abbr. HER2, values: negative/ positive), histological grade (abbr. grading, values: G1, G2, G3), tumor size (abbr. diameter, values: T1a, T1b, T1c, T2, T3, T4), lymph-node status (values: N0, N1, N2, N3), having performed the sentinel lymph-node biopsy (abbr. SLNB, values: not performed, negative, positive), having performed the axillary lymph-node dissection (abbr. ALND, values: not performed/performed), eradicated lymph-nodes, metastatic lymph-nodes, type of surgery (abbr. surgery, values: mastectomy/conserving-surgery), having received chemotherapy (abbr. chemotherapy, values: no/yes), chemotherapy scheme (values: not received, C1—anthracycline and taxanes, C2—anthracycline, C3—taxanes, C4—CMF, C5—other), having completed chemotherapy treatment (abbr. chemotherapy completion, values: yes/no), having received trastuzumab (abbr. trastuzumab, values: no/yes), having completed trastuzumab treatment (abbr. trastuzumab completion, values: yes/no), having performed hormone therapy (abbr. hormone therapy, values: no/yes), hormone therapy scheme (values: not received, H1—tamoxifen, H2—Lh-Rh, H3—tamoxifen and Lh-Rh, H4—Aromatase Inhibitor, H5—tamoxifen and Aromatase Inhibitor, H6—Lh-Rh and Aromatase Inhibitor, H7—other) and having completed hormone treatment (abbr. hormone therapy completion, values: yes/no).
An overview about the sample clinical properties is provided by Table 1.
An artificial intelligence-based approach for IDE prediction
We developed an artificial intelligence-based model to predict both 5-year and 10-year breast cancer IDE, after performing a stratified randomly sampling which allows to split both datasets in a hold-out training set, including 80% of patients, and a hold-out test set, comprising 20% of patients. The few missing data were initially estimated by means of a proximity algorithm which allows to replace the missing data of each patient with the data belonging to the patient having the minimum Euclidean distance from the patient under consideration and without missing values [33]. Potential missing features of newcome patients will be estimated comparing each incomplete observation to the 696 patients belonging to the dataset employed in this study.
All steps of the proposed approach depicted in Fig 1 have been implemented on both 5-year and 10-year datasets. All the analysis steps have been performed by using MATLAB R2022a (Mathworks, Inc. Natick, MA, USA) software.
(a) After transforming each patient represented by a vector of clinical information to an image form, (b) from each image we extracted radiomic features by means of a pre-trained CNN. (c) Finally, we identified a subset of significative features, and we trained a SVM classifier to classify breast cancer patients in the two classes, namely, IDE and no IDE. All steps of the proposed approach have been implemented on both 5-year and 10-year datasets.
1. Image transformation.
The first step consisted in transforming each patient, represented by a vector of 28 clinical information, to an image form (Fig 1A). Given the hold-out training set as an array X of dimension n × 28, where n represents the number of patients and 28 is the number of clinical features, we firstly computed X* as the transpose array of X. Then, we applied a norm-1 normalization on X* with the purpose of normalizing each feature by its minimum and maximum. Further details about the normalization procedure are discussed in the work of Sharma et al. [34].
At this step, with the aim of defining a two-dimensional plane on which projecting pixel frames for each patient, we implemented a Kernel Principal Component Analysis (K-PCA) technique with gaussian kernel to reduce the feature space dimensionality [37]. In this way, we transformed a manifold in a n-dimensional space (one dimension for each patient) to a projection onto a two-dimensional plane, where each point represents the location of a single feature, where new feature coordinates are identified in the basis of the first two principal components. K-PCA rather than standard PCA has been implemented since decision boundaries of our data were described by non-linear functions.
Afterwards, the minimal bounding rectangle of all points representing feature locations in the plane was identified by means of a convex hull algorithm [38]. After receiving in input coordinates of all points, this algorithm allowed to define the rectangle having minimum area and including all points. This bounding rectangle along with all points representing feature locations were finally rotated in the horizontal direction.
The final step consisted in mapping feature values to feature locations, for each patient. To this end, we first transformed point coordinates in pixel frames to determine the location of every feature in the image-form. Then, for each patient, we created the corresponding image assigning to these locations the feature values for the patient into account. Accordingly, we generated a unique image for each patient. Further details about the pixel conversion procedure are discussed in the work of Sharma et al. [34].
Finally, on both hold-out test sets, this final step has been implemented starting from both the bounding rectangle and all feature locations previously identified by the related hold-out training set.
2. Feature extraction.
As depicted in Fig 1B, the second step consisted in extracting radiomic features from each image by means of a pre-trained Convolutional Neural Network (CNN), namely, AlexNET [39], after resizing all images to 227×227 pixels, since the network requires images of such size as input. Specifically, we extracted features from the pool2 layer of the network architecture which corresponds to the second pooling layer after the second convolutional layer of the network. The pool2 layer has an output with dimensions of 13×13×256 that is flattening to a single 43.264-length vector. As consequence, the number of extracted features is 43.264 in total for each image.
3. Learning model.
The final step consisted in developing a learning model able to predict both 5-year and 10-year breast cancer recurrence event (Fig 1C). Specifically, for each temporal frame, this model was originally evaluated on the hold-out training set within a 10-fold cross-validation scheme over 10 rounds, and afterward, the model was validated on the hold-out test set exploiting only features resulted as important within every cross-validation cycle of every round.
First, exploiting features collected in the previous step, we performed a feature selection procedure with the aim of identifying a subset of significative features. To start with, we separately investigated the predictive power of each feature by means of a nonparametric statistical test, retaining the only features whose p-value resulted less than 0.005. Then, we implemented a Random Forest-based feature selection process to overall evaluate the importance of features previously identified as statistically significant. Features were considered as important when their weight computed by means of the so-called Gini impurity was greater than the median weight of all features involved [40].
Subsequently, we classified breast cancer patients in the two classes, namely, recurrence and no recurrence, via a linear Support Vector Machine (SVM) classifier trained on the identified subset of significative features [41].
For each temporal frame, performances on both the hold-out training and the hold-out test set were evaluated in terms of Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC) curve and other standard metrics such as accuracy, sensitivity and specificity computed by identifying the optimal threshold by means of a Youden’s index test [42]. Finally, with the aim of determining the statistical significance of our results, for each of the two temporal frame we performed a permutation test to assess whether the classifier has found a real class structure in the data [43]. We calculated a null distribution of the AUC value of our model on dataset after randomly permuting the patient labels 100 times. Thus, we estimated an empirical p-value as the ratio of the number of times an AUC value computed with permuted labels was less than or equal to the actual AUC value increased by 1 over the number of implemented permutations increased by 1.
Results
The stratified randomly sampling allowed to split both 5-year and 10-year datasets in a hold-out training set, including 80% of patients, and a hold-out test set, comprising 20% of patients. Consequently, the 5-year hold-out training set consisted of 557 patients, of which 136 (24.4%) IDE cases and 421 (75.6%) control cases, whereas the 5-year hold-out test consisted of 139 patients, of which 33 (23.7%) IDE cases and 106 (76.3%) control cases. Similarly, the 10-year hold-out training set counted 501 patients, of which 201 (40.1%) IDE cases and 300 (59.9%) control cases, and the 10-year hold-out test counted 125 patients, of which 50 (40.0%) IDE cases and 75 (60.0%) control cases.
To investigate existing relations among patients belonging to the same dataset, and, consequently, performing kernel PCA, we computed Spearman correlation coefficients among patients and the corresponding p-values [38]. As a result, two n×n matrices were obtained for each temporal frame. Fig 2 shows both correlation and p-value matrices associated to the 5-year dataset. The correlation heatmap show a significatively strong positive correlation among patients. Results related to the 10-year dataset are approximately the same and, therefore, are shown in a S1 Fig to not burden the discussion.
Correlation heatmap (a) computed through Spearman test and p-value heatmap (b) for patients belonging to the 5-year dataset. The correlation heatmap reported in Fig 2A demonstrates a strong positive correlation among patients, and the p-value heatmap depicted in Fig 2B confirms the statistical significance of these correlation.
Then, the image transformation procedure allowed to generate a unique image for each patient, mapping feature values of every patient to feature locations previously identified. Fig 3 provides some examples of generated images.
Images provided by Fig 3 highlight both similarities between patients belonging to the same class, and discrepancies between patients of different classes, in terms of pixel intensity values. Accordingly, examples of images computed for patients included in the 10-year dataset emphasize the same similarities and discrepancies and are depicted in a S2 Fig to not burden the discussion.
Examples of images generated within the image transformation procedure for patients belonging to the 5-year dataset. Images (a) and (b) referred to non-IDE patients, whereas images (c) and (d) belonged to IDE patients.
Classification performances achieved by both 5-year and 10-year predictive model are presented in Figs 4 and 5, respectively. Specifically, each radar chart includes performances obtained on both the hold-out training and the hold-out test set.
On the hold-out training set, performances were evaluated in terms of percentage median values and the first and the third quartile values. Reported percentages refers to results achieved on the hold-out test set.
On the hold-out training set, performances were evaluated in terms of percentage median values and the first and the third quartile values. Reported percentages refers to results achieved on the hold-out test set.
Both 5-year and 10-year models resulted particularly accurate in predicting breast cancer recurrence event, achieving on the hold-out test set a percentage accuracy equals to 88.71% and 88.82%, respectively. Besides, these performances reveal a perfect balance between the number of IDE cases correctly classified as IDE, i.e., the sensitivity percentage value, and the number of non-IDE cases correctly classified as non-IDE, i.e., the specificity percentage value. Radar charts also revealed classification performances achieved on the hold-out test set fall in the confidence interval calculated on performances reached on the hold-out training set.
The statistical significance of these results was confirmed by the outcome of the permutation test. Specifically, the permutation test confirmed that the AUC value predicted on the independent test does not belong to the null distribution of the AUC values obtained after randomly permuting the patient labels 100 times (p-value equal to 1 for each of two temporal frame). Therefore, the performed models have found a real class structure in the data.
Discussion
Breast cancer prognosis has always been adversely affected by the molecular heterogeneity of this tumor, precluding the design of targeted treatments for each tumor type [3, 4, 44, 45]. So far, several machine learning algorithms have been developed by the scientific community to support both the diagnosis of breast cancer and the clinical decision-making process in breast cancer treatment [14, 15]. Though these models achieved encouraging results also in predicting the occurrence of an IDE after a primary breast tumor, such as recurrence, metastasis, contralateral and second tumors, performances were not suitable for clinical practice [8, 46].
As documented by recent reviewers on the topic, the models proposed in literature to predict disease recurrence that use only clinical data are very few (Table 2). Recently, in the work of Fu et al. [46], authors developed a 5-year machine learning survival model exploiting demographics, pathological, diagnostics and therapeutic information of patients in their primary structured form. We also proposed different machine learning models to predict a breast cancer IDE analyzing the same dataset, though fewer in number due to the subsequent sample repopulation. We first developed an advanced ensemble algorithm able to identify the so-called confounding patients, having the same clinical and histological characteristics as other patients but responding differently to the adjuvant treatment [33]. Afterwards, we implemented an explainable artificial intelligence framework which offers clinicians a useful white tool for the interpretation of the predicted classification results. This algorithm was then optimized developing a complex grid search algorithm [30]. However, these models were not yet suitable to be implement in clinical practice. On ten independent tests, we achieved a median AUC of 77.1% and 76.3% and a median accuracy value of 75.5% and 71.3% for the 5-year follow-up and the 10-year follow-up, respectively.
With the aim of overcoming limits of our previous efforts in predicting the occurrence of a second breast cancer IDE by means of machine learning techniques, in this work, we proposed a novel approach exploiting the well-known capabilities of pre-trained CNNs in analyzing images to extract low-level quantitative imaging features from images obtained transforming each patient, represented by a vector of clinical information, to an image form. Thus, we developed a transfer-learning approach to predict the occurrence of an IDE at both 5-year and 10-year follow-ups. For this purpose, we considered a private database comprising 696 female patients with a first invasive breast cancer diagnosis and whose clinical and histological information was known. Then, we transformed the feature vector of each patient to an image form, and we analyzed them by means of a pre-trained CNN. Finally, we classified breast cancer patients in the two classes, namely, IDE and non-IDE, via a SVM classifier trained on a subset of significative features previously identified. Performance achieved by both 5-year and 10-year models resulted particularly accurate in predicting breast cancer recurrence event, also revealing a perfect balance between the number of IDE cases correctly classified as IDE, and the number of non-IDE cases correctly classified as non-IDE. Indeed, the 5-year and 10-year predictive models achieved an AUC value of 92.07% and 92.84%, an accuracy of 88.71% and 88.82%, a sensitivity of 86.83% and 88.06%, a specificity of 89.55% and 89.3% respectively.
The proposed approach allowed to significantly improve classification performances, despite the presence of confounding patients. The absence of patients with a strong negative correlation, thus presenting relevant differences in terms of clinical and histological characteristics, was highlighted by the correlation heatmaps (Figs 2A and S1A) and was then confirmed as statistically significant by the p-value heatmaps (Figs 2B and S1B). Hence, our study overcome biases resulting from the presence of confounding patients, being the first one proposing an approach which converts clinical data commonly collected in clinical practice into an image-form, by computing similarity measures among patients and, thus, detecting information not captured by common machine learning-based models. Moreover, this is the first time this approach is employed for predicting an IDE after a breast cancer diagnosis.
The proposed tool developed certainly has the advantage of being user-friendly and low-cost, as it only uses clinical characteristics commonly and directly collected in clinical practice by the medical oncologist in charge of the patient. Other methods proposed at the state-of-the-art, however, achieve performances comparable to those obtained by the proposed model but using other types of feature [22], such as radiomics and genomics, which may not be readily available to the medical oncologist for the assessment of the risk of recurrence.
However, this work has the limitation of losing in explainability with respect to our previous studies. Therefore, our future works will include both a validation process of the proposed model, and the implementation of an explainable artificial intelligence framework which can provide a more transparent assessment of the classification result, driving clinicians to be more confident in enclosing these artificial intelligence-based tools in clinical practice.
Conclusions
Despite the heterogeneous nature of our experimental population caused by the unavoidable evolution of oncological treatments, as well as of diagnostic procedures, our approach represents a step towards the definition of a deep learning-based decision support system for identifying patients at high risk of occurrence of an invasive disease event, exploiting CNNs to analyze clinical and histopathological information commonly collected in clinical practice and converted to an image form. The proposed model provides high-performing results with respect to state-of-the-art models which exploit raw data to train common machine learning algorithms. Thus, this method would allow clinicians to discriminate between low-risk and high-risk patients in a non-invasive and inexpensive way, with the aim of defining personalized oncological therapeutic treatments.
Supporting information
S1 Fig.
Correlation matrix heatmap (a) computed through Spearman test and p-value matrix heatmap (b) for patients belonging to the 10-year dataset.
https://doi.org/10.1371/journal.pone.0312036.s001
(PDF)
S2 Fig. Examples of images generated within the image transformation procedure for patients belonging to the 10-year dataset.
Images (a) and (b) referred to non-IDE patients, whereas images (c) and (d) belonged to IDE patients.
https://doi.org/10.1371/journal.pone.0312036.s002
(PDF)
References
- 1. Bray F, Ferlay J, Soerjomataram I, et al (2018) Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 68:394–424. pmid:30207593
- 2. Arnold M, Rutherford MJ, Bardot A, et al (2019) Progress in cancer survival, mortality, and incidence in seven high-income countries 1995–2014 (ICBP SURVMARK-2): a population-based study. Lancet Oncol 20:1493–1505. pmid:31521509
- 3. Hwang KT, Kim J, Jung J, et al (2019) Impact of breast cancer subtypes on prognosis of women with operable invasive breast cancer: A Population-based Study Using SEER Database. Clinical Cancer Research 25:1970–1979. pmid:30559169
- 4. Wang Y, Yin Q, Yu Q, Zhang J, Liu Z, Wang S, et al. A retrospective study of breast cancer subtypes: the risk of relapse and the relations with treatments. Breast Cancer Res Treat 2011 Nov;130(2):489–98. Epub 2011 Aug 12. pmid:21837481
- 5. Johnson KS, Conant EF, Soo MS (2021) Molecular Subtypes of Breast Cancer: A Review for Breast Radiologists. J Breast Imaging 3:12–24 pmid:38424845
- 6. Massafra R, Bove S, La Forgia D, et al (2022) An Invasive Disease Event-Free Survival Analysis to Investigate Ki67 Role with Respect to Breast Cancer Patients’ Age: A Retrospective Cohort Study. Cancers (Basel) 14:2215. pmid:35565344
- 7. Schmidt M, Victor A, Bratzel D, et al (2009) Long-term outcome prediction by clinicopathological risk classification algorithms in node-negative breast cancer ‐ Comparison between Adjuvant!, St Gallen, and a novel risk algorithm used in the prospective randomized Node-Negative-Breast Cancer-3 (NNBC-3) trial. Annals of Oncology 20:258–264. pmid:18824499
- 8. Wishart GC, Bajdik CD, Dicks E, et al (2012) PREDICT Plus: Development and validation of a prognostic model for early breast cancer that includes HER2. Br J Cancer 107:800–807. pmid:22850554
- 9. Tang G, Shak S, Paik S, et al (2011) Comparison of the prognostic and predictive utilities of the 21-gene recurrence score assay and adjuvant! for women with node-negative, ER-positive breast cancer: Results from NSABP B-14 and NSABP B-20. Breast Cancer Res Treat 127:133–142. pmid:21221771
- 10. Harbeck N, Sotlar K, Wuerstlein R, Doisneau-Sixou S (2014) Molecular and protein markers for clinical decision making in breast cancer: Today and tomorrow. Cancer Treat Rev 40:434–444 pmid:24138841
- 11. Bellotti R, de Carlo F, Massafra R, et al (2004) Topographic classification of EEG patterns in Huntington’s disease. Neurol Clin Neurophysiol 2004:37 pmid:16012598
- 12. Avanzo M, Porzio M, Lorenzon L, et al (2021) Artificial intelligence applications in medical imaging: A review of the medical physics research in Italy. Physica Medica 83:221–241. pmid:33951590
- 13. Scalchi P, Ciccotelli A, Felici G, et al (2017) Use of parallel-plate ionization chambers in reference dosimetry of NOVAC and LIAC ® mobile electron linear accelerators for intraoperative radiotherapy: a multi-center survey. Med Phys 44:321–332. https://doi.org/10.1002/mp.12020
- 14. McKinney SM, Sieniek M, Godbole V, et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94. pmid:31894144
- 15. Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56 pmid:30617339
- 16. Massafra R, Latorre A, Fanizzi A, et al (2021) A Clinical Decision Support System for Predicting Invasive Breast Cancer Recurrence: Preliminary Results. Front Oncol 11:. pmid:33777733
- 17. Amoroso N, Pomarico D, Fanizzi A, et al (2021) A roadmap towards breast cancer therapies supported by explainable artificial intelligence. Applied Sciences (Switzerland) 11:. https://doi.org/10.3390/app11114881
- 18. Basile TMA, Fanizzi A, Losurdo L, et al (2017) Hough transform for clustered microcalcifications detection in full-field digital mammograms. In: Tescher AG (ed) Applications of Digital Image Processing XL. SPIE, p 41
- 19. Mosayebi A, Mojaradi B, Naeini AB, Hosseini SHK (2020) Modeling and comparing data mining algorithms for prediction of recurrence of breast cancer. PLoS One 15:. pmid:33057328
- 20. LG A, AT E(2013) Using Three Machine Learning Techniques for Predicting Breast Cancer Recurrence. J Health Med Inform 04: https://doi.org/10.4172/2157-7420.1000124
- 21. Chakradeo K, Vyawahare S, Pawar P (2019) Breast Cancer Recurrence Prediction using Machine Learning. In: 2019 IEEE Conference on Information and Communication Technology. IEEE, pp 1–7
- 22. Kim W, Kim KS, Lee JE, et al (2012) Development of novel breast cancer recurrence prediction model using support vector machine. J Breast Cancer 15:230–238. pmid:22807942
- 23. Wang H, Li Y, Khan SA, Luo Y (2020) Prediction of breast cancer distant recurrence using natural language processing and knowledge-guided convolutional neural network. Artif Intell Med 110:. pmid:33250149
- 24. Yang PT, Wu WS, Wu CC, et al (2021) Breast cancer recurrence prediction with ensemble methods and cost-sensitive learning. Open Medicine (Poland) 16:754–768. pmid:34027105
- 25. Mohebian MR, Marateb HR, Mansourian M, et al (2017) A Hybrid Computer-aided-diagnosis System for Prediction of Breast Cancer Recurrence (HPBCR) Using Optimized Ensemble Learning. Comput Struct Biotechnol J 15:75–85. pmid:28018557
- 26. Abreu PH, Santos MS, Abreu MH, et al (2017) Predicting Breast Cancer Recurrence Using Machine Learning Techniques. ACM Comput Surv 49:1–40. https://doi.org/10.1145/2988544
- 27. Mazo C, Kearns C, Mooney C, Gallagher WM (2020) Clinical decision support systems in breast cancer: A systematic review. Cancers (Basel) 12 pmid:32041094
- 28. Mazo C, Aura C, Rahman A, et al (2022) Application of Artificial Intelligence Techniques to Predict Risk of Recurrence of Breast Cancer: A Systematic Review. J Pers Med 12 pmid:36143281
- 29. Huang S, Yang J, Fong S, Zhao Q (2020) Artificial intelligence in cancer diagnosis and prognosis: Opportunities and challenges. Cancer Lett 471:61–71 pmid:31830558
- 30. Massafra R, Comes MC, Bove S, et al (2022) A machine learning ensemble approach for 5- And 10-year breast cancer invasive disease event classification. PLoS One 17:. pmid:36121822
- 31. Zou L, Liu F hua, Shen P pei, et al (2018) The incidence and risk factors of related lymphedema for breast cancer survivors post-operation: a 2-year follow-up prospective cohort study. Breast Cancer 25:309–314. pmid:29397555
- 32. Hooning MJ, Aleman BMP, Hauptmann M, et al (2008) Roles of radiotherapy and chemotherapy in the development of contralateral breast cancer. Journal of Clinical Oncology 26:5561–5568. pmid:18854572
- 33. Massafra R, Fanizzi A, Amoroso N, et al (2023) Analyzing breast cancer invasive disease event classification through explainable artificial intelligence. Front Med (Lausanne) 10:. pmid:36817766
- 34. Sharma A, Vans E, Shigemizu D, et al (2019) DeepInsight: A methodology to transform a non-image data to an image for convolution neural network architecture. Sci Rep 9:. https://doi.org/10.1038/s41598-019-47765-6
- 35. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359
- 36. Comes MC, Fanizzi A, Bove S, et al (2021) Early prediction of neoadjuvant chemotherapy response by exploiting a transfer learning approach on breast DCE-MRIs. Sci Rep 11:14123. pmid:34238968
- 37. Schölkopf B, Smola A, Müller K-R. Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Neural Computation 10, 1299–1319
- 38.
Preparata FP, Shamos MI (1985) Convex Hulls: Basic Algorithms. In: Computational Geometry. Springer New York, New York, NY, pp 95–149
- 39. Russakovsky O., Deng J., Su H. et al. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis 115, 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
- 40. Schonlau M, Zou RY (2020) The random forest algorithm for statistical learning. Stata Journal 20:3–29. https://doi.org/10.1177/1536867X20909688
- 41.
Cyran K.A. et al. (2013). Support Vector Machines in Biomedical and Biometrical Applications. In: Ramanna S., Jain L., Howlett R. (eds) Emerging Paradigms in Machine Learning. Smart Innovation, Systems and Technologies, vol 13. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28699-5_15
- 42. Youden WJ (1950) Index for rating diagnostic tests. Cancer 3:32–35. pmid:15405679
- 43. Ojala M, Garriga GC (2010) Permutation Tests for Studying Classifier Performance. Journal of Machine Learning Research 11 (2010) 1833–1863
- 44. Harbeck N, Penault-Llorca F, Cortes J, et al (2019) Breast cancer. Nat Rev Dis Primers 5:. https://doi.org/10.1038/s41572-019-0111-2
- 45. Hennigs A, Riedel F, Gondos A, et al (2016) Prognosis of breast cancer molecular subtypes in routine clinical care: A large prospective cohort study. BMC Cancer 16:. https://doi.org/10.1186/s12885-016-2766-3
- 46. Fu B, Liu P, Lin J, et al (2019) Predicting Invasive Disease-Free Survival for Early Stage Breast Cancer Patients Using Follow-Up Clinical Data. IEEE Trans Biomed Eng 66:2053–2064. https://doi.org/10.1109/TBME.2018.2882867.