Fig 1.
During the first phase, the data modeling randomly selected two tissue slides as the test set: One from the responder group (red tissue slides), and the other from the non-responder group (blue tissue slides).
In the figure, slides 3 and 6 were selected as the test set from the responder and non-responder groups, respectively. The 33 remaining patients were used as the training set, to feed the machine learning system. All features from the 33 patients were extracted and analyzed to train the artificial intelligence. During the second phase, the convolutional neural networks (CNNs) ignored the belonging group of the two patients from the test set (gray tissue slides), and classified them as responders or non-responders based on their similarity to the training set tissue slides.
Fig 2.
A) Tissue slide of a spindle cell carcinoma of the lung. Three regions of interest (ROIs) were selected (blue squares) based on tumor heterogeneity. B–D) Close-up views of tumor cells within desmoplastic fibrosis (B), areas of necrosis within tumor aggregates (C), and prominent inflammation surrounding tumor cells (D).
Fig 3.
The sequence of crops and patches extracted from a single tumor.
Each group of cancer cells is contoured, defining 9 crops (areas contoured in red) within a single region of interest (ROI; blue square).
Fig 4.
AlexNet structure with key workflow.
Fig 5.
VGG structure with key workflow.
Fig 6.
MobileNet structure with key workflow.
Fig 7.
GoogLeNet structure with key workflow.
Fig 8.
GoogLeNet input layer.
Fig 9.
ResNet structure with workflow.
Fig 10.
ResNet input layer.
Fig 11.
ResNet residual block.
Fig 12.
ResNet residual learning layer.
Table 1.
Patients’ characteristics.
Fig 13.
Depiction of the utilised pipeline.
it consists of the following parts:
- Image acquisition: several images are acquired for each patient;
- Images segmentation: each image undergoes a manual segmentation process by an experienced physician to extract only the pixels containing the cancer cells;
- Model training: the selected pre-trained model is fine-tuned on all the training crops belonging to the selected fold. The validation technique is the leave-one-patient-out and ensures that all the crops extracted from a patient can compose either the training or the test set, thus preventing the model from being biased;
- Model test: all the crops in the test set of the selected fold are analysed by the trained model, which returns a prediction. In order to maintain balance between classes each test set always consists of the crops from two patients coming with different labels;
- Performance evaluation: once the model has predicted the outcome of all crops from each test fold, a global performance evaluation step calculates the metrics needed to compare the models
Table 2.
The performance of the convolutional neural network (CNN) architectures at the patient level, in terms of accuracy, F1 score, and precision.
Table 3.
The tables are the CMs obtained by each CNN in the test set.
Table 4.
The table reports the True positive rate (TPr) and the True negative rate (TNr).
Fig 14.
Two tissue slides that were failed by all the convolutional neural networks (CNNs).
A) Whole-slide image (WSI) of a responder patient affected by adenocarcinoma, who was incorrectly classified as a non-responder by all the CNNs (false negative). Scale bar: 5 mm. B) Close-up view showing fragmented and stretched tumor glands. Scale bar: 250 μm. C) WSI of a non-responder patient affected by spindle cell carcinoma, who was incorrectly classified as a responder by all the CNNs (false positive). Scale bar: 2.5 mm. D) Close-up view showing tumor infiltrating the submucosal stroma. Scale bar: 250 μm.
Fig 15.
The evolution of training loss across epochs is delineated for the various CNN architectures.
The solid line denotes the mean value encompassing the range of experiments, while the shaded colored region signifies the span of standard deviation.
Fig 16.
The flowchart outlines the potential practical implementation of deep-pathomics.
Following tissue collection, either through biopsy or surgery, and subsequent histological diagnosis, the tumor tissue slides undergo digitalization via Whole Slide Imaging, without further additional tissue consuming; therefore, the residual tumor tissue will be available for molecular testing and immunohistochemistry. Tumor tissue slides from eligible patients for chemoradiation will undergo assessment using Deep-Pathomics. Based on this analysis, different personalized therapeutic strategies can be employed accordingly. The illustration depicts two distinct patients affected by Stage IIIB adenocarcinoma of the lung. Despite the same diagnosis, these patients underwent different treatment approaches as determined by Deep-Pathomics analysis.