Fig 1.
The main analysis steps.
Fig 2.
Whole slide image of thyroid sample GTEX-11NSD-0126.
(A) Whole slide. (B) Detail. (C) Tile of size 512x512.
Table 1.
Convolutional neural network architectures used in this paper.
For ResNet34, the features considered involve only layers that are not “skipped over”, while for Inception_v3 only elementary layers with non-negative outputs (ReLU, max/avg-pooling) that are not on the auxiliary branch.
Table 2.
Classification accuracies for image tiles.
Table 3.
Classification accuracies for whole slides.
Table 4.
Numbers of significantly correlated gene-feature pairs for various network architectures.
Fixed correlation- and log2 gene expression thresholds are used: RT = 0.8, ET = 10. Numbers of unique genes, features and tissues involved are also shown (test dataset).
Table 5.
Numbers of significantly correlated gene-feature pairs for various correlation- and log2 gene expression thresholds.
Numbers of unique genes, features and tissues involved are also shown (VGG16 network, test dataset).
Fig 3.
Numbers of significantly correlated genes for the 31 layers of VGG16.
Significantly correlated genes were aggregated for all features (channels) belonging to a given layer (i.e. for all features of the form [layer]_[channel]). The correlated genes were broken down according to their tissues of maximal expression. Tissues are color-coded. Colors of the figure bars scanned bottom-up correspond to colors in the legend read top to bottom.
Table 6.
Developmental and transcription regulation genes correlated with visual features.
Fig 4.
Visualizations of select histological features.
The following features (of the form [layer]_[channel]) found correlated with specific genes are visualized on each row: row 1: 8_55—CTRC (r = 0.85), row 2: 18_13—CUZD1 (r = 0.866), row 3: 23_324—CELA3B (r = 0.868), row 4: 30_467—AMY2A (r = 0.928). Original image (column 1), guided backpropagation of the feature on the original image (column 2), synthetic image of the feature (column 3), immunohistochemistry image for the corresponding gene from the Human Protein Atlas (column 4). All visualizations are for pancreas sample tile GTEX-11NSD-0526_32_5.
Fig 5.
Visualizations of select histological features.
The following features (of the form [layer]_[channel]) found correlated with specific genes are visualized on each row: row 1: 22_405 NKX2-1 (r = 0.726) Thyroid GTEX-11NSD-0126_31_16, row 2: 27_343 TG (r = 0.801) Thyroid GTEX-11NSD-0126_31_16, row 3: 29_499 PAX8 (r = 0.802) Thyroid GTEX-11NSD-0126_31_16, row 4: 25_260 NEB (r = 0.831) Muscle—Skeletal GTEX-145ME-2026_39_19. Original image (column 1), guided backpropagation of the feature on the original image (column 2), synthetic image of the feature (column 3), immunohistochemistry image for the corresponding gene from the Human Protein Atlas (column 4).
Fig 6.
Visualizations of select histological features.
The following features (of the form [layer]_[channel]) found correlated with specific genes are visualized on each row: row 1: 29_234 AGXT (r = 0.939) Liver GTEX-Q2AG-1126_9_7, row 2: 29_118 CALR3 (r = 0.829) Testis GTEX-11NSD-1026_5_27, row 3: 30_244 KLK3 (r = 0.828) Prostate GTEX-V955-1826_8_13, row 4: 20_137 CYP11B1 (r = 0.850) Adrenal Gland GTEX-QLQW-0226_25_9. Original image (column 1), guided backpropagation of the feature on the original image (column 2), synthetic image of the feature (column 3), immunohistochemistry image for the corresponding gene from the Human Protein Atlas (column 4).
Table 7.
Numbers of significant gene-feature pairs with indirect dependencies gene-tissue-feature (g-t-f), feature-gene-tissue (f-g-t), gene-feature-tissue (g-f-t).
Conditional independence tests with α = 0.01 and respectively α = 0.05.