Fig 1.
Overview of the workflow of the multiple-instance learning (MIL) approach applied to hyperspectral chemical image data.
(a) A summary of the preprocessing steps for chemical images, including preparation procedures prior to training the deep learning model and optimising hyperparameters. (b) The skeletal structure of the linear model architecture utilised in this study, which is loosely inspired by the ResNet framework.
Table 1.
List of hyperparameters and their values for the grid search hyper parameter model optimisation.
Fig 2.
Training results for model architecture optimisation without residual connection using different hidden number of nodes accumulated over three trial runs.
Left panel shows the ‘half’ expansion type while eight panel shows ‘double’ expansion type. Top panel depicts the training and validation F1-score and bottom panel depicts the loss values accumulated during training phase. Shaded regions are the standard deviation x 0.05 scale factor.
Fig 3.
Average performance landscape of the validation dataset showing how different model architecture parameters impact the model prediction without residual connection.
Left panel show ‘half’ expansion type and right panel show ‘double’ expansion type.
Fig 4.
Training results for model architecture optimisation with residual connection using different hidden number of nodes accumulated over three trial runs.
Left panel shows the ‘half’ expansion type while right panel shows the ‘double’ expansion type. Top panel depicts the training and validation F1-score and bottom panel depicts the loss values accumulated during training phase. Shaded regions are the standard deviation x 0.05 scale factor.
Fig 5.
Average performance landscape of the validation dataset showing how different model architecture parameters impact the model prediction with residual connection.
Left panel shows the ‘half’ expansion type and right panel shows the ‘double’ expansion type.
Table 2.
Training results for learning rate parameter for both validation and training dataset over three trial run with standard deviation based on the optimum model architecture parameter.
Fig 6.
Attention maps from two representative samples from each group (left panel is CRC0076 and right panel CRC0344) obtained from normalised attention weight generated by the MIL mechanism in the model.
Grayscale image background is the tissue area under the curve projection while the coloured regions are the intensity of the normalised attention weights.
Fig 7.
The important wavenumber features based on the mean SHAP values of the two different classes that contribute to the model classification decision.
The black line indicates specific location of the molecular vibrations within the IR range while the dotted line indicates its range. ν (nu) represent stretching vibration of the atomic bonds (bond length) while δ (delta) represent bending vibration (bond angle, i.e.,; scissoring, rocking, wagging and twisting).
Fig 8.
Biological relevance of the most highly attended spectral instances.
a) UMAP clustering of the top-attended spectra, illustrating distinct groupings according to treatment type (top panel) and classification label (bottom panel). b) Mean absorbance profiles of the top-attended spectra, shown by classification group (top panel) and treatment type (bottom panel), with annotated protein-related wavenumber bands. Solid grey lines mark specific band positions, while dotted lines indicate the boundaries of the corresponding spectral regions.
Fig 9.
Correlation between selected top-attended spectral features and spectral ratios to the protein expression levels.
a) Normalised absorbance values and ratios at specific wavenumbers and corresponding relative protein expression levels for CRC0076 (top panel) and CRC0344 (bottom panel). b) Correlation analysis using Pearson’s correlation between spectral absorbance values and ratios to the relative protein expression levels for CRC0076 (top panel) and CRC0344 (bottom panel).