https://orcid.org/0000-0001-6334-0625
Figures
Abstract
Ninety percent of drugs fail during clinical trials, mainly due to lack of clinical efficacy. Recent developments in in vitro models such as 3D tumor heterospheroids have led to improvements in failure rates, but the relative lack of standardized evaluation methods for 3D cultures limits their utility in high-throughput screening. Optical coherence tomography (OCT) shows significant promise for high-throughput screening of 3D models; however, the optimal classification model and key image features for assessing drug efficacy in OCT images of spheroids has yet to be explored in detail. In this study, we investigate whether OCT combined with machine learning methods can be used to identify biomarkers of drug efficacy in 3D tumor spheroid models. We further compare the performance of two different models to determine the optimal configuration for accurate classification. Volumetric OCT images were acquired of co-cultured HT29 spheroids treated with 3 different concentrations of cisplatin. A two-dimensional multi-view ResNet model and a three-dimensional ResNet model were used to classify the images and to identify key image features associated with each group. Differences between spheroids treated with different concentrations of cisplatin are clearly visible in the OCT images. Our model was able to classify the images based on cisplatin concentration with 71.9% accuracy using the 2D multi-view model and 91.2% accuracy using the 3D model. Key features in the 3D model significantly improved the model accuracy. These results underscore the possibility that OCT could be used for high-throughput screening of drugs using 3D in vitro models and highlight key identifying features for further investigation.
Citation: Untracht GR, Kaminski J, Guldenring E, Nielsen BS, Holmstrøm K, Jensen KH, et al. (2026) A 3-dimensional Resnet model for assessment of drug efficacy in 3D cancer models using optical coherence tomography. PLoS One 21(7): e0353170. https://doi.org/10.1371/journal.pone.0353170
Editor: Kazunori Nagasaka, Teikyo University: Teikyo Daigaku, JAPAN
Received: January 3, 2026; Accepted: June 19, 2026; Published: July 24, 2026
Copyright: © 2026 Untracht et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All OCT image files are available from the DTU Data repository at DOI 10.11583/DTU.32105659. All software underlying the work presented in this manuscript can be found at https://github.com/alexandrainst/oct-spheroid.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Cancer is one of the leading causes of death worldwide, accounting for approximately 10 million deaths each year [1]. While cancer deaths have declined steadily over the last few decades due to earlier diagnosis, lifestyle changes such as a reduction in smoking, and the development of novel therapies, cancer remains a significant public health challenge [2]. As our knowledge of the disease evolves, new opportunities arise for the development of targeted and personalized therapies, underscoring the need for ongoing research and development of new drugs and treatment strategies [3].
While significant investment is made towards the development of new drugs, more that 90% fail in the clinical trial stage. This puts a significant economic burden on society, leading to an average cost of between $300 million and $4.5 billion for each new drug depending on the therapeutic area, with high cost passed on to patients [4]. Further, it takes an average of 6–12 years from initial development stages until new drugs are available on the market [5]. Research has shown that any strategy to reduce the time to develop new drugs would have a significant impact both on terms of the overall cost of drugs and the number of lives saved. In particular, high-throughput screening methods that can identify the most promising candidates quickly and early in the process could have significant societal impact.
One of the main challenges affecting the large failure rate is the use of simplistic models such as 2D cell culture platforms which cannot mimic the complex tumor microenvironment found in vivo [6]. Recent advances in 3D models such as multicellular tumor spheroids have significantly improved the success rate since they can more accurately model cell-cell and cell-environment interactions [7,8]. Additionally, 3D models can incorporate scaffold structures and extracellular proteins to more accurately replicate conditions that the drugs will experience in vivo. One approach for incorporating extracellular proteins is to use heterospheroid cultures comprising co-cultures of fibroblasts and cancer cells, which simulate the tumor microenvironment more closely to the in vivo conditions compared with monocultures of cancer cells. The fibroblast cells mediate spheroid formation, and the cancer cells determine the spheroid architecture; an extracellular matrix is generated, that provides the basis for assessment of in vivo-like therapeutic efficiency of drugs [8].
Since 3D models are relatively new, few analysis methods exist that can exploit the newly available volumetric data. Bright field microscopy is commonly used to visualize spheroids, however, this technique generates 2D projection images; volume and 3D shape measurements based on bright field images are inconstant depending on the estimation method used [9]. Histological analysis is commonly employed [10,11], but the spheroids must be fixed and sliced, which can impact microstructural information. Fluorescence microscopy is a gold standard method for volumetric imaging of cells [12]. Light-sheet fluorescence microscopy, in particular, shows promise for high-throughput assessment. Although fluorescent labelling may be required to acquire images quickly with sufficient contrast, which can disturb the cellular environment, fluorescently labeled variants can readily be developed for various cell lines. However, these methods are limited in terms of imaging depth and can otherwise be challenging due to, e.g., photobleaching.
One method which shows promise towards high-throughput assessment is optical coherence tomography (OCT) [13], which enables volumetric imaging of tissue microstructures over millimeter scale volumes at video-rate imaging speeds [14]. OCT is label free, with contrast based on optical scattering, and can visualize whole spheroid volumes. Since OCT is non-destructive, it can easily facilitate longitudinal assessments.
In order to detect subtle changes in the spheroid structure, various machine learning approaches including convolutional neural networks (CNNs) have been applied to improve the detection of key features relevant for assessing drug efficacy [15–18]. Due to the complexity and computational challenges of working with large datasets, many machine learning-based classification schemes rely on single 2D images of the object [19]. However, single images may contain insufficient information for an accurate decision. This is especially true for challenging classification problems. On the other hand, full 3D datasets can be very large and computationally intensive to work with. For this reason, the 2D multi-view model was proposed. The strength of 2D multi-view CNN compared to 3D CNN is reduced computational complexity, while efficiently capturing variation in especially shape and size of the objects. Data dimensionality and required dataset size are smaller and model training time shorter. On the other hand, the 3D model can be considered the purest approach, where input data is a full representation of the biological object and user interference is reduced to a minimum.
In this study, we explored several ResNet [20] CNN models to quantify the structural change in HT29 heterospheroids [8] after exposure to different concentrations of cisplatin. Our goal was to develop a model that could accurately classify images of spheroids based on the amount of cisplatin and to identify key distinguishing features between the measurement classes. We here also compared the performance of the 2D multi-view model and the 3D model and report that volumetric information is a critical contribution to the accuracy of the classification task. Our overall intention is to show the potential for OCT imaging combined with deep learning as a potential tool for high-throughput screening of drug candidates without the need for human intervention.
2. Methods
2.1. Cell culture and drug treatment
Heterospheroid cultures were prepared as described in Nielsen et al. [8]. Briefly, HT-29 (ATCC®, HTB-38™) colon cancer cells were co-cultured with 1BR.3.G fibroblasts (ECACC, Cat. 90020507) in ultra-low-attachment (ULA) 96-well plates (Corning®, Corning, NY, USA, #7007). Cancer cells and fibroblasts cells were seeded in a 1:2 ratio (2500:5000) per well. Prior to intervention, the co-cultures were incubated for 72 hours to allow spheroids to form.
Dilutions of cisplatin (Merck #C2210000) were prepared from a dimethylformamide (DMF) stock solution (33.3 mM) in culture media. The vehicle suspension contained 0.3% (v/v) DMF in culture medium. At day 3 (72 hrs) after seeding each well plate was treated with the vehicle solution (20 replicate wells), the 25 µM cisplatin solution (20 replicate wells) and the 100 μM cisplatin solution (20 replicate wells), respectively.
After 4 days (96 hrs) of drug exposure the spheroids were prepared for OCT-imaging using Histogel (Thermo Scientific #HG4000012). Briefly, the growth medium (190 µl) was removed carefully from each well and 50 µl liquefied Histogel was subsequently applied to each well. Histogel was added to modulate the refractive index difference between the spheroid, the bottom of the well plate, and the background. The Histogel solidified rapidly, and the plates were transported to the OCT facility for imaging in a portable incubator chamber. A total of 180 spheroids per treatment (total number = 540 spheroids) was produced for OCT-imaging.
Validation of the effect of the drug exposure was performed by acquiring parallel bright field images of the spheroids using a NYONE® Scientific Imager (Synentec GMBH, Elmshorn, Germany) and assessing the viability of the spheroids using the CellTiter-Glo® 3D Cell Viability Assay (Promega #G9681) following the manufacturer’s instructions. In order to assess the significance of the change in cell viability, we have used a one-way ANOVA using Dunnett’s multiple comparison test, assuming Gaussian distribution and equal standard deviations (SD) of the data points.
2.2. Histology
For histological examination, 5–6 spheroids from each of the 3 treatment groups were fixed in formalin for 3 days, then dehydrated and paraffin embedded using standard procedures [8]. Five-µm-thick sections were obtained from each paraffin block and used for hematoxylin and eosin staining or immunohistochemistry. For immunohistochemical staining, sections were processed using a Ventana Discovery Ultra instrument (Roche, Basel, Switzerland). Antibodies for staining of fibroblasts and cancer cells included anti-MRC2 (mAb OTI9G4 at 1:10,000, OriGene) and anti-CK (mAb MNF116 at 1:1,000, Dako-Agilent). Both were detected with OmniMap anti-Ms HRP followed by diaminobenzidine (DAB) substrate. Sections were counterstained with hematoxylin. The stained slides were scanned using an AxioScan Z1 slide scanner, and representative images acquired using Zeiss Zen software.
2.3. OCT imaging
OCT images were acquired using a spectral-domain OCT system (TELESTO II, Thorlabs inc., NJ, USA). This system employs a superluminescent diode with a central wavelength of 1310 nm and a bandwidth of 140 nm and provides an imaging resolution of 5.5 μm and 13 μm in air in the axial and lateral directions, respectively. Images were acquired with a line scan rate of 28 kHz over a 1 × 1 mm area with 1 μm pixel spacing. Each well plate was imaged within one day and was kept at room temperature (21 °c) in a temperature- and humidity-controlled laboratory during imaging. One or two images were acquired of each well with different focal plane placements depending on the size of the spheroid to ensure that the classification algorithm would work independent of the focus position in the spheroid. Volumetric OCT data was pre-processed using mean-standard deviation normalization to reduce the sensitivity of the classification model to variations in image acquisition parameters.
2.4. Model development, testing, and validation
A 2D multi-view classification model [19] was established in python based on a ResNet architecture [21]. Sets of orthogonal, cross-sectional images were extracted from each OCT volume representing different views of the same object. The first layer of the ResNet model concatenates all features from the different views, accumulating the information coming from different angles and regions of the spheroid.
A 3D ResNet18 model was established in python by adapting the 2D model [20]. The model applies three-dimensional convolutional filters to the volumetric data samples, capturing spatial and temporal features across width, height and depth, larger and smaller structures alike.
A stratified split was used to separate data into training, validation, and test datasets. Model accuracy was calculated as the percentage of correctly classified images in the test dataset: . Cross-entropy loss was used to calculate the model loss:
, where M is the number of classes (three, in this case),
is a binary indicator of whether c is the correct class for observation o, and
is this predicted probability that c is the correct class for observation o. Cross-entropy loss is the standard loss model for multi-class classification tasks. Confusion matrices were generated to visualize the true and false classifications stratified by class.
2.5. Qualitative analysis of activation maps
To gain insights into the decision-making process of the 3D CNN, we performed a qualitative analysis of super-imposed activation maps (attention maps) of the last layer of the neural network. These attention maps generated by GradCam++ [22,23] are a weighted combination of activation maps and provide insight to the most influential input regions used for the model’s prediction.
3. Results
3.1. Validation of spheroid model
The effect of the applied cisplatin can be visualized using bright field imaging (Fig 1). Clear differences can be observed between the spheroids treated with different concentrations of cisplatin. The spheroids treated with the vehicle appear intact, but generally larger 96 h after the treatment was applied. In contrast, the spheroids treated with 25 μM cisplatin do not appear to have grown after the treatment was applied. Evidence of degradation of the spheroids is visible, including a diffuse haze and increased debris visible in the media, indicating that the spheroid structures are breaking apart. The spheroids treated with 100 μM cisplatin show further evidence of degradation, including the distorted non-spherical shape and larger hazy region of cell debris surrounding the spheroid. The images are shown in triplicate to indicate that the treatment effects are highly controlled and repeatable.
Brightfield images of spheroids acquired after 72 h of growth before exposure to cisplatin (t = 0 h), and after 96 h of exposure to cisplatin (t = 96 h). Pairwise, images represent the same spheroid at the two time points, and for each treatment: Vehicle, 25 µM and 100 µM cisplatin, respectively. Three replicate examples are shown.
The effect of cisplatin was further explored using histological analysis. The cisplatin cytotoxic agent acts by killing cells, which in the case of HT29 spheroids cause disintegration of otherwise firmly bound cells in the structures. The killed and loosened cells will be lost during the dehydration and paraffin embedding steps and are therefore poorly retained, especially at the high cisplatin dose. The spheroids treated with the vehicle are thus larger than the cisplatin-treated spheroids. Fig 2a shows the spheroids treated with the vehicle and 25 μM cisplatin stained with H&E and by immunohistochemistry for cancer cell and fibroblast markers, cytokeratin and MRC2, respectively. Histology of the spheroids treated with 100 μM cisplatin could not be performed because of the extent of degradation of the spheroids and loss of structural compactness. An attempt to embed these spheroids in paraffin after fixation was unsuccessful due to disintegration of the spheroids. Images of the vehicle-treated spheroids show that the fibroblasts assemble in the interior of the spheroid while the cancer cells form the outer shell with a few cells infiltrating the core fibroblast region. Metabolic assessment with the CellTiterGlo 3D Cell Viability Assay (Fig 2b) confirms a significant decrease in metabolic activity between all three tested groups (p < 0.0001).
A) Sections of paraffin embedded spheroids were stained with hematoxylin and eosin (H&E) and using immunohistochemical (IHC) detection of cytokeratin and MRC2. Staining was achieved for detection of the cancer cells and fibroblasts, respectively. The spheroids had been treated with vehicle and 25 µM cisplatin for 96 h. B) Assessment of viability of the spheroids using the CellTiterGlo 3D Cell Viability Assay, where the ATP level is measured as an indication of metabolically active cells. For each condition (Vehicle, 25 µM and 100 µM cisplatin, respectively) 20 replicate samples were analyzed. **** indicates a statistically significant reduction in viability after cisplatin treatment at both concentrations (p < 0.0001).
We note that, while 540 spheroids we imaged, only 467 images were included in our dataset. Images were removed from the analysis for several reasons. First, during the process of embedding the spheroids in Histogel, some of the spheroids were destroyed, leading to empty wells. Additionally, in some cases, challenges in the embedding process led to spheroids that were improperly placed in the well. These ‘outlier’ images were removed from the analysis since their inclusion significantly reduced the performance of the classifier. Some examples of images removed from the analysis are shown in Fig 3.
Removed images comprised images where the spheroid was improperly placed in the well plate, for example, at the surface of the Histogel rather than embedded at the bottom of the well.
3.2. Modelling architecture
A classification model was developed using ResNet18 [21]; The ResNet is a mature CNN architecture type designed for training very deep networks effectively, readily available in PyTorch [24]. We selected ResNet18 to accommodate hardware limitations and ensure compatibility with consumer-grade systems. While deeper variants of ResNet could potentially yield improved performance, as demonstrated in other studies, our focus was on maintaining accessibility and efficiency. The CNN can efficiently capture spatial hierarchies and patterns in high-dimensional data such as image volumes and is still considered one of the leading architectures for image data classification. The imaged cancer spheroids display variation in size, shape and internal structure, both across classes and across examples from the same treatment class. As these variations do not follow a clear recognizable visual pattern, traditional feature-based methods would be less effective. For this reason, a CNN-based architecture is an ideal basis for a treatment classification model [25].
In order to assess the importance and influence of single voxels in the tumor spheroids on the treatment classification compared to global features such as shape and size, two different CNN designs were compared: a 2D multi-view CNN and a 3D CNN, described below. The 2D multi-view CNN processes multiple 2D slices of each sample volume, making it more light-weight computationally compared to the 3D CNN (Fig 4), both regarding memory requirements and training time. Here we can keep the original resolution and use sample slices of size 250x1000 pixels. The 3D CNN processes a resized version (described below) of the full 3D volume of the imaged cancer spheroids (Fig 5); it meets hard computational limitations related to sample volume dimensions due to the fact that multiple data samples must be loaded into GPU memory to obtain a proper batch size for the model training. On a GeForce RTX 4090 with 24GB RAM, this required a down-sampling of the training volumes to 200x500x500 voxels, representing the axial and lateral dimensions, respectively.
Four, cross-sectional slices are extracted from the spheroid volume and processed by a ResNet18 model. Slices 1 and 2 are orthogonal vertical cross-sectional slices. Slices 3 and 4 are vertical cross-sectional slices atan angle of 45 degrees to slices 1 and 2. Their output feature vectors are concatenated and passed through a multi-layer perceptron to produce a final class prediction.
The full 3D dataset is passed through the convolution and pooling layers of the model to extract a feature vector, which is classified by a final fully connected layer.
A stratified split was used to separate the data into training, validation and test datasets, as required for neural network model training. The splits and dataset size are described in Table 1.
3.3. Performance of the 2D multi-view and 3D classification models
Representative examples of OCT images of the spheroids in each class are shown in Fig 6. Comparable to the brightfield images, it can be seen that the spheroid decreases in size as the concentration of cisplatin increases. Notably, the spheroids treated with 100 μM of cisplatin are the smallest since, similar to paraffin embedding, most of the killed and loosened cells were removed during the process of embedding the spheroids in Histogel.
For each condition (Vehicle, 25 µM and 100 µM cisplatin, respectively), a representative OCT image is shown Top row: 3D visualizations of the spheroid. Bottom row: 2D cross-sectional image through the center of the spheroid. The bright curved line below the spheroid images in the bottom row (green arrow) indicates the bottom of the well plate. The bottom of the well plate is also visible in the 3D visualizations, indicated by the green dotted line.
The number of slices per volume for the best 2D multi-view model performance was experimentally found to be 4, which resulted in a model training time approximately 100 times faster than the 3D CNN. The 3D CNN model achieved an overall accuracy of 91.2% and the 2D multi-view CNN model achieved an overall accuracy of 71.9%. Learning curves and classification confusion matrices for the two models are shown in Fig 7. Although the 3D CNN dataset samples are much higher dimensional than the 2D multi-view CNN dataset samples, the training and validation loss of the 3D CNN are better decreasing towards a point of stability and show a more reasonable generalization gap between the training and validation samples.
Top row: 2D multi-slice model. Bottom row: 3D model.
3.4. Visual explanations of model predictions
Attention maps for each spheroid were generated to confirm which features or spatial regions in the spheroid images have the largest influence on the classification task; representative 2D and 3D visualizations of the attention maps are shown for examples from each class (Fig 8). In the control group (Vehicle), the 3D model shows a clear focus on the entire spheroid, indicating that the model identifies the full structure of the spheroid as relevant. With increasing cisplatin concentrations, the focus shifts towards the debris outside of the original spheroid, indicating that the model is adapting to the presence of the drug by focusing on the degradation state of the cells. In the 25 μM case the main spheroid structure still influences the model, however, increased cell debris on the bottom of the well has emerged as a notable feature. On the other hand, for the 100 μM class, the main spheroid structure is no longer identified as a key feature in the classification task; the surrounding disintegrating cell material is a stronger identifying feature. Notably, there is no significant attention on the background, indicating that it does not influence the decision-making process.
The top three rows show cross-sectional views of the attention maps overlayed on the spheroid images. The fourth row shows 3D images of the spheroids cut through the center for comparison with the corresponding 3D attenuation maps below. Left column Vehicle, middle column 25 µM and right column 100 µM cisplatin, respectively.
Qualitative inspection of the attention maps across all samples reveals three recurring spatial patterns: spheroid-focused attention, plate/debris-focused attention, and a mixed distribution combining both (Fig 9). When samples from the test dataset are classified into these three categories and stratified by condition, a clear dose-dependent shift emerges. In the control group (Vehicle), spheroid-focused attention is the dominant pattern with no plate-focused cases observed. The 25 µM group introduces a substantial proportion of mixed-signal samples alongside a small number of plate-focused cases. The 100 µM group shows the highest proportion of plate-focused cases and a considerable number of mixed cases, tracking directly with the degree of morphological disintegration at increasing cisplatin dose. Taken together, this dose-dependent gradient suggests that the plate-contact region, where cell debris accumulates following cytotoxic treatment, represents a genuinely informative spatial feature rather than a simple imaging artefact, and appears to contribute to classification alongside the spheroid-focused attention signal.
A) Example attention maps from spheroids treated with 25 µM cisplatin indicating three recurring spatial patterns: spheroid-focused (left), mixed signal (middle), and plate-focused (right). B) Distribution of the spatial patterns stratified by treatment group.
4. Discussion
The overall aim of this study was to identify the best model for quantifying structural changes in tumor spheroids in response to an applied treatment. We tested two ResNet CNN models, a 2D multi-view model and a 3D model. The experiment shows that the ResNet CNN architecture is capable of accurately identifying different treatment-induced levels of regression in the OCT data and that both approaches have potential for serving as the basis for a complete regression prediction model. However, tradeoffs exist between the performance of the model and the required computational power. Notably, the images of the spheroids treated with 25 uM cisplatin can be classified accurately with both models even though the effect may not be visible by eye, which shows the sensitivity of our method.
4.1. Performance comparison between the 2D multi-view and 3D models
Comparing the two models, the biggest tradeoff is accuracy vs computational efficiency. For the 2D multi-view model, the training time was approximately 100 times faster than the 3D CNN. The 3D model required about 3 hours training time for 100 epochs on a GeForce RTX 4090. Additionally, sample size limitations required down sampling of the data. Our experiments showed that improving the pixel spacing improved the accuracy of the model, therefore we have strong evidence that running the 3D model with the full-resolution data would further improve the performance. Full resolution data could improve the accuracy or reduce the requirements for the amount of training data required. Analyzing the model further, we found that structural variation in local voxel neighborhoods of the imaged cancer spheroids are as important as global measures of size and shape, making the 3D CNN model superior for the current dataset.
We chose the ResNet architecture to allow the fairest comparison between the 2D and 3D models. In an initial phase of the study, we confirmed that neither deeper nor shallower ResNet architectures improved the performance of the 2D model (or led to less overfitted models — results not shown); in fact, use of a shallower ResNet resulted in reduced accuracy. We note that alternative architectures for the 2D model could yield better classification performance than the model implemented here and could be a topic for future investigations.
4.2. Model generalizability
While the training curves of both models show ability to effectively learn to fit the training data, the huge gap between the training and validation loss of the 2D multi-view CNN indicates inability to generalize well; the model is overfitting. The main factors are limited data and the complexity of the model. With more shallow architecture experiments not showing significant improvement, this could indicate that the 2D slice sets do not provide sufficient information to learn the problem efficiently, i.e., that evidence of structural patterns associated with each of the 3 classes are left out in the slice sets. For both models, the validation loss shows noisy movements compared to the training loss. This indicates an unrepresentative validation dataset that does not provide sufficient information to evaluate the ability of the model to generalize. Increasing the validation set size might help but would decrease the training set and learning abilities. Both models would benefit from larger sample datasets. Introducing data augmentation could improve the generalizability of the model without the need to acquire additional data and could be investigated to further improve the model [26].
4.3. Perspectives and comparison with other studies
Several studies have demonstrated the useful application of OCT to spheroid imaging including for more accurate assessment of spheroid volume [27], detection of a necrotic core region [28,29], and to investigate apoptosis and cell viability [30–32]. Functional extensions of OCT can improve the specificity towards identifying different cell types or cell viability, for example dynamic OCT has shown promise as a label-free analogue of cell viability assays [31,33,34]. Furthermore, OCT can be used to detect changes in spheroids in response to applied drugs [17,18,34–36]. Drug-induced changes can include dramatically visible effects including spheroid disintegration, or more subtle effects that cannot be easily visualized by eye. However, most of these studies used only cross-sectional B-scans in their assessments. Our study shows that volumetric data can improve classification and segmentation tasks and further refine the set of key image features and metrics relevant for assessing drug efficacy.
Overall, we see several opportunities that arise from combining OCT imaging with deep learning approaches for assessing drug efficacy. First, fully automated assessment pipelines, such as the deep learning approach presented here, are required for high-throughput screening where human assessment of images would become a speed bottleneck. Additionally, while the OCT images in our study can easily be classified by eye, not all drug effects might be as readily visible. Our study shows not only that deep learning can be used to classify images based on the drug effects, but that it can also help to identify key distinguishing features between the images. Such aspects will be critical for models that are generalized to drugs with different mechanisms of action or for classifying differences between smaller variations in drug concentrations.
4.4. Study strengths and limitations
Our model was deployed using consumer-grade hardware and did not rely on access to specialized computing resources such as supercomputers. This demonstrates the potential for a wider impact of the model. However, we appreciate that the model could be improved by using such specialized computing resources since they can more easily handle large datasets, leading to improved accuracy and faster training times, although such resources can be costly to access and may not be accessible to standard users.
Modeling the different levels of structural change caused by treatment can be used to estimate an appropriate treatment dose to obtain sufficient cancer regression. In our study, the models were trained on images of spheroids treated with 0, 25, and 100 μM cisplatin to identify the most accurate model. The confusion matrices show that both models, when predicting the wrong class, picks the neighbor class, i.e., an image from the control group is never falsely predicted to belong to the 100 μM cisplatin group. Also, the most accurate model, the 3D CNN, seems to be unbiased, i.e., the model is trained on all the voxels in the image rather than relying on a user-defined set of 2D slices. These factors indicate the treatment causes regression patterns, the quantity of which changes linearly with the treatment dose, which is promising for building a model of structural change for arbitrary treatment dose. However, this will require a large amount of additional image data examples involving more treatment doses. In a future study, we intend to further investigate this dose-dependent response to confirm whether the model we have developed could be used to select the optimal concentration of a drug as part of the drug screening process.
In our study, the spheroids were embedded in Histogel before imaging with OCT. This significantly improved the signal-to-noise ratio in the images, particularly by reducing the reflection from the bottom of the well plate, due to the fact that Histogel has a higher refractive index than the cell culture media. However, it also presented some challenges for the analysis. Notably, for the spheroids treated with 100 μM of cisplatin, most of the cell debris was removed during the process of embedding the spheroids in Histogel. This could indicate that some information was lost during the transfer processes and may have implications for application of the model in future experiments if Histogel is not used. Additionally, since the histogel itself scattered some light, the background noise level in the images was increased, indicating some tradeoffs with the signal improvement from using histogel.
We note that, while 540 spheroids we imaged, only 467 images were included in our dataset due to the need to remove some images where the spheroid was damaged or improporly located in the well. These ‘outlier’ images were removed from the analysis since their inclusion significantly reduced the performance of the classifier. Given the impact on the performance from the outlier removal, it is critical to have clear, objective exclusion criteria. In the future, it could be an advantage to incorporate a 2D classifier to remove outliers automatically in advance since it is a clearly visible issue with the Histogel transfer protocol.
One limitation of our method is that the model we used was trained on images of the same spheroid type that were treated with the same drug. Given the sensitivity of such classification models to small variations in images, it is unclear how easily we will be able to expand the model to recognize the response to other drugs. This step is critical to use such a model for identifying novel drug candidates. An interesting next step could be to explore how we can connect the key distinguishing features from the classifier to particular mechanisms of action of different drugs. For example, while cisplatin-induced apoptosis caused the spheroids to break apart leaving cell debris that were recognized by our model, other drugs with different mechanisms of action could have different biomarkers of efficacy that appear differently in OCT images. It would be interesting to investigate the connection between specific drug effects or mechanisms of action with their OCT image hallmarks to build a more robust and generalized model. On the other hand, this study helped us to identify the optimal hyperparameters and classification model for this type of classification tasks. Our results show that there is strong promise for the use of 3D ResNet models for drug assessment tasks in the future.
5. Conclusion
Our study investigated the potential of 2D and 3D ResNet models for assessing drug efficacy in 3D cell culture platforms. While both models we tested showed promise, the 3D model performed significantly better at the cost of longer training times and higher computational needs. Nonetheless, these results show the benefits of working in 3D, both in terms of the biological model and the selected imaging modality. Our results further showed that maintaining sufficient special sampling to fully resolve the OCT pattern in 3D was important to ensure the most accurate classification. In the future, we aim to expand our model to work with different cell types and drugs with different mechanisms of action. These are important next steps towards the development of a model that can be used for identifying promising new drug candidates.
Acknowledgments
We would like to acknowledge the excellent technical assistance of Susanne Schmidt from Bioneer A/S in preparing all the 3D spheroids for OCT-imaging.
References
- 1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71:209–49.
- 2. Soerjomataram I, Bray F. Planning for tomorrow: Global cancer incidence and the role of prevention 2020-2070. Nat Rev Clin Oncol. 2021;18(10):663–72. pmid:34079102
- 3. Singh D, Dhiman VK, Pandey M, Dhiman VK, Sharma A, Pandey H, et al. Personalized medicine: An alternative for cancer treatment. Cancer Treat Res Commun. 2024;42:100860. pmid:39827574
- 4. Sertkaya A, Beleche T, Jessup A, Sommers BD. Costs of drug development and research and development intensity in the US, 2000-2018. JAMA Netw Open. 2024;7:e2415445.
- 5. Stewart DJ, Stewart AA, Wheatley-Price P, Batist G, Kantarjian HM, Schiller J, et al. The importance of greater speed in drug development for advanced malignancies. Cancer Med. 2018;7(5):1824–36. pmid:29601671
- 6. Chatzinikolaidou M. Cell spheroids: The new frontiers in in vitro models for cancer drug validation. Drug Discov Today. 2016;21(9):1553–60. pmid:27371506
- 7. Cukierman E, Pankov R, Yamada KM. Cell interactions with three-dimensional matrices. Curr Opin Cell Biol. 2002;14(5):633–9. pmid:12231360
- 8. Nielsen BS, Madsen NH, Larsen J, Skandorff I, Gad M, Holmstrøm K. Architectural organization and molecular profiling of 3D cancer heterospheroids and their application in drug testing. Front Oncol. 2024;14:1386097. pmid:39011470
- 9. Piccinini F, Tesei A, Bevilacqua A. Single-image based methods used for non-invasive volume estimation of cancer spheroids: A practical assessing approach based on entry-level equipment. Comput Methods Programs Biomed. 2016;135:51–60. pmid:27586479
- 10. Gaskell H, Sharma P, Colley HE, Murdoch C, Williams DP, Webb SD. Characterization of a functional C3A liver spheroid model. Toxicol Res (Camb). 2016;5(4):1053–65. pmid:27746894
- 11. Jeppesen M, Hagel G, Glenthoj A, Vainer B, Ibsen P, Harling H, et al. Short-term spheroid culture of primary colorectal cancer cells as an in vitro model for personalizing cancer medicine. PLoS One. 2017;12(9):e0183074. pmid:28877221
- 12. Daetwyler S, Fiolka RP. Light-sheets and smart microscopy, an exciting future is dawning. Commun Biol. 2023;6(1):502. pmid:37161000
- 13. Huang D, Swanson EA, Lin CP, Schuman JS, Stinson WG, Chang W, et al. Optical coherence tomography. Science. 1991;254(5035):1178–81. pmid:1957169
- 14. Drexler W, Liu M, Kumar A, Kamali T, Unterhuber A, Leitgeb RA. Optical coherence tomography today: Speed, contrast, and multimodality. J Biomed Opt. 2014;19(7):071412. pmid:25079820
- 15. Seesan T, Abd El-Sadek I, Mukherjee P, Zhu L, Oikawa K, Miyazawa A, et al. Deep convolutional neural network-based scatterer density and resolution estimators in optical coherence tomography. Biomed Opt Express. 2021;13(1):168–83. pmid:35154862
- 16. Wang B, Ganjee R, Khandaker I, Flohr K, He Y, Li G, et al. Deep learning based characterization of human organoids using optical coherence tomography. Biomed Opt Express. 2024;15(5):3112–27. pmid:38855657
- 17. Yan F, Mutembei B, Valerio T, Gunay G, Ha J-H, Zhang Q, et al. Optical coherence tomography for multicellular tumor spheroid category recognition and drug screening classification via multi-spatial-superficial-parameter and machine learning. Biomed Opt Express. 2024;15(4):2014–47. pmid:38633082
- 18. Zhang L, Wang L, Yang S, He K, Bao D, Xu M. Quantifying the drug response of patient-derived organoid clusters by aggregated morphological indicators with multi-parameters based on optical coherence tomography. Biomed Opt Express. 2023;14(4):1703–17. pmid:37078050
- 19. Seeland M, Mäder P. Multi-view classification with convolutional neural networks. PLoS One. 2021;16(1):e0245230. pmid:33434208
- 20.
Hara K, Kataoka H, Satoh Y. Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2017. 3154–60. https://doi.org/10.1109/iccvw.2017.373
- 21.
He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 770–8. https://doi.org/10.1109/cvpr.2016.90
- 22.
Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 2018. 839–47. https://doi.org/10.1109/wacv.2018.00097
- 23. Gotkowski K, Gonzalez C, Bucher A, Mukhopadhyay A. M3d-CAM: A PyTorch library to generate 3D data attention maps for medical deep learning. Informatik aktuell. 2020;:217–22.
- 24.
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In: Advances in Neural Information Processing Systems, 2019. 8024–35.
- 25. Cong S, Zhou Y. A review of convolutional neural network architectures and their optimizations. Artificial Intelligence Review. 2023;56:1905–69.
- 26.
Taylor L, Nitschke G. Improving Deep Learning with Generic Data Augmentation. In: 2018 IEEE Symposium Series on Computational Intelligence (SSCI), 2018. 1542–7. https://doi.org/10.1109/ssci.2018.8628742
- 27. Gil DA, Deming DA, Skala MC. Volumetric growth tracking of patient-derived cancer organoids using optical coherence tomography. Biomed Opt Express. 2021;12(7):3789–805. pmid:34457380
- 28. Yan F, Gunay G, Valerio TI, Wang C, Wilson JA, Haddad MS, et al. Characterization and quantification of necrotic tissues and morphology in multicellular ovarian cancer tumor spheroids using optical coherence tomography. Biomed Opt Express. 2021;12(6):3352–71. pmid:34221665
- 29. Huang Y, Wang S, Guo Q, Kessel S, Rubinoff I, Chan LL-Y, et al. Optical coherence tomography detects necrotic regions and volumetrically quantifies multicellular tumor spheroids. Cancer Res. 2017;77(21):6011–20. pmid:28904062
- 30. Jung Y, Klein OJ, Wang H, Evans CL. Longitudinal, label-free, quantitative tracking of cell death and viability in a 3D tumor model with OCT. Sci Rep. 2016;6:27017. pmid:27248849
- 31. Tan KH, Ang JLY, Yong ASK, Lim SZE, Kng JSJ, Liang K. Non-destructive viability assessment of cancer cell spheroids using dynamic optical coherence tomography with trypan blue validation. Biomed Opt Express. 2024;15(11):6370–83. pmid:39553864
- 32. Chiang C-C, Anne R, Chawla P, Shaw RM, He S, Rock EC, et al. Deep learning unlocks label-free viability assessment of cancer spheroids in microfluidics. Lab Chip. 2024;24(12):3169–82. pmid:38804084
- 33. El-Sadek IA, Miyazawa A, Shen LT-W, Makita S, Mukherjee P, Lichtenegger A, et al. Three-dimensional dynamics optical coherence tomography for tumor spheroid evaluation. Biomed Opt Express. 2021;12(11):6844–63. pmid:34858684
- 34. Abd El-Sadek I, Shen LT-W, Mori T, Makita S, Mukherjee P, Lichtenegger A, et al. Label-free drug response evaluation of human derived tumor spheroids using three-dimensional dynamic optical coherence tomography. Sci Rep. 2023;13(1):15377. pmid:37717067
- 35. Abd El-Sadek I, Miyazawa A, Tzu-Wei Shen L, Makita S, Fukuda S, Yamashita T, et al. Optical coherence tomography-based tissue dynamics imaging for longitudinal and drug response evaluation of tumor spheroids. Biomed Opt Express. 2020;11(11):6231–48. pmid:33282486
- 36. Yan F, Ha J-H, Yan Y, Ton SB, Wang C, Mutembei B, et al. Optical coherence tomography of tumor spheroids identifies candidates for drug repurposing in ovarian cancer. IEEE Trans Biomed Eng. 2023;70(6):1891–901. pmid:37015385