CELLULAR dataset.

The CELLULAR dataset [11] is a publicly available collection of fluorescence microscopy images of Drosophila melanogaster S2 cells, cultured under nutrient sufficiency (“fed”) and nutrient deprivation (“starved”). To monitor autophagy, cells were genetically engineered, as described in [12], to express the mRFP-EGFP-Atg8a reporter [9], consisting of monomeric red fluorescent protein (mRFP), enhanced green fluorescent protein (EGFP), and Atg8a protein. Under neutral pH conditions, both fluorescent proteins emit light and produce red and green fluorescence, respectively. However, in acidic environments, EGFP is quenched, yielding only red fluorescence. This differentiation allows the identification of autophagosomes (exhibit both green and red fluorescence, appearing yellow) and autolysosomes (red fluorescence only). Images were captured hourly over a four-hour period, producing sequences of five images per sample to document the progression of autophagy. Fig 2 shows an example sequence, highlighting the transition from basal to activated autophagy states in starved cells, contrasted with stable basal autophagy states in fed cells.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Time-lapse imaging from the CELLULAR dataset.

The top panel shows progressive changes in a cell population over five time points. The middle panel (Fed Cells Sequence) depicts an individual fed cell remaining in a basal autophagy state throughout the time course. The bottom panel (Starved Cells Sequence) shows a starved cell transitioning from a basal to an activated autophagy state.

https://doi.org/10.1371/journal.pone.0331045.g002

The dataset consists of 18,720 TIF files, with each file corresponding to a channel (green, red, transmitted light) across 6,240 samples. Among these, 53 images have been annotated by human experts. For the annotated images, merged color images at a resolution of 2048x2048 pixels are provided, along with segmentation masks, one mask per cell, bounding boxes, and class labels. Cells are classified according to their autophagy status into three categories: basal autophagy, activated autophagy, and unidentified. For this study, only the 53 annotated color images and their corresponding annotations were used.

The dataset, consisting of 53 images, was divided into training, validation, and test sets in a 35:8:10 split (train:validation:test). The test set includes two complete sequences: one of fed cells and one of starved cells. A file listing the exact filenames used for training, validation, and testing is available in our GitHub repository (data_division.txt).

We analyzed annotated images to understand their content and temporal evolution. Each image contains between 140 and 386 cells, with 44.9% unidentified, 32.8% showing basal autophagy, and 22.8% with activated autophagy. We analyzed transitions between cell classes over the five time points under fed (4 sequences) and starved (3 sequences) conditions. Only complete image sets with annotations for all five time points were included. Under starvation, basal autophagy cells decreased while activated autophagy cells increased. In fed conditions, most cells consistently showed basal autophagy (see S1 Table in Supplementary Material).

Using segmentation masks, we analyzed cell morphology, focusing on basal and activated cells. Cell area and circularity were calculated from pixel counts, revealing that basal autophagy cells are 12.65% larger and 5.97% less circular than activated autophagy cells. A t-test confirmed significant differences in both metrics (see S2 Table in Supplementary Material).

RNA interference (RNAi) dataset.

For the RNAi dataset, 2000 S2 mRFP-EGFP-Atg8a cells in ESF921 medium were cultured with 4000 ng dsRNA against Atg1, mTor or Luciferase and incubated at 25°C for five days. Cells were imaged in a 384-well glass-bottom plate (Cellvis, P384-1.5H-N) pre-coated with 0.5 mg/mL concanavalin A (Sigma, L7647). Live-cell time-lapse imaging was conducted at room temperature using an ImageXpress Micro Confocal High-Content Imaging System (Molecular Devices). Imaging was performed in wide-field acquisition mode, capturing three different visiting points per well with a 40× Plan Fluor ELWD air objective. For each field, images were collected in three channels: green fluorescence (GFP), red fluorescence (RFP), and transmitted light (TL). Live-Cell Imaging Solution (Molecular Probes, A14291DJ) supplemented with 2 mg/mL glucose (Formedium, GLU03) was used to induce starvation and the cells were followed over time.

Western blot.

S2-mRFP-EGFP-Atg8a were either kept fed in Schneider’s Drosophila Medium (GIBCO 21,720,001) supplemented with 10% heat-inactivated FBS (GIBCO, F7524) and 1% penicillin-streptomycin (GIBCO 15,140–122) or subjected starvation by using 1X phosphate-buffered saline (PBS; Gibco 14,190–094) supplemented with 2 mg/mL D(+)glucose (Formedium, GLU03). At the end of the treatment period, cell lysates were prepared using ice cold RIPA Buffer (50 mM Tris-HCl, pH 7.4, 150 mM NaCl, 0.25% deoxycholic acid, 1% NP-40, and 1 mm EDTA; Millipore, 20–188) supplemented with phosphatase (Roche 04,906,837,001) and protease (Roche 05,056,489,001) inhibitors. The cell lysates were centrifuged for 15 min at 14,000×g at 4°C before the supernatant was transferred to a new tube while the pellet was discarded. Supernatant was used for protein quantification using the bicinchoninic acid (BCA) Protein Assay Kit (Thermo Fisher 23,227). Protein extract was mixed with 4X sample buffer (Thermo Fisher, NP0008) and 10X 1 M DTT (Sigma-Aldrich, D0632) to a final concentration of 1X and boiled for 5 min at 95°C. The samples were separated by SDS-PAGE on 4–20% gradient gels (Bio-Rad, 567–1094, 567–1095). Proteins were then transferred onto LF PVDF membrane (Bio-Rad, 161–0374) using the semi-dry Trans-Blot^® Turbo^™ Transfer System (Bio-Rad 1,704,150). The membrane was air dried for 15 min before incubation with the primary antibodies rabbit anti-β-actin/Act5C 1:1000 (Abcam, ab8227) and goat anti-mCherry 1:500 (Acris, AB0040–200) and rotated overnight at 4°C. Next day, the membrane was washed three times for 10 min in Tris-buffered saline (TBS; Bio-Rad 10,026,938) with 0.1% Tween 20 (Sigma-Aldrich, P1379), and then incubated with the horseradish peroxidase (HRP)-conjugated secondary antibodies anti-rabbit 1:5000 (Jackson, 111-035-144) and anti-goat 1:5000 (Jackson, 705-035-147) in TBS-Tween 20 with 5% skim milk powder (Millipore 70,166) for 1 h at room temperature. The antibodies were detected by chemiluminescence, using SuperSignal West DURA Extended Duration Substrate (Thermo Fisher 11,593,440) and captured using ChemiDoc MP system (Bio-Rad).

Flow cytometry.

EGFP and mRFP expression levels were assessed in fed and starved mRFP-EGFP-Atg8a S2 cells by flow cytometry. Prior to analysis, the cells were centrifuged twice at 500 g for 5 minutes before being resuspended in 1 mL IPL-41 media (Sigma, I7760) for fed condition or 1 mL Live-cell imaging solution (Molecular probes, A14291DJ) supplemented with 2 mg/mL glucose (Formedium, GLU03) for starved condition. Hoechst (Hoechst 33258, Thermo Fisher, H3569) was added at a final concentration of 0.008 μg/mL to enable removal of dead cells during analysis. Cell suspensions were subsequently passed through 35 μm filter cap tubes (Falcon, 352235) before analysis on an LSR II flow cytometer (BD). Laser 488 nm with bandpass filter 525/30 nm and longpass Dichroic filter 505 nm was used to detect EGFP, and laser 561 nm with bandpass filter 582/15 nm and longpass Dichroic filter 570 nm was used to detect mRFP. Approximately 20.000 events (cells) were used for the analysis using the software FlowJo v10 (BD). S2 cells with no reporter expression were used as a negative control.

Hardware specifications.

We trained the deep learning models using an NVIDIA GeForce RTX 4080 Super GPU with 16GB of VRAM and an i9-13900K processor and 128GB of RAM. The code was written in Python 3.12 and the models were implemented using the PyTorch 2.5 framework with CUDA support. For each of the models below, a requirements file specifying the required package versions has been provided in the GitHub repository.

Object detection.

Object detection is the process of identifying and categorizing objects within images or video. In this context, it involves detecting the presence of cells and determining their positions, depicted by rectangular bounding boxes. For the detection task, we used the well-established You Only Look Once (YOLO) model [14]. Given the successful applications of YOLO models in related works [15], the state-of-the-art YOLOv8 model was selected for this study.

Two pre-trained YOLOv8 models of different sizes were utilized: YOLOv8x (extra-large) with 67 million parameters and YOLOv8l (large) with 43 million parameters. Both models were fine-tuned with a batch size of 4, preserving the original resolution and using YOLO’s default augmentations, including flipping, scaling, rotation, and others. The training was carried out until the validation loss stagnated for 50 consecutive epochs, after which the model checkpoint with the best performance was saved.

Segmentation.

Cell segmentation is the process of isolating cells from the background to enable further morphological analysis. This involves classifying each pixel in the image into cell or no-cell classes based on its characteristics. Initially, instance segmentation of entire cell images was prioritized, treating all cells uniformly without autophagic class differentiation.

We selected two models: the Segment Anything Model (SAM) [16], recognized for its remarkable performance and generalizability in instance segmentation tasks [17]; and Cellpose 2.0 [18], a widely used framework specifically designed for cell segmentation. Both were fine-tuned on the CELLULAR dataset. For SAM, we followed a community guide [19] targeting only the decoder and using the pre-trained vit_h model. Images were resized to pixels. The training was conducted over 100 epochs with a batch size of 4, using the Adam optimizer (learning rate: 0.001, weight decay: 0.0005), and combined focal and dice loss [16]. For Cellpose 2.0, we fine-tuned the cyto2torch3 model using existing [11]. This also spanned 100 epochs with a batch size of 4, using the SGD optimizer (learning rate: 0.2, weight decay: 0.00001), and combined mean squared error (MSE) and binary cross-entropy (BCE) loss.

Due to frequent misidentification by these models, we transitioned to integrating segmentation with object detection for better control (see Section Object Detection). Object detection models with adjustable confidence thresholds allow for higher accuracy by isolating and subsequently segmenting detected cells. SAM was selected again for this task due to its superior performance compared to Cellpose 2.0. In addition, the DeepLabV3+ and U-Net++ models were included for comparison. Both utilized a pre-trained ResNet50 encoder and were trained with normalized images, a batch size of 32, dice loss, the Adam optimizer (learning rate: 0.0001), a learning rate scheduler (patience: 10 epochs), and early stopping (patience: 30 epochs).

We used a custom test script, available in our GitHub repository, to ensure consistent model evaluation on the test dataset. For SAM, we assessed both pre-trained and fine-tuned models, while DeepLabV3+ and U-Net++ evaluations were based on versions trained from scratch on the CELLULAR dataset.

Classification.

This study classifies cells according to their autophagy status: basal autophagy, activated autophagy, or unidentified. We employed well-established architectures such as VGG-16, ResNet50, and Vision Transformer (ViT) due to their proven performance in image classification [20–22].

Our strategy involved both training these models from scratch and fine-tuning models pre-trained on large datasets like ImageNet [23]. We also compared the use of cropped images containing a single cell per image with segmented images where cells are isolated against a black background to evaluate the impact on model performance. VGG-16, ResNet50, and ViT models were obtained from the torchvision.models subpackage, adapting their final linear layer for 3-class classification. The images were normalized using the dataset’s mean and standard deviation and resized to 224x224 pixels. Training employed a batch size of 32, cross-entropy loss, the Adam optimizer (learning rate: 0.0001), a learning rate scheduler reducing the learning rate if validation loss did not improve over 10 epochs, and early stopping if no improvement was seen over 30 epochs, with checkpoints saved for the lowest validation loss.

Tracking.

Tracking is essential for monitoring individual cells’ progression across the five frames in our time-lapse dataset. This allows the analysis of cellular responses to nutritional factors like starvation or nutrient presence. However, the CELLULAR dataset lacks tracking annotations, posing a significant challenge.

We explored several object tracking algorithms, focusing on two widely used multiple object tracking methods: SORT [24] and DeepSORT [25], the latter being an extension of SORT. Building on these, we developed a custom algorithm specifically tailored to our dataset. Our method utilized the Hungarian algorithm for data association while omitting Kalman filters due to the limited number of frames. To extract feature vectors, it leveraged a pre-trained ResNet50 encoder. The algorithm distinguished between active (currently tracked) and inactive cells (previously tracked but currently undetected), preserving historical data for potential re-detection and addressing inconsistencies in human annotations. However, due to the absence of ground truth annotations for tracking, we are unable to quantitatively assess the performance of the models.

Explainability.

Deep learning models perform complex calculations, making them difficult to interpret, which raises trust concerns, especially in sensitive fields like healthcare [26]. Explainable AI (XAI) addresses this by developing techniques to enhance model interpretability. One widely used XAI method is Class Activation Mapping (CAM) [27], designed for convolutional neural networks (CNNs) to highlight key input features influencing predictions via heatmaps. However, CAM requires a specific architecture, limiting its applicability. To overcome this, we selected three CAM extensions—GradCAM [28], EigenCAM [29], and AblationCAM [30]—each offering a unique approach to importance calculation. EigenCAM applies principal component analysis (PCA) to feature maps, capturing dominant patterns without relying on a specific class score. GradCAM uses output gradients to identify regions contributing to a prediction, while AblationCAM determines importance by systematically removing feature map components and measuring the impact.

Although initially designed for CNNs, these methods can be adapted for transformers. We modified the [31] repository to suit our ViT model, which achieved the highest classification accuracy. In transformers, selecting the target layer for Grad-CAM is crucial since decisions rely on the class token in the final attention block. As the last layer lacks gradients, we used the preceding one layer_11.ln_1 layer for visualization. Model outputs were reshaped from [batch, 50, 768] to exclude the class token, focusing on the 7x7 image patches with 768 channels.

tSNE.

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a technique for visualizing high-dimensional data by reducing its dimensionality for representation in 2D plots. It achieves this by projecting data from a high-dimensional space to a lower-dimensional one, optimizing the placement of points to align the probability distributions of pairwise similarities. This process preserves both the local and global structure of the data.

We used t-SNE to visualize test dataset cell images cropped, segmented, and tracked by our models. The aim was to uncover potential relationships or patterns within the data. Feature vectors were extracted from the processed cells using a custom encoder UNet specifically trained on this dataset. These feature vectors were then used to generate a t-SNE plot.