celldeath: A tool for detection of cell death in transmitted light microscopy images by deep learning-based visual recognition

Cell death experiments are routinely done in many labs around the world, these experiments are the backbone of many assays for drug development. Cell death detection is usually performed in many ways, and requires time and reagents. However, cell death is preceded by slight morphological changes in cell shape and texture. In this paper, we trained a neural network to classify cells undergoing cell death. We found that the network was able to highly predict cell death after one hour of exposure to camptothecin. Moreover, this prediction largely outperforms human ability. Finally, we provide a simple python tool that can broadly be used to detect cell death.


Introduction
Cell death is a complex event found in normal and pathological contexts (1 ).For this reason, it is widely studied in biomedical research and it is a hallmark of many experiments, particularly in the context of drug discovery (2 , 3 ).Many different assays have been developed in the past decades in order to analyze cell death.All of them involve the analysis of particular features of a dying cell, including DNA fragmentation, cell membrane protein flipping, protein modifications, etc (4 -6 ).In any case, there is need for time and money in order to perform these assays.However, morphological changes are easily seen when a cell is dying, and this is clearly evident for the trained human eye.Cell detachment from the basal membrane, pycnosis, or cell shrinkage, for example, are hallmarks frequently observed in cell culture.
In the past few years there has been an increasing interest in artificial intelligence.The combination of newer algorithms and increasing computational capacities have sparked an overwhelming amount of research.In particular, deep learning (DL) algorithms have proved to be powerful.These algorithms are based on neural networks (NN), which adjust themselves by a method called backpropagation (7 ).This combination allows to classify complex and huge information, including digital images.Therefore, one of the most active field is image recognition (8 , 9 ).We have recently published that NN can be used to classify transmitted light microscopy (TLM) images (10 ).We were able to correctly classify pluripotent stem cell differentiation at one hour or even less, with an accuracy higher than 99%.Of note, changes in morphology were subtle.Hence, we demonstrated that applying DL over TLM images can be an incredible powerful technology for specific purposes: we can identify a complex process in a very short time, with nearly no money spent and with high precision.Identifying differentiation otherwise would require the use of an assay often involving time and money in several orders of magnitude.We are confident that our experience and that of many others will radically change the way fields in biology are approached (11 , 12 ).
In the present work we aimed to develop an easy and fast way to accurately classify cell death in culture by using simple TLM images.We expand our previous work by analyzing cell death instead of cell differentiation, and by developing a simple script to be used in any scientific lab running cell death experiments.

Cell culture and cell death induction
The four cancer cell lines and the three pluripotent stem cells used in this analysis were kept in a humidified air-filtered atmosphere at 37ºC and 5% CO Nonspecific secondary antibody binding was evaluated in the absence of primary antibodies.Images from four fields of three independent replicates were processed and analyzed automatically using custom macro scripts (ImageJ software) to determine mean fluorescent intensity per nucleus and statistical significance between CPT-treated and vehicle-treated cell populations was evaluated by Welch Two Sample t-test using R. , incubated with reagents provided in the kit and finally ran on BD Accuri Flow Cytometer.Results from three independent replicates were analyzed using FlowJo (v7.6) software and statistical significance between CPT-treated and vehicle-treated cell populations was evaluated by Welch Two Sample t-test using R. ).E) Flow cytometry analysis with AnnexinV-PE of iPS1 cells treated with CPT 1µM (light blue) for 3h compared to vehicle (red).Incubation with 7-AAD was performed to discriminate dead cells (Q2) from early apoptotic (Q3).Number of events (cells) in each quadrant is presented as mean percentage of total population ± SEM of three independent replicates.Statistical significance between conditions in Q3 was evaluated with Welch Two-Sample t-test (*p-value=2.5e−2 ).F) MCF7 cancer cells treated with CPT 10µM (light blue) for 6h were analyzed as in E, though using AnnexinV-FITC instead of PE.

Transmitted light imaging
wad 1920x1440 pixels originally, though we applied a short python script (image-slicer) to slice them into four parts in order to obtain four images from each one.

Deep learning analyzes
For deep learning training and prediction, we used fast.ai(v1.0.60), a frontend of PyTorch (v1.4).Briefly, training was done by using several different convolutional neural networks.ResNet50 architecture ( 14), however, was chosen among different options because it rendered excellent results and it is widely known.For analyzes, images were split in four, as previously explained, and hence image shape was (3,360,480).A total of 15226 images were used.We randomly split them into 60% for training, 20% for validation, and 20% for testing.
We also applied a pretrained model (imagenet), but found similar results either way.A python script with details is available in GitHub.

Cell Death Induction
We develop a cell death model in all cell lines used in this paper -three pluripotent stem cell (PSC) lines and four cancer cell (CC) lines-by incubating them with camptothecin (CPT), a topoisomerase I inhibitor.We have previously demonstrated that this molecule induces a very rapid cell death signaling in human embryonic stem cells that derives in apoptosis (15 ).In each of the seven cell lines we titrated drug concentration and exposure time and took TLM images hourly in both vehicle (DMSO) and CPT-treated cells.
To confirm that these cell lines were undergoing apoptosis we performed different assays.Inhibition of topoisomerase I results in replicationdependent DNA double strand breaks (DBSs) (16 ), which lead to the phosphorylation of H2AX (γH2AX) and activation of tumor suppressor protein p53 (17 , 18 ).Consistently, iPS1 pluripotent stem cells treated with CPT 1µM for 1.5h showed an increment in nuclear signal of γH2AX as well as accumulation of p53 (Figure 1A).Compared to vehicle (DMSO), the distributions of nuclear signals were significantly different for both marks (Figure 1B).We observed similar results in H9 embryonic stem cells and in iPS2 induced pluripotent stem cells.
Significant CPT-dependent activation and nu-clear localization of γH2AX and p53 (vs.vehicle) were also found in MCF7 cancer cell line at 3h of treatment (Figure 1C and D).All CC lines showed similar results between 3 and 6h of treatment with CPT.Interestingly, although CC lines generally evince high proliferation rates, they were practically unaffected by 1µM treatment with CPT and only a concentration of 10µM was sufficient to induce the apoptogenic signaling (data not shown).
Longer treatments with CPT resulted in a steady γH2AX and p53 nuclear signal in iPS1 and MCF7 cells compared to vehicle (Figure S1A and B, respectively), indicating that CPT treatment effectively triggers a sustained response to damaged DNA in both PSC and CC lines.
Apoptosis is a complex process and one of its most earlier characteristic features is phosphatidylserine (PS) exposure on the outer side of the cell membrane (19 ).Identification of PS residues on the surface of intact cells through its interaction with Annexin V protein enables detection of early stages of apoptosis by flow cytometry analysis.Treatment with CPT between 3 and 6h significantly increased the percentage of PS + /7-AAD − cells (Q3) compared to vehicle in both iPS1 and MCF7 cells (Figure 1E and F, respectively).Positive values for each quadrant were determined using single stained and double stained untreated samples (Figure S1C and D).
Taken together, these results indicate that CPT treatment induced damage to DNA which eventually resulted in cell death by apoptosis in PSC and CC lines.

Deep learning findings
Transmitted light microscopy images from cell death induction with CPT were taken at 1, 2 and 3 hour post-induction of cell death with CPT.As it can be seen in Figure 2, minor morphological changes, if any, are already observed in all cell lines at one hour.Induced and profound observation is needed to observe minimal changes in some cell lines.For example, in pluripotent stem cell lines some degree of cell-to-cell detachment is seen.A similar finding can be seen in T47D cells.In PC3 cells, some increase in cell volume is observed.After the first hour, morphological changes are more pronounced.
Considering these minor morphological changes, we challenged 5 experienced researchers (who  never saw the images before) to assess if they could correctly classify CPT exposure in a random set of 50 images (pre-training).We then allow them to train by looking at 500 labeled images.Finally, we ask them to classify another set of 50 images (post-training).Images were randomly taken from all cell lines and both control (DMSO) and cell death (CPT).As expected due to the minimal morphological changes seen, classification perfor-mance by investigators was completely random, both before and after training (Figure 3A).Moreover, there was no pattern of classification, since in some images all investigators agree or disagree and in some others they randomly disagree between them (Figure S2).
We used images at one hour to train a visual CNN for classification.We present results based on ResNet50 NN architecture, but others showed similar results.First, we trained the NN to classify each cell line in each condition (all versus all).Classification was excellent, with a final accuracy of 98.8 ±0.5 in the validation set, and 0.987 in the test set (mean of five trainings)(Table 1).A confusion matrix shows a very low number of misclassification, mostly corresponding to the MCF7 cells (Figure 3C).We then grouped all cell lines and trained the neural network.In this case, the final goal is to train a model were, irrespective of cell basal morphology, the CNN was able to identify cell death.Classification was again very high in this context, reaching a final accuracy of 99.4% when we compare all non-exposed cell images versus all-exposed cell exposure (Figure 3 and Table 1).Again, the confusion matrix showed a very low number of misclassification.Finally, to further investigate the ability of the CNN to identify cell death in each cell line, we ran an analysis where each group was treated independently (Table 1), and again classification was in general excellent.
Finally, we performed several trainings where one cell line was set aside, and training was performed.Then, we used the non-trained cell line and predicted cell death on it.We found that the CNN was not able to correctly classify those cell images which previously were not seen (data not shown).Hence, we conclude that those features found useful for classification in each cell line do not translate to an unseen cell line.

Discussion
In this paper we showed that convolutional neural networks can be trained to recognize very early features of cell death.We trained the NN with images taken just after one hour of starting cell death induction.At this early point in time, the human eye was unable to identify cell morphology changes and then to correctly classify a set of images.As usual with neural networks, it is not possible to identify which image features are selected in order to make an accurate classification.Most probably, NN are able to identify subtle changes in cell membrane, cytoplasmic vesicles and/or change in the nuclear morphology proper of the ongoing cell death process.
A few findings on this paper deserve attention.First, we found that these algorithms can reach a very high accuracy for detection of morphological changes in a LTM image.In everyday laboratory practice, this may be a significant advantage for running experiments.It is possible to think about experiments where the output is read through a technology that is easy, cheap and fast.In particular, the use of these technologies in automation may change the way readings are performed.We have already previously shown that this may be the case using a stem cell differentiation model (10 ).Hence, deep learning algorithms may substitute everyday assays in some circumstances.For example, in those situations where a specific assay is expected to be repetitive, DL analysis of LTM images may be practical, cheap and time-saving.Moreover, the addition of automation like live imaging may increase applications.
A second finding is that DL analysis significantly outperformed humans, at least when detecting early features.We conducted a simple test where several trained investigators from our institution assessed a set of images and attempt to classify them if they were undergoing cell death or not.We allowed them to train after, and retested again.The investigators were unable to properly identify the very early changes of cell death.In fact, their results were random.Moreover, they agreed in many images, but differ in some other in a random way.However, low performance of the investigators may be related to the fact that any regular cell culture may have some degree of cell death, and in fact our experiments show that some cells in the control group do translocate annexin V (Figure 1E).Hence, we believe that the investigators were not only unable to identify subtle changes corresponding to the incubation with CPT but also the extent of the undergoing cell death.Finally, our findings may well relate to other publication were deep learning was able to predict protein expression findings from non-labeled protein targets (20 , 21 ).
Deep learning tecnniques are being increasingly used in the biomedical field (12 , 22 ).Specifically for morphological changes in LTM images, we (10 ) and others (21 , 23 -25 ) have previously applied deep learning for different experimental approaches using LTM.For example, Ounkomol et al provided evidence that a DL model can predict immunofluorescence in LTM cells (21 ).Jimenez-Carretero et al predicted fluorescent toxicity looking at changes in stained cell nuclei (25 ).In a similar paper than ours, Richmond et al applied a CNNs in LTM in order to predict phototoxic- Confusion matrix: CPT v. DMSO ity, but their accuracy was approximately 94.5%, probably related to a shallow network they used.Moreover, it took them 16h of training to reach this level, whereas our model gets 99% accuracy in approximately 3 hours using a similar hardware.Finally, they did not provide any easy way to reproduce and apply their findings.
Beside the proof of concept regarding the ability of NN for cell death detection, we also provide a script for an easy application of this technology.With minimal knowledge about deep learning, any user can type a few lines in the command line to get similar results than ours with their images.
In conclusion, we found that deep learning can be applied for cell death recognition in simple transmitted light microscopy images.We provide an easy tool to be used in any lab working on cell death.

Figure 1 :
Figure 1: Camptothecin treatment induced apoptosis in both iPS1 pluripotent stem cell and MCF7 cancer cell lines.A) Immunostaining with anti-γH2AX and anti-p53 of iPS1 pluripotent cell line treated (CPT 1µM) or not (vehile: DMSO) with CPT for 1.5h.Both marks were merged with DAPI to reveal cell nuclei and scale was set to 200um (white bar).Images are representative of four different microscopic fields.B) Distribution of mean signal intensity per nucleus in all fields from A, measured in arbitrary units (log10 a.u.) for γH2AX (left) and p53 (right) marks.Statistical significance between CPT and vehicle was evaluated by Welch Two-Sample t-test (*p-value=2.2e−16 ).C) Immunostaining as in A for MCF7 cancer cell line treated (CPT 10µM) or not (vehicle) with CPT for 6h.D) Mean signal intensity quantification and statistical significance were determined as in B ( # p-value=4.89e−7 ; *p-value=2.22e−16).E) Flow cytometry analysis with AnnexinV-PE of iPS1 cells treated with CPT 1µM (light blue) for 3h compared to vehicle (red).Incubation with 7-AAD was performed to discriminate dead cells (Q2) from early apoptotic (Q3).Number of events (cells) in each quadrant is presented as mean percentage of total population ± SEM of three independent replicates.Statistical significance between conditions in Q3 was evaluated with Welch Two-Sample t-test (*p-value=2.5e−2 ).F) MCF7 cancer cells treated with CPT 10µM (light blue) for 6h were analyzed as in E, though using AnnexinV-FITC instead of PE.

Figure 2 :
Figure 2: Transmitted light images used for visual deep learning analysis.Representative images of vehicle (DMSO)-and CPT-treated cell lines for 1, 2 and 3h.

Figure 3 :
Figure 3: Results of CNN training.A) Comparison of human performance versus CNN.Human pre (48.8 ±3.03) and post-training (46.4 ±3.57) is shown, compared to results of CNN training for a validation (98.73 ±0.5) and a test set (98.77 ±0.18).Human accuracy was random, whereas CNN showed a consistent high accuracy.B) CNN training for 50 epochs.Final convergence is shown.C) Confusion matrix for all-versus-all analysis.Overall, and excellent performance is reached, and only MCF7 cell line showed some degree of inaccuracy.D) Confusion matrix of CPT versus DMSO.A very high accuracy was leading to a very low false positive (14) and false negative (9).

Figure S1 : 5 Figure S2 :
Figure S1: Effect of longer CPT exposure times on γH2AX and p53 staining and flow cytometry controls.A) iPS1 cells were treated or not (vehicle) with CPT 1uM for 3 and 5h.Cells were stained with anti-γH2AX or anti-p53 and nuclei were revealed with DAPI.Scale was set to 200um (white bar).B) MCF7 cells were treated or not (vehicle) with CPT 10uM for 8h.Cells were stained as in A. C) Controls used for setting background levels in iPS1 flow cytometry experiments.D) Controls used for setting background levels in MCF7 flow cytometry experiments.

Table 1 :
Training performance of the different conditions.Highest achieved value of several trainings are presented.