FalseColor-Python: A rapid intensity-leveling and digital-staining package for fluorescence-based slide-free digital pathology

Robert Serafin; Weisi Xie; Adam K. Glaser; Jonathan T. C. Liu

doi:10.1371/journal.pone.0233198

Abstract

Slide-free digital pathology techniques, including nondestructive 3D microscopy, are gaining interest as alternatives to traditional slide-based histology. In order to facilitate clinical adoption of these fluorescence-based techniques, software methods have been developed to convert grayscale fluorescence images into color images that mimic the appearance of standard absorptive chromogens such as hematoxylin and eosin (H&E). However, these false-coloring algorithms often require manual and iterative adjustment of parameters, with results that can be inconsistent in the presence of intensity nonuniformities within an image and/or between specimens (intra- and inter-specimen variability). Here, we present an open-source (Python-based) rapid intensity-leveling and digital-staining package that is specifically designed to render two-channel fluorescence images (i.e. a fluorescent analog of H&E) to the traditional H&E color space for 2D and 3D microscopy datasets. However, this method can be easily tailored for other false-coloring needs. Our package offers (1) automated and uniform false coloring in spite of uneven staining within a large thick specimen, (2) consistent color-space representations that are robust to variations in staining and imaging conditions between different specimens, and (3) GPU-accelerated data processing to allow these methods to scale to large datasets. We demonstrate this platform by generating H&E-like images from cleared tissues that are fluorescently imaged in 3D with open-top light-sheet (OTLS) microscopy, and quantitatively characterizing the results in comparison to traditional slide-based H&E histology.

Citation: Serafin R, Xie W, Glaser AK, Liu JTC (2020) FalseColor-Python: A rapid intensity-leveling and digital-staining package for fluorescence-based slide-free digital pathology. PLoS ONE 15(10): e0233198. https://doi.org/10.1371/journal.pone.0233198

Editor: Jerry Chun-Wei Lin, Western Norway University of Applied Sciences, NORWAY

Received: April 28, 2020; Accepted: September 16, 2020; Published: October 1, 2020

Copyright: © 2020 Serafin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The authors have provided a link to their GitHub repository in the paper: https://github.com/serrob23/falsecolor.

Funding: We acknowledge research grants from the Department of Defense (DoD) Prostate Cancer Research Program through W81XWH-18-10358 (JTCL and LDT); the National Science Foundation through 1934292 HDR: I-DIRSE-FW (JTCL and RS); and the NIH / NCI through K99CA240681 (AKG) and R01CA175391 (JTCL). The content is solely the responsibility of the authors and does not represent the official views of the National Institutes of Health, the U.S. Department of Defense, the National Science Foundation, or the United States Government.

Competing interests: A.K.G. and J.T.C.L. are co-founders and shareholders of Lightspeed Microscopy Inc. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Introduction

Modern microscopy methods enable life scientists and clinicians to visualize complex tissue structures, where recent technological advancements have radically enhanced our understanding of biological processes and disease pathologies. However, clinical diagnostic practices have not taken full advantage of these modern microscopy techniques. In particular, the gold-standard diagnostic method of histology is based on centuries-old technologies, where tissues are preserved in harsh fixatives, destructively sectioned onto glass slides, stained with simple chromogens (most-commonly with hematoxylin and eosin, i.e. H&E), and manually imaged with analog brightfield microscopes. In order to improve throughput, non-destructiveness, sampling extent, and in some cases, to provide 3D information, several slide-free microscopy techniques have recently been explored for use in clinical settings. For example, techniques such as confocal microscopy [1, 2], multiphoton microscopy [3–6], microscopy with UV surface excitation (MUSE) [7–9], and structured illumination microscopy (SIM) [10, 11] have been explored as slide-free alternatives to frozen-section histology for rapid interoperative guidance. Additionally, light-sheet microscopy technologies, such as open-top light-sheet (OTLS) microscopy [12–14], when used in conjunction with tissue-clearing techniques [15], have been explored for slide-free nondestructive 3D pathology. These techniques generally rely on fluorescence collected with a sensitive monochrome detector. Since pathologists are accustomed to certain color schemes generated by standard chromogens, such as the pink and purple hues associated with H&E staining, the ability to render grayscale fluorescent images with color palettes that mimic standard histology can play a major role on the ability of pathologists to interpret and adopt slide-free pathology methods in the future.

Several groups have published software methods to convert two-channel fluorescence images into “virtual H&E” images. One such virtual H&E algorithm was published in 2009 by Gareau [16]. Using an additive model, this method rendered the reflectance contrast generated by collagen and cytoplasm, and the fluorescence generated by a hematoxylin analog, to mimic H&E staining. Bini et al. in 2011 further refined this model by analyzing the transmitted spectra of multiple slides that were independently stained with either hematoxylin or eosin [17]. A limitation of the additive approach is that in standard H&E histology, the classic pink and purple hues are the result of spectral mixing of the two dyes present in the specimen, according to a nonlinear absorption process (Beer-Lambert-law attenuation). Additive models rely on the linear superposition of intensities, which is non-physical and does not reliably mimic the appearance of conventional H&E histology.

To address these limitations, a false-coloring model based on the Beer-Lambert law of absorption was developed by Giacomelli et al. in 2016 [18]. The Beer-Lambert model accounts for the spectral mixing of chromogenic (absorption-based) dyes and their wavelength-dependent nonlinear attenuation of light (i.e. exponential decay as a function of concentration). As a result, this model can accommodate multiple fluorescent stains with overlapping spectra without loss of contrast or non-physical results. Note that in addition to H&E, other chromogens can be modeled, such as the DAB stain most-commonly used for immunohistochemistry (IHC) [13].

The previous approaches described above, for false coloring fluorescent images to mimic conventional chromogenic (absorption-based) stains, have a number of practical shortcomings that prevent them from generating consistent results. In particular, three main points motivate the work described in this article:

Significant variations in the appearance of slide-free digital pathology images can occur due to uneven staining within a specimen (intra-specimen variability). In particular, thick unsectioned tissues exhibit diffusion barriers that often lead to nonuniform staining. This is more pronounced with large agents such as antibodies but also affects smaller agents. Additionally, when imaging optically cleared tissues, samples that are not fully cleared can exhibit a scattering-induced loss of signal as a function of imaging depth.
The appearance of both standard histology and slide-free fluorescence images can vary greatly between institutions, or even within an institution, due to day-to-day variations in sample preparation, staining protocols and imaging parameters (inter-specimen variability) [19]. In the presence of these variabilities, simple false-coloring algorithms can fail unless software parameters are manually tweaked, often by trial-and-error.
Finally, 3D slide-free digital pathology datasets are often hundreds of gigabytes to terabytes in size, which is 3 to 4 orders of magnitude larger than standard 2D whole slide images. To facilitate clinical translation, false-coloring algorithms must be able to process these large 3D datasets in an efficient and scalable manner.

It bears mentioning that several deep-learning methods have been developed to render H&E-like images from unstained tissue sections imaged with label-free imaging modalities such as brightfield and autofluorescence microscopy [20–22], multi-photon microscopy [23], and quantitative phase-contrast imaging [24]. Similar machine-learning-based approaches could be used to create H&E-like images from tissues labeled with exogenous fluorophores (i.e. fluorescent analogs of H&E). However, these methods not only require large amounts of training data but are also highly sensitive to pre-analytical variations such as staining protocols, imaging parameters, and hardware settings. In contrast, the ability to use an explainable physics-based approach for digital staining is likely attractive for many end users and regulatory agencies, allowing for easier error identification, debugging, and compatibility with different imaging platforms [25].

In this manuscript we present FalseColor-Python, which enables rapid and robust digital staining of fluorescence images to mimic the appearance of chromogenic stains using the Beer-Lambert model. As a pre-processing step prior to false-coloring, an intensity-leveling routine, analogous to flat fielding, is used to locally and globally normalize image intensities, thereby mitigating the effects of both intra- and inter-sample variability. In particular, the code presented here is specifically optimized to generate H&E-like images of tissues stained with a two-channel fluorescent analog of H&E (i.e. a fluorescent nuclear stain that mimics hematoxylin, along with the stromal stain, eosin, which is naturally fluorescent) and imaged with 3D open-top light-sheet (OTLS) microscopy [12–14]. We show that FalseColor-Python enables accurate and reproducible H&E false-coloring that qualitatively and quantitatively matches the appearance of H&E whole slide images, but with less variability than is seen with standard histology. GPU acceleration is used for efficient and scalable processing of large 3D datasets.

Note that while H&E false-coloring is demonstrated here, FalseColor-Python can be tailored to accommodate other preferred fluorescent staining combinations and individual color preferences. For example, FalseColor-Python can be adapted to false-color fluorescence images to mimic other chromogens, such as DAB (used in standard immunohistochemistry [13]) or other special stains (e.g. PAS, Masson’s Trichrome, Toluidine Blue). In addition, while FalseColor-Python is optimized for large 3D datasets, it can easily be applied to 2D images as well.

FalseColor-Python workflow

Fig 1 outlines the workflow for processing large two-channel 3D datasets using FalseColor-Python. For our specific implementation of this code, input data is stored on disk in the Hierarchical Data Format (HDF5), which is a common multi-resolution 3D image format (Fig 1A). The advantage of multi-resolution file formats like HDF5, or similar formats such as N5, is that they contain multiple down-sampled versions of the imaging data. For our 3D microscopy data, we use a 16x down-sampled version of the data (low-resolution) to rapidly calculate an approximate global background level for each channel. The background level is calculated by first ignoring all pixels that are below a threshold value (i.e. black pixels), which we have defined as 4 standard deviations above the detector noise floor. The intensity of the 20^th percentile of the remaining pixels is defined as the background value, which is due to a combination of tissue autofluorescence and nonspecific staining. We have found that for all of the 3D datasets we have examined, this 20% threshold is effective. However, for tissues that are very densely stained, in which there are few “background” pixels, this threshold may need to be adjusted. Once calculated, this background level is uniformly subtracted from the datasets, whether down-sampled as in Fig 1B or full-resolution as on the left side (numerator) of Fig 1F. Next, the down-sampled dataset is partitioned into uniformly sized data cubes. In our case, each data cubes corresponds to a 100 x 100 x 100 μm³ volume of tissue (where 100 μm corresponds to 256 pixels in our raw datasets) (Fig 1C). We use this partitioned data to create a low-resolution 3D “intensity-leveling map,” which will subsequently be used to remove global intensity fluctuations from the original dataset (see Introduction). The 3D intensity-leveling map is generated by calculating the median pixel value for every local data cube (Fig 1D). Given the size of the full-resolution 3D image data, the subsequent steps are sequentially performed on individual 2D image planes (i.e. different depths) within the full-resolution 3D dataset. Each full-resolution 2D intensity-leveling map (Fig 1E) is generated from the down-sampled 3D leveling map (Fig 1D) through linear interpolation. By using linear interpolation, abrupt intensity differences between adjacent data cubes are smoothed out, preventing grid artifacts in the final leveling map. All of the full-resolution 2D intensity-leveling maps are scaled by an empirical weighting factor for each channel, α, which is held constant for the entire 3D dataset. This weighting factor is used to control the relative intensity of each fluorescence channel, which can be tuned by users to generate a desired appearance for the final false-colored RGB images (see below). Intensity leveling is achieved by taking a pixel-by-pixel ratio of each full-resolution image plane (background-subtracted) and its corresponding interpolated intensity-leveling map (Fig 1F). The result is an image in which the median value of every local region (i.e. data cube) is approximately equal (due to the ratioing step), in which many of the large-scale (gradual) intensity irregularities are reduced (Fig 1G). Finally, the leveled images from both fluorescence channels are passed into the Beer-Lambert false-coloring algorithm, which generates a color (RGB) image (Fig 1H). The intensity-leveling and virtual H&E routines are accelerated with a GPU using the CUDA framework from Python’s Numba library [26]. A diagram illustrating the full workflow is shown in the methods section.

Download:

Fig 1. FalseColor-Python workflow for virtual H&E rendering of a two-channel fluorescent analog of H&E.

The following operations are performed on both channels, but only one is shown for simplicity. (A) 3D data (I) is loaded from disk. (B) A down-sampled version of the dataset (16x down-sampled here) is extracted, and a background level is calculated for each channel, which is uniformly subtracted from both the down-sampled and full-resolution original datasets. (C) The down-sampled data is further subdivided into cubes. (D) A preliminary down-sampled 3D leveling map is generated by calculating the median pixel value for each data cube. (E) A full-resolution 2D leveling map at each depth, αM, is then generated by linear interpolation. Here, α, is an empirically determined constant for each channel, applied to the entire 3D dataset, which allows users to tune the appearance of the final false-colored RGB image. (F) To achieve intensity leveling, each 2D plane (i.e. different depth) from the full-resolution data, I, is divided by its corresponding leveling map, αM. This evens out coarse non-uniformities within the image. (G) The leveled images, I_f for both channels are input into the Beer-Lambert model to generate color (RGB) virtual H&E images, (H).

https://doi.org/10.1371/journal.pone.0233198.g001

Note that the size of the data cubes (Fig 1C) that are used for generating an intensity-leveling map should be chosen based on the spatial scale of the intensity nonuniformities that one wishes to correct. For images that exhibit very gradual or minimal intensity variations, larger cube sizes can be used. Small cube sizes will remove finer-scale intensity fluctuations but may also remove fluctuations/contrast that are due to real tissue structure rather than staining artifacts.

To demonstrate the utility of this method, a comparison of virtual H&E staining with and without intensity-leveling is shown in Fig 2. Here a single plane from a 3D microscopy dataset of an optically cleared lung specimen is shown. The specimen was stained with a fluorescent nuclear stain TOPRO-3 Iodide (Cat: T3605, Thermo-Fisher) and was optically cleared with Ethyl-Cinnamate (Cat: 112372, Sigma-Aldrich) [14]. The cleared tissue volume was then imaged using OTLS microscopy [27]. The Beer-Lambert false-coloring algorithm was applied to a single grayscale image without intensity leveling (Fig 2A). In this example, the exterior of the tissue was stained more heavily than the interior. In Fig 2B, the intensity-leveling procedure was incorporated into the false-coloring routine, such that the false-colored nuclei appear much more uniform across the entire image.

Download:

Fig 2. False-coloring with and without intensity-leveling.

Only one channel is shown for simplicity. (A) Images are shown of a thick tissue specimen that is labeled (nonuniformly) with a fluorescent nuclear stain (TO-PRO-3), optically cleared, and then imaged with 3D microscopy. The image is false colored to mimic hematoxylin without intensity-leveling applied. A line profile within the inset clearly shows that nuclei in the inner region of the tissue (yellow arrow) are dimmer than nuclei on the exterior (blue arrow). When false colored, the nuclei on the exterior appear much darker (bottom panel). (B) The same field of view is shown after intensity leveling is applied. Nuclei across the field of view exhibit similar intensities, and when false colored, are much more uniform in appearance. Scale bars: 100 μm.

https://doi.org/10.1371/journal.pone.0233198.g002

Colorimetry to mimic standard histology

For the clinical adoption of slide-free digital pathology, virtual H&E algorithms must consistently render virtual H&E images that qualitatively and quantitatively match the coloration of standard histology. To ensure that FalseColor-Python’s virtual H&E algorithm mimics the color palettes of standard histology, we measured the color properties of 65 publicly available whole slide images (H&E) of prostate adenocarcinoma from the Cancer Digital Slide Archive [28] (Fig 3A). 10 regions of interest (ROIs) were selected from each whole slide image at a magnification of 200x (20x objective), (Fig 3B). Hematoxylin and eosin stains were segmented from each region of interest using color deconvolution [29, 30]. The color deconvolution algorithm generated a probability map for each dye, where pixel values represent the probability of that pixel belonging to one of the top-two color components of the image (i.e. hematoxylin or eosin), (Fig 3C & 3D). These probability maps were used to create binary masks for each structure using Otsu’s thresholding method [31]. The binary masks yielded segmented images of the nuclei (hematoxylin stain) and stroma (eosin stain). This process was repeated for all ROIs and the median color properties of each stain were quantified and plotted based on the hue, saturation, value (HSV) color model (Fig 3E & 3F). We used these measured HSV values as a target for our virtual H&E images. In other words, in FalseColor-Python, we have adjusted the color parameters used in the Beer-Lambert false-coloring algorithm so that the median HSV color properties of the virtual H&E images approximate that of standard whole slide images.

Download:

Fig 3. Measuring the color properties of histology images.

(A) A whole slide image of prostate adenocarcinoma from the Cancer Digital Slide Archive, cancer.digitalslidearchive.org. Scale bar: 2 mm (B) Magnified inset of (A). Scale bar: 75 μm (C, D) Process to segment the hematoxylin and eosin components. Using color deconvolution, a probability map is generated for the top two components of each ROI from the whole slide image (right panel). A binary mask is created for each of the two components by applying Otsu’s thresholding to the probability map (middle panel). This yields a segmented image of the nuclei (hematoxylin stain) and cytoplasm (eosin stain) (left panel). Scale bar: 75 μm (E, F) After segmentation of all ROIs, the median values of hue, saturation, and value (HSV color model) are quantified and plotted for both the hematoxylin-stained and eosin-stained tissue components.

https://doi.org/10.1371/journal.pone.0233198.g003

For the clinical adoption of slide-free pathology, virtual H&E images not only need to mimic the color palette of standard H&E but must also be consistent and robust in the presence of variations between samples, for example due to differences in staining protocols or imaging parameters. By incorporating intensity-leveling, our virtual H&E algorithm rendered images with similar appearance despite inter-sample intensity differences. As an example, fluorescent images of two prostate tissue samples are shown in Fig 4A, in which significant differences in fluorescence intensities are seen (when imaged with the same device and settings). The intensity levels for each fluorescence channel (TO-PRO-3 and eosin) are shown in the accompanying histograms. Despite these differences in intensity, the resulting virtual H&E images of each sample are qualitatively and quantitatively similar. On the right-most column of Fig 4A, a histogram of the value (V) component of each virtual H&E image is plotted and is comparable for both images after the intensity-leveled false-coloring routine is applied.

Download:

Fig 4. Inter-specimen consistency of virtual H&E images.

(A) OTLS images of prostate tissue with different staining intensities. Despite significant differences in intensity between example I and II, the virtual H&E images appear qualitatively and quantitatively similar. Histograms are shown of the fluorescence intensities within each image, and the value (V) component of the virtual H&E images. (B) Distribution of median color properties for 650 standard H&E and 2100 virtual H&E images of prostate tissue. Whiskers on the box plots indicate the 10^th and 90^th percentiles for each distribution. The distributions for standard H&E exhibit the high degree of color variation seen in conventional histology. The virtual H&E images rendered with FalseColor-Python accurately mimic the coloration of standard H&E with significantly less variation in color parameters than conventional H&E. Standard deviations for each distribution are listed below each plot.

https://doi.org/10.1371/journal.pone.0233198.g004

To quantitatively compare the consistency of FalseColor-Python with standard H&E histology, we measured the HSV color model parameters for 2100 virtual H&E images from 14 samples of prostate tissue imaged with 3D OTLS microscopy and then processed with FalseColor-Python. To measure the color properties of the virtual H&E images, we used the same color-deconvolution method that was used to measure the color properties of whole slide images from the Cancer Digital Slide Archive (described previously). It should be noted that while the color parameters were adjusted to allow our virtual H&E images to match the color parameters of standard H&E images, once these values were determined, all virtual H&E images used in this analysis were processed in a fully automated fashion using the exact same parameters for intensity-leveling and coloration. Our results show that the color properties of virtual H&E data processed with FalseColor-Python match the measured values of standard H&E, as expected (Fig 4B). Furthermore, the color properties of virtual H&E images processed with FalseColor-Python are much more consistent (less standard deviation) than the color properties of standard H&E-stained whole slide images. Table 1 lists the median and standard deviation of each color property for both standard and virtual H&E. Finally, an image atlas is shown in Fig 5 to demonstrate that FalseColor-Python renders consistent virtual H&E images across tissue types and to demonstrate that FalseColor-Python is easily adjusted to mimic other color spaces in standard histology (DAB staining).

Download:

Fig 5. False-coloring image atlas.

Thick tissues were stained with a fluorescent analog of H&E (A, B, D, E), or an antibody targeting the high molecular weight keratin (HMWK), CK-8, along with the nuclear stain, TO-PRO-3 (C, F). All tissues were optically cleared and imaged in 3D with an open-top light-sheet (OTLS) microscope. (A) Skin. (B) Kidney. (C) Prostate. (D) Basal layer of the epidermis. (E) Kidney tubules. (F) Prostate glands (carcinoma). Virtual H&E images were processed with identical code parameters in a fully automated fashion. Virtual IHC images were processed with an identical code, but with coloring parameters changed to mimic the chromogen DAB (brown stain) and hematoxylin (blue stain). These codes and parameters are provided in our GitHub repository (see provided link). Scale bars: 500 μm for A-C, 50 μm for D-F.

https://doi.org/10.1371/journal.pone.0233198.g005

Download:

Table 1. Measured median color properties of standard and virtual H&E images in HSV color space.

Standard H&E properties are measured from 10 ROIs from each of 65 whole slide images (prostate). Virtual H&E properties are measured from 2100 two-dimensional “optical sections” from 14 prostate specimens.

https://doi.org/10.1371/journal.pone.0233198.t001

Discussion

We have developed FalseColor-Python, a rapid intensity-leveling and digital-staining package for converting grayscale fluorescence images into color images that mimic conventional chromogenic (absorption-based) stains. In particular, we demonstrate the rendering of virtual H&E images from thick tissues stained with a fluorescent analog of H&E and imaged in 3D. To improve the consistency of our false-coloring method in the presence of both inter-sample and intra-sample variations in staining/intensity, we have developed and incorporated a 3D intensity-leveling routine (Figs 1 & 2). We analyzed the color properties of standard H&E images (Fig 3) and used this data to ensure that the virtual H&E images rendered by FalseColor-Python are representative of standard histology. Our results show that the virtual H&E images rendered by FalseColor-Python are qualitatively and quantitatively similar to standard H&E histology regardless of variations in intensity, as for example due to differences in sample preparation, imaging device, and/or imaging parameters (Fig 4). In particular, we have shown that FalseColor-Python renders virtual H&E images that not only quantitatively match the appearance of standard H&E images, but with less variability in coloration than is seen with standard histology (Fig 4B).

The ability to render quantitatively and qualitatively consistent virtual H&E images is of critical importance for the adoption of fluorescence-based imaging methods in anatomic pathology. Staining non-uniformities and depth-dependent intensity variations are common data-quality issues in fluorescence imaging, particularly for nondestructive slide-free 3D imaging modalities. In developing this intensity-leveling technique, we have taken advantage of the multiple down-sampled versions of an imaging dataset that are stored in a multi-resolution image format such as HDF5 (Fig 1A). However, down-sampling of imaging data is straightforward and intensity-leveling maps can be generated from any 3D imaging data. For our 3D microscopy data, optimal performance has been seen with data cubes of 100 x 100 x 100 μm³ however, users can make adjustments as needed to best suit their data. This tunability is demonstrated in an example located on our GitHub repository. Finally, for certain images with intensity nonuniformities, enhanced performance is seen using an implementation of contrast-limited adaptive histogram equalization (CLAHE) as a pre-processing step before intensity-leveling is performed. This is included as an optional method within FalseColor-Python but was not used in the examples shown in this study. More specifically, CLAHE is included as an optional method for use in cases where intensity changes within the sample are large over small distances such that intensity leveling alone may be insufficient to yield qualitatively acceptable results. For example, thick tissues stained with large agents that have difficulty diffusing into tissue interiors may see improved results by applying CLAHE before false coloring to enhance relative signal strength between high- and low-signal regions. However, CLAHE should be used judiciously since it can also increase noise in the images.

In terms of limitations, our intensity-leveling method is best suited for images where staining is present throughout the entirety of the sample, but where the spatial variations in that signal are gradual compared with the high-resolution features of interest (e.g. Fig 2) In cases in which large-scale (gradual) intensity variations are biologically real and informative to end users (e.g. pathologists), there is the possibility that our intensity-leveling methods will mask (i.e. flatten) such global-intensity variations. However, this could be mitigated by tuning the size of the data-partitioning cubes as outlined in Fig 1.

A significant outcome of this study is that virtual H&E images rendered by FalseColor-Python exhibit significantly less variability in color parameters than standard histology. This is not surprising considering the obvious differences in appearance seen in histology images generated by independent labs, and even within individual labs at different times. The measured median color-property values of our virtual H&E images were all well within one standard deviation of the color properties of standard H&E images.

We recognize that the standard H&E data used in this analysis represents a subset of all possible color presentations found in histology. Further, we acknowledge that there is a high degree of subjectivity and personal preference regarding the optimal coloration for standard H&E images. Therefore, it is possible to tune the appearance of false-colored images if so desired. For simplicity, RGB color parameters, which are easily measured from any image, are included as arguments in FalseColor-Python’s Beer-Lambert false-coloring algorithm so that users can make adjustments as needed. The leveling constant, α, is used to control the relative intensity of each channel, which affects the appearance of the final virtual H&E image. For our OTLS datasets, we found that α values of 1.5 and 3.7 were appropriate leveling constants for the hematoxylin and eosin channels, respectively. These particular values allowed the color properties of our virtual H&E data to consistently match the measured values derived from standard H&E images (Fig 3). Lower values of α result in a darker/stronger staining appearance. Further discussion of adjusting color settings is provided in an example on the online repository (see GitHub link at the end of this article).

Note that all methods used for colorimetry analysis of standard and virtual H&E images (e.g. Fig 3) are included in FalseColor-Python. As mentioned in the introduction, FalseColor-Python is easily adapted for other fluorescence-to-chromogenic staining transformations, for example to render images that mimic chromogenic immunohistochemistry (i.e. DAB stain) (Fig 5C & 5F).

FalseColor-Python contains several methods that use GPU acceleration. For example, CUDA-based implementations of the Beer-Lambert false-coloring algorithm are included in FalseColor-Python as well as several preprocessing steps such as intensity-leveling, image sharpening, and background subtraction. For users without access to GPU hardware, equivalent CPU-based processing methods are available as alternatives in FalseColor-Python. In a simple speed comparison, we measured the average time to process 200 two-channel 16-bit images (2048 x 2048 pixels each) with FalseColor-Python, using either GPU- or CPU-based Beer-Lambert false-coloring algorithms and preprocessing steps (i.e background subtraction, intensity-leveling). The GPU-based process was faster by over a factor of 6 (116 +/- 17 ms per image vs. 789 +/- 18 ms per image). There is room for further improvement by developing objects and methods within FalseColor-Python that can process large 3D datasets in parallel. For example, implementing multi-resolution file formats that allow for parallelized read operations (e.g. N5 instead of HDF5) will enable FalseColor-Python to process 3D datasets even more efficiently. Parallelized processing in FalseColor-Python is discussed in an example on the GitHub repository.

In summary, FalseColor-Python renders virtual H&E images that qualitatively and quantitatively match the coloration of standard H&E images, as well as other chromogenic stains. A key component of FalseColor-Python is the incorporation of an intensity-leveling method that corrects for intensity non-uniformities commonly seen in 3D fluorescent datasets, both within one sample and between different samples (i.e. intra- and inter-specimen variability). The color settings in FalseColor-Python are customizable to suit user preferences, with interpretable parameters that facilitate error identification and troubleshooting. FalseColor-Python is equipped with GPU-accelerated methods that allow it to process large 3D datasets efficiently. Developers who are interested in contributing to FalseColor-Python should submit pull requests via the GitHub repository.

Detailed methods

OTLS imaging

Tissue samples were stained with a fluorescent nuclear stain, TO-PRO-3 (Cat: T3605, Thermo-Fisher) at a 1:2000 dilution and eosin (Cat: 3801615, Leica Biosystems) at a 1:2000 dilution for 4 hours at room temperature with light shaking. Samples were optically cleared with ethyl-cinnamate (Cat: 112372, Sigma-Aldrich). Stained and cleared samples were imaged on a custom OTLS system [14]. A 660-nm laser was used to excite the nuclear dye, TO-PRO3, and a 561-nm laser was used to excite eosin. Each channel was imaged separately, in succession, with a 16-bit sCMOS camera. For more information on OTLS imaging see our previous publications [12–14].

Image processing

Two channel OTLS datasets were stored on disk in the HDF5 format with metadata in an XML file structured for analysis using BigStitcher [27]. A custom compression filter (B3D) was used to provide 10x compression. The fine alignment of all OTLS data was performed in BigStitcher and fused to disk as a separate HDF5 file. Before virtual H&E rendering (as described in the manuscript), an optional sharpening routine was used on each image to enhance edges. The full workflow is outlined in the diagram of Fig 6.

Download:

Fig 6. Diagrammatic flow.

Processes occur for each channel consecutively. Operations that act on 3D image data are highlighted in green and 2D operations are highlighted in blue. Input arguments such as file path, image planes to process (identified with HDF5 z-index), and desired intensity-leveling constants are parsed for processing. Low-resolution data is read into memory and the initial intensity-leveling map is generated. Background levels are computed from the low-resolution data. Full-resolution images are read into memory for processing. Full-resolution data undergoes background subtraction and image sharpening as preprocessing steps. The full-sized intensity-leveling map is generated via linear interpolation. Intensity leveling is applied to each image. The intensity-leveled images undergo the Beer-Lambert false-coloring process, and the resulting RGB image is saved to disk. Writing to disk occurs in a separate thread for efficiency, indicated by the dashed box.

https://doi.org/10.1371/journal.pone.0233198.g006

Intensity-leveling maps (Fig 1D) are generated by calculating the median intensity value within a specified 3D image volume (data cube). For an input image I, with specified background threshold b, the pixel value M for the intensity leveling map is given by: (1) (2)

Here i, j, k indicates the index. The pixel value of the intensity leveling map is set to the median value of the data cube above the background threshold, Eq (1). If the median value within the data cube falls below the background threshold, the level is set to a constant value, A, which is equal to the 95^th percentile of I by default, Eq (2). This ensures that low-signal background regions appear black in the intensity-leveled image (Fig 1G) and that they appear white in the false-colored (e.g. H&E mimicking) RGB image (Fig 1H).

Virtual H&E rendering of two-channel fluorescence images was achieved using the Beer-Lambert false-coloring algorithm [18]. Grayscale intensities are converted into RGB images via: (3) where H, E are fluorescent images, M_H,E is the 2D image leveling map for each channel, α is the intensity-leveling constant for each channel, j_n, k_n are the color settings for hematoxylin and eosin respectively and n = R, G, B. Based on the colorimetry measurements described in the manuscript (Fig 3), we adjusted FalseColor-Python’s hematoxylin and eosin color parameters until the measured color properties of virtual H&E images consistently matched those of standard H&E. After this calibration no further adjustments were necessary. Table 2 lists the RGB color settings used for processing OTLS images. For leveling our OTLS images, α_H = 1.5 and α_E = 3.7 were chosen as scaling constants for the majority of our datasets.

Download:

Table 2. Reference j_n, k_n values for virtual H&E coloration in RGB space.

https://doi.org/10.1371/journal.pone.0233198.t002

GPU acceleration for image processing was done using the cuda.jit decorator from the Numba library [26]. Implementation of GPU acceleration is done in such a way that the user needs no experience with the CUDA framework to accelerate their code (see examples on the GitHub repository). To achieve GPU acceleration, users only require a virtual environment equipped with the Anaconda’s cudatoolkit [32] and a CUDA capable GPU. GPU processing was done with a Nvidia Quadro P4000. A script containing the workflow as described in Fig 1 can be found on the GitHub repository.

Histology data collection

To quantify the HSV color space of traditional histopathology, 65 whole slide images of prostate adenocarcinoma biopsies from the Cancer Genome Atlas, cancer.digitalslidearchive.org [28] were examined, 10 equally sized fields of view from each whole slide image were taken at 20x magnification.

Color-Space quantification

To accurately analyze the color space of histology and OTLS images, the hematoxylin and eosin channels were separated from one another via a binary mask generated by color deconvolution from the scikit-image.color python package [30]. This color deconvolution-based segmentation was used for both standard and virtual H&E images. A binary mask was generated for each structure by first applying a median filter to the result of the color deconvolution, and then applying Otsu’s thresholding method [31]. Small objects were removed from the initial mask using an area threshold. A binary opening with a structuring element of a disk, r = 3, was applied to the resulting image. The input RGB image was converted to HSV space and then each mask was applied to generate the final segmented image for each structure. This process was repeated across 650 regions of interest taken from publicly available prostate adenocarcinoma whole slide images and 2100 OTLS virtual H&E images from 14 prostate samples. Once each structure was segmented, the median value for each color property (HSV) was recorded from each segmented image. Zero valued areas, which resulted from the application of the binary mask, were ignored.

Code availability

The full FalseColor-python code, example data, and colorimetry analysis methods, are publicly available at https://github.com/serrob23/falsecolor. The repository includes instructions for installation and annotated examples.

Programming language

Python 3.6+ is used in FalseColor-Python. Some features may be unavailable on older versions. A full list of dependencies and requirements is available on GitHub.

Acknowledgments

The authors would like to thank Drs. Suzanne Dintzis, Nicholas P. Reder, and Lawrence D. True for assistance regarding histology coloration as well as Lindsey Barner and Kevin Bishop for helpful technical discussions.

References

1. Longo C, Ragazzi M, Rajadhyaksha M, Nehal K, Bennassar A, Pellacani G, et al. In Vivo and Ex Vivo Confocal Microscopy for Dermatologic and Mohs Surgeons. Dermatol Clin. 2016;34(4):497–504. pmid:27692455
- View Article
- PubMed/NCBI
- Google Scholar
2. van Royen ME, Verhoef EI, Kweldam CF, van Cappellen WA, Kremers GJ, Houtsmuller AB, et al. Three dimensional microscopic analysis of clinical prostate specimens. Histopathology. 2016;69(6):985–92. pmid:27353346
- View Article
- PubMed/NCBI
- Google Scholar
3. Cahill LC, Fujimoto JG, Giacomelli MG, Yoshitake T, Wu Y, Lin DI, et al. Comparing histologic evaluation of prostate tissue using nonlinear microscopy and paraffin H&E: a pilot study. Mod Pathol. 2019 Jul 1;32(8):1158–67. pmid:30914763
- View Article
- PubMed/NCBI
- Google Scholar
4. Tao YK, Shen D, Sheikine Y, Ahsen OO, Wang HH, Schmolze DB, et al. Assessment of breast pathologies using nonlinear microscopy. Proc Natl Acad Sci U S A. 2014;111(43):15304–9. pmid:25313045
- View Article
- PubMed/NCBI
- Google Scholar
5. Sun CK, Kao CT, Wei ML, Chia SH, Kärtner FX, Ivanov A, et al. Slide-free imaging of hematoxylin-eosin stained whole-mount tissues using combined third-harmonic generation and three-photon fluorescence microscopy. J Biophotonics. 2019;(September 2018):1–21. pmid:30636033
- View Article
- PubMed/NCBI
- Google Scholar
6. You S, Tu H, Chaney EJ, Sun Y, Zhao Y, Bower AJ, et al. Intravital imaging by simultaneous label-free autofluorescence-multiharmonic microscopy. Nat Commun [Internet]. 2018;9(1). Available from: http://dx.doi.org/10.1038/s41467-018-04470-8 pmid:29844371
- View Article
- PubMed/NCBI
- Google Scholar
7. Fereidouni F, Harmany ZT, Tian M, Todd A, Kintner JA, McPherson JD, et al. Microscopy with ultraviolet surface excitation for rapid slide-free histology. Nat Biomed Eng. 2017 Dec 1;1(12):957–66. pmid:31015706
- View Article
- PubMed/NCBI
- Google Scholar
8. Xie Weisi, Chen Ye, Wang Yu, Wei Linpeng, Yin Chengbo, Glaser Adam K., et al, “Microscopy with ultraviolet surface excitation for wide-area pathology of breast surgical margins,” J. Biomed. Opt. 24(2), 026501 (2019), pmid:30737911
- View Article
- PubMed/NCBI
- Google Scholar
9. Yoshitake T, Giacomelli MG, Quintana LM, Vardeh H, Cahill LC, Faulkner-Jones BE, et al. Rapid histopathological imaging of skin and breast cancer surgical specimens using immersion microscopy with ultraviolet surface excitation. Sci Rep [Internet]. 2018;8(1):1–12. Available from: http://dx.doi.org/10.1038/s41598-018-22264-2 pmid:29311619
- View Article
- PubMed/NCBI
- Google Scholar
10. Wang M, Tulman DB, Sholl AB, Kimbrell HZ, Mandava SH, Elfer KN, et al. Gigapixel surface imaging of radical prostatectomy specimens for comprehensive detection of cancer-positive surgical margins using structured illumination microscopy. Sci Rep [Internet]. 2016;6(May):1–16. Available from: http://dx.doi.org/10.1038/srep27419 pmid:27257084
- View Article
- PubMed/NCBI
- Google Scholar
11. Wang M, Kimbrell HZ, Sholl AB, Tulman DB, Elfer KN, Schlichenmeyer TC, et al. High-resolution rapid diagnostic imaging of whole prostate biopsies using video-rate fluorescence structured illumination microscopy. Cancer Res. 2015;75(19):4032–41. pmid:26282168
- View Article
- PubMed/NCBI
- Google Scholar
12. Glaser AK, Reder NP, Chen Y, Mccarty EF, Yin C, Wei L, et al. Light-sheet microscopy for slide-free non-destructive pathology of large clinical specimens. Nat Biomed Eng. 2018;1(7):1–22. pmid:29750130
- View Article
- PubMed/NCBI
- Google Scholar
13. Reder NP, Glaser AK, McCarty EF, Chen Y, True LD, Liu JTC. Open-Top Light-Sheet Microscopy Image Atlas of Prostate Core Needle Biopsies. Arch Pathol Lab Med. 2019;143(9):1069–75. pmid:30892067
- View Article
- PubMed/NCBI
- Google Scholar
14. Glaser AK, Reder NP, Chen Y, Yin C, Wei L, Kang S, et al. Multi-immersion open-top light-sheet microscope for high-throughput imaging of cleared tissues. Nat Commun [Internet]. 2019;10(1):1–8. Available from: http://dx.doi.org/10.1038/s41467-019-10534-0 pmid:30602773
- View Article
- PubMed/NCBI
- Google Scholar
15. Rocha MD, Düring DN, Bethge P, Voigt FF, Hildebrand S, Helmchen F, et al. Tissue clearing and light sheet microscopy: Imaging the unsectioned adult zebra finch brain at cellular resolution. Front Neuroanat. 2019;13(February):1–7.
- View Article
- Google Scholar
16. Gareau DS, Li Y, Huang B, Eastman Z, Nehal KS, Rajadhyaksha M. Confocal mosaicing microscopy in Mohs skin excisions: feasibility of rapid surgical pathology. J Biomed Opt. 2008;13(5):054001. pmid:19021381
- View Article
- PubMed/NCBI
- Google Scholar
17. Bini J, Spain J, Nehal K, Hazelwood V, DiMarzio C, Rajadhyaksha M. Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance. J Biomed Opt. 2011;16(7):076008. pmid:21806269
- View Article
- PubMed/NCBI
- Google Scholar
18. Giacomelli MG, Husvogt L, Vardeh H, Faulkner-Jones BE, Hornegger J, Connolly JL, et al. Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging. PLoS One. 2016; pmid:27500636
- View Article
- PubMed/NCBI
- Google Scholar
19. Macenko M, Niethammer M, Marron JS, Borland D, Woosley JT, Guan X, et al. A method for normalizing histology slides for quantitative analysis. Proc—2009 IEEE Int Symp Biomed Imaging From Nano to Macro, ISBI 2009. 2009;1107–10.
20. Rana A, Yauney G, Lowe A, Shah P. Computational Histological Staining and Destaining of Prostate Core Biopsy RGB Images with Generative Adversarial Neural Networks. Proc—17th IEEE Int Conf Mach Learn Appl ICMLA 2018. 2019;828–34.
21. Rivenson Y, Wang H, Wei Z, De Haan K, Zhang Y, Wu Y, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng [Internet]. 2019;3(June). Available from: http://dx.doi.org/10.1038/s41551-019-0362-y pmid:31142829
- View Article
- PubMed/NCBI
- Google Scholar
22. Rizwan A, Bulte C, Kalaichelvan A, Cheng M, Krishnamachary B, Bhujwalla ZM, et al. Metastatic breast cancer cells in lymph nodes increase nodal collagen density. Sci Rep. 2015 May 7;5. pmid:25950608
- View Article
- PubMed/NCBI
- Google Scholar
23. Borhani N, Bower AJ, Boppart SA, Psaltis D. Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy. Biomed Opt Express. 2019;10(3):1339. pmid:30891350
- View Article
- PubMed/NCBI
- Google Scholar
24. Rivenson Y, Liu T, Wei Z, Zhang Y, de Haan K, Ozcan A. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci Appl [Internet]. 2019;8(1). Available from: http://dx.doi.org/10.1038/s41377-019-0129-y pmid:30728961
- View Article
- PubMed/NCBI
- Google Scholar
25. Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell [Internet]. 2019;1(5):206–15. Available from: http://dx.doi.org/10.1038/s42256-019-0048-x
- View Article
- Google Scholar
26. Lam SK, Pitrou A, Seibert S. Numba: A LLVM-based python JIT compiler. Proc Second Work LLVM Compil Infrastruct HPC—LLVM ‘15. 2015;1–6.
27. Hörl D, Rojas Rusak F, Preusser F, Tillberg P, Randel N, Chhetri RK, et al. BigStitcher: reconstructing high-resolution image datasets of cleared and expanded samples. Nat Methods. 2019;16(9):870–4. pmid:31384047
- View Article
- PubMed/NCBI
- Google Scholar
28. Gutman DA, Khalilia M, Lee S, Nalisnik M, Mullen Z, Beezley J, et al. The digital slide archive: A software platform for management, integration, and analysis of histology for cancer research. Cancer Res. 2017;77(21):e75–8. pmid:29092945
- View Article
- PubMed/NCBI
- Google Scholar
29. Ruifrok A. C. & Johnston D. A. Quantification of histochemical staining by color deconvolution. Anal. Quant. Cytol. Histol. 23, 291–299 (2001). pmid:11531144
- View Article
- PubMed/NCBI
- Google Scholar
30. Van Der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, et al. Scikit-image: Image processing in python. PeerJ. 2014;2014(1):1–18.
- View Article
- Google Scholar
31. Otsu N. THRESHOLD SELECTION METHOD FROM GRAY-LEVEL HISTOGRAMS. IEEE Trans Syst Man Cybern. 1979;
32. Anaconda Software Distribution. cudatoolkit version: 10.2.89 anaconda.org/anaconda/cudatoolkit

[ref1] 1. Longo C, Ragazzi M, Rajadhyaksha M, Nehal K, Bennassar A, Pellacani G, et al. In Vivo and Ex Vivo Confocal Microscopy for Dermatologic and Mohs Surgeons. Dermatol Clin. 2016;34(4):497–504. pmid:27692455
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. van Royen ME, Verhoef EI, Kweldam CF, van Cappellen WA, Kremers GJ, Houtsmuller AB, et al. Three dimensional microscopic analysis of clinical prostate specimens. Histopathology. 2016;69(6):985–92. pmid:27353346
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Cahill LC, Fujimoto JG, Giacomelli MG, Yoshitake T, Wu Y, Lin DI, et al. Comparing histologic evaluation of prostate tissue using nonlinear microscopy and paraffin H&E: a pilot study. Mod Pathol. 2019 Jul 1;32(8):1158–67. pmid:30914763
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Tao YK, Shen D, Sheikine Y, Ahsen OO, Wang HH, Schmolze DB, et al. Assessment of breast pathologies using nonlinear microscopy. Proc Natl Acad Sci U S A. 2014;111(43):15304–9. pmid:25313045
View Article
PubMed/NCBI
Google Scholar

[14] View Article

[15] PubMed/NCBI

[16] Google Scholar

[ref5] 5. Sun CK, Kao CT, Wei ML, Chia SH, Kärtner FX, Ivanov A, et al. Slide-free imaging of hematoxylin-eosin stained whole-mount tissues using combined third-harmonic generation and three-photon fluorescence microscopy. J Biophotonics. 2019;(September 2018):1–21. pmid:30636033
View Article
PubMed/NCBI
Google Scholar

[18] View Article

[19] PubMed/NCBI

[20] Google Scholar

[ref6] 6. You S, Tu H, Chaney EJ, Sun Y, Zhao Y, Bower AJ, et al. Intravital imaging by simultaneous label-free autofluorescence-multiharmonic microscopy. Nat Commun [Internet]. 2018;9(1). Available from: http://dx.doi.org/10.1038/s41467-018-04470-8 pmid:29844371
View Article
PubMed/NCBI
Google Scholar

[22] View Article

[23] PubMed/NCBI

[24] Google Scholar

[ref7] 7. Fereidouni F, Harmany ZT, Tian M, Todd A, Kintner JA, McPherson JD, et al. Microscopy with ultraviolet surface excitation for rapid slide-free histology. Nat Biomed Eng. 2017 Dec 1;1(12):957–66. pmid:31015706
View Article
PubMed/NCBI
Google Scholar

[26] View Article

[27] PubMed/NCBI

[28] Google Scholar

[ref8] 8. Xie Weisi, Chen Ye, Wang Yu, Wei Linpeng, Yin Chengbo, Glaser Adam K., et al, “Microscopy with ultraviolet surface excitation for wide-area pathology of breast surgical margins,” J. Biomed. Opt. 24(2), 026501 (2019), pmid:30737911
View Article
PubMed/NCBI
Google Scholar

[30] View Article

[31] PubMed/NCBI

[32] Google Scholar

[ref9] 9. Yoshitake T, Giacomelli MG, Quintana LM, Vardeh H, Cahill LC, Faulkner-Jones BE, et al. Rapid histopathological imaging of skin and breast cancer surgical specimens using immersion microscopy with ultraviolet surface excitation. Sci Rep [Internet]. 2018;8(1):1–12. Available from: http://dx.doi.org/10.1038/s41598-018-22264-2 pmid:29311619
View Article
PubMed/NCBI
Google Scholar

[34] View Article

[35] PubMed/NCBI

[36] Google Scholar

[ref10] 10. Wang M, Tulman DB, Sholl AB, Kimbrell HZ, Mandava SH, Elfer KN, et al. Gigapixel surface imaging of radical prostatectomy specimens for comprehensive detection of cancer-positive surgical margins using structured illumination microscopy. Sci Rep [Internet]. 2016;6(May):1–16. Available from: http://dx.doi.org/10.1038/srep27419 pmid:27257084
View Article
PubMed/NCBI
Google Scholar

[38] View Article

[39] PubMed/NCBI

[40] Google Scholar

[ref11] 11. Wang M, Kimbrell HZ, Sholl AB, Tulman DB, Elfer KN, Schlichenmeyer TC, et al. High-resolution rapid diagnostic imaging of whole prostate biopsies using video-rate fluorescence structured illumination microscopy. Cancer Res. 2015;75(19):4032–41. pmid:26282168
View Article
PubMed/NCBI
Google Scholar

[42] View Article

[43] PubMed/NCBI

[44] Google Scholar

[ref12] 12. Glaser AK, Reder NP, Chen Y, Mccarty EF, Yin C, Wei L, et al. Light-sheet microscopy for slide-free non-destructive pathology of large clinical specimens. Nat Biomed Eng. 2018;1(7):1–22. pmid:29750130
View Article
PubMed/NCBI
Google Scholar

[46] View Article

[47] PubMed/NCBI

[48] Google Scholar

[ref13] 13. Reder NP, Glaser AK, McCarty EF, Chen Y, True LD, Liu JTC. Open-Top Light-Sheet Microscopy Image Atlas of Prostate Core Needle Biopsies. Arch Pathol Lab Med. 2019;143(9):1069–75. pmid:30892067
View Article
PubMed/NCBI
Google Scholar

[50] View Article

[51] PubMed/NCBI

[52] Google Scholar

[ref14] 14. Glaser AK, Reder NP, Chen Y, Yin C, Wei L, Kang S, et al. Multi-immersion open-top light-sheet microscope for high-throughput imaging of cleared tissues. Nat Commun [Internet]. 2019;10(1):1–8. Available from: http://dx.doi.org/10.1038/s41467-019-10534-0 pmid:30602773
View Article
PubMed/NCBI
Google Scholar

[54] View Article

[55] PubMed/NCBI

[56] Google Scholar

[ref15] 15. Rocha MD, Düring DN, Bethge P, Voigt FF, Hildebrand S, Helmchen F, et al. Tissue clearing and light sheet microscopy: Imaging the unsectioned adult zebra finch brain at cellular resolution. Front Neuroanat. 2019;13(February):1–7.
View Article
Google Scholar

[58] View Article

[59] Google Scholar

[ref16] 16. Gareau DS, Li Y, Huang B, Eastman Z, Nehal KS, Rajadhyaksha M. Confocal mosaicing microscopy in Mohs skin excisions: feasibility of rapid surgical pathology. J Biomed Opt. 2008;13(5):054001. pmid:19021381
View Article
PubMed/NCBI
Google Scholar

[61] View Article

[62] PubMed/NCBI

[63] Google Scholar

[ref17] 17. Bini J, Spain J, Nehal K, Hazelwood V, DiMarzio C, Rajadhyaksha M. Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance. J Biomed Opt. 2011;16(7):076008. pmid:21806269
View Article
PubMed/NCBI
Google Scholar

[65] View Article

[66] PubMed/NCBI

[67] Google Scholar

[ref18] 18. Giacomelli MG, Husvogt L, Vardeh H, Faulkner-Jones BE, Hornegger J, Connolly JL, et al. Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging. PLoS One. 2016; pmid:27500636
View Article
PubMed/NCBI
Google Scholar

[69] View Article

[70] PubMed/NCBI

[71] Google Scholar

[ref19] 19. Macenko M, Niethammer M, Marron JS, Borland D, Woosley JT, Guan X, et al. A method for normalizing histology slides for quantitative analysis. Proc—2009 IEEE Int Symp Biomed Imaging From Nano to Macro, ISBI 2009. 2009;1107–10.

[ref20] 20. Rana A, Yauney G, Lowe A, Shah P. Computational Histological Staining and Destaining of Prostate Core Biopsy RGB Images with Generative Adversarial Neural Networks. Proc—17th IEEE Int Conf Mach Learn Appl ICMLA 2018. 2019;828–34.

[ref21] 21. Rivenson Y, Wang H, Wei Z, De Haan K, Zhang Y, Wu Y, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng [Internet]. 2019;3(June). Available from: http://dx.doi.org/10.1038/s41551-019-0362-y pmid:31142829
View Article
PubMed/NCBI
Google Scholar

[75] View Article

[76] PubMed/NCBI

[77] Google Scholar

[ref22] 22. Rizwan A, Bulte C, Kalaichelvan A, Cheng M, Krishnamachary B, Bhujwalla ZM, et al. Metastatic breast cancer cells in lymph nodes increase nodal collagen density. Sci Rep. 2015 May 7;5. pmid:25950608
View Article
PubMed/NCBI
Google Scholar

[79] View Article

[80] PubMed/NCBI

[81] Google Scholar

[ref23] 23. Borhani N, Bower AJ, Boppart SA, Psaltis D. Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy. Biomed Opt Express. 2019;10(3):1339. pmid:30891350
View Article
PubMed/NCBI
Google Scholar

[83] View Article

[84] PubMed/NCBI

[85] Google Scholar

[ref24] 24. Rivenson Y, Liu T, Wei Z, Zhang Y, de Haan K, Ozcan A. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci Appl [Internet]. 2019;8(1). Available from: http://dx.doi.org/10.1038/s41377-019-0129-y pmid:30728961
View Article
PubMed/NCBI
Google Scholar

[87] View Article

[88] PubMed/NCBI

[89] Google Scholar

[ref25] 25. Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell [Internet]. 2019;1(5):206–15. Available from: http://dx.doi.org/10.1038/s42256-019-0048-x
View Article
Google Scholar

[91] View Article

[92] Google Scholar

[ref26] 26. Lam SK, Pitrou A, Seibert S. Numba: A LLVM-based python JIT compiler. Proc Second Work LLVM Compil Infrastruct HPC—LLVM ‘15. 2015;1–6.

[ref27] 27. Hörl D, Rojas Rusak F, Preusser F, Tillberg P, Randel N, Chhetri RK, et al. BigStitcher: reconstructing high-resolution image datasets of cleared and expanded samples. Nat Methods. 2019;16(9):870–4. pmid:31384047
View Article
PubMed/NCBI
Google Scholar

[95] View Article

[96] PubMed/NCBI

[97] Google Scholar

[ref28] 28. Gutman DA, Khalilia M, Lee S, Nalisnik M, Mullen Z, Beezley J, et al. The digital slide archive: A software platform for management, integration, and analysis of histology for cancer research. Cancer Res. 2017;77(21):e75–8. pmid:29092945
View Article
PubMed/NCBI
Google Scholar

[99] View Article

[100] PubMed/NCBI

[101] Google Scholar

[ref29] 29. Ruifrok A. C. & Johnston D. A. Quantification of histochemical staining by color deconvolution. Anal. Quant. Cytol. Histol. 23, 291–299 (2001). pmid:11531144
View Article
PubMed/NCBI
Google Scholar

[103] View Article

[104] PubMed/NCBI

[105] Google Scholar

[ref30] 30. Van Der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, et al. Scikit-image: Image processing in python. PeerJ. 2014;2014(1):1–18.
View Article
Google Scholar

[107] View Article

[108] Google Scholar

[ref31] 31. Otsu N. THRESHOLD SELECTION METHOD FROM GRAY-LEVEL HISTOGRAMS. IEEE Trans Syst Man Cybern. 1979;

[ref32] 32. Anaconda Software Distribution. cudatoolkit version: 10.2.89 anaconda.org/anaconda/cudatoolkit

Figures

Abstract

Introduction

FalseColor-Python workflow

Colorimetry to mimic standard histology

Discussion

Detailed methods

OTLS imaging

Image processing

Histology data collection

Color-Space quantification

Code availability

Programming language

Acknowledgments

References