The authors have declared that no competing interests exist.
In a wide range of biological studies, it is highly desirable to visualize and analyze three-dimensional (3D) microscopic images. In this primer, we first introduce several major methods for visualizing typical 3D images and related multi-scale, multi-time-point, multi-color data sets. Then, we discuss three key categories of image analysis tasks, namely segmentation, registration, and annotation. We demonstrate how to pipeline these visualization and analysis modules using examples of profiling the single-cell gene-expression of
Multidimensional microscopic image data sets (
(a) A confocal image of kinetochores (EGFP labeled) and chromosomes (histone-mCherry labeled) used in studying the first meiotic division in mouse oocytes
In this primer, we briefly introduce the basic concepts and methods of 3D microscopic image visualization and analysis, which are the two core components for a number of bioimage informatics applications. We emphasize fluorescent microscopic images as examples, and occasionally also mention other types of image data in our discussion. On the other hand, the essential visualization and analysis methods introduced here can be applied to a wide range of data, including many of those not explicitly discussed. Due to the length limitations of this educational note, here we do not intend to comprehensively survey software tools or biological applications, which can be found in a few previous reviews
Visualizing 3D microscopic images helps better understand the data. It also helps determine appropriate analysis methods or parameters. In addition, visualizing analysis results on top of, or side-by-side with, the input image(s) is critical for checking the meaningfulness of an analysis and making necessary corrections (“proof-editing”
Two-dimensional (2D) cross-sectional display (
Visualization mode | Dimension | Often Applied to | Pros | Cons | Example Figure(s) |
|
2D | FM/WM/EM | Fast | Not 3D | — |
|
2D | FM/EM | Fast | Partial 3D | 3a |
|
3D | FM | 3D | Hardware-limited (HL) | 1a, 1b, 1c, 1e, 2b |
|
3D | FM/EM | Surface-display effect | HL | 1d, 2a |
|
4D | FM | 3D | Need color-blending (CB), HL | 1a, 1b, 1c, 1e, 2b |
|
5D | FM | 3D | CB, HL | 2b |
|
6D | FM | Hardware-friendly, 3D | Need 3D interaction of image content | 2a |
|
Heterogeneous 3D/4D/5D/6D | FM/EM | Allow proof-reading/editing | HL | 2a, 4b |
CB, color-blending; EM, electron microscopic images; FM, fluorescent microscopic images (often laser-scanning-microscopic images); HL, hardware-limited; WM, wide-field light microscopic images.
However, cross-sectional views are not able to visualize the 3D information of volumetric images. Visualizing the
3D image visualization calls for depth-blended views from any angle. Maximal (or minimal) intensity projection (MIP or mIP) and alpha-value blended views (
In many cases, each voxel in a 3D microscopic image could have multiple color components that correspond to various features of the biological entities (e.g., different fluorephores with different wavelengths in fluorescent imaging). Visualizing multi-channel (MC) 3D image stacks (thus four-dimension [4D], see
Live imaging experiments produce multi-time-point (MT) multi-color 3D image series (thus five-dimension [5D], see
(a) The hierarchical (multi-scale) 3D visualization of a fluorescent confocal image of fruit fly (
Surface-object rendering (
Interactive visualization techniques are important for microscopic image analysis. Through interactions, users can collect much more information of the multi-dimensional data than passively observing the 3D rendered data. Interacting with 3D rendered surface-objects is straightforward. It is more difficult to directly interact with 3D rendered volumetric data to define interesting 3D locations, 3D curves, and other objects. The concept of 3D-WYSIWYG (what you see is what you get) was recently proposed in the Vaa3D system to define an unambiguous 3D location (point) using one computer mouse click, or define a unique 3D curve using one mouse stroke on the 2D computer screen. This approach has been demonstrated to boost both the reconstruction speed and accuracy of 3D neuron morphology
In practice, 3D visualization of multi-dimensional image data may involve many other considerations. For instance, in both 3D tomographic EM imaging and laser scanning microscopy, anisotropy is an often seen property of the data. Software tools (e.g., Vaa3D) can reslice the data in the 3D rendering based on the relative pixel size in three dimensions, thus providing a more realistic display of the data. In Vaa3D, this auto-slicing function is combined with some image analysis functions (e.g., fibrous structure tracing) discussed below to generate various 3D reconstructions of the image objects. In addition, data filtering techniques (e.g., non-linear anisotropic diffusion, recursive median filtering, bilateral filtering, etc.) have been provided in many software tools (e.g., ImageJ). Integrating all these tools together could lead to more interesting insight in the data (see the last section on “pipelining”).
The overarching goal of microscopic image analysis is to quantitatively measure “objects” in microscopic images, preferably in an automatic manner. Various labeled molecules (e.g., proteins or protein complexes), sub-cellular organelles, cells, or super-cellular objects (e.g., neuron populations or cell lineages) often need to be extracted, named, and compared with each other, before they can be measured. Most microscopic image analysis techniques can be categorized into three major classes, namely
Segmentation is the process of partitioning an image into multiple regions, so that voxels within each region share certain common features. Image segmentation is often used to locate objects and their boundaries (lines, curves, etc., e.g.,
Registration
Annotation is the process to label/name images or image objects (e.g., cells) or assign their phenotypic properties with predefined terms. For example, controlled vocabularies of ontology have been assigned to images for annotating gene expression patterns (e.g.,
In many biological applications, different image analysis techniques need to be used as a whole pipeline. For instance, for profiling the gene expression at the single nucleus resolution of
(a) Tri-view display of a confocal image of
Pipelining image analysis modules and other more sophisticated data analysis/mining modules is a powerful way to generate quantitative biology. One such pipeline is shown in
(a) A flowchart of the key steps in building a fruit fly brain atlas. (b) A 3D digital atlas of 269 stereotyped neurite tracts reconstructed from GAL4-label fruit fly brains
Visualization and analysis methods are critical for understanding and using 3D microscopic images for various cell biology, structural biology, neurosciences, and systems biology applications. These tools become indispensable for the ever-increasing need to screen tens of gigabytes to many terabytes of microscopic images. Pipelining these tools and other data analysis/mining methods is a new trend for producing interesting biology.
We thank Yang Yu, Ting Zhao, Hang Xiao, and Yinan Wan for discussion of this article, Jan Ellenberg, Philipp Keller, Rex Kerr, Stuart Kim, Tomoya Kitajima, Xiao Liu, Davi Bock, Clay Reid, and Julie Simpson for providing microscopic images, and Christine Morkunas for proofreading the manuscript.