Fig 1.
Qualitative 2D illustration of the 3D mask simulation pipeline.
The simulation process generates an annotation mask in three refinement steps, starting with organism shape generation, adding cell positions within the foreground region and final nuclei or membranes structure generation at each respective positions.
Fig 2.
Overview of possible spherical shapes initialized with statistical shape models.
Different shape variations are visualized, with statistics learned from a public data set [24].
Fig 3.
Overview of possible spherical shapes initialized with spherical harmonics.
Shapes are shown for different values of γ and colors indicate a positive (red) or negative (blue) deviation from a perfect sphere.
Fig 4.
Schematic of the conditional generative adversarial network.
The generator transforms annotation masks m into realistic images , which are assessed by the discriminator using realistic image data
. The positional conditioning is used to generate and assess positional image characteristics. Note that, for simplicity, visualizations are provided in 2D, despite processing being done in 3D.
Fig 5.
Qualitative 2D overview of our 3D data generation pipeline.
It comprises the acquisition or simulation of annotations and the image synthesis. Annotations can be obtained from manual, automated, and simulation approaches, and final cellular annotations are used to generate corresponding instance segmentations.
Fig 6.
Different views of real image data (blue columns) in comparison to synthetic image data (red columns) generated by our GAN approach. Examples are shown for different experiments using manually corrected masks from the data set [24] (Membranemanual), and automatically annotated masks for membranes (Membraneautomatic) and nuclei (Nucleiautomatic) from the
data set [34]. Additionally, the spectra and intensity profiles of different slices are shown as qualitative metrics.
Table 1.
Quantitative assessment of image quality.
Quality scores obtained for the different synthetic data sets.
Fig 7.
Different views of synthetic nuclei image data (top) and synthetic membrane image data (bottom) generated by our GAN approach using the simulated annotation masks.
Fig 8.
Segmentation scores obtained for different data setups.
Multi-class segmentation scores obtained with the approach from [38] (top) and instance segmentation scores obtained with the approaches from [38, 35] (bottom) trained on real data (Real2Real, Real2Syn) and synthetic data (Syn2Real, Syn2Syn) and a mixed data set containing real and synthetic data (Mix2Real, Mix2Syn). Trained models are applied to real and synthetic data, respectively.
Fig 9.
Qualitative assessment of altered image quality.
2D slices of 3D synthetic image data generated by our GAN approach using the same manually corrected mask from [24], with different foreground distance scalings α (top). For the center xz-slice, intensities are integrated over the z dimension and plotted along the x dimension (bottom).
Table 2.
Quantitative assessment of altered image quality.
Quality scores obtained for synthetic image data generated on different quality levels by varying α from Eq 6, including the normalized root mean square error (NRMSE), structural similarity index measure (SSIM), zero mean normalized cross-correlation (ZNCC) and peak signal-to-noise ratio (PSNR).
Fig 10.
Segmentation scores obtained on altered image quality.
Multi-class segmentation scores [15] (top) and instance segmentation scores [15, 35] (bottom) obtained for synthetic image data generated on different quality levels by varying α from Eq 6. Dashed lines represent results obtained on the original test data.
Fig 11.
Samples of the first published data set, including generated 3D image data of nuclei and cellular membrane (top) and the corresponding automatically obtained instance segmentations for nuclei (middle) and membranes (bottom).
Fig 12.
Public altered quality data set.
Samples of the second published data set, including 3 different quality levels of the same cellular membrane structures. Corresponding instance segmentations (bottom) are available from [24].