Figure 1.
Architecture of WaveTracer software.
(A) Graph illustrating the automatic real-time control of the number of molecules detected per frame. The number of localization (green line) fluctuates between a set maximum and minimum (red line). It is controlled by a 405 nm laser (blue line). When the number of localized molecules falls outside the minimum and maximum thresholds, the laser power is automatically adjusted to keep the density of molecules ideal for accurate localization. (B) Different computation steps for real-time super-resolution reconstruction. The 2D localization algorithm and the visualization of the super-resolved image are performed in real-time with an automatic feed-back control based on the statistic extraction. 3D coordinates extraction is performed at the end of the acquisition. If required, at the end of the acquisition process, the fitting of the preliminary localized molecules is performed. If a fiduciary marker is present, it will be tracked for image registration.
Figure 2.
Implementation details of WaveTracer software.
(A) 2D real-time localization steps: 1) During the acquisition process, images are temporarily transferred to the CCD camera buffer. 2) The current image is transferred to the GPU memory for processing. 3) The image is split into 16×16 overlapping sub-images and sent to different processors of the GPU. 4) Wavelet filtering is performed in parallel on each sub-image. 5) Sub-images are stitched back to reconstruct the filtered image. 6) The filtered image is transferred to the CPU for thresholding, watershed processing and centroid extraction. 7) The super-resolution image is then reconstructed and the localized molecule coordinates are saved into the memory for later 3D analysis. (B) 3D post-acquisition localization steps: 1) Images are split into 7×7 sub-images centered on localized molecule coordinates. 2) Anisotropic Gaussian fitting is performed on each sub-image in parallel on GPU. 3) Axial coordinate retrieval of each localized molecule is performed in parallel on GPU. 4) 3D reconstruction is made from the (X,Y,Z) coordinates of all the localized molecules.
Figure 3.
(A) Performance of Gaussian fitting, with (full lines) or without (dashed lines) using GPU, for two different sizes of fitted region. A speed-up factor of about 70 is obtained for the GPU implementation versus the CPU implementation. (B) Performance of the localization algorithms in real-time mode. The 2D localization is performed frame by frame in real-time with CPU (green line) and GPU (blue line). Gaussian fitting using CPU (red line) can only process few molecules in 50 ms. Both algorithms are benchmarked on an Intel Xeon E5645@2.4 GHz personal computer equipped with a Nvidia Quadro 4000 graphic card. (C) Convergence of the NLLS minimization iterative process for anisotropic Gaussian fitting performed on a 7×7 pixels ROI. Measurements were average from 1,000 molecules, simulated with 200 and 1000 photons per molecule.
Figure 4.
Real-time super-resolution imaging.
(A) Diffraction-limited epifluorescence image of microtubules labeled with Alexa Fluor 647. (B) 2D super-resolved image of the cell in figure (A), reconstructed in real-time from 20,000 frames and 1.2 million single-molecule localizations. (C) 3D super-resolved image of the microtubules of figure (A) obtained only 15 seconds just after the end of the acquisition. Colors encode for the axial position, in µm. (E) A selected region of interest (ROI) from figure (A). (F) A selected ROI from figure (B). (G) Corresponding ROI from figure (C). (D), (H) Intermediate real-time visualization obtained after 1,000 and 4,000 frames respectively. (I) Diffraction limited epifluorescence image of microtubules labeled with Alexa Fluor 647. (J), (K) 2D super-resolved images of the cell in figure (I), reconstructed in real-time from 3,000 frames, without (78,341 localizations) and with (96,298 localizations) feedback control respectively. (L) Graph of the number of localizations over the number of images, without (red) and with (green) feedback loop control. Solid and dashed black lines represent their respective trends. For better clarity, only one point over 25 points is displayed.
Figure 5.
(A) 3D test patterns made of 40 sunburst alternating black and white stripes, used for single molecule based super-resolution microscopy simulations. The pattern is 1 µm thick, with consecutive stripes distant from 50 nm in the axial direction. (B) Diffraction limited image of the test pattern. (C) Example of isolated single point emitters located inside the test pattern, convolved with 3D astigmatic point spread function (PSF). (D–F) Test pattern reconstruction performed by WaveTracer software for 1000, 200 and 100 photons/molecule respectively. (G–I) Test pattern reconstruction performed by RapidSTORM software for 1000, 200 and 100 photons/molecule respectively. (J–L) Test pattern reconstruction performed by QuickPALM software for 1000, 200 and 100 photons/molecule respectively. Scale bar is 1 µm.
Figure 6.
(A) Speed benchmarking for 2D and 3D localization, in offline and real-time modes, and comparison with QuickPALM and RapidSTORM software. Benchmarking was performed on an image stack of 30,000 planes composed of 286,665 molecules. The 0* value mentioned for 2D online means that the localizations are performed in parallel with the acquisition, and that no extra processing time is required. (B) Localization accuracy benchmarking in 2D, 3D and for 3 different numbers of photons per molecule, and comparison with QuickPALM and RapidSTORM software. (C) Recall and precision detection rates benchmarking for 3 different numbers of photons per molecule, and comparison with QuickPALM and RapidSTORM software.