Fig 1.
Simulation of AO-SLO image with eye motion artifacts.
(a) A cone mosaic phantom. (b) A simulated eye movement traced, using a self-avoiding walk model of fixational eye movements. Units on both axes are visual angle in arcmin. (c) Simulated effect of eye movements in (b) on raster-scanned image of (a). Horizontal and vertical components of eye movements are visible in shearing and compression or expansion of image features, respectively.
Fig 2.
Schematic of the multi-modal AO-SLO system.
DM, deformable mirror; SHWS, Shack–Hartmann wavefront sensor; AL, achromatic lens; PMT, photomultiplier tube; SM, spherical mirror; FM, flat mirror; BS, beam splitter; DBS, Dichroic beam splitter; KEP, knife edge prism; RS, resonant (x) scanner; GS, galvanometer (y) scanner; BD, beam dump.
Fig 3.
Strip-based registration of synthetic AO-SLO images.
Registration begins with selection of a reference frame (a) from the series of 200 images to be registered. Next, each target image in the series is partitioned into a series of horizontal strips. Using two-dimensional cross-correlation, these strips are registered and aligned with the reference frame. Two example targets are shown in (b) and (c), after alignment to the reference. Displacements between position of a strip in the reference image and the aligned target are referred to as lags. Iterating through the target images, a running sum of the aligned images is stored, shown in (d). Because the retina is moving randomly, some parts of it are imaged more than other parts. As such, while calculating the sum of aligned targets, it is necessary to keep a counter image (e), which stores the number of strips contributing to any part of the sum. Once all of the target images have been partitioned, registered, aligned, and added, the sum image (d) is divided by the counter image (e) to produce a registered average (f). Artifacts of eye motion are clearly visible in the registered average (f). These artifacts are also manifest in the counter image (e), with horizontal motion causing lateral warp in the counter and vertical motion causing variations in its amplitude. Units in (e) are number of strips, and the units in the other images are arbitrary measures of intensity.
Fig 4.
Motion correction using lag biases.
The reference image (left) can be considered as a discrete set of image strips. During motion correction, the reference image is cut into strips, which are then registered, by cross-correlation, to each of the target images. Because eye motion is uncorrelated among the target images, the average position of a given reference strip is an indication of the eye’s position during its acquisition. The lag biases were used to estimate eye movements during acquisition of the reference frames, and then computationally removed from the reference frames using two-dimensional linear interpolation.
Fig 5.
Estimation of simulated motion from strip-registration lag biases.
Traces of the simulated eye position are plotted with solid lines, separately for horizontal (left) and vertical (right) components. The lag bias estimates and
are plotted with dashed lines. Estimates bear qualitative similarity to the simulated movement trace, consistent with high goodnesses of fit to the simulated movement trace (
and
).
Fig 6.
Removal of motion artifacts from reference image.
After reconstructing reference frame eye movements from lag biases, the reference frame is interpolated from its natural coordinates into a set of motion-free coordinates. The original object is shown in (a). The image used as a reference for strip-based registration is shown in (b). The motion-corrected reference is shown in (c). The shear, compression, and expansion artifacts in (b) are visibly reduced in (c). In order to demonstrate better correspondence of the corrected reference to the object, horizontal and vertical profiles of whole-image cross-correlations with (a) are shown. Horizontal and vertical profiles are offset for clarity. (d) shows the autocorrelation of (a), with characteristic central peak and side lobes due to the regularly spaced cones. (e) and (f) show the whole-image cross-correlations between (b) and (a) and between (c) and (a), respectively. Correlation of the corrected image with the object is significantly higher, with the residual mismatch between the latter two limited by finite oversampling of images during cross-correlation.
Fig 7.
Comparison with previously published approach.
The method proposed here and the method proposed by Bedggood and Metha, 2017 are slightly different. Our method does not utilize rigid body registration of whole frames prior to strip registration. As such, we may expect minor differences in the reconstructed eye movement traces, owed in part to the higher probability for strips to be misplaced in our approach. In order to quantify this error, we compared eye motion reconstructions using the Bedggood and Metha method with our own. The resulting pairs of x and y movement traces are shown on the left and right. The reconstructed traces from the two methods are qualitatively similar, and have R2 values of 1.0 and 0.97, respectively.
Fig 8.
Removal of motion artifacts from real AO-SLO images.
(a) and (b) show an average of 100 AO-SLO frames strip-registered to two separate reference frames; (e) and (f) show the motion-corrected versions of (a) and (b), respectively; (c) and (g) show the cross-correlations a ⋆ b and e ⋆ f, respectively. Clearly, smearing of the cones in (a) and (b) has reduced the cross-correlations between them, and after motion-correction of the references, the cross-correlation in both x- and y- direction become sharper and its value increases. (d) and (h) show a pseudocolor overlay of (a,b) and (e,f), respectively.
Fig 9.
Motion analysis of the averaged AO-SLO image before and after motion correction.
(a) Averaged AO-SLO image before motion correction; (b) and (c) Logarithmic scale DFT of the area outlined in blue and red strips in panel (a). Distortion analysis based on the coefficient of determination (R2) obtained from the linear regression of the maxima across columns shows R2 values of 0.327 and 0.276 for the area outlined in blue and red, respectively. (d) Averaged AO-SLO image after motion correction algorithm; (e) and (f) logarithmic scale DFT of the area outlined in blue and red strips in panel (d). The proposed motion correction algorithm reduced the distortion of the averaged image where the R2 values reduced to 0.004 and 0.042 for the area outlined in blue and red in panel (d), respectively.
Fig 10.
Strip-based registration of non-confocal AO-SLO channels.
(a) confocal, (b) split-detector, and (c) dark-field at 1° temporal to the fovea. The high-contrast image provided by the confocal channel permits strip-registration and motion correction of images in the other channels, since the images are acquired concurrently.
Fig 11.
Correcting axial motion in AO-OCT volumetric images.
Slow-axis B-scans of the IS/OS and COST layers. The uncorrected scan is shown in (a), with visible axial eye motion. En face projection and other visualization and analysis techniques require correction of axial warp, and commonly employed techniques include alignment of B-scans by center of mass, gradient-based edge detection, and maxima segmentation, shown in (b), (c), and (d), respectively. While the outer segments of cones in any region of interest are similar, they are not identical, and the reflective surfaces forming their boundaries are axially staggered with respect to one another. A consequence of this staggering is that aligning by these methods results in artifacts (white arrows) as well as alteration in the apparent flatness of the surfaces. The latter artifact is especially evident in (d), where the IS/OS is artificially flattened while the COST is artificially made rougher. Lag-bias reconstruction (e) avoids these artifacts, and is presumed to yield a more faithful representation of the roughness of these surfaces.
Table 1.
Roughness of outer retinal surfaces depends on axial dewarping strategy.
Fig 12.
Axial lag bias reconstruction improves accuracy of cellular morphometry.
AO-OCT volumes permit many sorts of quantitative cellular morphological measurements. The curvature of the cone bands results in variations in en face projections of the cone mosaic along the slow (vertical) axis of the image, as shown in (a) and (d). The green line indicates the location of the slow-axis B-scans shown in Fig 11(a) and 11(e), respectively. Automated three-dimensional segmentation of inner (IS) and outer segments (OS) permits visualization of their lengths. Uncorrected IS and OS lengths are shown in (b) and (c), while corrected maps are shown in (e) and (f). While a ground-truth comparison is not possible, the corrected maps appear to suffer from fewer errors and bear better similarity to the smooth appearance of the layers in OCT B-scans and other morphometric studies [38]. The IS map (b) and (e), which requires localization of the relatively dim ELM, benefits especially from careful correction.
Fig 13.
Removal of motion artifacts from AO-OCT images.
Averaging the AO-OCT volume in the two lateral dimensions produces a longitudinal reflectance profile, shown (a) in log scale. The labeled peaks correspond to the nerve fiber layer (NFL), inner plexiform layer (IPL), outer plexiform layer (OPL), external limiting membrane (ELM), inner-outer segment junction (ISOS), and outer segment tips (COST). The axial extents of the Henle fiber layer (HFL), cone outer segments (COS), and retinal pigment epithelium (RPE) are depicted on the plot with red, green, and blue shaded boxes. By extracting and averaging together corresponding depths of interest from the motion-corrected volumetric image, projections of these layers can be produced, shown in (b), (c), and (d), respectively. Some of the variation in brightness of HFL is likely due to shadows cast by overlying blood vessels; these can be observed in the much brighter COS mosaic as well. Other factors may be segmentation errors and directional effects [39, 40]. Each pixel in the images consists of an average of between 90 and 135 separate measurements. Images were centered 0.5° temporal to the fovea, subtending 1° and 0.5° in the vertical and horizontal dimensions, respectively.