Optimal model-based sensorless adaptive optics for epifluorescence microscopy

We report on a universal sample-independent sensorless adaptive optics method, based on modal optimization of the second moment of the fluorescence emission from a point-like excitation. Our method employs a sample-independent precalibration, performed only once for the particular system, to establish the direct relation between the image quality and the aberration. The method is potentially applicable to any form of microscopy with epifluorescence detection, including the practically important case of incoherent fluorescence emission from a three dimensional object, through minor hardware modifications. We have applied the technique successfully to a widefield epifluorescence microscope and to a multiaperture confocal microscope.


Introduction
Adaptive optics can be employed in fluorescence microscopy to compensate for system or sample induced phase aberrations in the optical path, increasing the quality of the acquired images, especially in thick and turbid samples. Due to the low brightness of fluorescence emission and to the absence of guide stars in most samples, the correction of aberrations is generally performed through the optimization of an image-based metric. A common method for metric optimization is a model-based procedure, using a-priori assumptions on the dependence of the metric from the aberration, providing fast convergence speed with good reliability. Model-based aberration correction in fluorescence microscopy techniques is usually performed [1][2][3] with a hill climbing optimization procedure, fitting of an image-based performance metric M within an N-dimensional aberration space generated by a base of aberration functions. For any base X, any phase aberration, neglecting its piston component, can be described as The value of a well chosen metric can be approximated, for small aberrations, as a quadratic where B ≽ 0 is a constant, positive semi-definite N × N matrix, and a is the vector of coefficients, a T = (a 1 , . . ., a n ).
In the particular case in which matrix B is diagonal, the metric function can be expressed as and correction of the wavefront through an adaptive optical element (AOE), such as a deformable mirror (DM), can be performed on each element of the base separately through a quadratic fit, therefore requiring only 2N + 1 measurements of M for full correction of any aberration.
The main drawback in the use of such a correction method is that, given an image metric M, a generic base for aberration space, such as the Zernike polynomials or the influence functions of the actuators of an AOE, generally does not satisfy condition (4). Moreover, the quadratic model of function (3), depending on the metric used, is generally only valid for small aberrations [4].
Several methods have been devised to generate, given a metric M, a suitable base X satisfying condition (4). In general the methods are based on a practical calibration procedure involving O(N 2 ) measurements of the metric M on a sample [5]. This may only work for small aberrations, and is based on the reasonable but generally not rigorously proven assumption that after calibration on a given field of view, condition (4) will be satisfied by the same base and metric function in any other field of view in the sample.
In this paper we propose a new approach to model-based hill-climbing optimization in fluorescence microscopy, involving small hardware modifications of the microscope, in order to enable measuring of a sample independent metric. The modification applied allows to measure the second moment of the fluorescence emission distribution from a point-like excitation spot, which constitutes a sample independent metric. Furthermore, we prove that such a metric respects condition (4) for an analytically defined base set (gradientorthogonal aberrations), for aberrations of any amplitude. Finally, we present a method for calibrating any AOE with a reference source and a wavefront sensor, in order to generate such modes without requiring sample dependent calibration procedures. The main practical advantages of the method are its validity for aberrations of any amplitude, and the speed and reliability of the wavefront sensor-based calibration procedure, as compared to image-based calibrations, which are generally slower, and can be severely affected by noise.
Second moment methods [6] have already been proven ideal for hill-climbing optimization in regular sensorless adaptive optics, when imaging the point spread function of the system from a coherent point-like source [7]; However, in order to successfully prove the validity of this method for the case of fluorescence microscopy in thick samples, the model should be extended to incoherent emission from a three dimensional source, (see section "Physical model and proof of validity").
Alternatively, image-based metrics, in particular the total intensity of a selected frequency range of the power spectrum of the image, have been shown to respect condition 4 for gradient-orthogonal bases, such as the Lukosz polynomials [8,9]. The main advantage in using image-based methods, compared to the method proposed in this paper, is the relatively simpler hardware setup, where the only modification required to the optical setup is the addition of the AOE. However, such methods work only for small aberrations, as the dependence of the metric from the aberration is Lorentzian, and can only be approximated with a quadratic function in a small range. Additionally, the frequency range considered must be carefully calibrated based on the nature of the sample, and especially on the contribution of out of focus fluorescence emission. Moreover, they would not work for aperture-based scanning setups, as the physical image formation process is different.
Due to these limitations, to the knowledge of the authors, calibration-less imagebased methods have never been reported working on thick fluorescent samples, but only on brightfield images of bidimensional samples [8], or on coherent images of pure scatterers [9].

Correction technique
The correction technique can be applied in any fluorescence microscope based on camera detection (e.g. Epifluorescence microscopy, Structured illumination microscopy, lightsheet microscopy, localization superresolution microscopy), with the addition of a secondary excitation source generating an array of point-like spots in the image plane. This can be achieved with a wide variety of methods such as the use of an incoherent source and a digital micromirror device or pinhole array, or with a coherent source and a diffractive optical element or microlens array.
In alternative, the technique can be applied to a scanning confocal fluorescence microscope, with single (e.g. confocal laser scanning microscopy, STED microscopy) or multiple (e.g. Spinning disk microscopy, programmable array microscopy) apertures, by using a confocal aperture created on a reflective surface. The aperture surface should be slightly tilted with respect to the optical axis, in order to allow for the reflection of fluorescence light to be imaged on a camera, as shown in Fig 1. In order to correct aberrations, an AOE should be present in a pupil plane of the system shared by excitation and emission light. The AOE should be pre-calibrated with a wavefront sensor, in order to introduce aberrations described by coefficients of a gradient orthogonal base [8] (rigorously defined in the section "Physical model and proof of validity").
The metric used is the second moment of the spatial distribution at the image plane of fluorescence emission from a diffraction limited, point-like excitation source, averaged over multiple positions in the field of view.
Each measurement of the metric should be performed on the average of images of pointlike excitation spots in several locations in the field of view. This would happen differently in a camera based system or in a aperture based confocal microscope.
In a single aperture microscope, shown in Fig 1A, the pinhole can be exposed for the entire duration of the image scan. Then the metric can be computed by averaging the pinhole images, sampled in arbitrary number of bright pixels evenly distributed in the field of view, in a "random access" fashion [10]. In a multi-aperture confocal microscope, a similar procedure can be applied using the rejected light image.
In a camera based system, shown in Fig 1B, once a pattern is projected, the brightest spots should be selected and averaged on a single image.
Correction of an aberration can then be performed as described in the introduction, by acquiring for each element x n of the gradient orthogonal base the three measurements of the second moment of the distribution M 0 = M(φ), M + = M(φ + ax n ) and M − = M(φ − ax n ), where a is an arbitrarily chosen factor, and determining the optimal correction as the minimum of the quadratic fit of the three points. Since the measurement of M 0 is the same for all elements of the base, the procedure only requires 2N + 1 measurements.

Physical model and proof of validity of the method
Definition of image moments. Describing an image as a bidimensional distribution of light intensity I(x, y) in a field of view F, its first moment, or center of mass, is defined as and the central second moment sm is defined as Definition of gradient orthogonal base. Given two phase aberrations φ 1 and φ 2 , a gradient dot product operator ; can be defined as the dot product between the gradients of φ 1 and φ 2 A gradient-orthogonal base G = {g 1 , g 2 , . . .} is a base in which, for any couple of vectors of the base A practical consideration is that the displacement Zernike polynomials Z 2 , Z 3 , and Z 4 (tip, tilt, and defocus) verify the previous condition. A convenient gradient-orthogonal base should therefore include these three displacement modes, and aberration correction should be performed on all remaining modes, to ensure that the correction procedure does not displace the field of view.
Physical model of the optical system. A generic microscopy setup is modeled as a telescope system formed by a tube lens and an objective lens. A scheme reporting the conventions in the systems of coordinates used is reported in Fig 2. The fluorescence image I cam (x@, y@) of a point-like excitation spot is dependent on the phase aberration φ(x, y) at the pupil plane, on the aperture of the optical system, and on the three-dimensional distribution O(x 0 , y 0 , z 0 ) of fluorophore concentration at the sample.
In particular, consider a field at the pupil plane where P(x, y), in the assumption of evenly illuminated pupil, is the aperture function The field E o (x 0 , y 0 , z 0 ) of excitation light at the object coordinates is approximated by coherent propagation as with f being the focal length of the objective lens. E o , representing the complex point spread function of the system, is equal to the Fourier transform of E p at the focal plane (where z 0 = 0), and the Fourier transform of E pz ¼ Pðx; yÞe iφ z elsewhere, where with Z 4 equal to the defocus Zernike polynomial. The distribution I o (x 0 , y 0 , z 0 ) of fluorescence emission is proportional to the intensity of excitation light multiplied by the spatial distribution of the fluorophore O: Due to the incoherent nature of fluorescence emission, the spatial fluorescence intensity distribution I em (x@, y@, z@) at the camera will simply be the convolution of I o and the three dimensional point spread function of the system (the coordinates system is considered the same, neglecting the magnification of the optical system). As the aberration affecting the point spread function is still φ, but applied to light propagating in the opposite direction, the point spread function is equal to |E o | 2 , with inverted coordinates, leading to Since image detectors are two dimensional, and positioned at the focal plane of the system, the detected intensity I cam at the camera plane can be just expressed as: I cam ðx 00 ; y 00 Þ ¼ I em ðx 00 ; y 00 ; 0Þ: ð15Þ Optimal adaptive optics for microscopy It is important to notice that, considering two different samples O 1 (x 0 , y 0 , z 0 ) and O 2 (x 0 , y 0 , z 0 ), due to the incoherent nature of fluorescence, the sum of the corresponding images is exactly equal to the image generated by a sum of the two objects O 1 + O 2 .
As a direct consequence, averaging on a wide enough variety of random objects, which can be achieved by averaging images in multiple excitation spots over the same field of view, is practically equivalent to sampling a single object with uniform fluorophore distribution. In this scenario, through the rest of this formal proof, we will assume illumination is generated in a single point, and the fluorophore distribution is assumed constant: Proof of validity of the method. This section of the paper demonstrates that, considering the image of fluorescence light emitted by a point-like excitation spot, the variation in second moment with respect to a diffraction limited condition can be used as a metric M, for which condition (4) is verified by a gradient-orthogonal base G.
In a previous publication [6], the authors proved that, in the presence of a phase aberration, the variation of the second moment of both the image of a coherent point source and of an incoherent two-dimensional extended source is proportional to the mean square gradient magnitude of the phase aberration. In mathematical terms, for any two-dimensional object O 2D (x 0 , y 0 ), in the presence of a phase aberration φ(x, y) this can be written as: Let us now apply Eq (17) to the model described in the paragraph "Physical model of the optical system". Defining sm o (φ, z 0 ) as the second moment of excitation intensity distribution at the object location for a given aberration φ at an axial distance z 0 from the image plane, Eq (17) can be written as: where φ z 0 is as defined in Eq (12), c o (z 0 ) is a constant only depending on z 0 , and sm o (0, 0) is the second moment of excitation intensity distribution at the focal plane in the absence of aberrations.
Under assumption (16), the fluorescence emission is imaged at the camera plane as an incoherent extended source with the same intensity distribution as the excitation light.
If fluorescence emission would not be affected by any aberration, the image obtained at the camera plane would be equivalent to the extended source, and therefore the second moment sm cam of intensity at the camera plane would be sm cam (0, z 0 ) = sm o (φ, z 0 ).
In the case of epifluorescence imaging, however, fluorescence light from a plane at a distance z from the focal plane in object space is affected by an aberration ϑ z 0(x, y) = φ z 0 (−x, −y). Observing that Z P jrW z 0 j 2 dx dy ¼ we can apply Eq (17) again, so that the second moment of the camera image sm cam (ϑ, z 0 ) where c(z 0 ) = c cam (z 0 ) + c o (z 0 ). The actual image at the camera plane is the integral over the z 0 range of the images of the single planes. Neglecting the loss in excitation light due to absorption, and therefore considering equal total excitation power for all values of z 0 , the second moment sm of the final image is the average second moment over z 0 : where T is defined as the distance for which sm o ð0; TÞ ) sm o ð0; 0Þ: The choice of limiting the integral to T is due to the observation that objects too far from the depth of field of the objective will have no distinguishable details, and negligible intensity compared to the objects close to the focal plane, and can therefore be excluded from the integral. Decomposing the aberration φ on a gradient orthogonal base G which includes the Zernike displacement modes (in Noll's index notation) and assuming it free of displacement modes, the aberrations φ z 0 can be described as It can therefore be derived that Denoting the sample dependent second moment of the image in the absence of an aberration as a constant smð0Þ ¼ it can be seen that the metric M(φ) = sm(φ) respects condition (4) independently of the sample shape.

Assumption and limitations.
A few assumptions were made in the physical modeling of the system, and proof of validity of the method. This section lists the most apparent, and the effect they can have on the experimental implementation of the system.
• Aberrations are assumed to be isoplanatic, meaning that a single phase aberration profile is present in the pupil plane of the system, resulting in a constant effect over the full field of view. While, considering the geometry of a sample induced aberration in microscopy system this is clearly not true, this is a common assumption in adaptive optics for microscopy. Image metrics calculated over the whole field of view are generally used, in the assumption the optimization of the metric will lead to the correction of the average aberration over the field of view. Since the metric described here is computed over multiple positions in the field of view, this assumption holds validity.
• Aberrations are considered constant over the whole thickness interval T. While it can be assumed that the order and amplitude of aberrations increases with depth in the sample, data reported in literature [11] suggests that the variation is small over the Rayleigh length of high numerical aperture objectives.
• The method is supposedly working for samples of any thickness. This is true as long as the intensity contribution to the spot image of the signal from within the distance T of the system is much brighter than that of out of focus light. This is true for a single, unaberrated spot, and it holds true in the presence of aberrations which can be reasonably compensated with a low order commercial AOE. This assumption could be an issue in the case of parallelized spots creation, as the contribution of multiple beams could sum up out of focus. As in any parallelized confocal system, the relationship between sample thickness, numerical aperture, and spots spacing must be carefully considered in order to successfully correct aberrations, as well as maintaining optical sectioning.
• The method proposed is mathematically valid for aberrations of any amplitude. However there are, obviously, physical limits to validity of the system. The main limit is the size of the detection area for the second moment metric, limited by the sensor size, and by the spacing between points in the case of a microscopy with multiple illumination points. High amplitude aberrations would require bigger sensors or lower magnification, and if necessary an increase in the spacing between illumination points. In an extreme situation, if the light is spread over an area too wide, the detection signal to noise ratio could be insufficient to correctly determine the second moment of the distribution.
• The method proposed, as in most adaptive optics methods for non monochromatic systems, does not consider dispersion in the sample, assuming the same aberration is present for all wavelengths in both excitation and emission spectrum. This is generally a valid assumption for most common fluorescent specimens, exhibiting small Stokes shifts, but could be a challenge for future attempts to implement the method in multiphoton excitation microscopes.

AOE calibration procedure
In a practical setup, the gradient dot product (Eq 7) between the phase correction introduced by an AOE can be estimated from the centroids displacements on a Shack-Hartmann wavefront sensor. The wavefront sensor is only required for calibration of the mirror, which could be performed on a separate optical setup, without the need of incorporating it in the microscope. In practice, the AOE can be calibrated on a separate optical setup, and then included in the microscope. In particular, given a Shack-Hartmann sensor with N sh centroids, each with a displacement measured in two components dx i and dy i , introducing two aberration φ 1 and φ 2 , the dot product can be estimated as: In order to obtain a gradient orthogonal base expressed as a set of inputs for the N act actuators of the AOE, the centroids displacements for each influence function φ i of each actuator should be measured, and a gradient products matrix G of size N act × N act can be estimated, with each element equal to: estimated as in (28).
Performing singular values decomposition on the matrix G, so that G = USV, the lines of matrix V are a set of coefficients, with singular values equal to the values of the diagonal of S, constituting a gradient orthogonal base for the AOE. For practical implementation of the correction method, the three elements of the base most similar to tip, tilt and defocus Zernike terms should be neglected, as well as elements of the base with neglectable singular values. The phase distribution of a representative gradient orthogonal base are reported in Fig 3. It is to be noted that, depending on the design of the optical system, the image of the AOE on the back aperture of the microscopy system could be slightly larger than the optical aperture. If that is the case, this should be carefully taken into account, and computation of the gradient dot product as in Eq (28) should be only performed on the centroids within the aperture area effectively used in the microscope.

Experimental results
In order to prove the proposed method works in experimental conditions, test measurements were performed both on a standard epifluorescence setup, and a multiaperture confocal microscope based on a digital micromirror device. Performances of the gradient orthogonal base were compared to the use of a simulated Zernike base, often used in hill-climbing optimization when no calibration is available [12,13]. The epifluorescence microscope is a simple, low cost adaptive setup, based on LED excitation at 470nm (M470L3, Thorlabs, US), a 43 actuators piezoelectric DM (DMP40/M-P01, Thorlabs, US) and an industrial CMOS camera (UI-3060CP, IDS, Germany). An array of focal spots was generated in an image plane through a single mode solid state laser (sapphire 488 nm LP, Coherent, US), and a microlens array (MLA300-14AR-M). The spacing between illumiation spots was fixed by the geometry of the camera and the microlens array, resulting in a usable array of 38x24 spots, with a spacing of 300μm on the camera, resulting of a 7.5μm at the sample plane in experimental conditions. It is to be noticed that, while the laser source used has a maximum power of 100mW, power levels < 5mW were sufficient to perform the test measurements reported.
The pinhole based setup is a custom made adaptive multi aperture confocal microscope, recently used for a model free optimization application [14]. The setup is based on incoherent LED illumination (PT121B, Luminous, US), and a Digital Micromirror Device (Lightcrafter 6500 EVM, Texas Instruments, US) acting both as an array of point sources and as a confocal array of pinholes. Exploiting the binary programmable nature of the array, isolated active pixels can act as pinholes, while the surrounding inactive pinholes can act as a tilted reflective surface. A secondary camera (Optimos, QImaging, Canada) is used to image such plane. A static array of equally spaced pinholes is created on the DMD. The number and spacing of spots is customizable through the DMD. In experimental conditions, an array of 15 by 8 spots with a spacing of 812μm is generated at the camera plane, resulting in a spacing of approximately 24μm at the sample plane in experimental conditions. The images of all pinholes are cropped from the camera image, and averaged, and the second moment of the average distribution was computed according to Eq (6). The AOE used for aberration correction is a 69 magnetic actuators DM (DM-69, Alpao, France). Calibration of the DMs was performed with two separate Shack Hartmann detectors. The lower order piezoelectric DM of the epifluorescence microscope was calibrated with a CCD based wavefront sensor (WFS150-7AR, Thorlabs, US) with a resolution of 1280 × 1024 pixels, 5.95 × 4.76mm 2 area sensor, with 150 μm pitch and 5mm focal length, for a total of approximately 700 subapertures in the pupil plane, while the higher order magnetic actuators DM for the confocal setup was calibrated with a high resolution Shack-Hartmann wavefront sensor (Flexible Optical B.V., the Netherlands) with a resolution of 2000 × 2000 pixels, 10 × 10 mm 2 area sensor, with 63 μm pitch and 2mm focal length, for a total of approximately 18000 subapertures in the pupil plane. All measurements, on both microscopes were performed with a 40X, 1.25 numerical aperture, oil immersion objective Optimal adaptive optics for microscopy (Leica, Germany), and a standard set of Green fluorescence protein filters (MDF-GFP, Thorlabs, U.S.). The sample used for experimental tests on the epifluorescence microscope is a prepared slide with BPAE cells, with microtubules stained with Bodipy (Fluocells prepared slide # 2, Invitrogen, US). The sample for experimental tests on the confocal microscope is a 16 μm thick prepared slide of mouse kidney stained with Alexa 488 (Fluocells prepared slide # 3, Invitrogen, US). Severe aberrations, for testing purposes, were introduced by drying a 2% agarose solution on the coverglass.
The calibration procedure of the DM resulted in the generation of a base of 50 gradientorthogonal aberrations for the high order DM, and 32 for the low order DM, excluding tip, tilt and defocus. In order to verify that the base respects condition (4), with the high order DM, for every couple of base elements φ i , φ j , measurements of the metric were performed for aberrations φ ij = a i φ i + a j φ j , with −3μm < a i , a j < 3μm, for 7 uniformly distributed values of a i and a j , resulting in a total of 49 measurements per couple of base elements. The resulting functions were fitted with the quadratic model , equivalent to a two dimensional version of Eq (3), which respects condition (4) in case the angle θ is Zero, or A 1 = A 2 .
In order to quantify the deviation of the metric function from (4), the parameter P was used: Optimal adaptive optics for microscopy which is equal to zero if φ i and φ j respect condition (4), and infinite if φ i / φ j . As a term of comparison, the same measurement was performed on a simulated Zernike base up to the eighth order. Results are reported in Fig 4. It can be observed that for Zernike polynomials every base element has at least one clearly not orthogonal other base member. For the optimal base, as expected, the values of parameter P have much lower values, and are generally randomly distributed.
In order to verify the performance of aberration correction, sequential 2N + 1 measurements corrections as described in the introduction were performed on the samples with severe aberrations introduced by an agarose layer. In both microscopes, optimization was performed at 100Hz, resulting in an optimization time of 0.65s for the low order epifluorescence microscope, and 1.01s for the high order confocal microscope. In the case of the confocal microscope, the system was tested separately with a clear coverslip, correcting for the sample induced aberration at a depth of % 10μm. The same test could not be performed on the epifluorescence microscope, as the sample did not show any detectable sample induced aberration. Usage of the gradient orthogonal base proved to produce the optimal correction in a single iteration for small aberrations, and in two iterations for more severe aberrations, possibly due to the experimental error in the metric calculation and in the fit of the metric function. On the other hand, performing the same correction procedure with a Zernike base, even small aberrations required several iterations to reach optimal correction. Some representative results are reported in Fig 5.

Conclusion
In this paper, a novel sensorless method for modal aberration correction in fluorescence microscopes has been presented. The method is based on sample-independent pre-calibration of an orthogonal set of modes, describing the connection between the adaptive optics action and the performance metric expressed as the second moment of the fluorescence distribution detected in epifluorescence from a point-like excitation pattern. As a result of such pre-calibration, the optimal correction can be achieved in 2N + 1 measurements of the metric, for aberrations of any amplitude. The methodology can be implemented, with the addition of a secondary excitation source producing an array of point-like spots, in camera based systems (e.g. epifluorescence microscope, structured illumination microscope, localization based superresolution) or, with the addition of a reflective aperture surface and a camera detector, in aperture-based confocal systems (e.g. laser scanning confocal, spinning disk microscopes, STED). The validity of the method has been experimentally proven in epifluorescence microscopy, and on a programmable multiaperture confocal microscope. Verhaegen.