Modular Scanning Confocal Microscope with Digital Image Processing

In conventional confocal microscopy, a physical pinhole is placed at the image plane prior to the detector to limit the observation volume. In this work, we present a modular design of a scanning confocal microscope which uses a CCD camera to replace the physical pinhole for materials science applications. Experimental scans were performed on a microscope resolution target, a semiconductor chip carrier, and a piece of etched silicon wafer. The data collected by the CCD were processed to yield images of the specimen. By selecting effective pixels in the recorded CCD images, a virtual pinhole is created. By analyzing the image moments of the imaging data, a lateral resolution enhancement is achieved by using a 20 × / NA = 0.4 microscope objective at 532 nm laser wavelength.


Introduction
Confocal microscopy has been used extensively in many areas of science since its invention in 1955 by Marvin Minsky [1]. The ability to obtain optical sections by the aid of a pinhole and detector combination sets it apart from traditional widefield microscopy. Over the last few decades, a variety of confocal microscopes have been developed based on the same principle that a pinhole equivalent blocks out-of-focus light. For example, the use of a single mode fiber emulates the result of the traditional pinhole-photodetector system [2,3]. A semiconductor laser diode cavity also has the same spatial filtering as a pinhole [4]. With the development of micro electromechanical system (MEMS) technology, digital micromirror device (DMD) based confocal microscopes have gained popularity owing to the fact that DMD can be programmed to create a size-changeable reflection pinhole [5][6][7]. Image scanning microscopy (ISM), proposed by Sheppard [8], uses pixel reassignment of an array detector, based on the fact that using a displaced point detector has a narrowing effect on the effective point spread function, which improves the lateral resolution of confocal microscopy. ISM was recently experimentally demonstrated by Müller and Enderlein [9]. They used a configuration which places an EMCCD camera behind the confocal pinhole. Another ISM approach uses a specifically-designedpoint detector array in which each displaced photodiode unit acts individually. Thus, multiple images are acquired with each image representing a different phase of the same sample. These images are then combined using sophisticated algorithms to generate the final sample image, contributing to a resolution improvement in both lateral and axial directions [10].
In this paper, we present a modular scanning confocal microscope in which the pinhole and detector are replaced by a CCD camera [11]. Instead of monitoring the light intensity, the entire beam profile is collected. After data acquisition, the beam profile images are processed to yield a sample image. By eliminating the physical pinhole and using an off-the-shelf CCD camera instead of a custom-designed detector, it reduces system complexity as well as alignment effort. This approach differs from that of Ref [12], which selected active CCD pixels to create a synthetic pinhole. With our microscope, the area of the detector plane is fully analyzed to enhance contrast or determine surface topography. Additionally, our approach provides a way to create multiple pinhole sizes on a single scan. On another note, confocal microscopy has been a mature market in which many commercial microscopes are available, but with a high cost. For this reason, many researchers opt to build custom designed microscopes to balance cost and functionality [13]. Our microscope, which uses low-cost components, could fill an important market need.

Experiment
The scanning microscope system as shown in Fig 1 is built around Thorlab's 30 mm cage system with a few custom designed parts. The entire system consists of four primary modules: 1. The light source module uses a collimated laser diode of 4.5 mW at 532 nm wavelength. A beam attenuation unit made of a Glan-Taylor polarizer P 1 and a half waveplate is used to adjust the incident laser power. A Keplerian type beam expander expands the laser beam to slightly overfill the back aperture of the microscope objective O 1 (Zeiss LD Plan-Neofluor 20 × /0.4 Corr). A 50 μm diameter confocal pinhole is inserted at the internal focal point acting as a spatial filter. The expanded beam is then guided into the main microscope optical train by a beam steering mirror pair and a beam splitter cube. The reflected (or emitted) light is then directed by various beam splitter cubes (or dichroic mirrors) into the camera detection module and fiber-optic detection module.
2. The camera module includes two arms. CCD 1 (The Imaging Source, DMK 23U618 monochrome camera) in the vertical arm collects the reflected light pattern, saved as a 640 × 480 resolution bitmap for later image processing. A line filter at 532 nm is placed in front to only let reflected laser light pass through. CCD 2 (The Imaging Source, DFK 23U274 color camera) in the horizontal arm is used for wide field inspection and initial positioning, and a notch filter at 532 nm is used to keep laser light from entering the camera. A 200 mm focal length tube lens is used in both arms.
3. The fiber detection module uses a 4 × Olympus microscope objective to couple light into a 25 μm diameter multimode fiber, which acts as the confocal pinhole.
4. The motion module uses a piezoelectric objective scanner (Physik Instrumente, PIFOC 1 P-725.4CD) for vertical scan, and a piezoelectric nanopositioning stage (Physik Instrumente, P-611.2S) for lateral scan. A motorized scan table (Physik Instrumente, KT-120) and a 3-axis manual stage are used for large area translation.
In addition, a shearing interferometer is used for beam collimation and intensity monitoring. A LabJack U3-LV DAQ device is used for collecting data from the photodiodes. The data (image) acquisition and motion control software is written in the C++ programming language. Image processing and data analysis are done in MATLAB 1 and Origin 1 .
We choose sample scanning instead of laser scanning because the design is optically simple and less susceptible to optical aberrations. Additionally, the field of view (FOV) of a sample scan microscope can be independently chosen, without changing the microscope's optical design.
By selecting a region of interest (ROI) on recorded images, a virtual pinhole can be created. The ROI only includes the pixels corresponding to the diffraction limited spot on the sample plane, which abides by the confocal principle. In practice, we choose a square pinhole of N × N pixels in our experiment. The choice of N value will be discussed in the following section. Beyond a virtual pinhole, the image can be analyzed to bring out additional contrast. Once the scan is complete, the recorded images are stored in the same order as their spatial coordinates. Then computer algorithms are used to extract information from the recorded images.

Image Moment
Images are analyzed by calculating image moments. The concept of moments is extensively used in classical mechanics [14]. A general definition of a two-dimensional (p+q) th order moment of a density distribution f(x,y) is given by [15]: Its form in image processing is provided as follows: Thus, the 0 th order image moment M 00 is equivalent to the irradiance. As discussed in the following section, higher-order moments yield novel contrast.

Two-dimensional Scan
To illustrate our microscope's operation and its performance, we started with a two-dimensional scan of a 100 μm × 100 μm area on a 3 mm × 3 mm chip carrier sample with total scan points of 200 × 200 at a 0.5 μm step size along both X and Y. Final images are obtained by summing the 0 th order image moment M 00 over a ROI of N × N pixels on CCD 1 recorded images using Eq 2. The Nyquist-Shannon sampling theorem requires at least two pixels per resolvable element. We choose a square pinhole of 2 × 2 pixels as the smallest to start with. Additional cropping sizes (N = 6, 11, 21, 31, 41, 51, 101) are used to determine the optimal pinhole size.
As a comparison, we used the fiber-optic module to represent the standard confocal signal detection. A 25 μm diameter multimode fiber was used to collect reflected light from the same field of view.
Before running an actual sample scan, a calibration process was carried out to ensure that both the fiber facet and the CCD are in a conjugate plane to the focal plane of the objective lens. Fig 2 shows the measured axial response curves of the fiber-optic detection and the CCD based detection. These relatively symmetrical plots share a common axial zero/focal point suggesting that they are optically aligned to the same conjugate plane. The fiber-optic response curve is relatively broader than those in the CCD plot up to 31 × 31. In the CCD plot, the width of the curve increases with the crop size. The full width at half maximum (FWHM) is 7 μm for N = 2, and 25.72 μm for N = 21. Beyond 31 × 31, a well-defined peak is not found.
An intensity map revealing the geometric profile of the sample from the fiber-optic confocal scan is shown in Fig 3. The CCD based detection results are shown in Fig 4. By manual control of the PI-611.2S piezoelectric stage to translate across the width of the top-right and bottom-left electrodes, a visual estimate of the electrode width AB and CD were obtained when laser spot vanished at the opposite edges in the inspection camera CCD 2 , i.e., AB, 28-29 μm, CD, 27-30 μm. In a more objective approach, a computer program was written to calculate the average width of the top-right and bottom-left electrodes, AB and CD, instead of a discrete set of hand-picked line pairs. It first calculates a threshold to completely separate the gold plated electrodes and markers from the substrate, and then computes the width of the electrodes above this threshold at each X or Y position. Thus, an array of width values is The diffraction limited spot diameter in the focal plane of the objective lens (Zeiss LD Plan-Neofluor 20 × /0.4 Corr) is 1.62 μm at 532 nm. It is projected by the microscope optics onto an area of diameter 39.56 μm on the CCD, which is enclosed by 7 × 7 pixels. The projected area on the fiber facet in the fiber-optic detection arm is of diameter 8.9 μm. Thus the 25 μm diameter multimode fiber core encloses 2.81 × Airy units (AU), which corresponds to an equivalent area of 20 × 20 pixels on the CCD. Case 2 × 2, which is at the size limit of point detection by a CCD, shows the least amount of deviations in the width measurement. Larger pinhole sizes, 6 × 6 and 11 × 11, shows slightly larger width values, but maintains comparable standard deviation values to 2 × 2 case. Case 21 × 21 shows slight noise increase. Above 31 × 31, the noise level increases drastically, and the resolution decreases as seen from the fact that the edge profiles of the bottom-right gold position markers become more and more rounded. Case 101 × 101, which corresponds to widefield imaging, shows the largest standard deviations and the highest noise level. The increase of noise in the larger pinhole cases can be attributed to the inclusion of more non-photon-receiving pixels subject to read noise and dark current noise outside the diffraction limited spot, The fiber-optic case shows larger numerical values due to its moderate core diameter size. In theory, an infinitesimal pinhole could give the best spatial resolution, but that small pinhole also rejects photons which could have been used to counter the noise. Case 6 × 6 gives the closest match to of the 2 × 2 case, and still maintains a sizeable amount of pixels. Thus we choose 6 × 6 as the optimal pinhole size for three-dimensional scans which require more complex analysis and the best performance, but 21 × 21 as the pinhole size for two-dimensional scans which requires a meaningful comparison between fiber and CCD experiments and less computational overhead.
In our experimental configuration, the total scan time was found to only depend on the detectors' readout speed. The CCD and the photodiode were configured to read out sequentially within each programming loop. A high-precision timing routine determined that the readout time for each point was 18.5 ms (740 seconds in total), i.e., 16.7 ms on CCD readout (60 fps) and 1.8 ms on photodiode readout. Since the photodiode's rise time is on the order of 1 ns, the 1.8 ms was the single point readout time of the LabJack U3-LV DAQ device through which the photodiode was connected to the computer.

Lateral Resolution Test
To better understand our microscope's performance, we ran a two-dimensional scan of a USAF resolution target (Ready Optics, California, up to Group 11, Element 6).

Higher Order Moments
In addition to the 0 th order image moment, higher order image moments are also used. Fig 8  shows the results of M 10 and M 01 normalized by M 00 for a 21 × 21 pixel pinhole size. These two quantities represent the (x, y) coordinate of the centroid of the image: The patterns shown in Fig 8(a) and 8(b) resemble the behavior of an edge filter. By looking at the color of stripe patterns, we found that the centroid show an inward-moving trend on the edge of stripes at the laser intensity level used in this experiment. More specifically, hxi increases on the left edge, and decreases on the right edge. hyi increases on the lower edge, and decreases on the upper edge. By carefully analyzing the line profile of intensity data, we found the spacing between peaks in Fig 8(c) and (d) match the designed width of a line-pair closely .  Fig 8(c) gives a line width of 1.91 ± 0.25 μm comparable to that of Group 8, Element 1 (line width of 1.953 μm), and a line width of 1.13 ± 0.25 μm comparable to that of Group 8, Element 6 (line width of 1.096 μm) . Fig 8(d) gives a line width of 1.56 ± 0.25 μm comparable to that of Group 8, Element 3 (line width of 1.550 μm), and a line width of 1.22 ± 0.25 μm comparable to that of Group 8, Element 5 (line width of 1.230 μm). This means, a higher order image moment analysis of the laser spot can reveal the slight shift of the centroid distribution, and provide a precise and straightforward determination of the dimensions of the pattern of interest without the need for a threshold. This method provides a view of how laser spot's internal intensity distribution changes when the laser overlaps a sharp edge. Fig 9 shows another example of higher order moment application. In this case, extra details of the feature on the electrode surface are brought out, compared to the 0 th order image moment case and the fiber-optic confocal case, and the edge profiles are more refined. Overall, image-moment analysis leads to an enhancement of the reconstructed image.

Image Differences
Besides the above mentioned techniques, by calculating the difference between recorded images of adjacent points on the sample, a more detailed plot of the edges can be obtained. Fig  10 shows the result by doing subtraction of images from adjacent points along X and Y directions. An orientation-dependent pattern is obtained, which can be used to refine the boundaries of the acquired image by aforementioned image moment method. The algorithm to compute the difference between points (m+1, n) and (m, n) along the X direction is given by Diff m;n ðXÞ ¼ sqrtð likewise, the difference between points (m, n+1) and (m, n) along Y direction is given by Diff m;n ðYÞ ¼ sqrtð where N × N is the size of the image (in this case, N = 6).

Three-dimensional Scan
In order to evaluate the three dimensional scan performance, a piece of photolithographyetched silicon wafer was tested. A total volume of 100 μm × 100 μm × 30 μm was scanned with a step size of 1 μm for all XYZ on this geometrically simple object. Fig 11 shows the reconstructed sample images of the fiber-optic confocal scan and the CCD based confocal scan. A pinhole of 6 × 6 pixels was used in the CCD scan for optimal result. The height value for each XY coordinate was determined by the location of maximum intensity or maximum image moment M 00 along the Z direction. Then a topographic surface was reconstructed using these height values. Both images show that the etched surface is lower than the un-etched surface by a certain depth value, and is also grainier.
To determine this depth value, a computer program first computes a threshold value to completely separate high ground and low ground, and then calculates the average height value for each of the two surfaces and their standard deviations. The final average depth is the subtraction of the two height values, and the sum of the two standard deviations gives the combined standard deviation. The numerical results of three scans are shown in Table 2 and in Fig  12. Each scan took about 1 hour and 30 minutes, or 18.5 ms per point. Overall, the CCD based scan gives result in agreement with the fiber-optic confocal scan within experimental uncertainty. The CCD scan generates a final image that is smoother than the fiber-optic confocal scan.

Summary
We have demonstrated a modular scanning confocal microscope using a CCD camera and image processing. It has been used to map out a two dimensional surface of a chip carrier and a three dimensional depth profile of an etched silicon wafer. The obtained dimensional  information was compared with that acquired by a fiber-optic confocal microscope module at the same time. The performance of this CCD and image processing based approach is on par with the fiber-optic module. However, the CCD has the advantage that its images can be analyzed in a variety of ways to enhance contrast. Additionally, a CCD does not require the precise alignment needed for an optical fiber or pinhole. In this work, we used a off-the-shelf monochrome industrial CCD camera which has a sensitivity of 0.015 lux. For low photon budget experiment, e.g., fluorescence measurement, choices of more light-sensitive scientific camera are available in current market, such as EMCCD and sCMOS. In conclusion, our work provides an easy to use, fully expandable as well as affordable approach to custom build a scanning confocal microscope based on postproccesing image data. Future work includes adding spectroscopic imaging module, providing scanning photoluminescence measurement capability.