Instrumentation

FLIO instrumentation has been described in detail elsewhere [17, 18]. Thus, only a short description is given here. A schematic of the FLIO instrumentation is shown in Fig 1A.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic of the FLIO instrumentation and the instrument response functions.

(A) shows a schematic of the FLIO instrumentation. A 473 nm pulse laser is fed into a scanning laser ophthalmoscope to excite the autofluorescence of the eye. The fluorescence emission is transmitted by a multimode fiber to a dichroic mirror (DM), which divides the fluorescence signal into two spectral channels: 498–560 nm and 560–720 nm. Hybrid photomultiplier tube detectors convert the fluorescence photons into electrical pulses, which are processed by a TCSPC device for each detector. A continuous wave (CW) infrared laser (IR) illuminates the fundus for online image registration. Blocking filters (BF) protect the detectors from excitation and infrared light. The FLIO instrument response functions are given in (B).

https://doi.org/10.1371/journal.pone.0131640.g001

The basis is a confocal scanning laser ophthalmoscope (cSLO, HRA-2, Heidelberg Engineering GmbH, Heidelberg, Germany). Fundus images (30°, 256x256 pixels) are recorded in high-speed mode at 8.8 frames/s. A pulsed diode laser with a wavelength of 473 nm (BDL-473-SMC, Becker & Hickl GmbH), a pulse width of approximately 70 ps (full width at half maximum) and a repetition rate of 80 MHz is fiber-coupled (single-mode) into the cSLO to excite the autofluorescence. The laser power in the corneal plane is circa 150 μW, well below the exposure limits set by the ANSI standards for durations of up to 8 h [19]. A multimode fiber collects the fluorescence photons and transmits them to filters to block the excitation light. A dichroic mirror (edge-wavelength 560 nm) splits the fluorescence photons into two spectral channels (498–560 nm and 560–720 nm), and there is one detector for each channel (HPM-100-40, Becker & Hickl GmbH). Each detector is connected to a time-correlated single photon-counting (TCSPC) device (SPC-150, Becker & Hickl GmbH). The TCSPC technique [9, 20, 21] generates time-, space- and spectrum-resolved fluorescence decay datasets.

The instrument response function (IRF) was measured using a 25 μM Eosin Y solution which additionally contained 5 M potassium iodide. The Eosin Y solution was prepared by dissolving Eosin Y powder (Sigma-Aldrich Chemie GmbH, Taufkirchen, Germany) in a small volume of dimethyl sulfoxide (DMSO) first. Then water was added to obtain a stock solution of 2 mM which finally was further diluted down to 25 μM using a solution of 5 M potassium iodide. The Eosin Y fluorescence can be excited from 350 to 500 nm and ranges between 450 and 680 nm with a sufficient intensity. Based on the reported Rose Bengal fluorescence lifetime of ca. 16 ps when dissolved in 5 M potassium iodide [22] it is reasonable to assume that Eosin Y, another Fluorescein derivative, shows similar characteristics which could be confirm by in-house measurements. Additionally, no differences in terms of shape and width were found in the IRFs based on the Eosin Y fluorescence in comparison to the IRFs measured using scattered excitation laser light. To measure the Eosin Y fluorescence based IRF, a flat cylindrical quartz cuvette with a detachable window and a volume of 90 μL (124–0.5–40, Hellma GmbH & Co. KG, Müllheim, Germany) was placed slightly tilted in front of the FLIO device. The acquisition time was set to 2–3 minutes which is comparable to typical measurements in volunteers. The IRF images showed no sign of spatial variation. The full width at half maximum of the IRF is 172 ps for channel 1 (498–560 nm) and 153 ps for channel 2 (560–720 nm). The IRFs of both spectral channels are shown in Fig 1B. All IRFs used in this work have similar properties as those in Fig 1B, including experiment 5.

Fluorescence of the Eye

To measure the fluorescence of the retina using the FLIO instrumentation discussed above, the excitation light has to pass through the lens. A crystalline lens emits a strong autofluorescence when excited at 473 nm [23], which lies mostly in the short wavelength channel (498–560 nm) of the FLIO instrumentation. The excitation pulse generates fluorescence in the crystalline lens before the pulse travels through the vitreous and reaches the retina. The fluorescence of the crystalline lens is isotropic and thus also reaches the retina. The reflectance of the human retina is very low (≤ 2%) as it absorbs most of the incoming light [24]. Further, the confocal properties of the instrument result in a much smaller contribution of the crystalline lens fluorescence reflected from the retina in comparison to the directly detected autofluorescence of the crystalline lens. Because of the large volume and the strong autofluorescence of the crystalline lens, the FLIO instrumentation is not able to suppress the autofluorescence signal of the crystalline lens entirely. The crystalline lens fluorescence will bias the approximated fluorescence lifetime of the retina, especially if the relative contribution of the crystalline lens becomes larger, e.g. in older patients with beginning cataract. The time shift tc caused by the distance d between the crystalline lens and the retina is defined as: (1) where n is the refractive index and c is the speed of light. For the average Gullstrand Schematic Eye [25], d is 22.2 mm (the center of the crystalline lens to the retina) and n of the vitreous is 1.3668. From these parameters, the 12.2 ps time resolution of the FLIO instrumentation is sufficient to resolve the resulting tc of 202.3 ps. The autofluorescence of the crystalline lens is visible as a shoulder in the rising edge of the fluorescence signal (see section Approximation of the Fluorescence Lifetime for One Pixel).

Binning

In FLIO, the spatially and time resolved TCSPC datasets are typically collected with an average number of 1000 photons per pixel in the macular region. This number is a compromise between signal-to-noise ratio (SNR) and acquisition time. If the number of photons is too low, e.g., for a multi-exponential approximation using three exponential functions, the photons of neighboring pixels are combined. This process sums up the decay signals for a square shaped window around each pixel and is called static binning. The edge length l of the window is: (2) where f is the binning factor. Static binning is effectively a moving average filter in the spatial dimensions. Often, a binning factor of one is used for multi-exponential approximation with two exponential functions, and a binning factor of two is used for multi-exponential approximation with three exponential functions.

The fluorescence intensity of the human retina is not homogeneously distributed. The optic disc is generally weakly fluorescent. High concentrations of macular pigment in the fovea centralis absorb more of the excitation light than is absorbed in the outer regions, and vessels block the excitation light, prohibiting retinal fluorescence from underlying tissue. Another important factor that can contribute to the inhomogeneous distribution of fluorescence is suboptimal alignment of the instrument to the patient. In darker regions with low fluorescence signals, static binning with a fixed binning factor may not collect enough photons for the chosen approximation model, resulting in inaccurate estimates of the fluorescence lifetimes. Choosing a higher static binning factor would collect a sufficient number of photons in darker regions, but at an unnecessary cost of spatial resolution in brighter regions. Furthermore, the number of pixels involved in static binning increases with the binning factor by a power of two, making high binning factors unfavorable.

We propose a new adaptive binning approach that overcomes the disadvantages of static binning by using a circular window centered on each pixel. The radius of the circle is iteratively increased until a threshold with a predefined minimum number of photons is reached. Thus, adaptive binning ensures that the required number of photons per pixel is collected at the highest possible spatial resolution. Fig 2 shows a comparison of static and adaptive binning for measuring the fluorescence intensity from a patient with diabetes mellitus. Specifically, it shows the effects of static binning with a binning factor of two (Fig 2G) and adaptive binning with a threshold of 100,000 photons (Fig 2K) for the same patient using identical color scaling. It can be seen that adaptive binning results in a much more homogeneous intensity distribution because each pixel possesses at least 100,000 photons. Furthermore, the intensity drop off at the borders of the image is no longer present. Ideally, the fluorescence intensity distribution after adaptive binning would be completely homogeneous. The loss of global structural information in the fluorescence intensity is irrelevant because the fluorescence lifetime is usually independent of the fluorescence intensity, as stated above. Consequently, adaptive binning has no relevant effect on the fluorescence lifetime distributions. The data in the photon histograms were analyzed using a multi-exponential model (Eq 3) with three exponential functions in combination with incomplete decay (Eq 9). The resulting fluorescence lifetimes as well as the figure of merit χ² (Eqs 10 and 11) are shown next to the photon histogram. The fluorescence lifetimes for the bright pixel differ between static (Fig 2H) and adaptive binning (Fig 2L) for less than 6%. In case of a dark pixel, the fluorescence lifetimes differ between static (Fig 2F) and adaptive binning (Fig 2J) for 17%–44%.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Comparison of static and adaptive binning.

TCSPC data obtained from a 39 year old diabetes patient is evaluated. The top row of the photon histograms displays the raw data (C, E), the middle row shows the photon histograms after static binning (F, H) and the bottom row is after adaptive binning (J, L). In the magnified insets (A, B), static binning uses all of the pixels inside the gray box, while adaptive binning uses the white pixels (note the different sizes of the insets). The small black square indicates the seed pixel for the binning. Static binning uses more neighboring pixels for each bright pixel than adaptive binning (right inset, B), while adaptive binning uses more neighboring pixels for each dark pixel than static binning (left inset, A). The left column (C, F, J) shows photon histograms for one pixel (black square in A) in a dark region of the intensity image, while the right column (E, H, L) shows photon histograms for one pixel in a bright region. The middle column displays the fluorescence intensity image (170 x 170 pixels, 59 x59 μm²/pixel) of the raw data (D), after static binning (G) and after adaptive binning (K). The greater loss of spatial contrast with adaptive binning yields a more balanced SNR in the photon histograms of the underlying binned pixels. The data in the photon histograms were analyzed using a multi-exponential model (Eq 3) with three exponential functions in combination with incomplete decay (Eq 9). The resulting curve (red) and the corresponding fluorescence lifetimes, the average fluorescence lifetime, the figure of merit χ² (Eqs 10 and 11) as well as the total number of photons are shown next to the photon histogram.

https://doi.org/10.1371/journal.pone.0131640.g002

Modelling of TCSPC Data

Different approaches can be used to describe the decay of the time-resolved fluorescence data [9, 26]. This work concerns the multi-exponential approach, the stretched exponential approach, the spectral global analysis approach, the layer-based approach and the lens-corrected approach, as well as modelling of incomplete decays, all of which are implemented in FLIMX.

The multi-exponential approach describes the data with a sum of the exponential decay curves: (3) where I is the time-dependent fluorescence intensity, IRF is the instrument response function, α is the amplitude, τ is the fluorescence lifetime, i is the index of the exponential, the asterisk denotes a convolution integral, and b is the background, e.g., from thermal noise of the detector or background light. This method seems beneficial for measuring the time-resolved autofluorescence of the human eye in different spectral channels to determine differences between patients suffering from early AMD and healthy controls [27].

It is not feasible to use more than three exponential functions in Eq 3 because of the extremely high number of photons required [28]. According to our experience, between 100.000 and 400.000 photons are required for a reliable modelling with three exponential functions. In FLIO measurements, usually 1.000–10.000 photons/pixel are acquired. Thus, the required number of photons/pixel has to be achieved by binning adjacent pixels using an approach described above. The number of fluorophores in the human eye is larger than three, because there are a number of known fluorophores, including lipofuscin / N-retinyliden-N-retinylethanolamin (A2E), retinal, advanced glycation end products, collagens, flavines and possibly nicotinamide adenine dinucleotide in its reduced form (NADH). Thus, alternative modeling approaches such as the stretched exponential have been introduced [16]. A stretched exponential is able to model the distribution of fluorescence lifetimes by introducing the stretching exponent β: (4)

Some fluorophores are present in one of the two spectral channels, while others might be present in both spectral channels. Because the fluorescence lifetime is independent of a fluorophore’s spectral properties, the fluorescence lifetime is identical in both spectral channels. To exploit this fact, a spectral global analysis for certain exponential functions can be performed. This method can improve the fluorescence lifetime approximation because more photons are used in the analysis. (5) where λ is the identifier of the spectral channel, and at least a single fluorescence lifetime is identical in both spectral channels, τ_i,λ1 = τ_i,λ2.

The layered structure of the eye causes fluorescence photons from structures that are farther away from the laser scanner ophthalmoscope to arrive later because of the additional travel time required for the excitation light and the fluorescence photons. To account for distance, Schweitzer et al. [29] enhanced the multi-exponential model by a time shift parameter tc, which is called the layer-based approach here: (6)

Stretched exponentials (Eq 4) and the layer-based approach (Eq 6) can be combined for different layers of stretched exponentials: (7)

Time-resolved fluorescence measurements of the crystalline lens show that a single exponential function is not sufficient to describe its decay behavior [30]. Instead of approximating the fluorescence lifetimes in a mixture of the crystalline lens fluorescence and the retinal fluorescence, the fluorescence decay of the crystalline lens can be directly embedded into the model. Therefore, the fluorescence decay of the crystalline lens I_lens has to be measured separately. As the distance between FLIO instrument and eye is different for the measurement of the crystalline lens for technical reasons, its fluorescence signal has to be shifted in time using tc_lens. The contribution of the crystalline lens fluorescence is modeled by α_lens. This approach is called the lens-corrected approach here: (8)

Often, the fluorescence signal does not decay sufficiently in the time period t_R between two laser pulses, which is the reciprocal of the pulse repetition rate. This so-called incomplete decay can be taken into account using an analytic approach [31] which does not correct the pre-excitation interval. Thus, a numerical approach is used here which also corrects the pre-excitation interval: (9) where I_C is the fluorescence intensity calculated according to Eqs 3–8 and j is the number of time periods to consider. The choice of j depends on the fluorescence lifetimes occurring in the sample, as the exponential components have to be decayed sufficiently in the j∙t_R time-frame. In FLIO with a t_R = 12.5 ns, a j = 1 is usually enough for the expected fluorescence lifetimes of up to circa 5 ns.

Approximation of the Fluorescence Lifetime for One Pixel

An example of a time-resolved fluorescence signal from a single fundus-pixel from a healthy volunteer, approximated using the multi-exponential model (Eq 3) and the lens-corrected approach (Eq 8), is shown in Fig 3. There are three main intervals in the fluorescence signal: the pre-excitation interval, the fluorescence rising edge and the fluorescence decay. Usually in FLIM, the shape of the excitation pulse, the detector response function, dispersion in the optical pathway as well as relaxation processes directly after excitation and immediately before fluorescence emission determine the rising edge. In FLIO, the autofluorescence of the crystalline lens also affects the rising edge, as discussed above. The autofluorescence of the crystalline lens is visible as a shoulder in the rising edge in Fig 3 (inset).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Example of TCSPC data approximated by the multi-exponential model and the lens-corrected approach.

TCSPC data obtained from a healthy volunteer is evaluated using adaptive binning (black) and approximated by the multi-exponential approach (left) using three exponential functions and the lens-corrected approach (right) using two exponential function and a separate measurement of the crystalline lens. The data are divided into three intervals: the pre-excitation interval (A), the fluorescence rising edge (B) and the fluorescence decay (C). The measured data and the multi-exponential model diverge due to the fluorescence of the crystalline lens in interval (B), which is magnified in the inset. The lens-corrected approach utilizes a scaled and shifted curve of a separate measurement of the crystalline lens to correct for the influence of the crystalline lens fluorescence in interval (B). For better visibility, the fluorescence intensity is plotted on a logarithmic scale. The fluorescence lifetimes of the exponential components (τ₁, τ₂, τ₃), the average fluorescence lifetime τ_m as well figure of merit χ² (Eqs 10 and 11) are shown next to the photon histogram.

https://doi.org/10.1371/journal.pone.0131640.g003

To approximate the fluorescence lifetime of the retina using Eqs 3–5, only the decay of the fluorescence signal is important. For Eqs 6–8, the rising edge of the fluorescence signal must be taken into account, in addition to the decay.

If the fluorescence lifetimes of a sample are sufficiently short, the pre-excitation interval can be used to calculate the background b. If incomplete decays are considered (Eq 9), it is often useful to include the pre-excitation interval data as well.

To approximate the fluorescence lifetime parameters, a global optimum characterized by the smallest possible figure of merit must be found. To quantify the figure of merit, the χ² error in its reduced form χ_r² is used: (10) where m is the number of time channels of the photon histogram, w(t_j) is the weighting in time channel j, I_M(t_j) is the number of measured photons, I_c(t_j) is the number of calculated photons using one of the models discussed above and p is the number of free parameters in the model. The ideal weighting is w(t_j) = ϭ_j², where ϭ_j is the standard deviation of the number of measured photons, which is the square root of the expected value for data with a Poisson distribution. To approximate the weighting, the Neyman approach [32] is used: (11)

In addition, other weighting approaches such as Pearson [32], fitted weighting [33] and Warren [15] have been implemented in FLIMX as the choice of the weighting is a critical factor [33].

To find the global optimum of χ_r², a minimization scheme is required. The summation of the exponentials is a linear operation. Thus, a linear minimization algorithm can determine the amplitudes as well as the background. The remaining parameters must be determined by a non-linear minimization algorithm. Generally, there are two types of non-linear minimization algorithms: stochastic approaches, such as the evolution strategies [34] and the particle swarm method [35], and deterministic approaches, such as the Levenberg-Marquardt method [36] and the Nelder-Mead simplex method [37]. Deterministic approaches are dependent on their initial solution and often cannot determine the global optimum. Stochastic approaches are based on random variation and can find the global optimum, but at the cost of a higher computational effort.

The following minimization algorithms are implemented in FLIMX: differential evolution [38], a particle swarm variant [39], and a modified version of the Nelder-Mead simplex method [37]. Any of them can be applied to a pixel-wise fluorescence lifetime approximation. The stochastic minimization algorithms deliver robust results but require 10 to 100 times more computation time. Thus, a stochastic minimization algorithm can be used, for example, to estimate an initial solution from a single fluorescence decay profile that is based on an integral of the fluorescence decay profiles across the image. In a second step, a pixel-wise approximation utilizes a deterministic algorithm, with this initial solution as starting point. Using this procedure and Eq 3 to compute the fluorescence lifetimes of an image with 256 x 256 pixels, 33 / 44 / 100 minutes were required using one / two / three exponential functions (dataset from experiment 3, 4.00 GHz Intel Core i7 4790K quad-core processor with 16 GB of memory).

All minimization algorithms implement parameter constraints to allow only positive values for the fluorescence amplitudes, the fluorescence lifetimes and the background. The stretching exponent is restricted to values between zero and one. Custom constraints can also be defined to exploit a priori knowledge, e.g., if a certain fluorescence lifetime is expected to occur in the sample.

Handling of Artifacts in the Fluorescence Lifetime Signal

Reflections in the optical pathway cause artifacts in the photon histogram of the fluorescence decay. The fluorescence signal decreases over time, starting at the excitation. A reflection of the fluorescence photons at two or more reflective surfaces/interfaces, e.g., at the surface of a detector or an optical filter, causes a sudden increase in the fluorescence signal at some point in the decay because the reflected photons travel an additional distance and, therefore, arrive at a later point in time. An example of a reflection artifact is shown in Fig 4. Usually such artifacts are eliminated by avoiding surfaces perpendicular to the optical pathway and by applying antireflection coating to surfaces and fiber ends. However, some residual reflection artifacts often remain. Thus, an algorithm to detect and remove reflection artifacts was developed. First, because the positions of the reflection artifacts in the time dimension are invariant for all pixels, all of the pixels of the image are binned into a single fluorescence decay, resulting in the best possible SNR. The algorithm then searches the gradient of the fluorescence decay for sections with a rising gradient to find the starting time of the artifact. The end time is approximated by adding three times the interval from the starting time to the peak time of the reflection to the starting time. This procedure was determined empirically using FLIO data. For data with longer fluorescence decays than usually occurring in FLIO the estimation of the end time would have to be adjusted. In Fig 4, the detected reflection artifacts are marked in red, with the sections of the rising gradient colored in dark red and the estimated decay of the artifact in light red. In contrast to Fig 3, there is much less noise in the fluorescence decay due to the binning of all of the pixels in the image. The detected reflection artifacts are removed by excluding the time intervals of the artifacts from all of the computations for the figures of merit for each pixel.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Reflection artifacts in TCSPC data after binning all of the pixels of an image.

TCSPC data after binning all of the pixels of an image from a healthy volunteer (black). The automatically detected artifacts caused by reflections in the optical pathway are divided into the rise of the artifact (dark red) and the decay of the artifact (light red). The latter is approximated as three times the width of the rise. For better visibility, the fluorescence intensity is plotted on a logarithmic scale. The insets show the magnified segment of the TCSPC data taken from a single pixel (gray). The results of a multi-exponential analysis (Eq 3) using three exponential functions are shown as orange (without removal of the reflection artifacts) and blue curves (with removal of the reflection artifacts).

https://doi.org/10.1371/journal.pone.0131640.g004

Improving Approximation by Exploiting Spatial Information

A number of factors affect the approximation of the fluorescence lifetime parameters when utilizing the model function and minimization algorithm discussed above. For example, noise due to a low number of measured photons can lead to unstable results in the approximation. An ill-posed model function, e.g., due to a higher number of excited fluorescent substances than employed exponential functions, can increase the ill-posedness of the problem. Furthermore, the minimization algorithm may be stuck at a local minimum and, thus, will not be able to find the global minimum. Spatial a priori information can be used to improve the approximation. In FLIO with a spatial resolution of circa 34 x 34 μm² per pixel, the fluorescence lifetime properties of a biological tissue are expected to change only moderately from pixel to pixel. Thus, the fluorescence lifetime parameters for a certain pixel should also provide reasonable initial approximation for its surrounding pixels. To model a pixel, a figure of merit may also be computed for the adjacent pixels to achieve a spatially smoother fluorescence lifetime approximation. The resulting is defined as the combination of the figure of merit according to Eq 10 for the central pixel and the k’th adjacent pixels fitted using the parameters from central pixel: (12) where n is the number of adjacent pixels.

Iterative Algorithm for the Treatment of Outliers

Because outliers in the fluorescence lifetime images may be present, a separate treatment of these outliers might be needed to improve the fluorescence lifetime approximation. To search for outliers after the fluorescence lifetime is approximated for all of the pixels of an image, a sliding window of a user-defined size is moved over the image. A window size of 7 x 7 pixels showed good results in practical tests. The window is moved through the images of the fluorescence lifetime parameters and the figure of merit. In each window, the central pixel is compared to the median of all of the pixels inside the window. If the relative difference between a pixel and the median is above a user-defined threshold, the approximation of this pixel is repeated using the fluorescence lifetime parameters from the pixel with the best figure of merit inside the window as initial values. The result of the repeated fluorescence lifetime approximation is accepted only if its figure of merit is better than the original approximation. The applied threshold depends on the application and the expected changes in the parameter space. Thresholds of circa 30 percent demonstrated good results in our applications. Fig 5 illustrates the outlier correction procedure. In the example, 259 outliers were detected and of those, 178 could be improved in one iteration of the algorithm described above. Multiple iterations of the outlier treatment could improve even more outliers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Example of the iterative algorithm for the treatment of outliers.

Comparison of a 75 x 75 pixels section (59 x59 μm²/pixel) of fluorescence lifetime τ₁ from the left eye of a healthy volunteer before (left) and after (right) correction for outliers. The color scaling of both fluorescence lifetime plots is identical. The 259 detected outliers are colored black in the middle subplot, of which 178 could be improved to generate the corrected image. To provide better orientation, a gray scale image of the fluorescence intensity has been added as an overlay to all three subplots. The macula is in the lower right corner of the image, where the shortest fluorescence lifetimes (red) occur.

https://doi.org/10.1371/journal.pone.0131640.g005

Average Fluorescence Lifetime and Spatial Filtering

For some applications, the average fluorescence lifetime τ_m is a good overview parameter. It can be derived from the fluorescence amplitudes and lifetimes: (13)

To reduce noise in the fluorescence parameter space, spatial mean- or median-filtering can be applied to the obtained images. A reasonable size for the filter kernel is either 3 x 3 or 5 x 5 pixels, depending on the amount of noise in the data. The statistics discussed in the section below are calculated using the filtered data.

Regions of Interest

Although visual analysis is important for a general judgment of the image quality, quantitative analysis is required for the detection of pathologic alterations. Dysli et al. [40] proposed to use the ETDRS grid introduced by the Age-Related Eye Disease Study group [41] to define standardized regions of interest (ROI). FLIMX implements the ETDRS grid (see the results sections of experiments 1 and 2 for exemplary applications of the ETDRS grid). Three concentric circles in the center of the macula and four radial lines at 45°, 135°, 225° and 315° compose the grid. The radii of the circles are related to the diameter of the optic disk of an average eye, which is 1500 μm. Specifically, the radius of the inner circle corresponds to 1/3 of the optic disk diameter, the radius of the middle circle is equal to 1 optic disk diameter, and the radius of the outer circle is equal to 2 optic disk diameters. Hence, these radii of the inner, middle, and outer circles are 500, 1500, and 3000 μm, respectively. Based on this grid, nine subfields are defined: central, inner superior, inner inferior, inner nasal, inner temporal, outer superior, outer inferior, outer nasal, and outer temporal. Users can also define a custom rectangular ROI. For each subfield of the ETDRS grid or a custom ROI, descriptive statistics including the mean, median, mode, standard deviation, variance and confidence intervals are used to quantitatively compare groups of patients. Histograms with user-defined class widths are also computed for the ETDRS grid subfields and custom ROIs.

Group Comparison

The Holm-Bonferroni method [42] is used for group comparisons. First, a histogram for a certain fluorescence lifetime parameter, e.g., τ₁, is computed for each volunteer. Then, for each class in the histogram, a two-sided Wilcoxon rank sum test [43] is performed on the patient data against the controls. This approach tests the null hypothesis that the data from the patients and controls are samples from continuous distributions with equal medians against the alternative that they are not, at a certain significance level. The p-values from the test are compared to a threshold th: (14) where s is the significance level and nC is the number of histogram classes for the fluorescence lifetime parameter. Histogram classes with p-values smaller than the threshold possess a significant difference in medians between the patients and controls. If such a significant difference is found for a histogram class, it can be used as a classifier. Subsequently, its specific cut off point as well as its sensitivity and specificity are determined. The Holm-Bonferroni method can then be applied to the full image or to a region of interest, such as a subfield of the ETDRS grid. In the latter case, only pixels inside the ROI are used for the histogram and for the statistics of the Holm-Bonferroni method. The Holm-Bonferroni method is applied to an exemplary group comparison in experiment 4.

Implementation

FLIMX was implemented in MATLAB (The MathWorks, Inc., Natick, Massachusetts, USA), mostly in an object-oriented programming style, as shown in S1 Fig. FLIMX includes a patient database and has the ability to group patients in studies. For each patient, single measurements and, if calculated, the corresponding results of the fluorescence lifetime approximation, both divided into spectral channels, are stored in the database. The backbone of FLIMX is a tree-like data structure, which stores studies and patients and handles disk access, as depicted in S2 Fig. The measured data are imported into FLIMX only once and are then saved internally. Auxiliary information, such as age, gender, disease state, can be stored for each patient and can later be used to define subgroups in a study. Patients can be copied or moved between studies. The parameters for binning, data modelling, minimization, constraints and computation are also saved by FLIMX, making it possible to trace the study settings from calculated results. Computational routines are separated into modules, which implement the methods described in this work. FLIMX’s fluorescence lifetime approximation requires the user to supply an IRF. The IRFs used in this work and the measurement procedure are described in the instrumentation section. Graphical user interfaces are available for all aspects of FLIMX, such as changing settings, altering the data structure, visualizing fluorescence lifetime parameters and statistics, as well as for access to export abilities.

A tool for generating synthetic, time-resolved fluorescence data is also included in the software package. It can be used for the simulation processes, e.g., for the construction of the fluorescence decays based on the fluorescence lifetimes of known fluorophores. In addition, a batch job manager is included, which allows for the assembly and batch processing of approximation jobs. The software can handle an arbitrary number of studies and patients with different parameter sets. The computation of the fluorescence lifetime parameters can be very time consuming, depending on the selected minimization algorithm. Thus, FLIMX is able to run the computation on different pixels in parallel, utilizing multiple CPU cores. A custom built backend allows for the user-transparent distribution of work units across multiple compute servers. Therefore, a slave process is started on each server, which scans a network folder for the work unit files generated by the FLIMX software. Each work unit can contain several pixels, which are computed in parallel by the slave process using up to 12 CPU cores (the current maximum number of parallel processes per MATLAB session). The results of the computation are saved in the shared folder, and FLIMX uses the results to assemble the images of the fluorescence lifetime parameters. File access is synchronized using semaphores. The CPU core activity and memory requirements for a typical distributed fluorescence lifetime approximation using the lens-corrected approach on a 256 x 256 pixel dataset, obtained from a healthy volunteer, are given in Fig 6A. The critical code paths of the fluorescence lifetime approximation have been optimized for processing speed. An overview of the CPU time spent on the different stages of the fluorescence lifetime approximation is shown in Fig 6B. A large amount of the computation time is not spent in specific stage of the algorithm, but on other things such as memory allocation, consistency checks, object creation, function calls and general MATLAB overhead. The benchmarks were performed on a 2.93 GHz Intel Core i7 940 quad-core processor with 12 GB of memory.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Profile of the CPU and memory requirements for FLIMX.

(A) shows the fractional CPU core activity for a quad core processor and MATLAB’s total memory consumption (red) during a distributed fluorescence lifetime approximation of a measurement. (B) shows a breakdown of the CPU time spent on the different stages of the algorithm (C).

https://doi.org/10.1371/journal.pone.0131640.g006

To analyze the approximation results, fluorescence lifetime parameters can be visualized in two- and three-dimensions. The three-dimensional view can be freely rotated. Arithmetic operations between different fluorescence lifetime parameters and different spectral channels enable an in-depth analysis of the approximation results.

Example Data

Experiment 1: Patient with Diabetes Mellitus

Fig 7 shows the fluorescence intensity of static and adaptive binning as well as the average fluorescence τ_m (Eq 13) and fluorescence lifetimes τ₁, τ₂ and τ₃ for both spectral channels. As the patient has no diabetic retinopathy, no pathologic changes are visible in the images. The amount of detected fluorescence photons is low in the lower left part of the image. This causes in case of static binning a prolongation of especially fluorescence lifetimes τ₂ in both spectral channels and τ_m in spectral channel 2. Adaptive binning shows the expected rather homogeneous fluorescence lifetimes around macular and optic disc.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Comparison of static and adaptive binning in a diabetes mellitus patient without diabetic retinopathy.

The images (149 x 169 pixels, 59 x59 μm²/pixel) of the fluorescence intensity, the fluorescence lifetimes τ_m, τ₁, and τ₂ are shown in the rows from top to bottom. The columns are static binning and adaptive binning for both spectral channels respectively. The fluorescence lifetimes were determined using the lens-corrected approach (Eq 8), with two exponential functions, ß set to 1 and a separate crystalline lens measurement. The color scaling is identical for the fluorescence intensity and the fluorescence lifetimes in each spectral channel for better comparison. The ETRS grid is drawn on each subplot for orientation. The low amount of detected fluorescence photons in the lower left part of the image causes a prolongation of especially fluorescence lifetimes τ₂ in both spectral channels as well as the average fluorescence lifetime τ_m in spectral channel 2 in case of static binning. The largest differences are highlighted by white arrows.

https://doi.org/10.1371/journal.pone.0131640.g007

Experiment 2: Healthy Volunteer

Fig 8A depicts the fluorescence intensity as well as the average fluorescence lifetime τ_m (Eq 13) and fluorescence lifetime τ₁ (Fig 8B) in spectral channel 1 for the different fluorescence lifetime approximation models. In the case of the stretched exponentials, only two exponential functions were used, compared to the three exponential functions used in the other cases. Furthermore, the fluorescence lifetime of a stretched exponential is not directly comparable to the other approaches because it must be interpreted in conjunction with the corresponding β.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Comparison of the different fluorescent lifetime modelling approaches in a healthy volunteer.

The fluorescence intensity as well as the fluorescence lifetimes τ_m (A) and τ₁ (B) in spectral channel 1 are shown (154 x 154 pixels, 59 x59 μm²/pixel). The color scaling of the fluorescence lifetime plots is identical in each subfigure. The ETDRS grid is drawn on each subplot, and the mean values are given for each subfield of the grid. The fluorescence intensity image is the measured signal before binning.

https://doi.org/10.1371/journal.pone.0131640.g008

The approximated average fluorescence lifetimes were relatively similar, regardless of the different models underlying the approximation. The stretched exponentials produced shorter fluorescence lifetime values in comparison to the multi-exponential, spectral global analysis and layer-based approaches but also showed very similar features, such as longer fluorescence lifetimes at the thicker blood vessels, the optic disc and the very low fluorescent spot at the upper left part of the image. The lens-corrected approach did not show such prolonged fluorescence lifetimes, except for the optic disc. Furthermore, the average fluorescence lifetimes of the lens-corrected approach were shorter and more homogenous in comparison to the multi-exponential, spectral global analysis and layer-based approaches.

The fluorescence lifetimes τ₁, typically the exponential function with the highest amplitudes, showed larger differences. Only the multi-exponential model detected longer fluorescence lifetimes in the optic disc. Interestingly, the results of the layer-based approach and the lens-corrected approach were very similar, although their average fluorescence lifetimes showed differences, especially at the locations of the larger vessels.

Experiment 3: Patient with Macular Hole

Fig 9 depicts the infrared image (Fig 9A), the average fluorescence lifetime τ_m obtained using static binning (factor two) and a multi-exponential model with three exponential functions (Fig 9B and 9C) as well as the average fluorescence lifetime τ_m obtained using adaptive binning (threshold 100,000 photons) and the lens-corrected approach with two exponential functions and a separate measurement of the crystalline lens (Fig 9D and 9E) of a patient with a macular hole. The average fluorescence lifetime is presented in a three-dimensional view (Fig 9B and 9D) beginning at a vertical cross-section through the fovea. The three-dimensional view of the average fluorescence lifetime (Fig 9B and 9D), in combination with a detailed illustration of a user defined cross-section in the image (Fig 9C and 9E), allowed for an in-depth analysis of the local changes in the patient. Cross-sections can be placed at any horizontal or vertical position of an image. Clearly, the average fluorescence lifetimes of the lens-corrected approach are much shorter in comparison to the multi-exponential model because the influence of the crystalline lens is eliminated. The missing retina tissue inside the macular hole results in a longer average fluorescence lifetime. Without lens-corrected approach, it is not possible to visualize the macular hole in the average fluorescence lifetime τ_m.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Overview of FLIMX’s visualization capabilities and comparison of static binning + multi-exponential model and adaptive binning + lens-corrected approach for a patient with macular hole.

(A) shows the infrared image of the fundus (256 x 256 pixels). The FLIO data are analyzed using a multi-exponential model with three exponential functions based on static binning (factor two) and by the lens-corrected approach with two exponential functions and a separate crystalline lens measurement based on adaptive binning (threshold 100,000 photons per pixel). A vertical cross-section through the fovea centralis at pixel 128 on the x axis is highlighted as a black line. (B) and (D) show the remaining fundus section (pixels 128 to 256 on the x axis, all y pixels) in a three-dimensional view of the average fluorescence lifetime τ_m in spectral channel 1 for static binning + multi-exponential model and adaptive binning + lens-corrected approach respectively. The average fluorescence lifetimes τ_m along the cross-section are shown in detail in (C) and (E). The average fluorescence lifetimes are shorter in (D) and (E) because of the eliminated influence of the crystalline lens.

https://doi.org/10.1371/journal.pone.0131640.g009

Experiment 4: Diabetes Patients and Healthy Controls

As stated above, the motivation of this experiment is to show the ability of our analysis chain to provide group comparison for groups of patients or volunteers because of the high inter-individual variability. Normalized histograms comparing the diabetes patients and controls for all of the fluorescence amplitudes and lifetimes, as well as for both spectral channels, are shown in Fig 10. A shift to longer fluorescence lifetimes in the diabetes patients is clearly visible in all fluorescence lifetimes and both spectral channels. Significantly different populated histogram classes were found for all of the fluorescence lifetimes, except for τ₂ in spectral channel 2. No significant differences were observed for the fluorescence amplitudes. For each histogram class with the highest significance level, the corresponding receiver operating characteristic (ROC) curve is given next to the histogram. The area under the ROC curve (AUC), as a measure of the ROC curve’s accuracy, achieves the largest value of 0.85 for fluorescence lifetime τ₁ in spectral channel 2. Thus, its cut-off point as best trade-offs between true positive rate and false positive rate would result in the best achievable diabetes detection using only the FLIO data from this experiment.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Results of the Holm-Bonferroni method applied to FLIO measurements to allow for group comparisons between diabetes patients and controls.

The normalized histograms of the fluorescence amplitudes α and lifetimes τ in both spectral channels are obtained from a multi-exponential approximation using three exponential functions, for controls (blue) and diabetes patients (red). Histogram classes with significant differences, according to the Holm-Bonferroni method, are colored in light gray. The class with the highest significance level (the smallest p value) is indicated in dark gray. Only the fluorescence lifetimes showed significant differences, except for τ₂ in spectral channel 2. For the class with the highest significance level, the corresponding receiver operating characteristic curve (orange) is shown next to the histogram. The cut-off point as best trade-offs between true positive rate and false positive rate is colored in light blue. The AUC is given under the ROC curve.

https://doi.org/10.1371/journal.pone.0131640.g010

Experiment 5: Ganglion Cell Layer in a Porcine Retina Sample

The fluorescence intensity and the average fluorescence lifetime τ_m for spectral channels 1 and 2 are shown in Fig 11. In spectral channel 1, the ganglion cells possess much shorter average fluorescence lifetimes (700 ps–900 ps) than their surroundings. In spectral channel 2, the cell bodies also possess shorter average fluorescence lifetimes (450 ps–650 ps) than their surroundings while the cell nuclei possess the shortest average fluorescence lifetimes (350 ps–450 ps).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Fluorescence intensity and average fluorescence lifetime of the ganglion cell layer in a porcine retina ex vivo sample.

The 256 x 256 pixels images (34 x34 μm²/pixel) of the fluorescence intensity before binning (left) and the average fluorescence lifetime τ_m (right) of the ganglion cell layer in a porcine retina sample are shown in two spectral channels (top: 500–560 nm; bottom: 560–700 nm). Adaptive binning with a threshold of 10,000 photons per pixel was applied. A multi-exponential model with two exponential functions was used to determine the fluorescence lifetimes. The length of the white bar is 20 μm.

https://doi.org/10.1371/journal.pone.0131640.g011