Theory

Under stationary conditions, the photons stemming from light emitting molecules, such as those used in fluorescence imaging, are described by Poisson statistics. This statistics is controlled by a single parameter—the number of emitted photons per time unit. Moreover, sums of random numbers drawn from a Poisson distribution will again yield Poisson statistics. For a detector exposed to light from multiple sources, the number of incoming photons is therefore also described by a Poisson distribution.

The first step in the EMCCD camera response is converting photons to electrons. The number of input electrons n_ie approximately follows a Poisson distribution, such that , with the Poisson parameter λ = QΛ + c, which is determined by the average number of photons Λ during the exposure time, the quantum efficiency Q, as well as the extra spurious charge c generated in the conversion process.

In the next step, the electronic signals are typically amplified with a certain gain through many electron multiplication steps. Each of these steps introduces noise, and the number of output electrons, n_oe, after these steps is usually modeled by a Gamma distribution with shape parameter n_ie and scale parameter g, such that [11] (1) where g is the electronic gain factor, Γ(⋅) is the gamma function and is the Kronecker delta-function.

The electrons are then converted to a digital image count n_ic by the readout process. This process is usually modeled by a normal distribution, such that (2) where f is the analog-to-digital conversion factor, r is the readout noise and Δ is an offset which is added to ensure that n_ic > 0.

Finally, the image count is rounded to the nearest integer to get the rounded image count n_icr. The error introduced in this step is commonly assumed to be described by a uniform distribution [14]. Note that rounding was not included in the study by Ryan et al. [16].

The set of distributions above determine the probability mass function (PMF) for the rounded image count, n_icr, in terms of the Poisson parameters λ and the chip parameters, chipParams = {g, f, r, Δ}. We here make use of the characteristic function, CF (the Fourier transform of the PMF) for n_icr. As we show in our derivation (see Sec. S1 in S1 Text), this CF takes a simple analytic form: (3) The PMF for n_icr (here referred to as the EMCCD-PMF) is obtained by the inverse Fourier-transform of Eq (3), see Sec. S2 in S1 Text for details.

The characteristic function given in Eq (3) can, through Taylor-expansions in p, be used to calculate cumulants and moments to any order. For instance, the mean is (4) and the variance is given by (5) Note that this variance includes the readout noise and the rounding effect (the factor 1/12).

For EMCCD devices one can also turn off the gain (electron multiplication steps). For this case (see Sec. S1 in S1 Text), we instead have: (6) while (7)

Camera parameter estimation

For our unsupervised probabilistic image thresholding method (see next two subsections), we need the four camera specific parameters, i.e. chipParams = {g, f, r, Δ}, as input. We here show how to determine these parameters using a set of calibration experiments. To this end, we adapt the image stacking method [11] as described below. Note that the calibration needs to be done only once for a given camera setting.

We first generate experimental calibration data (see Sec. S3 in S1 Text for experimental details). To that end, we record a stack of images of (non-moving) fluorescent beads with the gain set to its lowest setting or turned off, and the illumination intensity ranging from 0% to 100%. We then record another image stack at a specific gain setting. For each pixel (here labeled by j) we then have a time series of image counts. From these time series we compute sample estimates for the mean and for the associate variance S^2(j) for each pixel.

In order to extract the camera parameters, we next perform a mean-variance analysis of the experimental data. For the no-gain case, Eqs (6) and (7) yield a mean-variance relation (8)

For the case with gain, Eqs (4) and (5), give the mean-variance relation (9) Note that both expressions above yield a linear relation between the variance and the mean. To extract the parameters we then fit straight lines to a variance versus mean plot for the experimental data, i.e. to the dataset consisting of . In this step, we removed 40% intensity value for the gain = 100 setting, as well as bottom and top 1% of pixels from each experiment to reduce the number of outliers. The slope of the linear fit, , to the no-gain data gives us an estimate (see Eq (8)). The slope of a linear fit, , to the data obtained with gain, given an estimate (see Eq (9)). Additionally, the intersections at and gives estimates of Δ and r, (10) To estimate errors in the parameters, we use a random sub-sampling of the data into sub-sets of 1000 pixels. As the output we calculate median and interquartile ranges for each of the parameters. As an alternative to random subsampling one could use bootstrapping [21] of the original data.

Estimating the background illumination strength, λ_bg

With the camera parameters, chipParams, estimated once and for all, we are set to perform photophysical image analysis. To this end, we assume that the background pixels in the image at hand are described by a single Poisson parameter λ_bg. For such an image, we estimate λ_bg using a procedure which makes no assumptions about the signal-pixel intensity distribution in the image.

In our method for estimating λ_bg, for an image which contains both background and signal pixels, we fit a truncated EMCCD-PMF to the lower part of the image count histogram with λ_bg as a fit parameter (chipParams are known). The lower part of is here defined as all image counts below an optimal truncation point (threshold), . This truncation point is chosen such that the lower part of the histogram contain image counts with an overwhelming majority coming from true background pixels. The remaining part of the histogram then contains “outliers” and consists of counts from signal and remaining (uncertain) background pixels. In order to determine , we combine maximum likelihood estimation of λ_bg for truncated data with a χ² goodness-of-fit tests at several different truncation points. The quantity is then the largest truncation point for which we obtained an acceptable fit.

The details of our procedure are:

• Sort the image counts for each pixel, thereby yielding a sorted image count list:
  . Here, m is the total number of pixels.
• Estimate λ_bg using maximum likelihood estimation applied to the truncated data for a set of truncation points (lowest is set to 25% of data). In this step, we form the joint truncated probability , where the PMF is obtained by numerically evaluating the inverse Fourier transform of the characteristic function in Eq (3), the CDF is the cumulative sum over PMFs (see Sec. S1 and S2 in S1 Text) and θ = {chipParams, λ_bg}. In the truncated MLE procedure, we set the initial estimate as (see Eq 4).
• Calculate the goodness-of-fit χ² score for each of the truncation points. In this procedure we divide the data into five unique image count intervals, based on quantiles for the fitted PMF, to keep the number of degrees of freedom in the test (see below) the same for each truncation point.
• Set as the largest truncation point passing the χ² test (with one estimated parameter) [22] at a significance level set by a p-value threshold p_GoF. In here, we use p_GoF = 0.01.

As the output from the procedure above, we obtain an estimate for λ_bg, but also obtain an intensity threshold, , such that the truncated distribution fit is good (i.e., we are “certain” that the image counts below originate from background pixels).

Note that our procedure implicitly assumes that the image is acquired under uniform illumination (so that λ_bg does not vary over the image region of interest).

Unsupervised probabilistic image thresholding

Image thresholding (binarization) is based on setting a threshold for the recorded image counts. Pixels which have image counts are turned “black” and pixels with an image count are turned “white”. As we illustrate in S9 Fig (panels c and d) in S1 Text, two common unsupervised thresholding approaches, the Otsu method and adaptive thresholding, for automatically determining can yield uncontrolled results.

The main novelty of our new thresholding method, as detailed below, is that we provide a priori error estimates through two conditional probabilities: p(black|s)—the probability that an actual signal (s) pixel is deemed black at a given intensity threshold, and p(white|bg)—the probability that an actual background (bg) pixel is deemed white at this threshold. In Sec. S6 in S1 Text, we relate these two quantities to commonly used statistical measures such as the false positive rate (FPR) = fraction of incorrectly classified background pixels, the false negative rate (FNR) = fraction of incorrectly classified signal pixels, the accuracy (ACC) = fraction of correctly classified pixels among all the pixels, FDR = fraction of false positives among positives and FOR = fraction of false negatives among negatives.

Our unsupervised probabilistic image thresholding method uses as input a predetermined p-value threshold, p_binarize (i.e. predetermined value of the “wanted” FPR in the thresholded image). To convert p_binarize to an intensity threshold , we use the full PMF for the background photons (including camera-induced noise), see previous subsection. From this PMF, we can calculate the probability that an actual background pixel (bg) would yield an image count which exceeds (and hence be deemed to be a white pixel) as: (11) By setting the left-hand-side above = p_binarize and inverting, we estimate the image count threshold at which the probability (p-value, p_binarize) is no more than, say, 1%. Applying the threshold onto the image at hand then yields a black and white (binarized) image, where the error rate p(white|bg) is known a priori.

Given the image binarization threshold , we can also estimate the other error rate, p(black|s). To achieve this task, we first notice that the number of background pixels n_bg in the image can be determined through: (12) for . Hence, by plotting (# pixels with an image count ≤ N_j) as a function cdf(N_j| θ), we extract n_bg as the slope of a fitted linear function. The total number of signal pixels is then estimated through n_s = m − n_bg, where m is the total number of pixels in the image, as before. The number of black pixels among the background pixels is: (13) The number of black pixels in the binarized image is and hence the number of black pixels among the signal pixels is (14) As a consequence, we finally arrive at the probability that a signal pixel is deemed black in the binarized image: (15) The +1 and +2 terms in the numerators and denominators above are introduced in order to avoid zero probabilities [23].

Image segmentation

The goal of the image segmentation is to identify “objects” (in here, fluorescently-labeled lung cancer cell nuclei or fluorescent beads) in an image. These objects often correspond to contiguous regions of pixels, which one seeks to identify and to classify as signal (s) or background (bg) regions. In here, to keep things at a general-purpose level, by an “object” we refer to a single cell/bead, or to a cluster of touching cells/beads (separating touching objects would this would be a post-processing step which will require knowledge of the geometry of the object).

Let us now, to illustrate how our image binarization result can potentially be used in downstream image analysis tasks, introduce a heuristic image segmentation method. This is not a main novelty of this study, but included in our software package as an example application.

Our unsupervised image segmentation method assumes that erroneously classified pixels in our binarized image (see above) have a short-range spatial correlation. This in effect forces false background and false signal regions to be “small”. Our method makes use of one parameter, allowedGapLength (set = 1 in all examples) and is:

1. Find connected components of white pixels in the thresholded image (white regions) and connected components of black pixels (black regions).
2. Flip all the pixels in the white regions smaller than w_white and flip all the pixels in black regions smaller than w_black from the binarized image. By “flip” we refer to the operation of turning a white pixel black, or turning a black pixel white. Our procedure for obtaining the two thresholds, w_black and w_white, is described below.
3. For each region, we calculate a p-value, p_seg, based on the sum of image counts for all the pixels in a region, see Sec. S1 in S1 Text for the characteristic function used to estimate the p-values for each of the regions. This p-value serves as a quality control of the segmentation.

To determine w_black in step 2 above we find all black regions and sort these according to size (with the smallest regions size first). We then proceed through the sorted list and stop when the gap between the present region size (w_black) and the next region size exceeds allowedGapLength. The procedure for determining w_white is analogous. For the case when there is no gap larger than allowedGapLength we keep no connected components/objects.

Note that our method makes no assumption about the geometry or topology of the “objects” in the image. For instance, in case the objects would have “doughnut” shapes, our method should be able to detect the holes in these objects (however, we do not have any such objects in our examples in Results).

Estimating the camera chip parameters using calibration experiments

As a pre-processing step we need to determine the four camera chip parameters, gain g, analogue-to-digital unit conversion factor f, read-out noise parameter r, and camera offset (bias) parameter Δ (unless these are known from a previous calibration). To this end, we perform linear fits of the variance in image counts as a function of mean image counts for three different gain settings (see Methods for details). This procedure yields parameter estimates (median and the interquartile range) as reported in Table 1. Note that the accuracy (here quantified by the interquartile ranges) of the estimates can be improved by using a larger set of calibration experiment at different illumination intensities, or using an MLE approach [16]. It is, however, not a main aim of this study to provide a new method for estimation of the camera parameters, and we therefore settled at the current accuracy. Note that the data from differing gain settings yield consistent estimates of r and Δ, as it should. Since the chip parameters, except the gain, are static properties of the chip, these experiments only need to be performed once.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Estimates of the chip parameters for an EMCCD camera at different gain settings.

The calibration data shown in S1 Fig in S1 Text was used, and the chip parameters were calculated using Eqs (4)–(7) in Materials and Methods.

https://doi.org/10.1371/journal.pone.0300122.t001

Estimating the Poisson parameter for the background

The results obtained from our novel method for estimating the Poisson parameter of the background (λ_bg) directly from an input image which contains both background and signal is shown in Fig 1. As input we use an experimental image of 250 nm diameter fluorescent polystyrene beads adsorbed onto a surface, where the image was split into several tiles. Panel a) shows an example of such an image acquired with the settings labeled “Gain 100” in Table 1. In panel b), we show image count histogram for one of the tiles in panel a). In order to estimate λ_bg from this histogram, we fit a truncated PMF to the data at several different truncation points. For each truncation point, we run a goodness-of-fit test and choose the optimal threshold, , as the largest truncation point which passed the tests, see Methods for details. Our estimate of λ_bg is the value obtained by the fit to the data at a threshold . In the histogram, blue bars represent intensities below (“certain” background pixels). The orange bars represent the outliers, i.e. image count values which are not certain to be background (remaining background or signal pixels). Our procedure yields an estimate λ_bg = 22, i.e., in this particular tile on average 22 photons are incident on each background pixel during the exposure time (assuming quantum efficiency = 1, and neglecting the spurious charge, see Methods). Notice that our novel procedure automatically makes estimates of the statistical properties of the background, although the image contains both background and signal pixels. The fitted EMCCD-PMF, extended to the full image count range, is given by the black curve in Fig 1(b).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Estimating λ_bg for an image which contains both background and signal.

(a) Experimental fluorescence microscopy image of fluorescent beads. Here, the image is split into tiles of size 64x64 pixels, where each tile is given a label {row, column}, where in this example row, column = 1, …, 8. The contrast is set to better display the background noise and glass slide artifacts. (b) Estimating λ_bg: A histogram of the image counts for a single tile, here tile {4, 1}. The blue bars represent pixels regarded as true background, while the orange bars represent the outliers (not true background or signal pixels). The image counts threshold, , separating the blue and orange bars was determined using a p-value threshold, p_GoF = 0.01, for the goodness-of-fit tests. The dashed black curve shows the fitted PMF for the estimated λ_bg, extended to the full range of image counts (in our method, we fit a truncated PMF to the blue bars). In S5 Fig in S1 Text, we provide the estimates of λ_bg and for all tiles in panel a). In S8 Fig in S1 Text, we also provide two more examples of image count histograms with overlaid fits. Examples of fits to histograms for synthetic images at varying SNR is found in Sec. S7 in S1 Text. A major novelty of our method is that we are able to estimate λ_bg for a arbitrary fluorescence image even though the image contains signal pixels.

https://doi.org/10.1371/journal.pone.0300122.g001

The reason that we split the image into tiles is that the illumination intensity varies slightly over the image at hand, thus violating the model assumptions from Methods. By tiling the image we get roughly constant illumination over each tile, thereby making sure that the model assumptions are satisfied and, as a consequence, most of the tiles pass the goodness-of-fit p-value threshold, see S5 Fig in S1 Text. We leave it as a future challenge to extend our approach to situations of strong non-uniform illumination.

In the experimental image in Fig 1, the vast majority of the pixels are background pixels and the signal-to-noise ratio is rather high. To show that our procedure works also for smaller SNR, S4 Fig in S1 Text show examples of fits to histograms for synthetic images at varying SNR, where we demonstrate that our procedure for estimating λ_bg works also for smaller SNRs.

Unsupervised probabilistic image thresholding

The estimates of the camera parameters and our automatically determined λ_bg allow us to perform unsupervised probabilistic image thresholding, as described in Methods. To quantitatively test out our image thresholding approach (see Methods), we apply it to synthetically generated images of fluorescent beads. Our procedure for generating such images is described in Sec. S4 in S1 Text. The SNR in these images is defined [24], such that where λ_sig is the Poisson parameter for a signal pixel. Since we use synthetic images, we know the ground truth, i.e., which pixel is a background pixel and which is a signal pixel, and can use this information to estimate the quality of the thresholding. To this end, we use five observables: FPR, FNR, ACC, FDR, FOR. The main novelty of our method is that we can estimate all of these rates a priori, i.e., even if the ground truth is not known.

Our thresholding (binarization) method uses a p-value threshold, p_binarize (not to be confused with the p-value threshold used in the goodness-of-fit test, p_GoF) which controls the FPR. The results for the FPR, FNR, 1-ACC, FDR and FOR are shown in Fig 2 where we compare to our a priori predictions (see Methods) and to the Otsu method. Notice that with our probabilistic thresholding method, we have excellent control over the FPR and, unlike the Otsu method, can predict the other statistical observables a priori. The details of our procedure for a priori estimation of the FPR, FNR, 1-ACC, FDR and FOR is described in the Methods section and in Sec. S6 in S1 Text. The success of our unsupervised probabilistic thresholding method is quantified by the good agreement between the blue marks (binarization at a threshold set by p_binarize vs ground truth) and the orange curves (a priori estimates).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Photophysical image binarization.

Synthetic images of fluorescent beads were generated as described in Sec. S4 in S1 Text at different signal-to-noise ratios (with known values of λ_bg and λ_sig). For these images we know the ground truth pixel identity, i.e., which pixels are background and which are signal. (a) Estimated λ_bg compared to the ground truth value. (b-f) Our image binarization method was applied to the synthetic images (with binarization threshold p_binarize). The result was then compared to ground truth pixel identity, with success rates quantified through five statistical observables (blue). The orange marks correspond to our a priori prediction of the same observables (obtained without the knowledge of the ground truth). We also show Otsu’s method compared to ground truth pixel identities (red). In our method we have excellent control over the FPR, i.e., the fraction of white pixels in background regions. We also have good control over the FNR. Particularly important to notice is that our a priori prediction (orange) for all statistical observables agree very well with ground truths (blue). This agreement is the strength of our novel thresholding method, which hence opens up for unsupervised image thresholding where the error for the classification for each pixel is obtained a priori. The image count threshold for the binarization was determined through a p-value threshold p_binarize = 0.01.

https://doi.org/10.1371/journal.pone.0300122.g002

In S9 Fig in S1 Text, we applied our image thresholding method to the experimental data from Fig 1. Compared to two common unsupervised thresholding approaches, the Otsu method and adaptive thresholding, it is visually clear (no ground truth is here available) that our method gives good control over the FPR.

Image segmentation

The first application of our unsupervised image segmentation method is the image in Fig 1. The result is shown in Fig 3(a), where we see that, visually (no ground truth is available here), our method performs very well. Our segmentation method uses the binarized image in S9 Fig (panel b) in S1 Text as input. For the image in Fig 3(a), the p-values for the regions, p_seg, were all below 0.01.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Photophysical image segmentation.

(a) Beads image with detected regions (connected components of white pixels). The original image can be found in Fig 1a). The yellow pixels are boundary points to the detection regions. The binarized image used as input to our segmentation approach is found in S9 Fig (panel b) in S1 Text (b) Lung cell nuclei image with detected regions. The original image can be found in S7 Fig in S1 Text (a). Segmenting larger object with lower magnification is more challenging, since the background illumination will vary. We here tile the image and set p-value thresholds p_GoF = 0.01 and p_binarize = 0.01 and region size gap threshold = 1. An example of the associated image count histogram with an overlaid fit is found in S7 Fig in S1 Text. Notice that our image segmentation method performs visually well on both experiments.

https://doi.org/10.1371/journal.pone.0300122.g003

We next apply our image segmentation framework to more challenging images, namely images of stained lung adenocarcinoma A549 cell nuclei. These images are taken with a lower magnification (20 X) and the gain setting “Gain 100” in Table 1. At this magnification, the illumination varies across the field of view, violating our assumption of constant background. In addition, the non-uniform staining of such cell nuclei, as well as their varying shapes and sizes, should make the image harder to segment. Despite these challenges, our method, after tiling of the image (see Materials and methods), visually performs very well, as the segmentation results shown in Fig 3(b) illustrate.

To illustrate that our method is insensitive to the geometry of the objects, S10 Fig in S1 Text shows segmentation results for a synthetic image with objects of more complicated shapes.