Fig 1.
Method parameter optimization and evaluation.
Simulated SMVs can be applied to evaluate data analysis algorithms (left) and for comparison with experimental data (right). In the former application, MC simulations are performed to generate a set of SMVs characterized by defined VSPs, followed by analysis using the algorithm to be evaluated. The results (output parameters of the method) are then compared to the input parameters of the simulation to quantify the performance of the algorithm. Method specific parameters (MPs) are varied to maximize the agreement between input and output parameters in order to reach maximum accuracy and efficiency (see Section 4 for further details). Using pre-optimized parameter sets to analyze experimental data (application) yields reliable results of the molecular system under study.
Fig 2.
Schematic of smFRET data simulation with TIR illumination and camera detection as implemented in the MASH-FRET simulation tool. A large number of experimental or video simulation parameters can be independently set, ensuring the flexibility of the simulation tool.
Table 1.
Default values for the simulations carried out with the MASH-FRET simulation tool.
Fig 3.
(A) Example of simulated PSFs with wdet,0 = 1 pixel (top) and wdet,0 = 2 pixel (bottom) and their appearance depending on their subpixel localization. (B) Representative averaged SMV featuring randomly positioned SMs and an inhomogeneous illumination profile wex,x,0 = wex,y,0 = 256 pixel.
Fig 4.
Representative experimental camera noise of an Andor iXon3 DU 897D camera, following EMVA Standard 1288 notation [42]. (A) Standard deviation of camera noise as determined from single pixel temporal intensity fluctuations over 100 frames of single Cy3-labeled RNA with EM gain = 300, tbin = 100 ms and a readout rate of 10 MHz. Excitation intensities were varied to yield mean signal intensity rates between 0 and 6000 image counts per frame. Fitting with Eq (13) yields: K = 57.7 IC e-1, μic,dark = 113 IC e, σd = 0.067 e and σq = 0 IC (B) SNR characteristics of the dark count corrected intensity signal in comparison to an ideal image sensor. The camera units (image counts) were converted into photon counts according to Eq (12). (C) Histogram of the experimentally observed image counts with closed shutter (dark image) for the characterization of CIC noise. Pixel intensities were collected from L = 100 video frames (512×512) using the same settings as in (A). Fitting with the NexpN model in Eq (9) resulted in μoe = 1069 IC.s-1, ACIC = 0.02 and τCIC = 205 IC s-1. PGN noise model parameter (50000 samples): μph = 0 pc, CIC = 0.02 e, others as determined in (A). N noise model parameter (50000 samples): μph = 0.02 pc (D) Histogram of experimentally observed image counts of a single time trace (1000 frames) of Cy3 labelled RNA. PGN noise model (50000 samples): μph = 85 pc, CIC = 0.02, others as determined in (A). Parameters of the NexpN noise model in Eq (9) and the Normal distribution in Eq (10) are chosen in the same way as for the PGN noise model.
Fig 5.
MP Optimization for single molecule spot detection.
(A) Classification criteria: Black filled circles mark the ground truth (GT) of simulated SM coordinates (provided as VSP). Dotted circles illustrate a user-defined tolerance radius used for classification of detected SMs (open circles) into I) true positive (TP, blue assigned by black arrow), false positive (FP, red) and II) false negatives (FN, green) based on closest distance to a GT coordinate (greedy approach) and being located within the tolerance radius. Example III) and IV) compare a greedy and Gale-Shapely based approach where both show the same detection of SM but different classifications. For details, see text. (B) A sub-image of 256×30 pixels from a 512×512 pixels video with a total of 24 × 12 single molecules. The total emission intensity (given in photon counts per frame) decreases from the left to the right. White circles mark the GT coordinates, blue, red and green mark TP, FP and FN, respectively. (C) Optimization of ISS algorithm parameterization for SM detection using SMV category (i) (Table D in S1 File) with wdet,0 = 1 pixel. (C, left) Variation of model parameters (MP): 35 combinations of ISS input parameters Ithresh (intensity threshold) and NhoodSize (spot size) and their obtained color-coded recall, precision, and accuracy are shown for molecules of 40 pc total emitted intensity Itot,0. The respective range for a maximum (= 1) in recall, precision and accuracy are indicated by red bounding boxes. (C, middle and right) Variation of input parameters: Heat maps represent optimal parameterization of Ithresh,opt (middle) and NhoodSizeopt (right) to achieve maximum recall (1st row), precision (2nd row), and accuracy (3rd row) as function of PSF size wdet,0 and SM total emitted intensities Itot,0. Note that Itot,0 is varied from 1–300 pc within a SMV, while wdet,0 varies from video to video (category (i), Table D in S1 File). The corresponding example parameter optimization results for Itot,0 = 40 pc and wdet,0 = 1 pixel on the left are highlighted by grey squares in all heatmaps.
Table 2.
Spot detection algorithms tested.
Fig 6.
Evaluation of single molecule spot detection after MP optimization.
SM detection was performed with four different algorithms, ISS, HP, Sch and 2tone, using simulated SMV of category (i) (Table D in S1 File) with and wdet,0 = 1 pixel varying molecules total emitted intensity Itot,0 form 1 to 300 pc. (left) Recall, precision, and accuracy values of exemplarily chosen Itot,0 values. (right) Ranking of the four algorithms for recall, precision, and accuracy.