Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration
Fig 3
ZephIR analysis workflow and results for tracking GCaMP fluorescence from neuronal nuclei in 3D volumes of freely behaving C. elegans.
A: Plot of mean distance (in similarity space) to the nearest reference frame vs the number of reference frames (left), and the first three median frames (maximum intensity projections of shape 200 x 512) recommended by ZephIR’s k-medoids clustering algorithm (right). The first three median frames clearly represent the three main postures that the worm cycles through as it crawls. B: Accuracy (higher is better) and precision (lower is better) vs the number of reference frames. Accuracy is measured as the average percentage of neurons correctly tracked, where a neuron is considered correctly tracked if the tracked coordinate is within the volume of the neuron as identified via a manual annotator. Precision is presented as the average RMS error between the predicted position and the manually annotated position of each neuron in pixels. Note that once the majority of the postures present in the data is well-represented by the first three reference frames, subsequent additions returns diminished improvements. Last data point shows ZephIR’s accuracy using 10 reference frames with 10 partial annotations across all frames (panel C). C: 10 neurons were randomly selected to be verified or corrected to serve as partial annotations. Traces of 5 of these neurons extracted using the initial ZephIR results with 10 reference frames (left), and those using verified true positions (right) are shown, along with 5 other randomly selected neurons. Traces are calculated as fold change of the ratio between GCaMP and RFP fluorescence of each neuronal nuclei over the baseline, where the baseline is defined as the ratio in the first frame. Tracking quality for these 10 neurons can also be seen in individual crops around the neurons averaged across all frames (sharper image of the cell at the center reflects better accuracy and precision in tracking). Note how the five unannotated neurons show improvements in tracking quality after the addition of partial annotations, exemplifying the effects of partial annotations on the unannotated neurons in the same frame. D: Neuronal activity traces from 178 neurons, extracted using results from ZephIR with 10 reference frames and 10 partial annotations in all frames. Traces are calculated as fold change of the ratio between GCaMP and RFP fluorescence of each neuronal nuclei over the baseline, where the baseline is defined as the ratio in the first frame. Behavior is shown in the ethogram below the heatmap. Trajectory of the worm (t = 0 at bottom right) is also colored with the behavior state at the time. Trajectory of the worm matches changes in behavior over time as expected, and many of the neuronal activity traces show strong correlation with behavior. E: Accuracy vs the number of reference frames for tracking 79 neurons in a publicly available dataset of freely moving C. elegans [14]. Since the spatiotemporal patterns in the data are similar to the previously tracked data, we can reuse the same parameters and follow the same procedure to track the 79 neurons in the head of the worm. Each volume has been centered and rotated but no further straightening has been done. Since this particular dataset has also been used to benchmark a number of recently developed algorithms, we may also compare ZephIR’s accuracy with Neuron Registration Vector Encoding (NeRVE) [14], fast Deep Neural Correspondence (fDNC) [11], 3DeeCellTracker [23], and CeNDeR [46].