Fig 1.
Steps of our proposed tracking algorithm based on an input image sequence with erroneous segmentation data. After processing the image sequence through the tracking pipeline, the cells are tracked and segmentation errors are corrected. The node IDs in the tracking graph indicate the assigned track ID to the segmented objects.
Fig 2.
(a) shows a graph constructed from an image sequence with erroneous segmentation. Each segmented object is assigned an unique ID i. Nodes corresponding to a segmented object share the same ID i, however, depending on the node type these nodes are assigned to different time points t in the graph. We link segmented objects over a maximum time span of Δt = 2 frames by adding for each object node oi,t a skip node xi,t+1, which models a missing segmentation mask. The segmented objects are assigned to tracks by finding optimal paths—highlighted in black—through the graph. (b) visualizes how cell behavior and segmentation errors are modeled in the graph example (a). Annotations c(⋅, ⋅) on the edges are assigned edge costs. To model mitosis, edges which are connected to pairs of “daughter” nodes are pairwise coupled—highlighted in green.
Fig 3.
Extracted features to link segmented objects.
Shown are two correctly segmented objects at time point t and a single segmented object due to an under-segmentation error at time point t + 1. To calculate cost terms, for each segmentation mask i at time point t the mask centroid pi,t—shown as a cross –, a set of mask points —shown in a lighter shade—and a bounding box
—shown as a rectangle—are extracted. The Euclidean distance between the mask centroid pj,t+1 and the propagated mask centroid
is large, which can result in wrong links. The minimal Euclidean distance between the propagated mask centroid
and the set of mask points
, in contrast, is small.
Fig 4.
The tracking graph is modified by applying untangling operations (a) such that each track has at most one predecessor and at most two successors—to model cell division. Different combinations of untangling operations, however, all lead to valid tracking graphs (b). We model the problem of selecting a set of untangling operations as an optimization problem and choose the set of untangling operations that induces the fewest modifications on the graph—highlighted in green.
Table 1.
Information about the number of frames, tracks and cells of the CTC data sets.
Fig 5.
Simulated segmentation errors.
Shown is a raw image of the Fluo-N2DH-SIM+ 01 data set with corresponding ground truth segmentation masks and modified segmentation masks with simulated segmentation errors, highlighted with white arrows.
Fig 6.
Influence of the post-processing on Fluo-N2DH-SIM+ 01.
Scores of a single run are shown as circles, while + shows a CTC measure score averaged over N = 5 runs. Per run a fixed fraction of ground truth segmentation masks is modified randomly to simulate segmentation errors. “untangle” refers to the untangling step, which transforms the tracking graph such that each track has at most one predecessor and two successors, whereas “masks” refers to adding missing segmentation masks. Over lined post-processing steps indicate that the post-processing step is missing.
Fig 7.
Influence of the post-processing on Fluo-N3DH-SIM+ 01.
Scores of a single run are shown as circles, while + shows a CTC measure score averaged over N = 5 runs. Per run a fixed fraction of ground truth segmentation masks is modified randomly to simulate segmentation errors. “untangle” refers to the untangling step, which transforms the tracking graph such that each track has at most one predecessor and two successors, whereas “masks” refers to adding missing segmentation masks. Over lined post-processing steps indicate that the post-processing step is missing.
Fig 8.
Comparing tracking algorithms on Fluo-N2DH-SIM+ 01.
Shown are the CTC measure scores DET, SEG, and TRA of tracking algorithms on 2D data set Fluo-N2DH-SIM+ 01 when provided with the same erroneous segmentation data. Scores of a single run are shown as circles, while + shows a CTC measure score averaged over N = 5 runs.
Fig 9.
Comparing tracking algorithms on Fluo-N3DH-SIM+ 01.
Shown are the CTC measure scores DET, SEG, and TRA of tracking algorithms on 3D data set Fluo-N2DH-SIM+ 01 when provided with the same erroneous segmentation data. Scores of a single run are shown as circles, while + shows a CTC measure score averaged over N = 5 runs.
Table 2.
Run-times of tracking algorithms.
Run times of the tracking algorithms on 2D and 3D data sets when provided with perfect ground truth (GT) segmentation as well as when provided with erroneous segmentation data.
Table 3.
Cell Tracking Benchmark (CTB) results (6th CTC edition).
Top 3 rankings as team KIT-Sch-GE(2) in the overall performance measure OPCTB—average of SEG and TRA scores—are written in bold. The latest CTB leader board is available on the CTC website. State of the results: May 10th 2021.