Merged detection.

The average distance between objects becomes shorter for the sake of dense distribution. In that case, occlusion and merged detections would happen more frequently. Tracking algorithms that ignore such a phenomenon can lead to poor performance in multi-object tracking.

Prior work on dense small object tracking with merged detections is sparse, but there are some other works facing the similar problem. (1)One is about unresolved measurement [14, 15], which basically is a resolution problem rather than our segmentation problem. They need to model resolution capability of the used sensor while we don’t. Actually it’s very difficult to get the reliable description of the resolution capability of a given sensor. [14, 15] consider the problem with only two objects, there is no guarantee of such conditions in our application. (2)Severe occlusion produces merged measurement in visual tracking [16, 17], where detection results of pedestrians are bounding boxes. Visual tracking usually handles fewer objects while small object case is denser. Besides, rich appearance information is a very important cue for visual tracking to distinguish objects, but small object is featureless. So, it’s improper to directly employ the above-mentioned approaches for our case.

To tackle the merged detection, we do the followings: (1)For detection, we propose multi-appearance local contrast segmentation(MALC) to detect and clearly segment merged objects. MALC employs the multi-threshold idea and achieves one threshold for one object. (2)Most MHT approaches mainly utilize one-to-one association and neglect merged detection. To deal with merged detections during tracking, we employ one-to-many association to count for the occlusion caused by merged detection, namely that one detection can associate with multiple tracks. Extended 0-1 programming is proposed to achieve our one-to-many association.

High complexity.

In fact, high complexity is an inherent problem of MHT, especially when dealing with a large number of objects. Dense small object presents exactly such situation. Cluttered background and occlusions cause large ambiguity of detection. Moreover, to tackle some small objects that would change speed rapidly, it is necessary to apply a loose gating threshold. These facts significantly boost the number of hypotheses in MHT.

Although some efficient approaches had been proposed [9, 10], they still can’t achieve a good tradeoff between performance and efficiency for the special scene with dense small objects.

We approach the problem of complexity from two aspects: (1)batch optimization: we find that in MHT a considerable part of the computation is spent on compatibility processing, which accounts for the compatibility between hypotheses and is performed at every time step. In fact, such frequent processing may not be totally necessary, because the compatibility relations won’t change for every time step. Instead, it remains similar for a short period. Appropriately extending the time interval between two consecutive compatibility processes could be beneficial to reduce the computation. (2)hypothesis pruning: batch optimization will increase the hypothesis number, so we design and impose two-stage pruning technologies to control the growth of hypothesis. Meanwhile, to improve the quality of hypothesis, we also propose a new motion score to evaluate the likelihood of hypothesis, which helps for small object tracking.

Batch optimization and new pruning technologies complete each other. Hence, the frequency of compatibility processing is reduced while the scale of hypothesis is still under control.

Intention.

We provide a simple example in the Fig 1, which indicates that different thresholds for segmentation produce totally different results. As the Fig 1 showing, the low threshold can not distinguish objects and noisy points accurately, which produces merged detection. High threshold would lose some objects. The appropriate threshold should change for the different objects in image.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. An instance of multiple segmentation slides.

In a1, some objects are merged into a single object. As for a3, the segmentation with high threshold loses the sight of certain object.

https://doi.org/10.1371/journal.pone.0206168.g001

Each segmentation of the object demonstrates one of its appearances, which contains the contour information under current segmentation and can be interpreted as one slice measurement. From one slice measurement, we can obtain the corresponding object score according to its appearance. Then, we will use those appearance scores to choose the appropriate segmentation. Sequential layers of appearance and affiliated connections together form a multi-appearance tree. Using the example in Fig 2, we explain the affiliated relationship between the layers. As shown in Fig 2, the multi-appearance tree is built to count for the exclusion relation between selections of thresholds for different objects. For instance, the candidate combination for threshold selections in the tree with root node S1_3 can only be one from {S1_3}, {S2_2, S2_3} and {S3_2, S2_3}. That is based on the constraints indicated by the tree structure that one can’t select the S1_3 and S3_2 simultaneously, since the threshold for a local region is unique. The critical problem in segmentation is to select the most appropriate threshold for each object from all the layers. We introduce Eq 3 as the criteria for segmentation. The goal is to minimize ∑S_appearance, i.e. sum of S_appearance for all segmented objects.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Multi-appearance structure with tree relation.

https://doi.org/10.1371/journal.pone.0206168.g002

MALC method.

MALC is composed of three stages: For the first stage, the gray image is transformed into the corresponding map using Eqs 2 and 1. In the second stage, a multi-level binarization is performed on the transformed image and the tree structure is built based on the multiple binary images. Finally, during the third stage, we introduce a deep first segmentation selection(Alg. 1), which is executed for each multi-appearance tree.

For the first stage, we use the same Eqs 2 and 1 as C.L.P.Chen’s work [24]: (1) (2) where MI_n is the mean intensity of the neighbor area of I_n. The neighbor area of one pixel is the region between outer square windows and inner windows with current pixel at center. The outer and inner width are set to 9 and 3, respectively. N_u is the number of pixels in the neighbor area, and I_j is the gray level of the jth pixel in neighbor area. I_n represents the gray level of the central pixel and C_n is the final correspond value. Unlike original approach, we remove the minimum search and keep the sharp edge of object.

Secondly, we calculate the mean and standard deviation of C_n. The global threshold is the sum of mean and K times of standard deviation. Using global threshold as the reference middle value, n layers’ thresholds are set with equal interval (n and the interval are pre-defined parameters). For each threshold, the corresponding binarization is performed. The tree with the relationship between objects from adjacent layers is built.

Algorithm 1 Deep first segmentation selection

Require: n_root

Ensure: S_root

1: S_{child_score_sum} = 0

2: Calculate S_appearance according to Eq 3.

3: for each n_child satisfying that n_child is child node of n_root do

4:  S_{child_score_sum} ← S_{child_score_sum} + Deep first branch adjustment(n_child)

5: end for

6: if S_{child_score_sum} < S_appearance then

7:  S_root ← S_{child_score_sum}

8: else

9:  mark n_root as candidate node

10:  S_root ← S_appearance

11: end if

For the third stage, we introduce Eq 3, which provides each node in the tree with an appearance score, to evaluate a segmented object. We define the appearance score as the product of three score components, namely intensity, shape and bubble punishment terms: (3) (4) where S_intensity evaluates the object by the consideration that standard deviation of intensity will maintain higher stability inside the separated object. (5) (6) (7)

We use S_shape to measure the impact of object geometric shape. Accurate segmentation should impel the object contour to be smooth and regular. S_shape is defined as the variance of distance between edge pixel and object center. O_edge is the set of edge pixels, and n is the number of edge pixel. d_{x, y} is the distance between a certain pixel in object and the object center(), is the mean of d_{x, y}. And a punishment factor S_{bubble_punishment} is used for the regularization: (8) where bubble, the non-detected pixels in the object, indicates inappropriate segmentation.

The goal is to minimize ∑S_appearance, i.e. sum of S_appearance for all segmented objects. We propose Alg. 1 to perform traverse and search for the optimal segmentation with minimum ∑S_appearance. Alg. 1 compares the score of current node with the score sum of children to choose the proper segmentation (current level or deeper child level). Then, those chosen nodes are marked. Finally, only those top marked nodes, with no other marked nodes in the paths between them and root, are used to compose the optimal segmentation. We use a breadth-first search algorithm to pick out the top marked nodes.

Tracking framework.

The tracking framework is illustrated as the working flow diagram in Fig 3 where hypothesis generation and selection are performed. Hypothesis generator produces the candidate hypotheses of every tree, which are sent to the hypothesis selection component as input data. Then, extended 0-1 programming would determine the set of preserved hypotheses according to the compatible relationship among hypotheses. Extended 0-1 programming is a ‘soft’ method with more flexibility to handle complex occlusion circumstances. In other words, it relaxes the constraint when treating some incompatible tracks.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. The proposed tracking framework.

https://doi.org/10.1371/journal.pone.0206168.g003

Hypothesis generation and management.

We integrate some technologies for hypothesis generation and management. Since our tracking method is designed to be of batch optimization, 0-1 programming will not be executed at every frame but over a larger span. Thus, the hypothesis tree will grow to an uncontrollable scale if no special restriction measure is employed. The main function of the proposed technologies in this part is to decrease the number of selected hypotheses while preserving reliable ones.

In this section, we first present the mathematic expression form for tracking description. Then, along with the pipeline of hypothesis generation and management in Fig 3, we introduce our new score definition, gating and pruning technologies.

Tracking description. Assuming a sensor scans the surveillance region periodically, the set of detections received at frame t is denoted by D(t): (9) where N is the number of frames, is the ith detection received at frame t, and N_t is the number of detections received at frame t. In addition, a dummy observation is defined for each frame t to denote possible missed detections. A track hypothesis (we would use hypothesis for short in following content) at frame t is defined as a sequence of observations: (10) (11) (12)

This definition constitutes a restriction that one track can contain at most one detection at a particular frame. is the set of hypothesis tracks with all the in it possessing same root(in one connected tree). O^t is the set of , which represents all tree hypotheses at frame t.

Score based on autoregressive model of acceleration. Firstly, we defined S_AA to represent an autoregressive model of acceleration, which would be used for score test gating. (13) (14) where the following notations are used:

depth of hypothesis j, which will not exceed the depth of practical hypothesis tree.

score of leaf hypothesis j at frame t for autoregressive model of acceleration;

the velocity of at frame t assumed association between and is built;

detection i at frame t;

prediction deduced from the ;

mahaldist(⋅): function to get mahalanobis distance.

The velocity and prediction mentioned above are acquired through correction of Kalman filter. After transforming , we can get following autoregressive formula of , where l is the frame number. The initial is set as zero, so the constant term of this autoregressive formula is zero. (15) (16)

The coefficient a_l decays along with the l, and motion information from recent time will contribute more.

Score test for association gating. For each leaf node in hypothesis tree, a is calculated and maintained for every time step. Then, we use the Eq 18 to filter out(or called gating) unsatisfying hypotheses before branch growth. A_ij = 0 means the corresponding association won’t pass gating. (17) (18) (19)

th_n: multi-stage threshold for score test;

A_ij: association filter mask between detection i and hypothesis j, A_ij = 1 indicates permission of association between detection i and hypothesis j;

N_S: the number of sustaining frames;

α, β, γ, δ: parameters for multi-stage threshold.

We define S_ST to describe the strength for a track to maintain its previous motion pattern, which is based on the assumption that absolute acceleration of a track will change very slowly in a certain degree. MD_{m, P} reflects the difference between prediction velocity and practical one, namely absolute acceleration. As Eq 15 shows, is the previous weighted acceleration. |MD_{m, P} − S_AA(T_j)| is the absolute difference between previous weighted absolute acceleration and current one, which reflects the deviation between current movement and expected one. Eq 18 would reject a hypothesis with sudden and big acceleration change. Therefore, our score testing is capable to capture the coarse pattern of movement. Multi-stage threshold th_n is controlled by parameters α, β, γ and δ, which is a polyline as in Fig 4. The loose threshold at beginning facilitates the start of new object because new object may be unstable and come with high S_ST. When object trajectory becomes stable, following small threshold helps to retain robust objects and discard unreliable ones.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Illustration of score testing.

Motion score used here is S_ST. The fold line above the score curve is our multi-stage threshold th_n.

https://doi.org/10.1371/journal.pone.0206168.g004

We use Fig 4 to illustrate score test. Threshold th_n is the polyline with two stages. Once the S_ST crosses the threshold line, the corresponding hypothesis will be abandoned. For the motion score of T1 (the curve in Fig 4(a) with frequently changing motion pattern), three potential hypotheses are given. The diagram of the corresponding motion score is drawn in the right side. H1 and H2 are rejected since these two smooth tracks deviate far away for its original winding motion pattern. In Fig 4(b), H2 is distinguished from T2 by its sudden change of movement while H1 follows previous smoothness.

Hypothesis score. To indicate track’s likelihood, each hypothesis is assigned with a hypothesis score, which is used for hypothesis selection and pruning. The score of hypothesis j at frame t is defined as follows: (20) where and are the scores of hypothesis j in the consideration of long-term motion and short-term motion respectively. ω_LTM and ω_STM are the weights for long-term motion and short-term motion.

is the original score formulation [8] in traditional MHT. We introduce to capture the short-term motion and to enhance the sensitivity of for rapid motion variation. The original score is a slowly changing value and will increase with time for the potential track. After a long period of update, it reaches a rather high level and partly loses its sensitivity for motion change. Although some methods like SQRT employ a threshold to detect the change, this hard threshold measure can’t provide enough agile information to reflect short motion. indicates the likelihood of hypothesis from a long-term and overarching perspective.

Firstly, we revise , which uses cumulative log-likelihood ratio. (21)

If hypothesis is associated with detection at frame t, then the increment of hypothesis score is given by: (22) where the following notations are used:

vector of detection i at frame t;

p(⋅): the probability density function (PDF) of detection conditioned on the one-step prediction of hypothesis ;

P_D: detection probability;

λ_fa: the expected number of false alarms per unit volume of the detection space per frame(spatial density of clutter);

λ_nt: spatial density of new targets.

The initial hypothesis score is given as .

Then, we define as Eq 23. (23)

We use the ratio of S_ST to represent the short-term score. With S_ST, we design a short-term score according to the variability of recent acceleration, which assumes a constant acceleration model. Since we abandon those hypotheses with S_ST higher than th_n in score test, so th_n would be the upper limit of S_ST. can be roughly regarded as a likelihood rate(LLR) of S_ST toward acceleration smoothness. Then S_STM is the accumulation of this LLR, which reflects recent acceleration deviation trend. IC is the number of effective emergencies, which counts the times when a valid detection is assigned to the current object.

Hypothesis pruning. After score test, two stages of pruning are carried out to remove incompatible hypothesis.

The first stage is strong hypotheses pruning, which is performed between different trees. A comparison between two hypotheses, one with the highest score and the other one with the highest IC, is conducted. They are the historic best one and the temporary best one respectively. Hypotheses would be incompatible if they share too many detections in the recent period. We apply such operation because those wrong strong hypotheses could become more stubborn if not being handled at the beginning.

For the second stage, we use Alg. 3 to detect loop path and solve it by pruning low-probability hypothesis and retaining only one. Loop path means that two hypotheses start from same node and end with same node, but have different paths. This phenomenon may bring the exponential growth of hypothesis number if no special treatment is employed. Immediate pruning is performed in our algorithm so that unnecessary ambiguity is removed before development.

Algorithm 2 Pruning stage 1: Strong hypotheses pruning(inter-tree)

Require: O^t

Ensure: O^t

1: for each do

2:  for each satisfying do

3:   find with max .

4:   find with max .

5:   if and the number of shared detections exceeds tree depth then

6:    if then

7:     

8:    else

9:     

10:    end if

11:   end if

12:  end for

13: end for

Algorithm 3 Pruning stage 2: Loop hypotheses pruning(intra-tree)

Require: O^t

Ensure: O^t

1: for each do

2:  for each and do

3:   if and then

4:    if then

5:     

6:    else

7:     

8:    end if

9:   end if

10:  end for

11: end for

For inter-tree hypothesis pruning, hypothesis score is employed as the criteria. As for intra-tree pruning, S_AA is used, which is the proposed autoregressive model of acceleration.

Birth and death of hypothesis. Those detections that don’t link to any hypothesis would be regarded as the starts of new objects. We employ SQRT test [8] to terminate a hypothesis that has no linked detection for the update. We rank the terminated hypotheses at the last frame of a batch by their (Eq 20). Then we preserve only top 20 percent for hypothesis selection to reduce computation.

Hypothesis selection as an extended 0-1 programming.

When handling the merged detections in dense small object tracking, one-to-many association may be more appropriate than one-to-one. Most of past studies assume that at most one object is associated with each detection. As for the occlusion with approximative motion, two or more objects are detected as only one detection. As Fig 5 illustrates, if we insist the assumption of one-to-one association, big gaps may occur since some tracks lose necessary detections for association, which would influence the correct formation of tracks. In comparison, one-to-many association facilitates the complete formation of each hypothesis through the occlusion.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Two tracks with merged detections.

https://doi.org/10.1371/journal.pone.0206168.g005

Formula for one-to-many association. Track selection is naturally a binary linear programming problem in the view of using binary variable to control the selection of hypothesis. The traditional formula of one-to-one association (such as in [25]) is like following: (24) (25) (26) where is defined as the cost of hypothesis n. is a binary representation of T_i. To efficiently apply one-to-many association, we redesign an extended 0-1 linear programming with adjunctive binary variable I_ij as following formula: (27) (28) (29) (30) where I_ij represents the incompatibility between hypothesis i and j. I_ij will equal to 1 while hypothesis i and j own same detection. I_ij = 0 indicates that hypothesis i and j are totally irrelevant with each other. The constraints given by Eqs 28 and 29 ensure that while T_i and T_j are both selected, I_ij will be 1, otherwise I_ij equals to 0. represents the cost of incompatibility to select both hypothesis i and j.

We utilize adjunctive binary variables I_ij to count for one-to-many assignment. Without the I_ij, the problem would turn into a quadratic one. This is because it will be necessary to use quadratic term like to count for the flexible compatibility cost. It is much easier to solve a linear programming problem than a quadratic one. Besides, via the introduction of I_ij and , the exclusion restriction between incompatible hypotheses becomes roughly controllable. We slightly loosen exclusion restriction to encourage longer hypothesis formation.

Details of extended 0-1 linear programming. We use following criteria as the cost of hypothesis. (31) where is the score of leaf hypothesis T_n at frame t. K is used as adjustment coefficient. As for the cost of incompatible hypotheses couple, we use following formula: (32) where is denoted to be the number of incompatible detections between T_i and T_j. C_IN is a constant coefficient to count for incompatibility of single detection sharing. N_T is the threshold of tolerable maximum number. If exceeds N_T, a positive infinite value will be assigned to to indicate that at most one in T_i and T_j can be selected.

Then, after the building of 0-1 programming for each batch of frames, we use lpsolve to solve this problem. Owning to the rational assumptions and optimized constraints, solving progress is not complex. Detailed time consumption and analysis will be presented in the next section.

Segmentation methods.

• LCM: Using contrast based template operator would help to extract salient points, and also will be conducive to local optimization [24].
• Top-hat: Top-hat operation can extract small elements and details from image, which had been found truly useful for small object detection.
• MPCM: Multiscale patch-based contrast measure(MPCM) [26] is also a contrast measure-based method.

Tracking methods.

• Cox’s MHT: We use cox’s implementation [9] of MHT with default parameter settings. We also compared the speed of cox’s with our method when running on the same platform and implementing in the same language (C++).
• SGTS: The number of semi-greedy solutions generated before selection was set at 60.
• MDA: The duality gap of the termination criterion was set to 0.02. We defined the maximum number of iterations to 100. The Lagrange multiplier updating scheme applied was the heuristic price update, because it’s believed to be more efficient and can fully exploit the structure of the intermediate feasible solutions found by the Auction algorithm.
• GRASP-MHT: Greedy randomized adaptive search was applied in multi-object tracking by Murphey et.al [27] and Robertson et.al [28]. We used a MATLAB implementation from ren’s [10], where GRASP is used as an engine of hypothesis generation in the MWISP formulated TOMHT [10]. Parameters nv, np, and nitr, which were used to control the amount of computation in candidate construction, were set to 20, 20, and 3, respectively.

Larva.

Three segments of video data with movements of micro-animal are used in this dataset. They were captured under different conditions. The object number in Larva_s2 is much higher than others. In Larva_s3, focal length changes for several times, namely focus drift. Image could suddenly become blurry in few frames, and detection failure happens more frequently.

We sampled the video images at a regular interval for experiments. The ground truth is produced partly based on the tracking results of the few tracking methods mentioned before. The videos used for pre-designation are of full frame without down sampling, so it’d be of less ambiguity because more information is provided to fill the uncertain gap between time steps. Using those results as the reference we checked and corrected the trajectories manually, especially focusing on key frames where occlusions or false alarms happen. The ground-truth trajectories are presented in Fig 6b, 6c and 6d.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Ground-truth trajectories.

(a)Verti_Hat (b)Larva_s1 (c)Larva_s2 (d)Larva_s3.

https://doi.org/10.1371/journal.pone.0206168.g006

Verti_Hat.

With a vertical view, around 60 persons with hats walk around in this pedestrian video. After adding disturbance and removing some detections randomly, this scene becomes considerably difficult. This video comes from [10]. We intercepted the first 600 frames of the pedestrian video, then down sampled it at 5Hz as the author did. Finally, 120 frames of images were used in this scenario test. Unlike only 60 frames used in [10]’s experiment, we used twice longer(not frequency) sequence to produce more fair evaluation.

Scenario Verti_Hat.

The results are presented in Table 2. Our tracking method outperforms others in the principal metrics such as OSPA-T, MOTA and IDSW. OSPA-T is a critical metric in the traditional tracking scenario. Our tracking method achieves better performance with near 20% decreasing in OSPA-T compared with the second method, i.e GRASP-MHT. Besides, lower IDSW is a remarkable feature of our method, which can be noticed in the following experiment results as well.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Performance comparison between our tracking method and others methods for the Verti_Hat dataset.

https://doi.org/10.1371/journal.pone.0206168.t002

Scenario Larva.

Larva tracking is a difficult problem with dense object in the clutter background. We collected three sequences with different qualities of image to represent different tracking conditions.

The final row in Table 3 is the weighted average of metrics from three sequences, where for most metrics the weight is the corresponding frame number divided by the total frame number. As for IDSW, FN and FP, we use the sum of three sequences. For this scenario, our method outperforms other methods and ranks first in 6 of total 10 metrics. The recall figure of our method is at an ordinary level, since we try lots of efforts to reduce the number of hypotheses, which may restrict part of correct hypothesis.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Performance comparison between our tracking method and others methods for the Larva dataset.

https://doi.org/10.1371/journal.pone.0206168.t003

We present part of trajectories in Fig 8. Each line of the pictures shows the tracking results in a sequence clip, including two frame images(last frame and twentieth from the end) and a global projection graphic displaying all trajectories. Object ID is marked on the image. Only last 30 frames of trajectories are drawn to keep the picture more readable. From top to bottom, four lines of pictures indicate the Verti_Hat,Larva_s1,Larva_s2 and Larva_s3 respectively.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Results on the datasets Larva and Verti_Hat.

Each line of the figure shows the tracking results in a sequence clip, including two frame images(last frame and twentieth from the end) and a global projection graphic displaying all trajectories. From top to bottom, four lines of pictures indicate the Verti_Hat,Larva_s1,Larva_s2 and Larva_s3 respectively.

https://doi.org/10.1371/journal.pone.0206168.g008

Impact of one-to-many assumption.

By using different constraints for track selection, we also provide an experiment to test the effect of one-to-many assumption. The results of flexible association (one-to-many) and hard association (one-to-one) are listed as Table 4. One-to-many constraint boosts the performance notably according to some critical metrics, where OSPA-T decreases by 22% and MOTA increases by more than 20%. Meanwhile, some metrics appear moderate degeneration such as IDSW because of descent recall. Even though, the one-to-many constraint is effective according to the obvious improvement on critical metrics.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Performance comparison between our method and its variation under one-to-one constraint for the Larva_s2 dataset.

https://doi.org/10.1371/journal.pone.0206168.t004

One-to-one association may refuse some correct hypotheses and produce worse performance in dense situations. In the complex dense scenes, some strong hypotheses encounter long occlusion or merged detection as in Fig 5, where competition for detection occurs. One-to-one association will reject a strong hypothesis that loses in competition while the major part of this hypothesis is correct. Usually, if a tracker could not track an object with consistent ID, it may still produce some small fragments and redeem part of trajectories. Unfortunately, we already removed those weak and short hypotheses at first to keep the number of hypotheses in a tractable scale. Thus, performance will degenerate if we reject objects with small flaws. In comparison, with a looser constraint one-to-many association could preserve the practicable strong hypothesis.

We only provide moderate relaxation for one-to-many constraint. Only a limited number of sharing detections is permitted.

Impact of pruning.

We evaluate the impact of the proposed pruning technologies on video Verti_Hat by comparing the number of hypothesis per time step for three profiles, namely removing all pruning stages, using only stage 1 and using all pruning stages. As Fig 9 shows, the proposed pruning technologies(stage 1 and 2) significantly reduce the number of hypotheses. The result without any pruning has much higher and unstable hypothesis number, which sometimes rises to near 500 and certainly requires more computations. We also find that stage 2 (loop pruning) produces a better effect than stage 1 by pruning much more hypotheses, which indicates that loop detection based pruning has a significant impact in computation reduction.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. Hypothesis number per time step for three pruning profiles on Verti_Hat.

https://doi.org/10.1371/journal.pone.0206168.g009

Analysis of parameters.

To deal with the complicated tracking problem, we pre-define many parameters in our algorithm, which produces some difficulties to understand the characteristics of the proposed method. We evaluate the effect and sensibility of different parameters via the experiment presented in Fig 10. We only test on the parameters introduced by ourself. With respect to traditional MHT parameters like P_D, λ_fa, λ_nt, we use default ones as in [9, 10]. Since we set ω_STM + ω_LTM = 2, the result for ω_STM also indicates ω_LTM’s. We assess different parameters in terms of the MOTA score, OSPA-T.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 10. Sensitivity analysis for parameters on Larva_s1.

https://doi.org/10.1371/journal.pone.0206168.g010

As in Fig 10, our parameter setting(ω_STM = 1, ω_LTM = 1, K = 5, C_IN = 1, N_T = 5) basically ensure both high MOTA score and low OSPA-T, except for tree depth, where deeper tree produces better result. We choose 6 as the depth since deeper tree also requires more computation and memory. The results also show that our algorithm is less sensitive to the most parameter changes. For those parameters with less sensitivity(like K, N_T), it’s not necessary to precisely locate the highest or lowest point in the flat curve, we can roughly choose a value and the result is still totally comparable. Actually our method is more robust for less dependency on parameter setting.

Runtime comparison.

Our tracking method achieves comparable performance while maintaining a considerable speed advantage. Cox’s MHT algorithm possesses both well tracking capability and efficient implementation according to the study of Chanho Kim et.al [33], so we use it for our speed test. As the Table 5 shows, our tracking method is more than ten times faster than Cox’s MHT. The platform we used is a PC with E5(3.3GHz) CPU. The two methods listed in the table are implemented by the same language(C++) for a fair comparison. The fast implementation of our method is achieved based on the following factors: We screen out weak hypothesis in due course and preserve strong ones; Meanwhile, we employ batch optimization instead of compatibility checking frame by frame.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Consuming time(second) comparison between our tracking method and others methods for the Larva dataset.

https://doi.org/10.1371/journal.pone.0206168.t005