2.1 Tracking framework

Online tracking often refers to a tracking-by-detection framework, wherein objects are detected in each frame and linked to existing tracklets [1,3,14]. Kalman filtering [26] is widely used for motion prediction, whereas the Hungarian method [27] is usually used to address the data association problem. A cost matrix using Intersection over Union (IoU) or a mix of IoU [16] and appearance features is used to associate detections with existing tracks [28]. To address occlusions and crowded settings, multiple models include appearance similarity with motion cues. To reduce false positives and ghost tracks, various methods exclude low-confidence detections and associate only with high-confidence tracklets. Recent studies indicate that incorporating low-confidence detections associated with a second stage could recover missed objects and improve recall [29,30]. Multiple studies [31–33] have proposed multi-stage association frameworks that highlight more recently updated or higher-confidence tracklets during the first stages. Although successful, these multi-stage techniques may cause more identity switches due to inconsistent associations between stages. On the other hand, BoostTrack [18] and BoostTrack++ [17] use a single-stage association method. They use confidence boosting, Mahalanobis distance, and appearance similarities, such as visual embeddings and shape similarity, to achieve robust association. BoostTrack++ improves confidence modeling, generating superior outcomes.

2.2 Occlusion handling

Occlusion presents a continual challenge in multi-object tracking (MOT), often disrupting detections and fragmenting trajectories. Researchers have developed various methods to maintain track continuity in partial or complete occlusions. TrackFormer [22] uses a transformer-based attention mechanism to represent contextual and spatial relationships between objects. It collects long-range relationships that enhance occlusion resilience; however, it suffers from high computational demands because of huge data requirements. MeMOT [34] uses a transformer-based architecture, including memory encoding for simultaneous detection and data association. It is highly effective in crowded environments with recurrent occlusions. SMILEtrack [14] uses a Siamese Learning Module and Patch Self-Attention to improve feature matching in situations of occlusion. Nonetheless, its fixed similarity metric limits adaptability to varying occlusion levels, potentially reducing association accuracy over time. Occlusion-Aware Attention (OAA) [35] uses a self-supervised mask loss to reduce occluded background areas, therefore focusing the model’s attention on visible features. Although it improves robustness in dense environments, its detection-centric methodology neglects motion continuity, leading to identity switches. DetTrack [36] addresses full occlusion by combining spatio-temporal information with motion cues. It predicts object positions based on motion in missing detections, ensuring continuity without the necessity for explicit visual input. MSPNet [37] (Motion-guided and Occlusion-aware Multi-Object Tracking) integrates motion-guided aggregation with an occlusion-aware attention mechanism. Its hierarchical spatial association considerably improves performance in situations with frequent occlusion and camera movements. PSMOT [38] uses position-sensitive pooling to maintain spatial context within feature maps, boosting robustness against occlusion and lower identity changes in noisy environments.

2.3 Re-identification

Re-identification (Re-ID) ensures stable identity tracking across frames, especially after occlusion or missed detections. It uses visual, motion, or combined cues to accurately re-associate missing tracklets. FairMOT [21] integrates Re-ID directly into its joint detection and tracking (JDT) framework, producing embeddings simultaneously with detections, allowing real-time identity association. BoT-SORT [28] adopts a deep CNN-based model, such as OSNet, to extract distinctive appearance features, providing more resilience in occluded and crowded environments. BoostTrack [18] combines motion-based prediction with a simplified appearance model (e.g., SHIP) for hybrid association. It achieves outstanding efficiency without significant feature extraction, but it can fail when appearance or motion features lack distinctiveness. MOTFR [39] uses a Feature Purification Module (FPM) that eliminates identity-discriminative features via feature recoding, thereby providing accurate object tracking on visually complex occasions, including during occlusions. Posture-guided Re-ID methods [40] utilize human position estimation and spatial attention to coordinate visible body parts across frames. These methods can improve re-ID in scenarios with partial occlusion; nevertheless, they generally need large labeled datasets., hence raising complexity and training expenses. YOLO11-JDE [41] includes a self-supervised Re-ID component inside the YOLO detection framework. It uses triplet loss with semi-hard negative mining to acquire embeddings without identity-labeled data, enabling rapid real-time implementation. However, overall tracking accuracy remains relatively low. imprAsso [42] uses a deep Re-ID model to extract appearance features from each detection. The cosine similarity between embeddings allows accurate track association, even in an environment of occlusion. ConfTrack [43] uses a Re-ID model to produce embeddings for detections and tracks. It associates them through cosine similarity, modified by detection confidence for improved reliability. These methods fail in situations of heavy occlusion and lack the capacity for dynamic similarity adjustments. Adaptive Feature Fusion with Local High Discriminant Features effectively combines motion cues, global appearance, and detailed local features. By updating fusion weights according to the occlusion condition, it maintains excellent Re-ID performance even in visually ambiguous or crowded environments [44].

Despite their developing ability, several methods are impeded by high computational costs, restricting their use on resource-limited platforms. In addition, identity switches remain, eventually affecting key performance indicators such as HOTA, Multiple Object Tracking Accuracy(MOTA), and IDF1.

3.1 Novelty of proposed components

Although each component draws inspiration from existing MOT techniques, our design places strong emphasis on computational reduction and collaborative functionality for deployment under resource constraints. Specifically,

• LAE generates compact 32-dimensional embeddings, reducing computational cost by approximately 80% compared to conventional ReID encoders [28].
• TBSS employs a single-layer transformer with restricted attention computation, avoiding the overhead of multi-layer transformer architectures [22,34].
• CMOH maintains only the most recent features, ensuring identity continuity during short-term occlusions without expensive long-range memory modules.
• ASW enables real-time similarity fusion conditioned on occlusion density, in contrast to static or fixed similarity weighting [14].

These design choices achieve a favorable balance between robustness and efficiency that prior approaches have not addressed within a unified framework.

Each component provides distinct enhancements over existing techniques, focusing on computational efficiency and tracking robustness. The LAE uses MobileNetV3-Small to create compact 32-dimensional ReID features. In contrast to more complicated encoders such as OSNet in BoT-SORT [28], which prioritize feature richness at a more computational cost, or the simplified SHIP model in BoostTrack [18], which affects discriminative power, LAE optimizes MobileNetV3-Small for MOT-specific ReID tasks, establishing a balance between efficiency and feature quality. The TBSS uses a single-layer transformer with four attention heads for similarity evaluation, combining appearance and IoU features. Unlike TrackFormer [23], which uses multi-layer transformers for concurrent detection and tracking, or TransMOT [24], which uses spatial-temporal graph transformers, TBSS focuses on lightweight association, making it appropriate for real-time applications. The CMOH maintains continuity during occlusions by maintaining the K = 10 most recent appearance features, a simpler method compared to transformer-based memory in MeMOT [34] or graph neural networks [24]. This design prioritizes efficiency and implementation simplicity, making it suitable for resource-limited environments. The ASW dynamically balances IoU and appearance similarities according to occlusion weight, in contrast to the static metrics utilized in SMILEtrack [14], which are ineffective with changeable occlusion levels. By adjusting weights by a sigmoid function, ASW improves association robustness across many contexts. These components empower LightTrack-ReID to successfully tackle occlusion challenges, with performance evaluated in the experiments section.

3.2 Framework overview

LightTrack-ReID processes video frames , in which YOLOX produces detections , with indicating the bounding box and indicating the confidence score. Tracklets retain previous detections and visual features. The SimilarityModel calculates similarity evaluations between detections and tracklets, enhanced by contextual memory and adaptive weighting, for Hungarian matching. Training improves the SimilarityModel for robust association, especially in an environment of occlusion.

YOLOX detections are first processed by the Lightweight Appearance Encoder (LAE) to extract 32-dimensional appearance features. The mentioned features, along with geometric information, are passed to the Transformer-Based Similarity Scoring (TBSS) module for the computation of pairwise similarities. The Context Memory for Occlusion Handling (CMOH) preserves short-term appearance history to ensure identity continuity during occlusion. The Adaptive Similarity Weighting (ASW) module integrates appearance and IoU similarities based on occlusion density, producing the final cost matrix for Hungarian matching.

3.3 Frame-level tensor caching (FLTC)

Frame-level tensor caching improves training efficiency by storing a single tensor for each frame, so increasing performance without affecting inference accuracy. For a sequence , each frame’s tensor contains detection boxes , 224x224 RGB images , and ground-truth IDs : Located in tensor_cache_dir, this reduces I/O from about 100,000 pair tensors to about two thousand frame tensors, reducing loading time from hours to roughly 2–5 minutes (uncached) or less than 30 seconds (cached, achieving a speedup of approximately 3–5 times). Caching ensures reproducible training and contributes to slight improvements in HOTA and IDF1 scores, as well as reduced identity switches. The cost is around 0.001 GFLOPs per tensor input/output.

3.4 Adaptive pair sampling (APS)

Adaptive pair sampling balances the training dataset (( samples, with 80% assigned for training and 20% for validation) by restricting pairings to MAX_PAIRS_PER_FRAME = 50 For frame t, the positive (identical ID) and negative (different ID) pairings are:

(1)

using labels . This training-exclusive feature improves data equilibrium at a minimal expense ( GFLOPs).

3.5 Lightweight appearance encoder (LAE)

The appearance encoder extracts 32-dimensional ReID features via MobileNetV3-small (). For an input image , the encoder produces a compact feature embedding:

(2)

where the encoder is defined as:

(3)

The parameters θ are trained for producing distinctive appearance features suitable for person Re-Identification (ReID), with inference happening approximately 0.01 seconds per image.

In practice, the feature dimensionality was determined empirically. Among the 16D, 32D, and 64D configurations tested, 32-dimensional embeddings provided the best balance between accuracy and speed, achieving near-optimal tracking performance while minimizing computation for real-time inference.

3.6 Transformer-based similarity scoring (TBSS)

The SimilarityModel evaluates a similarity score between detection and tracklet . Inputs include boxes , and IoU:

(4)

and appearance features , forming a feature vector:

(5)

A linear projection, single-layer transformer (4 heads, around 0.1 GFLOPs), and sigmoid activation provide:

(6)

The transformer combines appearance and spatial features for robust association. Reason for Architectural Design. The single-layer transformer was chosen to optimize computational efficiency and association accuracy in environments with restricted resources (e.g., Ubuntu 16.04, GTX 1080). In contrast with complex frameworks like TrackFormer [22], which uses multi-layer transformers for simultaneous detection and tracking, our single-layer design is only dedicated to tracklet association, hence reducing computing cost. The selection of four attention heads follows standard practices in lightweight trans formers [45], optimizing the integration of features (appearance, IoU) while preserving efficiency. Initial experiments indicated that four heads offer adequate capacity for MOT settings, including brief occlusions, with performance specifics described in the Ablation Study Section. A single-layer transformer was utilized to balance attention modeling capacity and computational efficiency. Experiments with more complex configurations produced minor accuracy improvements but significantly raised inference costs; thus, the one-layer design was selected as the most pragmatic option for real-time tracking.

3.7 Context memory for occlusion handling (CMOH)

To reduce occlusion-induced track fragmentation, a CMOH stores the most recent 32-dimensional appearance features for each tracklet :

(7)

where denotes the age of the tracklet. For occluded tracklets, a contextual feature is calculated as

(8)

supporting similarity scoring through

(9)

This simple memory buffer was selected compared to more complicated alternatives, such as transformer-based memory in MeMOT [34] or graph-based contextual modeling [24], because of its minimal processing cost and suitability for MOT17/MOT20 situations, characterized by common short-term occlusions. Limiting memory to K = 10 improves resource use and helps maintain tracking continuity. The effect on efficiency is evaluated in Ablation Study Section. This choice of K = 10 was empirically validated, as larger memory sizes offered negligible performance gains while increasing computational and memory costs.

3.8 Adaptive similarity weighting (ASW)

Adaptive similarity weighting balances IoU and appearance similarities with respect to occlusion density. The cost matrix for Hungarian matching is delineated as

(10)

where , with indicating the number of occluded detections (i.e., IoU overlap ), and denoting the overall number of detections. This dynamic weighting ensures a strong association across various environments, improving the HOTA score by adapting to different amounts of occlusion.

3.9 Training Objective and Rationale

The SimilarityModel, including the appearance encoder parameters and Transformer weights, is trained to identify appearance features and predict identity-matching scores. To ensure that the appearance embeddings can distinguish between various identities, a triplet loss is used:

(11)

where , and , , and represent the anchor, positive, and negative samples, respectively. The model simultaneously trains to predict similarity scores or that accurately reflect identity associations, utilizing a binary cross-entropy loss function:

(12)

where indicates if detection i and tracklet j correspond to the exact same identity. The total loss is the straightforward sum of the two elements:

(13)

Training uses MOT17/MOT20 (80% train, 20% validation) for 20 epochs with Adam (lr = 0.001). Images are resized to 224x224, normalized to [0,1], with augmentations: random flip (50%), crop (10% padding), color jitter (0.2). Hyperparameters (, ) were tuned on MOT17 validation.

Algorithm 1 LightTrack-ReID Tracking Procedure

1: Input: frames

2: Output: Object tracks

3: Initialize tracks , memory buffer , max size

4: for to T do

5:  Apply CMC

6:  Get from YOLOX on frame

7:  Filter by (e.g., )

8:  Forecast track states with Kalman filter

9:  Extract appearance features

10: for each track do

11:   Prepare context memory

12:   Get context-aware feature

13:   end for

14:  Get similarity scores using transformer with and ▷(TBSS: Transformer-Based Similarity Scoring)

15:  Get occlusion weight ▷ ASW: Adaptive Similarity Weighting

16:  Define cost matrix:

17:   ▷ (Combines TBSS and ASW before Hungarian matching)

18:  Hungarian algorithm on

19:  for in match set do

20:   Update track with detection , Kalman update

21:   end for

22:   for d in unmatched detections do

23:   Initialize new track and add to

24:   end for

25:  for in unmatched tracks do

26:   Mark as occluded or terminate if inactive for too long

27:   end for

28:  Update context memory for each track

29:  Tracklet management

30: end for

31: return

4.1 Experimental settings

Datasets. We evaluate our method using two common pedestrian tracking benchmarks: MOT17 [57] and MOT20 [58], following the “private detection” procedure. MOT17 has sequences captured using fixed and moving cameras; MOT20 highlights crowded environments. Both datasets provide training and test sets, even though they do not include formal validation sets. In our ablation studies, we adhere to the procedures described in [29,59], using the first half of each MOT17 training set for training and the remaining half for validation.

Metrics. We measure performance with the standard CLEAR metrics [60], which contain Multiple Object Tracking Accuracy (MOTA), False Positives (FP), False Negatives (FN), and ID Switches (IDSW). In addition, we use IDF1 [61] and Higher-Order Tracking Accuracy (HOTA) [15] for evaluating identity maintenance and complete tracking performance. Tracking speed (in FPS/Hz) can be measured, though it can change according to the hardware. MOTA is derived from false positives (FP), false negatives (FN), and ID switches (IDSW), and primarily reflects detection performance due to the dominant influence of FP and FN. IDF1 focuses on detection association, whereas HOTA combines detection, association, and localization accuracy into one metric.

Implementation details. LightTrack-ReID is implemented on a resource-limited computer running Ubuntu 16.04, containing an NVIDIA GTX 1080 GPU (8GB VRAM) and an Intel Core i7-6700 CPU (3.4 GHz, 32GB RAM). To clarify, the 0.6 GFLOPs figure refers exclusively to the association stage, which encompasses the Lightweight Appearance Encoder (about 0.5 GFLOPs) and the single-layer transformer similarity block (around 0.1 GFLOPs). The YOLOX-S detector contributes to approximately 26.8 GFLOPs, being the predominant portion of end-to-end computation. The proposed association consequently results in an additional computational expense of less than 3%, enabling the system to maintain approximately 30 FPS real-time performance on GTX1080 hardware.

The software package uses Python 3.8 and PyTorch 1.9.1, using CUDA 10.2 for GPU acceleration. Dependencies include OpenCV 4.5.3 for image processing, NumPy 1.19.5 for numerical calculations, and the TrackEval package for multi-object tracking evaluation. The model uses a pretrained YOLOX from ByteTrack [29] for detection and uses MobileNetV3-small to extract 32-dimensional ReID features. A lightweight transformer with four heads calculates similarity using appearance and soft Intersection over Union (IoU). Occlusion is handled with memory buffers and adaptive weighting. The model is trained on MOT17 and MOT20 for 20 epochs with triplet loss and the Adam optimizer (learning rate = 0.001), consuming approximately 10 hours on a GTX 1080 GPU.

4.2 Ablation study

We evaluate the impact of each inference-stage component of LightTrack-ReID on the MOT17 train set using common tracking metrics: HOTA, MOTA, IDF1, and IDS. The baseline uses YOLOX detections, motion prediction using a Kalman filter, development of an IoU-based cost matrix, and Hungarian matching for association. It additionally incorporates confidence-based filtering and exponential moving average (EMA) for state smoothing, elements frequently used in modern tracking systems. We progressively integrate LAE derived appearance encoder for 32-dimensional ReID features, TBSS, CMOH temporal appearance, and ASW to balance IoU and appearance scores. Training-specific methods are excluded.

To assess the impact of every element in our tracking framework, we performed an ablation study using the MOT17 and MOT20 validation datasets. Table 1 and Figure 4 present the results on MOT17, whereas Table 2 and Figure 5 report the outcomes on MOT20, highlighting the effect of adding each module individually to the baseline. Both figures use a dual-axis configuration: HOTA, MOTA, and IDF1 are shown as line graphs on the left vertical axis, while the number of ID switches (IDSW) is shown as green bars on the right. The x-axis begins with the baseline configuration, followed by incremental additions of each component.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Ablation results for individual components on the MOT17 public validation split.

https://doi.org/10.1371/journal.pone.0342246.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Individual Component Ablation on MOT20 (Validation Split).

https://doi.org/10.1371/journal.pone.0342246.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Ablation study: individual component performance on the MOT17 validation set.

https://doi.org/10.1371/journal.pone.0342246.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Ablation study: individual component performance on the MOT20 validation set.

https://doi.org/10.1371/journal.pone.0342246.g005

The cumulative ablation study on MOT17 (Table 3, Figure 6) begins with the baseline configuration, which achieves a HOTA of 66.13, MOTA of 74.8, IDF1 of 77.3, and IDSW of 227, indicating clear limitations in handling occlusions. Incorporating the Lightweight Appearance Encoder (LAE) improves HOTA to 70.88, MOTA to 79.0, IDF1 to 81.97, while reducing IDSW to 168, demonstrating its effectiveness in enhancing appearance representation. The incorporation of Transformer-Based Similarity Scoring (TBSS) elevates HOTA to 73.38, MOTA to 81.2, and IDF1 to 84.47, while decreasing IDSW to 138, thus enhancing association precision. The incorporation of Context Memory for Occlusion Handling (CMOH) enhances performance to HOTA 74.88, MOTA 82.6, IDF1 86.07, and significantly reduces IDSW to 80, demonstrating its efficacy in preserving track continuity during occlusion.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Cumulative component ablation on MOT17 (Public Validation Split).

https://doi.org/10.1371/journal.pone.0342246.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Ablation study: cumulative component performance on the MOT17 validation set.

https://doi.org/10.1371/journal.pone.0342246.g006

On MOT20 (Table 4, Figure 7), the baseline achieves HOTA 56.17, MOTA 69.92, IDF1 73.72, and IDSW 1120, highlighting the significant challenges associated with dense crowd tracking. With LAE, performance improves to HOTA 60.38, MOTA 73.51, IDF1 77.15, while IDSW reduces to 952. The addition of TBSS raises HOTA to 63.94, MOTA to 76.61, and IDF1 to 80.01, while lowering IDSW to 882, confirming its efficacy in enhancing reliable matching. The implementation of CMOH increases HOTA to 65.74, MOTA to 78.21, IDF1 to 81.51, and substantially reduces IDSW to 701, thereby dramatically reducing occlusion-related mistakes. The concluding phase, incorporating ASW, achieves HOTA 66.7, MOTA 78.9, and IDF1 82.3, but IDSW stays unchanged at 701, providing incremental enhancements with minimal supplementary effect.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Cumulative component ablation on MOT20 (Validation Split).

https://doi.org/10.1371/journal.pone.0342246.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Ablation study: cumulative component performance on the MOT20 validation set.

https://doi.org/10.1371/journal.pone.0342246.g007

4.3 Comparison with other methods

Table 5 presents an in-depth evaluation of the proposed method, LightTrack-ReID, compared to other state-of-the-art multi-object tracking (MOT) methods on the MOT17 and MOT20 benchmarks. Figure 1 presents a graphical representation of the HOTA and IDF1 scores, highlighting the superior effectiveness and robustness of our model. LightTrack-ReID shows outstanding results across all major evaluation standards, achieving the best HOTA 66.92, MOTA 82.81, and IDF1 82.52 scores on MOT17 while, at the same time, maintaining a relatively low number of identity switches (IDSW). In the challenging MOT20 dataset, which features common occlusions and high object density, the proposed method achieves a HOTA of 66.6, MOTA of 79.1, and IDF1 of 82.2, while also keeping the lowest number of ID switches (753) when compared to all other methods tested.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparison of state-of-the-art methods on MOT17 and MOT20 benchmarks.

https://doi.org/10.1371/journal.pone.0342246.t005

The results clearly show the success of LightTrack-ReID in maintaining unchanged object identification, especially in conditions with significant occlusion. The method includes a highly effective Re-Identification (ReID) module that enhances identification association across frames, allowing accurate continuous tracking even when objects are temporarily hidden or disappear. Compared to current methods like BoostTrack++, ConfTrack, and ImprAsso, which similarly use ReID techniques, LightTrack-ReID demonstrates enhanced identity retention and reduced tracking failures in occluded environments.

The results indicate that LightTrack-ReID provides an accurate and occlusion-resistant tracking method, suitable for real-world applications that include frequent object interactions and occlusions. With a frame rate of 30 FPS on the MOT17 benchmark, it achieves real-time performance.

4.4 Limitations

Despite LightTrack-ReID’s outstanding tracking accuracy and low computational demands, specific limitations remain. The global Adaptive Similarity Weighting (ASW) uses a uniform occlusion weight across the entire frame, potentially overlooking local occlusion variations, whereas the Context Memory for Occlusion Handling (CMOH) has limitations in addressing long-term or recurrent occlusions due to its limited buffer capacity (K = 10). Moreover, the appearance encoder trained on MOT17/MOT20 may exhibit limited generalization to novel domains characterized by diverse illumination or camera movement. Future enhancements will focus on localized ASW weighting, hierarchical memory architectures, and domain-adaptive training to improve robustness and flexibility.