- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces UAV-Diverse, a test-only benchmark for multi-person tracking in UAV videos. The dataset contains 44 videos, 37,363 frames, 1,498 trajectories, and approximately 821,900 bounding boxes. It covers three major scenario categories (sports, wilderness search-and-rescue/patrol, and urban/indoor environments) and nine sub-scenarios. The benchmark also preserves identity consistency when people leave the field of view and later re-enter.
The authors compare UAV-Diverse with VisDrone and MMOT using normalized entropy and other statistical measures. They argue that UAV-Diverse presents greater challenges in viewpoint variation, camera motion, object scale, appearance, and re-entry events. They also suggest that existing multi-object tracking (MOT) methods have limited cross-scene generalization.
The paper further proposes UAV-Tracker, a training-free baseline built on a frozen SAM3 model. It consists of three components: object lifecycle management, a context-based trajectory stabilizer, and scene fingerprint-based policy adaptation. UAV-Tracker achieves a HOTA score of 61.90 on the proposed benchmark.
Critical strengths
1. A valuable benchmark for UAV-based person tracking.
UAV-Diverse provides a person-centered tracking benchmark with diverse scenarios. It also maintains consistent identity annotations when people leave and re-enter the field of view. This makes the dataset valuable for studying long-term identity preservation and person re-identification. In addition, three annotators who were not involved in bounding-box annotation independently labeled viewpoint, camera motion, and appearance conditions in a blinded manner. This is a well-designed annotation procedure.
2. Detailed experimental analysis.
The appendix provides results grouped by different conditions, including re-entry events, object scale, viewpoint, and camera motion. The authors also report threshold sensitivity in Table 4. Importantly, they acknowledge potential confounding factors between these subsets. These analyses help readers better understand the benchmark's challenges and the limitations of the reported results.
Critical weaknesses
1. The main claims about limited cross-scene generalization and benchmark difficulty lack sufficient evidence. Some results even contradict these claims.
-
Detector-related confounding factors. The methods in the Original Pipeline group use detectors trained on VisDrone (HOTA 26–35), while those in the Shared SAM3 group use a foundation model detector (HOTA 52–62). This large gap may mainly reflect differences in detection quality rather than cross-scene tracking ability. Since the benchmark has no in-domain training set, it is also difficult to separate domain shift from the inherent difficulty of the dataset.
-
UAV-Diverse appears easier than VisDrone under the same detector. With SAM3 detection, ByteTrack achieves 54.56 HOTA on UAV-Diverse but only 33.15 on the VisDrone pedestrian subset. Similarly, UAV-Tracker achieves 61.90 and 37.47, respectively (Table 2 vs. Table 10). Although differences in dataset composition may affect this comparison, these results weaken the claim that UAV-Diverse is more challenging.
-
The proposed statistics do not directly measure difficulty. Normalized entropy measures how evenly category labels are distributed, not how difficult the tracking task is. The reported 97.5% conditional re-entry rate largely reflects the annotation protocol. UAV-Diverse reuses the original identity when a person re-enters, while datasets such as VisDrone may assign a new identity. Moreover, only 7.9% of trajectories (118 trajectories) contain valid disappearance events.
2. The baseline offers limited improvements, and the evaluation lacks important comparisons for identity association.
-
UAV-Tracker relies heavily on hand-crafted rules. Equations 15–23 introduce many fixed coefficients and thresholds, which the authors tune on five rugby videos. This makes the method relatively complex for a "simple baseline" and raises concerns about its general applicability.
-
The performance gains are modest. Compared with AMOT+SAM3Feat, UAV-Tracker improves HOTA by only 2.11 points. However, its MOTA is lower (58.20 vs. 62.83). The authors also report results from only a single run, without variance estimates.
-
The ReID comparisons are insufficient. For identity recovery, UAV-Tracker only uses cosine similarity between SAM3 features. The authors do not compare it with trackers that use dedicated ReID models, such as StrongSORT, Deep OC-SORT, or BoT-SORT with OSNet/CLIP-ReID features. Therefore, it is unclear whether the improvement comes from the proposed lifecycle management or the quality of the appearance features.
-
Potential annotation bias. The benchmark uses SAM2 for pre-annotation, while the proposed tracker relies on SAM3. Any remaining annotation bias may favor SAM-based methods. The paper does not sufficiently examine this possibility.
-
Dataset availability and reproducibility. Some videos come from online sources, and it is unclear whether the authors have permission to redistribute the original footage. This may affect the reproducibility and long-term availability of the benchmark.
Questions for authors
-
Under the same SAM3 detector, tracking scores on UAV-Diverse are much higher than those on the VisDrone pedestrian subset. How do the authors justify the claim that UAV-Diverse is more challenging? Could the authors provide an in-domain training upper bound, for example using cross-validation splits, to separate the effects of domain shift from dataset difficulty?
-
How do VisDrone and MMOT handle identity annotations when a person leaves and re-enters the field of view? How much of the difference between 97.5% and 26.1% comes from annotation protocols rather than actual scene characteristics?
-
Could the authors include trackers with dedicated ReID features as additional baselines? It would also be useful to replace the SAM3 features in UAV-Tracker with ReID features to evaluate their respective contributions.
-
What are the licensing conditions for the videos collected from online sources? If the original videos cannot be redistributed, how can other researchers access the complete test dataset?
-
Could the authors report results across multiple runs or different hyperparameter settings to assess performance variance? A sensitivity analysis of UAV-Tracker's scene fingerprint coefficients would also help evaluate its robustness.
5 comments
Response to Reviewer cBL5
3. Baseline Design, Performance Gains, and ReID Comparisons (W2.1–W2.3, Q3)
Identity management and mechanism contributions.
UAV-Tracker improves identity management and trajectory continuity on top of frozen SAM3. Table 3 evaluates lifecycle management (OLM) under identical SAM3 observations.
| Configuration | HOTA ↑ | AssA ↑ | IDF1 ↑ |
|---|---|---|---|
| Native SAM3 | 51.64 | 61.16 | 61.05 |
| SAM3 + OLM | 59.48 | 68.95 | 68.06 |
| Improvement | 7.84 (+15.2%) | 7.79 (+12.7%) | 7.01(+11.5%) |
The full UAV-Tracker further achieves 61.90 HOTA, a 10.26-point (+19.9%) improvement over Native SAM3. To assess whether simpler mechanisms suffice, we evaluate simplified OLM reactivation, a fixed CS propagation budget, and disabled SF under identical SAM3 observations.
| Configuration | Mechanism changed | HOTA ↑ | AssA ↑ | IDF1 ↑ | MOTA ↑ |
|---|---|---|---|---|---|
| Simplified OLM Reactivation | Cosine threshold only (0.85) | 61.1 | 67.71 | 70.63 | 58.15 |
| Fixed CS Budget | Fixed propagation budget B=5 | 61.62 | 69.2 | 71.82 | 58.09 |
| Without SF | SF disabled | 61.35 | 69.07 | 71.17 | 56.2 |
| Full UAV-Tracker | Original configuration | 61.9 | 69.43 | 71.97 | 58.2 |
The full method improves HOTA by 0.28–0.80 points over the simplified alternatives. As the reviewer notes, all parameters were selected using five external rugby videos and applied unchanged to all 44 UAV-Diverse sequences (Appendix A.6), supporting cross-scene applicability without target-domain tuning.
Dedicated ReID comparisons.
We appreciate the reviewer's helpful suggestion to strengthen the ReID comparison. Table 2 includes BoT-SORT with a VisDrone-trained FastReID SBS-S50 model and ReID enabled. Following the reviewer's suggestion, we further replace UAV-Tracker's SAM3 appearance features with the same frozen FastReID model while keeping its other mechanisms unchanged.
| Method | Appearance | HOTA ↑ | AssA ↑ | IDF1 ↑ |
|---|---|---|---|---|
| BoT-SORT | FastReID | 54.51 | 60.14 | 63.64 |
| UAV-Tracker | SAM3 | 61.9 | 69.43 | 71.97 |
| UAV-Tracker | FastReID | 62.02 | 69.78 | 72.19 |
With the same FastReID model and shared SAM3 detections, UAV-Tracker outperforms BoT-SORT by 7.51 HOTA (+13.8%), 9.64 AssA (+16.0%), and 8.55 IDF1 (+13.4%). Replacing SAM3 appearance features with FastReID also slightly improves UAV-Tracker's HOTA from 61.90 to 62.02. These results, together with the OLM ablation, support the contribution of the proposed mechanisms beyond the choice of appearance features.
Performance gains and evaluation metrics.
Compared with AMOT+SAM3Feat, UAV-Tracker improves HOTA by 2.11, AssA by 3.29, and IDF1 by 3.17 points, while achieving lower MOTA (58.20 vs. 62.83). We acknowledge this trade-off. As discussed by Luiten et al. (IJCV 2021, Sec. 9.1), MOTA is less informative about identity association, whereas HOTA balances detection and association accuracy. Given the importance of identity continuity in our benchmark, we emphasize HOTA, AssA, and IDF1 while also reporting MOTA for a comprehensive evaluation.
4. Evaluation Stability and Annotation Reliability (W2.1, W2.4, Q5)
Evaluation stability.
As detailed in Appendix A.6, all inference parameters were fixed across evaluation datasets. OLM, SF, and adaptive propagation make deterministic tracking decisions without random sampling given identical SAM3 observations and inference settings. A full-dataset rerun reproduced the archived predictions across all 44 UAV-Diverse sequences, with identical track IDs and bounding-box coordinates in the evaluation coordinate system.
We further evaluate Scene Fingerprint coefficient sensitivity under four configurations, independently varying the coverage and identity-risk weights by ±20% while retaining scene-conditioned adaptation. Across all 44 sequences, HOTA remains between 61.8982 and 61.8987, indicating stable performance within the tested parameter range.
Annotation Quality and Potential SAM2 Pre-annotation Bias.
We recognized the importance of reliable identity annotations. SAM2 was used only to initialize annotations, rather than to define the final ground truth. As detailed in Appendix A.1, all bounding boxes and identity labels were manually reviewed and refined in CVAT, with ambiguous cases resolved through additional review and consensus. All trackers are evaluated against the same manually verified ground truth.
Response to Reviewer cBL5
5. Dataset Availability and Licensing (W2.5, Q4)
We appreciate the reviewer's concern about dataset accessibility, licensing, and reproducibility. For online-source videos, we will adopt a URL-based release policy, following established practices in TrackVerse (ICCV 2025), PreVAD (ICLR 2026), and VALUE (NeurIPS 2021). We will not host or redistribute third-party video files or extracted frames. Instead, we will release source URLs, precise timestamps, frame mappings, MOT annotations, and evaluation code, allowing researchers to reconstruct the test clips from the original sources where access remains available and permitted.
The original videos remain subject to their owners' rights and applicable licensing terms, while our annotations and code will have separate licenses. We have updated the Ethics and Reproducibility Statements in the revised PDF to clarify this release policy and privacy safeguards.
We hope these results and clarifications help address the reviewer's concerns. Thank you again for your time and consideration.
Response to Reviewer cBL5
We appreciate the reviewer's careful assessment and recognition of UAV-Diverse's diverse scenario coverage, identity-consistent annotations, and detailed challenge analyses. We also value the constructive suggestions regarding ReID comparisons, robustness, and dataset reproducibility. UAV-Diverse is the primary contribution of our paper. Below, we provide clarifications and additional experiments to address these concerns.
1. Evaluation Protocol and Cross-Dataset Performance (W1.1, W1.2, Q1)
Cross-dataset performance and benchmark scope.
As documented in Appendix A.5.3 and Figure 6, we identified annotation issues in VisDrone, including oversized ground-truth boxes and unannotated visible people. These issues can affect human-only tracking scores. We recognize the importance of reliable annotations for cross-dataset comparisons. However, comprehensively reannotating VisDrone would require a separate dataset curation effort beyond the scope of this work.
We would like to clarify that UAV-Diverse does not claim to be inherently harder than VisDrone. Our focus is evaluating cross-scene transfer across diverse UAV environments. The reported statistics characterize challenge coverage and diversity, not absolute tracking difficulty.
Evaluation protocols.
Section 5.1 defines two complementary protocols. Original Pipeline evaluates five VisDrone-trained trackers using their native inference pipelines. Shared SAM3 Detections controls detection inputs to compare tracking strategies.
UAV-Diverse is intentionally test-only. Its purpose is to evaluate cross-scene transfer under domain shift, without target-domain training or adaptation. In-domain cross-validation would instead evaluate dataset-specific adaptation under a different protocol.
2. Diversity Statistics and Identity Reappearance (W1.3, Q2)
Diversity versus difficulty.
We would like to clarify the distinction between challenge diversity and tracking difficulty. Section 3.2 uses normalized entropy to characterize challenge diversity. UAV-Diverse achieves 89.0% viewpoint entropy (VisDrone: 28.6%, MMOT: 0.0%), 91.0% camera-motion entropy (74.4%, 39.6%), and 89.6% appearance entropy (14.3%, 15.6%). Its target-scale range spans 14.9×, compared with 7.3× and 3.6×.
These results demonstrate broader coverage of heterogeneous UAV conditions. Appendix A.3 further examines tracking performance under these challenges. For example, in the smallest-target regime, AMOT+SAM3Feat achieves only 18.35 HOTA, compared with 38.33 for UAV-Tracker (Appendix Table 6), highlighting the challenges of small-target tracking and the differences between tracking methods.
Identity consistency and reappearance.
We would like to clarify that the reported 97.5% is a conditional reappearance rate. Although valid disappearance events involve 7.9% of trajectories, reappearance events occur in 24 of the 44 sequences. Under the same 15-frame criterion, 115 of 118 trajectories with valid disappearance events satisfy the reappearance criterion in UAV-Diverse (97.5%), compared with 71/272 (26.1%) in VisDrone and 32/54 (59.3%) in MMOT. Both VisDrone-MOT and MMOT explicitly specify temporally consistent identity annotations in their annotation protocols. Appendix Table 4 confirms that the relative ordering remains unchanged across 10-, 15-, and 30-frame thresholds.