- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The underlying motivation of this paper is interesting: not all ground-truth annotations are equally important, and the costs of prediction errors are not necessarily equal. In the absence of a perfect solution, a reasonable second-best approach is to prioritize correcting errors with higher costs rather than those with lower costs. However, the proposed method is overly complicated and relies on too many intricate design choices. Moreover, the effectiveness of some of its contributions has not been adequately evaluated, and some of the reported results contain factual errors.
Critical strengths
As stated in our summary, the authors' proposed idea of non-uniform supervision allocation appears to be a feasible approach, and the underlying concept is innovative.
Critical weaknesses
-
I have serious concerns about the reliability of the experiments presented in this paper. In Table 6, the authors report MOTIP experiments under different conditions, with and without motion prediction. However, to my knowledge, MOTIP does not contain any motion prediction module at all. Moreover, the reported validation score of around 64.0 is close to MOTIP's original validation performance. This makes it difficult for me to understand the purpose and setting of this experiment, and I have serious doubts about the reliability of the reported results.
-
The computation of future preferences involves numerous complicated and largely heuristic design choices, including the selection and weighting of different evaluation metrics and the construction of four verification channels. However, the authors do not provide individual ablation studies for these channels. This makes it difficult to assess whether all four channels are necessary, how they complement each other, and whether each contributes meaningfully to the final performance. Furthermore, there is considerable overlap among the metrics used across different channels, such as the repeated use of ID-switch penalties and identity correctness. What is the rationale behind this design? Why is such redundancy necessary, and why not consolidate these overlapping metrics into a simpler, unified evaluation function?
-
Although the proposed method indeed outperforms the baseline under the same number of training steps, the on-policy collection and counterfactual rollouts performed at the beginning of each epoch may introduce substantial additional computational overhead. However, the authors do not provide a detailed analysis of the computational cost or wall-clock time associated with these operations. I believe that a comprehensive evaluation of these additional computational costs is essential for assessing the actual efficiency and practical value of the proposed method.
-
Statements such as Wall-clock time and GPU-hour measurements are unavailable and The label ‘w/o contrast’ leaves the changed operation unspecified should not appear in a finalized research manuscript, especially the latter. Furthermore, the description of Ordinary Training in Figure 4 is also unclear. These issues raise serious doubts about the completeness and provenance of the submitted manuscript, as well as the extent of the authors’ actual involvement in verifying the work.
-
On page 6, the authors introduce three auxiliary losses without sufficient justification. Why is KL loss alone insufficient? What are the intuitive motivations and empirical evidence supporting the additional ranking and value losses? The lack of individual ablations further raises concerns about their necessity.
-
Although this concern may be difficult to address during the rebuttal, I would still like to emphasize that, while the underlying idea is appealing, its implementation is overly complicated, arguably even more so than traditional hand-crafted methods. This seems to contradict the fundamental philosophy of end-to-end MOT research, which aims to minimize manual design and learn directly from data with simpler frameworks.
Questions for authors
Regarding each point raised in the Weaknesses section:
-
The authors must adequately address this issue. Otherwise, I will lower my rating to 1 due to serious concerns about the authenticity of the reported results.
-
The authors need to provide appropriate ablation studies to clearly establish the effectiveness of at least the key components.
-
The authors must provide a comprehensive analysis of the actual computational cost of the entire training pipeline, rather than simply stating that runtime measurements are unavailable.
-
The authors need to provide point-by-point clarifications for all ambiguously described aspects of the paper.
-
The authors need to provide the corresponding ablation results or relevant references.
-
This is a difference in research philosophy, and the authors are not required to respond. However, I hope that other reviewers, ACs, and SACs will take this point into consideration in their overall assessment. I would also welcome further discussion in the discussion section to hear different perspectives on this issue. Ultimately, this is a matter of philosophical judgment rather than factual correctness.
12 comments
Responses to Reviewer doZx (Part 2 of 3)
Question 3 (W3 / Q3): What is the computational cost of the entire training pipeline, including collection, future branching, verification, and optimization? How often is collection repeated?
A3. Future evidence is prepared at frozen snapshots and reused across updates of the existing association model. Histories and verified preferences are reused within an epoch; the teacher is rebuilt from current predictions. The next epoch refreshes the snapshot. With MOTIP's detector frozen, branches share detector features for the same transformed frames.
The single-RTX-3090 profile below uses shared feature/history caches and 40 training sequences. It contains one preparation sweep, with 117 states and 238 branches at a 15-frame horizon, and separately measured 4k optimization:
| Phase | Continue (s) | Track-OPD (s) |
|---|---|---|
| Loading and training setup | 22.309 | 22.354 |
| Collection and future continuations, one sweep | 0.000 | 925.352 |
| Future verification and record construction | 0.000 | 7.940 |
| 4k optimization | 1,397.222 | 1,252.706 |
The measured optimization phase takes 10.3% less time for Track-OPD. Peak allocated GPU memory is 0.728 GiB during setup, 0.288 GiB during its collection, and 8.792 GiB during optimization for both methods; verification runs on the CPU. Thus, the largest measured GPU-stage peak is the same.
Initial common-cache generation and validation evaluation are separate. Future supervision adds preparation at every snapshot refresh, reused across the subsequent updates; the cached optimization retains the same measured peak memory and is faster in this profile.
Question 4 (W4 / Q4): Please clarify the unfinished runtime and “w/o contrast” statements, and the meaning of Ordinary Training in Figure 4.
A4. Ordinary Training is native training from scratch; Table 8 changes the two operations specified below. A3 gives the measured costs. In Figure 4, the native baseline reaches 43.282 HOTA. All branches end at the same final epoch. Introducing Track-OPD at epochs 0, 2, 4, or 8 yields 28.904, 37.675, 43.751, and 46.139 HOTA. The epoch-8 result is 2.857 higher, supporting introduction after initial association learning.
In Table 8, “w/o contrast” sets the coefficient of that configuration's explicit margin term to zero. Candidate-score centering is a separate teacher-construction operation. “w/o dynamic weight” disables record-specific dynamic weighting. Table 8 compares complete configurations; A5 gives the matched KL-only comparison for the current KL/rank/value objective in Appendix B.5.
Figure 4 therefore examines when to introduce future supervision, while the Table 8 labels identify the training operations being changed.
Question 5 (W5 / Q5): Why use rank and value losses in addition to KL, and what ablations support their contributions?
A5. KL matches the full teacher distribution; rank and value emphasize its pairwise probabilities and confidence. Rank fits the teacher's soft pairwise probabilities, emphasizing verified competitors. For a two-class support with mass , before common normalization and weighting, where the KL gradient is . Rank therefore gives the verified pair greater weight when its current support mass is small. Value uses a Huber loss to match tanh-transformed log odds, emphasizing the confidence of the same comparison. These gradients follow from the loss definitions in Appendix B.5. All terms share the fixed point .
The matched 4k, seed-42 comparison is:
| Objective | HOTA | AssA | IDF1 |
|---|---|---|---|
| KL-only | 65.078 | 56.408 | 70.374 |
| Track-OPD | 65.645 | 56.417 | 71.560 |
The joint objective gains 0.567 HOTA and 1.186 IDF1, with similar AssA. This supports our choice to fit both the distribution and its verified identity comparisons.
Responses to Reviewer doZx (Part 3 of 3)
Question 6 (W6 / Q6): The implementation appears more complex than hand-crafted methods. Does this contradict end-to-end MOT's aim of reducing manual design and learning associations through a simpler framework?
A6. Thank you for recognizing the idea's value. We use efficient training to help the existing tracker learn more from its videos. Track-OPD retains the host architecture, parameter count, online inference, and model complexity.
GT defines the reference identity. Score definitions and distillation settings are fixed across the evaluated trackers and datasets, while channel availability and reliability follow each comparison's observable evidence. These rules define the training signal; the association rule is learned by the original network. Detector features are shared across branches when the detector is frozen. Saved histories and verified preferences are reused within an epoch, while the teacher is rebuilt from current predictions. Table 4 and A3 report the resulting quality and optimization cost:
| Reported comparison | Continue | Track-OPD |
|---|---|---|
| HOTA / AssA / IDF1, Table 4 | 64.934 / 56.184 / 69.727 | 65.353 / 56.754 / 70.336 |
| Cached 4k optimization (s), A3 | 1,397.222 | 1,252.706 |
| Optimization peak allocated GPU memory (GiB), A3 | 8.792 | 8.792 |
Fewer updates reach higher quality. Cached 4k optimization takes 10.3% less time at the same peak memory; A3 gives preparation costs and the refresh schedule.
We see an important advantage of end-to-end MOT in learning temporal associations directly from well-annotated video. DanceTrack's similar appearances and diverse motion make temporal reasoning especially relevant [1]. MOTIP illustrates how associations can be learned by predicting identities from trajectory history [2]. This motivates us to improve how the model uses the temporal information already present in its training videos.
Even with correct annotations, occlusion and similar appearances make current observations ambiguous. A locally plausible match can contaminate the identity history and cause errors after the objects separate. GT identifies the correct correspondence; future verification reveals how competing decisions affect later identity continuity. Fixed identity bindings and common observable annotations make those consequences comparable. The local teacher then turns them into bounded corrections, helping the original model learn associations that remain correct beyond the ambiguous observation.
Table 1 reports HOTA gains of 1.03–1.36 over native Continue across four tracker–dataset comparisons, with AssA and IDF1 improving throughout. Table 5 reports bidirectional transfer AssA gains of 4.880 and 5.030 with shared coefficients fixed. These results support better use of training evidence and association behavior that transfers across the tested domains.
We see a useful direction here for future unified trackers: a common training principle that turns temporal identity outcomes into supervision across architectures and videos, while keeping deployment simple. Track-OPD takes a step in this direction by making future identity consequences usable for learning within existing end-to-end models.
We will include these analyses and clarify Tables 6 and 8 and Figure 4 in the revision.
References
[1] Peize Sun, Jinkun Cao, Yi Jiang, Zehuan Yuan, Song Bai, Kris Kitani, and Ping Luo. DanceTrack: Multi-object tracking in uniform appearance and diverse motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20993–21002, 2022.
[2] Ruopeng Gao, Ji Qi, and Limin Wang. Multiple object tracking as ID prediction. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 27883–27893, 2025.
Responses to Reviewer doZx (Part 1 of 3)
Thank you for the detailed questions about experimental reliability and the training design. We clarify what Table 6 changes, then give the requested cost measurements and comparisons of the scoring and losses.
A1. Table 6 changes trajectory-history augmentation during training; both settings retain MOTIP's architecture. The operations simulate occluded history segments and identity switches. The parameters
AUG_TRAJECTORY_OCCLUSION_PROBandAUG_TRAJECTORY_SWITCH_PROBare 0.5/0.5 in the default configuration and 0/0 without augmentation. Trajectory feature processing and the ID decoder remain active.The experiment tests whether future-informed training benefits both settings. All results use the same 25,508 validation frames; these paired checkpoints are separate from the matched 4k study:
The approximately 64 HOTA value in your comment is the default-configuration Continue. The second comparison starts from the official no-augmentation checkpoint. The gains in both comparisons therefore show that future-informed post-training benefits the existing association model under both trajectory-history augmentation settings.
A2. The four channels contribute to one score, with normalization and reliability determining how observable evidence is combined. Geometry and identity continuity describe different aspects of the same trajectory; interaction and recoverability add evidence when those events are observable. A geometrically plausible continuation that follows another person should still count as an identity error. This is why identity correctness and switch penalties recur across channels.
The aggregate defined in Appendix A.3 is a single function of primitive scores and penalties: repeated identity terms combine into one effective identity weight, and switch terms into one switch weight. The primitive-score coefficients are fixed. Channel availability, score normalization, and reliability determine how they combine for each record. Reliability follows observability and candidate-score separation.
We compared simpler scoring rules at 4k updates with seed 42:
The normalized control sets available-channel reliability and aggregate confidence to one; raw averaging additionally removes normalization. Both reapply selection to the same 114 sampled states. The retained supervision is:
Positive targets count states with a nonzero teacher correction. The last column counts training uses, including repeated uses of a state; native supervision continues on all valid predictions.
Track-OPD has higher HOTA and IDF1 than both controls, while raw averaging has slightly higher AssA. The controls measure calibration together with the supervision it retains. This sample contains observable geometry, identity/appearance, and interaction evidence. These comparisons support normalizing available evidence and accounting for reliability when constructing association targets.