- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
ReTrack addresses recovery after target loss in visual object tracking. Conventional trackers typically search the nearby region of the previous prediction and may drift onto background clutter when occlusion or rapid motion causes the target to leave the search region. The proposed framework combines a local tracker, a full-frame proposal head, and a lightweight decision network to enable global recovery. During training, staying with the current prediction and switching to different candidates are rolled out from the same tracker state. Differences in their subsequent mean IoU provide supervision for the decision network. At inference, the network estimates switching benefits to select a recovery candidate or retain the local prediction. Extensive experiments demonstrate the effectiveness of the proposed framework comparing to the conventional trackers.
Critical strengths
-
The proposed framework is technically sound, and the paired rollout design provides a clear supervision signal for recovery decisions. By comparing staying and switching from the same tracker state, it directly measures the subsequent benefit of different switching decisions and generates labels for each decision using only existing bounding-box annotations. A lightweight decision network predicts these benefits at inference, enabling informed recovery decisions with little additional computational cost.
-
ReTrack improves over the base tracker across all four main benchmarks and achieves the highest AUC on three. The component ablations further support the contributions of full-frame recovery and the learned decision policy.
Critical weaknesses
- The switching behavior and temporal stability of the recovery mechanism are insufficiently characterized. The paper does not report the average number of switches per sequence, the spatial displacement associated with each switch, or the full-sequence trajectory visualizations. These statistics are needed to characterize switching frequency and abrupt trajectory changes during difficult sequences. Additional experiments and analysis would help establish the stability of the recovery mechanism.
Questions for authors
-
Could the authors provide sensitivity analyses for the confidence threshold and periodic interval used to trigger decision points?
-
Given that the full-frame proposal head is trained with bounding-box localization and confidence losses, how does it distinguish the designated target from other objects, and what evidence demonstrates target identity recognition beyond reliance on spatial cues?
-
Could the authors provide full-sequence trajectory visualizations and quantitative analyses of switching behavior, including the average number of switches per sequence and the displacement of each switch, to assess recovery stability and repeated global relocalization?
4 comments
Response to Reviewer bhFx (2/2)
Q2: Given that the full-frame proposal head is trained with bounding-box localization and confidence losses, how does it distinguish the designated target from other objects, and what evidence demonstrates target identity recognition beyond reliance on spatial cues?
The full-frame head proposes target-like locations, and the policy identifies the target among them by comparing each proposal with the current track.
(1) Evidence beyond spatial cues (Tab. 5). The policy compares each proposal with the current track through four groups of relative features: tracker reliability (the tracker confidence and its smoothed value), proposal quality (the proposal score and its rank), the spatial relation (the center distance and the scale ratio), and the time since the last switch. The tracker confidence is derived from matching the initial template with the search region, so it carries the appearance evidence of the target. Removing the tracker confidence lowers AUC from 49.07% to 43.17% (Tab. 5(b)), whereas removing the geometric features lowers AUC to 46.69% (Tab. 5(d)). Therefore, the policy relies more on the appearance evidence of the template than on the spatial cues.
(2) Evidence with similar distractors. UAV-Anti-UAV annotates whether a sequence contains a visually similar distractor. The table below reports the AUC on the two subsets.
| Method | All sequences (420) | Similar distractor (222) | No similar distractor (198) |
|---|---|---|---|
| LoRAT-ID | 42.22 | 37.06 | 48.02 |
| ReTrack without switching | 41.21 | 36.93 | 46.02 |
| Hand-crafted rule | 45.97 | 40.24 | 52.40 |
| ReTrack | 49.07 | 43.60 | 55.20 |
ReTrack achieves the highest AUC on both subsets, and its margin over the hand-crafted rule is larger with a similar distractor (3.36% versus 2.80%), where comparing each proposal with the current track is most critical.
In Fig. 6(a) of the revised paper, the current track has drifted onto a car with a confidence of 0.37, the full-frame head responds most strongly at the drone, and the policy switches to it. Over the next 30 frames with the target visible, ReTrack reaches a mean IoU of 0.80, while ReTrack without switching remains lost.
We hope these responses address the reviewer's questions, and we are glad to discuss any remaining points.
I thank the authors for their response. They have addressed my concerns, and I have no further questions.
Thank You to Reviewer bhFx
We thank the reviewer for confirming that our responses address the concerns, and for the positive assessment of ReTrack. The suggestions on switching behavior and on the decision-point settings led to Fig. 6 and Tab. 8 in §4.4 of the revised paper.
Response to Reviewer bhFx (1/2)
We thank the reviewer for the positive assessment and the constructive suggestions, and for finding the framework "technically sound" and the paired rollout design "a clear supervision signal for recovery decisions".
The switches are sparse and restore tracking after failures.
(1) Full-sequence visualizations. We added them to §4.4 of the revised paper as Fig. 6. For two long sequences of 5,236 and 7,444 frames, Fig. 6(b, c) plots the per-frame IoU of ReTrack and of the same network without switching. ReTrack recovers the target after most losses, raising the sequence AUC from 21.6% to 57.5% and from 14.5% to 55.3%, whereas the tracker without switching remains lost for most of each sequence.
(2) Switching statistics. The table below summarizes the full run on the 420 evaluation sequences of UAV-Anti-UAV. The policy switches at 4.4% of the decision points, and the mean number of switches exceeds the median because a few difficult sequences account for most switches.
A switch moves the box by a quarter of the image diagonal at the median, well beyond the local search region around a small target. Repeated relocalization appears as several switches within a short time. For every group of switches spaced less than 30 frames apart, we measure the IoU 30 frames after the last switch of the group, at the 1,159 such frames with the target visible. ReTrack achieves a mean IoU of 0.479 at these frames, compared with 0.189 for the same network without switching, indicating that repeated switches converge on the target rather than resulting in drift.
ReTrack is insensitive to both settings, and AUC stays between 48.78% and 49.09% across a wide range of these settings. The two settings only decide which frames become decision points, and the learned policy then decides whether and to which proposal to switch. The table below varies the confidence threshold csoft and the period M of the decision points on UAV-Anti-UAV while keeping the tracker and the policy fixed.
From csoft = 0.4 to 0.7 and from M = 5 to no periodic trigger, the decision points range from 20.9% to 37.2% of the frames, while AUC changes by at most 0.29%. Even the extreme settings, csoft = 0.8 with 57.8% of the frames as decision points and a decision at every frame, reach 48.07% and 46.84% AUC. Thus, the reported setting lies within a stable range rather than at a sharp optimum. We added this analysis to §4.4 of the revised paper (Tab. 8).