acceptodds
Withdrawn from ICLR 2027

SRTrack: Self-Recovering Visual Tracking via Re-Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual object tracking has made significant progress through spatio-temporal context modeling. However, current approaches lack explicit spatial priors from historical predictions to guide localization, relying on feature similarity for attention computation. More critically, existing trackers implicitly assume reliable predictions throughout the sequence and lack specific mechanisms to detect and recover from tracking failures. To address these challenges, we propose SRTrack, a self-recovering tracking framework. For failure recovery, a tracking quality estimator monitors confidence at every frame and triggers a re-localization module to predict a coarse anchor on an extended search region. For robust localization, a trajectory-guided feature sampler leverages historical positions as geometric priors for feature aggregation, and a feature refinement head performs coarse-to-fine localization through iterative deformable attention with Gaussian gates. Experiments on eight benchmarks demonstrate that SRTrack achieves competitive results, with an AO of 80.9% on GOT-10k and an AUC of 74.4% on LaSOT, surpassing HIPTrack by 3.5% and 1.7%, respectively. The code and results will be made publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual object tracking, tracking failure recovery, geometric priors, re-detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_txRPOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

SRTrack builds on a one-stream ViT tracker and aims to make it recover from target loss and localize more precisely. It adds a quality estimator that triggers a re-localization module on a larger search region, a deformable-attention sampler guided by the past boxes, and a three-stage refinement head. With a frozen ViT-L backbone, the paper reports 80.9 AO on GOT-10k and 74.4 AUC on LaSOT, above HIPTrack, and results on eight benchmarks at 24.6 FPS.

Critical strengths
  • Recovering from target loss is a relevant problem for local-search trackers, and the re-localization module is lightweight compared with a full re-detector.
  • Results are reported on eight benchmarks, with an ablation for each added module.
Critical weaknesses
  • The recovery contribution is limited in both design and effect. Failure detection is a separately trained IoU estimator with a hand-set threshold, and recovery searches an enlarged crop around the previous box rather than the full frame used by earlier verifier-based trackers such as LTMU and SPLT, so a target that reappears far away cannot be recovered. The estimator is never compared with the tracker's own confidence score. The effect is also small: Table 3 shows 0.3 AO on GOT-10k and nothing on LaSOT, and the crop size barely changes the UAV123 results, so the gains over HIPTrack come mainly from the larger backbone, the refinement head and the sampler.
  • The evaluation is tilted toward the method. The failure threshold is set per dataset and is sensitive, and the sweeps for it and the other hyperparameters are run on the evaluation benchmarks themselves. I am not asking for state-of-the-art results, but RTracker, the one directly comparable recovery method, is higher on LaSOT and is compared only by parameter count, and the strongest trackers in Table 1 are absent from the OTB2015 and NFS30 tables and from the UAV123 attribute analysis, where SRTrack is reported as best.
Questions for authors
  • Why use a separately trained quality estimator with a dataset-specific threshold instead of the tracking head's confidence or a learned failure decision, and how do they compare?
  • How does recovery behave when the target reappears outside the enlarged crop, and what is the trade-off between crop size, cost and recovery?
  • Can you show where the sampling points and Gaussian gates fall on real frames, and whether the refinement gain holds against a head of similar capacity?
  • Where do the frozen ViT-L weights come from, and what exactly is row 1 of Table 3?
Reviewer_6EDrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SRTrack adds confidence-triggered re-localization to a local visual tracker. An IoU estimator scores each prediction. When the score falls below a threshold, a DINOv2-based module proposes a coarse anchor in an enlarged search region, and the tracker runs again around that anchor. Historical boxes guide deformable feature sampling in the local tracker, which also uses a three-stage refinement head. SRTrack reports 80.9 AO on GOT-10k and 74.4 AUC on LaSOT, with results on six further benchmarks. The main question is how much of the gain comes from recovery after failure, as opposed to better localization during ordinary tracking.

Critical strengths
  1. Table 3 separates the gains from the local tracking components. After the ViT-L backbone change, the refinement head raises GOT-10k AO from 78.4 to 79.7, and trajectory-guided sampling raises it to 80.6. LaSOT AUC increases from 73.0 to 73.9 and then 74.4. Appendix Table A1 shows that deformable sampling alone is insufficient: Gaussian guidance contributes to the gain. These controls explain the localization improvements more convincingly than the overall comparison with HIPTrack.
  2. The recovery module only needs to propose a coarse location, and the existing local tracker then handles precise localization. This is a practical division of work. The eight-benchmark evaluation and Appendix Table A4 also help assess the system’s use in practice. The reported 24.6 FPS includes recovery calls, so it reflects the full inference procedure.
Critical weaknesses
  1. The evidence for SR’s recovery benefit is less conclusive than the evidence for the local tracking components. Adding SR changes GOT-10k AO from 80.6 to 80.9, LaSOT AUC from 74.4 to 74.4, and UAV123 AUC from 70.9 to 71.4 (Table 3). Figure 6 and Figure A3 provide attribute-wise results, Table 6 compares re-localizer designs, and Figure 7 and the appendix illustrate successful recoveries. However, the attribute plots compare complete trackers, and Table 6 evaluates the re-localizer’s direct predictions. Neither measures recovery frequency, delay, or harmful triggers separately within the full tracker. Small aggregate gains could conceal useful improvements on rare failures, and an event-level analysis would help determine whether this is the case. A comparison with the cited RTracker, SPLT, or LTMU, holding the local tracker and search budget fixed, would also help assess the coarse-anchor design. Table 5 reports trainable parameters, and Table A4 reports hardware-independent MACs alongside hardware-dependent FPS. These characterize cost but do not isolate the recovery mechanism’s contribution.
  2. The failure threshold materially affects the results, but its selection cannot be reproduced from the description. Section 4.1 uses different thresholds across datasets. In Figure A2, GOT-10k AO drops from 80.9 at 0.65 to 77.1 at 0.70, and UAV123 has a different optimum (Tables A2–A3). The paper does not identify the held-out data or criterion used to select the deployed thresholds. This omission is not evidence of test-set tuning. The protocol nevertheless needs clarification because the estimator determines both when to attempt recovery and which prediction to retain. Calibration, trigger rates, and the selection procedure are needed to judge whether the small net SR gain is stable across operating points.
Questions for authors
  1. Which split and objective were used to select the failure threshold for each benchmark? Please give the selected values and clarify whether the GOT-10k sweep in Figure A2 uses validation or test evaluation. Were all thresholds fixed before the final test results were obtained?
  2. Can the existing evaluation traces show how often SR triggers, how often the second prediction is accepted, and whether replacement improves or worsens overlap? Recovery rate and delay on identified target-loss events would help interpret the small average gains in Table 3.
  3. Section 3.2 defines normalized historical boxes, and Algorithm 1 specifies confidence-gated template updates. For the trajectory bank used during sampler training, are the boxes predictions or perturbed ground truth, and which crop coordinate system defines their normalization? At inference, does successful re-localization append the selected box, or reset or filter the preceding trajectory history?
  4. Is there an existing comparison with another recovery mechanism using the same local tracker and search budget? If so, please provide recovery and runtime results. A broader benchmark study can be left to future work. It is not required for the rebuttal.
Reviewer_LbCROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose SRTrack, a self-recovering tracking framework. It designs a tracking quality estimator monitors confidence at every frame and triggers a re-localization module for failure tracking recovery. A trajectory-guided feature sampler leverages historical positions as geometric priors for feature aggregation, and a feature refinement head performs coarse-to-fine localization through iterative deformable attention with Gaussian gates. Experiments on eight benchmarks demonstrate that SRTrack achieves competitive results.

Critical strengths
  1. The paper is clearly written and well-organized, with a straightforward presentation and easy to follow. The motivation is clear, and the overall framework is intuitive.
  2. The trajectory-guided sampling and iterative refinement designs are well motivated and effective. The tracking performance is competitive across the eight benchmarks, particularly on GOT-10k.
Critical weaknesses
  1. The contribution of the self-recovering module appears limited. Although the paper emphasizes recovery through re-detection as its main contribution, this module provides only a 0.3-point AO improvement on GOT-10k and no AUC improvement on LaSOT. The author should conduct more important experiments, such as recovery success rates when target is lost, to better demonstrate its effectiveness.
  2. How does the proposed module compare with existing recovery strategies under the same tracking framework? Comparisons with other re-detection strategies would help establish advantages of the proposed design.
  3. Although the paper reports an overall speed of 24.6 FPS and provides computational cost estimates, the runtime of the tracking and recovery processes remains unclear.
  4. Comparisons with relevant recent methods are incomplete. The main experimental comparisons lack recent 2026 methods. The authors should include more recent state-of-the-art trackers to provide a more comprehensive evaluation of the proposed method.

I encourage the author to provide detailed comment on my concerns above. My final recommendation will be made upon consideration of their rebuttal.

Questions for authors
  1. What are the speeds of the tracker without recovery, the re-localization module, and the individual proposed components? The author should make a further clarification.
  2. The recovery module is evaluated only within the HIPTrack-based framework. Can it consistently improve other trackers with different architectures and prediction heads?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.