acceptodds
Withdrawn from ICLR 2027

ReliTrack: Pseudo-State Reliability Learning for Self-Supervised Visual Tracking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Self-supervised visual tracking reduces reliance on dense bounding-box annotations by deriving training signals from unlabeled videos. A key challenge, however, is that model-generated target states are not equally reliable: inaccurate pseudo-states can corrupt the target context used for subsequent self-supervised optimization. To address this issue, we propose ReliTrack, a reliability-aware self-supervised tracking method designed to mitigate pseudo-state contamination. ReliTrack first generates a forward pseudo-trajectory and estimates its reliability from the sharpness of the corresponding localization responses. Concentrated responses indicate confident target localization, whereas flat or ambiguous responses suggest unreliable pseudo-states. The resulting frame-level estimates are aggregated into a trajectory-level reliability score, which is converted into a conservative soft weight to regulate backward localization learning. This weighting suppresses contaminated supervision without discarding training samples. The reliability estimation requires neither additional annotations nor auxiliary networks and introduces no inference-time computation. Experiments demonstrate consistent improvements in tracking performance, while further analysis shows that ReliTrack shifts the effective training signal toward higher-quality pseudo-trajectories and reduces the contribution of inaccurate ones.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision tracking, self supervised

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QETROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReliTrack, a reliability-aware self-supervised visual tracking framework that mitigates pseudo-state contamination in forward-backward training. It estimates pseudo-state reliability from response-map sharpness, aggregates frame-level confidence into a trajectory-level score, and uses the resulting soft weight to regulate the backward localization loss. The method consistently improves over the reproduced SSTrack baseline on four benchmarks.

Critical strengths
  1. The paper identifies a practically relevant issue in self-supervised tracking, namely that inaccurate intermediate predictions can contaminate subsequent supervision.

  2. The proposed reliability weighting is simple, lightweight, and introduces no additional inference-time cost.

  3. The correlation analysis between response sharpness and pseudo-state IoU provides useful empirical evidence for the proposed reliability signal.

Critical weaknesses
  1. The proposed reliability measure is essentially a PSR-inspired standardized peak statistic, followed by a simple linear loss reweighting scheme. While effective, the technical contribution appears incremental relative to the SSTrack framework on which it is built.

  2. The ablation study varies the aggregation strategy and hyperparameters, while keeping the reliability cue fixed. Comparisons with alternative uncertainty signals, such as response entropy, maximum confidence, peak margin, or consistency-based confidence, would better justify the specific choice of response sharpness.

  3. Each training sample contains only three forward search frames, although the method is formulated around trajectory-level reliability and temporally propagated pseudo-state contamination. Experiments with longer pseudo-trajectories would better demonstrate whether the proposed weighting remains effective under accumulated tracking errors.

  4. A sharply peaked response can still correspond to a confident but incorrect distractor, especially after target drift. The current correlation analysis shows overall effectiveness but does not sufficiently characterize such high-confidence failure cases.

  5. The main table includes only TADS and SSTrack in the self-supervised category, and the proposed method is primarily evaluated as an improvement over SSTrack. Broader comparisons or experiments on additional self-supervised tracking frameworks would better establish the generality of the proposed reliability mechanism.

  6. Some improvements are relatively modest, particularly on TrackingNet, where the gains over SSTrack are below approximately one point for several metrics. Reporting results over multiple runs or confidence intervals would help establish the robustness of these improvements.

Questions for authors

Please refer to Weaknesses.

Reviewer_oYavOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ReliTrack addresses errors in model-generated pseudo-states used as target context in self-supervised tracking by making each training sample’s influence depend on estimated trajectory reliability. It aggregates tracking-response sharpness along each forward trajectory to down-weight the backward localization loss for samples with less reliable context, modifying the training objective without adding inference computation. Benchmark comparisons report improvements over the underlying tracker, while hyperparameter ablations and analyses of pseudo-state quality examine whether the reliability signal identifies useful training examples and produces the intended weighting effect.

Critical strengths

The reported gains over reproduced SSTrack occur across all four benchmarks, including 3.10 points in GOT-10k AO and 1.53 points in LaSOT AUC, without additional inference cost. The analysis provides evidence that sharpness is informative: Section 4.4 reports a correlation of 0.681 with pseudo-state IoU, and Table 3 shows that weighting shifts the effective training signal toward more accurate pseudo-trajectories.

Critical weaknesses

The evaluation does not sufficiently establish that the reported gains generalize beyond the settings used to select the weighting scheme. Section 4.3 motivates the adopted aggregation function, minimum weight, and sharpness scale using their performance on LaSOT and LaSOT-extension, without identifying a separate validation split for these choices. This omission is consequential because performance is sensitive to the selected settings: changing the sharpness scale from 16 to 12 reduces LaSOT AUC from 66.37 to 63.91, a 2.46-point decrease compared with the reported 1.53-point gain over SSTrack. Independent-run variability is also not reported. Consequently, the manuscript leaves unresolved how much of the improvement reflects a robust training benefit under an independently selected configuration. A documented selection protocol and evaluation of the resulting fixed configuration are central to the empirical claim.

The experiments do not sufficiently establish the contribution of the proposed reliability measure beyond a basic confidence-weighting approach. The method combines a PSR-inspired response statistic with bounded loss weighting, while all ablations in Table 2 retain the same reliability cue. The correlation analysis and effective-IoU measurements demonstrate that the cue contains useful information, but they do not establish whether this particular construction offers a meaningful advantage over simpler measures such as peak response or response entropy. Since reliability estimation and weighting constitute the main methodological addition, comparing these alternatives under the same weighting function and selection protocol is important for assessing the contribution. The current evidence establishes improvement over uniform weighting but leaves the value of the specific proposed design insufficiently resolved.

Questions for authors
  1. Clarification: Which data selected the aggregation function, λ, and τ? Were LaSOT or LaSOT-extension evaluation results used in those decisions, and how much do the gains vary across independent training runs?
  2. How does response sharpness compare with peak response and response entropy when the weighting function, training settings, and hyperparameter-selection protocol are held constant?
Reviewer_3nGNOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper tackles self-supervised visual tracking built on the SSTrack forward–backward paradigm, where pseudo trajectories generated by the tracker itself provide supervision. It argues that these pseudo states suffer from pseudo-state contamination: unreliable states are self-generated, evolve during training, and propagate over time, while uniform averaging in the baseline lets low-quality trajectories exert the same optimization influence as high-quality ones.

Critical strengths

Treating pseudo-state contamination as endogenous noise — self-generated, evolving with the model, and propagating along time — rather than static label noise is a correct and well-argued characterization of the self-supervised tracking setting.

Critical weaknesses
  1. The most closely related work — ULAST (Shen et al., 2022) — is not cited. ULAST performs forward–backward unsupervised tracking with quality-based dynamic loss reweighting (mask-guided reweighting), structurally isomorphic to ReliTrack's core mechanism. The Related Work section only surveys cycle/synthetic/contrastive lines and never engages this prior art, which directly undermines the novelty claim in Contribution 1. This is the single most damaging issue and would be flagged in the first round of review.

  2. No "remove-the-weighting" ablation is reported. Table 2 ablates aggregation/λ/τ but lacks a w(q)≡1 control row; the effect of the 20-epoch warm-up is never isolated from the weighting itself, so the contribution of the core mechanism cannot be attributed.

  3. No variance / multiple-seed reporting. All results are single runs; gains vary widely across benchmarks (+3.10 vs +0.86), making it impossible to rule out training noise for the smaller gains.

  4. the key assumption is that eq10, how to make the constraint Cov(w(q), D) < 0 hold?

Questions for authors

Does the sharpness–quality correlation (Fig 4, Spearman 0.679) still hold on the weighted trained model, given that weighting alters the response distribution? If it drifts, how robust is the fixed threshold τ=16?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.