acceptodds
Withdrawn from ICLR 2027

PureTrack: Template Purification with Temporal Learning for Efficient Tracking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Efficient tracking requires a careful balance between inference speed and tracking accuracy. Many efficient trackers adopt a single fixed template to reduce computational cost. However, background and distractor content in the template can become entangled with the target representation and induce false responses during subsequent template-search interactions. In addition, a fixed template cannot effectively adapt to appearance changes over time. To address these issues, we propose PureTrack, an efficient framework that combines template purification with temporal learning to improve template reliability and temporal adaptability. First, PureTrack decouples template initialization from subsequent template-search interaction and establishes an independent template-processing stage. An Attention-Guided Template Purification (AGTP) module then uses global template-relevance cues from the spatio-temporal token to suppress background interference while preserving the complete spatial structure of the template, producing more reliable response maps. Second, the spatio-temporal token is propagated across frames as a compact state to supplement the temporal context missing from the fixed template. Finally, building on the cleaner response maps produced by AGTP, we introduce a Bidirectional Temporal Consistency (BTC) learning paradigm to enforce cross-frame localization consistency. BTC spatially aligns response maps according to target displacement and imposes bidirectional consistency constraints during training, improving localization stability under target motion without introducing additional inference-time computation. Experiments show that PureTrack improves the reliability and temporal stability of fixed-template representations while achieving a favorable accuracy-efficiency trade-off across different platforms. It achieves 72.5% AO on GOT-10k and 208/76/84 FPS on GPU/CPU/AGX.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Efficient Tracking, Token Learning, Temporal Consistency Learning, Spatial-Temporal consistency

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wnSzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PureTrack combines three ideas for efficient single-object tracking: purifying the initial template once, passing a compact spatio-temporal token between frames, and applying a bidirectional response-map consistency loss during training. The template is then cached rather than re-encoded at every frame. The paper reports 72.5 AO on GOT-10k and 208/76/84 FPS on GPU/CPU/Jetson AGX.

Critical strengths
  1. Separating template initialization from online tracking is a sensible way to cut repeated computation. In Table 3, the decoupled version improves GPU speed from 162 to 211 FPS while also slightly improving AO.

  2. The evaluation covers several tracking datasets and reports speed on three platforms. The AGX measurements are particularly relevant to the paper's efficiency objective.

Critical weaknesses
  1. The ablation study does not clearly separate the effects of the proposed components.

a) I had trouble reconciling Eq. (1) with Table 3. AGTP uses attention from the initial spatio-temporal token to the template tokens, yet configurations #3 and #7 include AGTP with the “s-t token” column unchecked. Does that column mean that cross-frame propagation is disabled, rather than that the token is absent? If so, the table should say so. This distinction matters when interpreting the small change from #7 (72.3 AO) to #8 (72.5 AO).

b) The evidence for the claimed complementarity between AGTP and BTC is also weak. AGTP improves AO from 71.2 to 71.8 when added to the decoupled model, but adds only 0.1 point when BTC is already present (72.2 to 72.3). The joint model is slightly better, but this alone does not establish a meaningful interaction between the two components.

c) Temporal token propagation is not new to tracking; ODTrack and LETrack are relevant examples. The paper cites related work, but I would like a clearer account of what the proposed token adds beyond these approaches, particularly given its 0.2-point gain in the reported ablation.

  1. The training setup and some reported comparisons need clarification.

a) Section 4.1 lists five training datasets, including LaSOT, TrackingNet, COCO2017, GOT-10k, and VASTTrack. It is not clear whether the GOT-10k result and the ablations in Table 3 come from a GOT-10k-only model or the jointly trained model. This should be stated explicitly because it affects how the results can be compared with other trackers.

b) The baseline in Table 3 already reaches 70.6 AO, but the source of the Fast-iTPN-Tiny pretrained weights and the baseline initialization are not specified clearly enough. Since the full method improves on this baseline by 1.9 points, these details are important for interpreting the gains and reproducing the results.

c) There is also a concrete error in the TrackingNet discussion on page 6. ARTrack is called the fastest non-real-time tracker, and PureTrack is said to be 4.4 times faster than ARTrack on AGX. Table 1 instead gives 84 FPS for PureTrack and 8 FPS for ARTrack-256, a ratio of 10.5. The 4.4 ratio matches the comparison with OSTrack-256 (19 FPS), which is itself faster than ARTrack-256 on all three reported platforms. The text appears to mix up the two trackers and should be corrected.

Questions for authors
  1. In Table 3, does “s-t token” refer specifically to recurrent propagation? What token is used to compute AGTP in configurations #3 and #7?

  2. What is the main technical difference between PureTrack's temporal token and those used in ODTrack and LETrack? Is there additional evidence of a benefit beyond the 0.2 AO gain in Table 3?

  3. Were the GOT-10k results and ablations obtained with GOT-10k-only training? What pretrained weights and initialization were used for the 70.6 AO baseline?

  4. Please check the ARTrack/OSTrack speed comparison on page 6 against Table 1 and correct the ranking and speed ratio.

  5. Some Mistakes a) The Figure 3 caption on page 9 begins “isualization”; it should read “Visualization”. The abbreviations “w” and “w/o” could also be made consistent. b) The conclusion contains “spatio-temporal token serve”; the number and verb agreement should be corrected.

Reviewer_N6RfOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PureTrack, an efficient framework combining template purification with temporal learning for fixed-template tracking. It decouples template initialization from subsequent template-search interaction and introduces an Attention-Guided Template Purification (AGTP) module to suppress background interference while preserving the template’s spatial structure. A lightweight spatio-temporal token propagates target-relevant context across frames. In addition, Bidirectional Temporal Consistency (BTC) learning aligns response maps according to ground-truth target displacement and enforces cross-frame localization consistency during training, without introducing additional inference-time computation. PureTrack achieves 72.5% AO on GOT-10k and 208/76/84 FPS on GPU/CPU/AGX.

Critical strengths
  1. The paper offers a perspective on efficient visual tracking by emphasizing template quality and temporal adaptability.
  2. The proposed approach achieves a favorable balance between tracking accuracy and inference efficiency, with promising results across multiple benchmarks and hardware platforms.
Critical weaknesses
  1. AGTP uses attention from a learnable token to reweight template features, and the paper does not analyse why these attention weights reliably distinguish target-relevant content from background distractors. The author needs more precise justification for this module.
  2. It is unclear whether BTC remains reliable under scale variation, deformation, or occlusion, since the response maps may differ even after translation alignment. This concern becomes more relevant for larger temporal gaps. The paper should specify the frame-sampling interval and examine how BTC behaves across different temporal gaps and challenging attributes.
  3. Lack of ablation studies on important hyperparameters and internal module designs, such as the BTC loss weight and reweighting strategies in AGTP.
  4. Comparisons with relevant recent methods are incomplete. Other relevant recent approaches, such as LETrack and UETrack from CVPR 2026, are absent from the experimental comparisons.
Questions for authors
  1. Do the compared methods use the same backbone and training datasets, particularly regarding the use of VastTrack? The authors should clarify any differences in these settings and explain whether they ensure a fair comparison.
  2. why don't you provide more detailed ablation analyses on the hyperparameter and module design?
Reviewer_sxUROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Considering the background and distractor content in the template, this paper presents PureTrack, an efficient fixed-template tracking framework that addresses background interference and limited temporal adaptability. PureTrack decouples template initialization from subsequent template-search interaction and establishes an independent template-processing stage. An Attention Guided Template Purification (AGTP) module then uses global template-relevance cues from the spatio-temporal token to suppress background interference. Then, the spatio-temporal token is propagated across frames as a compact state to supplement the temporal context missing from the fixed template. Finally, authors introduce a Bidirectional Temporal Consistency (BTC) learning paradigm to enforce crossframe localization consistency.

Critical strengths

PureTrack combines a simple architecture designed for practical deployment with reported results showing a competitive balance between accuracy and efficiency across multiple benchmarks and hardware platforms.

Critical weaknesses

Several methodological details are insufficiently specified, and inconsistent notation reduces the clarity and overall quality of the manuscript.

Questions for authors

1.PureTrack uses an initial spatio-temporal token to obtain the purified template, but how is this token initialized? Is it randomly initialized? If so, how does AGTP learn to emphasize target features and suppress background information when processing the initial template?

2.The spatio-temporal token is continuously updated across frames. If the target undergoes motion blur in a particular frame, the updated token may incorporate background information irrelevant to the objective and affect the object feature extraction in subsequent frames. How does PureTrack prevent or mitigate such error propagation?

3.The equations and figures require more careful and consistent presentation. Equation (1) obviously contains an error, and the bidirectional temporal consistency loss shown in Figure 2 uses notation that is inconsistent with Equation (6).

4.Although PureTrack achieves high inference speeds, the experimental evaluation lacks comprehensive comparisons with the latest and the state-of-the-art tracking methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.