PureTrack: Template Purification with Temporal Learning for Efficient Tracking

Efficient tracking requires a careful balance between inference speed and tracking accuracy. Many efficient trackers adopt a single fixed template to reduce computational cost. However, background and distractor content in the template can become entangled with the target representation and induce false responses during subsequent template-search interactions. In addition, a fixed template cannot effectively adapt to appearance changes over time. To address these issues, we propose PureTrack, an efficient framework that combines template purification with temporal learning to improve template reliability and temporal adaptability. First, PureTrack decouples template initialization from subsequent template-search interaction and establishes an independent template-processing stage. An Attention-Guided Template Purification (AGTP) module then uses global template-relevance cues from the spatio-temporal token to suppress background interference while preserving the complete spatial structure of the template, producing more reliable response maps. Second, the spatio-temporal token is propagated across frames as a compact state to supplement the temporal context missing from the fixed template. Finally, building on the cleaner response maps produced by AGTP, we introduce a Bidirectional Temporal Consistency (BTC) learning paradigm to enforce cross-frame localization consistency. BTC spatially aligns response maps according to target displacement and imposes bidirectional consistency constraints during training, improving localization stability under target motion without introducing additional inference-time computation. Experiments show that PureTrack improves the reliability and temporal stability of fixed-template representations while achieving a favorable accuracy-efficiency trade-off across different platforms. It achieves 72.5% AO on GOT-10k and 208/76/84 FPS on GPU/CPU/AGX.