CSPTrack: Causal State Prediction with Ridge-Aware Token Pruning for Visual Tracking

Current template–search trackers often fail to sufficiently model the temporal evolution of target states, lack effective constraints for maintaining consistent target representations across frames, and densely process a large number of redundant search tokens, thereby limiting both tracking robustness and computational efficiency. To address these issues, we propose CSPTrack, a robust and efficient visual tracking framework with three complementary designs. First, the causal state prediction module leverages current search features and historical hidden states to explicitly predict short-term future-state representations, thereby modeling the temporal evolution of target states. Second, the instance-consistency objective brings the template and search representations of the same target closer while separating the representations of different target instances, enhancing both cross-frame representation consistency and target discrimination. Third, the ridge-aware token pruning strategy evaluates the importance of search tokens based on ridge-reconstruction errors, selectively retains informative search tokens, aggregates information from discarded tokens, and preserves all template tokens. Extensive experiments on six challenging benchmark datasets demonstrate the effectiveness and efficiency of CSPTrack. Notably, without token pruning, CSPTrack-B224 achieves a new state-of-the-art performance of 75.4% AUC on LaSOT under similar settings. Even after reducing the number of search tokens by 30%, CSPTrack-B224 incurs only a marginal 0.2% AUC drop on LaSOT while improving tracking speed by 10%. These results demonstrate that CSPTrack effectively reduces redundant computation while maintaining high tracking accuracy.