acceptodds
Withdrawn from ICLR 2027

Spatial-Frequency Collaborative Mamba Tracker for Visual Object Tracking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual object tracking has achieved substantial progress, yet robust tracking in complex dynamic scenes remains chal lenging because spatial-domain appearance representations are vulnerable to fast motion, illumination variations, oc clusions, and background distractors. Existing Mamba-based trackers improve long-range dependency modeling and com putational efficiency, but mainly focus on spatial-temporal representations, leaving the temporal evolution of frequency domain motion information insufficiently explored. To ad dress this limitation, we propose a spatial-frequency collab orative tracking framework based on Mamba, termed SFMT, which jointly models spatial-temporal appearance representa tions and frequency-domain motion cues. First, a Frequency Domain Awareness (FDA) module employs Mamba to model the temporal evolution of phase spectra and uses the re sulting motion cues to guide amplitude filtering. Second, a Spatial-Frequency Fusion (SFF) module performs global sequence-level and local point-level interaction to effectively integrate complementary information across the two domains. Finally, a pyramid multi-scale decoder enhances high- and low-frequency representations and dynamically balances their contributions through gated adaptive fusion for accurate tar get localization. Extensive experiments on six challenging benchmarks demonstrate the effectiveness of SFMT, achiev ing competitive tracking performance, including an AUC of 74.1% on LaSOT and an AO of 77.2% on GOT-10K.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual object tracking, Mamba, Frequency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eEFoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SFMT, a spatial-frequency collaborative framework for single-object visual tracking. Its Frequency-Domain Awareness (FDA) module models temporal phase information using Mamba and uses the resulting features to guide amplitude filtering. A Spatial-Frequency Fusion (SFF) module combines global sequence-level interaction with local point-wise interaction, followed by a multi-scale decoder for target localization. Experiments on six benchmarks show competitive performance, including 74.1% AUC on LaSOT and 77.2% AO on GOT-10K. The reported ablations support the effectiveness of the overall design, although the evidence for the proposed motion-modeling mechanism remains incomplete.

Critical strengths
  1. The paper investigates temporal phase information as a complement to spatial appearance features. This is a meaningful direction for tracking under appearance changes and motion-related disturbances. The separation of phase modeling, amplitude filtering, and spatial-feature fusion provides a coherent design rationale.
  2. FDA, SFF, and the multi-scale decoder address distinct stages of the tracking pipeline. In particular, combining global interaction with local correspondence is a reasonable way to integrate the two feature branches. The architecture presents a concrete implementation of spatial-frequency collaboration rather than relying on frequency information only as an auxiliary training signal.
  3. The paper evaluates SFMT on six datasets. It achieves the highest LaSOT AUC among the methods listed in Table 1 and remains close to the leading reported results on TrackingNet and UAV123. This breadth of evaluation provides useful evidence that the approach is not effective only on a single benchmark.
Critical weaknesses
  1. Table 3 only adds FDA, SFF, and the decoder in a fixed order. It does not separate temporal phase modeling from amplitude filtering, additional historical information, or increased model capacity. Comparisons against current-frame-only phase processing, guidance-free amplitude processing, and a capacity-matched spatial branch are missing. Similarly, the global/local fusion paths and the decoder’s multi-scale and gating components are not evaluated separately.
  2. Equation (2) constructs a global sequence with (2N_x) tokens but subsequently reshapes the Mamba output into an (H\times W\times C) feature map without explaining how the token count is reduced. Figure 3 also depicts an addition operation in the amplitude path, whereas Equation (1) specifies multiplicative gating. The use of the original phase for inverse FFT is not clearly reflected in the diagram. Furthermore, the subsequent use of template features is insufficiently described.
  3. The robustness and scope of the performance gains remain unclear. The LaSOT improvement over DreamTrack is 0.3 percentage points, but no repeated-run variability is reported to contextualize this margin. SFMT also trails the best listed methods by 4.3 points on GOT-10K AO and 5.1 points on TNL2K AUC.
Questions for authors

Please see weakness above.

Reviewer_acRuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses visual object tracking under motion, occlusion, and appearance changes through a spatial–frequency collaborative Mamba architecture. Its Frequency-Domain Awareness module models temporal phase spectra to guide amplitude filtering, while a dual-path fusion module and a gated multi-scale decoder support localization. Evaluation covers LaSOT, LaSOT-ext, TrackingNet, GOT-10K, UAV123, and TNL2K, reporting 74.1% LaSOT AUC and 77.2% GOT-10K AO. The results demonstrate competitive tracking performance, but do not establish that phase-based motion modeling explains the gains.

Critical strengths
  1. Temporal phase modeling provides a plausible inductive bias. Using phase evolution to condition amplitude filtering is more technically motivated than generic frequency-feature concatenation and could provide useful motion-sensitive representations.
  2. The reported LaSOT improvement is substantial over the internal baseline. The increase from 71.0% to 74.1% AUC suggests that the complete architecture benefits long-sequence tracking, although the contribution of its central mechanism remains unresolved.
Critical weaknesses
  1. The motion interpretation is not established. The Fourier shift theorem concerns translated signals, whereas independently cropped search regions contain changing backgrounds, scale changes, and crop-coordinate shifts; without alignment controls or motion-specific experiments, temporal phase variation cannot be attributed to target displacement.
  2. Figure 3 routes processed phase to iFFT, while the text specifies original phase, and Figure 4 depicts initial addition rather than the stated dual concatenations.
  3. GOT-10K protocol compliance is unclear. Section 4.0.1 specifies joint training on COCO, LaSOT, GOT-10K, and TrackingNet without identifying a separately trained GOT-10K-only model; consequently, comparability under the standard restricted-training protocol is unverified. The benchmark’s one-shot design specifically targets generalization to unseen categories. GOT-10K benchmark paper
  4. The experiments do not isolate the claimed advance over existing frequency-aware tracking. FPDT already combines frequency-based denoising, spatial–frequency interaction, and multi-scale decoding; SFMT’s principal distinction therefore rests on temporal phase modeling, yet Table 3 provides only cumulative ablations without phase-free, shuffled-history, or capacity-matched spatial controls.
Questions for authors
  1. What exact operations reduce the doubled sequence length in Equation (2), and which implementation—equations, figures, or prose—produced the reported results?
  2. It would strengthen the paper if the authors compared temporal phase modeling against current-frame-only phase, shuffled phase histories, a capacity-matched spatial temporal branch, and motion-specific performance.
Reviewer_udfBOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, the authors propose a spatial-frequency collaborative tracking framework based on Mamba, which jointly models spatial-temporal appearance representations and frequency-domain motion cues. The idea is simple and the proposed method is easy to unstandard. The experimental results show that the method gets some improvement on several tracking benchmarks.

Critical strengths

[1] The method is simple and easy to understand, and the experiments also show some improvement.

Critical weaknesses

[1] Frequency-domain information has already been widely used in tracking; the authors do not present any novel ideas, nor do they provide a comparative analysis against existing tracking methods that also leverage frequency-domain information. [2] The authors merely propose a different module for utilizing frequency-domain information, yet they lack a theoretical analysis of this module,specifically, what kind of information it actually learns and why it is able to improve tracking performance. [3] The proposed method appears to be a general-purpose module, but it lacks validation across multiple baseline trackers.

Questions for authors

[1] As seen in Figures 2 and 3, the inputs to the architecture consist of individual template and search images; so, how is FDA able to learn motion information? Furthermore, how can one verify that what has been learned is indeed motion information, rather than something else? [2] The dual-branch fusion module also lacks theoretical and experimental ablation analysis. [3] If the FDA has learned motion information, it should demonstrate superior performance regarding certain motion-related attributes; the authors need to provide a detailed verification and analysis of this.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.