- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Vision State-Space Token Mixer (ViS), a state-space operator retaining 2D data format. Instead of flattening an image into a 1D sequence with a handcrafted scan order, ViS lets every token run its own recurrence over spatial Chebyshev-distance rings around it, inputting rings from farthest to nearest. This position-wise recurrence, under a scalar value parameterization of the matrix A in SSM, admits an exact closed form as structured masked attention within the SSD framework (Mamba-2), enabling GEMM-friendly parallel computation. To restore directional information lost by the isotropic formulation of the ring, the authors add a sector gain on the mask and a rotary embedding on the B/C projections. Experiments on classification (ImageNet-1k), detection (COCO), semantic segmentation (ADE20K), and class-conditional image generation (ImageNet-1k) show superior accuracy and zero-shot resolution transfer.
Critical strengths
- Novelty and soundness of the spatial state space model: The reframing of SSMs as per-token context aggregation over spatial distance rather than sequence position after flattening patches is conceptually novel, and the reviewer agree with the merit.
- zero-shot resolution robustness: The injected spatial prior (direction rather than absolute grid position) is well-motivated to be robust to the resolution difference and actually gives the best accuracy at every tested resolution up to 768², with a widening margin.
Critical weaknesses
- Claims regarding inference speed might be overstated: Although ViS computes point-wise recurrences, it claims to achieve high speed through GEMM operations. However, the primary baselines are Mamba-based methods that especially combine 1D scans with 2D convolutions, which incur overhead from data format conversions. The reviewer believes the paper should also discuss methods such as MambaVision [1], SF-Mamba [2], and Adventurer [3], which do not use 2D convolutions and can better leverage the computational efficiency of Mamba itself. Furthermore, including the convolution-based MambaOut in Fig. 6 seems to suggest that ViS loses to MambaOut in terms of the throughput-performance trade-off. While ViS is parameter-efficient, is it also throughput-efficient?
- Insufficient technical details: The paper states that it adopts a Mamba-2-style computation, but several implementation details remain unclear. For example, does it use multiple heads for the hidden state at each point, and what is the head dimension? Could the point-wise hidden state be achieved while reducing computational costs by adjusting these parameters?
[1] MambaVision: A Hybrid Mamba-Transformer Vision Backbone, CVPR2025
[2] SF-Mamba: Rethinking State Space Model for Vision, ICML2026
[3] Adventurer: Optimizing Vision Mamba Architecture Designs for Efficiency, CVPR2026
Questions for authors
- Details of the experimental setup: Could the authors provide more details about the state space model configurations used in the experiments?
- Ablation study on computational cost: Given the use of point-wise hidden states and a quadratic-complexity computation scheme, ViS appears to lose the advantages that state space models and linear attention typically have over full attention. I would be interested in seeing how inference speed and FLOPs scale with increasing image resolution under both the quadratic and recurrent formulations.
- Robustness to resolution changes: Previous studies have shown that incorporating RoPE can improve robustness to changes in resolution [4]. Comparing ViS with methods that incorporate RoPE, such as RoPE-ViT [4] and MILA [5], could help establish whether the proposed 2D state space model offers additional benefits beyond those attributable to RoPE alone.
[4] Rotary Position Embedding for Vision Transformer, ECCV2024.
[5] Demystify Mamba in Vision: A Linear Attention Perspective, NeurIPS2024.