- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper argues that the SSM state update can be read as one gradient-descent step on the quadratic energy , and concludes from this that stacking SSM layers amounts to successive optimisation steps. On that basis it proposes LoopMamba, a four-stage hierarchical backbone with a single block per stage, in which a weight-shared Self-loop Layer is applied four or eight times. Each iteration also receives an Iteration-specific Lightweight State Fusion module, which combines three dilated depth-wise convolutions over the hidden state with a per-iteration low-rank adapter. On ImageNet-1K, LoopMamba-T-4 and T-8 reach 79.7% and 80.2% with 8.1M and 8.5M parameters, and LoopMamba-B-4 and B-8 reach 82.1% and 82.4% with 17.8M and 18.6M parameters at 6.4 and 9.6 GFLOPs. The paper also reports Mask R-CNN 1× results on COCO, UperNet results on ADE20K, ablations of the fusion components and of loop depth, an analysis showing that a hidden-state entropy decreases with loop depth, which it offers as evidence for the energy view, and qualitative feature maps compared with VMamba.
Critical strengths
The design is simple and parameter-efficient: with one block per stage and shared weights across loop iterations, LoopMamba reaches 82.1–82.4% top-1 with fewer than 19M parameters, and the depth sweep in Table 5 shows that additional iterations buy accuracy with almost no growth in parameters. The component ablations in Tables 4 and 5 are also informative and candid, since they show that most of the gain comes from the per-iteration low-rank adapters rather than from repeating the shared transition.
Critical weaknesses
1. Neither the looped architecture nor its energy-based motivation is new or specific to the design. Sharing one block across depth while adding lightweight, non-shared parameters at each iteration is the core design of SReT [1] and MiniViT [2], which are neither compared nor discussed. SReT already reports the finding that LoopMamba presents in Table 5, that naive recursion barely helps, from 72.2% to 72.5% in SReT and from 79.0% to 79.3% in LoopMamba, and that the gain appears only with non-shared per-iteration parameters. SReT-S also reaches 81.9% with 20.9M parameters and 4.2 GFLOPs, on par with LoopMamba-B-4 at 82.1% and 6.4 GFLOPs. The theoretical motivation also does not fill this gap. Theorem 1 rewrites a symmetric affine update as gradient descent on the quadratic in Eq.5, but the identity holds for any affine map with a symmetric linear part, and for Mamba's diagonal it simply says that each coordinate's exponential moving average is a gradient step on a one-dimensional parabola. More importantly, it concerns the update along the sequence inside one scan, whereas LoopMamba loops over depth through convolutions, adapters, normalisation and an FFN, for which no energy is defined. the entropy plot in Fig. 4 is undefined, uses truncated per-subplot axes and has no non-looped control.
2. The model is not competitive at matched compute, and its efficiency is never measured. The paper claims "the best performance" and a "favorable accuracy–efficiency trade-off", but looping converts saved parameters into extra FLOPs and sequential depth. VMamba-T, which the paper uses in Fig. 5 but omits from Table 1, reaches 82.6% at 4.9 GFLOPs, ahead of LoopMamba-B-8 at 82.4% with 9.6 GFLOPs, and also leads on COCO and ADE20K with about 47.3 box AP, 42.7 mask AP and 47.9 mIoU against 46.1, 41.9 and 45.8. VSSD-Micro [3] reaches 82.5% with 14M parameters and 2.3 GFLOPs, beating LoopMamba-B-8 on accuracy, parameters and FLOPs at once. The training recipe also adds MESA and EMA, which several baselines did not use. There is no measured latency, throughput or memory, although inference latency grows with loop depth and training memory with the number of stored activations, and Tables 2 and 3 list the classification-backbone parameters and FLOPs for the detectors and segmenters, hiding exactly the resolution-dependent cost that matters for a looped backbone.
References
[1] Z. Shen, Z. Liu, E. Xing. Sliced Recursive Transformer. ECCV, 2022.
[2] J. Zhang, H. Peng, K. Wu, M. Liu, B. Xiao, J. Fu, L. Yuan. MiniViT: Compressing Vision Transformers with Weight Multiplexing. CVPR, 2022.
[3] Y. Shi, M. Dong, M. Li, C. Xu. VSSD: Vision Mamba with Non-Causal State Space Duality. arXiv:2407.18559, 2024.
Questions for authors
-
How does LoopMamba relate to SReT and MiniViT, and what does the SSM setting add beyond them? Please also report an unshared stack with the same number of layers per stage at the same FLOPs, trained with your exact recipe.
-
What are the throughput, latency and peak memory of LoopMamba-T-4, T-8, B-4 and B-8 compared with VMamba-T, VSSD-Micro or similar SoTA models on the same GPU?
12 comments
Response to reviewer's comments: Regarding naive recursion in SReT and LoopMamba
We would like to clarify the reviewer's interpretation of the naive recursion experimental results. The improvement from 72.2% to 72.5% cited by the reviewer comes from Table 3 of SReT, where the baseline and the recursive model without NLL have 5.7M and 3.8M parameters, respectively. However, Table 1 of SReT provides a more direct comparison, showing that one recursion improves DeiT-Tiny's Top-1 accuracy from 72.2% to 74.0% (+1.8 percentage points) while maintaining 5.7M parameters. Therefore, characterizing the benefit of naive recursion as merely 0.3 percentage points does not fully reflect the reported results. Similarly, Table 5 of LoopMamba shows that increasing the loop depth from 4 to 8 without ISLA improves accuracy from 79.0% to 79.3% without increasing parameters. These observations suggest that recurrent depth can benefit representation learning even without additional iteration-specific adaptation, although the improvement may be limited by the capacity of shared transformations.
Response to reviewer's comments: Regarding the comparison with SReT-S
We believe that accuracy, parameter count, and computational complexity should be considered jointly. SReT-S achieves 81.9% Top-1 accuracy with 20.9M parameters and 4.2G FLOPs, whereas LoopMamba-B-4 achieves 82.1% with 17.8M parameters and 6.4G FLOPs. Although LoopMamba-B-4 requires more computation, it reduces the parameter count by approximately 14.8% while achieving slightly higher accuracy. These results highlight different efficiency trade-offs: SReT offers lower computational complexity, whereas LoopMamba demonstrates better parameter efficiency at this model scale.
Response to reviewer's comments: Relation to SReT and MiniViT
We acknowledge the relevance of SReT and MiniViT to our work. However, unlike these Transformer-based approaches, LoopMamba focuses on the state transition mechanism of SSMs. Specifically, motivated by an energy-based interpretation of SSM state updates, we introduce VPST, which combines shared state transitions with iteration-specific state fusion. MSSF and ISLA further enable multi-scale hidden-state aggregation and iteration-specific low-rank adaptation, respectively. These designs target the iterative refinement of SSM hidden states under an interpretable view, rather than simply applying conventional parameter sharing to another backbone. We now include the discussion of SReT and MiniViT in Appendix I.