acceptodds
Withdrawn from ICLR 2027

OCShell-Mamba: Object-Centric Shell Propagation for Vision State Space Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision state space models (SSMs) achieve efficient visual modeling by serializing 2D feature maps into 1D sequences. However, serialization is not representation-invariant: the scan order determines how information accumulates in the recurrent state. Conventional scan paths, such as raster and bidirectional scans, often disrupt local neighborhoods and fragment object structures, while existing adaptive scans remain token-centric and do not explicitly preserve object-level coherence. We propose OCShell-Mamba (Object-Centric Shell routing Mamba), an object-centric vision SSM that propagates states over object-like regions instead of individual tokens. Without requiring external supervision, OCShell-Mamba learns an objectness field and organizes state propagation using a shell routing strategy that expands from each region center toward its boundary. Its core hell-wise bject-entric ruting (SOCO) module partitions each region into concentric shells, aggregates features within each shell, and processes shell descriptors from inner to outer. This design better aligns sequence adjacency with image geometry while preserving local and object-level coherence. We further introduce ersistent tpology mory (PoMe), a fixed-size memory bank that maintains object-level representations across resolution changes. On ImageNet-1K, ADE20K, and COCO, OCShell-Mamba consistently outperforms a matched DAMamba baseline, achieving 83.9% top-1 accuracy, 50.8% mIoU, 49.1% box AP, and 43.7% mask AP. These results show that vision SSMs benefit not only from adaptive scan paths but also from explicitly modeling object-centric state propagation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision State Space Models, Mamba, Object Centric

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8PM2OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes OCShell-Mamba, a hierarchical vision SSM built on the DAMamba block that replaces DAMamba's adaptive scan with an object-centric token order. In the routing module, SOCO, a lightweight conv head predicts an objectness map, and per-image Otsu thresholding and connected-component labelling turn it into object candidates. In the deep stages, each candidate's tokens are traversed shell by shell from an objectness-weighted centre outwards, candidates are concatenated by confidence, and the background follows in raster order; the shallow stages instead sort all tokens into confidence bins. The SSM runs on the permuted sequence and the output is permuted back. The second component, PoMe, which the paper calls Persistent Topology Memory, is a bank of 64 latent tokens of width 256 that is read and written once per stage by cross-attention and carries an auxiliary classification loss during ImageNet pre-training. The objectness head is regularised by an annotation-free border-background prior and a routing-coherence loss. Only a Tiny model with 27M parameters and 5.7 GFLOPs is reported. It reaches 83.9% top-1 on ImageNet-1K, against 83.8% reported for DAMamba-T and 83.6% for the authors' reproduction; 50.8 versus 50.3 mIoU on ADE20K with UperNet; and 49.1 box AP and 43.7 mask AP versus 48.5 and 43.4 on COCO with Mask R-CNN 1×. All component and hyperparameter ablations are run on a custom 100-class mini-ImageNet split.

Critical strengths

The core idea is clear and intuitive: making each object candidate a contiguous, centre-out segment of the recurrence is a natural content-dependent alternative to geometric and deformable scans, and the construction (Otsu threshold, connected components, centre and radius, shell binning, inverse permutation) is fully specified, with pseudocode in Alg. D.1, so it is reproducible and could be grafted onto other SSM backbones. The comparison is also made against a strong and closely matched baseline, using DAMamba's own recipe and evaluation protocols on classification, segmentation and detection, and the improvement over DAMamba has the same sign on all three tasks.

Critical weaknesses

1. The state-of-the-art claim is not supported, and there is no evidence of a better accuracy–efficiency trade-off. On ImageNet the model improves over its direct baseline DAMamba-T by only 0.1 points, from 83.8% to 83.9%, at 19% more FLOPs, and the dense-prediction gains of 0.5 mIoU, 0.6 box AP and 0.3 mask AP also come at higher cost. Several recent backbones of the same size reach equal or higher accuracy at equal or lower cost and are absent from Table 1: Mamba2D-T [1] reaches 84.0% with 26.5M parameters and 4.7 GFLOPs, A2Mamba-S [2] 84.7% with 31M and 5.4G, OverLoCK-T [3] 84.2% with 33M and 5.5G, and TransNeXt-Tiny [4] 84.0% with 28M and 5.7G. Some included rows are also wrong or not comparable: the VSSD row combines the accuracy of VSSD-Micro with the FLOPs of VSSD-Tiny, and ZigMa and MaIR are, to my knowledge, a diffusion model and an image-restoration model that do not report ImageNet-1K top-1. The method adds per-image Otsu thresholding, connected-component labelling, sorting, gather–scatter permutations and memory cross-attention at every stage, which are irregular and hard to fuse on GPUs, yet only parameters and FLOPs are reported.

2. The evidence does not show that the object-centric routing is responsible for the gain. Only one model size is reported, and all ablations are run on a custom 100-class mini-ImageNet split, where the full model gains 1.43 points over a retrained DAMamba-T, from 87.48% to 88.91%, whereas on full ImageNet-1K the gain shrinks to 0.1 points over the reported and 0.3 points over the reproduced baseline. The controls are missing that the same backbone with the same losses and PoMe but with SOCO replaced by raster scan or by DAMamba's adaptive scan, and the model with the auxiliary losses switched off, even though Lmem\mathcal L_{mem} is an auxiliary classifier that could explain part of the gain by deep supervision alone. In addition, the description of the routing conflicts with the appendix. Sec. 3.1 and Fig. 3 state that the objectness head is learned from task gradients, but routing uses a stop-gradient copy of the map, and according to App. B the head is trained only by the border-background prior and the coherence loss, so the resulting map is a background-prior saliency map rather than an object structure learned from the task.

[1] E. Baty, A. Hernández Díaz, R. Davidson, C. Bridges, S. Hadfield. Mamba2D: A Natively Multi-Dimensional State-Space Model for Vision Tasks. arXiv:2412.16146, 2024.

[2] M. Lou, Y. Fu, Y. Yu. A2Mamba: Attention-augmented State Space Models for Visual Recognition. arXiv:2507.16624, 2025.

[3] M. Lou, Y. Yu. OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels. CVPR, 2025.

[4] D. Shi. TransNeXt: Robust Foveal Visual Perception for Vision Transformers. CVPR, 2024.

Questions for authors

(1) Please give the source of every row in Table 1, correct the VSSD row, remove or justify the ZigMa and MaIR rows, and add recent same-scale backbones such as Mamba2D-T, A2Mamba-S, OverLoCK-T and TransNeXt-Tiny.

(2) What are the throughput and peak memory of OCShell-Mamba, DAMamba-T and the strongest Table 1 baselines on the same GPU, at 224² for classification and at 1280×800 for detection, and what fraction of runtime goes to the routing and memory operations?

(3) On full ImageNet-1K, what accuracy does the same architecture with the same losses reach when SOCO is replaced by raster scan and by DAMamba's adaptive scan, and when λmem\lambda_{mem}, λobj\lambda_{obj} and λcoh\lambda_{coh} are set to zero?

(4) Please confirm how the objectness head is trained, whether DAMamba's adaptive scan is kept, and give the stage depths and widths with a per-component breakdown of parameters and FLOPs that explains the extra 0.9 GFLOPs over DAMamba-T.

Reviewer_K5XJOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes an object-oriented ordering of image tokens for hierarchical vision state space models. SOCO predicts an objectness field, thresholds it using Otsu's method, extracts connected components, and orders each component in radial shells around an objectness-weighted center. Component sequences are concatenated before SSM processing, and an inverse permutation restores spatial positions afterward. Confidence-bin ordering is used in stages 1-2, while component-based shell ordering is restricted to stages 3-4. PoMe adds a fixed-size memory bank, initialized from stem features and updated through cross-attention across stages.

Critical strengths
  1. The paper treats serialization as a modeling choice with consequences for recurrent information accumulation. Grouping candidate regions before propagation is an understandable way to address object fragmentation, and the motivation is stronger than simply proposing another fixed traversal pattern.

  2. The experiments include classification, semantic segmentation, detection, and instance segmentation. Relative to the tabulated DAMamba results, the full model improves ADE20K by 0.5 mIoU, COCO box AP by 0.6, and mask AP by 0.3.

  3. The writting of this paper is easy to follow, and the figures are neat.

Critical weaknesses
  • Section 3.1, immediately after Eq. (2), says that the head receives no routing gradient but is updated through task gradients flowing into its input features. Gradients entering an input tensor do not, by themselves, update parameters in a downstream branch whose output is detached. Under the described graph, the task loss has zero derivative with respect to the objectness-head parameters through routing.

  • Table C.1 is informative, but the DAMamba row is explicitly an external reference, and the PoMe-only variant uses an outside-in spiral scan. There is no same-architecture configuration with both SOCO and PoMe disabled under that same scan. Consequently, the +1.02 and +1.12 gains cannot cleanly be interpreted as independent module effects relative to a shared base. I would like to see the corresponding four configurations with all other blocks, training losses, and scan conventions held fixed. The existing full-versus-SOCO-only comparison does support a memory-associated gain of 0.41 points on this subset, and full-versus-PoMe-only supports a 0.31-point routing-associated gain, but the broader attribution remains incomplete.

  • SOCO combines multiple changes: shallow confidence bins, connected components, component ranking, and center-out shell ordering. Tables C.1-C.2 do not determine which of these matters. In particular, processing exactly the same components in raster order would already ensure candidate-contiguous segments. Comparing this control with shell ordering, preferably also with reversed shell order, would directly test the central center-to-boundary hypothesis. A small stage-policy ablation would also help distinguish the contribution of the early confidence-based scans from that of the deep object-based scans. Without these controls, the experiments support the package more convincingly than the specifically proposed propagation geometry.

Questions for authors
  • Under the implemented computational graph, which losses produce gradients for the objectness-head parameters?

  • What happens when the same detected components are traversed in raster order rather than radial shells? Can you provide a common-base SOCO/PoMe factorial ablation and clarify which scan and auxiliary losses are retained in every row?

Reviewer_VcNFOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OCShell-Mamba, an object-centric vision SSM that organizes token-level state propagation into contiguous segments corresponding to object-like regions. The method comprises two main components: (1) Shell-wise Object-Centric Routing (SOCO), (2) Persistent Topology Memory (PoMe). Experiments are reported on ImageNet-1K, ADE20K, and COCO 2017.

Critical strengths
  1. Organizing 1D SSM state propagation around semantic foreground regions is intuitive.
  2. The pseudocode in App. D is complete.
Critical weaknesses
  1. limited performance improvement with more computation: On full ImageNet-1K, OCShell-Mamba improves over DAMamba by a marginal +0.1% Top-1 accuracy (83.9% vs. 83.8%) while requiring +18.75% more FLOPs (5.7G vs. 4.8G), extra parameters (27M vs. 26M), and an auxiliary deep-supervision classification loss. Gains are within noise and compute is not matched.

  2. No component ablations separating SOCO and PoMe are provided on full ImageNet-1K, ADE20K, or COCO. When inspecting the mini-ImageNet ablation, the numbers undermine the paper's core narrative: adding the entire SOCO objectness head, Otsu thresholding, connected-component labeling, and shell routing on top of PoMe only yields a mere +0.31% on mini-ImageNet.

  3. Routing uses a stop-gradient copy of the objectness map in Eq. 2, and the task loss only ever sees that detached copy in Eqs. 4, 9. The claim that Φ_s “is updated via the task-driven gradients flowing into its input” is a non sequitur. Under the main text’s description Φ_s would stay at initialization; it trains only because App. B adds two losses the main text does not mention: a classical border prior (background := mean feature of image-border tokens) and a smoothness term along the route. The “objectness” is therefore a center-bias saliency map learned without task signal, which will look object-like on ImageNet by construction and is questionable for ADE20K/COCO, where border tokens are routinely foreground. Candidate quality is never measured.

  4. No latency, and the mechanism acts on 49 tokens. Otsu + connected components + sorting run per image per stage, unbatchable, excluded from FLOPs; App. A.1 says profiling is required and does not do it. Shell routing is active only at stages 3–4 (196 and 49 tokens at 224²), where a candidate has at most 3–4 shells.

  5. Missing to compare and illustrate the difference with DefMamba: Deformable Visual State Space Model [CVPR 2025] and Deformba: Vision State Space Model with Adaptive State Fusion [ICML 2026].

Questions for authors
  1. Could the authors do component ablation on full imagenet dataset?
  2. Objectness quality against ImageNet-S or COCO masks; what fraction of candidates overlap a ground-truth object?
  3. Training and inference throughput vs other baseline on the same GPU.
  4. Why confidence-bin sorting at shallow stages, given Sec. 1?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.