acceptodds
Withdrawn from ICLR 2027

AAMamba: Not All Voxels Deserve Equal Tokens in Mamba-Based 3D Medical Image Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Medical volumes are highly nonuniform. Most voxels belong to homogeneous background or simple tissue, while the evidence needed for accurate segmentation concentrates at lesion margins, organ interfaces, and small irregular structures. State-space models such as Mamba process long 3D sequences efficiently, yet existing Mamba-based segmentation models still build tokens on uniform grids and serialize them in fixed spatial orders. We identify this anatomy-agnostic volume-to-sequence construction as a central bottleneck of 3D state-space segmentation. Uniform tokenization gives background the same representational budget as lesions and boundaries, and naive scan orders can place neighboring voxels far apart once the volume is flattened. We propose AAMamba, an anatomy-aware Mamba framework in which token allocation, sequence serialization, and dense reconstruction are designed jointly. AAMamba estimates a structural density field from intensity deviation, boundary strength, and curvature response, and uses it to assign denser tokens to anatomically complex regions while still covering the whole volume. The adaptive tokens are then ordered by ROI-guided bidirectional Hilbert scanning, which keeps spatial neighbors close in the sequence. ROI guidance is applied only during training to emphasize foreground structures, so inference requires no ROI input. Because adaptive sampling can disturb voxel-level correspondence, a coordinate-aware decoder fuses the adaptive semantic features with grid-aligned spatial references to reduce spatial drift in the dense prediction. AAMamba achieves 84.2% mean Dice on BraTS, 91.85% DSC on ACDC, and 84.26% average Dice on Synapse. The largest gains appear on small or morphologically complex structures, including the enhancing tumor, myocardium, gallbladder, and pancreas, which suggests that anatomy-aware sequence construction is an effective principle for 3D medical state-space models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Mamba, State-Space Models, 3D Medical Image Segmentation, Adaptive Tokenization, Anatomy-Aware Representation, Hilbert Curve Scanning, Volume-to-Sequence Serialization, Efficient Segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hDZJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AAMamba, a compact Mamba-based model for 3D medical image segmentation. It argues that how a volume is turned into a token sequence is a main bottleneck. A content-adaptive tokenizer warps the sampling grid toward regions with strong intensity change or edges or curvature. The adaptive tokens are ordered with a bidirectional Hilbert curve and a training-time ROI mask boosts foreground tokens. A decoder then fuses the adaptive features with a grid-aligned stream. With 5.7M parameters it reports 84.2 mean Dice on BraTS and 91.85 on ACDC and 84.26 on Synapse and beats SegFormer3D on all three.

Critical strengths
  1. I like the question the paper asks. How to turn a 3D volume into a 1D sequence for Mamba is a real design issue and most medical Mamba work just uses a fixed scan. The density map and the dual stream idea are simple and easy to understand.
  2. The model is small at 5.7M parameters and still improves over SegFormer3D on every dataset. The gains are largest on small structures like ET and gallbladder and pancreas which fits the motivation.
  3. The ablation in Table 4 covers CAT and the grid stream and Hilbert order on all three datasets. It also shows that CAT matters most and that Hilbert order adds only a little.
Critical weaknesses
  1. The central claim that sequence construction matters as much as the sequence model is not supported by the ablation. Removing Hilbert in favor of raster costs only 0.2 Dice on BraTS and 0.14 on Synapse and 0.13 on ACDC. With one run and no error bars this is within noise. Most of the gain comes from CAT which is a token allocation idea and not a scan order idea.
  2. The ROI guidance does not match what the paper says. The text and Figure 4 describe lesion-first scanning but Eq. 2 only multiplies tokens by 1 plus alpha times the ROI value. Nothing is reordered. The ROI is also taken from the ground truth mask during training and set to zero at test time so the model sees a different input distribution at inference. There is no ablation of alpha or of removing ROI guidance.
  3. The tokenizer is under-specified. The displacement field Delta and the gate m are never defined. It is unclear whether they are learned or hand made and how fold-overs are prevented. Table 5 talks about control points P which never appear in the method section.
  4. The experimental comparison is weak. Only one Mamba model appears and only on BraTS and the numbers look copied from other papers. The BraTS and Synapse baselines such as TransUNet at 64.4 on BraTS are far below what is usually reported. There is no nnU-Net or U-Mamba or VM-UNet and no matched retraining. SegFormer3D has 4.5M parameters and AAMamba has 5.7M so the gap is not purely free. Only Dice is reported with no HD95 and no FLOPs or latency and no significance test even though the paper claims efficiency and thin structure gains.
Questions for authors
  1. How exactly are Delta and m computed and are they learned? What are the control points P in Table 5
  2. Does ROI guidance reorder tokens or only scale them? Please add an ablation over alpha including alpha equal to zero
  3. Were all baselines retrained under the same protocol and splits? Please add nnU-Net and U-Mamba or SegMamba on all three datasets.
  4. Can you report mean and standard deviation over several seeds and HD95 and also FLOPs and memory and latency with and without CAT and Hilbert?
Reviewer_9JZvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AAMamba, a compact framework for 3D medical image segmentation. Its central argument is that converting a volume into a sequence is an important limitation of existing Mamba-based segmentation models: uniform tokenization allocates equal resources to regions of different complexity, while fixed scan orders can disrupt spatial locality. With 5.7 million parameters, the model reports DSC of 84.2% on BraTS, 91.85% on ACDC, and 84.26% on Synapse.

Critical strengths
  1. The paper addresses the mismatch between nonuniform anatomical complexity and uniform spatial representation. Considering token placement, sequence ordering, and reconstruction together is a coherent approach to dense volumetric prediction.

  2. Retaining a grid-aligned stream is a sensible response to the reconstruction difficulties introduced by adaptive sampling. The ablation results support the usefulness of this stream, although they do not establish that it explicitly corrects spatial drift.

  3. The reported gains span brain tumor MRI, cardiac MRI, and abdominal CT. Improvements on several small or thin structures are consistent with the method’s motivation and suggest potential beyond a single anatomical setting.

  4. The reported accuracy with 5.7 million parameters is promising. Relative to SegFormer3D, the model improves performance with a modest increase from 4.5 million parameters.

  5. Removing adaptive tokenization, the grid-aligned stream, or Hilbert ordering reduces performance on all three datasets. This provides useful preliminary support for the architecture, with adaptive tokenization showing the largest contribution.

Critical weaknesses
  1. Equation (2) multiplies each token by the positive scalar 1+αR1+\alpha R, but Equation (3) immediately applies LayerNorm. Consequently, the stated foreground amplification may have little effect in the formulation presented. The normalization axes, actual implementation, and measured effect of ROI guidance require clarification.
  2. The text and Figure 4 describe scanning foreground tokens first and background afterward. The equations instead specify a Hilbert permutation followed by token scaling, without an ROI-dependent permutation. These are different mechanisms. If ROI-first reordering is implemented, its effect on locality and its replacement at inference must be explained.
  3. The paper defines the density field but does not adequately explain how it produces the displacement field Δn\Delta_n, gate mnm_n, or control points PP. Displacement bounds, boundary handling, interpolation, and regularization are also unclear. These omissions prevent faithful reproduction. Moreover, an unconstrained displaced grid does not by itself guarantee full-volume coverage: samples can cluster, leave gaps, or move outside the valid domain.
  4. The decoder reshapes adaptive tokens and concatenates them with grid-aligned features. Reshaping does not reverse a spatial deformation, and no coordinate-conditioned interpolation, inverse warp, or explicit scattering operation is described. The grid stream may compensate empirically, but the claim that the decoder prevents spatial drift needs direct evidence.
  5. The supplied manuscript does not specify dataset versions, patient splits, preprocessing, voxel spacing, patch sizes, augmentation, losses, optimization, training schedules, or inference procedures. It is also unclear which baseline results were reproduced under the same protocol. Strong CNN baselines such as nnU-Net are absent, and Mamba comparisons are limited despite the paper’s central focus on state-space segmentation.
  6. There is no isolated evaluation of ROI guidance or bidirectionality, nor comparisons against alternative locality-preserving orders or a conventional deformable sampling baseline. Hilbert ordering improves aggregate scores by only 0.13–0.20 DSC points. Without repeated runs or confidence intervals, the reliability of these small gains is unclear.
  7. Parameter count alone does not establish computational efficiency. Resampling, dual streams, sorting, and bidirectional processing can add runtime and memory costs. Likewise, feature variation and edge strength are proxies for structural complexity, not direct measures of anatomical importance; they can emphasize noise or strong irrelevant boundaries while overlooking low-contrast lesions.
  8. The introduction reports a 1.87-point Synapse improvement, whereas Table 3 supports approximately 2.11 points. The limitations section says comparisons focus on models below 100 million parameters, yet Table 2 includes models above 300 million. The SETR rows cite a reference whose listed title concerns influence functions. These inconsistencies weaken confidence in the reporting.
Questions for authors
  1. How does ROI amplification remain effective after LayerNorm? Please specify the normalization axes and provide an isolated ROI-guidance ablation.

  2. Does the implementation change token order using the ROI, amplify token features, or perform both operations? Please give training and inference pseudocode and quantify any resulting distribution shift.

  3. How are the density field, displacement field, gate, and control points connected? What constraints prevent sampling collapse, uncovered regions, or out-of-bounds coordinates?

  4. Which coordinates determine Hilbert ordering after deformation? Are these reference-grid coordinates or displaced sampling coordinates, and how are collisions or quantization handled?

  5. How are adaptive features aligned with the output voxel grid? Is an inverse mapping implemented? Can spatial alignment be measured directly rather than inferred from DSC?

  6. Can the full experimental protocol and baseline provenance be provided? In particular, were all comparisons made with identical patient splits, preprocessing, and inference settings?

  7. Do the improvements persist across random seeds and matched-capacity baselines? Comparisons with a tuned CNN, a compact Mamba model, and conventional deformable sampling would help isolate the proposed contribution.

  8. What are the actual deployment costs and boundary-quality gains? Please report latency, peak memory, computational cost, and boundary-sensitive metrics such as HD95 or surface DSC.

Reviewer_gSSXOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper identified anatomy-aware volume-to-sequence construction as a key design problem in 3D state-space segmentation, and propose a new architecture based on it. It achieves accuracy gains in multiple datasets.

Critical strengths
  1. very well-written, good story-telling technique
  2. many comparison methods
  3. multiple datasets
Critical weaknesses
  1. The most recent comparison method is in 2024.
  2. The most cited baseline nnUnet in this domain is not included.
  3. number of parameters is reported. But GPU memory usage is not.
Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.