acceptodds
Under review as a conference paper at ICLR 2027

Content-Aware Scanning in Vision State Space Models: Is Grouping Enough?

Abstract

Content-aware scanning for vision state space models (SSMs) rests on a proximity hypothesis: placing informative tokens next to each other shields their interaction from the recurrence's decay. We test it in a controlled SSM restoration testbed (1,108 trained models) that varies the grouping of informative tokens independently of the information an order carries, and the hypothesis fails in both directions. Sorting tokens by training-free importance scores, such as constant false alarm rate (CFAR) detection, raises grouping without adding information and performs like a random permutation (within 0.1 dB), even when the scores find the targets with AUC 0.98. Ground-truth orders that place informative tokens farther apart than a raster scan still help, in all 72 runs, so "oracle" orders measure leakage rather than an attainable gain. In a factorial design that sets the precision of an order (the informative fraction of the tokens it selects) and its grouping independently, raising precision from 0 to 1 adds 3.5 dB at fixed grouping, and grouping helps only in proportion to precision: grouping alone explains 0.62 of the variance, grouping and precision with their interaction 0.96. The same information delivered as an input channel instead of an order helps as much or more (+9.82 against +8.03 dB for the ground-truth support; +0.78 against +0.69 dB for a supervised CNN scorer). Finally, because the recurrence has no history at the start of a scan, position is readable there: a ground-truth block placed first gains 3–6 dB more than mid-sequence in a causal scan, also with the scan convolution removed and with a variable number of targets. We release a protocol with region-aware metrics and random-permutation, locality, leakage, and input-channel controls.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
state space models, Mamba, scan order, content-aware scanning, token reordering, information leakage, evaluation protocol, image restoration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XWS2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether informative-token grouping explains the effectiveness of content-aware scanning in vision SSMs. By disentangling Grouping, Information, Locality, and Position through controlled experiments, the authors challenge the conventional proximity hypothesis and propose an information-based explanation of scan-order performance.

Critical strengths
  1. Insightful experimental methodology. The paper's main contribution is its systematic separation of four interacting factors, grouping, information, locality, and position, rather than attributing performance improvements solely to token proximity. The factorial experiments and input-channel controls are particularly valuable.
  2. Comprehensive and transparent analysis. The authors provide extensive controlled experiments, including random permutations, oracle leakage analysis, scan-convolution removal, and causal/bidirectional comparisons. The proposed evaluation protocol is useful for future research.
Critical weaknesses

1. The proximity hypothesis may oversimplify existing content-aware scanning mechanisms.

Several underlying insights have been established in prior work, which the paper cites but does not sufficiently distinguish from its own contributions. QuadMamba (NeurIPS 2024) and MaIR (CVPR 2025) already emphasize spatial locality preservation; REOrder (NeurIPS 2025) demonstrates that ordering effectiveness depends on model- and task-specific properties; and Mamba-Reg (CVPR 2025) reveals positional sensitivity in Vision Mamba.

More fundamentally, H1 assumes that improvement depends solely on informative-token proximity. However, Content-Aware Mamba (ICLR 2026) groups content-similar tokens, not necessarily high-importance tokens. CFAR-based importance sorting does not reproduce this mechanism. Therefore, rejecting H1 may demonstrate the failure of an oversimplified hypothesis rather than invalidate the actual mechanisms of existing content-aware scanning methods.

2. The strongest improvements depend on oracle information rather than practical scanning methods.

In Figure 5, ground-truth ordering improves performance by +8.03 dB, whereas the supervised CNN scorer achieves only +0.69 dB and CFAR sorting degrades performance by −0.45 dB. The same information delivered through an input channel performs comparably or better.

Although oracle leakage is explicitly acknowledged and appropriately used for diagnostic experiments, the large oracle gains do not establish that practical content-aware scanning offers comparable benefits. The evidence supporting the proposed mechanism under deployable scanning strategies remains relatively limited.

3. The simplified testbed limits the generalizability of the conclusions.

Despite 1,108 trained models, the experiments primarily rely on a 0.69M-parameter SSM backbone, 32×32 inputs, and synthetic sparse restoration. The baseline learns the synthetic task relatively weakly, and longer training provides little improvement. The radar and natural-image controls are useful, but do not replace validation on representative Vision SSM architectures.

Consequently, it remains unclear whether the observed mechanisms generalize to established content-aware methods such as CAM or MambaIRv2. The conclusions appear broader than the experimental evidence supports.

4. The interpretation of “Information” in H3 needs greater precision.

For an input-derived permutation π = f(X), the ordering introduces no additional Shannon information beyond X. Its benefit instead concerns how effectively the backbone can access information already present in the input.

Furthermore, the factorial experiments manipulate selected-set spacing and selection precision, whereas Equation (1) measures grouping over the true informative set. At zero precision, both configurations report approximately zero true-support grouping gain, so this control does not demonstrate that substantially changing informative-token grouping has no effect without information.

The positive-precision matched comparisons remain valuable, but the proposed information-based explanation should distinguish target-related selection accuracy, realized grouping, and information accessibility.

Questions for authors
  1. How does the proposed H1 capture semantic-similarity-based grouping in existing methods, particularly CAM, rather than importance-based sorting alone?
  2. Can the authors clarify how the large oracle gains support the practical relevance of the proposed mechanism, given the relatively modest improvements from deployable scorers?
  3. Can the main conclusions be validated using an established, well-trained content-aware Vision SSM?
  4. Can the authors distinguish information-theoretic information, selection precision, and backbone accessibility, and clarify the zero-precision grouping control?
2 comments
AuthorsOpenReview ↗

Response - similarity grouping, oracle vs. deployable gains, validation, and terminology

Thank you for a clear summary and for judging the soundness concerns fixable. The revision addresses each of them.

W1 / Q1 (H1 oversimplifies; CAM groups similar tokens). We agree. Section 2 now labels H1 as the strong form we formalise and states that its failure leaves room for a local effect of grouping. An importance sort does not reproduce similarity grouping, so we added it in both settings: k-means clusters of the local mean and standard deviation (MambaIRv2/CAM-style), with tokens ordered by cluster.

  • Testbed (Section 6, 3 seeds). The cluster order gives −0.21 [−0.30, −0.13] dB, because it scatters spatial neighbours (P 0.45 vs. 0.54 for raster). The same clusters as an input channel give +0.13 [+0.07, +0.21] dB.
  • Realistic backbone (Section 8). Computed from the noisy input, the clusters give +0.04 [+0.02, +0.06] dB at 4k steps and +0.02 dB at 12k, within the 0.03–0.04 dB that MambaIRv2 reports for its semantic grouping. Computed from the clean image, they give +2.55 and +3.14 dB.

Section 9 now separates what is known from what we add. Known: locality preservation (QuadMamba, MaIR), the dependence of the best order on model and task (REOrder), and positional artefacts in Vision Mamba (Mamba-Reg). Added: random-permutation, locality, leakage and input-channel controls for per-image orders, and the anchor measured as a distance from a state reset.

W2 / Q2 (oracle gains vs. deployable scorers). We agree that the oracle gains say nothing about what deployable orders gain; ground-truth orders measure what the target information is worth. The deployable evidence is:

  • training-free importance sorts ≈ random permutation in six settings, including the realistic backbone;
  • a supervised CNN scorer: +0.69 dB as the order, +0.78 dB as a channel;
  • input clusters in the realistic backbone: +0.02 to +0.04 dB.

So a content-aware order helps by roughly as much as its scorer adds information the backbone does not already use. The protocol (Section 10) measures this with a random permutation, the same map as an input channel, and leakage controls.

W3 / Q3 (validation on an established content-aware SSM). Partially. We could not train the official CAM or MambaIRv2 code during the discussion. Instead we added a MambaIR-style network with a 3×3 depthwise convolution in every block, a four-direction scan and channel attention, trained on DIV2K denoising for up to 12k steps (Section 8). We also repeated the core testbed comparisons at 3.96M parameters (Appendix I).

  • What carries over: importance sorts match a random permutation, and oracle orders show the leakage pattern.
  • What does not: the channel matching the order. The same clean-image cluster map helps more as the order (+2.55 vs. +1.74 dB).

This is not a validation on an established, fully trained model. Section 10 states this limitation, and the title and abstract now state the controlled scope.

W4 / Q4 (information, precision, accessibility; zero precision). We agree on both points.

  • Terms (Section 2). We now distinguish three things. Target information is unavailable at test time (ground-truth orders, i.e. leakage). Selection precision q is the fraction of selected tokens in the support, measured against the truth. Accessibility is how readily the backbone uses information already in y. For π = f(y), any gain is accessibility, and H3 is stated in these terms.
  • Zero precision (Section 6). At q=0 the true support keeps raster's grouping, so these cells do not test the grouping of informative tokens. The revision removes that claim, marks both q=0 points in Fig. 4a, and adds two controls:
    • content-agnostic scans that raise the true support's log-grouping by +0.10 to +0.56 change L by −0.16 to +0.23 dB, where the full-information slope predicts +1.0 to +1.3 dB (Section 4);
    • a truth-independent selection whose own grouping ranges from 0.07 to 12.75 changes L by −0.07 to −0.09 dB.
  • Table 9 lists the selected-set and true-support grouping of every cell. As you suggest, the matched positive-precision comparisons are now the primary evidence: raising q from 0.5 to 0.75 at equal grouping adds +1.35 [+1.20, +1.50] dB.

We sincerely thank you again for your time and thoughtful review of our submission.

We apologize that our response was posted relatively late in the discussion period, and we understand this may have left you with limited time to review it. As the discussion period closes tomorrow, we would be most grateful if you could let us know whether our response has sufficiently addressed your concerns.

If any concerns remain, we would be happy to address them before the deadline. If they have been resolved, we would appreciate it if you could consider reflecting this.

Thank you for your consideration.

Reviewer_LPeKOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper tests the proximity hypothesis behind content-aware scanning in vision SSMs, say, grouping informative tokens in the scan shields their interaction from the recurrence’s decay. In a controlled testbed (a 0.69M-parameter SSM without 2-D convolution, mostly on synthetic sparse restoration), the authors vary grouping and information independently. They find that training-free importance sorts perform like random permutations, ground-truth orders help through leakage rather than grouping, and grouping helps only in proportion to the information an order carries, which works as well when supplied as an input channel. They also identify a start-of-scan position effect and propose an evaluation protocol.

Critical strengths
  • The paper asked a useful question. Content-aware scan papers rarely include a random-permutation control or separate what an order groups from what it reveals.
Critical weaknesses
  • A 0.69M-parameter SSM without 2-D convolution is not enough to support the claims in this paper. The setting maximizes locality cost by construction, removing the 2-D convolutions and multi-directional scans that real vision SSMs use to compensate for it. The reference model also barely learns the main task: its whole-image PSNR falls below that of an all-zero output. Under these conditions, the negative results (e.g., “training-free sorts ≈ random permutation”) may reflect the testbed rather than content-aware scanning itself. The paper would need much more replication of the core comparisons in a realistic backbone to justify the conclusions stated in the title and abstract.

  • The paper’s stated target is the rationale behind methods such as MambaIRv2, QuadMamba, and VAMamba. However, these method are never implemented or tested in the paper? The title and abstract read as a verdict on content-aware scanning in general, but the evidence covers a simplified proxy.

Questions for authors
  • Please add real task and real baselines to validate the claims. I don't think synthetic sparse restoration and 0.69M-parameter SSM without 2-D convolution is suitable to support any claims in this paper.
3 comments
AuthorsOpenReview ↗

Response - backbone on natural images; clarifications and the remaining limitation

Thank you for the review. We agree that a 0.69M-parameter SSM without 2-D convolution, mostly on synthetic data, cannot support a verdict on content-aware scanning in general. The revision narrows the claims:

  • Existing methods. Section 1 states that we do not evaluate existing content-aware methods. Their orders use no ground truth, so nothing in our results implies that their gains are leakage.

A realistic backbone on natural images (new Section 8, Table 2)

  • Setup. A MambaIR-style restoration network that we implemented in PyTorch; it is not the official code. It has 0.99M parameters, and each of its 12 blocks has a 3×3 depthwise convolution, a four-direction selective scan with input-dependent decay, and channel attention.
  • Task. Grey Gaussian denoising (σ=50) on DIV2K: 64×64 patches from 750 training images, and 512 test patches from 50 held-out images.
  • The reference learns normally. It rises from 15.05 dB (noisy input) to 28.49 dB after 12k steps, with +0.04 dB over the last 2k steps.

PSNR change against the standard four-direction scan:

order4k steps (2–3 seeds)12k steps (1 seed)
random permutation−0.03 [−0.04, −0.01]−0.03
local-contrast importance sort−0.03 (equal to random within 0.01)—
similarity clusters of the noisy input (MambaIRv2/CAM-style), as the order+0.04 [+0.02, +0.06]+0.02
same clusters, as an input channel−0.03 [−0.05, −0.01]—
clusters of the clean image (oracle), as the order+2.55 [+2.42, +2.69]+3.14
clusters of the clean image, as a channel+1.74 [+1.64, +1.83]—
  • Orders available at test time. Their effects are small, about the size of the semantic-grouping ablation reported for MambaIRv2 (0.03–0.04 dB).
  • Oracle orders. Clean-image orders gain more than 2.5 dB, the leakage pattern of the testbed.
  • What does not transfer. The testbed's "a channel helps as much" does not transfer, and the revision states this.

The core testbed comparisons at 5.7× the parameters (3.96M, 3 seeds; Appendix I):

  • CFAR sort − random permutation: +0.02 [−0.02, +0.06];
  • the spread lattice still gains, +3.35 dB.

"The reference model barely learns the task." We agree that the synthetic objective stays high: 0.26, against 0.353 for an all-zero output.

  • Why it scores below an all-zero output. On sparse targets the background dominates whole-image PSNR, while the reference does learn the informative region: 7.4 dB, against 2.8 dB for a zero output and 4.1 dB for the noisy input.
  • Other settings. This weak-learning regime is why the paper includes radar, where the reference learns well. The realistic backbone now adds a standard task on which the reference learns normally, and the importance sort still matches the random permutation there.

"Removing 2-D convolutions and multi-directional scans." A small clarification:

  • Scan directions. The testbed is bidirectional by default; causal scans appear only in the anchor experiments.
  • What is missing by design. It lacks four-direction scans and 2-D convolution, so that all spatial mixing passes through the scan. We agree that this maximises the locality cost.
  • In the realistic backbone. The new backbone restores both, and the locality cost of a random permutation shrinks: −0.03 dB in PSNR, against −0.47 dB in L in the testbed (Sections 4–5).

Real baselines. We could not reproduce the official implementations of MambaIRv2, QuadMamba or VAMamba within the discussion period.

  • What the new runs cover. They implement the ingredient these methods share, a per-image order computed from content, together with the controls we propose: a random permutation, the same map as an input channel, and a clean-image oracle.
  • What they do not cover. They do not implement the methods themselves. Section 10 states this as the main limitation.

We understand that this may not fully meet the request for real baselines. The revised paper presents controlled counterexamples, a check in a realistic backbone, and an evaluation protocol, rather than a verdict on existing methods.

We sincerely thank you again for your time and thoughtful review of our submission.

We apologize that our response was posted relatively late in the discussion period, and we understand this may have left you with limited time to review it. As the discussion period closes tomorrow, we would be most grateful if you could let us know whether our response has sufficiently addressed your concerns.

If any concerns remain, we would be happy to address them before the deadline. If they have been resolved, we would appreciate it if you could consider reflecting this.

Thank you for your consideration.

Your reply doesn't address my concern at all. MambaIR-style restoration network in 0.99M parameters is still not enough. We need baselines from the real, official works in multiple areas to confirm that your claim is correct.

Reviewer_dECmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  1. This paper explores whether content-aware scanning order can improve the performance of visual state-space models by arranging information-rich markers more closely. The study distinguishes four typically intertwined factors: grouping, information revealed by the order, preservation of spatial locality, and relative position to the scan start position.

  2. The experiments employed a small Mamba-2-based restoration backbone network without using 2D convolutions. The key results indicate that grouping alone is insufficient. CFAR-based ordering can increase the grouping rate of information-rich markers by approximately 1.6 to 2.1 times, but its region-balanced PSNR change is -0.50 dB, close to -0.47 dB for random arrangement. Conversely, even when the true order arranges information-rich markers further apart than the grid order, they remain effective, although these orders reveal target information.

  3. A factorial experiment further demonstrates that, with a fixed grouping rate, increasing the precision of the selected markers improves performance by approximately 3.5 dB per unit of precision. Passing the same information as the input channel yields performance comparable to or even better than encoding in scan order: the performance improvement is +9.82 dB when using the real mask as the channel, compared to +8.03 dB when encoding in sequence; the performance improvement of the learned CNN is +0.78 dB and +0.69 dB, respectively. The paper also points out that empty loop states lead to a significant scan initiation effect.

Critical strengths
  1. Separating grouping, information, locality, and scan location is conceptually useful. In content-aware scanning papers, these factors often change simultaneously, making reported performance improvements difficult to interpret.

  2. The experimental controls are comprehensive. The paper includes random permutations, locality-preserving variants, decoy blocks, diffusion-based true value lattices, input channel control, causal and bidirectional scans, non-convolution variants, variable support sizes, and longer training times.

  3. The hypotheses are clearly derived. H1 was determined before the main experiment, H2 through H4 were proposed after observing initial failures, and a second set of 200 runs was designed to prospectively test their predictions.

  4. Statistical comparisons are paired based on seed, training batch, and evaluation images. Reporting region-specific metrics and simple baselines is particularly important because full-image PSNR can be misleading for sparse targets.

  5. This paper candidly discloses negative results and experimental issues, including failures in CT content-aware runs, incomplete computational matching controls, and corrected pairing errors.

  6. Many control experiments rather than simply comparing different paths: Section 4–7 incorporates random per-image arrangement, reverse sorting, background decoy, dispersed ground truth lattice, and input of the same signal channels. These controls effectively weaken the simplistic notion that "higher grouping is necessarily better."

  7. The proposed evaluation protocol is not limited by test platforms. Random permutation, locality, leakage, location, and backup information path control will help improve future evaluations of adaptive scan sequences.

Critical weaknesses
  1. Overturning the sufficiency of grouping does not equate to excluding the proximity mechanism.
    H1 requires that the benefit increases monotonically with grouping and is independent of the source and location of the ordering, which is stronger than "shortening the sequence distance helps interaction." CFAR ordering simultaneously destroys locality, and GT lattice introduces unusable answer cues. Therefore, its results overturn the sufficiency of a single variable and cannot exclude the possibility that grouping has an independent but offset positive effect. This paper observed the grouping benefit in a stride sweep with fixed truth cues. Therefore, statements such as proximity is not a mechanism and grouping only amplifies information still lack evidence to exclude other models.

  2. Orthogonal design of precision and stride does not equate to orthogonality between information and actual grouping. q and stride of IG can be set independently, but the actual measured C is the coupling of all real support tokens, not just the tightness of the selected set, and therefore also changes with q. Table 5 shows that the log-grouping for both strides is 0.00 when q=0, failing to form a test of "changing the real grouping without information". q=0, using the truth value to ensure the selected set is entirely background with negative labeling cues, is not equivalent to statistically independent zero information; the expected precision of random selection should be close to the support proportion s. The grouping ranges for different q values ​​in the figure do not completely overlap. Therefore, R²=0.96 and the slope of the precision of fixed grouping cannot be directly considered as two mechanisms being independently identified.

  3. The scanning initiation effect has not yet been located as a causal consequence of an empty initial state.
    Removing convolutions, changing orientations, and varying K effectively eliminates several alternative explanations, but does not directly intervene in the initial state. The GT block also alters the background contamination before entering support and the length of the continuously output background after processing support; the paper points out that the background error distribution changes significantly. These factors can create an initial advantage without proving that the model actually reads out the absolute position. Therefore, "anchor rests on the empty recurrent state" is more causally indicative than existing evidence.

  4. The distance exponent C is not a validated effective interaction quantity of the model. Under fixed decay, the distance kernel is a reasonable geometric surrogate, but actual SSM interactions also include input and output projections, B/C, gating, five-layer combination, and scan convolution. Learned decay is input-dependent, and a single effective horizon may not represent the cumulative decay of each head/layer. If C cannot reflect actual interactions, dismissing the interaction distance mechanism based on C's poor interpretability may simply be a measurement mismatch. Appendix F also acknowledges that the linear trend of locality is mainly determined by random extreme points, and P cannot be considered a complete cost model.

  5. Incomplete Matching of Channels and Rankings: Clues Revealed. Continuous score channels provide numerical values ​​for each spatial location, while ranking typically only retains the rank; ground truth support channels more directly identify the token identity, and random location GT blocks require decoding from sequence position and observation. Both come from the same map, but are not equally direct or have the same model input capacity. The results support channels are a useful alternative for this task, and it cannot be proven that ranking works solely through a single information medium. Furthermore, for fixed training parameters θ, neither untrained nor supervised scorer fθ(y) adds Shannon information beyond y during the test period; priors obtained from supervised training are not equivalent to answer leakage.

  6. Equivalence and Ineffectiveness Conclusions Require Direct Differences and Sufficient Seed Evidence. The close relative intervals of the two methods to the raster do not equate to equivalence. While some direct paired differences are presented, most only have 2–4 seeds per cell; hierarchical bootstrap cannot create unobserved seed variations. The contrast sort of natural images is 0.46 dB higher than random, indicating that "identity" varies depending on the task and allowable difference. Shared datasets, raster references, and seeds correlate errors across cells; 72/72 positive signs cannot be considered 72 independent confirmations. For the statement "CNN channels and order are equally good," the difference range [−0.03, +0.22] does not automatically prove equivalence.

  7. Negative results from controlled small models still lack connection to practical applications. Currently, the main limitations are a custom SSD with 32×32 resolution, 0.69M image size, no 2D local mixing, and short training. The objective function of the synthetic raster only decreased from 0.353 to approximately 0.26, and even after a three-fold increase, it only reached approximately 0.255. The radar provides support for different learning states, but it still uses simulated data. Natural images are only controlled by a small-size CIFAR-10, and the CT adaptive branch fails. These limitations have been honestly revealed, yet the commented methods have not been tested; the pure scalar importance sort does not represent a complete design for semantic grouping, local rearrangement, or schema traversal.

  8. The impact of flaws in the original experiments still needs to be defined using sensitivity analysis of valid data. The background and balance errors of the radar are reconstructed from the average support ratio, which is an approximation and cannot be directly guaranteed by the formula for fixed support; and the radar is an important part of external support. Discrepancies between E4 support and nominal s, and premature cessation of the E10 raster, may also change specific relative effects. Actively identifying problems is good, but it does not mean that all summary results are robust to them. The supplementary records mentioned in the PDF are not provided this time, therefore it is not yet possible to check which conclusions depend on the affected run.

Questions for authors
  1. Retain "grouping is not enough" as the main conclusion, and clearly distinguish between sufficiency, necessity, and local causal effects. Label H1 as the strong version proposed in this paper, without implying that all existing studies promise invariance of its source and location. Supplement the comparison by manipulating actual grouping under the same clues, locality, origin distribution, and model, or narrow the causal exclusion statement to the current experimental conditions.

  2. Define the clustering degree of the selected set and the true support C separately; add random selection completely independent of the truth value, and different generators with the same precision. Report the common distribution of q, stride, C, P, location, and target cues; compare only in common support regions, and disclose the regression equations, design matrix, and resampling method. Re-estimate the effects and interactions of permutations generated by exact matching or constraints.

  3. Compare zero, learnable, random, and background burn-in initial states; add control over resetting the state within the sequence to see if the advantage shifts with the reset. Examine the readout mechanism using position probes, hidden-state trajectories, and support/background sub-errors, and control the amount of background before and after support and the block boundary structure.

  4. Retain C and P as descriptive statistics, and separately measure the cumulative transition product, implicit interaction weights, or input-output Jacobian. Report the effective distance distribution layer by layer and head by head; recalculate the geometric C with a common τ under learned decay conditions to avoid mixing model decay differences with permutation differences. Test whether the grouping and information interpretation still hold after incorporating actual interaction quantities.

  5. Compare continuous score, rank-only, binary membership, and normalized sequence position channels separately, using the same frozen scorer and paired checkpoints/initializations, and exposing the inputs for each branch. Separate input-derived representation, prior learned through training, and test-target side information; define "information the backbone lacks" as measurable read/computational accessibility, rather than ambiguous information increments.

  6. It is not necessary to reproduce all methods, but at least one existing restoration SSM should be selected, and randomization, locality-matched, and channel control should be added under normal local mixing and training convergence conditions. Higher-resolution natural images should be used, and scale/training expansion should be provided. If additional experiments are not possible, contributions should be explicitly limited to controlled counterexamples and evaluation criteria, rather than the judgment of general visual scanning mechanisms.

  7. Calculate the MSE, Charbonnier, and actual support size for each sample region directly from the original predictions, and recalculate the radar L; rerun the references that stopped early, and use actual support analysis for E4. List the inclusion/exclusion list for each conclusion, providing the difference between the original analysis and only valid data; fail data should not be silently deleted. Clearly define the range of 908, 948, 1108, and 1148, the number of failed runs, and the number of duplicate configuration seeds.

  8. Please compare the following papers: [1] Y. Wu et al., “InfoScan: Information-Efficient Visual Scanning via Resource-Adaptive Walks,” ICLR, 2026. [2] L. Liu et al., “DefMamba: Deformable Visual State Space Model,” CVPR, pp. 8838–8847, 2025.

  9. Permutation Notation: π(i) represents the sequence position of token i, and it should be explicitly stated that gather uses its inverse mapping; the entire text and released code follow the same convention.

  10. In Eq. (1): Explains the base of log, the definition of K≤1, and the causal version that only counts predecessors; C is referred to as the distance-weighted support neighbor count to avoid implying empirically effective coupling.

  11. Figure 5a: Marks nearly identical true C values ​​at two points where q = 0; shows the common x ranges comparable for each q using different line types or tables to avoid implying orthogonality of measurement constructs by parametric design orthogonality.

  12. Figure 6: Plots the error contributions of support and background on the same graph, noting that this graph compares separately retrained models or the same checkpoint intervention; the causal issues differ between the two.

5 comments
AuthorsOpenReview ↗

Response (1/4) - sufficiency vs. proximity

Thank you for a detailed review and a concrete list of controls; the revision runs most of them. We agree with the central point that our evidence refutes the sufficiency of grouping, not the existence of an independent effect of proximity.

W1 / Q1 (sufficiency, necessity, H1 as our strong version).

  • Main conclusion. "Grouping is not enough" is now the main conclusion (title, abstract, Sections 4 and 10).
  • H1. Section 2 labels H1 as the strong form that we formalise. It states that existing methods motivate grouping by proximity without committing to H1's independence from source and position.
  • Causal statements. The revision no longer says "proximity is not the mechanism". The takeaway of Section 4 now notes the local positive effect of grouping seen in the stride sweep (Fig. 2c).

New comparisons that change grouping while holding the cue and the model fixed:

  • GF, grouping without target information (Section 4; clustered supports, s=0.03, 3 seeds). Content-agnostic scans (snake, NSS, Hilbert) raise the true support's log-grouping by +0.10 to +0.56 and change L by −0.16 to +0.23 dB; the largest change is Hilbert, +0.23 [+0.02, +0.42]. The slope with full information (IG, q=1) would predict +1.0 to +1.3 dB for NSS and Hilbert. These scans also raise neighbour retention, so we present this as a bound on the joint effect of grouping and locality without information, not as an isolation of grouping.
  • SG, similarity grouping (Section 6; k-means as in MambaIRv2/CAM). As the order: −0.21 [−0.30, −0.13]. As an input channel: +0.13 [+0.07, +0.21].

W2 / Q2 / Q11 (designed orthogonality is not orthogonality of the measured quantities). We agree. At q=0 the true-support log-grouping is 0.00 at both strides, so these cells test the grouping of selected background tokens. Because q=0 uses the truth to pick background, it carries negative information rather than none. The revision:

  • Removes the claim. "Without information grouping does nothing" is gone from the Fig. 4 caption and text. The caption now says that the q=0 order groups background tokens only, and Fig. 4a marks both q=0 points at log gain 0.00.
  • Separates the two groupings. It defines the grouping of the selected set, CselC_\text{sel}, separately from that of the true support, CC. Table 9 lists q, σ, CselC_\text{sel}, C, the log gain, the seed count and L for all 22 cells, so the grouping ranges shared across q are visible.
  • Adds a truth-independent selection (RS) with expected precision s. At stride 1, 8 and spread, CselC_\text{sel} = 12.75, 2.69 and 0.07, while C stays at raster's 0.58. L = −0.07, −0.08 and −0.09 dB (3 seeds).
  • Leads with the matched comparisons. For example, raising q from 0.5 to 0.75 at equal grouping adds +1.35 [+1.20, +1.50] dB within seeds; R² is now described as descriptive.
  • Discloses the regression (Appendix C). It is OLS over 66 (cell, seed) rows, with intervals for the precision coefficient over rows [2.8, 4.1] and over whole seeds [3.4, 3.5]. With 3 seeds the seed-level interval is coarse, and we say so.
AuthorsOpenReview ↗

Response (2/4) - the anchor; what C measures

W3 / Q3 / Q12 (the anchor is not causally located). We agree that the first set did not intervene on the state. Section 7 now reports interventions: a causal scan without the scan convolution, s=0.02, a ground-truth block, and 4 seeds per arm, each arm a separately trained model.

conditiontokens since the last reset or startL gain (dB)
block at start0+9.94
middle block, state reset at its first token0+8.74
16 learned prefix tokens, then start block16+10.17
64 background tokens, then start block64+9.68
middle block, reset 64 / 128 / 256 tokens before64 / 128 / 256+7.98 / +6.85 / +4.46
502 background tokens, then start block502+3.84
middle block, no reset502+3.83
  • The advantage moves with the reset. On the informative region, a reset at the block equals the start block (S difference −0.01 [−0.05, +0.03]).
  • It falls step by step. Each step from 64 to 128 to 256 tokens to no reset lowers L: +1.13 [+0.27, +2.18], +2.39 [+1.35, +3.12], +0.63 [+0.37, +0.93].
  • What matters is the number of tokens, not absolute image position or whether the tokens are scored. A 502-token background prefix makes a start block equal to a middle block (+0.01 [−0.79, +0.81]).
  • Failed prediction. Our earlier prediction that 64 background tokens would remove the advantage failed. We report it and restated H4 as "an informative block is read more easily the fewer tokens separate it from the last state reset". The fourth set then tested this form with predictions recorded in advance (Table 8).
  • The background contributes, as the reviewer suspected. A start block and a block reset in the middle share S but not the background PSNR (33.3 vs. 20.4 dB). Fig. 13 (Q12) plots the support and background error contributions on the same axes, for E5 and for every intervention. Both captions state that each bar or point is a separately trained model.

W4 / Q4 / Q10 (C is not a validated interaction measure). We agree. C is renamed the distance-weighted support-neighbour count and described as the geometry of an order (Section 2). Appendix I adds:

  • Measured decay. 300 runs with τ=16 record how the trained model's input–output Jacobian decays with scan distance. The fitted horizons have medians of 19–21 tokens across scan direction, convolution and width (pooled interquartile range 18–21), and the exponential fits have median R² of 0.96–0.97. The stacked model thus decays at about the nominal horizon. This is one output token per run, not a distribution over layers or heads, and the anchor experiments show reset information persisting over 128–256 tokens. We therefore keep C descriptive.
  • Prediction (b) of H1 does not depend on τ. In a simulation with uniformly placed supports, the spread lattice groups the support less than raster for every s ∈ {0.005, 0.02, 0.07, 0.35} and τ ∈ {4, …, 1024}.

Eq. 1. Section 2 now states natural logarithms, C = 0 for K ≤ 1, and that counting only predecessors (the causal version) gives exactly C/2.

AuthorsOpenReview ↗

Response (3/4) - channels and information; practical relevance

W5 / Q5 (channel and order are not matched; what "information" means).

  • Channel formats (Section 6). For CFAR: continuous score −0.07, rank only −0.05 [−0.10, −0.00], binary top-K +0.00 [−0.05, +0.05] dB, against −0.45 dB for the CFAR order. The format does not change the result.

  • What we did not run. We did not test a normalised-position channel. The CNN channel and the CNN order were trained separately, not from one frozen scorer with paired initialisations; the revision says so.

  • RB points the other way. In the realistic backbone, the same cluster map helps more as the order (Section 8). We therefore no longer claim that an order works only through one information route. The channel is now presented as a control that separates what a map contains from what reordering adds, and the protocol asks for the gap between the two routes.

  • Terminology (Section 2). We separate three things:

    • target information, which is unavailable at test time; ground-truth orders carry it, i.e. leakage;
    • precision, which is measured against the truth;
    • accessibility, which covers both input-derived representations (a fixed function of y adds no Shannon information) and priors learned from labels (the supervised CNN, which is not leakage).

    "Information the backbone lacks" is now operational: the gain of a route over the raster backbone given y alone.

W6 (equivalence and seed evidence).

  • Definition (Section 3). Two orders are equivalent when the paired mean difference is below 0.1 dB and its 95% interval lies within ±0.2 dB.

  • The direct differences (Appendix C). Sort − random:

    settingdifference (dB)
    first setat most 0.05 per seed pair
    no convolution−0.07 [−0.13, −0.02]
    3× training−0.03 [−0.07, −0.00]
    3.96M testbed+0.02 [−0.02, +0.06]
    radar+0.09 [+0.03, +0.16]
    realistic backbone−0.003 [−0.012, +0.006]
  • Exceptions, now stated in the main text.

    • Natural images: contrast sort − random is +0.46 [+0.43, +0.50] (Section 5).
    • CNN channel vs. order: +0.09 [−0.03, +0.22] does not establish equivalence (Appendix C).
  • The 72 spread runs. They are now described as 24 (sparsity, horizon) cells × 3 seeds that share data, references and seeds. All 24 cell means gain (+0.61 to +6.56 dB), but these are not independent confirmations.

  • Seeds. Appendix C states that with 2–4 seeds the intervals cannot reflect seed variation we did not observe.

W7 / Q6 (connection to practical models). Section 8 adds a MambaIR-style network with normal local mixing, our implementation rather than the official code (0.99M parameters). Each block has a 3×3 depthwise convolution, a four-direction selective scan and channel attention. It is trained on grey DIV2K denoising (σ=50) for 4k steps (2–3 seeds) and 12k steps (1 seed). The reference rises from 15.05 dB (input) to 28.49 dB and gains only +0.04 dB over its last 2k steps. The table gives PSNR change vs. the standard scan:

order4k steps12k steps
random permutation−0.03−0.03
importance sortequal to random (−0.003)—
clusters of the input+0.04 [+0.02, +0.06]+0.02
clusters of the clean image (oracle)+2.55+3.14

The random-permutation and oracle patterns carry over; the channel matching the order does not. These runs test the shared ingredient of content-aware methods, a per-image order computed from content, and not the methods themselves. As the reviewer suggests, the contribution is now framed as controlled counterexamples, a realistic-backbone check, and evaluation criteria.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revision uploaded: what the new evidence confirms, and how we narrowed the claims

We thank the reviewers for careful and constructive reviews. We uploaded a revised PDF, a revised supplement, and a new title and abstract. During the discussion period, we trained 149 additional models. Before each set, we recorded the predictions of each competing account; the plans are in the supplement, and Table 8 lists every outcome, including the predictions that failed.

What the new evidence confirms

  1. Importance sorts behave like a random permutation, now in six settings, all with paired differences below 0.1 dB:

    • the first set;
    • no scan convolution;
    • 3× longer training;
    • a 5.7× larger testbed (3.96M parameters): +0.02 [−0.02, +0.06];
    • radar with exact region errors: +0.09 [+0.03, +0.16];
    • a MambaIR-style backbone on natural images: −0.003 [−0.012, +0.006] dB.

    Section 4 and Appendix C list them.

  2. Grouping without target information changes little.

    • Content-agnostic scans that raise the true support's log-grouping by +0.10 to +0.56 on clustered supports change L by −0.16 to +0.23 dB. The grouping slope measured with full information would predict +1.0 to +1.3 dB (Section 4).
    • A truth-independent selection whose own grouping ranges from 0.07 to 12.75 changes L by −0.07 to −0.09 dB (Section 6).
  3. "Oracle" orders measure leakage, also in a realistic backbone (Section 8, Table 2).

    • k-means clusters of the clean image gain +2.55 dB (4k steps) and +3.14 dB (12k steps).
    • The same clustering of the noisy input gains +0.04 and +0.02 dB, about the size of the semantic-grouping ablation reported for MambaIRv2.
  4. Interventions on the recurrent state locate the start-of-scan effect (Section 7, Fig. 5c).

    • A state reset at a middle block restores the start block's gain on the informative region.
    • The gain falls with the number of tokens since the last reset: L +8.74, +7.98, +6.85 and +4.46 dB at 0, 64, 128 and 256 tokens, against +3.83 dB without a reset.
    • A 502-token background prefix turns a start block into a middle block (difference +0.01 [−0.79, +0.81]).

How we narrowed the claims

  • Title and abstract. The new title is "Is Grouping Enough? Controlled Tests of Content-Aware Scanning in Vision State Space Models". The abstract states the controlled scope.
  • H1. We now call H1 the strong form that we formalise (Section 2). Its failure shows that grouping is not sufficient; it leaves room for a local effect of proximity, which our stride sweep shows.
  • What we do not claim. We do not evaluate existing content-aware methods, and we no longer state that "proximity is not the mechanism".
  • "Information". We split it into target information (leakage), selection precision, and accessibility (Section 2).
AuthorsOpenReview ↗

Failed predictions, corrections, and a map of the changes

Predictions that failed

  • (a) Channel vs. order in the realistic backbone. The same cluster map helps more as the order than as an input channel. Channel − order is −0.81 [−0.95, −0.68] dB for clean-image clusters and −0.07 [−0.10, −0.04] dB for input clusters. In the testbed the channel helps more (+1.64 dB at 3.96M parameters).
    • The claim that a channel helps "as much or more" is now restricted to the testbed.
    • The protocol now asks for both routes and the gap between them (Sections 6, 8, 10).
    • In RB the channel enters only at the stem, while the order acts in every block; we state this.
  • (b) The empty-state form of the anchor. A 64-token background prefix and a reset 64 tokens before a middle block both kept most of the start advantage. This led to the reset-distance form of H4, which the fourth set then tested prospectively (Section 7).
  • (c) Reproducibility. Repeats of one configuration and seed are not always bit-identical (next list).

Corrections

  • Determinism (Appendix D). The submitted Appendix H claimed that training is deterministic given the configuration and seed. That claim is wrong.
    • Bit-identical repeats: 2 of 3 synthetic references and 10 of 19 radar reruns.
    • The others differ by up to 0.02 dB in mean L (synthetic) and 0.46 dB in mean whole-image PSNR (radar).
    • The likely cause is that cuDNN autotuning was enabled.
  • "Stopped early" runs (Appendix D). The three first-set runs we listed as stopped early had in fact been resumed from a checkpoint and completed. Each record holds an evaluation at the final step, and a from-scratch rerun of the E10 reference reproduces it exactly.
  • Radar. Radar L is now exact (Appendix B). The approximation had shifted levels by 0.17–0.19 dB, but not the comparison: CFAR − random is +0.09 exactly, against +0.10 approximated.
  • Equivalence (Section 3, Appendix C). We now define it explicitly: a paired mean difference below 0.1 dB with a 95% interval within ±0.2 dB. The exceptions are stated: natural images (contrast sort − random +0.46 dB), and the CNN's channel vs. order, where equivalence is not established.

Map of the changes

WhereWhat changed
Section 2H1 as strong form; permutation convention (gather uses π⁻¹); C renamed as the distance-weighted support-neighbour count; three kinds of information
Section 3equivalence definition; the later sets
Section 4grouping on clustered supports (GF)
Section 5locality in RB; natural-image exception
Section 6q = 0 reworded; truth-independent selection; channel formats; similarity clusters
Section 7reset interventions (H4 restated; Fig. 5c)
Section 8realistic backbone (Table 2)
Section 9InfoScan and DefMamba; positioning against QuadMamba, MaIR, REOrder, Mamba-Reg
Section 10narrowed limitations and conclusion
Appendix Cregression details, seed-level bootstrap, counting units
Appendix Drepeats; evidence used by each conclusion (Table 4)
Appendix Ithe new sets: predictions and outcomes (Table 8), backbone details, measured decay, support/background errors (Fig. 13), all IG cells (Table 9), all new arms (Table 10)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.