acceptodds
Under review as a conference paper at ICLR 2027

Beyond Sequential Scanning: Rethinking Visual State Propagation with Neighborhood State Interaction

Abstract

Visual state space models (VSSMs) provide an efficient mechanism for modeling long-range dependencies in vision tasks. However, many existing visual state space models mainly rely on sequential scanning to organize state propagation, which introduces indirect interactions between spatially adjacent locations and makes local information exchange dependent on intermediate states. In this work, we rethink visual state propagation beyond sequential scanning and introduce Neighborhood State Interaction (NSI), a novel state interaction mechanism that enables direct interactions among neighboring states during each update. Instead of propagating states along predefined scan paths, NSI performs synchronous neighborhood interactions on two-dimensional feature maps and progressively expands contextual information through multiple propagation steps. To make multi-step state interaction efficient and stable, we generate input-conditioned interaction kernels and share them across propagation steps, while using state-dependent retention to adaptively control accumulated information and incremental readout to combine representations from different propagation stages. Extensive experiments on image restoration tasks demonstrate that our approach consistently improves over SS2D, achieving PSNR gains of 0.50 dB on the GoPro dataset for deblurring, 1.15 dB on the O-HAZE dataset for dehazing, and 1.37 dB on the Haze4K dataset for dehazing, while reducing MACs by 25.1% and 30.2% for deblurring and dehazing models, respectively.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual State Space Models, Mamba, State Propagation, Neighborhood State Interaction, Image Restoration, Efficient Vision Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DDaiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper focuses on the design of fundamental visual operators and architectural improvements for image restoration. It investigates whether visual state propagation can move beyond Mamba's one-dimensional sequential scanning and operate directly in two-dimensional image space. To this end, the authors propose a new token mixer, Neighborhood State Interaction (NSI), which enables multi-step state propagation through synchronous neighborhood interactions. It further incorporates input-conditioned interactions, state-dependent retention, and incremental readout to improve propagation efficiency and stability. Overall, the paper addresses a meaningful research problem, and the proposed method has a certain theoretical foundation. However, my main concern is the limited scope of the experimental evaluation.

Critical strengths
  1. The research problem is meaningful. The paper rethinks visual state propagation by considering the mismatch between two-dimensional spatial adjacency and one-dimensional scan order. This is a problem with fundamental research significance.

  2. The module design is relatively comprehensive. The proposed method combines input-conditioned interactions, state-dependent retention, parameter sharing across propagation steps, and incremental readout. These components are well aligned with the research motivation. In addition, the authors provide a theoretical analysis of the stability of multi-step propagation.

Critical weaknesses
  1. The choice of backbone is relatively limited. Although the authors conduct experiments on 3 datasets and 2 image restoration tasks, all experiments are based on the VIVNet backbone. Given that NSI is proposed as a general-purpose token mixer, the lack of evaluation on other network architectures leaves its generalizability across different backbones unclear.

  2. The comparisons with other scanning mechanisms are insufficient. The authors mainly compare NSI with VMamba's SS2D and additionally include MaIR and Spatial-Mamba. However, only SS2D is evaluated on all 3 datasets, while MaIR and Spatial-Mamba are evaluated only on O-HAZE, which contains just 5 test images. Furthermore, the paper lacks quantitative comparisons with other relevant scanning mechanisms.

Questions for authors
  1. Can NSI preserve Mamba's advantages in long-range dependency modeling? With the default 5 propagation steps, a single NSI module has a direct propagation range of only 11 × 11. By replacing sequential scanning with local neighborhood interactions, NSI may lose one of Mamba's key advantages: efficient long-range dependency modeling. Could the authors further discuss this potential limitation?

  2. Does NSI generalize across different backbones? The current experiments mainly demonstrate that NSI outperforms SS2D within VIVNet, but it remains unclear whether similar improvements can be achieved with other network architectures.

  3. Could the authors provide additional comparisons with other scanning mechanisms? I suggest including more comparisons with different scanning mechanisms to further validate the effectiveness of NSI relative to alternative scanning strategies.

9 comments
AuthorsOpenReview ↗

Response to Reviewer DDai (Part 1 of 2): Q1 and W1/Q2

Thank you very much for your thoughtful review, and for recognizing both the significance of the research problem and how the module design follows from it. We have revised the manuscript based on your feedback. Below, we quote each point and describe the revision and where it appears. Please let us know whether we have addressed your concerns.


Q1 – Long-range dependency modeling

Question 1: "Can NSI preserve Mamba's advantages in long-range dependency modeling? With the default 5 propagation steps, a single NSI module has a direct propagation range of only 11 × 11. [...] Could the authors further discuss this potential limitation?"

A1. Thank you for raising this. A single NSI module does not keep Mamba's global path: with K=5K = 5, its direct reach is 11×11 positions at that module's resolution, and we now state this trade-off in the conclusion (Section 6, p. 10). What NSI keeps from Mamba is selective state propagation, with selection that depends on the input and on the evolving state. What it gives up is the ordering of that propagation along a 1D sequence.

In the network, long-range context comes from composition across scales and blocks. The theoretical receptive field of VIVNet-B with NSI is 776 pixels, and the measured ERF is wide and nearly isotropic (Table R1).

Table R1: ERF over 47 test images and three seeds (Table 15, p. 28).

GoProO-HAZEHaze4K
r90r_{90}, NSI (pixels)91118152
r90r_{90}, CSPN-style (pixels)72126153
Axis-to-diagonal ratio, NSI / SS2D1.15–1.21 / 3.9–21.7

On the tasks we tested, this did not cost accuracy: under VIVNet's full Haze4K schedule NSI is 0.35 dB above SS2D, and on GoPro at 200K updates it is 0.12 dB above (Table 3, p. 8). The number of steps KK trades reach against cost without affecting stability (Table 5(a), p. 9). Combining depth-indexed propagation with a lightweight global path is a natural extension that we leave to future work.


W1, Q2 – Generalization across backbones

Weakness 1: "The choice of backbone is relatively limited. [...] all experiments are based on the VIVNet backbone. Given that NSI is proposed as a general-purpose token mixer, the lack of evaluation on other network architectures leaves its generalizability across different backbones unclear."

Question 2: "Does NSI generalize across different backbones? [...]"

A2. Thank you for this suggestion. We added a second backbone: EVSSM (CVPR 2025), a native Mamba deblurring network. We replaced all 48 of its SS2D mixers with NSI and kept every other component. NSI reaches 31.29 vs. 30.88 dB with 24.4% fewer MACs and 25% fewer parameters, and is better on 866 of 1,111 frames and on 10 of 11 test videos (two-sided video-level sign test, p=0.012p = 0.012). Our evaluation of the released EVSSM checkpoint gives 34.53 dB against the 34.5 dB reported.

Since all our evidence is on restoration, we have also revised the title to end with "for Image Restoration", and we no longer present NSI as a general-purpose token mixer.

Revision: Table 4 (p. 8) and Appendix E (p. 33); title, abstract and Section 6 (p. 10).

AuthorsOpenReview ↗

Follow-up: have our responses addressed your concerns?

Dear Reviewer DDai,

Thank you again for your thoughtful review and for your recognition of the research problem. As the discussion period closes on November 18 (AoE), we would like to check whether our responses and the revised paper have addressed your concerns. Since our first response, we have also trained EAMamba and C2SSM under VIVNet's full Haze4K schedule: on the 1,000-image test set NSI is above both, including EAMamba, which was slightly ahead on the five-image O-HAZE (Table 22). In particular:

  1. A second backbone, the native Mamba network EVSSM (Table 4; Appendix E).
  2. Additional scan mechanisms on the 1,000-image Haze4K test set (Table 22), where NSI is the most accurate at the lowest MACs.

If any point remains unclear, we would be glad to clarify during the discussion period. If you find that your concerns have been resolved, we would be grateful if you would consider updating your assessment.

AuthorsOpenReview ↗

Response to Reviewer DDai (Part 2 of 2): W2/Q3

W2, Q3 – Comparisons with other scanning mechanisms

Weakness 2: "The comparisons with other scanning mechanisms are insufficient. [...] only SS2D is evaluated on all 3 datasets, while MaIR and Spatial-Mamba are evaluated only on O-HAZE, which contains just 5 test images. Furthermore, the paper lacks quantitative comparisons with other relevant scanning mechanisms."

Question 3: "Could the authors provide additional comparisons with other scanning mechanisms? [...]"

A3. Thank you for this suggestion. We added four scanning mechanisms and evaluated them on the 1,000-image Haze4K test set under VIVNet's full training schedule (Table R1): window-ordered scans (Local4, as in LocalMamba), eight-direction scans (RS8, as in RS-Mamba), EAMamba and C2SSM.

Table R1: Scanning mechanisms under VIVNet's full Haze4K schedule (seed 42; 1,000 test images; Table 22, p. 34).

MixerPSNR (dB)SSIMMACs (G)
EAMamba35.320.990827.8
SS2D35.160.990236.7
RS835.070.990051.7
C2SSM34.240.989534.5
Local433.610.987736.7
Ours35.510.992225.6

NSI is the most accurate of these scans at the lowest MACs. The comparison also shows how strongly a scan depends on its routes: RS8 is within 0.1 dB of SS2D, whereas Local4, which keeps every SS2D component and changes only two of its four routes, is 1.55 dB lower. NSI has no route to choose.

We also retrained the same four scans on O-HAZE with the main protocol (Table 26, p. 36). They reach 26.67, 26.38, 26.94 and 26.86 dB, against 26.78 dB for NSI. With five test images and one seed, differences of 0.1–0.2 dB are within the seed variation of Table 2, so the 1,000-image Haze4K comparison above is the more reliable one. In addition, SS2D is now compared over three seeds on all three main datasets (Tables 1–2, pp. 7–8).

For MaIR and Spatial-Mamba, we verified each transfer against the official code (forward error 0, gradients within 1e-6, all 22 locations running the official modules; Appendix I, p. 35), and showed that their low O-HAZE scores come mainly from the evaluation resolution (Appendix D.3, p. 32).

Revision: Table 22 and Appendix F (pp. 33–34); Table 26 and Appendix I (pp. 35–36); Appendix D.3 (p. 32).


Thank you again for your encouraging assessment of the problem and for suggestions that broadened our evaluation. We would be glad to provide any further clarification before the discussion period closes.

Reviewer_WkrMOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper critiques 1D sequential scanning in Visual State Space Models (VSSMs) for image restoration, noting that 1D scan paths separate spatially adjacent (especially diagonal) neighbors by many recurrent steps. The authors propose Neighborhood State Interaction (NSI), which replaces the SS2D scan in VIVNet’s token mixer with K=5 synchronous 3×3 neighborhood updates using input-conditioned 2×2 pair kernels shared across steps, a state-dependent retention nonlinearity, an incremental readout over step increments, and spectral-norm gain control.

Critical strengths
  1. Complete implementation details. The authors document random seed variance on GoPro (Table 9), numerical equivalence of their custom Triton kernel, and the distinction between independently retrained controls and fixed-weight interventions.
  2. Well mathematical Rigor and Transparency. The Jacobian norm derivation for groupwise RMS retention in Appendix A.2 is mathematically thorough, and the empirical verification of internal state bounds in Table 13 is good.
Critical weaknesses
  1. This paper proposes a local operator, so the whole VSSM framing is wrong. Information travels at most 11×11 as mentioned in Sec. 3.1. Tab. 11 shows an ERF smaller than SS2D’s and on par with DWConv. The paper drops the property that motivates state propagation and then benchmarks MACs against a global operator. The real baselines are absent, like dynamic conv, involution, CSPN++ and NLSPN.

  2. The two novel components are inert by the paper’s own table. Tab. 5: removing retention changes GoPro by +0.011 dB; removing incremental readout by +0.021. Those are the only substantive differences from CSPN.

  3. GoPro is severely under-trained and the ported baselines are broken. 49K steps at 128² crops (Tab. 6) leave every GoPro number 2–3 dB below the literature (NSI 30.87 vs. NAFNet/Restormer 32.9–33.7).

Questions for authors
  1. At matched MACs, how does NSI compare to a 5-step dynamic/involution-style 3×3 operator and to CSPN++/NLSPN inside the same VIVNet block? If NSI is a local operator, these — not SS2D — are the relevant baselines.

  2. Given Tab. 5, what is the justification for keeping retention and incremental readout?

  3. Please train NSI and SS2D on GoPro at a standard budget (≥200K iterations, 256² crops) and on Haze4K at VIVNet’s official schedule (300K iterations, batch 8). Does the 0.5 dB margin survive? Also, why do ported MaIR and Spatial-Mamba score 3 dB below plain Swin in Tab. 15, were their official configurations verified to reproduce their published numbers?

6 comments
AuthorsOpenReview ↗

Response to Reviewer WkrM (Part 1 of 3): W1/Q1, state-space framing and local baselines

Thank you sincerely for the time you put into this review, and for acknowledging our implementation details and the rigor of the retention analysis. Your critique was direct and specific, and the experiments you asked for have made the paper considerably stronger. Below, we quote each point and describe the revision and where it appears.


W1, Q1 – Local operator, VSSM framing and baselines

Weakness 1: "This paper proposes a local operator, so the whole VSSM framing is wrong. Information travels at most 11×11 as mentioned in Sec. 3.1. Tab. 11 shows an ERF smaller than SS2D's and on par with DWConv. The paper drops the property that motivates state propagation and then benchmarks MACs against a global operator. The real baselines are absent, like dynamic conv, involution, CSPN++ and NLSPN."

Question 1: "At matched MACs, how does NSI compare to a 5-step dynamic/involution-style 3×3 operator and to CSPN++/NLSPN inside the same VIVNet block? If NSI is a local operator, these — not SS2D — are the relevant baselines."

A1. We have taken this point seriously. Our original text did not separate two questions clearly enough: how the state is updated, and how far it propagates. We answer them in turn.

(a) How the state is updated: NSI is a state-space recurrence over depth. For a fixed module input ff,

hk+1=Lf(df+Rf(hk)),zk+1=zk+ωk+1(hk+1−hk),y=Wout(Cf⊙zK).h_{k+1} = L_f\big(d_f + R_f(h_k)\big),\qquad z_{k+1} = z_k + \omega_{k+1}(h_{k+1} - h_k),\qquad y = W_{\mathrm{out}}(C_f \odot z_K).

Table R1 (Table 27 of the revision) maps each state-space component of SS2D to NSI. With identity retention, the update reduces to a conditional linear state-space recurrence. Locality fixes only the support of the transition; state-space models on spatial grids with local shift operators are standard in control [1, Section III]. What NSI changes is the recurrence axis: propagation depth on the 2D grid instead of a 1D token order.

Table R1: State-space components (Table 27, p. 37).

ComponentSS2DNSI
StatePer-channel state along a 1D sequence2D field of channel-pair states
Recurrence indexToken position along each scanPropagation step kk
Input termCurrent tokendfd_f, injected at every step
SelectionToken-conditioned coefficientsInput-conditioned kernels and state-dependent retention
ReadoutInput-dependent CCIncrement accumulation, then CfC_f and projection
Decay / boundNegative real part of AAFixed-input contraction (Proposition 1)

(b) How far it propagates. 11×11 is the direct reach of one module; across VIVNet's scales and blocks, the theoretical receptive field is 776 pixels. Over 47 test images and three seeds, NSI's ERF radius r90r_{90} is 91–152 pixels, close to CSPN-style's (72–153), and nearly isotropic: the axis-to-diagonal ratio is 1.15–1.21 for NSI versus 3.9–21.7 for SS2D (Table 15, p. 28). A single NSI module has no global path, and the conclusion states this limitation.

(c) Why we still compare with SS2D. For a VSSM designer, the question is whether the global scan buys accuracy. Under the budgets you requested (Part 3), NSI is above SS2D with 25–30% fewer MACs, so on these restoration tasks the global path did not translate into higher accuracy.

(d) The local baselines you named. We added all of them inside the same VIVNet block, at similar parameters and MACs (Table R2).

Table R2: Local baselines (PSNR, dB).

BaselineGoProO-HAZEHaze4K, 300K
Involution-style, 5 steps (Table 4)28.3825.24—
NLSPN-style [2] (Table 25)—25.74—
CSPN++-style [3] (Table 25)—26.13—
CSPN-style, three seeds30.2026.3333.98
Ours30.8726.44 (seed 42: 26.78)35.51

Recurrent dynamic convolution, which regenerates kernels from the state at every step, gives lower or similar PSNR at 24.5–32.9% more MACs (Table 5(b), p. 9).

Revision: Algorithm 1 and the state-space statement in Section 3.1 (pp. 4–5); Appendix J (Table 27, p. 37); ERF in Appendix C.4 (Table 15, p. 28); local baselines in Table 4 (p. 8) and Appendix H (Table 25, p. 35).

[1] Bamieh, Paganini and Dahleh, Distributed control of spatially invariant systems, IEEE TAC 2002, https://web.mit.edu/dahleh/OldFiles/www/pubs/bamieh01.pdf [2] Park et al., NLSPN, https://arxiv.org/abs/2007.10042 [3] Cheng et al., CSPN++, https://arxiv.org/abs/1911.05377

AuthorsOpenReview ↗

Response to Reviewer WkrM (Part 2 of 3): W2/Q2, retention and incremental readout

W2, Q2 – Retention and incremental readout

Weakness 2: "The two novel components are inert by the paper's own table. Tab. 5: removing retention changes GoPro by +0.011 dB; removing incremental readout by +0.021. Those are the only substantive differences from CSPN."

Question 2: "Given Tab. 5, what is the justification for keeping retention and incremental readout?"

A2. Thank you for pointing this out. You are right about GoPro: removing either component changes PSNR by only +0.01 / +0.02 dB, and we no longer claim that they help on every task. Our justification for keeping them rests on four pieces of evidence.

(a) They matter on dehazing. Table R1 (Table 11 of the revision) reports the independently retrained controls on all three datasets.

Table R1: Retrained component controls (seed 42; PSNR, dB; Table 11, p. 24).

ConfigurationO-HAZEGoProHaze4K
Full model26.7830.8930.11
Identity retention26.44 (−0.33)30.90 (+0.01)30.05 (−0.07)
Final-state readout26.32 (−0.45)30.91 (+0.02)29.33 (−0.78)
Both replaced26.26 (−0.52)30.88 (−0.01)29.92 (−0.19)

On Haze4K, the readout comparison is repeated over three seeds: 29.93 ± 0.29 with incremental readout vs. 29.66 ± 0.33 dB with final-state readout (Section 5.3, p. 10).

(b) The trained model uses the spatial pattern of retention. With the trained O-HAZE model fixed, replacing each retention map by its spatial mean lowers PSNR by 0.22 dB even after output magnitudes are matched, and lowers it on all five test images; cyclically shifting the maps by half the grid lowers it by 0.40 dB (Table 16, p. 29). This intervention cannot be compensated by retraining, so it shows directly that the network relies on where retention is strong or weak.

(c) They are not the only differences from CSPN. With both components removed, NSI is still 0.59 dB above the seed-matched CSPN-style mixer on GoPro (30.88 vs. 30.29 dB, seed 42; Tables 11 and 12, pp. 24–25). The remaining gap comes from the input-generated pairwise channel transitions shared across steps and from gain control, which CSPN-style lacks.

(d) Their roles in the analysis. Retention with positive decay bounds the contribution of any previous state (Appendix A.4, p. 16), and incremental readout weights the information added at each depth, which yields the readout bound of Appendix A.3. Neither component adds MACs.

We therefore keep one design across tasks and describe the effects of retention and readout as task-dependent.

Revision: Section 5.3 and Table 6 (p. 10); Tables 11 and 12 (pp. 24–25); Table 16 (p. 29); Appendix A.3–A.4 (p. 16).

AuthorsOpenReview ↗

Follow-up: have our responses addressed your concerns?

Dear Reviewer WkrM,

Thank you again for your detailed review. As the discussion period closes on November 18 (AoE), we would like to check whether our responses and the revised paper have addressed your concerns. Since our first response, we have also trained EAMamba, C2SSM and Swin under VIVNet's full Haze4K schedule; NSI remains the most accurate of all eleven mixers (Table 22). In particular:

  1. Standard budgets you requested (Table 3; Appendix F): NSI is above SS2D on GoPro at 200K updates, and attains the highest PSNR and SSIM of all eleven mixers under VIVNet's full Haze4K schedule.
  2. Local baselines you named (Tables 4 and 25): involution-, CSPN-, NLSPN- and CSPN++-style mixers inside the same block, all below NSI at similar cost.

If any point remains unclear, we would be glad to provide further clarification or results during the discussion period. If you find that your concerns have been resolved, we would be grateful if you would consider updating your assessment.

Reviewer_iKJ7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Neighborhood State Interaction (NSI), which replaces one-dimensional visual state-space scanning with synchronous 3-by-3 neighborhood propagation. NSI uses input-conditioned kernels shared across propagation steps, repeated input injection, state-dependent retention, and incremental readout. A gain-normalization scheme makes the internal fixed-input update contractive. Within VIVNet, NSI improves deblurring and dehazing performance over SS2D while using fewer parameters and MACs.

Critical strengths
  1. The method is clearly defined and the controlled replacement of VIVNet's token mixer is reasonably fair.

  2. The ablations cover the main components, including propagation depth, kernel sharing, retention, readout, and gain control.

  3. GoPro improvements are consistent across three seeds, and the paper provides unusually detailed implementation and evaluation protocols.

  4. The internal contractivity analysis offers a useful explanation for stable recurrent propagation.

Critical weaknesses
  1. The novelty relative to dynamic convolution, recurrent convolution, and CSPN-style spatial propagation is not sufficiently clear. NSI appears closer to a dynamic spatial propagation layer than to a new state-space formulation. Stronger parameter- and compute-matched spatial propagation baselines are needed.

  2. The experimental scope is narrow. All main results use one VIVNet backbone and only deblurring/dehazing tasks. This supports effectiveness within the tested restoration setting, but not the broader claim of rethinking visual state propagation.

  3. Statistical evidence is uneven. O-HAZE has only five test images and uses one training seed; Haze4K and most ablations also use one seed. Small improvements in the ablations may therefore reflect training variance.

  4. The theoretical and efficiency claims need qualification. Contractivity applies only to the fixed-input internal propagation before output scaling and projection, not to the complete block or network. Moreover, 25%--30% MAC reductions yield only about 3%--4% measured latency improvement.

Questions for authors
  1. How does NSI compare with stronger, parameter-matched CSPN and recurrent dynamic-convolution baselines?

  2. Does it remain advantageous in another backbone or a non-restoration task?

  3. Can the authors provide multi-seed O-HAZE and Haze4K results?

  4. What limits the wall-clock speedup despite the large MAC reduction?

4 comments
AuthorsOpenReview ↗

Response to Reviewer iKJ7 (Part 1 of 2): W1/Q1 and W2/Q2

Thank you very much for your careful and balanced review, and for recognizing the clarity of the method, the coverage of the ablations and the detail of our protocols. Below, we quote each point and describe the revision and where it appears. Please let us know whether we have addressed your concerns.


W1, Q1 – Novelty relative to dynamic convolution, recurrent convolution and CSPN

Weakness 1: "The novelty relative to dynamic convolution, recurrent convolution, and CSPN-style spatial propagation is not sufficiently clear. [...] Stronger parameter- and compute-matched spatial propagation baselines are needed."

Question 1: "How does NSI compare with stronger, parameter-matched CSPN and recurrent dynamic-convolution baselines?"

A1. NSI shares iterative neighborhood propagation with CSPN [1]; the difference lies in how the state is updated. Table R1 (Table 19 of the revision) summarizes the design axes. CSPN, NLSPN and GSPN are linear in the state once their affinities are predicted, and SS2D selects from the input alone along a 1D order. NSI updates all positions synchronously and selects from both the input and the evolving state: interaction kernels are generated once from the input and shared across steps, while retention is recomputed from the state at every step, which makes the update nonlinear. A gain bound then guarantees contraction of this nonlinear update.

Table R1: Design comparison (Table 19, p. 31).

OrderNeighbors per updateSelection fromUpdateStability
CSPNSynchronous8 (fixed 3×3)InputLinearAffinity normalization
NLSPNSynchronous8 (learned, nonlocal)InputLinearAffinity normalization
GSPNLine by line3 (preceding line)InputLinearNormalized weights
SS2D1-D scans1 (predecessor)InputLinearDecay below one
NSISynchronous8 (fixed 3×3)Input and stateNonlinearContraction

We trained four matched alternatives inside the same VIVNet block (Table R2); each has parameters and MACs within a few percent of NSI in its backbone (VIVNet-B for GoPro, VIVNet-S for dehazing).

Table R2: Matched baselines (PSNR, dB). ± denotes three seeds; other entries use seed 42.

BaselineParams / MACsGoProO-HAZEHaze4K
CSPN-style [1]5.79M / 59.1G30.20 ± 0.0926.33 ± 0.2627.07 ± 0.45
NLSPN-style [2]2.27M / 24.7G—25.74—
CSPN++-style [3]2.25M / 27.6G—26.13—
Involution-style, 5 steps [4]6.20M / 61.8G28.3825.24—
Ours5.81M / 61.4G30.87 ± 0.0526.44 ± 0.29 (seed 42: 26.78)29.93 ± 0.29

For recurrent dynamic convolution, regenerating the kernels from the state at every step adds 24.5–32.9% MACs and changes PSNR by −0.44, −0.98 and +0.06 dB on O-HAZE, Haze4K and GoPro (Table 5(b), p. 9). Under the full Haze4K schedule, NSI leads CSPN-style by 1.53 dB (Table 22, p. 34).

Revision: the distinction from spatial propagation networks is added to Section 2 (p. 3) and Appendix D.1 (Table 19, p. 31); NLSPN- and CSPN++-style controls are in Appendix H (Table 25, p. 35); the involution-style control is in Table 4 (p. 8).


W2, Q2 – Backbone and task scope

Weakness 2: "The experimental scope is narrow. All main results use one VIVNet backbone and only deblurring/dehazing tasks. This supports effectiveness within the tested restoration setting, but not the broader claim of rethinking visual state propagation."

Question 2: "Does it remain advantageous in another backbone or a non-restoration task?"

A2. Thank you for this suggestion. We added a second backbone, the native Mamba deblurring network EVSSM (CVPR 2025). We replaced all 48 of its SS2D mixers with NSI and kept every other component, the initialization of the remaining weights and the batch order. NSI reaches 31.29 vs. 30.88 dB with 24.4% fewer MACs and 25% fewer parameters, and is better on 866 of the 1,111 test frames and on 10 of the 11 test videos. Our evaluation of the released EVSSM checkpoint gives 34.53 dB against the 34.5 dB reported, which checks the evaluation pipeline.

We agree that our evidence supports the restoration setting, and we have narrowed the claim accordingly: the title now ends with "for Image Restoration", and the conclusion states that NSI has been evaluated only on image restoration.

Revision: EVSSM results are in Table 4 (p. 8) and Appendix E (p. 33); the scope statement is in the title, the abstract and Section 6 (p. 10).

[1] Cheng et al., CSPN, https://arxiv.org/abs/1810.02695 [2] Park et al., NLSPN, https://arxiv.org/abs/2007.10042 [3] Cheng et al., CSPN++, https://arxiv.org/abs/1911.05377 [4] Li et al., Involution, https://arxiv.org/abs/2103.06255

AuthorsOpenReview ↗

Follow-up: have our responses addressed your concerns?

Dear Reviewer iKJ7,

Thank you again for your balanced review. As the discussion period closes on November 18 (AoE), we would like to check whether our responses and the revised paper have addressed your concerns. Since our first response, we have also completed three seeds for DWConv and Swin, so every mixer in Tables 1–2 now has three seeds. In particular:

  1. Matched spatial propagation and dynamic-convolution baselines (Tables 4, 5 and 25).
  2. A second backbone, the native Mamba network EVSSM (Table 4; Appendix E), and the title now scoped to image restoration.
  3. Three seeds for every mixer on O-HAZE and Haze4K (Tables 1–2).

If any point remains unclear, we would be glad to clarify during the discussion period. If you find that your concerns have been resolved, we would be grateful if you would consider updating your assessment.

AuthorsOpenReview ↗

Response to Reviewer iKJ7 (Part 2 of 2): W3/Q3 and W4/Q4

W3, Q3 – Seeds and statistical evidence

Weakness 3: "Statistical evidence is uneven. O-HAZE has only five test images and uses one training seed; Haze4K and most ablations also use one seed. Small improvements in the ablations may therefore reflect training variance."

Question 3: "Can the authors provide multi-seed O-HAZE and Haze4K results?"

A3. Thank you for this suggestion. We retrained every mixer of Table 2 with seeds 43 and 44, so all entries on O-HAZE and Haze4K are now three-seed means (Table R3). NSI is above SS2D for every seed on both datasets.

Table R3: Dehazing over seeds 42–44 (PSNR, dB; Table 2, p. 8).

MixerO-HAZEHaze4KMACs (G)
VIVNet-S (native)26.05 ± 0.2229.88 ± 0.1931.78
DWConv26.21 ± 0.2829.57 ± 0.0428.36
Swin25.56 ± 0.0727.78 ± 0.2234.52
CSPN-style26.33 ± 0.2627.07 ± 0.4524.30
SS2D25.58 ± 0.3528.31 ± 0.3936.70
Ours26.44 ± 0.2929.93 ± 0.2925.61

Some margins are smaller than seed variation: over native VIVNet on Haze4K (0.05 dB) and over CSPN-style and DWConv on O-HAZE (0.12 and 0.23 dB). The revised text reports them as such. For the ablations, the Haze4K comparison between incremental and final-state readout is now repeated over three seeds: 29.93 ± 0.29 vs. 29.66 ± 0.33 dB (Section 5.3, p. 10). Haze4K (1,000 test images) and GoPro (1,111 frames) carry the main test coverage.

Revision: Table 2 and its discussion in Section 4.2 (p. 8); Section 5.3 (p. 10).


W4, Q4 – Scope of the theory and wall-clock speed

Weakness 4: "The theoretical and efficiency claims need qualification. Contractivity applies only to the fixed-input internal propagation before output scaling and projection, not to the complete block or network. Moreover, 25%–30% MAC reductions yield only about 3%–4% measured latency improvement."

Question 4: "What limits the wall-clock speedup despite the large MAC reduction?"

A4. We agree with both qualifications. The revision states the scope of the theorem wherever it is mentioned (contribution list, Section 3.5 and the conclusion): it concerns the nonlinear state update at a fixed module input, before output scaling and projection, not the complete block or network. Within that scope the argument has three steps (Appendix A, pp. 14–17):

  1. With channel-pair norm NN and kernel gain qf<1q_f < 1, the retention Jacobian satisfies ∣DRf∣N≤1\\|DR_f\\|_N \le 1 whenever a retention group has at most eight channels, so N(Tf(h)−Tf(v))≤qf,N(h−v)N(T_f(h) - T_f(v)) \le q_f\\, N(h - v) (Appendix A.2, p. 15). For positive decay, the eight-channel condition is also necessary.
  2. From zero initialization, the state increments decay geometrically: N(Δhk)≤qf,k−1N(Δh1)N(\Delta h_k) \le q_f^{\\,k-1} N(\Delta h_1).
  3. With nonnegative, nondecreasing readout weights that sum to KK, the readout norm is bounded independently of KK (Appendix A.3, p. 16).

Both results are also checked in Lean 4 (supplementary material).

On wall-clock speed, the recurrence is memory-bound. Hardware counters (Nsight Compute) show that NSI's two recurrence kernels run at 90–92% of peak DRAM throughput but only 8–11% of SM throughput, so removing arithmetic barely shortens them. In VIVNet, the measured latency gain is 3.4–3.9%. In EVSSM, whose SS2D scans a single direction, NSI is 4–9% slower. We therefore state efficiency in MACs and report measured latency for each backbone separately; fusing propagation steps to reduce state traffic is the corresponding optimization.

Revision: Section 3.5 (p. 6) and Appendix A (pp. 14–17) for the theory; Appendix G (Table 24, p. 34) for the latency breakdown and EVSSM timing.


Thank you again for your constructive comments. We hope the revision addresses your concerns, and we would be glad to clarify anything further before the discussion period closes.

AuthorsOpenReview ↗

Your first question: NSI against a parameter-matched CSPN

Dear Reviewer iKJ7,

Your review lists as its first weakness that NSI "appears closer to a dynamic spatial propagation layer than to a new state-space formulation", and asks for stronger parameter-matched CSPN and recurrent dynamic-convolution baselines. Since our first reply, the clearest answer to this question has come from one control, which we summarize here.

Same topology, different state update. NSI and CSPN both update a 2D grid over several neighborhood steps. CSPN diffuses each channel separately with scalar affinities of unit total gain. NSI updates coupled channel pairs with input-conditioned 2×2 rotation–scaling transitions under a gain bound below one, which makes the update a contraction. A 2×2 rotation–scaling is multiplication by a complex number, the form of state update adopted in recent sequence SSMs such as Mamba-3 (ICLR 2026).

Matched control (Table 11, p. 24). We gave CSPN the same kernel generator as NSI (2.231 M vs. 2.248 M parameters; 24.1 vs. 25.6 GMACs) and removed NSI's retention and incremental readout, so the two differ only in the state transition.

Haze4K (1,000 images)O-HAZE (5 images)
NSI, identity retention + final-state readout29.92 dB26.26 dB
CSPN-style, NSI's generator27.65 dB23.93 dB
Difference+2.27 dB+2.32 dB

These runs use one seed; on Haze4K the margin is at least five times the seed standard deviation of either mixer (0.29 and 0.45 dB, Table 2). Our first reply attributed the difference to state-dependent retention; this control shows that the transition itself carries it, and the revised abstract and Contribution 1 now say so.

Recurrent dynamic convolution. Regenerating the kernels from the state at every step adds 24.5–32.9% MACs and changes PSNR by −0.44, −0.98 and +0.06 dB on O-HAZE, Haze4K and GoPro (Table 5(b)). A MAC-matched five-step involution is 2.49 dB below NSI on GoPro (Table 4).

Your other three questions are answered in the revision: EVSSM as a second backbone, +0.41 dB with 24% fewer MACs (Table 4); three seeds for every mixer on O-HAZE and Haze4K (Tables 1–2); and the scope of the contraction theorem and the memory-bound latency, stated in Section 3.5 and Appendix G.

If this comparison resolves the first concern in your review, we would be grateful if you would take it into account in your final recommendation. Thank you for your careful review.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of revisions

Update (Nov 17): Figure 1 is revised (routes traced on the input image, a sharper base map, panel (d)), and the abstract, introduction, contributions and conclusion now focus on deblurring and dehazing with the backbone and training budget fixed. Individual responses are organized by part, with exact revision locations.

We sincerely thank all three reviewers for their time and careful reading. Your comments were specific and constructive, and addressing them has substantially improved the paper. They converged on three questions: (i) how NSI differs from spatial propagation and dynamic convolution, (ii) whether its gains hold beyond our original setting, and (iii) what the stability and efficiency results establish. We have uploaded a revised PDF and supplementary material; the supplementary includes the evaluation records of every new result and a revision map.


Revision map – Where each concern is addressed in the revised PDF.

ConcernWhat we addedLocation
(ii) Short training budgetsGoPro 200K and Haze4K with VIVNet's full 300K schedule; eleven mixers on Haze4KTable 3 (p. 8); App. F, Tables 22–23 (pp. 33–34)
(i) Spatial propagation / dynamic convolutionInvolution-, CSPN-, NLSPN- and CSPN++-style mixers, per-step kernel regeneration, all at matched cost in the same blockTable 4 (p. 8), Table 5b (p. 9); App. H, Table 25 (p. 35)
(i) Design differencesDesign comparison with SPNs and scans; explicit state-space viewTable 19 (p. 31); App. J, Tables 27–28 (p. 37)
(ii) Other backbonesAll 48 SS2D mixers of EVSSM replaced by NSITable 4 (p. 8); App. E (p. 33)
(ii) Statistical reliabilityThree seeds for every mixer on GoPro, O-HAZE and Haze4K; NH-HAZETables 1–2 (pp. 7–8); App. K, Table 29 (p. 37)
(ii) Restoration-specific scansLocal4, RS8, EAMamba, C2SSM; per-module check of the MaIR and Spatial-Mamba transfersTables 22, 26 (pp. 34, 36); App. I (p. 35)
(i) Role of retention and readoutComponent controls on three datasets; fixed-weight interventions; ERF over images, positions and seedsTables 11, 16 (pp. 24, 29); Tables 14–15 (pp. 27–28)
PresentationFigure 1: routes traced on the input image, enlarged insets, panel (d) on why scans separate neighborsFig. 1 (p. 2)
(iii) Stability and efficiencyAlgorithm 1; contraction proof (Lean-checked); bound check on a trained model; latency breakdownAlg. 1 (p. 5); App. A (pp. 14–17), Table 17 (p. 30); App. G, Table 24 (p. 34)

Main result – NSI stays ahead under standard budgets.

As shown in Table R1, NSI stays ahead of SS2D on GoPro at 200K updates, and attains the highest PSNR and SSIM of all eleven mixers under VIVNet's full Haze4K schedule, including the restoration scans EAMamba and C2SSM, with the second-lowest MACs. Inside EVSSM, NSI gains +0.41 dB with 24% fewer MACs.

Table R1: Standard training budgets (seed 42; PSNR dB / SSIM; MACs at 256×256).

MixerGoPro, 200K updatesHaze4K, 300K updatesMACs (G)
SS2D31.23 / 0.918135.16 / 0.990282.1 / 36.7
EAMamba—35.32 / 0.9908— / 27.8
RS8—35.07 / 0.9900— / 51.7
CSPN-style—33.98 / 0.9904— / 24.3
Ours31.34 / 0.918935.51 / 0.992261.4 / 25.6

Scope and title – We narrowed the claim and revised the title accordingly.

For image restoration, selective state propagation in visual SSMs need not follow a scan order. NSI makes the recurrence run over propagation depth on the 2D grid instead of over a 1D token order, and selects from both the input and the evolving state. We study this design in image restoration, and to make this scope explicit we have revised the title to "Beyond Sequential Scanning: Rethinking Visual State Propagation with Neighborhood State Interaction for Image Restoration"; the abstract, introduction and contributions now name the two tasks and the controlled setting.


The individual responses (iKJ7: 2 parts; WkrM: 3 parts; DDai: 2 parts and follow-ups) address each question in turn. Thank you again for helping us improve this work; we would be glad to clarify anything further before the discussion period closes on November 18 (AoE).

AuthorsOpenReview ↗

Revision: positioning of the state transition (no new experiments)

We uploaded a revised PDF. It adds no experiments; it restates what distinguishes NSI, using results already in the paper.

Change 1 – Abstract and Contribution 1 lead with the state transition

NSI shares its spatial topology with spatial propagation networks such as CSPN: multi-step neighborhood updates on a 2D grid. The two differ in the state dynamics. CSPN diffuses each channel independently with scalar affinities of unit total gain. In NSI, each step is a state-space recurrence over coupled channel pairs: input-conditioned 2×2 rotation–scaling transitions, generated once and shared across steps, under a gain bound below one that makes the update a contraction. State-dependent retention and incremental readout adapt how accumulated context is carried and combined; they are not the main difference from CSPN. Rotation–scaling (complex-valued) state updates are also used in recent sequence SSMs (Mamba-3, ICLR 2026, arXiv:2603.15569), which we now cite.


Change 2 – Evidence for this reading, from Table 11

Table R1: NSI without retention and readout versus CSPN with NSI's own kernel generator (seed 42, Table 11, p. 24).

DatasetNSI, identity retention + final-state readoutCSPN-style, NSI's generatorDifference
O-HAZE26.26 dB23.93 dB+2.32 dB
Haze4K29.92 dB27.65 dB+2.27 dB

With the same kernel generator and with retention and readout removed, NSI's state transition alone stays 2.27–2.32 dB above CSPN's diffusion. Removing NSI's gain control costs 1.97–3.74 dB (Table 6), and a MAC-matched five-step involution is 2.49 dB below NSI (Table 4).


Change 3 – Correction in Appendix D.1

One sentence stated that the CSPN-style control is on par with NSI under VIVNet's full Haze4K schedule. Table 22 shows it 1.53 dB below NSI; the sentence now says so.

All other text, tables and numbers are unchanged. Thank you for your time throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.