acceptodds
Under review as a conference paper at ICLR 2027

Correcting State-Dependent Propagation Defects for Operator Learning

Abstract

Neural operators provide efficient surrogate models for the time evolution of parameterized partial differential equations (PDEs). Existing spectral neural operators reuse a shared response across input states, while nonlinear dynamics exhibit state-varying local responses. To address this structural discrepancy, we introduce State-Dependent Propagation Defect Correction (SDC), which enables an observable one-step propagation defect between a numerical reference map and a frozen neural propagator. We establish the connection between state-varying local responses and the propagation defect and show that standard prediction-space calibration is equivalent to regression of the observable defect. SDC-based operator learning is designed by a low-rank complex spectral update that adapts global mode-wise channel responses and a bias-free bottleneck that provides nonlinear state-conditioned correction from globally mixed hidden features. Both correction paths are zero-initialized and the pretrained propagator remains frozen during calibration, preserving the original propagation map at initialization. Experiments on time-dependent PDEs and weather forecasting demonstrate improved autoregressive prediction with minimal parameters and computational overhead.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operator, Nonlinear Dynamics, Partial Differential Equations, Defect Correction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FWd9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SDC, which augments frozen pretrained neural operators with low-rank spectral updates and nonlinear bottleneck modules to learn state-dependent prediction corrections through supervised training. The authors evaluate autoregressive forecasting performance on spherical shallow-water dynamics, WeatherBench weather forecasting, and planar Navier–Stokes flows. They also conduct state–correction pairing experiments to examine whether the learned corrections depend on the current input state.

Critical strengths
  • Experiments span multiple forecasting tasks. The method improves upon the corresponding backbone models on spherical shallow-water dynamics, WeatherBench weather forecasting, and planar Navier–Stokes flows. Results for WeatherBench and planar FNO are reported across three random seeds.

  • A comparison at similar computational cost supports the performance gains. In the shallow-water experiments, SDC reduces the average 5-hour and 10-hour forecast errors by 13.3% and 14.4%, respectively, relative to a GSNO trained for 60 epochs, with a comparable end-to-end runtime.

  • Diagnostic experiments support the input dependence of the learned corrections. On 500 test states, correctly pairing each state with its learned correction reduces the single-step error by approximately 34.7%. Shuffling these pairings increases the error above the backbone baseline, indicating that the corrections contain state-specific information.

Critical weaknesses
  1. Insufficient differentiation from LoRA and adapter tuning. The proposed method combines low-rank spectral updates with nonlinear bottleneck adapters. However, the paper lacks comparisons with standard LoRA and adapter tuning under matched parameter budgets, as well as separate ablations of the two correction branches. This makes it difficult to identify the added value of the proposed design.

  2. Insufficient validation of the motivation for state-dependent correction. FNOs are already nonlinear models with state-dependent local responses. Although the state-shuffling experiment demonstrates that the learned corrections depend on the input, it does not establish that the underlying errors primarily arise from shared spectral weights. Perturbation-response experiments or Jacobian analyses would help substantiate this interpretation.

  3. Insufficient evidence of long-horizon rollout stability. The experiments primarily evaluate 10–20-step rollouts, leaving error accumulation over longer horizons insufficiently examined. The theoretical bounds assume uniformly bounded one-step errors and still permit error amplification over time. The results therefore support improved accuracy within the evaluated horizons, while longer-term stability remains unverified.

Questions for authors

see Weaknesses.

4 comments

We sincerely thank the reviewer for the constructive feedback. We appreciate the reviewer's recognition of the broad experimental evaluation, favorable accuracy–efficiency trade-off, and diagnostic evidence supporting state-dependent corrections.


1. Differentiation from Matched LoRA/Adapter Controls & Branch Ablations (Weakness 1)

We thank the reviewer for this valuable suggestion. We agree that comparisons with parameter-matched adaptation methods and individual correction components are important for evaluating the contribution of SDC. We have evaluated independently retrained branch ablations (Spectral-Only, Eq. 64; Bottleneck-Only, Eq. 65) alongside two parameter-matched PEFT controls under the identical 50-epoch calibration schedule across three seeds:

  • Generic Residual Adapter: Matches SDC's exact trainable budget of 115,520 real scalar degrees of freedom (DOF).
  • Spectral LoRA: Applies a standard rank-4 complex low-rank update to the five native GSNO spectral kernels (114,688 real scalar DOF, within 0.72% of SDC).
Model / ControlTrainable Tensor Entries / Real Scalar DOF5 h MRE10 h MRE
GSNO-50 Reference (Table 1)797,537 / 797,5370.1492090.155611
Bottleneck-Only (Retrained ablation, Eq. 65)832 / 8320.147647 ±\pm 0.0083790.151505 ±\pm 0.006602
Spectral-Only (Retrained ablation, Eq. 64)57,344 / 114,6880.119134 ±\pm 0.0066320.120879 ±\pm 0.003407
Generic Residual Adapter (DOF-matched)— / 115,5200.130348 ±\pm 0.0009720.135999 ±\pm 0.001323
Spectral LoRA (0.72% DOF diff.)— / 114,6880.114420 ±\pm 0.0027260.118312 ±\pm 0.001815
Full SDC (Ours)58,176 / 115,5200.108345 ±\pm 0.0008780.112183 ±\pm 0.000755

Comparison with Matched PEFT Controls: Full SDC consistently achieves lower rollout error than both the DOF-matched generic residual adapter and the spectral LoRA at 5h and 10h.

Branch Contributions: Recalibrating the shared spectral weights (Spectral-Only) accounts for 73.6% of the total 5h MRE reduction and 80.0% of the 10h reduction. Combining the compact 832-DOF state-conditioned bottleneck with the spectral path provides the remaining 26.4% (5h) and 20.0% (10h) reduction (0.119134 →\to 0.108345 at 5h; 0.120879 →\to 0.112183 at 10h) and reduces cross-seed standard deviation by nearly an order of magnitude (±0.006632→±0.000878\pm 0.006632 \to \pm 0.000878).

The matched controls and branch ablations have been highlighted in Appendix A.13.

2. Validating the Motivation: Error Attribution, Perturbation, and Jacobian Analysis (Weakness 2)

We appreciate the reviewer's careful consideration of the theoretical motivation. As noted in Section 3.2 (Eq. 4) and Appendix A.4, the complete GSNO/FNO network is already nonlinear and state-dependent through its pointwise activations and bypass paths. Our statement concerns specifically the nonlocal spectral propagation weights (ϑ0\vartheta_0 in Eq. 3 and W0,oil(b)W_{0,oil}^{(b)} in Eq. 61), which are statically shared across input states, as clarified in Figure 1 and Section 3.2.

To further examine the mechanisms, we paired the retrained ablations above with hidden-state perturbation / directional-derivative diagnostics and post-training branch interventions on the trained checkpoint (with all parameters fixed):

Diagnostic / Inference Intervention SetupEvaluated MetricEmpirical Result
Frozen GSNO-50 ReferenceOne-step MRE (50 fixed test states)0.041465
Full SDC (Both branches active)One-step MRE (50 fixed test states)0.028816
Spectral Path Only (Bottleneck disabled at inference)One-step MRE (50 fixed test states)0.066268
Bottleneck Only (Spectral path disabled at inference)One-step MRE (50 fixed test states)0.058117
1% Hidden-State Perturbation (Directional derivative Jc(z)vJ_c(z)v)Relative change in correction output ∥δcψ(z)∥/∥cψ(z)∥\Vert \delta c_\psi(z) \Vert / \Vert c_\psi(z) \Vert14.65% (mean over 16 states)
  1. Shared Spectral Weights as the Dominant Source of Improvement: In the retrained ablations (Section 1), updating the statically shared spectral weights alone (Spectral-Only) captures 73.6%–80.0% of the total multi-step error reduction, whereas training the bottleneck alone without recalibrating the shared spectral weights yields minimal gain (0.149209 →\to 0.147647).
  2. Co-Adaptation vs. Independent Ablations: While Spectral-Only captures most of the gain when trained independently, disabling either branch at inference time after joint calibration increases one-step MRE (0.066268 and 0.058117) above the uncorrected GSNO-50 baseline (0.041465). This is expected: the retrained ablations measure what each path can learn alone, whereas the inference intervention shows that both paths co-adapt and operate jointly in the calibrated solution.
  3. Perturbation & Jacobian Verification of State Sensitivity: Applying a 1% relative perturbation to the hidden state zz before the first propagation block changes the fixed-parameter SDC correction output cψ(z)c_\psi(z) by 14.65% on average in relative norm (Eq. 12 & Eq. 65). Combined with the state-shuffling diagnostic (Table 2 & Appendix A.11), this confirms that the learned correction actively responds to the current state rather than acting as a fixed residual field.

The distinction and diagnostics have been included in Section 3.2 and Appendices A.13–A.14.


3. Extended Rollout Stability & Theoretical Bound Scope (Weakness 3)

We thank the reviewer for raising this important point regarding long-horizon stability. In Appendix A.9 (Eq. 81–91), the finite-horizon bound ek≤ϵψLk−1L−1e_k \le \epsilon_\psi \frac{L^k - 1}{L - 1} formalizes how reducing the one-step defect ϵψ\epsilon_\psi scales down the error envelope over finite horizons kk. We agree that when L>1L > 1, this bound permits error growth over time and does not claim unconditional asymptotic (k→∞k \to \infty) stability; the finite-horizon scope is stated in Appendix A.9.

To examine error accumulation well beyond 10–20 steps, we evaluated a 30-hour (30-step) closed-loop rollout across 50 fixed SWE test trajectories under 1-hour autoregressive feedback:

Model5 h MRE10 h MRE20 h MRE25 h MRE30 h MRE
GSNO-50 (Frozen Backbone)0.1308230.1762160.2498930.2933810.332341
Full SDC (Ours)0.1108800.1531730.2158100.2521710.283806

Both models remain stable through 30 autoregressive steps without entering a runaway regime, and Full SDC consistently maintains lower error across all horizons, reducing 30h MRE by 14.6% relative to the frozen GSNO-50 backbone. The extended rollout evaluation has been highlighted in Appendix A.16.


We sincerely thank the reviewer again for the constructive comments and valuable suggestions. We have incorporated the additional experiments and analyses into the revised manuscript, especially the parameter-matched adaptation comparisons, component-wise ablations, state-sensitivity diagnostics, and extended 30-hour rollout evaluation.

Thank you for your response and clarifications. While I consider the proposed method a reliable approach, my concern about its limited novelty remains, so I will maintain my original score.

We sincerely thank you for your further feedback and for recognizing the reliability of our approach. We understand your remaining concern regarding the novelty.

To further address your concern and build on the reviewers' constructive suggestions, we have further strengthened our work through additional operator-level theoretical analysis and new matched multi-step experiments. These findings clarify how SDC's complementary correction mechanisms interact with closed-loop optimization and demonstrate that its architectural advantage persists over generic residual and spectral-only adaptations under the same multi-step training objective. Importantly, these results further strengthen the methodological novelty of SDC by highlighting its structured, operator-aligned correction mechanism, whose benefits extend beyond generic residual adaptation and remain evident under closed-loop optimization.

For further details, please refer to our response to Reviewer fQSt, entitled "Further Clarification of SDC's Methodological Novelty and New Multi-Step Validation", as well as Sections 4 and Appendix A.19-20 of the revised manuscript, where the relevant theoretical analysis and experimental results have been incorporated and highlighted in red.

We hope these additional findings help address your concern and would greatly appreciate your reconsideration of the overall assessment.

Reviewer_fQStOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces State-dependent Propagation Defect Correction (SDC) to correct single-step prediction errors of pretrained neural operators. The method keeps the pretrained neural operator frozen and corrects the errors by a low-rank complex spectral update and a bottleneck that provides nonlinear state-conditioned correction. Several experiments on time-dependent PDEs, including a two-dimensional Navier–Stokes equation, and weather forecasting show that the method improves autoregressive prediction with minimal extra parameters and computational costs.

Critical strengths
  1. Reducing the errors of neural operators, while also controlling the errors by iteratively applying the operator in the long term has recently been a problem of focus to people from different backgrounds.
  2. The choice of the numerical examples, and the way the numerical results are reported are solid and look honest and understandable. Also, the numerical results show some promises of the current methodology.
Critical weaknesses
  1. The paper lacks the level of novelty needed to be published in a top conference, because essentially it is about efficient residual fitting using standard techniques, and then applying the residual fitting to dynamical systems problems. To be a more solid work, I would like to see the paper using more physical/dynamical properties (e.g. the equations/generators behind the dynamics), or enforcing some long-term accuracy/stability during the training, so that it is not just some work applying some residual fitting tricks to dynamical problems.
  2. The theoretical results are not organized in a very condensed manner, and there seems to be a gap between the theoretical results and the rest part of the paper. In particular, I do not see a convincing theoretical justification or an intuitive motivation/interpretation behind the correction techniques the authors use. Also, the paper lacks a comprehensive literature review on related literature trying to correct the neural operators, or some fair comparisons with these types of methods in the numerical results. For instance, by doing a simple search, I can find the paper F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning, which seems to be closely relevant to this work but not discussed by the authors. The authors are mainly comparing with neural operator themselves, which is not a fair comparison. Albeit the authors in the end try to permute the state assignments, it is still not enough to show advantages over other correction approaches.
Questions for authors
  1. Am I missing something about the motivation for the choice of techniques used for correction, in particular from the perspectives of dynamical systems? e.g. can you characterize what are the main error sources for those neural operators, and why your technique can help substantially in this regard?
  2. Can you clarify how each of the two individual correction approaches would contribute to the error reduction?
  3. Can you discuss more on for what type of dynamical system (e.g. depending on the source, type, and magnitude of nonlinearity behind the system), you expect your method works better than previous correction approaches, and when not? Even some intuitive descriptions would help, e.g. does weather forecasting represent a particular type pf dynamical systems that your method help more?
  4. Neural operators have some good properties, e.g. resolution invariance/transferability, symmetry, … After the correction, do those good properties still hold?
  5. Have you tried, in the same system, different step-sizes, and see how error change with step-sizes quantitatively?
  6. Have you considered incorporating some properties of dynamical systems to enforce long term stability?
  7. This may be asking too much, but have you considered using correction at multiple timescales and letting simulators with different timescales interact with each other?
  8. Have you considered what the remaining errors are mostly coming from after using your correction method?
9 comments

We sincerely thank the reviewer for the constructive feedback. We appreciate the reviewer's positive assessment of the experimental design and numerical results.


1. Dynamical properties, long-term accuracy/stability, and distinction from generic residual fitting (Weakness 1)

We appreciate the reviewer's concern regarding the dynamical interpretation and long-term behavior of SDC. Rather than applying an undifferentiated spatial residual trick, SDC decomposes the one-step propagation defect into two dynamics-motivated components inside the operator blocks (Section 4.3, Eq. 13–14; Appendix A.6, Eq. 63–69): a mode-wise spherical-harmonic transfer update (calibrating global modal dispersion/dissipation drift) and a state-conditioned hidden bottleneck (capturing localized nonlinear interactions).

  • Superiority over Generic Residual Fitting: Under strictly matched trainable degrees of freedom (115,520 real scalar DOF) and training budgets (50 epochs), SDC consistently outperforms a generic residual adapter (0.108345 vs. 0.130348 at 5h; 0.112183 vs. 0.135999 at 10h; see Table in Weakness 2).
  • Long-Term Closed-Loop Stability: Over an extended 30-hour (30-step) closed-loop rollout (detailed in Question 6), SDC remains stable and reduces 30h MRE by 14.6% relative to the frozen GSNO-50 backbone (0.283806 vs. 0.332341).

The correction design, matched controls, and extended rollout have been highlighted in Section 4.3 and Appendices A.13 and A.16.

2. Connection between theory and correction techniques, F-Adapter discussion, and fair comparisons (Weakness 2)

2.1 Theoretical Motivation and Correction Mechanisms. As established in Section 4.1–4.2, a nonlinear PDE generator Fdyn\mathcal{F}_ {dyn} induces trajectory-dependent local responses (Eq. 7–10), so the one-step defect A∗(x)=ΦΔt(x)−Mθ0(x)A^{\ast}(x) = \Phi_{\Delta t}(x) - M_{\theta_0}(x) (Eq. 11) exhibits a state-dependent tangent discrepancy DA∗[x]=DΦΔt[x]−DMθ0[x]DA^*[x] = D\Phi_{\Delta t}[x] - DM_{\theta_0}[x] (Eq. 12, derived in Appendix A.5, Eq. 52–53).

As noted in Appendix A.5, this discrepancy combines (i) a systematic mismatch in the shared nonlocal spectral response across states, and (ii) a state-varying local residual that depends on the current flow configuration.

Our parameterization in Section 4.3 (Eq. 13–14) directly mirrors this split: the low-rank complex spectral update (ΔWoil(b)\Delta W_{oil}^{(b)} in Eq. 64) recalibrates the shared mode-wise transfer response, while the nonlinear bottleneck (Hψ(b)(z)H_\psi^{(b)}(z) in Eq. 65) provides the state-varying response required by Eq. 12.

2.2 Discussion of F-Adapter and Related Literature. Thank you for pointing out F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning. The revision has discussed F-Adapter in Section 2 and distinguishes its scope from SDC in Appendix A.18. While both methods use parameter-efficient adaptation in transform coordinates, they differ in three core aspects:

  • Adaptation Objective: F-Adapter studies frequency-adaptive parameter allocation for pretrained scientific operators and can support downstream PDE transfer, whereas SDC formulates in-domain autoregressive defect correction around a frozen propagator to suppress multi-step rollout drift (Eq. 11–12 & Eq. 16).
  • Spectral Parameterization: F-Adapter partitions Cartesian Fourier modes into frequency bands and applies band-specific adapters inside attention/MLP layers. In contrast, SDC operates directly on the Driscoll–Healy spherical-harmonic contraction weights W0,oil(b)W_{0,oil}^{(b)} via a degree-wise complex low-rank update ΔWoil(b)\Delta W_{oil}^{(b)} (Appendix A.6, Eq. 61–64), preserving spherical geometry.
  • Explicit State Conditioning: Beyond static spectral weight updates, SDC couples its modal update with a zero-anchored state-conditioned bottleneck Hψ(b)(z)H_\psi^{(b)}(z) on the globally mixed spherical field (Eq. 65–69) to capture trajectory-specific local residuals.

Because F-Adapter is built specifically for planar DPOT backbones and its public release does not provide the checkpoints or spherical harmonic operators required for a matched SWE evaluation, we benchmark SDC directly against strictly DOF-matched spectral and residual PEFT controls on the same spherical GSNO backbone in Subsection 3 below.

2.3 Numerical Comparisons & Branch Ablations (Matched DOF & Budget). On the same frozen GSNO-50 checkpoint, SWE cache, and 50-epoch calibration protocol (3 seeds), we compared Full SDC against its individual branches and two parameter-matched PEFT controls—a Generic Residual Adapter (115,520 real DOF) and a Spectral LoRA (rank-4 complex update on the native GSNO spectral kernels; 114,688 real DOF, within 0.72% of SDC):

Correction Method / AblationTrainable Real DOF5 h MRE10 h MRE
Frozen GSNO-50 (Uncorrected Reference)797,537 (frozen)0.1492090.155611
Bottleneck-Only (State-conditioned branch, Eq. 65)8320.147647 ±\pm 0.0083790.151505 ±\pm 0.006602
Spectral-Only (Low-rank spectral branch, Eq. 64)114,6880.119134 ±\pm 0.0066320.120879 ±\pm 0.003407
Generic Residual Adapter (DOF-matched)115,5200.130348 ±\pm 0.0009720.135999 ±\pm 0.001323
Spectral LoRA (0.72% DOF diff.)114,6880.114420 ±\pm 0.0027260.118312 ±\pm 0.001815
Full SDC (Ours)115,5200.108345 ±\pm 0.0008780.112183 ±\pm 0.000755

Under matched parameter budgets and training protocols, Full SDC outperforms both the generic residual adapter and spectral LoRA at both horizons.

The matched controls and branch ablations have been highlighted in Appendix A.13.

3. Dynamical motivation for the correction techniques & main error sources in neural operators (Question 1)

From a PDE propagation perspective, the one-step error d(x)=y−Mθ0(x)d(x) = y - M_{\theta_0}(x) of a spectral neural operator stems from two primary sources:

  1. Shared Modal Propagation Error: In spectral operators, the nonlocal kernel weights W0,oil(b)W_{0,oil}^{(b)} (Appendix A.6, Eq. 61) are statically reused across all states, introducing systematic amplitude/phase (dispersion and dissipation) drift across global modes.
  2. State-Dependent Nonlinear Interaction Error: Local advection, steepening gradients, and wave–vortex interactions depend on the instantaneous state configuration, which a statically shared spectral kernel cannot adapt to dynamically.

The low-rank spectral update recalibrates the shared spherical-harmonic transfer weights across states, while the nonlinear bottleneck evaluates a state-conditioned correction from the globally mixed hidden state zˉ(b)\bar{z}^{(b)} (Eq. 63–69). As shown in our state-matching diagnostics (Section 5.3, Table 2 & Figure 4), the learned correction achieves a mean cosine alignment of 0.764 with the true one-step defect and removes 58.2% of the defect energy when paired with the correct state, whereas permuting state assignments raises error above the uncorrected backbone. The formulation and state-matching evidence appear in Sections 4.1–4.3 and 5.3, with additional diagnostics in Appendix A.11.


4. How each of the two individual correction approaches contributes to error reduction (Question 2)

We evaluated both independently retrained branch ablations (Table in Weakness 2 above) and post-training branch interventions (disabling one branch at inference time in a jointly calibrated checkpoint):

Inference Intervention on Jointly Calibrated CheckpointOne-Step MRE (50 fixed test states)
Frozen GSNO Reference0.041465
Full SDC (Both branches active)0.028816
Spectral path only (bottleneck disabled at inference)0.066268
Bottleneck only (spectral path disabled at inference)0.058117
  • Independent Retrained Contributions: In the retrained ablations (Weakness 2 Table), Spectral-Only accounts for 73.6% of the total 5h MRE reduction (and 80.0% at 10h) by correcting the shared modal response. While the 832-DOF Bottleneck-Only branch is close to the reference alone, combining it with the spectral update lowers 5h/10h MRE from 0.119134 / 0.120879 to 0.108345 / 0.112183 (capturing the remaining 26.4% / 20.0% reduction) and reduces cross-seed standard deviation by nearly an order of magnitude (±0.006632→±0.000878\pm 0.006632 \to \pm 0.000878).
  • Post-Training Co-Adaptation: Disabling either branch without retraining increases one-step MRE (0.066268 and 0.058117) above the uncorrected GSNO-50 baseline (0.041465), showing that the two paths co-adapt and operate jointly in the calibrated solution.

The two paths target complementary parts of the one-step defect: the spectral update corrects shared modal drift, while the hidden bottleneck captures state-dependent nonlinear residuals. This explains why disabling either path increases the one-step MRE and why their joint use performs best.

The ablations and inference interventions have been highlighted in Appendices A.13 and A.14.


5. Applicability and Limitations Across Dynamical Systems (Question 3)

We appreciate the reviewer's question regarding the types of dynamical systems for which SDC may be particularly effective and its potential limitations.

  • Expected Applicability: SDC is particularly relevant to autoregressive systems governed by advection and multiscale wave–flow interactions where the frozen operator captures the dominant large-scale background modes, but accumulates modal drift and localized nonlinear residuals over rollouts. Weather forecasting (WeatherBench, Section 5.4) and spherical shallow-water dynamics (SWE, Section 5.2) exemplify this regime: planetary Rossby/gravity waves provide a strong shared modal backbone, while localized synoptic systems and nonlinear advection create state-varying defects around that response. Planar Navier–Stokes (Appendix A.12) similarly benefits during vortex advection.
  • Potential Limitations: SDC may be less effective when the dominant error arises from (i) severe backbone collapse in extreme chaotic regimes (e.g., in fully developed extreme isotropic turbulence where the frozen reference operator fails to capture even single-step dominant structures, a compact low-rank correction cannot reconstruct the operator from scratch), (ii) missing stochastic forcing, or (iii) purely diffusive systems where physical viscosity rapidly damps errors on its own.

The evaluated examples and applicability boundaries are reported in Sections 5.2 and 5.4 and Appendices A.12 and A.17.

Reviewer_CW18OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a parameter-efficient defect-correction approach to fine-tune existing spectral neural operators with fixed global mode-wise responses, with the goal of mitigating state-dependent propagation errors. The proposed module has two components: a low-rank spectral update ΔR=1rUV\Delta R=\frac{1}{r}UV, whose parameters are learned during calibration but fixed during inference, and a nonlinear bottleneck HH, whose output depends on the current input state during both training and inference. The proposed module improves long-rollout performance on the Shallow Water Equations and WeatherBench over its GSNO backbone.

Critical strengths
  1. The problem is well-motivated and supported by theoretical analysis.

  2. The two-stage full-model training + lightweight fine-tuning formulation seems to provide an efficient training pipeline compared with training the full model further, with the training-time comparison provided by the authors.

  3. The proposed correction components have a reasonable division of roles, with ΔR\Delta R correcting the spectral response and HH providing the input-dependent correction.

Critical weaknesses
  1. There is no component-wise ablation study. In particular, there is no comparison between using only ΔR\Delta R, using only HH, and using both components.

  2. There is no experiment comparing backbone freezing with training the whole model together with the correction components in an end-to-end manner.

  3. The choice of training settings for the GSNO backbone is a bit difficult to understand. In particular, the authors compare a GSNO trained for 60 epochs with SDC trained using 50 backbone epochs + 50 correction epochs. While the authors match these settings based on total training time, it is still not clear to me how much of the gain comes from the correction components and how much comes from the different training procedure.

  4. There is no discussion or comparison with input-adaptive spectral operators. Similar ideas have already been explored in papers such as Adaptive Mamba Neural Operators (AMO), and I think this comparison is relevant if the main motivation is the limitation of globally fixed spectral responses.

Questions for authors
  1. Can the authors provide the component-wise ablation study mentioned in the Critical Weaknesses section?

  2. Likewise, can the authors provide experimental results from training the whole model together with the correction components in an end-to-end manner?

  3. Can the authors provide an epoch-matched comparison, for example, 100 epochs of full GSNO versus 50 epochs of GSNO + 50 epochs of correction training? I understand that the current comparison is matched by training time, but I think the epoch-matched result would also help clarify the effect of the different training procedures.

Optional

  1. If possible, can the authors provide an experimental comparison with AMO? If not, can the authors discuss how the proposed approach differs from AMO?
2 comments

We sincerely thank the reviewer for the constructive comments and for recognizing the motivation, theoretical analysis, and complementary design of the proposed correction components. We appreciate the suggestions for strengthening the experimental evaluation.


1. Component-Wise Ablation Study (Weakness 1 & Question 1)

Thank you for this valuable suggestion. We agree that evaluating the two correction components separately is important for understanding their individual contributions. We evaluated the Low-Rank Spectral path alone (Eq. 64), the State-Conditioned Bottleneck alone (Eq. 65), and the complete SDC module under the identical 50-epoch calibration protocol across three random seeds:

ModelTrainable Tensor Entries / Real Scalar DOF5 h MRE10 h MRE
GSNO-50 (Frozen Reference)797,537 / 797,5370.1492090.155611
Bottleneck-Only832 / 8320.147647 ±\pm 0.0083790.151505 ±\pm 0.006602
Spectral-Only57,344 / 114,6880.119134 ±\pm 0.0066320.120879 ±\pm 0.003407
Full SDC (Ours)58,176 / 115,5200.108345 ±\pm 0.0008780.112183 ±\pm 0.000755
  • Complementary Roles: The Spectral-Only path provides the large individual improvement. While the compact 832-DOF Bottleneck-Only path yields a modest gain in isolation when the global spectral bias remains uncorrected, combining both components further lowers 5h / 10h MRE from 0.119134 / 0.120879 to 0.108345 / 0.112183.
  • Variance Reduction: Coupling the state-conditioned bottleneck with the spectral update also reduces the cross-seed rollout standard deviation by nearly an order of magnitude (from ±0.006632\pm 0.006632 to ±0.000878\pm 0.000878 at 5h). These results support the complementary contributions of the spectral update and the state-conditioned bottleneck to prediction accuracy and consistency.

The ablation results have been highlighted in Revision (Appendix A.13).


2. Frozen Backbone vs. End-to-End Joint Training (Weakness 2 & Question 2)

We appreciate the reviewer's suggestion to examine end-to-end joint training. To investigate the role of backbone freezing, we additionally trained the GSNO backbone and SDC correction components jointly from initialization, allowing both sets of parameters to update throughout training:

SeedInitial Val MREEpoch 1 MREEpoch 2 MREEpoch 5 MRETarget RMS (Ep 1)Target RMS (Ep 5)Backbone Drift (Ep 5)
6670.064217.091.28×1031.28\times 10^32.19×1052.19\times 10^523.11.12×1081.12\times 10^80.315
6680.06428.648.26×1028.26\times 10^23.61×1063.61\times 10^612.19.54×1079.54\times 10^70.305
6690.06427.517.42×1027.42\times 10^25.41×1045.41\times 10^410.58.89×1078.89\times 10^70.307

Under the tested joint-training configuration, optimization became unstable across all three seeds. By Epoch 2, the defect-target cosine similarity drops to 0.955–0.963, while the target RMS grows by 1.5×1031.5\times 10^3 to 2.8×1032.8\times 10^3.

In joint training, the defect relative to the current backbone is dθt(x)=y−Mθt(x)d_{\theta_t}(x)=y-M_{\theta_t}(x), which varies as θt\theta_t changes. Freezing the pretrained backbone instead provides a fixed reference map and a stationary defect target throughout calibration. These observations highlight the practical stability of the frozen-reference formulation.

The joint-training diagnostic has been highlighted in Revision (Appendix A.13).

3. Epoch-Matched Training Schedules (Weakness 3 & Question 3)

Thank you for highlighting the importance of controlling the training schedule. In addition to the original wall-clock-matched comparison, we conducted the requested 100-epoch matched comparison under identical settings (128×256128\times 256 cache, seed 666, batch size, and learning rate):

Training ScheduleTotal Epochs5 h MRE10 h MRE
GSNO-100 (Full backbone training)1000.0859410.091317
GSNO-90 + SDC-10 (Equal 100-epoch match)1000.0818040.085268
GSNO-100 + SDC-50 (Post-hoc calibration on GSNO-100)100 + 500.0738730.075747

Equal 100-Epoch Comparison (GSNO + SDC vs. GSNO): At the exact same 100-epoch budget, reallocating the final 10 epochs to SDC calibration outperforms full 100-epoch GSNO training at both 5h and 10h while requiring slightly less training time (205.8s vs. 217.7s).

Post-Hoc Gain on a 100-Epoch Backbone (GSNO-100 + SDC-50): Even after the GSNO backbone has been trained for a full 100 epochs, adding SDC calibration further reduces 5h and 10h MRE by 14.0% and 17.1% (0.085941 / 0.091317 →\to 0.073873 / 0.075747). This demonstrates that the performance gain comes from the correction module resolving structural propagation defects rather than from training schedule differences.

The epoch-matched results have been highlighted in Revision (Appendix A.13).


4. Relation to Input-Adaptive Spectral Operators (Weakness 4 & Optional Question)

We thank the reviewer for highlighting input-adaptive spectral operator. We agree that AMO is relevant to the broader study of adaptive neural operator representations.

AMO modifies the core operator family using state-space sequence modeling so that the primary propagation map itself is input-adaptive. In contrast, SDC retains a pretrained, frozen spherical spectral operator backbone and learns a compact one-step defect correction. Thus, AMO develops an adaptive operator architecture, whereas SDC provides a efficient correction framework for pretrained propagators.

Regarding experimental comparison, AMO's official implementation is currently not publicly available. Reproducing AMO under our spherical SWE benchmark would additionally require adapting its architecture and training pipeline to the same spherical representation and autoregressive evaluation protocol. Nevertheless, extending SDC to input-adaptive spectral operators such as AMO represents an interesting direction for future work, allowing us to investigate whether defect correction can further improve their autoregressive prediction.

We have expanded the Related Work section of the revised manuscript to further discuss input-adaptive spectral operators, including AMO, and clarify their relationship to SDC (Section 2 and Appendix A.18).


We sincerely thank the reviewer again for the constructive feedback and valuable suggestions. We have incorporated the additional experiments and analyses into the revised manuscript (highlighted in red), including the component-wise ablation study, the investigation of end-to-end joint training, the epoch-matched comparisons, and the discussion of input-adaptive spectral operators.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Revisions

Dear Area Chair and Reviewers,

Thank you very much for your time. We sincerely thank the Area Chair and reviewers for their constructive feedback and thoughtful discussions. Below, we briefly summarize the main contributions and improvements. All revisions are highlighted in red.

1. Main Contributions

  • Operator-level formulation: SDC formulates adaptation around state-dependent propagation defects through complementary spectral and nonlinear correction pathways.
  • Performance and efficiency: Consistent improvements across spherical SWE, WeatherBench, and Navier–Stokes, supported by multi-seed evaluations and efficiency comparisons.
  • Mechanistic validation: Matched architectural controls, perturbation analyses, and branch interventions provide evidence for the proposed correction mechanisms.

2. Key Revisions

Following the reviewers' suggestions, we made several substantial revisions:

  • Theory and formulation: Clarified the distinction between shared spectral responses and the nonlinear full operator, and strengthened the propagation-defect formulation (Sections 3–4).
  • Architectural comparisons: Added three-seed ablations and parameter-matched generic residual and spectral-LoRA controls (Appendix A.13).
  • Training fairness: Added equal-epoch comparisons, frozen versus joint calibration, and experiments with a fully trained GSNO-100 backbone (Appendices A.10 and A.13).
  • Robustness and diagnostics: Added perturbation and branch-intervention analyses, cross-resolution evaluations, and 30-hour rollouts (Appendices A.14–A.16).
  • Related work and limitations: Expanded discussions of F-Adapter, AMO, physical constraints, time-step dependence, and simulator interactions (Section 2; Appendices A.17–A.18).

We appreciate Reviewer CW18's increased rating and acknowledgment that the concerns were addressed, as well as the constructive follow-up comments from Reviewers fQSt and FWd9. We understand that the novelty remains a concern.

3. Additional Evidence Addressing the Novelty

Motivated by the reviewers' suggestions, we further investigated SDC's operator-level correction structure and its extension to closed-loop multi-step optimization.

  • Structured correction mechanism: SDC jointly modifies native spectral propagation and state-conditioned hidden responses. Matched controls support the benefit of these complementary correction directions beyond generic residual adaptation (Sections 4.2–4.3; Appendix A.13).

  • Multi-step theoretical analysis: We derived how future trajectory errors jointly update both correction pathways through full BPTT, connecting SDC's structured tangent space to closed-loop optimization (Section 4.4; Appendix A.19).

  • Multi-step refinement experiments: Extending SDC to 3-, 5-, and 7-step optimization improved long-horizon forecasting. Seven-step BPTT reduced 30-hour MRE from 0.287286 to 0.273430, relative to the historical single-step continuation (Appendix A.20, Table 13).

  • Matched multi-step comparisons: Under the same seven-step optimization objective, full SDC achieved a 30-hour MRE of 0.275089, outperforming generic residual adaptation (0.295932) and spectral-only adaptation (0.288244) (Appendix A.20, Table 14).

These findings provide further evidence that SDC's methodological contribution lies in its complementary operator-level correction directions, which remain effective under closed-loop optimization, rather than in multi-step training alone.


We sincerely appreciate the reviewers' suggestions, which have helped strengthen our theoretical analysis and experimental validation. We hope these revisions and additional findings provide a clearer basis for subsequent discussion and evaluation.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.