acceptodds
Withdrawn from ICLR 2027

Integral-Form Refinement Suppresses Error Accumulation in Neural Operator Rollout

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural operators (NOs) forecast PDE trajectories rapidly, but in autoregressive rollout each window becomes the next input and errors compound until the forecast is unusable. We refine each predicted window against the governing equation before feeding it forward, with the NO frozen. The residual is the integral (Duhamel) form of the equation, or the discrete balance of a numerical scheme where no suitable linear part exists. This lowers rollout error by 40–72% on Burgers, 83% on turbulent Navier–Stokes and 33–61% on the highly nonlinear degenerate Richards equation of variably saturated flow. The improvement is substantial for the tested NOs, and refinement largely equalizes them, though our primitive NO keeps an edge at small correction budgets. More importantly, the refinement stops rollout error from compounding on non-chaotic systems. The uncorrected Richards error grows 2.7-fold over a 38-day forecast, whereas the corrected error settles three to thirteen times lower. In turbulent flow, where chaos forbids any bound, the error still grows but stays 56 times lower. The integral-form residual is critical: at the coarse frame spacing of NOs, both the standard pointwise residual and a one-step scheme residual are dominated by their own time-discretization error, and refining against them worsens the rollout or diverges. On Burgers, our training-free correction also outperforms learned refiners, which need per-system training. Furthermore, the benefit can be anticipated from the residual’s own error, measured once per system. This simple screen correctly identifies when refinement helps. Lastly, the correction can be strategically enhanced: semi-Lagrangian transport lowers Navier–Stokes error from 0.050 to 0.0083 in 17 ms per trajectory, below what any residual-minimizing refiner can reach. Together, these results show NO rollout errors to be measurable, predictable and suppressible, and establish test-time refinement as a practical tool for long forecasts.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neural operators, Physics-based refinement, Autoregressive rollout, Integral residual, Error-targeted correction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Mq7eOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how to reduce roll out errors for neural operators with window refinement. In the so-called window refinement, the authors propose to minimize ∥∣R(h)∥∣2\||\mathcal{R}(h)\||^2 where R(h)\mathcal{R}(h) denotes the integral residual obtained by solving ∂tu=Lu+N(u)\partial_tu=Lu+N(u) with an exponential integrator. However, the paper is full of AI slop and barely readable. It lacks substantial technical details needed to justify its contribution.

Critical strengths
  • Reducing roll out errors for neural operators is an important problem to study.
Critical weaknesses
  • The paper does not provide a clear definition of the research problem. Section 3.1 sketches a problem setting, which is supposed to lead to a problem definition, but the authors jump to an evolution equation that is split into linear and nonlinear parts.
  • The paper does not clarify the proposed algorithm. Section 3.2 simply describes how to solve the split equation with an exponential integrator. It is not clear how this could improve rollout errors of neural operators.
  • The rationale of this paper is unclear. If one can afford the computational cost of using exponential integrators and the governing equations are given, why use neural operators in the first place?
Questions for authors

N/A

Reviewer_ZQYhOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes test-time refinement of neural-operator rollout windows using the integral/mild form of the governing PDE. Each predicted window is optimized to reduce a physics residual and then fed into the next rollout step. For systems where the residual is insensitive to dominant errors, the authors introduce problem-specific projected corrections, including column-pressure corrections for Richards flow and semi-Lagrangian transport for Navier–Stokes.

Critical strengths
  1. Addresses the practically important problem of autoregressive error accumulation without retraining the neural operator.
  2. Corrected feedback is appropriately separated from independent window postprocessing.
  3. Evaluation covers several PDEs, distribution shifts, multiple backbones, runtime measurements, and some classical-solver comparisons.
  4. The paper is transparent about computational cost and cases where refinement fails.
Critical weaknesses
  1. The residual-floor criterion is heuristic. It requires reference trajectories both to measure the floor and to estimate the forecast error, and no independent validation of the thresholds.
  2. The method can be expensive: generic refinement costs approximately 20–130 neural-operator forward passes on Burgers. Accuracy–runtime Pareto curves against well-tuned classical solvers would be more informative than selected operating points.
Questions for authors
  1. How can the floor diagnostic be used when reference trajectories are unavailable
  2. Were residual weights and correction subspaces selected on validation data, and how sensitive are results to these choices
Reviewer_m81GOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses autoregressive error accumulation in neural operator (NO) rollouts through training-free, test-time window refinement. With the NO weights frozen, each predicted multi-frame window is adjusted in state space to reduce the governing PDE residual before being fed forward as the input to the next window (corrected feedback). The authors' core contributions are: (1) formulating the refinement residual in the integral (Duhamel/mild) form—or the discrete mass balance of an implicit scheme when no suitable linear part exists—so that linear dynamics are propagated exactly across coarse frame intervals where pointwise finite differences and one-step scheme residuals fail; (2) introducing low-wavenumber spectral weighting and problem-specific subspace/transport corrections to address dominant error modes (such as phase shifts or column pressure offsets) to which raw residuals are weakly sensitive; and (3) defining a pre-measurable diagnostic, the residual floor FF (the discrete residual evaluated on true trajectories), which predicts whether refinement will improve (F/E<0.8F/E < 0.8) or degrade (F/E>1.2F/E > 1.2) a forecast of error EE. The approach is validated across six PDE systems (including 1D/2D Burgers, variable-coefficient advection–diffusion, the nonlinear Richards equation, and 2D turbulent Navier–Stokes) and three NO backbones (FNO, Transolver++, and a physics-augmented "primitive NO"), alongside comparisons against learned refiners (PDE-Refiner, Metamizer), one-step correction (PhysicsCorrect), and GPU-matched classical solvers.

Critical strengths
  1. Principled resolution of the coarse frame-spacing bottleneck in test-time refinement (with a predictive diagnostic): Prior test-time physics corrections rely either on pointwise time derivatives (PINO) or single-step numerical schemes (PhysicsCorrect). This paper convincingly demonstrates—both analytically and empirically (Section 4.3, Tables 8–9)—that when NO predictions jump across multiple solver steps (Δt≫δtsolver\Delta t \gg \delta t_{\text{solver}}), the temporal truncation error of pointwise and one-step residuals exceeds the network's forecast error and causes refinement to degrade or diverge. Formulating the residual in Duhamel integral form eliminates linear time-stepping truncation error, and the proposed residual floor ratio (F/EF/E, Section 4.4) provides a practical, network-agnostic criterion for predicting when test-time refinement helps versus hurts. This is consequential because it explains when and why equation-based test-time compute works in realistic operator-learning regimes.
  2. Rigorous empirical controls and error-propagation analysis: Following McGreivy & Hakim (2024), the authors benchmark runtime and accuracy against converged classical GPU solvers (Appendix G), reproduce prior baselines (PhysicsCorrect) to machine precision before ablating frame spacing, and validate their turbulent Navier–Stokes pipeline on a newly generated cohort after freezing all hyperparameters (Appendix J). Furthermore, their autoregressive recurrence analysis (en+1=ρen+ϵe_{n+1} = \rho e_n + \epsilon, Table 13) cleanly shows that corrected feedback stops error compounding on non-chaotic systems by contracting inherited error (ρ<1\rho < 1).
Critical weaknesses
  1. Generic integral-residual minimization is slow and fails on turbulent flow; practical performance relies on bespoke, PDE-specific solvers: Minimizing the integral residual via L-BFGS or Adam is computationally expensive (20–130×20\text{--}130\times the cost of an uncorrected rollout at batch 8, Table 1) and fails on 2D turbulent Navier–Stokes (where Picard iteration diverges and 100 Adam steps only reduce error from 4.97%4.97\% to 3.94%3.94\% in 271 ms271\text{ ms}, Table 5). To achieve competitive speed and accuracy, the authors must inspect reference trajectories to identify each PDE's dominant error mode and hand-craft a problem-specific numerical solver step (a 9-mode Fourier Newton step for Burgers, a vertical-line + column-offset solver for Richards, and semi-Lagrangian vorticity advection for Navier–Stokes). This is consequential because it significantly limits the generality of the method as a plug-and-play test-time wrapper.
  2. The headline turbulent Navier–Stokes improvement (83%83\% error reduction / 56×56\times lower final error) comes almost entirely from a classical solver rather than NO refinement: On turbulent Navier–Stokes, the proposed "transport correction" discards the NO's predicted vorticity field and instead integrates the advection–diffusion equation from the incoming state using classical semi-Lagrangian transport along the NO-predicted velocity field (Section 4.2, Appendix E). Buried in Appendix G (lines 1170–1175), the authors report that running the exact same classical semi-Lagrangian transport stepper without any neural operator achieves nearly identical accuracy and runtime (0.00866 at 16.63 ms classical vs. 0.00833 at 16.86 ms with FNO over 3 windows; 0.00223 at 176.93 ms vs. 0.00212 at 163.07 ms over 9 windows). Omitting this control from Table 5 and the main text overstates the contribution of the neural operator and of residual refinement on the paper's most challenging benchmark.
Questions for authors
  1. Clarification (Navier–Stokes classical control): In Appendix G (lines 1170–1175), a purely classical semi-Lagrangian stepper without the NO achieves 0.00866 in 16.63 ms (3 windows) and 0.00223 in 176.93 ms (9 windows), matching your 3-pass and finer transport-corrected NO results within ~4–5% in both error and wall-clock time. Given that the transport step reconstructs vorticity from w0w_0 rather than minimizing the integral residual R(h)\mathcal{R}(h), will you include this classical semi-Lagrangian baseline directly in Table 5 and clarify in the abstract/main text that on turbulent Navier–Stokes, the NO provides only a modest velocity warm-start over a standalone classical semi-Lagrangian solver at matched cost?
  2. Clarification (Selecting targeted corrections without reference trajectories): The fast targeted corrections (Section 3.3 / 4.5) require knowing the dominant error subspace PP (e.g., low-kk modes, column pressure offsets, or Lagrangian transport), which was identified by comparing NO forecasts against reference trajectories. How would a practitioner systematically construct PP on a new PDE without first analyzing ground-truth rollout errors, or can PP be extracted automatically (e.g., via SVD/POD of single-window training errors)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.