acceptodds
Withdrawn from ICLR 2027

DRIFT OR DYNAMICS: RESIDUAL CORRECTION OF LONG-HORIZON NEURAL PDE ROLLOUTS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learned simulators offer a way to make expensive physical simulation much faster, with potential applications across science and engineering, from fluid and material dynamics to weather and climate modeling. But using a learned model to predict far into the future remains difficult: each prediction becomes the input to the next, so small errors can accumulate until the simulation no longer resembles the true system. Existing approaches improve long-horizon behavior by training models to make longer sequences of predictions, repeatedly refining their outputs, or adding mechanisms that pull predictions back toward physically or statistically plausible states. These strategies can extend the useful prediction horizon, but they leave an important question unanswered: what should we do when a prediction moves away from the states seen during training? Pulling it back seems natural—but a physical system may be genuinely evolving into a state the model has never seen before. We introduce a geometry-aware correction that pulls back only the part of a prediction outside the principal directions of the training data, leaving the in-subspace component untouched. In controlled experiments, this targeted correction reduces long-horizon error by up to 90%. Importantly, we find that the geometry of the correction matters, and when it removes meaningful dynamics, error can instead increase by up to 130×. Our results show that long-horizon stability is not simply a matter of keeping predictions close to the training data. The critical question is whether a departure from the data is an error or a part of the dynamics we are trying to predict.

Primary area:
learning on time series and dynamical systems
Keywords:
Scientific Machine Learning, Neural PDE Solvers, Long-Horizon Prediction, Neural Operators, Dynamical Systems, Surrogate Models, Geometric Stabilization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_EeCaOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors use PCA on the training data to find the main directions in which the ground truth states vary. They then split each predicted state into two parts: one inside the PCA subspace and one outside of it. If the part outside of the subspace is too large, they shrink it. The idea behind their method is to reduce errors that build up during long rollouts. However, the authors also show that this correction can make predictions much worse if the removed part contains important dynamics. They test the method on several PDE datasets and show when the correction is useful, does nothing, or hurts the prediction.

Critical strengths
  • The problem of misaligned autoregressive rollouts is a difficult and important problem. I appreciate the authors investing time, and effort into this topic.

  • The biggest strength for me is that the authors also show where their method fails. They do not only report improvements, but also cases where the correction makes the predictions much worse. I think this is useful, especially since it shows that moving predictions closer to the training data is not always a good idea.

  • I also appreciate the ablations. The authors test different correction strengths, different PCA subspaces, and different types of errors. These experiments help explain why the correction works in some cases and fails in others.

Critical weaknesses
  1. The paper is very difficult to read/follow The ideas is simple, but the paper makes it hard to follow. It appears like large parts have been written by AI, and there was no human-review afterwards. As a reader, I repeatedly had to look ahead for explanations, guess what certain symbols mean (as some are explained after their introduction or not at all), or search in the appendix to find out what has been evaluated. Some of these issues can be understood from context, but a reader should not have to guess what the authors mean. I think the paper needs a major rewrite. This is not just a matter of fixing a few sentences or adding some definitions. The method and experiments need to be explained in a much clearer order.

  2. The comparison with existing correction methods In the paper, the authors did a good job in their ablation of the method itself, but I am not entirely convinced that the paper properly establishes what is new compared to previous work. There are already several methods that correct or stabilize predictions during PDE rollouts. Some appear closely related to the presented method. For example:

  • Li et al. (2022), Learning Dissipative Dynamics in Chaotic Systems, already worked on how to stabilize learned dynamics by applying a correction when predictions leave a certain region.
  • Oommen et al. (2024), Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling, used diffusion models for corrections of neural operators.
  • McGreivy and Hakim (2023), Invariant Preservation in Machine Learned PDE Solvers via Error Correction, correct neural PDE predictions using physical constraints.
  • Wells et al. (2015), An Evolve-Then-Filter Regularized Reduced Order Model, showed filtering numerical solutions using POD-based methods. This seems particularly relevant since the proposed method also uses a PCA/POD subspace. I wonder where in this the proposed method fits it, and more important, maybe a direct comparison to similar methods.
Questions for authors
  1. What is the main novelty compared to the existing stabilization/corrections methods? In particular, how does the proposed method differ from dissipative stabilization methods such as Li et al. (2022) and POD-based filtering methods such as Wells et al. (2015)?
  2. Why are there no direct comparisons against simpler correction methods? For example, how does the proposed method compare against standard PCA/POD projection or a simple dissipative correction when applied?
  3. Why do 14 of the 15 APEBench benchmark rows not use the proposed method?
Reviewer_id8EOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a test-time “residual anchor” for neural PDE rollouts. A PCA subspace (W) is estimated from training states, and each predicted state is decomposed into an in-subspace component and an orthogonal residual. When the residual exceeds a calibrated threshold, it is partially contracted while the in-subspace component is unchanged. This correction helps when the residual is rollout drift but can catastrophically harm predictions when it contains valid dynamics.

Critical strengths
  1. Includes both positive and strongly negative results instead of reporting only favorable tasks.
  2. Evaluates very long rollouts, including 100,000-step Kolmogorov experiments, multiple PDE families, SGNO, and APEBench models.
  3. The method requires no additional neural-network evaluation, and Proposition 1 correctly guarantees contraction of the PCA residual.
  4. Clearly distinguishes distributional drift from genuine unseen dynamics.
Critical weaknesses
  1. Proposition 1 only proves that the PCA residual decreases, not that physical or rollout error decreases. The mathematical result is straightforward but provides no dynamical stability guarantee.
  2. The correction can be extremely harmful: up to (130\times) in 2D KS and a 222% degradation when composed with SGNO on 2D Burgers. This makes an unavailable prospective gate essential.
Questions for authors
  1. How sensitive are results to normalization, PCA rank, the 99.95% variance threshold, and spatial resolution?
  2. What are the memory and runtime costs of storing and applying (W) at realistic simulation resolutions.
Reviewer_kSVQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a projection method to correct rollouts predicted by a neural surrogate. It relies on precomputing PCA directions from the training set, then projecting generated trajectories using directions of maximum variance. The method works for some systems, and not others; in general, the effectiveness seems to be ad-hoc and not convincing.

Critical strengths
  • The method seems to be explained well. The motivation is also clear.
Critical weaknesses
  • The primary weakness is that the method doesn’t seem to work for a lot of systems (such as for KS). The projection damps out features that are actually present in the PDE, causing autoregressive drift and error accumulation. The results support this almost overwhelmingly (most systems have decreased performance in Figure 3, and always using the projection is more regret than never using it).
    • Fitting a filter to adjust this also cannot be constructed a priori, since this requires access to a ground truth during inference.
    • The paper also mentions that using unrolled training has much more of an impact than the proposed projection
  • The abstract is therefore rather misleading, claiming that the method achieves up to 90% error reduction (which is technically true for one case), but in majority of cases the method does not work.
  • In general, precomputing the projection based on a training set also relies on having access to a training set that is similar to the test set, which is generally not the case for many applications we are interested in.
Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.