acceptodds
Under review as a conference paper at ICLR 2027

Do Neural PDE Solvers Learn the Right Dynamics?

Abstract

Neural PDE solvers can achieve low prediction errors, but do they reproduce the dynamics of the systems they model? Prediction scores alone offer an incomplete answer: they measure agreement with reference solutions but provide limited insight into how errors accumulate, nearby states diverge, or extreme events arise. We propose an evaluation framework that directly examines these behaviors in deterministic and stochastic neural solvers. By evolving ensembles of nearby initial states and comparing them with direct numerical simulation, we assess three complementary aspects of learned dynamics: error formation, ensemble geometry, and extreme events. Experiments on two-dimensional Kolmogorov flow reveal limitations that conventional scores can obscure. Smaller trajectory errors can reflect weaker error amplification despite less accurate local updates. Models can match an ensemble's overall spread and effective dimension while failing to capture the spatial directions where nearby states diverge. Similarly, matching overall event frequencies can conceal failures to predict persistent extreme events. These findings show that improved prediction accuracy does not necessarily imply greater dynamical fidelity. Our framework makes this distinction measurable, providing concrete criteria for evaluating whether advances in neural PDE solvers better capture the underlying dynamics.

Primary area:
learning on time series and dynamical systems
Keywords:
neural PDE solvers, model evaluation, dynamical systems, ensemble forecast

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Zfv6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents a suite of different evaluation methods for neural PDE surrogates, based on generating perturbed initial condition ensembles with a trained surrogate and having access to a DNS solver. There are several conclusions: firstly, the step-wise residual error does not usually correlate with rollout error, second, matching the ensemble spread is generally not enough, and lastly, rare events are generally under or over-predicted by surrogates even with low bias.

Critical strengths
  • The evaluation seems to be thorough (plenty of surrogate benchmarks). Access to a DNS solver allows for some interesting experiments

  • The error decomposition is interesting, and can highlight where some of the rollout error comes from.

Critical weaknesses
  • My main concern is that the paper is written somewhat poorly and the main figures are hard to follow. This makes it difficult to understand the main points of the paper. For example:

    • H1/ H4 doesn’t seem to be explained in Figure 2 or before line 191, when it is introduced.
    • I’m also assuming the reference state x_k comes from the surrogate? (if so, reference state usually means ground truth which would be a bit confusing)
    • What are the triangles/circles in Figure 3? These don’t seem to be labeled.
    • I would recommend using variables to denote plotted/measured quantities (instead of “conditional spread” or “dynamics-solver subspace overlap”). I was able to find these quantities in the text, but using variables would be more clear as to what metric is plotted.
  • Somewhat of a costly metric (needs access to DNS to generate a ground truth ensemble). This probably isn’t feasible for unstructured solvers, or for high Reynolds numbers / more complex fluids. Therefore, it is unlikely that similar analyses can be done for other systems (for example weather/climate?) and these takeaways may be restricted to Kolmogorov Flow.

Questions for authors
  • I am curious about the interpretation of the first experiment (Figure 2). Unet H1 has better rollout error but more residual error, but then the next plot for FNO shows that higher residual errors correspond to failed members. If this interpretation is correct, does it mean that residual error isn’t indicative of rollout stability? These seem to contradict each other.
5 comments
AuthorsOpenReview ↗

Response to Reviewer Zfv6 (Part 1/3)

We thank Reviewer Zfv6\color{blue}{\textbf{Reviewer Zfv6}} for the thoughtful review and for finding our decomposition and analysis interesting. Below, we respond to the presentation concern, clarify the interpretation of error formation experiments, and discuss the computational cost of the metrics.

1. The presentation concerns (W1)

Following the reviewer's suggestion, we have carefully clarified definitions and notation in revised paper, and have updated the figure captions and legends for better clarity.

To directly answer the reviewer's specific concerns:

  • Definitions of H1/H4 (Figure 2): We have explicitly defined H1 and H4 in the caption of Figure 2. HiH_i indicates that the training input uses the past ii frames. While originally defined in Section 2 ("Solvers and Forecast Evaluation"), repeating these definitions at the point of specific elaboration is indeed necessary for clarity.

  • "Ground Truth" vs. "Reference": The reference state xkx_k is computed via direct numerical simulation (DNS). Unlike weather forecasting, where a true physical system exists alongside its numerical approximation, our task directly learns from numerical simulations. Therefore, following the convention in the neural solver literature, we directly refer to the DNS results as the "ground truth" or "reference."

  • Clarifying the markers in Section 3.2 and Figure 3c: The red triangles in the original Figure 3c represent the noise-trained DPOT. The original Figure 3c aimed to highlight one key finding: while the noise-trained DPOT with lower rollout prediction error compresses the ensemble cloud during propagation, this compression does not bring the ensemble cloud closer to the ground truth than DNS. The detailed definitions are in the Section 3.2. With clearer definitions, we now convey this finding more effectively through text and notation. Consequently, we have replaced Figure 3c to align with the broader discussion of neural solver performance in Section 3.2.

  • Notation Optimization: We have optimized our notation and use variables as clear indicators. These adjustments cover Sections 3.1, 3.2, and Section 4.

AuthorsOpenReview ↗

Response to Reviewer Zfv6 (Part 2/3)

2. Clarify the "contradiction" about local prediction error (Q1)

We thank the reviewer for this insightful question. We realize that our original presentation lacked clarity, making these two observations appear contradictory. In fact, they are entirely consistent and together reveal a critical flaw in evaluating models solely by rollout prediction error.

For clarity, we define the rollout prediction error as ek=x^k−xke_k = \hat{x}_k - x_k and the local prediction error as rk=x^k−F(x^k−1)r_k = \hat{x}_k - F(\hat{x}_{k-1}) (formerly termed the "residual"). All notation here and below follows revised Section 3.1. The cumulative local prediction error is J=∑i=1k∥ri∥2.J = \sum_{i=1}^{k} \lVert r_i \rVert^2.

Our core motivation is simple: a good solver should first and foremost have good local accuracy, which is why we choose to focus on it explicitly.

  • The U-Net H1 vs. H4 case demonstrates how a lower ∥ek∥2\|e_k\|^2 can obscure a larger cumulative local prediction error JJ.
  • The FNO case demonstrates the harm of this obscuring effect. Before blow-up happens, the blown-up FNO members have ∥ek∥2\|e_k\|^2 almost identical to stable members. However, their local prediction error is consistently and significantly higher. A low ∥ek∥2\|e_k\|^2 here masks a severely flawed local model, making it a precursor to hidden failure modes.

In summary, these two cases jointly motivate our explicit focus on the local prediction error (rkr_k). A low rollout prediction error (∥ek∥2\|e_k\|^2) can obscure a large rkr_k (U-Net H1 vs. H4) and mask a fragile state that precedes hidden failure modes (FNO).

To explain this mechanism, we have added Figure R1 in Section 3.1 and a detailed discussion in the revised manuscript. As shown in Figure R1b, the propagation of existing errors by the ground-truth dynamics (GG) is highly complex and fluctuates in sign. Its negative short-term contribution can mask the severe local prediction error (JJ) (Figure R1c), leading to a deceptively low ∥ek∥2\|e_k\|^2 (Figure R1a). This demonstrates that the error formation process is far more complex than simple autoregressive amplification.

AuthorsOpenReview ↗

Response to Reviewer Zfv6 (Part 3/3)

3. Discussion on computational cost and Clarification of our purpose (W2)

We thank the reviewer for raising this practical and important concern. We agree that the ensemble evaluation is computationally demanding. We would like to clarify two aspects: (1) the varying computational costs of our proposed metrics and their feasibility, and (2) our core purpose, which is to reveal certain problems with neural PDE solver based on physical observation rather than a universal benchmark.

3.1. Computational cost and feasibility

We acknowledge the computational cost and breakdown the cost different metrics:

  • Low-cost component (Section 3.1): The error decomposition (g,j,cg, j, c) only requires single trajectory. It can be computed by launching a short numerical simulation at a selected model state x^k\hat{x}_k to obtain F(x^k)F(\hat{x}_k), making it extendable to other systems.
  • High-cost components (Sections 3.2, 4, 5): The ensemble error formation, subspace geometry, and extreme event analysis indeed require a full numerical simulation ensemble.

Regarding feasibility:

  • For solvers and simulation methods: The paradigm applies to any neural solver trained on numerical simulation data. Provided that the underlying simulation code is accessible, that specific simulation solver can directly serve the exact same role as DNS in our framework.
  • For systems: We agree that for high-Reynolds-number turbulence, complex fluids, or weather/climate models, the cost of generating sufficient DNS ensembles will increase drastically, limiting the direct application of the ensemble metrics.

3.2 Core purpose: pointing out unphysical behaviors in neural solvers

We emphasize that the goal of this work is not to position these ensemble metrics as universal benchmarks. Instead, the ensemble acts as a dense observational tool to reveal physical inconsistencies that standard rollout errors mask.

Our motivation stemmed from several physical intuitions that puzzled us when considering neural solver works. We provide the DPOT example:

  • Fluid dynamics is a deterministic chaotic system. The mapping from xkx_k to xk+1x_{k+1} is deterministic, and due to positive Lyapunov exponents, ensemble of xk+1x_{k+1} should expand compared to xkx_k.
  • However, DPOT's denoising training maps these perturbed states to the same clean target, despite them being supposed to map to distinct states. This physically corresponds to a contraction of the ensemble, contrary to the true dynamical expansion.
  • Intuitively, this means its mapping is relatively more contractive than the true dynamics.
  • Despite this physically contradictory behavior, DPOT achieves very good rollout prediction error (∥ek∥2\|e_k\|^2) across broad cases.

Ensemble evaluation is a appropriate and necessary tool to reveal this hidden contradiction. Indeed, we find that this simultaneous occurrence of lower error and greater contraction is widespread across other neural solvers(Section 3.2).

This prompted us to investigate two follow-up questions: the origin and the consequences of this contraction design.

Regarding its origin, the common intuition is that autoregressive models suffer from iterative error amplification. Motivated by this, we further decomposed the error to understand error formation mechanism.

Regarding its consequences, we realized that this artificial contraction might suppress the natural dynamical amplification of the chaotic system. Consequently, we deliberately evaluated false-negative cases in extreme event forecasts to uncover this impact. These inquiries collectively form the core motivation of our work.

In summary, we appreciate the reviewer's concern and have clarified the cost breakdown and limitations in the paper. We hope the reviewer agrees that while computationally expensive and restricted on some fronts, the ensemble perspective provides unique physical insights that single-trajectory metrics fundamentally cannot offer. While we have not yet conducted broader tests, we believe that, based on the logic of our physical reasoning, these findings are not specific to the Kolmogorov flow but reflect general behaviors of deterministic chaotic dynamics.

Reviewer_oy6HOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper illustrates a series of subtle failure modes that occur in PDE surrogate models and suggests that current evaluation metrics are insufficient. This includes both local dynamical properties and statistical properties. They also evaluate a broad array of recent architectures and training methods.

Critical strengths

This is a nice, clear exposition of a significant issue with evaluation in the field. It provides a path forward for better evaluations of PDE surrogate models.

Critical weaknesses

The actual results, especially in section 3, don't read as comprehensive and instead look like a set of cherry-picked examples of failures. While this is still useful, the authors should consider providing a broader analysis across the entire set of configurations/architectures that they tested. Broader trends should also be discussed based on this analysis. The authors should also consider testing and discussing additional alternative training losses/methods (e.g., https://openreview.net/forum?id=zIHEctNlcP, https://openreview.net/forum?id=Llh6CinTiy).

Questions for authors
  1. How do different architectures, training losses, and deterministic vs. stochastic models behave relative to each other? Do they all essentially exhibit the same failure modes, or is there a clear trend where one type benefits over another?
  2. More specifically, how do we evaluate deterministic vs. stochastic models differently? Are stochastic models even well-defined using the current training methods, and what theoretical distribution are they actually learning?
  3. You only test on a single chaotic system. It would be nice to see more diversity to see if the trends you identify hold broadly. Also, how do the failure modes you identify change as you scale data and compute?
  4. Does your study suggest a more comprehensive approach to evaluation? You currently provide some brief discussion, but it might be nice to recommend perhaps one metric per class of failure mode to include in future model evals?
  5. In Figure 5, I would like to see a DNS reference that shows the mismatch from purely different initial states. How much mismatch is due to noise/chaos? More generally, can you provide reasonable error bars in all your plots/tables?
8 comments
AuthorsOpenReview ↗

Response to Reviewer oy6H (Part 1/4)

We thank Reviewer oy6H\color{green}{\textbf{Reviewer oy6H}} for the comprehensive and insightful feedback. We address the boarder analysis, the assessment of stochastic models, metric recommendation, the member-wise scaling experiment and other issues below.

1. Providing broader analysis across all configurations in Section 3 (W1)

We thank the reviewer for this constructive feedback. We agree that our original presentation may have appeared focused on specific examples rather than a comprehensive analysis. To address this, we have expanded our analysis across all tested architectures and configurations, both in Section 3.1 and 3.2.

In Section 3.1, we have added Figure R1, which provides a comprehensive breakdown of the GG, JJ, and$C$ percentages across all tested models. The key takeaway is that the effect of GG, the dynamical propagation of existing error, is highly complex, often fluctuating in sign and inducing significant error reduction. This reveals that the commonly assumed mechanism of autoregressive iterative amplification is insufficient to fully explain the error dynamics.

We used the U-Net H1/H4 and FNO cases as entry points driven by our core premise: a good solver should first and foremost have good local accuracy.

  • The H1/H4 case reveals how lower ∥ek∥2\|e_k\|^2 can obscure larger JJ due to small or negative GG and CC.
  • The FNO case demonstrates that this obscuring effect can be harmful for long-horizon stability.

By dissecting these failures, we obtained a more general understanding of the error formation mechanism: the propagation of ground-truth dynamics GG can strongly contract existing errors, masking severe local inaccuracies.

In Section 3.2, we have updated Figure 3c–d and added a comprehensive discussion.

  • This figure maps all tested solvers and cases based on their relative ensemble size change (horizontal axis DD) and their relative deviation from the true dynamics (vertical axis TT).
  • Ensemble contraction is a widespread phenomenon.
  • As the rollout progresses, the distribution concentrates along the diagonal, suggesting that approaching the ground-truth ensemble is consistently accompanied by stronger ensemble contraction.

These broader results confirm that our case studies are more than isolated examples; they provide effective entry points to uncover a deeper mechanism that generalizes across diverse neural solver architectures.

AuthorsOpenReview ↗

Response to Reviewer oy6H (Part 2/4)

2. Assessment of stochastic models (Q1 & Q2)

We address them by clarifying our experimental settings for stochastic models, discussing whether such models are well-defined for deterministic systems, and providing additional comparative experiments.

2.1 Experimental setting for stochastic models

ACDM and PDE-Refiner have diffusion-style noise sampling and denoising. Given xtx_t, repeated inputs yield a distribution rather than a deterministic output, so we call them stochastic models.

For a given initial condition at a specific time step, all ensemble members share the same noise, while different initial conditions and time steps use independently sampled noise (detailed in Appendix C).

This setting does not fully preserve the generative randomness and is not the standard practice of the original papers. However, it is supported by their codebases and is crucial for observing a clean ensemble evolution, rather than turning off noise entirely or applying the same noise field across the whole rollout.

2.2. Are stochastic models well-defined for these tasks?

Both models were originally developed for deterministic chaotic systems, aligning with our test systems.

  • PDE-Refiner targets 1D KS equations and 2D Kolmogorov flow (Re=1000). The original paper explicitly notes that the underlying PDE solutions are deterministic; noise is introduced for better accuracy on low-amplitude frequencies, not to model physical stochastic drivers.
  • ACDM tests on periodic cylinder wakes (Inc), highly chaotic transonic cylinder flows (Tra), and 2D slices from 3D isotropic turbulence (Iso). The Iso task is a key exception: 2D slices do not form a complete dynamical state. Unobserved 3D motions affect the future, making the problem severely under-informed and justifying conditional distribution sampling.

We argue that using randomness to capture the distribution of chaotic behavior is scientifically incorrect but engineering-wise valuable. We believe neural solver design should lean towards extremes: either pursue scientific accuracy (faithfully reflecting physics) or engineering efficiency (giving sufficiently usable results in practical scenarios).

2.3 Comparative experiments: shared vs. independent noise

We further conducted experiments comparing shared noise versus independent noise across ensemble members. The results and visualizations are in Appendix H.

  • The impact of independent noise is consistent across both solvers. For error formation, the proportion and trends of GG, JJ, and CC are minimally affected. For geometry, ensemble spread and effective rank significantly increase and deviate from DNS, while subspace overlap decreases. For extreme events, both excess and deficit significantly increase.
  • Discussion: A key motivation for introducing noise in these original models is to add more details to a single trajectory via generative design. Our ensemble comparison demonstrates that this introduces an ensemble distortion problem.
AuthorsOpenReview ↗

Response to Reviewer oy6H (Part 3/4)

3. Recommended metrics and motivation statement (Q4)

We thank the reviewer for this constructive suggestion. Below, we summarize our recommendations, their feasibility, and the physical motivations that drove this evaluation framework.

3.1 Recommended metrics

  • Error formation: We recommend tracking the squared local prediction error jk+1=∥rk+1∥2=∥x^k+1−F(x^k)∥2j_{k+1} = \|r_{k+1}\|^2= \|\hat{x}_{k+1} - F(\hat{x}_k)\|^2, aligning with our premise that a good solver should first and foremost have good local accuracy. It only requires a single trajectory, making it feasible and extendable to other systems(provided the numerical simulation solver as F(x^k)F(\hat{x}_k) is accessible).

  • Ensemble geometry: We recommend evaluating the subspace geometry overlap, which provides a high-density observation of the dynamics intermediate between an instantaneous Jacobian and standard point-wise metrics. As shown in Figure 5, coarse DNS and resampling references yield high overlap, confirming its stability, while a different-cases reference yields low overlap, confirming its discriminability across initial conditions. However, it requires a full simulation ensemble.

  • Extreme events: The definition of extreme events is highly system-dependent. Therefore, we refrain from recommending a universal metric for this class, though our false-negative analysis highlights its importance.

3.2 Motivation: a problem-driven framework

We emphasize that this evaluation suite is problem-driven and is not intended to be a universal benchmark. Instead, the ensemble acts as a dense observational tool to reveal physical inconsistencies that standard rollout errors mask.

Our motivation stemmed from physical reasoning about neural solver behaviors. For example, consider DPOT:

  • Fluid dynamics is a deterministic chaotic system with positive Lyapunov exponents. The ensemble of xk+1x_{k+1} should naturally expand compared to xkx_k, mapping perturbed states to distinct future states.
  • However, DPOT's denoising training maps perturbed states to the same clean target. This physically corresponds to a contraction of the ensemble, fundamentally contradicting the true dynamical expansion.
  • Despite this, DPOT achieves a very low rollout prediction error (∥ek∥2\|e_k\|^2) across broad cases.

Ensemble evaluation is a necessary tool to reveal this hidden contradiction. This prompted us to investigate the artificial contradiction design's origin ( autoregressive error amplification bias, probed via error decomposition) and its consequences (suppressed dynamical expansion, probed via false-negative extreme event forecasts).

In summary, this evaluation framework was formed by following physical motivations to point out hidden problems in neural solvers. While computationally restricted, we believe it provides unique physical insights that standard metrics cannot offer.

Reviewer_oMxGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper uses ensembles of perturbed initial conditions to study "error formation", "ensemble geometry", and "extreme events" of neural PDE solvers.

Critical strengths

The paper touches on some important problems, the empirical results are believable, and some implications, particularly the second, are reasonable.

Critical weaknesses

The presentation is poor--this makes reading the paper (and spotting its strengths) difficult. I encourage the authors to spend more time on the presentation. In particular, several key terms are never explicitly defined in the main paper (e.g., "dynamical fidelity", "later-failing", "non-crossing", etc.), others are used differently from the literature (e.g., "residual" to mean the single-step error, "trajectory error" to mean error at forecast step k instead of time-aggregated error, etc.), and in general, terms should be defined before or at first use. Also, there are several notation issues: include F in x_k+1^hat = F(x_k^hat) so that r_k is not misread to be 0, disambiguate the overloaded terms r_k, r_90, r, R_3, R_k, etc.

Major issues with the experiments/main findings: Section 3.1's findings are mostly well-known including by cited works (PDE-Refiner, APEBench, etc.). Experiments in Sections 4 and 5 are not convincingly shown to be specific to neural solvers: by step 80 the solvers’ relative L2 errors are 0.32–1.11 (Table 4), so low overlap at steps 50–90 may reflect decorrelated trajectories. Experiments to produce Figure 5 should include a coarse numerical solver with comparable rollout error, and same for experiments to produce Table 1.

Questions for authors

Can you repeat experiments to produce Figure 5 with a coarse numerical solver with comparable rollout error, and same for experiments to produce Table 1?

5 comments
AuthorsOpenReview ↗

Response to Reviewer oMxG (Part 1/3)

We thank Reviewer oMxG\color{orange}{\textbf{Reviewer oMxG}} for the thoughtful feedback. Below, we address these points: presentation concerns, coarse numerical references, justification for the geometry overlap, and the findings on error formation.

1. The presentation concerns (W1) We have carefully clarified all definitions and notation, ensuring that no undefined terms remain in the main text. Following the reviewer's suggestion, we have adopted more general terminology for key terms and revised the related sentences to make the definitions clearer. Additionally, we have updated the figure captions and legends for better clarity.

2. Coarse numerical references (Question 1) Following the reviewer's suggestion, we performed coarse numerical simulations using the exact same numerical scheme as the original resolution, initialized from the same initial conditions as the other neural solvers. The coarse DNS settings and comparison protocol are detailed in Appendix C (Coarse DNS Resolution Controls).

The coarse DNS experiments exhibit low relative L2L_2 errors to 1282128^2 origin DNS field, defined as ∥x^−x∥2/∥x∥2\|\hat{x} - x\|_2/\|x\|_2. As summarized in Appendix Table 4, the errors are approximately two orders of magnitude lower than those of the neural solvers.

ResolutionStep 20Step 80
64²0.00480.0101
96²0.00060.0075

We have added coarse DNS references for geometry overlap to the top two rows of Figure 5, and for extreme event forecasting to the first two columns of Table 1. The coarse DNS references exhibit significantly higher geometry overlap than the neural solvers. The excess and deficit in the coarse DNS forecasts are significantly lower than those in the neural solvers' predictions.

AuthorsOpenReview ↗

Response to Reviewer oMxG (Part 2/3)

3. Justification for the geometry overlap (W3)

The reviewer raises a valid concern: when rollout prediction error is large, a low geometry overlap could simply reflect decorrelated trajectories rather than a failure in matching the ensemble geometry directions. We address this from three perspectives: definition of centered ensemble, experimental controls, and underlying motivation.

3.1 The overlap is computed on the centered ensemble.

Let Z\mathcal{Z} denote the centering operator (subtracting the ensemble mean), so Zxk\mathcal{Z}\mathbf{x}_k is the centered ensemble. Using the ensemble-averaged norm ∣A∣M2=1MD∑m=1M∑i=1DAm,i2\\|A\\|_M^2 = \frac{1}{MD}\sum_{m=1}^{M}\sum_{i=1}^{D} A_{m,i}^2 (averaging over MM members and DD grid points), we decompose the error at step kk into three components:

  • Original error: RMSEraw2=∥x^k−xk∥M2\mathrm{RMSE}_{\mathrm{raw}}^2 = \|\hat{\mathbf{x}}_k - \mathbf{x}_k\|_M^2
  • Ensemble mean error: RMSEmean2=∥x^‾k−x‾k∥M2\mathrm{RMSE}_{\mathrm{mean}}^2 = \|\overline{\hat{\mathbf{x}}}_k - \overline{\mathbf{x}}_k\|_M^2
  • Centered error: RMSEcentered2=∥Zx^k−Zxk∥M2\mathrm{RMSE}_{\mathrm{centered}}^2 = \|\mathcal{Z}\hat{\mathbf{x}}_k - \mathcal{Z}\mathbf{x}_k\|_M^2

These satisfy exactly: RMSEraw2=RMSEmean2+RMSEcentered2\mathrm{RMSE}_{\mathrm{raw}}^2 = \mathrm{RMSE}_{\mathrm{mean}}^2 + \mathrm{RMSE}_{\mathrm{centered}}^2, where x‾k\overline{\mathbf{x}}_k is the ensemble mean field. The median values across test cases at step 80 are:

SolverOriginal RMSEEnsemble mean RMSECentered RMSE
DNS 64²0.018200.014460.00867
DNS 96²0.012880.011070.00635
U-Net H41.59491.53060.3737
U-Net H4 P41.28561.16870.3647
U-Net H4 K81.40041.32820.2935
U-Net H11.20441.09540.3026
DySLIM H1 K101.64681.60450.3841
AFNO H42.01891.89080.6129
AFNO H4 K21.37351.19580.3581
FNO H10 K100.61370.58880.2277
DPOT H101.65341.59730.3924
DPOT H10 P41.74971.64100.3775
PDE-Refiner H11.12311.05670.2810
ACDM H21.78031.68370.3797

After removing the ensemble mean, the centered RMSE is significantly smaller than the original RMSE for all solvers. This indicates that the rollout prediction error is largely dominated by the mean bias. Therefore, the low overlap is a genuine reflection of misaligned geometry directions, rather than an artifact of trajectory decorrelation.

3.2 Control experiments confirm that the geometry overlap is a stable and discriminative metric.

We appreciate the reviewer's inspiring suggestion regarding the coarse DNS reference. Beyond this, we have added two additional references in Figure 5: a resampling reference (randomly drawing 256 members from the 1024-member pool 2000 times) and a different cases reference (overlap between ensembles from different initial conditions).

  • The coarse DNS reference yields high overlap, demonstrating stability across different numerical solvers.
  • The resampling reference also yields high overlap, showing robustness for ensembles of nearby trajectories.
  • The different cases reference yields significantly low overlap, confirming clear discrimination across different initial conditions.

These controls prove that the geometry overlap is a stable and discriminative metric.

3.3 Motivation: Treating the ensemble as a high-density observation of the dynamical system.

Compared to single-trajectory metrics that only ask "is this trajectory correct?", we aim to ask a deeper question: how does the dynamics deform a cluster of nearby trajectories in the surrounding region?

Our metric represents an observation resolution that is intermediate between the Jacobian matrix and standard point-wise metrics:

  • Conventional evaluations treat trajectories as isolated samples, providing a sparse observation of the dynamics. Our ensemble metric provides a higher-density observation.
  • Compared to the Jacobian matrix propagated along a single trajectory (which captures instantaneous local linearization), our ensemble spans a finite neighborhood and evolves nonlinearly over time. While offering a coarser local spatial resolution than the Jacobian, it captures the cumulative, long-term temporal effects of the dynamics.
AuthorsOpenReview ↗

Response to Reviewer oMxG (Part 3/3)

4. Clarify the findings on error formation (W2)

Our decomposition isolates and observes a previously underexplored mechanism in error formation. While the community widely acknowledges that errors propagate and grow in autoregressive rollouts, we explicitly extract the one-step propagation of existing error by the ground-truth dynamics FF (implenmented by DNS). Through this decomposition, we demonstrate that the ground-truth dynamics can strongly contract existing errors. This contraction directly explains why a model with low rollout prediction error ∥ek∥2\|e_k\|^2 can simultaneously have significantly high local prediction error (i.e., one-step prediction error). This mechanism directly answers our naive motivation: a good solver should first and foremost have good local accuracy.

Prior works have indeed established related error-growth phenomena, but they focus on fundamentally different aspects or mechanisms:

  • Lippe et al. [1] (PDE-Refiner) note that "the error propagates over time" and discuss how high-frequency errors propagate into lower frequencies via the nonlinear term in the KS equation. McCabe et al. [2] (Towards Stability of Autoregressive Neural Operators) state that autoregressive stepping "can lead to uncontrolled error growth over time and eventual instability." Schiff et al. [3] (DySLIM) connect this to "positive Lyapunov exponents, which exponentially amplify errors in the learned dynamics." These works characterize the overall rollout behavior of errors, emphasizing amplification and instability. Our finding complements this by isolating the propagation itself at both single-step and cumulative level, revealing that it can also induce strong contraction.

  • Prior work has analyzed error propagation through local linearizations of learned and reference dynamics. Ainslie et al. [4] propagate existing errors through the learned-map Jacobian evaluated along the true trajectory. Bangera and Gupta [5] additionally construct a counterfactual that propagates the same measured one-step defects through the true solver Jacobian. Our decomposition differs by directly evaluating the full nonlinear reference evolution from the predicted state, F(x^k)−F(xk)F(\hat{x}_k)-F(x_k), rather than relying on a first-order Jacobian approximation along the true trajectory. This yields an exact finite-error decomposition at both single-step and cumulative levels, revealing that the reference dynamics can strongly contract existing errors, not only amplify.

  • APEBench [6] uses the one-step DNS update from the predicted state as a training objective, which aligns with our use of F(x^k)F(\hat{x}_k) in the decomposition. ANCHOR [7] alternates DNS and neural solvers, noting that DNS "consistently rectifies accumulated errors." However, neither work explicitly discusses the specific mechanism of how this single-step DNS update contributes to error formation or propagation. Our work extracts and quantifies this specific contribution.

We have revised Section 3.1 to better present our findings and revised Appendix B to include these discussions and separate our mechanism findings from the broader observations of error growth.

We also recognize that the previous presentation of Section 3.2 may have hindered the understanding of our findings. We have revised this section accordingly and encourage the reviewer to read it, as it provides a more comprehensive picture of the error formation process.


[1] Lippe et al. (2023), “PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers,” NeurIPS 2023.

[2] McCabe et al. (2023), “Towards Stability of Autoregressive Neural Operators,” TMLR 2023.

[3] Schiff et al. (2024), “DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems,” ICML 2024

[4] Ainslie et al. (2026), “Eigenanalysis Framework for Autoregressive Neural Emulators of Multi-scale Chaotic Dynamics,” arXiv:2608.16084.

[5] Bangera and Gupta (2026), “When Does Jacobian Fine-Tuning Stabilize Neural PDE Emulators?,” arXiv:2610.42013.

[6] Koehler et al. (2024), “APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs,” NeurIPS 2024.

[7] Roy et al. (2026), “ANCHOR: Error-Controlled Adaptive Numerical Correction for Neural Operator Time Marching,” arXiv:2512.19643.

Comments on the paper

2 comments
AuthorsOpenReview ↗

To All Reviewers: Updated Manuscript and Appendix

We thank the reviewers for their thoughtful feedback. We have revised the manuscript and appendix in response to the comments. The main changes are summarized below.

  1. @Reviewer oMxG\color{orange}{\textbf{Reviewer oMxG}}, Reviewer Zfv6\color{blue}{\textbf{Reviewer Zfv6}} and Reviewer oy6H\color{green}{\textbf{Reviewer oy6H}}: Ensemble error formation. Revised presentation and clearly presented the definitions and findings in Section 3.2. Changed Figure 3c.
  2. @Reviewer oMxG\color{orange}{\textbf{Reviewer oMxG}} and Reviewer Zfv6\color{blue}{\textbf{Reviewer Zfv6}}: Clarifying the definitions. Explicitly defined key variables and used them to replace unclear wording in Section 3.1, Section 4, Appendix E and Appendix G.
  3. @Reviewer oMxG\color{orange}{\textbf{Reviewer oMxG}} and @Reviewer Zfv6\color{blue}{\textbf{Reviewer Zfv6}}: Figure readability. Revised the legend and caption of Figure 2-4.
  4. @Reviewer oMxG\color{orange}{\textbf{Reviewer oMxG}}: Coarse DNS reference. Added coarse DNS experiments reference in Fig.5, Table.1 and Table.4.
  5. @Reviewer oMxG\color{orange}{\textbf{Reviewer oMxG}}: Distinct findings on error formation. Added related work analysis in Appendix B.
  6. @Reviewer oy6H\color{green}{\textbf{Reviewer oy6H}}: Boarder analysis across entire set. Added Figure.R1 for overall behavior of error formation. Added analysis beyond failure modes in the last paragraph of Section 3.1.
  7. @Reviewer oy6H\color{green}{\textbf{Reviewer oy6H}}: Assessment of stochastic models. Performed experiment regarding stochastic models. Details in Appendix H.
  8. @Reviewer oy6H\color{green}{\textbf{Reviewer oy6H}}: Reference experiments of geometry overlap. Added resampling and intial condition control experiments in Figure 5.
  9. @Reviewer oy6H\color{green}{\textbf{Reviewer oy6H}}: Member-wise scaling experiment. Added experiments, the settings and results are in Appendix I.

We appreciate the reviewers’ suggestions, which helped us present our findings more accurate and clear.

AuthorsOpenReview ↗

To All Reviewers: The Second Updated Manuscript and Appendix

We thank the reviewers for their insightful feedback. We have further revised the manuscript and appendix in response to the comments. The main changes are summarized below.

  1. @Reviewer Zfv6\color{blue}{\textbf{Reviewer Zfv6}}: Spread-skill ration and ensemble distance. Motivated by your insightful comment, we have made a preliminary attempt to apply ensemble forecasting concepts from the weather prediction domain to our setting, with details added to Appendix J.
  2. @Reviewer oy6H\color{green}{\textbf{Reviewer oy6H}} and @Reviewer oMxG\color{orange}{\textbf{Reviewer oMxG}}: Clarifying the Additional Information Provided by Geometry Overlap. Analyzed situations with similar errors but distinct overlaps, and added Figure 5d with updated captions, along with two sentences of discussion at the end of Section 4.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.