acceptodds
Under review as a conference paper at ICLR 2027

Predicting Unseen Dynamics with Neural Emulators

Abstract

Can neural emulators predict dynamical regimes that are absent from their training data? Our main results concern two-dimensional Kolmogorov flow: an emulator trained only on chaotic aspect ratios predicts laminar dynamics beyond an -bifurcation, provided the architecture does not carry a symmetry the equations lack. A standard convolution is equivariant to every -shift, an invariance the forced equations do not possess, and drifts unphysically along that unconstrained direction; an -aware convolution, equivariant only to the discrete -shifts the shift-reflect symmetry permits, does not. As supporting multi-system evidence, a Kuramoto–Sivashinsky emulator recovers excluded relaminarisation, warm-up, and kink transitions and extrapolates to domains twice as large as those seen in training, while a partially observed beta-plane emulator recovers coalescence, nucleation, and stationary four-jet states excluded from training. Our failures are as informative as our successes: the chaotic transient approaching the Kolmogorov attractor is not recovered, and the learned map retains a dependence on the emulator timestep that a converged evolution map would not have. Both point to what matters being whether the target is represented locally in the training measure, at the scale the architecture sees, rather than whether the whole state has been seen before.

Primary area:
learning on time series and dynamical systems
Keywords:
Machine Learning, Fluid Dynamics, Dynamical Systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_22DXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is junk and makes no meaningful contribution.

Critical strengths

None

Critical weaknesses
  1. The paper does not have a clear explanation of the research problem it aims to address.
  2. The paper is not written in a way that human can understand.
Questions for authors

Have the authors read the paper before submitting it?

3 comments

We thank the reviewer for their time. We would welcome specific points on which the research question or the presentation was unclear, and would be glad to clarify them. A summary of the revisions made in response to the other reviews is posted above.

The introduction has only one sentence that vaguely states the goal of the paper: “Throughout, we study recovery of regimes deliberately excluded from training rather than interpolation across parameter values.” It is not clear what “recovery of regimes” means.

The introduction begins by discussing “predicting events that are unseen in the training set” and calls it a “less common” ML application. This is not true. There are a lot of papers (including some of those cited in section 2) in the scientific ML literature that address generalization to unseen data.

Generalization in neural emulators is a very broad topic. The literature addresses generalization across time, space, initial conditions, boundary conditions, system parameters, and other dimensions. To make a clear and meaningful contribution, the authors need to specify:

  • Which types of generalization does this paper address?
  • What metric is used to define success/failure?

Without a clear problem definition, the paper jumps into model architecture and experimental settings too quickly, making it read like a technical report. The conclusions drawn from the experiments, particularly those on lines 439, 443, 461, and 467, do not offer a useful takeaway. What can the audience learn from these conclusions?

The authors should clearly identify the central contribution and design experiments around it: is it a new method, a new explanation, or a new insight? That being said, these concerns are not simply writing problems that can be fixed quickly. The work requires substantial improvement in conceptualization and experimental design. I maintain my original score.

We thank the reviewer for these specific comments. The paper addresses one type of generalisation: recovering dynamical regimes that are deliberately excluded from the training data, as opposed to interpolating between parameter values or initial conditions seen in training. Each experiment defines the excluded regime explicitly (laminar states beyond a bifurcation for Kolmogorov flow, kink transitions and larger domains for Kuramoto–Sivashinsky, and coalescence, nucleation and four-jet states for the beta-plane), and success is judged against DNS of the same regime. We agree that this definition, and the criteria for success and failure, should be stated more prominently in the introduction, and that the central takeaway should be drawn out more explicitly. We will take these points on board in the revised manuscript.

Reviewer_vQyPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses an important question in data-driven PDE modeling: can neural emulators trained on a restricted dynamical regime generalize to previously unseen regimes? Here, unseen regimes refer not merely to new initial conditions or parameter interpolation, but to qualitatively different dynamical behaviors, e.g., involving new equilibrium structures or transitions across bifurcations. Such changes may arise either through state evolution at fixed parameters or through variations in the governing parameters.

The authors investigate this question using three systems: 2D Kolmogorov flow, KS equation, and beta-plane. Although the experimental settings and architectures differ, all three experiments restrict the training data to selected dynamical regimes and evaluate generalization beyond them. Examples include predicting laminar dynamics from chaotic training data, extrapolating to larger spatial domains, and recovering unseen regimes under partial observations.

The results demonstrate that neural emulators can reproduce certain unseen dynamical regimes, while highlighting the importance of appropriate inductive biases. A particularly instructive example is the Kolmogorov flow: here, the governing equations possess only a discrete translation symmetry, whereas standard CNNs impose a stronger, full translation equivariance, leading to unphysical predictions. Introducing a symmetry-aware convolution substantially improves the results. This provides a concrete demonstration that incorrectly imposed symmetries or inductive biases can be detrimental to generalization. The authors also carefully discuss other failure cases, including timestep sensitivity and unsuccessful recovery in certain regimes.

To explain the observed generalization, the authors conjecture that local coverage may be the key factor. Even when a test regime is globally unseen, its constituent local structures may already be represented in the training data, allowing the emulator to generalize through their recombination. However, this explanation remains a conjecture rather than a theoretically established result, and may not fully account for all experiments.

Critical strengths

S1. I found the paper interesting and generally well-motivated. Predicting unseen dynamical regimes from restricted training data is an important challenge in scientific modeling, as it provides a meaningful test of whether a learned model captures the underlying dynamical and topological structures rather than merely interpolating observed trajectories. This paper systematically investigates this challenge across complex PDE systems using neural emulators.


S2. The three systems considered exhibit rich dynamical behaviors, including chaos, regime transitions, and bifurcations. Systematically investigating model generalization across these systems provides valuable insights that may extend beyond the specific systems and model architectures studied.

Critical weaknesses

W1. The main limitation is that the mechanisms enabling generalization to unseen dynamical regimes remain largely unexplained theoretically. In particular, the proposed local-coverage hypothesis is not quantitatively established or supported by a theoretical framework. Nevertheless, I recognize that this is a challenging problem, and the present study could serve as a valuable foundation for future investigations.


W2. This is a minor comment rather than a weakness. The related work section provides a comprehensive overview of PDE modeling and neural emulators. However, a few relevant studies on generalization to unseen dynamical regimes in ODE-based system reconstruction are missing [1, 2]. Specifically, ref. 1 theoretically investigates the limitations of OOD generalization, showing that flexible black-box models, such as neural ODEs, cannot guarantee generalization to unseen attractor basins without additional structural priors. Ref. 2 investigates whether neural ODEs trained on pre-bifurcation data can predict post-bifurcation dynamics, including cases where the model hallucinates spurious bifurcations.

Although differing in scope and methodology, these studies address closely related questions and emphasize the importance of appropriate inductive biases for unseen-regime prediction. Briefly discussing them would better contextualize the contributions of the present paper and strengthen the literature review.

[1] Göring, N., Hess, F., Brenner, M., Monfared, Z., & Durstewitz, D. Out-of-domain generalization in dynamical systems reconstruction. ICML 2024.

[2] Huh, I., Jeong, C., & Alam, M. Context-Informed Neural ODEs Unexpectedly Identify Broken Symmetries: Insights from the Poincaré-Hopf Theorem. ICML 2025.

Questions for authors

Q1. My understanding is that the discrete translation symmetry of the Kolmogorov flow is a property of the governing equations and therefore holds in both the training and test regimes. If so, does the standard CNN already struggle to fit the training data due to its excessive translation equivariance? If not, is the effect of this symmetry mismatch more pronounced in the laminar regime than in the chaotic training regime? A brief clarification would help explain why the architectural limitation becomes particularly evident under the distribution shift.


Q2. More generally, how can we determine whether the symmetries imposed by a neural emulator are consistent with those of the underlying physical system, especially when the governing equations are not fully known? If the physical symmetries are regime-independent, could training losses or the physical validity of learned solutions provide useful evidence for assessing the appropriateness of the imposed symmetry? Conversely, how should this assessment change when the relevant symmetry constraints or their dynamical consequences depend strongly on the regime?

1 comment

We thank the reviewer for the positive and thoughtful review, and for the helpful references.

W1. We agree that a theory is missing, but the local-coverage account is tested against controls in the paper, not only inferred after the fact. With the training data held fixed, a local-attention emulator whose receptive field spans only 6.5% of the L=400L = 400 domain extrapolates zero-shot to L=400L = 400, whereas an FNO, whose trainable modes are fixed by the smallest training domain, does not, and an MLP cannot be evaluated off its training grid (Appendix B, Figs. 8–9). The restricted-training experiments remove the target transitions, and the states leading up to them, from the training data, so the emulator cannot have learned them from examples. The account also explains which of our results succeed and which fail. KS warm-up is driven by the same local kink-forming instability that the training data already sample on the chaotic attractor, and it is recovered. The Kolmogorov chaotic transient explores states weakly represented in training, and it is not. The beta-plane emulator's receptive field spans the whole domain, so we do not use it as evidence for this argument.

W2. Both references bear on unseen-regime prediction and the role of inductive biases (Göring et al., 2024; Huh et al., 2025), and we have added them to the related work.

Q1. We thank the reviewer for this insightful question. The reviewer is correct that the governing equations have the same discrete symmetry in both training and test regimes. The excessive equivariance of standard convolution is therefore present in both. Unphysical translation in the forcing direction can also be observed in chaotic rollouts, but it is harder to distinguish from the underlying motion of coherent structures. In the laminar regime, the physical solution is stationary relative to the forcing, making this drift immediately apparent. The laminar regime also provides a clearer comparison between S²-aware convolution and forcing injection: both suppress the drift, but only the former produces substantially steadier predictions in our experiments.

Q2. Training loss alone is not a reliable indicator: the standard convolution fits the chaotic training data well despite its excess equivariance. A practical check is whether rollouts drift persistently along a direction the equations do not leave invariant. A symmetry that holds only in some regimes should not be imposed beyond them, since imposing it more broadly risks the opposite error.

We thank the reviewer again for the encouraging assessment and the helpful suggestions.

Reviewer_c2MFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors study generalization to unseen parameter settings in Kolmogorov flow, the KS equation, and beta-plane turbulence. For Kolmogorov flow, the authors restrict to the setting of Re=50 and study the generalization to new aspect ratios in the domain. They find that an emulator trained on chaotic flows can produce stationary vorticity fields. For KS, the authors look at the generalization to new domain lengths which is the control parameter for chaos and find that a trained emulator can generalize to larger domain lengths. For beta-plane turbulence, the authors investigate the generalization capability under partial observability. The authors qualitatively show that the model can emulate dynamics with 4 jets which were not in the training distribution.

Critical strengths
  • The authors demonstrate positive generalization results for the KS and beta-plane experiments. However, reproducibility needs to be verified (Weakness 1) and statistical robustness needs to be quantified for the beta-plane experiments (Weakness 4).
Critical weaknesses
  1. This work is not reproducible. The reproducibility statement points to Appendices A and C for architecture and optimization settings, but these describe the KS and beta-plane models. The modified Kolmogorov emulator is not sufficiently specified: its depth, channel widths, kernel sizes, optimizer, and training budget are unclear. Appendix F also introduces a phase-increment loss without specifying how phase is computed. Also, the authors should provide code because withholding code for anonymity concerns is not a real barrier - code can be easily de-anonymized and also the anonymous4openscience service can be used which is standard practice for anonymous review.

  2. There is no systematic evaluation of generalization performance. The evidence for generalization mostly comprises of qualitative, selected examples. If i'm not mistaken, there is no evaluation of aggregated OOD metrics e.g. forecast error on a set of held out trajectories or initial conditions. The only evidence that goes beyond single examples seems to be Fig. 5 and Fig. 6c.

  3. The Kolmogorov flow results do not convincingly demonstrate generalization. My main concern is the sensitive dependence of the results on the timestep. As mentioned, the fact that the main results of paper only hold for Δt=0.5\Delta t = 0.5 indicate that the model is not actually generalizing to the correct dynamics which is also supported by Fig. 1 and 2 which show that the laminar attractors are completely wrong despite correctly representing stationary dynamics. This dependence on the timestep and the incorrect predictions at this selected timestep substantially weaken the generalization claim.

  4. The proposed convolution is not sufficiently supported. The authors propose a convolution which explicitly respects the translational symmetries of the kolmogorov flow domain but do not fairly compare against other convolutions. If the authors intend to propose a novel method, then fair comparisons over multiple seeds with thorough quantification of improvement over baselines. The authors only provide a qualitative comparison for a single instance in Fig. 3 and the baseline convolutions are not even parameter-matched (which the authors note in Sec 4.1).

  5. Statistical validation is lacking. Given that the beta-plane is a probabilistic latent variable model, the forecast uncertainty should be reported since the model is trained as an ensemble forecaster. And a wider evaluation would be informative beyond the single example rollouts of the mean zonal velocity in Fig 6 and 7. Moreover, the details of estimator, normalization, and measurements of confidence interval or uncertainty are missing for the transition probability and return period results for beta-plane and KS experiments respectively.

Questions for authors
  1. (Regarding Weakness 1) Can the authors release anonymized code.

  2. (Regarding Weakness 2) Can the authors quantify the results on more held out examples and initial conditions across all the experiments.

  3. (Regarding Weakness 4) Can the authors perform a proper parameter-matched comparison of the standard conv and forced conv against the proposed conv.

  4. (Regarding Weakness 5) Can the authors measure ensemble spread or uncertainty for the beta plane model in an aggregate evaluation over a set of held out initial conditions (Q2). Further, can the authors clarify more details about how the return periods for KS and the transition probabilities for beta-plane are estimated.

  5. Can the authors also report some sort of spectral metric for all of the PDEs. Especially for KS in the chaotic regime, pointwise error is meaningless beyond some point in time.

6 comments
AuthorsOpenReview ↗

Response to c2MF (1/4): reproducibility and code

We thank the reviewer for the detailed and constructive review, which has helped us strengthen both the paper and its supporting evidence.

The architecture and training configuration of the S²-aware Kolmogorov-flow emulator are below. These details will be included in the revised manuscript.

ParameterValue
Architecture
Hidden channels64
Convolutional blocks6 (two convolutions each)
Convolution kernel size3×3
Downsampling stages2 (stride 2)
Upsampling stages2 (bilinear interpolation)
Transformer blocks8
Attention heads1
Attention window9×9
Transformer MLP ratio4
Attention/MLP projections1×1 convolutions
ActivationGELU
NormalisationGroupNorm (32 groups), LayerNorm
Dropout0
Phase alignmentStreamwise (xx) only
Trainable parameters18,120,162
Training
OptimiserAdam
Learning rate10−410^{-4}
Adam (β1,β2)(\beta_1, \beta_2)(0.9, 0.999)
Batch size30
Training epochs70
Training fraction per epoch5%
Train/validation/test split80% / 10% / 10%

Regarding the phase loss, the streamwise phase is extracted from the first nonzero Fourier mode of the vorticity field, ϕx=arg⁡a1\phi_x = \arg a_1 with a1=∑j,ℓω(yj,xℓ) e−2πiℓ/Nxa_1 = \sum_{j,\ell} \omega(y_j, x_\ell)\, e^{-2\pi i \ell / N_x}. Its increment between consecutive snapshots is wrapped to the interval from −π-\pi to π\pi and represented by its cosine and sine. A separate prediction head outputs a normalised two-component vector, and the loss is the mean-squared difference between the predicted and reference vectors. The predicted increment is used for streamwise alignment; no alignment is performed in the forcing direction.

Code is available in an anonymised repository: https://anonymous.4open.science/r/neural-emulators-unseen-dynamics-288A/. It contains the shared emulator package and the code for the Kolmogorov, Kuramoto–Sivashinsky and beta-plane experiments.

AuthorsOpenReview ↗

Response to c2MF (2/4): timestep sensitivity, laminar attractors and the S²-aware convolution

We thank the reviewer for raising this important point. We agree that the timestep sensitivity is a limitation of the Kolmogorov results. Related difficulties have been observed by Chuang and Barba (2023), who found that PINNs can converge spuriously to steady solutions in flows that should exhibit vortex shedding, potentially owing to numerical dissipation and dispersion. Their work does not investigate timestep sensitivity directly, but illustrates a related failure in reproducing the correct dynamics. Our observation that larger emulator timesteps favour excessive laminarisation may reflect a related numerical effect, although establishing the underlying mechanism requires further investigation. Importantly, the timestep of 0.5 was selected a priori from the temporal spectra of the chaotic training data, resolving approximately 95% of the input-energy variability, rather than by tuning against the unseen laminar regime. We therefore distinguish between conditional generalisation at a fixed temporal discretisation and recovery of the correct underlying evolution operator. Our results support the former, but not the latter.

Regarding the apparent separation of the laminar attractors in Figure 1, the axes cover very narrow ranges, which exaggerates the visual differences. At aspect ratios 0.8500 and 0.8250, the normalised energy discrepancies are approximately 0.028 and 0.038 respectively, corresponding to 2.8% and 3.8% of the reference laminar energy. Nevertheless, we acknowledge that the equilibria are not exact: DNS continuation shows that the predicted states relax towards the true solutions. Our claim is recovery of the previously unseen laminar regime, not exact recovery of its attractors. We will clarify the axis scaling in Figure 1.

We thank the reviewer for raising this concern and agree that a more systematic comparison is necessary to establish the empirical benefits of the proposed convolution. We have therefore carried out two additional experiments. First, we tested the translation-equivariance properties of the three convolutions using shifts of 30–34 grid points in the forcing direction (this test was performed on a synthetic input using untrained convolutional kernels). The S²-aware convolution is equivariant under the physically admissible shift of 32 points, but not neighbouring shifts. Standard convolution is equivariant under all five shifts. Adding a fixed forcing channel can also break the unwanted symmetry, provided that the forcing contributes nontrivially to the output. This demonstrates the different ways in which the architectures handle the forcing symmetry.

Second, we conducted a capacity-controlled comparison using three independent training seeds and ten matched initial conditions per seed, at aspect ratio 0.85. The S²-aware model retains 64 hidden channels and one attention head (18.12 million parameters), while the standard and forcing-injection baselines use 192 channels and three heads (21.04 million parameters each). All models therefore retain 64 channels per attention head, with the baselines having slightly more total parameters. We measured temporal steadiness through the relative change in predicted vorticity between consecutive timesteps, normalised by the timestep, and forcing-relative drift through the phase of the vorticity Fourier mode at the forcing wavenumber. The comparison reveals distinct shortcomings of the two baseline architectures. The standard convolution exhibits substantial unphysical drift in the yy-direction, with an absolute drift rate of 0.01733±0.009290.01733 \pm 0.00929 and a temporal steadiness error of 0.07648±0.017830.07648 \pm 0.01783. Introducing the forcing as an additional input effectively suppresses this drift, but produces less steady predictions, with a temporal steadiness error of 0.12944±0.010990.12944 \pm 0.01099. By contrast, the proposed S²-aware convolution also suppresses the unphysical drift while achieving the lowest temporal steadiness error of 0.01177±0.005940.01177 \pm 0.00594, approximately 11 times smaller than that of the forcing-injection baseline. These results suggest that removing the spurious translation equivariance alone is insufficient to maintain approximately stationary laminar dynamics, and that the manner in which the correct symmetry structure is incorporated into the architecture also matters.

We will include the additional equivariance comparison and multi-seed results in the revised manuscript.

AuthorsOpenReview ↗

Response to c2MF (3/4): repeated KS experiment with bootstrap intervals

We thank the reviewer for this thoughtful point. It prompted us to repeat the experiment at a much larger scale, and the results are below. The return periods in the paper are computed over 10,000-step autoregressive rollouts for both DNS and emulator, so they are statistics accumulated along long trajectories, not comparisons of single events. We agree that intervals make them easier to read, particularly for the rarest transitions. The table below reports the repeated experiment for both restricted training sets (states with at most four kinks, and data with every kink-count transition removed), using a retrained emulator (3 seeds × 32 rollouts × 10,000 steps), DNS restarted from the same initial states, and 95% bootstrap intervals over rollouts.

Return period in ML steps (1 step = 1 time unit), 1/P(transition∣k kinks)1/P(\text{transition} \mid k \text{ kinks}):

KinksTransitionMax 4 kinks: DNSMax 4 kinks: MLNo transitions: DNSNo transitions: ML
33→413 [11, 16]13 [11, 15]11 [9.9, 13]16 [13, 19]
3no change1.2 [1.2, 1.3]1.3 [1.2, 1.3]1.3 [1.2, 1.3]1.2 [1.2, 1.2]
44→5255 [165, 391]257 [171, 391]18 [8.3, 33]13 [8.7, 25]
44→314,477 [8,640, 26,310]12,618 [7,437, 24,666]808 [366, 1,537]261 [148, 503]
4no change1.0 [1.0, 1.0]1.0 [1.0, 1.0]1.1 [1.0, 1.2]1.1 [1.1, 1.2]
55→613 [13, 14]13 [12, 13]13 [13, 13]12 [12, 13]
55→4101 [95, 107]97 [91, 102]95 [92, 99]91 [85, 97]
5no change1.1 [1.1, 1.1]1.1 [1.1, 1.1]1.1 [1.1, 1.1]1.1 [1.1, 1.1]
66→7105 [101, 108]95 [91, 100]103 [100, 106]92 [89, 96]
66→520 [20, 21]21 [21, 22]21 [20, 21]22 [22, 23]
6no change1.1 [1.1, 1.1]1.1 [1.1, 1.1]1.1 [1.1, 1.1]1.1 [1.1, 1.1]
77→679 [50, 120]74 [49, 111]125 [88, 174]103 [65, 157]
7no change1.0 [1.0, 1.0]1.0 [1.0, 1.0]1.0 [1.0, 1.0]1.0 [1.0, 1.0]
8no change1.4 [1.3, 1.5]1.2 [1.2, 1.3]1.2 [1.2, 1.4]1.2 [1.2, 1.2]

Transitions with fewer than 5 events are omitted.

In both restrictions, the emulator's return periods agree with DNS to within about 20% for transitions from 5–7 kinks, including the 5–7-kink states that the max-4-kinks emulator never saw in training. At 6 kinks the differences are small (5–10%) but larger than the statistical uncertainty. The largest discrepancy is the rare 4→3 transition under the no-transitions restriction, which the emulator produces about three times more often than DNS.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.