acceptodds
Under review as a conference paper at ICLR 2027

CasEm: A Cascade Architecture for Long-Horizon Neural Emulation

Abstract

Autoregressive neural emulators can drift or diverge over long rollouts despite accurate short-term predictions. We introduce Cascaded Emulation (CasEm), a one-way rollout architecture that augments an existing full-state backbone with an independently evolving model of physically specified aggregates. Its forecasts guide corrections to full-state predictions, without feedback from the backbone to the aggregate model. Effective guidance requires aggregates that cover substantial backbone error, remain accurately predictable, and support useful full-state corrections. We derive a finite-horizon error bound that clarifies these three factors and use empirical diagnostics to guide subsystem selection. Across four ODE/PDE benchmarks, CasEm reduces long-horizon rollout errors across diverse backbones and suppresses the trend toward error divergence in both diffusion tasks using Fourier neural operator backbones. In global climate emulation, CasEm with a regional total-water subsystem reduces 10-year full-state time-mean error by and for frozen ACE and Spherical DYffusion backbones, respectively, while adding less than to inference time. The implementation can be found at https://anonymous.4open.science/r/CasEm_code-6807.

Primary area:
learning on time series and dynamical systems
Keywords:
Neural emulation, Cascade architecture, Long-horizon prediction, Dynamical systems, Climate emulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PhSwOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2(−1)
Read review
Paper summary

The paper introduces CasEm, a method to improve ML-based long-horizon forecasts that are usually subject to accumulated errors or instability due to the use of autoregressive rollout (chaining output of the previous inference as the input of the next one). CasEm proposes a cascade rollout where a smaller sidecar model is used to correct the main model over the prediction of big quantities. The feedback is unidirectional and hence guarantees the evolution of the sidecar model without influence of the main model errors. The paper derives an error bound expressing a trade-off between coverage, modelability, and correctability, and provide guidance to choose the sidecar. The approach is tested in four ODE/PDE benchmarks and a 10-year climate emulation, showing significant reduction of long-horizon forecast error at little computational cost.

Critical strengths

The problem explanation and the objective are clear. The proposed solution is simple to understand and to apply, and effectively cheap. The unidirectional feedback ablation directly tests the paper design choice. Code and configurations are provided

Critical weaknesses

The evaluation baselines are not strong enough, and results are not discussed enough. One example is related to the climate emulation, where the climatology may impact the interpretation. In Table 18, a constant 10-year-mean target gives full-state TM 0.046 vs 0.040 for full CasEm, compared with 0.079 for Base S-DY (Table 2), which suggest most of the gain survives a time-constant target and may come from a training climatology. Table 14 shows robustness results averaged over IC1–11, but IC1-10 were used to fit the subsystem (section E.2) The reading is difficult, the main content relying on many results provided in appendices.

Questions for authors

A. Could you add a simple climatology-based subsystem (not learned) to clearly state if CasEm beats it?

B. Were the correction hyperparameters (gain, cap, interval, grid) selected on IC11? Do the 11 trajectories share the same forcing? An evaluation under different forcing would strengthen the generalization claim

A clear win over the climatology baseline in A and more robust evaluations would raise my score

3 comments
AuthorsOpenReview ↗

R1–R2: Questions A–B

R1 : climatology baseline (A)

We thank the reviewer for this suggestion. We have added a simple, non-learned climatology baseline, in which the water targets are computed directly as training-set means and held fixed throughout the ten-year rollout. This differs from the existing ten-year-mean ablation, whose constant targets are obtained by averaging the learned subsystem’s rollout and therefore retain information learned by that subsystem.

MethodFull-state TMFull-state STTotal-water TMTotal-water ST
Training-set climatology0.066350.823090.136600.88142
Ten-year mean of the water-subsystem rollout0.045760.819830.072050.76813
Full CasEm0.040110.803380.066520.73852

Results and analysis. Full CasEm clearly outperforms the non-learned training-set climatology baseline on all four metrics, reducing full-state TM and ST by 39.5% and 2.4%, and total-water TM and ST by 51.3% and 16.2%, respectively. Both the training-set climatology and the rollout-mean target are constant in time, yet the latter yields lower errors. This shows that the gains retained by the ten-year-mean ablation cannot be reproduced by simply imposing the training climatology. Full CasEm further improves all four metrics over the rollout-mean target, indicating an additional benefit from retaining the subsystem’s time-varying forecasts.

R2: Hyperparameter selection and robustness under different forcing

Were the correction hyperparameters selected on IC11? No. The aggregation grid, correction interval, and gain were chosen using development data IC10 and fixed for the IC11 evaluation. The default pseudoinverse correction itself is determined by HH. Although we used a gain of 0.5 to control the correction magnitude, our additional experiments show that this scaling is not necessary: removing it (i.e., using a gain of 1) has little effect on the final performance.

Do the 11 trajectories share the same forcing? Yes. They form an initial-condition ensemble with the same prescribed forcing, including annually repeating climatological SST and solar forcing, and fixed CO₂ and other greenhouse-gas concentrations. This follows the FV3GFS benchmark used by ACE [1] and Spherical DYffusion [2]. These trajectories therefore test sensitivity to initial conditions rather than generalization across forcing conditions.

Additional evaluation under different forcing. We added experiments using the SHiELD AMIP-constant-CO₂ dataset, which uses historically varying SST rather than the annually repeating SST climatology of the original benchmark, with CO₂ held at its 1979 level [3]. This also changes the reference simulator from FV3GFS to SHiELD.

Since S-DY was not trained on this SHiELD setting, we evaluate its transfer over seven independently initialized three-year windows. We also evaluate ACE2S-SHiELD+, pretrained on SHiELD simulations including AMIP data, abbreviated as ACE2S below. Both backbones remain frozen. The total-water subsystem is fitted on 2001–2006 and validated on 2007–2008; evaluation uses 1980–2000.

BackboneFull-state TM reductionFull-state ST reductionTotal-water TM reductionTotal-water ST reduction
S-DY22.6%9.0%34.1%24.8%
ACE2S30.3%6.8%55.7%19.7%

Results. The table above summarizes the average relative error reductions across the seven three-year evaluation windows. CasEm reduces all four error metrics relative to Base for both backbones in every window, supporting robustness across the tested initialization years and prescribed forcing trajectories. We have also added this experiment to Appendix E.3.2, where Table 18 reports the per-window results, including Oracle.

References

[1] Watt-Meyer, O., et al. (2023). ACE: A fast, skillful learned global atmospheric model for climate prediction. arXiv:2310.02074.

[2] Rühling Cachay, S., Henn, B., Watt-Meyer, O., Bretherton, C. S., & Yu, R. (2024). Probabilistic Emulation of a Global Climate Model with Spherical DYffusion. Advances in Neural Information Processing Systems, 37.

[3] Clark, S. K., et al. (2026). Disentangling the effects of sea surface temperature and CO₂ in global machine learned weather-climate emulators. arXiv:2606.07928.

Thank you for suggesting the climatology baseline and the evaluation under different forcing; we have added both experiments and clarified that the correction settings were chosen without using the held-out test trajectory. We hope the new results answer your questions about robustness and would be happy to discuss any remaining concerns.

Thank you for taking the time to add the new experiments, they strengthen your evaluation and clarify the impact of your approach. I still have some concerns about the overall reading of the article, for which I have no clear hints you took my remark into consideration.

Reviewer_VXnrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a new way to stabilize long-horizon neural emulation, leveraging subsystems with simpler dynamics (ex Fourier coefficients) to guide the full state. Cascaded Emulation (CasEm) does not need retraining of the autoregressive backbone, a low-dim model g_psi (learned) and a coordination map C_phi (learned or prescribed) are integrated to guide and correct the backbone predictions. The specificity is that the subsystem evolves independently, and that the influence is one-way: the backbone never modifies the subsystem dynamics, to avoid joint accumulation of errors. They provide a bound and protocol to determine the most efficient subsystem, and evaluate across ODE/PDE benchmarks (heat equation, Fisher-KPP, Alanine, BVE) across various backbone architectures, and on a large scale atmospheric emulator.

Critical strengths
  • Authors introduce novel methods to combine a reduced-order system with an emulator and provide results indicating it improves long term stability over several physical systems. The method combines easily with existing emulators and improves their results with low computational overhead.
  • They show that their prediction structure with no feedback from the full state system to the subsystem is more stable and less prone to error accumulation; it also reduces a bias.
  • Large scale example on long term atmospheric forecasting
  • Generalisability / Interpretability: Authors both show results with learned and prescribed coordination map
Critical weaknesses
  1. Positioning on the paper

The idea of having a lower-dimensional system running in parallel to a higher-dimensional system without feedback on the subsystem is not new (e.g., multi-layer RNNs), nor is the idea of using 'intelligent' gates to combine the two systems (e.g. GRUs, LSTMs and a lot of variants). In this context, one might consider the fully learned setting (g, f, C) to be simply a multilayer recurrent network (with a special care of not having a feedback from the high dimensional representation to the low dimensional one). Thus, in my opinion, the core novelty of this paper is that C is an interpretable projection and the subsystem can be tailored to exist in an understandable space (e.g. the low-frequency part of the spectrum). However, I feel that this aspect is underdeveloped in the paper, and we really have to work through the appendix/experiment section to get a good understanding of it. I think the paper would benefit from more comments on that.

From what I understand of the current setup:

  • You have a pretrained emulator f that simulates the evolution of the full state.
  • Then, you define an operator H that goes from a full state to an 'aggregated/reduced' state z.
  • Given the time series of z (i.e., the data projected by H), you can learn your simplified model g.
  • Now, you 'define/learn' C_ϕ to combine the subsystem and the full state system.

In Fig. 8, the authors present a method for selecting H (which defines the subsystem), but this is unclear to me.

  1. How do you learn C_\phi when you have your subsystem prediction? (in an unrolled way?)
  2. Globally, Fig. 8 looks like an expensive/complex optimization (in every step, you have to train a bunch of systems). How hard would it be to do this on a realistic system? How can we 'guess' H in practice?.

In my opinion, this is precisely the contribution of the paper, but it's not made clear in the text. The paper would benefit from a description of how to choose H and perform this optimization in the main text. A potential risk is that the method doesn’t generalise well to arbitrary systems where there is no obvious subsystem so I think this is worth a comment.

  1. “Climate” test case

Secondly, running an atmospheric simulation over 10 year is not climate research, which is interested in running long term (several decades) coupled (with ocean and ice model) simulations under various scenarios (e.g of carbone diaoxyde emissions). Phrasing this evaluation as climate is in my opinion overselling it and could be harmful to climate & ML community in general, blurrying the standard. I propose "long term atmospheric simulation".

Questions for authors
  • In Annex D.2.1, it seems that you do not use the same process for defining C for FNO or ResCNN: why is that ? Did you perform the experiment with the C FNO architecture and what were the results ?

  • The claim that the failure of HINE (l.300) is due to « unnecessary feedback from full-state predictions into closed aggregate dynamics » may be too strong. It does not seem to be only related to that since even the Oracle explodes for ResCNN (Figure 3.c), while SO keeps decreasing. Can you further comment on the stability of SO and instability of ResCNN in that case ? Could it be linked to the choice of C ?

  • If the choice of the subsystem seems relatively natural for the low dimensional exemples, for the atmospheric model it seems more complicated. Why did you choose the water subsystem, did you try others that failed or succeeded ? Is the C learned in that case ? If that is the case, CasEm would necessitate the backbone to be differentiable, but classical numerical models for weather or climate are not usually (the recognized ones used in Model Intercomparisons exercises). How could you adapt CasEm to this bottleneck ?

  • Overall, the whole procedure feels a bit heavy (when SO already performs really well in the long term for BVE and Alanine) and resembles hybrid models (see Kochkov et al. ML accelerated computational fluid dynamics, learned correction and learned interpolation), did you perform an experiment with these types of models to compare with your method ?

5 comments
AuthorsOpenReview ↗

R1: Question 1

R1: Correction differences between FNO and ResCNN (Q1)

Before explaining the reasons for these design choices, we first clarify how the FNO and ResCNN correction maps are related and how they differ. Both are built on the same parameter-free default pseudoinverse correction, which makes the smallest Euclidean change to the backbone prediction needed to match the subsystem forecast. Its role as the default correction was not sufficiently emphasized in the original method section. We have now clarified it at the end of Section 3.1, with the additions highlighted in blue. FNO generally uses the default pseudoinverse correction, whereas ResCNN adds a learned residual network to adjust the complementary components. An exception to this pattern is BVE, where both ResCNN and SFNO add a learned residual network to adjust the remaining vorticity structure.

R1.1: Why does ResCNN use a different correction from FNO?

The default pseudoinverse correction makes the smallest Euclidean change needed to satisfy the aggregate constraint, leaving the complementary components unchanged. Here, the subsystem variables are selected Fourier modes, and FNO also operates in Fourier space, so this correction is naturally aligned with its representation. Because ResCNN relies on local spatial patterns without explicitly separating Fourier modes, it may be more sensitive to mismatches between the corrected aggregate modes and the unchanged complementary components. We therefore introduce a residual network to learn their dependencies and adjust the complementary components accordingly.

R1.2: ResCNN with the correction used for FNO

We evaluated ResCNN with two correction settings. Default (FNO) denotes the additional comparison using FNO’s default pseudoinverse correction without a learned residual network. Default + learned residual denotes the original ResCNN setting in the manuscript, which adds a learned residual network to adjust the complementary components.

Heat / ResCNN

HorizonBaseDefault (FNO)Default + learned residual
100 steps0.006019 ± 0.0012950.003614 ± 0.0009210.003521 ± 0.001007
500 steps0.009641 ± 0.0035300.004438 ± 0.0016060.004150 ± 0.001582

Fisher–KPP / ResCNN

HorizonBaseDefault (FNO)Default + learned residual
100 steps0.02708 ± 0.002570.01451 ± 0.005090.01202 ± 0.00543
500 steps0.04492 ± 0.022300.03432 ± 0.026200.01350 ± 0.00303

Results and analysis. The correction used by FNO also improves ResCNN on both tasks. Adding the residual network provides only modest gains on Heat but substantially improves long-horizon accuracy on Fisher–KPP. Compared with Heat, where Fourier modes evolve independently, nonlinear mode interactions in Fisher–KPP may make ResCNN more sensitive to mismatches between the corrected aggregate modes and the unchanged complementary components. The learned residual helps reconcile these components.

AuthorsOpenReview ↗

R2–R3: Questions 2–3

R.2: HINE’s failure and the stability of SO and ResCNN (Q2)

HINE’s aggregate prediction errors. We acknowledge that, based on the main-text tables alone, our original attribution was too strong. Additional diagnostics show that, over the first 10,000 rollout steps, the center-of-mass position RMSEs of HINE’s two internal aggregate prediction heads are approximately 10 nm with GNS and 7 nm with EGNN, compared with 0.00565 nm for the independently evolved subsystem. These diagnostics confirm that HINE indeed fails to accurately predict the aggregate variables in this example. The problem therefore already arises at the subsystem-prediction level, unlike the Oracle failure discussed below, which occurs even with exact aggregate targets. We have revised the corresponding passage to clarify the scope of the claim and include these additional results, with the changes highlighted in blue.

Oracle’s divergence and the role of CC. Even with exact aggregate targets, full-state predictions may still diverge, as you noted in Figure 3(c). Oracle accurately corrects the state along the aggregate directions, but errors in the complementary subspace may continue to grow during autoregressive rollout. The correction map CC can affect this process: the default pseudoinverse correction adjusts only the aggregate directions, leaving the complementary components unchanged. A neural network can learn dependencies between these components and the aggregate variables, enabling complementary adjustments that improve consistency with the corrected aggregates. The learned correction used with ResCNN follows this approach. Our additional comparisons show that it reduces long-horizon error, although it does not guarantee the elimination of divergence. We have added this explanation of the choice of correction map at the end of Section 3.1, immediately before Remark 1.

Why SO remains stable. In contrast, SO autoregressively advances only the low-dimensional aggregate variables and reconstructs full-state outputs without feeding those reconstructions into the next step. SO therefore does not recursively evolve the complementary components, and its simpler low-dimensional dynamics make stability easier to maintain over long rollouts.

R3: Subsystem choice and correction-map training (Q3)

Why water, and what about other subsystems? For S-DY, we explored several candidate subsystems and now illustrate our selection process in Appendix E.1.3 (Table 15). Among the reported candidates, error coverage ρ\rho first narrows the choice to water and wind. Water also shows favorable modelability qq, whereas wind performs less well. Wind further exhibits low correctability γ\gamma, leading to the final selection of water. Additionally, the energy subsystem yielded positive results with ACE, as reported in Appendix E.3.3, with its construction detailed in Appendix E.1.2.

Is the correction learned? For the water subsystem in our main experiments, no. It uses the default pseudoinverse correction. For the energy subsystem in Appendix E.1.2, yes: a conditional diffusion network is trained to generate temperature corrections.

Does learning the correction require a differentiable backbone? No. The energy subsystem in the appendix uses a conditional diffusion network to generate temperature corrections. We train this network on paired predictions from the frozen backbone and corresponding reference states, using their temperature differences as target correction residuals. Conditional denoising uses the energy mismatch, external forcing, and predicted temperature and pressure fields as inputs. Data collection and network optimization are performed separately, with no gradients propagated through the backbone or the rollout. Thus, this training procedure does not require a differentiable backbone.

AuthorsOpenReview ↗

R4: Question 4

R4: Subsystem-only prediction and comparison with hybrid models (Q4)

R4.1: Why use CasEm when SO already performs well?

SO is indeed simpler and can perform well in some settings, but this does not mean that it can generally replace CasEm. Its simplicity comes from evolving only the low-dimensional aggregates and reconstructing the full state, rather than modeling the evolution of the remaining degrees of freedom.

On Alanine, independent center-of-mass prediction helps avoid global translational drift, while a trained decoder reconstructs the atomic state using relationships learned from training data. This combination can yield good full-state accuracy when those relationships remain applicable to the test states. However, the same center-of-mass state can correspond to different internal configurations. This reconstruction therefore depends on the states and relationships covered by the training data, rather than predicting subsequent evolution from the current internal configuration and relative velocities.

For BVE, SO instead uses a fixed spectral reconstruction, retaining only the three degree-1 spherical-harmonic components and setting the others to zero. This discards the remaining vorticity structure but avoids recursively amplifying prediction errors in the discarded modes. Consequently, this simple reconstruction can remain competitive when errors in full-state rollouts grow, even though it does not describe the remaining vorticity dynamics.

CasEm retains the full-state backbone precisely to preserve dynamical modeling of these degrees of freedom, rather than relying entirely on low-dimensional reconstruction. Its additional computation advances the current full state and applies macroscopic corrections using stable subsystem forecasts. We include SO primarily as a diagnostic reference for long-horizon subsystem stability, illustrating that prediction errors in these low-dimensional aggregates change little as the forecast horizon increases, rather than positioning it as a replacement for Base or CasEm for full-state prediction.

R4.2: Comparison with hybrid models.

We added comparisons with learned interpolation (LI) and learned correction (LC), following the hybrid modeling approaches of Kochkov et al. [2]. We use the two-dimensional Kolmogorov flow case adapted from Constante-Amores and Graham [1], with Reynolds number Re=12\mathrm{Re}=12, a doubly periodic domain [0,2π]2[0,2\pi]^2, and sinusoidal forcing f=(sin⁡2y,0)\mathbf f=(\sin 2y,0). The vorticity fields are represented on a 32×3232\times32 grid, with 0.5 time units per reported forecast step.

Base is a residual MLP that predicts increments to the full vorticity field, using two hidden layers of 128 units with SiLU activations. CasEm retains the same frozen backbone and introduces an 8-dimensional subsystem comprising the leading proper orthogonal decomposition (POD) coefficients of the vorticity field. The POD modes capture dominant spatial patterns extracted from the training data. A separate MLP predicts their coefficients autonomously, and pseudoinverse writeback uses these forecasts to correct the full-state predictions. LI instead learns interpolation within the numerical solver’s advection calculation, while LC learns a correction to the solver’s predicted velocity field. Both hybrid methods are adapted to the same flow setting.

All methods use the same chronological training/validation/test split and are evaluated from the same 16 test starts. The table reports full-field vorticity mean squared error (MSE), averaged over the spatial grid and test starts, at each forecast horizon.

HorizonBaseCasEmLILC
203.918×10−63.918 \times 10^{-6}4.028×10−64.028 \times 10^{-6}2.059×10−6\mathbf{2.059 \times 10^{-6}}4.344×10−64.344 \times 10^{-6}
501.210×10−51.210 \times 10^{-5}1.224×10−51.224 \times 10^{-5}6.267×10−6\mathbf{6.267 \times 10^{-6}}9.469×10−69.469 \times 10^{-6}
2001.015×10−41.015 \times 10^{-4}9.684×10−59.684 \times 10^{-5}5.062×10−55.062 \times 10^{-5}3.550×10−5\mathbf{3.550 \times 10^{-5}}
5001.408×10−31.408 \times 10^{-3}6.008×10−4\mathbf{6.008 \times 10^{-4}}1.059×10−31.059 \times 10^{-3}9.860×10−49.860 \times 10^{-4}
10005.960×10−25.960 \times 10^{-2}3.903×10−3\mathbf{3.903 \times 10^{-3}}1.441×10−21.441 \times 10^{-2}4.245×10−24.245 \times 10^{-2}

Results and analysis. LI and LC perform well at relatively short horizons, but their errors grow more rapidly over longer rollouts. CasEm achieves substantially lower errors at 500–1,000 steps, highlighting its advantage in long-horizon prediction. Forecasts of the dominant flow structures provide sustained guidance for the full-state rollout, helping control error accumulation.

Reference

[1] Constante-Amores, C. R., & Graham, M. D. (2024). Data-driven state-space and Koopman operator models of coherent state dynamics on invariant manifolds. Journal of Fluid Mechanics, 984, R9.

[2] Kochkov, D., et al. (2021). Machine learning–accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21), e2101784118.

Reviewer_xSLnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a relatively simple, plug-and-play correction framework that can be integrated with existing predictors without modifying their architectures. With limited additional inference overhead, it shows improvements in controlling long-horizon prediction errors. The method is evaluated across multiple datasets against existing baselines, with supporting theoretical analysis and experimental results. However, further investigation may be needed regarding experimental scalability, computational cost-effectiveness, and generalizability to more complex systems.

Critical strengths
  1. The authors study an important problem, the long-term rollout accuracy.

  2. The proposal stems from the predictor-corrector numerical schema is easy to understand.

Critical weaknesses
  1. The submission lacks readability:
  • at Section 3.1, what's the connection between rtr_t and the subsystem model?
  • What is G⋆G^{\star}, and how does it connect to the design choice of gψg_\psi?
  • The method section only described an abstract idea, where the experiments show that many implementation choices remain task dependent?
  1. The selection of the aggregate readout H is highly case-dependent. Although the paper provides evaluation criteria, it lacks a systematic method for identifying suitable subsystems, potentially limiting its applicability to complex, high-dimensional systems.

  2. Training the subsystem and selecting suitable aggregate variables require additional computation beyond backbone inference. These costs are not fully quantified and may affect the overall computational cost-effectiveness.

  3. Mapping low-dimensional subsystem corrections back to the full-state space requires careful correction design. As suggested by the climate ablations, different correction allocations can significantly affect prediction accuracy, raising questions about generalizability to more complex systems.

  4. Theorem 1 provides an interpretable decomposition of ideal correction gains, subsystem prediction errors, and their propagation. However, its practical predictive value remains unclear because it depends on oracle-rollout performance and a sensitivity constant B for which a usable general upper bound is not established. Evaluating bound tightness, horizon dependence, and whether its diagnostics predict successful subsystem choices on held-out trajectories would strengthen the theoretical contribution.

Questions for authors
  1. The paper considers several correction mechanisms. Could the authors compare alternative correction maps while holding the subsystem forecasts and backbone fixed, and evaluate whether maps selected on development data remain effective under unseen initial conditions or physical regimes?

  2. For the fair benchmark, could you run the experiments on datasets like realPDEbench or directly on the dataset HINE used? It is difficult to evaluate the results considering the limited amount of the ablations reported: the choice of the backbones, the implementation details, and the choice of hyper-parameters matter.

  3. Could the authors report subsystem-training and design-selection costs alongside inference overhead, and show accuracy versus total computational cost for different numbers of deployed rollouts?

10 comments
AuthorsOpenReview ↗

R1: Question 1 — Part 1

R1: Correction-map comparisons and generalization (Q1)

R1.1: Different correction maps built on a common default

To clarify the requested comparison, we group the correction maps used in these experiments into two categories.

Default correction. Our default is parameter-free pseudoinverse writeback, which makes the smallest Euclidean change to the backbone prediction needed to match the subsystem forecast. In the original submission, this rule was mentioned in Theorem 1 and explicitly formulated in Appendix B.1 (“Correction rule”), but its role as the default correction was not sufficiently emphasized. We have now clarified this point in blue at the end of Section 3.1. The Heat and Fisher–KPP FNOs use this rule directly. The climate water and Alanine experiments use it with task-specific adjustments: a fixed correction gain of 0.5 for climate water, and a simplification of H†H^\dagger to a uniform translation operator for Alanine, applying the same position and velocity corrections to all atoms.

Default correction augmented with a learned residual. The ResCNNs and BVE/SFNO additionally use a residual network to adjust the complementary components. The default correction leaves these components unchanged, although they may be coupled to the corrected aggregates. The network learns these dependencies and makes corresponding adjustments. We have also added this explanation at the end of Section 3.1. In practice, when evaluating correctability γ\gamma in Section 3.2, we start with the default pseudoinverse correction. If it performs poorly, a learned residual network can be added to adjust the complementary components.

R1.2: Comparing correction maps with fixed backbone and subsystem forecasts

We compare these correction choices to assess the effects of the non-learned adjustments and the additional residual networks, holding the backbone and the entire subsystem forecast sequence fixed within each comparison. The five tables below summarize the results. In the tables, “Default” denotes the unmodified pseudoinverse correction without task-specific adjustments or a learned residual network; † marks the correction settings used in the original manuscript, while unmarked settings correspond to the additional comparisons.

Task / BackboneCorrection map10050016,000
Heat / ResCNNDefault0.003610.00444NF (0/12)*
+ Learned residual †0.003520.00415NF (0/12)*
Heat / FNODefault †(1.31) × 10⁻⁴(0.892) × 10⁻⁴(1.32) × 10⁻⁴
+ Learned residual(1.69) × 10⁻⁴(1.07) × 10⁻⁴(1.39) × 10⁻⁴
Task / BackboneCorrection map1005001,000
Fisher–KPP / ResCNNDefault0.01450.0343(6.11) × 10⁴
+ Learned residual †0.01200.01350.443
Fisher–KPP / FNODefault †0.01260.01540.0181
+ Learned residual0.01360.02600.0314
Task / BackboneCorrection map505001,000
BVE / ResCNNDefault0.65212.164.0
+ Learned residual †0.4832.174.96
BVE / SFNODefault0.1450.9722.30
+ Learned residual †0.1540.7731.72
Task / BackboneCorrection map5002,00010,000
Alanine / GNSUniform COM translation †1.261.281.27
Default1.251.251.31
Alanine / EGNN-velocityUniform COM translation †1.211.251.21
Default1.251.281.29
Task / BackboneCorrection mapFull-state STFull-state TMQ8 STQ8 TM
Climate / ACEGain 0.5 †0.80170.052680.74710.08402
Default0.80390.049520.74750.07186
Climate / S-DYGain 0.5 †0.80340.044520.72770.06412
Default0.79940.041500.73080.06223

Results and analysis. The non-learned task-specific adjustments are not essential: the default correction achieves comparable accuracy in the climate and Alanine experiments. By contrast, adding a learned residual to adjust the complementary components substantially reduces long-horizon errors on BVE with both backbones, and on Fisher–KPP with the ResCNN backbone.

AuthorsOpenReview ↗

R1: Question 1 — Part 2

R1: Correction-map comparisons and generalization (Q1)

This continues our response to Q1. Sections R1.1–R1.2 are provided in our previous comment.

R1.3: Generalization to unseen initial conditions and physical regimes

Unseen initial conditions. The original evaluations already use held-out initial conditions. The correction maps and their hyperparameters were determined using training and development data and remained fixed during testing.

Unseen physical regimes. For Heat, we increased the spatially varying part of the external forcing by 25% at each step, kept its spatial mean fixed, and used the original models. For Fisher–KPP, we reduced the reaction rate from r=1r=1 to r=0.75r=0.75 and fine-tuned the output layers of the backbone and subsystem dynamics model on separate training trajectories. The correction maps themselves were neither retrained nor retuned in either experiment. The table reports mean full-state NRMSE at 100 and 500 steps over 12 trajectories per setting. Each Raw–CasEm pair uses the same backbone and trajectories.

Task / BackboneRaw (100)CasEm (100)Raw (500)CasEm (500)
Heat / ResCNN0.006840.003650.01430.00716
Heat / FNO5.90 × 10⁻⁴1.46 × 10⁻⁴1.00 × 10⁻³9.65 × 10⁻⁵
Fisher–KPP / ResCNN0.02800.01280.1350.0284
Fisher–KPP / FNO0.05010.01580.6640.0294

Results and Analysis. CasEm reduces error in all eight comparisons under the new physical regimes while keeping the correction maps unchanged. In Heat, the gains persist under altered forcing without retraining either predictor. In Fisher–KPP, the backbone and subsystem predictors are adapted to the new reaction rate, but the original correction maps remain effective. These results show that adapting the predictive dynamics does not necessarily require redesigning or retraining the correction map, supporting its reuse under the tested changes in physical conditions.

AuthorsOpenReview ↗

R2: Question 2

R2: Additional benchmarks (Q2)

We added two complementary experiments beyond our original benchmarks. A published Kolmogorov-flow case [1] tests whether CasEm can improve long-horizon flow prediction while better preserving coherent structures and long-term statistics. The second follows the HINE Navier–Stokes setting and extends the forecast horizon to examine whether the comparison changes beyond its original evaluation range.

R2.1: External Kolmogorov-flow benchmark

We independently reconstructed the relative periodic orbit of two-dimensional Kolmogorov flow at Re=12\mathrm{Re}=12 described in [1]. All methods use the same 32×3232\times32 vorticity fields, chronological data split, and 16 test starts, with 0.5 time units per forecast step. CasEm uses the coefficients of the leading eight proper orthogonal decomposition (POD) modes as its subsystem. These modes capture dominant spatial patterns extracted from the training vorticity fields, and an MLP predicts their coefficients autonomously. HINE instead uses spatially averaged vorticity fields at 8×88\times8 and 4×44\times4 resolutions as coarse states within its hierarchical prediction network.

Method100 steps500 steps1,000 steps
Base3.59e−51.51e−36.24e−2
CasEm3.62e−56.30e−43.94e−3
HINE4.96e−48.54e−12.57

We also compare mean kinetic energy and the relative L1L_1 error of the time-averaged energy spectrum over the 1,000-step rollout.

MethodMean kinetic energyEnergy spectrum error
Reference0.81904—
Base0.821680.323%
CasEm0.818950.101%
HINE0.6794917.25%

Results and Analysis. CasEm’s advantage emerges primarily over longer horizons: its full-field error is similar to Base at 100 steps but decreases relative to Base by 58% and 94% at 500 and 1,000 steps. The subsystem diagnostics help explain this improvement. At 1,000 steps, the autonomously predicted POD coordinates have an MSE of 0.000252 in standardized coordinates, compared with 0.1207 for the corresponding projection of Base. These more accurate aggregate forecasts provide a reference for correcting drift along the selected POD directions. The accompanying improvements in mean kinetic energy and energy spectrum show that the benefit extends beyond pointwise prediction error to long-term flow statistics.

R2.2: Comparison using the original HINE configuration

To examine long-horizon performance in the HINE Navier–Stokes setting, we evaluated both the original 100-step horizon and an extended horizon of 1,000 steps. With a time interval of 0.05, these correspond to 5 and 50 time units, respectively. HINE uses spatially averaged vorticity fields at 32×3232\times32 and 8×88\times8 resolutions as its coarse states. The table below reports full-field MSE.

HorizonBaseHINECasEm
10.0003100.0003200.006266
100.0101060.0053270.012942
250.0525980.0332230.058276
500.2236620.1695850.204424
750.4659340.3689080.397674
1000.8188800.6202890.760575
2001.4280001.4721201.436653
5001.5577101.6068171.128986
1,0001.7186151.6476891.241662

Results and Analysis. HINE is more accurate than CasEm over the original 100-step evaluation range, whereas CasEm achieves lower errors than both HINE and Base at 500 and 1,000 steps. This reversal shows that the short-horizon ranking does not persist over longer rollouts and supports evaluating the methods over the longer horizons targeted by CasEm.

R2.3. Backbone choices, ablations, and implementation details

We evaluate two different backbones for each task and report subsystem-choice ablations in Section 4.2 of the original manuscript. Implementation details are provided in Appendices D.1–D.3 and E.1, and our code and experimental configurations are publicly available.

[1] C. R. Constante-Amores and M. D. Graham. Data-driven state-space and Koopman operator models of coherent state dynamics on invariant manifolds. Journal of Fluid Mechanics, 984, R9, 2024.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Closing summary of the discussion

Dear Area Chair,

As the discussion draws to a close, we would like to summarize the main revisions and additional experiments prompted by the reviewers’ feedback.

First, we revised Section 3.1 to clarify aspects of CasEm’s design that we had not explained clearly enough in the original manuscript:

  • Unlike a stacked RNN, CasEm uses an explicit, physically motivated aggregation rule to extract subsystem trajectories from the full-state training data. This turns the evolution of the selected variables into a separate forecasting task. When their dynamics are simpler, they can be easier to predict over long horizons. Because the subsystem evolves without feedback from the backbone, its forecasts can provide a stable reference for correcting the full-state predictions.
  • CasEm provides a default pseudoinverse correction, which we use in most of our experiments. It requires no training and makes the smallest Euclidean change to the backbone prediction needed for its aggregate values to match the subsystem forecast. This leaves the complementary components unchanged. When these components are strongly coupled to the aggregates, a residual network can learn to adjust them in response to the aggregate correction (as in our ResCNN experiments on Fisher–KPP and BVE).

Second, we clarified how we select a subsystem in practice. In Section 3.2, we organized the three existing diagnostics into a step-by-step procedure for screening candidates. We also added the screening results to Appendix E.1.3 to show why we selected water over the other atmospheric candidates.

Third, we expanded the empirical evaluation. We reported CasEm’s training and inference costs and tested its long-horizon accuracy on additional benchmarks and under different atmospheric forcing. On the RealPDEBench cylinder-flow task, adding CasEm improved long-horizon forecast accuracy for both tested backbones. In the atmospheric experiment, which used historically varying rather than annually repeating sea surface temperatures, CasEm reduced both time-mean and spatiotemporal errors for the full state and for total water. These gains held for both backbones in each of the seven three-year evaluation windows.

We thank the reviewers for their thoughtful feedback and appreciate your consideration of the revised manuscript and the additional results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.