acceptodds
Under review as a conference paper at ICLR 2027

Beyond One-Step Accuracy: Trajectory, Invariant, and Dissipative Fidelity in Modelling of PDE Dynamics

Abstract

Popular approaches for modelling PDE dynamics in a data-driven way include direct one-step predictors like neural operators. Recently, continuous-time generator-based methods like Trajectory Flow models have also shown strong results in this setting. These approaches typically have good one-step accuracy. However, long-horizon trajectory fidelity under autoregressive composition and physical consistency are also important metrics for physical systems. Low one-step prediction error alone does not determine long-horizon trajectory fidelity or physical consistency. We analyse why continuous-time generators achieve stronger long-horizon fidelity than the direct one-step predictors. We also introduce GENERIC structure into the Trajectory Flow model to better preserve known physical invariants while retaining a continuous-time generator formulation. Our model reduces invariant violations by orders of magnitude. This substantial gain in invariant preservation comes at the cost of some trajectory accuracy, although the structured model retains strong long-horizon fidelity. Moreover, invariant preservation alone does not ensure quantitatively correct dissipation. The structured model can evolve dissipative functionals in the correct direction while misestimating their rate. We therefore extend GENERIC Flow with explicit functional-change supervision, which partially recovers dissipative-rate fidelity.

Primary area:
learning on time series and dynamical systems
Keywords:
long-horizon dynamics, continuous-time generators, autoregressive rollout, conservation laws, GENERIC, Trajectory Flow, PDE surrogate modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kUdsOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies trajectory accuracy, invariant preservation, and dissipative-rate fidelity in learned PDE dynamics. It compares direct direct one-step predictors with Trajectory Flow. It then introduces GENERIC Flow to incorporate prescribed conservative and dissipative structure, with additional functional-change supervision for dissipative rates. Experiments report stronger trajectory prediction for Trajectory Flow, improved invariant preservation at the expense of trajectory accuracy for GENERIC Flow, and partial recovery of dissipative fidelity through rate supervision.

Critical strengths

None

Critical weaknesses
  • The claimed advantage of continuous-time generators is insufficiently justified: The paper claims “We analyse why continuous-time generators achieve stronger long-horizon fidelity than the direct one-step predictors.” However, the argument in Sec. 4 does not establish this advantage. It provides error-propagation expansions but does not establish that generators yield smaller errors or weaker amplification than direct predictors. Importantly, a direct one-step predictors is not inherently restricted to one-step supervision, it can be trained on unrolled trajectories, with losses on intermediate and terminal predictions. Yet the local-error and Jacobian diagnostics in Table 1 cover only the basic direct predictors. The conclusion should be restricted to the tested configurations, not presented as an explanation of a broader model-class advantage.

  • The motivation and contribution are unclear: The paper moves from general claims about continuous-time generators to Trajectory Flow without sufficiently justifying why this specific training formulation should represent the broader model class. The subsequent GENERIC extension addresses physical constraints, but its connection to the preceding temporal-representation analysis remains underdeveloped. Moreover, the reported improvements in invariant preservation come at the expense of trajectory. These observations demonstrate trade-offs within the tested framework, not inherent incompatibility between the objectives.

Questions for authors
  • Clarify the central contribution: Please clarify the central contribution and the connection between continuous-time generators and trajectory, invariant, and dissipative-rate fidelity? The paper first argues for continuous-time generators, then selects Trajectory Flow, introduces GENERIC constraints, and reports the resulting fidelity trade-offs. However, the unifying research question remains unclear: how does the temporal-representation analysis motivate this particular model and its structural extension? Please clarify whether the primary contribution is a general explanation of dynamical fidelity, a physics-constrained modelling method, or an empirical investigation of trade-offs, and distinguish conclusions about continuous-time generators broadly from observations specific to the chosen framework.

  • Abnormal data in experimental results: Please clarify the reliability and consistency of the reported results: Table 2 reports catastrophic FNO rollout errors, including an actual nRMSE@100 of 2.999×1052.999\times10^5 on Advection; The same CNN Trajectory Flow baseline has different nRMSE@100 values in Tables 2 and 3 after unit conversion, for example, 5.072×10−45.072\times10^{-4} versus 5.804×10−45.804\times10^{-4} on Heat. Please reconcile these differences and identify the runs and evaluation scripts underlying both tables.

  • Readability and proofreading: Table 2 uses text substantially smaller than the 10-point size specified in the ICLR template, making the numerical results difficult to read. Page 10 retains template instructions, such as “This section is required and does not count toward the page limit”. Additionally, the formula in the “Skew operator” box of Figure 4 contains an unmatched closing parenthesis. Additionally, please clarify whether Figure 4 was directly generated by AI tools.

7 comments
AuthorsOpenReview ↗

Response to reviewer kUds

Thank you for your thoughtful review.

Weakness 1

The claimed advantage of continuous-time generators is insufficiently justified. Sec. 4 provides error-propagation expansions but does not establish that generators yield smaller errors or weaker amplification than direct predictors.

We would like to clarify the nature of the argument in Section 4 and why the error-propagation framework is important.

TFM and direct one-step predictors use very different parameterisations and training methodologies. TFM learns a vector field that is integrated numerically (Sec. 3.2), while a one-step predictor directly learns the finite-step map (Sec. 3.1). So we cannot directly compare their internal parameters to understand the effect of the time parameterisation.

However, since both models are deployed in an autoregressive setting, the error-propagation framework (Eq. 2) gives a standard way to compare them through the quantities that are responsible for long-horizon error. The framework identifies two terms: the local transition defect δk\delta_k and the input Jacobian JkJ_k of one deployed step. The input Jacobian is the Jacobian with respect to the state, not to be confused with the parameter Jacobian. Long-horizon error depends jointly on both terms (l. 194-195).

Next, we measure both terms directly on matched test states in Table 1. TFM has a much lower local defect for both backbones. It also has fewer expansive steps.

Finally, we give a mechanistic explanation of why continuous-time vector-field learners can have a better inductive bias (Eq. 3, l. 197-203). In TFM, the finite-step transition and its sensitivity both come from the same learned vector field, so learning the vector field correctly constrains both of them together. A direct one-step predictor only fits the endpoints, which leaves the map and its sensitivity less tightly coupled (l. 204-207).

The purpose of Section 4 is therefore not to prove that continuous-time generators are always better. It is to identify the quantities that govern autoregressive error, measure them, and give an explanation that is consistent with these measurements. In the revised version, we also extend these diagnostics to the strengthened direct predictors (App. C; pointer in l. 220), as discussed in our response to the next point.

Importantly, a direct one-step predictor is not inherently restricted to one-step supervision; it can be trained on unrolled trajectories, with losses on intermediate and terminal predictions. Yet the local-error and Jacobian diagnostics in Table 1 cover only the basic direct predictors.

We agree, and this is also the setting we study in the paper. In Section 3.1, we explicitly consider multistep rollout-aware training (with losses on all intermediate and terminal predictions of an MM-step rollout) and input-Jacobian regularisation, to check whether the failures of direct predictors can be fixed purely through stronger training objectives. These strengthened predictors are motivated in the Related Work (l. 097-100), evaluated in Table 2, and discussed in Section 6.2. The strengthened predictors improve some settings, particularly with the CNN backbone, but do not consistently recover the long-horizon accuracy of TFM.

The purpose of Table 1 is to compare the two temporal parameterisations through the two quantities in Eq. 2, which is why it reports the basic formulation of each. The strengthened objectives target exactly these quantities: multistep training targets the error accumulated along the rollout, and Jacobian regularisation targets the input Jacobian.

Following the reviewer's suggestion, we have now measured the same quantities for every training strategy in Table 2.

BackboneModelOne-Step MSE ↓\downarrowMedian σ^max⁡\hat{\sigma}_{\max}P95 σ^max⁡\hat{\sigma}_{\max}Frac. σ^max⁡>1\hat{\sigma}_{\max}>1
CNNOne-step8.154×10−68.154\times10^{-6}0.99781.012122.3%
CNNMultistep5.284×10−55.284\times10^{-5}0.99791.030534.5%
CNNOne-step + Jacobian1.768×10−51.768\times10^{-5}0.99731.017322.9%
CNNMultistep + Jacobian2.466×10−52.466\times10^{-5}0.99771.017119.0%
CNNTFM8.049×10−108.049\times10^{-10}0.99701.004114.0%
FNOOne-step2.297×10−52.297\times10^{-5}1.05711.393094.0%
FNOMultistep2.041×10−32.041\times10^{-3}1.15971.926093.5%
FNOOne-step + Jacobian3.416×10−53.416\times10^{-5}1.00711.046781.2%
FNOMultistep + Jacobian2.329×10−32.329\times10^{-3}1.09851.303693.8%
FNOTFM1.154×10−81.154\times10^{-8}0.99931.056722.9%

The conclusion of Table 1 holds for every training strategy: TFM keeps a local defect several orders of magnitude lower than every direct predictor, for both backbones.

AuthorsOpenReview ↗

Response to reviewer kUds continued..

Weakness 1 continued..

As expected, multistep training gives up some one-step accuracy at reference states, since it optimises the error along the model's own rollout rather than the one-step error. To measure this effect directly, we also evaluate every model at the state it actually reaches during deployment. Starting from the true initial state, each model is rolled out for 100 steps on its own predictions. At the state where it ends up, we measure the rollout error, the local defect (one step of the model compared with one step of the exact PDE solver from the same state), and the Jacobian amplification.

BackboneModelRollout MSE ↓\downarrowLocal defect ↓\downarrowMedian σ^max⁡\hat{\sigma}_{\max}P95 σ^max⁡\hat{\sigma}_{\max}Frac. σ^max⁡>1\hat{\sigma}_{\max}>1
CNNOne-step4.06×10−14.06\times10^{-1}3.73×10−13.73\times10^{-1}0.99771.039921.2%
CNNMultistep5.65×10−45.65\times10^{-4}2.00×10−72.00\times10^{-7}0.99931.021043.6%
CNNOne-step + Jacobian1.07×10−11.07\times10^{-1}2.92×10−52.92\times10^{-5}0.99741.015717.1%
CNNMultistep + Jacobian4.35×10−44.35\times10^{-4}4.17×10−84.17\times10^{-8}0.99861.006031.0%
CNNTFM9.04×10−79.04\times10^{-7}1.25×10−81.25\times10^{-8}0.99591.008914.5%
FNOOne-step2.39×10122.39\times10^{12}1.20×10111.20\times10^{11}1.12112.109798.6%
FNOMultistep2.52×10−12.52\times10^{-1}2.70×10−52.70\times10^{-5}1.32432.8286100.0%
FNOOne-step + Jacobian3.02×10403.02\times10^{40}∞†\infty^{\dagger}1.02131.326889.5%
FNOMultistep + Jacobian2.69×10−12.69\times10^{-1}2.31×10−52.31\times10^{-5}1.13881.4370100.0%
FNOTFM7.74×10−57.74\times10^{-5}1.28×10−61.28\times10^{-6}0.99931.201825.5%

†^{\dagger}Three KdV trajectories diverge, so the exact solver cannot be restarted from these states. We keep them rather than drop them.

On these states, the local defect ranks the models in the same order as their long-horizon accuracy. For CNN the order is identical, and for FNO the only swap is between Multistep and Multistep + Jacobian, whose values are nearly equal. This shows how multistep training helps the direct predictors: it lowers the local defect on the states the model actually visits. Lower amplification alone is not sufficient; for example, FNO + Jacobian still diverges on three KdV trajectories. Both observations are consistent with Eq. 2, where long-horizon error depends on the two quantities together. In both settings, TFM has the lowest local defect of all models.

Both diagnostics are now included in the revised version (App. C), with a pointer from Section 4.1 (l. 220).

Weakness 2: The motivation and contribution are unclear: The paper moves from general claims about continuous-time generators to Trajectory Flow without sufficiently justifying why this specific training formulation should represent the broader model class. The subsequent GENERIC extension addresses physical constraints, but its connection to the preceding temporal-representation analysis remains underdeveloped. Moreover, the reported improvements in invariant preservation come at the expense of trajectory accuracy.

As described in the general comment, the unifying research question of the paper is how different temporal parameterisations and structural constraints affect learned PDE surrogates under autoregressive deployment, in both long-horizon trajectory accuracy and physical consistency. In the revised version, this question and the three connected questions below are stated explicitly in the Introduction (l. 050-062):

  • How does temporal parameterisation affect long-horizon trajectory accuracy? (Sec. 4.1, Sec. 6.2)
  • Does better trajectory accuracy also imply physical consistency? (Sec. 4.2, Sec. 5 and Sec. 6.3)
  • Does preserving invariants and enforcing admissible dissipation also give the correct dissipation rate? (Sec. 6.4)

Why Trajectory Flow : The distinction we study in Section~4 is between directly learning the finite-step map and learning a continuous-time vector field du/dt=fθ(u,t)\mathrm{d}u/\mathrm{d}t = f_\theta(u,t), whose numerical integration gives the transition. TFM is very close to Neural ODEs in this respect. It learns the same kind of vector field and integrates it at inference. The only difference is in training, where TFM learns the vector field directly from observed transitions without differentiating through the solver (Related Work, l. 108-115; Sec. 3.2). So TFM is not an arbitrary choice. Its continuous-time formulation is exactly the property being studied. We also chose TFM because, within this model class, it is a strong representative. Continuous Flow Operators (CFO; Hou et al., 2026), which we cite in the Related Work (l. 114-115), compare TFM-based training against Neural ODEs and PDE-Refiner and report the strongest results among them.

AuthorsOpenReview ↗

Response to reviewer kUds Continued...

Weakness 2 Continued: These observations demonstrate trade-offs within the tested framework, not inherent incompatibility between the objectives.

We agree, and we would like to clarify that we never claim the objectives are mathematically incompatible. This is clearly not the case. The reference PDE solutions themselves have accurate trajectories, preserve the invariants, and dissipate at the correct rate, so all three objectives can hold at the same time. Our claim is that trajectory accuracy, invariant preservation and dissipative-rate fidelity are distinct notions of fidelity, and improving one does not necessarily improve the others (Introduction, l. 035-042; Conclusion, l. 478-480).

The question we study is therefore whether learned surrogates can satisfy these objectives together, and how this can be achieved. We study this systematically. Unrestricted TFM achieves strong trajectory accuracy but violates invariants (Fig.1). Adding hard GENERIC structure restores invariant preservation at some cost in trajectory accuracy (Table 3). Explicit rate supervision then partly recovers the dissipation rate (Sec. 6.4; Fig. 3). This is similar in spirit to APEBench (Koehler et al., 2024), which systematically studies how different emulator architectures and training choices handle different PDE dynamics. Our study does the same along the axes of temporal parameterisation and physical structure.

  1. Felix Koehler, Simon Niedermayr, Rüdiger Westermann, and Nils Thuerey. APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs. NeurIPS Datasets and Benchmarks Track, 2024.

  2. Xianglong Hou, Xinquan Huang, and Paris Perdikaris. CFO: Learning Continuous-Time PDE Dynamics via Flow-Matched Neural Operators. ICLR, 2026.

Reviewer_tuFwOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper shows the trade-offs between accurate trajectory prediction, the preservation of invariants, and the precise estimation of dissipation rates in modeling PDE dynamics. It then introduces a GENERIC-inspired generator, and analyze how the incorporation of physical structure influences the outcomes.

Critical strengths
  • This paper shows the trade-offs between trajectory accuracy, invariant preservation, and dissipation accuracy, which helps explain that a single metric is not enough to give a complete picture of model quality.
  • Trajectory Flow achieves lower nRMSE than the best baseline. GENERIC Flow also reduces invariant drift by several orders of magnitude.
Critical weaknesses
  • The novelty of the GENERIC Flow is not very significant compared to existing methods since GENERIC constraints and flow-matching methods are already established.
  • In Table 1, Flow uses 16 evaluations per step and achieves low one-step error than the direct predictors, which makes it difficult to isolate the benefits of continuous-time modeling.
Questions for authors
  • Could you better illustrate the novelty of this paper?
  • Does Flow still outperforms when one-step error or inference cost is matched?
  • Does rate supervision still help when both variants use the same slack projection?
3 comments
AuthorsOpenReview ↗

Response to reviewer tuFw

Thank you for taking the time to review our submission and providing valuable feedback. We greatly appreciate your insightful comments and suggestions, which have helped us improve the quality and clarity of our work.

Weakness 1 and Question 1: The novelty of the GENERIC Flow is not very significant compared to existing methods, since GENERIC constraints and flow-matching methods are already established.

We have cited prior works that used GENERIC in the Related Work (Sec. 2 l.124-126 mentions GFINNs; Zhang et al., 2022 and MCFM; Baheri & Lindemann, 2025). However, trajectory flow matching (TFM) for surrogate physics solvers is a relatively recent technique.

We would like to mention that combining TFM with GENERIC is non-trivial. As discussed in l. 270-274, restricting the entire learned vector field to the GENERIC form is too rigid, and optimisation under these structural constraints is difficult (l. 291-292). We therefore added flexibility components (the tangent and slack branches, l. 274-278), an endpoint loss (l.279-292), and a rollout-aware curriculum (l. 293-307) to make training stable while keeping trajectory accuracy. The endpoint-loss ablation (App. I.2, Tables 19-20) shows that this term is needed to preserve trajectory accuracy under the structural constraints.

Since our main intention was to use the GENERIC framework to enforce structure in ordinary TFM, we also performed the required ablations to check where the invariant preservation actually comes from. Both were included in the original submission (App. G–H there; App. H–I in the revised version) and are summarised in Section 5 (l. 308–315):

  • A TFM model trained with the same rollout-aware curriculum, with and without additional soft physics losses, still violates the invariants in 45.85-97.28% of updates, compared with 0-0.127% for GENERIC Flow (App. H, Table 16).

  • Removing only the null-space projections in the structured core, while keeping the flexibility components and the full training pipeline, increases invariant drift by about five orders of magnitude on Burgers and KdV (App. I.1, Tables 17-18). So the invariant preservation comes from the hard GENERIC structure, not from the flexibility components or the rollout training.

To the best of our knowledge, no prior work combines GENERIC with TFM over a large range of PDE families, including a 2D setting, achieves stable training, and performs these ablations. As described in the general comment, we use GENERIC Flow as an investigative tool to study the trade-off between trajectory accuracy and physical fidelity, not as the main contribution of the paper.

Weakness 2 and Question 2: In Table 1, Flow uses 16 evaluations per step and achieves lower one-step error than the direct predictors, which makes it difficult to isolate the benefits of continuous-time modelling.

The required ablations were already included in the original submission, in Appendix B (l.833-909 in the revised version, the appendix and table numbers are unchanged). We study two controls there.

First, in Table-7, we compare the direct one-step predictor with TFM using a single Euler step, so that both methods use exactly one neural-function evaluation (NFE) per step. Even with this matched budget, TFM achieves lower nRMSE@100 and longer accuracy horizons on all five PDE families and for both CNN and FNO backbones. For example, on Advection with CNN, nRMSE@100 drops from 14.749 to 0.1270 (in units of 10−210^{-2}), and the accuracy horizon increases from 362.5 to 2000 steps. Table 8 further shows that higher-order solvers (midpoint, RK4) give only a small additional improvement over one-step Euler. The main advantage therefore comes from the continuous-time parameterisation, not from the extra evaluations.

Second, we study the wall-clock time in Table 9 (discussed in App. B.2). With the CNN backbone on Burgers, the direct one-step predictor takes 1.16042,s and one-NFE TFM takes 1.16021,s, so the runtime is almost identical while TFM keeps its large trajectory-fidelity advantage.

In the revised version, Section 6.2 now points to them directly (l.368-370).

  1. Zhen Zhang, Yeonjong Shin, and George Em Karniadakis. GFINNs: GENERIC Formalism-Informed Neural Networks for Deterministic and Stochastic Dynamical Systems. Philosophical Transactions of the Royal Society A, 380(2229), 2022.

  2. Ali Baheri and Lars Lindemann. Metriplectic Conditional Flow Matching for Dissipative Dynamics. 2025.

AuthorsOpenReview ↗

Response to reviewer tuFw Continued..

Question 3: Does rate supervision still help when both variants use the same slack projection?

We thank the reviewer for this question. We trained a control with the same slack projection but without the rate losses, using the same two seeds and training recipe. The ratios below compare GENERIC Flow + Rate with this projection-only control (values below one indicate improvement):

PDEFunctional trajectory errorFunctional increment error
Advection1.1370.984
Heat0.9961.021
Burgers0.4030.559

With the same projection, rate supervision clearly helps on Burgers, where it reduces the functional trajectory and increment errors to 0.40×0.40\times and 0.56×0.56\times. On Advection and Heat, the effect is smaller. We have added this control to the revised version (App. I.3, Table 21), with a pointer from Section 6.4 (l.461-462).

We hope that our revisions have resolved the issues you raised. If you find our responses satisfactory and the changes adequate, we kindly ask you to consider reflecting this in your overall evaluation.

Thank you once again for taking the time to review our submission. We understand that the review process is demanding, and we truly appreciate the effort and time you dedicate. However, we noticed that there hasn’t been a response to our rebuttal or clarification. We would kindly like to follow up to see if there is anything further we can clarify or provide to assist in the review process. We are happy to address any concerns you may have regarding the rebuttal.

If you find our responses satisfactory and the changes adequate, we kindly ask you to consider reflecting this in your overall evaluation.

Reviewer_Vy77OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper systematically studies the long-term prediction error caused by the local error injected at each time step and the propagation error caused by the learned one-step transition. Based on the GENERIC principle, the authors develop the structured vector field with extra flexibility, which automatically preserves the physical laws of underlying systems. Numerical results further validate the efficacy and the advantages over the existing methods of the proposed method.

Critical strengths
  1. The introduction of GENERIC principle to the construction of the neural odes is a novel and interesting idea, which endows physical rules to the parametrised vector fields instead of solely force the neural network fits data.

  2. The performance of the proposed GENERIC FLOW achieve better performance than the existing methods.

Critical weaknesses
  1. The current method is not an integral-free method as Trajectory Flow since it needs to calculate the transition flow FθF_\theta during training stage, which may lead to high computational complexity when applied to realistic scenarios instead of the toy equations considered at present.

  2. The numerical experiments mainly focus on classic PDEs instead of real data, I do not see the practical value from the current manuscript.

Questions for authors

Due to the introduction of GENERIC structure, can the proposed method discover new physical laws instead of just preserves known physical laws?

4 comments
AuthorsOpenReview ↗

Response to reviewer Vy77

First of all, we would like to thank you for appreciating our work. We would also like to sincerely thank you for your detailed review and constructive feedback, which has helped us enhance the presentation of our submission.

Weakness 1:The current method is not an integral-free method as Trajectory Flow since it needs to calculate the transition flow during training stage, which may lead to high computational complexity when applied to realistic scenarios instead of the toy equations considered at present.

We agree that GENERIC Flow is not fully integral-free. The standard TFM model in our study is integral-free (Sec. 3.2). GENERIC Flow adds two terms that go through the integrator. However, the cost of both is limited:

  • Endpoint loss. This uses a single explicit midpoint step, which is only two neural-function evaluations (App. D.2, Eq. 12-14). It does not backpropagate through an adaptive solver or a long trajectory, as in standard Neural ODE training.
  • Rollout-aware training. This uses only 20 rollout updates per epoch, compared with 400 pointwise updates (App. A.1, l. 699-701). The rollout horizon follows a curriculum up to 64 steps.

We use these terms because the hard structural constraints make optimisation difficult. Without the endpoint loss, trajectory accuracy drops noticeably (App. I.2, Table 19). Rollout-aware training is also a standard choice for autoregressive surrogates, and the multistep direct baselines in our study unroll in the same way (Sec. 3.1, l. 151-152).

At deployment, GENERIC Flow uses one midpoint substep (2 neural-function evaluations) per stored step, while the main TFM model uses eight substeps (16 evaluations) (App. A.1). In our controlled wall-clock measurement, GENERIC Flow costs 2.68×2.68\times the direct one-step predictor per step, compared with 2.01×2.01\times for TFM at the same two evaluations (App. B.2, Table 9). The extra cost comes from the structured branches and projections.

AuthorsOpenReview ↗

Response to reviewer Vy77 continued...

Weakness 2:The numerical experiments mainly focus on classic PDEs instead of real data, I do not see the practical value from the current manuscript.

We use classic PDEs deliberately, because our evaluation needs quantities that real data usually cannot provide:

  • known invariants, to measure invariant drift.
  • known dissipative functionals, to measure the dissipation rate.
  • an exact reference solver, to measure the local defect from any state, including the model's own rollout states (App. C).

Without these, the separation between trajectory, invariant and dissipative-rate fidelity, which is the main finding of the paper, cannot be measured. This follows standard practice in the literature on neural PDE surrogates.

Neural operators such as FNO were introduced and evaluated on Burgers', Darcy and Navier-Stokes equations (Li et al., 2021), and message-passing PDE solvers on Burgers', KdV and wave-type equations (Brandstetter et al., 2022). Benchmarks for autoregressive emulators and PDE surrogates, such as APEBench (Koehler et al., 2024) and PDEBench (Takamoto et al., 2022), are likewise built on classic PDE families including advection, diffusion, Burgers' and KdV. Structure-preserving models are also typically evaluated on systems with known invariants or thermodynamic structure (Greydanus et al., 2019; Zhang et al., 2022; Yu et al., 2021), as are studies of invariant drift in learned surrogates (McGreivy & Hakim, 2023). We include five qualitatively different PDE families and a 2D setting (Sec. 6.1), and we acknowledge in the limitations (l. 481-483) that broader multidimensional validation is an important direction.

We believe the practical value of the paper lies in the trade-offs we observe for popular PDE surrogate methods. When building a surrogate, a common choice is between direct one-step predictors (CNN, FNO), the same predictors with stronger training (multistep or Jacobian regularisation), and continuous-time models such as TFM. The right surrogate depends on what the application needs. Physical consistency has to be checked separately, because good trajectory accuracy does not guarantee it.

  1. Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier Neural Operator for Parametric Partial Differential Equations. International Conference on Learning Representations, 2021.

  2. Johannes Brandstetter, Daniel E. Worrall, and Max Welling. Message Passing Neural PDE Solvers. International Conference on Learning Representations, 2022.

  3. Felix Koehler, Simon Niedermayr, Rüdiger Westermann, and Nils Thuerey. APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs. NeurIPS Datasets and Benchmarks Track, 2024.

  4. Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. PDEBench: An Extensive Benchmark for Scientific Machine Learning. NeurIPS Datasets and Benchmarks Track, 2022.

  5. Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian Neural Networks. Advances in Neural Information Processing Systems, 2019.

  6. Zhen Zhang, Yeonjong Shin, and George Em Karniadakis. GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 380(2229), 2022.

  7. Haijun Yu, Xinyu Tian, Weinan E, and Qianxiao Li. OnsagerNet: Learning Stable and Interpretable Dynamics Using a Generalized Onsager Principle. Physical Review Fluids, 6(11):114402, 2021.

  8. Nick McGreivy and Ammar Hakim. Invariant Preservation in Machine Learned PDE Solvers via Error Correction. ICLR Workshop, 2023.

AuthorsOpenReview ↗

Response to Vy77 continued..

Question 1: Due to the introduction of GENERIC structure, can the proposed method discover new physical laws instead of just preserves known physical laws?

We thank the reviewer for this question. In its current form, GENERIC Flow does not discover new physical laws. The energy-like invariant EE and the dissipative functional SS are declared from known physics and the model learns how the state evolves consistently with them. Our goal is to test whether imposing known structure improves invariant preservation and how it affects trajectory accuracy and the dissipation rate, which requires the invariants to be known so that their drift can be measured.

Extending the framework to discovery is possible in principle, by parameterising EE and SS with neural networks. However, this raises an identifiability problem: many combinations of (E,S,L,M)(E, S, L, M) generate the same dynamics, so learned functionals would not be uniquely determined from trajectory data without additional assumptions. Moreover, even when such functionals are learned, it may be difficult to interpret them as new physical laws when they are represented by neural networks. Addressing these challenges, for example through additional structural assumptions or interpretable parameterisations of EE and SS, is a promising direction for future work.

We hope that our revisions have resolved the issues you raised. If you find our responses satisfactory and the changes adequate, we kindly ask you to consider reflecting this in your overall evaluation.

Thank you once again for taking the time to review our submission. We understand that the review process is demanding, and we truly appreciate the effort and time you dedicate. However, we noticed that there hasn’t been a response to our rebuttal or clarification. We would kindly like to follow up to see if there is anything further we can clarify or provide to assist in the review process. We are happy to address any concerns you may have regarding the rebuttal.

If you find our responses satisfactory and the changes adequate, we kindly ask you to consider reflecting this in your overall evaluation.

Comments on the paper

1 comment

We are deeply grateful to the reviewers for their effort and time spent in reviewing our work. We value their critical and insightful comments, which have greatly helped us improve the quality of our work. Furthermore, we appreciate the suggestions that have enabled us to improve the organisation and the presentation of our paper.

We would first like to clarify the central contribution of our work. The unifying research question is how different forms of time parametrisation and structural constraints affect PDE surrogates under autoregressive deployment, both in terms of long-horizon accuracy and physical consistency.

Prior work reports that direct one-step predictors often have poor long-horizon accuracy under autoregressive deployment (l. 035-037). More recently, Continuous Flow Operators (Hou et al., 2026; discussed in l. 112-115) showed that Trajectory Flow Matching (TFM) can substantially improve this accuracy. However, CFO does not investigate why continuous-time parameterisation may be better than direct one-step prediction. The first part of our paper addresses this gap.

The paper follows three connected questions:

  1. How does temporal parameterisation affect long-horizon trajectory accuracy? (Sec. 4.1, 6.2) We use the error-propagation framework (Eq. 2) to identify the two quantities that govern long-horizon error: the local transition defect and the input-Jacobian amplification. We measure both (Table 1) and explain how a learned vector field couples them (Eq. 3). Stronger training improves the direct predictors, but TFM keeps better long-horizon accuracy across the tested settings (Table 2).

  2. Does better trajectory accuracy imply physical consistency? (Sec. 4.2, 5, 6.3) It does not. TFM still violates conserved quantities (Fig. 1). We therefore use GENERIC as an investigative tool to study whether hard physical structure improves invariant preservation, and how it affects trajectory accuracy. Even though GENERIC constraints were considered earlier (as we have mentioned GFINN, MCFM in Sec. 2), incorporating it into TFM was non-trivial, which is why Section 5 describes the model choices in detail, with ablations.

  3. Do preserved invariants and admissible dissipation give the correct dissipation rate? (Sec. 6.4) They do not. Explicit rate supervision only partly recovers it.

These are the three connected questions which we study systematically. We study how the most widely used and effective model classes for PDE surrogates handle these notions of fidelity. They are examined across advection, diffusion, nonlinear dissipative, reaction-diffusion and dispersive systems, with multiple physical parameters, in both 1D and 2D (Sec. 6.1; App. A, Table 6).

In the revised version of the paper, we explicitly state the unifying research question in the Abstract (l. 019-024) and the Introduction, which now lists the three connected research questions and answers them in order (l. 050-081). We also present GENERIC Flow as an investigative tool, in the Abstract (l. 022-024), Introduction (l. 074), contributions (l. 082-090) and Related Work (l. 126-129).

Apart from this, we also add the following to the revised version:

  • the Table 1 diagnostics for every training strategy in Table 2, both at reference test states and at each model's own state after 100 autoregressive steps (App. C, Tables 11-12; pointer in Sec. 4.1, l. 220).
  • a pointer in the main text to the NFE- and wall-clock-matched controls in Appendix B (Sec. 6.2, l. 368-370).
  • a matched slack-projection control for the rate-supervised extension (App. I.3; pointer in Sec. 6.4, l. 461-462).
  • the corrected 3-seed Trajectory Flow (CNN) row in Table 2, with the coefficient-level origin of the FNO Advection value in its caption (l. 379-384).
  • Table 2 reformatted to the template font size, with accuracy horizons moved to App. C, Table 10.
  • a corrected Figure 4, disclosed in the AI-use statement, and removal of the leftover template text.

All line, appendix and table references in our responses refer to the revised version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.