acceptodds
Under review as a conference paper at ICLR 2027

UniDynamic: Unified Surrogate Modeling for PDE and SPDE via Flow Matching

Abstract

Pretraining shared models across deterministic and stochastic PDEs requires accommodating both diverse dynamics and different levels of predictive uncertainty. With sufficient observations, deterministic dynamics yield a unique continuation, whereas unobserved future stochastic forcing induces a conditional distribution of possible futures. A controlled Kolmogorov-flow experiment reveals complementary strengths of point prediction for deterministic accuracy and conditional generation for stochastic distribution matching. We therefore introduce UniDynamic, a flow-matching framework that jointly learns both modes across deterministic and stochastic PDEs. A shared backbone, conditioned on multiscale windowed log-signature history representations, supports one-pass prediction and iterative sampling. We construct a paired benchmark with repeated-future DNS ensembles for evaluating conditional moments, marginal distributions, and spatial fluctuations. UniDynamic achieves competitive accuracy across 11 deterministic PDE subsets. On the three stochastic PDEs, it attains the lowest lead-20 marginal energy distance among the compared methods. Furthermore, pretraining improves few-shot performance on unseen equations, while adding deterministic training data improves marginal-distribution accuracy on stochastic Kolmogorov flow and reaction–diffusion. These results support a unified flow-matching framework that combines accurate point prediction, conditional generation, and transfer across equations and regimes.

Primary area:
learning on time series and dynamical systems
Keywords:
Neural operator, Flow matching, PDE Solver

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_eRqrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

My read is that UniDynamic combines point prediction and stochastic sampling in one network. A zero-time branch produces a source-independent point forecast. Multistep flow matching generates possible futures. Log-signature features summarize changes in the observed history. The reported gains look strong, but I'm less convinced by the experimental controls.

Critical strengths

The repeated-future DNS benchmark is the strongest part for me. It evaluates spread and distributions against many possible futures. The numerical checks and reference-size tests add useful credibility.

Those marginal energy-distance gains are substantial. Lead-20 scores improve by roughly 40–62% over the best compared baselines. I also appreciate that the advantage persists at 50 forecast steps.

Critical weaknesses

The full training recipe still feels under-tested. A one-seed encoder ablation doesn't establish why the two-mode model works. I'd want the gate, prediction loss, and flow loss isolated.

Matching parameter counts doesn't settle comparison fairness. Training exposure, checkpoint selection, and sampling cost matter too. I don't get a sufficiently clear account of those controls.

Winning marginal ED doesn't settle joint-distribution quality. Damped-wave fluctuation error is 0.0977, versus DPOT-FM's 0.0645. I'd want stronger tests of spatial and temporal dependence.

Questions for authors

Which components actually earn the improvement? Please isolate the gate, direct prediction loss, and history encoder on the same backbone. I'd want several training seeds.

Were the main baselines trained and selected under matched budgets? Please report loss weights, update counts, checkpoint rules, and seed variability. Measured inference costs would also affect my assessment.

Does removing the relative-L2 term change distributional calibration? I'd want that ablation alongside spatial and temporal dependence tests. Lower marginal ED alone wouldn't settle this for me.

How much uncertainty comes from the unresolved fine-grid state? Please test its effect while keeping the observed history consistent. That would clarify whether the DNS references match the model's forecasting target.

9 comments

Response to Reviewer eRqr (Part 1/3)

We thank Reviewer eRqr for the detailed feedback and for recognizing the value of our repeated-future DNS benchmark. We clarify the roles of the model components, the experimental controls, and the scope of our distributional evaluation below.


1. Component Contributions and Training Objectives (W1, Q1, Q3)

History encoder. We have moved the controlled encoder ablations to the main paper (Section 6.5, Table 4), extending the original stochastic evaluation to deterministic forecasting. The comparisons keep the backbone and training setup fixed while varying history encoding. Relative to the full encoder, DPOT [1] temporal pooling with the same frame encoder increases mean deterministic rollout L2RE by 6.1% and stochastic Energy Distance by 12.9%. Removing multiscale windows or second-order order-sensitive information also worsens the corresponding average results. These experiments isolate the encoder contribution from the choice of generative objective; the controlled encoder study uses one training seed.

Source cancellation versus output gate. The term “gate” previously referred to two different mechanisms. Source cancellation at flow time zero is a structural part of the dual-mode parameterization: it guarantees a noise-independent one-step point prediction by construction, rather than serving as an optional reconstruction enhancement. The detail-adaptive output gate (Appendix C.1) is a separate, auxiliary refinement module. Its ablation improves horizon-averaged normalized ED from 0.011608 to 0.010146 and the reported relative error from 0.097512 to 0.093594. We therefore distinguish this auxiliary contribution from the central dual-mode mechanism.

Prediction, flow, and Relative-L2 losses. The zero-time prediction loss supervises the direct point readout, whereas positive-time CFM loss learns the conditional generative velocity. Both operate on the same shared network; the channelwise Relative-L2 term additionally balances physical-space errors across variables with different magnitudes (Section 4.4; Appendix C.2). Removing the flow loss would remove supervision of the positive-time velocity, so such a model would not be a controlled test of generative quality under the same training task. By contrast, removing direct prediction supervision or Relative-L2 would be useful for isolating cross-objective effects and calibration. Those independent loss-removal and multi-seed ablations are not part of the current controlled study. Our claims concern the observed performance of the jointly trained system, rather than an independently established improvement from every individual loss term.

Reference

  • [1] Hao et al. DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training. arXiv:2403.03542, 2024.

Response to Reviewer eRqr (Part 2/3)

2. Baseline Fairness, Training Protocols, and Inference Costs (W2, Q2)

Comparable baselines. We added a 30M-parameter FNO and PDE-Transformer [1] SC-S (46.6M parameters) to the revised comparisons. Across eleven deterministic subsets, UniDynamic (30.7M) achieves mean full-rollout L2RE 0.04496, versus 0.10327 for FNO (30M) and 0.10703 for PDE-Transformer SC-S. PDE-Transformer is also evaluated on all three stochastic systems; UniDynamic retains lower lead-20 marginal Energy Distance on each (Table 2).

Splits and checkpoint selection. All compared baselines use the same train/validation/evaluation partitions. Both deterministic and stochastic comparisons select the best validation checkpoint. Deterministic experiments use 500 epochs with validation every 50 epochs. In the stochastic setting, UniDynamic uses 96k joint updates, batch size 32, across three systems; separately trained FNO, DPOT [2], DPOT-FM, ACDM, and GenCFD receive 32k updates per system. FNO-CRPS and DPOT-CRPS additionally use point-prediction pretraining followed by CRPS fine-tuning (32k updates for each stage per system). The 96k joint updates correspond to approximately 32k updates of training exposure per system, but do not imply identical GPU compute per update. Loss weights are specified in revised Section 4.4. The motivating DPOT/DPOT-FM study additionally reports mean and standard deviation over three training seeds (Appendix B.3, Table 5).

Measured inference cost. We benchmarked the same single-frame inference setting on an A100 80GB GPU, FP32, batch size 1, using synthetic inputs. For 1-NFE point prediction, UniDynamic takes 12.68 ms versus 42.03 ms for PDE-Transformer. For 32-NFE conditional generation, the respective times are 327.99 ms and 1297.91 ms. Peak measured memory is 0.236 GiB for UniDynamic and 0.214 GiB for PDE-Transformer. Thus, UniDynamic is about 3.3 times faster for point prediction and 4.0 times faster for 32-NFE generation in this measured setting. We distinguish inference-time NFE and wall-clock latency from training-update counts rather than treating parameter matching alone as a complete compute control.

References

  • [1] Holzschuh et al. PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations. ICML, 2025.
  • [2] Hao et al. DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training. arXiv:2403.03542, 2024.

Response to Reviewer eRqr (Part 3/3)

3. Distributional Evaluation and Sources of Uncertainty (W3, Q3, Q4)

Why multiple complementary metrics? We agree that marginal Energy Distance alone does not fully assess spatial and temporal dependence. However, a high-dimensional joint distribution over entire physical fields and multi-step trajectories is difficult to evaluate reliably from finite conditional ensembles. Unlike image or video generation, where perceptual feature-based metrics such as FID/FVD are available, stochastic PDE forecasting has no single broadly established, physically meaningful counterpart that simultaneously validates all spatial and temporal dependencies. Recent benchmarks therefore combine complementary diagnostics rather than relying on one scalar: StocBench [1] evaluates conditional mean, standard deviation, Energy Distance, and long-rollout enstrophy spectra; SPDEBench [2] similarly emphasizes a suite of Sobolev and distributional metrics beyond L2 error. This is the evaluation philosophy adopted in our paper, not an attempt to equate marginal ED with joint-distribution recovery.

Evidence beyond marginal ED. Our repeated-future DNS protocol evaluates conditional mean error, conditional spread error, pointwise marginal ED, fluctuation-spectrum error, and autoregressive behavior (Section 5.3). Appendix E.2 shows individual conditional trajectories; Appendix G.3 (Figures 27–29) compares spatial fields of generated versus DNS ensemble means and standard deviations; Appendix F extends forecasts to 50 steps. UniDynamic's three-system macro fluctuation-spectrum error is 0.0936 versus 0.1345 for DPOT-FM at lead 20 and 0.1180 versus 0.1687 at lead 50. We also report the exception transparently: DPOT-FM has lower damped-wave fluctuation-spectrum error at lead 20 (0.0645 versus 0.0977). The evidence therefore covers several physically relevant aspects of stochastic forecasts, without claiming that any one metric certifies the full spatiotemporal law. Explicit lagged-correlation diagnostics would be a useful further extension.

Uncertainty from unresolved fine-grid states (Q4). We thank the reviewer for this insightful observation. Uncertainty arising from unresolved fine-scale states is an interesting question, also investigated in recent work such as Spitzer et al. [3].

We conducted controlled experiments to examine unresolved fine-grid states magnitude in our setting. Using deterministic Kolmogorov dynamics at 256×256 DNS resolution, we constructed ensembles sharing the same resolved low-frequency states but differing in their unresolved high-frequency components. Over 20-step forecasts (8 initial conditions, 32 members each), normalized pairwise spread was 0 (no measurable divergence) at 128×128, 1.79 × 10^-7 at 64×64, and 5.85 × 10^-4 at 32×32.

We further examined conditioning on 10 consecutive observed frames in the more challenging 32×32 setting (4 initial conditions, 16 members each). Compared with single-frame conditioning, enforcing consistency with the 10-frame observed history reduced future spread by approximately 3–4 orders of magnitude.

Together, these experiments indicate that uncertainty from unresolved fine-grid states is negligible under our tested observation conditions, particularly at the 128×128 resolution used by UniDynamic. Our primary focus is therefore uncertainty induced by unobserved future stochastic forcing, isolated through repeated-future DNS ensembles starting from the same high-resolution state (Section 5.1; Appendix D).

References

  • [1] Pfister, Holzschuh, and Thuerey. StocBench: A Benchmark for Generative Modeling of Stochastic Dynamics. arXiv:2608.22309, 2026.
  • [2] Zhu et al. SPDEBench: An Extensive Benchmark for Learning Stochastic PDEs. arXiv:2505.18511, 2025 (revised 2026).
  • [3] Spitzer et al. Generative Forecasting from Coarse-Grained Observations: When Does Cascading Help? arXiv:2611.00686, 2026.

We appreciate the reviewer’s suggestions, which helped us distinguish the validated claims of the benchmark and unified model from additional possible dependence and component-level diagnostics. We welcome further discussion.

Reviewer_rsc4OpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The work proposes a method for training a unified model for PDEs (a foundation model), trained on several PDE datasets (11 deterministic and 3 stochastic).

The proposed approach extends the existing DPOT deterministic foundation model by combining it with the Conditional Flow Matching (CFM) framework. They unified these two approaches, achieving a single model that produces deterministic and stochastic forecasting; modelling stochasticity is also motivated by the advantage of a lower marginal energy distance (ED) with respect to the deterministic method, capturing variability of the dynamics (not only focusing on the prediction accuracy). When it comes to DPOT, they adapt the modelling of the history patches by integrating some statistics of history states, which include consecutive temporal state distances of the patch and the orientation of the path (clockwise vs. counterclockwise). On the CFM side, they adapt it such that the model can produce deterministic and stochastic output using the same model. The final loss includes the minimisation of the deterministic and stochastic model.

They demonstrate and evaluate the performance of the proposed method on predictions of the dynamics, conditional generations of the stochastic models: accounting for uncertainty, spread and forecasting validations. Finally, they also compared fine-tuning performance against scratch training and the main baseline, DPOT. They used standard benchmark experiments from PDEBench, PDEArena and FNO-v, and introduced other experiments from high-resolution Direct Numerical Simulation (DNS) for Kolmogorov, Reaction Diffusion and Damped wave dynamical systems, assessing stochasticity.

Critical strengths

The critical strength of the paper relates to the experiments and the evaluation protocol, which I appreciated quite a lot.The authors demonstrate substantial gains across a broad set of tasks, and they additionally validate the approach, including some baselines, on conditional generative tasks for stochastic dynamics.

Another interesting point that might support acceptance relates to the method for encoding the history states, but this should be further investigated by some extra ablation studies.

Critical weaknesses

Method

Some parts of the submission are unclear and hard to follow, with undefined notation. The method is also not yet clear or convincing to me yet.

1) Encoding dynamics with the windowed log signature. This section should be explained better, and the notation should be improved to remove doubts.

  • The paragraph on encoding ordered changes is hard to follow. xix_i and xi+1x_{i+1} are not defined. Do they represent the features of consecutive frames, fi(p)f_i(p) and fi+1(p)f_{i+1}(p)? Which dimension do they have? Does fi(p)f_i(p) include all the channels? Should the window satisfy 1≤a<b≤T1 \leq a < b \leq T?
  • Once the notation is clarified, AwA_w should also be explained better. What does it actually capture? Is this matrix tracking the orientation of the path through its sign changes? The statement that "AwA_w contains signed-area terms that capture order-sensitive interactions between increments" should be made more explicit.
  • Since the encoding of the history is one of the paper's contributions, an ablation should be included to quantify its impact against a simple history modelling as in DPOT (where attention is used). This ablation should appear in the main body of the paper, reporting both axes: forecast error and generative performance. I would run it on the deterministic canonical tasks, to assess its effectiveness and to better isolate the generative component. If the encoding turns out not to be effective, this should be reported as well.
  • The last paragraph is somewhat easier to follow, but I would avoid introducing the notation ϕ\phi and instead extend the window notation with the existing subscripts l,jl, j, so that SS and AA become Swl,jS_{w_{l,j}} and Awl,jA_{w_{l,j}}.

2) Unified point prediction and conditional generation. I did not find this modelling straightforward.

  • It is difficult to distinguish rθr_\theta from vθv_\theta in the formulation. Is there a real difference between the two, given that both take the same input? If not, there is no need to define rθr_\theta. Or is vθv_\theta a head layer of the network?
  • Equation (4):
    • The "source-independent" model is not truly source independent, since it still takes noise as input. I would avoid describing it as such.
    • More importantly, this formulation breaks the boundary conditions and assumptions of Conditional Flow Matching and Stochastic Interpolants (Albergo et al., 2025) for the probability path. The resulting flows are not valid, as they do not respect the marginals between the source and target distributions. A simple proof of concept: at time 0 the model gives a deterministic output, but after a small subsequent step it returns to Gaussian noise, which changes the whole ODE integration. Have the authors considered this? I also encourage the authors to consider alternatives for autoregressive flow matching, such as trajectory flow matching (Zhang et al., 2025), or approaches designed specifically for physical dynamics, such as Variational Grey-Box Dynamics Matching (Singh et al., 2026) and CFO (Hou et al., 2025; learning continuous-time PDE dynamics via flow-matched neural operators).
  • The relative-ℓ2\ell_2 error is not well described. Please explain the role of D(y^τ)\mathcal{D}(\hat{y}_\tau) and why it is important. Is this operator ultimately used to learn a drift from the mean and variance statistics of the history?
  • The values of the hyperparameters in the loss are not reported. They are likely to play an important role. If I missed them somewhere in the paper, please point me to them.

Missing Strong Baseline

Among foundation models for PDEs, a closely related work is PDE-Transformer (Holzschuh et al., 2025). It addresses both deterministic and generative modelling for PDEs, using Conditional Flow Matching. All of its models are open source, including the small S variant, which has a similar number of parameters (about 33M).

That work also builds on the DiT architecture and adapts it to PDE systems, defining a transformer that is channel-agnostic. This avoids, for example, redundant channel padding (they call this feature Separate Channel): the channels are first processed independently, and their dependencies are then learned in the embedding space. It also uses patches and attention between non-overlapping local windows.

Therefore, the submission should at least mention this work, discuss the differences (including the configuration settings), and include it as an additional baseline in at least some settings. If adding it as a baseline is not possible in any of the settings, the authors should provide a strong justification for its absence.

Questions for authors

The authors should address all the important points raised above. I am also adding some questions related to the experiments and the method.

Questions

  1. Baseline adaptation. Were the baselines (e.g., FNO, GNOT) adapted to each dataset, or were they run with the same hyperparameters as in the reference works? If they were adapted, how? Note that some baselines in this category have a number of parameters that is more than 11 times smaller than that of the foundation model. Thus, i am concerned about the fairness in the comparison between baselines that have such a big gap in parameter complexity. For example in the work of the PDE transformer, they also compared the FNO baseline with their fine-tuned model, but using an FNO of similar model complexity, like 42.72M parameters.

  2. DNS resolution. For the datasets generated from DNS, it is not clear whether the high resolution is present in both the training and the validation/test sets, or whether the high-resolution DNS reference is used only at evaluation time. If it is used only at evaluation time, does this mean that the model is trained on a coarse grid and evaluated on a higher-resolution one? I am referring only to the spatial resolution, not the temporal one.

  3. Rollout lengths. Across the datasets, the number of rollout steps ranges from 4 to 91. A rollout of 4 steps seems very short. Could the authors justify this choice and specify where it applies?

  4. History length vs. rollout in the DNS experiments. In the experiments, the input history is 10 state frames. However, for the DNS experiments the text states: "The main stochastic comparison uses 20-step rollouts against 128 same-history DNS futures". I am confused by this sentence. Is the history made of 128 steps, while the rollout involves only 20 steps? Please clarify what the 128 refers to (e.g., the number of DNS ensemble members or future trajectories sharing the same history) and how the history is defined in this setting.

7 comments

Response to Reviewer rsc4 (Part 1/4)

We thank Reviewer rsc4 for the detailed feedback. We have clarified the method, added controlled history-encoder ablations, and incorporated the requested PDE-Transformer baseline. We address the theoretical and experimental questions below.


1. Clarification and Ablation of Windowed Log-Signature Encoding (W1)

(1) Improved formulation and interpretation. We have substantially rewritten Section 4.2 to clarify the construction of temporal feature trajectories and their log-signature statistics [1, 2]. Specifically, we now explicitly define xi(g)(p)x_i^{(g)}(p) as the feature vector for group gg at frame ii and spatial patch pp, consisting of qq learned feature dimensions and one relative-time coordinate, and di(g)(p)d_i^{(g)}(p) as the difference between consecutive frames. We also simplify the window notation, consistently use SwS_w and AwA_w, and remove the unnecessary intermediate notation Φ\Phi.

To make the second-order statistics more intuitive, we introduce a simple example of two feature sequences with identical endpoints but different orders of intermediate changes. While their first-order statistics are identical, the second-order matrix AwA_w distinguishes them through order-sensitive interactions between feature increments. Its antisymmetric entries correspond to signed areas in pairwise feature projections, rather than a global orientation of the entire trajectory.


(2) Controlled ablations moved to the main paper. We have moved the history-encoder ablations from Appendix A into the main paper (Section 6.5, Table 4) and extended the evaluation to both deterministic forecasting and stochastic generation. The comparison keeps the backbone, training objectives, and evaluation settings fixed while changing the history encoder, helping isolate its contribution from the generative training mechanism. The full encoder achieves the lowest mean error in both settings; this controlled ablation uses one training seed, so we interpret the differences as evidence of average improvements rather than statistical significance. Compared with our full encoder, DPOT temporal pooling using the same frame encoder increases deterministic forecasting error by 6.1% and stochastic generation error by 12.9%, with comparable parameter counts. Further ablations show that replacing signed areas with feature covariance or removing local windows and second-order features degrades aggregate performance in both settings. These results support the benefits of multiscale, order-sensitive history encoding for both deterministic prediction and conditional generation.

References

  • [1] Chevyrev and Kormilitzin. A Primer on the Signature Method in Machine Learning. arXiv:1603.03788, 2016.

  • [2] Morrill et al. A Generalised Signature Method for Multivariate Time Series Feature Extraction. arXiv:2006.00873, 2020.

Response to Reviewer rsc4 (Part 2/4)

2. Unified Point Prediction, Source Cancellation, and Conditional Flow Matching (W2)

We have revised Section 4.3 to clarify the theoretical motivation for source cancellation and its relationship to standard conditional flow matching.


(1) Shared network and source-independent point prediction.

The distinction between rθr_\theta and vθv_\theta is that rθr_\theta denotes the only learnable field network, whereas vθv_\theta is the velocity constructed from its output for ODE integration. They are not separate networks, and no additional prediction head is introduced.

Importantly, source independence refers to the final one-step point prediction, not the initial velocity itself. At τ=0\tau=0, the network evaluates rθ(cH,0,0)r_\theta(c_H,0,0) without receiving the sampled source noise, while the velocity is constructed as vθ(cH,z,0)=rθ(cH,0,0)−zv_\theta(c_H,z,0)=r_\theta(c_H,0,0)-z. Consequently, a single Euler step gives

y^point=e+vθ(cH,e,0)=rθ(cH,0,0),\hat y_{\mathrm{point}}=e+v_\theta(c_H,e,0)=r_\theta(c_H,0,0),

which is exactly independent of ee by construction.


(2) Consistency with conditional flow matching.

We would like to clarify that source cancellation is motivated by the optimal source-boundary velocity of standard conditional flow matching, rather than an arbitrary modification of the probability path. For the linear interpolation zτ=(1−τ)e+τyz_\tau=(1-\tau)e+\tau y, minimizing the expected squared-error flow-matching loss yields

v⋆(cH,z,τ)=E[y−e∣cH,zτ=z].v^\star(c_H,z,\tau)=\mathbb E[y-e\mid c_H,z_\tau=z].

Since z0=ez_0=e and the Gaussian source is independent of the history and target, the optimal velocity at τ=0\tau=0 becomes

v⋆(cH,z,0)=E[y∣cH]−z.v^\star(c_H,z,0)=\mathbb E[y\mid c_H]-z.

This has precisely the same structure as our initial velocity rθ(cH,0,0)−zr_\theta(c_H,0,0)-z, where the shared network receives direct next-state prediction supervision at zero flow time. At positive flow times, the same network learns the conditional velocity through standard flow matching. Thus, the source-boundary parameterization is consistent with the optimal conditional flow-matching velocity under the ideal squared-error objective.

We also clarify the reviewer’s concern regarding multistep ODE integration. For N>1N>1 Euler steps, the first update is

z1=(1−1N)e+1Nrθ(cH,0,0).z_1=\left(1-\frac1N\right)e+\frac1N r_\theta(c_H,0,0).

Therefore, the first step retains dependence on the Gaussian source and does not collapse the generative trajectory to a deterministic prediction. Subsequent integration proceeds using the positive-time velocity field, without resetting the state to Gaussian noise. Only the single-step case N=1N=1 produces exact source cancellation in the final output.

The probability path is unchanged. We train the positive-time velocity with the standard linear-interpolation CFM target; source cancellation is a boundary parameterization, not a new interpolation path or a reset of the ODE state. At time zero, the construction agrees with the optimal CFM boundary velocity. For positive flow times, the velocity is learned from the same standard CFM regression target as in conventional conditional flow matching; no additional discontinuous reset or alternative probability path is introduced.

Response to Reviewer rsc4 (Part 3/4)

(3) History normalization, Relative-L2L_2 supervision, and loss weights.

As described in Section 4.1 and detailed in Appendix C.1, we employ history-only channel-wise normalization to accommodate the different physical scales of heterogeneous PDE variables. For example, velocity may be on the order of 10−110^{-1} while pressure reaches 10210^2, causing unnormalized squared errors to be dominated by larger-scale variables. The operator DHD_H simply restores normalized predictions to physical coordinates. For each channel c, it multiplies the normalized prediction by the history-dependent scale and adds the corresponding history mean. It contains no learnable parameters and does not estimate or learn any physical drift from historical statistics. History statistics set the normalization scale; the conditional velocity is learned through the CFM objective, not constructed as a drift from the historical mean and variance. We use this inverse transformation to compute an unsquared, channel-wise Relative-L2L_2 loss in physical space, complementing normalized MSE by measuring prediction errors relative to each target field’s magnitude. Relative-L2L_2 is also an established training objective in multi-PDE neural operator learning, as adopted by DPOT [1] and NESTOR [2]. All normalization statistics are computed exclusively from the observed history to prevent future information leakage and are recomputed from the updated history during autoregressive inference.

We have additionally specified the loss weights in Section 4.4: λpred=λFM=0.5\lambda_{\mathrm{pred}}=\lambda_{\mathrm{FM}}=0.5, with both objectives optimized on every training batch.

References

  • [1] Hao et al. DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training. arXiv:2403.03542, 2024.

  • [2] Sun et al. NESTOR: A Nested MOE-Based Neural Operator for Large-Scale PDE Pre-Training. CVPR, 2026.


3. PDE-Transformer Baseline and Fairness of Model Comparisons (W3, Q1)

We have added the reviewer-requested PDE-Transformer baseline rather than relying only on the original comparisons. This provides a more closely related CFM-based transformer comparator; the additional comparison results are included in the revised main experimental tables in Section 6.2. We have added PDE-Transformer [1] to the related work discussion (Section 2) and included additional large-scale baselines in both main experimental tables (Section 6).


(1) Relationship to PDE-Transformer. PDE-Transformer is a closely related DiT-based architecture that supports both supervised prediction and conditional flow-matching training. Its Separate Channel (SC) design embeds physical variables independently and models their interactions in the embedding space, while shifted-window attention captures spatial dependencies. In contrast, UniDynamic uses a unified padded-channel interface and explicitly encodes temporal feature evolution through multiscale windowed log-signatures. Our primary methodological distinction is the joint optimization of zero-time point prediction and positive-time flow matching within a single shared checkpoint, enabling both source-independent one-step forecasting and multi-step conditional generation without separate prediction heads. This unified framework also enables knowledge sharing across deterministic and stochastic PDEs, with Section 6.4 (Table 3 and Figure 6) demonstrating the benefits of joint training across regimes and few-shot transfer to new equations. We also clarify that the approximately 33M-parameter PDE-Transformer S configuration corresponds to the Mixed Channel variant (MC-S), whereas the Separate Channel variant used in our comparison (SC-S) has 46.6M parameters.


(2) Additional large-scale baseline comparisons. To address the concern about parameter-count disparities, we have additionally evaluated a 30M-parameter FNO and the 46.6M-parameter PDE-Transformer SC-S against our 30.7M-parameter model. On the eleven deterministic PDE subsets (Table 1), UniDynamic achieves a mean full-horizon L2RE of 0.04496, compared with 0.10327 for FNO (30M) and 0.10703 for PDE-Transformer SC-S. We further evaluate PDE-Transformer on all three stochastic systems (Table 2). Its lead-20 normalized Energy Distances are 0.0247, 0.0110, and 0.0101 on Kolmogorov, reaction–diffusion, and damped waves, respectively, compared with 0.0175, 0.0102, and 0.0057 for UniDynamic. These additional comparisons show that our advantages in deterministic forecasting and stochastic marginal-distribution accuracy persist against substantially stronger and more comparable-capacity baselines.

Reviewer_SnpDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The proposed paper introduces a model to a new form of generalization towards deterministic and stochastic models. Based on a conditional DiT, the proposed method provides both a deterministic and a stochastic prevision based on flow matching. The objectif behind is to keep the best of both worlds. The paper first introduces the model and the different generation processes and then provides some experiments to illustrate its capacities.

Critical strengths
  • I found the paper clear and well written
  • I found the idea of combining deterministic and stochastic forecasting in one model an interesting research question
  • The proposed method is evaluated on a large set of datasets and models, both deterministic and stochastic
Critical weaknesses
  • The paper and title claim dealing with SPDE. Is it to mention a form of parametric setting of PDEs where different forcing terms give different solutions ? If so, what is the difference with the neural operators framework that generate datasets on a same PDE with different forcings terms or initial conditions ?
Questions for authors
  • Could you explain why designing a dedicated encoding of dynamics and not using standard methods ? (eg temporal aggregation, attention or convolution) ? What are the benefits of this encoding technique wrt to existing ones ?
  • Have you studied what is outputted by the deterministic branch when τ>0\tau > 0 ? Is the deterministic prevision consistently the same despite the noise level changing ?
  • In practise, does it really makes sense to have models dealing with both modes ?
  • is the training method adaptable to different model architecture ? Have you considered to directly use the proposed training strategy to existing models ? (or with minimal modifications?)
  • What about the cost of the method ? does the joint training induced an additional cost ? Could you provide training time and memory and inference latency ?
  • Have you compared the results of the deterministic branch with the mean outputted by the stochastic branch ? Are the results consistent ? What is the benefit of have both predictions beyond a cost reduction to have the mode of the distribution ?
4 comments

Response to Reviewer SnpD (Part 1/3)

We sincerely thank Reviewer SnpD for the constructive feedback and for recognizing the value of unifying deterministic and stochastic forecasting. We address the SPDE formulation, history encoding, the two inference modes, architecture compatibility, and computational costs below.


1. Distinction Between SPDEs and Parametric PDEs (W1)

Our three systems follow additive-noise SPDE formulations with prescribed noise laws that shape solution distributions and correlations (Appendix D.1). For sufficiently regular additive noise, fixing its realization permits pathwise PDE solution learning, including with neural operators. Unlike arbitrary variations of deterministic forcing parameters, stochastic forcing follows a prescribed probability law, whose covariance, spatiotemporal correlations, and stochastic increment structure influence the statistical properties of the solution. Accordingly, SPDE modeling concerns not only individual solution trajectories but also their probability distributions, correlations, and long-term statistical behavior. Moreover, future stochastic forcing is unobserved in our forecasting setting. The model receives only physical state history HH and targets the conditional predictive distribution p(y∣H)p(y\mid H), capturing multiple possible continuations and their uncertainty.

To isolate forcing-induced variability, we generate 128 DNS continuations from the same saved high-resolution state, independently resampling only future forcing (Section 5.1). Our controlled experiments further show that point prediction provides better rollout accuracy under deterministic dynamics, while conditional generation better captures marginal distributions under stochastic dynamics (Section 3). These findings motivate our unified framework supporting both forecasting modes. We clarify the SPDE formulation, inference-time inputs, and distinction between pathwise solution learning and conditional distribution modeling in revised Section 2.


2. Why Windowed Log-Signature Encoding? (Q1)

Our motivation is to introduce an explicit temporal inductive bias for learning shared history representations across heterogeneous PDEs, allowing the model to directly access structured information about how physical states evolve. The key insight is that dynamical forecasting requires understanding both the current state and its evolution. First-order log-signatures capture net changes, while second-order signed areas characterize temporal ordering and, with time coordinates, the relative timing of changes. Multiscale windows capture global trends and local transitions, complementing last-frame features to represent both the current state and its recent dynamics.

We have moved our controlled history-encoder ablations from Appendix A to the main paper (Section 6.5, Table 4) and extended them to both deterministic forecasting and stochastic generation. Compared with our full encoder, DPOT [1] temporal pooling using the same frame encoder increases deterministic forecasting error by 6.1% and stochastic generation error by 12.9%, with comparable parameter counts. Further ablations show that multiscale windows and order-sensitive second-order features improve the average performance across both prediction modes.

Reference: [1] Hao et al. DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training. arXiv:2403.03542, 2024.

Response to Reviewer SnpD (Part 2/3)

3. Two Inference Modes, Statistical Consistency, and the Value of Unified Modeling (Q2, Q3, Q6)

(1) Inference mechanism (Q2). UniDynamic uses a single shared network rather than separate deterministic and generative branches. Direct point prediction is defined at flow time τ=0\tau=0, where source cancellation gives

\hat y_{\mathrm{point}}&=e+v_\theta(c_H,e,0)\\ &=e+[r_\theta(c_H,0,0)-e]\\ &=r_\theta(c_H,0,0). \end{aligned}$$ Thus, the point prediction is **exactly independent of sampled source noise by construction**. At $\tau>0$, the same network instead predicts the conditional flow velocity for generative sampling; these outputs are not point predictions evaluated at different noise levels. Both inference modes use the same checkpoint and parameters, without separate prediction branches. **(2) Point predictions versus generated ensemble means (Q6).** Under a pure squared-error objective, the optimal point predictor and the expectation of an ideal conditional generator both equal $\mathbb E[y\mid H]$. Our actual point objective combines normalized MSE with channelwise relative-$L_2$ loss (Appendix C.2). This mixed objective, finite model capacity, optimization error, numerical integration, and finite ensemble sampling mean that the two outputs need not coincide numerically. We additionally **compared them directly**, using the same UniDynamic checkpoint. For each PDE, we evaluated 8 histories, with 128 generated members per history and 32 NFE. We report the L2 distance between the point prediction and generated ensemble mean, normalized by the L2 norm of the DNS ensemble mean: | System | Lead 1 | Lead 5 | Lead 20 | |---|---:|---:|---:| | Kolmogorov | 0.42% | 1.54% | 11.56% | | Reaction–diffusion | 0.35% | 1.12% | 4.70% | | Damped wave | 0.45% | 1.33% | 3.23% | The modes agree closely at one step (below 0.5% for all three systems), but their discrepancy grows during autoregressive rollout. This is not necessarily inconsistent with the conditional-mean interpretation: recursively propagating point estimates generally differs from averaging stochastic trajectories under nonlinear dynamics, and the two procedures condition on increasingly different predicted histories. Approximation and sampling effects may also contribute. **For additional insight into how the generated ensemble compares with the DNS reference**, Appendix G.3 (Figures 27–29) visualizes generated and DNS ensemble mean and standard-deviation fields across all three systems at multiple forecast leads. Appendix E.2 (Figures 17–19) also provides side-by-side examples of generated trajectories and DNS continuations from the same observed histories. **(3) Practical benefits (Q3, Q6).** Unified modeling offers more than a cheaper point estimate: it supports **beneficial transfer from deterministic training data to stochastic distribution modeling**. Repeated-future DNS ensembles are expensive, whereas deterministic trajectories are often more accessible. In Section 6.4 (Table 3), under the same total training-update budget, adding deterministic trajectories reduces stochastic horizon-averaged marginal Energy Distance by **13.99% on Kolmogorov flow** and **11.78% on reaction–diffusion**, despite fewer stochastic training updates. Figure 6 further shows few-shot adaptation benefits, including on unseen PDEs. Together, these findings support data-efficient transfer and adaptation across dynamical regimes.

Response to Reviewer SnpD (Part 3/3)

4. Generalizability to Other Backbones (Q4)

Our dual-mode training strategy is not tied to the DiT architecture; its core mechanism relies on the velocity parameterization and joint training objectives. We have also investigated adapting an existing neural operator architecture. In Section 3 and Appendix B.2, we extended DPOT [1] into DPOT-FM while retaining its AFNO backbone, introducing noisy-state input fusion and flow-time conditioning. This demonstrates that conditional flow matching can be incorporated into an established neural operator architecture beyond DiT.

For existing flow-conditioned architectures, the proposed dual-mode strategy requires only the zero-time source-cancellation construction and its corresponding supervision. For conventional deterministic predictors such as FNO or DPOT, noisy-state inputs and flow-time conditioning must first be introduced, as illustrated by our DPOT-FM adaptation. Thus, the proposed training mechanism is architecture-agnostic under these input-output requirements, although its implementation requires appropriate conditioning interfaces.


5. Computational Cost of Joint Training and Inference (Q5)

Jointly optimizing the prediction and flow-matching objectives increases training cost because both objectives are evaluated on every batch. Under the same three-system stochastic training setup (96k updates, batch size 32, four A100 40GB GPUs), point-only training requires 10.01 GPU-hours, whereas joint training requires 15.12 GPU-hours, a 51.0% increase.

At inference, the same UniDynamic checkpoint supports both modes without retraining. On an A100 80GB GPU (FP32, batch size 1, synthetic inputs), point prediction takes 12.68 ms per frame (1 NFE), while conditional generation takes 327.99 ms per frame (32 NFE), with measured peak inference memory of 0.236 GiB. For comparison, PDE-Transformer [2] requires 42.03 ms and 1297.91 ms for the corresponding modes under the same measurement conditions. Thus, joint training incurs additional training cost but preserves a low-cost one-step inference option alongside conditional generation.


References

[1] Hao et al. DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training. arXiv:2403.03542, 2024.

[2] Holzschuh et al. PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations. ICML, 2025.

We thank Reviewer SnpD again for the helpful questions. These clarifications and additional comparisons explain the roles of stochastic forcing, history encoding, joint training, and the two inference modes. We welcome further discussion.

I thank the authors for their answer to my questions, the additional experiments and the modifications brought to the manuscript. I will maintain my score.

Comments on the paper

2 comments

To All Reviewers: Updated Manuscript

We thank all reviewers for their constructive feedback. In addition to our individual responses, we have revised the manuscript to clarify the formulation, strengthen the experimental comparisons, and make the evaluation protocol more transparent. The main changes, organized by the concerns they address, are summarized below:

  1. @Reviewer SnpD and @Reviewer rsc4: PDE versus SPDE forecasting (Section 2). We explicitly formulate the additive-noise SPDE setting and distinguish learning a pathwise solution with a known forcing realization from forecasting a conditional distribution when future stochastic forcing is unobserved. We also clarify the deterministic limit.

  2. @Reviewer SnpD and @Reviewer rsc4: Windowed log-signature representation (Section 4.2, Eqs. (2)–(3)). We clarify the shared frame features, temporal increments, multiscale windows, first-order changes, and second-order signed-area terms. We explain how temporal ordering and the last observed frame enter the history representation.

  3. @Reviewer rsc4: CFM formulation and source cancellation (Sections 4.3–4.4, Eqs. (4)–(9)). We make the standard interpolation path, conditional flow-matching objective, zero-time optimal velocity, source-canceling construction, and direct prediction supervision explicit. We also clarify the role of the first Euler update and report the training loss weights.

  4. @Reviewer rsc4 and @Reviewer eRqr: Stronger baseline comparisons (Tables 1–2, Section 6.2, Figure 4). We add a matched-scale 30M-parameter FNO and the closely related PDE-Transformer baseline to deterministic comparisons, and include PDE-Transformer in the stochastic comparison. Figure 4 now includes its rollout curves, allowing the main claims to be assessed against stronger alternatives.

  5. @Reviewer SnpD, @Reviewer rsc4, and @Reviewer eRqr: Controlled encoder ablations (Section 6.5, Table 4). We move the history-encoder study into the main text and expand it to cover both deterministic rollout error and stochastic lead-20 Energy Distance. The comparisons include DPOT-style temporal pooling, removal of local windows or second-order features, and replacement of signed areas with covariance features. Appendix A specifies the controlled setup.

  6. @Reviewer rsc4 and @Reviewer eRqr: DNS and forecasting protocol (Sections 5.1 and 6). We clarify the high-resolution DNS generation and the model observation grid, and explicitly distinguish the 10 observed history frames, 20 forecast steps, and 128 independent same-history DNS futures used for stochastic evaluation. This makes the conditioning and reference-ensemble construction easier to verify.

We sincerely thank all reviewers for their constructive feedback, which has helped us improve the quality of our manuscript.

To All Reviewers: Second Manuscript Update

We have uploaded a further revision that adds the analyses from our follow-up responses. All new content is in the appendix and marked in blue; the main text is unchanged.

  1. @Reviewer rsc4: Integration from source to target (Appendix C.3, Figure 8). A new subsection examines whether the zero-time step is consistent with the rest of the flow: the actual 32-step Euler trajectory, the integrated marginals against the CFM reference along the flow, and the terminal distribution under step refinement.
  2. @Reviewer eRqr: Temporal dependence and matched controls (Appendix H, Tables 13–17). Appendix H.1 reports the trajectory Energy Distance and the lag-correlation error, together with the DNS and shuffled controls (Tables 13–14). Appendix H.2 reports the matched controls for the two Relative-L2 terms, joint training, and the zero-time parameterization (Tables 15–17, corresponding to Tables A–C in our follow-up response).
  3. Figure numbering. The new Figure 8 shifts the subsequent appendix figures by one. Figures 17–19 (Appendix E.2) and 27–29 (Appendix G.3), cited in our first-round responses, are now Figures 18–20 and 28–30.

We thank the reviewers again for the continued discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.