acceptodds
Under review as a conference paper at ICLR 2027

A Shortcut to Statistically Steady-State Turbulence with Flow Matching

Abstract

Many nonlinear physical systems exhibit an initial transient phase in which perturbations grow before nonlinear interactions lead to a statistically steady state. While this saturated regime is of primary interest, direct numerical simulations must resolve the full transient dynamics to reach it, incurring significant computational cost. In Computational Fluid Dynamics, reduced-order approaches such as Large Eddy Simulation mitigate computational cost by modeling small-scale dynamics, enabling tractable approximations of turbulent flows. In contrast, for systems such as gyrokinetics, comparably effective closures for the full dynamics are not generally available, and high-fidelity simulations remain necessary. Existing surrogate modeling approaches for these systems are autoregressive, hence they suffer from accumulating error. We instead propose to bypass explicit time evolution by directly modeling the distribution of saturated states under an ergodicity assumption, stating that ensemble averages over samples are equivalent to time averages of a single long simulation. We introduce GyroFlow, a latent generative model that directly estimates steady-state statistics of gyrokinetic turbulence in 5D phase space, without resolving the transient phase. At the same time, GyroFlow generates saturated snapshots from noise, conditioned on dimensionless operating parameters and matches the strongest autoregressive and tabular baselines, without requiring a saturated solver state, while outperforming other generative approaches. GyroFlow attains a speedup compared to nonlinear gyrokinetics simulators and speedup compared to autoregressive baselines. To evaluate generation quality we propose , a distributional metric computed in the latent space of a pretrained gyrokinetic model, and show that it correlates with downstream flux accuracy and the spectral divergence of warm-started solver runs. Finally, GyroFlow can be used to warm-start the numerical code used to produce the data, which shortens its heat-flux transient by about .

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
flow matching, latent generative models, gyrokinetics, plasma turbulence, fusion, scientific machine learning, surrogate modeling, statistically steady state, ergodicity, turbulent transport

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_da54OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors present a generative learning approach to generate snapshots for 5D gyroscopic turbulence, which is described by a nonlinear kinetic equation and is applied in fusion reactor design. This equation is known to show chaotic behavior. The fact that it is solved in five dimensions makes it computationally difficult to treat. Therefore the benefit from using generative AI surrogates is especially high.

The authors argue that learning a time dependent trajectory is not required, as evaluation can be directly conducted on sampled snapshot data, as in the presence of chaos one can appeal to ergodic hypothesis (1).

The authors train a flow matching algorithm using the approach from stable diffusion, that is training an VAE an then generating with FM on the latent representations. The training is conducted on data provided by Paischner et al 2025 which contains 250 trajectories with four parameters representing operating conditions. The evaluation and comparison with baselines is conducted based on the Frechet distance and on physical downstream tasks.

The authors find dramatic speedup when comparing to classical simulation and a significant one when compared to other machine learning models with about the same accuracy. Testing is performed in and out of distribution.

Critical strengths
  • The fact that gyrokinetics is a 5D partial differentioa equation problem requires considerable attention, as other physics based machine learning is often executed on 3D partial differential equation. This pushes the boundary of physics machine learning.
  • Gyrokenetics is a much less frequently studied problem in physics machine learning and therefore of interest. The paper here makes a very professional impression, representing good physics understanding and professional machine learning at a good level of ML technology.
Critical weaknesses
  • One of the main arguments, the 'shortcut' (1) using the ergodic hypotesis, is not completely new. See Drygala et al, Generative Modeling of Turbulence, Physics of Fluids 34 (2022), eq. (3) and Theorem 1 including a detailed theoretical analysis for the statistical learning theory of GAN on ergodic data.
  • The model is faster, but not a big leap forward against the GyroSwirn baseline, which comes with similar precision metrics in average over all tests. Also other components of the GyroFlow model, the GyroSwirn autoencoder or the DiT seem to be rather recycled that new breaktrough designs.
Questions for authors

Can you better elaborate on the novelties of your model design, in case this goes significantly beyond the combination of known modules.

3 comments

Thank you for the positive assessment of the physical scope and of the relevance of 5D gyrokinetics to physics machine learning.

W1: Ergodicity is not new

We did not claim that generative modeling relying on the ergodicity assumption is our novelty. The main novelty of our work is the development of a methodology that scales to one of the most computationally demanding frontiers of modern science. In gyrokinetics each snapshot has about 2.2107 degrees of freedom on a 5D grid, larger than a 4K image and over 100 the 256256 images on which latent diffusion transformers are usually benchmarked. This requires careful design decisions (folding into channels, zonal flow separation, -wise normalization and attention stabilization).

Furthermore, previous methodologies like GyroSwin are limited as they require a saturated snapshot to start their rollout from, hence its applicability in practice is limited as typical plasma scenario evaluation requires running thousands of simulations. We leverage the ergodicity assumption which enables sampling saturated 5D snapshots from noise which has not been shown in any prior work. We thank the reviewer for raising additional support (Drygala et al. (2022)) on ergodicity for turbulent dynamical systems.

W2: GyroSwin

While accuracy of GyroFlow is comparable to GyroSwin, the main improvement is in terms of efficiency. GyroSwin in its published form has limited applicability in practice for reactor design as it requires an initial saturated snapshot to start from. GyroFlow sidesteps this issue and is 4000x more efficient than GyroSwin at similar accuracy. GyroFlow for the first time provides an efficiency-accuracy trade-off that surpasses commonly used reduced-order models and neural surrogates to be integrated and used in transport models, optimization loops and large parameter scans, where GyroSwin would be too expensive.

W2/Q: Novelties in model design

Our core novelty does not lie in model design, but in providing a functional generative model for 5D plasma turbulence with an improved accuracy-efficiency trade-off. Furthermore, measuring generation quality in 5D latent space and demonstrating practical utility of solve speedup via warm-starting are valuable contributions for practitioners and have not been shown in any prior work to the best of our knowledge.

Concretely, our contributions are:

  • a conditional generator of saturated 5D turbulence that, by skipping the transient and the rollout, is over 1000 faster than autoregressive surrogates such as GyroSwin at comparable accuracy;
  • FGyD, a distributional metric for 5D gyrokinetics that correlates with downstream accuracy;
  • generated states that warm-start the solver, shortening the transient by about 3.
Reviewer_da54OpenReview ↗

Degrees of freedom - how is this defined? The Frechet distance is rather standard...?!

Dear authors, thank you for the reply. I don't understand the 2.2107 (?!) degrees of freedom on a 5D grid. Taking the 2.2107^(1/5) you end up with ~7.4?! Is that the latent space or the decoded space? If 22107 is correct in latent space and you re-used the Autoencoder from prior work, your generative part with a standard DiT processing this number of parameters seems not to be a really a too big problem to solve, but this seems strange and I might have gotten something wrong. So where was the the particular novelty? E.g. something like 5D positional encoding?

With FGyD aka the good old Frechet distance as a metrics in latent space - it is only relevant in the world defined by your frozen auto-encoder from GyroSwirn and it is not a truly physical metrics, as the AE has some kind of bias. So I see this as a weaker metric compared to Frechet in original state space. But even this metric does only process 2nd order moment information which does not fix the distribution and I'm not a fan of it, especially in the context of physics where exactness counts.

I like the warm-start thing and the evaluation protocol that follows up with it.

Thank you for the quick answer. Sorry about the confusion on the degrees of freedom, 2.2107 meant 2.2e7 counted in physical space. Each snapshot is a complex 5D field on a (vpar, mu, s, kx, ky) grid of 32 x 8 x 16 x 85 x 32 = 2.2 * 10^7 values. The autoencoder is not reused from prior work (it shares the swin nd blocks of GyroSwin, different architecture). For completeness, the latent is 7e4 values. Learning this compression stably at this scale required the tricks listed in other answers (QK norm, mu into channels etc).

Yeah you are right on FGyD, it compares only the mean and covariance of the features, so equal values do not guarantee equal distributions. However, the features are nonlinear functions of the field, so these two moments also reflect higher-order statistics of the field, as in FID. Frechet in the original space is not feasible (samples<<2.2e7). As in FID, a pretrained feature space is needed, and for gyrokinetics the only existing one is GyroSwin (kind of like a UNet wiht multiple decode branches). This model is trained to autoregressively predict the fields and the heat flux, so we use its features (not from our autoencoder) with the assumption that they encode physics. FGyD is never used alone. In all our evaluations we measure accuracy on physical quantities (heat flux, spectra etc) alongside FGyD and the two correlate. The FID tracks the flux error and the warm start KS distance (Fig 3), which compares full distributions per mode. Physical metrics remain the primary evaluation.

Reviewer_gSB8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces GyroFlow, a conditional generative framework designed to directly synthesize statistically saturated states of five-dimensional gyrokinetic turbulence. Rather than explicitly simulating the costly transient dynamics, the method learns a conditional distribution of saturated states from operating parameters. It combines a Swin-based autoencoder with a conditional Diffusion Transformer trained via rectified flow matching. The paper also introduces Fréchet GyroSwin Distance (FGyD) to assess generation quality in a physics-informed latent space and explores the use of generated states to initialize nonlinear gyrokinetic simulations. The reported results demonstrate substantial inference speedups and an approximately threefold reduction in the time needed to meet the heat-flux saturation criterion.

Overall, I find the problem important and the proposed framework promising, particularly in its potential to accelerate scientific simulations. However, several concerns regarding the conditional OT formulation, dataset evaluation protocol, and warm-start validation need to be addressed to establish the reliability and practical advantages of the approach. If the authors can satisfactorily resolve these concerns and provide the necessary clarifications and supporting evidence, I would be willing to revise my assessment toward acceptance.

Critical strengths
  1. The paper addresses a major computational bottleneck in gyrokinetic turbulence simulations: the high cost of reaching a statistically saturated regime. Modeling the conditional distribution of saturated states provides a promising alternative to conventional time-dependent simulation, particularly when the primary goal is to estimate steady-state observables.
  2. Applying latent generative modeling to full 5D gyrokinetic phase-space data is a technically meaningful contribution. The proposed framework combines established generative modeling components into a unified pipeline for conditional state generation, extending their application to a challenging scientific computing problem.
  3. The paper goes beyond evaluating generated samples in isolation by testing their utility as initial conditions for numerical simulations. Across 33 operating conditions, the reported median time required to meet the heat-flux saturation criterion decreases from 90 to 24 simulation time units. This provides encouraging evidence that the generated states can reduce simulation costs in practical settings.
Critical weaknesses
  1. The paper does not specify whether OT matching respects operating conditions. Cross-condition matching could alter the conditional source distribution and create a mismatch with Gaussian initialization at inference. Clarification and a comparison with independent pairing are needed.
  2. The main paper and appendix report different test-set sizes. The authors should clarify the data partitions used for each experiment and ensure that the reported results are reproducible.
  3. The speedup over cold starts is promising, but comparisons with simpler initializations are missing. Testing nearest-neighbor states and alternative convergence criteria would better establish the benefit of GyroFlow.
  4. The framework relies largely on established generative modeling techniques. Since the appendix shows that latent EDM performs competitively on some metrics, broader comparisons are needed to establish the specific benefits of flow matching in generation quality, efficiency, and solver initialization.
Questions for authors

How are samples assigned during minibatch optimal transport when operating conditions vary within a batch? Please clarify how the coupling preserves consistency between the training and inference distributions.

Could the authors clarify the dataset partitions used across the main experiments and appendices, particularly the differences in the reported numbers of ID and OOD test cases?

How does GyroFlow compare with initializing the solver from the nearest available saturated snapshot in parameter space, using the same time-to-saturation evaluation protocol?

3 comments

Thank you for the constructive review and for the clear direction on what evidence would help.

W1/Q1: Minibatch OT and conditions

We are very thankful to the reviewer, because while checking this we found that due to a indexing bug minibatch OT resulted in a no-op in our trained models (neugk/diffusion/run.py, line 564). The reported GyroFlow therefore effectively used independent pairing. To provide results on OT pairing, we adopt it as in Tong et al. (2024), who report straighter paths and fewer integration steps.

We corrected Sections 4.3 (and Appendix) accordingly, since independent pairing outperforms minibatch OT in our case. The table below shows the supporting results.

CouplingQ RMSE IDQ RMSE OOD
GyroFlow19.020.4
OT, whole batch24.227.7
OT, within condition28.932.4

W2/Q2: Dataset partitions

Thank you for flagging this inconsistency. We upsampled the original dataset used in Paischer et al. (6 ID, 5 OOD simulations) to 31 ID and 28 OOD simulations which we used throughout the paper. We updated Appendix B and Table 10 accordingly in the revision.

W3/Q3: Nearest snapshot and convergence criteria

Thank you for the suggested addition. We added a comparison to the nearest-snapshot baseline w.r.t. the time-to-saturation protocol of Section 5.3 (and Appendix H). We take the closest operating conditions from the training set and start the solver from three of its saturated snapshots, which resembles sampling with GyroFlow. Saturation is determined if the mean flux over the last 24 time units lands inside the reference 95% band, as stated in the paper. Everything else (400 time units per run, last 80 GKW flux points as reference) is unchanged. We add paired speed-up and the count of runs saturating quicker than their cold run. In the tables below we report median time to convergence and the percentage of runs for which the criteria is met at different timestamps for the original cold-start and both nearest snapshot and GyroFlow.

In distribution:

ColdNearest snapshotGyroFlow
Median time to criterion ↓902424
Criterion met (by t=24) ↑0.0%64.6%74.7%
Criterion met (by t=96) ↑72.7%93.9%94.9%
Relative flux error (t=96) ↓47.9%18.3%18.2%
Flux KS distance (t=96) ↓0.7350.4590.412
Spectral KS distance (t=96) ↓0.4790.4490.377

Out of distribution:

ColdNearest snapshotGyroFlow
Median time to criterion ↓94.82424
Criterion met (by t=24) ↑0.0%56.8%55.6%
Criterion met (by t=96) ↑51.9%82.7%86.4%
Relative flux error (t=96) ↓49.4%17.5%18.6%
Flux KS distance (t=96) ↓0.7820.5310.497
Spectral KS distance (t=96) ↓0.5310.3880.356

GyroFlow generally starts closer to the true saturated state as indicated by the lower flux and spectral KS. Furthermore, GyroFlow warm-started simulations converge significantly faster than nearest snapshot ones on in-distribution data (at t=24 74.7% are converged compared to 64.6% for nearest snapshot). On OOD simulations a larger percentage (86.4% vs 82.7%) is converged at t=96. Finally, flux error is mostly consistent between both.

W4: Benefit of flow matching

To provide a broader comparison, we compare all three generative approaches in terms of generation quality, efficiency and solver initialization. The different methods share autoencoder and training data. We report results in the table below.

ModelQ RMSE IDQ RMSE OODW(ky) Pearson IDW(ky) Pearson OODQ(ky) Pearson IDQ(ky) Pearson OOD
Latent DDPM29.954.60.9880.8460.8110.668
Latent EDM23.323.30.9880.8490.5680.695
Flow matching19.020.40.9890.8590.9380.874

For completeness, FGyD at all four feature depths (Table 11):

Modelskip L=1skip L=2decoderflux head
Latent DDPM4.801041.271056.50105446.6
Latent EDM61891.021042.1710542.1
Flow matching34741.111042.5710538.2

Flow matching as utilized in GyroFlow gives the lowest Q RMSE and the highest spectral correlations on both splits, as well as the best FGyD at the first skip connection and at the flux head.

Reviewer_gSB8OpenReview ↗

Reply base on authors comments

First, I believe the authors have found a bug in the code, but I find this point insufficiently rigorous, which raises concerns about the other results. In addition, I feel that the novelty of applying Flow Matching to gyrokinetics is not sufficient. Taking both the implementation rigor concerns and the limited methodological novelty into account, I will maintain my current rating.

Thank you for the quick follow up. We disagree with both points.

The bug is isolated to OT pairing as yu can see from the code in the supplementary, and it does not invalidate any other result. The reported numbers are exactly those of the model we trained and released, which used independent pairing, and we have re-run the OT variants to show that independent pairing is in fact the better choice (Table R3). Bugs of this kind happen in complex codebases. We found it because of the reviewer's question and chose to disclose it rather than quietly update the description, and we do not think transparency should be penalized. The rebuttal served its purpose and measurably improved our paper. All other results come from separate pipelines, and to the best of our knowledge they are robust and reproducible: during the rebuttal we independently re-implemented the evaluation and reproduced the GyroFlow results, and the new experiments (ablations, nearest-snapshot baseline, DDPM/EDM comparison) are consistent with the original findings. We remain available for additional checks reviewers may find useful.

On novelty, our reply listed the contributions beyond flow matching. Reducing the paper to just "applying flow matching on gyrokinetics" does not engage with them.

Reviewer_1KCrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GyroFlow, a generative surrogate for gyrokinetic turbulence that directly models the statistically steady state instead of simulating the expensive transient dynamics. By assuming approximate ergodicity, GyroFlow replaces long time averages with ensemble averages from generated saturated states. The method combines a Swin5D autoencoder with a conditional Diffusion Transformer trained using rectified flow matching. Given four operating parameters, it generates full 5D plasma states from Gaussian noise. The authors also introduce FGyD to evaluate distributional quality. GyroFlow achieves competitive physical accuracy, more than 1000 ×\times speedup over nonlinear simulations, and about 3$\times$ faster convergence when warm-starting numerical solvers.

Critical strengths
  1. The paper reframes gyrokinetic surrogate modeling from autoregressive dynamics prediction to direct conditional modeling of the statistically steady-state distribution, thereby bypassing the costly transient phase altogether.

  2. The combination of a Swin5D autoencoder and conditional latent flow matching alignes with the high-dimensional 5D gyrokinetic state space and the need for efficient conditional generation.

3.The paper evaluates not only downstream physical observables, but also OOD parameter scans, distributional quality through FGyD, and proximity to the physical attractor via warm-started solver runs.

Critical weaknesses
  1. The core components, Swin-based autoencoding, DiT, and rectified flow matching, are already exsiting, so the main weakness lies in the methodology novelty.

  2. GyroFlow is built on the assumption that time averages can be replaced by ensemble averages from a stationary distribution; it is unclear how robust the approach would be in regimes with multiple attractors, long-lived metastability, or weaker ergodicity.

Questions for authors

1.Could the authors better elaborate the paper’s novelty and contributions, particularly in distinguishing the proposed problem formulation and system-level contributions from the relatively standard modeling components used in GyroFlow?

2.Could the authors clarify the construction of the evaluation splits? Appendix B states that six trajectories are held out as the ID test set, whereas Table 1 reports results on 31 ID and 28 OOD operating conditions (and Table 10 refers to the “original” six ID and five OOD conditions). It would be helpful to explain how these sets are related, how the additional evaluation conditions were obtained, and whether they are fully disjoint from the training data.

  1. Could the authors provide quantitative ablations for the main stabilization and flow-matching design choices, including per-μ\mu-shell normalization, QK normalization, gated attention, logit-normal time sampling, and optimal-transport coupling? While the appendix provides qualitative motivation for these components, isolating their effects on training stability, downstream accuracy, and generation quality would help establish which components are essential.
2 comments

Thank you for the positive assessment of the problem formulation and of the evaluation, including the OOD parameter scans.

W1/Q1: Novelty

The main novelty of our work is the development of a methodology that scales to one of the most computationally demanding frontiers of modern science. In gyrokinetics each snapshot has about 2.2107 degrees of freedom on a 5D grid, larger than a 4K image and over 100 the 256256 images on which latent diffusion transformers are usually benchmarked. This requires careful design decisions (folding into channels, zonal flow separation, -wise normalization and attention stabilization).

Furthermore, previous methodologies like GyroSwin are limited as they require a saturated snapshot to start their rollout from, hence its applicability in practice is limited as typical plasma scenario evaluation requires running thousands of simulations. We leverage the ergodicity assumption which enables sampling saturated 5D snapshots from noise which has not been shown in any prior work.

Concretely, our contributions are:

  • a conditional generator of saturated 5D turbulence that, by skipping the transient and the rollout, is over 1000 faster than autoregressive surrogates such as GyroSwin at comparable accuracy;
  • FGyD, a distributional metric for 5D gyrokinetics that correlates with downstream accuracy;
  • generated 5D states that warm-start the solver, shortening the transient by about 3.

W2: Ergodicity

Ergodicity is the assumption underlying our methodology. Dynamical systems with weaker ergodicity are therefore not an intended use case, which is a limitation of our approach. However, many dynamical systems exhibit this property, e.g. fully developed fluid turbulence and strongly driven plasma turbulence. Long-lived metastability and multiple attractors can occur near stability thresholds, and has been observed in a few instances when kinetic electrons or electromagnetic effects are included. These regimes are significantly more complex, and data generation alone requires substantial effort, hence we leave them for future work, for which our current paper would provide the only available basis to start from.

Q2: Evaluation splits

Thank you for flagging this inconsistency. We upsampled the original dataset used in Paischer et al. (6 ID, 5 OOD simulations) to 31 ID and 28 OOD simulations which we used throughout the paper. We updated Appendix B and Table 10 accordingly in the revision. Below we report the difference in input parameters for both sets.

SetR/LTR/Lnsq
Train2.60-11.970.00-6.990.51-4.711.55-8.98
ID original (6)6.11-11.890.03-6.160.72-2.145.35-8.68
ID added (25)7.04-11.660.06-1.740.80-1.775.13-8.85
OOD original (5)7.50-11.130.72-1.540.57-1.885.32-8.80
OOD added (23)5.03-11.801.28-6.490.66-4.731.88-8.56

Q3: Ablations

Thank you for the suggested ablations. We report additional ablations for normalization, QK normalization and gated attention in the Autoencoder, as well as time sampling and coupling to the DiT.

AE: training is expensive and rebuttal time limited, so we use 30 random training trajectories (10% of the dataset). Results are shown in the table below.

mu normQK normGated attentionGPU hoursRecon RMSE ID / OODRelative Q error ID / OOD
yesyesyes9.31.05 / 0.768.3% / 9.8%
yesyesno8.11.04 / 0.758.5% / 10.7%
noyesyes10.01.28 / 0.9234.3% / 40.5%
noyesno9.70.97 / 0.6911.7% / 20.5%
anynoany-nannan

mu normalization has the largest impact on flux error. Gated attention (Qiu et al., 2025) has marginal effect on flux and reconstruction error but improves training stability.

DiT: We train on the entire latent set produced by the original autoencoder, for 140 epochs; the GyroFlow row is the reported model. All rows use 15 Euler steps. NOTE: while running this experiments, we found a bug in the code in the way OT minibatch pairing was applied (neugk/diffusion/run.py, line 564). This means the model reported in the paper is effectively without OT (see W1/Q1 to gSB8).

CouplingSamplingQ RMSE ID / OODW(ky) / Q(ky) Pearson ID
paperlogit-normal19.0 / 20.40.989 / 0.938
OTlogit-normal24.2 / 27.70.991 / 0.920
OT, in conditionlogit-normal28.9 / 32.40.990 / 0.879
No OTuniform18.7 / 21.80.990 / 0.915

As shown in the table above, OT does not improve over independent pairing. Furthermore, we provide results on comparing uniform and logit-normal integration timestep sampling. Uniform time sampling degrades with fewer steps, from Q RMSE 18.7 / 21.8 at 15 Euler steps to 25.6 / 25.1 at 10. This is in line with the motivation of logit-normal sampling, concentrating training on the intermediate times that matter most for few-step sampling (Esser et al., 2024), hence resulting in faster integration during inference.

Thanks for the response. I will maintain my score.

Comments on the paper

1 comment

We thank the reviewers for their efforts. The revision highlights changes in yellow, with new tables numbered R1-R5:

  • AE and DiT ablations (Tables R2, R3); minibatch OT was inactive in the reported model.
  • Nearest-snapshot warm-start baseline, ID and OOD (Tables 8, R5).
  • Latent DDPM and EDM comparison (Table R4).
  • Clarified test splits (Appendix B, Table R1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.