- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes the Invariant Measure Generator (IMG), a flow-matching model that learns to sample the long-time stationary distribution (invariant measure) of chaotic/turbulent systems without stationary training data. At each update, the current model generates samples, these are evolved by the physical solver for a random short time δt ∈ (0, ΔT), and the evolved states are used as fixed targets for conditional flow matching. A theorem bounds W2(q_θ, P_t q_θ) by the flow-matching loss for fixed targets. In a few-data setting, the same loss is added to standard data-trained CFM, optionally using a frozen neural operator instead of the solver (IMG-neural). Experiments on Lorenz–63, Kuramoto–Sivashinsky (KS), and 2D Hasegawa–Wakatani (HW, 131k dimensions) show IMG outperforming physics-supervised baselines (FNO, U-Net, PI-DeepONet, WGS) on PDF, spectrum, and increment statistics, and improving data-trained CFM in the few-data setting.
Critical strengths
- Simple, scalable idea. Training a generator on its own short-time evolved samples avoids estimating high-dimensional distribution distances and works at 131k dimensions, far beyond prior physics-supervised samplers (~100 dimensions). This is the main reason the paper is of interest.
- Thorough and honest evaluation. The paper uses multiple physically meaningful metrics (spectra, increments, flatness), tests stability under further evolution, and openly reports limitations (drift under evolution, weak discriminative power of SW2, no demonstrated speedup over DNS). The KS few-data gains also hold across 3 training seeds.
Critical weaknesses
- "Short-time supervision" hides long total evolution, and cost is not competitive. IMG is essentially a fixed-point iteration q ← P_δt q; over 20,000 updates the model population is effectively evolved for thousands of physical time units, far more than the DNS burn-in (t = 400). The full HW run takes 52.7 GPU hours, while DNS burn-in takes about 212 s and 41 fields about 423 s. The paper itself states that no speedup over DNS is established, so the main motivation (avoiding long-time evolution) is not supported in practice.
- "Data-free" is overstated. DNS validation trajectories are used for checkpoint selection and hyperparameter search (including ΔT and λ), so stationary DNS data still shapes the final model.
- Weak theory. Theorem 1 holds only for frozen targets; the gradient identity is the standard CFM result, and convergence of the actual iterative procedure is not analyzed. The constant C_t grows exponentially with t for chaotic systems (κ > 0), so the bound is nearly vacuous at long horizons. Lipschitz constants are assumed uniform in θ, which is unrealistic. Measure selection is unresolved: trivial invariant measures (e.g., the zero field for KS/HW) also satisfy self-consistency, and avoiding collapse is shown only empirically.
- Learned samples are not actually invariant. Under further HW evolution, IMG's spectrum, generalized enstrophy, and particle flux drift noticeably more than the DNS control (Tables 14 and 16), which directly weakens the claim of learning the invariant measure.
- Missing or weak baselines. Several physics-supervised baselines (PI-DeepONet, WGS) produce near-zero-amplitude fields (Fig. 12), so beating them says little. The most natural baseline, plain DNS or an evolved ensemble at the same compute budget, is missing; the particle-pool variant (essentially ensemble DNS) appears only as an ablation. In the few-data setting, data CFM is given only 15 KS snapshots or 10 short HW windows; a fair comparison would give it as much DNS data as IMG's training compute could produce.
- Limited statistical evidence. Main tables use one training seed, and error bars reflect sampling only. HW evaluation uses only 64 fields per repeat, and the authors show SW2 cannot distinguish the stronger models from DNS-to-DNS noise. Lorenz–63 uses a single run, and IMG's density is visibly more diffuse than DNS.
Questions for authors
N/A
6 comments
Official Response to Reviewer ZC88 (2/5)
(response to weakness 1, continued)
(3) We clarify that our training signal is beneficial under fair comparisons with a data-driven invariant measure sampler. Thus, our contribution of turning dynamical evolution into training signals for invariant measure learning holds whether or not our training is faster than one DNS.
In a new HW experiment, under the same solver budget and matched training wall-clock time, we compare IMG (not trained with DNS data), data CFM (first generate data with DNS and then train CFM on it), and mixtures of IMG and data CFM. The details are shown in Section 5.3 in the revised manuscript, and we summarize the results below:
| Solver budget ratio | Spectrum L1 ↓ | PDF-TV ↓ | Flatness rel-L1 ↓ | Increment PDF-TV ↓ | Flux MAE ↓ | A100 GPU hours | |
|---|---|---|---|---|---|---|---|
| IMG | DNS 0% IMG 100% | 0.41513 | 0.06627 | 0.07918 | 0.04703 | 0.08858 | 17.86 |
| Data CFM + IMG 25% | DNS 25% IMG 75% | 0.27143 | 0.05048 | 0.09402 | 0.05337 | 0.05292 | 17.90 |
| Data CFM + IMG 50% | DNS 50% IMG 50% | 0.26310 | 0.05122 | 0.09861 | 0.05619 | 0.10214 | 17.90 |
| Data CFM + IMG 75% | DNS 75% IMG 25% | 0.29599 | 0.05642 | 0.09116 | 0.05530 | 0.10227 | 17.88 |
| Data CFM | DNS 100% IMG 0% | 0.33345 | 0.05905 | 0.08877 | 0.06483 | 0.11923 | 17.90 |
The results demonstrate that comibning invariance supervision is better than pure data supervision even under controlled compute budget.
Besides, IMG can utilize the dynamics information hidden in trajectory data, which previous data-driven generative samplers are not able to use, to supervise the learning of the invariant measure.
In the original manuscript, this ability has shown benefits in the few-data HW experiment (section 5.2, with a new 3-seed result summarized below), where IMG-neural outperforms data-driven CFM. There, neural operators train on the same limited dataset, and then executes dynamical evolution in IMG. Thereby, IMG-neural does not require extra solver calls than data CFM.
| Method | KS | KS | KS | HW | HW | HW | HW |
|---|---|---|---|---|---|---|---|
| Coarse SW₂ ↓ | Spectrum L₁ ↓ | PDF-TV ↓ | Spectrum L₁ ↓ | PDF-TV ↓ | Increment PDF-TV ↓ | Flatness L₁ ↓ | |
| Data CFM 3-seed mean ± train std | 0.1041 ± 0.0052 | 0.1624 ± 0.0121 | 0.0744 ± 0.0012 | 0.2096 ± 0.0625 | 0.0426 ± 0.0105 | 0.0382 ± 0.0099 | 0.0621 ± 0.0123 |
| Data CFM + IMG-neural (ours) 3-seed mean ± train std | 0.0714 ± 0.0013 | 0.0987 ± 0.0121 | 0.0370 ± 0.0010 | 0.1228 ± 0.0049 | 0.0306 ± 0.0054 | 0.0284 ± 0.0066 | 0.0736 ± 0.0220 |
We have added these results into the revised manuscript, and we edited the introduction to differentiate learned samplers and DNS.
(4) Additionally, we improved the training efficiency by engineering optimizations in the training pipeline. Specifically, we reuse samples in training steps and adopted a replay buffer mechanism with details in Appendix E.13.
| Method | Spectrum rel. L1 ↓ | PDF-TV ↓ | Increment PDF-TV ↓ | Flatness rel. L1 ↓ | Training wall-clock time (h) |
|---|---|---|---|---|---|
| IMG | 0.0630 | 0.0116 | 0.0094 | 0.0273 | 52.71 |
| IMG-RB | 0.0652 | 0.0134 | 0.0080 | 0.0286 | 3.73 |
These results suggest that training IMG has the potential to be further accelerated. The training may be faster than a single DNS rollout especially in more computationally expensive systems or slowly-mixing systems (see the double-well result in Appendix E.12).
[1] T. Matsumoto et al. “Correlation function and linear response function of homogeneous isotropic turbulence in the Eulerian and Lagrangian coordinates.” Journal of Fluid Mechanics, 919, A9, 2021.
[2] R. Nies, et al. “Saturation of magnetized plasma turbulence by propagating zonal flows.” Physical Review Research, 8, 013295, 2026.
[3] A. M. Dimits et al. “Comparisons and physics basis of tokamak transport models and turbulence simulations.” Physics of Plasmas, 7, 969–983, 2000.
Weakness 2: "Data-free" is overstated. DNS validation trajectories are used for checkpoint selection and hyperparameter search (including ΔT and λ), so stationary DNS data still shapes the final model.
We thank the reviewer for this comment. We agree that we use the validation data for hyperparameter search and checkpoint selection, but we argue that our central claim is not weakened by the term.
We would like to clarify that our claim is that IMG can learn a stationary-state sampler without using stationary samples as training targets, and we stated this scope in the original manuscript p. 6, line 313. The validation data is also used for the physics-supervised baselines as they follow the same checkpoint selection protocal. The training signal, which is vital to model training, is still free of invariant measure samples.
Official Response to Reviewer ZC88 (3/5)
(reponse to weakness 2, continued)
We also clarify that the search concerns the separate few-data setting, where weights the self-consistency term relative to the data-based CFM objective. This setting explicitly uses stationary training samples and is not presented as data-free.
To avoid potential misunderstandings in the term "data-free", we have replaced "data-free" with more accurate wording throughout the manuscript.
Weakness 3: Weak theory. Theorem 1 holds only for frozen targets; the gradient identity is the standard CFM result, and convergence of the actual iterative procedure is not analyzed. The constant C_t grows exponentially with t for chaotic systems (κ > 0), so the bound is nearly vacuous at long horizons. Lipschitz constants are assumed uniform in θ, which is unrealistic. Measure selection is unresolved: trivial invariant measures (e.g., the zero field for KS/HW) also satisfy self-consistency, and avoiding collapse is shown only empirically.
We thank the reviewer for raising these questions, but we respectfully disagree that this makes the theory weak. We established an upper bound of the target distributional invariance residual (which we aim to minimize) to be the marginal CFM loss. Our claim was intuitively that, if the training loss is minimized, we minimize the upper bound of the target. Under our assumptions, we prove this result and provide justifications for our method design.
Next, we clarify the questions on the convergence, the constant and measure selection.
(1) We did not claim that the distributional residual will strictly decrease during training. We agree that studying the distributional convergence behavior of IMG is an important direction for future work.
(2) For the exponential growth in Ct, we provide an additional proof. If the dynamical system satisfies the mixing condition in Appendix B.2 (which holds for certain chaotic dynamical systems including the stochstic L63 in our experiments), we show that a finite-time residual controls the residual at all later times, and thus stops the exponential growing bound (see Appendix B.2 in the revised manuscript). Besides, we would like to clarify that the dependence of the error constant on physical evolution time is common, which appears in multiple previous works, including DeepONet (Theorem 3) and Markov Neural Operator (Theorem 1, Appendix B.2) [1-2]. We note that our original says that, when the marginal flow matching loss vanishes, the distributional invariance residual also vanishes.
The Lipshitz constant being uniform in theta (or equivalently, limiting the theory to a model class that has a uniform Lipshitz constant) is another regularity condition adpoted in classical generative model works including WGAN (Theorem 3, Section 3), NeuralODE (Theorem 1), and consistency models (Theorem 1) [3-5].
(3) We agree that measure selection remains an open problem. However, we clarify that the supervision comes from the invariant measure alone cannot distinguish the physical measure from other invariant measures by definition. Similarly, WGS cannot guarantee to recover the physical measure in such systems either.
In the few-data-assisted scenario, though, IMG has more information and may be able to select the physical measure. We believe that the theoretical analysis in this case is an interesting future direction.
Furthermore, we would like to point out that invariant measures other than the physical measure are not always trivial or undesired. In the modified HW system, our observations suggest that our IMG can find candidate distributions of states that are slowly evolving. These states are not observed in our DNS reference (see Appendix E.12). This result is relevant because identifying almost invariant structures can reveal organization of state space that is difficult to uncover through direct numerical simulation [6], which is an important potential application of IMG.
[1] Li et al., Learning Chaotic Dynamics in Dissipative Systems, NeurIPS, 2022.
[2] Lu et al., Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, 2021.
[3] Arjovsky et al., Wasserstein Generative Adversarial Networks, ICML, 2017.
[4] Chen et al., Neural Ordinary Differential Equations, NeurIPS, 2018.
[5] Song et al., Consistency Models, ICML, 2023.
[6] Froyland et al., Estimating Long-Term Behavior of Flows without Trajectory Integration: The Infinitesimal Generator Approach, SIAM Journal on Numerical Analysis, 2013.
Official Response to Reviewer ZC88 (1/5)
We thank the reviewer for the detailed and careful comments. We are encouraged that the reviewer finds our method simple and scalable, and the evaluation is thorough. Below, we address the concerns of the reviewer point-by-point.
We thank the reviewer for the comment. We agree that the training of IMG reported in the main table is expensive, involving many solver calls which surpasses the cost of a DNS run.
However, we would like to clarify that
IMG training is not identical to evolving a state population.
our IMG has a lower inference time than DNS, so the high training cost can be amortized. This is relevant because several applications need repetitive sampling.
our contribution remains valid under fair comparisons where the total training budget (DNS data generation + model training) is matched.
there is room for further accelerating IMG training.
These points are elaborated as follows:
(1) We agree that IMG uses the distributional invariance condition q=Ptq to construct self-consistent training targets, but its actual updates are not identical to applying the physical evolution operator Pt on the distribution q. First, we use a random δt within each batch. Second, the neural network parameterization and CFM loss induce a preconditioner on the network updates, which changes the distributional update dynamics (details in Appendix A.2 in the revised manuscript). This may improve convergence relative to direct evolution. We show such possibility in an example of a slowly-mixing double well (see Appendix E.12 in the revised manuscript), where IMG converges faster than directly evolving the same ensemble.
(2) We clarify that IMG is useful even if its training is more expensive than DNS. We aim to learn a generator of invariant measure states, which is substantially different from direct numerical simulation (DNS). Once trained, IMG can synthesize samples from the invariant measure with a lower per-sample wall-clock inference cost than DNS. We report the inference time per second per field on an A100 GPU in the table below.
Therefore, the training cost can be amortized when many samples are required. In HW, the original ~50 hour training time can be amortized if about 900 samples are needed. An example of potential application is perturbation–response studies where many states from the invariant measure are perturbed and evolved to probe the system response as in [1,2].
Furthermore, IMG can be conditioned on physical parameters, amortizing its training cost across multiple physical settings. Once IMG is trained, it can generate invariant measure samples faster than DNS in new physical conditions.
This is of practical use in high-fidelity tokamak plasma simulations [3], where the heat and particle flux induced by saturated microturbulence (the invariant measure of HW equation in our experiment is an instance) needs to be computed repetitively with different background physical parameters.
We conduct a new experiment to demonstrate that IMG can approximate invariant measure statistics conditioned on different physical parameters (density-gradient drive κ and electron adiabaticity α) in the HW equation. The results are summarized in the table below, with each cell showing predicted particle flux / reference particle flux:
Despite visible error, the trend of the particle flux's dependency on α and κ is captured. This demonstrates that IMG can be trained to learn invariant measures conditioned on physical parameters, and therefore obtain efficiency gains over DNS after training.
More details are included in Section 5.3 in the edited manuscript.