- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper attempts to distinguish tracking, state-space, and path-space fidelity in chaotic forecasting. From Energy Score, this paper proposes a post-hoc calibration for generators with per-time stochastic inputs, in which a single coupling parameter correlates these inputs across forecast times with the learned weights frozen, selected with an unbiased estimator of a temporal Variogram Score. Experiments on Kolmogorov flow, 2D Kuramoto-Sivashinsky, and Hasegawa-Wakatani show substantial changes in temporal correlations, and spectral improvements on two systems, with small changes in joint Energy Score.
Critical strengths
-
The method is straightforward and easy to understand. Temporal dependence is adjusted through a single coupling parameter in the stochastic inputs, while the trained network remains fixed. This provides a simple calibration procedure without retraining.
-
The experimental results support the usefulness of the approach. Across three chaotic systems, calibration improves the reported within-window temporal correlations. Temporal spectra also improve on Kolmogorov flow and HW2D, while the joint Energy Score remains nearly unchanged.
Critical weaknesses
-
Sensitivity to endpoint prediction errors remains unclear. Since Jump–Complete conditions the intermediate trajectory on a predicted distant endpoint, the paper should clarify whether, and to what extent, inaccuracies in that endpoint degrade the quality and temporal fidelity of the completed path. Although Appendix F.6 examines the conditioning mismatch, it does not quantify how completion quality changes as endpoint prediction errors increase.
-
The empirical results do not establish a consistent advantage over baselines. Calibration improves several diagnostics relative to the uncalibrated model, but the resulting temporal fidelity is not consistently better than competing methods. Moreover, the comparison is limited to DYffusion, PDE-Refiner, and an endpoint-free Joint-Forecast control. Additional comparisons with approaches that explicitly model joint trajectories or temporal coherence would strengthen the evidence for the method’s comparative advantage.
-
Limited discussion of the underlying chaotic dynamics. The experiments evaluate state distributions and temporal dependence, but the paper does not sufficiently explain how these design and results relate to characteristic properties(i.e., the Lyapunov Exponent) of chaotic systems.
-
The applicability of the ES analysis to chaotic dynamics is insufficiently discussed. The main scaling result is established for zero-mean Gaussian trajectories with separable covariance. Although Appendix D provides a distribution-free decomposition, the stated high-dimensional decay rate relies on Gaussian moment bounds to control the remainder. The manuscript does not adequately examine whether the required concentration and covariance assumptions approximately hold for the chaotic trajectory distributions considered.
Questions for authors
-
Inconsistent table formatting. Please standardize the use of boldface for the best results and clearly indicate metric preferences across tables. For example, Table 1 neither highlights the best results in bold nor indicates the preferred direction for ES, whereas Tables 12–14 do both. For signed correlation errors, consistently specify that values closer to zero are better.
-
How does the proposed method relate to existing approaches that jointly address trajectory consistency and distributional fidelity in chaotic forecasting, and what are its main methodological differences?
-
What’s the key contribution of this work in terms of the methodology or theoretical perspectives for chaotic forecasting?
5 comments
Response to Reviewer Zne3 (2/2): Q1-Q3 / References
Q1: Table formatting.
Tables now bold best means, indicate lower ES is better, and specify zero as optimal for signed correlation errors.
Q2: Relationship to existing approaches.
Our method explicitly controls temporal dependence after training, rather than modifying the learned model or generated states. We clarified this in a new Positioning paragraph (new App. A). We adjust only the joint law of per-time stochastic inputs, keeping weights, conditioning, and one-time latent marginals fixed. Unlike approaches implicitly constraining temporal coherence, we explicitly calibrate temporal dependence using a dependence-sensitive validation criterion and assess it with lag-resolved diagnostics.
Q3: Central contribution.
Our contribution is the diagnosis-to-remedy connection: we quantify why joint ES poorly resolves temporal dependence and use this insight for post-hoc calibration. Beyond known qualitative limitations of ES, we derive effective-dimensionality scaling and finite-sample detection budgets, empirically validated in chaotic systems. The novelty lies in this diagnosis-guided calibration, not the coupling family alone.
References
[1] Yifan Chen, Mark Goldstein, Mengjian Hua, Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Probabilistic forecasting with stochastic interpolants and Föllmer processes. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research. PMLR, 2024.
[2] David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research. PMLR, 2024.
[3] Johannes Brandstetter, Daniel E. Worrall, and Max Welling. Message passing neural PDE solvers. In International Conference on Learning Representations, 2022.
As the discussion period is coming to an end, we wanted to ask whether our rebuttal has adequately addressed your concerns or whether any points would benefit from further clarification. We would be happy to respond promptly to any remaining questions before the deadline.
Response to Reviewer Zne3 (1/2): W1-W4
We thank the reviewer for their feedback, which motivated seven new experiments, additional comparisons, and clearer contributions (in blue in the revision of the pdf document).
Our central contribution links evaluation and correction: we quantify why the joint Energy Score (ES), despite being proper, can be insensitive to temporal-dependence errors, and address this through post-hoc calibration. We correlate stochastic inputs across forecast times with a single parameter ρ, selected with an unbiased temporal Variogram Score (VSt).
W1: Sensitivity to endpoint prediction errors.
Following the reviewer's suggestion, we quantify endpoint-error sensitivity and show that calibration remains effective under perturbations. Adding Gaussian endpoint noise degrades completion quality for both JUMP-COMPLETE and DYffusion, but JUMP-COMPLETE consistently performs better. Without retuning, calibration maintains near-zero within-window correlation error and reduces VSt relative to the native coupling ( ρ=0) (new App. F.7, Fig. 15).
W2: Comparisons with temporally coherent forecasting methods.
Competitive temporal fidelity at substantially lower cost. We added two new baselines, stochastic interpolants (SI) [1] and Rolling Diffusion [2], the latter jointly denoising future frames (Table 1, new App. C.6). Calibrated JUMP-COMPLETE achieves the lowest within-window correlation error on KS2D and HW2D, and ranks second on Kolmogorov, narrowly behind PDE-Refiner (0.069 vs. 0.060). On Kolmogorov, it requires 23-800$\times$ fewer network evaluations than DYffusion, PDE-Refiner, SI, and Rolling (Table 7). Thus calibration provides competitive temporal accuracy at a fraction of the computational cost.
The new baselines reinforce the limitation of joint ES. On Kolmogorov, Rolling and SI achieve lower joint ES despite 1.7$\times$ and 1.6$\times$ larger within-window correlation errors than calibrated JUMP-COMPLETE (new results in Table 1). ES-based selection also favors the uncalibrated coupling (ρ=0) on all three systems despite worse temporal correlations. Hence, a better joint ES does not necessarily indicate more faithful temporal dynamics.
Calibration is not architecture-specific. We clarified JOINT-FORECAST as an endpoint-free temporal-bundling baseline [3] (new App. C.6). Calibration also improves this baseline (App. F.8), demonstrating that the mechanism is not restricted to endpoint-conditioned JUMP-COMPLETE.
W3: Connection to chaotic dynamics.
We added a Lyapunov-time analysis linking predictability to completion, evaluation, and calibration difficulty (new App. C.2). Estimated Lyapunov times are 17.6, 16.5, and 8.4 frames respectively for Kolmogorov, KS2D, and HW2D (new Table 3). Completion windows span 0.91, 0.97, and 1.43 Lyapunov times, respectively, making endpoints sensitive to initial-condition errors. The 496-frame rollouts span tens of Lyapunov times across all three systems, where tracking loses relevance and temporal fidelity matters. HW2D has the longest window in Lyapunov units, lowest selected coupling, and smallest calibration gain, consistent with greater calibration difficulty over longer relative windows.
W4: Applicability of the ES analysis.
On Kolmogorov, KS2D, and HW2D, both Gaussian concentration and covariance separability assumptions are substantially violated (as assessed through the measures in the new Table 8). We performed new tests showing that ES sensitivity analysis remains predictive on the three equations. The theoretical approximation predicts temporal-dependence ES gaps within 8%. New Figure 7 in the appendix shows that both the loss of ES sensitivity to temporal-dependence errors and its consequences for detectability persist in the nonlinear 2D systems. ES sensitivity to temporal-dependence errors decreases approximately inversely with effective spatial dimension, while sensitivity to variance errors remains nearly constant. Consequently, detecting temporal-dependence errors at full resolution requires an estimated 104 to 5×104 forecast cases, compared with only 50 available (new Fig. 7).
References are listed in the following comment.