- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Second-Order Flow Matching (SoF) for learning the second-order time derivative, i.e., acceleration, of the flow ODE. The theoretical motivation is that integrating the flow ODE with quadratic approximation using acceleration reduces local truncation error to from of tangent-line approximation. To support efficient training and inference, the paper proposes a shared-decoder dual head network, where the first head predicts velocity and the second head predicts acceleration. This architectural design enables velocity and acceleration inference with a single forward pass. In additional to these core contributions, the paper also discusses various design choices such as interpolant parametrization (Sec. 2.3), time sampling (Sec. 2.5), and adaptive integrator variants (Sec. 2.6). The combination of the choices is shown to provide a non-trivial gain in FID scores on standard generative benchmarks such as CIFAR10, CelebA, and LSUN.
Critical strengths
None.
Critical weaknesses
Inconsistency between theory and practice. While the theory regarding the quadratic integration where in Section 2.1 and 2.2 are sound, I am not sure whether the SoF acceleration learned by solving proposed in Section 2.4 Eq. (10) actually converges to the right quantity . In short, my concern is that in general. The right-hand side is discussed in Sections 2.1 and 2.2, whereas the left-hand side is the acceleration learned and used in practice within the paper.
Insufficient empirical evidence. Using learned acceleration shows inconsistent benefits across number of function evaluations (NFEs). For instance, in Table 2, CIFAR10, 2nd Euler starts under-performing 1st Euler for NFE >= 20. Likewise for DPM++ in the same table. Furthermore, the experiment settings are unclear, so it is difficult to comprehend exactly how much the learned acceleration contributes to the final performance. For instance, while Table 2 in the main paper implies 1st Heun vs. 2nd Heun is the comparison isolating the effect of learned acceleration, Table 10 in the Appendix reports 2nd Heun (C,S) vs. 2nd Heun (C,S,Z) where "Z = zero acceleration". The paper should clarify the exact training and inference settings for reported numbers, so one may disentangle the gain from auxiliary training and inference techniques from the gain from core idea of second-order integration with learned acceleration.
Insufficient discussion of closely related works. The paper does not adequately acknowledge closely related prior works such as [1,2,3]. In particular, GENIE already introduced a lightweight neural head for predicting higher-order ODE derivatives and using them for single-evaluation Taylor sampling. It also demonstrated the approach on realistic image-generation benchmarks.
[1] GENIE: Higher-Order Denoising Diffusion Solvers, NeurIPS, 2022
[2] NRFlow: Towards Noise-Robust Generative Modeling via High-Order Mechanism, UAI, 2025
[3] High-Order Matching for One-Step Shortcut Diffusion Models, arXiv, 2025
Questions for authors
- Can the authors verify whether the optimum of the SoF acceleration loss equals ?
- Can the authors provide an ablation table which clearly disentangles the effects of training techniques, inference techniques, and using acceleration for ODE integration?
- Can the authors explain the methodological novelty and empirical strengths compared to prior related works such as GENIE?