- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(−1)
- Soundness
- 2(+1)
Read review
Paper summary
This paper studies the statistical learnability of long-horizon flow maps for one-step generation. It introduces a transport loss connecting flow-map regression with MeanFlow and consistency models, and derives polynomial learning bounds under low-rank Gaussian mixture assumptions. Based on these insights, the authors propose GMRFlow, combining analytical Gaussian flow maps with neural residual corrections. Experiments on CelebA80 and FFHQ64 demonstrate improved one-step FID over several baselines.
Critical strengths
-
The transport-loss formulation provides a useful unified perspective on existing flow-map learning objectives.
-
GMRFlow demonstrates promising empirical improvements, suggesting that Gaussian mixture priors provide a useful inductive bias for one-step generation.
Critical weaknesses
-
Gap between theory and practical training. The transport loss and learning guarantees in Theorem 6.6 depend on the true marginal velocity, which is unavailable during practical training. MeanFlow instead uses conditional velocities and stop-gradient. The corresponding squared objectives are not equivalent, and the paper does not establish that its statistical guarantees extend to the practical training procedure. This significantly weakens the main theoretical contribution.
-
Theory–algorithm mismatch and incomplete evaluation. The theoretical analysis considers a layered Gaussian-mixture Euler network rather than the proposed U-Net-based GMRFlow architecture. Moreover, Table 1 omits the vanilla MeanFlow baseline on FFHQ64, and GMRFlow slightly underperforms the hard-cluster baseline in one-step FID on the ImageNet64 subset. These limitations weaken the claimed generality of the theoretical and empirical contributions.
Questions for authors
-
How do the learning guarantees extend from the oracle transport loss to the practically implemented conditional-velocity MeanFlow objective with stop-gradient?
-
Which theoretical guarantees apply to the actual GMRFlow architecture rather than the layered network analyzed in Section 6?
-
Could the authors provide the missing vanilla MeanFlow baseline on FFHQ64 under matched settings?
6 comments
Weakness 2 and Questions 2–3: Theoretical guarantees for GMRFlow, FFHQ64 MeanFlow baseline, and ImageNet-64 results.
We thank the reviewer for these constructive comments. We address each point below:
- Theoretical guarantees for GMRFlow (Question 2; Sections 4, 6, 7.1, 8): In the revised Section 6.2, we clarify that the analyzed layered network (soft conditioning without labels) and GMRFlow (hard conditioning with cluster labels) stem from the same hierarchical Gaussian mixture. Specifically, three theoretical guarantees apply directly to GMRFlow:
- Subspace-confined residual target (Proposition 6.2): Under the subspace assumption, the true flow map decomposes into an explicit ambient scaling plus a residual . Since already matches the exact scaling on , the non-Gaussian residual target left for to learn lies entirely within the low-dimensional subspace (consistent with Section 7.1, Fig. 4).
- Exact Gaussian prior (Proposition 6.4): solves the exact component flow and recovers the exact mixture marginals when .
- Global surrogate control (Theorem 4.2, Corollary 4.3): Both bounds hold for any map satisfying (enforced in Eq. 5 via ) and thus bound GMRFlow's regression and generation errors.
- Added vanilla MeanFlow baseline on FFHQ64 (Question 3; Table 1): We added the matched vanilla MeanFlow baseline on FFHQ64 in Table 1; under identical settings, GMRFlow lowers 1-NFE FID from 76.70 (MF) to 53.90, outperforming all baselines (CTM: 76.33, HC: 70.72).
- ImageNet-64 results and new single-Gaussian-per-class experiment (Appendix G, Table 6): On the 100-class ImageNet-64 subset, subdividing each class into subclusters () leaves only samples per component. This limited sample size leads to covariance estimation noise, causing GMRFlow (subcluster) () to slightly trail HC () while still improving over MF (). Rather than tuning training hyperparameters, we kept the training recipe and architecture identical and switched to a prior better aligned with Theorem 6.7's component-count vs. sample-size trade-off (and Table 2): pooling all samples per class into a single Gaussian flow map per class (, GMRFlow (per-class)) mitigates this estimation bottleneck and reduces 1-NFE FID to 62.86, outperforming MF (72.86), HC (66.98), and GMRFlow (subcluster) (67.44) by 10.00, 4.12, and 4.58 points, respectively.
Proof A (1D Realizable Gaussian Counterexample without Stop-Gradient). Let , , and , with independent of and . Consider the single-parameter family and , where and . For , the marginal velocity is , so integrating gives ; thus realizes the exact flow map .
(1) Residual identity: Writing , we have and (holding fixed). Substituting into the MeanFlow residual yields , so .
(2) Exact variance decomposition: Since are jointly Gaussian with , , and , the true marginal velocity is and . Writing and conditioning on cancels the cross term, giving
(3) Strict positivity at : Since solves the transport equation, is a global minimum, so . On , uniformly on , so dominated convergence justifies differentiating under the expectation:
because , , and on . By contrast, the stop-gradient semi-gradient satisfies , restoring .
Weakness 1. and Question 1.
We thank the reviewer for raising this central question. To directly address the gap between the oracle transport loss Ltr and practical training, we (i) extend our finite-sample guarantees to observable conditional-velocity labels in the revised manuscript (Section 4.2, Theorem 4.4; Section 6.3, Corollary 6.9; Appendices A.4 and C.6), and (ii) provide two formal results below (Proofs A and B) explaining why stop-gradient optimization separates the optimization and statistical ERM questions:
Why the oracle transport loss captures the statistical core & exact extension to Improved MeanFlow (iMF): The oracle transport loss Ltr captures the shared statistical core of flow-map learning: distillation-based methods (e.g., continuous Consistency Models) correspond directly to Ltr, while our ERM analysis extends with minimal additional work to improved MeanFlow (iMF; Geng et al., 2026), which is a variant of MeanFlow and trains directly on observable conditional-velocity labels. Specifically, replacing the unknown marginal velocity v with the model's diagonal boundary value wθ(ξ,t):=uθ(ξ,t,t) yields the empirical squared loss LiMF,n(θ) against conditional velocity ε−x0 :
Dimension-free population equivalence (Theorem 4.4, Appendix A.4): With fixed positive diagonal r=t sampling rate and bounded spatial Jacobian, the iMF population training loss LiMF(θ) and the transport loss Ltr(θ) are equivalent up to a constant factor independent of the ambient dimension D (LiMF(θ)−σ2≍Ltr(θ)), where σ2=E[∥(ε−x0)−v∥2] is the irreducible regression noise.
Identical ERM finite-sample rate (Corollary 6.9, Appendix C.6): Over the same Gaussian-mixture Euler class Θm, any empirical risk minimizer θiMF of LiMF,n satisfies, with probability at least 1−δ,
LiMF(θiMF)−σ2≤C′(m2(1+d∗)4+rn,m),
matching both the approximation order and the statistical rate rn,m of Theorem 6.7 (and hence controlling Ltr(θiMF) via Theorem 4.4). Crucially, the proof directly reuses the oracle approximation, intrinsic moment, and covering-number bounds from Theorem 6.7, requiring only a concise noise-cancellation argument to eliminate the common empirical noise energy Pn∥ζ∥2 (ζ=Y−v) and absorb the noise multiplier into the empirical residual.
Why stop-gradient in MeanFlow decouples optimization from statistical ERM (Proofs A and B below): Our paper focuses on statistical learnability rather than optimization dynamics. For sample-based MeanFlow, two formal results below clarify why separating optimization from statistical ERM is both natural and mathematically sound: