acceptodds
Under review as a conference paper at ICLR 2027

Learning Long-Horizon Flow Maps in High Dimensions: From a Regression Surrogate to Efficient Statistical Learning

Abstract

Establishing statistically efficient neural flow-map learning for one-step generation requires a valid training objective in the absence of direct flow-map supervision. To this end, we develop (i) a flow-map regression surrogate perspective and (ii) a dimension-efficient statistical learning theory that exploits intrinsic mixture structure of real data. Specifically, we introduce transport loss, which identifies the true flow map and controls its regression error, providing a common framework for MeanFlow, CM, and sCM objectives. We then investigate statistical learnability under this objective. While generic learning bounds suffer from the curse of dimensionality, we establish learning guarantees with polynomial dependence on intrinsic and latent dimensions under a low-rank Gaussian mixture assumption, using a layered residual network construction to control approximation and estimation errors. These insights motivate Gaussian Mixture Residual Flow (GMRFlow), which combines analytic Gaussian component flow maps with a learned neural residual correction. Experiments on CelebA80 and FFHQ64 show that GMRFlow achieves improved one-step FID over MeanFlow baselines.

Primary area:
generative models
Keywords:
one-step generation, flow maps, flow matching, Mean Flow, consistency models, learning theory, intrinsic dimension, Gaussian mixtures

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_K8myOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(−1)
Soundness
2(+1)
Read review
Paper summary

This paper studies the statistical learnability of long-horizon flow maps for one-step generation. It introduces a transport loss connecting flow-map regression with MeanFlow and consistency models, and derives polynomial learning bounds under low-rank Gaussian mixture assumptions. Based on these insights, the authors propose GMRFlow, combining analytical Gaussian flow maps with neural residual corrections. Experiments on CelebA80 and FFHQ64 demonstrate improved one-step FID over several baselines.

Critical strengths
  1. The transport-loss formulation provides a useful unified perspective on existing flow-map learning objectives.

  2. GMRFlow demonstrates promising empirical improvements, suggesting that Gaussian mixture priors provide a useful inductive bias for one-step generation.

Critical weaknesses
  1. Gap between theory and practical training. The transport loss and learning guarantees in Theorem 6.6 depend on the true marginal velocity, which is unavailable during practical training. MeanFlow instead uses conditional velocities and stop-gradient. The corresponding squared objectives are not equivalent, and the paper does not establish that its statistical guarantees extend to the practical training procedure. This significantly weakens the main theoretical contribution.

  2. Theory–algorithm mismatch and incomplete evaluation. The theoretical analysis considers a layered Gaussian-mixture Euler network rather than the proposed U-Net-based GMRFlow architecture. Moreover, Table 1 omits the vanilla MeanFlow baseline on FFHQ64, and GMRFlow slightly underperforms the hard-cluster baseline in one-step FID on the ImageNet64 subset. These limitations weaken the claimed generality of the theoretical and empirical contributions.

Questions for authors
  1. How do the learning guarantees extend from the oracle transport loss to the practically implemented conditional-velocity MeanFlow objective with stop-gradient?

  2. Which theoretical guarantees apply to the actual GMRFlow architecture rather than the layered network analyzed in Section 6?

  3. Could the authors provide the missing vanilla MeanFlow baseline on FFHQ64 under matched settings?

6 comments

Weakness 1. and Question 1.

We thank the reviewer for raising this central question. To directly address the gap between the oracle transport loss Ltr\mathcal{L}_{\mathrm{tr}} and practical training, we (i) extend our finite-sample guarantees to observable conditional-velocity labels in the revised manuscript (Section 4.2, Theorem 4.4; Section 6.3, Corollary 6.9; Appendices A.4 and C.6), and (ii) provide two formal results below (Proofs A and B) explaining why stop-gradient optimization separates the optimization and statistical ERM questions:

  1. Why the oracle transport loss captures the statistical core & exact extension to Improved MeanFlow (iMF): The oracle transport loss Ltr\mathcal{L}_{\mathrm{tr}} captures the shared statistical core of flow-map learning: distillation-based methods (e.g., continuous Consistency Models) correspond directly to Ltr\mathcal{L}_{\mathrm{tr}}, while our ERM analysis extends with minimal additional work to improved MeanFlow (iMF; Geng et al., 2026), which is a variant of MeanFlow and trains directly on observable conditional-velocity labels. Specifically, replacing the unknown marginal velocity vv with the model's diagonal boundary value wθ(ξ,t):=uθ(ξ,t,t)w_\theta(\xi,t) := u_\theta(\xi,t,t) yields the empirical squared loss L^iMF,n(θ)\widehat{\mathcal{L}}_{\mathrm{iMF},n}(\theta) against conditional velocity ε−x0\varepsilon-x_0 :

    • Dimension-free population equivalence (Theorem 4.4, Appendix A.4): With fixed positive diagonal r=tr=t sampling rate and bounded spatial Jacobian, the iMF population training loss LiMF(θ)\mathcal{L}_{\mathrm{iMF}}(\theta) and the transport loss Ltr(θ)\mathcal{L}_{\mathrm{tr}}(\theta) are equivalent up to a constant factor independent of the ambient dimension DD (LiMF(θ)−σ2≍Ltr(θ)\mathcal{L}_{\mathrm{iMF}}(\theta) - \sigma^2 \asymp \mathcal{L}_{\mathrm{tr}}(\theta)), where σ2=E[∥(ε−x0)−v∥2]\sigma^2=\mathbb E[\Vert (\varepsilon-x_0)-v\Vert ^2] is the irreducible regression noise.

    • Identical ERM finite-sample rate (Corollary 6.9, Appendix C.6): Over the same Gaussian-mixture Euler class Θm\Theta_m, any empirical risk minimizer θ^iMF\widehat\theta_{\mathrm{iMF}} of L^iMF,n\widehat{\mathcal{L}}_{\mathrm{iMF},n} satisfies, with probability at least 1−δ1-\delta,

LiMF(θ^iMF)−σ2≤C′((1+d∗)4m2+rn,m),\mathcal{L}_{\mathrm{iMF}}(\widehat\theta_{\mathrm{iMF}}) - \sigma^2 \le C'\left(\frac{(1+d^\ast)^4}{m^2} + r_{n,m}\right),

matching both the approximation order and the statistical rate rn,mr_{n,m} of Theorem 6.7 (and hence controlling Ltr(θ^iMF)\mathcal{L}_{\mathrm{tr}}(\widehat\theta_{\mathrm{iMF}}) via Theorem 4.4). Crucially, the proof directly reuses the oracle approximation, intrinsic moment, and covering-number bounds from Theorem 6.7, requiring only a concise noise-cancellation argument to eliminate the common empirical noise energy Pn∥ζ∥2P_n\Vert \zeta\Vert ^2 (ζ=Y−v\zeta = Y - v) and absorb the noise multiplier into the empirical residual.

  1. Why stop-gradient in MeanFlow decouples optimization from statistical ERM (Proofs A and B below): Our paper focuses on statistical learnability rather than optimization dynamics. For sample-based MeanFlow, two formal results below clarify why separating optimization from statistical ERM is both natural and mathematically sound:

    • Failure of MeanFlow without stop-gradient (Proof A): In a 1D realizable Gaussian MeanFlow model with optimal parameter λ⋆\lambda^\star, the full-gradient sample loss without stop-gradient incurs a Jacobian-dependent conditional-variance bias B(λ)B(\lambda) such that LMF′(λ⋆)=B′(λ⋆)>0L_{\mathrm{MF}}'(\lambda^\star) = B'(\lambda^\star) > 0 for every time distribution on 0≤r<t≤10 \le r < t \le 1; hence λ⋆\lambda^\star is not even a stationary point.
    • Coercivity and convergence of MeanFlow stop-gradient dynamics (Proof B): Although stop-gradient in MeanFlow produces a non-conservative semi-gradient field G(θ)=2E[(Dθuθ)⊤Areθ]G(\theta) = 2\mathbb{E}[(D_\theta u_\theta)^\top \mathcal{A}_r e_\theta], under uniform time sampling t∣r∼Unif⁡[r,1]t \mid r \sim \operatorname{Unif}[r,1] the linear operator Ar=I+(t−r)Dt\mathcal{A}_r = I + (t-r)\mathcal{D}_t satisfies the coercivity bound ⟨h,Arh⟩≥12∥h∥L22\langle h, \mathcal{A}_r h \rangle \ge \frac{1}{2}\Vert h\Vert _{L^2}^2, yielding global exponential convergence ∥us−u∗∥L2≤e−s∥u0−u∗∥L2\Vert u_s - u^\ast\Vert _{L^2} \le e^{-s}\Vert u_0 - u^\ast\Vert _{L^2} in infinite-dimensional function space. This shows that analyzing stop-gradient MeanFlow is fundamentally an optimization-dynamics question rather than an ERM problem, whereas replacing vv with wθw_\theta via iMF yields a genuine ERM objective L^iMF,n(θ)\widehat{\mathcal{L}}_{\mathrm{iMF},n}(\theta) to which Corollary 6.9 directly applies.

Weakness 2 and Questions 2–3: Theoretical guarantees for GMRFlow, FFHQ64 MeanFlow baseline, and ImageNet-64 results.

We thank the reviewer for these constructive comments. We address each point below:

  1. Theoretical guarantees for GMRFlow (Question 2; Sections 4, 6, 7.1, 8): In the revised Section 6.2, we clarify that the analyzed layered network (soft conditioning without labels) and GMRFlow (hard conditioning with cluster labels) stem from the same hierarchical Gaussian mixture. Specifically, three theoretical guarantees apply directly to GMRFlow:
    • Subspace-confined residual target (Proposition 6.2): Under the subspace assumption, the true flow map decomposes into an explicit ambient scaling rtz\frac{r}{t}z plus a residual h∈Vh \in V. Since X^kG\hat{X}_k^G already matches the exact rt\frac{r}{t} scaling on V⊥V^\perp, the non-Gaussian residual target left for FθF_\theta to learn lies entirely within the low-dimensional subspace VV (consistent with Section 7.1, Fig. 4).
    • Exact Gaussian prior (Proposition 6.4): X^kG\hat{X}_k^G solves the exact component flow and recovers the exact mixture marginals when Fθ=0F_\theta=0.
    • Global surrogate control (Theorem 4.2, Corollary 4.3): Both bounds hold for any C1C^1 map satisfying Xθ(z,t,t)=zX_\theta(z,t,t)=z (enforced in Eq. 5 via 1−r/t1-r/t) and thus bound GMRFlow's L2L^2 regression and W22W_2^2 generation errors.
  2. Added vanilla MeanFlow baseline on FFHQ64 (Question 3; Table 1): We added the matched vanilla MeanFlow baseline on FFHQ64 in Table 1; under identical settings, GMRFlow lowers 1-NFE FID from 76.70 (MF) to 53.90, outperforming all baselines (CTM: 76.33, HC: 70.72).
  3. ImageNet-64 results and new single-Gaussian-per-class experiment (Appendix G, Table 6): On the 100-class ImageNet-64 subset, subdividing each class into 55 subclusters (K=500K=500) leaves only ∼200\sim 200 samples per component. This limited sample size leads to covariance estimation noise, causing GMRFlow (subcluster) (67.4467.44) to slightly trail HC (66.9866.98) while still improving over MF (72.8672.86). Rather than tuning training hyperparameters, we kept the training recipe and architecture identical and switched to a prior better aligned with Theorem 6.7's component-count vs. sample-size trade-off (and Table 2): pooling all ∼1,000\sim 1{,}000 samples per class into a single Gaussian flow map per class (K=100K=100, GMRFlow (per-class)) mitigates this estimation bottleneck and reduces 1-NFE FID to 62.86, outperforming MF (72.86), HC (66.98), and GMRFlow (subcluster) (67.44) by 10.00, 4.12, and 4.58 points, respectively.

Proof A (1D Realizable Gaussian Counterexample without Stop-Gradient). Let x,ϵ∼i.i.d.N(0,1)x, \epsilon \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,1), zt=(1−t)x+tϵz_t = (1-t)x + t\epsilon, and V=ϵ−xV = \epsilon - x, with (r,t)∼μ(r,t) \sim \mu independent of (x,ϵ)(x,\epsilon) and μ({0≤r<t≤1})=1\mu(\lbrace 0 \le r < t \le 1\rbrace ) = 1. Consider the single-parameter family Xλ(z,r,t)=aλ(r,t)zX_\lambda(z,r,t) = a_\lambda(r,t)z and uλ(z,r,t)=(z−Xλ(z,r,t))/(t−r)u_\lambda(z,r,t) = (z - X_\lambda(z,r,t))/(t-r), where Cs(λ)=s2+(1−s)2λC_s(\lambda) = s^2 + (1-s)^2\lambda and aλ(r,t)=Cr(λ)/Ct(λ)a_\lambda(r,t) = \sqrt{C_r(\lambda)/C_t(\lambda)}. For x∼N(0,λ)x \sim \mathcal{N}(0,\lambda), the marginal velocity is vλ(z,t)=t−(1−t)λCt(λ)z=∂tCt(λ)2Ct(λ)zv_\lambda(z,t) = \frac{t-(1-t)\lambda}{C_t(\lambda)}z = \frac{\partial_t C_t(\lambda)}{2C_t(\lambda)}z, so integrating dztdt=vλ(zt,t)\frac{\mathrm{d}z_t}{\mathrm{d}t} = v_\lambda(z_t,t) gives zr=aλ(r,t)zt=Xλ(zt,r,t)z_r = a_\lambda(r,t)z_t = X_\lambda(z_t,r,t); thus λ⋆=1\lambda^\star = 1 realizes the exact flow map X1X_1.

(1) Residual identity: Writing Δ=t−r>0\Delta = t-r > 0, we have ∂tuλ=−(∂tXλ+uλ)/Δ\partial_t u_\lambda = -(\partial_t X_\lambda + u_\lambda)/\Delta and ∂zuλ=(1−∂zXλ)/Δ\partial_z u_\lambda = (1 - \partial_z X_\lambda)/\Delta (holding z,rz,r fixed). Substituting into the MeanFlow residual Rλ=uλ−V+Δ(∂tuλ+∂zuλV)R_\lambda = u_\lambda - V + \Delta(\partial_t u_\lambda + \partial_z u_\lambda V) yields Rλ=−(∂tXλ+∂zXλV)R_\lambda = -(\partial_t X_\lambda + \partial_z X_\lambda V), so LMF(λ):=E[Rλ2]=E[(∂tXλ+∂zXλV)2]L_{\mathrm{MF}}(\lambda) := \mathbb{E}[R_\lambda^2] = \mathbb{E}[(\partial_t X_\lambda + \partial_z X_\lambda V)^2].

(2) Exact variance decomposition: Since (zt,V)(z_t, V) are jointly Gaussian with Var⁡(zt)=Ct(1)\operatorname{Var}(z_t) = C_t(1), Cov⁡(V,zt)=2t−1\operatorname{Cov}(V, z_t) = 2t-1, and Var⁡(V)=2\operatorname{Var}(V) = 2, the true marginal velocity is v⋆(z,t)=E[V∣zt=z]=2t−1Ct(1)zv^\star(z,t) = \mathbb{E}[V \mid z_t=z] = \frac{2t-1}{C_t(1)}z and Var⁡(V∣zt)=2−(2t−1)2Ct(1)=1Ct(1)\operatorname{Var}(V \mid z_t) = 2 - \frac{(2t-1)^2}{C_t(1)} = \frac{1}{C_t(1)}. Writing V=v⋆(zt,t)+(V−v⋆(zt,t))V = v^\star(z_t,t) + (V - v^\star(z_t,t)) and conditioning on (zt,r,t)(z_t, r, t) cancels the cross term, giving

LMF(λ)=Loracle(λ)+B(λ),B(λ):=Er,t[(∂zXλ)2Var⁡(V∣zt)]=Er,t[Cr(λ)Ct(λ)Ct(1)].L_{\mathrm{MF}}(\lambda) = L_{\mathrm{oracle}}(\lambda) + B(\lambda), \qquad B(\lambda) := \mathbb{E}_{r,t}\big[(\partial_z X_\lambda)^2 \operatorname{Var}(V \mid z_t)\big] = \mathbb{E}_{r,t}\left[\frac{C_r(\lambda)}{C_t(\lambda)C_t(1)}\right].

(3) Strict positivity at λ⋆=1\lambda^\star = 1: Since X1X_1 solves the transport equation, Loracle(1)=0L_{\mathrm{oracle}}(1) = 0 is a global minimum, so Loracle′(1)=0L_{\mathrm{oracle}}'(1) = 0. On λ∈[1/2,3/2]\lambda \in [1/2, 3/2], Cs(λ)≥12(s2+(1−s)2)≥14C_s(\lambda) \ge \frac{1}{2}(s^2+(1-s)^2) \ge \frac{1}{4} uniformly on 0≤r≤t≤10 \le r \le t \le 1, so dominated convergence justifies differentiating under the expectation:

LMF′(1)=B′(1)=Er,t[(1−r)2Ct(1)−(1−t)2Cr(1)Ct(1)3]=Er,t[(t−r)(t+r−2rt)Ct(1)3]>0,L_{\mathrm{MF}}'(1) = B'(1) = \mathbb{E}_{r,t}\left[\frac{(1-r)^2 C_t(1) - (1-t)^2 C_r(1)}{C_t(1)^3}\right] = \mathbb{E}_{r,t}\left[\frac{(t-r)(t+r-2rt)}{C_t(1)^3}\right] > 0,

because t−r>0t-r > 0, t+r−2rt=t(1−r)+r(1−t)>0t+r-2rt = t(1-r)+r(1-t) > 0, and Ct(1)≥1/2C_t(1) \ge 1/2 on 0≤r<t≤10 \le r < t \le 1. By contrast, the stop-gradient semi-gradient gsg(λ)=2E[Rλ∂λuλ]g_{\mathrm{sg}}(\lambda) = 2\mathbb{E}[R_\lambda \partial_\lambda u_\lambda] satisfies E[R1∣zt,r,t]=−(∂tX1+∂zX1v⋆(zt,t))=0\mathbb{E}[R_1 \mid z_t,r,t] = -(\partial_t X_1 + \partial_z X_1 v^\star(z_t,t)) = 0, restoring gsg(1)=0g_{\mathrm{sg}}(1) = 0.

Reviewer_LXCGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a "transport loss" for flow-map learning that it argues unifies MeanFlow, CM and sCM, and introduces GMRFlow, which combines analytic Gaussian component flow maps with a neural residual.

Critical strengths
  1. The question of when flow maps can be learned efficiently in high dimensions is well motivated and relevant to one-step generation.
Critical weaknesses
  1. The transport loss is not new. It is the Eulerian map distillation loss of [1], which already shows that the flow map minimizes it, bounds the generation error by it, and relates it to consistency distillation; [2] further treats CM and MeanFlow as special cases of the same formulation. The paper cites both only in passing and presents the loss as its own proposal.

  2. Assumption 5.1 requires the noised distributions to have bounded support, which fails under Gaussian interpolation, so Theorem 5.3 does not apply. The constants in Theorems 6.5–6.6 grow exponentially with the separation between mixture components, so the bound is polynomial only in the dimensions.

  3. The theory does not describe the proposed method. The analysis uses soft mixture weights and an Euler residual network, whereas GMRFlow uses hard DINO cluster labels and a U-Net. The ablations also run against the theory: a higher Gaussian rank, more components, or fitting the mixture in pixel space all worsen FID (Table 2).

  4. FFHQ64 has no MeanFlow baseline, the main comparison uses an off-diagonal fraction that the paper's own ablation finds worst, the best result (FID 24.11) uses a different recipe without a matching baseline, and on ImageNet-64 the hard-cluster baseline beats GMRFlow at one step (Table 6).

  5. The experiments are small-scale.

  6. Missing citations [3–7].

[1] Flow Map Matching with Stochastic Interpolants: A Mathematical Framework for Consistency Models. TMLR 2025.

[2] How to Build a Consistency Model: Learning Flow Maps via Self-Distillation. NeurIPS 2025.

[3] Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion. ICLR 2024.

[4] One Step Diffusion via Shortcut Models. ICLR 2025.

[5] Align Your Flow: Scaling Continuous-Time Flow Map Distillation. arXiv 2025.

[6] Gaussian Mixture Flow Matching Models. ICML 2025.

[7] The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications. TMLR 2024.

Questions for authors

See weakness.

7 comments

Weakness 1: Prior introduction of the EMD objective and our distinct contributions.

We thank the reviewer for highlighting the connection to Boffi et al. (2024; 2025). While Ltr(θ)\mathcal{L}_{\mathrm{tr}}(\theta) coincides with the Eulerian map distillation (EMD) objective of Boffi et al. (2024) and they already link this formulation to consistency distillation and MeanFlow, our focus is not on proposing a new objective, but on establishing a flow-map regression surrogate framework and finite-sample statistical learning theory in high dimensions:

  1. Global L2L^2 regression control & sharper constants (Theorem 4.2, Corollary 4.3): Unlike Boffi et al. (2024, Proposition 3.10), which uses Young's inequality and a Gronwall argument to bound the terminal W22W_2^2 distance with factor ee, our direct trajectory integration with Cauchy–Schwarz and Fubini–Tonelli controls the global L2L^2 flow-map regression error across all (r,t)∈Tϵ(r,t) \in \mathcal{T}_\epsilon (LL2(θ)≤14Ltr(θ)\mathcal{L}_{L^2}(\theta) \le \frac{1}{4}\mathcal{L}_{\mathrm{tr}}(\theta)) and sharpens the terminal W22W_2^2 constant from ee to (1−ϵ)2≤1(1-\epsilon)^2 \le 1.
  2. sCM equivalence & CM discretization bias (Section 4.2): Beyond the continuous-time links in Boffi et al. (2024; 2025), we prove exact equivalence (both objective and gradient) between sCM and stop-gradient MeanFlow, and establish a Θ(Δ2)\Theta(\Delta^2) discretization bias lower bound for CM (Theorem 4.5).

We have revised the Introduction (Section 1.1), Related Work (Section 2), and Section 4.1 to explicitly credit Boffi et al. (2024; 2025) and clarify our positioning.

Weakness 2: Bounded support in Assumption 5.1 and exponential dependence on component separation.

We thank the reviewer for these sharp theoretical observations. We address both points below:

  1. Removal of the bounded-support assumption (Assumption 5.1, Theorem 5.3, Appendix B): In the revised manuscript, Assumption 5.1 no longer assumes bounded support. Instead, we assume σ0\sigma_0-sub-Gaussian clean data pdatap_{\mathrm{data}} under unbounded Gaussian interpolation, and prove Theorem 5.3 via high-probability ball truncation at radius Rn=∥m0∥2+2σˉD+4log⁡(4n/δ)R_n = \Vert m_0\Vert _2 + 2\bar{\sigma}\sqrt{D + 4\log(4n/\delta)} (Lemmas B.1–B.2).

  2. Component separation, curvature parameter LL, and CelebA80 diagnostics (Theorems 6.6–6.7): In our analysis, separation between mixture components enters the bounds through the one-sided velocity Jacobian condition Dξv⪰−LIdD_\xi v \succeq -L I_d in Assumption 6.5, appearing as the prefactor eLe^L. While worst-case separation yields exponential prefactors, on real image distributions the effective negative curvature LL typically remains moderate and does not blow up with the ambient dimension DD.

    To empirically verify this on CelebA80 (D=6,400D=6,400), we measured the empirical local negative curvature ℓ(z,t)\ell(z,t) corresponding to LL in Assumption 6.5(v) across 256 validation images, as well as its trapezoidal integral ∫0.011ℓ(zt,t)dt\int_{0.01}^1 \ell(z_t,t) \mathrm{d}t along 8 ODE trajectories:

Model (D=6,400D=6,400)Intermediate t∈[0.4,0.67]t \in [0.4, 0.67] (Mean / P95 / Max)Near-Data t=0.01t=0.01 (Mean / P95 / Max)Trajectory Integral ∫0.011ℓdt\int_{0.01}^1 \ell \mathrm{d}t (Median / Max)
MeanFlow4.21 / 6.03 / 10.9440.83 / 58.83 / 69.006.35 / 7.19
GMRFlow4.19 / 6.00 / 10.5320.43 / 27.98 / 38.155.62 / 7.69

Despite the 6,400-dimensional ambient space, ℓ(z,t)\ell(z,t) remains single-digit at intermediate times and increases moderately near t=0.01t=0.01 in line with the theoretical ϵ−1\epsilon^{-1} scaling; moreover, the full trajectory integral stays around 5.65.6–6.46.4 in median because the near-data region occupies a narrow time window. GMRFlow's cluster conditioning removes cross-component switching, roughly halving the near-data curvature relative to unconditional MeanFlow. In this sense, the stability prefactor remains well-controlled in this 6,400-dimensional pixel space.

Weakness 3: Connection between theory and GMRFlow, and interpretation of Table 2 ablations.

We thank the reviewer for this comment. In the revised manuscript, we clarify how our theoretical model and GMRFlow relate and why Table 2 aligns with our framework:

  1. Complementary conditioning schemes (Section 6.2, Section 6.4, Section 8): We restructure Section 6.2 to show that both the analyzed soft-conditioned Euler network and the proposed hard-conditioned GMRFlow stem from the same hierarchical Gaussian mixture. Soft conditioning analyzes unlabeled marginal flow learning, capturing neural approximation and estimation errors, better reflecting how neural networks learn flow maps. Hard conditioning yields exact analytic Gaussian component maps ΦkG\Phi_k^G (Proposition 6.4) as an inductive bias; GMRFlow (Eq. 5) pairs per-cluster Gaussian maps X^kG\hat{X}_k^G with a shared U-Net residual FθF_\theta to absorb non-Gaussian deviations in real data.
  2. Table 2 ablations (Section 7.2):
    • Clustering rule: Pixel K-means (41.70) and Pixel EM (41.68) operate purely in pixel space without external representations and achieve nearly identical FID. DINO clustering (37.33) improves further because it groups images in a pretrained visual feature space, leveraging additional semantic similarity that raw Euclidean distances in pixel space cannot easily capture; thus, this gap reflects the benefit of semantic partitioning rather than a contradiction with pixel-space modeling.
    • Rank and KK: On FFHQ64 (n=70Kn=70\text{K}, D=12,288D=12{,}288), larger KK reduces the per-cluster sample count (n/Kn/K) and higher rank RR fits noisier low-variance directions into each empirical covariance Σ^k\hat{\Sigma}_k, introducing estimation noise into the analytic prior X^kG\hat{X}_k^G; compact priors (R=16,32R=16,32; K=4,8K=4,8) avoid this overfitting. This empirical trade-off mirrors the statistical complexity scaling in Theorem 6.7, where estimation error grows with the intrinsic rank and component count KK when sample size nn is fixed.

Weakness 4: Empirical baselines, training recipes, and ImageNet-64 results.

We thank the reviewer for carefully examining the experimental setup.

  1. Added FFHQ64 MeanFlow baseline (Table 1): We added the matched vanilla MeanFlow baseline on FFHQ64; under identical settings, GMRFlow lowers 1-NFE FID from 76.70 (MF) to 53.90, outperforming all baselines (CTM: 76.33, HC: 70.72).

  2. Fairness of the main comparison and role of Table 2 ablations (Section 7.2; Appendix F, Table 5): Table 1 evaluates all four variants under the exact same vanilla MeanFlow objective and identical hyperparameters (including the 0.75 off-diagonal fraction) to ensure a strictly fair comparison that isolates architectural gains. Table 2 uses a unified alternative recipe solely for internal design ablations across GMRFlow variants, and its 100K-update run (24.11 1-NFE / 16.80 2-NFE FID) reports extended training under that same ablation setup rather than a direct comparison against Table 1.

  3. ImageNet-64 and new single-Gaussian-per-class experiment (Appendix G, Table 6): In the original ImageNet-64 setup (100100 classes, 1,0001{,}000 images per class, D=12,288D=12{,}288), further partitioning each class into 55 subclusters (K=500K=500 total) leaves only ∼200\sim 200 samples per subcluster. While HC only consumes discrete cluster labels, GMRFlow (subcluster) relies on per-subcluster covariance estimates; with only ∼200\sim 200 high-dimensional samples per component, covariance estimation noise offsets the benefit of the analytic Gaussian prior, which is consistent with GMRFlow (subcluster) (67.4467.44) slightly trailing HC (66.9866.98) while still improving over MF (72.8672.86). Rather than tuning training hyperparameters, we kept the training recipe and architecture identical and switched to a prior better aligned with Theorem 6.7 and our FFHQ64 component-count ablations (Table 2): pooling all ∼1,000\sim 1{,}000 samples per class into a single Gaussian flow map per class (K=100K=100, GMRFlow (per-class)) mitigates this sample-per-component bottleneck and lowers 1-NFE FID to 62.86, outperforming MF (72.86), HC (66.98), and GMRFlow (subcluster) (67.44) by 10.00, 4.12, and 4.58 points, respectively.

Weakness 5: Experimental scale.

We thank the reviewer for this feedback. As our primary focus is establishing high-dimensional statistical learning theory for flow maps, our experiments are designed to provide strictly controlled validation directly in high-dimensional pixel space—spanning CelebA80 (100K100\text{K}, D=6,400D=6,400), FFHQ64 (70K70\text{K}, D=12,288D=12,288), and a 100-class ImageNet-64 benchmark (100K100\text{K}, D=12,288D=12,288; Appendix G). Training directly in pixel space avoids confounding factors from pretrained autoencoders and cleanly tests our theoretical mechanism for overcoming ambient dimension dependence.

Weakness 6: Missing citations [3–7].

We thank the reviewer for pointing out these relevant works. In the revised manuscript, we have added and discussed all five references in the corresponding sections.

Reviewer_6MWyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the theoretical learnability of neural flow maps for one-step generative modeling. The authors introduce a transport loss as a surrogate for direct flow-map regression, which is generally intractable because the ground-truth flow map is not available as a training target. The paper further establishes statistical learning guarantees under a low-rank Gaussian mixture assumption and proposes Gaussian Mixture Residual Flow (GMRFlow), which combines analytic Gaussian component flow maps with a learned neural residual correction. Experiments on CelebA80 and FFHQ64 demonstrate improved one-step generation performance over the considered baselines.

Critical strengths
  1. The transport-loss formulation provides a principled way to study flow-map learning without requiring direct supervision from the ground-truth flow map.

  2. The low-rank Gaussian mixture assumption allows the authors to derive learning guarantees with more favorable dependence on intrinsic and latent dimensions, highlighting the potential importance of exploiting data structure.

  3. Experiments on CelebA80 and FFHQ64 demonstrate improved one-step generation performance over the considered baselines.

Critical weaknesses

The paper develops a theoretical framework connecting the transport loss to flow-map regression and existing one-step learning objectives, including MeanFlow and consistency-based methods. However, it is not clear whether the proposed formulation provides a practical optimization advantage over existing objectives, particularly MeanFlow. The theoretical characterization of a loss and its relationship to the desired regression target do not necessarily imply better optimization, convergence, or sample quality in practical neural-network training. Moreover, the main GMRFlow comparisons use the same MeanFlow objective across the considered model variants, so the reported FID improvements do not isolate the contribution of the transport-loss formulation itself.

The paper establishes more favorable learning bounds under a low-rank Gaussian mixture assumption. This is an interesting theoretical result, but the guarantees rely on substantial structural assumptions about the data distribution and the corresponding flow map. It remains unclear how well these assumptions characterize the image distributions considered in the experiments and how sensitive the theoretical advantages are to deviations from the assumed structure. In particular, the improvement in statistical complexity appears to arise from the combination of low-dimensional structure and the Gaussian mixture assumption, rather than from the mixture representation alone.

Questions for authors

Can the authors provide more controlled ablations to separate the effects of residual parameterization, cluster conditioning, and Gaussian component flow maps?

Why are experiments not conducted on ImageNet at 256×256 resolution? Since ImageNet provides more diverse semantic categories and visual structures than face-image datasets, could the authors evaluate the method on this benchmark to better assess its generality and scalability?

4 comments

Thank you for your valuable comments and constructive suggestions. We provide our detailed response to each point below. For the revised paper, we highlight the newly added or updated parts in blue.

Weakness 1: The role of the transport loss, its equivalence to MeanFlow, and the source of practical optimization gains.

Rather than proposing a competing training objective to replace MeanFlow, we introduce the transport loss Ltr(θ)=E[∥DXθ(zt,r,t)∥2]\mathcal{L}_{\mathrm{tr}}(\theta) = \mathbb{E}[\Vert \mathcal{D}X_\theta(z_t, r, t)\Vert ^2] as a unifying theoretical surrogate framework to explain and analyze why continuous one-step objectives work. Under the standard average-velocity parameterization Xθ(zt,r,t)=zt−(t−r)uθ(zt,r,t)X_\theta(z_t, r, t) = z_t - (t-r)u_\theta(z_t, r, t), the oracle MeanFlow objective LMF(θ)\mathcal{L}_{\mathrm{MF}}(\theta) is algebraically identical to the transport loss (LMF(θ)=Ltr(θ)\mathcal{L}_{\mathrm{MF}}(\theta) = \mathcal{L}_{\mathrm{tr}}(\theta)). Establishing this equivalence allows us to prove that optimizing MeanFlow upper-bounds the intractable L2L_2 flow-map regression error (LL2(θ)≤14Ltr(θ)\mathcal{L}_{L^2}(\theta) \le \frac{1}{4}\mathcal{L}_{\mathrm{tr}}(\theta) in Theorem 4.2). Hence the transport loss framework aims to explain the effectiveness of continuous one-step losses including MeanFlow loss, not beating them.

Furthermore, because MeanFlow is already an exact instantiation of the transport loss Ltr\mathcal{L}_{\mathrm{tr}}, the practical optimization and sample-quality gains in our framework arise not from changing the loss formula, but from our theory-motivated structural parameterization of the flow map XθX_\theta.

Weakness 2: Characterization of real image distributions, the complementary roles of low-dimensional and mixture assumptions, and sensitivity to structural deviations.

We thank the reviewer for this insightful question. We clarify the complementary roles of our structural assumptions and how our guarantees behave under model misspecification:

  1. Complementary roles of subspace and mixture structures: The statistical gain stems from two distinct mechanisms: (i) the linear subspace VV decouples orthogonal ambient noise, reducing the effective dimension from DD to d≪Dd \ll D; (ii) the low-rank Gaussian mixture models the curved data manifold embedded inside VV via multiple local Gaussian approximations.

    Empirically, our CelebA spectral diagnostics (Section 7.1) confirm strong linear concentration in pixel space. Moreover, Gaussian mixtures are widely used to characterize image distributions [1, 2]. Wang et al. [1] (Section 2.4, Table 1) show on real image benchmarks that analytical mixtures of low-rank Gaussians capture the coarse structure of trained U-Nets much better than a single full-rank Gaussian. Across 10K10\text{K} shared noise inputs, they avoid severe blurring and consistently achieve a smaller average Euclidean distance to the U-Net outputs:

DatasetSingle Full-Rank GaussianMixture of Low-Rank Gaussians (MoLRG)
FashionMNIST72.6257.56
MNIST69.1262.53
CIFAR-1033.5531.29
FFHQ36.7535.78
  1. Population-level stability under misspecification and connection to GMRFlow: While our finite-sample rates are established under the stated low-rank mixture assumptions, Theorem 4.2 also yields a population-level stability bound under velocity misspecification. Let pp be the true data distribution and qq an admissible low-rank Gaussian mixture, with velocities vtrue,vqv_{\mathrm{true}},v_q and exact flow maps Xtrue,XqX_{\mathrm{true}},X_q. Suppose E(r,t)∼ν,zt∼pt[∥vtrue(zt,t)−vq(zt,t)∥2]≤εmis2\mathbb{E}_{(r,t)\sim\nu, z_t\sim p_t}[\Vert v_{\mathrm{true}}(z_t,t) - v_q(z_t,t)\Vert ^2] \le \varepsilon_{\mathrm{mis}}^2, where ν\nu is the time-sampling distribution in Theorem 4.2. Under the one-sided Jacobian condition on vqv_q in Assumption 6.5, backward-flow stability gives ∥DzXq∥op≤eL\Vert D_z X_q\Vert _{\mathrm{op}} \le e^L. Since DqXq=0\mathcal{D}_qX_q=0, we have DpXq=DzXq(vtrue−vq)\mathcal{D}_pX_q=D_zX_q(v_{\mathrm{true}}-v_q). Applying Theorem 4.2 under the true distribution therefore yields

E(r,t)∼ν,zt∼pt[∥Xq(zt,r,t)−Xtrue(zt,r,t)∥2]≤14Ltr,p(Xq)≤14e2Lεmis2.\mathbb{E}_{(r,t)\sim\nu, z_t\sim p_t}\big[\Vert X_q(z_t,r,t)-X_{\mathrm{true}}(z_t,r,t)\Vert ^2\big] \le \frac{1}{4}\mathcal{L}_{\mathrm{tr},p}(X_q) \le \frac{1}{4}e^{2L}\varepsilon_{\mathrm{mis}}^2.

This establishes stability of the population flow-map target; extending the finite-sample bound in Theorem 6.7 to misspecified data requires additional analysis.

  1. Validation from our image generation experiments (Section 7.2, Tables 1–2; Appendix G, Table 6): Our generative experiments further confirm that low-rank Gaussian mixture priors capture useful structure on real image distributions. On FFHQ64 (Table 1), adding the analytic low-rank Gaussian component maps X^kG\hat{X}_k^G on top of cluster conditioning (HC →\to GMRFlow) reduces 1-NFE FID by 16.82 points (70.72→53.9070.72 \to 53.90). Likewise, on the 100-class ImageNet-64 subset (Appendix G, Table 6), fitting one Gaussian flow map per class (GMRFlow (per-class)) lowers 1-NFE FID to 62.86, outperforming both HC (66.98) and MF (72.86).

Question 1: Disentangling the effects of residual parameterization, cluster conditioning, and analytic Gaussian component maps.

We thank the reviewer for this helpful suggestion. In fact, the four controlled variants already evaluated in Section 7.2 (Table 1), now including the matched vanilla MeanFlow baseline on FFHQ64, form a progressive ablation chain under the exact same MeanFlow training recipe and U-Net backbone that isolates each design component:

VariantParameterization XθX_\thetaPreconditioned ResidualCluster Cond. (kk)Gaussian Map (X^kG\hat{X}_k^G)CelebA80 1-NFE FIDFFHQ64 1-NFE FID
MF (FFHQ added)z−(t−r)uθ(z,r,t)z - (t-r)u_\theta(z,r,t)NoNoNo59.7676.70
CTMrtz+(1−rt)Gθ(z,r,t)\frac{r}{t}z + (1-\frac{r}{t})G_\theta(z,r,t)YesNoNo55.1876.33
HCrtz+(1−rt)Gθ(z,r,t;k)\frac{r}{t}z + (1-\frac{r}{t})G_\theta(z,r,t;k)YesYesNo50.5570.72
GMRFlowX^kG(z,r,t)+(1−rt)cout(t)Fθ(z,r,t;k)\hat{X}_k^G(z,r,t) + (1-\frac{r}{t})c_{\mathrm{out}}(t)F_\theta(z,r,t;k)YesYesYes40.6553.90

This progression isolates the individual contribution of each component across both CelebA80 and FFHQ64:

  1. Residual parameterization (MF→CTM\text{MF} \to \text{CTM}): Switching from standard average-velocity parameterization to the preconditioned residual parameterization yields a modest baseline improvement on both datasets.
  2. Cluster conditioning (CTM→HC\text{CTM} \to \text{HC}): Supplying the cluster index kk to the residual parametrization improves generation quality, showing the benefit of partitioning multimodal data.
  3. Analytic Gaussian component map (HC→GMRFlow\text{HC} \to \text{GMRFlow}): Replacing the rtz\frac{r}{t}z term in the residual parametrization with the cluster-conditioned Gaussian component flow map X^kG\hat{X}_k^G yields the largest single improvement on both datasets, confirming that letting the analytic Gaussian prior X^kG\hat{X}_k^G capture the coarse component structure makes the remaining residual much easier for FθF_\theta to learn.

Question 2: Evaluation on ImageNet at 256×256 resolution and generality across diverse semantic categories.

We thank the reviewer for raising this important point regarding semantic diversity and scalability. We clarify our experimental scope and ImageNet validation below:

  1. Because our primary focus is establishing high-dimensional statistical learning theory for flow maps, our experiments are designed for controlled validation directly in pixel space from scratch, rather than pursuing state-of-the-art benchmark performance.
  2. To directly assess generality beyond face datasets (CelebA80 and FFHQ64) on diverse semantic categories and visual structures, we evaluate our method in D=12,288D=12{,}288 pixel space on a 100-class subset of ImageNet-64 (100K100\text{K} training images, 1,0001{,}000 per class; Appendix G, Table 6). Under fixed 100K updates, GMRFlow (per-class) achieves 62.86 1-NFE FID, outperforming MF (72.86) by 10.0010.00 points and HC (66.98) by 4.124.12 points (while GMRFlow (subcluster) at 67.4467.44 reflects the component-count vs. sample-size (K/nK/n) trade-off highlighted in Theorem 6.7).
  3. Scaling pixel-space GMRFlow to higher resolutions such as 256×256256\times 256 and full ImageNet-1K remains an important direction for future work.

[1]. Wang, P., Zhang, H., Zhang, Z., Chen, S., Ma, Y., & Qu, Q. (2026). Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions. Journal of Machine Learning Research, 27(179), 1-50.

[2]. Yang, R., Li, Y., Jiang, B., Chen, C., & Li, S. (2026). Multi-subspace multi-modal modeling for diffusion models: Estimation, convergence and mixture of experts. arXiv preprint arXiv:2601.01475.

Thank you for your detailed responses and clarifications. Your explanations have been helpful in improving my understanding of the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

Dear Area Chair,

Thank you for your time and effort in handling our submission. To facilitate a quick review of our revisions and the discussion outcomes, we provide a concise summary below.

We are grateful to all reviewers for their constructive feedback throughout the discussion. Reviewer K8my raised their rating from 2 to 3, noting that our additional theory and experiments addressed many of their concerns. Reviewer 6MWy (rating 3) also noted that our clarifications improved their understanding of the paper. Reviewer LXCG maintained a rating of 2.

The key updates in the revised manuscript (highlighted in blue) address the shared concerns raised across reviews, including prior-work positioning, removing the bounded-support assumption, extending our guarantees to practical conditional-velocity training, and adding matched baselines and multi-metric evaluations. Below, we summarize these advances and clarify the remaining disagreement with Reviewer LXCG regarding the theoretical contribution and the role of the experiments.

(a) Theoretical Contributions Beyond the Transport-Loss Formulation

We explicitly credit Boffi et al. for the EMD objective. Our main theoretical contributions lie in (i) a flow-map regression surrogate perspective and (ii) finite-sample learnability under intrinsic data structure and the oracle transport loss, which further extends to the practical improved MeanFlow (iMF) training objective with observable conditional-velocity labels (Theorem 4.4, Corollary 6.9). We establish polynomial dependence on the intrinsic and latent dimensions with fixed regularity constants (e.g., velocity spatial curvature LL), thereby overcoming exponential dependence on the ambient dimension DD and escaping the curse of dimensionality, which is standard in the scope of statistical learning theory. Our diagnostics on CelebA80 (D=6400D=6400) further indicate that the empirical velocity curvature remains moderate in high-dimensional pixel space.

(b) Relationship Between GMRFlow and the Theory

We clarify that the analyzed soft-conditioned network and hard-conditioned GMRFlow stem from the same low-rank Gaussian mixture model, serving complementary roles. We focus our finite-sample learning analysis (Theorem 6.7) on the soft-conditioned network (which approximates the nonlinear marginal flow map without supplied component labels) because this setting captures both neural approximation and statistical estimation errors, better reflecting how neural networks learn flow maps. While hard conditioning shifts the statistical burden to external component identification, it provides a direct way to encode a Gaussian-mixture inductive bias into one-step generative models via exact analytic component maps (Proposition 6.4). Under the subspace assumption, this analytic prior absorbs the ambient-space scaling so that GMRFlow's neural residual only needs to learn low-dimensional deviations within the data subspace (Assumption 6.1, Proposition 6.2), and our general L2L^2 regression and W22W_2^2 generation error bounds (Theorem 4.2, Corollary 4.3) apply directly to GMRFlow.

(c) Assessing the Experiments Against the Scope of a Theory-Driven Paper

Reviewer LXCG's remaining objection centers on the gap between our reported FIDs and SoTA results. We acknowledge that GMRFlow does not aim to establish SoTA generation performance. Our primary goal is to establish statistical learnability and examine whether the theoretically motivated residual construction is effective under controlled comparisons.

For a theoretical work, our empirical validation is extensive, spanning three pixel-space datasets (CelebA80, FFHQ64, and a 100-class ImageNet-64 subset) and addressing several distinct scientific questions: (i) spectral and ODE-trajectory curvature diagnostics examine the plausibility of our assumptions; (ii) progressive MF–CTM–HC–GMRFlow comparisons under matched recipes isolate each architectural component; (iii) ablations on covariance rank, cluster count, and clustering methods evaluate the design choices of the Gaussian prior; and (iv) our follow-up experiments provide multi-metric evaluations (KID, Precision, Recall, IS) across all three datasets.

Together, these results provide empirical support for our theoretical predictions and the GMRFlow construction. Scaling to large ImageNet-256 benchmarks is a valuable engineering direction for future work, but its absence does not diminish the theoretical advances or the controlled empirical evidence already established.

We thank the Area Chair and reviewers again for their careful consideration and constructive discussion. We hope the paper will be assessed on its theoretical contributions and controlled empirical validation rather than proximity to SoTA ImageNet-256 FID.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.