acceptodds
Under review as a conference paper at ICLR 2027

How Many Steps Does Flow Matching Really Need? A Frequency-Domain Analysis of Discretization Error in the Base-ODE Regime

Abstract

Flow matching generates high-quality images in as few as 4–50 ODE integration steps, yet existing convergence theory provides only scalar bounds that treat all frequencies equally. Under base-ODE sampling (guidance ), we derive an explicit frequency-decomposed characterization of discretization error for polynomial ODE solvers applied to the flow matching interpolant, showing that the ODE acceleration scales as for a target power spectrum , producing a spectral ceiling above which high-frequency content is exponentially attenuated. We validate this prediction exactly on Gaussian targets () and reproduce the predicted convergence pattern on five frontier text-to-image models spanning five independent teams (SD3.5-Medium 2.5B, FLUX.1-dev 12B, AuraFlow v0.3, CogView4-6B, Lumina-Image-2.0), with monotonic high-band recovery in latent space and midpoint dominating Euler at matched NFE in every case (a – low-NFE advantage). A single reflow iteration eliminates the ceiling entirely on a controlled synthetic setting, isolating ODE trajectory curvature as the mechanism. Distillation (FLUX.1-schnell) also eliminates the ceiling but reverses the frequency-band convergence order, providing a spectral signature of the distillation mechanism. We further report an overshoot drift at large step counts (), where solvers exceed the reference power by +, which an exact-velocity control experiment attributes to learned velocity-field error rather than discretization. These findings yield actionable guidelines for the base-ODE regime: the midpoint solver recovers – more high-frequency energy per function evaluation than Euler, and the optimal noise-schedule shift lies in across models.

Primary area:
learning theory
Keywords:
flow matching, diffusion models, discretization error, frequency-domain analysis, ODE solvers, spectral methods, image generation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_Bf1tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper studies frequency-dependent discretization error in flow matching. In a Gaussian setting, it derives frequency-dependent acceleration and a spectral cutoff that changes with the number of sampling steps.
  • Experiments on five pretrained image-generation models examine spectral recovery, solver choice, and timestep schedules. Further experiments investigate reflow, distillation, and spectral overshoot at large step counts.
Critical strengths
  • Frequency-resolved diagnostics reveal behavior that aggregate metrics can miss. The observation that VAE decoding can reverse the measured spectral trend is particularly useful.
  • The exact Gaussian setting provides a controlled starting point. The reference-convergence checks also expose an important source of error in evaluating sampling methods.
Critical weaknesses
  • The higher-order scaling contradicts itself: the main text says solver order preserves the cutoff exponent, while Appendix A.5 predicts N4/βN^{4/\beta} for midpoint.
  • Appendix A.5 also contains an algebraic inconsistency. A correction proportional to N−2∣k∣βN^{-2}|k|^\beta gives kc∝N2/βk_c\propto N^{2/\beta}, not N4/βN^{4/\beta}.
  • Appendix A.4 restricts the expansion to frequencies well below the cutoff. Setting that expansion to order one does not, by itself, establish its accuracy at the cutoff.
  • The additional-model experiments use Euler-50 references without the convergence checks reported for SD3.5 and FLUX. Their absolute spectral-recovery and overshoot measurements therefore remain reference-dependent.
  • Power above a finite-step reference does not establish error in the learned velocity field. The synthetic exact-velocity control supports a possible explanation, but does not isolate the cause of overshoot in the pretrained models.
Questions for authors
  • What is the correct midpoint cutoff scaling, and how should Appendix A.5 be reconciled with the main text?
  • Do the additional-model solver rankings and overshoot observations persist against independently converged references?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Bf1t: Midpoint Scaling & ISD Bridge, Theorem 1 Bound, & Extended-Model References

We thank Reviewer Bf1t for recognizing the methodological importance of latent-space measurement, our multi-team frontier validation, and the SOSS negative result. Every weakness and question you raised pointed to places needing tighter rigor, all now incorporated into the revised PDF.


1. Midpoint Scaling Contradiction, N2/βN^{2/\beta} vs. Frontier Data, & ISD Bridge (W1, W2, Q1)

  1. Resolution of A.5 (Theorem 2, §3.2, App. A.3): Appendix A.5's N4/βN^{4/\beta} derivation was erroneous. For explicit midpoint on x^˙k=c(t,k)x^k\dot{\hat{x}}_k = c(t,k)\hat{x}_k, ∫01c(t,k)3 dt=−14(P−1−P)\int_0^1 c(t,k)^3\,\mathrm{d}t = -\frac{1}{4}(P^{-1}-P) cancels [c′/24+c2/8]01=124(P−1−P)[c'/24 + c^2/8]_0^1 = \frac{1}{24}(P^{-1}-P), making the nominal O(N−2)∣k∣βO(N^{-2})|k|^\beta global log-error vanish identically. The leading error is third-order, ln⁡Rmid(k,N)=−π128∣k∣3β/2C3/2N3(1+o(1))\ln R_{\mathrm{mid}}(k,N) = -\frac{\pi}{128}\frac{|k|^{3\beta/2}}{C^{3/2}N^3}(1+o(1)), giving (3β/2)/3=β/2  ⟹  kcmid(N)∝N2/β(3\beta/2)/3 = \beta/2 \implies k_c^{\mathrm{mid}}(N) \propto N^{2/\beta}—the exact same exponent as Euler, with a β\beta-independent 3.54×3.54\times step reduction (1.77×1.77\times NFE). Table 1 verifies δmid=2/β\delta_{\mathrm{mid}} = 2/\beta across (β,d)(\beta, d) to R2=0.9999R^2 = 0.9999 (1.386,1.023,0.6771.386, 1.023, 0.677 vs. 1.333,1.000,0.6671.333, 1.000, 0.667; classical RK4 rebounds past 11 with min⁡kR≈0.934>e−1\min_k R \approx 0.934 > e^{-1}, so N2p/βN^{2p/\beta} is withdrawn).
  2. Why kc∝N2/βk_c \propto N^{2/\beta} cannot be fit on 64×6464\times 64 latents, & the ISD\mathrm{ISD} bridge (§3.3, App. D.10): With K=31K=31 integer bins, frontier kck_c jumps from 0–10\text{--}1 (N=2–4N=2\text{--}4) to saturation (3131, N=5–12N=5\text{--}12), leaving 2–62\text{--}6 quantized points. Instead, ISD(N)\mathrm{ISD}(N) provides a parameter-free bridge: asymptotically (N→∞N \to \infty, fixed KK), Theorems 1–2 give ISDEuler∝N−1\mathrm{ISD}_{\mathrm{Euler}} \propto N^{-1} (α→1\alpha \to 1) and ISDmid∝N−3\mathrm{ISD}_{\mathrm{mid}} \propto N^{-3} (α→3\alpha \to 3). At finite K=31K=31 over N∈[2,50]N \in [2,50], exact Gaussian recursions at β∈[1.95,2.0]\beta \in [1.95, 2.0] predict αEuler∈[0.31,0.58]\alpha_{\mathrm{Euler}} \in [0.31, 0.58] and αmid∈[1.43,2.03]\alpha_{\mathrm{mid}} \in [1.43, 2.03] (reaching 2.50–2.58>22.50\text{--}2.58 > 2 on [5,50][5,50] toward α→3\alpha \to 3), matching SD3.5 (0.460.46 / 1.341.34) and FLUX-dev (0.660.66 / 1.201.20) with zero free parameters.

2. Rigor of Appendix A.4 & Non-Asymptotic Attenuation Bound (W3)

We replaced A.4's informal telescoping remark and decay phrasing with Theorem 1 (App. A.2): using gi(k)=(N−i)2+i2P(k)g_i(k) = (N-i)^2 + i^2 P(k) and gigi+1−φi2=N2P(k)g_i g_{i+1} - \varphi_i^2 = N^2 P(k) yields REuler(k,N)=∏i=0N−1 ⁣(1−N2P(k)gi(k) gi+1(k))  ≤  exp⁡Φ(N2P(k)),Φ(ρ)=∑j=1∞ln⁡ ⁣(1−ρ(ρ+j2)(ρ+(j−1)2)),R_{\mathrm{Euler}}(k,N) = \prod_{i=0}^{N-1}\!\left(1 - \frac{N^2 P(k)}{g_i(k)\,g_{i+1}(k)}\right) \;\le\; \exp\Phi\bigl(N^2 P(k)\bigr), \quad \Phi(\rho) = \sum_{j=1}^{\infty}\ln\!\left(1 - \frac{\rho}{(\rho+j^2)(\rho+(j-1)^2)}\right), valid for every N≥1N \ge 1 and all k≠0k \ne 0. Since Φ(ρcEuler)=−1\Phi(\rho_c^{\mathrm{Euler}}) = -1 at ρcEuler≈0.7030\rho_c^{\mathrm{Euler}} \approx 0.7030, N2P(k)≤ρcEulerN^2 P(k) \le \rho_c^{\mathrm{Euler}} guarantees REuler(k,N)<e−1R_{\mathrm{Euler}}(k,N) < e^{-1} non-asymptotically.


3. Extended-Model References & Per-Model Guidance (W4 & Q2)

  1. Extended-model reference caveat (§4.2, App. D.6): Unlike SD3.5/FLUX-dev (evaluated against uniform-midpoint references; see Table 5), AuraFlow, CogView4, and Lumina2 used default-shift Euler-5050 references without an N=100N=100 midpoint check. Internal table values confirm Euler-5050 is slightly under-converged: midpoint crosses rhigh=1.0r_{\mathrm{high}} = 1.0 by N=3–8N=3\text{--}8 on all three, and Lumina2 Euler reaches 1.0451.045 at N=20N=20. Crucially, matched-NFE Euler-vs.-midpoint ratios at fixed NN divide by the identical reference PSD, so the reference cancels mode-by-mode (App. D.7 also confirms schedule-ranking invariance). We added this caveat in §4.2/App. D.6 and excluded extended-model rhigh>1r_{\mathrm{high}} > 1 from large-NN drift claims.
  2. Guidance (§4.1, App. B.2): SD3.5, AuraFlow, CogView4, and Lumina2 run at w=1.0w=1.0; single-pass guidance-distilled FLUX.1-dev runs at its native scale 3.53.5 (Meng et al., 2023); FLUX.1-schnell runs at w=0w=0.

4. Large-NN Drift Mechanism (W5)

Agreed: linear float64 Gaussian controls (REuler≤1R_{\mathrm{Euler}} \le 1 proved in Theorem 1(i)) cannot separate nonlinear velocity error from bfloat16 arithmetic. §4.4 and Fig. 3 now frame this as broadband spectral drift (rlow=1.21,rmid=1.49,rhigh=1.61r_{\mathrm{low}}=1.21, r_{\mathrm{mid}}=1.49, r_{\mathrm{high}}=1.61 at N=200N=200, continuing through N=400N=400 and σmin⁡=0.02\sigma_{\min}=0.02 clipping), with bfloat16 step accumulation (Lewandowski et al., 2026; He et al., 2023) and small-σ\sigma velocity bias as explicit candidate mechanisms.

AuthorsOpenReview ↗

Follow-Up to Bf1t: Uniform Midpoint N=100 Checks Across All Three Extended Models (W4) & fp32 Precision Ablation (W5)

Following up on our initial reply, we have now completed both empirical follow-ups addressing W4 (additional-model reference convergence) and W5 (mechanism behind large-NN drift):

  1. Uniform midpoint N=100 reference checks on all three extended models (W4, outputs/a1v2_solvers_{cogview4,lumina2,auraflow}_convcheck.json, n=64,512px,w=1.0n=64, 512\text{px}, w=1.0): Re-evaluating CogView4-6B, Lumina-Image-2.0, and AuraFlow-v0.3 with converged uniform midpoint N=100 (NFE=200) references eliminates every prior >1.0>1.0 ratio observed against default-shift Euler-50 references (previously 1.070/1.062 on CogView4, 1.297 midpoint / 1.045 Euler on Lumina2, and 1.099/1.110 on AuraFlow). All curves remain strictly <1.0<1.0 and monotonic in NN, and Midpoint consistently outperforms Euler at matched NFE:

    • CogView4-6B: Euler (N=20,28,50,100N=20,28,50,100) latent rhigh=0.8384,0.8772,0.9169,0.9722r_{\mathrm{high}} = 0.8384, 0.8772, 0.9169, 0.9722 (ISD:0.1707→0.0303\mathrm{ISD}: 0.1707 \to 0.0303); Midpoint (N=20,28,50N=20,28,50) rhigh=0.9647,0.9674,0.9689r_{\mathrm{high}} = 0.9647, 0.9674, 0.9689 (ISD:0.0336→0.0260\mathrm{ISD}: 0.0336 \to 0.0260). At NFE=40\text{NFE}=40, Midpoint (0.9647) exceeds Euler NFE=50\text{NFE}=50 (0.9169) with 5.1×5.1\times lower ISD at N=20N=20.
    • Lumina-Image-2.0: Euler (N=20,28,50,100N=20,28,50,100) rhigh=0.7471,0.8066,0.8716,0.9679r_{\mathrm{high}} = 0.7471, 0.8066, 0.8716, 0.9679 (ISD:0.3500→0.0703\mathrm{ISD}: 0.3500 \to 0.0703); Midpoint (N=20,28,50N=20,28,50) rhigh=0.9294,0.9354,0.9401r_{\mathrm{high}} = 0.9294, 0.9354, 0.9401 (ISD:0.0983→0.0635\mathrm{ISD}: 0.0983 \to 0.0635). The spurious Lumina2 Euler@20 >1>1 artifact (1.0448) disappears completely.
    • AuraFlow-v0.3: Euler (N=20,28,50,100N=20,28,50,100) rhigh=0.6247,0.6979,0.7918,0.9309r_{\mathrm{high}} = 0.6247, 0.6979, 0.7918, 0.9309 (ISD:0.3781→0.0727\mathrm{ISD}: 0.3781 \to 0.0727); Midpoint (N=20,28,50N=20,28,50) rhigh=0.8864,0.9008,0.9194r_{\mathrm{high}} = 0.8864, 0.9008, 0.9194 (ISD:0.1112→0.0787\mathrm{ISD}: 0.1112 \to 0.0787).
  2. Precision ablation isolating the large-NN drift mechanism (W5, outputs/sd35_precision_drift_ablation.json): We compared matched-seed SD3.5 (Euler, uniform, n=16, N=50..400) under bf16_default, bf16_vel_fp32_acc (bf16 velocity forward pass + fp32 ODE accumulator), and full_fp32. Relative to full_fp32@100 (band_high=0.0604), bf16_default inflates high-band power 2.221x at N=200N=200 and 4.424x at N=400N=400 (band_mid: 1.946x / 3.428x). Retaining only the ODE step accumulator in fp32 (bf16_vel_fp32_acc) eliminates >97%>97\% of the high-band inflation (1.047x at N=200N=200, 1.069x at N=400N=400), nearly matching full_fp32 (1.034x / 1.054x). This experimentally separates numerical step-accumulation precision from learned velocity error as the dominant driver of large-NN broadband drift.

AuthorsOpenReview ↗

Response to Reviewer Bf1t: Midpoint Scaling, Exact Bound, 100-Step Reference Checks, & fp32 Ablation

We thank Reviewer Bf1t for highlighting the value of our latent-versus-pixel observation, multi-model evaluation, and reference-convergence checks. Because our earlier reply suffered from OpenReview formula-rendering glitches, below is a clean, plain-language summary of how every weakness (W1–W5) and question (Q1–Q2) is resolved in the newly polished manuscript (main.pdf).

1. Reconciling Midpoint Cutoff Scaling & Bridging to 64x64 Frontier Latents (W1, W2, Q1; §3.2–§3.4, Theorem 2, Table 1, Apps. A.3 & D.9)

  • Analytical reconciliation (W1, W2): Both points you raised on old Appendix A.5 were spot-on: it contradicted §3.2 and contained an algebra slip (a second-order k^beta / N^2 error already implies N^(2/beta), not N^(4/beta)). Re-deriving explicit midpoint (Theorem 2, App. A.3) uncovered an even cleaner property: along the flow-matching path, midpoint's second-order global error cancels completely between the trajectory interior and endpoints, leaving a third-order leading error (k^(3*beta/2) / N^3). Dividing both exponents by 3 yields the exact same N^(2/beta) cutoff exponent as Euler (Table 1, R^2 = 0.9999), while boundary rescaling near t = 1 proves midpoint reaches any target cutoff in 3.54x fewer steps (1.77x fewer evaluations) regardless of beta. (Classical RK4 rebounds above 1/e past its stability limit, scoping the monotonic 1/e cutoff to first- and second-order polynomial solvers.)
  • Parameter-free bridge to 64x64 latents (Q1): Because 64x64 latents have only 31 radial rings, the 1/e cutoff reaches the grid edge within 5–12 steps (leaving 2–6 quantized integer points, too coarse to fit a power-law cutoff exponent directly). Instead, full-spectrum distance (ISD, §3.3) connects theory to frontier models with zero free parameters: evaluating exact Gaussian recursions on a 31-ring grid over N = 2..50 predicts effective decay slopes of 0.31–0.58 (Euler) and 1.43–2.03 (midpoint, App. D.9), matching SD3.5-Medium (0.46 vs. 1.34) and FLUX.1-dev (0.66 vs. 1.20).

2. Non-Asymptotic Attenuation Bound Across the Cutoff (W3; §3.2, Theorem 1, App. A.2) We replaced Appendix A.4's small-error expansion argument with an exact closed-form product identity (Theorem 1) valid for every N >= 1 and every frequency. Bounding each step factor directly from t = 1 proves—with zero small-error or asymptotic approximations—that retained power falls strictly below 1/e whenever N^2 * P(k) <= 0.7030, establishing the N^(2/beta) cutoff rigorously right across the threshold.

3. Converged 100-Step Midpoint Reference Checks on All Three Additional Models (W4 & Q2; §4.2, App. D.6) Directly answering Q2: we re-evaluated AuraFlow v0.3, CogView4-6B, and Lumina-Image-2.0 (n = 64 matched seeds, unguided w = 1.0, FLUX.1-dev at its native embedded scale 3.5, FLUX.1-schnell at w = 0; §4.1, App. B.2) against converged 100-step uniform-midpoint (200 NFE) references (App. D.6):

  • Every prior > 1.0 overshoot value (caused solely by normalizing against slower-converging 50-step shifted Euler baselines, including Lumina-Image-2.0 Euler 1.05 at N = 20) disappears completely.
  • Across N = 20, 28, 50, all Euler and midpoint curves stay strictly below 1.0, converge monotonically with N, and preserve midpoint's strong advantage at matched budgets: at 40 evaluations (20-step midpoint), high-frequency recovery reaches 0.965 (CogView4-6B), 0.929 (Lumina-Image-2.0), and 0.886 (AuraFlow v0.3), beating 50-step Euler (50 evaluations: 0.917, 0.872, and 0.792) on every model.

4. Experimental Isolation of the Large-N Drift Mechanism (W5; §4.4, Figure 4, App. D.10) Directly resolving W5: holding the SD3.5-Medium velocity network fixed on matched seeds (N = 50..400), comparing default bfloat16, full float32, and hybrid precision (bfloat16 velocity forward pass + float32 ODE state accumulator) separates numerical rounding from velocity-field error. Default bfloat16 inflates high-frequency power 2.22x at N = 200 and 4.42x at N = 400 (vs. full float32 at N = 100). Keeping only the ODE state accumulator in float32 suppresses N = 400 inflation to 1.07x (matching full float32 at 1.05x), eliminating over 97% of the overshoot at zero runtime overhead and establishing bfloat16 step-accumulation rounding as the primary cause.

Reviewer_fmuoOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies discretization error in lower NFE regime for flow matching in the frequency domain. The authors assume a multivariate Gaussian target distribution with covariance diagonal in the Fourier basis, and derive the acceleration of each Fourier mode and a “spectral ceiling,” a frequency cutoff defined by a 1/e power spectral ratio threshold for a N Euler steps. They also compare output power spectra against references (from high NFE) on multiple pretrained T2I models and conduct analyses such as solver type (Euler vs RK2 midpoint), noise schedules, reflow, overshoot, etc.

Critical strengths

The paper analyzes the discretization error of flow matching in the spectral domain, which I found somewhat interesting. Under Gaussian assumption, it derives how the numerical error depends on frequency and N step, consistent with Table 1. On several pretrained models, the authors compute latent-space PSDs and decompose the error by frequency, showing a consistent pattern of the error is larger at high frequencies and shrinks as N grows (which is expected).

Critical weaknesses

The major issue is that the manuscript reads like an AI-assisted research report that still needs careful human verification. Beyond the multiple inaccuracies and inconsistencies that need to be addressed, I still don’t get what is the main take-home message here.

Part of writings are wordy and difficult to follow, sometimes with quite a longshot context, and several connections are not sufficiently justified. There are also overstatements. For example, regarding overshoot, the absence of overshoot in an exact Gaussian control does not single out learned velocity-field error as the cause in complex real world models.

Also observe multiple inconsistencies: for example, on ceiling exponent, Sec 3.2 mentions as 2/β2/\beta for higher-order solvers, where as Appendix A.5 tells 4/β4/\beta for midpoint. In addition, B.4 uses 100-step midpoint as the benchmark for checking reference convergence, whereas D.3 uses 200-step Euler. This difference is confusing and needs clarification. The paper also omits comparisons with close works using multivariate Gaussian for target distribution such as [A]. [A] already develops a spectral analysis under this assumption and the paper use the analysis to design noise schedules (which has clear message).

Overall, the writing, tables, figures, and analyses suggest that the manuscript still needs more careful verification. My concern is not AI use itself or the fact that this is an analysis paper, rather, the manuscript needs a lot of verification, its scientific message is unclear, thus I am not convinced that current submission is right for this venue.


[A] Spectral Analysis of Diffusion Models with Application to Schedule Design (NeurIPS 2025)

Questions for authors

Have no particular question.

3 comments
AuthorsOpenReview ↗

Response to Reviewer fmuo: Take-Home, Benita et al. (NeurIPS 2025), Audit Fixes, & Drift Reframing

We thank Reviewer fmuo for the candid, incisive critique. All four points—clarifying the take-home message, citing [A] Benita et al. (NeurIPS 2025), fixing A.5/B.4/D.3, and reframing large-NN drift—identified substantive weaknesses that we have corrected in the revised manuscript.


1. Crisp Take-Home Message & Practical Payoffs

Take-home: Because the FM ODE's acceleration grows with spatial frequency as amax⁡(k)∝∣k∣β/2a_{\max}(k) \propto |k|^{\beta/2}, finite-NN polynomial solvers impose a provable ceiling kc(N)∝N2/βk_c(N) \propto N^{2/\beta} whose prefactor—not exponent—changes with solver order (3.54×3.54\times fewer steps for midpoint than Euler), turning latent spectral ratios into a quantitative tool for choosing solvers, schedules, and step budgets on frontier models.

This yields four practitioner-ready rules: (i) use explicit midpoint for NFE<40\text{NFE} < 40 (2–3×2\text{--}3\times higher rhighr_{\mathrm{high}} than Euler at NFE≤16\text{NFE} \le 16) and switch to DPM-Solver++(2S) at NFE≥40\text{NFE} \ge 40 (Fig. 5(a)); (ii) set σ\sigma-shift s∈[1,2]s \in [1,2] for high-band fidelity, and avoid unnormalized curvature-weighted schedules (SOSS), which collapse near t=0t=0 due to latent DC dominance (App. C); (iii) audit samplers in latent space before VAE decoding, which inverts high-band trends (App. D.1); and (iv) use high-to-low band ratios (12×12\times gap between FLUX.1-schnell and FLUX.1-dev) to fingerprint step distillation (Fig. 2(c)).


2. Missing Reference [A] Benita, Elad, & Keshet (NeurIPS 2025)

Thank you for pointing out [A] (Spectral Analysis of Diffusion Models with Application to Schedule Design, NeurIPS 2025, arXiv:2502.00180). Missing it was our oversight; we now cite and contrast [A] (with Pierret & Galerne, 2024; Wang & Vastola, 2024) in §1 and §6. While [A] derives closed-form per-eigenmode transfer functions for discrete DDIM/DDPM chains under VP/VE parameterizations to optimize schedules, our work complements [A] along four axes:

  1. Multi-order FM cutoff theory (Theorems 1–2): On the linear FM ODE, we prove an all-NN Euler bound and prove midpoint's O(N−2)∣k∣βO(N^{-2})|k|^\beta log-error cancels identically, giving O(N−3)∣k∣3β/2O(N^{-3})|k|^{3\beta/2} superconvergence to the same kc∝N2/βk_c \propto N^{2/\beta} exponent with a universal 3.54×3.54\times step gain (while classical RK4 lacks a 1/e1/e cutoff).
  2. Exact reflow mechanics (App. A.4, Fig. 2(a)): 1-reflow eliminates step roll-off (Rself≡1R_{\mathrm{self}} \equiv 1) while freezing fixed-target error at the N0N_0-step pair ceiling.
  3. Frontier latent diagnostics (§4.2–4.3): Across six 2.5B–12B models, we expose VAE pixel inversion and distillation's 12×12\times band-order reversal.
  4. Large-NN broadband drift (§4.4): Solver-agnostic bfloat16 power inflation at N≥128N \ge 128.

3. Resolution of A.5, B.4/D.3, and Verification Audit

Your concern about AI-assisted algebraic/reporting errors prompted a full symbolic and raw-JSON audit (reviews/verify_*.py in code_supplement.zip + updated AI use statement):

  • Appendix A.5 (Theorem 2): Replaced A.5's N4/βN^{4/\beta} claim with the exact proof that ∫01c3 dt=−14(P−1−P)\int_0^1 c^3\,\mathrm{d}t = -\frac{1}{4}(P^{-1}-P) cancels midpoint's O(N−2)O(N^{-2}) integral identically, leaving ln⁡Rmid≈−π128∣k∣3β/2/(C3/2N3)\ln R_{\mathrm{mid}} \approx -\frac{\pi}{128}|k|^{3\beta/2}/(C^{3/2}N^3) and preserving kc∝N2/βk_c \propto N^{2/\beta} (Table 1: δmid=2/β\delta_{\mathrm{mid}} = 2/\beta, R2=0.9999R^2 = 0.9999).
  • Reference convergence (97.6%, B.4, D.3 →\to Table 5): You caught a real misattribution across evaluation chains: 0.9760.976 (SD3.5) and 0.9900.990 (FLUX-dev) in unifref are midpoint N=28N=28 vs. 5050, not N=50N=50 vs. 100100. On SD3.5's convladder2 (vs. midpoint N=100N=100), midpoint N=50N=50 is 0.8420.842 (84.2%84.2\%) and Euler N=100N=100 is 0.9440.944; in ref200, uniform Euler N=50N=50 vs. default Euler N=200N=200 is 0.5080.508. Table 5 reports all three chains from JSON, removing self-referential 1.000 and unsupported FLUX Euler-5050 rows.
  • Other fixes: Replotted Fig. 2(a) (non-EMA weights) and Fig. 3(b) (N≤400N \le 400 / 102421024^2 ladders); synced JSON values in App. C/D.1; and corrected two bibliography entries.

4. Reframing Large-NN Drift (§4.4, Fig. 3)

We agree that without an exact ground truth on real models, attributing >1.0>1.0 ratios to learned velocity bias overreached. §4.4 now frames this strictly as large-NN broadband spectral drift: Euler and midpoint agree at N=100N=100 (0.9440.944 vs. 1.0001.000) before co-drifting across low (1.211.21), mid (1.491.49), and high (1.611.61) bands at N=200N=200 (continuing through N=400N=400 and σmin⁡=0.02\sigma_{\min}=0.02 clipping), while Theorem 1(i) proves REuler≤1R_{\mathrm{Euler}} \le 1 in float64, leaving bfloat16 step accumulation (Lewandowski et al., 2026) and small-σ\sigma velocity drift as open hypotheses.

AuthorsOpenReview ↗

Follow-Up to fmuo: Isolating the Large-N Drift Mechanism via Matched-Seed fp32 Ablation & Extended Reference Checks

Following up on our previous response, we want to highlight one decisive follow-up result prompted directly by your methodological critique ("the exact Gaussian control does not isolate learned velocity-field error as the cause of large-NN overshoot"), along with two completed verification runs (outputs/):

  1. Causal isolation of large-NN drift (outputs/sd35_precision_drift_ablation.json): On matched-seed SD3.5 (Euler, uniform, n=16, N=50,100,200,400), we held the velocity network fixed and contrasted bf16_default, bf16_vel_fp32_acc (bf16 velocity inference + fp32 ODE accumulator), and full_fp32:
    • Relative to full_fp32@100, bf16_default exhibits (low, mid, high) ratios of (1.285x, 1.946x, 2.221x) at N=200N=200 and (1.078x, 3.428x, 4.424x) at N=400N=400.
    • Keeping only the ODE accumulator in fp32 (bf16_vel_fp32_acc) reduces N=400 ratios to (1.108x, 1.061x, 1.069x), virtually identical to full_fp32 ((1.114x, 1.050x, 1.054x)).
    • Complementarily, in outputs/dpmsolver_spectral_sd35-medium_cfg1_ext.json (n=64n=64, after fixing sample_dpm2m boundary/scaling bugs), exponential integrators (dpm2s, dpm2m) stay bounded (tot_pow = 0.9610 / 0.9641 at N=200N=200) without Euler/Midpoint's additive bf16 blowup (tot_pow = 1.2672 / 1.3382). Together, these experiments confirm your critique was spot-on and establish bfloat16 step-accumulation rounding as the root cause.
  2. Extended-model reference checks & curvature verification: All three additional models (CogView4-6B, Lumina-Image-2.0, AuraFlow-v0.3) re-run against uniform midpoint N=100 (a1v2_solvers_*_convcheck.json, n=64n=64) remove prior >1.0>1.0 Euler-50 reference artifacts completely (all curves strictly <1.0<1.0), and direct SimpleUNet (n_eval=512) measurements confirm 1-reflow cuts trajectory straightness MSE by 329x (reflow_curvature_direct.json).
AuthorsOpenReview ↗

Response to Reviewer fmuo: Clear Take-Home, Manuscript Rewrite, Drift Root Cause, & Benita et al. (2025)

We thank Reviewer fmuo for the candid, sharp critique. Your feedback that the initial draft felt wordy, buried its main take-home message, overstated the cause of high-step overshoot, contained cross-section discrepancies, and missed [A] Benita et al. (NeurIPS 2025) prompted two decisive actions: six targeted verification experiments and a complete top-to-bottom plain-language rewrite of the manuscript (main.pdf).

1. One Clear Take-Home Message & Plain-Language Manuscript Overhaul (§1–§7) We removed dense inline symbol piles, wordy caveats, and cross-section inconsistencies so the entire paper centers on one plain-language insight:

  • Core message: Along a flow-matching trajectory, coarse layout follows gentle paths whereas fine spatial details bend sharply near the clean-image end (t -> 1). Coarse solver steps cut across those sharp late turns, so a finite step budget acts as a spatial low-pass filter whose resolution limit grows as N^(2/beta)—and raising polynomial solver order shifts the constant prefactor (3.54x fewer steps for midpoint than Euler) without changing the scaling exponent.
  • Concrete practical payoffs (§5): This gives practitioners three immediate rules on 2.5B–12B models: (i) use explicit midpoint up to 16 evaluations (2–3x higher detail recovery than Euler) and multistep DPM-Solver++ from 20 evaluations upward (App. D.12); (ii) keep timestep shift in [1, 2] rather than heavy defaults (s = 3) or unnormalized curvature schedules (SOSS, which fail because large coarse-scale latent energy crowds steps near t = 0, App. C); and (iii) audit samplers in latent space before nonlinear VAE decoding introduces edge ringing (App. D.1).

2. Causal Isolation of High-Step Overshoot — Confirming Your Critique (§4.4, Figures 4 & 7, App. D.10) You were completely right that our exact Gaussian control did not prove learned velocity-field error caused high-step overshoot in real models. Holding the SD3.5-Medium velocity network fixed across matched seeds (50 to 400 steps), we compared standard bfloat16, full float32, and hybrid precision (bfloat16 velocity forward pass + float32 ODE state accumulator):

  • Relative to full float32 at 100 steps, default bfloat16 inflates high-frequency power 2.22x at 200 steps and 4.42x at 400 steps (causing blocky mosaic corruption in Figure 7 and 8.4% mean pixel clipping across 64 images).
  • Keeping only the ODE state accumulator in float32 cuts 400-step high-frequency ratios from 4.42x down to 1.07x, matching full float32 (1.05x) and removing over 97% of the blowup at zero runtime overhead (148.8s vs. 149.0s, vs. 960.7s for full float32). Exponential integrators (DPM-Solver++ 2S/2M) likewise stay bounded (0.96 total power at 200 steps) because their analytical decay damps rounding buildup. Thus bfloat16 state-accumulation rounding, not velocity-field bias, is the root cause.

3. Relation to [A] Benita, Elad, & Keshet (NeurIPS 2025) (§1 & §6) Thank you for pointing out [A], now cited and contrasted in §1 and §6. While [A] models discrete DDIM/DDPM chains under variance-preserving/exploding diffusion schedules as per-mode transfer functions to optimize noise schedules, our study complements [A] by: (i) proving multi-order scaling on the flow-matching ODE (midpoint's second-order error cancels completely to give third-order accuracy and a universal 3.54x step reduction to the same N^(2/beta) cutoff); (ii) proving and measuring how reflow straightens paths (329x lower straightness error, App. B.5) while locking fidelity to teacher pairs (App. A.4); (iii) exposing VAE pixel inversion and step-distillation's 12x band-order reversal across six 2.5B–12B models (§4.2–§4.3); and (iv) isolating bfloat16 accumulation drift (§4.4).

4. End-to-End Verification of Equations, Tables, and Reference Protocols (Table 1, Table 3, App. D.6, code_supplement.zip) Every equation, table, and figure was audited end-to-end (reviews/verify_*.py in code_supplement.zip): Theorem 2 and Table 1 reconcile old Appendix A.5 (R^2 = 0.9999), Table 3 (App. D.3) unifies the 50-step midpoint, 100-step midpoint, and 200-step Euler reference checks clearly, all three extended models (CogView4-6B, Lumina-Image-2.0, AuraFlow v0.3) are verified against converged 100-step midpoint references (App. D.6), and Figure 3a and Figure 5 are replotted from verified checkpoints and matched references.

Reviewer_X4mLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the loss of high frequency detail in Flow Matching at low sampling step counts. The authors propose a frequency domain analysis framework, derive the relation between the spectral cutoff frequency and the number of sampling steps, and analyze the spectral fidelity of different numerical solvers through experiments on multiple generative models. The experiments show that the midpoint method usually preserves more high frequency energy than the Euler method at low step counts. The study further provides guidance on the choice of the sampling step count and the noise schedule parameter. The relation between the spectral metrics and human perceptual judgments still requires further validation.

Critical strengths

An interpretable frequency-domain characterization of flow-matching sampling errors. The Gaussian analysis provides an analytically tractable connection between the target power spectrum, ODE trajectory dynamics, and the frequency-dependent behavior of numerical integration. The exact per-frequency gain formula is particularly useful because it enables controlled numerical verification rather than relying exclusively on aggregate image-quality metrics. This is consequential because it offers a potentially useful perspective on sampling efficiency beyond conventional scalar convergence measures.

Critical weaknesses
  1. The central theoretical claims require substantial correction or stronger justification. There is a direct inconsistency between Section 3.2 and Appendix A.5 regarding the spectral cutoff scaling of higher-order solvers. Appendix A.5 states that the leading midpoint log-error scales as (k^\beta/N^2), but then concludes that the cutoff scales as (N^{4/\beta}). Applying a fixed-error threshold to the stated expression instead gives (N^{2/\beta}).An independent evaluation using the paper's exact Gaussian velocity field and the standard explicit midpoint scheme also shows an approximately linear cutoff in N when beta = 2, rather than the quadratic scaling predicted in Appendix A.5. These issues are consequential because the spectral cutoff theorem and its interpretation across solver orders constitute a central theoretical contribution.
  2. The paper lacks a systematic cross validation between the spectral metrics and human subjective evaluation. It therefore does not yet demonstrate that improved spectral fidelity translates into better visual quality.
  3. The qualitative analysis of the high frequency energy overshoot is insufficient. The paper does not clarify how this overshoot appears in real generated images and what its potential impact is.
  4. The paper compares ISD with LSD, IS divergence and rFID in terms of correlation, but it does not establish the distinct advantage of ISD over existing spectral metrics.
  5. The recommendation on the shift parameter rests mainly on spectral metrics, and it lacks sufficient validation with other quality metrics and with qualitative generation results. The practical conclusions therefore remain to be confirmed.
  6. The paper does not propose a concrete training or inference optimization method based on the observed spectral behavior. The contribution concentrates on error analysis and on parameter selection advice, so the methodological novelty and the direct application value are limited.
Questions for authors
  1. Can the authors reconcile the cutoff scaling statements in Section 3.2 and Appendix A.5? In particular, how does the stated midpoint error proportional to (k^\beta/N^2) imply (k_c\propto N^{4/\beta})? Can the cutoff scaling be established without extending a perturbative approximation beyond its stated domain of validity?
  2. Could the authors clarify the exact reference solver, schedule, precision, and normalization used for the overshoot experiments, and provide a consistent convergence comparison for the same learned velocity field at N = 50, 100, 200, and higher accuracy? This would help determine whether the overshoot persists relative to an independently converged solution and whether it can truly be separated from discretization error.
  3. Does the disappearance of the spectral cutoff after reflow persist when evaluated against a fixed target spectrum rather than each model's own high-step reference? Is there direct evidence that the change in frequency-resolved sampling error is explained by reduced trajectory curvature, rather than changes in the learned endpoint distribution? 4.Can the authors verify the correlation between ISD and perceptual quality through a systematic human subjective evaluation study, and further clarify whether ISD reflects the visual quality of generated images more accurately than existing spectral metrics?
5 comments
AuthorsOpenReview ↗

Response to Reviewer X4mL (1/2): Midpoint Superconvergence, Large-N Drift, Reference Audit, & Reflow

We thank Reviewer X4mL for an exceptionally thorough review and for independently computing the Gaussian midpoint cutoff. Your numerical check is completely right—and tracing why it holds uncovered an exact analytical cancellation on the Gaussian FM ODE. Part 1 covers W1/Q1, W3/Q2, and Q3; Part 2 covers W2/W4/Q4, W5, and W6.


1. Midpoint Cutoff Scaling: Why kcmid∝N2/βk_c^{\mathrm{mid}} \propto N^{2/\beta} Holds Analytically (W1 & Q1)

Your check (kcmid/kcEuler≈3.55k_c^{\mathrm{mid}}/k_c^{\mathrm{Euler}} \approx 3.55 at β=2\beta=2, kcmid∝N2/βk_c^{\mathrm{mid}} \propto N^{2/\beta}) matches our Gaussian data (m1_gaussian_beta2.0_dim256.json: kcmid=20,41,84,210k_c^{\mathrm{mid}} = 20, 41, 84, 210 vs. kcEuler=5,11,23,59k_c^{\mathrm{Euler}} = 5, 11, 23, 59 at N=5,10,20,50N=5,10,20,50; a4_beta_values.json: δmid=1.386,1.012,0.800,0.656\delta_{\mathrm{mid}} = 1.386, 1.012, 0.800, 0.656 for β=1.5..3.0\beta = 1.5..3.0). Both A.5's N4/βN^{4/\beta} result and its assumed O(N−2)∣k∣βO(N^{-2})|k|^\beta global log-error were wrong (Theorem 2, §3.2, App. A.3):

  1. Exact O(N−2)∣k∣βO(N^{-2})|k|^\beta cancellation: One midpoint step Mi=1+hc(tm)(1+h2c(ti))M_i = 1 + h c(t_m)(1 + \frac{h}{2}c(t_i)) has local log-error −h3(124c′′+14cc′+16c3)∣tm+O(h4)-h^3(\frac{1}{24}c'' + \frac{1}{4}c c' + \frac{1}{6}c^3)|_{t_m} + O(h^4). Via v=t/(1−t)v = t/(1-t), ∫01c3 dt=−14(P−1−P)\int_0^1 c^3\,\mathrm{d}t = -\frac{1}{4}(P^{-1} - P) holds for all P(k)>0P(k) > 0. With c(0)=−1,c(1)=1c(0)=-1, c(1)=1 and c′(1)−c′(0)=P−1−Pc'(1)-c'(0) = P^{-1}-P, the O(N−2)O(N^{-2}) integral cancels identically: ∫01 ⁣(124c′′+14cc′+16c3)dt=[124c′+18c2]01+16∫01c3 dt=P−1−P24−P−1−P24=0.\int_0^1\!\left(\frac{1}{24}c'' + \frac{1}{4}c c' + \frac{1}{6}c^3\right)\mathrm{d}t = \left[\frac{1}{24}c' + \frac{1}{8}c^2\right]_0^1 + \frac{1}{6}\int_0^1 c^3\,\mathrm{d}t = \frac{P^{-1}-P}{24} - \frac{P^{-1}-P}{24} = 0.
  2. Superconvergence & universal gain: Expanding to O(h4)O(h^4) gives, for P(k)≪1≪ρ=N2P(k)P(k) \ll 1 \ll \rho = N^2 P(k): ln⁡Rmid(k,N)=−π128 ∣k∣3β/2C3/2N3(1+o(1))=−π128 ρ−3/2(1+o(1)).\ln R_{\mathrm{mid}}(k,N) = -\frac{\pi}{128}\,\frac{|k|^{3\beta/2}}{C^{3/2} N^3}\bigl(1+o(1)\bigr) = -\frac{\pi}{128}\,\rho^{-3/2}\bigl(1+o(1)\bigr). Setting ln⁡Rmid=−1\ln R_{\mathrm{mid}} = -1 gives (3β/2)/3=β/2  ⟹  kcmid∼(C/ρcmid)1/βN2/β(3\beta/2)/3 = \beta/2 \implies k_c^{\mathrm{mid}} \sim (C/\rho_c^{\mathrm{mid}})^{1/\beta} N^{2/\beta} (ρcmid≈0.05612\rho_c^{\mathrm{mid}} \approx 0.05612 vs. ρcEuler≈0.7030\rho_c^{\mathrm{Euler}} \approx 0.7030). At matched NN, kcmid/kcEuler→3.5392/βk_c^{\mathrm{mid}}/k_c^{\mathrm{Euler}} \to 3.539^{2/\beta} (3.543.54 at β=2\beta=2, matching your 3.553.55!), while the matched-kck_c step reduction NEuler/Nmid→3.54×N_{\mathrm{Euler}}/N_{\mathrm{mid}} \to 3.54\times is independent of β,C\beta, C. Asymptotic ISDmid∝N−3\mathrm{ISD}_{\mathrm{mid}} \propto N^{-3} (α→3\alpha \to 3, explaining α=2.50–2.58>2\alpha = 2.50\text{--}2.58 > 2 on [5,50][5,50] in App. D.10); classical RK4 rebounds past 11 (min⁡kR≈0.934>e−1\min_k R \approx 0.934 > e^{-1}), so we withdraw N2p/βN^{2p/\beta} and scope the 1/e1/e cutoff to p≤2p \le 2.

2. Large-NN Drift to N=400N=400 & Reference Audit (W3 & Q2)

  1. No saturation to N=400N=400 (§4.4, Fig. 3): On matched-noise SD3.5 (bfloat16, n=8n=8), total latent power rises 0.892→1.010→1.218→1.2650.892 \to 1.010 \to 1.218 \to 1.265 at N=50,100,200,400N=50,100,200,400, and high-band power climbs 1.00×→1.27×→2.23×→4.74×1.00\times \to 1.27\times \to 2.23\times \to 4.74\times (max pixel clip 21.6%21.6\%). On convladder2 (n=64n=64), Euler and midpoint agree at N=100N=100 (0.9440.944 vs. 1.0001.000) then co-drift broadband at N=200N=200 (rlow=1.213,rmid=1.491,rhigh=1.608/1.617r_{\mathrm{low}}=1.213, r_{\mathrm{mid}}=1.491, r_{\mathrm{high}}=1.608/1.617). Clipping σmin⁡=0.02\sigma_{\min}=0.02 still drifts (1.3161.316 at N=256N=256).
  2. Reframing & reference audit (Table 5): Without exact real-model endpoints, attributing drift solely to vθ−v∗v_\theta - v^* overreached. Theorem 1(i) proves REuler≤1R_{\mathrm{Euler}} \le 1 in float64 (tested to N=5000N=5000), leaving bfloat16 step accumulation (Lewandowski et al., 2026; He et al., 2023) and small-σ\sigma drift as open hypotheses. Auditing references also fixed B.4/D.3: 0.9760.976 (SD3.5) and 0.9900.990 (FLUX-dev) were midpoint N=28N=28 vs. 5050 (unifref), while SD3.5 midpoint N=50N=50 vs. 100100 (convladder2) is 0.8420.842 (Table 5, Fig. 3(b)).

3. Reflow Against Fixed Ground-Truth Target (Q3)

App. A.4 proves that for Gaussian FM trained on N0N_0-step Euler pairs (x0,GN0(k)x0)(x_0, G_{N_0}(k)x_0), 1-reflow's vref=GN0(k)−1(1−t)+tGN0(k)x^kv_{\mathrm{ref}} = \frac{G_{N_0}(k)-1}{(1-t)+tG_{N_0}(k)}\hat{x}_k has zero curvature and is integrated to ∼10−15\sim 10^{-15} by Euler at every N≥1N \ge 1: Rself(k,N)≡1R_{\mathrm{self}}(k,N) \equiv 1 (vs. reflow N=128N=128), while Rfixed(k,N)≡Rbase(k,N0)R_{\mathrm{fixed}}(k,N) \equiv R_{\mathrm{base}}(k,N_0) (vs. P(k)P(k) or base N=128N=128) freezes at the N0=32N_0=32 pair ceiling. Auditing Fig. 2(a) showed a3_reflow_iter1.json plotted unconverged EMA weights (decay=0.9999, 3,9003{,}900 steps   ⟹  67.7%\implies 67.7\% init). Replotting Fig. 2(a) with converged non-EMA weights from the same checkpoint confirms both laws: Rself∈[0.87,1.00]R_{\mathrm{self}} \in [0.87, 1.00] (N=2..32N=2..32), while RfixedR_{\mathrm{fixed}} freezes along base N0=32N_0=32 (ISDfixed∈[0.24,0.28]\mathrm{ISD}_{\mathrm{fixed}} \in [0.24, 0.28]; slope 2.062.06 vs. data 1.931.93).

AuthorsOpenReview ↗

Response to Reviewer X4mL (2/2): Spectral Metrics vs. FID, Sigma-Shift Scope, & Diagnostic Positioning

(Part 2 of 2 — addressing W2/W4/Q4, W5, and W6)

4. Spectral Metrics (ISD) vs. Perceptual Quality & Role Alongside FID/LPIPS (W2, W4, Q4)

We completely agree that phase-insensitive radial PSD summaries cannot replace perceptual or human evaluation—a spatial white-noise scramble with matched radial amplitudes has ISD=0\mathrm{ISD}=0 while looking like pure noise. Our claim is narrower and diagnostic (§4.5, App. D.8): within deterministic trajectories of a fixed trained generator, ODE truncation acts as a structured zero-phase radial low-pass filter (GN(k)>0G_N(k) > 0 per mode) rather than a random phase scrambler, making latent PSD ratios a fast, seed-matched probe of which spatial scales a sampler attenuates or inflates before VAE decoding.

Concretely:

  1. Why latent spectral diagnostics catch what pixel/Inception metrics miss: In App. D.1, VAE decoding nonlinearly hallucinates sharp edges on under-resolved latents, inverting high-band trends (latent rhighr_{\mathrm{high}} rises 0.046→0.6970.046 \to 0.697 over N=2..20N=2..20, whereas pixel rhighr_{\mathrm{high}} stays >1>1, moving 1.680→1.858→1.2511.680 \to 1.858 \to 1.251). Similarly, InceptionV3's texture/aliasing sensitivity (Stein et al., NeurIPS 2023) conflates VAE ringing with genuine latent detail.
  2. Quantitative alignment on valid solver trajectories: Across n=12n=12 (solver, NN) configurations on SD3.5 (App. D.8), ISD\mathrm{ISD}, log-spectral distance (LSD), and Itakura–Saito divergence correlate monotonically with ResNet-50 rFID (ρ=0.825,p=9.5×10−4\rho = 0.825, p = 9.5\times 10^{-4} for ISD\mathrm{ISD}; ρ=0.923,p=1.9×10−5\rho = 0.923, p = 1.9\times 10^{-5} for LSD and IS-div), reproducing midpoint's 2.3×2.3\times NFE=8 advantage in rFID (146146 vs. Euler 221221).
  3. Explicit scope: §4.5 states clearly that ISD\mathrm{ISD} complements rather than supersedes FID/CMMD/human preference, especially when comparing across distinct checkpoints or guidance regimes.

5. σ\sigma-Shift Recommendation (s∈[1,2]s \in [1,2]) vs. Deployed Guided Practice (s=3s=3) (W5)

Your distinction between base-ODE spectral fidelity and CFG-guided perceptual quality is well taken, and we sharpened §4.1 and §5 (Guideline 2) to scope this explicitly:

  1. Per-model guidance disclosure (§4.1, App. B.2): SD3.5-Medium, AuraFlow v0.3, CogView4-6B, and Lumina-Image-2.0 run at w=1.0w=1.0 (unamplified velocity field). Because FLUX.1-dev is guidance-distilled into a single-pass model (Meng et al., CVPR 2023), its native base-ODE operating point uses embedded guidance 3.53.5 (where σ\sigma-shift still peaks at s≈1.5s \approx 1.5, rhigh=0.904r_{\mathrm{high}} = 0.904), and FLUX.1-schnell runs at w=0w=0.
  2. Why deployed defaults use s=3s=3 (Esser et al., 2024): Higher shifts place more steps near t→0t \to 0 (high noise) to stabilize global layout at 102421024^2 and under strong CFG (w=4–7w=4\text{--}7), trading away steps near t→1t \to 1 where Theorem 1 shows high-frequency curvature peaks (t∗(k)=1/(1+P(k))→1t^*(k) = 1/(1+P(k)) \to 1). Our sweep makes this layout-vs.-detail trade-off quantitative: in App. C (SD3.5, N=20N=20), s=3s=3 edges s=1s=1 on low frequencies (rlow=0.842r_{\mathrm{low}} = 0.842 vs. 0.6820.682) at the cost of high-frequency recovery (rhigh=0.697r_{\mathrm{high}} = 0.697 vs. 0.8600.860). Thus s∈[1,2]s \in [1,2] is specifically the base-ODE spectral-fidelity optimum rather than a blanket replacement for high-CFG layout tuning.

6. Positioning as a Theory-Driven Diagnostic & Measurement Study (W6)

We propose no new sampler by design; our contribution follows the measurement-and-diagnosis tradition in generative modeling—such as Lin et al. (WACV 2024) diagnosing nonzero terminal-SNR flaws, Karras et al. (NeurIPS 2022) mapping the EDM design space, and Stein et al. (NeurIPS 2023) auditing diffusion evaluation metrics. With Theorems 1–2 and the Gaussian reflow proposition now proved in exact closed form, the paper provides four concrete payoffs:

  1. Exact solver-order law on FM: Proving why explicit midpoint superconverges at O(N−3)∣k∣3β/2O(N^{-3})|k|^{3\beta/2} yet preserves kc∝N2/βk_c \propto N^{2/\beta} with a universal 3.54×3.54\times step gain (while classical RK4 lacks a 1/e1/e cutoff).
  2. Sampler-switching rule: Locating the low-NFE midpoint vs. high-NFE DPM-Solver++(2S) crossover at NFE≈40\text{NFE} \approx 40 (Fig. 5(a)).
  3. Training-regime forensics: Separating reflow's curvature elimination (and fixed-target pair freezing) from step distillation's 12×12\times band-order reversal (Fig. 2), and explaining SOSS collapse via unnormalized latent DC power (App. C).
  4. Large-NN bfloat16 drift benchmark: Documenting broadband spectral inflation past N≥128N \ge 128 across resolutions.
AuthorsOpenReview ↗

Follow-Up to X4mL: Decisive bf16 Accumulation Ablation (Q2), n=64 N=400 & Grids (W3), Direct Curvature (Q3), & rFID@Shift (W5)

Following up on our initial response, we have now executed all four GPU/trajectory follow-up experiments discussed for W3/Q2, Q3, and W5:

  1. Precision isolation of large-NN drift (closing Q2's open hypothesis): Running matched-seed SD3.5 (Euler, uniform, n=16, N=50,100,200,400) across bf16_default, bf16_vel_fp32_acc (bf16 model forward pass + fp32 ODE accumulator), and full_fp32 isolates bfloat16 step-accumulation rounding as the primary cause:
    • Relative to full_fp32@100 (band_low=6.5013, mid=0.1190, high=0.0604), bf16_default inflates (mid, high) power to (1.946x, 2.221x) at N=200N=200 and (3.428x, 4.424x) at N=400N=400 (tot_mean: 0.9267→1.0514→1.2476→1.26430.9267 \to 1.0514 \to 1.2476 \to 1.2643).
    • Retaining only the ODE accumulator in fp32 (bf16_vel_fp32_acc) suppresses N=400 high-band inflation to 1.069x (low=1.108x, mid=1.061x), matching full_fp32 (tot_mean: 0.9098→0.9976→1.0453→1.08040.9098 \to 0.9976 \to 1.0453 \to 1.0804; N=400 (1.114x, 1.050x, 1.054x)).
    • Additionally, in our extended fixed DPM-Solver++ sweep (dpmsolver_spectral_sd35-medium_cfg1_ext.json, n=64n=64), exponential integrators (dpm2s, dpm2m) stay bounded (tot_pow = 0.9610 / 0.9641 at N=200N=200, zero NaNs) rather than exhibiting additive bf16 drift (Euler 1.2672, Midpoint 1.3382), while dpm2m reaches rhigh=0.8702r_{\mathrm{high}} = 0.8702 at NFE=28\text{NFE}=28.
  2. Formal n=64n=64 N=400N=400 ladders & qualitative grids (W3): At n=64n=64 (overstep_sd35-medium_{euler,midpoint}_n64.json), per-image N=400 vs 50 high-band ratios reach 4.72x (Euler, median 4.45x) and 4.20x (Midpoint, median 3.90x); pixel clipping escalates 0.413% -> 1.306% -> 4.436% -> 8.377% (Euler, max 42.14%) and 1.499% -> 2.249% -> 5.525% -> 8.615% (Midpoint, max 42.57%). Paired matched-seed PNG grids (outputs/qualitative_overshoot/) confirm clean imagery at N=50N=50, over-contrast/ringing at N=100–200N=100\text{--}200, and severe high-frequency mosaic corruption by N=400N=400.
  3. Direct trajectory curvature measurement for reflow (Q3b): Evaluating SimpleUNet 64x64 (outputs/reflow_curvature_direct.json, n_eval=512, n_fine=128, fp32) directly confirms curvature elimination:
    • iter0 -> iter1_non_ema -> iter2_non_ema: straightness_mse drops 329x (0.024006→0.000073→0.0000620.024006 \to 0.000073 \to 0.000062), path_length_ratio 1.00723→1.00004→1.000031.00723 \to 1.00004 \to 1.00003, accel_rms 3.4023→1.0149→0.64533.4023 \to 1.0149 \to 0.6453, and high-band acceleration PSD 51285.6 -> 3335.0 (15.4x) -> 1267.0 (40.5x).
    • Correspondingly, iter1_non_ema achieves Euler@1 r_hi_self=0.7751 and Euler@2 r_hi_self=0.9181 while freezing vs. the fixed target (r_hi_fixed = 0.5036 @ N=1 -> 0.5970 @ N=2 -> 0.6468 @ N=128), whereas unconverged iter1_ema / iter2_ema blow up against the fixed target (Euler@2 r_hi_fixed = 91.79 / 68.29).
  4. Perceptual rFID @ shift & qualitative triplets (W5): Running ResNet-50 rFID and matched-seed grids at N=20N=20, s∈{1,2,3}s \in \{1,2,3\} (perceptual_shifts_sd35-medium_{cfg1,cfg7}.json):
    • At w=1.0w=1.0 (n=64n=64, ref mid100), (s=1,2,3)(s=1,2,3) gives rFID = 139.42 -> 114.03 -> 107.44, ISDL1=0.4384,0.4205,0.4423\mathrm{ISD}_{L_1} = 0.4384, \mathbf{0.4205}, 0.4423, LSD=0.5779,0.5613,0.6267\mathrm{LSD} = 0.5779, \mathbf{0.5613}, 0.6267, while latent rhighr_{\mathrm{high}} falls monotonically 0.558 -> 0.516 -> 0.457 (r_lo: 0.554→0.6890.554 \to 0.689).
    • At w=7.0w=7.0 (n=32n=32, ref mid50), (s=1,2,3)(s=1,2,3) gives rFID = 279.70 -> 200.35 -> 186.01, with s=2s=2 again minimizing ISDL1\mathrm{ISD}_{L_1} (0.2660) and LSD\mathrm{LSD} (0.3192) and peaking in rhighr_{\mathrm{high}} (0.701), while s=1s=1 visually suffers heavy contrast collapse. This directly quantifies our refined §5 guidance: s∈[1,2]s \in [1,2] balances base-ODE high-band fidelity (s=1–2s=1\text{--}2) against guided structural stability (s≥2s \ge 2).
Reviewer_xxjROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how limited sampling steps affect different spatial frequencies in flow matching. It derives a frequency-dependent resolution limit for Gaussian targets and examines related behavior in five text-to-image models, mainly without classifier-free guidance. The experiments also explore solver choice, timestep schedules, reflow, and distillation.

Critical strengths
  1. The frequency-domain perspective offers a useful explanation for why fine details require more sampling steps. The Gaussian setting makes this behavior accessible to analysis.
  2. The experiments connect the analysis to several large models. Comparisons at matched numbers of model evaluations and checks on reference convergence make the empirical findings useful beyond the synthetic setting.
Critical weaknesses
  1. The central resolution-limit argument needs stronger justification. Appendix A.4 derives an approximation for small errors, then uses it to infer the cutoff where errors are substantial. Although the appendix acknowledges this distinction, it does not adequately establish the claimed scaling near the cutoff or exponential attenuation beyond it.
  2. The higher-order solver analysis is internally inconsistent. Section 3.2 says higher-order solvers preserve the cutoff exponent, whereas Appendix A.5 predicts a different exponent for midpoint. This affects the paper's theoretical explanation of the solver comparisons, even if the measured improvements remain useful.
Questions for authors
  1. Can the authors establish the cutoff scaling in the regime where the small-error approximation breaks down, and clarify which attenuation claims are exact versus approximate?
  2. Which cutoff scaling and asymptotic convergence rate should hold for midpoint, and which empirical conclusions remain supported after reconciling the conflicting statements?
4 comments
AuthorsOpenReview ↗

Response to Reviewer xxjR: Non-Asymptotic Euler Bound (Theorem 1) & Midpoint Superconvergence (Theorem 2)

We thank Reviewer xxjR for the thoughtful evaluation and for highlighting the clarity of our central question, empirical consistency across synthetic and frontier models, distillation band-order reversal, and large-NN analysis. Both questions pinpointed places where Appendix A needed exact non-asymptotic derivations rather than informal expansions. We rewrote §3.2, Table 1, and Appendix A, adding CPU verification scripts (reviews/verify_*.py) in code_supplement.zip.


1. Crossover Rigor & Non-Asymptotic Bound at k∼kc(N)k \sim k_c(N) (W1 & Q1)

You are right that naive leading-order truncation ln⁡R(k,N)≈−2α1∣k∣β/2/N\ln R(k,N) \approx -2\alpha_1 |k|^{\beta/2}/N requires ∣R−1∣≪1|R-1| \ll 1, whereas kc(N)k_c(N) lies at ln⁡R=−1\ln R = -1 (O(1)O(1) attenuation), and that "exponential attenuation" overstated the decay profile. Theorem 1 (§3.2, App. A.2) resolves both points:

  1. Exact product for all N≥1,k≠0N \ge 1, k \ne 0: With gi(k)≔(N−i)2+i2P(k)g_i(k) \coloneqq (N-i)^2 + i^2 P(k) and φi(k)≔(N−i)(N−i−1)+i(i+1)P(k)\varphi_i(k) \coloneqq (N-i)(N-i-1) + i(i+1)P(k), one Euler step scales x^k\hat{x}_k by φi/gi\varphi_i / g_i. Polynomial expansion gives gigi+1−φi2=N2P(k)g_i g_{i+1} - \varphi_i^2 = N^2 P(k), telescoping the NN-step ratio into REuler(k,N)=∏i=0N−1 ⁣(1−N2P(k)gi(k) gi+1(k)).R_{\mathrm{Euler}}(k,N) = \prod_{i=0}^{N-1}\!\left(1 - \frac{N^2 P(k)}{g_i(k)\,g_{i+1}(k)}\right).
  2. All-NN upper bound across and above kc(N)k_c(N): Re-indexing from t=1t=1 via j=N−i∈{1,…,N}j = N - i \in \{1,\ldots,N\} with ρ≔N2P(k)=CN2∣k∣−β\rho \coloneqq N^2 P(k) = C N^2 |k|^{-\beta} gives gN−j(k)≤ρ+j2g_{N-j}(k) \le \rho + j^2. Hence for every N≥1N \ge 1: ln⁡REuler(k,N)  ≤  Φ(ρ)≔∑j=1∞ln⁡ ⁣(1−ρ(ρ+j2)(ρ+(j−1)2)).\ln R_{\mathrm{Euler}}(k,N) \;\le\; \Phi(\rho) \coloneqq \sum_{j=1}^{\infty}\ln\!\left(1 - \frac{\rho}{(\rho+j^2)\bigl(\rho+(j-1)^2\bigr)}\right). As N→∞N \to \infty at fixed ρ\rho, ln⁡REuler(k,N)→Φ(ρ)\ln R_{\mathrm{Euler}}(k,N) \to \Phi(\rho) pointwise across the entire crossover ρ∈(0,∞)\rho \in (0,\infty), including Φ(ρcEuler)=−1\Phi(\rho_c^{\mathrm{Euler}}) = -1 at ρcEuler≈0.70296\rho_c^{\mathrm{Euler}} \approx 0.70296 (independent of β,C\beta, C). Thus REuler(k,N)<e−1R_{\mathrm{Euler}}(k,N) < e^{-1} for all ∣k∣≥(C/ρcEuler)1/βN2/β|k| \ge (C/\rho_c^{\mathrm{Euler}})^{1/\beta} N^{2/\beta} at every N≥1N \ge 1 without asymptotics (replacing informal decay wording and removing the old A.4 Step 3 remark).
  3. First-order Euler–Maclaurin accuracy up to kck_c: Via v=t/(1−t)v = t/(1-t), ∫01c2 dt=π4(P+P−1/2)−1\int_0^1 c^2\,\mathrm{d}t = \frac{\pi}{4}(\sqrt{P} + P^{-1/2}) - 1 (α1=π/8\alpha_1 = \pi/8). Keeping the +2/N+2/N boundary constant gives −ln⁡REuler≈(π4(P+P−1/2)+1)/N-\ln R_{\mathrm{Euler}} \approx (\frac{\pi}{4}(\sqrt{P}+P^{-1/2}) + 1)/N, within 6.6%6.6\% relative error of the exact product up to k=kck = k_c (N=50–400N=50\text{--}400).

2. Higher-Order Cutoff Scaling & Midpoint Validation (W2 & Q2)

Checking Appendix A.5 symbolically and numerically revealed a remarkable cancellation on the Gaussian FM ODE (Theorem 2, §3.2, App. A.3):

  1. Exact O(N−2)∣k∣βO(N^{-2})|k|^\beta cancellation & third-order superconvergence: One explicit midpoint step has local log-error −h3(124c′′+14cc′+16c3)∣tm+O(h4)-h^3(\frac{1}{24}c'' + \frac{1}{4}c c' + \frac{1}{6}c^3)|_{t_m} + O(h^4). Using ∫01c(t,k)3 dt=−14(P−1−P)\int_0^1 c(t,k)^3\,\mathrm{d}t = -\frac{1}{4}(P^{-1}-P) and c(0)=−1,c(1)=1,c′(1)−c′(0)=P−1−Pc(0)=-1, c(1)=1, c'(1)-c'(0) = P^{-1}-P: ∫01 ⁣(124c′′+14cc′+16c3)dt=124(P−1−P)+0−124(P−1−P)=0.\int_0^1\!\left(\frac{1}{24}c'' + \frac{1}{4}c c' + \frac{1}{6}c^3\right)\mathrm{d}t = \frac{1}{24}(P^{-1}-P) + 0 - \frac{1}{24}(P^{-1}-P) = 0. The nominal O(N−2)∣k∣βO(N^{-2})|k|^\beta global log-error vanishes identically! Expanding to O(h4)O(h^4) gives the leading error: ln⁡Rmid(k,N)=−π128 ∣k∣3β/2C3/2N3(1+o(1))=−π128 ρ−3/2(1+o(1)).\ln R_{\mathrm{mid}}(k,N) = -\frac{\pi}{128}\,\frac{|k|^{3\beta/2}}{C^{3/2} N^3}\bigl(1 + o(1)\bigr) = -\frac{\pi}{128}\,\rho^{-3/2}\bigl(1+o(1)\bigr).
  2. Cutoff exponent & step gain: Setting ln⁡Rmid=−1\ln R_{\mathrm{mid}} = -1 gives (3β/2)/3=β/2(3\beta/2)/3 = \beta/2, so midpoint shares the same exponent kcmid(N)∝N2/βk_c^{\mathrm{mid}}(N) \propto N^{2/\beta} as Euler (not N4/βN^{4/\beta}), with ρcmid≈0.05612\rho_c^{\mathrm{mid}} \approx 0.05612 and a β\beta-independent 3.54×3.54\times step reduction (1.77×1.77\times NFE), reconciling A.5 with §3.2.
  3. Validation across β,d\beta, d (Table 1): Exact Gaussian midpoint recursions (N∈{5,…,200}N \in \{5,\ldots,200\}) confirm δ=2/β\delta = 2/\beta (R2=0.9999R^2 = 0.9999):
    • (β=1.5,d=512)(\beta=1.5, d=512): δpred=1.333\delta_{\mathrm{pred}} = 1.333, Euler 1.6181.618, Midpoint 1.3861.386
    • (β=2.0,d=256/512/1024)(\beta=2.0, d=256 / 512 / 1024): δpred=1.000\delta_{\mathrm{pred}} = 1.000, Euler 1.052/1.042/1.0331.052 / 1.042 / 1.033, Midpoint 1.023/1.018/1.0151.023 / 1.018 / 1.015
    • (β=3.0,d=256)(\beta=3.0, d=256): δpred=0.667\delta_{\mathrm{pred}} = 0.667, Euler 0.6880.688, Midpoint 0.6770.677 For classical RK4, ∣z∣4/24|z|^4/24 rebounds RRK4R_{\mathrm{RK4}} past 11 (min⁡kR≈0.934>e−1\min_k R \approx 0.934 > e^{-1}), so we withdraw N2p/βN^{2p/\beta} and scope the 1/e1/e cutoff to p≤2p \le 2.
AuthorsOpenReview ↗

Follow-Up to xxjR: Fixed DPM-Solver++(2M) Sweep, Precision Isolation of Large-N Drift, & Extended-Model N=100 Checks

As a brief addendum to our first-round response (§1–2), we completed three additional empirical verification experiments (outputs/):

  1. Fixed DPM-Solver++(2M) across N=4…200N=4\dots 200 (dpmsolver_spectral_sd35-medium_cfg1_ext.json, n=64n=64): After fixing a log(0) singularity at σ=1\sigma=1 and the hcurr/hprevh_{\mathrm{curr}}/h_{\mathrm{prev}} step-ratio formula in sample_dpm2m, dpm2m (1 NFE/step, zero NaNs) reaches latent rhigh=0.8163r_{\mathrm{high}} = 0.8163 at NFE=20\text{NFE}=20 and 0.8702 at NFE=28\text{NFE}=28 (vs. Euler 0.54370.5437 / 0.63480.6348), while midpoint still leads at very low budgets (NFE=8\text{NFE}=8: 0.60760.6076 vs. 0.42110.4211). Unlike Euler (tot_pow = 1.2672) and Midpoint (1.3382), dpm2s and dpm2m remain bounded at N=200N=200 (0.9610 / 0.9641).
  2. Large-NN drift isolated to bfloat16 step accumulation (sd35_precision_drift_ablation.json, n=16n=16): Keeping only the ODE accumulator in fp32 (bf16_vel_fp32_acc) reduces SD3.5 N=400 high-band inflation from 4.424x (bf16_default vs. full_fp32@100) down to 1.069x, matching full_fp32 (1.054x).
  3. Extended-model midpoint N=100 convergence checks (a1v2_solvers_{cogview4,lumina2,auraflow}_convcheck.json, n=64n=64): Under converged uniform midpoint N=100 references, all Euler and Midpoint curves (N=20,28,50N=20,28,50) on CogView4 (mid 0.965..0.969), Lumina2 (mid 0.929..0.940), and AuraFlow (mid 0.886..0.919) stay strictly <1.0<1.0 and confirm consistent midpoint superiority at matched NFE.
AuthorsOpenReview ↗

Response to Reviewer xxjR: Exact All-N Cutoff Bound, Midpoint Scaling, & Empirical Verification

We thank Reviewer xxjR for the clear, constructive review. Both of your questions pinpointed where our theoretical arguments needed non-asymptotic rigor and reconciliation. Below is a plain-language summary of how both points are resolved in the revised manuscript (main.pdf, §3.2–§3.4, §4.2–§4.5, and Appendix A), which has also been rewritten throughout in clear, accessible prose.

1. Cutoff Scaling Without Small-Error Approximations, and Exact vs. Approximate Claims (W1 & Q1; §3.2, Theorem 1, App. A.2) You rightly noted that a small-error expansion valid far below the cutoff cannot establish scaling at the 1/e cutoff where attenuation is large, and that "exponential attenuation" overstated high-frequency decay. Theorem 1 (§3.2, App. A.2) separates exact and approximate statements cleanly:

  • Exact for every N >= 1 and every frequency (zero small-error approximation): Each uniform Euler step multiplies a mode's power by an exact fraction strictly between 0 and 1. Multiplying all N steps gives a closed-form algebraic product (Theorem 1) proving two exact facts without Taylor truncation: (i) generated power never exceeds target power in exact arithmetic (0 <= R <= 1), and (ii) bounding each factor directly from the clean endpoint (t = 1) proves retained power falls strictly below 1/e whenever N^2 * P(k) <= 0.7030. This establishes the N^(2/beta) cutoff directly across and above 1/e for all N >= 1, replacing informal "exponential attenuation" wording.
  • Approximate (sub-ceiling regime only): The continuum integral formula describes gentle damping below the cutoff and stays within 6.6% of the exact product all the way up to 1/e (App. A.2).

2. Reconciled Midpoint Scaling, Convergence Rate, and Supported Empirical Conclusions (W2 & Q2; §3.2–§3.4, Theorem 2, Table 1, App. A.3)

  • Correct midpoint cutoff scaling and convergence rate: Old Appendix A.5's N^(4/beta) formula was incorrect and is replaced by Theorem 2 (§3.2, App. A.3). Along the Gaussian flow-matching path, midpoint's second-order global error cancels completely between the trajectory interior and endpoints, yielding third-order asymptotic convergence (N^(-3)) once all grid modes are resolved (§3.3). Right at the 1/e cutoff, rescaling time near t = 1 collapses the exact multi-step midpoint update onto the same N^2 * P(k) scaling variable as Euler (App. A.3), proving that midpoint shares the exact same N^(2/beta) cutoff exponent (R^2 = 0.9999 across all grids in Table 1) while needing 3.54x fewer steps (1.77x fewer evaluations) for any target cutoff, independent of beta. (Classical RK4 rebounds above 1/e past its stability limit, scoping the monotonic 1/e cutoff to first- and second-order polynomial solvers.)
  • Supported empirical conclusions (strengthened by new follow-up experiments): Every empirical conclusion in the paper aligns with Theorem 2:
    1. Constant-factor low-budget speedup across frontier models (§4.2, §4.5, App. D.6): Midpoint recovers 2–3x more high-frequency power than Euler at matched low budgets across all five non-distilled 2.5B–12B models. Re-evaluating CogView4-6B, Lumina-Image-2.0, and AuraFlow v0.3 against converged 100-step midpoint references removes all prior > 1.0 Euler-50 baseline artifacts (all curves < 1.0 and monotonic).
    2. Parameter-free ISD decay gap (§3.3, §4.2, App. D.9): On 31-ring 64x64 latents over N = 2..50, exact Gaussian recursions predict effective decay slopes of 0.31–0.58 (Euler) vs. 1.43–2.03 (midpoint), matching SD3.5-Medium (0.46 vs. 1.34) and FLUX.1-dev (0.66 vs. 1.20).
    3. Solver crossovers and zero-cost float32 fix (§4.4–§4.5, §5, Apps. D.10 & D.12): Sweeping multistep DPM-Solver++(2M) and 2S across N = 4..200 shows midpoint leading up to 16 evaluations, 2M excelling across 20–50 evaluations (0.82–0.87 high-band recovery vs. Euler 0.54–0.63), and switching only the ODE state accumulator to float32 removing over 97% of 400-step bfloat16 drift at zero runtime cost.

Thanks for the authors' response and clarifications. I appreciate the effort put into addressing my concerns. While I acknowledge the strengths of the paper, I feel that my main concerns have not been fully resolved. These issues still affect my overall assessment of the work. Therefore, I will maintain my original score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Response: Exact Non-Asymptotic Euler Bound, Midpoint Superconvergence, & Revision Summary

We thank the Area Chairs and all four reviewers (xxjR, X4mL, fmuo, Bf1t) for their rigorous, constructive critiques—especially X4mL's independent numerical test of the midpoint cutoff, fmuo's pointer to Benita et al. (NeurIPS 2025) and cross-section audit, xxjR's questions on crossover rigor, and Bf1t's scrutiny of reference convergence. Below we summarize our core contribution and the updates in the revised PDF and anonymized code_supplement.zip.

Core take-home. In the base-ODE regime, trajectory acceleration of the linear flow-matching interpolant grows with spatial frequency as amax⁡(k)∝∣k∣β/2a_{\max}(k) \propto |k|^{\beta/2} for P(k)∝∣k∣−βP(k) \propto |k|^{-\beta}, imposing a provable spectral ceiling kc(N)∝N2/βk_c(N) \propto N^{2/\beta} shared by Euler and explicit midpoint up to a β\beta-independent 3.54×3.54\times step factor. Applied to six 2.5B–12B frontier models, this yields concrete rules—midpoint up to NFE≈40\text{NFE}\approx 40 (DPM-Solver++(2S) above 40), σ\sigma-shift ∈[1,2]\in [1,2], latent-space evaluation (avoiding VAE inversion), and large-NN drift monitoring—plus latent spectral signatures of reflow and step distillation.

1. Exact Non-Asymptotic Euler Bound & Midpoint Superconvergence (§3.2, Theorems 1–2, App. A; all reviewers). Appendix A.5 previously stated kcmid∝N4/βk_c^{\mathrm{mid}} \propto N^{4/\beta}. Re-deriving the exact log-gain expansion establishes two stronger, verified results:

  • Exact Euler product & all-NN bound (Theorem 1): With gi(k)=(N−i)2+i2P(k)g_i(k) = (N-i)^2 + i^2 P(k), the identity gigi+1−φi2=N2P(k)g_i g_{i+1} - \varphi_i^2 = N^2 P(k) gives REuler(k,N)=∏i=0N−1 ⁣(1−N2P(k)gi(k) gi+1(k))∈[0,1](∀N≥1,  k≠0),R_{\mathrm{Euler}}(k,N) = \prod_{i=0}^{N-1}\!\left(1 - \frac{N^2 P(k)}{g_i(k)\,g_{i+1}(k)}\right) \in [0,1] \qquad (\forall N \ge 1,\; k \ne 0), proving exact Gaussian no-overshoot and the all-NN bound REuler(k,N)≤exp⁡Φ(N2P(k))R_{\mathrm{Euler}}(k,N) \le \exp\Phi(N^2 P(k)) with Φ(ρ)=∑j≥1ln⁡(1−ρ(ρ+j2)(ρ+(j−1)2))\Phi(\rho) = \sum_{j\ge 1}\ln\bigl(1 - \frac{\rho}{(\rho+j^2)(\rho+(j-1)^2)}\bigr). Thus N2P(k)≤ρcEuler≈0.7030  ⟹  REuler(k,N)<e−1N^2 P(k) \le \rho_c^{\mathrm{Euler}} \approx 0.7030 \implies R_{\mathrm{Euler}}(k,N) < e^{-1} for every N≥1N \ge 1 without perturbative approximations. Also, ∫01c2 dt=π4(P+P−1/2)−1\int_0^1 c^2\,\mathrm{d}t = \frac{\pi}{4}(\sqrt{P}+P^{-1/2})-1 holds exactly, and −ln⁡R≈(2+∫01c2 dt)/N-\ln R \approx (2+\int_0^1 c^2\,\mathrm{d}t)/N stays within 6.6%6.6\% across 1≤k≤kc(N)1 \le k \le k_c(N).
  • Midpoint third-order superconvergence (Theorem 2): On the Gaussian FM ODE, ∫01c3 dt=−14(P−1−P)\int_0^1 c^3\,\mathrm{d}t = -\frac{1}{4}(P^{-1}-P) cancels the boundary terms in ∫01(124c′′+14cc′+16c3) dt=0\int_0^1(\frac{1}{24}c'' + \frac{1}{4}c c' + \frac{1}{6}c^3)\,\mathrm{d}t = 0 identically. Midpoint's nominal O(N−2)∣k∣βO(N^{-2})|k|^\beta log-error vanishes, leaving ln⁡Rmid(k,N)=−π128 ∣k∣3β/2C3/2N3(1+o(1)).\ln R_{\mathrm{mid}}(k,N) = -\frac{\pi}{128}\,\frac{|k|^{3\beta/2}}{C^{3/2} N^3}\bigl(1+o(1)\bigr). Setting ln⁡Rmid=−1\ln R_{\mathrm{mid}} = -1 gives (3β/2)/3=β/2  ⟹  kcmid(N)∝N2/β(3\beta/2)/3 = \beta/2 \implies k_c^{\mathrm{mid}}(N) \propto N^{2/\beta} (ρcmid≈0.05612\rho_c^{\mathrm{mid}} \approx 0.05612), asymptotic ISDmid∝N−3\mathrm{ISD}_{\mathrm{mid}} \propto N^{-3} (α→3\alpha \to 3), matched-NN cutoff ratio 3.542/β3.54^{2/\beta}, and a β\beta-independent 3.54×3.54\times step reduction (1.77×1.77\times NFE) for any target kck_c. Revised Table 1 verifies δmid=2/β\delta_{\mathrm{mid}} = 2/\beta to R2=0.9999R^2 = 0.9999. Classical RK4 rebounds past 11 (min⁡kRRK4≈0.934>e−1\min_k R_{\mathrm{RK4}} \approx 0.934 > e^{-1}), so we withdraw N2p/βN^{2p/\beta} and scope the 1/e1/e-cutoff theorem to p≤2p \le 2.

2. Reframed Large-NN Broadband Drift (§4.4, Fig. 3; X4mL, fmuo, Bf1t). We revised §4.4 from causal attribution ("signed high-frequency bias in vθ−v∗v_\theta - v^*") to phenomenological characterization: on SD3.5 (bfloat16), Euler and midpoint agree at N=100N=100 (0.9440.944 vs. 1.0001.000) before co-drifting across all bands at N=200N=200 (rlow=1.21,rmid=1.49,rhigh=1.61r_{\mathrm{low}}=1.21, r_{\mathrm{mid}}=1.49, r_{\mathrm{high}}=1.61), continuing unsaturatingly to N=400N=400 (4.74×4.74\times high-band vs. N=50N=50) and under σmin⁡=0.02\sigma_{\min}=0.02 clipping, whereas float64 Gaussian controls satisfy R≤1R \le 1 for all NN. Both bfloat16 step accumulation and small-σ\sigma learned-velocity drift are stated as candidate mechanisms.

3. Literature, Reference Audit, & Verification (fmuo, Bf1t, X4mL). We added Benita et al. (NeurIPS 2025) and related works (§1, §6); proved exact Gaussian reflow freezing against fixed targets and replotted Fig. 2(a) with converged non-EMA weights (§4.3, App. A.4/B.5); unified the 3-chain reference audit (Table 5: 97.6%/99.0%97.6\%/99.0\% is midpoint N=28N=28 vs. 5050, while SD3.5 midpoint N=50N=50 vs. 100100 is 84.2%84.2\%); disclosed per-model guidance (§4.1); synced Fig. 3(b), App. C, App. D.1, references, and the AI use statement; and included reviews/verify_*.py in code_supplement.zip.

AuthorsOpenReview ↗

Addendum to Global Response: Completed Follow-Up Experiments (bf16 Drift Isolation, N=400 n=64, DPM++2M, Direct Reflow Curvature, rFID@Shift, & Extended-Model Checks)

Since posting our initial response, we completed six targeted GPU and trajectory experiments (outputs/) and updated our revised manuscript PDF (§4.2–§4.5, §5, §7, Apps. B.5, D.6, D.10–D.12) and code supplement accordingly, resolving the open mechanisms noted in §4.4 and closing remaining empirical checks raised during review:

  1. Decisive isolation of large-NN broadband drift (outputs/sd35_precision_drift_ablation.json): On matched-seed SD3.5 (Euler, uniform, n=16, N=50..400), comparing bf16_default, bf16_vel_fp32_acc (bf16 UNet/DiT velocity + fp32 ODE accumulator), and full_fp32 shows that bfloat16 step-accumulation rounding—not bf16 velocity evaluation—drives the large-NN blowup:
    • Against full_fp32@100, bf16_default inflates (low, mid, high) band power by (1.285x, 1.946x, 2.221x) at N=200N=200 and (1.078x, 3.428x, 4.424x) at N=400N=400 (tot_mean: 0.927→1.2640.927 \to 1.264).
    • Keeping only the ODE accumulator in fp32 (bf16_vel_fp32_acc) eliminates >97%>97\% of the high-band inflation: (1.066x, 1.041x, 1.047x) at N=200N=200 and (1.108x, 1.061x, 1.069x) at N=400N=400, matching full_fp32 ((1.071x, 1.030x, 1.034x) at N=200N=200; (1.114x, 1.050x, 1.054x) at N=400N=400).
  2. Formal n=64n=64 N=400N=400 ladders & qualitative grids (outputs/overstep_sd35-medium_{euler,midpoint}_n64.json): At n=64n=64 (w=1.0w=1.0, matched seeds), per-image N=400 vs 50 high-band power rises 4.72x (Euler, median 4.45x) and 4.20x (Midpoint, median 3.90x); mean pixel clipping grows 0.413% -> 8.377% (Euler, max 42.144%) and 1.499% -> 8.615% (Midpoint, max 42.570%), with visual grids (outputs/qualitative_overshoot/) showing progressive over-sharpening (N=100–200N=100\text{--}200) followed by severe high-frequency mosaic corruption across all sampled seeds at N=400N=400.
  3. Fixed DPM-Solver++(2M) & high-NFE stability (outputs/dpmsolver_spectral_sd35-medium_cfg1_ext.json): Fixing the σ=1\sigma=1 log⁡(0)\log(0) singularity and 0.5*h_curr/h_prev scaling in sample_dpm2m eliminates all NaNs across N=4…200N=4\dots 200 (n=64n=64, ref midpoint N=100). At 1 NFE/step, dpm2m reaches latent rhigh=0.8163r_{\mathrm{high}} = 0.8163 at NFE=20\text{NFE}=20 and 0.8702 at NFE=28\text{NFE}=28 (vs. Euler 0.54370.5437 / 0.63480.6348). Crucially, exponential integrators (dpm2s, dpm2m) avoid additive bf16 step blowup up to N=200N=200 (tot_pow = 0.9610 / 0.9641 vs. Euler 1.2672 and Midpoint 1.3382).
  4. Direct Reflow curvature measurement (outputs/reflow_curvature_direct.json): On SimpleUNet 64x64 (n_eval=512, n_fine=128, fp32), 1-reflow (non-EMA) cuts trajectory straightness MSE by 329x (0.024006→0.0000730.024006 \to 0.000073), RMS acceleration 3.4023→1.01493.4023 \to 1.0149, and high-band acceleration PSD by 15.4x (iter2: 40.5x), directly explaining why self-step dependence vanishes (Euler@2 r_hi_self=0.9181) while fixed-target recovery freezes (r_hi_fixed=0.5970 -> 0.6468). Unconverged EMA checkpoints blow up against the fixed target (Euler@2 r_hi_fixed=91.79 / 68.29).
  5. Perceptual rFID @ shift (outputs/perceptual_shifts_sd35-medium_{cfg1,cfg7}.json): Under both w=1.0w=1.0 (n=64n=64) and w=7.0w=7.0 (n=32n=32) at N=20N=20, s=2s=2 minimizes spectral error (ISDL1=0.4205\mathrm{ISD}_{L_1}=0.4205 / 0.26600.2660, LSD=0.5613\mathrm{LSD}=0.5613 / 0.31920.3192), whereas rFID drops monotonically s=1→2→3s=1 \to 2 \to 3 (139.42→114.03→107.44139.42 \to 114.03 \to 107.44 at w=1w=1; 279.70→200.35→186.01279.70 \to 200.35 \to 186.01 at w=7w=7) alongside stronger low-frequency structure preservation; in the base-ODE regime (w=1w=1), latent rhighr_{\mathrm{high}} still peaks at s=1s=1 (0.558>0.516>0.4570.558 > 0.516 > 0.457).
  6. Uniform midpoint N=100 checks on all 3 extended models (outputs/a1v2_solvers_{cogview4,lumina2,auraflow}_convcheck.json): Re-evaluating CogView4-6B, Lumina-Image-2.0, and AuraFlow-v0.3 (n=64,w=1.0n=64, w=1.0) against uniform midpoint N=100 removes every prior >1.0>1.0 overshoot artifact (previously caused by under-converged Euler-50 references): all Euler and Midpoint curves (N=20,28,50N=20,28,50) stay strictly <1.0<1.0 and monotonic in NN, and Midpoint consistently outperforms Euler at matched NFE.
AuthorsOpenReview ↗

Global Summary: Core Insight, Exact Theory, & Six Verification Experiments

We thank the Area Chairs and all four reviewers (xxjR, X4mL, fmuo, Bf1t) for their constructive feedback throughout the discussion. Alongside our theoretical updates and six follow-up experiments, we rewrote the entire manuscript (main.pdf, §1–§7 and Appendices) in clear, plain language—removing dense inline notation so the core insight, proofs, and practical takeaways read effortlessly.

Core message of the comment. Along a flow-matching path, coarse image layout moves gently while fine spatial details bend sharply near the end of generation. Coarse solver steps cut across those sharp late turns, so a finite step budget acts as a spatial low-pass filter that blurs details above a sharp frequency ceiling. Understanding this single geometric bottleneck explains solver speedups, how trajectory straightening (reflow) differs from step distillation, and why low-precision arithmetic drifts at high step counts.

How all review concerns are resolved in the revised PDF and code supplement:

  1. Exact resolution scaling without small-error approximations (§3.2, Theorems 1–2, Table 1, Appendix A; all reviewers).

    • We fixed the earlier contradiction between §3.2 and Appendix A.5: both Euler and explicit midpoint follow the exact same N^(2/beta) cutoff-frequency scaling (R^2 = 0.9999 across all spectra in Table 1).
    • Neither proof extrapolates a small-error approximation to the cutoff. For Euler (Theorem 1, App. A.2), an exact closed-form product valid for every N >= 1 bounds retained power directly across and above the 1/e threshold. For explicit midpoint (Theorem 2, App. A.3), the second-order global error cancels completely along the flow-matching path (yielding third-order accuracy), and rescaling time near the clean endpoint collapses the exact multi-step update onto a universal curve with zero small-error assumptions. Midpoint reaches any target cutoff in 3.54x fewer steps (1.77x fewer evaluations) than Euler regardless of the spectrum, whereas classical fourth-order RK4 amplifies unresolved high frequencies past its stability limit.
  2. Root cause and zero-cost fix for high-step blowup (§4.4, Figures 4 & 7, Appendix D.10; X4mL, fmuo, Bf1t).

    • Pushing standard bfloat16 sampling to 200–400 steps inflates high-frequency latent power over 4x, clips 8.4% of pixels on average (up to 42.6% across 64 images), and corrupts outputs into blocky mosaics (Figure 7).
    • Keeping the SD3.5-Medium network in bfloat16 while switching only the ODE state accumulator to float32 eliminates over 97% of the 400-step inflation (4.42x down to 1.07x, matching full float32 at 1.05x) at zero runtime cost (148.8s vs. 149.0s, vs. 960.7s for full float32). Rounding in bfloat16 state accumulation—not velocity-network error—drives high-step drift.
  3. Direct trajectory curvature proof for reflow vs. step distillation (§4.3, Figure 3, Appendices A.4 & B.5; X4mL).

    • Measuring 512 trajectories directly shows one reflow pass straightens paths by 329x and cuts high-frequency acceleration power by 15.4x (40.5x after two passes). Thus 2-step Euler retains 87–100% of its own 128-step detail while accuracy against the true data target stays locked at the 32-step teacher ceiling. Step distillation (FLUX.1-schnell) instead synthesizes sharp textures first, inverting the high-to-low band ratio by 12x vs. FLUX.1-dev.
  4. 100-step midpoint checks, multistep solvers, and perceptual validation (§4.2, §4.5, §5, Appendices D.3, D.6, D.11–D.12; all reviewers).

    • Re-evaluating CogView4-6B, Lumina-Image-2.0, and AuraFlow v0.3 against converged 100-step midpoint references removes every > 1.0 baseline artifact (all curves < 1.0) and confirms midpoint's 2–3x low-budget lead across all five 2.5B–12B models (App. D.6).
    • Multistep DPM-Solver++(2M) and 2S sweeps across 4–200 steps establish clear budget handoffs (midpoint up to 16 evaluations, 2M across 20–50, 2S above 40, App. D.12), perceptual rFID and visual grids (Figure 8, App. D.11) verify why shift = 2 balances detail and contrast, and §1/§6 now discuss Benita et al. (NeurIPS 2025).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.