- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
- The paper studies frequency-dependent discretization error in flow matching. In a Gaussian setting, it derives frequency-dependent acceleration and a spectral cutoff that changes with the number of sampling steps.
- Experiments on five pretrained image-generation models examine spectral recovery, solver choice, and timestep schedules. Further experiments investigate reflow, distillation, and spectral overshoot at large step counts.
Critical strengths
- Frequency-resolved diagnostics reveal behavior that aggregate metrics can miss. The observation that VAE decoding can reverse the measured spectral trend is particularly useful.
- The exact Gaussian setting provides a controlled starting point. The reference-convergence checks also expose an important source of error in evaluating sampling methods.
Critical weaknesses
- The higher-order scaling contradicts itself: the main text says solver order preserves the cutoff exponent, while Appendix A.5 predicts for midpoint.
- Appendix A.5 also contains an algebraic inconsistency. A correction proportional to gives , not .
- Appendix A.4 restricts the expansion to frequencies well below the cutoff. Setting that expansion to order one does not, by itself, establish its accuracy at the cutoff.
- The additional-model experiments use Euler-50 references without the convergence checks reported for SD3.5 and FLUX. Their absolute spectral-recovery and overshoot measurements therefore remain reference-dependent.
- Power above a finite-step reference does not establish error in the learned velocity field. The synthetic exact-velocity control supports a possible explanation, but does not isolate the cause of overshoot in the pretrained models.
Questions for authors
- What is the correct midpoint cutoff scaling, and how should Appendix A.5 be reconciled with the main text?
- Do the additional-model solver rankings and overshoot observations persist against independently converged references?
3 comments
Follow-Up to Bf1t: Uniform Midpoint N=100 Checks Across All Three Extended Models (W4) & fp32 Precision Ablation (W5)
Following up on our initial reply, we have now completed both empirical follow-ups addressing W4 (additional-model reference convergence) and W5 (mechanism behind large- drift):
-
Uniform
midpoint N=100reference checks on all three extended models (W4,outputs/a1v2_solvers_{cogview4,lumina2,auraflow}_convcheck.json, ): Re-evaluating CogView4-6B, Lumina-Image-2.0, and AuraFlow-v0.3 with convergeduniform midpoint N=100(NFE=200) references eliminates every prior ratio observed against default-shiftEuler-50references (previously1.070/1.062on CogView4,1.297midpoint /1.045Euler on Lumina2, and1.099/1.110on AuraFlow). All curves remain strictly and monotonic in , and Midpoint consistently outperforms Euler at matched NFE:- CogView4-6B: Euler () latent (); Midpoint () (). At , Midpoint (
0.9647) exceeds Euler (0.9169) with lower ISD at . - Lumina-Image-2.0: Euler () (); Midpoint () (). The spurious Lumina2 Euler@20 artifact (
1.0448) disappears completely. - AuraFlow-v0.3: Euler () (); Midpoint () ().
- CogView4-6B: Euler () latent (); Midpoint () (). At , Midpoint (
-
Precision ablation isolating the large- drift mechanism (
W5,outputs/sd35_precision_drift_ablation.json): We compared matched-seed SD3.5 (Euler, uniform, n=16, N=50..400) underbf16_default,bf16_vel_fp32_acc(bf16velocity forward pass +fp32ODE accumulator), andfull_fp32. Relative tofull_fp32@100(band_high=0.0604),bf16_defaultinflates high-band power2.221xat and4.424xat (band_mid:1.946x/3.428x). Retaining only the ODE step accumulator infp32(bf16_vel_fp32_acc) eliminates of the high-band inflation (1.047xat ,1.069xat ), nearly matchingfull_fp32(1.034x/1.054x). This experimentally separates numerical step-accumulation precision from learned velocity error as the dominant driver of large- broadband drift.
Response to Reviewer Bf1t: Midpoint Scaling, Exact Bound, 100-Step Reference Checks, & fp32 Ablation
We thank Reviewer Bf1t for highlighting the value of our latent-versus-pixel observation, multi-model evaluation, and reference-convergence checks. Because our earlier reply suffered from OpenReview formula-rendering glitches, below is a clean, plain-language summary of how every weakness (W1–W5) and question (Q1–Q2) is resolved in the newly polished manuscript (main.pdf).
1. Reconciling Midpoint Cutoff Scaling & Bridging to 64x64 Frontier Latents (W1, W2, Q1; §3.2–§3.4, Theorem 2, Table 1, Apps. A.3 & D.9)
- Analytical reconciliation (W1, W2): Both points you raised on old
Appendix A.5were spot-on: it contradicted§3.2and contained an algebra slip (a second-orderk^beta / N^2error already impliesN^(2/beta), notN^(4/beta)). Re-deriving explicit midpoint (Theorem 2,App. A.3) uncovered an even cleaner property: along the flow-matching path, midpoint's second-order global error cancels completely between the trajectory interior and endpoints, leaving a third-order leading error (k^(3*beta/2) / N^3). Dividing both exponents by 3 yields the exact sameN^(2/beta)cutoff exponent as Euler (Table 1,R^2 = 0.9999), while boundary rescaling neart = 1proves midpoint reaches any target cutoff in3.54xfewer steps (1.77xfewer evaluations) regardless ofbeta. (Classical RK4 rebounds above1/epast its stability limit, scoping the monotonic1/ecutoff to first- and second-order polynomial solvers.) - Parameter-free bridge to
64x64latents (Q1): Because64x64latents have only 31 radial rings, the1/ecutoff reaches the grid edge within 5–12 steps (leaving 2–6 quantized integer points, too coarse to fit a power-law cutoff exponent directly). Instead, full-spectrum distance (ISD,§3.3) connects theory to frontier models with zero free parameters: evaluating exact Gaussian recursions on a 31-ring grid overN = 2..50predicts effective decay slopes of0.31–0.58(Euler) and1.43–2.03(midpoint,App. D.9), matchingSD3.5-Medium(0.46vs.1.34) andFLUX.1-dev(0.66vs.1.20).
2. Non-Asymptotic Attenuation Bound Across the Cutoff (W3; §3.2, Theorem 1, App. A.2)
We replaced Appendix A.4's small-error expansion argument with an exact closed-form product identity (Theorem 1) valid for every N >= 1 and every frequency. Bounding each step factor directly from t = 1 proves—with zero small-error or asymptotic approximations—that retained power falls strictly below 1/e whenever N^2 * P(k) <= 0.7030, establishing the N^(2/beta) cutoff rigorously right across the threshold.
3. Converged 100-Step Midpoint Reference Checks on All Three Additional Models (W4 & Q2; §4.2, App. D.6)
Directly answering Q2: we re-evaluated AuraFlow v0.3, CogView4-6B, and Lumina-Image-2.0 (n = 64 matched seeds, unguided w = 1.0, FLUX.1-dev at its native embedded scale 3.5, FLUX.1-schnell at w = 0; §4.1, App. B.2) against converged 100-step uniform-midpoint (200 NFE) references (App. D.6):
- Every prior
> 1.0overshoot value (caused solely by normalizing against slower-converging 50-step shifted Euler baselines, includingLumina-Image-2.0Euler1.05atN = 20) disappears completely. - Across
N = 20, 28, 50, all Euler and midpoint curves stay strictly below1.0, converge monotonically withN, and preserve midpoint's strong advantage at matched budgets: at 40 evaluations (20-step midpoint), high-frequency recovery reaches0.965(CogView4-6B),0.929(Lumina-Image-2.0), and0.886(AuraFlow v0.3), beating 50-step Euler (50 evaluations:0.917,0.872, and0.792) on every model.
4. Experimental Isolation of the Large-N Drift Mechanism (W5; §4.4, Figure 4, App. D.10)
Directly resolving W5: holding the SD3.5-Medium velocity network fixed on matched seeds (N = 50..400), comparing default bfloat16, full float32, and hybrid precision (bfloat16 velocity forward pass + float32 ODE state accumulator) separates numerical rounding from velocity-field error. Default bfloat16 inflates high-frequency power 2.22x at N = 200 and 4.42x at N = 400 (vs. full float32 at N = 100). Keeping only the ODE state accumulator in float32 suppresses N = 400 inflation to 1.07x (matching full float32 at 1.05x), eliminating over 97% of the overshoot at zero runtime overhead and establishing bfloat16 step-accumulation rounding as the primary cause.
Response to Reviewer Bf1t: Midpoint Scaling & ISD Bridge, Theorem 1 Bound, & Extended-Model References
We thank Reviewer
Bf1tfor recognizing the methodological importance of latent-space measurement, our multi-team frontier validation, and the SOSS negative result. Every weakness and question you raised pointed to places needing tighter rigor, all now incorporated into the revised PDF.1. Midpoint Scaling Contradiction, N2/β vs. Frontier Data, & ISD Bridge (W1, W2, Q1)
A.5(Theorem 2,§3.2,App. A.3): Appendix A.5's N4/β derivation was erroneous. For explicit midpoint on x^˙k=c(t,k)x^k, ∫01c(t,k)3dt=−41(P−1−P) cancels [c′/24+c2/8]01=241(P−1−P), making the nominal O(N−2)∣k∣β global log-error vanish identically. The leading error is third-order, lnRmid(k,N)=−128πC3/2N3∣k∣3β/2(1+o(1)), giving (3β/2)/3=β/2⟹kcmid(N)∝N2/β—the exact same exponent as Euler, with a β-independent 3.54× step reduction (1.77× NFE).Table 1verifies δmid=2/β across (β,d) to R2=0.9999 (1.386,1.023,0.677 vs. 1.333,1.000,0.667; classical RK4 rebounds past 1 with minkR≈0.934>e−1, so N2p/β is withdrawn).§3.3,App. D.10): With K=31 integer bins, frontier kc jumps from 0–1 (N=2–4) to saturation (31, N=5–12), leaving 2–6 quantized points. Instead, ISD(N) provides a parameter-free bridge: asymptotically (N→∞, fixed K), Theorems 1–2 give ISDEuler∝N−1 (α→1) and ISDmid∝N−3 (α→3). At finite K=31 over N∈[2,50], exact Gaussian recursions at β∈[1.95,2.0] predict αEuler∈[0.31,0.58] and αmid∈[1.43,2.03] (reaching 2.50–2.58>2 on [5,50] toward α→3), matching SD3.5 (0.46 / 1.34) and FLUX-dev (0.66 / 1.20) with zero free parameters.2. Rigor of Appendix A.4 & Non-Asymptotic Attenuation Bound (W3)
We replaced A.4's informal telescoping remark and decay phrasing with Theorem 1 (
App. A.2): using gi(k)=(N−i)2+i2P(k) and gigi+1−φi2=N2P(k) yields REuler(k,N)=∏i=0N−1(1−gi(k)gi+1(k)N2P(k))≤expΦ(N2P(k)),Φ(ρ)=∑j=1∞ln(1−(ρ+j2)(ρ+(j−1)2)ρ), valid for every N≥1 and all k=0. Since Φ(ρcEuler)=−1 at ρcEuler≈0.7030, N2P(k)≤ρcEuler guarantees REuler(k,N)<e−1 non-asymptotically.3. Extended-Model References & Per-Model Guidance (W4 & Q2)
§4.2,App. D.6): Unlike SD3.5/FLUX-dev (evaluated against uniform-midpoint references; seeTable 5), AuraFlow, CogView4, and Lumina2 used default-shift Euler-50 references without an N=100 midpoint check. Internal table values confirm Euler-50 is slightly under-converged: midpoint crosses rhigh=1.0 by N=3–8 on all three, and Lumina2 Euler reaches 1.045 at N=20. Crucially, matched-NFE Euler-vs.-midpoint ratios at fixed N divide by the identical reference PSD, so the reference cancels mode-by-mode (App. D.7also confirms schedule-ranking invariance). We added this caveat in§4.2/App. D.6and excluded extended-model rhigh>1 from large-N drift claims.§4.1,App. B.2): SD3.5, AuraFlow, CogView4, and Lumina2 run at w=1.0; single-pass guidance-distilled FLUX.1-dev runs at its native scale 3.5 (Meng et al., 2023); FLUX.1-schnell runs at w=0.4. Large-N Drift Mechanism (W5)
Agreed: linear
float64Gaussian controls (REuler≤1 proved in Theorem 1(i)) cannot separate nonlinear velocity error frombfloat16arithmetic.§4.4andFig. 3now frame this as broadband spectral drift (rlow=1.21,rmid=1.49,rhigh=1.61 at N=200, continuing through N=400 and σmin=0.02 clipping), withbfloat16step accumulation (Lewandowski et al., 2026; He et al., 2023) and small-σ velocity bias as explicit candidate mechanisms.