acceptodds
Under review as a conference paper at ICLR 2027

Explicit Error Propagation for Improving Timestep Scheduling of Generative Models

21 views

Abstract

Discrete timestep schedules are among the last heuristic components of diffusion and flow-based samplers. Common choices, such as time-uniform and logSNR-uniform, rely on unrealistic simplifying assumptions: locally stationary score errors, slowly varying noise levels, and uniform error propagation along the sampling trajectory. We introduce EP-Flow, a principled framework for timestep scheduling based on explicit error propagation. By modeling the sampler as a discrete-time dynamical system, we derive a tractable surrogate functional decomposing total generation error into three components: (i) a persistent directional bias arising from score approximation error, (ii) a residual variance from step-local stochastic error, and (iii) a local truncation error. Each component is weighted by trajectory-dependent propagators quantifying how errors at each step accumulate at the final sample. This objective applies to any initial schedule: EP-Flow both improves heuristic spacings and provides a complementary signal for refining already-learned ones. Across four standard datasets—CIFAR-10, FFHQ, AFHQv2, and ImageNet-64—EP-Flow consistently improves few-step generation over the widely used Karras schedule. It generalizes across training recipes (EDM2), refines learned schedules in Stable Diffusion v1.5, and extends to large-scale text-to-image generation with FLUX.1-dev.

Primary area:
generative models
Keywords:
Diffusion Models, Flow Matching, Generative Models, Sampling, Timestep Scheduling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DbQqOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a novel framework for offline optimization of discrete timestep schedules for diffusion and flow-based samplers, named EP-Flow. It derives a tractable surrogate cost that decomposes terminal sampling error into bias, variance, and discretization terms weighted by trajectory-dependent propagators. Experiments across EDM/EDM2, Stable Diffusion v1.5, and FLUX.1-dev validate improved few-step generation quality at matched solver, endpoints, and NFE.

Critical strengths

1.A principled, propagation-aware objective with exact degenerate cases. Eqs. 8 and 13 explain how local errors propagate along the trajectory, while Appendix F derives uniform-t and logSNR-uniform spacings under idealized conditions, highlighting deviations in modern models (as shown in Fig. 1).

2.Broad evaluation across model families. Table 1 covers EDM, EDM2, SD v1.5, and FLUX.1-dev under matched solvers, endpoints, and NFE settings.

Critical weaknesses

1.Overstated improvements. Table 1 shows a slight regression on FFHQ 64×64 at NFE=18 (FID 9.86 vs. 9.74 for Karras), contradicting the claimed consistent gains across all EDM datasets. The authors should qualify this claim.

2.Limited validation. Section 4.2.1 introduces a piecewise-kernel formulation to address the failure of A1 on SD v1.5, but neither the two-regime choice nor schedule sensitivity to segmentation is ablated.

Questions for authors

1.Table 1 shows a regression on FFHQ 64×64 at NFE=18 (9.86 vs. 9.74 for Karras). How does the objective behave as headroom shrinks at higher NFE, and can the claim in the abstract be scoped (e.g., to NFE ≤ 14 or to low-NFE regimes)?

2.For the piecewise kernel on SD v1.5 (as in Appendix C.2), how sensitive is the optimized schedule and the resulting GenEval/HPSv2 gain to the number of segments and their boundaries? A sensitivity study would substantiate the "robustness beyond stationarity" claim in Section 5.1.

Reviewer_fY1DOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EP-Flow, an offline timestep-scheduling method for diffusion and flow-based generative models. It constructs a surrogate objective that accounts for the propagation and temporal correlation of model approximation and discretization errors. Experiments on EDM/EDM2, Stable Diffusion v1.5, and FLUX.1-dev show improvements, particularly in low-NFE settings, without additional inference-time model evaluations.

Critical strengths
  1. Explicitly incorporating error propagation provides an interesting perspective on timestep scheduling.
  2. The experiments cover several model families and show substantial improvements over standard schedules in some low-NFE settings.
  3. The method does not require model retraining or additional inference-time evaluations.
Critical weaknesses
  1. The related work and empirical comparisons are seriously incomplete. The paper omits schedule-optimization methods such as GITS [1], LD3 [2], Optimal Stepsize [3], and HSO [4]; learned solver methods such as S4S [5] and GAS [6]; and model-specific solvers such as BNS [7] and AMED-Solver [8]. The main experiments likewise contain few recent competitive baselines, making the novelty and competitiveness difficult to assess.
  2. The surrogate relies on many approximations whose combined error is unclear. These include trajectory linearization, separate bounds for model and discretization errors, the far-field kernel decomposition, leading-order discretization errors, scalar Jacobian propagation, and an empirical anisotropic correction. The paper does not show that the final surrogate reliably tracks the true endpoint error in the low-NFE regime. The central role of propagation is also not isolated. A Γj=I\Gamma_j=I ablation with all other terms and optimization settings unchanged is necessary.
  3. The results depend strongly on the anisotropic correction. In Table 16, the α=0\alpha=0 variant is worse than cosine at 4, 6, and 8 NFE. The gains after tuning α\alpha are small. Since α\alpha is calibrated on GenEval, it is unclear whether the reported GenEval gains use an independent validation protocol.
  4. The optimization robustness and reproducibility are unclear. Figure 2 shows that different initializations reach different local optima. The method uses multiple restarts, but the restart count, initialization strategy, and optimization hyperparameters are not fully reported. No variance across optimization runs or restart budgets is provided.
  5. The practical reuse of the offline statistics is insufficiently studied. Statistics collection takes about three A6000 hours, followed by separate optimization for each NFE. It is unclear which statistics remain valid after changing the solver, endpoints, CFG scale, conditioning distribution, or model parameterization. Solver changes require at least a new discretization objective, while guidance changes may also invalidate the Jacobian and error statistics.

Reference

[1] Chen D, Zhou Z, Wang C, et al. On the trajectory regularity of ode-based diffusion sampling[J]. arXiv preprint arXiv:2405.11326, 2024.

[2] Tong V, Hoang T D, Liu A, et al. Learning to discretize denoising diffusion odes[C]//International Conference on Learning Representations. 2025, 2025: 47244-47282.

[3] Pei J, Hu H, Gu S. Optimal stepsize for diffusion sampling[J]. arXiv preprint arXiv:2503.21774, 2025.

[4] Zhu A, Su R, Zhao Q, et al. Hierarchical schedule optimization for fast and robust diffusion model sampling[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2026, 40(16): 13907-13915.

[5] Frankel E, Chen S, Li J, et al. S4s: Solving for a diffusion model solver[C]. International Conference on Machine Learning (ICML 2025), 2025.

[6] Oganov A, Bykov I, Neudachina E, et al. GAS: Improving Discretization of Diffusion ODEs via Generalized Adversarial Solver[C]//International Conference on Learning Representations. 2026, 2026: 39711-39747.

[7] Shaul N, Singer U, Chen R T Q, et al. Bespoke non-stationary solvers for fast sampling of diffusion and flow models[J]. arXiv preprint arXiv:2403.01329, 2024.

[8] Zhou Z, Chen D, Wang C, et al. Fast ode-based sampling for diffusion models in around 5 steps[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024: 7777-7786.

Questions for authors
  1. Could the authors provide a Γj=I\Gamma_j=I ablation that retains both model approximation and discretization errors while removing propagation weighting? Model-error-only and discretization-error-only variants would also help clarify the contributions of the different terms.

  2. Could the authors compare EP-Flow with methods such as GITS, LD3, Optimal Stepsize, and HSO under matched models, solvers, and NFE budgets, where feasible? A comparison of offline optimization costs would also be useful.

  3. How is α\alpha selected, and is a separate validation set used? Also, how much do the optimized schedules and generation metrics vary across different initializations or restart budgets?

3 comments
AuthorsOpenReview ↗

Response to the Reviewer fY1D

We thank the reviewer for the feedback. We address each concern with additional experimental evidence and clarifications below.

Q2 & W1 (Comparison with existing methods).

We clarify a central misunderstanding: EP-Flow neither distills reference trajectories nor redesigns numerical solvers. Related to AYS [9] and Xue et al. [10], EP-Flow advances analytical schedule optimization from local errors to global error propagation.

Learned solvers (S4S [5], GAS [6], BNS [7], AMED [8]) modify integration rules and thus fall outside our fixed-solver scope. Teacher-based methods (GITS [1], LD3 [2], OSS [3]) instead distill reference trajectories. All these methods are cited in the revised manuscript, with their distinctions further clarified in Appendix B (Table 11).

Beyond AYS and CTR in the main paper, we further refine HSO [4] on SD v1.5 and OSS [3] on FLUX.1-dev. Table 12 shows improvements under matched solvers, endpoints, and NFEs, demonstrating the complementary value of our objective across different approaches.

Q1 & W2 (Theoretical validity and propagation ablation).

We respectfully clarify that our framework does not rely on six compounding approximations. The listed components differ fundamentally:

  • Rigorous bounds: Error separation follows from the triangle inequality; far-field decomposition yields an exact sandwich bound (Appendix E).
  • Leading-order analysis: Trajectory linearization and discretization error have explicit order guarantees (Appendix D).
  • Modelling approximations: Without directional alignment information, gˉ\bar g provides an unbiased prior for propagation.

Optimizing principled bounds rather than exact endpoint errors is standard practice. AYS [9] optimizes a KL upper bound, while Xue et al. [10] optimize a discretization-error upper bound. Neither claims to characterize the exact endpoint error. Our cost agrees with FID rankings across Karras schedules (Appendix A.2, Table 6), with further evidence in Tables 7–8.

Excerpt from Table 6 (EDM, CIFAR-10):

NFE = 11ρ=1\rho=13579
Cost ↓106.673.260.810.560.48
FID ↓174.1622.0715.4713.9013.35

The reviewer-requested ablation further strengthens our central claim. Removing propagation (Γ≡I\Gamma\equiv I) with all other settings unchanged worsens FID in 15/16 settings (Appendix A.1, Table 5).

Q3 & W3 (Anisotropic correction).

All main results use the isotropic formulation without tuning. Anisotropic correction is studied solely to validate the theoretical prediction, using GenEval Overall as a quality proxy.

W5 (Statistics reuse).

Statistics are reusable across NFEs, endpoints, and equivalent parameterizations, and jointly collectible across solvers (full details in Appendix C, Table 13). The 3 A6000 GPU-hour collection cost can be reduced through batching and shared evaluations.

Q3 & W4 (Optimization robustness).

Different local optima are expected for our non-convex objective. Each comparison starts from its baseline schedule. We use 3 restarts (seed 42, under 1 minute each) and select the lowest-cost solution. Hyperparameters are in Appendix I. Across restarts, lower cost generally corresponds to better FID (Tables 7–8), supporting the reliability of cost-based schedule selection.

References

[9] Sabour, A., Fidler, S., and Kreis, K. Align Your Steps: Optimizing Sampling Schedules in Diffusion Models. ICML, 2024.

[10] Xue, S., Liu, Z., Chen, F., Zhang, S., Hu, T., Xie, E., and Li, Z. Accelerating Diffusion Sampling with Optimized Time Steps. CVPR, 2024.

Thank you for the detailed rebuttal and substantial revision. After checking the revised manuscript, I acknowledge that I misinterpreted the role of the anisotropic correction.

The revision addresses my core concerns and strengthens the manuscript, including those regarding the analytical approximations, empirical validation, related work, and reproducibility. The additional evaluations involving HSO and OSS are also useful. I have therefore raised my rating.

Dear Reviewer fY1D,

Thank you for your thoughtful reassessment and encouraging feedback. Your constructive suggestions have helped strengthen our manuscript, and we sincerely appreciate your time and effort throughout the review process.

Best,

Authors

Reviewer_zVmAOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes EP-Flow, an offline method for optimizing timestep schedules in pretrained diffusion and flow-matching models. It decomposes terminal sampling error into model approximation, discretization, and propagation effects, then estimates model-level statistics to optimize a fixed schedule without retraining or inference overhead. Experiments cover EDM/EDM2, Stable Diffusion v1.5, and FLUX.1-dev.

Critical strengths
  1. The propagation-forcing viewpoint is conceptually valuable: it recognizes that a local error matters according to both its magnitude and its downstream amplification.

  2. The evaluation spans several model families and solvers, with meaningful low-NFE improvements over Karras and CTR schedules and no added inference cost.

Critical weaknesses
  1. The theory begins with matrix-valued propagators, but the implemented objective replaces them with scalar trace-based quantities. The stated leading-order theorem justifies trajectory linearization, not this matrix-to-scalar substitution. Therefore, it does not establish that the optimized surrogate approximates terminal error.

  2. The required model error is defined relative to the unknown true marginal vector field, yet it is estimated using training-residual MSE. Such residuals generally also contain irreducible conditional-target variance. In addition, gains over the strongest FLUX baseline (cosine) are very small and obtained after tuning the anisotropy parameter on GenEval, which is the reported evaluation benchmark.

Revision:

Following the authors’ rebuttal, I am revising my evaluation to reflect the additional experimental evidence and clarifications, which address some of my concerns and strengthen the empirical support for the proposed approach.

Questions for authors
  1. Can you justify or empirically validate the replacement of matrix propagators by products of scalar Jacobian traces, by comparing the surrogate with measured terminal trajectory error?

  2. How are the true approximation error variance and cross-timestep kernel estimated from training residuals after removing irreducible conditional-target variance?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive feedback and thoughtful evaluation of our submission. We have carefully revised the manuscript in light of their comments, incorporating additional experiments and clarifications (revisions are marked in blue):

Related work and comparisons (fY1D): expanded discussion of relevant scheduling and solver optimization methods (App. B, Table 11), and additional comparisons involving HSO and OSS (Table 12).

Role of propagation and the cost functional (fY1D, zVmA): a Γ=I\Gamma=I ablation (App. A.1, Table 5) and an examination of the cost against generation quality across schedules (App. A.2, Tables 6–8).

Theoretical structure and error statistics (fY1D, zVmA): clarification of the components of the surrogate, namely theoretical bounds (App. E), leading-order analysis (App. D), and the motivation for the scalar propagation proxy in the absence of directional information (§4.2.2, App. F); and an empirical study on FLUX.1-dev isolating the effect of the Bayes irreducible variance on the error statistics (App. A.3, Table 9); and a discussion of the scalar propagator and the irreducible variance, together with directions for future work (App. M).

Robustness and reproducibility (DbQq, fY1D): a sensitivity study of the piecewise kernel (App. A.4, Table 10), optimization details (App. I), and reuse of offline statistics (Table 13).

Scope and clarifications (DbQq, fY1D, zVmA): clarification of the few-step scope, the isotropic formulation (α=0\alpha=0) used in all main results, and the role of the anisotropic correction in examining the predicted scaling α⋆∝1/NFE\alpha^\star \propto 1/\mathrm{NFE}.  

We are sincerely grateful to all reviewers for their time and constructive engagement throughout the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.