- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a proportional-integral (PI) adaptive solver for stochastic diffusion sampling, using current and previous local errors to adjust step sizes. It introduces a diffusion noise-normalized error estimator and a method for averaging adaptive trajectories into reusable static sampling schedules. Experiments on Gaussian mixtures, FFHQ, ImageNet-64, and LM1B show improvements over several baselines in sampling quality at matched NFE. The results also suggest that the benefits of per-sample adaptivity are task-dependent, with static schedules sometimes performing comparably or better.
Critical strengths
1. Practical and reusable data-driven sampling schedules. A notable strength is the conversion of per-sample adaptive trajectories into reusable static schedules. These schedules can be integrated with existing samplers without modifying or retraining the underlying diffusion model. The improvements observed with stochastic Heun and EDM-churn suggest that the learned allocation of sampling steps captures useful information about the diffusion dynamics. This makes the approach practically relevant beyond the proposed adaptive solver itself.
2. Evaluation across multiple modalities with informative ablations. The paper evaluates the approach on synthetic distributions, image generation, and language diffusion. The ablations help isolate the contributions of PI control and noise-normalized error estimation. Importantly, the authors also acknowledge settings where per-sample adaptivity provides limited benefits. These results offer useful empirical insights into when adaptive sampling or static schedules may be preferable.
Critical weaknesses
W1. Limited Comparison with Existing Optimized Sampling Schedules. The paper demonstrates promising improvements over the handcrafted EDM schedule and includes comparisons with the entropy-based schedule. However, the experimental evaluation does not cover other relevant data-driven scheduling methods already discussed by the authors in the Related Work section, such as Align Your Steps (Sabour et al., 2024) and Adaptive Time-Stepping Schedules (Chen et al., 2024). Although these methods may target different solver settings, additional comparisons where applicable, or a more detailed discussion of their differences and applicability, would help better establish the effectiveness of the proposed PI Static schedule relative to existing approaches.
W2. Limited evidence for practical efficiency gains over strong baselines. The adaptive PI solver does not consistently outperform the strongest existing sampler, EDM-churn. In particular, at 49 NFE, its FID is substantially worse on both FFHQ and ImageNet-64. Some of the strongest results instead come from combining the proposed static schedule with EDM-churn. Furthermore, efficiency is evaluated primarily through NFE, without reporting wall-clock sampling time or the computational cost of constructing the static schedules. These omissions make it difficult to assess the practical efficiency advantage of the proposed approach, particularly when per-sample adaptive execution introduces additional implementation overhead.
Questions for authors
Q1. Comparison with Existing Schedules. Could the authors provide comparisons with other optimized scheduling methods discussed in the Related Work, such as Align Your Steps and Adaptive Time-Stepping Schedules, where applicable, or clarify why these methods are not directly comparable?
Q2. Practical Efficiency. Could the authors report wall-clock sampling time in addition to NFE, and clarify the computational overhead of constructing the PI Static schedule?
Q3. Adaptive vs. Static Sampling. Why does the PI adaptive solver perform substantially worse than EDM-churn at low NFE, while the PI Static schedule combined with EDM-churn achieves better results? Could the authors further explain this difference?
2 comments
Thank you for the additional experiments and clarifications. The new wall-clock results and the explanation of the differences between adaptive and static sampling have addressed most of my concerns. While I understand the challenges of directly comparing different scheduling approaches, the relative advantages of PI Static over existing methods remain insufficiently established. Therefore, I will maintain my original rating.
Rebuttal by the Authors
Thank you for the review and the provided feedback. Our responses follow.
Q1 Comparison with Existing Schedules
We have looked at implementing the Align Your Steps method and Adaptive Time-Stepping Schedules. Neither method supplies an implementation that works for the EDM pipeline (Align Your Steps does not supply an implementation at all). Therefore, both methods would need to be implemented and tuned for the EDM pipeline, which would introduce additional confounds and make these hard to use as a proper baseline.
Q2 Practical Efficiency
The difference in wall-clock time between the PI solver and other methods is fairly small. We ran an experiment comparing total wall-clock time required for generating 10 batches. The overhead created by the more complex sampling algorithm is about 1-2% compared to simpler algorithms such as Euler-Maruyama or EDM. Compared to the Gotta Go Fast sampler, our algorithm is about 2-5% faster in terms of seconds/sample at matched NFE. The difference between method likely shrinks with larger models, as larger models require more compute per sample, while the compute required by our algorithm remains the same. For a full table of the results, refer to section 6.4 of the updated manuscript.
The time it takes to compute a static schedule from sampled PI trajectories depends on the number sampled. In this paper, we used 50000 sampled trajectories to compute the static schedule at matched NFE. Aggregating these sampled trajectories into a single static schedule takes about 5 seconds, and only has to be done once before the schedule is usable. Sampling trajectories is the computationally expensive part. Sampling 50000 images takes about 18 hours on ImageNet-64 and 6 hours on FFHQ at 75NFE. However, this large sample size is likely not required to obtain good schedules due to how homogeneous the individual per-sample schedules are.
Q3 Adaptive vs Static Sampling
EDM-churn with PI static performs substantially better than PI adaptive at 49 NFE because these use different sampling mechanisms. EDM-churn does not directly sample from an SDE. Instead, it samples from the probability flow ODE, increasing the noise level at step ti to γσ(ti), then denoising this in a predictor and corrector steps to ti+1. Instead, PI adaptive uses stochastic Heun’s method on the reverse SDE. This adds a noise increment Δwi to both the predictor and corrector step. This different mechanism of adding noise in the two samplers likely explains the large difference in performance between these two samplers. Even though this difference in samplers exists, we still show that our PI Static schedule can improve performance over the EDM schedule in a sampler using a different sampling method.