- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper asks what timestep schedules in few-step diffusion sampling should actually optimize. The authors observe that existing scheduling objectives, discretization error, state- or trajectory-level error, and paired feature error, all derive from the sampling dynamics, whereas generation quality is measured by FID in Inception feature space. In the extreme few-step regime of 4–16 NFE, the two do not share the same optimization geometry.
Critical strengths
-
The paper's decomposition of Q, together with its analysis of the finite-sample bias, is excellent.
-
The main experiment compares different objectives under the same parameter settings, initialization, and search budget at the same time point, thus effectively isolating the influence of the optimization objective itself.
-
The authors decompose the per-sample feature error into "the offset of the overall feature mean" and "the error fluctuation between samples," and optimize only the former. This design is based on the fact that closer images are to the reference result does not automatically mean that the FID of the entire batch of generated images is lower.
Critical weaknesses
The Stable Diffusion experiments used an input range for feature extraction that does not match the expected one, which affects the interpretation of these results with respect to standard FID. This setting is used both for the feature optimization objective and for the FID evaluation in this part.
The reliability of the method depends on a suitable initial timestep, but the optimization objective alone cannot determine whether the search has entered an unreliable region. The paper's local analysis depends on the directional relationship between the reference objective and the real-data objective, and these conditions involve the feature statistics of real data.
Questions for authors
-
Please confirm what preprocessing was used for the real COCO reference statistics in Stable Diffusion. After extracting features for the generated images, the model reference images, and the real images all through a correct and consistent pipeline, re-evaluate the existing timesteps and re-run the objective optimization. Do the method ranking and the gaps in Table 1 remain?
-
Apart from the empirical behavior of the standard initialization, does the current method include any constraint or diagnostic that does not require real-data statistics, in order to identify the cases in Table 15 where the objective value is lower but the FID is worse? If not, beyond CIFAR-10, which existing experiments can support that the standard initialization still keeps the search in the reliable region?
2 comments
Response to g7yQ: Detecting Unreliable Searches
W2/Q2. Detecting unreliable searches
The submitted method used alone to select a schedule. In the CIFAR-10 example from Table 15 of the submission (Table 18 in the revision), the schedule found from random seed 21 had a lower on the 2,000 search samples than the schedule found from the standard Karras start (6.249 vs. 6.264), yet its FID against real images was worse (20.82 vs. 16.93). We have added a diagnostic that flags this case after the search. Computing the diagnostic, setting its threshold, and deciding whether to flag a schedule all use only samples from the model.
How the diagnostic works. For a candidate schedule , we compute on 10,000 samples. Both sets of images come from the same model. One set is generated with , and the other with a reference sampler that uses many more steps. We compute the same distance for the schedule obtained by optimizing from the standard Karras start and use that value as the baseline. We estimate sampling variability by splitting the model samples into two disjoint halves and computing the distance on each half. We flag a candidate when the percentage increase over the baseline exceeds the threshold derived from this variability.
On CIFAR-10, the diagnostic flagged all four failed runs (Table R8), including the run with initialization seed 21. For this run, exceeded the baseline by 26.4%, well above the 2.8% threshold. Most of this increase came from the covariance term, which explains why the diagnostic flagged this schedule even though its on the search samples was slightly lower.
Table R8. Diagnostic results for six random starts per dataset using DPM-Solver++ at 4 NFE. To evaluate the diagnostic, we used FID against real images to determine which runs succeeded. A run counted as successful if its FID on 50,000 generated samples was within 1.6% of the FID obtained from the standard Karras start.
| Dataset | Flag threshold | Failed runs flagged | Successful runs flagged |
|---|---|---|---|
| CIFAR-10 | 2.8% | 4/4 | 0/2 |
| FFHQ | 3.3% | 3/3 | 0/3 |
| AFHQv2 | 4.1% | 1/1 | 0/5 |
Evidence beyond CIFAR-10. We tested whether two practical starting schedules led to similar final FIDs on FFHQ and AFHQv2. On FFHQ, changing the Karras parameter from to changed the initial FID from 29.51 to 33.22, yet refinement reached 18.167 and 18.175, respectively. On AFHQv2, the initial FIDs were 15.65 and 20.22, and the final FIDs were 12.983 and 13.082. We also checked the local search direction at the standard start. The angle between the gradient of and the gradient of FID against real images was 7° on FFHQ and 10–23° on AFHQv2 (Appendix C.3, Table 12).
Across the 18 random starts, the diagnostic flagged all eight observed failures and none of the ten successful runs. Its scope is to assess a schedule found from an alternative start relative to the schedule obtained from the standard start. Accordingly, we have added the diagnostic and its results to Section 5.5 (Table 3). Appendices D.1 and D.2 report the initialization comparisons and results for individual runs (Tables 14 and 18–20).
Response to g7yQ: Stable Diffusion Preprocessing
Thank you for the careful analysis of our decomposition and experimental controls. Your two concerns require separate answers: correcting the SD evaluation protocol, and stating what the local analysis does and does not guarantee.
W1/Q1. Stable Diffusion preprocessing
We reported the input range mismatch in Appendix A of the submission. To determine its effect on the SD1.5 results, we performed both requested tests. When we kept the submitted schedules fixed and corrected the evaluation, the absolute FIDs increased, but the best surrogate at each NFE stayed the same. When we also repeated the search with the original budget, MR achieved the lowest FID among the three surrogates at all four NFEs.
COCO preprocessing. We computed the real reference statistics from all 40,504 COCO val2014 images. We resized the shorter side to 512 pixels using PIL bicubic interpolation and took a 512 × 512 center crop. In the submitted experiments, the feature extractor received values in [0,1] where it expected [−1,1]. This affected the real COCO images, the model reference images, and the generated images. We recomputed features for all three using the standard
pytorch-fidInceptionV3 pipeline. Images are rounded touint8before conversion to [0,1], and the extractor resizes them to 299 × 299 and maps them to [−1,1].Table R7. SD1.5 FID on the same 30,000 evaluation prompts and latents. “Fixed” evaluates the submitted schedules with corrected preprocessing. “New search” repeats the optimization with corrected preprocessing, using the same initialization, search samples, and search budget as the submission (one round of Nelder–Mead). Bold marks the lowest FID among the three surrogates. The final column gives the percentage gap between MR and direct FID.
Effect of correcting the evaluation. With the submitted schedules fixed, P retained the lowest FID among the three surrogates at 4 NFE, and MR retained the lowest FID at 5, 6, and 7 NFE. Each surrogate’s percentage gap to direct FID changed by less than 0.8 percentage points relative to the submission.
Effect of repeating the search. Using the same search budget as in the submission, MR achieved the lowest FID among the three surrogates at every NFE tested. At 4 NFE, it reached 16.59, compared with 16.78 for P; this is a 32.1% reduction in FID relative to the default schedule, compared with 30.2% in the submission. The gap between MR and direct FID narrowed from 2.10% to 0.11% at 5 NFE and widened from 3.78% to 5.96% at 6 NFE.
Accordingly, we have documented the corrected preprocessing in Appendix A and updated the SD1.5 results in Table 1. The revised Table 1 reports results after three search rounds, for which MR also achieved the lowest surrogate FID at every NFE tested. Appendix E.2, Table 23 reports both the one-round and three-round results. The eight pixel-space settings used a separate preprocessing pipeline and were unaffected by the SD1.5 correction.