acceptodds
Under review as a conference paper at ICLR 2027

What Should Few-Step Diffusion Schedules Optimize? Feature-Aligned Timestep Refinement Beyond Numerical Error

Abstract

Few-step diffusion sampling is highly sensitive to timestep placement, yet existing scheduling objectives are typically derived from numerical or trajectory-level accuracy, while generation quality is evaluated in a nonlinear feature space. We show that this mismatch produces markedly different optimization geometry in the extreme few-step regime: schedules with similar state-space error can have very different Fr\'echet Inception Distance (FID), whereas the feature-mean discrepancy to a high-accuracy reference closely follows the local FID landscape. We therefore use this reference feature-mean discrepancy, the mean component of the FID to the reference, as a local schedule-refinement objective. Rather than assuming global equivalence to data FID, we characterize when its gradient follows the data-mean component and when decreasing that component also decreases FID. We further show that the naive finite-sample plug-in estimator reintroduces a paired feature-dispersion term and derive an unbiased -statistic estimator. The resulting method requires neither model retraining nor real-data statistics during schedule search. Across eight pixel-space settings spanning three datasets, four solvers, and 4–16 function evaluations (NFE), our schedules lie – above best-found direct-FID solutions (median ), versus median gaps of for paired feature error and for state-space error. The same objective also transfers to LD3's richer two-sequence parameterization, improving its released schedule from FID to at 4 NFE and from to at 10 NFE.

Primary area:
generative models
Keywords:
diffusion models, few-step sampling, timestep scheduling, feature-aligned optimization, training-free sampling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_g7yQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks what timestep schedules in few-step diffusion sampling should actually optimize. The authors observe that existing scheduling objectives, discretization error, state- or trajectory-level error, and paired feature error, all derive from the sampling dynamics, whereas generation quality is measured by FID in Inception feature space. In the extreme few-step regime of 4–16 NFE, the two do not share the same optimization geometry.

Critical strengths
  1. The paper's decomposition of Q, together with its analysis of the finite-sample bias, is excellent.

  2. The main experiment compares different objectives under the same parameter settings, initialization, and search budget at the same time point, thus effectively isolating the influence of the optimization objective itself.

  3. The authors decompose the per-sample feature error into "the offset of the overall feature mean" and "the error fluctuation between samples," and optimize only the former. This design is based on the fact that closer images are to the reference result does not automatically mean that the FID of the entire batch of generated images is lower.

Critical weaknesses

The Stable Diffusion experiments used an input range for feature extraction that does not match the expected one, which affects the interpretation of these results with respect to standard FID. This setting is used both for the feature optimization objective and for the FID evaluation in this part.

The reliability of the method depends on a suitable initial timestep, but the optimization objective alone cannot determine whether the search has entered an unreliable region. The paper's local analysis depends on the directional relationship between the reference objective and the real-data objective, and these conditions involve the feature statistics of real data.

Questions for authors
  1. Please confirm what preprocessing was used for the real COCO reference statistics in Stable Diffusion. After extracting features for the generated images, the model reference images, and the real images all through a correct and consistent pipeline, re-evaluate the existing timesteps and re-run the objective optimization. Do the method ranking and the gaps in Table 1 remain?

  2. Apart from the empirical behavior of the standard initialization, does the current method include any constraint or diagnostic that does not require real-data statistics, in order to identify the cases in Table 15 where the objective value is lower but the FID is worse? If not, beyond CIFAR-10, which existing experiments can support that the standard initialization still keeps the search in the reliable region?

2 comments
AuthorsOpenReview ↗

Response to g7yQ: Stable Diffusion Preprocessing

Thank you for the careful analysis of our decomposition and experimental controls. Your two concerns require separate answers: correcting the SD evaluation protocol, and stating what the local analysis does and does not guarantee.

W1/Q1. Stable Diffusion preprocessing

We reported the input range mismatch in Appendix A of the submission. To determine its effect on the SD1.5 results, we performed both requested tests. When we kept the submitted schedules fixed and corrected the evaluation, the absolute FIDs increased, but the best surrogate at each NFE stayed the same. When we also repeated the search with the original budget, MRM_R achieved the lowest FID among the three surrogates at all four NFEs.

COCO preprocessing. We computed the real reference statistics from all 40,504 COCO val2014 images. We resized the shorter side to 512 pixels using PIL bicubic interpolation and took a 512 × 512 center crop. In the submitted experiments, the feature extractor received values in [0,1][0,1] where it expected [−1,1][-1,1]. This affected the real COCO images, the model reference images, and the generated images. We recomputed features for all three using the standard pytorch-fid InceptionV3 pipeline. Images are rounded to uint8 before conversion to [0,1][0,1], and the extractor resizes them to 299 × 299 and maps them to [−1,1][-1,1].

Table R7. SD1.5 FID on the same 30,000 evaluation prompts and latents. “Fixed” evaluates the submitted schedules with corrected preprocessing. “New search” repeats the optimization with corrected preprocessing, using the same initialization, search samples, and search budget as the submission (one round of Nelder–Mead). Bold marks the lowest FID among the three surrogates. The final column gives the percentage gap between MRM_R and direct FID.

NFEProtocolDefaultDirect FIDMRM_RQQPPMRM_R gap
4Submitted23.0916.4016.1217.2615.91−1.72%
4Fixed24.4217.2216.8118.1016.72−2.38%
4New search24.4216.8916.5917.4416.78−1.77%
5Submitted15.4412.2112.4713.8012.69+2.10%
5Fixed16.3212.9213.2214.6513.42+2.34%
5New search16.3212.9312.9413.9113.22+0.11%
6Submitted13.0611.3611.7912.8412.32+3.78%
6Fixed13.8511.9212.3813.5013.00+3.87%
6New search13.8511.6712.3613.4913.03+5.96%
7Submitted12.1610.7811.1312.8612.44+3.21%
7Fixed12.8211.3111.6413.4512.97+2.90%
7New search12.8211.2811.7412.9812.95+4.13%

Effect of correcting the evaluation. With the submitted schedules fixed, PP retained the lowest FID among the three surrogates at 4 NFE, and MRM_R retained the lowest FID at 5, 6, and 7 NFE. Each surrogate’s percentage gap to direct FID changed by less than 0.8 percentage points relative to the submission.

Effect of repeating the search. Using the same search budget as in the submission, MRM_R achieved the lowest FID among the three surrogates at every NFE tested. At 4 NFE, it reached 16.59, compared with 16.78 for PP; this is a 32.1% reduction in FID relative to the default schedule, compared with 30.2% in the submission. The gap between MRM_R and direct FID narrowed from 2.10% to 0.11% at 5 NFE and widened from 3.78% to 5.96% at 6 NFE.

Accordingly, we have documented the corrected preprocessing in Appendix A and updated the SD1.5 results in Table 1. The revised Table 1 reports results after three search rounds, for which MRM_R also achieved the lowest surrogate FID at every NFE tested. Appendix E.2, Table 23 reports both the one-round and three-round results. The eight pixel-space settings used a separate preprocessing pipeline and were unaffected by the SD1.5 correction.

AuthorsOpenReview ↗

Response to g7yQ: Detecting Unreliable Searches

W2/Q2. Detecting unreliable searches

The submitted method used MRM_R alone to select a schedule. In the CIFAR-10 example from Table 15 of the submission (Table 18 in the revision), the schedule found from random seed 21 had a lower MRM_R on the 2,000 search samples than the schedule found from the standard Karras start (6.249 vs. 6.264), yet its FID against real images was worse (20.82 vs. 16.93). We have added a diagnostic that flags this case after the search. Computing the diagnostic, setting its threshold, and deciding whether to flag a schedule all use only samples from the model.

How the diagnostic works. For a candidate schedule SS, we compute FID(S,R)\mathrm{FID}(S,R) on 10,000 samples. Both sets of images come from the same model. One set is generated with SS, and the other with a reference sampler RR that uses many more steps. We compute the same distance for the schedule obtained by optimizing MRM_R from the standard Karras start and use that value as the baseline. We estimate sampling variability by splitting the model samples into two disjoint halves and computing the distance on each half. We flag a candidate when the percentage increase over the baseline exceeds the threshold derived from this variability.

On CIFAR-10, the diagnostic flagged all four failed runs (Table R8), including the run with initialization seed 21. For this run, FID(S,R)\mathrm{FID}(S,R) exceeded the baseline by 26.4%, well above the 2.8% threshold. Most of this increase came from the covariance term, which explains why the diagnostic flagged this schedule even though its MRM_R on the search samples was slightly lower.

Table R8. Diagnostic results for six random starts per dataset using DPM-Solver++ at 4 NFE. To evaluate the diagnostic, we used FID against real images to determine which runs succeeded. A run counted as successful if its FID on 50,000 generated samples was within 1.6% of the FID obtained from the standard Karras start.

DatasetFlag thresholdFailed runs flaggedSuccessful runs flagged
CIFAR-102.8%4/40/2
FFHQ3.3%3/30/3
AFHQv24.1%1/10/5

Evidence beyond CIFAR-10. We tested whether two practical starting schedules led to similar final FIDs on FFHQ and AFHQv2. On FFHQ, changing the Karras parameter from ρ=7\rho=7 to ρ=11\rho=11 changed the initial FID from 29.51 to 33.22, yet refinement reached 18.167 and 18.175, respectively. On AFHQv2, the initial FIDs were 15.65 and 20.22, and the final FIDs were 12.983 and 13.082. We also checked the local search direction at the standard start. The angle between the gradient of MRM_R and the gradient of FID against real images was 7° on FFHQ and 10–23° on AFHQv2 (Appendix C.3, Table 12).

Across the 18 random starts, the diagnostic flagged all eight observed failures and none of the ten successful runs. Its scope is to assess a schedule found from an alternative start relative to the schedule obtained from the standard start. Accordingly, we have added the diagnostic and its results to Section 5.5 (Table 3). Appendices D.1 and D.2 report the initialization comparisons and results for individual runs (Tables 14 and 18–20).

Reviewer_LFanOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method to optimize sampling distribution of timesteps via optimizing an unbiased estimator of square of expected feature mean. Across datasets, solvers, and sampling budgets, the resulting training-free objective acts as an effective local refinement of standard schedules.

Critical strengths
  1. Acrossg CIFAR-10, FFHQ, and AFHQv2, DPMSolver++ (2M), Euler, Heun, and iPNDM, and different sampling budgets, the resulting training-free objective acts as an effective local refinement of standard schedules. The experiment results shows that the unbiased estimator of square of expected feature mean is a good proxy for FID.
Critical weaknesses
  1. The writing and presentation of this paper is orderless and hard to follow. The authors spend three pages and nearly fifteen equations on things related to FID calculation and the final unbiased estimator of square of expected feature mean. But the logic of writing requires substantial effort for understanding. Also, many notations in Figure 1 are not defined and clarified.I would lean toward a weak accept if the authors improved the paper’s organization and clarified the logic of the presentation.

  2. The experiments are not sufficient. Most analysis focus on 4 steps sampling on CIFAR-10. Lack of more modern text-to-image and text-to-video results.

Questions for authors

I have no other questions, please see the weakness.

4 comments
AuthorsOpenReview ↗

Response to LFan: Presentation Issues

Thank you for highlighting our finding that the squared difference in mean features between samples from a few-step sampler and those from a more accurate reference sampler is a useful proxy for FID when refining diffusion schedules. We agree that the original presentation may make it difficult to follow how this observation leads to our method. We have rewritten Section 4 and redrawn Figure 1 to explain why we match feature means and how we estimate their difference from a finite sample. We have also extended the analysis beyond CIFAR-10 at 4 NFE and compared all four objectives on SD3.5 Medium.

W1 Presentation Issues

We agree that the original Section 4 introduced too many equations before explaining how we choose a schedule. The revised section starts by explaining how we refine the interior timesteps for a fixed model, solver, and number of denoiser evaluations.

How we find a schedule. We draw KK independent noise inputs. For each input, we run the same model with many steps and save the Inception features of the resulting reference image. We start from the model’s default schedule and use Nelder-Mead to adjust the positions of its interior timesteps. Our parameterization keeps the timesteps in order and the endpoints fixed. Each time the optimizer tries a candidate schedule, we run it on the same KK noise inputs and score the resulting images against the cached reference features using our estimate of MRM_R. We return the schedule with the lowest score found during the search.

Why the estimator matters. For each noise input, let Δφ\Delta\varphi be the candidate image’s feature vector minus the cached reference feature vector. The paired error Q=E∥Δφ∥2Q=\mathbb{E}\|\Delta\varphi\|^2 measures the squared feature distance for each image pair and then averages over inputs. Our objective MR=∥E[Δφ]∥2M_R=\|\mathbb{E}[\Delta\varphi]\|^2 measures the squared distance between the two population feature means. For example, if two inputs produce feature differences in opposite directions with equal magnitude, those differences cancel in the mean. Both still contribute to QQ.

In practice, we have only KK inputs. A natural estimate of MRM_R is the squared distance between the two sample feature means, ∥Δφ‾∥2\|\overline{\Delta\varphi}\|^2, where Δφ‾\overline{\Delta\varphi} is the average feature difference over those inputs. Eq. 9 shows that this estimate partly measures QQ:

= \left(1-\frac{1}{K}\right)M_R + \frac{1}{K}Q.$$ This matters when comparing schedules. If two schedules have the same $M_R$, the one with smaller $Q$ receives a lower score on average. To remove this bias, Eq. 10 averages dot products between feature differences from distinct noise inputs. Because the inputs are independent, the expected dot product is exactly $M_R$ for any fixed schedule. The optimizer uses this unbiased estimate to compare candidate schedules. **How this relates to FID.** Our objective measures the squared distance between the candidate and reference feature means. Final FID compares generated images with real images and also depends on feature covariances. Section 4.4 derives a sufficient condition under which the negative gradient of $M_R$ with respect to the timesteps is also a local descent direction for FID. This condition accounts for the difference between the reference and real data means and for how the covariance term changes as the timesteps move. Accordingly, Section 4 now presents the objective (4.1), its estimator (4.2), and the complete search procedure in Algorithm 1 (4.3) before turning to the local justification (4.4). We also moved intermediate definitions into the text and removed the table of FID components. **Figure 1.** We redrew the figure and revised its caption to explain the notation and how to read the plots. Each coordinate $\lambda_i=-\log\sigma_i$ specifies a timestep in terms of its noise level $\sigma_i$. For a candidate image and a reference image generated from the same initial noise, $\Delta\varphi$ is the difference between their Inception feature vectors. In this example, $P$ measures the pixel error between paired images, $Q$ measures their mean squared feature distance, and $M_R$ measures the squared distance between the two population feature means. The upper panels vary $\lambda_1$ and $\lambda_2$ while holding $\lambda_3$ at the value found by direct FID optimization. The star marks the schedule with the lowest FID on this slice, and the circle marks the schedule with the lowest value of the objective shown in that panel. For $M_R$, the two markers coincide. As $\lambda_3$ varies in the lower panel, $M_R$ closely follows the FID curve, while $P$ and $Q$ change little over the same range.
AuthorsOpenReview ↗

Response to LFan: Experimental Coverage and Newer Generative Models

W2 Experimental coverage and newer generative models

Beyond CIFAR-10 at four steps. We agree that the original analysis of why MRM_R tracks FID focused mainly on CIFAR-10 at 4 NFE. We extended the gradient analysis to the eight pixel-space settings in Table 1. These cover four solvers on CIFAR-10 at budgets ranging from 4 to 16 NFE, as well as DPM-Solver++ on FFHQ and AFHQv2 at 4 NFE. At the standard starting schedule, MRM_R had the smallest measured angle to the FID gradient among the three surrogates in all eight settings (Appendix C.3, Table 12). On FFHQ, for example, the angle was 7° for MRM_R, 11–15° for QQ, and 28–33° for PP. The gradient estimates were reliable in the six settings with four solver steps, including Heun at 8 NFE. At 10 and 16 NFE, estimates of the FID gradient from two disjoint sets of samples differed by about 10° and 15°, respectively. We therefore base our comparison at these two budgets on the final FIDs in Table 1. At 10 NFE, optimizing MRM_R gave a FID of 2.672 versus 2.656 for direct FID optimization. At 16 NFE, the corresponding FIDs were 2.106 and 2.073.

SD3.5 Medium at 4 NFE. We compared all four objectives on SD3.5 Medium, a text-to-image model trained with rectified flow. Table R6 reports the results on 30,000 COCO prompts held out from the search.

Table R6. SD3.5 Medium at 4 NFE, evaluated on 30,000 COCO prompts. Default is the model’s standard schedule. Each remaining column reports results for the schedule found by optimizing the indicated objective under the same search budget. We kept each schedule fixed and recomputed FID, CLIP, and HPSv2.1 with a second evaluation seed.

MetricDefaultDirect FIDMRM_RQQPP
FID@30k (seed 1) ↓71.5522.4823.0722.9450.79
FID@30k (seed 2) ↓71.6322.6623.1823.1050.95
CLIP ×100 (seed 1) ↑22.6726.5826.5426.6325.56
CLIP ×100 (seed 2) ↑22.6826.5926.5426.6325.55
HPSv2.1 ×100 (seed 1) ↑15.3921.8921.8021.9517.89
HPSv2.1 ×100 (seed 2) ↑15.3921.9021.8021.9517.86

On both evaluation seeds, MRM_R and QQ reached FIDs within 3% of direct FID optimization, with QQ slightly ahead of MRM_R (by less than 0.13 FID), while PP remained about 125% above it. Both also improved text alignment (CLIP) and predicted human preference (HPSv2.1) over the default schedule. Whether these findings extend to text-to-video models remains open.

Accordingly, we have expanded the gradient analysis in Section 5.2 and report the results for all eight settings in Appendix C.3 (Table 12). We have also added SD3.5 Medium to Table 1, with the full evaluation in Appendix E.3 (Table 26).

I thank the authors for the detailed response and the revised manuscript. The restructured draft makes the logic easier to follow. And the new experiments on text to image addressed my concerns. I have raised my score to 3.

Thank you for considering our responses and the revised manuscript. We appreciate your constructive feedback.

Reviewer_zUcDOpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a feature-mean discrepancy objective for refining few-step diffusion schedules. It aligns candidate samples with a high-accuracy reference in Inception feature space and introduces an unbiased finite-sample estimator. Experiments show consistent improvements across several datasets, solvers, and sampling budgets.

Critical strengths

The paper identifies a meaningful mismatch between numerical error and FID-oriented generation quality.

The proposed objective is simple, training-free, and supported by careful theoretical and empirical analysis.

The experiments cover multiple datasets, solvers, sampling budgets, and schedule parameterizations.

Critical weaknesses

The method is expensive, requiring thousands of samples and many objective evaluations for each setting.

Its justification is local and depends strongly on initialization from an already reasonable schedule.

The evaluation focuses primarily on FID and provides limited evidence for perceptual quality, semantic alignment, or broader diffusion models.

Questions for authors

See weakness

3 comments
AuthorsOpenReview ↗

Response to zUcD: Computational Cost and Initialization

Thank you for highlighting the simplicity of our approach to refining diffusion timesteps by matching the mean features of samples from a few-step sampler to those from the same model run with many more steps. We also appreciate your positive assessment of our theoretical and empirical analysis of how this objective relates to FID. To address your concerns, we report how the search budget and initialization affect the resulting schedules, and we evaluate text alignment and perceptual quality on SD1.5 and SD3.5 Medium using CLIP and HPSv2.1.

W1 Computational cost

We optimize the schedule once offline for each combination of model, solver, and NFE. The resulting schedule can be reused without additional denoiser evaluations during generation. To test how much search is needed, we reduced both the number of samples per evaluation and the number of objective evaluations. Using one sixth of the full search budget changed the final FID by less than 1% on each of the three datasets (CIFAR-10, FFHQ, and AFHQv2).

Table R1 reports FID@50k and measured search time on one RTX 3090. All searches optimized MRM_R from the same Karras initialization for each dataset using DPM-Solver++ at 4 NFE. The budget fractions are based on the total number of samples generated across objective evaluations.

Table R1. Search time and final FID under three search budgets.

Search budgetSamples per evaluationObjective evaluationsCIFAR-10 FID (time)FFHQ FID (time)AFHQv2 FID (time)
Full2,000543 (3 rounds)16.927 (162 min)18.167 (340 min)12.983 (333 min)
1/61,000181 (1 round)16.765 (27 min)18.326 (54 min)13.064 (55 min)
1/12500181 (1 round)17.649 (14 min)18.097 (28 min)13.090 (27 min)

At one twelfth of the full budget, FID was 4.3% higher on CIFAR-10 and differed by less than 1% on FFHQ and AFHQv2, relative to the full search on each dataset. The reported times exclude reference generation and final FID evaluation. For SD1.5, three rounds of MRM_R search took 20.6 to 40.8 hours on one H100 across 4 to 7 NFE. For SD3.5 Medium at 4 NFE, one round took 21.6 hours on four H200 GPUs (86.4 GPU hours). Generating the 2,000 SD3.5 reference samples took a further 27 minutes on the same four GPUs (1.8 GPU hours), and all four objectives shared those samples.

Accordingly, we have added a discussion of search cost to Section 5.1. Appendix A reports the measured costs for SD1.5 and SD3.5 in Table 5 and the full budget comparison in Table 6.

W2 Local validity and initialization

We agree that the justification is local. In practice, we start from the model’s standard schedule, which is available without evaluating FID on real data. We tested whether the choice among several practical starting schedules materially affects the result. Across all three datasets, the final FIDs differed by less than 1% across the tested initializations, even when their starting FIDs differed substantially.

Table R2. FID before and after optimizing MRM_R from different starting schedules with DPM-Solver++ at 4 NFE.

DatasetStarting scheduleInitial FIDFinal FID
CIFAR-10Standard Karras36.0616.927
CIFAR-10Clamp, rmax⁡=3r_{\max}=320.1716.954
CIFAR-10Clamp, rmax⁡=2r_{\max}=227.8516.926
FFHQStandard Karras, ρ=7\rho=729.5118.167
FFHQKarras, ρ=11\rho=1133.2218.175
AFHQv2Standard Karras, ρ=7\rho=715.6512.983
AFHQv2Karras, ρ=11\rho=1120.2213.082

On CIFAR-10, we selected the clamp setting rmax⁡=3r_{\max}=3 using initial FID. Although this gave a much lower starting FID than the standard Karras schedule (20.17 vs. 36.06), the two searches reached nearly identical final FIDs (16.954 vs. 16.927). Selecting the starting schedule using real data therefore offered no advantage in this comparison.

With random initialization (six random starts per dataset, DPM-Solver++ at 4 NFE), 10 of 18 runs reached a final FID within 1.6% of the result obtained from the standard Karras start. Accordingly, we limit our claim to refinement from practical starting schedules. We further evaluated a post-search diagnostic using FID(S,R), which requires no real-data statistics. Using 10,000 samples, we flagged schedules whose FID(S,R) exceeded that of the optimized Karras solution by more than 3√2·s (2.8–4.1%), a threshold derived from the split-half variability s of FID(S,R). In these 18 random-start trials, this detected all eight unsuccessful runs without flagging any of the ten successful runs.

Accordingly, Section 5.5 and Appendices D.1 and D.2 report the initialization experiments and the diagnostic.

AuthorsOpenReview ↗

Response to zUcD: Perceptual Quality and Model Generalization

W3 Perceptual quality, semantic alignment, and model coverage

Text alignment and perceptual quality. We evaluated the SD1.5 schedules on 30,000 COCO prompts using CLIP for text alignment and HPSv2.1 as a learned proxy for human preference. We reran the SD1.5 searches and recomputed FID using the corrected Inception preprocessing for generated images, model reference images, and real COCO images (see our response to Reviewer g7yQ, W1/Q1).

Table R3. SD1.5 results with the default schedule and the schedule obtained by optimizing MRM_R with three search rounds. Each entry shows default →\rightarrow optimized.

NFEFID@30k ↓CLIP ×100 ↑HPSv2.1 ×100 ↑
424.42 → 16.5925.65 → 25.9021.78 → 22.40
516.32 → 12.7326.21 → 26.3423.78 → 24.22
613.85 → 12.3726.46 → 26.5424.82 → 25.25
712.82 → 11.7526.58 → 26.5425.37 → 25.38

On SD1.5, optimizing MRM_R lowered FID at every NFE tested. At each NFE, CLIP scores varied by less than 0.5 points across all schedules (Table 24), and the change from the default schedule to the MRM_R schedule ranged from −0.04 to +0.25 points. HPSv2.1 increased by 0.62, 0.44, and 0.43 points at 4, 5, and 6 NFE, respectively, and was nearly unchanged at 7 NFE (25.37 to 25.38). Thus, the refined schedules maintained comparable text alignment across all four budgets and improved predicted human preference at 4 to 6 NFE.

Accordingly, we have expanded the SD1.5 discussion in Section 5.3. Table 1 now reports the corrected SD1.5 FID results, and Appendix E.2 reports the new CLIP and HPSv2.1 evaluations for all schedules in Tables 24 and 25, respectively.

Transfer to SD3.5 Medium. We also tested all four objectives on SD3.5 Medium at 4 NFE, using the same starting schedule and search budget. On both evaluation seeds, the schedules found with MRM_R and QQ achieved FIDs within 3% of direct FID optimization. The FIDs of the two schedules differed by less than 0.13 on each seed. On both seeds, optimizing MRM_R also raised CLIP from about 22.7 to 26.54 and HPSv2.1 from 15.39 to 21.80 (Table R4).

Table R4. SD3.5 Medium at 4 NFE. Each column identifies the objective used to optimize the schedule. We kept each schedule fixed and recomputed FID, CLIP, and HPSv2.1 with a second evaluation seed.

MetricDefaultDirect FIDMRM_RPaired feature error QQLatent state error PP
FID@30k (seed 1) ↓71.5522.4823.0722.9450.79
FID@30k (seed 2) ↓71.6322.6623.1823.1050.95
CLIP ×100 (seed 1) ↑22.6726.5826.5426.6325.56
CLIP ×100 (seed 2) ↑22.6826.5926.5426.6325.55
HPSv2.1 ×100 (seed 1) ↑15.3921.8921.8021.9517.89
HPSv2.1 ×100 (seed 2) ↑15.3921.9021.8021.9517.86
AuthorsOpenReview ↗

Response to zUcD: Evaluation with DINOv2 Features

Evaluation with DINOv2 features. We first evaluated the original schedules optimized with MRM_R in Inception feature space. Using DINOv2 features for evaluation, these schedules achieved a lower Fréchet distance than the default schedule in seven of eight settings. The exception was CIFAR-10 with Heun at 8 NFE. We then repeated the searches for MRM_R, QQ, and direct Fréchet distance optimization using DINOv2 features. Since PP is computed in the sampler state, we evaluated its existing schedules. In this comparison, MRM_R achieved a lower DINOv2 Fréchet distance than both QQ and PP in all eight settings.

Table R5. Fréchet distance in DINOv2 feature space on 50,000 generated images (lower is better). The two MRM_R columns differ only in the features used during search. Bold marks the lowest distance among the three surrogates in the final three columns.

DatasetSolverNFEDefaultMRM_R (Inception)Direct DINOv2 FDMRM_R (DINOv2)QQ (DINOv2)PP
CIFAR-10DPM-Solver++4559.4397.9393.4396.6456.6434.6
CIFAR-10Euler4766.2574.3570.0569.8647.3926.0
CIFAR-10Heun8585.8637.4503.3510.0532.5543.6
CIFAR-10iPNDM4529.9362.8359.4368.2414.2544.7
CIFAR-10iPNDM10200.4180.4169.1180.7182.9205.1
CIFAR-10DPM-Solver++16181.4166.0156.6166.1172.5179.8
FFHQDPM-Solver++4461.1406.5373.3401.2414.7401.3
AFHQv2DPM-Solver++4388.1356.6356.1357.6463.9374.2

Accordingly, we have added this comparison to Section 5.3. Appendix E.6 reports the full results under both DINOv2 and Inception evaluation (Tables 29 and 30).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.