acceptodds
Under review as a conference paper at ICLR 2027

ENCORE: Training-Free Few-Step Diffusion Sampling with Terminal Moment Matching

Abstract

Diffusion models achieve state-of-the-art quality in visual and audio generation, but sampling requires many evaluations of an expensive denoising network, and fewer steps severely degrade sample quality. Existing approaches either design fast ODE solvers or rely on auxiliary trained networks for post-hoc refinement. We show that even with optimized ODE solvers, few-step generation deviates systematically from many-step runs in two distributional ways: (i) a loss of power across all frequency bands and (ii) a mean shift in a feature space. Because both are distribution-level properties, they cannot be precisely measured or corrected along a single sampling trajectory: the large per-sample variance masks the overall mean shift and distorts frequency measurements. To address this issue, we propose ENCORE, a training-free, post-hoc correction that reuses the differences between denoising outputs at consecutive steps, which the sampler otherwise discards, and thus requires no additional network evaluation. In a spectral-then-mean (S2M) pipeline, ENCORE first restores the per-band power via magnitude scaling and then realigns the sample centroid in a pre-trained feature space by ridge regression over these differences. The two stages match a second moment per frequency band and a first moment in feature space, after the last step. Because it operates after sampling, ENCORE is plug-and-play and yields further gains on top of sampler-level optimizations. Evaluations across pixel-space, image latent, and audio latent diffusion models demonstrate that ENCORE improves sample quality with negligible overhead.

Primary area:
generative models
Keywords:
diffusion models, few-step sampling, training-free, post-hoc correction, moment matching, diffusion ODE solvers

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_L8RWOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper introduces ENCORE to improve the quality of generated samples. The method is built on two systematic failures of few-step sampling: power loss in frequency bands and mean shift in feature space. It uses samples generated with more steps as reference and performs two stages. In the first stage (Stage S), the average band powers of the reference and generated sets are used to compute a per-band scaling factor. In the second stage (Stage M), given the last K differences between consecutive denoising outputs, transformed by Stage S, a least-squares regression is fitted so that the feature mean of the corrected samples matches that of the reference set. The method is tested on pixel-space, image-latent, and audio-latent diffusion models and demonstrates improvements across settings.

Critical strengths
  • Experiments across different models show consistent improvement. The ablation study shows the importance of both components.
  • The method is based on statistics from the sets and a closed-form least-squares formulation. The computational overhead is small.
Critical weaknesses
  • For image generation, matching the mean Inception features to the mean features of the reference set might directly improve FID, since FID uses the same features. The result would be more convincing if other metrics rather than just FID also improve.
  • The linear-displacement assumption is not clearly explained and formalized. It’s unclear how large the approximation error can be. I see the way aKa_K is optimized is to propose N−1N-1 good enough candidates, then select the best based on J(K)J(K) - the quantity we actually want to minimize. I wonder what the performance would be if a were optimized by directly minimizing this objective.
Questions for authors
  • In Figure 2a, the band power after correction is above 100%. Does this mean that as long as you increase the band power, it’s alright? Is the band power of xNcorrectedx_N^{corrected} the same as x~\tilde x? If not, performing stage M also hurts stage S.
  • Optional: Provide other metrics for image generation and what if aKa_K is optimally trained without the need for the linear-displacement assumption.
7 comments
AuthorsOpenReview ↗

Response to Reviewer L8RW (1/2)

We thank the reviewer for the careful and constructive review. We answer the two questions first and then the remaining point of the weaknesses. The revised PDF has the changes in blue, with the new tables in Appendix C and the human study in Section 5.3.

Q1. Band power above 100% in Figure 2(a)

The band power of the corrected output is not the same as that after Stage S. Stage S matches the band power of the sample to the target, and Stage M then adds the rescaled differences, which changes the band power again. Appendix C.9 reports the band power after each stage (Table 19). After Stage S alone, the 16 band powers lie between 0.90 and 1.08 of the target in all 64 settings. After both stages, they lie between 0.73 and 4.12, which is the overshoot in Figure 2(a).

Stage M therefore does move the band power away from the target that Stage S matched. Matching the band power is the goal of Stage S, but not of the correction as a whole: Stage M fits the feature mean, and the band power of its output can exceed the target. Increasing the band power is not the goal, and keeping it matched does not help either. S2M performs better than Stage S alone in 61 of the 64 settings (Table 10), and matching the band power again after Stage M makes FID worse. In a separate run on CIFAR-10 with Euler (50,000 samples), applying Stage S once more after S2M gives a higher FID at every NFE:

FID, CIFAR-10 EulerNFE 4NFE 6NFE 8NFE 10
Baseline67.3235.8722.4915.85
S only40.7322.8414.8910.92
S2M (ENCORE)24.5813.028.556.47
S2M2S (band power matched again after M)26.9116.5610.758.16

Q2. Other metrics, and optimizing aka_k without the linear-displacement assumption

Stage M fits the Inception mean, which is one of the two terms of FID, so we first split FID into its mean term and its covariance term (Appendix C.12, Table 23). Over the 64 pixel-space settings, the covariance term is lower than the baseline in 57 settings, although Stage M does not fit it, and in those settings its decrease accounts for 47% of the FID improvement on average. As the reviewer suggested, we also reported metrics other than FID on the same 64 settings. ENCORE achieves a lower KID than the baseline in 60 settings and a lower FD-DINOv2 in 46 settings (Table 24). It also achieves a higher precision in 51 settings and a higher recall in 56 settings. Recall is lower in 6 settings, five of them at NFE 10, by at most 0.02 (Table 25). On Stable Diffusion 1.4, KID, FD-DINOv2, precision, recall and the two human preference scores HPSv2.1 and ImageReward all improve at every NFE (Table 27). Section 5.3 also adds a blind user study on DiT-XL/2 with 32 participants, who chose ENCORE in 97.7%, 92.3% and 86.9% of the pairs at NFE 4, 6 and 8 (Table 4).

The closed form for aka_k relies on the linear-displacement assumption, so we also minimized the feature-mean discrepancy directly with Gauss–Newton, without this assumption and with the same K and band gains (Appendix C.10, Table 21). Compared with the closed form, directly minimizing the objective lowers FID in 30 of the 64 settings, leaves it unchanged in 14 and gives a higher FID in 20, with an average change of −0.13, so the closed form is close to the direct optimum in most settings.

AuthorsOpenReview ↗

Response to Reviewer L8RW (2/2)

Weakness 1. Metrics other than FID

Please refer to our reply to Q2.

Weakness 2 (remaining point). The linear-displacement assumption and its approximation error

We thank the reviewer for pointing this out. Section 4.2 now states the linear-displacement assumption as Equation 5, with its approximation error r(a)r(a):

E[φ(x~+∑kakd~k)]=E[φ(x~)]+∑kakΦˉk+r(a),Φˉk=E[φ(x~+d~k)−φ(x~)],\mathbb{E}\big[\varphi(\tilde x + \textstyle\sum_k a_k \tilde d_k)\big] = \mathbb{E}[\varphi(\tilde x)] + \sum_k a_k \bar\Phi_k + r(a), \qquad \bar\Phi_k = \mathbb{E}[\varphi(\tilde x + \tilde d_k) - \varphi(\tilde x)],

where φ\varphi is the feature map, x~\tilde x is the rescaled sample and d~k\tilde d_k are the rescaled directions. The assumption is that the feature-mean shift is linear in the coefficients, so the closed-form solve for aka_k sets r(a)r(a) to zero. To see how large the error is, Appendix C.10 measures r(a)r(a) on all 64 settings (Table 20). We compared the feature-mean shift predicted by Equation 5 with the shift measured after Stage M is applied on top of Stage S. The relative error, the norm of r(a)r(a) divided by the norm of the measured shift, lies between 0.05 and 0.81 with an average of 0.31. The cosine between the predicted and the measured shift is at least 0.90 in 59 of the 64 settings, and the lowest is 0.69. The effect of this error on FID is in our reply to Q2, where the closed form is compared with minimizing the discrepancy directly.

Thank you for the detailed responses and the additional experiments.

Q1, W1 Partially addressed, concern remains

Thanks for adding Appendix C.9 and Table 19. I agree that empirically, S2M is better than "M only" in most settings, except for FFHQ-64, so Stage S does help. However, S2M changes the band power again, which doesn’t match the stated contribution that the method “restores band power”. I see your goal is both to “restore band power” and “realign the feature centroid”, but to keep it simple, you optimize them in sequence, and you prefer M since it matters more ("M only" is better than "S only"). The S2M2S result also suggests that matching band power doesn’t matter in the end, so the role of S is unclear.

Why does S still matter even though, in the end, the objective S optimized has been changed?

Q2, W2 Addressed with additional questions

Thanks for the additional experiments and metrics. They addressed my concern that the gains might be limited to FID. On the linear-displacement assumption, the relative error r(a)r(a) lies between 0.05 and 0.81, with an average of 0.31, which is fairly large for an assumption of zero error. The high cosine shows the predicted shift has the right direction, but not necessarily the right magnitude. However, Table 21 shows the closed form performs close to direct minimization, so this error matters less in practice. Table 23 indeed shows that the direction dkd_k is important, possibly helping the linear assumption work.

Is it the case that dkd_k has a lower error r(a)r(a) than random vectors? If so, do you have any idea about the properties of dkd_k that would make this the case?

Reviewer_aZY7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ENCORE, a training-free post-hoc correction method for improving few-step diffusion sampling without additional denoiser evaluations. The authors identify two systematic discrepancies between few-step and many-step outputs: reduced spectral power and shifted feature-space means. ENCORE addresses these through frequency-band magnitude rescaling and ridge-regression-based correction using previously computed denoising predictions. Experiments across image and audio diffusion models demonstrate consistent improvements in generation quality.

Critical strengths
  1. Simple, cheap and plug-and-play. The correction is a closed-form, post-hoc step that requires no training, no real data and no additional network evaluations, and adds only about 1 to 2% to the cost of each sample. It composes with existing solvers and accelerated samplers such as PAS, GITS, LA-DPM, RX-DPM and STORK.

  2. Broad and consistent empirical gains. Improvements hold across four pixel-space datasets, four solvers, two image latent models and one audio latent model. Gains are largest at very low NFE (for example, 47.2 to 16.1 FID for DiT-XL/2 at 4 NFE).

Critical weaknesses
  1. The optimization objective overlaps with the evaluation metric. Stage M explicitly aligns the Inception feature mean, which is one term of the reported Inception FID. Part of the large gains may therefore reflect optimizing the metric rather than improving sample quality. Solving in DINOv2 and evaluating audio in other feature spaces partly addresses this, but FID decomposed into its mean and covariance terms, FD-DINOv2 and evaluation in other feature spaces are needed. For audio, the CLAP score is reported even though CLAP is the space in which the coefficients are solved.

  2. Effects on diversity and fidelity are untested. A global extrapolation combined with band amplification may sharpen samples at the cost of recall or prompt alignment. The paper reports no precision and recall, no CLIP score for Stable Diffusion 1.4 and no perceptual study.

Questions for authors

For Stable Diffusion 1.4, does CLIP score or a human preference metric improve along with FID? How do you explain the FFHQ results, where Stage S often hurts and the M2S order outperforms the deployed S2M order?

2 comments
AuthorsOpenReview ↗

Response to Reviewer aZY7

We thank the reviewer for the careful and constructive review. We answer the two questions first and then the remaining points of the weaknesses. The revised PDF has the changes in blue, with the new tables in Appendix C and the human study in Section 5.3.

Q1. CLIP score and human preference on Stable Diffusion 1.4

We added the CLIP score and two human preference metrics (HPSv2.1 and ImageReward) for Stable Diffusion 1.4, now calibrated on captions disjoint from the evaluation captions (Appendix C.12, Table 27). Both human preference metrics improve at every NFE together with FID. The CLIP score also improves at every NFE, but only by at most 0.09, so text alignment is maintained rather than improved.

Q2. The FFHQ-64 results

On FFHQ-64, Stage S alone raises FID in 3 of the 16 settings, the reverse order M2S reaches a lower FID than the deployed order S2M in 12 of the 16 settings, and Stage M alone reaches a lower FID than S2M in 15 of the 16 settings (Table 10). To relate this to what Stage S corrects, we measured the band-power deficit of the baseline on the calibration pairs of every setting (Appendix C.15, Table 32). In 13 of the 16 settings, FFHQ-64 has the smallest deficit of the four datasets, and in 11 of the 16 settings the FID reduction from Stage S alone is also the smallest on FFHQ-64 (Table 10). This suggests that Stage S has little to correct on FFHQ-64, so that most of the gain comes from Stage M and the order of the two stages matters less there. Section 5.4 now states the FFHQ-64 exception and relates it to the deficit.

Weakness 1. The optimization objective overlaps with the evaluation metric

We agree that Stage M fits the Inception mean, which is one term of FID, and we added the checks the reviewer asked for (Appendix C.12). First, FID decomposed into its mean and covariance terms: over the 64 pixel-space settings, ENCORE achieves a lower covariance term than the baseline in 57 settings, although Stage M does not fit this term, and in those settings the decrease of the covariance term accounts for 47% of the FID improvement on average (Table 23). Second, evaluation in a feature space that the calibration never sees: ENCORE achieves a lower FD-DINOv2 than the baseline in 46 of the 64 settings (Table 24). Third, audio: we agree that the CLAP score in Table 3 is circular, since the coefficients are solved in CLAP features, so we solved them in VGGish features instead. The CLAP score still improves at every NFE (Table 17). Section 5.3 now notes that CLAP is also the space in which the coefficients are solved and refers to this result in Appendix C.7.

CLAP scoreNFE 4NFE 8NFE 16
Baseline0.1180.3290.397
+ ENCORE (coefficients solved in VGGish)0.1510.3780.425

Weakness 2 (remaining points). Precision and recall, and the perceptual study

On the 64 pixel-space settings, ENCORE achieves a higher precision than the baseline in 51 settings and a higher recall in 56 (Table 25). Recall is lower in 6 settings, five of them at NFE 10, by at most 0.02, so diversity increases rather than decreases in most settings. On Stable Diffusion 1.4, precision and recall both improve at every NFE, and the CLIP score is maintained as answered in Q1 (Table 27).

As the reviewer asked for a perceptual study, we ran a blind user study on DiT-XL/2 in which participants chose between two images (Section 5.3, Table 4). We drew 100 pairs at random for each of NFE 4, 6 and 8 before any rating. Each pair shows the baseline sample and the same sample with ENCORE in random order, and 32 participants were asked, for all 300 pairs, which image looks more realistic and has better visual quality. Participants chose ENCORE in 97.7%, 92.3% and 86.9% of the pairs at NFE 4, 6 and 8, so the improvement is visible to human raters and not only in FID.

AuthorsOpenReview ↗

Requesting feedback on our rebuttal

Dear Reviewer aZY7,

Thank you again for the time and effort you spent reviewing our paper. As the discussion period is nearing its end, we would be grateful to know whether our response and the revised PDF (changes in blue) have addressed your concerns. For each of the points you raised, we added measurements and revised the main text, and the results are summarized in our reply above. If any question or concern remains, we would be glad to answer it before the deadline.

Reviewer_gXkvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ENCORE is a post-hoc correction applied after the last step of a few-step sampler. Stage S rescales 16 radial Fourier bands with gains from the target-to-few-step power ratio. Stage M adds a combination of the last K denoiser differences, with coefficients from a ridge regression that matches the Inception (or CLAP) feature mean to a 64-step target. It is calibrated once per (model, solver, NFE) on 1,000 self-generated pairs, adds no NFEs and 0.3–2% wall-clock. FID drops in 63/64 pixel-space cells, and it stacks on PAS, GITS and others, and works on DiT-XL/2, SD 1.4 and AudioLDM 2.

Critical strengths
  • It is cheap and practical, adds no NFEs, and works across solvers, stacked methods and three model families (pixel, image-latent, audio-latent) (e.g., Euler 4-NFE CIFAR-10 FID 67.3 to 24.6).

  • The ablations are thorough (stages, order, target, feature space, K) and include failure cases.

  • The cost tables (11, 12) and the calibration-size study (Table 10) are honest and complete.

Critical weaknesses

Major weaknesses:

  1. For images, the headline metric is partly what Stage M optimizes, and no independent evaluation is given.

Stage M shifts the mean in Inception space and FID is measured there. The DINOv2 ablation is still scored with Inception FID. For audio, CLAP fits the coefficients and CLAP score is reported as a gain (Table 3), which is circular. We need FD-DINOv2, KID, precision/recall, CLIP or ImageReward for SD, and LPIPS/PSNR against the 64-step output to back “restores details”. SD gains are small (e.g., 16.55 to 14.61 at NFE 10)". The NFE 8 gain is larger (16.97→14.46) with no text-alignment evidence.

  1. Claims exceed the evidence.

The paper says it matches a second moment per band, but Fig. 2(a) shows overshoot after Stage M, and the 97.9–109.2% figure is not clearly the final output (it may be the Stage S state). Fitting a 2048-d mean with K ≤ 9 scalars leaves residual error that is never reported numerically (Fig. 2(b) shows only a 1-D projection). The linear-displacement assumption (Eq. 5) is never validated. Also, the method is close to lookahead extrapolation (LA-DPM) and related extrapolation methods (RX-DPM, PFDiff), and the paper does not isolate what differs, and 'orthogonal' is not demonstrated beyond stacking results (gains on GITS are small). There is no test with random or PCA directions in place of d_k, no fixed linear sharpening and extrapolation baseline, and no BFR comparison.

Other:

Calibration is per (model, solver, NFE) at a single fixed guidance scale, and transfer is untested. SD calibration prompts are unspecified, and overlap with the 30k eval captions is not ruled out. The rule-selected K can miss the oracle (e.g., SD NFE 6: +1.74 FID, Table 13). There are no seeds or CIs on FID, and many NFE 8–10 gains are tiny (e.g., AFHQ iPNDM(4M) NFE 8: 2.84 to 2.83).iPNDM(3M)+PAS on ImageNet-64 NFE 4 is worse than plain iPNDM(3M) (59.07 vs 31.43; PAS reports ImageNet-64 only from NFE 5), so that stacked result is not a real test.

Minor:

No flow-matching or large modern text-to-image model is tested. Typical a_k values and their stability are not reported. The name ENCORE is also used by arXiv:2610.02538 (replica-exchange control for diffusion generation, posted Oct 2026).

Questions for authors

Please:

  1. Report FD-DINOv2, KID, precision/recall, a CLIP-type score on SD, and LPIPS/PSNR vs the 64-step output. Do the gains hold when the evaluation space differs from the fitting space, and does diversity (recall) change?

  2. Give final band-power ratios and the residual feature-mean error before and after correction.

  3. Replace d_k with random or PCA directions, and add a fixed linear extrapolation or sharpening baseline.

  4. Test transfer across guidance scale, prompt set and NFE, and say how the SD calibration captions were chosen.

  5. Add FID seed variance or bootstrap CIs, and report typical a_k values.

5 comments
AuthorsOpenReview ↗

Response to Reviewer gXkv (1/3)

Thank you for the detailed review. We answer the five questions first and then the remaining points of the weaknesses. The revised PDF has the changes in blue, with the new tables in Appendix C and the human study in Section 5.3.

Q1. Metrics other than FID, and recall

We agree that FID alone is not enough, because Stage M fits the Inception mean that FID also measures. We therefore added the metrics the reviewer asked for on all 64 pixel-space settings (Appendix C.12). KID, which uses the same Inception features but a different statistic, is lower than the baseline in 60 settings. FD-DINOv2, computed in a network that the calibration never sees, is lower in 46 (Table 24). Precision is higher in 51 settings (Table 25). Recall does not fall: it is higher in 56 settings and lower in 6, five of them at NFE 10, by at most 0.02 (Table 25). Within FID itself, the covariance term, which Stage M does not fit, is lower in 57 settings (Table 23). For Stable Diffusion 1.4, KID, FD-DINOv2, precision and recall improve at every NFE. HPSv2.1 and ImageReward, which are derived from models trained on human preference data, also rise at every NFE. The CLIP score rises by at most 0.09, so text alignment is kept rather than improved (Table 27). We agree that the CLAP score in Table 3 is circular, since the coefficients are solved in CLAP features. We therefore solved them in VGGish features instead. The CLAP score still improves at every NFE (Table 17). Section 5.3 now notes that CLAP is also the space in which the coefficients are solved and refers to this result in Appendix C.7.

CLAP scoreNFE 4NFE 8NFE 16
Baseline0.1180.3290.397
+ ENCORE (coefficients solved in VGGish)0.1510.3780.425

On LPIPS and PSNR against the 64-step sample, ENCORE lowers LPIPS in 39 of the 64 settings and raises PSNR in 33 (Table 26). ENCORE corrects the statistics of the sample set, not each sample, so these per-sample distances change little. We have rephrased the caption of Figure 1 from "restores fine structural details lost by the baseline" to "shows finer details than the baseline".

Q2. Final band-power ratios and residual feature-mean error

The 97.9–109.2% range in Section 4.3 is the band power after Stage S, which the original manuscript did not state clearly. We have revised Sections 4, 4.1 and 4.3 to state that Stage S matches the band power and that Stage M then adds the rescaled differences, which changes the band power again. The band power and the residual feature-mean error, reported as the mean distance after each stage, are in Appendix C.9 (Table 19). After Stage S alone, the 16 band powers lie between 0.90 and 1.08 of the target in all 64 settings, and the residual is already lower than that of the baseline in 55 settings. After Stage M is applied on top of Stage S, the band powers lie between 0.73 and 4.12, so the final sample does not keep the band power that Stage S matched, and the residual is lower than that of the baseline in 63 of the 64 settings and equal in the remaining one.

Q3. Random and PCA directions, and fixed extrapolation and sharpening baselines

We ran the two tests the reviewer asked for. First, we replaced d_k by random directions of the same norm and solved the coefficients again. This makes FID worse than the baseline in 61 of the 64 settings (Appendix C.12, Table 23). A random direction of the same size does not reproduce the gain, so the directions d_k are needed. Second, we compared ENCORE with three fixed corrections on all four datasets at NFE 4 and 8 (Appendix C.16, Table 33): PCA directions in place of d_k, fixed extrapolation x_N + c d_1, and sharpening by unsharp masking. The extrapolation coefficient c and the sharpening strength are chosen on the calibration pairs. ENCORE achieves the lowest FID in 28 of the 32 settings, and fixed extrapolation in the remaining 4. Sharpening and PCA directions do not achieve a lower FID than ENCORE in any setting.

AuthorsOpenReview ↗

Response to Reviewer gXkv (2/3)

Q4. Transfer, and the SD calibration captions

We thank the reviewer for pointing out the overlap of the calibration captions. In the original manuscript, 731 of the 1,000 calibration captions belonged to images in the 30,000 evaluation captions. We recalibrated Stable Diffusion 1.4 on 1,000 MS-COCO val2014 captions that are not among the 30,000 evaluation captions and replaced the numbers in Table 2. With the disjoint captions, FID is lower than with the original calibration at NFE 4 and 6 and higher at NFE 8 and 10, and ENCORE still improves over the baseline at every NFE. Section 5.1 now states that for text-conditional models the calibration prompts are disjoint from the evaluation prompts.

NFE 4NFE 6NFE 8NFE 10
Baseline23.4918.0016.9716.55
+ ENCORE, original calibration (overlapping captions)21.7616.4814.4614.61
+ ENCORE, disjoint captions20.6716.1814.5515.23

We tested transfer along the three axes the reviewer listed: prompt set, NFE and guidance scale. In each test, the calibration is done in one setting and applied unchanged in a different setting (Appendix C.13). Across prompt sets, the calibration on MS-COCO captions raises HPSv2.1 and ImageReward on PartiPrompts at every NFE, and the calibration on PartiPrompts lowers FID on MS-COCO at every NFE (Table 28). Across NFE, the calibration at NFE 6 applied at NFE 4 and 8 lowers FID in 24 of the 32 settings, in all 16 with Euler and DPM++(2M) and in 8 of the 16 with UniPC(3M) and iPNDM(4M) (Table 29). Across guidance scale, the calibration of DiT-XL/2 at guidance 1.5 is applied at guidance 1.0, 1.25, 2.0 and 4.0, and the calibration of Stable Diffusion 1.4 at guidance 7.5 is applied at 3.0 and 5.0. FID is lower than the baseline at 5 of these 6 scales. The exception is DiT-XL/2 at guidance 4.0, the farthest of its four scales from 1.5, where FID rises from 15.09 to 16.46 (Table 30). The calibration therefore carries over to other prompt sets, NFE and guidance scales in most settings.

Q5. Seed variance of FID, and typical a_k values

Appendix C.14 repeats all 64 settings with three evaluation seeds and reports the mean and standard deviation of FID (Table 31). In 59 of the 64 settings, ENCORE is lower than the baseline by more than the standard deviation across seeds. In 4 settings the difference is within the standard deviation, including the 2.84 to 2.83 cell the reviewer quoted (AFHQv2-64, iPNDM at NFE 8, which is 2.91 to 2.90 over three seeds), and in one, FFHQ-64 with iPNDM at NFE 10, the mean rises from 3.90 to 4.01.

Appendix C.11 lists a_1 and a_2 with Euler and DPM++(2M) on all four datasets, each with its standard deviation over three calibration seeds (Table 22). In these 32 settings the coefficients lie between −1.68 and 2.73, and a_1 is positive in all 16 settings at NFE 4 and 6, so the correction moves the sample further along its last denoising difference. The standard deviations in Table 22 show how much each coefficient changes with the calibration seed.

AuthorsOpenReview ↗

Response to Reviewer gXkv (3/3)

Major weakness 2 (remaining points)

Equation 5: in Section 4.2 we have rewritten Equation 5 with its approximation error r(a), and tested the assumption in Appendix C.10. The cosine between the feature-mean shift predicted by Equation 5 and the shift measured in Inception feature space after Stage M is at least 0.90 in 59 of the 64 settings, and the lowest is 0.69. The relative error, the norm of r(a) divided by the norm of the measured shift, lies between 0.05 and 0.81 with an average of 0.31 (Table 20). Minimizing the discrepancy directly with Gauss–Newton instead of the closed form lowers FID in 30 settings, leaves it unchanged in 14 and raises it in 20, with an average change of −0.13 (Table 21).

Lookahead and Richardson extrapolation: as noted in Section 2, LA-DPM and RX-DPM also reuse the predictions the sampler has already computed, but they operate within individual sampling paths and address per-sample trajectory errors rather than distribution-level discrepancies, while ENCORE is applied once after the last step. Appendix C.2 of the paper combines ENCORE with both methods (Table 9), which we reproduce here for convenience. We have also replaced the "orthogonal axis" sentence in Section 1 with "Existing approaches either improve the sampling trajectory or refine samples after generation".

FID on CIFAR-10NFE 10NFE 12NFE 15
LA-DPM15.579.787.19
+ ENCORE6.464.643.92
RX-DPM5.224.183.36
+ ENCORE3.963.323.05

BFR: since no checkpoint of BFR is publicly available and its repository lists the reproducible experiments as still being updated, we trained the refiner with the official code and default settings for 400,000 steps and compared on CIFAR-10 with Euler at NFE 4 (Appendix C.16, Table 34). BFR adds 26 network evaluations per sample with its default adaptive solver and lowers FID from 67.31 to 58.61. When run as a single step, it adds one evaluation and FID rises to 216.47. ENCORE adds no network evaluation and lowers FID to 24.58, with lower FD-DINOv2 and higher precision and recall.

Other (remaining points)

K far from the oracle: yes. K is selected on the held-out half of the calibration pairs without any evaluation sample, and Table 14 reports its gap to the oracle K.

PAS at ImageNet-64 NFE 4: PAS does not report this setting, and its schedule there is worse than iPNDM(3M) alone, so we no longer count that pair. The stacked result in Section 5.2 now reads "ENCORE lowers FID in 14 of 15 settings of Table 1, leaving out PAS on ImageNet-64 at four steps", instead of 15 of 16.

Minor

Flow matching: Appendix C.17 applies ENCORE to SD3-medium and SD3.5-medium, two recent flow-matching text-to-image models, in their official configuration at NFE 4 and 8 on 5,000 MS-COCO captions (Table 35). Over these four settings, ENCORE lowers FID and raises HPSv2.1 and ImageReward in all four, and raises the CLIP score in three. The one decrease is for SD3-medium at NFE 4, from 28.24 to 28.17.

The name: we thank the reviewer for the suggestion and will take it into consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the three reviewers for their careful reading and helpful comments. We are encouraged that the reviewers found ENCORE "cheap and practical" (gXkv) and "simple, cheap and plug-and-play" (aZY7), with "small" computational overhead (L8RW) and consistent improvements across models (all).

We have uploaded a revised PDF with the changes in blue.

  • As all three reviewers asked whether the gain only reflects the fitted Inception mean, we evaluated with metrics other than FID (Appendix C.12). We reported the mean term and the covariance term of FID separately, and added KID, FD-DINOv2, precision and recall, and LPIPS and PSNR against the 64-step sample on all 64 settings. For Stable Diffusion 1.4 we added the CLIP score for text alignment and two learned human-preference scores, HPSv2.1 and ImageReward. For audio, where the coefficients are solved in CLAP features, we also reported the CLAP score when they are solved in VGGish features instead (Appendix C.7).
  • As aZY7 asked for a perceptual study, we ran a blind two-alternative forced-choice study with 32 participants on DiT-XL/2 (Section 5.3, Table 4).
  • As gXkv and L8RW asked what each stage matches and how large the error of Equation 5 is, we measured the band power and the feature mean after each stage (Appendix C.9) and the linear-displacement assumption (Appendix C.10). As L8RW suggested, we also minimized the discrepancy directly with Gauss–Newton instead of the closed form (Appendix C.10), and as gXkv asked, we listed the coefficients with their spread over three calibration seeds (Appendix C.11). Section 4 and Equation 5 are reworded accordingly.
  • As gXkv asked whether the gain comes from the directions d_k and how ENCORE compares with plain extrapolation, sharpening and BFR, we replaced d_k by random directions (Appendix C.12) and by PCA directions, and added fixed extrapolation, sharpening and BFR as baselines (Appendix C.16).
  • As gXkv also asked, we added transfer across prompt sets, NFE and guidance scale (Appendix C.13), FID over three evaluation seeds (Appendix C.14), and two flow-matching models (Appendix C.17). We recalibrated Stable Diffusion 1.4 on captions disjoint from the evaluation captions (Section 5.1, Table 2), left PAS out of the stacked count on ImageNet-64 at NFE 4 (Section 5.2), and removed the "orthogonal axis" sentence (Section 1).
  • As aZY7 asked about the FFHQ-64 results, we added the band-power deficit of the baseline on each dataset, which is smallest on FFHQ-64 (Appendix C.15).

We answer each remaining point in the individual responses. We thank the reviewers again for their feedback and would be glad to hear any further suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.