acceptodds
Under review as a conference paper at ICLR 2027

Correction Training for Diffusion Models

Abstract

Neural networks inevitably produce imperfect predictions, and accumulate errors in iterative sampling scenarios like generative diffusion models. In this paper, we introduce an iterative correction step for diffusion models. The corrector takes the base model’s own prediction as input and is trained against the same ground-truth target as the base model. This makes correction an explicit learning task, with training inputs provided by the base model. During sampling, at each denoising step, we iteratively refine the denoiser’s clean-sample prediction. Although the corrector is trained only on predictions from the base denoiser, we find that it can be applied repeatedly to its own outputs at inference time for even better results. This simple framework substantially improves sample quality while largely preserving diversity. We extensively evaluate our method across widely used architectures and training frameworks, including DiT, SiT, REPA, and JiT. Our method substantially improves upon previously reported FID scores for unguided generation and, notably, unconditional generation, where producing high-quality samples without guidance has been particularly challenging. Our method achieves performance competitive with classifier-free guidance (CFG), and combining correction with CFG further improves diversity and reduces oversaturation typical of CFG. Finally, we show that the additional inference cost of correction can be amortized by distilling the refined predictions back into the base diffusion model, retaining the quality gains without additional inference-time overhead. We hope that our findings encourage further efforts to move beyond diffusion models' long-standing reliance on guidance techniques.

Primary area:
generative models
Keywords:
generative models, diffusion models, flow matching, classifier-free guidance, image generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SXZPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The work introduces Correction Training, which is a technique that helps in improving the quality of samples produced by diffusion processes by learning to enhance the predictions of the denoiser. The suggested corrector receives as input the clean sample predicted by denoiser and is trained to recognize the same ground-truth target without needing further supervision. Though the corrector is trained with one step of correction, it can still be used iteratively for enhancing the generated samples during inference.

This approach is tested with conditional as well as unconditional ImageNet generation by applying the four following types of architecture (DiT, SiT, SiT-REPA, and JiT) and also with using NanoGen for creating images from text. The results show significant improvement in FID values especially for unconditional generation. Furthermore, it offers results comparable to those acquired by the classifier-free guidance (CFG) technique utilizing the same inference budgets.

Critical strengths
  • The proposed correction seems simple and without any modifications in diffusion objective or proposing any additional supervision, it learns to improve the denoiser’s output by correcting it based on ground truth targets.
  • The work validates itself on four different ImageNet architectures (DIT, SiT, SiT-REPA, and JiT) and in all of them, as shown in table 1, it shows consistent improvement in FID.
  • In unconditional generation, shown in table 2, the work shows that it can improve FID (24.82 -> 5.37) and can outperform autoguidance (9.37) with same NFE and without using class conditioning or external guidance.
Critical weaknesses
  • Most quantitative experiments use ~ 250 NFEs. Even the distilled model in table 6 is evaluated in 83 NFEs. I understand that the goal is to reduce numerical integration error as much as possible and focus on modeling error but it is currently unkown how the proposed method works in low NFE, a budget that is more commonly used in practice, and whether the the numerical integration error becomes significant or not. a FID-versus-NFE curves at 10, 20, 50, 100 NFE and comparing the baseline, baseline + correction, and distilled student can better help us gain insight in how the correction works in low NFE settings.
  • Table 14 uses different correction iteration counts and active interval across different architectures used in the experiment. For example, DiT uses K=2K = 2 for the full trajectory but JiT uses K=9K = 9 for t=1→0.8t = 1 \rightarrow 0.8. This should complicate the usage of correction to a new method because it appears that it might be helpful for some if it is applied for full trajectory and for some in subset of trajectory and with different value of K.
  • The work mentions “reduced diversity” as one of the main issues of CFG. However ifigure 6 and 8 show that increasing K makes precision better but reduces recall. Although the method can achieve a more favorable trade-off than CFG, it still suffers from underlying loss of distributional coverage that CFG has.
  • Although the work has tested several ImageNet architectures, for T2I configuration in section 3.2 it only tests a DiT with a DDT head and there is no evidence if it can transfer to another T2I architecture or a substantially larger production-scale model that can generate with higher-resolution. This concern is also related with number of NFEs because with those production-scale models 250 NFE is not practical and correction should work in low NFE.
  • For the human evaluation in table 4, even though both CFG baseline and CFG + correction use 50 denoising steps, the CFG baseline has 100 NFE but in CFG + correction we have (i) the CFG scale is different and (ii) with K=8 we have 50 * (2+8) =500 NFE that is 5x the CFG baseline. Therefore it is not an apple-apple comparison as the CFG + correction uses more NFE and different CFG scale.
Questions for authors
  • How the hyperparameters in table 14 is selected?
  • Could the authors explain how the prediction error (e.g., mean square error of predicted vs. actual labels) changes when K increases at different noise levels. This would help understand if the enhancement of quality of generated samples is achieved by correct denoising predictions or due to moving the produced distribution to more visual samples.
5 comments
AuthorsOpenReview ↗

Response to Reviewer SXZP (1/3)

We thank the reviewer for their careful reading of our manuscript and detailed feedback, which helped us strengthen the experimental evaluation and explore how iterative correction improves sample quality. We are happy that the reviewer found the proposed correction "simple" and noted that it improves the denoiser's output "without any modifications in diffusion objective or proposing any additional supervision". Below, we address the reviewer's concerns and clarify a few points about our method and experimental setup.


C1. "Most quantitative experiments use ~ 250 NFEs. … a FID-versus-NFE curves at 10, 20, 50, 100 NFE and comparing the baseline, baseline + correction, and distilled student can better help us gain insight in how the correction works in low NFE settings."

C4-2. "This concern is also related with number of NFEs because with those production-scale models 250 NFE is not practical and correction should work in low NFE."

Thank you for this suggestion. We would like to note that 250 NFE is indeed a standard setting for many academic papers on ImageNet generation, and works including DiT, SiT, REPA all adopt this setting, and we follow the standard practice. Nonetheless, we are happy to further examine both the effect of using few denoising steps and performance under a limited total inference budget, which may provide better insight on how the method works in more commonly used budget in practical text-to-image generation

First, when the number of denoising steps is fixed, adding correction improves FID even in the few-step regime (Table A). This shows that correction remains effective when the denoising trajectory is coarsely discretized.

Table A. FID@5K for the baseline and the baseline with correction (K=2K=2) at matched denoising step counts. See Fig. 18 in Appendix C.1 for the corresponding plot.

Denoising stepsBaselineCorrection (K=2)
10107.1564.18
2040.2515.07
5020.227.96
10016.777.56
25014.357.44

Second, Table B reports FID for the baseline, baseline + correction, and distilled student at matched total NFE budgets, including 10, 20, 50, and 100 NFEs. Under very small budgets, allocating evaluations to correction leaves too few denoising steps (e.g., only 3 denoising steps in total NFE=10), so an increased numerical integration error may outweigh the benefits of correction. By distilling one denoising step + multiple correction steps into a single student step, the distilled student can outperform the baseline even in the low NFE regime.

Table B. FID@5K across different NFEs on baseline, baseline + correction, distilled student (SiT-XL/2, guidance scale w=1). See Fig. 19 in Appendix C.1 for the full FID-versus-NFE curves.

Total NFEBaselineBaseline + Correction (K=2)Distilled Student
10107.147381.59469.842
2040.255154.81417.454
5020.21519.6668.700
10016.7749.1637.978
25014.3467.5087.884
50014.0307.4547.819
100013.9357.5037.752

We also note that, at higher inference budgets, adding correction yields FID scores that the baseline cannot match even with additional denoising steps. Together, baseline + correction and the distilled student form the Pareto frontier for inference compute (total NFE) and sample quality (FID), achieving lower FID than the baseline at both practical low-NFE budgets and higher inference budgets.

AuthorsOpenReview ↗

Response to Reviewer SXZP (2/3)


C2. "Table 14 uses different correction iteration counts and active interval across different architectures used in the experiment. For example, DiT uses K=2K=2 for the full trajectory but JiT uses K=9K=9 for t=1.0→0.8t=1.0 \rightarrow 0.8."

Q1. "How the hyperparameters in table 14 is selected?"

Correction has two inference-time hyperparameters (correction count KK and correction interval) analogous to CFG's guidance scale ww and guidance interval. Both can be adjusted using the same trained checkpoint without retraining, making them relatively cheap to tune. Such inference-time sweeps are also standard practice for guidance parameters, as in REPA and JiT. As with CFG, suitable settings can vary with the model family, size, and whether generation operates in pixel or latent space. Also, In practice, users can inspect a few samples to identify plausible settings.

For Table 14, we sweep K∈1,…,10K \in \\{1, \ldots, 10\\} using FID@5K and evaluate the selected setting with FID@50K. We follow each baseline’s sampling conventions when deciding whether to tune the interval. For DiT and SiT, we apply correction throughout the trajectory, as their baselines do for CFG. For SiT-REPA and JiT, the baseline evaluations include guidance-interval sweeps, so we also sweep correction intervals. We search interval endpoints on a coarse grid 1.0,0.8,0.6,0.4,0.2,0.0\\{1.0, 0.8, 0.6, 0.4, 0.2, 0.0\\} and try nearby values in steps of 0.10.1 when needed.


C3. "The work mentions “reduced diversity” as one of the main issues of CFG. However ifigure 6 and 8 show that increasing K makes precision better but reduces recall. Although the method can achieve a more favorable trade-off than CFG, it still suffers from underlying loss of distributional coverage that CFG has."

Thank you for pointing out this. Our wording was not very clear: we meant to say that while reduced diversity is one of the main issues of CFG, it is much less pronounced with our method, as can be seen in qualitative figures (Fig. 9 in the update manuscript). However, this qualitative improvement is not well captured by the Recall metric in Fig. 6 and Fig. 8, due to its well-known limitations: outliers can inflate the estimated manifold of generated samples and increase recall without improving coverage of realistic samples [1]. We therefore complement Recall with the Coverage metric [1], which was proposed to address this outlier sensitivity. As can be seen in the table below, correction improves coverage over the unguided baseline despite lower recall, suggesting that recall alone may not fully characterize the change in sample diversity. We revised the manuscript to add the coverage metric (Fig. 6 and Fig. 8) and cleaned up this explanation.

AuthorsOpenReview ↗

Response to Reviewer SXZP (3/3)


C4. "Although the work has tested several ImageNet architectures, for T2I configuration in section 3.2 it only tests a DiT with a DDT head and there is no evidence if it can transfer to another T2I architecture or a substantially larger production-scale model that can generate with higher-resolution."

Thank you for raising this point. Our study centers on ImageNet, following the evaluation scope of architectural studies such as DiT and JiT. We validate correction across four architectures spanning latent and pixel spaces, and additionally demonstrate its applicability to text-to-image generation. Broader evaluation on other T2I architectures and production-scale models would be valuable, but these experiments are too costly to perform and are beyond the scope of this paper. We do agree with the reviewer that larger-scale T2I experimentation is an important future direction.


C5. “For the human evaluation in table 4, ... it is not an apple-apple comparison as the CFG + correction uses more NFE and different CFG scale.”

Thank you for pointing out this confounder. We repeated the human evaluation using the same CFG scale (w=6), the same total inference budget (250 NFEs). Under these controlled settings, CFG + correction (K=6) remains preferred over CFG alone for both quality (62.9%) and diversity (55.8%). We have updated Table 4 with these results.

MethodQuality ↑Diversity ↑
CFG baseline37.1%44.2%
CFG + Correction62.9%55.8%

Q2. "Could the authors explain how the prediction error (e.g., mean square error of predicted vs. actual labels) changes when K increases at different noise levels. This would help understand if the enhancement of quality of generated samples is achieved by correct denoising predictions or due to moving the produced distribution to more visual samples."

Thank you for this insightful question. We have updated our discussion section in an attempt to provide better intuition for why this method works. In Fig. 27, we report the prediction MSE across different correction counts KK and noise levels, as requested by the reviewer. Interestingly, the MSE increases as we apply more correction iterations (note that the error is measured on noisy real data, not on inference states). But the final sample quality improves.

This may seem counterintuitive at first glance. One possible explanation for this is the training–inference gap. During diffusion sampling, prediction errors due to neural network approximation can accumulate across steps, so the denoiser encounters states different from the noisy real images seen during training, potentially degrading sample quality. By sharpening and refining predictions before constructing the next noisy state, correction may bring subsequent inputs closer to the training distribution. We quantify the gap with a two-sample classification test, and it shows that iterative correction narrows this gap (Sec. D.3 and Fig. 28). Bringing the inputs to subsequent denoising steps closer to the training distribution may help explain the improvement in sample quality. We present this analysis in Section 4 and Sections D.3–D.4 of the appendix.

Reviewer_q5eMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes correction training for diffusion models. A corrector is trained on the denoiser’s predictions using the original clean-data targets, and is applied repeatedly within each sampling step at inference time. The approach is evaluated across several image-generation architectures and text-to-image generation. The paper also investigates parameter sharing and distillation to reduce resource requirements. The substantial improvements in unguided generation are promising, and the extensive ablations strengthen the empirical contribution.

Critical strengths
  1. The improvement is clear. Table 1 reports large FID improvements across DiT, SiT, REPA, and JiT, covering both latent-space and pixel-space generation. The main quantitative comparison is promising.

  2. Parameter sharing retains much of the improvement, while distillation transfers substantial gains into a single denoiser. These experiments increase the practical relevance of the method.

Critical weaknesses
  1. The contribution needs clearer differentiation from closely related work ProSeCo. ProSeCo already combines denoising and correction training using model predictions, clean targets, and stop-gradient, followed by repeated correction during sampling. This closely related work is not discussed. The manuscript should clarify its additional technical contributions and more precisely distinguish its approach from previous work. The concern is the scope of novelty.

  2. The most results are not fully specified by the main-text formulation, which are introduced in Algorithms 1–2 and Equations (3)–(4). Table 13 and Equations (8)–(9) show that DiT, SiT, REPA, and the text-to-image experiments use gated prediction, only JiT uses direct (x)-prediction.

Questions for authors
  1. What are the principal technical contributions beyond releated works?

  2. Does correction improve the best achievable text-to-image alignment when CFG strength and sampling steps are tuned under the same inference budget?

3 comments
AuthorsOpenReview ↗

Response to Reviewer q5eM (1/2)

We appreciate the reviewer's feedback and valuable comments that help improve our manuscript. We are happy that the reviewer found "The improvement is clear" and especially that "I learned something interesting or useful". Here we address the questions raised.


C1. "The contribution needs clearer differentiation from closely related work ProSeCo. ... ProSeCo already combines denoising and correction training using model predictions, clean targets, and stop-gradient, followed by repeated correction during sampling."

Q1. "What are the principal technical contributions beyond releated works?"

Thank you for pointing out highly related work ProSeCo [1]. We will make sure to cite this work, and in fact we have already included a citation and discussion in the subsequent versions of our manuscript. Both are included in the updated submission (main text, L528–531, and Appendix D.1, L1748–1760).

Respectfully, we would like to note ICLR 2027 policy on concurrent work:

"We consider papers contemporaneous if they are published within the last two months. That means, since our full paper deadline is September 16, if a paper was published (i.e., at a peer-reviewed venue) on or after July 17, 2026, authors are not required to compare their own work to that paper. Note that arXiv is not considered a peer-reviewed venue." (https://iclr.cc/Conferences/2027/ReviewerGuidelines)

Since ProSeCo was accepted to NeurIPS’26 after July 17, 2026, it is to be considered contemporaneous work. However, we agree that it would be valuable to discuss the connection and technical contribution of our work here and in the manuscript.

While ProSeCo and our work share several design choices in training (feeding the model's own clean-data predictions back into the model and using stopping-gradients, as in self-conditioning [2]) and in sampling (applying iterative correction), they address fundamentally different domains and problems:

  • ProSeCo targets discrete diffusion (specifically, masked diffusion models) over categorical data in language modeling and molecular design). Since the model predicts a categorical distribution at each token position and generation proceeds by progressively unmasking positions, tokens remain fixed once unmasked, which naturally motivates ProSeCo to refine already decoded tokens.
  • In contrast, we propose correction training for continuous diffusion models on continuous signals, focusing on image generation. In continuous diffusion, intermediate states can be updated at every denoising step (unlike the fixed token issue in masked diffusion models), yet errors still accumulate and substantially degrade sample quality, especially when classifier-free guidance (CFG) is not used. This motivated us to find a more principled method that can make diffusion models generate high-quality samples without CFG.

We would like to emphasize that continuous and categorical data are in fundamentally different state spaces and that the corresponding diffusion formulations differ in how they parameterize predictions, define training objectives, and generate samples. So, in general, not all methods can be trivially transferred between the two settings.

In the image generation task, our approach achieves state-of-the-art performance and significant visual quality improvement among guidance-free methods without external perceptual losses. Reaching this level of visual quality without CFG is itself nontrivial, given how heavily the image generation community has relied on it in continuous diffusion models. Further, we show we can amortize multiple correction steps by distilling it into one denoiser step, providing a practical way for industry and the community to use our correction method.

In addition to practical performance, we provide extensive analysis to uncover the mechanisms underlying these gains:

  • We provide insights specialized to continuous image signals. For example, frequency-domain analysis (Fig. 36 in the update manuscript) shows that additional correction iterations restore attenuated high-frequency components, aligning the power spectrum with natural images.
  • In the updated manuscript, we quantify the training-inference gap from error accumulation via a two-sample classification test (Sec. D.3) and demonstrate that correction narrows this gap.
  • We show that multi-step correction during training shifts the optimal correction step count proportionally without significant gains, suggesting that improvements stem primarily from repeatedly applying the learned mapping from noisy predictions to coherent clean targets.
  • We show that jointly fine-tuning the denoiser exposes the corrector to a wider range of model biases compared to freezing the denoiser, making it more robust to higher iteration counts while preserving the original structure better than freezing the denoiser.
AuthorsOpenReview ↗

Response to Reviewer q5eM (2/2)

In summary, we have cited ProSeCo and discussed the principle of iterative self-correction that our approaches share. In continuous image generation, our method achieves state-of-the-art quality among guidance-free methods without external perceptual losses. We also demonstrate its use in text-to-image generation and make correction practical through distillation. Our analyses help explain why correction works in continuous diffusion by showing that it restores attenuated high-frequency components in image signals and narrows the training–inference gap.


C2. "The most results are not fully specified by the main-text formulation, which are introduced in Algorithms 1–2 and Equations (3)–(4)."

Thank you for raising this point. We put the simplest version in the main paper for easier understanding, and detailed the application to the epsilon and the velocity prediction in Appendix B.2. Following the reviewer’s comment, we added the sentence in the main text (L199) that the epsilon and velocity prediction is shown in Appendix B.2. We also added the intuition behind the equations in Appendix B.2 and summarize it in Table 15.


Q2. "Does correction improve the best achievable text-to-image alignment when CFG strength and sampling steps are tuned under the same inference budget?"

Thank you for asking this. Following the experimental setup in Table 3, we repeated the evaluation across different CFG scales ww and numbers of correction steps KK. We extended the sweep to w∈2,4,6,…,12,16,20w \in \\{2, 4, 6, \ldots, 12, 16, 20\\} and K∈0,2,4K \in \\{0, 2, 4\\} under a fixed inference budget and report the results in Table A.

Correction generally improves text-to-image alignment at matched guidance scales in the commonly used range from w=4w=4 to w=12w=12, including w=6w=6, the officially recommended setting for NanoGen. At very high guidance scales, however, the uncorrected baseline achieves the highest alignment scores over the full sweep, with GenEval peaking at w=16w=16 and DPG-Bench at w=20w=20.

To complement these prompt-following metrics, we also report a human preference metric (PickScore [3]) in Table B. With correction, PickScore peaks at moderate guidance scales (w=6w=6 or w=8w=8, depending on the prompt set). The best tested correction setting improves PickScore at every guidance scale for both prompt sets, consistent with the preference gains we observed in Fig. 10.

Table A. DPG-Bench and GenEval scores across guidance scales ww under a fixed inference budget, following Table 3. The asterisk (*) marks the officially recommended CFG scale for NanoGen’s text-to-image evaluation [4].

ww│DPG-Bench ↑│GenEval ↑
│No correctionBest with correction│No correctionBest with correction
2│80.1481.42│56.4262.70
4│83.3883.69│68.0870.48
6∗6^{*}│84.5384.56│72.2073.28
8│85.0585.01│73.6474.68
10│85.2785.46│74.5975.32
12│85.6585.49│74.8675.68
16‾\underline{16}│85.8385.99│76.13‾\underline{\mathbf{76.13}}74.91
20‾\underline{20}│86.09‾\underline{\mathbf{86.09}}86.01│75.9074.91

Table B. PickScore on GenEval and DPG-Bench prompts across guidance scales ww, using the same fixed inference budget and notation as in Table A.

ww│GenEval│DPG-Bench
│No correctionBest with correction│No correctionBest with correction
2│21.5222.26│20.5321.00
4│22.2022.59│20.9121.14
6‾∗\underline{6}^{*}│22.4122.68│21.0321.17‾\underline{\mathbf{21.17}}
8‾\underline{8}│22.5022.70‾\underline{\mathbf{22.70}}│21.0621.17‾\underline{\mathbf{21.17}}
10│22.5422.69│21.0821.15
12│22.5222.65│21.0721.15
16│22.5122.58│21.0621.09
20│22.4722.52│21.0321.05

We have included the full results in Appendix C.4 of the updated manuscript and added a reference to them in the caption of Table 4.


[1] Schiff, Yair, et al. "Learn from your mistakes: Self-correcting masked diffusion models." arXiv preprint arXiv:2602.11590 (2026).

[2] Chen, Ting, Ruixiang Zhang, and Geoffrey Hinton. "Analog bits: Generating discrete data using diffusion models with self-conditioning." arXiv preprint arXiv:2208.04202 (2022).

[3] Kirstain, Yuval, et al. "Pick-a-pic: An open dataset of user preferences for text-to-image generation." Advances in neural information processing systems 36 (2023): 36652-36663.

[4] Leng, Xingjian, et al. "DiffusionBench: On holistic evaluation of diffusion transformers." arXiv preprint arXiv:2606.24888 (2026).

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer q5eM,

Thank you for reviewing our manuscript. With the author–reviewer discussion period ending soon, we wanted to follow up and would be happy to answer any remaining questions.

We've cited ProSeCo in the main text and added a comparison in Appendix D.1 that clarifies contributes of our work.

(In summary, we achieve state-of-the-art sample quality among guidance-free methods without using external perceptual losses. We demonstrate the practical value of our method by applying it to text-to-image generation and distilling multiple correction passes into a single denoiser step to accelerate generation. We also provide unique analyses of why the method works, including a two-sample classification test to measure the training–inference gap, extensive ablation studies, and an analysis of the power spectrum of continuous image signals.)

Thank you again for your time and for helping us improve the manuscript.

Best,
The Authors

Reviewer_ztW9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper trains a corrector to refine a diffusion model’s clean-sample predictions using the original training targets. Although trained for one correction, the corrector can be applied repeatedly during sampling. Experiments cover conditional, unconditional, and text-to-image generation, alongside parameter sharing and distillation.

Critical strengths

The improvements are substantial at approximately matched inference budgets. SiT’s unguided FID improves from 8.55 to 1.92, and unconditional FID from 24.82 to 5.37. Continued-training and capacity controls strengthen the evidence. Parameter sharing and distillation also demonstrate practical deployment options.

Critical weaknesses

The human evaluation does not isolate correction’s contribution. The main comparison changes guidance strength from 6 to 4 and adds correction passes while keeping denoising steps fixed. Both guidance and total compute therefore differ, limiting conclusions about simultaneous quality and diversity improvements.

Repeated correction has meaningful trade-offs. Unconditional recall decreases from 0.699 to 0.595, and excessive correction can degrade results. Performance also depends on the correction count and interval, whose transferability remains insufficiently established.

Questions for authors

Do the human-rated quality and diversity improvements persist at the same guidance strength and matched total inference compute?

How are the correction count and interval selected, and how well do these choices transfer across checkpoints?

1 comment
AuthorsOpenReview ↗

Response to Reviewer ztW9

Thank you for the review. We are happy that you found that "the improvements are substantial" and especially that you "gained a substantially deeper understanding of the problem or question." Here we address the questions raised.


C1. "The human evaluation does not isolate correction’s contribution. The main comparison changes guidance strength ... and adds correction passes while keeping denoising steps fixed."

Q1. "Do the human-rated quality and diversity improvements persist at the same guidance strength and matched total inference compute?"

We repeated the human evaluation using the same CFG scale (w=6), the same total inference budget (250 NFEs). Adding correction (K=6) to CFG remains preferred over CFG alone for both quality (62.9%) and diversity (55.8%). These results address the guidance-strength and compute confounds in the original comparison. We updated Table 4 with this guidance-scale- and inference-budget-controlled experiment.

MethodQuality preferenceDiversity preference
CFG baseline37.1%44.2%
CFG + Correction62.9%55.8%

C2-1. "Repeated correction has meaningful trade-offs. Unconditional recall decreases from 0.699 to 0.595, and excessive correction can degrade results."

We agree that increasing KK can reduce diversity relative to the unguided baseline, and excessive correction can also degrade sample quality. Our intended claim is that correction offers a more favorable fidelity/diversity trade-off than CFG. In the manuscript, we were trying to say that our method preserves diversity better than CFG, not that it preserves the diversity of the unguided baseline. However, we found that some sentences (like in the abstract) could be misleading. We have now updated the corresponding sentences to make this point more clear.

Also, it has been pointed out that the automated Recall metric [1], although widely used, conflates outliers with the true diversity [2]. When computing Recall, the generated manifold is estimated as a union of hyperspheres in feature space, each centered on a generated sample with a radius equal to the distance to its kk-th nearest neighbor. An isolated outlier can therefore create a very large hypersphere and artificially increase recall. We therefore complement Recall with Coverage metric, proposed in [2] as a more reliable diversity metric. In our conditional and unconditional evaluations, correction improves Coverage over the unguided baselines and achieves higher peak coverage than the corresponding guidance methods. We added “Coverage-Density” plots in Fig 6 and Fig 8 and updated the manuscript.


C2-2. "Performance also depends on the correction count and interval, whose transferability remains insufficiently established."

Q2. "How are the correction count and interval selected, and how well do these choices transfer across checkpoints?"

We appreciate the question on hyperparameters, and we should note that Correction count KK is similar to CFG guidance scale, while Correction interval originates from CFG guidance interval. Thus, our method shares a very similar hyperparameter combination to CFG. Since both can be tuned with the same trained checkpoint without retraining, it’s not costly to tune them in practice. For quantitative evaluation, we sweep K∈1,…,10K \in \\{1, \ldots, 10\\} using FID@5K and evaluate the selected setting with FID@50K.

We use the full trajectory for DiT and SiT, and additionally sweep intervals for SiT-REPA and JiT, whose baseline configurations use guidance intervals. We search interval endpoints on a coarse grid 1.0,0.8,…,0.0\\{1.0, 0.8, \ldots, 0.0\\} and refine locally when needed. For JiT, we favor early correction (t=1.0t=1.0-0.80.8), since applying correction later can oversmooth samples. In practice, users can inspect a few samples to identify promising settings before a quantitative sweep, making adaptation to a new checkpoint possible without retraining.


[1] Kynkäänniemi, Tuomas, et al. "Improved precision and recall metric for assessing generative models." Advances in neural information processing systems 32 (2019).

[2] Naeem, Muhammad Ferjad, et al. "Reliable fidelity and diversity metrics for generative models." International conference on machine learning. PMLR, 2020.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.