- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a novel approach to achieving few-step previews that closely match the results of full-step denoising by directly predicting the noise residual to select a better starting point. This approach can help users quickly identify better images or perform best-of-N selection for test-time inference scaling. Experimental results on SDXL and SD1.5 demonstrate that the proposed method outperforms previous approaches across different benchmarks, achieving lower MSE and higher PSNR.
Critical strengths
- The key strength of this paper is its novel approach to updating the noise rather than designing a new scheduler. This makes the noise corrector simple yet effective. The quantitative results (Tables 1 and 2) demonstrate strong performance on toy examples, outperforming previous approaches. The qualitative results (Figure 11) also show promising outcomes, further supporting the effectiveness of the proposed method.
Critical weaknesses
- One important weakness is that the trajectory correction is mainly motivated by empirical observations, with limited theoretical explanation. It would be helpful to provide more insight into how this correction affects the denoising process and why it is effective. For instance, is there any connection to consistency models or existing scheduler designs, but with a simpler formulation? Such an explanation could strengthen the overall contribution and provide more insight into the proposed method. A similar concern applies to the transfer step, which also seems to be an empirical design choice (Appendix C.3). In particular, it is unclear why the squared term helps improve the norm. Further explanation or justification would make this design more convincing.
- Another concern regarding trajectory correction is whether it introduces additional training difficulties, as it may require backpropagation through multiple denoising steps, potentially making training more expensive or challenging for users. It would be helpful to report the training resources and computational costs, or clarify whether this is actually a concern in the proposed method. If so, discussing possible ways to mitigate this issue would further strengthen the paper.
Questions for authors
- It would be helpful if the authors could provide a more mathematical explanation of how the proposed method relates to the denoising trajectory. This does not necessarily require developing a new theory, but establishing a connection to existing approaches, such as consistency models or scheduler designs, could provide more insight into why the method works.
- Could the authors provide additional details on the training resources and computational time? It would also be helpful to clarify whether trajectory correction or input correction requires backpropagation through multiple denoising steps, and whether this introduces any training difficulties or instability. If so, how are these challenges addressed?
3 comments
I appreciate the authors' response and the additional theoretical analysis. I maintain a positive outlook on the paper.
My final suggestion is to adopt a more modern model that is more widely used in real-world applications today, as SD1.5 and SDXL are less commonly utilized now.
We agree, and thank you for your positive review and helpful suggestion! Your feedback has helped us substantially strengthen the paper.
Thank you for the thoughtful review and for recognizing the simplicity and effectiveness of our approach!
Summary of changes: We added a mathematical decomposition of trajectory error and a supporting SD1.5 experiment on per-step consistency. We also add information on the training cost.
The new SD1.5 analysis and reported cost are highlighted in blue in the revised paper, with their locations specified below.
1. Mathematical explanation of trajectory correction
Great point! We fully agree that our method would benefit from a deeper theoretical understanding.
Following your suggestion, we added a per-step consistency analysis and supporting experiments to address two questions: (1) Does LoRA correction make each student step more consistent with the teacher? (2) How does each corrected step affect the final reconstruction error, individually and through interactions with other steps?
Our starting point is Proposition A.1, which decomposes the squared L2 reconstruction error between student and teacher into per-step defect energies and their cross-step interactions (E=S+C). A step's defect measures how much taking that student step changes the final output obtained by teacher continuation. This isolates each step's effect and shows how the effects combine.
We then measure these quantities on SD1.5 using 32 prompt–seed pairs, comparing our three-step LoRA-corrected student with the raw three-step DDIM baseline. The results are summarized below.
These results show two findings. First, LoRA indeed makes every step more consistent with teacher continuation. Our method thus aligns with the consistency model’s principle of preserving the teacher-predicted endpoint along the sampling trajectory. Second, interestingly, the remaining defects partially cancel across steps, further reducing the final reconstruction error. The decomposition shows how smaller per-step defects and their partial cancellation contribute to the final error reduction.
Please see Section 3.3 and Appendix A.5 (blue text), including Proposition A.1, Figure 9, and Tables 6–7.
Step-count transfer. For input correction, we expect a smaller correction at larger sampling budgets because raw samples are typically closer to the full-step reference. The power term controls this decay; normalization preserves the original noise norm. Experimentally, stronger decay (p=2.5) gives higher PSNR, while weaker decay (p=1.7) gives higher HPS. The cause of this tradeoff requires further study. Please see Section 2 and Appendices A.4 and C.2 (Table 8).
2. Backpropagation and training requirements
Both methods backpropagate through the few-step student (rather than the teacher model) during training, updating the input corrector or step-specific LoRA adapters.
Training cost: The SD1.5/SDXL trajectory models use 8,192 updates on 4,000 prompts at five CFG scales. Training takes about 30.3 L40S GPU-hours for SD1.5 and 86.2 GPU-hours for SDXL.
FLUX uses 36,864 paired inputs and 1,548 updates, with gradient clipping and EMA. The selected EMA checkpoint is at update 1,280, with a measured training-loop cost of about 28.66 GH200 GPU-hours.
We report these numbers in Appendix B.3 of the revised manuscript.
Training stability and memory. We did not observe training instability in our experiments. We added a small profiling experiment to measure training memory and runtime on a single GH200 GPU with 95.1 GiB of visible memory. At 512×512, rank 64, and microbatch 1, our four-step FLUX student uses 37.7 GiB peak allocated memory and 3.02 s per training update. The same setup supports 16-step training updates with 61.1 GiB peak allocated memory, while 32-step training runs out of memory. Thus, our current few-step setup fits within the available GPU memory. To reduce backward computation, one can potentially detach the denoiser input.
We hope these additions make the connection between individual denoising updates and final reconstruction fidelity clearer. Thank you again for the helpful suggestions!