- Rating
- 4(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes a new method for CFG distillation by using a mixer to encode guidance scale and timesteps and using attention manipulation for negative guidance.
Critical strengths
- The paper proposes a new and effective method for distilling few-step models.
- The proposed method considers both image and video models with different architectures.
- The math behind the paper supports the method well.
- The human evaluation supports the first claim about distillation
Critical weaknesses
- The negative guidance control deviates from how negative prompts are intended to be used. A negative prompt is usually not directly conflicting with the positive prompt. For example, one might ask for a
positive prompt: housenegative prompt: windows. However, in the examples shown in Figure 7, the positive prompt directly mentions the negative element: the positive prompt asks for a wooden table with sunglasses, while the negative prompt asks for no sunglasses. This raises the question if sunglasses do not appear, is the model following the negative prompt or just failed to follow the positive one? If the author can provide more experiments on a small set of non-conflicting prompts (like the NegGenBench used in VSF [1] or DCS-Bench from [2]. Some examples from [2]: ("A Christmas tree", "gift boxes") ("A home office desk at night", "desk lamp"). The author could also try a few de-biased examples like (“a machine learning researcher”, “male”). - For the guidance part, the authors compared enough baseline traditional distillation methods like DMD and LCM. However, for the negative guidance part, the authors did not compare with training-free methods like NASA (which is what the PP-CFG is built on top of), VSF, and NAG.
- Following the last one, the novelty of this paper seems limited. It is just NASA with two condition transformations trained with LCM or DMD loss.
- The authors did not specify what distillation datasets were used besides “internal-Image-10k”. With no information about it, it is unclear if it overlaps with the testing datasets.
[1] https://arxiv.org/abs/2508.10931
[2] https://arxiv.org/pdf/2605.29390
[3] https://arxiv.org/abs/2505.21179
Questions for authors
Could the author provide more information about the training dataset?
I am impressed by the author's detailed rebuttal, and my concerns are resolved. I changed my score from 2 to 4. The NegGenBench results solved my concern, and the new comparison with the training-free method is valid. I would not want to take points off for novelty concerns, given that the comparison with the base is added. The details of the dataset are also informative. I am also impressed by the rebuttal the authors had for other reviewers.