- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DA-DMD, a weighting scheme for distribution matching distillation based on disagreement between the teacher and fake score models. It starts from the observation that larger disagreement tends to coincide with higher update energy, particularly at high noise levels. The proposed rule attenuates distillation updates according to disagreement within a selected high-noise region. A distribution-mixture construction motivates the rule, and a smoothness analysis gives conditions under which attenuation improves a descent bound. Experiments across several diffusion backbones report improvements in image quality and prompt following.
Critical strengths
-
The method makes a small, targeted change to DMD training: it uses the existing teacher–fake disagreement to control update strength in the high-noise region. This addresses unusually large updates without additional teacher evaluations or inference cost, making the reported improvements practically relevant.
-
The experiments support the approach from several directions. Improvements are reported across five backbones, and the noise-region ablation supports restricting the intervention to high noise. The image comparisons on counting, text rendering, and attribute binding also show what the quantitative gains correspond to in generated outputs.
Critical weaknesses
-
The experiments do not show whether disagreement-dependent weighting is better than simply using smaller updates in the same high-noise region. Since , even samples with little disagreement are attenuated. A constant weight applied in the same region and matched for average retained update strength is needed to determine what the adaptive weighting adds.
-
The analysis explains when attenuation can help, but the evidence connecting that condition to disagreement is incomplete. Whether a smaller update improves the descent bound depends on update energy relative to useful gradient alignment, . The reported correlation with update energy alone does not show that disagreement identifies updates whose magnitude is excessive relative to their useful contribution.
-
The protocol for selecting is not described precisely enough to reproduce. Appendix A provides the selected values and mentions a small candidate set, and Figure 4 examines sensitivity on SD3.5-M. However, the validation prompts and selection criterion are not specified. Since the preferred attenuation varies across backbones, these details are needed to assess the tuning required to obtain the reported gains.
Questions for authors
-
How does DA-DMD compare with a constant attenuation factor applied in the same high-noise region, matched for average retained update strength?
-
What evidence supports the assumption that increases with disagreement during training, beyond the observed correlation with update energy?
-
Which candidate values, validation prompts, and selection criterion were used to choose for each backbone?
-
How is the average pairwise win rate computed? Please specify the judge, how results are aggregated across prompts, and how ties are handled.