- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Contraction-Aware Guidance (CAG), a training-free sampling method for generative dataset distillation using diffusion models (DMs). The authors observe that although stronger classifier-free guidance (CFG) improves the class consistency of generated images, it can reduce intra-class diversity during the later stages of denoising, thereby limiting the complementary training information contained in a small synthetic dataset.
CAG preserves the early CFG sampling trajectory and modifies the predicted clean latent only during the later denoising steps. Experiments on five ImageNet-derived benchmarks show moderate performance improvements, albeit at the cost of increased generation time.
Critical strengths
The authors make an interesting observation that, although stronger classifier-free guidance (CFG) improves the class consistency of generated images, it can reduce intra-class diversity during the later stages of denoising. This finding is insightful and may inspire further research into the dynamics of guidance in diffusion-based dataset distillation. The paper also proposes a simple and intuitive approach to mitigate this issue.
Critical weaknesses
-
The paper is highly related to Diffusion Models as Dataset Distillation Priors (DAP) [1], an ICLR 2026 paper that also uses training-free, feature-based guidance to modify the reverse diffusion process for dataset distillation. Although the two methods optimize different feature objectives, they share a closely related problem setting. However, the authors neither cite nor discuss DAP, leaving the novelty of CAG and its distinction from prior work insufficiently clarified.
-
The paper is often difficult to follow due to its dense and overly qualified writing. For example, the fifth paragraph of the Introduction introduces several caveats and distinctions before clearly communicating the main idea, which obscures rather than clarifies the contribution.
[1] D. Su, H. Wu, H. Chen, Y. Shi, Y. Wang, X. Ye, and J. Zhu. “Diffusion Models as Dataset Distillation Priors.” ICLR 2026. https://proceedings.iclr.cc/paper_files/paper/2026/hash/2c7b9949db45ccc7fa5bf857a37330da-Abstract-Conference.html
Questions for authors
-
CAG and DAP [1] substantially overlap in their methodological framework and are evaluated on four common datasets. Could the authors provide a controlled comparison with DAP or explain why a fair comparison is infeasible?
-
In Figure 1, the “CFG only” and “CAG” legends are placed awkwardly. Could the authors revise the legend placement and overall layout to improve readability?
[1] D. Su, H. Wu, H. Chen, Y. Shi, Y. Wang, X. Ye, and J. Zhu. “Diffusion Models as Dataset Distillation Priors.” ICLR 2026. https://proceedings.iclr.cc/paper_files/paper/2026/hash/2c7b9949db45ccc7fa5bf857a37330da-Abstract-Conference.html