CAG: Contraction-Aware Guidance for Diffusion Dataset Distillation

Recent diffusion-based dataset distillation has emerged as a promising approach for constructing compact training sets from pretrained diffusion models. Under extremely low sample budgets, the semantic fidelity and class discriminability of generated samples are important determinants of its effectiveness. Existing training-free methods therefore commonly employ classifier-free guidance (CFG) with large guidance scales. However, although strong CFG improves conditional fidelity, it also sacrifices mode coverage and intensifies late-stage within-mode contraction, causing synthetic samples to share redundant local structures and provide insufficiently complementary training signals. To resolve this tension, we propose Contraction-Aware Guidance (CAG), a training-free framework that introduces timestep-gated manifold correction during inference. CAG reconstructs clean estimates from noisy sampling states, constructs a contraction-aware objective from feature variance, and activates the resulting correction only during late-stage denoising. This design counteracts excessive trajectory concentration while preserving the class semantics established by CFG, thereby promoting richer fine-grained and intra-class variation. Extensive experiments demonstrate improved diversity and cross-architecture generalization, with state-of-the-art performance across multiple extremely low-budget dataset-distillation benchmarks.