MaskDD: Sculpting Discriminative Features for Dataset Distillation

Dataset Distillation (DD) aims to compress large-scale datasets into a compact set of synthetic samples while preserving downstream performance. Recent diffusion-based DD methods improve scalability and visual quality by leveraging pretrained generative priors, but they suffer from a fundamental misalignment between generative realism and discriminative utility. Specifically, diffusion models optimized with maximum likelihood estimation tend to emphasize dominant generic features while overlooking the fine-grained discriminative signals essential for accurate categorization. Consequently, existing methods often generate visually realistic yet weakly discriminative synthetic datasets. To address this issue, we propose MaskDD, a diffusion-based dataset distillation framework that jointly optimizes data synthesis and model structure. Our method introduces a class-aware masking mechanism to selectively preserve task-relevant feature channels and suppress redundant representations. In addition, we develop a dynamic rollback strategy inspired by the Lottery Ticket Hypothesis to stabilize optimization and preserve favorable subnetworks during discriminative refinement. Extensive experiments across multiple datasets and architectures demonstrate that MaskDD consistently outperforms existing state-of-the-art methods in dataset distillation.