Dataset Distillation via Saliency-Guided Initialization and Adaptive Gaussian Mixture Frequency Sampling

Dataset distillation aims to compress a large training set into a compact synthetic set while preserving its training utility. However, in existing distribution matching methods, representation construction and frequency sampling are typically treated as independent components, which may introduce alignment bias between different representation states. In this paper, we propose a novel dataset distillation framework, termed Saliency-guided Initialization with Gaussian Mixture Sampling (SIGMS), which addresses both weak synthetic-data initialization and the limited adaptability of frequency sampling to varying representation states through two complementary mechanisms. Firstly, we propose SaliPatch, a saliency-guided patch initialization module that selects and stitches informative patches from real images, reducing background interference and improving the quality of synthetic initialization. Secondly, we jointly introduce Stage-Conditioned Feature Interpolation and Adaptive SampleNet, where intermediate representations are constructed between the frozen initial and expert models, while frequency sampling is adaptively adjusted to the corresponding representation states through a shared stage variable, thereby reducing the mismatch between feature representations and frequency probing during distribution alignment. Extensive experiments on CIFAR-10 and CIFAR-100 demonstrate that SIGMS achieves state-of-the-art performance at IPC=1 and maintains competitive performance at higher IPC settings.