- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SIGMS, a dataset distillation framework that combines SaliPatch initialization with stage-conditioned characteristic-function matching. Its key motivation is that existing methods typically treat representation-state construction and frequency-space probing as independent components, which may cause a mismatch between feature representations and sampled frequency arguments. SIGMS addresses this issue by coupling Stage-Conditioned Feature Interpolation with Adaptive SampleNet through a shared stage variable. It consistently outperforms NCFM on several datasets at IPC=1, demonstrating its effectiveness under highly compressed settings. However, SIGMS requires approximately 3.5 times the distillation time of NCFM, while providing only modest improvements at higher IPC settings. Overall, this is a reasonably well-evaluated but incremental work with limited computational efficiency.
Critical strengths
-
Existing methods typically treat representation-state construction and frequency-space probing independently, so the sampled frequency arguments may not adapt well to different representation states. Coordinating the two through a shared stage variable is a reasonable and insightful idea.
-
SIGMS consistently outperforms its direct baseline, NCFM, across multiple datasets at IPC=1. The ablation results also show that shared stage conditioning is more effective than using Adaptive SampleNet independently, providing empirical support for the proposed stage-coordinated design.
Critical weaknesses
-
SIGMS incurs substantial computational overhead. On CIFAR-10 at IPC=1, its distillation time increases from 3.02 hours for NCFM to 10.65 hours—approximately 3.5 times longer—while the improvements at higher IPC settings are generally modest.
-
The technical novelty appears limited. SaliPatch is closely related to existing patch-based initialization methods, while feature interpolation and learnable frequency sampling build on DANCE and NCFM, respectively. The main distinction of SIGMS is the use of a shared stage variable to coordinate these components, and the significance of this contribution requires stronger justification.
Questions for authors
-
SIGMS requires approximately the distillation time of NCFM. Could the authors provide a comparison under a matched computational budget or a similar sampler parameter count to determine whether the gains mainly result from longer optimization and a larger Adaptive SampleNet?
-
According to Eqs. (7)–(9), the shared stage variable primarily affects frequency sampling through the temperature and interpolation coefficient , while the means and variances of the Gaussian mixture do not appear to be directly conditioned on . Could the authors clarify how Adaptive SampleNet adapts to different representation states and how this mechanism differs from a predefined stage schedule?