- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PSM improves decoupled dataset distillation by accounting for sample difficulty that existing methods ignore. It scores image difficulty with GPS, splits each class into difficulty groups, and uses group-specific BN statistics (SUA) and matching initializations (ISS) to supervise distilled samples. Experiments show PSM broadens the difficulty range of distilled data and boosts performance across datasets and architectures.
Critical strengths
-
This paper is simle but effective ,easy to follow.
-
The thought of sample difficulty in statistical matching-based methods is new to me, which maybe important for DD.
-
The insight is intuitive and the experiments are robust.
Critical weaknesses
-
Although the motivation regarding sample difficulty is intuitive, it still requires stronger experimental evidence or support from previous research findings. This is because earlier studies have also shown that under a smaller IPC, only simple samples are needed. This may be worth further investigation.
-
I still have some concerns about the fairness and security of this type of DD paradigms. They rely on soft labels, and prior research has shown that under identical settings they can even perform worse than randomly selected images. In addition, they introduce the risk of teacher information leakage. Moreover, evaluation must use the same image augmentation scheme and training logic employed during distillation, which also compromises the flexibility of downstream tasks.
-
PSM appears to be usable as a plug-in that can be integrated into existing methods, such as SRE2L and GVBSM. The corresponding experimental results should be presented
minor: line 68 :Imagenette->ImageNette
Questions for authors
-
Regardless of the results presented in this paper, is there any relevant evidence or analysis showing that covering the range of difficulty in the original dataset is important for DD?
-
Regarding ISS, for example, when IPC = 10, does it mean dividing the data into 10 difficulty groups of equal size, and then randomly selecting one sample from each difficulty group, or choosing the one with medium difficulty, as the initialization?
-
Refer to Weaknesses 3. I still think that, under identical settings, a comparison with coreset-based methods is necessary regardless of whether the results are better or worse. Could the authors provide the results of methods such as random, k-center, and herding under this framework?
-
Is the teacher statistics update performed in real time, or is each batch updated using the original statistics as an anchor?
Research on difficulty in DD is not novel, but the idea of directly combining difficulty with statistics and updating teacher statistics is insightful and may have a positive impact on DD research. Therefore, if the authors can properly address all of my concerns, I would be happy to raise my score.