- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies Continual Dataset Distillation (CDD), where class groups arrive sequentially without access to past real data, and the maintained synthetic memory is evaluated by training a fresh classifier on all observed classes. It identifies Local–Global Discrimination Collapse (LGDC), measured by LGGap and CSGap, and proposes ReCDD, which replays and refines historical synthetic samples and adds a noise-initialized auxiliary class during condensation. ReCDD is applied to DC, DSA, MTT, FTD and DATM on MNIST, CIFAR-10/100 and Tiny ImageNet, and improves over sequential baselines in most settings.
Critical strengths
(1) The CDD setting is clearly defined, and comparing CSGap against a Joint control is a reasonable way to isolate cross-step interference.
(2) ReCDD serves as a plug-in for both gradient- and trajectory-matching methods, with consistent gains at IPC=10/50 on smaller datasets.
Critical weaknesses
(1) Limited novelty. LGDC is a restatement of inter-task confusion, a well-known issue in class-incremental learning: classes from different tasks are never trained together and are therefore confused at test time. CSGap likewise reuses the standard comparison between predicting with and without task identity. The method components also have clear precedents in works the paper itself cites: joint distillation of current data with previous memory in DDN (Liao et al., 2025), and refinement of stored exemplars across phases in Mnemonics (Liu et al., 2020). The paper does not explain how ReCDD differs from them methodologically.
(2) The experiments do not support the paper’s narrative. In the ablation (Table 4), the auxiliary reference alone reaches 45.4% on CIFAR-10 versus 45.6% for the full model, so replay and refinement, the core of the method, contribute little; the auxiliary reference itself is left unexplained. The diagnostics also overstate the problem: on CIFAR-10, Joint condensation already has a CSGap of about 36 pp versus about 42 pp for sequential distillation, so most cross-step confusion is inherent to the task. Figure 1 omits this Joint control. Finally, no competing method, such as DDN or LoDM (Yang et al., 2023), and no simple global fine-tuning baseline is compared.
Minor: The paper gives no background on the five base methods and omits backbone details. ConvNet appears only in the caption of Figure 3; its depth, the training schedule and cross-architecture results are not reported, which limits readability and reproducibility.
Questions for authors
- How does LGDC differ from inter-task confusion in class-incremental learning?
- Why does the auxiliary reference help, and why is Frozen replay about 10 pp worse than no replay?
- Figure 1 reports 78.0% global accuracy, well below the Table 3 mean for the same setting (85.5 ± 2.3%). Is it a single run or a selected class order?
- For MTT/FTD/DATM, how are expert trajectories built at each step, and are they retrained per step?
- Were sequential baselines tuned under the same budget as ReCDD? Some degrade at larger IPC. What is ReCDD’s computational overhead?