- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies checkpoint selection in decoupled dataset distillation, where the preferred maturity of recovery and labeling checkpoints depends on the IPC budget. The authors propose PAC. PAC first builds a checkpoint pool from accuracy milestones and forms local ensembles of consecutive checkpoints as candidates. It then recovers one shared image bank at the largest budget using a curriculum over all checkpoints, so that only labeling checkpoints need to be selected per budget. To avoid profiling every budget, the paper introduces exposure-indexed rank transfer. Labeling candidates are therefore profiled once at the reference budget, and the ranking at the matched epoch is read out for smaller budgets.
Critical strengths
- Reading candidate rankings from intermediate epochs of a single reference run is simple and easy to implement. It can produce selections for unseen budgets below the reference budget at no extra cost.
- The gains at low IPC are large and consistent across datasets and student architectures.
- The paper reports the reduction in alignment time relative to joint search and the effect of prefix elimination on profiling time.
Critical weaknesses
- The paper introduces many terms that are not standard in dataset distillation, such as maturity level, alignment, profiling, selection frontier, and readout. Some are never defined explicitly, and others are defined only implicitly or long after their first use.
- The conceptual novelty is limited. The finding that the preferred teacher training stage depends on the budget has been established by DATM and CheckpointKD. Sampling smaller IPC sets from a larger distilled set is common practice and is also used for the baselines in Table 1. Curriculum recovery over multiple teachers is a combination of existing components. The only new ingredient is the exposure matching rule, which is a heuristic.
- Exposure-indexed rank transfer, the core idea of the paper, is assumed rather than verified. Eq. (8) simply assumes that training at IPC 1 for 300 epochs is equivalent to training at IPC 50 for 6 epochs, which is the pairing the paper uses in Appendix B.1. The paper never checks whether candidate rankings actually agree under such matched settings across datasets. For the same reason, the transfer error δ_s in Corollary 1 is never measured, so the regret bound cannot be assessed.
- Theorem 1 restates its own assumption. It assumes the best candidate never shifts back to a less mature one, then concludes that less mature candidates can be safely removed. The theorem thus adds no insight beyond the assumption.
- PAC combines several components, including full-pool recovery, intermediate-checkpoint labeling, exposure matching, and prefix elimination. An ablation that adds each component to FADRM+ separately is missing.
- Most ablations and analyses (Tables 3 to 12) are run only on CIFAR-100 with ResNet-18, mostly from single runs without standard deviations. The evidence for rank transfer in Figures 2 and 3 does not state the dataset clearly.
- Eq. (6) does not specify how Π_s selects s images per class from the shared bank, for example randomly or by some criterion.
Questions for authors
- Were the 9 architectures in Table 2 each profiled separately, or do they reuse the selections from one architecture?
- In Table 13 the least mature window Ω1 is selected at IPC 1, the smallest budget tested. How sensitive are the IPC 1 results to the lower end of the milestone range?
- Table 16 lists “Number of ARC 4” without explanation. What does ARC refer to?
- In Figure 4, is profiling repeated at each stage as the number of classes grows?