- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates what recovery training actually recovers after structured pruning. The authors point out that evaluating recovery training solely by final overall accuracy may obscure important instance-level changes, because the repair of pruning damage, the correction of errors made by the original model, and the loss of originally correct predictions may offset one another. To address this issue, the paper starts from the same original model and constructs both a post-pruning recovery-training path and a matched continued-training path without pruning. It then analyzes overall accuracy, performance across predefined groups, and prediction changes for individual examples on Waterbirds, CelebA, MetaShift, and CIFAR-100.
Critical strengths
- The paper correctly identifies the limitations of considering only final accuracy. Two models with the same accuracy may make correct predictions on entirely different examples, resulting in different reliability and group-level risks.
- In addition to comparing the original, pruned, and recovered models, the authors include an unpruned model that undergoes continued training with the same data, number of training steps, and learning-rate schedule. This design helps distinguish changes specific to pruning recovery from those caused by continued training in general.
- The main experiments include three network architectures and three datasets with predefined groups, supplemented by experiments on CIFAR-100 using different pruning strategies and pruning ratios. The authors also report results from multiple independently trained original models and repeated training runs.
Critical weaknesses
- The methodological novelty is relatively limited. The core analysis is based mainly on existing concepts. Therefore, the contribution is closer to a systematic empirical study than to a new pruning or recovery method.
- The proposed has not been directly validated. The paper proposes using to select recovery checkpoints or pruned models, but it does not actually conduct an ablation or comparative evaluation. At present, the results establish only that the metric is intuitively meaningful, not that it leads to better model-selection outcomes.
- The scope of pruning in the main experiments is relatively narrow. The main experiments focus on approximately 10% FLOP reduction and primarily use the authors' group-sensitive channel-pruning method. Although the CIFAR-100 experiments add 10% and 20% pruning levels and random pruning, this is still insufficient to establish whether the conclusions generalize to a wider range of settings.
Questions for authors
- Why are the original models trained with GroupDRO, while both recovery and continued training use cross-entropy? Could the authors add a control experiment that uses GroupDRO throughout, to distinguish the effects of the change in training objective from those of pruning recovery?
- Does actually improve checkpoint selection? I suggest directly comparing selection based on overall accuracy, worst-group accuracy, negative-flip rate, and , and evaluating the resulting models on an independent test set.
- Do the main conclusions still hold under more aggressive pruning? When the FLOP reduction reaches 30%, 50%, or even higher levels, does the offsetting relationship between error corrections and negative flips change substantially?
- How sensitive are the conclusions to the pruning and recovery strategies? If unstructured pruning, learning-rate rewinding, weight rewinding, knowledge distillation, or other recovery methods are used, do the phenomena observed in the paper still hold?