- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Plug-and-Prune, which allows for adjusting the sparsity of each layer to improve the performance of the pruned model when the total pruning ratio is fixed. Specifically, the proposed method, based on the allocation of existing methods, measures the layer-by-layer reconstruction response generated by the selected pruning backend and updates the weight retention ratio accordingly using regularization. The update process penalizes deviations from the initial allocation while satisfying global budget constraints and upper and lower bounds for each layer's ratio, while the intra-layer pruning mask is still constructed by the original pruning backend. Extensive experiments have demonstrated that the proposed method can further improve the performance of pruning methods such as OWL, DLP, and AlphaPruning.
Critical strengths
- The proposed algorithm is a plugin that can be applied to other pruning algorithms.
- Extensive experiments have demonstrated the effectiveness of the proposed method.
Critical weaknesses
- Why are most experiments conducted at a 70% pruning rate? It is recommended to conduct experiments within a pruning rate range of 50%-90%.
- The cost of the proposed algorithm needs to be reported further.
- The existing experimental models are relatively old. Should we conduct experiments on some of the latest GLM, Qwen and other models?
- Could you analyze why some layers can have high sparsity while others have relatively low sparsity?
Questions for authors
The performance gains in Table 2 are very small; could this be due to randomness?
Dear Reviewer fQj9,
With the discussion period nearly over, we are writing to gently follow up on our initial response to your review. We would be very grateful to know whether our explanations and the additional results have addressed your concerns, and if there are any remaining issues that we can help clarify.
Thank you again for your time and consideration.
Best regards,
The Authors