- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CrePE, which incorporates a two-dimensional local neighborhood term into the row- and column-normalized importance score of RIA and balances the three types of information through hyperparameters. They further propose PHO, which uses the Gini coefficient of pruned importance scores as a surrogate objective and reduces the search. Extensive experiments have demonstrated the effectiveness of the proposed method.
Critical strengths
- CRePE does not require retraining; it primarily performs pruning by modifying importance scores, making it easy to integrate with existing score-based pruning processes.
- The ablation experiment is performed thoroughly.
Critical weaknesses
- There is a lack of experiments on 30b or 70b models.
- Current datasets primarily focus on commonsense reasoning tasks, lacking experiments on other types of tasks such as mathematics and coding.
- The latest baseline is RIA, but comparisons with newer methods are lacking.
- There is a lack of theoretical proof that the proposed method is better than RIA.
Questions for authors
Can the generalization ability of the proposed method be further demonstrated on larger and newer models?