acceptodds
Withdrawn from ICLR 2027

CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Post-Training Pruning (PTP) reduces the memory and computational costs of Large Language Models (LLMs) by removing weights without retraining. RIA, a state-of-the-art scoring method, normalizes each weight by its row and column sums, but uses only 1D cross-shaped information and weights the two directions equally. We propose CRePE (Convolution-aware Relative Importance in Post-training Pruning with Efficient Search), which adds a 2D local neighborhood term and adaptive per-term coefficients to Relative Importance scoring. Since searching these coefficients by perplexity (PPL)-based hill climbing takes about 11 hours on LLaMA-2-7B, we further propose PHO (Proxy-based Hyperparameter Optimization), which uses the Gini coefficient of pruned importance scores as a surrogate objective and reduces the search to about 20 minutes, a 30 speedup with nearly the same PPL; the resulting coefficients transfer from LLaMA-2-7B to LLaMA-2-13B without re-tuning. Across LLaMA-1/2/3 under 2:4 and 4:8 sparsity and LLaMA-1/2 under unstructured sparsity, CRePE consistently improves over scoring-only baselines, with margins that widen under tighter structural constraints or higher sparsity. Under 2:4 sparsity, CRePE lowers PPL by 0.43 on average relative to RIA. CRePE also combines with channel permutation, non-uniform sparsity allocation, quantization, and re-pruning.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
post-training pruning, large language models, semi-structured sparsity, N:M sparsity, pruning metric, hyperparameter optimization, proxy objective

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Q1p7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CrePE, which incorporates a two-dimensional local neighborhood term into the row- and column-normalized importance score of RIA and balances the three types of information through hyperparameters. They further propose PHO, which uses the Gini coefficient of pruned importance scores as a surrogate objective and reduces the search. Extensive experiments have demonstrated the effectiveness of the proposed method.

Critical strengths
  • CRePE does not require retraining; it primarily performs pruning by modifying importance scores, making it easy to integrate with existing score-based pruning processes.
  • The ablation experiment is performed thoroughly.
Critical weaknesses
  • There is a lack of experiments on 30b or 70b models.
  • Current datasets primarily focus on commonsense reasoning tasks, lacking experiments on other types of tasks such as mathematics and coding.
  • The latest baseline is RIA, but comparisons with newer methods are lacking.
  • There is a lack of theoretical proof that the proposed method is better than RIA.
Questions for authors

Can the generalization ability of the proposed method be further demonstrated on larger and newer models?

Reviewer_BmQ5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes CRePE, a pruning method that extends RIA with a 2D local neighborhood term and three coefficients to weight the row, column, and neighborhood terms. It also introduces PHO, which uses the Gini coefficient of pruned importance scores from the first transformer block’s q_proj layer to screen coefficient settings before full model PPL evaluation. The authors report improved perplexity over the tested scoring baselines and a reduction in search time from about 11 hours to 20 minutes on LLaMA-2-7B.

Critical strengths

The method is simple, requires no retraining or weight updates, and can be combined with channel permutation, nonuniform sparsity allocation, and quantization. The paper is clearly written, with visually appealing figures that make the method easy to understand and follow.

Critical weaknesses
  1. The novelty of this paper compared with existing methods is limited. The main changes involve the importance score and the search for its coefficients. For the importance score, the method only adds a 2D local neighborhood term to RIA and uses three hyperparameters to weight the three terms.

  2. The authors use the q_proj layer in the first transformer block to compute the Gini coefficient and guide the search for the weighting coefficients. They report a strong correlation between this proxy and PPL. The high correlation may mainly reflect the strong impact of this early layer on model performance, since errors introduced at this stage can build up through later layers. This does not mean that the coefficients found using this layer are also suitable for other layers. Although the authors show some positive results, the strategy still seems to lack a solid basis. Furthermore, the author does not appear to show the final values ​​set for these coefficients in the experiments.

  3. Table 7 shows that PHO takes much longer than pruning itself. However, these results are reported only for a relatively small 7B model, and the search cost on larger models is not provided. This cost could be much higher for larger models, since each candidate selected for PPL evaluation requires forward passes on the validation set. This raises concerns about whether PHO can scale to larger models and may limit its practical use.

Questions for authors

Please refer to Critical Weaknesses, which is my main concern.

Additionally, I have a few questions.

  1. Does the proposed local neighborhood outperform an equally sized random neighborhood, showing that local structure matters beyond adding another normalization term?

  2. Beyond the reported correlations, what supports applying coefficients selected using the first q_proj layer to all layers?

Reviewer_aGrqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CRePE extends RIA’s pruning criterion with a local 2D neighborhood normalization term and tunable coefficients for its three components. PHO searches these coefficients using the Gini coefficient of pruned importance scores as a proxy, followed by validation-PPL evaluation of shortlisted candidates. Experiments evaluate unstructured and semi-structured pruning across several model families, alongside downstream tasks and combinations with other compression methods.

Critical strengths
  1. Consistent empirical improvements over RIA. CRePE achieves lower PPL in all 16 model/pattern combinations in Table 3, with an average reduction of 0.43 under 2:4 sparsity. On LLaMA-2-13B, average downstream accuracy improves from 53.31% to 55.01% (Table 5). Appendix Table 10 extends the evaluation to additional architectures, including a substantial improvement on Mistral-Small 3 under 2:4 sparsity: 15.59 to 11.84 PPL.

  2. The coefficient-only ablation supports an additional contribution from the neighborhood term. Table 7(a) shows that tuning the row/column coefficients improves PPL from 11.29 to 11.06, while adding the 5×5 neighborhood further reduces it to 10.69. This provides useful evidence that the gains are not entirely attributable to reweighting the existing RIA terms.

Critical weaknesses
  1. The central claim about local 2D structure is insufficiently substantiated. The motivation in §3.1–3.2 treats the two-dimensional representation of a weight matrix as a reason to exploit neighboring entries. However, matrix dimensionality alone does not establish meaningful locality between channel indices. The proposed criterion is a relatively small extension of RIA, so demonstrating why this particular neighborhood contributes useful information is important to establishing its methodological significance. Table 7(a) shows that the added normalization helps, but does not distinguish local structure from the effect of adding another normalization term. Matched-size random or nonlocal neighborhoods are not evaluated. Consequently, the experiments support the usefulness of the heuristic more strongly than the claimed structural insight.

  2. The efficiency evaluation does not establish a competitive advantage for PHO’s proxy. Figure 2 reports a reduction from 648 to 21.9 minutes while nearly matching hill climbing’s PPL, but this comparison depends on a baseline whose evaluation budget and stopping criterion are not adequately specified. The alternative samplers in Figure 2 all use PHO; there is no comparison against direct PPL-based optimization at the same wall-clock budget. This comparison is particularly relevant because common positive scaling of the coefficients leaves the pruning mask unchanged (§4.2), leaving only two effective coefficient ratios to optimize. Furthermore, the strong global correlations in Table 1 do not establish reliable discrimination among the best candidates. The current evidence therefore does not isolate how much practical benefit comes from the proposed proxy.

Questions for authors
  1. Requested evidence: Does the local 5×5 neighborhood outperform matched-size random or nonlocal neighborhoods under the same tuning budget, with the original N groups held fixed?

  2. Requested comparison: At approximately 22 minutes, how does PHO compare with direct PPL-based random search or CMA-ES, accounting for the coefficients’ scale redundancy?

  3. Clarification: What evaluation count and stopping criterion were used for hill climbing? How were the configurations in the proxy-correlation study sampled, and does the correlation remain useful within the best-performing region?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.