acceptodds
Under review as a conference paper at ICLR 2027

Sensitivity-Weighted Reconstruction Mitigates Priority Dilution in LLM Pruning

11 views

Abstract

One-shot pruning compresses large language models by matching sparse module outputs to dense counterparts. Existing reconstruction objectives assign unit weight to every calibration token occurrence, despite differences in how local reconstruction errors affect the global loss. We define occurrence sensitivity as the squared norm of the loss gradient at the module output. Across 11 models, the most sensitive 10% of occurrences account for a median of 63% of the sensitivity mass. This concentration exposes Uniform reconstruction to priority dilution: abundant low-sensitivity occurrences can reduce the reconstruction priority of more sensitive ones. We propose Sensitivity-Weighted Reconstruction (SWR), which converts raw occurrence sensitivities into normalized, clipped reconstruction weights and inserts them into existing solvers. With fixed pruning masks, adding the least sensitive half at unit weight raises perplexity in 8 of 11 models; replacing those unit weights with SWR weights reduces this degradation in 10 of 11. Across five pruning methods and 11 models at 70% unstructured sparsity, SWR lowers perplexity in 51 of 55 configurations, with a 3.0% median relative reduction, and improves zero-shot accuracy in 51 of 55 configurations. Further analysis finds that high-sensitivity occurrences are more strongly associated with the token being processed for K/V and generally with the token being predicted next for Q/O/FFN.

Primary area:
generative models
Keywords:
Large Language Models, Model Pruning, One-shot Pruning, Loss-aware Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9kiTOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates "priority dilution" in one-shot LLM pruning, where uniformly weighted reconstruction can overemphasize low-sensitivity calibration token occurrences. It proposes Sensitivity-Weighted Reconstruction (SWR), which derives occurrence sensitivity from squared module-output loss gradients, then normalizes and clips these values to reweight existing pruning solvers. Experiments span five pruning methods and 11 language models, including fixed-mask controls and structured-sparsity evaluations. At 70% unstructured sparsity, SWR improves WikiText-2 perplexity and mean zero-shot accuracy in 51 of 55 configurations, with a median 3.0% relative perplexity reduction. In addition, the authosrs provide detailed analyses and ablation studies to investigate module-specific sensitivity patterns. This paper is well-organized and technically sound.

Critical strengths
  1. Well-motivated and broadly applicable approach. The paper identifies priority dilution in uniform reconstruction and motivates occurrence sensitivity through an empirical-Fisher surrogate. SWR modifies only reconstruction weights, making it compatible with multiple existing pruning methods without changing their optimization procedures. This offers a simple and practical improvement.

  2. Extensive experimental validation with consistent improvements. Experiments across five pruning methods and 11 LLMs demonstrate consistent perplexity reductions and zero-shot accuracy improvements. Controlled experiments, including fixed-mask comparisons, alternative sensitivity statistics, and calibration-size analyses, provide further evidence supporting the effectiveness of sensitivity-weighted reconstruction.

Critical weaknesses
  1. Limited Baselines. This paper adopts five pruning methods to showcase the effectiveness of the proposed SWR. However, these five methods are fairly old and out-of-the-date. They are not the SOTA baselines. If the authors can provide the experimental results on more powerful method like DenoiseRotator[1]. It will strengthen this paper.

  2. Modest practical gains relative to computational overhead. Although SWR consistently improves pruning performance, the improvements are relatively modest, with a median 3.0% perplexity reduction and only 0.38 percentage points of zero-shot accuracy improvement. Meanwhile, computing sensitivities requires additional backward passes, increasing pruning time and GPU memory consumption. This raises concerns about the practical significance and cost-effectiveness of the proposed method.

  3. ** Ignore related works ** For one-shot pruning method, the authors ignore a very relavant work [2].

[1] DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration. NeurIPS 2025.

[2] One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models. ICASSP 2024.

Questions for authors
  1. Comparison with stronger baselines. Can the authors provide comparisons with more competitive baselines such as DenoiseRotator (NeurIPS 2025) or demonstrate whether SWR can further improve their performance?

  2. Ignore reference to related work. Can the authors discuss the "One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models." in the related work section?

4 comments

Dear Reviewer 9kiT,

We are grateful for your thoughtful and practical feedback. Your comments helped us assess the practical value of SWR more concretely. Please find our point-by-point responses below.

W1 and Q1: Integration with recent pruning methods

Thank you for suggesting DenoiseRotator and highlighting the practical cost. We integrated SWR with DenoiseRotator using SparseGPT and additionally with ROSE, evaluating both on Llama-3.1-8B and Qwen3-8B. The results are reported in the General Response; for your convenience, we reproduce them below.

For DenoiseRotator, we follow its setup and use the default SWR settings from our paper. We evaluate 50% and 70% unstructured sparsity and 2:4 sparsity.

Table R1. PPL (↓) and seven-task mean accuracy (%, ↑) results across sparsity settings. Each row reports DenoiseRotator → DenoiseRotator+SWR.

ModelSparsityWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Llama-3.1-8B50%7.58 → 7.5412.09 → 12.0112.49 → 12.4763.19 → 63.39
70%14.29 → 13.8520.49 → 20.1820.40 → 20.1555.19 → 55.69
2:49.81 → 9.6915.15 → 14.9515.24 → 15.0759.07 → 59.18
Qwen3-8B50%7.74 → 7.7213.25 → 13.1813.64 → 13.5965.32 → 65.65
70%11.74 → 11.3018.91 → 18.1519.80 → 19.1156.61 → 56.98
2:49.23 → 9.1215.27 → 15.1015.81 → 15.7162.41 → 62.63

In all six configurations, SWR lowers PPL on all three datasets, with a median relative reduction of 1.15% (range 0.16–4.02%), and improves seven-task mean accuracy, with a median increase of 0.28 percentage points (range 0.11–0.50).

For ROSE, we apply SWR to its reconstruction Gram matrix while preserving and rerunning its ordering algorithm.

Table R2. PPL (↓) and seven-task mean accuracy (%, ↑) results with ROSE at 50% and 70% unstructured sparsity. Each row reports ROSE → ROSE+SWR.

ModelSparsityWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Llama-3.1-8B50%9.34 → 9.0914.14 → 13.8814.58 → 14.4560.89 → 60.95
70%39.40 → 35.3552.80 → 47.7559.22 → 57.9444.15 → 45.16
Qwen3-8B50%8.49 → 8.4813.98 → 13.9714.83 → 14.9764.74 → 64.96
70%19.40 → 19.2228.80 → 28.4132.30 → 32.3550.20 → 50.48

SWR lowers PPL in 10 of 12 dataset comparisons, with a 1.14% median relative reduction, and improves seven-task mean accuracy in all four configurations, with a median increase of 0.25 percentage points.

In addition, DenoiseRotator exhibits priority dilution, the issue our paper identifies, and SWR mitigates it (General Response, Section 3).

W2: Practical improvements and computational cost

1. Improvement relative to more calibration data. SWR can outperform Uniform that uses four to eight times as many sequences. With DenoiseRotator, SWR with 128 reconstruction sequences outperforms Uniform with 1,024 on both models (Table R4). In the paper's calibration-scaling experiment, SWR with 256 calibration sequences outperforms Uniform with 1,024 in 9 of 11 models. On Llama-2-7B, this fourfold reduction lowers pruning time, excluding data preparation and model loading, from 1,019.6 to 498.6 seconds (51.1%) while reducing PPL from 37.35 to 36.71.

2. Measured cost. We prune Llama-2-7B with SparseGPT at 70% sparsity on one RTX PRO 6000 Blackwell in FP16, using 128 calibration sequences of 2,048 tokens and batch size 1. Times exclude data preparation, model loading, and evaluation. Appendix E used batch size 2 for sensitivity collection and reports a peak memory of 37.3 GiB for SWR on Llama-2-7B; this measurement uses batch size 1.

Table R11. Sensitivity-collection time, total pruning time, peak GPU memory, and WikiText-2 PPL (↓) on Llama-2-7B with SparseGPT at 70% sparsity.

ConfigurationSensitivity collection (s)Total time (s)Peak GPU memory (GiB)WikiText-2 ↓
Native SparseGPT—190.8219.4628.35
SWR, fresh sensitivities75.35270.5721.9026.16
SWR, cached sensitivities—193.2619.4626.16
Low-memory SWR, fresh sensitivities87.99288.4015.5026.16

On first use, SWR adds 79.75 seconds (41.8%) and 2.44 GiB (12.5%) of peak memory over Native SparseGPT. For the same dense model state, calibration tokens, loss and gradient definition, and numerical settings, one sensitivity collection can be reused across compatible masks, sparsity levels, and solvers. With cached sensitivities, SWR takes 2.44 seconds (1.28%) longer than Native SparseGPT at the same peak memory and produces the same pruning results as fresh-sensitivity SWR. If one collection is amortized over five compatible runs, the average overhead is 9.38%. The low-memory implementation, which combines activation checkpointing with CPU calibration buffers, lowers peak memory to 15.50 GiB, below Native SparseGPT, at 6.59% more time than standard SWR and with the same pruning results.

We will replace the measurements in Appendix E with those in Table R11 and distinguish first-use and cached costs.

W3 and Q2: Relation to sensitivity-aware mixed-sparsity pruning

Thank you for bringing this relevant work [4] to our attention. We will include it in Related Work. It combines curvature-based weight selection with Hessian-based sensitivity for allocating sparsity across layers or weight matrices. SWR addresses a different decision: how to weight calibration-occurrence reconstruction errors within each module. Our fixed-mask control experiments isolate this effect while keeping the pruning mask and sparsity allocation unchanged. We will clarify this distinction in the revised paper.

[4] Hang Shao, Bei Liu, and Yanmin Qian. One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models. ICASSP, 2024.

Reviewer_9kiTOpenReview ↗

Reply to Authors' Comments

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns. The new results provide useful evidence and further strengthen the paper's empirical contributions.

1. Comparisons with Stronger Pruning Methods. I appreciate the additional experiments integrating SWR with DenoiseRotator and ROSE. The consistent improvements across different models and sparsity settings provide further evidence of the general applicability of SWR. These results substantially address my concern regarding the lack of comparisons with more recent and competitive pruning methods.

2. Practical Improvements and Computational Overhead. The additional analysis of sensitivity caching, calibration efficiency, and GPU memory usage provides a more comprehensive understanding of the computational costs. In particular, the ability to reuse sensitivity statistics across compatible pruning configurations is a useful practical advantage. However, the performance gains on stronger pruning baselines remain relatively modest, and the additional computational overhead for first-time sensitivity collection is still non-negligible. Therefore, I believe the practical cost-effectiveness of SWR remains an important consideration.

3. Related Work. Thank you for clarifying the differences between SWR and One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models (ICASSP 2024). I find the explanation reasonable and appreciate the authors' commitment to including this relevant work in the revised paper.

Overall Assessment

Overall, I find the rebuttal convincing and acknowledge that the additional experiments have substantially addressed my original concerns. Therefore, I will maintain my rating of 3 (Weak Acceptance).

Thank you for your follow-up and for recognizing that the additional experiments address your main concerns. We appreciate your point about practical cost-effectiveness and will make this trade-off explicit in the revised paper. Thank you again for your time and constructive feedback.

Reviewer_VUekOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

They propose Sensitivity-Weighted Reconstruction to converts raw occurrence sensitivities into normalized, clipped reconstruction weights and inserts them into existing solvers. Across five pruning methods and 11 models at 70% unstructured sparsity, SWR lowers perplexity in 51 of 55 configurations, with a 3.0% median relative reduction, and improves zero-shot accuracy in 51 of 55 configurations.

Critical strengths
  1. The motivation for this work is clear, especially as shown in Figure 1 where the sensitivity is visualized.
  2. The experimental data in this work is comprehensive, especially the additional experiments presented in the appendix. Most experiments have demonstrated the effectiveness of the method.
Critical weaknesses
  1. My main concern is that the comparison methods presented in this article (OBS, Table 1; Wanda, Table 2) are outdated. Although SWR is used as a plugin, I hope the author considers integrating it with the latest methods.

  2. Recently, there have been many layer pruning methods. Can they be combined with the coarse-grained One-shot pruning method?

[1] Qianhao Yuan, Qingyu Zhang, Yanjiang Liu, Jiawei Chen, Yaojie Lu, Hongyu Lin, Jia Zheng, Xianpei Han, and Le Sun. ShortV: Efficient multimodal large language models by freezing visual tokens in ineffective layers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 329–339, 2025.

[2] Xinrui Chen, Hongxing Zhang, Fanyi Zeng, Yongxian Wei, Yizhi Wang, Xitong Ling, Guanghao Li, and Chun Yuan. Prune&Comp: Free lunch for layer-pruned LLMs via iterative pruning with magnitude compensation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 20316–20324, 2026.

  1. Although pruning can share the loss reconstruction with quantization, the authors cited numerous papers on quantization in the "Related Works" section. However, this work does not discuss the applicability of the proposed method to quantization. If the author believes that the validity of the discussion is limited to pruning, then the "Related Works" section may need to be reorganized.
Questions for authors

My main concern remains that the combination method is outdated.

  1. The author did not provide the performance data of the original model. Table 2-3, although achieving a relative performance advantage, suffered a significant performance loss compared to the original model, and I think the improved performance can be disregarded.
  2. In fact, the results of the WIKI-2 dataset on the 7B model were approximately 10. The methods combined by the author have already caused the model to collapse. I think the actual dialogue performance (not PPL/QA bench) of the testing model can demonstrate this.
  3. If the author can solve the problem of the comparison method, I would be more than willing to raise the score.
4 comments

Dear Reviewer VUek,

We sincerely appreciate your thorough review. Your suggestions prompted several additions that strengthen the paper: integration with recent pruning methods, dense-model references, and an evaluation of multi-turn instruction following. We address your comments one by one below.

W1 and Q3: Integration with recent pruning methods

Following your suggestion, we integrated SWR into two recent pruning methods, DenoiseRotator with SparseGPT and ROSE, on Llama-3.1-8B and Qwen3-8B. The results are reported in the General Response; for your convenience, we reproduce them below.

With DenoiseRotator, we evaluate 50% and 70% unstructured sparsity and 2:4 sparsity.

Table R1. PPL (↓) and seven-task mean accuracy (%, ↑) results across sparsity settings. Each row reports DenoiseRotator → DenoiseRotator+SWR.

ModelSparsityWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Llama-3.1-8B50%7.58 → 7.5412.09 → 12.0112.49 → 12.4763.19 → 63.39
70%14.29 → 13.8520.49 → 20.1820.40 → 20.1555.19 → 55.69
2:49.81 → 9.6915.15 → 14.9515.24 → 15.0759.07 → 59.18
Qwen3-8B50%7.74 → 7.7213.25 → 13.1813.64 → 13.5965.32 → 65.65
70%11.74 → 11.3018.91 → 18.1519.80 → 19.1156.61 → 56.98
2:49.23 → 9.1215.27 → 15.1015.81 → 15.7162.41 → 62.63

In all six configurations, SWR lowers PPL on all three datasets, with a median relative reduction of 1.15% (range 0.16–4.02%), and improves seven-task mean accuracy, with a median increase of 0.28 percentage points (range 0.11–0.50).

We apply SWR to ROSE's reconstruction Gram matrix while preserving and rerunning its ordering algorithm.

Table R2. PPL (↓) and seven-task mean accuracy (%, ↑) results with ROSE at 50% and 70% unstructured sparsity. Each row reports ROSE → ROSE+SWR.

ModelSparsityWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Llama-3.1-8B50%9.34 → 9.0914.14 → 13.8814.58 → 14.4560.89 → 60.95
70%39.40 → 35.3552.80 → 47.7559.22 → 57.9444.15 → 45.16
Qwen3-8B50%8.49 → 8.4813.98 → 13.9714.83 → 14.9764.74 → 64.96
70%19.40 → 19.2228.80 → 28.4132.30 → 32.3550.20 → 50.48

SWR lowers PPL in 10 of 12 dataset comparisons, with a 1.14% median relative reduction, and improves seven-task mean accuracy in all four configurations, with a median increase of 0.25 percentage points.

In addition, DenoiseRotator exhibits priority dilution, the issue our paper identifies, and SWR mitigates it (General Response, Section 3).

W2: Combination with Prune&Comp and the scope of ShortV

Prune&Comp. For Prune&Comp on Llama-3-8B (32 → 27 layers; fixed deletion order), we apply sensitivity weighting to the calibration statistics of its magnitude compensation. Both the native and the sensitivity-weighted compensation are then followed by the same reconstruction of the preceding block's MLP down-projection, a relative-error least-squares update following ReplaceMe [3].

Table R8. PPL (↓) and seven-task mean accuracy (%, ↑) results with Prune&Comp on Llama-3-8B.

ConfigurationWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Native Prune&Comp11.8716.9817.4659.45
Native compensation + reconstruction9.8416.3116.0959.33
Sensitivity-weighted compensation + reconstruction9.8216.2916.0759.52

Relative to native compensation with the same reconstruction, sensitivity weighting lowers all three PPLs by 0.02 and raises mean accuracy by 0.19 percentage points.

ShortV. Thank you for mentioning ShortV. SWR weights calibration occurrences when model weights are updated through reconstruction or compensation. ShortV instead freezes visual-token updates in selected layers without changing model weights, so SWR does not directly apply to its native procedure.

W3: Quantization references

Thank you for pointing out this mismatch between our Related Work and the scope of our experiments. We cite these works because reconstruction-based pruning and quantization share local output-reconstruction objectives, while our method and experiments address pruning. We will revise the paper accordingly: the introduction will state that our method and experiments address one-shot pruning; the Related Work section will keep only the quantization methods directly relevant to the weighted reconstruction objective and explain their relation to SWR.

Q1: Missing dense-model results and whether SWR's improvements matter despite performance degradation

Missing dense-model results. Thank you for pointing out that Tables 2–3 of the paper lack these references. We report the dense-model results for all 11 models below and will include them in the revised main tables, so readers can assess both the gap to the dense models and the improvement from SWR.

Table R9. WikiText-2 PPL (↓) and seven-task mean accuracy (%, ↑) results on the dense models.

ModelWikiText-2 ↓Mean acc. ↑
LLaMA-1-7B5.6762.06
LLaMA-1-13B5.0964.42
LLaMA-1-30B4.1067.47
Llama-2-7B5.4762.26
Llama-2-13B4.8865.01
Llama-3-8B6.1465.96
Llama-3.1-8B6.2466.57
Mistral-7B5.2566.52
OPT-6.7B10.8654.65
Qwen2.5-7B6.8565.39
Qwen3-8B6.9965.88

whether SWR's improvements matter. At 70% sparsity, all tested one-shot pruning methods remain far from the dense models (e.g., SparseGPT raises the WikiText-2 PPL of Llama-3.1-8B from 6.24 to 43.24; Table R6), so pruning methods are compared at the same sparsity by how much of this gap they close. SWR narrows this gap without retaining additional weights or adding inference-stage operations. On Llama-3.1-8B at 70% sparsity, SparseGPT+SWR reaches 39.23 WikiText-2 PPL, comparable to ROSE without SWR (39.40); adding SWR to ROSE lowers PPL further to 35.35 (Table R2). With DenoiseRotator, whose pruned models are closer to their dense counterparts, SWR still lowers PPL and improves mean accuracy in all settings (Table R1).

Q2: Generated responses and multi-turn instruction following

To assess dialogue ability directly, we evaluate generated responses rather than PPL or multiple-choice QA. IFEval scores single-turn responses against verifiable instructions, and Multi-IF scores three-turn conversations in which each model continues from its own preceding responses. We prune Llama-3.1-8B-Instruct to 50% sparsity and evaluate 541 IFEval prompts and 909 English Multi-IF conversations with official scoring, greedy decoding, and a 2,048-token output limit. Thirteen Multi-IF conversations contain only two annotated turns, leaving 896 examples for turn 3; all models use the same subset for each turn.

Table R10. WikiText-2 PPL (↓) and instruction-following accuracy (%, ↑) results on Llama-3.1-8B-Instruct. Pruned models use 50% sparsity. Accuracy is strict prompt-level, scored separately for each Multi-IF turn.

ConfigurationWikiText-2 ↓IFEval ↑Multi-IF T1 ↑T2 ↑T3 ↑
Dense7.2174.6876.6863.2648.44
DenoiseRotator8.5271.7274.0457.6540.85
DenoiseRotator+SWR8.3972.8373.9358.7541.41

The pruned models retain 96–98% of the dense model's IFEval accuracy and at least 84% of its Multi-IF accuracy in every turn. SWR improves IFEval by 1.11 percentage points and Multi-IF turns 2–3 by 1.10 and 0.56 percentage points; turn 1 decreases by 0.11 points, a difference of one conversation out of 909.

[3] ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization. NeurIPS 2025.

I acknowledge that the additional experiments have substantially addressed my original concerns. Therefore, I will raise my score to 3 (Weak Acceptance). Furthermore, please incorporate the supplementary experiments into the final version.

Dear Reviewer VUek,

Thank you for taking the time to read the additional experiments and for updating your assessment. We are glad that the new results addressed your concerns. Your suggestions helped us improve the paper, particularly the evaluation with recent pruning methods and the comparison with the dense models.

We have uploaded the revised PDF and incorporated the supplementary experiments. Blue marks the revisions in response to your comments, including the dense-model results in Tables 2 and 3, the revised quantization discussion in Related Work, and the Prune&Comp and instruction-following experiments in Appendix Q.3 and Q.4. The DenoiseRotator and ROSE experiments in Appendix Q.1 and Q.2 are marked in purple because they address a concern shared by multiple reviewers.

Reviewer_bkwcOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Sensitivity-Weighted Reconstruction (SWR) to improve one-shot pruning of large language models. The core argument is that uniform reconstruction assigns equal importance to all calibration token occurrences, ignoring their different impacts on the global loss. To address this, the paper introduces a sensitivity-based weighting strategy that uses the squared norm of the loss gradient at each module output to construct normalized and clipped reconstruction weights, and incorporates them into existing pruning solvers. Experiments are conducted on 11 LLMs across five mainstream pruning methods, using perplexity as well as several zero-shot benchmarks.

Critical strengths
  1. The motivation is reasonable, as token importance is an important factor to consider in pruning and reconstruction-based error compensation.
  2. The experimental evaluation is extensive and covers multiple aspects, including comparisons across different models and pruning methods, as well as extensive ablation studies and sensitivity analyses.
Critical weaknesses
  1. The methodological novelty is limited. The core idea of using global loss gradients to guide local reconstruction and construct a weighted Gram matrix has already been explored in GuidedQuant [1] for quantization. Although this paper extends the idea to pruning and introduces different weighting strategies, the methodological novelty remains limited.

  2. The experimental results are not sufficiently convincing. The improvements are generally larger at high sparsity levels (e.g., 70%), while gains at moderate sparsity levels (e.g., 50%) are often marginal. Moreover, the authors frequently use the number of improved settings (e.g., 51 out of 55 configurations) to demonstrate the effectiveness of SWR. However, this metric does not adequately reflect the magnitude of improvements or regressions, making it difficult to assess the actual benefits of the proposed method.

[1] Kim et al. GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance. ICML, 2025.

Questions for authors
  1. Could the authors clarify how the proposed SWR differs theoretically from GuidedQuant [1], beyond per-sequence normalization, magnitude clipping, and its application to pruning? Could the authors directly apply GuidedQuant to pruning and compare its downstream task performance with SWR across different models and sparsity levels?
  2. How were the candidate clipping thresholds and their search ranges initially determined? Although the Appendix provides some ablation results, could the authors conduct a more comprehensive sensitivity analysis across different sparsity levels, pruning methods (particularly SparseGPT), and model architectures?
  3. Could the authors extend the evaluation to larger models, such as LLaMA2-70B, to further demonstrate the scalability and effectiveness of SWR?

[1] Kim et al. GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance. ICML, 2025.

7 comments

Dear Reviewer bkwc,

Thank you for the time and care you devoted to reviewing our paper. Your questions on the relation to GuidedQuant and the choice of clipping thresholds led us to clarify the derivation and add direct comparisons, and your comment on how we present the results will make the revised paper easier to assess. Below we respond to each point in turn.

W1 and Q1: Differences from GuidedQuant and direct comparison in pruning

1. Theoretical differences.

Beyond normalization, clipping, and the pruning setting, the two methods differ in how the weighting is derived. GuidedQuant ignores cross-output-channel Fisher interactions and averages channel-specific blocks within groups, yielding an approximation in which more groups preserve finer channel-specific information at higher cost. SWR starts from the occurrence-level surrogate (gt⊤et)2(g_t^\top e_t)^2, where gtg_t is the output gradient and ete_t the reconstruction residual, and bounds it by ∥gt∥22∥et∥22\lVert g_t\rVert_2^2\lVert e_t\rVert_2^2 (Section 3.1). Because ∥gt∥22I\lVert g_t\rVert_2^2 I is the smallest multiple of the identity that dominates gtgt⊤g_t g_t^\top, this bound yields one Gram matrix shared by all output rows. With one group, the two raw Gram matrices are proportional: GuidedQuant obtains this matrix by group averaging, whereas SWR obtains it from this occurrence-level bound.

2. Priority dilution and the role of normalization and clipping.

Beyond the form of the weighting, our paper identifies and isolates priority dilution. With the mask and the reconstruction weights of the more-sensitive half fixed, adding the least-sensitive half of occurrences at unit weight raises PPL in 8 of 11 models (median 0.97%); weighting the same occurrences by sensitivity instead lowers PPL in 10 of 11 models (median 1.06%) (Table 7 of the paper). A shuffled control with the same weight distribution lowers PPL in only 3 of 11 models, compared with 10 of 11 models for SWR (Table 1 of the paper), so the weights must be assigned to the correct occurrences. Normalization and clipping determine how effectively the weights address this issue: relative to mean-one, unclipped weights, the default clipping lowers SparseGPT's PPL in all six settings (Table R6), and relative to GuidedQuant's raw weighting (G=1G=1), SWR improves mean accuracy in all eight configurations (Table R5).

3. Direct comparison in pruning.

We adapted GuidedQuant's weighting with G=1G=1 and G=4G=4 to Wanda+OBS and SparseGPT.

Table R5. Seven-task mean accuracy (%, ↑) results on Llama-3.1-8B and Qwen3-8B. G=1G=1 and G=4G=4 denote GuidedQuant.

ModelMethodSparsityG=1G=1G=4G=4SWRTime ratio (G=4G=4 / SWR)
Llama-3.1-8BWanda+OBS50%60.2760.3760.923.34×
70%37.4737.7537.863.41×
SparseGPT50%60.2660.8161.293.58×
70%42.4042.6944.113.93×
Qwen3-8BWanda+OBS50%62.8762.9563.253.50×
70%42.7942.7643.643.53×
SparseGPT50%63.6163.5064.333.55×
70%48.1748.5350.523.92×

SWR achieves higher accuracy in all eight configurations, with a median increase of 0.79 percentage points over G=1G=1 and 0.69 over G=4G=4. Since G=1G=1 uses SWR's raw Gram matrix up to scale, the gap over G=1G=1 measures the effect of normalization and clipping. Increasing GuidedQuant to four groups does not close the gap and requires 3.34–3.93× SWR's reconstruction time.

W2: Improvements at 50% sparsity and presentation of results

One explanation for the smaller improvements at 50% is that the pruned models are closer to the dense models, leaving less to recover. In mask-generation methods (Wanda, GBLM, and DSnoT), SWR changes only the reconstruction on a fixed mask, and their median WikiText-2 PPL reductions at 50% range from 0.01% to 0.43%. In joint methods such as SparseGPT, SWR also changes the mask, and the improvement at 50% is larger: in the paper's sparsity sweep, SWR lowers SparseGPT's WikiText-2 PPL at 50% by a median of 1.46%, in 10 of 11 models.

Regarding the presentation of results, Section 4.2 of the paper reports a median relative WikiText-2 PPL reduction of 3.0% and a median seven-task mean accuracy increase of 0.38 percentage points at 70% unstructured sparsity. Tables 2 and 3 of the paper provide the per-configuration results. We agree that summarizing results by the number of improved configurations made their magnitude difficult to assess. In the revision, we will replace these counts in the abstract and Section 4.2 with the median and range of relative change.

Q2: Clipping-threshold selection and sensitivity

We chose the candidate ranges heuristically for sensitivities normalized to mean one: lower bounds {0,0.01,0.1,0.2}\{0,0.01,0.1,0.2\} test zero and small positive floors, while upper bounds {5,10,20}\{5,10,20\} vary the strength of outlier clipping. Across intervals, Table 11 of the paper shows a median relative PPL spread of 0.87%. No single interval consistently performs best across models and methods. These results support retaining [0.2,5][0.2,5] as a fixed default without tuning thresholds for individual configurations.

Beyond the Wanda, GBLM, and DSnoT results in Table 11 of the paper, we extend the same grid to SparseGPT on Llama-3.1-8B, Qwen2.5-7B, and OLMo-2-7B (an additional architecture beyond our 11 models) at 50% and 70% sparsity.

Table R6. WikiText-2 PPL (↓) results with SparseGPT under different clipping settings. Each cell reports 50% / 70% sparsity.

Reconstruction weightsLlama-3.1-8BQwen2.5-7BOLMo-2-7B
Uniform (native SparseGPT)9.34 / 43.248.43 / 20.949.06 / 26.04
Mean-one, unclipped9.22 / 40.288.40 / 19.938.54 / 23.72
[0.2,∞)[0.2,\infty), lower only9.26 / 40.858.39 / 19.618.20 / 23.25
[0,5][0,5]9.10 / 39.358.35 / 18.978.21 / 21.98
[0,10][0,10]9.11 / 39.008.37 / 19.038.21 / 22.09
[0,20][0,20]9.17 / 39.028.37 / 19.158.24 / 22.23
[0.01,5][0.01,5]9.09 / 39.498.35 / 19.078.20 / 22.15
[0.01,10][0.01,10]9.12 / 39.068.35 / 19.108.21 / 21.99
[0.01,20][0.01,20]9.11 / 39.578.37 / 19.158.21 / 22.40
[0.1,5][0.1,5]9.13 / 39.188.38 / 18.998.26 / 22.63
[0.1,10][0.1,10]9.15 / 38.658.37 / 19.078.22 / 22.65
[0.1,20][0.1,20]9.15 / 37.518.37 / 19.248.21 / 22.63
[0.2,5][0.2,5], default9.14 / 39.238.37 / 19.468.40 / 22.51
[0.2,10][0.2,10]9.14 / 37.768.37 / 19.148.28 / 22.85
[0.2,20][0.2,20]9.19 / 37.498.38 / 19.298.23 / 22.63

All twelve intervals improve on Uniform in all six settings, and in each setting the PPL range across intervals is smaller than the improvement of the worst interval over Uniform. Thus, SWR's improvement is not specific to the default [0.2,5][0.2,5] interval.

Q3: Evaluation on Llama-2-70B

We evaluated the requested Llama-2-70B with SparseGPT at 50% and 70% unstructured sparsity.

Table R7. PPL (↓) and seven-task mean accuracy (%, ↑) results on Llama-2-70B with SparseGPT. Pruned rows report Native → SWR.

ConfigurationWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Dense3.325.7724.2569.31
50% sparsity4.24 → 4.206.51 → 6.4826.50 → 26.2868.44 → 68.60
70% sparsity9.35 → 9.3412.77 → 12.7350.73 → 49.7860.76 → 60.95

SWR lowers PPL on all three datasets at both sparsity levels and improves mean accuracy by 0.16 and 0.19 percentage points, respectively.

Dear Reviewer bkwc,

Following up on our earlier reply, we have uploaded the revised PDF, with changes addressing your comments marked in red. In response to your concern about novelty, we revised the manuscript to clarify what Sensitivity-Weighted Reconstruction (SWR) contributes beyond GuidedQuant. We also explain why the improvements are often modest at 50% sparsity and report their magnitude across pruning methods and sparsity levels.

What does SWR contribute beyond GuidedQuant? SWR addresses a problem we identify in one-shot pruning.

  1. Our central finding is priority dilution. When all calibration tokens receive equal weight, many less sensitive tokens can reduce the influence of more sensitive ones during reconstruction. With the pruning mask and the weights on the more sensitive half fixed, adding the less sensitive half at unit weight raises perplexity (PPL) by a median of 0.97%. Applying SWR weights to those same tokens lowers PPL by a median of 1.06% relative to unit weighting (Section 5).
  2. SWR derives its weights from the effect of each reconstruction error on the loss. GuidedQuant averages channel-wise Fisher approximations within each output group. SWR bounds the squared first-order loss change caused by reconstruction error at each token occurrence. This derivation gives one sensitivity weight per occurrence for the whole module (Section 3.1).
  3. SWR improves accuracy over GuidedQuant applied directly to pruning. We compared SWR and GuidedQuant using Wanda+OBS and SparseGPT on two models at 50% and 70% sparsity. SWR improves seven-task mean accuracy by a median of 0.79 percentage points over one-group GuidedQuant and 0.69 percentage points over four-group GuidedQuant (Section 3.2). Four-group GuidedQuant takes 3.54 times as long as SWR for reconstruction, based on the median of the eight time ratios in Table R5 of our earlier reply.

Why are the improvements often smaller at 50% sparsity? Pruning causes less degradation at 50%, and the amount SWR can correct depends on whether it can change which weights are kept.

  1. Pruned models are closer to their dense counterparts at 50% sparsity. The median WikiText-2 PPL gap to the dense models is 1.5 points at 50% sparsity and 25.8 points at 70%.
  2. Wanda+OBS uses SWR only to adjust the values of the retained weights. We fix the set of weights retained by Wanda to isolate the effect of adjusting their values. This limits what SWR can correct, and the median PPL reduction at 50% sparsity is 0.10%.
  3. SparseGPT improves more when SWR can also change which weights are kept. Using SWR to select weights and adjust their values yields a median PPL reduction of 1.46% at 50% sparsity. In the SparseGPT ablation at 70% sparsity, the median PPL reduction is 4.92% when SWR is used for weight selection and reconstruction, 1.94% when it is used only for weight selection, and 0.30% when it is used only for reconstruction.

The revised manuscript incorporates the requested experimental results, including the expanded SparseGPT clipping study and the Llama-2-70B evaluation.

As the discussion deadline approaches, we would appreciate hearing whether these results and revisions address your concerns. We would be happy to answer any remaining questions.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response(Part I)

We thank all reviewers for their constructive feedback. A shared concern is whether SWR can be integrated with more recent pruning methods while retaining its improvements. Additional experiments show that it can. Tables in this response are numbered R1–R11 to distinguish them from the tables of the paper.

  • Following the reviewers' suggestion, we evaluated SWR with DenoiseRotator [1], a recent pruning method, using SparseGPT on Llama-3.1-8B and Qwen3-8B. SWR improves DenoiseRotator across the tested sparsity settings. We further evaluated SWR with ROSE [2], another recent pruning method, and observed improvements in most evaluated configurations.
  • DenoiseRotator also exhibits priority dilution, the issue our paper identifies: adding lower-ranked occurrences at unit weight raises PPL, whereas weighting them by sensitivity lowers it.
  • We release anonymized code supporting the experiments in Tables R1, R2, R8, and R11.

1. SWR improves DenoiseRotator across sparsity settings.

We evaluate 50% and 70% unstructured sparsity and 2:4 sparsity.

Table R1. PPL (↓) and seven-task mean accuracy (%, ↑) results across sparsity settings. Each row reports DenoiseRotator → DenoiseRotator+SWR.

ModelSparsityWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Llama-3.1-8B50%7.58 → 7.5412.09 → 12.0112.49 → 12.4763.19 → 63.39
70%14.29 → 13.8520.49 → 20.1820.40 → 20.1555.19 → 55.69
2:49.81 → 9.6915.15 → 14.9515.24 → 15.0759.07 → 59.18
Qwen3-8B50%7.74 → 7.7213.25 → 13.1813.64 → 13.5965.32 → 65.65
70%11.74 → 11.3018.91 → 18.1519.80 → 19.1156.61 → 56.98
2:49.23 → 9.1215.27 → 15.1015.81 → 15.7162.41 → 62.63

In all six configurations, SWR lowers PPL on all three datasets, with a median relative reduction of 1.15% (range 0.16–4.02%), and improves seven-task mean accuracy, with a median increase of 0.28 percentage points (range 0.11–0.50).

2. SWR also improves ROSE.

We apply SWR to ROSE's reconstruction Gram matrix while preserving and rerunning its ordering algorithm.

Table R2. PPL (↓) and seven-task mean accuracy (%, ↑) results with ROSE at 50% and 70% unstructured sparsity. Each row reports ROSE → ROSE+SWR.

ModelSparsityWikiText-2 ↓C4 ↓PTB ↓Mean acc. ↑
Llama-3.1-8B50%9.34 → 9.0914.14 → 13.8814.58 → 14.4560.89 → 60.95
70%39.40 → 35.3552.80 → 47.7559.22 → 57.9444.15 → 45.16
Qwen3-8B50%8.49 → 8.4813.98 → 13.9714.83 → 14.9764.74 → 64.96
70%19.40 → 19.2228.80 → 28.4132.30 → 32.3550.20 → 50.48

SWR lowers PPL in 10 of 12 dataset comparisons, with a 1.14% median relative reduction, and improves seven-task mean accuracy in all four configurations, with a median increase of 0.25 percentage points.

AuthorsOpenReview ↗

General Response(Part II)

3. Priority dilution with DenoiseRotator.

At 70% sparsity, we repeat the prefix experiment in Figure 2 of the paper, using identical sensitivity-ranked subsets from 128 calibration sequences; Table R3 summarizes the expansion from the top 20% to all occurrences.

Table R3. WikiText-2 PPL (↓) results with DenoiseRotator at 70% sparsity.

ModelUniform: 20% → 100%SWR: 20% → 100%
Llama-3.1-8B13.92 → 14.2914.06 → 13.85
Qwen3-8B11.49 → 11.7411.47 → 11.30

Adding lower-ranked occurrences increases PPL under Uniform but decreases it under SWR, consistent with the priority-dilution effect.

We further examine whether more reconstruction data closes the gap between Uniform and SWR. Rotations learned from 128 sequences are held fixed across budgets. Uniform and SWR use identical reconstruction sequences of 2,048 tokens each. For budget NN, SWR reads only the corresponding NN sequences' sensitivities from a shared 1,024-sequence cache.

Table R4. WikiText-2 PPL (↓) results with DenoiseRotator at different reconstruction budgets and 70% sparsity. Each row reports Uniform → SWR.

Reconstruction sequencesLlama-3.1-8BQwen3-8B
3216.12 → 15.8112.83 → 12.56
6414.94 → 14.7711.99 → 11.67
12814.29 → 13.8511.74 → 11.30
51214.17 → 13.4811.58 → 11.04
1,02414.12 → 13.4011.51 → 11.00

Increasing the reconstruction budget improves Uniform but does not eliminate SWR's advantage: from 32 to 1,024 sequences, SWR's relative PPL reduction grows from 1.9% to 5.1% on Llama-3.1-8B and from 2.1% to 4.4% on Qwen3-8B. On both models, SWR with 128 reconstruction sequences outperforms Uniform with 1,024.

[1] DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration. NeurIPS 2025.

[2] ROSE: Reordered SparseGPT for More Accurate One-Shot Large Language Models Pruning. CPAL 2026.

We thank the reviewers for their constructive feedback. We have uploaded the revised PDF.

  1. Method and main results. Section 3 clarifies the theoretical relationship between SWR and GuidedQuant and summarizes a direct comparison of the two methods in pruning experiments. Section 4.2 explains how the gains vary with sparsity and pruning method.
  2. Dense baselines and supplementary experiments. Tables 2 and 3 now include the dense-model results. Appendix Q includes experiments with DenoiseRotator, ROSE, and Prune&Comp, as well as the instruction-following evaluation. The appendix also includes the expanded SparseGPT clipping study and the Llama-2-70B evaluation.
  3. Pruning cost. The main text reports pruning time and peak GPU memory with and without SWR for all five methods. Appendix E describes the timing protocol and reports results for sensitivity reuse and the low-memory configuration.

In the revised PDF, red marks changes in response to Reviewer bkwc, blue marks changes in response to Reviewer VUek, and teal marks changes in response to Reviewer 9kiT. Purple marks revisions that address shared concerns or incorporate experimental results from our General Response.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.