- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper formulates layer-wise sparsity allocation as discrete optimization over candidate pruning states. DKNS assigns gradient-weighted deletion costs, UDPD constrains cumulative costs relative to uniform allocation, and a lexicographic MILP selects the allocation. Experiments cover five models, three local pruning rules, four sparsity levels, and six downstream tasks. The claimed contribution is a combination of NLL-sensitive cost modeling, depth-prefix constraints, and exact optimization of the resulting surrogate.
Critical strengths
- Substantial performance improvements in several settings. Tables 1 and 5 show the best PPL in most configurations.
- Practical deployment evidence. The SpInfer experiments demonstrate that the resulting non-uniform sparse models can achieve end-to-end acceleration, reaching approximately 1.68–1.69× speedup at 70% sparsity on the two evaluated OPT models.
- Feasible and flexible prefix constraints. UDPD preserves the uniform allocation as a feasible reference while allowing non-monotonic sparsity profiles. This provides a structured way to protect shallow layers without prescribing a fixed depth-dependent schedule.
Critical weaknesses
- The experiments do not establish the contribution of the proposed components. Section 3.3 varies only grid resolution and candidate range; there is no comparison against DKNS Cost-Only Allocation, an otherwise identical allocator with , or removal of the secondary objectives. Consequently, the results do not establish whether gradient weighting, UDPD, or exact selection explains the gains. A controlled component ablation and a comparison between predicted costs and measured held-out NLL changes are necessary, particularly for SparseGPT, whose compensation updates are excluded from the state cost.
- Downstream superiority over competitive allocators is much weaker than the headline suggests. In Table 2, Qwen3.5-4B/Wanda improves by 4.06 percentage points over Uniform, but only 0.14 points over DLP; Qwen3.5-2B/SparseGPT improves by 0.09 points over ATP. Conversely, Qwen3.5-4B/SparseGPT obtains 43.54% versus ATP’s 45.23%, despite achieving better PPL. With one calibration seed and no uncertainty estimates, these results support substantial improvements over Uniform in selected settings, but do not establish reliable downstream superiority over strong allocators. Repeated calibration samples and paired uncertainty estimates would clarify this distinction.
- The methodological novelty requires clearer clarification. Constrained layer-wise allocation using precomputed costs already appears in SPDY[1], while protecting early layers against propagated pruning errors motivates ATP[2]. The paper should clarify this distinction and demonstrate the additional benefit of UDPD over cost-only allocation and established depth-aware allocation strategies.
References
[1] Frantar et al. SPDY: Accurate Pruning with Speedup Guarantees
[2] Huang et al. Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective
Questions for authors
- What are the actual parameter-weighted sparsities and retained-weight counts for each Qwen3.5 allocation, and do the advantages persist when these budgets are matched?
- How much does UDPD improve held-out PPL over DKNS Cost-Only Allocation, and does DKNS predict actual pruning-induced NLL changes better than the corresponding activation-only cost?
8 comments
Response to W2 (Reviewer j7pF)
Response to W2 (Reviewer j7pF)
Thank you for suggesting paired reruns to assess the effect of calibration samples.
At a target mean layer sparsity of 70%, we repeat each comparison using five C4 calibration seeds. For each seed, we sample 128 sequences at each calibration stage. We report the mean accuracy across six zero-shot tasks (Mean) and WikiText-2 perplexity (PPL). For each method, the mean ± sample standard deviation (SD) summarizes its average performance and variability across the five seeds. denotes the mean paired difference, computed as NS-UDPD minus the baseline. Brackets show two-sided 95% paired confidence intervals (CIs), which quantify uncertainty in the estimated mean difference due to calibration sampling. Here, pp denotes percentage points.
Table 4. Paired rerun results for five C4 calibration seeds at a target mean layer sparsity of 70%.
| Model + Pruner | Baseline | Baseline Mean (%) ↑ | NS-UDPD Mean (%) ↑ | Baseline PPL ↓ | NS-UDPD PPL ↓ | Mean (pp) [95% CI] | PPL [95% CI] |
|---|---|---|---|---|---|---|---|
| Qwen3.5-4B + Wanda | DLP | 41.16 ± 0.14 | 41.35 ± 0.09 | 91.89 ± 0.13 | 38.54 ± 0.07 | +0.19 [0.12, 0.27] | −53.35 [−53.43, −53.27] |
| Qwen3.5-2B + SparseGPT | ATP | 39.15 ± 0.12 | 39.30 ± 0.11 | 64.92 ± 0.15 | 44.37 ± 0.10 | +0.15 [0.07, 0.23] | −20.55 [−20.74, −20.36] |
| Qwen3.5-4B + SparseGPT | ATP | 44.98 ± 0.18 | 43.79 ± 0.16 | 35.66 ± 0.09 | 25.88 ± 0.05 | −1.19 [−1.54, −0.83] | −9.78 [−9.89, −9.66] |
Across the six zero-shot tasks, NS-UDPD improves mean accuracy by 0.19 percentage points over DLP with Qwen3.5-4B + Wanda and by 0.15 percentage points over ATP with Qwen3.5-2B + SparseGPT. Both paired confidence intervals lie entirely above zero, supporting a mean accuracy advantage in these two settings across different calibration samples.
In the WikiText-2 language modeling evaluation, the 95% confidence intervals for the paired PPL differences lie entirely below zero in all three comparisons, supporting lower mean PPL for NS-UDPD in all three settings. NS-UDPD also has a lower sample SD for PPL than the corresponding baseline in each setting, indicating less PPL variability across these five sets of calibration samples.
These results suggest that NS-UDPD outperforms the baselines in most experimental configurations and tasks. We will report these results in Appendix J of the revised manuscript.
Response to W3 (Reviewer j7pF)
Response to W3 (Reviewer j7pF)
Thank you for pointing out the connections between NS-UDPD, SPDY, and ATP.
SPDY [1] uses local search to estimate layer-wise error scores and dynamic programming to select layer-wise sparsities that minimize accuracy loss subject to a target inference speedup. ATP [2] sets layer-wise sparsities as an arithmetic progression that increases with depth and searches for the common difference. The lower sparsities assigned to shallow layers mitigate the propagation and amplification of reconstruction errors in subsequent layers. NS-UDPD exactly matches the target mean layer sparsity and separately bounds the cumulative DKNS state cost at every depth prefix. At each depth , allocation must satisfy , where and are the cumulative DKNS state costs over the first layers under and Uniform at the same budget, respectively. Bounding each prefix prevents cost savings in deeper layers from offsetting excess costs in shallower prefixes. If the cumulative state cost of the first layers is below its bound, the DKNS state cost of layer may exceed that layer's cost under Uniform by up to . This slack gives the joint optimization room to adjust layer-wise sparsities while exactly meeting the budget and satisfying all prefix bounds.
We use SparseGPT on Qwen3.5-4B at a fixed target mean layer sparsity of 70%. All variants share the DKNS cost table and candidate grid. As shown in Table 5, Cost-Only minimizes only the full-depth DKNS state cost . UDPD (Uniform) constructs prefix bounds using 70% sparsity in every layer. ATP-shaped projects the ATP sparsity schedule, which increases with depth, onto the same grid while preserving the budget, then constructs prefix bounds from the projected sparsities. DKNS-ranked reorders the same projected sparsities, assigning higher sparsities to layers with smaller DKNS cost increases when sparsity is raised by one grid step from . Each of the three reference-based variants jointly selects layer-wise sparsities under its own prefix bounds, sequentially minimizing the full-depth DKNS state cost, the worst normalized depth-prefix ratio, and the layer-wise sparsity variance.
Compared with Cost-Only, which minimizes only the full-depth DKNS state cost, full UDPD (Uniform) reduces WikiText-2 PPL from 29.28 to 25.85 and increases six-task mean accuracy from 42.82% to 43.54%. At the same budget and with the same solver stages, UDPD (Uniform) reduces PPL by 22.12% relative to ATP-shaped and by 18.27% relative to DKNS-ranked, while increasing six-task mean accuracy by 0.88 and 0.81 percentage points, respectively. ATP-shaped assigns lower reference sparsities to shallow layers, which may tighten the corresponding prefix bounds and limit how much pruning less sensitive shallow layers can accommodate. DKNS-ranked orders the entire set of layer sparsities using one-step DKNS cost increases at , and this ordering may vary across sparsity levels. These results support using Uniform as the default reference in this setting and show the benefit of full UDPD over minimizing the full-depth DKNS state cost alone. We will include this analysis in Appendix H of the revised manuscript.
Table 5. Comparison of allocation strategies on Qwen3.5-4B with SparseGPT at a target mean layer sparsity of 70%. ✓ and ✗ indicate enabled and disabled, respectively.
| Allocation Strategy | Prefix Constraints | Stage 1 | Stage 2 | Stage 3 | WikiText-2 PPL ↓ | Six-Task Mean (%) ↑ |
|---|---|---|---|---|---|---|
| Cost-Only | ✗ | ✓ | ✗ | ✗ | 29.28 | 42.82 |
| UDPD (Uniform) | ✓ | ✓ | ✓ | ✓ | 25.85 | 43.54 |
| ATP-shaped | ✓ | ✓ | ✓ | ✓ | 33.19 | 42.66 |
| DKNS-ranked | ✓ | ✓ | ✓ | ✓ | 31.63 | 42.73 |
[1]Frantar, Elias, and Dan Alistarh. "SPDY: Accurate pruning with speedup guarantees." International conference on machine learning. PMLR, 2022.
[2]Huang, Weizhong, et al. "Determining layer-wise sparsity for large language models through a theoretical perspective." arXiv preprint arXiv:2502.14770 (2025).
Response to W1 and Q2 (Reviewer j7pF)
Thank you for your constructive feedback and for highlighting NS-UDPD’s ppl improvements and the flexibility of its depth-prefix constraints. We respond to your comments point by point below.
Response to W1 and Q2 (Reviewer j7pF)
Thank you for asking about each design's contribution. We conduct three controlled ablations on Qwen3.5-4B at a target mean layer sparsity of 70% and analyze how DKNS costs rank actual NLL changes. We will add these experiments results to Appendix G of the revised manuscript. In the tables, ✓/✗ means UDPD prefix constraints are enabled/disabled or the stage is run/skipped.
1. The Three DKNS Factors
With Wanda, we test all seven nonempty combinations of W, A, and G, along with Uniform as a control. W is the squared weight wℓ,m,i,j2, A the input-activation second moment Aℓ,m,j, and G the squared output-NLL-gradient energy Gℓ,m,i. All seven combinations use the same calibration samples and follow the same Wanda rule for generating candidate deletion masks. For each, we set omitted factors to 1, compute uniform prefix bounds from its costs, and run all three stages. Uniform targets 70% sparsity in every layer. Table 1 reports the results.
Under the same allocation procedure, restoring G to WA, A to WG, and W to AG lowers PPL from 107.34, 234.79, and 48.79 to 38.50 and raises Mean by 4.55, 1.85, and 1.17 percentage points, respectively. Both metrics improve in each comparison. The WA–WAG comparison supports collecting output-gradient statistics during allocation.
DKNS Costs and Actual NLL Changes. For each pruner, we score the same model-wide allocations using full-depth costs from DKNS (WAG) and its activation-weighted variant (WA, G=1) and measure their full-model NLL changes. Spearman’s ρ is the Pearson correlation between cost ranks and NLL-change ranks. Switching from WA to WAG raises ρ from 0.214 to 0.976 for Wanda and from −0.286 to 0.952 for SparseGPT. In both settings, full DKNS cost ranks agree more closely with actual NLL-change ranks than those of WA.
Table 1. DKNS factor ablation on Qwen3.5-4B with Wanda.
2. Cost-Only Comparison with Fixed DKNS Costs
For each pruner (Wanda or SparseGPT), both variants share the candidate set and full DKNS (WAG) cost table. NS-CostOnly disables UDPD prefix constraints and runs only Stage 1 to minimize the full-depth DKNS state cost. Full NS-UDPD enforces these constraints in all three stages.
Table 2 reports the results. Compared with NS-CostOnly, full NS-UDPD reduces PPL from 53.72 to 38.50 with Wanda and from 29.28 to 25.85 with SparseGPT, raising Mean by 0.27 and 0.72 percentage points, respectively. These results show the combined benefit of UDPD prefix constraints and the secondary optimization objectives.
Table 2. NS-UDPD versus NS-CostOnly with fixed WAG costs.
3. Secondary Optimization Objectives
For Wanda and SparseGPT, we fix full DKNS (WAG) costs and run the stages sequentially under UDPD prefix constraints. Stage 1 minimizes the full-depth DKNS state cost. Stage 2 preserves that optimum within numerical tolerance and minimizes the worst normalized depth-prefix ratio relative to the uniform reference allocation. Stage 3 preserves both prior optima within their respective tolerances and minimizes layer-wise sparsity variance.Table 3 reports the results.
For Wanda and SparseGPT, respectively, Stage 2 lowers PPL by 7.33 and 5.17 and raises Mean by 0.32 and 0.43 percentage points. Stage 3 further lowers PPL by 2.13 and 0.45 and raises Mean by 0.11 and 0.36 percentage points. Together, Stages 2 and 3 add 2.175 s and 0.232 s of MILP solve time over Stage 1 alone, respectively. The measured gains and small additional solve times support retaining both objectives.
Table 3. Three-stage ablation with fixed WAG costs and UDPD prefix constraints.