acceptodds
Under review as a conference paper at ICLR 2027

-Support Pruning: Tight Convex Relaxation and Principled Optimization for N:M LLM Pruning

Abstract

Semi-structured N:M sparsity—keeping N of every M weights, the form that supported sparse kernels can accelerate—is the dominant hardware-realizable way to prune large language models. Choosing which N weights survive is combinatorial, and current methods approximate that choice rather than characterize it: one-shot pruners commit to a ranking fixed before optimization, while learnable-mask methods optimize a nonconvex surrogate at the cost of gradients through the model. We show that this choice has a convex geometry of its own. Per block, the convex hull of the N-sparse unit Euclidean ball is exactly the unit ball of the k-support norm with k=N: the tight per-block relaxation and, among the classical structured penalties we consider, the one that carries the budget N itself; for N greater than or equal to 2, group and exclusive lasso do not. The relaxation is constructive. Its exact closed-form proximal operator is governed by one water level per block, found in a single sorted pass: weights above it survive and are shrunk, while the rest become exactly zero, followed by a least-squares refit that restores the survivors. It turns the layerwise pruning objective into a convex program solved in the one-shot post-training setting and certifies per block that rounding the relaxed solution back to N:M incurs no additional reconstruction loss. Magnitude and activation-scaled saliency reappear as zero-iteration gates, and under diagonal curvature with a separation condition, hardening and refitting the solution returns the combinatorial optimum. Read across matrices rather than within a block, the same water level prices the sparsity budget, so one convex object decides both which weights to keep and where sparsity is spent. Across seven models from four architecture families, KSP achieves the best accuracy in six of seven columns under 2:4 sparsity and the best results in eight of ten cells for each metric at 75% sparsity. On a 70B model, allocating the budget improves over uniform N:M sparsity at matched measured latency, and pricing the allocation by latency provides further gains.

Primary area:
optimization
Keywords:
N:M sparsity, large language model pruning, structured pruning, convex optimization, k-support norm

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_x8VMOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes K-Support Pruning (KSP), a framework for post-training N:M pruning of LLMs. It uses blockwise, activation-weighted squared k-support norms to formulate a convex regularized layer reconstruction problem, followed by mask hardening and a masked least-squares refit. The theoretical analysis provides a weighted proximal operator, a conditional lossless-hardening certificate, and support recovery under diagonal curvature. Extending the norm to fractional budgets yields a water-level interpretation for allocating sparsity across matrices, with discrete patterns selected under parameter or latency budgets. Evaluation spans seven models across four architecture families, plus a Llama-3.1-70B deployment check, and includes uniform sparsity comparisons, component ablations, and measured latency experiments. KSP generally achieves competitive or improved quality, while the ablations highlight the substantial contributions of refitting and budget allocation.

Critical strengths
  1. The paper provides a unified framework for N:M pruning and sparsity allocation through the k-support norm. The weighted proximal operator and water-level interpretation make the framework computationally constructive.
  2. Experiments cover multiple model families and sparsity levels, with shared-refit and latency-matched controls. These comparisons demonstrate competitive pruning quality and the practical value of compensation and budget allocation.
Critical weaknesses
  1. The theoretical guarantees are narrower than some claims suggest: tightness applies to the scaled per-block sparse ball, while exact recovery requires diagonal curvature. The paper also does not report how often the lossless-hardening certificate holds in its LLM experiments.
  2. The comparisons do not fully isolate the benefit of the proposed regularizer under matched computation and compensation. For example, on Llama-3.2-3B, KSP improves perplexity over ALPS from 18.20 to 17.93 but takes 2200 versus 1570 seconds, while the shared-refit ablation excludes ALPS and SparseGPT.
Questions for authors
  1. Clarification of novelty: Which parts of the variational representation and proximal algorithm are new beyond the box-norm framework of McDonald et al. (2016) [1], which already provides an O(dlog⁡d)O(d\log d) algorithm for the squared-norm prox? Please distinguish the weighted extension and pruning-specific results from inherited machinery.

  2. Evidence connecting theory to practice: What fraction of blocks satisfy the hardening certificate at the actual returned iterates? Please report the associated hardening loss and convergence residuals, and clarify which guarantees apply to the finite-budget, validation-selected ADMM output.

  3. Evidence for attribution: Under matched optimization time and the same final masked-CG refit, how does KSP compare with ALPS and SparseGPT on representative 2:4 and 75%-sparsity settings? Does the advantage persist across calibration seeds?

  4. Clarification of allocation: Which weights are used to compute GT(n)G_T(n), and is the jointly convex weight–budget program actually solved, or is allocation computed with fixed weights before pruning? This distinction is important for interpreting the allocation optimality claim and its relationship to the deployed procedure.

[1] McDonald A M, Pontil M, Stamos D. New perspectives on k-support and cluster norms[J]. Journal of Machine Learning Research, 2016, 17(155): 1-38.

1 comment
AuthorsOpenReview ↗

Response to Reviewer x8VM

Thank you for the precise questions; each is answered below and in the revised PDF.

W1 — Scope of the guarantees. We agree, and now state the scope where each claim is made: tightness is per block, for the ℓ2-bounded N-sparse set (abstract, Prop. 1); exact recovery holds under diagonal curvature with a separation condition (abstract, §3.3); and the hardening result is a per-block condition (Cor. 1), whose measured frequency is reported under Q2.

Q1 — Novelty beyond McDonald et al. (2016). Delineated next to Thm. 1. Inherited: the variational box-norm form, the unweighted squared-norm prox with its O(d log d) algorithm, and non-integer budgets (Argyriou et al. 2012; McDonald et al. 2016). New: identifying this geometry as the tight per-block relaxation of N:M sparsity, and the pruning-specific results it yields — the prox with separate loss and penalty weights and its selection statistic ρ_j; matched scaling (the weighting, unique up to scale, under which ρ_j ranks by the loss); the lossless-hardening condition; support recovery under diagonal curvature; and the zero-iteration limit that recovers magnitude and Wanda.

Q2 — Certificate frequency, hardening loss, residuals, and what holds for the returned output (Tabs. 23–24). The reported solver (Alg. 3, App. C) is a splitting with a ridge variable B, a masked variable D given by exact blockwise support selection (the hardened, debiased counterpart of the prox step) and a dual V; B is the variable hardened. Diagnostics at the returned iterate (seed 0; residuals are medians over pruned matrices, median 33 iterations):

Model / patternNear-integral blocksHardening lossPrimal ‖B−D‖/‖D‖Dual
Llama-3.2-3B 2:495.6%0.0%0.036%0.36%
Llama-3.2-3B 2:883.6%0.1%0.091%0.35%
Llama-2-7B 2:491.0%0.0%0.16%0.86%
Llama-2-7B 2:880.9%0.1%0.15%0.53%
  • What holds for the returned output. Every block holds exactly N weights; the refit is a masked least-squares solve on the returned support; polish rounds are kept only if the layerwise objective drops. The diagnostics support the stability of the returned supports in the evaluated configurations; the formal convergence guarantees apply to the convex reference solver (Alg. 1).
  • Certificate. Cor. 1 is a conditional result. Evaluated at the prox input, as it requires, at the final λ of our schedules, it holds on about 0.01% of blocks for Alg. 1 and on 1.0–1.3% for a prox-driven splitting at 2:4 (App. E.3). We therefore present it as a diagnostic of the relaxed solution, and rely on the measured hardening loss above for the practical solver.
  • Selection. Configuration selection uses a pre-registered rule on a C4 slice disjoint from every reported column (App. C); it chooses among outputs and enters no guarantee.

W2 / Q3 — Same refit, matched time, seeds (Tab. 2, Tabs. 20–22, Fig. 10). Every mask now receives the identical masked-CG refit from W0; WikiText-2, mean ± sd over calibration seeds:

KSPALPSSparseGPT
3B 2:418.08 ± 0.1718.42 ± 0.3119.76 ± 0.25
7B 2:49.80 ± 0.1110.01 ± 0.1910.46 ± 0.14
3B 2:8 (75%)122.6 ± 8.4131.4 ± 1.5162.1 ± 2.8
7B 2:8 (75%)58.8 ± 1.858.6 ± 2.674.4 ± 4.4
  • Seeds. At 2:4, KSP is lower than ALPS on 5/5 paired seeds at both scales (paired t = −3.2 and −5.1) and lower than SparseGPT on 3/3 (t = −15.3 and −8.4). At 2:8 it is lower than ALPS on 3/3 seeds at 3B and comparable at 7B. Zero-shot accuracy is on par with ALPS.
  • Time. You cited 2,200 vs 1,570 s from the submission. Re-timed at 2:4 on an otherwise idle GPU with the final code (permutation search vectorized, bit-identical outputs), KSP takes 438 s and ALPS 1,077 s of mask time at 3B (670 vs 2,045 s at 7B). At matched or lower time KSP keeps its lead: at about a third of ALPS's mask time it reaches 18.15 vs 18.42 and 9.95 vs 10.01.
  • Isolation. These controls isolate the support search under matched compensation; Tab. 22 further varies only the block operation.

Q4 — Allocation: which weights, and is the joint program solved? (App. E.3, Alg. 5). The allocation is computed once, with fixed weights, before the final pruning pass, and then held fixed; the final prune restarts from W0. The deployed allocator is a greedy descent on the latency-priced knapsack of §4, each step valued by the measured KL change of the whole model with previously committed matrices in place (probe sequences from the C4 and WikiText-2 training splits, disjoint from all evaluation data). Thm. 2 characterizes the separable surrogate G_T, evaluated on the dense weights W0; in our tests the measured value performed better, so it is the one deployed. The jointly convex weight–budget program of App. D is a theoretical object and is not solved in the pipeline; the revision states the deployed procedure explicitly (Alg. 5).

Reviewer_T8QsOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes KSP, a one-shot N:M pruning method built on the k-support norm. Through a relaxation, the authors use the squared per-block norm as a regularizer on the standard layerwise reconstruction objective. The norm has a variational form with per-weight gates and a closed-form “water-filling” solution. The authors derive an exact weighted proximal operator. The solver consists of a continuation over the penalty strength, then hardening, then a masked conjugate-gradient refit of the survivors. Experiments cover seven models from 1B to 70B.

Critical strengths
  1. Using the per-block k-support norm as the convex relaxation of N:M is natural and that's a neat and useful identification of the paper.

  2. The method has very low costs. Everything runs on one GPU without backpropagation through the model.

  3. The allocation results are very insightful, and the covered model sizes span a large range, making the experiments in this paper very helpful.

Critical weaknesses
  1. The paper’s own analysis shows its main contribution matters least. The paper ranks the selector (the convex relaxation, the paper’s headline contribution) as the smallest of the four effects. At the reduced solver budget used for many controls, KSP’s uniform 2:4 is worse than ALPS.

  2. Following weakness 1, the largest gains come from refit and allocation, which don’t depend on the relaxation (masked least-squares refit and free dense layers).

  3. Some major baselines such as ADMM [1] and MaskLLM [2] are not included in the results section.

  4. The “2.7× more MMLU retained” claim compares the allocated model at 62.5% density against uniform arms at 50%. (I'm aware the the paper mentions this honestly in the results section, but the abstract is misleading when it discusses allocation effects).

  5. Results switch between a “reported” and a “reduced” solver budget across tables, which makes a fair comparison challenging.


[1] Boza, Fast and Effective Weight Update for Pruned Large Language Models, TMLR 2024

[2] Fang et al., MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models, NeurIPS 2024

Questions for authors
  1. How does your method compare agains MaskLLM and ADMM? I understand that running MaskLLM from scratch is expensive. But they have released a few of their checkpoints in their github repository. On the other hand, ADMM is a very fast method to run.

  2. At intermediate densities, PATCH [3] learns which tiles stay dense and which become 2:4, and reports results between 25% and 45% sparsity, close to the 62.5% density used in your manuscript. I believe PATCH has released their checkpoints on their github repository as well. How does your method compare to them?

3- Can the authors report MMLU and GSM8K for the allocated and uniform configurations at matched measured latency, rather than at 62.5% versus 50% density?


[3] Hourri et al., PATCH: Learnable Tile-level Hybrid Sparsity for LLMs, NeurIPS 2026

6 comments
AuthorsOpenReview ↗

Response to Reviewer T8Qs

Thank you for the careful review. Your points led directly to the controls below, all in the revised PDF.

W1 — the selector is the smallest effect; KSP is behind ALPS at the reduced budget.

  • What the selector bar measures. In Fig. 3a the selector bar is KSP's remaining margin over ALPS, the strongest baseline, whereas the refit bar switches a component on or off; it sizes a lead over the best competitor, not the value of the formulation. With the identical refit and a uniform pattern (new Tab. 2, now opening §5.2), KSP's supports beat ALPS on all five paired seeds at both scales — 18.08 ± 0.17 vs 18.42 ± 0.31 at 3B (paired t = −3.2) and 9.80 ± 0.11 vs 10.01 ± 0.19 at 7B (t = −5.1) — and ALPS is essentially unchanged by the shared refit (18.39 → 18.42), so the gap lies in the support. At 2:8, KSP leads ALPS at 3B (122.6 vs 131.4, 3/3 seeds) and is comparable at 7B.
  • Reduced budget. The reduced setting is a cheap configuration used only inside one-factor controls, not for comparing KSP with ALPS. The reported configuration is itself 2.5–3.1× faster than ALPS in mask time (438 vs 1,077 s; 670 vs 2,045 s; re-timed on an idle GPU with the final code), and at about a third of ALPS's mask time KSP still matches or beats it (18.15 vs 18.42; 9.95 vs 10.01).
  • Where the gain comes from (Tabs. 20, 22). With the rest of the search fixed, exact OBS support selection improves on a top-k projection by 0.18 on average over three paired seeds at 7B, while top-k is slightly better at 3B (17.94 vs 18.05); with either block operation our search is ahead of ALPS at 3B. The complete support search thus contributes beyond refit and allocation, while the benefit of the block operation alone varies across models.

W2 — the largest gains come from refit and allocation (free-dense layers).

  • Refit. The masked least-squares refit is standard, which is why Tab. 2 gives it to every method; what remains is a comparison between supports.
  • Allocation is not just free-dense. On Llama-3.2-3B every projection has a real 2:4 speedup (1.36–1.65×), so free-dense plays no role; at matched measured latency (108.60 vs 108.73 ms, Tab. 16) our allocation improves perplexity from 14.50 to 12.68 and zero-shot from 51.44 to 53.42 over a random dense selection; OWL does no better than random at a matched parameter budget (14.77 vs 14.55; ours 11.91). At 70B, beyond free-dense, pricing the budget by latency rather than parameters is worth a further 0.89 perplexity at a matched 1.078× (Tab. 18). The allocator is the latency-priced knapsack of §4: the norm gives its analytical price (Thm. 2), and deployment evaluates the same knapsack with a measured value (Alg. 5, App. E.3).

W3 / Q1 — ADMM and MaskLLM (Tabs. 1, 2, 26). ADMM (Boža, official code, our calibration) is now in Tab. 1: KSP has lower perplexity on all six models where both run (e.g., 7B 9.70 vs 10.24; Mistral 9.32 vs 9.96), and under the shared refit on every seed (3B 18.08 vs 19.07; 7B 9.80 vs 10.16). MaskLLM's released masks, learned end to end on ~2B tokens, reach higher native quality (7B: 7.34 vs 9.71); we report them with their training cost (Tab. 28; ours uses 0.26M calibration tokens). We also corrected one transcription error (Tab. 1, SparseGPT 3B zero-shot: 47.27).

Q2 — PATCH (Tab. 27). PATCH (learned, ~2B tokens, 128×128 tiles) reaches higher quality, and the gap is small at lighter sparsity: on Llama-2-7B at ~25% sparsity KSP reaches 6.38 vs 6.26 with about 10⁴× less data. Among one-shot configurations, our allocation gives the lowest perplexity at every tested 1B and 7B operating point, and it also improves SparseGPT and Wanda over the OWL-based allocation.

Q3 / W4 — MMLU and GSM8K at matched latency (Tab. 3, Tab. 25). With the same profiled latency budget, selector and refit, our allocation gives the best WikiText-2, zero-shot accuracy, MMLU (40.6 ± 0.6 vs 38.5 ± 1.7 random, 35.8 ± 2.0 OWL) and GSM8K (7.1 vs 4.5 / 3.7) at the main 3B operating point; measured whole-model speedups are 1.08× (ours), 1.05× and 1.09×. Uniform 2:4, the fastest configuration on this model (1.17×), is a separate tradeoff point: moving from 1.17× to 1.08× raises MMLU from 32.5 to 40.6. Across budgets (Tab. 25), it has the best WikiText-2 and zero-shot accuracy at every 3B budget. The abstract's allocation sentence now states what is matched (measured latency), and the 2.7× figure in §5.3 is labelled with its densities.

W5 — reported vs reduced. App. C now defines each procedure and what its results establish (Tab. 6): the convex reference solver (Alg. 1), for which the guarantees are stated; the reported solver (Alg. 3), whose block step is the hardened, debiased counterpart of the prox (exact blockwise support selection); and the reduced setting. The main comparisons (Tabs. 1–3) use the reported configuration; controls marked "reduced" hold the KSP configuration fixed across arms and vary one factor.

I thank the authors for their clarifications and the additional results.

My remaining concern is the role of the relaxation: the reported solver uses exact blockwise support selection rather than the k-support prox, your search beats ALPS with either block operation, and the deployed allocator uses measured values rather than the water-level price. This suggests the gains come mainly from the search, which the title and abstract should reflect.

I retain my score, but I’m open to raising it in case additional clarifications are made regarding this observation.

AuthorsOpenReview ↗

Follow-up: role of the relaxation; title and abstract revised

Thank you for the follow-up, and for being open to revisiting the score. We agree with your observation, and we have revised the title and abstract accordingly.

We agree on the facts. The reported solver selects supports by exact blockwise selection rather than by the k-support prox; our search is ahead of ALPS at 3B with either block operation; and the deployed allocator uses measured values rather than the water-level price. The measured gains therefore come from the search and the measured-value allocator. We do not claim that solving the convex program produces them, and the guarantees are stated for the convex program only (§3.2).

Changes in the revised PDF.

  • Title: now "k-Support Pruning: The Convex Geometry of N:M Sparsity and a Practical Support Search for LLM Pruning" (previously "… Tight Convex Relaxation and Principled Optimization for N:M LLM Pruning").
  • Abstract: a new sentence separates the two parts: "The convex program is our analytical reference; the empirical gains come from the practical method it leads to, an exact blockwise support-selection search on the same objective and a latency-priced allocator driven by measured values."
  • Contributions: item (iii), the practical method, is now labelled as the source of the measured gains.

What the relaxation contributes, stated without overreach. The paper has two parts, and it now presents them as such.

  1. An analysis of the N:M constraint. Per block, the convex hull of the constraint is the k-support ball (Prop. 1); its weighted prox is closed-form (Thm. 1); under diagonal curvature, hardening and refitting returns the combinatorial optimum (Prop. 2); magnitude and Wanda are its zero-iteration case (Prop. 6); and the water level gives a closed-form price for budget (Thm. 2). These are results about the constraint and its relaxation, not about any particular search.
  2. A practical method designed from that analysis. The block step of the search is the hardened, unshrunk counterpart of the prox: under diagonal curvature, exact support selection keeps exactly the top-N ranking the matched-scaling prox selects, which at W0 is the optimal support of Prop. 2 (App. C). It departs from the prox in two ways, both practical: it applies no shrinkage, and with dense curvature it also uses the coupling within a block. For allocation, §4 supplies the latency-priced knapsack that the deployed allocator solves; the allocator probes each candidate matrix with the zero-iteration gates (Alg. 5) and values it by measured KL in place of the closed-form price.

So the relaxation is the paper's theoretical contribution and the origin of the method's design, while the empirical contribution is the search and the allocator, supported by the controlled comparisons of Tabs. 2–3 and 20–25.

We hope the revised title and abstract now reflect this division accurately, and we are glad to adjust the wording further if you have a preference.

Reviewer_waMUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a pruning method based on k-support norm, called KSP, to solve the problem of discrete weight selection in semi-structured N:M pruning of LLMs. Specifically, the authors derive a closed-form weighted proximal operator, establish theoretical properties related to support recovery and hardening, and further extend the formulation to non-uniform sparse allocations using a water level-based budget pricing mechanism. Extensive experiments have demonstrated the effectiveness of KSP.

Critical strengths
  • Weight selection and sparse budget allocation across matrices are derived from the same mathematical framework.
  • The experiments cover a variety of models from 1B to 70B, and are compared with multiple pruning methods, including Wanda, RIA, SparseGPT, ALPS, MaskPro and ProxSparse, etc.
  • The theoretical derivation of this paper is very complete.
  • The ablation experiment, especially Figure 4, is very well done.
Critical weaknesses
  • Lack of measurement of actual acceleration effects, such as latency, etc.
  • Can the proposed KSP be further generalized to VLM or MoE models?
  • Although the authors provide pruned runtime and peak GPU memory comparisons in Table 4 and Figure 9, the computational efficiency of KSP relative to ALPS is still not fully established.
Questions for authors
  • The current calibration dataset is C4. Will there be any issues if I switch to another dataset?
3 comments
AuthorsOpenReview ↗

Response to Reviewer waMU

Thank you for the positive assessment and the concrete questions. Each is answered below with new measurements, all included in the revised PDF.

W1 — Measured acceleration (Tab. 3, Tab. 30). We now report latency next to quality: whole-model prefill at 4096 tokens, batch 1, fp16, CUDA graphs and stock cuSPARSELt 2:4 kernels on one GPU, mean ± sd over 60 repetitions.

ModelConfigurationPrefill (ms)Speedup
Llama-3.2-3Bdense130.29 ± 0.241.00×
uniform 2:4111.57 ± 3.011.17×
allocated, budget 0.625115.90 ± 0.151.12×
Llama-2-7Bdense286.95 ± 5.971.00×
uniform 2:4225.41 ± 6.561.27×
allocated, budget 0.625237.13 ± 0.581.21×
  • Speed and quality together (Tab. 3). On Llama-3.2-3B, uniform 2:4 runs at 1.17× with WikiText-2 18.06 and MMLU 32.5; our latency-priced allocation runs at 1.08× with 12.37 and 40.6 (dense MMLU: 56.7), and it beats random and OWL allocations of the same latency budget on perplexity, zero-shot accuracy, MMLU and GSM8K. An intermediate budget runs at 1.13× with 13.90 and 36.0 (Tab. 25).
  • 70B (decoder estimates from individually timed layers with real weights). The 2:4 kernel's speedup differs sharply across projections: on Llama-3.1-70B, sparsifying only gate or up gives 0.98× / 1.00×, while down alone gives 1.06×. Keeping gate/up dense is therefore both faster than uniform 2:4 (1.077× vs 1.030×) and more accurate (WikiText-2 5.38 vs 8.20); on Llama-2-70B it matches uniform 2:4 in speed (1.068× vs 1.071×) while lowering perplexity by 1.28.

W3 — Efficiency relative to ALPS (Tab. 2, Tab. 7, Tab. 20, Fig. 10). We re-timed every method at 2:4 on an otherwise idle GPU with the final code (KSP's permutation search is now vectorized, with bit-identical outputs), and compare quality with the identical refit applied to every mask:

Method3B: mask / +refit (s)3B WikiText-27B: mask / +refit (s)7B WikiText-2
SparseGPT224 / 38319.76176 / 28210.46
ALPS1077 / 131218.42 ± 0.312045 / 221510.01 ± 0.19
KSP438 / 59618.08 ± 0.17670 / 7769.80 ± 0.11
KSP, 100 iterations366 / 52518.15556 / 6629.95

KSP is 2.5–3.1× faster than ALPS in mask time and 2.2–2.9× including the refit, with lower perplexity on all five paired calibration seeds at both scales. The cost is tunable: at about a third of ALPS's mask time KSP still matches or beats it, and doubling the budget improves 3B further (17.79 at 536 s, still half of ALPS's time). Fig. 9 reports wall-clock and peak GPU memory from 1B to 70B; every run, including 70B, fits on a single 96 GiB GPU with no backpropagation through the model, and the allocation is computed on a smaller model and transferred, adding no search cost at 70B.

Q1 — Other calibration data (Tab. 29). Switching to another broad corpus causes no issue. We repeated the uniform and allocated runs with FineWeb and WikiText-2 calibration (Llama-3.2-3B, 3 seeds; WikiText-2 / C4 / PTB perplexity, then zero-shot accuracy):

CalibrationKSP 2:4SparseGPT 2:4KSP allocated
C418.06 / 23.05 / 24.91, 49.021.44 / 27.38 / 29.22, 47.112.91 / 17.44 / 17.59, 53.4
FineWeb17.43 / 23.31 / 24.72, 49.520.98 / 27.85 / 28.06, 47.212.52 / 17.38 / 17.21, 52.9
WikiText-213.02 / 29.36 / 25.86, 43.515.61 / 31.60 / 29.56, 45.910.67 / 18.61 / 17.97, 52.5

FineWeb matches C4, and KSP keeps its lead over SparseGPT on all three perplexities under every corpus. A narrow corpus such as WikiText-2 fits its own domain but transfers less well, and KSP's zero-shot accuracy is then the more affected, so we recommend a broad web corpus (C4 or FineWeb). The allocation is stable: 60 ± 4, 62 ± 5 and 61 ± 3 matrices stay dense under the three corpora, and 174 of 196 matrices keep the same budget between C4 and WikiText-2 calibration. Llama-2-7B behaves the same (FineWeb vs C4, single seed: 9.69 vs 9.83 uniform; 7.24 vs 7.37 allocated).

W2 — VLM and MoE. KSP touches a model only through its linear layers and their calibration Grams, with no architecture-specific component: the same code prunes four families (Llama-2, Llama-3, Qwen2.5, Mistral) from 1B to 70B, and an allocation computed on one model transfers unchanged to another (Llama-2-70B → Llama-3.1-70B, closing 52–75% of the gap to dense; Fig. 5). The linear layers of a vision encoder, a projector or an MoE expert present the same interface. What needs separate validation is calibration — multimodal inputs for VLMs, and per-expert Grams built only from routed tokens for MoE — so we prefer not to claim it without experiments and leave it to future work.

Presentation. To reduce reading effort, §5.2 now opens with the controlled comparison against ALPS, the latency and calibration results above are collected in App. E.3, and the appendix gives complete pseudocode for the method.

Thanks to the author for the reply, most of my concerns have been resolved and I choose to keep it as is (weak accept)

Thank you for reading our response and for your continued support of the paper. If any concern remains open, we would be glad to address it during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.