acceptodds
Under review as a conference paper at ICLR 2027

CoCurve: Cross-Module Co-Pruning Curvature for Structured LLM Pruning

15 views

Abstract

Resource-constrained deployment requires sustaining large language model (LLM) capabilities as models scale under fixed memory and computation budgets. Structured pruning advances this deployment frontier with smaller dense checkpoints, yet deciding what to prune remains bottlenecked by interactions among jointly removed units. We introduce Cross-Module Co-Pruning Curvature (CoCurve), which formulates structured pruning as set-dependent predictive risk over a unified inventory of attention heads and feed-forward groups. A co-pruning graph built from single-unit forward ablations assigns individual risk to nodes and reinforcement or cancellation to interaction edges, conditioning each decision on the units already removed. We evaluate 6 LLMs (3B–70B) and 3 vision–language models (VLMs) across 3 perplexity corpora, 12 language tasks, and 7 multimodal benchmarks. Across the five-model 20–40% grid and the 70B 10–50% sweep, CoCurve ranks first in 53/60 corpus comparisons (15/15 at 70B) and permits 2.2–6.6 more pruning points at matched quality in 9/10 interpolated comparisons, while leading all 6 VLM Avg7 blocks. After lightweight recovery, its retained structures remain strongest through 50% pruning, where the 8B checkpoint regains 10.8 Avg12 points; physical slicing delivers 1.58× dense prefill throughput with 41% lower peak memory. Mechanism analysis across 10 LLMs and 7 VLMs further finds organized within- and cross-module edge structure; matched removals of low-saliency, high-coupling units degrade 19/20 capability groups by up to 23.7 points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Structured Pruning, Model Compression, Co-Pruning Interactions, Efficient Inference, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mpNvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies structured LLM pruning when the effect of removing a component depends on which other components are removed. CoCurve builds a graph from single-unit forward ablations, representing individual pruning costs and interactions among attention heads and FFN groups. It then selects components jointly and chooses the interaction strength using teacher KL on separate calibration data. Experiments across LLMs and VLMs show improved quality retention over the evaluated selectors, including after common LoRA recovery and quantization. Physically sliced checkpoints also demonstrate inference throughput and memory improvements.

Critical strengths
  1. Broad, consistent evaluation. The study covers six LLMs, three VLMs, multiple pruning levels, recovery, and quantization. The quality gains therefore extend beyond one checkpoint or an unrecovered pruning result (Section 3; Appendices C–D).
  2. Controls isolate the practical contribution. Fixed-allocation and diagonal controls show gains from interaction-aware selection, while matched physical slicing separates retained quality from differences in memory or throughput (Table 7; supplementary Tables 18 and 30).
Critical weaknesses
  1. The named baselines are evaluated as modified selectors. Appendix C.3 removes FLAP's bias compensation, SlimGPT's reconstruction, and SlimLLM's regression. This is useful for isolating mask selection, but does not establish superiority over their complete published pipelines. Common LoRA recovery addresses a different comparison; the headline claims should preserve this distinction.
  2. The additional offline cost lacks a matched-compute comparison. Table 19 reports 17.4 hours for the 8B five-ratio sweep and 62.8 hours for 70B, including extensive candidate evaluation. The cost is disclosed, but the study does not establish whether comparable resources spent on alternative selection or recovery would retain the same quality ordering.
Questions for authors
  1. Are any existing results available with the baselines' native compensation/reconstruction enabled? Otherwise, can the claimed improvement be explicitly limited to the shared selector protocol?
  2. From the recorded runs, what are selection cost and quality for one requested pruning ratio, and how do these compare with the strongest baseline under the same device and resource accounting?
5 comments
AuthorsOpenReview ↗

Response to mpNv (1/3): Native Pipelines and Shared Recovery

Thank you for your constructive feedback and for recognizing our consistent empirical gains, controlled analyses, and practical deployment value. We address complete native pipelines in Part 1 and single-target cost and matched-compute comparisons in Part 2.

Part 1. Native Pipelines (W1 / Q1)

We have added complete pipelines with FLAP's, SlimGPT's, and SlimLLM's native compensation/reconstruction enabled. The paper's shared selector protocol isolates structure selection itself and establishes CoCurve's selection advantage (best PPL in 38/45 comparisons across five models; highest Avg12 in 8/8 configurations on 8B/24B). We first directly compare complete native pipelines in Part 1.1, then test whether the quality advantage persists when every pipeline receives equal lightweight adaptation in Part 1.2.

Part 1.1. Full Pipelines

Takeaway. Both CoCurve implementations lead in most configurations against complete native pipelines across three models.

Specifically, we retain each baseline's native selection and recovery: FLAP's channel allocation and bias compensation, SlimGPT's sequential selection and OBS reconstruction, and SlimLLM's inter-layer allocation, channel selection, and local regression. We implement CoCurve's complete pipeline with two existing reconstruction approaches, comparing each against the three native pipelines. CoCurve + OBS fixes the original mask and uses OBS to compensate retained weights. CoCurve + FFN Refinement preserves CoCurve's selected attention heads and per-layer quotas, reusing SlimGPT's channel selection and reconstruction within FFNs. All pipelines are evaluated after selection and recovery.

The table reports Avg12 (%) for three models. Each cell lists 30%/40%/50% pruning. No pipeline uses LoRA.

PipelineLlama-3.2-3BLlama-3.1-8B-InstructMistral-Nemo-12B
FLAP native42.05 / 39.44 / 34.9847.73 / 44.04 / 40.5048.02 / 40.68 / 36.32
SlimGPT native38.82 / 37.05 / 35.7952.40 / 46.45 / 42.4842.31 / 38.20 / 38.43
SlimLLM native43.23 / 38.94 / 35.7652.83 / 45.82 / 37.7349.32 / 43.50 / 38.24
CoCurve + OBS46.82 / 40.74 / 36.6753.35 / 46.84 / 41.3750.31 / 45.28 / 40.34
CoCurve + FFN Refinement47.23 / 41.79 / 36.9053.56 / 48.46 / 43.0853.83 / 47.48 / 42.63

Against the strongest native baseline in each configuration, CoCurve + OBS achieves higher Avg12 in 8/9 configurations, while CoCurve + FFN Refinement achieves higher Avg12 in 9/9 and the lowest C4 in 8/9. CoCurve's structure-selection advantage carries through to complete pipelines across all three models.

Part 1.2. Shared LoRA

Takeaway. After equal recovery, CoCurve still leads in most configurations on each of Avg12 and C4.

The paper's common LoRA comparison tests structural recovery potential under the same budget (Tables 3, 16). We further start from each complete native pipeline's output and apply the same one-epoch Alpaca LoRA (rank 16), testing whether CoCurve's structural advantage persists when the baselines also receive adaptation. Each cell lists 30%/40%/50%, with C4↓/Avg12↑ at each ratio.

Pipeline + same LoRA8B12B
FLAP native19.40/52.87 · 23.34/49.23 · 29.16/46.8415.47/56.03 · 19.55/52.09 · 25.85/47.63
SlimGPT native17.38/56.98 · 20.15/52.31 · 24.35/47.6613.10/56.92 · 15.22/46.85 · 18.12/48.90
SlimLLM native17.62/56.97 · 21.57/53.43 · 27.49/47.9113.45/58.09 · 16.02/53.81 · 20.60/47.99
CoCurve + FFN Refinement16.05/56.67 · 18.72/52.45 · 23.10/48.0012.38/58.58 · 14.47/54.17 · 18.16/48.99

Across six configurations, the CoCurve combination achieves the lowest C4 in five and the highest Avg12 in four. Complete native results directly establish end-to-end competitiveness. Common recovery further shows that the better structural starting point remains valuable during subsequent adaptation. The revision will include native results and specify which conclusions the shared selection protocol, complete native pipelines, and common recovery each support in the headline claims.

AuthorsOpenReview ↗

Response to mpNv (2/3): Single-Target Cost and Graph Reuse

Part 2. Matched Compute (W2 / Q2)

Graph construction is an offline investment shared by multiple subsequent deployment versions. The resulting graph and pruned structures can also be delivered directly to users. Its return is a better structural starting point, reducing the recovery training needed to reach a given quality. We first show the demonstrated reuse scope and deployment costs in Part 2.1, then include amortized graph construction in the total cost to test the quality ordering when comparable resources go to alternative selection or recovery in Part 2.2.

Part 2.1. Cost and Reuse (Q2)

Takeaway. A base model's graph is reusable across ratios, and released selection results remain usable for reconstruction, adaptation, and low-bit deployment.

① Reuse is realized through ready-to-use graphs and structures. With the base model and structural units fixed, one graph supports searches at all pruning ratios. It occupies only 2.36 MB for 8B and under 15 MB for 70B. The reproduction package provides graphs for 9 models and 42 pruned structures, with exact CPU replay verified at all 42 operating points. For these base models, users can directly use the available graphs and selection results without rebuilding the graph:

  • New pruning ratio: Reuse the graph and only search and score for that target. On an A100, searching 30%/50% structures for 8B takes 40.8/27.4 minutes.
  • Target-ratio calibration coefficients available: Generate the target structure on CPU. For 8B at 20%/50%, this takes only 39/79 milliseconds.
  • Released structure available: Skip search and compilation, proceeding directly to reconstruction or adaptation. Selection results remain usable for training-free reconstruction, LoRA recovery, and physically sliced low-bit deployment. See Part 1 and vvVx Part 4.

② Reuse costs correspond to actual quality gains. Separate cost runs compare complete preparation and quality at 50% pruning without LoRA, using A5000/A100 for 8B/12B, respectively. Δ denotes CoCurve + FFN Refinement relative to SlimGPT native.

ModelCoCurve prep. (min)SlimGPT prep. (min)ΔC4 PPL (%) ↓ΔAvg12 (pp) ↑
8B51.648.7-40.0%+0.65
12B54.350.0-9.3%+4.58

8B requires only about 3 additional minutes of preparation, with 40.0% lower C4 and 0.65 percentage points higher Avg12. 12B requires about 4.4 additional minutes of preparation, with 9.3% lower C4 and 4.58 percentage points higher Avg12. The structural advantage produced by selection can thus reach users at subsequent preparation costs close to the native baseline's. The complete common-recovery table in Part 1.2 further shows that these selection results provide a better starting point for adaptation.

③ First-run single-target cost and amortization. For one requested pruning ratio, 8B at 50% on A5000 takes 21.85 hours for first-run selection (20.69 for construction and 1.16 for search/scoring), or 24.61 hours for the complete pipeline including slicing, FFN reconstruction, and one LoRA epoch. On the same device type, SlimGPT takes 0.81/2.95 hours for native preparation/the complete pipeline after the same one-epoch recovery. Both timings exclude evaluation. CoCurve's output has 6.3% lower C4 and 0.23 percentage points higher Avg12.

The 20.69 hours produce a shared graph: across 20%/30%/40%/50%, each ratio bears only 5.17 hours of construction, with its own search and recovery charged separately. The complete pipeline for a new target reusing the graph takes 3.92 hours (sum of measured stages). Construction time varies by device: 12.6/3.70/8.41 hours for 8B on RTX 6000 Ada/H100/A100. Table 19 in the submission (Table 22 in the revision) reports 17.4/62.8 hours for total selection across five ratios on 8B/70B, respectively. We next use complete resource comparisons to show the quality return from sharing this investment.

AuthorsOpenReview ↗

Response to mpNv (3/3): Quality–Cost Trade-offs Across Compute Budgets

Part 2.2. Quality–Cost Trade-off (W2 / Q2)

Takeaway. Including amortized construction, CoCurve achieves higher quality in less time. Its advantage also holds for complete native pipelines.

① Including amortized construction: better selection reduces recovery needs. To test the opportunity cost of additional selection, we use the same LoRA recipe on structures pruned under the paper's shared selection protocol, allowing LLM-Pruner more recovery compute. The table gives the full recovery curve for 8B at 50% on the same A5000 device type: CoCurve includes graph construction amortized across four ratios, target-specific search, slicing, and recovery. LLM-Pruner counts only training and saving.

8B, 50% pruningTime (h)C4 PPL ↓Avg12 (%) ↑
CoCurve + 1 LoRA epoch≤8.7033.8945.15
CoCurve + 4 LoRA epochs≤14.6632.4445.61
LLM-Pruner + 1 LoRA epoch1.97 (training + save)37.1444.57
LLM-Pruner + 4 LoRA epochs7.89 (training + save)34.9245.09
LLM-Pruner + 8 LoRA epochs19.39 (training + save)39.2145.13
LLM-Pruner + 16 LoRA epochs33.98 (training + save)83.5943.00

One CoCurve epoch takes less than half the time of the baseline's best downstream setting, reaching similar Avg12 with 13.6% lower C4. Four epochs are still faster than that setting and improve both metrics. Compared with the baseline's best measured value for each metric, four-epoch CoCurve has 0.48 percentage points higher Avg12 and 7.1% lower C4. Even charging all first-run graph construction, CoCurve's four-epoch total-time upper bound of 30.18 hours is below the baseline's 33.98 hours for training and saving alone at sixteen epochs. Thus, the offline investment both reduces recovery training needed for the same quality and is amortizable across pruning budgets.

② Complete native pipelines: the advantage persists at similar total time.
i) Shared four-epoch recovery. We keep native selection and reconstruction and apply four LoRA epochs with the same recipe. In this A5000 matched-compute comparison, CoCurve sums measured search, slicing, and recovery stages after graph reuse. Baseline times include full native preparation and training. Evaluation is excluded.

8B, 50% pruningTime (h)C4 PPL ↓Avg12 (%) ↑
CoCurve + FFN Refinement9.7123.0048.16
SlimGPT native9.3724.0948.11
SlimLLM native8.1727.7247.83
FLAP native9.2328.3948.02

Within roughly 8–10 hours, CoCurve achieves the lowest C4 (4.5–19.0% lower) and the highest Avg12. Reconstruction and additional LoRA both optimize within the retained structure, so better structure selection still affects the final recoverable quality, consistent with Part 1.2.

ii) Recovery budget sweep. On A100, CoCurve and LLM-Pruner reuse released structures. The three native baselines retain their selection and reconstruction. Time includes loading, slicing, and recovery. Each cell lists time (h)/C4↓/Avg12↑, excluding evaluation time. CoCurve’s four-epoch Avg12 is from a separate A100 run; time and C4 are from the original run.

8B, 50% pruning1 epoch2 epochs4 epochs8 epochs
CoCurve + FFN Refinement1.68 / 23.08 / 48.192.63 / 22.82 / 48.544.54 / 23.18 / 48.719.05 / 26.38 / 48.21
SlimGPT native1.81 / 24.37 / 47.673.08 / 23.89 / 47.425.24 / 24.14 / 48.0011.02 / 27.68 / 47.45
SlimLLM native1.20 / 27.67 / 48.212.29 / 27.20 / 48.434.47 / 27.72 / 48.649.04 / 34.19 / 47.50
FLAP native1.26 / 29.15 / 46.902.47 / 28.34 / 47.724.86 / 28.33 / 48.149.67 / 31.65 / 48.07
LLM-Pruner0.98 / 37.13 / 44.601.92 / 35.53 / 44.813.83 / 34.92 / 45.097.68 / 39.23 / 44.89

CoCurve has the lowest C4 at every dose and the highest Avg12 at two, four, and eight epochs. Its two-epoch C4 (22.82) beats every baseline setting in the sweep. Against SlimGPT, one epoch takes 1.68 vs. 1.81 h (C4 −5.3%, Avg12 +0.51 pp), and four take 13.4% less time (C4 −4.0%, Avg12 +0.71 pp).

③ Alternative selection: the advantage persists with equal tuning opportunities. Beyond recovery, we give the interaction-aware selector SOSP-I the same calibration text and candidate limit to test whether selection tuning closes the gap. CoCurve still leads by 0.99–4.08 percentage points at all four 8B ratios from 20–50%. See vvVx Part 2.2.

Graph construction costs are shared across targets, while structural quality gains remain in each deployed model. These comparisons show that additional selection compute yields better structures, reduces reliance on recovery training, and delivers quality gains in complete native pipelines. The supplied graphs and structures make these returns directly reusable by subsequent users.

Thank you again for recognizing the practical contribution and for the suggestions that have helped us strengthen the end-to-end and resource comparisons.

Reviewer_pgLcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper focuses on structured pruning for LLMs and VLMs. The authors propose CoCurve, which treats pruning as set-dependent predictive risk. It puts attention heads and FFN groups into a single pruning inventory, builds a signed co-pruning graph from single-unit forward ablations, and conditions each pruning decision on the units already removed. Experiments on 6 LLMs (3B–70B) and 3 VLMs across language and multimodal benchmarks suggest that modeling cross-module joint-removal interactions improves the quality–deployment frontier. The analysis also finds that low-saliency, high-coupling units can affect retained capability.

Critical strengths
  1. This paper asks a useful question and presents a clear framework to answer it. Most structured pruning methods score units one by one. They miss how removals interact. This paper treats attention heads and FFN groups as one inventory. It models signed interactions between removals. This addresses a real blind spot of unit-wise pruning. The estimator is also practical. It builds the graph from single-unit forward ablations. It avoids pairwise interventions, gradients, and weight updates. The core idea is well motivated and usable at LLM scale.

  2. This paper conducts broad experiments on multiple LLM and VLM scales. It also reports recovery, quantization, and physical slicing. This shows that the method is intended for practical deployment, not only for lower perplexity. It also gives reviewers several angles to judge its practical value.

Critical weaknesses
  1. The Conclusion claims that the method “traces the gain to joint-removal interactions rather than budget allocation alone.” (Lines 483-484) This attributes the gain to interaction modeling. But Table 7 shows only modest edge-specific gains: at 20%, ΔKL −0.049 and ΔAvg12 +1.2; at 30%, ΔKL −0.240 and ΔAvg12 +1.1. The paper does not separate the contribution of edges from joint allocation and held-out selection, so the attribution is not established.

  2. Section 4 claims that CoCurve “addresses” this blind spot by conditioning on the removal set (Line 439). However, the evidence is insufficient. Figure 8(b) only shows that low-saliency, high-coupling units exist. Figure 8(c) provides a single example of a decision reversal. Figure 8(d) shows that force-removing these units is damaging. None of these show that CoCurve systematically retains them during natural pruning, or that this retention explains the downstream gain.

  3. The Introduction moves from “Existing structured pruners… mostly assign unit-wise scores” (Line 053) directly to “These interactions matter because structured pruning is inherently a joint-removal problem” (Line 072). It does not explain why unit-wise scoring fails under shared residual streams.

  4. The paper moves from “mean normalized attention-FFN edge magnitude is 0.74–0.98× the within-module mean” (Line 076) to “We therefore ask: How can interactions…” (Lines 077-078). Non-trivial edges do not by themselves show that joint modeling is necessary.

  5. In Section 3.6, Table 7 shows that edges alone yield ΔAvg12 +1.2/+1.1. Joint competition yields +2.0/+1.4. The paper never analyzes where the remaining gain comes from.

Questions for authors
  1. How much of the reported gain comes from edge-conditioned reordering versus joint allocation, held-out selection, recovery, and tuning? Table 7 suggests that edges alone explain only a modest part.

  2. What are the exact conditions for the “1.58× prefill” and “41% lower peak memory” claims, and how do they relate to the smaller gains in Table 5 at 20%?

  3. How often do accumulated edges reverse a node-only decision, and does CoCurve systematically avoid bridge units during natural pruning?

4 comments
AuthorsOpenReview ↗

Response to pgLc (1/4): Gain Attribution—Reordering and Allocation

Thank you for your insightful analysis and for recognizing our clear, well-motivated framework, scalable estimator, and practical deployment value. We address gain attribution in Part 1, natural pruning behavior in Part 2, motivation and evidence for joint removal in Part 3, and efficiency conditions in Part 4.

Part 1. Gain Attribution (W1 / W5 / Q1)

CoCurve models pruning as selecting combinations, directly accounting for how removal perturbations reinforce or cancel one another. At each step, it therefore compares candidates' removal costs per parameter given the removal set, so what to remove and how much to remove from each part emerge naturally from the same sequence of decisions. This mechanism naturally yields three benefits, without a separate strategy for each: (1) better removal combinations and budget allocation in Part 1.1; (2) quality gains from specific unit-level relationships in Part 1.2; and (3) structural advantages under equal recovery in Part 1.3. The complete method consistently leads across models at the same pruning budgets: best PPL in 53/60 comparisons across six LLMs, and highest Avg12 in all eight configurations on 8B/24B.

We first address your concerns about modest gains and attribution to interaction modeling (W1/W5/Q1):

① Gain magnitude. The seemingly modest Avg12 increments in Table 7 (+1.2/+1.1 percentage points at 20%/30% pruning) come from the fixed-quota control in (1). The Diagonal starting point already retains CoCurve's complete-removal response scores: for example, its 8B Avg12 at 20%/30% is already 55.11/48.66, above the strongest external baseline's 54.49/47.98 in the paper's shared selection comparison. On top of this, edge-conditioned reordering still accounts for about 60% and 80% of the total gains when quotas are freed (+2.0/+1.4 percentage points).

② Interaction attribution. Both sides use the same single-removal scores and attention/FFN quotas. Only one adds calibrated edge-conditioned reordering. At 30% pruning, independent KL falls by 13.1–19.2% and WikiText-2 PPL by 15.4–25.3% on 3B/8B. Thus, interaction modeling further reduces prediction damage on top of the same single-removal signals.

The controlled comparisons below further establish interaction edges as the main source of structure-selection gains, test the stability of this contribution, and separate the gains from joint allocation, held-out selection, and recovery.

Part 1.1. Reordering and Joint Allocation (W1 / W5 / Q1)

Takeaway. Edge-conditioned reordering accounts for most downstream gains in Table 7. Joint allocation adds gains at 20% and 30%.

① Gains from edge-conditioned reordering and joint allocation. Diagonal retains CoCurve's estimated single-removal damage, disabling only interactions between candidates and removed units (λ=0\lambda=0). We first fix Diagonal's global attention/FFN quotas and add only edge-conditioned reordering, then free the quotas so the same rule performs joint allocation across types. Each edge-based branch calibrates λ\lambda on held-out text among its allowed candidates. The table follows Table 7's equally weighted 3B/8B means. Both Wiki PPL columns are relative to Diagonal.

PruningReordering (Avg12 pp)Wiki PPL Δ (fixed) ↓Extra allocation (Avg12 pp)Total (Avg12 pp)Wiki PPL Δ (free) ↓
20%+1.230−2.7%+0.739+1.969−2.1%
30%+1.124−20.4%+0.233+1.357−23.0%
40% (extension)+2.313−46.8%+0.018+2.331−39.9%

Edge-conditioned reordering under fixed quotas already provides most downstream gains at all three ratios. At 40% pruning, it lowers independent test KL on both 3B and 8B (−0.557 on average) and accounts for nearly all of the 2.3-point Avg12 gain. The remaining gains come from joint allocation: freeing quotas lets attention and FFN compete for the removal budget under the same risk criterion. This additional benefit varies with the pruning ratio.

② Interactions improve structure selection even with identical per-layer removal counts. Fixing total attention/FFN quotas still allows budget to move between layers. After further matching removal counts within each layer, type, and cost group, adding interactions improves Avg12 in 11/12 configurations on 3B/8B/24B and reduces independent KL by 9.2–31.5% at all four 24B ratios.

CoCurve vs. quota-matched node-only20%30%40%50%
3B ΔAvg12 (pp) ↑+0.58+0.53+0.39+0.58
8B ΔAvg12 (pp) ↑+0.69+1.00+0.39−0.06
24B ΔAvg12 (pp) ↑+0.62+1.11+0.13+0.63
24B independent KL reduction ↑15.52%22.96%9.21%31.45%

Six equal-total-budget comparisons across three VLMs also show that removing interaction terms lowers Avg7 by 0.9–7.6 percentage points.

AuthorsOpenReview ↗

Response to pgLc (2/4): Gain Attribution—Interactions, Held-Out Selection, and Recovery

Part 1.2. Unit-Level Interactions and Held-Out Selection (W1 / Q1)

Takeaway. Held-out selection improves predictions and stabilizes selection. With equal calibration, specific interaction edges still improve pruning quality.

① Held-out selection improves prediction quality and stabilizes structure selection. On the same graph and pruning budget, selecting interaction strength on text excluded from graph construction lowers independent test PPL relative to full-strength interactions in all nine configurations on both WikiText-2 and C4. Cells show relative PPL changes (WikiText-2 / C4).

Model20%30%40%
3B−1.1% / −12.9%−20.4% / −20.8%−51.4% / −54.1%
8B−0.2% / −9.0%−27.2% / −37.1%−89.3% / −83.6%
12B−9.3% / −15.2%−23.5% / −33.7%−66.0% / −69.0%

When graph-construction text is halved, the structures selected at 20% pruning on four models also stay closer to their respective full-data results, with mean overlap increasing by 0.13.

② With equal calibration, interaction modeling itself improves pruning quality. To separate interaction modeling from calibration, both sides in each of two controls receive equal calibration opportunities: first we compare which interaction information is retained, then how the same edges are used to select units.

i) Specific edges distinguish units of the same type. At 20% pruning on 8B, both sides start from the same graph with the same budget and solver. One retains each unit pair's relationship, while the other averages by type. Retaining specific relationships improves Avg12 by 0.79 percentage points (0.84/0.84/0.75/0.65 percentage points across the paper's four capability groups). Calibration adjusts interaction strength but cannot recover unit-level distinctions lost through averaging.

ii) Updating rankings given the removal set selects better combinations than static scores. On the same graph, updating signed risk given the removal set, rather than aggregating absolute edge values into fixed scores, improves Avg12 at all four 8B ratios from 20–50% (+1.96–4.95 percentage points). A candidate may amplify one removed unit's perturbation but cancel another's, so its removal cost must be assessed in the context of the removal set.

8B Avg12 (%) ↑20%30%40%50%
Static absolute scores54.4346.8337.8431.86
Signed conditional selection56.3950.5540.6536.81
ΔAvg12 (pp)+1.96+3.71+2.81+4.95

Together, these controls show that interaction modeling itself yields downstream gains under equal calibration. Specific edges distinguish which units are compatible for joint removal. CoCurve then updates rankings given the removal set to use this information to select better pruned structures.

Part 1.3. Recovery and Tuning (Q1)

Takeaway. Tuning adds gains. Under equal tuning, CoCurve's selected structures still attain higher downstream quality.

i) Fixing the structure to measure tuning gains. In the original Table 3 comparison, adding one LoRA epoch to each fixed 8B structure at 30%/40%/50% improves Avg12 by 4.6–10.8 percentage points.

ii) Fixing the tuning recipe to test CoCurve's structural advantage. After all methods receive one LoRA epoch, CoCurve leads on Avg12 at all four 8B ratios. At 30–50%, it exceeds the best baseline at each ratio by 0.7–1.8 percentage points, with lower Wiki PPL as well. CoCurve with reconstruction also retains its advantage against native pipelines across three models. See vvVx Part 1.1. Both comparisons support the same conclusion: CoCurve already preserves structures more useful for downstream tasks during pruning, and this advantage persists after recovery and tuning.

Attribution: interaction modeling is the main source of structure-selection gains. Even with fixed quotas, edge-conditioned reordering contributes most downstream gains and consistently improves pruning quality after per-layer quotas are matched. Joint allocation adds downstream gains at 20% and 30%. Held-out selection stabilizes structure selection, while recovery and tuning further improve the retained structures' performance.

AuthorsOpenReview ↗

Response to pgLc (3/4): Bridge Units in Natural Pruning

Part 2. Natural Pruning (W2 / Q3)

We agree that both retention and its downstream benefit need direct evidence from natural pruning. The submitted Figure 8(b–d), now Figure 11(b–d), connects coupling between units, removal-decision reversals, and downstream performance. We now test this mechanism systematically during natural pruning: first quantifying how often accumulated interactions change the next selection and whether CoCurve consequently retains more bridges in Part 2.1, then testing how these naturally retained units contribute to downstream gains in Part 2.2.

Part 2.1. Natural Decisions (W2 / Q3)

Takeaway. Accumulated edges change nearly four-fifths of stepwise choices, and CoCurve retains more bridges on average in all six models.

We replayed the original pruning trajectories of six language models at five ratios from 10%–50%. At each step, we fix the actual removal set and candidate pool, comparing the next removal with accumulated edges versus node-only scores. We also run node-only Diagonal from scratch to the same target ratio. For both methods, retention is the fraction of eligible bridge units retained. Each model is averaged equally across the five ratios, and the overall row weights all 30 configurations equally.

ModelChanged / total stepsBridge retention: Diagonal → CoCurveΔ retention (pp)
Falcon3-7B602/758 (79.4%)40.0% → 58.0%+18.0
Llama-3B819/972 (84.3%)51.7% → 60.3%+8.6
Llama-8B895/1044 (85.7%)60.2% → 61.7%+1.5
Llama-70B1743/2520 (69.2%)53.5% → 61.5%+7.9
Nemo-12B849/1308 (64.9%)56.1% → 56.3%+0.2
Mistral-24B985/1062 (92.7%)61.2% → 63.8%+2.6
Overall5,893/7,664 (76.89%)53.8% → 60.3%+6.5

Interactions reverse 5,893/7,664 node-only decisions (76.89%). In 90.11% of the distinct reversal contexts, no single edge explains the reversal. The benefit of reordering is reflected in the final structures: with per-layer quotas matched and only the selected units changed, adding interactions improves downstream performance in 11/12 comparisons in Part 1.1.

CoCurve retains more bridges on average in every model and, averaging across models, at every tested pruning ratio. Across the 30 configurations, mean retention rises from 53.8% to 60.3% (+6.5 pp). At 30%–50% pruning, the mean advantage across models is 9.2–11.2 pp. We next test how naturally retained units contribute to downstream gains in Part 2.2.

Part 2.2. Behavioral Effects (W2)

Takeaway. Units retained by CoCurve during natural pruning contribute to downstream gains.

To test the contribution of natural retention to downstream gains, we start from CoCurve's actual pruned structures and swap out units it retains but node-only scores are more inclined to remove, restoring units from the same layer, type, and cost group to preserve per-layer pruning budgets. On 8B at 20% pruning, swapping out four naturally retained units, three of them bridges, lowers Avg12 by 0.675 percentage points.

On 8B at 30% pruning, swapping out more naturally retained units increases both test KL and C4 PPL:

Matched swapsIndependent ΔKLC4 PPL
0+0.000031.989
4+0.002132.027
8+0.019032.583
16+0.112835.971

In a separate bridge-specific control on 8B and 12B at 20% and 30% pruning, removing naturally retained bridges raises test KL more than removing matched units. Thus, interactions change which units survive natural pruning, and these retained units are a source of downstream gains.

AuthorsOpenReview ↗

Response to pgLc (4/4): Joint Removal and Efficiency Conditions

Part 3. Why Joint Removal (W3 / W4)

Takeaway. Unit-wise scores omit how perturbations combine. Accounting for interactions improves pruning quality under fixed quotas.

We really appreciate your perceptive observation that the Introduction could make the case for joint removal more explicit. To keep the Introduction concise and avoid introducing method and experimental details prematurely, the paper uses shared residual streams to motivate joint removal, leaving the specific mechanism and validation to later sections. The revision will strengthen the logical link from unit-wise scoring to joint removal and point to the supporting evidence. We supply the two steps here: why unit-wise scores miss combination risk, then how selection behavior and quality validate the practical role of interactions.

① Why unit-wise scoring misses risk (W3). Attention and FFN outputs jointly affect final predictions through residual pathways. Single-removal scores measure individual perturbation magnitudes. Damage from combined removals also depends on how these perturbations reinforce or cancel in the same prediction space. Thus, two candidates with similar single-removal damage can cause different additional damage to an already-pruned model. This conditional dependence follows naturally from the unified predictive risk: the second-order teacher-KL expansion contains single-removal and interaction terms (Eqs. (1)–(2)). In the paper's estimated and calibrated risk approximation, taking the risk increment from SS to S∪{u}S\cup\{u\} and dividing by parameter cost yields the conditional marginal:

q(u∣S)=Huu2cu+λ∑v∈SHuvcuq(u\mid S)=\frac{H_{uu}}{2c_u}+\lambda\sum_{v\in S}\frac{H_{uv}}{c_u}.

The first term depends only on unit uu, while the second varies with the removal set SS. Fixed unit-wise scores omit the latter and cannot express the same unit's cost in different removal combinations, which is why pruning must be treated as a joint-removal problem. This information affects actual selection: across 30 natural-pruning trajectories on six LLMs, accumulated interactions change the next selection in 5,893/7,664 steps (76.89%), with the removal set and candidate pool fixed; see Part 2.1.

② From coupling magnitude to practical effect (W4). The observed 0.74–0.98× magnitude motivates testing cross-module relationships. The value of joint modeling is further supported by predicting combination damage and improving quality under fixed quotas. At a 20% budget on 8B, predicted risk and measured KL have a rank correlation of 0.866 across 26 approximately equal-budget removal sets, showing that the risk model distinguishes combinations with similar removal amounts but different damage. More directly, after matching removal counts within each layer, type, and cost group, interaction-aware selection improves Avg12 in 11/12 configurations on 3B/8B/24B and lowers independent KL by 9.2–31.5% at all four 24B ratios. See Part 1.1. With identical quotas, changing only which units are removed together reduces prediction damage and preserves more downstream capability, providing direct performance evidence for joint modeling.

Part 4. Efficiency Conditions (Q2)

Takeaway. The 1.58× prefill speedup and 41% peak-memory reduction both correspond to 50% pruning.

Table 5 compares physically sliced Llama-3.1-8B-Instruct with the original model on the same dedicated A100-SXM4-80GB (bf16/eager). Prefill uses batches of 32 with 2,949 valid input tokens in total and a per-sequence cap of 2,048; decode uses batches of 16 and generates 128 tokens. Each measurement is repeated 7 times.

ρ\rhoParams BPrefill speedupPeak GBPeak reduction %
20%6.6331.207×14.29016.83
30%5.9331.282×12.94424.66
40%5.2351.412×11.53532.87
50%4.5341.580×10.10341.20

All pruning ratios are defined over prunable projection parameters and use the same protocol. At 20%, more projection parameters remain, yielding 1.21× prefill speedup and 16.83% lower peak memory, while at 50%, total model parameters fall by 43.53%, and smaller matrices yield larger savings. We will specify 50% pruning alongside the abstract's efficiency figures and point to the measurement protocol.

Thank you for the detailed feedback. We will connect these attribution and natural-pruning results more directly to the framework in the revision, and would appreciate your reassessment of its presentation in light of these clarifications.

Reviewer_vvVxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CoCurve to address set-dependent structured pruning in LLMs and VLMs. By formulating structured pruning as set-dependent predictive risk via a local teacher-KL second-order expansion, CoCurve builds a reusable Fisher-Gram matrix from single-unit forward ablations. This captures both within- and cross-module interactions to guide conditional greedy pruning.

Critical strengths
  1. Sound mathematical motivation utilizing teacher-KL curvature without requiring training-loss stationarity, yielding a positive semidefinite Fisher-Gram estimator.

  2. Comprehensive empirical validation spanning 6 LLMs and 3 VLMs across perplexity corpora, 12 downstream tasks, and physical slicing deployments.

  3. Rigorous mechanism analyses including fixed-allocation controls, bridge units interventions, and cross-module interaction profiling.

Critical weaknesses
  1. Baseline comparisons isolate unified selection signals by stripping away native reconstruction steps , rather than providing end-to-end comparisons against full native pipelines.

  2. The novelty boundary relative to prior second-order and interaction-aware pruning methods requires more precise positioning.

  3. Non-negligible computational overhead for graph construction and candidate search, alongside potential inconsistencies between runtime masked evaluation and physically sliced low-bit models.

Questions for authors
  1. How does CoCurve's quality advantage and total cost scale when retaining the native compensation/reconstruction steps of FLAP, SlimGPT, or SlimLLM?

  2. What are the key empirical differences and isolated contributions relative to SOSP-I through a direct controlled comparison?

  3. Do the performance metrics of the runtime-masked version perfectly match those of the physically sliced low-bit checkpoints?

4 comments
AuthorsOpenReview ↗

Response to vvVx (1/4): Native Pipelines and Shared Recovery

Thank you for your incisive questions and for recognizing our sound mathematical motivation, broad validation, and rigorous mechanism analyses. We address complete native pipelines in Part 1, differences from related methods and SOSP-I in Part 2, quality and cost in Part 3, and low-bit agreement in Part 4.

Part 1. Native Pipelines (W1 / Q1)

The paper compares pruning outcomes produced directly by each method's structure selection within a standard evaluation framework, isolating the pruning strategy's contribution and establishing CoCurve's advantage in structure selection (best PPL in 38/45 comparisons across five models; highest Avg12 in all 8/8 configurations on 8B/24B).

To address W1/Q1, we retain the baselines' native compensation and reconstruction for end-to-end comparisons of complete native pipelines in Part 1.1, then test whether the quality advantage persists after equal recovery in Part 1.2. Each pipeline's total cost, the trade-offs after reuse, and quality comparisons at similar total times are reported in Part 3.

Part 1.1. Full Pipelines

Takeaway. Both CoCurve implementations lead in most configurations against complete native pipelines across three models.

Each baseline retains its native selection and recovery: FLAP's channel allocation and bias compensation, SlimGPT's sequential selection and OBS reconstruction, and SlimLLM's inter-layer allocation, channel selection, and local regression.

We implement CoCurve's complete pipeline with two existing reconstruction approaches, comparing each against the three native pipelines. CoCurve + OBS fixes the original mask and uses OBS to compensate retained weights. CoCurve + FFN Refinement preserves CoCurve's selected attention heads and per-layer quotas, reusing SlimGPT's channel selection and reconstruction within FFNs. All pipelines are evaluated after selection and recovery.

The table reports Avg12↑ for three models. Each cell lists 30%/40%/50% pruning. The first five rows use no LoRA. The last row, CoCurve + LoRA, reruns the released masks with the paper's one-epoch recipe.

PipelineLlama-3.2-3BLlama-3.1-8B-InstructMistral-Nemo-12B
FLAP native42.05 / 39.44 / 34.9847.73 / 44.04 / 40.5048.02 / 40.68 / 36.32
SlimGPT native38.82 / 37.05 / 35.7952.40 / 46.45 / 42.4842.31 / 38.20 / 38.43
SlimLLM native43.23 / 38.94 / 35.7652.83 / 45.82 / 37.7349.32 / 43.50 / 38.24
CoCurve + OBS46.82 / 40.74 / 36.6753.35 / 46.84 / 41.3750.31 / 45.28 / 40.34
CoCurve + FFN Refinement47.23 / 41.79 / 36.9053.56 / 48.46 / 43.0853.83 / 47.48 / 42.63
CoCurve + LoRA47.29 / 41.89 / 38.8353.68 / 49.62 / 45.3656.14 / 51.23 / 46.59

Against the strongest native baseline in each configuration, CoCurve + OBS achieves higher Avg12 in 8/9 configurations, while CoCurve + FFN Refinement achieves higher Avg12 in 9/9 and the lowest C4 in 8/9. Thus, CoCurve's structure-selection advantage carries through to complete pipelines across all three models.

Part 1.2. Shared LoRA

Takeaway. After equal recovery, CoCurve still leads in most configurations on each of Avg12 and C4.

The paper applies the same LoRA recovery to structures pruned by each selector, testing whether CoCurve's structures provide a better starting point for lightweight adaptation under the same budget (Tables 3, 16). We extend this comparison to complete native pipelines: each reconstructed model receives the same one-epoch Alpaca LoRA (rank 16), testing whether the quality advantage persists after equal adaptation. Each cell lists 30%/40%/50%, with C4↓/Avg12↑ at each ratio.

Pipeline + same LoRA8B12B
FLAP native19.40/52.87 · 23.34/49.23 · 29.16/46.8415.47/56.03 · 19.55/52.09 · 25.85/47.63
SlimGPT native17.38/56.98 · 20.15/52.31 · 24.35/47.6613.10/56.92 · 15.22/46.85 · 18.12/48.90
SlimLLM native17.62/56.97 · 21.57/53.43 · 27.49/47.9113.45/58.09 · 16.02/53.81 · 20.60/47.99
CoCurve + FFN Refinement16.05/56.67 · 18.72/52.45 · 23.10/48.0012.38/58.58 · 14.47/54.17 · 18.16/48.99

Across the six 8B/12B configurations, the CoCurve combination achieves the lowest C4 in five and the highest Avg12 in four, showing that its advantage persists after every pipeline completes reconstruction and receives the same adaptation opportunity.

AuthorsOpenReview ↗

Response to vvVx (2/4): Core Differences and SOSP-I Controls

Part 2. Core Differences (W2 / Q2)

CoCurve's novelty boundary lies in estimating prediction-preservation risk from finite responses to complete removals and using it for signed conditional selection. We contrast related methods in Part 2.1, then test empirical differences and isolated contributions against SOSP-I in Part 2.2.

Part 2.1. Core Differences

Takeaway. CoCurve uses signed responses to complete removals to decide what to remove next, given the current removal set.

Section 5 and Appendix F.1 discuss these methods. The table summarizes their differences and effects.

MethodModeling targetDecision ruleCoCurve: difference → effect
CBSLoss change from jointly zeroing weightsJoint selection; compensates remaining weightsWhole groups, fixed weights → direct slicing and smaller inference matrices
SOSP-ILocal training-loss gradients/curvatureAccumulates absolute sensitivities with the removal setSigned complete-removal responses → reinforcement vs cancellation
Kwon et al.Head/filter Fisher relations within blocksBlock quotas, then reordering/mask tuningJoint cross-block selection/allocation → lower-risk combinations
SAViTHead/hidden/embedding type interactionsType ratios, then importance-based unit removalSpecific unit pairs → distinguishes risk at identical ratios
LLM SurgeonWeight row/column curvature and compensation costPer-structure allocation; repeated re-estimation/compensationUpdates combination risk on one graph → reuse across ratios
BonsaiScalar performance of sampled submodelsFits additive unit scores; iterative pruningSigned interactions → distinguishes joint damage at equal summed scores
CoCurvePrediction changes from complete removalsUpdates risk/cost given removed units to select next unitMarginal combination risk → more compatible removals at fixed budget

Both SOSP-I and CoCurve update selection as the removal set grows. The key difference is how they estimate the next removal's damage. SOSP-I by default accumulates absolute local sensitivities of the training loss. CoCurve instead measures deviation from the frozen model's predictions using teacher-KL, which is zero before pruning. Complete-removal responses (isolated contribution 1) provide a signal closer to the actual removal decision. Two units with similar derivatives at the unpruned model can cause different prediction perturbations when fully removed. CoCurve incorporates this measured difference into the ranking.

Signed conditional selection (isolated contribution 2) further uses these response directions to determine which units are compatible for joint removal. Consider two candidates with equal single-removal damage and parameter cost, and equal coupling magnitudes to the removal set. If one amplifies the existing perturbation while the other partly cancels it, the latter is preferable at this step. CoCurve includes this distinction in marginal risk per parameter, favoring combinations expected to perturb predictions less under the same budget. The two controls in Part 2.2 hold the selection rule and response graph fixed, respectively, to test the quality gains from these two key changes relative to SOSP-I.

Part 2.2. Controlled Effects

Takeaway. CoCurve outperforms SOSP-I with equal calibration. Complete-removal responses and signed conditional selection each yield measured gains.

All comparisons use CoCurve control. We test overall empirical differences against recalibrated SOSP-I and two isolated contributions: complete-removal responses and signed conditional selection. The table reports pre-recovery Avg12 (%) on Llama-3.1-8B-Instruct. Variants share structural units, calibration text and candidate limits.

VariantInformation / selection20%30%40%50%
SOSP-ILocal / NLL + absolute, default53.5946.5836.9030.16
Calibrated SOSP-ILocal / NLL + absolute, calibrated55.4048.0738.3732.73
Teacher-localLocal / teacher-KL + signed conditional53.6148.8239.1334.31
Static finiteSame finite graph / fixed absolute coupling54.4346.8337.8431.86
CoCurve controlFinite / teacher-KL + signed conditional56.3950.5540.6536.81

① Overall difference. With equal calibration, CoCurve leads SOSP-I at all four ratios by 0.99–4.08 percentage points.

② Isolated contribution 1: complete-removal responses. Teacher-local uses SOSP-I's local derivatives with CoCurve's teacher-KL objective and signed conditional rule. Changing only to complete-removal responses improves all four ratios by 1.52–2.78 pp.

③ Isolated contribution 2: signed conditional selection. Static finite sums absolute couplings into fixed scores. On the same graph, signed risk updated for the removal set improves all four ratios by 1.96–4.95 pp.

Together, the overall comparison and two isolated controls support CoCurve's advantage over SOSP-I with measured gains from each design.

AuthorsOpenReview ↗

Response to vvVx (3/4): Graph Reuse and Quality–Cost Trade-off

Part 3. Quality and Cost (W3: computational cost / Q1)

We separate the computational overhead of graph construction and candidate search into first-run cost and reuse in Part 3.1, then assess the quality and cost of complete native pipelines in Part 3.2.

Part 3.1. Build Once, Reuse

Takeaway. Each base model needs one graph. A new pruning ratio only requires calibrating its own candidates.

For a fixed base model and structural units, one graph serves all pruning ratios (only 2.36 MB for 8B and under 15 MB for 70B). Structures selected at each ratio can also be used directly in reconstructed and LoRA-adapted versions of that base model. See pgLc Part 1.3.

① First-run cost. On an A5000, the first complete 8B pipeline at 50% pruning takes 24.61 hours: 20.69 hours for one-time graph construction, 1.16 hours for candidate search, and 2.76 hours for slicing, FFN reconstruction, and one LoRA epoch. Graph construction alone varies by device: 12.6 hours on RTX 6000 Ada, 3.70 hours on H100, and 8.41 hours on A100.

② Reuse of available results. As more results become reusable, less global selection work is needed:

  • Graph available: Generate and score candidates for the new ratio: 40.8/27.4 minutes for 8B at 30%/50% on A100.
  • Graph + target-ratio calibration coefficients available: Compile on CPU: 39/79 milliseconds for 8B at 20%/50%. All ten released masks across five models replay exactly.
  • Released target-ratio structure available: Skip search and compilation. At 50% pruning, calibration and reconstruction take only 37.6/48.8 minutes for 8B/12B on A100. Without LoRA, C4 and mean downstream performance remain better than the three native pipelines. See Part 1.1.

We next compare amortized time and final quality to show how reusable structure selection delivers higher quality at similar total cost in deployment. See Part 3.2.

Part 3.2. Quality–Cost Trade-off

Takeaway. CoCurve's complete pipelines retain a quality advantage at similar total times after reuse.

① Amortization: selection versus recovery. For 8B at 50% on A5000, CoCurve includes one quarter of graph-construction time plus search and recovery. LLM-Pruner uses its eight-epoch setting with the highest Avg12, counting only training and saving. Full budget curve: mpNv Part 2.2.

8B, 50% pruningTime (h)C4 PPL ↓Avg12 (%) ↑
CoCurve + 1 LoRA epoch≤8.70 (amortized)33.8945.15
CoCurve + 4 LoRA epochs≤14.66 (amortized)32.4445.61
LLM-Pruner + 8 LoRA epochs19.39 (training + save)39.2145.13

With one LoRA epoch, CoCurve reaches the downstream performance of LLM-Pruner's best setting in less than half the time, with lower C4. After four epochs, it still uses less time and improves both quality metrics. A better structural starting point reduces the recovery training needed to reach the same quality.

② Complete pipelines after reuse: quality and cost. Separate cost runs at 50% pruning use A5000 for 8B and A100 for 12B, excluding evaluation. Δ denotes CoCurve + FFN Refinement relative to the corresponding native pipeline.

i) Direct reuse of released structures. We first compare complete preparation time and quality without LoRA, against SlimGPT native.

ModelCoCurve prep. (min)SlimGPT prep. (min)ΔC4 PPL (%) ↓ΔAvg12 (pp) ↑
8B51.648.7-40.0%+0.65
12B54.350.0-9.3%+4.58

Both models achieve higher quality with similar complete preparation times.

ii) Equal recovery. Apply the common one-epoch LoRA from Part 1.2. CoCurve reuses its graph for 8B and released structure for 12B. Graph-reuse time sums measured stages.

ModelNative baselineCoCurve reuse (h)Baseline total (h)ΔC4 PPL (%) ↓ΔAvg12 (pp) ↑
8BSlimGPT3.9172.946-6.3%+0.23
8BFLAP3.9172.348-22.0%+1.05
8BSlimLLM3.9172.098-17.5%+0.19
12BSlimGPT2.3222.557+0.1%-0.15
12BFLAP2.3221.689-29.8%+1.14
12BSlimLLM2.3221.691-12.0%+0.90

After common recovery, CoCurve has lower C4 and higher mean downstream performance in all three 8B comparisons. At 12B, CoCurve leads FLAP and SlimLLM on both quality metrics.

iii) Similar total time. Each pipeline receives four LoRA epochs. CoCurve again sums measured stages after graph reuse.

8B, 50% pruningTime (h)C4 PPL ↓Avg12 (%) ↑
CoCurve + FFN Refinement9.7123.0048.16
SlimGPT native9.3724.0948.11
SlimLLM native8.1727.7247.83
FLAP native9.2328.3948.02

Within roughly 8–10 hours, CoCurve achieves the lowest C4 (4.5–19.0% lower) and the highest Avg12. The complete A100 recovery sweep is in mpNv Part 2.2.

Graph construction is performed offline, while the structural quality gains remain in the deployed models. Multiple pruned versions of the same base model share this construction cost. Together with the quality advantage at similar total times, this gives CoCurve a better quality–cost trade-off in practical deployment.

AuthorsOpenReview ↗

Response to vvVx (4/4): Low-Bit Agreement Between Masking and Slicing

Part 4. Low-Bit Agreement (Q3 / W3: deployment)

Takeaway. NF4 physical slicing closely matches runtime masking, with an Avg12 difference below 0.09 percentage points.

We compare runtime-masked and physically sliced Llama-3.1-8B-Instruct with the same mask: FP32 tests prediction agreement, while NF4 tests differences in task performance. Arrows run from masked to sliced.

8B comparisonDirect agreementNumerical gap / ΔPPL% (Wiki / PTB / C4)
FP32 numerical probe, 20% / 50%Top-1 agreement 100% / 100%Max logit gap 5.63e−5 / 4.00e−4
NF4, 30%Avg12: 47.31 → 47.37 (+0.057 pp)-0.26 / -0.35 / -0.29
NF4, 40%Avg12: 40.07 → 40.12 (+0.056 pp)+0.04 / -0.63 / +0.09
NF4, 50%Avg12: 34.43 → 34.34 (-0.089 pp)-1.22 / -2.03 / -1.42

Slicing applies the same head and channel indices to connected projections, removing the same functional paths as masking. Changes in matrix shapes and quantization groups introduce small numerical differences: top-1 predictions agree throughout the FP32 probes, and the maximum relative NF4 PPL difference across the three corpora is 2.03%. Physical slicing also delivers resource savings: in the paper's isolated A100 measurements, bf16 deployment of 8B at 50% pruning achieves 1.58× prefill and 41.20% lower peak generation memory.

Thank you again for your constructive feedback, which has helped us strengthen the comparisons and clarify CoCurve's contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.