acceptodds
Under review as a conference paper at ICLR 2027

TrueSlice: Zero Inference Overhead Pruning via Global Residual Stream Rotation

Abstract

To alleviate the significant memory and computational costs of large language models, numerous structural pruning methods have emerged, including residual stream width reduction. However, some residual width pruning methods introduce architectural changes and additional inference overhead. We propose TrueSlice, a calibration-based method that reduces residual width using a single global rotation derived from trace-normalized activation second moments. By retaining the original model architecture, TrueSlice can be applied to both standard transformer and modern hybrid networks, such as those in the Qwen3.5 family. Trace normalization balances contributions across the network, while the shared rotation enables slicing without additional residual rotation parameters or operations at inference. At matched total parameter reduction targets of 10-30%, TrueSlice achieves lower WikiText-2 perplexity than SliceGPT in 17 of 24 comparisons across eight Llama 3 and Qwen3 models without recovery training. On Llama-3.1-8B at 30% target reduction, TrueSlice achieves higher average task accuracy than SliceGPT after 3B recovery tokens. TrueSlice also complements attention head and MLP feature pruning: at a 50% target parameter reduction on Llama-3.1-8B, the best tested combination with Týr-the-Pruner raises average accuracy across five reasoning benchmarks from 44.86% to 48.01%, a gain of 3.15 percentage points over Týr alone.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, structural pruning, model compression, residual stream pruning, orthogonal rotations, trace normalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PUmjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TrueSlice selects one shared residual-stream subspace using a sum of trace-normalized activation second moments, then absorbs the rotation into dense weights and slices the residual width. A common basis removes the residual basis transformations required by layer-specific rotations. The method builds explicitly on SliceGPT and is evaluated across Llama 3, Qwen3, and a Qwen3.5 hybrid model, with additional recovery-training and combined-pruning experiments.

Critical strengths
  1. The method addresses a concrete deployment issue with a simple, understandable design. Trace normalization prevents high-magnitude residual locations from dominating pooled statistics, while a shared subspace yields ordinary reduced-width dense layers.
  2. Parameter accounting is careful: tied embedding/head untying is included, and comparisons match total parameter reduction rather than nominal width reduction. Appendix A.1 provides achieved counts.
  3. The empirical case is meaningful: normalization improves perplexity in 17/18 ablation comparisons; TrueSlice beats SliceGPT in 17/24 matched comparisons, including 15/16 at 10–20% reduction. At 30% on Llama-3.1-8B, recovery raises average accuracy to 58.0 versus 54.4 for SliceGPT. The paper also reports unfavorable settings rather than claiming uniform superiority.
Critical weaknesses
  1. The contribution is incremental: invariance and slicing follow SliceGPT, with trace-normalized global subspace selection as the main addition. A small comparison with alternative location-weighting rules would clarify its value.
  2. Table 3 shows only 1.4–3.4% throughput gains over SliceGPT in one batch-size-one setting, without measurement dispersion. At 30% compression, quality also improves in only 2/8 model comparisons. The practical benefit is therefore operating-regime dependent.
  3. Reproducibility needs clarification: Table 2 and Figure 4 use different perplexity procedures, and reduced-width RMSNorm formulas and recovery settings are insufficiently specified. Please explain these differences and provide the relevant configuration.
Questions for authors
  1. What explains the Table 2 versus Figure 4 perplexity difference, and can both methods be rescored consistently?
  2. Do hardware gains exceed repetition-level variability and persist at a larger batch or longer context?
  3. Please specify RMSNorm restoration and recovery settings, and clarify sensitivity to calibration-location weights.
5 comments
AuthorsOpenReview ↗

Response to Reviewer PUmj

Dear Reviewer PUmj,

Thank you for your careful review and detailed feedback. We address your concerns below.

Q1 Our previous explanation was misleading: Table 2 and Figure 4 use identical perplexity evaluation settings, but different model checkpoints. Table 2 uses WikiText-2 calibration; the recovery experiments use FineWeb-Edu calibration and Nemotron recovery training. Both evaluate on WikiText-2. Thus, the results reflect different model preparation, not different evaluation procedures. Rescoring with the same evaluation settings would not remove this distinction. We will clarify the datasets, splits, and checkpoint preparation in the revised paper.

Q2 We expanded the hardware evaluation to five repetitions at batch sizes 1 and 16 on a RunPod H100 80GB GPU using BF16, at matched approximately 30% total parameter reduction. Each sequence has 128 prompt tokens and 128 cached decode steps. We use 128 sequences per repetition at batch size 1 and 1,024 at batch size 16.

We randomize the initial method order and rotate it across repetitions, use five untimed warmup steps per batch, and synchronize the GPU around timed operations. We measure model-only cached decoding, excluding input preparation, transfers, and warmup; prefill is measured separately.

Mean throughput changes relative to SliceGPT, with approximate 95% paired-bootstrap intervals, are:

  • Llama-3.2-1B: batch 1, +25.8% [17.1%, 37.2%]; batch 16, +5.7% [−1.2%, 15.4%].
  • Llama-3.1-8B: batch 1, −2.7% [−17.0%, 18.0%]; batch 16, +21.6% [16.7%, 26.4%].
  • Qwen3-32B: batch 1, +10.7% [2.9%, 18.8%]; batch 16, +16.6% [7.4%, 27.8%].

Intervals use 2,000 bootstrap resamples of the five paired complete repetitions, not individual token timings. Four intervals lie above zero, including Llama-3.1-8B and Qwen3-32B at batch size 16; two remain inconclusive. With five repetitions, these intervals warrant caution and describe timing variation for fixed checkpoints and inputs.

The benefit is configuration-dependent. At batch size 16, TrueSlice’s mean throughput differs from the dense model by only −0.9% to +2.7%, so gains over SliceGPT do not imply comparable gains over the dense model. For Llama-3.1-8B with 2,048-token prompts and 128 decode steps at batch size 1, five repetitions of 128 sequences show an 11.9% throughput gain over SliceGPT (95% paired-bootstrap interval: [3.1%, 21.9%]). This supports a gain in the tested longer-context setting; mean throughput is only 1.6% above the dense model. We will report these results, repetition-level standard deviations, and uncertainty intervals.

Q3 After slicing from width D to d, parameterless normalization retains D in its denominator. We restore native RMSNorm at width d without changing epsilon by scaling the residual stream by a = sqrt(d/D) and setting the effective norm gain to 1/a:

(1/a)(ax)∣ax∣22/d+ϵ=x∣x∣22/D+ϵ.\frac{(1/a)(ax)}{\sqrt{|ax|_2^2/d+\epsilon}}=\frac{x}{\sqrt{|x|_2^2/D+\epsilon}}.

We absorb the scaling into embedding weights and attention/MLP output projections, including biases, and restore native norms, including the pre-head norm. This preserves the sliced computation in exact arithmetic without additional runtime scaling operations. Keeping epsilon unchanged avoids affecting internal query/key norms that share the configuration value. We will add this derivation and specify the recovery datasets, training budget, trainable parameters, optimizer, learning-rate schedule, batch size, and sequence length.

Trace normalization minimizes mean relative squared reconstruction error:

U⋆∈arg⁡min⁡U⊤U=Id1∣S∣∑s∈S∣Xs−XsUU⊤∣F2∣Xs∣F2.U^\star\in\arg\min_{U^\top U=I_d}\frac{1}{|S|}\sum_{s\in S}\frac{|X_s-X_sUU^\top|_F^2}{|X_s|_F^2}.

It does not explicitly account for downstream sensitivity. We therefore compared it with scalar Fisher weighting across 18 configurations: Qwen3.5-4B, Llama-3.1-8B, and Qwen3-14B; 10%, 20%, and 30% reduction; and WikiText-2 and Alpaca.

We use 128 sequences for activation statistics and 1,024 from the same training split for Fisher estimation, with maximum length 2,048 tokens. Squared activation gradients of summed next-token loss, scaled by mean squared activation magnitude, give location scores. We normalize these to mean-one weights and reweight the trace-normalized second moments. This sequence-level empirical Fisher approximation requires backward passes but no parameter updates.

Trace normalization performed better on Qwen3.5-4B; Fisher weighting performed slightly better on the other two models. Many Fisher improvements were below 0.1 PPL; the largest was 0.77 PPL, reducing WikiText-2 perplexity from 18.82 to 18.05 for Llama-3.1-8B at 30% reduction (4.1%).

Thanks to the authors for the response, which addresses most of my concerns. I have one additional request: at 30% compression, SliceGPT is usually better, and TrueSlice’s advantage is mainly at 10–20%. Could you briefly compare what information each method discards at 30% and clarify the more reliable compression range for TrueSlice? Please include this analysis in the final version.

AuthorsOpenReview ↗

Response to Reviewer PUmj - 2

Dear Reviewer PUmj,

Thank you for the follow-up. At 30% compression, both methods discard residual-stream directions, but they impose different constraints on which directions are retained. SliceGPT selects location-specific subspaces, allowing a direction important at one location to be retained there even if it is less important elsewhere. TrueSlice selects one shared subspace from aggregated, trace-normalized activation statistics. Consequently, directions important at particular locations may be discarded if they receive insufficient weight in the global objective. Trace normalization balances relative reconstruction contributions, but does not guarantee preservation of locally important information. However, neither SliceGPT nor TrueSlice guarantee the preservation of downstream-sensitive information.

As the retained width decreases, this shared-subspace constraint can become more restrictive. This provides a plausible explanation for TrueSlice’s weaker results at 30%, although our experiments do not isolate it as the sole cause. Based on our tested configurations, 10–20% total-parameter reduction is the better-supported range for standalone TrueSlice; we would not claim a consistent advantage over SliceGPT at 30%. We will include this explanation and clarify the supported operating range in the final version.

Please also see our new general comment, “Clarifying TrueSlice’s role: a complementary pruning step.” The completed experiments show that modest TrueSlice reductions can contribute to larger overall compression budgets when combined with Týr-the-Pruner, IntraSlice, or FLAP. This complementary use does not remove the standalone limitation you identify, but offers another practical role for TrueSlice.

Reviewer_t3ddOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TrueSlice, a residual-width pruning method that uses a single global rotation derived from trace-normalized activation second moments. Sharing the retained subspace across the network eliminates additional residual adapters. The paper evaluates pruning quality, hardware performance, recovery training, and combinations with complementary structural pruning.

Critical strengths
  1. I like the motivation. Removing residual transformation overhead while preserving a conventional computational structure is a useful objective for practical model compression.
  2. The method is simple, and Table 1 provides encouraging evidence for trace normalization, improving perplexity in 17 of 18 comparisons.
Critical weaknesses
  1. Shared global rotations have been explored in prior work. The main methodological novelty is trace normalization in Equation 8, but why trace normalization works remains insufficiently justified. Besides trace normalization, this paper does not have other contribution.

  2. Table 3 shows only a small speedup over SliceGPT. Throughput improves by 1.4–3.4%, and latency decreases only slightly.

  3. The main comparison is only against SliceGPT. Please include at least some other structural pruning baselines, such as RotPruner.

  4. The presentation needs improvement. The tables and figures are hard to follow, and their layout and formatting could be clearer. The paper as a whole looks like a lightly modified version of SliceGPT’s LaTeX manuscript, including its overall structure and presentation.

Questions for authors

Please see weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer t3dd

Dear Reviewer t3dd,

Thank you for your careful review and detailed feedback. We address your concerns below. Please also see our general comment, “Clarifying TrueSlice’s role: a complementary pruning step,” for the completed combination experiments, full weighting results, and clarification of the paper’s focus.

1 Trace normalization minimizes the mean relative squared reconstruction error:

U⋆∈arg⁡min⁡U⊤U=Id1∣S∣∑s∈S∣Xs−XsUU⊤∣F2∣Xs∣F2.U^\star\in\arg\min_{U^\top U=I_d}\frac{1}{|S|}\sum_{s\in S}\frac{|X_s-X_sUU^\top|_F^2}{|X_s|_F^2}.

This objective does not guarantee low error at every location, including early layers, or explicitly account for downstream sensitivity. To address your request for a stronger baseline, we compared it with scalar Fisher weighting across 18 configurations: Qwen3.5-4B, Llama-3.1-8B, and Qwen3-14B; 10%, 20%, and 30% reduction; and WikiText-2 and Alpaca.

We estimate activation statistics from 128 sequences and Fisher weights from 1,024 sequences from the same training split, with maximum length 2,048 tokens. For each location, we compute squared activation gradients of the summed next-token prediction loss, scaled by mean squared activation magnitude. This sequence-level empirical Fisher approximation requires backward passes but no parameter updates.

Trace normalization performed better on Qwen3.5-4B, while Fisher weighting performed slightly better on Llama-3.1-8B and Qwen3-14B. Many Fisher improvements were below 0.1 PPL; the largest was 0.77 PPL, reducing WikiText-2 perplexity from 18.82 to 18.05 for Llama-3.1-8B at 30% reduction (4.1%). These results show that the two weighting schemes are competitive across the tested configurations.

2 We acknowledge that the original throughput gains were modest and configuration-dependent. We also acknowledge TrueSlice’s weaker standalone performance at 30% reduction and the potential limitation imposed by a single shared subspace. Its strongest standalone results occur at 10–20% reduction.

The completed combination experiments provide additional evidence of practical value at higher compression. At the 50% total-reduction target, the best tested combinations improve average zero-shot accuracy from 44.90% to 48.17% for Týr-the-Pruner, 37.00% to 49.65% for IntraSlice, and 43.64% to 44.90% for FLAP. These gains depend on the allocation: the tested combinations do not improve the 25% average, and some allocations underperform at 50%.

We will revise the paper to emphasize TrueSlice’s role as a complementary pruning step. Testing three methods is not exhaustive, but provides encouraging evidence of broader compatibility. Our general comment presents the complete results and discusses this revised emphasis.

3 We have completed direct comparisons with standalone IntraSlice and FLAP, alongside combinations of each with TrueSlice, at common total-parameter reduction targets. Achieved reductions are reported because structural pruning does not always attain the target exactly. These experiments assess both standalone performance and the benefit of combining methods.

We could not locate public RotPruner code; implementing and validating it remains beyond this rebuttal’s scope.

4 We will restructure the method section to distinguish our contributions and explain RMSNorm restoration, and improve table grouping and readability.

After slicing from width D to d, parameterless normalization retains D in its denominator. We restore native RMSNorm at width d without changing epsilon by scaling the residual stream by a = sqrt(d/D) and setting the effective norm gain to 1/a:

(1/a)(ax)∣ax∣22/d+ϵ=x∣x∣22/D+ϵ.\frac{(1/a)(ax)}{\sqrt{|ax|_2^2/d+\epsilon}}=\frac{x}{\sqrt{|x|_2^2/D+\epsilon}}.

We absorb the scaling into embedding weights and attention/MLP output projections, including biases, and restore native norms, including the pre-head norm. This preserves the sliced computation in exact arithmetic without additional runtime scaling operations. Keeping epsilon unchanged avoids affecting internal query/key norms that share the configuration value. We will include the derivation in the revised method section.

Reviewer_cAshOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TrueSlice, a structural pruning method that reduces the residual stream width of LLMs without introducing inference-time overhead (e.g., residual adapters). Building upon the rotation-based framework of SliceGPT, TrueSlice computes a single global rotation matrix by summing trace-normalized activation second-moment matrices across all measurement locations. This shared subspace allows the rotation to be permanently absorbed into the linear projections, preserving the original transformer architecture and enabling standard dense inference. The method is evaluated on Llama 3, Qwen 3, and hybrid Qwen 3.5 models, showing competitive perplexity and accuracy at 10–20% parameter reduction compared to SliceGPT, while offering better inference throughput. It is also shown to complement attention/MLP pruning methods like T´yr-the-Pruner.

Critical strengths
  • By enforcing a single global rotation, TrueSlice successfully eliminates the residual stream adapters required by SliceGPT. This results in a truly "zero-overhead" compressed model that can be deployed using standard, highly optimized dense inference kernels without custom architectural modifications.
  • The evaluation is extensive, covering multiple model families (including the hybrid Qwen3.5), various parameter scales, and detailed hardware profiling. The demonstration that the method integrates seamlessly with orthogonal pruning techniques (T´yr-the-Pruner) adds practical value.
Critical weaknesses
  • The primary novelty—trace-normalized summation of second moments—is a naive heuristic that assumes all layers' activation spaces are equally important globally. It ignores the varying sensitivity of different layers (e.g., embeddings vs. middle layers) to subspace truncation, which is clearly reflected in the high reconstruction errors at early layers (Figure 6). Furthermore, the ablation study only compares against a strawman "unnormalized sum" baseline, ignoring more principled global subspace selection methods (e.g., Fisher Information, gradient-based weighting, or learnable rotations).
  • While TrueSlice outperforms SliceGPT at 10–20% reduction, it consistently and significantly loses to SliceGPT at 30% reduction across almost all models (Table 2). For structural pruning to yield meaningful deployment benefits on modern hardware, higher compression rates are typically required. The rigid global subspace constraint fundamentally bottlenecks the method's utility for deep compression.
  • The related work explicitly discusses DISP-LLM and IntraSlice, which also aim to achieve structural pruning without introducing residual adapters. However, the experimental section omits empirical comparisons with these zero-overhead baselines, making it difficult to assess TrueSlice's true standing in the current literature.
Questions for authors

See weaknesses above

1 comment
AuthorsOpenReview ↗

Response to Reviewer cAsh

Dear Reviewer cAsh,

Thank you for your careful review and detailed feedback. We address your concerns below. We have also posted a general comment, “Additional results: TrueSlice as a complementary pruning method,” with the completed IntraSlice and FLAP combination experiments and the full Fisher-weighting comparison. We invite you to read it alongside this response.

1 We agree that the original submission insufficiently justified trace normalization. It is equivalent to minimizing the mean relative squared reconstruction error:

U⋆∈arg⁡min⁡U⊤U=Id1∣S∣∑s∈S∥Xs−XsUU⊤∥F2∥Xs∥F2.U^\star \in \arg \min_{U^\top U=I_d} \frac{1}{|S|} \sum_{s\in S} \frac{\|X_s-X_sUU^\top\|_F^2} {\|X_s\|_F^2}.

This minimizes the mean relative squared reconstruction error, but does not guarantee a small error at every measurement location or explicitly account for downstream sensitivity. You are correct in that trace-normalization does not account for differences in how sensitive layers are to the compression. This leads us to the Fisher weighting comparisons you suggested.

However, we also understand your critique that we require a better baseline, thus to evaluate an alternative location-weighting rule, we compared trace normalization with scalar Fisher weighting. We estimate activation statistics from 128 calibration sequences and Fisher weights from 1,024 sequences drawn from the same dataset’s training split (WikiText-2 or Alpaca), with a maximum length of 2,048 tokens. For each measurement location, we estimate sensitivity using squared activation gradients of the next-token prediction loss, scaled by the location’s mean squared activation magnitude. This sequence-level empirical Fisher approximation requires backward passes through the frozen model but no parameter updates.

Trace normalization performs better in all six Qwen3.5-4B configurations and one Qwen3-14B configuration. Fisher weighting improves the remaining 11 configurations by 0.01–0.77 perplexity. Its largest improvement reduces WikiText-2 perplexity from 18.82 to 18.05 for Llama-3.1-8B at 30% reduction (4.1%). The complete results appear in our general comment and will be incorporated into Table 1. They support trace normalization as a competitive default that avoids gradient computation.

2 We acknowledge TrueSlice’s weaker performance relative to SliceGPT at 30% reduction and the potential limitation imposed by a single shared subspace. Its strongest standalone results occur at 10–20% reduction. We will clarify that its practical role also includes contributing modest residual-width reductions within a broader pruning pipeline.

We have now completed the IntraSlice and FLAP combination experiments, extending the existing Týr-the-Pruner results. At the 50% total-reduction target, the best tested combinations improve average zero-shot accuracy from 44.90% to 48.17% for Týr, 37.00% to 49.65% for IntraSlice, and 43.64% to 44.90% for FLAP. These results support allocating part of a larger compression budget to TrueSlice.

The benefit depends on the allocation: the tested combinations do not improve average accuracy at the 25% target, and some allocations also underperform at the 50% target. Testing three methods is not exhaustive, but provides encouraging evidence of broader compatibility. Our general comment presents all results and explains the revised emphasis on TrueSlice as a complementary pruning step.

  1. The completed experiments include standalone TrueSlice baselines as well as Tyr, IntraSlice and FLAP baselines and their combinations with TrueSlice at common total-parameter reduction targets. We report achieved reductions as well, since structural pruning does not always attain the target exactly. This allows readers to assess standalone performance and the benefit of combining methods.

For DISP-LLM, the publicly available implementation supports Llama-1 and Llama-2, whereas our experiments use different model families. A matched comparison would require adapting and validating that implementation. DISP-LLM also uses training during compression, while our experiments focus on training-free methods. These differences do not eliminate its relevance as a baseline without residual adapters; we will explicitly acknowledge the missing direct comparison as a limitation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Clarifying TrueSlice’s role: a complementary pruning step

We thank the reviewers for encouraging broader comparisons. We have now completed IntraSlice and FLAP experiments alongside Týr-the-Pruner, strengthening a case underdeveloped in the submission: TrueSlice as a complementary pruning step. Its strongest standalone results occur at modest reductions, outperforming SliceGPT in nearly all tested 10–20% settings. At higher compression, allocating part of the pruning budget to TrueSlice can improve the quality obtained by all three methods. We have retitled the paper “TrueSlice: Residual Width Pruning for Structural Compression Pipelines” and clarified this role, adding a reconstruction objective, RMSNorm derivation, and broader evaluations.

Our motivation is to make residual-width reduction simple to incorporate into an existing pruning pipeline. The Fisher comparison supports trace normalization as a computationally simple default: it remains competitive without the additional backward passes required for sensitivity estimation, making TrueSlice cheap to include in an existing pipeline. Although testing three methods is neither exhaustive nor conclusive, the combination results provide encouraging evidence of broader compatibility and support TrueSlice’s potential as a generally useful addition to pruning pipelines.

SliceGPT remains a central direct baseline because it addresses the same residual-width compression problem. Our extensive comparison assesses the tradeoff introduced by TrueSlice’s shared subspace; the combination experiments assess its complementary value. DISP-LLM is also relevant, but we could not locate a public implementation supporting the tested model families. The absence of that direct comparison remains a limitation.

Combination results

Zero-shot accuracy (%) on Llama-3.1-8B. “a+b” allocates target percentage points of original-model parameter reduction to the listed method + TrueSlice, applied TrueSlice first. Single values denote the method alone. Scores are normalized accuracy except for WinoGrande (WG); Avg. is their unweighted mean.

MethodCutPIQAWGHellaSwagARC-eARC-cAvg.
TrueSlice2566.7662.7554.5655.6432.0854.36
TrueSlice5053.7051.0727.9129.5022.7836.99
Týr2575.7968.6768.6073.8644.9766.38
Týr15+1075.1469.6166.6074.4544.4566.05
Týr5063.9352.6441.1242.5924.2344.90
Týr40+1064.8555.0942.3143.3526.9646.51
Týr30+2065.4554.9341.1251.6427.7348.17
Týr20+3061.1053.5936.0645.4526.8844.62
IntraSlice2575.7370.1769.4375.0046.3367.33
IntraSlice15+1075.3069.8566.5473.8244.4565.99
IntraSlice5052.9450.3629.9829.3822.3537.00
IntraSlice40+1056.9653.2035.9339.9824.4042.10
IntraSlice30+2065.1858.9642.3651.8129.9549.65
IntraSlice20+3061.3256.8338.2645.7527.6545.96
FLAP2573.9465.0458.8764.8636.5259.84
FLAP15+1073.8363.9357.1765.4537.7159.62
FLAP5060.6152.9634.9143.4326.2843.64
FLAP40+1061.8654.7839.0643.6425.1744.90
FLAP30+2061.3254.3037.9043.2725.0044.36
FLAP20+3059.3653.9135.1540.4025.1742.80

At the 50% target, the best tested combinations improve average accuracy by 3.27, 12.65, and 1.26 percentage points for Týr, IntraSlice, and FLAP, respectively. Gains are not universal: no tested combination improves the 25% average. Actual IntraSlice reductions are 24.52–24.68% and 49.48–49.72%; FLAP is within 0.03 points of its targets.

Weighting results

Perplexity at 10% / 20% / 30% reduction in each cell; lower is better. W2 = WikiText-2, Al = Alpaca, using disjoint calibration/evaluation splits. Activation statistics use 128 sequences; Fisher weights use 1,024 sequences, with maximum length 2,048.

ModelDataUnnormalizedTrace-normalizedFisher
Qwen3.5-4BW215.51 / 25.44 / 47.0014.27 / 21.18 / 34.6915.38 / 23.28 / 39.00
Qwen3.5-4BAl9.86 / 14.51 / 19.497.44 / 10.72 / 16.648.54 / 14.07 / 22.88
Llama-3.1-8BW27.87 / 12.33 / 27.697.56 / 10.65 / 18.827.54 / 10.34 / 18.05
Llama-3.1-8BAl3.47 / 4.41 / 7.153.38 / 3.97 / 5.473.37 / 3.91 / 5.31
Qwen3-14BW29.73 / 11.94 / 22.569.65 / 11.41 / 15.679.52 / 11.29 / 15.39
Qwen3-14BAl3.86 / 4.21 / 5.403.87 / 4.18 / 5.213.84 / 4.07 / 5.24

Trace normalization improves over unnormalized aggregation in 17/18 settings. It beats Fisher weighting in 7/18; Fisher’s improvements in the other 11 range from 0.01 to 0.77 perplexity. Our Fisher estimate requires gradients but no parameter updates.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.