- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TrueSlice selects one shared residual-stream subspace using a sum of trace-normalized activation second moments, then absorbs the rotation into dense weights and slices the residual width. A common basis removes the residual basis transformations required by layer-specific rotations. The method builds explicitly on SliceGPT and is evaluated across Llama 3, Qwen3, and a Qwen3.5 hybrid model, with additional recovery-training and combined-pruning experiments.
Critical strengths
- The method addresses a concrete deployment issue with a simple, understandable design. Trace normalization prevents high-magnitude residual locations from dominating pooled statistics, while a shared subspace yields ordinary reduced-width dense layers.
- Parameter accounting is careful: tied embedding/head untying is included, and comparisons match total parameter reduction rather than nominal width reduction. Appendix A.1 provides achieved counts.
- The empirical case is meaningful: normalization improves perplexity in 17/18 ablation comparisons; TrueSlice beats SliceGPT in 17/24 matched comparisons, including 15/16 at 10–20% reduction. At 30% on Llama-3.1-8B, recovery raises average accuracy to 58.0 versus 54.4 for SliceGPT. The paper also reports unfavorable settings rather than claiming uniform superiority.
Critical weaknesses
- The contribution is incremental: invariance and slicing follow SliceGPT, with trace-normalized global subspace selection as the main addition. A small comparison with alternative location-weighting rules would clarify its value.
- Table 3 shows only 1.4–3.4% throughput gains over SliceGPT in one batch-size-one setting, without measurement dispersion. At 30% compression, quality also improves in only 2/8 model comparisons. The practical benefit is therefore operating-regime dependent.
- Reproducibility needs clarification: Table 2 and Figure 4 use different perplexity procedures, and reduced-width RMSNorm formulas and recovery settings are insufficiently specified. Please explain these differences and provide the relevant configuration.
Questions for authors
- What explains the Table 2 versus Figure 4 perplexity difference, and can both methods be rescored consistently?
- Do hardware gains exceed repetition-level variability and persist at a larger batch or longer context?
- Please specify RMSNorm restoration and recovery settings, and clarify sensitivity to calibration-location weights.
5 comments
Thanks to the authors for the response, which addresses most of my concerns. I have one additional request: at 30% compression, SliceGPT is usually better, and TrueSlice’s advantage is mainly at 10–20%. Could you briefly compare what information each method discards at 30% and clarify the more reliable compression range for TrueSlice? Please include this analysis in the final version.
Response to Reviewer PUmj - 2
Dear Reviewer PUmj,
Thank you for the follow-up. At 30% compression, both methods discard residual-stream directions, but they impose different constraints on which directions are retained. SliceGPT selects location-specific subspaces, allowing a direction important at one location to be retained there even if it is less important elsewhere. TrueSlice selects one shared subspace from aggregated, trace-normalized activation statistics. Consequently, directions important at particular locations may be discarded if they receive insufficient weight in the global objective. Trace normalization balances relative reconstruction contributions, but does not guarantee preservation of locally important information. However, neither SliceGPT nor TrueSlice guarantee the preservation of downstream-sensitive information.
As the retained width decreases, this shared-subspace constraint can become more restrictive. This provides a plausible explanation for TrueSlice’s weaker results at 30%, although our experiments do not isolate it as the sole cause. Based on our tested configurations, 10–20% total-parameter reduction is the better-supported range for standalone TrueSlice; we would not claim a consistent advantage over SliceGPT at 30%. We will include this explanation and clarify the supported operating range in the final version.
Please also see our new general comment, “Clarifying TrueSlice’s role: a complementary pruning step.” The completed experiments show that modest TrueSlice reductions can contribute to larger overall compression budgets when combined with Týr-the-Pruner, IntraSlice, or FLAP. This complementary use does not remove the standalone limitation you identify, but offers another practical role for TrueSlice.
Response to Reviewer PUmj
Dear Reviewer PUmj,
Thank you for your careful review and detailed feedback. We address your concerns below.
Q1 Our previous explanation was misleading: Table 2 and Figure 4 use identical perplexity evaluation settings, but different model checkpoints. Table 2 uses WikiText-2 calibration; the recovery experiments use FineWeb-Edu calibration and Nemotron recovery training. Both evaluate on WikiText-2. Thus, the results reflect different model preparation, not different evaluation procedures. Rescoring with the same evaluation settings would not remove this distinction. We will clarify the datasets, splits, and checkpoint preparation in the revised paper.
Q2 We expanded the hardware evaluation to five repetitions at batch sizes 1 and 16 on a RunPod H100 80GB GPU using BF16, at matched approximately 30% total parameter reduction. Each sequence has 128 prompt tokens and 128 cached decode steps. We use 128 sequences per repetition at batch size 1 and 1,024 at batch size 16.
We randomize the initial method order and rotate it across repetitions, use five untimed warmup steps per batch, and synchronize the GPU around timed operations. We measure model-only cached decoding, excluding input preparation, transfers, and warmup; prefill is measured separately.
Mean throughput changes relative to SliceGPT, with approximate 95% paired-bootstrap intervals, are:
Intervals use 2,000 bootstrap resamples of the five paired complete repetitions, not individual token timings. Four intervals lie above zero, including Llama-3.1-8B and Qwen3-32B at batch size 16; two remain inconclusive. With five repetitions, these intervals warrant caution and describe timing variation for fixed checkpoints and inputs.
The benefit is configuration-dependent. At batch size 16, TrueSlice’s mean throughput differs from the dense model by only −0.9% to +2.7%, so gains over SliceGPT do not imply comparable gains over the dense model. For Llama-3.1-8B with 2,048-token prompts and 128 decode steps at batch size 1, five repetitions of 128 sequences show an 11.9% throughput gain over SliceGPT (95% paired-bootstrap interval: [3.1%, 21.9%]). This supports a gain in the tested longer-context setting; mean throughput is only 1.6% above the dense model. We will report these results, repetition-level standard deviations, and uncertainty intervals.
Q3 After slicing from width D to d, parameterless normalization retains D in its denominator. We restore native RMSNorm at width d without changing epsilon by scaling the residual stream by a = sqrt(d/D) and setting the effective norm gain to 1/a:
∣ax∣22/d+ϵ(1/a)(ax)=∣x∣22/D+ϵx.
We absorb the scaling into embedding weights and attention/MLP output projections, including biases, and restore native norms, including the pre-head norm. This preserves the sliced computation in exact arithmetic without additional runtime scaling operations. Keeping epsilon unchanged avoids affecting internal query/key norms that share the configuration value. We will add this derivation and specify the recovery datasets, training budget, trainable parameters, optimizer, learning-rate schedule, batch size, and sequence length.
Trace normalization minimizes mean relative squared reconstruction error:
U⋆∈argminU⊤U=Id∣S∣1∑s∈S∣Xs∣F2∣Xs−XsUU⊤∣F2.
It does not explicitly account for downstream sensitivity. We therefore compared it with scalar Fisher weighting across 18 configurations: Qwen3.5-4B, Llama-3.1-8B, and Qwen3-14B; 10%, 20%, and 30% reduction; and WikiText-2 and Alpaca.
We use 128 sequences for activation statistics and 1,024 from the same training split for Fisher estimation, with maximum length 2,048 tokens. Squared activation gradients of summed next-token loss, scaled by mean squared activation magnitude, give location scores. We normalize these to mean-one weights and reweight the trace-normalized second moments. This sequence-level empirical Fisher approximation requires backward passes but no parameter updates.
Trace normalization performed better on Qwen3.5-4B; Fisher weighting performed slightly better on the other two models. Many Fisher improvements were below 0.1 PPL; the largest was 0.77 PPL, reducing WikiText-2 perplexity from 18.82 to 18.05 for Llama-3.1-8B at 30% reduction (4.1%).