- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes FTC, a post-training compression framework for large language model attention layers that aims to reduce model storage while preserving accuracy. FTC jointly compresses query, key, and value projections using a shared Tucker representation, incorporates activation information to guide compression, and processes layers sequentially to account for changes introduced by earlier compression. It also optimizes the allocation of compression budgets between tensor ranks and sparse-core coefficients and handles the attention output projection separately.
Critical strengths
- FTC introduces an activation-induced metric to guide Tucker decomposition, improving compression quality by accounting for the functional importance of model weights.
- FTC uses activations from the current compressed model to guide subsequent layer compression, accounting for representation shifts caused by earlier compression.
Critical weaknesses
-
Although the paper reports compression overhead, it does not evaluate end-to-end inference latency or throughput. It remains unclear whether FTC's more complex compressed representation provides practical inference acceleration.
-
FTC compresses only attention projections, which account for a relatively small fraction of parameters in modern GQA models. Consequently, even aggressive attention compression yields modest whole-model storage savings, limiting its practical impact.
Questions for authors
See the weakness