- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Deep Low-Rank Projector (DLRP), a two-stage approach for KV-cache compression. It first trains deep linear projectors on a frozen LLM using KL divergence and a low-rank regularizer, then selects layer-wise compression ranks based on singular-value spectra. The reduced projectors are fine-tuned and folded into the attention weights for inference. Experiments on Qwen3 and Llama-3 models evaluate downstream accuracy, long-context performance, and serving efficiency.
Critical strengths
DLRP achieves better accuracy than the evaluated low-rank compression baselines at matched compression ratios, with relatively small performance degradation. The two-stage training and rank-allocation ablations provide useful evidence for the design choices. Unlike many KV-cache compression papers, the authors also implement a custom Triton kernel and report actual GPU memory usage and serving throughput. The ability to fold projectors into attention weights and combine DLRP with token-axis compression is practically useful.
Critical weaknesses
-
Unfair baseline comparisons. DLRP uses substantial additional training (100K samples in Stage 1 and 20K in Stage 2), while the compared methods are not given equivalent fine-tuning. Without training-matched comparisons, it is difficult to determine whether the improvements come from the proposed method or additional optimization.
-
Insufficient evidence for the proposed regularizer. The theoretical result provides an upper bound on the nuclear norm but does not establish that the regularizer improves compression performance. The paper lacks an α = 0 ablation at matched compression ratios, making its actual contribution unclear.
-
Inconsistent experimental comparisons. Table 5 appears to mix compression ratios for Qwen3-8B. Specifically, the full DLRP score of 66.30 corresponds to 30.1% compression, while several ablation results correspond to 32.4%. This inconsistency undermines the validity of the ablation conclusions.
-
Limited compression and serving efficiency evaluation. The evaluated compression ratios are relatively modest (approximately 23%–36%), leaving performance under more aggressive compression unexplored. Furthermore, decoding throughput decreases from 14.4 to 11.0 tokens/s at 64K context length, despite reduced KV-cache memory usage.
-
Limited methodological novelty. The framework primarily combines deep linear parameterization, low-rank regularization, SVD-based rank selection, and recovery fine-tuning. These components are individually well established, and the paper does not sufficiently demonstrate which aspects provide improvements beyond existing low-rank compression techniques.
Questions for authors
-
Implementation details: Are the projectors shared across attention heads or trained independently? How are Qwen3's Q/K normalization and RoPE handled after dimensionality reduction?
-
Long-context evaluation: Why does RULER Single-NIAH accuracy increase with context length for all compressed methods in Table 3? Could the authors evaluate more challenging RULER tasks?
-
Evaluation protocol: Could the authors clarify the dataset splits, evaluation metrics, and number of runs used for the short-context benchmarks? Some reported accuracy values appear inconsistent with standard test-set sizes.
-
Memory measurements: Could the authors report GPU memory consumption at identical batch sizes? Table 7 currently compares peak memory under different maximum batch sizes, making the reported memory savings difficult to interpret.
Thanks for detailed rebuttal
The rebuttal has adequately addressed my main concerns. I maintain my positive assessment of the paper and my Weak Accept recommendation.