- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a KV cache compression method called InfoKV. It introduces entropy and cross-layer representation evolution as signals to evaluate token importance compared to traditional methods that only use attention weights. The experiments cover long prefilling and long decoding scenarios, and the performance of InfoKV is competitive across multiple models, tasks and budgets.
Critical strengths
-
The paper focuses on the problem that attention fails to evaluate token importance accurately under some specific contexts. It introduces entropy based on forward influence analysis as an importance signal, and the strategy is aligned with the intuition.
-
The experiments cover multiple settings, especially in terms of model size and KV budgets.
Critical weaknesses
-
Although main hyperparameter ablation studies and sensitivity experiments are included, some design choices and parameter settings do not have clear explanations or justification, such as the lack of an ablation study on cross-layer representation, and the lack of evaluation on some latest model families, e.g. Qwen3/3.5.
-
It should include more recent baselines, e.g. KeyDiff and R-KV.
-
The efficiency experiment only uses 128-token inputs, and does not quantify the extra computation introduced by entropy and cross-layer representation evolution under long prefilling.
Questions for authors
-
How does InfoKV perform under long prefilling? For example, how does it compare with baselines in terms of TTFT, peak GPU memory, etc.?
-
The authors may consider adding KeyDiff and R-KV as baselines. KeyDiff uses key similarity as a signal, while R-KV focuses on long-decoding scenarios.
-
Currently, the sensitivity analysis of (\tau) does not isolate the effect of cross-layer representation distance.
-
What are the parameter settings of the 14B model? Do they align with those of the 7B model?
-
The model DeepSeek-R1-Distill-Qwen-7B is based on Qwen2.5, which is relatively old. The authors may consider verifying the performance on some more recent model families. The Qwen3.5 series utilizes hybrid attention, which may make adaptation difficult; perhaps Qwen3 could be considered.
Dear Reviewer GnYe,
As the authors' response deadline is fast approaching, please read through their rebuttal and let them know whether your concerns have been addressed or further clarification is still required.
Thank you.
AC