- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes GraceKV, a training-free KV-cache compression method that jointly allocates cache resources across layers, KV heads, and context regions. The method builds multi-resolution prototype trees and uses global utility-based allocation to balance information coverage and local resolution. It also introduces curvature-guided segmentation, multi-hop relevance estimation, and a singleton-preservation mechanism.
Experiments on LongBench and RULER with multiple language models show strong performance across different compression ratios, especially for question answering and retrieval tasks.
Critical strengths
-
The paper considers KV-cache compression as a global resource allocation problem instead of using fixed eviction or merging rules. The multi-resolution representation provides a flexible way to balance information coverage and local resolution.
-
The method is evaluated on several models and long-context tasks with different compression ratios. It achieves competitive or superior performance in many settings, and the ablation studies support the importance of multi-resolution representation and global allocation.
Critical weaknesses
- GraceKV combines curvature-based segmentation, multi-hop relevance, head sensitivity, prototype distortion, and a singleton floor. Although the ablations show their effectiveness, it is still unclear whether the full complexity is necessary or whether a simpler global allocation method could achieve similar results.
Questions for authors
Can the authors compare GraceKV with a simpler global allocation method using only token importance or fixed-size merging? This would help clarify whether the multi-resolution prototype trees are necessary.
3 comments
Thank you for your rebuttal. I have no further questions.
Thank you for your time and effort!
We sincerely appreciate your careful review and constructive feedback on our work.
We sincerely thank you for your constructive comments — Experiments on global token selection and fixed-granularity merging.
Simple global token selection or fixed-granularity merging cannot match GraceKV’s performance.
We added two types of controls: global token selection and fixed-granularity merging. All methods use Qwen2.5-7B-Instruct with identical inputs and decoding configurations. We evaluate 200 samples each from the LongBench single-document QA and multi-document QA subsets, matching the actual context-cache budget for every sample.
Table 1. Comparison of GraceKV with simplified global allocation methods. All metrics are F1. The fixed block length is set to r tokens, where r is the numerical value of the compression ratio: 8 tokens at 8× and 64 tokens at 64×.
As shown in Table 1, GraceKV outperforms these controls across all the above settings, supporting the necessity of multi-resolution prototype trees.