- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CoFeatKV is a training-free method that compresses post-RoPE keys to shrink the KV cache in long-context inference. It slices keys into fixed-size pages along the sequence, compressing each page once it is full while keeping the active page uncompressed. Inside a sealed page it groups feature dimensions by attention importance, represents each group with one shared direction, and stores only a coefficient plus a group index per feature — much lighter than a dense projection matrix. At decode time it computes attention directly in this compressed space, so it never rebuilds keys or re-applies RoPE. On Qwen3-8B / WikiText-2 it saves up to ~45% of key storage and lifts decode throughput ~23–26%, with perplexity close to full KV cache.
Critical strengths
- The storage saving is structural rather than tuned, because each feature stores only one coefficient and a group index instead of a dense projection matrix, which cuts per-page projection cost by roughly an order of magnitude.
- Decoding gets faster instead of slower, since it needs no key reconstruction and no second RoPE pass, so the method does not trade speed away for the storage saving.
- It is training-free and streaming, computing grouping locally per sealed page with no retraining or global optimization, which makes it easy to drop into an existing inference pipeline.
Critical weaknesses
- RULER is already saturated, with both the method and full KV cache scoring near the ceiling. The evaluation would be more convincing with hard long-reasoning tasks such as AIME-style problems, which stress the cache far more than RULER or WikiText perplexity.
- There is no code release or reproducibility statement, the paper gives no repository link or supplementary material, so the results cannot be correctly verified or reproduced.
- Grouping is greedy and page-local with no global-optimization guarantee, and the calibration is frozen at sealing time, so it may mismatch queries that arrive later.
Questions for authors
The author stated that their benchmarks include MMLU and use Llama-2-7B. But I see no MMLU and Llama-2-7B experiments.
In your experiments, a page holds 32 tokens, so each group direction is a 32-dimensional vector. 32 dimensions can contain at most 32 independent directions. The paper asks for 64 to 96 group directions per page. That means most of those directions are just linear combinations of the first 32. Is this design reasonable?
I think this paper needs more efforts. A lot of things haven't been fully thought through.
We sincerely thank the reviewer for the careful review and constructive comments, and we apologize for the late response due to the time needed to run the additional experiments.
Q1: RULER is close to saturation. Should long-reasoning tasks such as AIME be included?
Response: We have added a long-reasoning evaluation on AIME 2024/2025. We use Qwen3-8B with thinking enabled on 60 problems in total, with a maximum generation length of 32,768 tokens and an average output length of about 16K tokens.
m=64m=80The accuracy of
m=80is close to that of FullKV;m=64decreases the overall accuracy by 2.50 percentage points, mainly on AIME 2025, reflecting the quality cost of stronger compression. These results show that CoFeatKV performs well in long-context settings beyond RULER: with outputs averaging about 16K tokens, m=80 matches FullKV (71.25% vs. 70.83%).Q2: The paper does not provide code, a repository link, or a reproducibility statement. How can the results be verified?
Response: We thank the reviewer for pointing this out. Our code is publicly available via an anonymous link: https://anonymous.4open.science/r/cofeatsvd-5155/.
Q3: The grouping is greedy and page-local, and it is frozen after sealing. Could it mismatch later queries?
Response: Paging, greedy assignment, and freezing after sealing are design choices made to control the cost of streaming inference; we do not claim global optimality. Once the grouping is determined, the shared direction of each group is the optimal solution to the weighted rank-one approximation problem for that group.
Regarding query mismatch, CoFeatKV can operate completely independently of queries. Denoting the query weights in the paper as wj, setting wj←1 for all j yields the Key-only variant: grouping and compression use only the keys themselves without any historical query information, so there is no mismatch with later queries. On Qwen3-8B at 32K with
m=64, the Key-only variant achieves 88.00 on FWE, comparable to 87.67 for the Q-weighted version; on multi-value NIAH, both achieve 99.17. This shows that the effectiveness of the method does not rely on query calibration.Q4: The paper mentions MMLU and Llama-2-7B. Why are there no corresponding results?
Response: We thank the reviewer for pointing this out. After checking, the MMLU experiment was actually conducted with Llama-3.1-8B-Instruct; the description of Llama-2-7B in the paper has no corresponding experimental results, which is our mistake. The MMLU experiment covers 57 subjects and 14,042 questions in the 0-shot setting. The results are as follows:
m=64)m=80)CoFeatKV
m=80is only 0.50 percentage points below FullKV, andm=64is 1.20 percentage points below, both higher than the other compression baselines. In the revised version, we will add these results, remove the descriptions related to Llama-2-7B, and unify the model and task lists in the abstract, experimental setup, and result tables.Q5: With a page length of only 32, why are 64–96 grouping directions needed? Does the linear dependence among directions indicate an unreasonable design?
Response: The observation about the rank is correct: a 32-dimensional space has at most 32 linearly independent directions. However, the number of groups m denotes the number of shared representative directions, not the rank of a matrix. In CoFeatKV, each feature can select only one representative direction and multiply it by one coefficient. Under this constraint, a direction may still be necessary even if it is a linear combination of other directions. For example, in a two-dimensional space, "northeast" can be obtained by combining "east" and "north," but it cannot be represented by scaling "east" or "north" alone.
Therefore, using more than 32 representative directions reduces the error of this "single direction + single coefficient" representation and does not require these directions to be linearly independent. If 32 basis vectors were used instead, each feature would typically require multiple combination coefficients, losing the current compact storage structure. Taking P=32, d=128, and m=80 as an example, the representative vectors, coefficients, and group indices take 5,504 bytes in total, 32.8% less than the 8,192 bytes of the original BF16 K. Therefore, the error and efficiency should be compared under the same byte budget, rather than judging the design solely by m>P.