acceptodds
Under review as a conference paper at ICLR 2027

QREAM: KV Cache Compression via Pivoted QR and Explicit Attention Matching

Abstract

Large key-value (KV) caches are a major bottleneck for long-context large language model (LLM) inference, motivating methods that reduce their size while preserving information needed for future queries. Attention matching (AM) has recently emerged as a practical and theoretically grounded framework for controlling attention behavior, with successful applications to prefix tuning and KV-cache compression. However, existing AM-based methods for KV-cache compression separately match the numerator and normalization factor of attention, requiring auxiliary per-key biases that are incompatible with standard KV-cache representations. In this work, we propose QREAM, a bias-free formulation of attention matching that directly matches the normalized attention output rather than its numerator and normalization factor separately. Once the compressed keys are selected, this formulation reduces attention matching to an efficient linear solve over the compressed values, requiring neither training nor modifications to the standard KV-cache representation. To select informative keys, QREAM further introduces a pivoted-QR-based procedure that favors linearly independent attention contributions, providing an efficient alternative to matching-pursuit methods. We evaluate QREAM across multiple Qwen model scales, long-context QA benchmarks, and compression ratios from 5× to 100×. QREAM is competitive with existing attention-matching methods while substantially outperforming token-scoring baselines under aggressive compression, despite its simpler and more deployment-friendly formulation.

Primary area:
optimization
Keywords:
KV cache compression, attention matching, long-context, inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rLRUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

QREAM does query-agnostic KV-cache compression. It drops AM's per-key bias by directly matching the normalized attention output, then solves the retained values by ridge toward the originals, and picks keys via pivoted QR on value-weighted attention columns. Tested on Qwen3-0.6B/4B/8B over QuALITY/LongHealth/QASPER at 5×–100×.

Critical strengths
  1. The bias-free reformulation is the paper's strongest selling point and it is well-argued: by matching the normalized output directly and re-solving the values, QREAM keeps a standard KV cache representation, so it drops into existing inference stacks without the kernel modification AM requires.
  2. The pivoted-QR key selection is well-motivated by the method's own math: since the re-solved values can only live in the row space spanned by the retained attention columns, picking keys for linear independence rather than raw attention magnitude is the correct criterion.
  3. The 100× result is a clean and useful empirical finding: every eviction-based baseline collapses to near-zero F1 while the attention-matched caches retain most performance. This sharply illustrates that "keeping high-scoring tokens" is not the same as "keeping the information needed to answer," and it gives the paper a real headline.
Critical weaknesses
  1. In your method, every retained value is re-solved, so it can overfit or break when the reference continuation is long. The benchmarks here (LongBench/HealthBench-type) have short outputs, so it needs a long-generation test like LongGenBench.
  2. No overhead analysis against traditional cheap methods. Your method must compare against StreamingLLM/SnapKV, and show the gain is worth the added compression-time cost.
  3. SnapKV scores are implausibly low; it is unclear if the question is dropped during compression, so this setup must be stated.
  4. H2O, the canonical attention-eviction baseline, is absent and should be included. The paper also lacks both a reproducibility statement and an ethics statement, which is a straightforward completeness gap for a submission.
Questions for authors

The paper states that QREAM and AM "use the same reference queries". However, for Qwen3-0.6B and 8B, QREAM uses the full ss+repeat probe (five reference-query families; qream_press.py L470-537) while AM is run as AM-fast with repeat-only (evaluate_registry.py L134/143, am_press.py L67). Is this design reasonable or a mistake?

Reviewer_RRGQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The author suggests QREAM, which is an attention matching without bias term for kv cache compression. QREAM directly matches the normalized attention output rather than its numerator and normalization factor separately. Namely, after selecting linearly independent contributions by devising a pivoted-QR-based procedure, QREAM reduces attention matching to an efficient linear solve over the compressed values. (explicit matching. Through extensive experiments across model scales, and benchmarks, diverse compression ratio, QREAM is competitive to original attention matching, with the benefits of easy deployment.

Critical strengths
  1. The tackling problem seems timely and important. Moreover, the motivation (bias-free for deployment) and the proposed method (explicit attention matching and pivoted QR) seems reasonable.

  2. The paper is generally understandable and the explanation on the methods are easy to follow.

  3. The experimental results are evaluated on various benchmarks with varying compression ratio, with reasonable baselines.

Critical weaknesses
  1. Insufficient Experimental Support for the Motivation and Practical Advantages Although the motivation for a bias-free formulation appears reasonable at first glance, I do not think it is sufficiently supported by the current experimental results or explanation.
  • Given that the proposed method achieves performance comparable to attention matching across most settings, with wins and losses in different scenarios, the primary advantage of the paper seems to lie in its ease of deployment. However, the authors mainly argue that rectifying the attention path is necessary, without sufficiently explaining how this process becomes technically challenging or time-consuming in practice. A more explicit analysis demonstrating the practical advantages of the bias-free formulation is needed.

  • A comparison of the compression costs of the proposed method, KVzip, and attention matching is also important to show the practicality of the proposed method, including both computational overhead and the time required to construct the compressed representation.

  • Although the method shows promising results at 50–100× compression ratios for both the 0.6B and 8B models, the practical significance of such aggressive compression settings remains unclear. The authors should compare these results with the performance of the original, uncompressed models and discuss whether the resulting performance trade-offs justify pursuing these compression ratios in practical applications.

  1. Lack of Ablation Studies
  • The paper does not appear to include sufficient ablation studies. The proposed method consists of several components, including ridge regularization, the on-policy procedure, and key selection. It is important to evaluate the contribution of each component to the final performance to establish the effectiveness of the proposed design choices.
  1. Unclear Advantages over Attention Matching
  • As noted above, the performance improvements seem relatively marginal, with results largely comparable to those of attention matching. The paper should more clearly show the specific advantages of the proposed method over this baseline, beyond its comparable performance, and provide stronger evidence to justify the proposed approach.
  1. Unclear Distinction Between Novel Contributions and Existing Methods
  • In some parts of the paper (e.g., [Lines 250–265]), it is difficult to distinguish which components are newly proposed by the authors and which are adopted from existing methods. The authors should clarify these distinctions to better highlight the paper's original contributions.
Questions for authors

Wrote above.

Reviewer_Qb26OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

QREAM is a training-free, query-agnostic KV-cache compression method. It selects cached keys using pivoted QR on value-weighted attention columns and reconstructs compressed values by directly matching normalized attention outputs through regularized least squares. Unlike the compared attention-matching method, QREAM introduces no auxiliary per-key biases. Reference queries are constructed from context repetition and self-study continuations, with layer-wise processing to address query-distribution mismatch. Experiments cover three Qwen3 model scales, three QA benchmarks, and compression ratios from 5× to 100×.

Critical strengths

Directly fitting normalized attention outputs provides a simple formulation that preserves the standard key–value cache representation. Eliminating auxiliary per-key biases offers a concrete compatibility advantage for attention implementations whose standard interfaces do not support these biases.

The results under aggressive compression are promising. At 100× compression, QREAM achieves the highest score among the evaluated methods in eight of nine model–benchmark combinations. On Qwen3-8B, it improves over Attention Matching by 5.63 points on QuALITY, 6.25 points on LongHealth, and 9.71 F1 on QASPER. The question-agnostic evaluation also tests reuse of a single compressed cache across multiple questions.

Critical weaknesses

Compression time, peak memory, and inference latency are not reported. Reference generation, layer-wise processing, pivoted QR, and double-precision solving may introduce substantial overhead, leaving the claimed efficiency and deployment advantages unverified.

There are no ablations separating bias-free matching, QR selection, value weighting, and on-policy construction, and no uncompressed baseline to quantify retained performance. The claim that all eviction methods have near-zero F1 at 100× compression also contradicts Table 3: ExpectedAttention achieves 20.82 F1 on Qwen3-8B, versus QREAM’s 27.93.

Questions for authors

What are the total compression time, peak memory, and inference latency compared with AM and KVzip at different context lengths?

How much does each component contribute when the selected keys, cache budgets, and fitting objectives are controlled?

What are the uncompressed-model scores, and how sensitive are the results to the 50-context QuALITY subset? Are the schedule-calibration articles disjoint from evaluation contexts?

How are A and B in Equation (5) constructed and normalized? Does on-policy fitting target the original model’s outputs or outputs computed from altered hidden states? How sensitive are the results to λ?

Why does AM remain near 25% on Qwen3-0.6B QuALITY across compression ratios? Has the reproduction been validated against the official implementation and budget schedules?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.