acceptodds
Under review as a conference paper at ICLR 2027

RRC-KV: Range-Driven Rotation and Centering for Two-Bit KV Caches

Abstract

Scalar INT2 KV caches have only four reconstruction levels per group, making accuracy sensitive to quantization coordinates. We present RRC-KV, which learns per-head key and per-layer value rotations through range-based objectives. A query-based auxiliary regularizer limits coordinate sensitivity concentration, while static key centering shifts the encoding origin with its contribution restored in attention logits. The method requires neither downstream labels nor quantizer backpropagation and supports mixed-precision attention with fused Triton kernels. Across four models and five reasoning and coding benchmarks, RRC-KV achieves a macro-average score of 79.54 versus 80.04 for BF16. Under the same KV storage layout, it outperforms the spectral-rotation baseline OSCAR in every tested RULER setting across two models and 8K–64K contexts (87.19 versus 81.61 on average), while a long-context gap to BF16 remains. Across three Qwen models, scores on two coding tasks vary by at most 3.05 points between two calibration configurations, compared with up to 34.69 points for OSCAR on Qwen3-32B. At 64K tokens, the KV representation provides 6.91× compression relative to BF16, including metadata and retained BF16 windows.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
KV cache quantization, low-bit quantization, efficient LLM inference, long-context inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ibzTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

RRC-KV learns orthogonal transformations for scalar INT2 KV-cache quantization. Keys use per-head rotations trained with a squared-range objective and a query-sensitivity regularizer while values use a layer-shared rotation trained with a smooth-range objective. Static key centering is applied afterward, with the center contribution restored in attention logits. Calibration avoids downstream labels and quantizer backpropagation.

RRC-KV outperforms spectral rotation baseline OSCAR on RULER benchmark. Additionally RRC-KV provides '6.91X compression relative to BF16 baseline at 64K context length.

Critical strengths

RRC-KV has following strengths:

  1. Strong practical accuracy results. RRC-KV achieves the highest five-task average among the evaluated low-bit methods on each model. Comparisons include recent OSCAR, OScaR, and TurboQuant methods.
  2. Useful component ablations. Table 4 separates learned key rotations, learned value rotations, random rotations, query regularization, and centering. With trained values fixed, learning the key rotation improves Q4 RULER64K by 12.68 points, supporting the central range-learning contribution.
  3. Careful attention-error diagnostics. The distinction between reconstruction error and downstream attention error is valuable. For example, OScaR reconstructs keys better in Figure 2 but produces larger attention-output error.
Critical weaknesses

RRC-KV weaknesses :

  1. The paper is very poorly written and extremely difficult to understand. It is not acceptable in it's current state.
  2. Learning rotations have already been shown to improve quantization performance [1]. Therefore it limits the novelty of this work.
  3. The main text in the paper only provides details on construction of cost function not how it is optimized.
  4. Long context evaluation is very limited. Although RRC-KV demonstrates improved performance compared to OSCAR on RULER, it needs to also be validated on more challenging long context settings.
Questions for authors

Please see weaknesses for concerns.

Reviewer_EgqVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

RRC-KV calibrates scalar INT2 KV caches using learned per-head post-RoPE key rotations, a shared value rotation per layer, and static key centering.

It learns rotations through range objectives with an auxiliary query-sensitivity regularizer, without backpropagating through the quantizer.

Its evaluation covers four models, five reasoning and coding benchmarks and shows the effectiveness and efficiency of this work.

Critical strengths
  1. The accuracy improvements are validated. Across four models and five tasks, RRC-KV reaches an average of 79.5 against 80.0 for BF16. It also exceeds official OScaR on all three Qwen models while using fewer representation bits at 64K. These results support the value of the proposed calibration scheme.
  2. The ablations establish gains beyond key centering. In Table 4, RRC-KV scores much higher than OSCAR with the same static centering. The random/trained K/V comparisons also separate the effects of learning each rotation. This makes the improvement over fixed rotations more convincing.
  3. The diagnostics distinguish reconstruction from attention error. Held-out text measurements and the value-centering intervention show that improved KV reconstruction can coexist with worse attention readout. This provides useful evidence for evaluating KV quantization beyond reconstruction MSE.
  4. The method has a concrete inference implementation. Fused Triton kernels support the proposed mixed-precision representation. Storage accounting includes metadata and retained BF16 tokens, giving more than 6x KV representation compression at 64K. Whole-model timings also expose the hardware-dependent costs of the transforms and decoding operations.
Critical weaknesses
  1. The methodological novelty is limited. Distribution-based rotation learning is established by KurTail, DartQuant, and InfoQuant. OSCAR already equalizes diagonal query sensitivity, and SageAttention establishes key centering. The distinct contribution is therefore the particular range objective and its integration into scalar INT2 KV caching. Concurrent OptR also combines learned rotations with static key centering..
  2. The benefit of the specific range objective is not sufficiently isolated. Table 4 shows that learning rotations helps, but does not compare range learning with existing distribution-shaping objectives under the same post-RoPE rotation granularity, quantizer, and centering. This matters because the specific objective is the main remaining methodological distinction. A focused comparison could establish whether its design adds value beyond general distribution-based rotation learning.
  3. The query regularizer has uncertain task-level value. It clearly reduces coordinate sensitivity concentration, but Table 11 shows only about a 3% change in readout MSE across the tested weights. Its gain on RULER64K has a paired 95% confidence interval in Appendix G. This supports the targeted statistical effect more strongly than an independent contribution to task accuracy.
  4. The practical contribution remains uneven across settings. The mean gain over OSCAR varies a lot. The tested B200 workloads show no consistent acceleration over BF16.
  5. Two additional references would improve the related-work discussion. I do not require new benchmark comparisons to these papers. (disclaimer: I am not an author of these 2 papers.)
  • TaDA: Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centering, ACL 2025. It quantizes deviations from shared K/V activations across heads. Its centering differs from the static channel means used here, but is relevant to residual KV representations.
  • KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks, arXiv 2606. It combines Hadamard rotation with online variance normalization along token and channel axes, providing a relevant alternative approach to accurate two-bit KV caching during generation.
Questions for authors
  1. Can the authors provide a focused comparison with an existing distribution-shaping objective, such as peak suppression or Whip loss, using the same rotation granularity, calibration data, quantizer, and centering? This would directly clarify the contribution of the proposed range objective.
  2. Beyond the sensitivity statistics, what evidence supports a task-accuracy benefit from the query regularizer given the uncertainty reported in Appendix G? Please clarify whether the intended claim is sensitivity control or a demonstrated improvement in downstream accuracy.
Reviewer_mY6DOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RRC-KV, a method for improving two-bit KV-cache quantization through learned orthogonal rotations and static key centering. The authors observe that scalar INT2 quantization is highly sensitive to coordinate ranges, since extreme values can substantially increase quantization errors.

The proposed method contains three main components. First, per-head key rotations are learned by minimizing squared coordinate ranges, together with a query-based regularizer that limits the concentration of attention sensitivity. Second, per-layer value rotations are optimized using a smooth range objective. Third, static key centering shifts the quantization origin toward the calibration mean, while restoring the center contribution in attention logits.

Experiments on four language models and five reasoning and coding benchmarks demonstrate strong performance under INT2 compression. RRC-KV achieves a macro-average accuracy of 79.54 compared with 80.04 for BF16, while providing approximately 6.91-fold KV-cache compression at 64K tokens. It also outperforms OSCAR across all evaluated RULER settings, achieving 87.19 versus 81.61 average accuracy. Additional experiments investigate calibration robustness, component contributions, attention errors, and inference efficiency.

Critical strengths

The paper shows a practical and well-motivated approach to handle the extreme KV-cache quantization. The combination of range-driven rotations, query sensitivity regularization, and key centering is computationally efficient and sensible, and provides a meaningful improvement over existing rotation-based methods.

The empirical evaluation is also comprehensive, on the other hand, covering multiple model scales and reasoning and coding benchmarks. The component ablations provide convincing evidence for the effectiveness of the proposed design, particularly the range-based key rotation objective.

Critical weaknesses

Even though the sensible design and performance improvement, the methodological and theoretical novelty is somewhat limited. Learned orthogonal rotations, range-based optimization, and key centering have been well explored in related work. From my perspective, the main contribution is the particular combination, while the theoretical results primarily justify surrogate quantization objectives without establishing stronger guarantees on attention-output error, which influences the most of downstream performance.

Questions for authors

Can the authors further clarify the methodological novelty relative to recent output-aware rotation methods, like OSCAR and OptR?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.