- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
RRC-KV learns orthogonal transformations for scalar INT2 KV-cache quantization. Keys use per-head rotations trained with a squared-range objective and a query-sensitivity regularizer while values use a layer-shared rotation trained with a smooth-range objective. Static key centering is applied afterward, with the center contribution restored in attention logits. Calibration avoids downstream labels and quantizer backpropagation.
RRC-KV outperforms spectral rotation baseline OSCAR on RULER benchmark. Additionally RRC-KV provides '6.91X compression relative to BF16 baseline at 64K context length.
Critical strengths
RRC-KV has following strengths:
- Strong practical accuracy results. RRC-KV achieves the highest five-task average among the evaluated low-bit methods on each model. Comparisons include recent OSCAR, OScaR, and TurboQuant methods.
- Useful component ablations. Table 4 separates learned key rotations, learned value rotations, random rotations, query regularization, and centering. With trained values fixed, learning the key rotation improves Q4 RULER64K by 12.68 points, supporting the central range-learning contribution.
- Careful attention-error diagnostics. The distinction between reconstruction error and downstream attention error is valuable. For example, OScaR reconstructs keys better in Figure 2 but produces larger attention-output error.
Critical weaknesses
RRC-KV weaknesses :
- The paper is very poorly written and extremely difficult to understand. It is not acceptable in it's current state.
- Learning rotations have already been shown to improve quantization performance [1]. Therefore it limits the novelty of this work.
- The main text in the paper only provides details on construction of cost function not how it is optimized.
- Long context evaluation is very limited. Although RRC-KV demonstrates improved performance compared to OSCAR on RULER, it needs to also be validated on more challenging long context settings.
Questions for authors
Please see weaknesses for concerns.