- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates generation failures under KV-cache eviction, and attribute the reason of failure to the misalignment between physical cache indices and original RoPE positions. The authors then propose Rotary Healing Space Topology (RHST), which restores the original causal coordinates using a mapping . The intended relative phase is
=(m-I_{\mathrm{offset}}(k))\theta_j =(m-n)\theta_j.$$ The correction is integrated into a Triton fused-attention kernel to avoid expensive tensor re-rotation. Experiments on Llama-3-8B-Instruct report substantial improvements under 90\% KV eviction, including 47.40\% accuracy on 2WikiMultihopQA and 21.50\% on MATH-500-Concat, compared with 6.47\% and 0\% for SnapKV.Critical strengths
The paper addresses an important issue in memory-constrained LLM inference: integrating coordinate correction into fused attention is a potentially useful systems optimization. Also, empirically, RHST reports substantial accuracy and latency improvements under extreme KV eviction. The matched-policy SnapKV ablation provides useful evidence supporting the proposed mechanism.
Critical weaknesses
However, from my perspective, physical KV eviction does not necessarily destroy RoPE positions. Correct implementations can preserve already-rotated keys and absolute query positions. Thus, RHST may primarily correct an avoidable implementation error rather than an inherent limitation of existing eviction methods.
Apart from that, there seems to be mathematical inconsistency. Appendix A uses an unrotated query and rotates keys by . However, under the standard RoPE convention,
=q^\top R((n-m)\theta)k.$$ The displayed implementation therefore appears to use the opposite rotation direction. This raises concerns about the correctness of the proposed kernel and its claimed exact equivalence.Questions for authors
Can the authors compare RHST against a correctly implemented KV-eviction baseline that preserves precomputed RoPE keys and absolute positional indices? Would RHST still provide substantial accuracy or latency improvements?