acceptodds
Under review as a conference paper at ICLR 2027

Rotary Healing Space Topology: Hardware-Fused Coordinate Re-mapping for Extreme KV Cache Eviction

Abstract

Memory constraints in autoregressive language models force the adoption of Key-Value (KV) cache physical truncation. While heuristic eviction methods successfully bound VRAM allocation, they introduce a deterministic generation collapse (e.g., infinite repetitive token loops or absolute syntactic disintegration) at extreme compression boundaries. We trace the root cause of this phase transition cliff to a topological fracture within Rotary Position Embeddings (RoPE). The mathematical failure does not originate from the physical memory voids. Instead, it arises because the underlying tensor framework forcefully re-indexes surviving tokens to contiguous physical positions to maintain memory continuity, thereby destroying the exact relative shift-invariance within the rotary subspace. To resolve this coordinate misalignment without incurring the latency penalty of physical tensor modifications, we propose Rotary Healing Space Topology (RHST). RHST operates as a hardware-level coordinate re-mapping operator with an macroscopic memory I/O footprint. Implemented as an extension to fused attention kernels, RHST bypasses memory bandwidth congestion by mapping physical indices back to their absolute causal coordinates directly at the hardware register level. Empirical evaluation across multi-hop retrieval and mathematical reasoning validates this algebraic intervention. Under extreme physical truncation, application-layer baselines suffer from absolute reasoning failure (0.00% accuracy on complex multi-step reasoning in MATH-500). RHST suppresses generation collapse and maintains deterministic reasoning integrity, effectively acting as a topological safety net that unlocks extreme heuristic compression safely.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.