RoFold: Reconstruction-Free Low-Rank KV Caching under RoPE

Low-rank KV caching reduces the storage and bandwidth costs of long-context inference, but pre-RoPE methods reconstruct compressed historical keys to full width before rotation and scoring. We ask whether keys can instead remain latent throughout the read path without giving up a data-adaptive basis. We show that exact latent execution, RoPE invariance of the decoder subspace, and commutation of its orthogonal projector with RoPE are equivalent, and that the execution error of a general subspace decomposes orthogonally into subspace escape and latent-dynamics mismatch. Based on this characterization, we introduce RoFold, which combines (i) a data-adaptive construction of same-frequency cross-head modes and an attention-aware left-inverse encoder fitted by a convex ridge problem without increasing cache width or retained rank, and (ii) a reconstruction-free key path that never rebuilds full-width keys. This path is exactly equivalent, in exact arithmetic, to reconstruct-then-rotate for the same compressed representation, and a frozen unit-level hybrid applies it to 41 of 64 units. On Llama-3.1-8B, at a matched 40.1% KV-reduction budget, RoFold's autoregressive decode advantage over the reconstruct-then-rotate PaLU baseline grows from 9.1% at 8K to 56.4% at 64K, while the relative PPL increase remains below 10% across all evaluated lengths.