Multi-anchor QK-coupling in LLMs: Shared Attention Routing and Efficient Learning

We identify multi-anchor QK-coupling, an anchor-relative geometry in which query and key trajectories remain coupled across layers, revealing shared routing structure in trained LLMs. We analyze this observation from three complementary perspectives: shared relational reconstruction error is jointly controlled by the common component's spectral tail and the mismatch measured by multi-anchor coupling; under spectral-dominance and relational-orthogonality conditions, common-core-preserving MLA-style compression can increase coupling; and private residual branches provide complementary first-order score-space correction directions around a fixed shared representation. Guided by these results, we propose a QK-coupled parameterization with a shared routing trunk and lightweight query-private/key-private residual branches. The design supports standard pre-training, MLA-style compressed-KV pre-training, and LoRA-style fine-tuning, reducing QK projection parameters, projection FLOPs, and training-state storage, with additional latent KV-cache savings on the evaluated SmolLM-style MLA backbones. Experiments show improved or competitive quality, including 75% QK-side FLOP/optimizer-memory savings with better perplexity in standard pre-training and a 50.57% QK-path projection-FLOP reduction with better final perplexity in a 3B, 200B-token MLA-style run.