Locate, Then Continue: A Framework for Shared KV Memory in Hybrid Attention Models

Hybrid attention models combine linear attention for efficient long-context modeling with periodic full-attention calls for exact retrieval, yet it is unclear how far those calls, though occurring at different depths, can share computation and state, and what minimal degree of sharing exact retrieval still requires. We answer this question with a descriptive framework that views each full-attention call as writing to and reading from an associative memory. Within this framework, exact retrieval composes two read types: associative lookup, which locates a record by matching its identity, and sequence continuation, which copies the record verbatim by matching predecessor windows against the generated prefix. Reads of one type share a single key function and differ only in their queries, so two read types suffice for direct, chained, and multi-target retrieval regardless of read count, hop count, content length, or model depth. Conversely, under an empirically motivated role-stability premise, that queries alone cannot reliably switch one cache between lookup and continuation, one layer does not suffice, making two layers the minimum, typically ordered as lookup then continuation. Experiments on 0.5B and 1.3B hybrids support the framework's predictions: the shallow group specializes in lookup and the deep group in continuation, merging two continuation groups preserves retrieval, and a single shared layer fails to recover verbatim copying. The resulting design, Group Share, reduces independent full-attention layers from seven to two at 1.3B, saving 10.4% of parameters and 57.4% of total cache at 16K while maintaining general capabilities; it even yields higher retrieval accuracy at context lengths from 2K to 16K.