Selective but Not Broad: Retrieval Capacity and Accessibility in State-Space Models
Abstract
State-space models (SSMs) such as Mamba solve selective retrieval (associative recall, MQAR) yet fail broad retrieval (uniform-over-positions discrimination) that requires every input position to stay recoverable (Position Recovery, PR). We argue these are two different memory problems: selective retrieval is content-addressed (recall by key), broad retrieval is position-addressed (keep every position discriminable), with distinct capacity requirements and distinct accessibility. Is the broad deficit a capacity limit or an optimization artifact? We model retrieval as distributing a fixed per-position signal-to-noise budget across the sequence. A single linear SSM channel provably cannot solve filler recall, which forces channel specialization: broad retrieval needs retrieval rank scaling with the sequence length, selective only with the number of keys, and the required timescales must be placed to match the queried ages. The theory predicts the structure of selective-retrieval collapse and the specialization signature of successful backbones; both appear in trained weights, including multi-scale RetNet tracking the theoretical 1/m line. A symmetric causal intervention then separates the two explanations: a curriculum unlocks broad retrieval on a standard SSM and transfers to selective, and unlocks selective at its own lower floor where standard SGD fails. At every scale tested, both deficits are accessibility-limited; the barrier persists through 122M parameters. A controlled Pareto frontier shows the objective is the lever: pure language modeling leaves broad at the random floor, and the PPL cost of adding broad is architecture-dependent. The theory holds for the linearized state map; every causal and theoretical claim is validated on this project's own checkpoints, and the frozen-probe absence of broad information replicates on 2.7B open pretrained checkpoints of both families.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.