RAVEL: Collision-Verified State Augmentation for Long-Context Recurrent Language Models

Fixed-state recurrent language models enable linear-time processing and constant-state decoding, but repeatedly compressing the prefix into a small state can cause interference between independent associations. We introduce RAVEL, a recurrent architecture with bounded collision-verified state augmentation. RAVEL maintains six fixed direct-address banks alongside the recurrent core, where each slot stores an exact token fingerprint, a support count, and a differentiable successor-value prototype. Reads precede writes, fingerprint mismatches abstain exactly, and a learned residual gate with a delayed horizon protects short-context behavior. The resulting augmentation requires no attention, nearest-neighbor search, relevance ranking, or unbounded token cache. We establish causal equivalence between teacher-forced and autoregressive execution, exact collision abstention, bounded state values and logit corrections, convergence of the running prototype, and an exponential-in-bank-count bound on surviving collisions. Across matched 6.1M, 11.7M, and 23.7M parameter scales, three-seed TinyStories experiments show that RAVEL improves over matched Mamba-3 at context lengths 128–4096 while remaining statistically indistinguishable at length 32, and improves over RWKV-7 at every measured length. A post-hoc TinyShakespeare study reproduces the long-context advantage. Frozen ablations show that one verified bank provides most of the gain, while additional banks offer diminishing collision redundancy. Together, these results demonstrate that RAVEL delivers a consistent and interpretable long-context quality advantage while preserving the bounded-state structure of recurrent language models.