acceptodds
Under review as a conference paper at ICLR 2027

StateBridge: State-Transition Shortcuts for Context Memory Reuse in SSM Serving

Abstract

State-space sequence models (SSMs) have emerged as a promising alternative to Transformers for efficient long-context language modeling, compressing context into a fixed-size recurrent state. However, unlike Transformers, which have well-established solutions for reusing precomputed KV caches of repeated text blocks across requests, SSMs lack any analogous mechanism. The reason is structural: Transformers handle context in an append-based manner that naturally supports modular reuse, whereas SSMs operate in an in-place manner where each block is fused into the recurrent state inseparably. This suggests a natural question: can we learn to approximate the effect of a fixed block on a different recurrent state, without actually prefilling it? In this paper, StateBridge answers this affirmatively, learning a compact input matrix for each reusable context block that approximates its in-place state update for different preceding contexts. At serving time, SSM scans this learned input matrix from the live recurrent state, applying a shared transition shortcut in place of the raw block to approximate the original behavior. Across four benchmarks, StateBridge retains %–% of performance and achieves up to speedup in time to first token (TTFT).

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.