Entity-Conditioned Addressing: Reducing Cross-Entity Interference in Fixed-Size Recurrent Models
Abstract
Fixed-size recurrent models compress events from many interleaved entities into a shared state. We identify a cross-entity interference failure mode that is distinct from long-range forgetting: performance can remain stable as sequence length increases yet collapse as the number of concurrently updated entities grows. We introduce Entity-Conditioned Addressing (ECA), which injects an entity-derived key into a recurrent model's write and read addresses. Our main variant, ECA-Hash, generates deterministic random keys from entity identifiers without maintaining a trainable entity table, thereby preserving streaming inference and state size independent of the entity count. On a controlled entity-state benchmark, matched identity codes supplied through the input fail as entity load grows, whereas injecting the same codes into the addresses raises accuracy at 16 tracked entities from at most 0.20 to 0.87 (Mamba-2) and 0.98 (Gated DeltaNet). On a second, independently generated benchmark, the effect transfers across official Mamba-1/2/3 blocks, Naju, and Gated DeltaNet, and approximately follows the predicted dependence on entity load relative to address width. On a real Windows endpoint state probe, address injection improves accuracy over the same model without it from 0.714–0.726 to 0.936–0.984 across official Mamba-1/2/3, Naju, and Gated DeltaNet (official Mamba-2: 0.722 to 0.984). However, it provides no consistent benefit when identity information alone or recent content suffices, and address injection can hurt content-dominated detection. These results characterize ECA as a structural prior for interference-bound entity tracking.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.