acceptodds
Under review as a conference paper at ICLR 2027

Entity-Conditioned Addressing: Reducing Cross-Entity Interference in Fixed-Size Recurrent Models

Abstract

Fixed-size recurrent models compress events from many interleaved entities into a shared state. We identify a cross-entity interference failure mode that is distinct from long-range forgetting: performance can remain stable as sequence length increases yet collapse as the number of concurrently updated entities grows. We introduce Entity-Conditioned Addressing (ECA), which injects an entity-derived key into a recurrent model's write and read addresses. Our main variant, ECA-Hash, generates deterministic random keys from entity identifiers without maintaining a trainable entity table, thereby preserving streaming inference and state size independent of the entity count. On a controlled entity-state benchmark, matched identity codes supplied through the input fail as entity load grows, whereas injecting the same codes into the addresses raises accuracy at 16 tracked entities from at most 0.20 to 0.87 (Mamba-2) and 0.98 (Gated DeltaNet). On a second, independently generated benchmark, the effect transfers across official Mamba-1/2/3 blocks, Naju, and Gated DeltaNet, and approximately follows the predicted dependence on entity load relative to address width. On a real Windows endpoint state probe, address injection improves accuracy over the same model without it from 0.714–0.726 to 0.936–0.984 across official Mamba-1/2/3, Naju, and Gated DeltaNet (official Mamba-2: 0.722 to 0.984). However, it provides no consistent benefit when identity information alone or recent content suffices, and address injection can hurt content-dominated detection. These results characterize ECA as a structural prior for interference-bound entity tracking.

Primary area:
foundation or frontier models, including LLMs
Keywords:
State Space Models, Selective State Space Models, Entity-Conditioned State Updates, Structured Memory, Associative Memory, Long-Sequence Modeling, Interference Reduction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VfvFOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies cross-entity interference, as distinct from sequence-length forgetting, in recurrent models that maintain a fixed-size shared state over interleaved entity events. It proposes Entity-Conditioned Addressing (ECA), which adds a stable, identifier-derived key to both the write and read addresses of a scan-compatible recurrence. Its principal ECA-Hash variant uses deterministic Rademacher keys without an entity-specific learned table. An idealized superposition analysis predicts expected crosstalk energy proportional to entity load divided by address width, O(M/N), under stated independence assumptions. Evaluation covers the synthetic e-SET entity-state benchmark, an independently constructed security-grammar task, official Mamba-1/2/3, Naju and Gated DeltaNet backbones, and probes on WinTap ACME, CTU-13 and Google cluster traces. The most substantial gains arise when a query must retrieve an entity’s stale state amid interleaved updates; the paper also reports cases where ECA is neutral or harmful.

Critical strengths
  1. Well-isolated evidence for the address-location mechanism. The comparison holds the identity code and input encoder fixed while varying whether the key is injected into recurrent read/write addresses (Table 2). Several input-only alternatives remain near the no-identity floor at M = 16, whereas ECA-Hash reaches 0.865 on Mamba-2 and 0.977 on Gated DeltaNet. The independent security grammar reproduces the effect across five recurrent backbones (Table 3). Key resampling, write/read mismatch, one-sided keying and query-swap tests (Appendices B and F) provide unusually direct evidence that stable, aligned addressing—not merely the presence of entity metadata—drives retrieval.
  2. A meaningful characterization of scope and cost, rather than an unqualified accuracy claim. The WinTap ACME entity-state probe shows sizeable gains across five backbones (Table 5), and the study varies entity load independently of horizon and address width (Fig. 2). The constant persistent-state property is explicit; Appendix F also reports parameter, state and throughput measurements. Importantly, the authors document negative results on attack detection, CTU-13 and GCT (Sec. 8–9), helping establish when this inductive bias is appropriate rather than presenting it as a universal security improvement.
Critical weaknesses
  1. Practical significance beyond a routing-specific diagnostic remains uncertain. The strongest real-data result is a deliberately constructed stale entity-state probe on ACME rather than a demonstrated gain on a native security objective. The paper itself reports no consistent benefit for ACME attack detection or the GCT probe; CTU-13 can even exhibit poorer scenario-held-out recall under keyed configurations (Sec. 8.1–8.3). Thus the evidence supports a conditional memory mechanism more strongly than it supports broad real-world utility. The paper would be more compelling if it quantified how often representative applications actually enter the interference-bound regime.
  2. The capacity argument does not directly guarantee scaling or efficiency in trained recurrent backbones. The O(M/N) law concerns an idealized random-key superposition with key-independent values and decay (Sec. 4), not the full learned gated systems. Fig. 2 already shows degradation at higher relative load, and the method requires reliable, stable entity identifiers (Sec. 9). Although state size remains constant, Appendix F reports non-fused reference-path throughput reductions for several official implementations; the available fused Naju measurement does not establish equal deployment efficiency across all backbones. This limits the strength of the claimed capacity/performance tradeoff. Figure/presentation concern (secondary): The compact ECA flow diagram (Fig. 1, PDF p. 4) uses small annotations, while the load–accuracy comparisons (Fig. 2 and Appendix Fig. A1) rely on fine dashed traces, small point markers and a crowded legend. Key capacity trends require zooming to inspect, and larger labels and better visual separation of baseline curves would improve interpretability. This visual issue does not independently undermine the reported mechanism.
Questions for authors
  1. Routing-bound prevalence and native-task impact (assessment-critical). Can the authors measure, on ACME and at least one additional real event stream, the fraction of evaluation queries that actually require retrieving a stale entity-specific value rather than recency/content-based inference? When results are stratified by this property, do ECA gains translate to a naturally occurring downstream objective? A demonstrated benefit would materially strengthen the significance assessment.
  2. Robustness to identifier instability and implementation budget (assessment-critical). Beyond the key-overlap stress tests already in Appendix F, what happens under realistic identifier churn, aliasing or occasional incorrect IDs on the ACME probe? Can matched-budget throughput or end-to-end latency be reported for a fused ECA implementation in more than the Naju host? These answers would clarify whether the method’s assumptions and computational overhead remain acceptable in its intended deployment regime.
Reviewer_vwMVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies cross-entity interference as a failure mode of fixed-size recurrent models that is distinct from long-range forgetting: a model may retain information over long sequences but fail when many interleaved entities compete for the same recurrent state. To address this issue, the authors propose Entity-Conditioned Addressing (ECA), which injects a stable entity-derived key directly into the model’s write and read addresses. The main variant, ECA-Hash, uses deterministic hash-based random keys, requiring no trainable per-entity table and preserving constant recurrent-state size with respect to the number of entities.

The paper provides an idealized superposition analysis predicting that cross-entity interference grows approximately with the entity load relative to address width, and validates the mechanism on controlled entity-state benchmarks. Across several recurrent backbones, including Mamba variants and Gated DeltaNet, address-level identity conditioning substantially outperforms supplying the same identity information only through the input. On real endpoint telemetry, ECA also gives large gains on an entity-state tracking probe, while showing limited or negative benefit on tasks where identity-specific routing is not the main bottleneck.

Critical strengths
  1. The paper identifies and cleanly isolates cross-entity interference as a failure mode distinct from long-range forgetting. The matched input-vs-address conditioning experiments make the central routing claim convincing and reproduce across multiple recurrent backbones.

  2. ECA is simple, efficient, and effective in the intended regime. ECA-Hash requires no trainable per-entity table, preserves fixed recurrent-state size, and achieves large gains on controlled entity-tracking tasks, with a simple superposition analysis explaining the observed capacity trend.

Critical weaknesses
  1. The practical scope of ECA appears relatively narrow. The controlled experiments strongly support the proposed mechanism, but the real-data gains are concentrated on routing-bound entity-state tracking. ECA provides little or no benefit when identity input or recent content already suffices, and can even hurt content-dominated tasks. This limits the evidence that cross-entity interference is a broadly important bottleneck in practical sequence modeling.

  2. The real-world validation remains limited in scale and diversity. The strongest real-data result comes from an endpoint state probe that closely matches the synthetic entity-tracking setting, while the other real tasks show mixed results. Additional applications where entity-specific retrieval arises naturally would strengthen the practical significance of the method.

Questions for authors
  1. Can the authors provide additional evidence that cross-entity interference is a practically important bottleneck beyond explicitly routing-bound state-tracking probes? In particular, are there real-world tasks with naturally occurring entity-specific retrieval where ECA consistently improves downstream performance?

  2. How sensitive is ECA to imperfect entity identifiers, such as identifier reuse, merging/splitting of entities, or noisy entity resolution? Since the method relies directly on persistent entity keys, robustness to such errors would materially affect its practical applicability.

Reviewer_w1GmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies cross-entity interference in fixed-size recurrent models, distinguishing it from long-range forgetting. The authors propose Entity-Conditioned Addressing (ECA), which injects deterministic random keys into the write and read addresses of recurrent memories. Experiments across architectures show substantial improvements over identity-input baselines on controlled entity-tracking benchmarks.

Critical strengths

1. Convincing experimental evidence. The improvements are substantial and consistent across several architectures. The matched identity-code comparisons and intervention experiments provide compelling evidence that explicit address conditioning improves entity-specific retrieval.

2. Extensive ablations. The authors carefully investigate key consistency, capacity scaling, alternative identity conditioning, and unseen entities. Reporting negative results on real-world datasets also helps characterize the method's limitations.

3. Well-written paper. Reading this paper was quite easy and enjoyable.

Critical weaknesses

1. Limited novelty and scientific significance.

The central mechanism essentially consists of using persistent random keys to address a shared associative memory, following well-established principles from distributed associative memories and fast weights. In particular, Associative Long Short-Term Memory (Danihelka et al., ICML 2016) already investigates associative storage in recurrent networks and the effect of interference on memory capacity. The paper's O(M/N)O(M/N) interference analysis is also a direct consequence of classical random-key superposition.

Furthermore, the related-work discussion overlooks relevant entity-tracking literature, particularly Chain and Causal Attention for Efficient Entity Tracking (Fagnou et al., EMNLP 2024) and Entity Tracking in Language Models (Kim & Schuster, ACL 2023). Although these works address different architectures, they already investigate the limitations of standard models for tracking mutable entity states and, in the case of ChaCAL, the benefits of specialized architectural mechanisms.

While demonstrating these principles on modern recurrent architectures is interesting, I am not convinced that the paper establishes a sufficiently substantial conceptual or methodological advance.

2. Limited evidence of broader practical relevance.

The largest improvements occur on benchmarks specifically designed around entity-state retrieval, where explicit associative addressing is naturally advantageous. The main real-world improvement concerns a constructed process-state probe rather than an operational security task.

On ACME attack-process detection and Google Cluster Trace, ECA provides no consistent benefit, while on CTU-13 it can substantially degrade botnet recall. Furthermore, experiments are limited to relatively small models, and the paper does not establish whether the same routing bottleneck persists in larger pretrained recurrent architectures.

Overall, the results convincingly demonstrate the usefulness of explicit addressing for these particular tasks, but do not sufficiently establish a broadly significant advance in recurrent sequence modeling.

Questions for authors
  1. What is fundamentally new about ECA beyond established associative-memory mechanisms, particularly those investigated by Danihelka et al. (2016)? Is the main contribution primarily the empirical demonstration of these principles in modern SSM architectures?

  2. Is there evidence that the identity-routing failure persists in larger or pretrained recurrent models, or with substantially more training data? This seems important for establishing the broader significance of the proposed mechanism.

I am open to revise my grade if the questions and weaknesses are coherently answered to.

Comments on the paper

2 comments
Area_Chair_ZcnXOpenReview ↗

Reviewers: please respond to the author rebuttal

Dear reviewers,

Thank you to those who have already engaged with the authors' responses. The author-reviewer discussion closes on November 18 (AoE).

If you have not yet replied, please read the authors' response to your review and post a reply here before then: whether your concerns have been addressed, what remains open, and whether you are updating your score. If your assessment is unchanged, please say so explicitly. Engaging with the other reviews and the ongoing discussion is also welcome.

Thank you, Area Chair

Area_Chair_ZcnXOpenReview ↗

Final reminder: discussion closes today (Nov 18 AoE)

The author-reviewer discussion closes today, November 18 (AoE).

Dear reviewers,

Thank you to those who have already responded. If you have not yet replied: Please read the authors' response and post a brief reply here before the deadline: whether your concerns have been addressed, what remains open, and whether you are updating your score. If your assessment is unchanged, please say so explicitly. Your response is important for the final recommendation.

Thank you, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.