acceptodds
Under review as a conference paper at ICLR 2027

Evidence Is Not Authority: A Causal Study of State Arbitration in Long-Horizon Agent Memory

Abstract

Long-horizon agents can retrieve both historical and current user-state evidence yet still act on the historical state. Because existing memory benchmarks primarily score final answers, they often cannot distinguish a lost update from one that was visible but ignored. We call this choice among jointly visible states state arbitration and study it through an evidence-to-action audit and paired interventions. Across 240 paired histories and three model families, retaining an early explicit declaration while holding later behavior fixed lowers current-state accuracy by 33–39 points. Our primary intervention changes only the lifecycle relation between two records already in context. Marking the current record active and the old record superseded raises current-state actions from 24% to 53–55% on two 70B-class models. Natural validity/provenance fields also help, whereas timestamps alone do not. The contrast also transfers to structured action versions of STALE, LongMemEval, and MemoryAgentBench, but incorrect labels can reverse the result: marking a valid record superseded reduces one model's correct actions from 241 to 8 of 400. In native paths, managers often fail either to expose new evidence or to link it to the state it replaces. These findings identify a read-time failure distinct from retrieval and suggest preserving history while making the currently applicable record explicit and inspectable.

Primary area:
foundation or frontier models, including LLMs
Keywords:
long-horizon agents, memory governance, lifecycle validity, state arbitration, LLM agents, agent memory, benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LWFgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies state arbitration in long-horizon agent memory: which state governs an agent’s action when both historical and current records are available. It introduces an evidence-to-action audit separating retention, retrieval, materialization, and arbitration, and uses paired interventions to distinguish early-declaration anchoring from the read-time influence of lifecycle representations.

The declaration experiments show that early explicit statements can suppress the influence of subsequent behavioral evidence. In the primary lifecycle experiment, marking jointly visible records as ACTIVE/SUPERSEDED improves current-state action accuracy by 28.6 and 31.3 percentage points on Qwen-72B and Llama-70B, respectively.

Validation includes alternative metadata representations, native memory-manager audits, structured-action transformations of external benchmarks, and stable-state and reversed-label controls. The main contribution is a conditional empirical analysis of how supplied lifecycle relations affect action, rather than an end-to-end method for reliably identifying and managing evolving states.

Critical strengths

The problem is well defined and interesting. Separating evidence availability from the choice of which state guides an action gives a useful way to diagnose stale-memory failures.

The paired experiments are convincing within their stated scope: record content, order, instructions, and tasks remain fixed while lifecycle labels change. This supports the claimed read-time effect.

Critical weaknesses

My main concern is practical significance. The primary gains depend on correctly materialized states and supplied lifecycle relations. The native LangMem result is also conditional on 24 successful histories out of 120. The paper establishes that correct labels help, but not how reliably a system can obtain and use them across all cases.

The design recommendation is less clear than the empirical finding. In Table 4, the current-only diagnostic substantially outperforms presenting both records with lifecycle labels. Historical records could remain in storage without entering every action prompt. Given the same correct state assignment, the paper does not establish when labeled joint readout is preferable to filtering.

Questions for authors
  1. Can the existing native experiments show a net benefit across all attempted histories, including those with missing or incorrect lifecycle relations, rather than only the successful subset?

  2. How should the current-only results in Table 4 affect the proposed design recommendation? Do any evaluated tasks show a benefit from retaining historical records in the action context?

Reviewer_pNcTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies an important problem in long-term agent memory. Even when both old and new information are available, an agent may still follow outdated information. This problem is called state arbitration. To study this issue, the authors conduct controlled experiments where old and current memories are kept the same, but their status labels are changed. Results show that marking old memories as "SUPERSEDED" and current memories as "ACTIVE" can significantly improve the agent's decisions. Additional experiments on existing memory systems and benchmarks support these findings.

Critical strengths
  1. An interesting and practical problem is studied. Having the correct information in memory does not always mean an agent will use it correctly.

  2. Results are promising, with large improvements across different models. Additional tests on existing memory benchmarks also help support the findings.

  3. Including experiments with incorrect labels is a good idea. They show that memory status information can be helpful but may also cause serious problems when assigned incorrectly.

Critical weaknesses
  1. Previous work, such as STALE, has already studied outdated memories and memory updates. Using labels like "ACTIVE" and "SUPERSEDED" is also quite simple. It is unclear how much new insight this work provides beyond existing research.

  2. Real user preferences are often more complex and may change depending on the task. Repeatedly using Python, for example, does not necessarily mean someone no longer prefers Go.

  3. Most of the main experiments assume that the correct current memory is already available and that the correct status labels can be provided. However, automatically deciding which memory is current is difficult in practice.

  4. Some tasks benefit much more than others. For example, improvements on tool preferences are relatively small, and some label formats have limited effects.

Questions for authors
  1. How is state arbitration different from the problems studied in STALE and other memory update methods? What is the main new contribution?

  2. How can an agent automatically decide whether an old memory should be marked as "SUPERSEDED", especially when user behavior is unclear or temporary?

  3. Since the main experiments focus on cases where the correct current memory is already available, how well would this approach work in a complete memory system without this assumption?

  4. Can the authors evaluate the approach on real conversation histories rather than only synthetic examples? This would help show whether the findings generalize to practical settings.

Reviewer_7ZafOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors study the problem of existing benchmarks not being able to attribute specific failures of memory usage. They mainly study the cases where state changes explicitly and implicitly and claim that existing benchmarks are unable to track both of these at sufficient granularity because of the outcome based scoring they perform. Their proposal is to mark the current state as "active" and the previous states of memory as "inactive" to ensure that agents are able to track the current setup and preferences in a live manner. The paper is generally extremely hard to read and it seems to be completely AI generated under no supervision.

Critical strengths

None

Critical weaknesses
  • The work is not novel and is extremely poorly written. The figures are not properly formatted and I suspect that this paper was completely AI generated along with its figures, tables and written content.
  • The evaluation metrics are very hard to understand and it is unclear to me why the authors structured these metrics in this manner
  • There is no reason provided for the specifically sampled 240 history pairs (no justification for methodology of sampling either).
  • The idea itself is not novel either. There exist several benchmarks, that the authors have ignored, that perform granular grading of traces to find inflection points to study forgetting and state tracking in memory systems. The authors do not use any of these as baselines.
  • There is no human solvability baseline provided
Questions for authors

I believe the paper lacks any unique direction and doesn't have any justifications for the decisions they make. I do not believe that enumerating questions in addition to my notes in the weaknesses section is going to impact my stance on this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.