acceptodds
Withdrawn from ICLR 2027

Distributed Evidence Memory: Deferring Irreversible Decisions for Reliable Long-Term Memory

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-term memory is essential for language-model agents operating over interaction histories beyond the context window. Existing memory systems typically compress past experiences into summaries, facts, embeddings, or structured representations, and then use a centralized mechanism to decide which memories enter the current context. We argue that this central-gating paradigm can create an irreversible evidence bottleneck: relevant memories may be excluded before their original content is read, conflicting versions may be resolved too early, and retrieval may stop before the necessary evidence is complete.We introduce Distributed Evidence Memory (DEM), which defers these decisions to query time. DEM lets raw-memory shards report query-conditioned evidence through a shared workspace, preserves conflicting versions with a Conflict Directed Acyclic Graph (CDAG), and uses Good-Turing Coverage Estimation (GTC) to stop retrieval based on evidence saturation rather than fixed budgets or model confidence. Across FactCon, LoCoMo, and LongMemEval, DEM consistently outperforms strong retrieval and memory baselines. On FactCon, DEM improves SubEM from 41.5 to 64.0 with DeepSeek-v4-flash, with the largest gains appearing in multi-hop, conflict-heavy, and large-memory settings. These results support a broader principle for long-term memory design: preserve uncertainty during storage and defer irreversible decisions until sufficient query-time evidence is available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
long-term memory, LLM agents, agent memory, memory retrieval, multi-hop retrieval, conflict resolution, adaptive retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PuqFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the problem that existing memory systems often construct query-independent representations and make early decisions, which may lead to irreversible evidence loss. The paper introduces Distributed Evidence Memory (DEM), a framework that preserves potentially relevant evidence and defers irreversible decisions until query-conditioned information becomes available. The framework consists of three main components:

  1. Raw-memory Shards: Enable distributed access to original memory episodes and retrieve query-conditioned evidence through a shared workspace.
  2. Conflict Directed Acyclic Graph (CDAG): Preserves conflicting memory versions for query-time aggregation instead of prematurely discarding them.
  3. Good-Turing Coverage Estimation (GTC): Provides an evidence-based adaptive stopping mechanism to determine when sufficient evidence has been collected.

Experiments on FactCon, LoCoMo, and LongMemEval show that DEM outperforms existing retrieval and memory baselines. Additional analyses suggest that improved evidence coverage accounts for a substantial portion of the performance gains.

Critical strengths
  • Interesting motivation and clear design principle. The paper identifies irreversible evidence loss as an important challenge in long-term agent memory and presents a coherent framework for deferring memory decisions until query-conditioned evidence becomes available.
  • Comprehensive evaluation and strong empirical results. The paper evaluates DEM across three benchmarks and seven backbones, with substantial performance improvements. The extensive ablations and mechanism analyses provide useful insights into evidence coverage, multi-hop retrieval, and conflict preservation.
Critical weaknesses
  • The statistical validity of GTC remains unclear, as adaptive multi-hop retrieval produces dependent and non-stationary evidence observations that may violate the assumptions underlying Good-Turing estimation.
  • Although DEM aims to defer irreversible memory decisions, its initial shard activation remains irreversible, as subsequent evidence-guided retrieval is restricted to the fixed activated set. This may prevent the discovery of relevant evidence in previously inactive shards, particularly in multi-hop queries.
  • Although DEM consistently improves SubEM across seven backbones, some models exhibit substantial declines in EM and F1 (e.g., Qwen3.5-9B: EM 31.0→10.0, F1 33.6→25.0), raising concerns about whether the reported SubEM gains consistently reflect improved answer correctness rather than more verbose or ambiguous outputs.
  • DEM's gains are not sufficiently separated from its 2.7× compute cost, and the necessity of CDAG over simpler version-preserving methods remains unclear.
  • CDAG's conflict definition is overly simplistic, as different values for the same subject–relation pair may represent compatible facts rather than genuine conflicts.
Questions for authors
  • Could the authors clarify GTC assumptions and provide an experiment examining the relationship between GTC's estimated unseen mass and the actual remaining ground-truth evidence, particularly when the stopping criterion is triggered?
  • Could the authors report the proportion of ground-truth evidence covered by the initially activated shards and compare against a dynamic shard-reactivation strategy under matched computational budgets?
  • Could the authors provide compute-matched comparisons and evaluate CDAG against a simple timestamped multi-version baseline?
  • Could the authors clarify how CDAG handles multi-valued relations and temporal validity, and evaluate its robustness to incorrectly identified conflicts?
Reviewer_NYRXOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to study a problem in the memory of LLM-agent. They claim that agents are making decisions too early. The authors propose DEM, which delays memory filtering, conflict resolution, and retrieval stopping until query time. They conducted experiments on FactCon, LoCoMo, and LongMemEval and show consistent improvements across seven backbones, mainly due to better evidence coverage.

Critical strengths
  1. The stopping criterion is efficient and requires no additional model calls. The ablation also clearly shows the trade-off between accuracy and compute.

  2. The authors provide ablations and a good analysis showing that 68% of the gains come from better evidence coverage. The memory-scaling and conflict-retention experiments are particularly convincing.

  3. The paper reports computational costs, includes a cheaper variant, and discusses limitations. The experiments across multiple backbones further strengthen the results.

Critical weaknesses
  1. My main concern is its Limited technical novelty. The system they proposed, including query-conditioned retrieval, iterative search, and conflict preservation, have been explored in prior work. The main contribution seems to be combining existing techniques under a common design principle, rather than introducing a fundamentally new mechanism.

  2. Another limitation of this work is its weak theoretical contribution. The information-theoretic argument is reasonable but fairly obvious. It explains why early compression can lose information, but does not quantify how often this matters or explain the improvements.

  3. For the experimental parts, it is unclear to evaluate the baseline fairness. Some baseline results come from previous papers and may not be directly comparable. More importantly, DEM uses substantially more retrieval compute, but no compute-matched baseline is provided. This makes it difficult to tell how much of the gain comes from the proposed design rather than additional computation.

Questions for authors

NA

Reviewer_os6WOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Distributed Evidence Memory (DEM), a framework designed to mitigate premature evidence selection in long-term LLM memory. DEM integrates query-conditioned retrieval across raw-memory shards, a Conflict Directed Acyclic Graph (CDAG) to retain conflicting evidence, and Good-Turing Coverage Estimation (GTC) to enable adaptive stopping. Experimental results on FactCon, LoCoMo, and LongMemEval show that DEM consistently outperforms existing memory baselines.

Critical strengths

The paper identifies premature evidence selection as an important limitation of existing memory systems, the research question is meaningful, and provides a clear motivation for the proposed framework.

The experimental analysis is solid. The paper includes component ablations, evidence coverage analysis, cross-backbone evaluations, and memory-size robustness experiments.

Critical weaknesses
  • Limited methodological novelty. Query-aware retrieval, iterative evidence collection, and conflict-aware memory management have been explored in prior work. DEM mainly integrates existing ideas, and comparisons with closely related adaptive retrieval methods such as MRAgent are missing.

  • Potential evidence loss during shard activation. The activated shard set remains fixed throughout retrieval. Relevant evidence in initially rejected shards cannot be recovered in later rounds, which may reproduce the early evidence exclusion problem that DEM aims to address.

  • High computational overhead. DEM requires 2.7× the input tokens of E-mem in the DeepSeek-v4-flash setting. All memory shards must also be screened at query time. The lack of compute-matched comparisons makes it difficult to assess the efficiency of the proposed design.

Questions for authors
  • How does DEM recover evidence from initially inactive shards? Could dynamic shard activation help?

  • Can the authors provide compute-matched comparisons to better isolate DEM's architectural benefits?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.