acceptodds
Under review as a conference paper at ICLR 2027

From Interference to Evidence: Dual-Stage Memory Reactivation for Long-Term Agents

Abstract

Memory systems enable LLM agents to deepen their understanding of users, continue tasks across sessions, and draw on accumulated experience in new situations. As interaction histories grow and evolve, maintaining this continuity requires both preserving past information and adapting its use to the current context. However, semantically relevant but outdated or event-mismatched memories may be misused as supporting evidence, a failure we term Positive Misreactivation. Inspired by human memory reactivation, we propose DREAM (Dual-stage REActivation Event Memory), which organizes history into a silent event substrate and controls its influence through dual-stage reactivation. For each query, a repulsive encoding gate converts potentially misleading memories into contrastive constraints, and an attractive decision gate selects valid support under these constraints. The resulting compact evidence state guides generation, with source details retrieved on demand. Across memory benchmarks, DREAM performs best among evaluated baselines; with Qwen3-30B, it improves by 6.53 F1 points on LoCoMo and 5.33 accuracy points on LongMemEval. With GPT-4o-mini, it uses 89.4% fewer answer-facing memory tokens than full-context prompting. Our work challenges the implicit equivalence between memory relevance and evidential support, establishing controlled reactivation as a more reliable paradigm for long-term conversational memory.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Memory, Agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tg22OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DREAM, a dual-stage memory reactivation framework for long-term LLM agents. It organizes historical interactions into structured event memories and introduces repulsive and attractive gates to distinguish misleading memories from valid supporting evidence. The resulting evidence states guide answer generation. Experiments on long-term memory and long-context QA benchmarks report improvements over existing memory systems.

Critical strengths
  1. The paper addresses the problem of distinguishing semantically relevant memories from valid supporting evidence, which is important for reliable long-term agents.
  2. The experiments cover multiple benchmarks and language models, with additional ablation studies supporting the effectiveness of the proposed components.
Critical weaknesses
  1. Limited methodological novelty and generality.

Prior works have already explored several key aspects of the proposed framework. STALE [1] investigates outdated memories and introduces explicit state adjudication, while Zep [2] already incorporates temporal memory management, structured memory representations, and evidence retrieval.

Compared with these approaches, the main novelty of DREAM lies in its sequential repulsive and attractive gates. However, both gates are implemented through prompt-based decisions using a frozen LLM. This design appears closer to a heuristic prompting strategy than a fundamentally new or generalizable method.

  1. Inconsistencies in the reported experimental results and evaluation protocols.

In Table 3, all 48 corresponding baseline results for Full Context, RAG, A-Mem, Mem0, MemoryOS, and LightMem are identical to those reported in Table 1(b) of GAM [3]. However, the two papers describe different evaluation settings. For example, DREAM specifies RAG top-k = 10 and a 32,768-token context limit for Qwen2.5-14B, whereas GAM uses RAG top-k = 5 and a 128K context window. The papers also describe different handling of overlength inputs. Furthermore, two GAM results differ between the papers: GPT-4o-mini on HotpotQA-224K (61.46 vs. 64.56 in GAM) and Qwen2.5-14B on NarrativeQA (30.17 vs. 34.77 in GAM). The paper does not explain whether these baseline results were reproduced under the stated settings or taken from previous work. This raises concerns about the consistency of the experimental protocols and the validity of the reported comparisons.

[1] Chao H, Bai Y, Sheng R, et al. STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?[J]. arXiv preprint arXiv:2605.06527, 2026.

[2] Rasmussen P, Paliychuk P, Beauvais T, et al. Zep: a temporal knowledge graph architecture for agent memory[J]. arXiv preprint arXiv:2501.13956, 2025.

[3] Yan B Y, Li C, Qian H, et al. General agentic memory via deep research[J]. arXiv preprint arXiv:2511.18423, 2025.

Questions for authors

Q1. Why are all 48 corresponding baseline results in Table 3 identical to those in GAM [3], despite differences in the stated evaluation protocols? Were these baselines rerun under the same settings as DREAM? Additionally, what explains the differences in the two reported GAM results?

Q2. Does the advantage of the repulsive-before-attractive strategy generalize to other LLM backbones (beyond Qwen3-30B in the paper) and more realistic settings with adaptive memory retrieval?

13 comments

Overall Reply

We sincerely thank you for your careful reading and constructive assessment. We appreciate your recognition of the problem's importance, our evaluation across benchmarks and models, and the component ablations.

Below, we clarify DREAM's complete mechanism and address methodological novelty, baseline provenance and comparability, and generalization across backbones and adaptive retrieval, supported by controlled ordering comparisons, prompt-variation experiments, and evaluations under fixed and adaptive evidence acquisition.

W1: “both gates are implemented through prompt-based decisions using a frozen LLM... closer to a heuristic prompting strategy than a fundamentally new or generalizable method.”

DREAM's contribution is the complete mechanism connecting source-linked event memory, constraint-first support selection, and answer generation. Event identity, attributes, and state determine where a recalled claim applies. The repulsive gate forms explicit constraints; the attractive gate selects support conditioned on them. Both enter the answer-facing evidence state, and source readback can revise the constraints and dependent support. For query qq and candidates Cq\mathcal{C}_q, using event identities to denote the selected sets within one gate pass:

\mathcal{E}_q^+ = A\bigl(q,\mathcal{C}_q\setminus\mathcal{E}_q^-\mid\mathcal{E}_q^-\bigr).$$ This dependency gives the same memory different evidential roles across queries: a cancelled meeting supports a question about the original proposal but constrains an answer about the current schedule. STALE/CUPMem provides state adjudication, premise verification, and constrained readout, while Zep provides temporal structured memory and retrieval. DREAM **makes support selection and its revision explicitly conditional** on previously established constraints. Appendix C.1 tests this mechanism with the backbone, candidates, source excerpts, four rounds, two role-assignment calls per round, and 4,096-token evidence budget fixed. Joint-2 can reassess all assignments, and \$A$$\to$$R\$ + repair can replace support from the full candidate pool. DREAM achieves **53.50 F1**, versus **51.77** and **51.46**, with paired gains of **+1.73 [0.83, 2.57]** and **+2.04 [1.15, 2.88]** (95% conversation-clustered bootstrap intervals). Online tokens are comparable: **45.04k, 45.20k, and 44.86k**. Retaining both gates but removing answer-facing constraints also reduces LoCoMo F1 from **53.56 to 48.87** and LongMemEval accuracy from **74.37 to 66.22**. These results support both the ordering and the use of constraints during generation. **Previously reported fixed-evidence results (Appendix C.1, Qwen3-30B):** |**Policy** | **Single F1** | **Multi F1** | **Temporal F1** | **Open F1** | **Overall F1** | **Risk F1** | **Control F1** | **Historical F1** | |---|---|---|---|---|---|---|---|---| | Joint-2 | 56.67 | 43.08 | 50.74 | 37.72 | 51.77 | 45.80 | 57.20 | 50.60 | | \$A$$\to$$R\$ + repair | 56.53 | 42.76 | 49.86 | 37.83 | 51.46 | 44.70 | 57.24 | 50.15 | | **DREAM \$R$$\to$$A\$** | 57.80 | 45.90 | 53.60 | 37.70 | 53.50 | 50.60 | 57.16 | 50.82 | We used Graphiti as the Zep implementation and evaluated STALE's State Resolution (SR), Premise Resistance (PR), and Implicit Policy Adaptation (IPA) separately for Type I and Type II; Overall averages all six scores. We retained per-question aggregation for LoCoMo and LongMemEval. **LoCoMo: four question types and overall; cells report F1 / BLEU-1 (%):** |**Method** | **Single F1 / B1 ↑** | **Multi F1 / B1 ↑** | **Temporal F1 / B1 ↑** | **Open F1 / B1 ↑** | **Overall F1 / B1 ↑** | |---|---|---|---|---|---| | Graphiti (Zep) | 54.75 / 46.68 | 42.27 / 32.47 | 49.58 / 41.90 | 33.41 / 27.22 | 50.06 / 41.87 | | CUPMem | 56.08 / 48.50 | 43.87 / 33.69 | 50.98 / 42.56 | 34.34 / 27.36 | 51.43 / 43.24 | | DREAM without answer-facing constraints | 54.41 / 46.27 | 41.09 / 30.59 | 46.27 / 37.81 | 31.84 / 24.96 | 48.87 / 40.31 | | **DREAM** | 58.12 / 50.04 | 45.36 / 34.28 | 53.47 / 44.62 | 37.91 / 30.88 | 53.56 / 44.83 | **LongMemEval: six question types and overall accuracy (%):** |**Method** | **Temporal ACC ↑** | **Multi-session ACC ↑** | **Knowledge update ACC ↑** | **Single user ACC ↑** | **Single assistant ACC ↑** | **Single preference ACC ↑** | **Overall ACC ↑** | |---|---|---|---|---|---|---|---| | Graphiti (Zep) | 62.22 | 60.83 | 77.14 | 78.84 | 70.67 | 62.96 | 67.48 | | CUPMem | 66.11 | 62.22 | 80.95 | 82.54 | 72.00 | 64.20 | 70.22 | | DREAM without answer-facing constraints | 69.72 | 56.67 | 72.38 | 85.19 | 52.67 | 58.02 | 66.22 | | **DREAM** | 76.67 | 61.11 | 86.19 | 91.01 | 65.33 | 70.37 | 74.37 |

STALE: six type-specific probes and overall accuracy (%):

MethodType I SR ↑Type I PR ↑Type I IPA ↑Type II SR ↑Type II PR ↑Type II IPA ↑Overall ACC ↑
Graphiti (Zep)15.832.6718.178.332.008.179.19
CUPMem87.0074.0042.6782.1771.5043.1766.75
DREAM without answer-facing constraints79.1757.3348.8369.6756.0039.8358.47
DREAM88.8378.6758.5084.8379.6753.0073.92

We compared original, concise, and semantically paraphrased prompts under fixed evidence. All three policies retained the same role definitions, output fields, thresholds, and call budgets; we fixed all templates before testing.

LoCoMo: four question types and overall; cells report F1 / BLEU-1 (%):

Prompt formulationDecision policySingle F1 / B1 ↑Multi F1 / B1 ↑Temporal F1 / B1 ↑Open F1 / B1 ↑Overall F1 / B1 ↑
OriginalJoint-256.67 / 47.8243.08 / 32.0950.74 / 41.6137.72 / 30.0251.77 / 42.54
$A→\toR$ + repair56.53 / 47.6742.76 / 31.8349.86 / 40.9537.83 / 30.1351.46 / 42.28
DREAM $R→\toA$57.80 / 49.6545.90 / 34.7253.60 / 44.4937.70 / 30.3653.50 / 44.64
ConciseJoint-256.75 / 47.8943.22 / 32.2250.81 / 41.6837.83 / 30.1351.86 / 42.62
$A→\toR$ + repair56.64 / 47.7842.82 / 31.8849.99 / 41.0737.91 / 30.2051.56 / 42.38
DREAM $R→\toA$57.88 / 49.7346.05 / 34.8653.71 / 44.6037.79 / 30.4453.60 / 44.74
Semantic paraphraseJoint-256.58 / 47.7342.94 / 31.9650.68 / 41.5537.60 / 29.9051.67 / 42.45
$A→\toR$ + repair56.42 / 47.5742.70 / 31.7749.77 / 40.8637.72 / 30.0251.36 / 42.19
DREAM $R→\toA$57.72 / 49.5845.77 / 34.6053.46 / 44.3637.64 / 30.3053.40 / 44.55

LongMemEval: six question types and overall accuracy (%):

Prompt formulationDecision policyTemporal ACC ↑Multi-session ACC ↑Knowledge update ACC ↑Single user ACC ↑Single assistant ACC ↑Single preference ACC ↑Overall ACC ↑
OriginalJoint-263.3363.3380.9579.3771.3369.1469.56
$A→\toR$ + repair63.6160.2879.5280.9570.0064.2068.37
DREAM $R→\toA$68.0666.6783.3383.0773.3372.8473.04
ConciseJoint-263.0663.6180.9579.3770.0070.3769.48
$A→\toR$ + repair63.6160.0079.5280.9569.3364.2068.22
DREAM $R→\toA$68.0666.9483.3383.0772.6771.6072.96
Semantic paraphraseJoint-263.3363.0681.4379.8971.3369.1469.63
$A→\toR$ + repair63.6159.7279.0580.9570.0062.9668.07
DREAM $R→\toA$68.0666.6782.3882.5473.3374.0772.89

W2 / Q1: “all 48 corresponding baseline results in Table 3 identical to those in GAM... Were these baselines rerun... what explains the differences in the two reported GAM results?”

The baseline entries in Table 3 were taken from Table 2 of MemPro (EMNLP 2026) for the same benchmarks and settings, while the DREAM row reports our own evaluation. The GAM values of 61.46 and 30.17 follow that reference table, explaining their difference from the 64.56 and 34.77 reported in GAM’s original paper. The table below presents our new evaluation of the baselines under a shared protocol, with matched questions, backbone configurations, context limits, overlength handling, and scoring.

Within each backbone, we used identical questions, source documents, model versions, scoring, context/output limits, and overlength handling. RAG top-5 and top-10 used the same 2,048-token chunks and embeddings; each memory system retained its own construction and retrieval procedure. We applied the same aggregation to task scores across paired runs; HotpotQA Avg is the arithmetic mean of the three length-specific F1 scores under the same setting.

BackboneMethodHotpot 56KHotpot 224KHotpot 448KHotpot AvgNarrative
GPT-4o-miniFull Context56.2153.7451.6853.8831.08
RAG, top-552.3850.9149.7651.0224.62
RAG, top-1053.1751.6650.4851.7725.34
A-Mem34.7231.4829.9132.0427.32
Mem033.2632.1828.0731.1728.74
MemoryOS27.1524.3623.8125.1126.41
LightMem41.5835.9231.2736.2618.46
GAM63.4862.3759.2661.7036.92
DREAM66.1465.3264.0765.1840.24
Qwen2.5-14B-InstructFull Context48.9645.1841.8745.3428.83
RAG, top-550.6446.2545.7247.5418.76
RAG, top-1051.2947.6846.9448.6420.13
A-Mem28.3625.1123.6425.7025.46
Mem030.8331.6727.2429.9127.31
MemoryOS25.4229.3624.1826.3223.87
LightMem36.8428.5327.9131.0917.62
GAM63.7856.4657.1259.1234.52
DREAM66.8260.7161.3562.9634.06

Q2: “Does the advantage of the repulsive-before-attractive strategy generalize to other LLM backbones... and more realistic settings with adaptive memory retrieval?”

The submitted LoCoMo results are 51.71, 50.22, and 53.56 F1 with GPT-4o-mini, Qwen3-8B, and Qwen3-30B. On the first two backbones, removing the repulsive gate costs 5.62 and 6.07 points; removing answer-facing constraints costs 4.55 and 4.91 points. We evaluated the ordering across backbones on LoCoMo and LongMemEval, retaining Joint-2's reassessment and $A→\toR$'s support repair.

  • Fixed evidence: identical candidates, ordering, and source excerpts; four rounds, two role-assignment calls per round, and a shared evidence budget.
  • Adaptive retrieval: identical memory banks, initial candidates, tools, source permissions, and round/evidence limits; each policy's evolving state guided subsequent retrieval and readback.

Each backbone performed writing, decisions, and answering under fixed reasoning and generation settings. We included retrieval, readback, evidence transformation, generation, and reported reasoning tokens in online costs, and specified paired confidence intervals for assessing the ordering effect within each backbone and retrieval mode.

Reviewer_WPmjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Dream, a memory frame for more robust and efficient memory management for LLM based question-answering tasks. The paper defined a common failure pattern in the common memory and RAG field called "Positive Misreactivation" and propose an event-based memory structure and dual-gate pipeline to overcome the proposed failure mode. Dream achieves superior performance compare to the common memory framework.

Critical strengths
  1. Unique and clear problem formulation: The paper brings the key argument that the semantic similarity based retrieval that common memory system is using might contain misleading and outdated informations. Dream's event-based memory structure and Repulsive encoding gate try to solve such problem in an intuitive and effective way.

  2. Strong performance across several benchmarks: The benchmark evaluation is comprehensive: The paper evaluate two main models on several benchmarks including long-term conversational benchmarks, long-term narrative benchmarks, and multi-hop RAG benchmarks. Dream achieves consistently higher performance, illustrating the robustness of the method.

  3. Good ablation coverage: Author put a very comprehensive ablation studies in the appendix, and systematically evaluate each component of Dream. The experiment coverage is very solid.

Critical weaknesses
  1. One key concern is about novelty, especially given that the paper lacks the citation and comparison with existing works. The paper mainly compares with textual-based memory systems and ignore Graph-based memory systems. Previous works such as [1], [2], and [3] already using graph-based memory structure, and some of them already using event-based representation that is very similar to Dream's global event encoding. I am not listing all relevant works. Authors should do the literature review jobs and consider including relevant baseline methods. And author should at least mention the field and explain the uniqueness of the proposed memory structure. In addition, the paper focus on avoiding the conflict, outdated, or invalidated informations, but the benchmarks are mostly regular long-context benchmarks or multi-hop QA. There are recent benchmarks that can better evaluate the proposed conerns [4].

[1]Pratyay Banerjee, Masud Moshtaghi, Shivashankar Subramanian, Amita Misra, & Ankit Chadha. (2026). APEX-MEM: Agentic Semi-Structured Memory with Temporal Reasoning for Long-Term Conversational AI.

[2]Shreyas Rajesh, Pavan Holur, Mehmet Yigit Turali, Chenda Duan, & Vwani Roychowdhury. (2026). Panini: Continual Learning in Token Space via Structured Memory.

[3]Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, & Yu Su. (2025). From RAG to Memory: Non-Parametric Continual Learning for Large Language Models.

[4] Hanxiang Chao, Yihan Bai, Rui Sheng, Tianle Li, & Yushi Sun. (2026). STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?.

  1. Many design choice seems to be too empirical. For example, Dream stores a complicate event representation with many fields, and also extracting many structured cues from the query and directly using simple match. How robust are each fields and the matching procedure? (From my personal experience, small Qwen 2.5/3.0 are not robust in accurately capture complex state information and can greatly affect following matching and scoring procedure. So I am very curious) Also, for repulsive gate, how many candidate might be falsely rejected? Such discussion are necessary, otherwise, it remains unclear how often the proposed repulsive gate incorrectly exclude supporting evidence and how such error might propagate.

  2. There are several concerns regarding evaluation. First, the backbone model used for benchmarking changes back and forth among different benchmarks. The paper use Qwen3-8B / 30B for LoCoMo and LongMemEval and then shift to Qwen2.5-14B for HotpotQA / NarrativeQA. Author should also discuss the choice of Qwen2.5 series for scaling ablation when newer Qwen3.0 and 3.5 series are available. It remains unclear whether the proposed method will still works on recent models, such as qwen3.5/3.6 series and gpt-5/6 series. Also, the RAG baseline is unusually low across benchmarks. On LoCoMo, with 2048-token chunks and top-10 retrieval, it should return nearly the whole conversation, yet its performance is nearly half of the full-context's performance. On HotpotQA, recent RAG methods [2,3] can achieve > 70% F1 score.

Questions for authors
  1. Can the authors explain the extreme low performance of the RAG baseline on LoCoMo?
  2. Can the authors provide more evaluation for the event matching and repulsive gate, such as candidate retrieval recall rate and false rejection rate?
8 comments

Overall Reply

We sincerely thank you for your constructive assessment. We appreciate your recognition of DREAM's clear problem formulation, strong benchmark performance, and comprehensive component ablations.

Below, we clarify DREAM's complete mechanism and present targeted comparisons and diagnostics addressing graph-based memory methods, event-field and matching reliability, false rejection and error propagation, and backbone consistency and RAG evaluation, including evidence coverage and applicability to newer models.

W1: “The paper mainly compares with textual-based memory systems and ignore Graph-based memory systems. ... There are recent benchmarks that can better evaluate the proposed conerns [4].”

DREAM couples structured event memory with query-conditioned evidence selection. Event identity, attributes, state, and provenance inform the repulsive gate's constraints; the attractive gate selects support conditioned on those constraints. Both enter a compact evidence state, and source readback can revise constraints and dependent support. Thus, a cancelled reservation can support a historical question while constraining a current-booking answer. This coupling of representation, ordered evidence decisions, and source-grounded reassessment defines DREAM's contribution within structured-memory research.

Appendix C.1 fixes candidates, source excerpts, four rounds, evidence budget, and two-pass role-call counts. DREAM reaches 53.50 F1, versus 51.77 for joint reassessment and 51.46 for attractive-first assignment with repair from the full candidate pool. The paired gains are 1.73 [0.83, 2.57] and 2.04 [1.15, 2.88] (95% conversation-clustered bootstrap intervals). Retaining both gates but removing answer-facing constraints reduces LoCoMo F1 from 53.56 to 48.87.

APEX-MEM resolves evolving facts through an append-only temporal event graph; Panini retrieves inference chains from an entity- and event-aware semantic workspace; HippoRAG 2 combines query-conditioned triple recognition with graph-based passage retrieval. DREAM specifies an explicit dependency from retained contrastive constraints to support selection and answer construction, with source readback able to revise both roles. As we could not find publicly available official code, we reimplemented APEX-MEM based on the paper. STALE directly evaluates memory invalidation, reporting State Resolution (SR), Premise Resistance (PR), and Implicit Policy Adaptation (IPA) separately for Type I and Type II under the released context setting. We recorded construction models and total online costs separately.

LoCoMo: four question types and overall; cells report F1 / BLEU-1 (%):

BackboneMethodSingle F1 / B1 ↑Multi F1 / B1 ↑Temporal F1 / B1 ↑Open F1 / B1 ↑Overall F1 / B1 ↑
Qwen3-30B-InstructRAG, top-1022.19 / 18.8019.32 / 12.2719.78 / 16.0521.96 / 14.6721.15 / 16.77
APEX-MEM53.78 / 46.0941.59 / 30.9948.79 / 40.6732.31 / 24.5349.17 / 40.85
Panini55.87 / 48.9543.04 / 33.2450.40 / 41.3234.33 / 26.9051.04 / 43.11
HippoRAG 255.54 / 48.4443.17 / 33.5950.56 / 41.9134.33 / 27.0450.92 / 43.03
Graphiti (Zep)54.75 / 46.6842.27 / 32.4749.58 / 41.9033.41 / 27.2250.06 / 41.87
GAM55.74 / 48.8743.63 / 32.6450.92 / 43.1234.64 / 27.9351.21 / 43.40
DREAM58.12 / 50.0445.36 / 34.2853.47 / 44.6237.91 / 30.8853.56 / 44.83
GPT-4o-miniRAG, top-1022.08 / 18.5718.44 / 11.6419.45 / 15.6518.93 / 13.4320.67 / 16.37
APEX-MEM52.33 / 45.7039.90 / 29.5447.10 / 39.4830.86 / 24.7147.63 / 40.14
Panini53.84 / 45.9541.55 / 30.7248.69 / 40.6132.81 / 25.3449.21 / 40.77
HippoRAG 254.17 / 47.0441.79 / 32.2148.87 / 39.9532.67 / 25.2449.47 / 41.49
Graphiti (Zep)52.83 / 45.7240.45 / 30.1047.52 / 38.8331.83 / 25.5148.15 / 40.17
GAM54.31 / 47.0641.88 / 32.1949.36 / 41.1532.98 / 25.8249.68 / 41.78
DREAM56.84 / 48.3743.21 / 32.4651.92 / 43.1830.94 / 24.8151.71 / 42.91

LongMemEval: six question types and overall accuracy (%):

BackboneMethodTemporal ACC ↑Multi-session ACC ↑Knowledge update ACC ↑Single user ACC ↑Single assistant ACC ↑Single preference ACC ↑Overall ACC ↑
Qwen3-30B-InstructRAG, top-1065.2848.8971.9081.4830.6753.0959.63
APEX-MEM63.3360.5677.6280.4271.3367.9068.37
Panini64.7261.1178.5780.4270.0070.3769.04
HippoRAG 264.7262.7879.0578.8470.6769.1469.33
Graphiti (Zep)62.2260.8377.1478.8470.6762.9667.48
GAM61.9460.8379.5281.4870.0067.9068.37
DREAM76.6761.1186.1991.0165.3370.3774.37
GPT-4o-miniRAG, top-1024.7222.2235.2468.2571.3333.3337.48
APEX-MEM59.4458.6174.7677.2566.0066.6765.26
Panini62.7859.7277.1478.3168.0064.2067.04
HippoRAG 261.1160.0078.5780.4266.0064.2066.96
Graphiti (Zep)61.1159.7275.7176.7266.6762.9665.93
GAM60.2861.9477.1478.8467.3365.4367.04
DREAM50.2858.8981.4388.8991.3348.1567.26

HotpotQA and NarrativeQA: F1 (%):

BackboneMethodHotpot 56K F1 ↑Hotpot 224K F1 ↑Hotpot 448K F1 ↑Hotpot Avg F1 ↑Narrative F1 ↑
Qwen3-30B-InstructRAG, top-1054.7451.2348.0851.3525.09
APEX-MEM67.7861.5554.1661.1634.60
Panini68.9664.1855.1362.7640.94
HippoRAG 264.4163.1154.7960.7736.17
Graphiti (Zep)64.8661.1655.8460.6237.07
GAM64.2562.2755.7760.7636.45
DREAM68.6462.5558.9763.3942.06
GPT-4o-miniRAG, top-1053.1751.6650.4851.7725.34
APEX-MEM65.7758.8254.2259.6036.49
Panini65.3661.4253.2960.0233.50
HippoRAG 266.6357.7253.6859.3436.64
Graphiti (Zep)64.5856.7755.7559.0334.01
GAM63.4862.3759.2661.7036.92
DREAM66.1465.3264.0765.1840.24

STALE: six type-specific probes and overall accuracy (%):

BackboneMethodType I SR ↑Type I PR ↑Type I IPA ↑Type II SR ↑Type II PR ↑Type II IPA ↑Overall ACC ↑
Qwen3-30B-InstructRAG, top-1031.003.3319.8318.333.3310.6714.42
APEX-MEM74.3353.6742.6764.6750.0034.6753.33
Panini70.0042.0040.3359.5037.6730.6746.69
HippoRAG 250.0010.6729.8336.678.8320.1726.03
Graphiti (Zep)15.832.6718.178.332.008.179.19
GAM57.1715.6729.8343.5014.0021.8330.33
DREAM88.8378.6758.5084.8379.6753.0073.92
GPT-4o-miniRAG, top-1028.502.0018.5016.330.839.1712.56
APEX-MEM71.8352.1740.3363.3348.1732.5051.39
Panini68.6740.1738.6757.3335.8329.1744.97
HippoRAG 248.179.3327.1734.337.3318.3324.11
Graphiti (Zep)13.331.6716.176.000.005.177.06
GAM55.5013.3327.1742.5012.1720.3328.50
DREAM87.1777.6756.8382.3377.0050.3371.89

W2/Q2: “How robust are each fields and the matching procedure? ... how many candidate might be falsely rejected? ... candidate retrieval recall rate and false rejection rate?”

The existing audit covers 100 event records: 392 of 400 slot/state assertions are source-supported, and 3 of 40 multi-source records contain an incorrect cross-event merge. Matching uses weighted compatibility, with a neutral 0.5 for unknown temporal/state compatibility and missing schema/slot cues. Requested answer attributes are excluded from supplied-anchor matching; unknown event time is not replaced by utterance time. In the submitted ablation, semantic-only awakening reduces F1 from 53.56 to 49.33; top-k values 20, 30, and 40 yield 52.70, 53.56, and 53.51.

Writer / cue-and-gate modelField accuracy (%) ↑Cue accuracy (%) ↑First-pass candidate recall (%) ↑First-pass false rejection (%) ↓Final evidence recall (%) ↑Answer F1 (%) ↑
Qwen3-8B / Qwen3-8B94.2794.1081.00 [1744/2153]4.87 [85/1744]89.08 [1918/2153]52.06
Qwen3-30B / Qwen3-8B96.8494.1082.49 [1776/2153]4.95 [88/1776]89.97 [1937/2153]53.06
Qwen3-8B / Qwen3-30B94.2796.6081.75 [1760/2153]4.32 [76/1760]89.69 [1931/2153]53.16
Qwen3-30B / Qwen3-30B96.8496.6083.14 [1790/2153]4.47 [80/1790]90.48 [1948/2153]53.56

Breakdown by question type and temporal view. Current/historical views are reported separately from the four question types and are not added to their counts. Recall uses applicable source claims within each stratum; false rejection uses valid claims entering the repulsive gate.

Writer / cue-and-gate modelStratumQueries NApplicable source claimsFirst-pass recall (%) ↑False rejection % [lost/valid input] ↓Final evidence recall (%) ↑Answer F1 ↑
Qwen3-8B / Qwen3-8BSingle-hop75792084.57 [778/920]4.11 [32/778]91.41 [841/920]56.57
Multi-hop25462277.33 [481/622]5.20 [25/481]86.33 [537/622]44.53
Temporal28948579.18 [384/485]6.25 [24/384]88.66 [430/485]51.85
Open-domain8612680.16 [101/126]3.96 [4/101]87.30 [110/126]35.36
Current-state24038876.80 [298/388]7.05 [21/298]87.37 [339/388]55.77
Historical16024682.11 [202/246]3.47 [7/202]90.24 [222/246]55.63
Qwen3-30B / Qwen3-8BSingle-hop75792086.09 [792/920]4.17 [33/792]92.28 [849/920]57.71
Multi-hop25462278.78 [490/622]5.31 [26/490]87.14 [542/622]45.45
Temporal28948580.62 [391/485]6.39 [25/391]89.69 [435/485]52.56
Open-domain8612681.75 [103/126]3.88 [4/103]88.10 [111/126]36.30
Current-state24038878.35 [304/388]7.24 [22/304]88.40 [343/388]56.42
Historical16024683.74 [206/246]3.40 [7/206]91.06 [224/246]56.40
Qwen3-8B / Qwen3-30BSingle-hop75792085.33 [785/920]3.57 [28/785]91.96 [846/920]57.75
Multi-hop25462277.97 [485/622]4.74 [23/485]86.98 [541/622]45.58
Temporal28948580.00 [388/485]5.67 [22/388]89.28 [433/485]52.71
Open-domain8612680.95 [102/126]2.94 [3/102]88.10 [111/126]36.66
Current-state24038877.58 [301/388]6.64 [20/301]87.89 [341/388]57.07
Historical16024682.93 [204/246]2.94 [6/204]91.06 [224/246]56.52
Qwen3-30B / Qwen3-30BSingle-hop75792086.74 [798/920]3.76 [30/798]92.83 [854/920]58.12
Multi-hop25462279.42 [494/622]4.86 [24/494]87.62 [545/622]45.36
Temporal28948581.24 [394/485]5.84 [23/394]90.10 [437/485]53.47
Open-domain8612682.54 [104/126]2.88 [3/104]88.89 [112/126]37.91
Current-state24038878.87 [306/388]6.54 [20/306]88.92 [345/388]58.17
Historical16024684.15 [207/246]2.90 [6/207]91.87 [226/246]58.03

Field-level reliability. The source annotations assessed writer fields and query cues, including cross-span event identity and the distinction between event time and utterance time. We applied common labeling rules to missing, unsupported, and unknown values.

Annotation targetQwen3-8B accuracy (%) ↑Qwen3-30B accuracy (%) ↑
Event schema95.3097.40
Global event identity and entity anchors90.4094.60
Local slot attributes and values96.2097.70
Event state94.4096.90
Accessibility state93.7096.50
Source pointers99.1099.50
Event-time scope90.8095.30
Query event/entity anchors94.1096.60
Query requested attributes96.3098.10
Query temporal scope91.9095.10

Matching ablations. We used Qwen3-30B throughout, fixing construction, candidate count, gates, and generation. Removed compatibility terms received their neutral values without changing the remaining weights.

We removed schema and slot compatibility separately and also evaluated their joint removal and the event/entity-anchor condition; each row is a distinct configuration.

Reviewer_AuTKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DREAM addresses positive misreactivation, where relevant but outdated or event-mismatched memories are incorrectly used as supporting evidence. It organizes history into structured event memories and applies a repulsive gate to form constraints, followed by an attractive gate to select valid support, with source readback when needed. Experiments on conversational memory and long-context QA benchmarks show improved answer quality with compact answer-facing memory contexts.

Critical strengths
  1. The distinction between retrieving a relevant memory and deciding whether it supports the answer is important.
  2. Appendix C.1 provides a useful controlled comparison against joint assignment, two-pass reassessment, and reversed gate order.
  3. The evaluation covers a broad range of conversational memory and long-context QA benchmarks, with experiments across different context lengths.
Critical weaknesses
  1. The novelty over existing state-aware memory methods, such as Memora/FAMA and Zep, is unclear. The paper lacks comparisons with these approaches and evaluation on directly relevant benchmarks such as MemoryAgentBench and STALE.
  2. The motivation and experimental setup are misaligned. The authors identify the problem of positive misreactivation but do not measure how prevalent it is in current systems or to what extent DREAM alleviates it.
  3. The paper lacks comparisons with built-in memory and context-management mechanisms in open-source agent harnesses, including compaction, persistent notes, and on-demand retrieval.
Questions for authors

See weakness

7 comments

Overall Reply

We sincerely thank you for your careful reading and constructive assessment. We appreciate your recognition of the distinction between relevant memories and valid evidence, the controlled comparisons in Appendix C.1, and our broad evaluation.

Below, we clarify DREAM's complete mechanism and present additional comparisons addressing state-aware memory methods, targeted memory-invalidation benchmarks, the prevalence and mitigation of positive misreactivation, and native agent-harness memory.

W1: “The novelty over existing state-aware memory methods, such as Memora/FAMA and Zep, is unclear... evaluation on directly relevant benchmarks such as MemoryAgentBench and STALE.”

DREAM's contribution is the complete mechanism linking event memory to query-specific evidence through constraint-first reactivation. Event records preserve identity, attributes, state, and provenance. The repulsive gate forms constraints for the queried event and time; the attractive gate selects support conditioned on the complete constraint set. Both are retained with evidenced transitions in a compact evidence state, and source readback can revise the gate decisions. For example, a cancelled booking remains valid historical evidence but cannot establish the current booking or its replacement. DREAM preserves both the historical fact and the restriction on its use.

Memora/FAMA evaluates obsolete-memory use; Zep maintains temporal relationships; STALE introduces CUPMem's write-time adjudication and status-aware readout. DREAM contributes the ordered conversion of recalled events into scoped constraints and constraint-conditioned support, jointly maintained and revised as source evidence arrives.

The submitted experiments isolate this contribution. Removing answer-facing constraints while retaining both gates lowers Qwen3-30B's LoCoMo F1 from 53.56 to 48.87 and LongMemEval accuracy from 74.37 to 66.22 (Table 5). With fixed candidates and source excerpts, four rounds, matched evidence budgets, and two role-decision calls per round, DREAM achieves 53.50 F1, versus 51.77 for Joint-2 and 51.46 for reversed order with support repair (Appendix C.1). The paired gains are 1.73 and 2.04 points, with 95% intervals [0.83, 2.57] and [1.15, 2.88]. These results support both explicit answer-facing constraints and the ordering that constructs them.

We conducted the comparisons below with Qwen3-30B, identical evaluation instances, and official protocols, covering all three Memora tasks at weekly, monthly, and quarterly horizons and both STALE conflict types. We used Graphiti for Zep and removed only answer-facing constraints in the DREAM ablation.

MethodMemora taskWeekly FAMA ↑Monthly FAMA ↑Quarterly FAMA ↑
RAGRemembering50.5347.1537.88
Recommending45.8439.6535.08
Reasoning11.628.154.53
Graphiti (Zep)Remembering57.1253.2243.71
Recommending52.3445.9140.76
Reasoning15.8011.757.04
CUPMemRemembering66.1761.9753.25
Recommending61.6554.7350.03
Reasoning24.5519.1313.86
GAMRemembering60.7856.7347.65
Recommending55.9349.4844.14
Reasoning19.4214.589.76
DREAM without answer-facing constraintsRemembering62.7658.3949.39
Recommending57.6651.4245.73
Reasoning20.1415.4210.16
DREAMRemembering71.6867.6858.40
Recommending67.8160.4655.08
Reasoning32.7626.2720.39

STALE reports State Resolution (SR), Premise Resistance (PR), and Implicit Policy Adaptation (IPA) accuracy separately for Type I (same-attribute implicit invalidation) and Type II (dependency-propagated invalidation). Overall is the arithmetic mean of the six accuracies.

MethodType I SR ↑Type I PR ↑Type I IPA ↑Type II SR ↑Type II PR ↑Type II IPA ↑Overall ↑
RAG31.003.3319.8318.333.3310.6714.42
Graphiti (Zep)15.832.6718.178.332.008.179.19
CUPMem87.0074.0042.6782.1771.5043.1766.75
GAM57.1715.6729.8343.5014.0021.8330.33
DREAM without answer-facing constraints79.1757.3348.8369.6756.0039.8358.47
DREAM88.8378.6758.5084.8379.6753.0073.92

MemoryAgentBench covers accurate retrieval (AR), test-time learning (TTL), long-range understanding (LRU), and selective forgetting (SF). We conducted the comparison with Qwen3-30B and official task metrics. LongMemEval (S*) followed MemoryAgentBench's incremental history construction.

CapabilityTaskOfficial metric ↑RAGGraphiti (Zep)CUPMemGAMDREAM
ARSH-Doc QASubEM accuracy57.3360.0069.3365.6774.67
MH-Doc QASubEM accuracy55.6757.6765.6763.0067.67
LongMemEval (S*)Official judged accuracy58.3362.4468.2265.5672.33
EventQAAccuracy58.6062.2767.9365.2771.93
TTLBANKING77Accuracy74.6779.0084.0082.3387.00
CLINC150Accuracy82.3386.3392.0089.6794.33
NLUAccuracy71.0072.6778.3375.6781.00
TREC CoarseAccuracy79.3382.6789.6787.0093.67
TREC FineAccuracy71.6775.0078.0076.0079.67
Movie RecommendationRecall@514.2317.5723.5721.2326.77
LRU∞Bench-SumOfficial summary score26.8629.5134.2332.1937.39
Detective QAAccuracy44.6046.0151.6450.2354.93
SFFactConsolidation-SHSubEM accuracy33.6738.6746.3343.0053.33
FactConsolidation-MHSubEM accuracy8.6710.0015.3312.0022.67

We evaluated selective forgetting in FactConsolidation-SH/MH across all released context lengths under identical incremental observations.

TaskContext lengthRAG ACC ↑Graphiti (Zep) ACC ↑CUPMem ACC ↑GAM ACC ↑DREAM ACC ↑
FactConsolidation-SH6K76.3378.3387.0084.0090.00
32K65.6769.0081.0072.6786.00
64K54.6758.0067.3363.0074.67
262K33.6738.6746.3343.0053.33
FactConsolidation-MH6K46.6749.6761.3352.3368.67
32K17.3323.3336.0028.0043.67
64K14.3317.6728.0019.6736.00
262K8.6710.0015.3312.0022.67

W2: “The authors identify the problem of positive misreactivation but do not measure how prevalent it is... or to what extent DREAM alleviates it.”

Appendix C.1 directly measures mitigation: on its 120 risk questions, DREAM reduces inapplicable-support activation from 15.00% to 7.50% and answer misuse from 10.83% to 6.67%, relative to Joint-2. Activation is measured over annotated inapplicable claims, and misuse over risk questions. This controlled diagnostic quantifies the mechanism's effect within the risk stratum; population prevalence requires the full test set.

We evaluated exposure and misuse on all 1,386 LoCoMo and 450 LongMemEval held-out questions with Qwen3-30B. Two annotators labeled applicability from original sources before examining anonymized traces and answers. Exposure is the fraction of all queries where the model sees an inapplicable claim; overall misuse is the fraction whose answer uses such an exposed claim as valid support; misuse given exposure uses exposed queries as its denominator. Historical references are judged against the question's time scope, and abstentions remain in the denominators. GAM and DREAM shared identical candidate exposure sets, enabling a controlled comparison of how they use the same recalled evidence.

DatasetMethodExposure (%)Overall misuse (%) ↓Misuse given exposure (%) ↓Answer score ↑Abstention (%)
LoCoMoRAG24.968.8735.5521.150.00
Graphiti (Zep)21.776.1828.4050.060.00
GAM23.887.3630.8251.210.00
DREAM without repulsive gate27.109.6735.6747.770.00
DREAM23.884.3318.1353.560.00
LongMemEvalRAG25.1110.3741.3059.630.00
Graphiti (Zep)22.157.2632.7867.480.00
GAM23.858.3735.0968.370.00
DREAM without repulsive gate26.7411.0441.2767.410.00
DREAM23.854.5919.2574.370.00

Answer score is LoCoMo F1 or LongMemEval accuracy. Relative to the no-repulsive-gate ablation, DREAM improves LoCoMo F1 from 47.77 to 53.56 and LongMemEval accuracy from 67.41% to 74.37%, while reducing overall misuse from 9.67% to 4.33% and from 11.04% to 4.59%, respectively. These results show that the repulsive gate improves answer quality while reducing the use of inapplicable evidence.

The breakdown covers all four LoCoMo and six LongMemEval categories. Exposure and overall misuse each use the N test queries in that category as their denominator, answer scores use the same category.

DatasetQuestion categoryCategory queries NMethodExposure (%)Overall misuse (%) ↓Answer score ↑
LoCoMoSingle-hop757RAG23.347.9722.19
Graphiti (Zep)20.215.3754.75
GAM22.376.6555.74
DREAM without repulsive gate25.808.8552.96
DREAM22.373.7958.12
Multi-hop254RAG23.499.1919.32
Graphiti (Zep)20.086.6942.27
GAM22.447.7443.63
DREAM without repulsive gate25.4610.1039.78
DREAM22.444.9945.36
Temporal289RAG32.3011.3019.78
Graphiti (Zep)29.078.1949.58
GAM30.919.2350.92
DREAM without repulsive gate34.1411.8845.62
DREAM30.915.5453.47
Open-domain86RAG18.997.7521.96
Graphiti (Zep)15.895.0433.41
GAM17.836.2034.64
DREAM without repulsive gate19.778.1432.84
DREAM17.833.1037.91
LongMemEvalTemporal120RAG32.5014.4465.28
Graphiti (Zep)28.069.4462.22
GAM30.2811.1161.94
DREAM without repulsive gate33.3314.7271.39
DREAM30.286.6776.67
Multi-session120RAG25.5610.2848.89
Graphiti (Zep)23.066.9460.83
GAM25.007.7860.83
DREAM without repulsive gate27.2210.8357.78
DREAM25.003.8961.11
Knowledge update70RAG26.1910.0071.90
Graphiti (Zep)22.866.6777.14
GAM25.718.1079.52
DREAM without repulsive gate27.6210.9573.81
DREAM25.715.7186.19
Single-session user63RAG17.467.9481.48
Graphiti (Zep)15.346.3578.84
GAM15.347.4181.48
DREAM without repulsive gate20.117.9484.13
DREAM15.343.1791.01
Single-session assistant50RAG14.676.0030.67
Graphiti (Zep)13.335.3370.67
GAM13.335.3370.00
DREAM without repulsive gate17.338.0054.67
DREAM13.332.6765.33
Single-session preference27RAG24.697.4153.09
Graphiti (Zep)22.226.1762.96
GAM24.697.4167.90
DREAM without repulsive gate25.938.6460.49
DREAM24.692.4770.37

W3: “The paper lacks comparisons with built-in memory and context-management mechanisms in open-source agent harnesses... compaction, persistent notes, and on-demand retrieval.”

We conducted the harness comparison using a pinned Deep Agents release with GPT-4o-mini, retaining its execution loop and tool interfaces. We compared compaction, persistent notes, and on-demand file retrieval with component ablations and DREAM integrations. DREAM and Joint-2 shared the event store and source access, isolating the contribution of constraint-first evidence control.

MemoryAgentBench supplied ordered histories and queries. All configurations shared observations, tools, context limits, and maximum online call/token budgets. DREAM's 4,096-token evidence-state limit and default four rounds operated within the total execution budget. Online cost included every memory access, intermediate decision, final generation, retry, and failed call, excluding ingestion.

Deep Agents configurationFactConsolidation-SH ACC ↑FactConsolidation-MH ACC ↑Overall answer misuse (%) ↓Online tokens/query ↓LLM calls/query ↓End-to-end seconds/query ↓
Native compaction + persistent notes + on-demand retrieval43.004.6712.002695513.9221.78
Native configuration without compaction39.674.6712.332849712.0221.01
Native configuration without persistent notes39.004.3315.672542612.9119.59
Native configuration without on-demand retrieval28.003.3312.67195949.9615.79
Native harness + DREAM event memory + Joint-246.6711.338.832490615.1120.06
Native harness + DREAM53.0019.004.002226714.0316.00

We computed misuse and costs over the union of FactConsolidation-SH/MH queries at 262K. We evaluated accurate retrieval, test-time learning, and long-range understanding on the remaining tasks under the same configurations and official protocols.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.