- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes DREAM, a dual-stage memory reactivation framework for long-term LLM agents. It organizes historical interactions into structured event memories and introduces repulsive and attractive gates to distinguish misleading memories from valid supporting evidence. The resulting evidence states guide answer generation. Experiments on long-term memory and long-context QA benchmarks report improvements over existing memory systems.
Critical strengths
- The paper addresses the problem of distinguishing semantically relevant memories from valid supporting evidence, which is important for reliable long-term agents.
- The experiments cover multiple benchmarks and language models, with additional ablation studies supporting the effectiveness of the proposed components.
Critical weaknesses
- Limited methodological novelty and generality.
Prior works have already explored several key aspects of the proposed framework. STALE [1] investigates outdated memories and introduces explicit state adjudication, while Zep [2] already incorporates temporal memory management, structured memory representations, and evidence retrieval.
Compared with these approaches, the main novelty of DREAM lies in its sequential repulsive and attractive gates. However, both gates are implemented through prompt-based decisions using a frozen LLM. This design appears closer to a heuristic prompting strategy than a fundamentally new or generalizable method.
- Inconsistencies in the reported experimental results and evaluation protocols.
In Table 3, all 48 corresponding baseline results for Full Context, RAG, A-Mem, Mem0, MemoryOS, and LightMem are identical to those reported in Table 1(b) of GAM [3]. However, the two papers describe different evaluation settings. For example, DREAM specifies RAG top-k = 10 and a 32,768-token context limit for Qwen2.5-14B, whereas GAM uses RAG top-k = 5 and a 128K context window. The papers also describe different handling of overlength inputs. Furthermore, two GAM results differ between the papers: GPT-4o-mini on HotpotQA-224K (61.46 vs. 64.56 in GAM) and Qwen2.5-14B on NarrativeQA (30.17 vs. 34.77 in GAM). The paper does not explain whether these baseline results were reproduced under the stated settings or taken from previous work. This raises concerns about the consistency of the experimental protocols and the validity of the reported comparisons.
[1] Chao H, Bai Y, Sheng R, et al. STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?[J]. arXiv preprint arXiv:2605.06527, 2026.
[2] Rasmussen P, Paliychuk P, Beauvais T, et al. Zep: a temporal knowledge graph architecture for agent memory[J]. arXiv preprint arXiv:2501.13956, 2025.
[3] Yan B Y, Li C, Qian H, et al. General agentic memory via deep research[J]. arXiv preprint arXiv:2511.18423, 2025.
Questions for authors
Q1. Why are all 48 corresponding baseline results in Table 3 identical to those in GAM [3], despite differences in the stated evaluation protocols? Were these baselines rerun under the same settings as DREAM? Additionally, what explains the differences in the two reported GAM results?
Q2. Does the advantage of the repulsive-before-attractive strategy generalize to other LLM backbones (beyond Qwen3-30B in the paper) and more realistic settings with adaptive memory retrieval?
13 comments
STALE: six type-specific probes and overall accuracy (%):
| Method | Type I SR ↑ | Type I PR ↑ | Type I IPA ↑ | Type II SR ↑ | Type II PR ↑ | Type II IPA ↑ | Overall ACC ↑ |
|---|---|---|---|---|---|---|---|
| Graphiti (Zep) | 15.83 | 2.67 | 18.17 | 8.33 | 2.00 | 8.17 | 9.19 |
| CUPMem | 87.00 | 74.00 | 42.67 | 82.17 | 71.50 | 43.17 | 66.75 |
| DREAM without answer-facing constraints | 79.17 | 57.33 | 48.83 | 69.67 | 56.00 | 39.83 | 58.47 |
| DREAM | 88.83 | 78.67 | 58.50 | 84.83 | 79.67 | 53.00 | 73.92 |
We compared original, concise, and semantically paraphrased prompts under fixed evidence. All three policies retained the same role definitions, output fields, thresholds, and call budgets; we fixed all templates before testing.
LoCoMo: four question types and overall; cells report F1 / BLEU-1 (%):
| Prompt formulation | Decision policy | Single F1 / B1 ↑ | Multi F1 / B1 ↑ | Temporal F1 / B1 ↑ | Open F1 / B1 ↑ | Overall F1 / B1 ↑ |
|---|---|---|---|---|---|---|
| Original | Joint-2 | 56.67 / 47.82 | 43.08 / 32.09 | 50.74 / 41.61 | 37.72 / 30.02 | 51.77 / 42.54 |
| $AR$ + repair | 56.53 / 47.67 | 42.76 / 31.83 | 49.86 / 40.95 | 37.83 / 30.13 | 51.46 / 42.28 | |
| DREAM $RA$ | 57.80 / 49.65 | 45.90 / 34.72 | 53.60 / 44.49 | 37.70 / 30.36 | 53.50 / 44.64 | |
| Concise | Joint-2 | 56.75 / 47.89 | 43.22 / 32.22 | 50.81 / 41.68 | 37.83 / 30.13 | 51.86 / 42.62 |
| $AR$ + repair | 56.64 / 47.78 | 42.82 / 31.88 | 49.99 / 41.07 | 37.91 / 30.20 | 51.56 / 42.38 | |
| DREAM $RA$ | 57.88 / 49.73 | 46.05 / 34.86 | 53.71 / 44.60 | 37.79 / 30.44 | 53.60 / 44.74 | |
| Semantic paraphrase | Joint-2 | 56.58 / 47.73 | 42.94 / 31.96 | 50.68 / 41.55 | 37.60 / 29.90 | 51.67 / 42.45 |
| $AR$ + repair | 56.42 / 47.57 | 42.70 / 31.77 | 49.77 / 40.86 | 37.72 / 30.02 | 51.36 / 42.19 | |
| DREAM $RA$ | 57.72 / 49.58 | 45.77 / 34.60 | 53.46 / 44.36 | 37.64 / 30.30 | 53.40 / 44.55 |
LongMemEval: six question types and overall accuracy (%):
| Prompt formulation | Decision policy | Temporal ACC ↑ | Multi-session ACC ↑ | Knowledge update ACC ↑ | Single user ACC ↑ | Single assistant ACC ↑ | Single preference ACC ↑ | Overall ACC ↑ |
|---|---|---|---|---|---|---|---|---|
| Original | Joint-2 | 63.33 | 63.33 | 80.95 | 79.37 | 71.33 | 69.14 | 69.56 |
| $AR$ + repair | 63.61 | 60.28 | 79.52 | 80.95 | 70.00 | 64.20 | 68.37 | |
| DREAM $RA$ | 68.06 | 66.67 | 83.33 | 83.07 | 73.33 | 72.84 | 73.04 | |
| Concise | Joint-2 | 63.06 | 63.61 | 80.95 | 79.37 | 70.00 | 70.37 | 69.48 |
| $AR$ + repair | 63.61 | 60.00 | 79.52 | 80.95 | 69.33 | 64.20 | 68.22 | |
| DREAM $RA$ | 68.06 | 66.94 | 83.33 | 83.07 | 72.67 | 71.60 | 72.96 | |
| Semantic paraphrase | Joint-2 | 63.33 | 63.06 | 81.43 | 79.89 | 71.33 | 69.14 | 69.63 |
| $AR$ + repair | 63.61 | 59.72 | 79.05 | 80.95 | 70.00 | 62.96 | 68.07 | |
| DREAM $RA$ | 68.06 | 66.67 | 82.38 | 82.54 | 73.33 | 74.07 | 72.89 |
W2 / Q1: “all 48 corresponding baseline results in Table 3 identical to those in GAM... Were these baselines rerun... what explains the differences in the two reported GAM results?”
The baseline entries in Table 3 were taken from Table 2 of MemPro (EMNLP 2026) for the same benchmarks and settings, while the DREAM row reports our own evaluation. The GAM values of 61.46 and 30.17 follow that reference table, explaining their difference from the 64.56 and 34.77 reported in GAM’s original paper. The table below presents our new evaluation of the baselines under a shared protocol, with matched questions, backbone configurations, context limits, overlength handling, and scoring.
Within each backbone, we used identical questions, source documents, model versions, scoring, context/output limits, and overlength handling. RAG top-5 and top-10 used the same 2,048-token chunks and embeddings; each memory system retained its own construction and retrieval procedure. We applied the same aggregation to task scores across paired runs; HotpotQA Avg is the arithmetic mean of the three length-specific F1 scores under the same setting.
| Backbone | Method | Hotpot 56K | Hotpot 224K | Hotpot 448K | Hotpot Avg | Narrative |
|---|---|---|---|---|---|---|
| GPT-4o-mini | Full Context | 56.21 | 53.74 | 51.68 | 53.88 | 31.08 |
| RAG, top-5 | 52.38 | 50.91 | 49.76 | 51.02 | 24.62 | |
| RAG, top-10 | 53.17 | 51.66 | 50.48 | 51.77 | 25.34 | |
| A-Mem | 34.72 | 31.48 | 29.91 | 32.04 | 27.32 | |
| Mem0 | 33.26 | 32.18 | 28.07 | 31.17 | 28.74 | |
| MemoryOS | 27.15 | 24.36 | 23.81 | 25.11 | 26.41 | |
| LightMem | 41.58 | 35.92 | 31.27 | 36.26 | 18.46 | |
| GAM | 63.48 | 62.37 | 59.26 | 61.70 | 36.92 | |
| DREAM | 66.14 | 65.32 | 64.07 | 65.18 | 40.24 | |
| Qwen2.5-14B-Instruct | Full Context | 48.96 | 45.18 | 41.87 | 45.34 | 28.83 |
| RAG, top-5 | 50.64 | 46.25 | 45.72 | 47.54 | 18.76 | |
| RAG, top-10 | 51.29 | 47.68 | 46.94 | 48.64 | 20.13 | |
| A-Mem | 28.36 | 25.11 | 23.64 | 25.70 | 25.46 | |
| Mem0 | 30.83 | 31.67 | 27.24 | 29.91 | 27.31 | |
| MemoryOS | 25.42 | 29.36 | 24.18 | 26.32 | 23.87 | |
| LightMem | 36.84 | 28.53 | 27.91 | 31.09 | 17.62 | |
| GAM | 63.78 | 56.46 | 57.12 | 59.12 | 34.52 | |
| DREAM | 66.82 | 60.71 | 61.35 | 62.96 | 34.06 |
Q2: “Does the advantage of the repulsive-before-attractive strategy generalize to other LLM backbones... and more realistic settings with adaptive memory retrieval?”
The submitted LoCoMo results are 51.71, 50.22, and 53.56 F1 with GPT-4o-mini, Qwen3-8B, and Qwen3-30B. On the first two backbones, removing the repulsive gate costs 5.62 and 6.07 points; removing answer-facing constraints costs 4.55 and 4.91 points. We evaluated the ordering across backbones on LoCoMo and LongMemEval, retaining Joint-2's reassessment and $AR$'s support repair.
- Fixed evidence: identical candidates, ordering, and source excerpts; four rounds, two role-assignment calls per round, and a shared evidence budget.
- Adaptive retrieval: identical memory banks, initial candidates, tools, source permissions, and round/evidence limits; each policy's evolving state guided subsequent retrieval and readback.
Each backbone performed writing, decisions, and answering under fixed reasoning and generation settings. We included retrieval, readback, evidence transformation, generation, and reported reasoning tokens in online costs, and specified paired confidence intervals for assessing the ordering effect within each backbone and retrieval mode.
Overall Reply
We sincerely thank you for your careful reading and constructive assessment. We appreciate your recognition of the problem's importance, our evaluation across benchmarks and models, and the component ablations.
Below, we clarify DREAM's complete mechanism and address methodological novelty, baseline provenance and comparability, and generalization across backbones and adaptive retrieval, supported by controlled ordering comparisons, prompt-variation experiments, and evaluations under fixed and adaptive evidence acquisition.
W1: “both gates are implemented through prompt-based decisions using a frozen LLM... closer to a heuristic prompting strategy than a fundamentally new or generalizable method.”
DREAM's contribution is the complete mechanism connecting source-linked event memory, constraint-first support selection, and answer generation. Event identity, attributes, and state determine where a recalled claim applies. The repulsive gate forms explicit constraints; the attractive gate selects support conditioned on them. Both enter the answer-facing evidence state, and source readback can revise the constraints and dependent support. For query q and candidates Cq, using event identities to denote the selected sets within one gate pass:
\mathcal{E}_q^+ = A\bigl(q,\mathcal{C}_q\setminus\mathcal{E}_q^-\mid\mathcal{E}_q^-\bigr).$$ This dependency gives the same memory different evidential roles across queries: a cancelled meeting supports a question about the original proposal but constrains an answer about the current schedule. STALE/CUPMem provides state adjudication, premise verification, and constrained readout, while Zep provides temporal structured memory and retrieval. DREAM **makes support selection and its revision explicitly conditional** on previously established constraints. Appendix C.1 tests this mechanism with the backbone, candidates, source excerpts, four rounds, two role-assignment calls per round, and 4,096-token evidence budget fixed. Joint-2 can reassess all assignments, and \$A$$\to$$R\$ + repair can replace support from the full candidate pool. DREAM achieves **53.50 F1**, versus **51.77** and **51.46**, with paired gains of **+1.73 [0.83, 2.57]** and **+2.04 [1.15, 2.88]** (95% conversation-clustered bootstrap intervals). Online tokens are comparable: **45.04k, 45.20k, and 44.86k**. Retaining both gates but removing answer-facing constraints also reduces LoCoMo F1 from **53.56 to 48.87** and LongMemEval accuracy from **74.37 to 66.22**. These results support both the ordering and the use of constraints during generation. **Previously reported fixed-evidence results (Appendix C.1, Qwen3-30B):** |**Policy** | **Single F1** | **Multi F1** | **Temporal F1** | **Open F1** | **Overall F1** | **Risk F1** | **Control F1** | **Historical F1** | |---|---|---|---|---|---|---|---|---| | Joint-2 | 56.67 | 43.08 | 50.74 | 37.72 | 51.77 | 45.80 | 57.20 | 50.60 | | \$A$$\to$$R\$ + repair | 56.53 | 42.76 | 49.86 | 37.83 | 51.46 | 44.70 | 57.24 | 50.15 | | **DREAM \$R$$\to$$A\$** | 57.80 | 45.90 | 53.60 | 37.70 | 53.50 | 50.60 | 57.16 | 50.82 | We used Graphiti as the Zep implementation and evaluated STALE's State Resolution (SR), Premise Resistance (PR), and Implicit Policy Adaptation (IPA) separately for Type I and Type II; Overall averages all six scores. We retained per-question aggregation for LoCoMo and LongMemEval. **LoCoMo: four question types and overall; cells report F1 / BLEU-1 (%):** |**Method** | **Single F1 / B1 ↑** | **Multi F1 / B1 ↑** | **Temporal F1 / B1 ↑** | **Open F1 / B1 ↑** | **Overall F1 / B1 ↑** | |---|---|---|---|---|---| | Graphiti (Zep) | 54.75 / 46.68 | 42.27 / 32.47 | 49.58 / 41.90 | 33.41 / 27.22 | 50.06 / 41.87 | | CUPMem | 56.08 / 48.50 | 43.87 / 33.69 | 50.98 / 42.56 | 34.34 / 27.36 | 51.43 / 43.24 | | DREAM without answer-facing constraints | 54.41 / 46.27 | 41.09 / 30.59 | 46.27 / 37.81 | 31.84 / 24.96 | 48.87 / 40.31 | | **DREAM** | 58.12 / 50.04 | 45.36 / 34.28 | 53.47 / 44.62 | 37.91 / 30.88 | 53.56 / 44.83 | **LongMemEval: six question types and overall accuracy (%):** |**Method** | **Temporal ACC ↑** | **Multi-session ACC ↑** | **Knowledge update ACC ↑** | **Single user ACC ↑** | **Single assistant ACC ↑** | **Single preference ACC ↑** | **Overall ACC ↑** | |---|---|---|---|---|---|---|---| | Graphiti (Zep) | 62.22 | 60.83 | 77.14 | 78.84 | 70.67 | 62.96 | 67.48 | | CUPMem | 66.11 | 62.22 | 80.95 | 82.54 | 72.00 | 64.20 | 70.22 | | DREAM without answer-facing constraints | 69.72 | 56.67 | 72.38 | 85.19 | 52.67 | 58.02 | 66.22 | | **DREAM** | 76.67 | 61.11 | 86.19 | 91.01 | 65.33 | 70.37 | 74.37 |