acceptodds
Under review as a conference paper at ICLR 2027

Remember Before You're Asked: MemDream for Self-Probing Memory Evolution

Abstract

Memory is essential for enabling LLM-based agents to maintain coherent, personalized behavior over long-horizon interactions. However, existing memory systems share a fundamental limitation: they never proactively test their own memory, repairing it only after real queries expose weaknesses. This reactive paradigm means every retrieval failure corresponds to a real interaction in which the cost has already been paid. We propose MemDream, a framework that enables self-probing memory evolution for LLM agents. Our framework periodically enters offline dream cycles where three specialized agents (Dreamer, Analyst, Consolidator) collaboratively probe, diagnose, and repair the memory graph before failures occur. A policy trained via Group Relative Policy Optimization learns which repair operations produce durable retrieval improvements, while a soft decay mechanism provides reversible forgetting driven by the same anticipatory signal. Experiments on LoCoMo and MemoryAgentBench demonstrate that MemDream improves answer F1 by 4.5 points on LoCoMo and achieves a 9.1-point higher overall score on MAB over the strongest reactive-evolution baselines.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Agent Memory, Memory Probing, Memory Consolidation, Multi-Agent Systems, Long-Horizon Dialogue

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_D6i3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MemDream, a framework for self-probing memory evolution that aims to automatically improve an LLM-based agent's memory by testing it, identifying weaknesses, and repairing them in the background before real queries fail. MemDream runs offline dream cycles with three agents: (1) a Dreamer that probes the memory graph with forward-looking queries, (2) an Analyst that diagnoses retrieval failures, and (3) a Consolidator that repairs the memory graph. The agents share a Llama-3.1-8B backbone with separate LoRA adapters, which are first trained with supervised fine-tuning on GPT-4.1 outputs, then individually optimized with Group Relative Policy Optimization (GRPO). Subsequently, Dreamer and Consolidator are jointly fine-tuned with GRPO while Analyst remains frozen. The framework also incorporates a soft-decay mechanism that provides reversible forgetting by reducing the retrieval priority of less relevant memory nodes rather than deleting them outright. MemDream is evaluated on LoCoMo and MemoryAgentBench (MAB) using GPT-4.1 and Llama-3.1-8B as reader models.

Critical strengths

The paper is well motivated, and its central idea is interesting. Following the key insight that a memory system should test itself, MemDream uses self-generated probing queries to find memory that is stored but cannot be retrieved, and repairs it before a real query fails. These probes are well grounded (Appendix H). Each expected answer is copied verbatim from the stored memory, so repairs cannot chase hallucinated targets, and the probes are shown not to overlap with the evaluation questions. The soft decay mechanism is simple, and the ablation confirms its importance.

Critical weaknesses

The paper lacks an experimental comparison with the closest methods. Auto-Dreamer, TrustMem, and RecMem are discussed only qualitatively (Appendix M). Could the authors provide a direct comparison under matched settings?

The ablation in Table 6 suggests that the gains may come mostly from the retrieval pipeline rather than the dream cycle. The "w/o Dreamer" variant performs no dream-guided maintenance, yet it still outperforms every baseline (52.09 vs. 51.16 average F1 for FluxMem). The dream cycle adds only about 3 points on top of this. The paper presents the dream cycle as the main source of improvement, but the main comparison does not isolate its contribution.

The rule-based probes outperform the LLM Dreamer in discriminative power and alignment (Figure 3d and Table 10), which weakens the contribution of the trained Dreamer. What empirical evidence justifies selecting the LLM Dreamer as the primary query generator?

Questions for authors
  1. The paper states that queries arriving during a dream cycle are answered with the existing memory and recent context. However, such queries may still suffer from the retrieval failures the cycle is meant to fix. Could the authors discuss this limitation and clarify whether any mechanism ensures response reliability during an ongoing dream cycle?

  2. The paper states that edges connect nodes sharing entities, but the entity extraction procedure is not specified. Since missing edges are precisely what the dream cycle is designed to detect and repair, this step is central to the method. Could the authors specify how entities are identified and matched across nodes?

  3. The values of λ1\lambda_1, λ2\lambda_2, λ3\lambda_3 in Eq. 3, and the selection threshold are not reported. Could the authors provide these values and explain how they were chosen? The construction of the local probe query set Q(v⋆)Q(v^\star) is unclear. Bridge partners are selected by co-retrieval frequency across these queries, yet the Dreamer generates its probe only after the pair is selected. How is Q(v⋆)Q(v^\star) constructed, including its size and initialization, and how does the method handle cases with few or no local queries?

  4. The authors state that pruning never loses information because decayed nodes remain accessible. However, a node with a very low weight may become effectively unretrievable, especially if it is pruned again in later cycles (e.g., 0.2, then 0.04). Since weight restoration in Eq. 6 depends on the success count, a node that is never retrieved may never be restored. Can an already pruned node be pruned again, and how can such nodes be recovered when they become relevant?

  5. Per-agent reward weights are said to be reported in the appendix, but they are missing. Fixing weights in advance reduces benchmark-specific tuning, but it does not guarantee that the reward design is unbiased or effective. Could the authors provide these values and explain how they were chosen?

  6. The paper does not explain why the Analyst is frozen during joint fine-tuning. Could the authors justify this choice and compare it with jointly fine-tuning all three agents or keeping all agents independently trained?

Minor comment: The abstract, introduction, and contributions describe GRPO as training a single repair policy, whereas Section 3.3 trains all three agents with GRPO. These sections should be revised to describe the training procedure accurately.

Reviewer_Xd79OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MemDream studies proactive maintenance for long-term agent memory. Existing systems usually discover fragmentation, redundancy, or staleness only after a real query fails. MemDream instead runs an offline dream cycle: the Dreamer generates probing queries for weak regions based on node isolation, redundancy, and staleness; the Analyst diagnoses retrieval failures as buried, scattered, redundant, or noisy; and the Consolidator applies extract, synthesize, merge, or prune operations. The system learns probing and repair policies through SFT and GRPO and replaces permanent deletion with soft decay whose node weights can later recover. MemDream generally outperforms multiple memory systems on LoCoMo and MemoryAgentBench, and removing soft decay reduces average F1 from 55.14 to 41.04, making reversible forgetting the strongest mechanism-level contribution. The paper also provides bootstrap confidence intervals, cost profiling, groundedness checks, and a 20-cycle stability analysis.

Critical strengths
  1. MemDream moves memory maintenance from repair after observed query failures to proactive probing between queries. The Dreamer identifies weak regions, the Analyst assigns failure types, and the Consolidator applies targeted repairs, giving each stage a defined role in the maintenance loop.
  2. The system implements selective forgetting by reducing retrieval weights rather than permanently deleting nodes, while retaining the option of later recovery. Replacing decay with hard deletion lowers average F1 from 55.14 to 41.04, a 14.10-point decrease and the strongest evidence for an individual design choice.
Critical weaknesses
  1. The paper includes w/o Dreamer, w/o Analyst, w/o Consolidator GRPO, and w/o Decay ablations, but lacks a minimal sufficient system such as rule probe + single repair agent + soft decay and does not directly compare with Auto-Dreamer under a shared reader, memory store, and compute budget. The appendix mainly contrasts mechanisms and therefore does not isolate the net value of the full three-agent and multi-stage GRPO design.
  2. The backbone study is narrow: the three agents are based on Llama-3.1-8B-Instruct with LoRA, and no newer backbone such as Qwen3.5 is evaluated. The reader models are also limited mainly to GPT-4.1 and Llama-3.1-8B, without recent stronger readers such as DeepSeek-V4-Pro, GPT-6, or Claude-Opus-5.5.
Questions for authors
  1. Under a common protocol, how do rule probe + single repair agent + soft decay, LLM probe + single repair agent, full MemDream, and Auto-Dreamer compare? This comparison would isolate the contributions of the LLM Dreamer, the Analyst, three-agent specialization, and multi-stage GRPO.
  2. Can the authors evaluate the three agents on newer backbones such as Qwen3.5 and report results with recent readers such as DeepSeek-V4-Pro, GPT-6, and Claude-Opus-5.5 to test whether the gains are robust across model families and stronger readers?
2 comments

The authors have adequately addressed my concerns. I will raise my score and recommend acceptance.

Thank you for raising your score and for taking the time to review our revisions and respond. We greatly appreciate your valuable feedback.

Reviewer_zXSVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MemDream performs offline maintenance of external memory for long-horizon language agents. Three LoRA-specialized agents trained with GRPO generate retrieval probes, diagnose failures, and repair memory, including soft down-weighting that retains pruned nodes for possible restoration. With GPT-4.1 as the reader, the manuscript reports a 3.98-point improvement in category-macro F1 over FluxMem on LoCoMo and a 9.1-point improvement in overall score over O-Mem on MemoryAgentBench.

Critical strengths
  1. Proactively finding retrieval blind spots before real user queries is a useful direction for memory maintenance. The probing, diagnosis, and repair roles are clearly separated, and Table 1 makes the diagnostic categories and repair operations easy to inspect.

  2. The paper reports aggregate improvements on both benchmarks with GPT-4.1 and Llama-3.1-8B readers. The component ablations and repeated-maintenance analysis in Appendix G provide useful evidence about the tested framework and memory preservation.

Critical weaknesses
  1. The incremental benefits of the main maintenance choices are not fully isolated. Removing the Dreamer also removes dream-guided maintenance, while Table 10 gives rule-based probes higher point estimates for discrimination (0.36 vs. 0.15) and alignment (0.37 vs. 0.35), without a corresponding final-QA comparison. Similarly, the decay study compares soft down-weighting with hard deletion but does not include unchanged weights (gamma = 1), and the specified restoration rule is not directly tested after repeated demotion and a later relevance change.

  2. Uncertainty in the strongest baseline comparisons remains unresolved. Appendix F explicitly notes the absence of paired FluxMem outputs, so the strongest LoCoMo comparison is a point estimate. On MemoryAgentBench, MH-QA contributes 6.0 of the 9.1 overall points relative to O-Mem. The other tasks still contribute positively on average, but paired outputs and an O-Mem MH-QA configuration/error audit would clarify the strength and source of the aggregate advantage.

Questions for authors
  1. Additional validation: With the same initial memories, retriever, reader, and total maintenance budget, how do learned and rule-based probes compare on final QA? For soft decay, a gamma = 1 control and a relevance-reversal test would help distinguish down-weighting benefits from simply retaining nodes.

  2. Reporting and verification: Can the paired FluxMem comparison identified as missing in Appendix F be completed? Please also provide the O-Mem MH-QA configuration and failure breakdown to clarify the unusually large task-level difference.

Comments on the paper

12 comments
AuthorsOpenReview ↗

A gentle reminder for discussion

Dear Reviewer zXSV,

Thank you for your constructive comments and valuable suggestions.

In our responses, we have (1) clarified the Dreamer configurations and the final-QA contribution of learned probing (52.09 → 55.14 F1); (2) added the unchanged-weight control and explained how retained memories recover through subsequent successful use; (3) completed the paired FluxMem analysis, showing a 3.30-point F1 improvement with a 95% CI of [1.68, 4.73]; and (4) provided the O-Mem execution-error analysis and sensitivity checks showing that the aggregate MAB advantage remains positive when the anomalous MH-QA result is replaced or excluded. The revised manuscript incorporates the Dreamer clarifications, decay control, recovery explanation, and paired statistical analysis.

Reviewer D6i3 raised overlapping concerns about the Dreamer ablation and the rule-guided versus LLM-only configurations, and has since confirmed that our clarifications addressed those concerns. We hope these explanations are also helpful in resolving the corresponding points in your review.

We would appreciate hearing whether these clarifications and additional results address your concerns. We would be happy to answer any further questions.

Best regards,

The Authors of Submission 36346

AuthorsOpenReview ↗

A gentle reminder for discussion

Dear Reviewer Xd79,

Thank you for your constructive comments and suggestions.

(1) We evaluated the minimal system you proposed, combining rule probes, a single repair agent, and soft decay. It achieves 45.20 F1, compared with 55.14 for full MemDream. Our responses also clarify the Dreamer configurations and the contribution of learned probing, and explain why differences in task settings and the lack of an available official implementation make a direct Auto-Dreamer comparison difficult.

(2) To address your concern about model coverage, we added experiments using Qwen3.5-9B for all three agents, with DeepSeek-V4-Pro, Claude-Opus-5.5, and GPT-6 Astra as readers. Dream-cycle maintenance improves F1 by 2.53, 3.81, and 3.67 points, respectively. BLEU-1 and ROUGE-L also improve for every reader. These results and clarifications are included in the revised manuscript.

Reviewer D6i3 raised a similar concern about comparisons with the closest methods. They have confirmed that our direct RecMem comparison under the same reader and protocol, together with our explanations regarding Auto-Dreamer and TrustMem, addressed that concern. We hope these results and explanations also help answer your questions about the comparison.

We would appreciate your feedback on whether our responses address your concerns and would be happy to discuss any remaining questions.

Best regards,

The Authors of Submission 36346

The authors have responded to your reviews. Please read their response and reply. Please specify which concerns are resolved and which remain. Please update your review and score if the assessment has changed.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.