- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents Memento, a continual learning framework that improves LLM agents through persistent memory without fine-tuning. Instead of relying on embedding similarity, Memento learns a retrieval policy to select useful past experiences. The framework is formalized as a Stateful Reflective Decision Process (SRDP), with theoretical guarantees based on KL-regularized soft policy iteration. Memento achieves performance comparable to an RL-trained baseline on DeepResearcher and outperforms existing memory-based agents across five multimodal benchmarks.
Critical strengths
- Effective learning without LLM fine-tuning: Memento offers a practical way to improve LLM agents by learning which past experiences to retrieve. Experiments show competitive performance against RL-trained agents, supported by ablations demonstrating the benefits of memory and learned retrieval.
Critical weaknesses
- Limited evidence of continual learning: It remains unclear whether Memento demonstrates continual learning beyond memory accumulation and retrieval policy updates. In Figure 3(a), even the no-CBR baseline improves over iterations, making it difficult to isolate Memento's contribution. The experiments also provide limited evidence of adaptation to changing tasks or retention of prior capabilities.
- Inconsistencies in the reported experimental results: The F1 scores in Figure 3(b) appear inconsistent with those in Tables 1 and 3, despite seemingly identical OOD evaluation settings. Additionally, the scores in Appendix Table 7 are substantially higher than those in Table 1. These discrepancies make it difficult to reliably assess the reported improvements.
Questions for authors
- How does the five-pass experiment differ from repeated training on a fixed dataset? Can Memento adapt to changing task distributions while retaining previously learned capabilities?
- Why do the F1 scores differ across Figure 3(b), Tables 1 and 3, and Appendix Table 7? Are different evaluation protocols, model settings, or learning iterations involved?
5 comments
Dear reviewer,
As the discussion period ends on 18th Nov, could you please let us know if our response and revisions above have resolved your concerns?
Best regards, Authors
Dear Reviewer,
As the discussion period closes on November 18 (23:59 AoE), we would appreciate your feedback on our rebuttal and additional experimental results. Could you please let us know whether our responses address your concerns, and which issues, if any, remain unresolved? We would welcome the opportunity to clarify them before the discussion ends.
Thank you again for your time and consideration.
Best regards, The Authors
W1 & Q1. Continual learning in the LLM-agent setting
We would like to clarify that continual learning in agentic memory is conventionally measured in two ways: repeated passes over a fixed task set with memory accumulating between them, reported as a per-pass learning curve, and evaluation on tasks the memory was never populated for. Recent training-free self-evolving methods use exactly this pairing — repeated-episode learning curves plus a disjoint-task retrieval condition — as do the memory-based agents we already compare against. Our Fig. 3a is the first of these and Fig. 3b the second.
Figure 3(a) reports Pass@N: the proportion of tasks solved within N attempts. Consequently, the no-CBR baseline can improve as N increases simply because it receives more opportunities to succeed; this does not imply that the baseline is learning. We include no-CBR precisely to account for this repeated-attempt effect. At each matched attempt budget, N = 1, 2, 3, 4 and 5, Memento achieves higher Pass@N than no-CBR. The relevant evidence is therefore its advantage at the same N, rather than the upward trend alone. Figure 3(b) complements this comparison by evaluating transfer from a case bank populated on NQ, TQ, HotpotQA and 2Wiki to MusiQue, Bamboogle and PopQA. Together, these results support the benefits of accumulated experience and its reuse beyond the tasks used to construct memory. We have clarifed the Pass@N metric in the figure caption and experimental description.
We agree that studying adaptation and retention under sequentially changing task distributions is also important. A systematic evaluation of non-stationary task streams and forgetting of earlier capabilities lies beyond the scope of this paper. Our present evaluation focuses on memory-based adaptation and transfer under the two protocols above.
W2 & Q2. Clarification of experimental settings
We apologise for not making the experimental settings sufficiently clear. Tables 1 and 3 use Qwen2.5-7B for both the planner and executor, whereas Figure 3(b) and Appendix Table 7 use a GPT-4.1 planner and an o4-mini executor; their absolute scores therefore come from different model configurations. We have revised the experimental description and the relevant captions to state these model configurations and aggregation rules explicitly.