acceptodds
Under review as a conference paper at ICLR 2027

Memento: Continual Learning of LLM Agents via Stateful Reflective Memory

11 views

Abstract

We present a unified method for continual learning of large language model (LLM) agents that fully eliminates the need to fine-tune the underlying model. Existing paradigms are either rigid (handcrafted reflection workflows) or computationally costly (gradient updates of LLM parameters), and neither scales to long-horizon tasks where environments drift faster than models can be retrained. We argue that the right primitive is reflection: revisiting past experience to adjust future action selection. We formalise this primitive as a Stateful Reflective Decision Process (SRDP), in which the agent alternately writes new experiences to an episodic memory and reads relevant cases to guide decisions. By augmenting the state space with the memory, we lift the SRDP to a Reflected Markov Decision Process amenable to classical control analysis. We then adapt KL-regularised policy iteration to this memory setting as Read–Write Reflective Learning, integrating Parzen-window retrieval with closed-form soft updates, and prove (i) convergence under bounded rewards and a fixed memory, (ii) two-time-scale convergence when memory evolves slowly, and (iii) asymptotic optimality as the memory densely covers the state space. We instantiate this method as a planner–executor Deep Research agent, Memento, with both non-parametric and parametric memory variants. Empirically, Memento statistically ties the strongest RL-trained baseline on the seven-dataset DeepResearcher suite without finetuning LLMs, and wins the average on five multimodal benchmarks. Furthermore, in end-to-end system-level comparison, Memento reaches Top-1 on the GAIA validation leaderboard, and surpasses prior agents on SimpleQA, HLE, and UIS-QA.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Continual Learning, LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qRgXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Memento, a continual learning framework that improves LLM agents through persistent memory without fine-tuning. Instead of relying on embedding similarity, Memento learns a retrieval policy to select useful past experiences. The framework is formalized as a Stateful Reflective Decision Process (SRDP), with theoretical guarantees based on KL-regularized soft policy iteration. Memento achieves performance comparable to an RL-trained baseline on DeepResearcher and outperforms existing memory-based agents across five multimodal benchmarks.

Critical strengths
  • Effective learning without LLM fine-tuning: Memento offers a practical way to improve LLM agents by learning which past experiences to retrieve. Experiments show competitive performance against RL-trained agents, supported by ablations demonstrating the benefits of memory and learned retrieval.
Critical weaknesses
  • Limited evidence of continual learning: It remains unclear whether Memento demonstrates continual learning beyond memory accumulation and retrieval policy updates. In Figure 3(a), even the no-CBR baseline improves over iterations, making it difficult to isolate Memento's contribution. The experiments also provide limited evidence of adaptation to changing tasks or retention of prior capabilities.
  • Inconsistencies in the reported experimental results: The F1 scores in Figure 3(b) appear inconsistent with those in Tables 1 and 3, despite seemingly identical OOD evaluation settings. Additionally, the scores in Appendix Table 7 are substantially higher than those in Table 1. These discrepancies make it difficult to reliably assess the reported improvements.
Questions for authors
  • How does the five-pass experiment differ from repeated training on a fixed dataset? Can Memento adapt to changing task distributions while retaining previously learned capabilities?
  • Why do the F1 scores differ across Figure 3(b), Tables 1 and 3, and Appendix Table 7? Are different evaluation protocols, model settings, or learning iterations involved?
5 comments

W1 & Q1. Continual learning in the LLM-agent setting

We would like to clarify that continual learning in agentic memory is conventionally measured in two ways: repeated passes over a fixed task set with memory accumulating between them, reported as a per-pass learning curve, and evaluation on tasks the memory was never populated for. Recent training-free self-evolving methods use exactly this pairing — repeated-episode learning curves plus a disjoint-task retrieval condition — as do the memory-based agents we already compare against. Our Fig. 3a is the first of these and Fig. 3b the second.

Figure 3(a) reports Pass@N: the proportion of tasks solved within N attempts. Consequently, the no-CBR baseline can improve as N increases simply because it receives more opportunities to succeed; this does not imply that the baseline is learning. We include no-CBR precisely to account for this repeated-attempt effect. At each matched attempt budget, N = 1, 2, 3, 4 and 5, Memento achieves higher Pass@N than no-CBR. The relevant evidence is therefore its advantage at the same N, rather than the upward trend alone. Figure 3(b) complements this comparison by evaluating transfer from a case bank populated on NQ, TQ, HotpotQA and 2Wiki to MusiQue, Bamboogle and PopQA. Together, these results support the benefits of accumulated experience and its reuse beyond the tasks used to construct memory. We have clarifed the Pass@N metric in the figure caption and experimental description.

We agree that studying adaptation and retention under sequentially changing task distributions is also important. A systematic evaluation of non-stationary task streams and forgetting of earlier capabilities lies beyond the scope of this paper. Our present evaluation focuses on memory-based adaptation and transfer under the two protocols above.

W2 & Q2. Clarification of experimental settings

We apologise for not making the experimental settings sufficiently clear. Tables 1 and 3 use Qwen2.5-7B for both the planner and executor, whereas Figure 3(b) and Appendix Table 7 use a GPT-4.1 planner and an o4-mini executor; their absolute scores therefore come from different model configurations. We have revised the experimental description and the relevant captions to state these model configurations and aggregation rules explicitly.

Dear reviewer,

As the discussion period ends on 18th Nov, could you please let us know if our response and revisions above have resolved your concerns?

Best regards, Authors

Dear Reviewer,

As the discussion period closes on November 18 (23:59 AoE), we would appreciate your feedback on our rebuttal and additional experimental results. Could you please let us know whether our responses address your concerns, and which issues, if any, remain unresolved? We would welcome the opportunity to clarify them before the discussion ends.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_Nx1NOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Memento, a memory-based approach to continual adaptation of LLM agents without fine-tuning the underlying language models. It formalizes the interaction between retrieval, action generation, and memory updates as a Stateful Reflective Decision Process, and constructs a Reflected MDP by augmenting the state with memory and treating case retrieval as the action. The proposed Read-Write Reflective Learning method combines a Parzen-window retrieval prior with KL-regularized policy updates. The authors present convergence and stability results under fixed or slowly evolving memory, together with an asymptotic-optimality claim under memory-coverage and LLM-imitation assumptions. The framework is instantiated as a planner-executor system with non-parametric and learned retrieval variants; the latter trains a lightweight case-utility predictor from task outcomes. Experiments on text-only and multimodal benchmarks report competitive performance and average gains from case memory and learned retrieval, alongside additional end-to-end system evaluations.

Critical strengths
  1. Practically meaningful direction. Improving agents through persistent experience and learned retrieval while keeping the underlying LLMs frozen is an appealing approach to deployment-time adaptation. The planner-executor design offers a concrete realization of this idea without requiring LLM parameter updates.

  2. Broad empirical coverage. The evaluation includes both text-only and multimodal tasks, comparisons with several memory-augmented methods, and additional end-to-end benchmarks. This provides evidence of the approach's utility beyond a single task setting.

  3. Useful component ablations. Tables 3-4 hold the planner-executor architecture fixed while comparing no case memory, similarity-based retrieval, and learned retrieval. Average text-task F1 increases from 50.4 to 52.4 and then 53.6; average multimodal accuracy increases from 54.0 to 56.6 and then 57.4. Reporting means and standard deviations over three runs for these configurations supports the average incremental benefits of memory and learned retrieval, while helping distinguish those benefits from the contribution of the underlying agent architecture.

Critical weaknesses
  1. The claimed novelty and theoretical significance are not sufficiently established. The practical mechanism is closely related to retrieval-based in-context learning and demonstration selection. The central formalization absorbs a frozen LLM into the environment dynamics and augments the state with memory, both of which are standard MDP modeling operations. Once this construction is adopted, much of the subsequent development appears to be a direct application or restatement of classical regularized MDP and convergence results. Although the authors acknowledge the classical nature of the constituent techniques, the paper does not sufficiently isolate the genuinely new technical contribution beyond their application to memory retrieval.

  2. Comparisons with the closest alternatives are incomplete. Appendix B identifies ReasoningBank and Memp as the closest concurrent works. ReasoningBank is evaluated on five multimodal and four text-only datasets, which is meaningful coverage; however, the comparisons omit the main seven-dataset DeepResearcher setting and the full GAIA, HLE, and UIS-QA system evaluations. Moreover, the text-only comparison in Table 13 includes only non-parametric Memento, leaving the learned-retrieval variant untested against ReasoningBank in that setting. Memp is not evaluated, and its omission is not explained. More systematic, matched comparisons are needed to establish the added value of the full method over its closest alternatives.

  3. The connection between theoretical guarantees and practical results is insufficiently justified. Assumption 10 presupposes that sufficiently nearby cases induce a conditional LLM policy close to an optimal environment policy, a strong property not sufficiently validated for the actual models or retrieval representations. The paper also does not clearly bridge single-case retrieval and policy iteration in the theory with top-K case combinations, single-step classifier training, and optional replanning in the implementation. Task-level improvements, five-pass learning curves, and OOD gains do not directly establish the memory-coverage, local-optimality, or two-time-scale conditions on which the guarantees depend. As a result, the extent to which the theory explains or guarantees the behavior of the evaluated system remains unclear.

Questions for authors
  1. What is the substantive technical novelty beyond standard MDP modeling and retrieval-based in-context learning? Once the frozen LLM is absorbed into the environment dynamics and memory is included in the state, which algorithmic or theoretical steps are not direct applications of existing KL-regularized policy optimization and convergence results? Please identify the specific new technical challenges and results, and clarify what the proposed formulation adds beyond learning to select useful demonstrations for a frozen model.

  2. Can the advantage over the closest alternatives be established more systematically? I acknowledge the ReasoningBank comparisons in Tables 2 and 13. However, why is Memp, identified as one of the closest concurrent works, not evaluated, and why does Table 13 include only non-parametric Memento? Can the authors provide matched comparisons of the learned-retrieval variant with ReasoningBank and Memp in representative text-only or end-to-end settings not covered by the current head-to-head comparisons, or explain the practical obstacles? The comparisons should control models, tools, memory sources, and inference budgets so that the contribution of the proposed method can be isolated.

  3. To what extent do the theoretical guarantees apply to the evaluated system? What evidence supports Assumption 10 for the actual frozen LLMs and retrieval representations, rather than merely assuming that nearby cases induce near-optimal behavior? Please explain how single-case retrieval and policy iteration in the analysis correspond to top-K case combinations, single-step classifier training, and optional replanning in the implementation. Which guarantees remain applicable under these implementation choices, and what measurements beyond task scores and five-pass learning curves support the relevant memory-coverage and two-time-scale conditions?

9 comments

Thank you for the detailed assessment and for recognizing the value of the controlled ablations. We address your concerns about the contribution, the closest alternatives, and the theory–implementation gap separately. We have completed a matched QA comparison including learned Memento, ReasoningBank, and MemP, and added diagnostics beyond task scores.

W1 & Q1. Technical contribution beyond classical ingredients

We agree with you that state augmentation, absorbing a frozen LLM into the transition kernel, KL-regularized improvement, and the underlying convergence tools are classical. We do not claim these operations as new general-purpose RL results. The contribution is their formulation and implementation for outcome-conditioned retrieval over an agent's evolving episodic memory: reading selects prior experience, execution supplies task feedback, and writing changes the candidate experiences and therefore subsequent retrieval decisions.

This formulation separates the similarity prior from the outcome-trained utility score and distinguishes optimisation with a fixed bank from adaptation as the bank changes. These distinctions identify what must be controlled experimentally and what additional assumptions an evolving-memory analysis needs. The specific technical roles and boundaries are summarised below:

ComponentContribution and boundary
Reflected MDP, Theorem 2A construction for the agent's coupled retrieval, execution, and memory writing; the Markovian lift itself uses standard modeling.
Prior-conditioned read rule, Eq. (4)Applies the classical KL-regularized solution to an outcome-trained score over episodic cases, with a semantic prior.
Fixed/evolving-memory analysis, Theorems 5 and 7Instantiates established policy-iteration and stochastic-approximation tools; changing memory creates a changing retrieval problem, whose tracking requires additional conditions.
Coverage argument, Theorem 3Relates memory coverage and a local LLM-imitation assumption to a conditional value-gap statement; it does not prove that the frozen LLM satisfies that assumption.

Tables 3–4 separately test the practical incremental value of learned retrieval under a fixed architecture.

W2 & Q2. Comparisons with ReasoningBank and MemP

Our main baseline comparison is the seven-dataset DeepResearcher setting in Table 1, covering prompt-based methods (CoT, CoT + RAG, Search-o1) and RL-trained agents (Search-R1, R1-Searcher, DeepResearcher) within the Qwen2.5-7B model family. We present this complete comparison together with the supplemental ReasoningBank/MemP comparison below.

Table R2-W2. Main baseline and supplemental memory-method comparisons.

(a) R2-W2a: Seven-dataset DeepResearcher comparison with new MemP and ReasoningBank results. F1 (%) ↑.

MethodNQ (ID)TQ (ID)HotpotQA (ID)2Wiki (ID)MusiQue (OOD)Bamboogle (OOD)PopQA (OOD)Avg. (7 datasets)
CoT19.845.624.426.48.522.117.023.4
CoT + RAG42.068.937.124.410.025.446.936.4
Search-o132.458.933.030.914.746.638.336.4
Search-R133.144.745.743.426.545.043.040.2
R1-Searcher35.473.144.859.422.864.842.749.0
ReasoningBank36.668.848.351.128.467.348.849.9
MemP35.967.744.249.732.274.546.750.1
DeepResearcher39.678.452.859.727.171.048.553.9
Memento39.2 ± 0.774.3 ± 0.852.3 ± 0.959.2 ± 1.029.6 ± 1.171.6 ± 1.848.7 ± 1.053.6 ± 0.6

In our controlled QA evaluation, all methods use the same frozen Qwen2.5-7B-Instruct planner/executor models, SimCSE encoder, and Wiki18/BM25 tools, with shared bootstrap trajectories and adaptation questions as memory sources. Each method constructs its own memory, which is frozen during evaluation. Inference budgets are matched: at most four retrieved entries, 2,048 memory tokens, three subtasks, two searches per subtask, and three passages per search.

Memento has the lowest mean evaluation token use in the three-domain QA evaluation: 200 questions each from HotpotQA, 2Wiki, and MusiQue over three seeds (1,800 evaluation episodes per method).

Table R2-Q2. Mean evaluation LLM tokens per question.

MethodMean evaluation LLM tokens/question ↓
ReasoningBank20,667.8
MemP18,034.4
Memento11,970.0

Tokens include prompt and completion tokens across all task calls and exclude memory construction.

W3 & Q3. Scope of the theoretical guarantees

We would like to clarify that Memento is an engineering instantiation of the theoretical framework. Formally analysing how its implementation choices affect the theoretical guarantees is beyond the scope of this paper.

The theoretical analysis and the empirical evaluation address complementary questions. The analysis specifies conditions under which improving retrieval can improve the memory-conditioned policy; the ablations test whether this mechanism is useful with the evaluated models and tools.

The guarantees have different scopes. Theorem 5 concerns the specified soft policy evaluation and improvement updates with fixed memory, bounded rewards, and discounting. Theorem 7 addresses evolving memory under the additional two-time-scale, boundedness, and noise conditions in Assumption 6. Theorem 3 relates the value gap to memory coverage and Assumption 10, which requires the frozen LLM's case-conditioned action distribution to approach an optimal policy as case distance decreases. This is an explicit transfer assumption, rather than a property guaranteed by semantic similarity.

The implementation retains the similarity prior and outcome-trained score in the read rule of Eq. (4). In the single-retrieval QA implementation, cross-entropy provides an outcome-prediction objective, but the selected cases share a task-level label and their individual contributions are not separately identified. Deterministic top-K selection and the full system's optional replanning also differ from the updates analyzed in the theorems. We therefore treat the implementation as an approximate instantiation of the framework.

Tables 3–4 provide controlled evidence for the practical mechanism by comparing no memory, similarity retrieval, and learned retrieval under fixed models and tools. They support the usefulness of outcome-based selection beyond memory alone. They do not directly measure worst-case memory coverage or establish the uniform local-imitation and asymptotic time-scale conditions.

Dear reviewer,

As the discussion period ends on 18th Nov, could you please let us know if our response and revisions above have resolved your concerns?

Best regards, Authors

Reviewer_QJMoOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This work addresses continual learning for LLM agents without fine-tuning underlying model weights. The authors formalize agent reflection as a Stateful Reflective Decision Process (SRDP), where the agent writes trajectory experiences to an episodic memory and reads relevant historical cases during decision-making. By state augmentation, SRDP is lifted to a Reflected Markov Decision Process (Reflected MDP). To optimize case selection, the paper proposes Read–Write Reflective Learning, pairing a Parzen-window retrieval prior with KL-regularized soft policy iteration. Theoretical proofs are provided for geometric convergence under fixed memory, two-time-scale convergence under evolving memory, and asymptotic optimality as memory density increases.

Critical strengths
  1. Elevating empirical case retrieval to a Reflected MDP provides a principled mathematical framework for analyzing convergence and asymptotic optimality in memory-augmented LLM agents.
  2. The framework achieves state-of-the-art results across challenging benchmarks without requiring computationally expensive gradient updates or model re-training.
  3. Decoupling high-level task decomposition from tool-based subtask execution effectively prevents context clutter and improves modularity in long-horizon research workflows.
Critical weaknesses
  1. While the theoretical framework assumes multi-step Reflected MDP Bellman updates, the actual implementation reduces to a single-step binary classification loss (LCE\mathcal{L}_{\text{CE}}) over a 2-layer MLP Q-scorer using frozen SimCSE embeddings, over-packaging what is essentially a case re-ranking heuristic.
  2. Continual learning is implemented merely as an append-only store of raw trajectory cases. Lacking active memory pruning, abstraction, or consolidation, context token overhead grows rapidly alongside retrieval noise.
  3. System-level gains on GAIA heavily stem from an aggressive engineering stack rather than the intrinsic superiority of the learned retrieval strategy alone.
  4. In Appendix K, replacing the fast planner with stronger reasoning backbones (e.g., o3) degrades performance, revealing architectural fragility and prompt-coupling when models bypass executor tools.
  5. The Parzen-window prior depends on fixed SimCSE embeddings that measure surface semantic similarity rather than task logic equivalence, risking retrieval of misleading historical context in complex domains.
Questions for authors
  1. How do the authors justify the theoretical optimality claims when substituting the multi-step Reflected MDP Bellman update with a single-step binary classification loss LCE\mathcal{L}_{\text{CE}} in practice?
  2. What automated safeguards prevent persistent Case Memory from being polluted by noisy or sub-optimal trajectories during open-ended deployment?
  3. Have the authors considered integrating memory consolidation or abstraction (e.g., rule extraction) to curb the exponential growth of context token usage in long-horizon tasks?
  4. How sensitive is the Planner–Executor loop to prompt drift across different commercial or open-source LLM backbones?
4 comments

Thank you for your thoughtful review and constructive questions about the theoretical framework, memory mechanism and experimental evaluation.

W1 & Q1

We would like to clarify that Memento is one instantiation of the theoretical framework, not the framework. The SRDP and the Reflected MDP are stated for a general horizon, and T = 1 in Memento is a property of the Deep Research deployment rather than a simplification we adopted for convenience. The planner is invoked at the top level of a task and the only reward available is terminal binary correctness, so the bootstrapped target has nothing to bootstrap from and the soft Bellman recursion collapses to a Monte-Carlo update on the observed reward, which is what Eq. (6) implements.

W2 & Q2 & Q3

Regarding context overhead, we would like to clarify that increasing the memory bank does not require increasing the retrieved-memory context: only the top-K retrieved cases are included in the LLM context.

For the noise from trajectories, only trajectories marked successful by task-outcome feedback are retained as cases in persistent memory. This provides a filter at the write stage, preventing unsuccessful trajectories from accumulating as future planning examples. Since a case that succeeded on one task may still be unsuitable for another, the trained retrieval scorer additionally assesses its utility for the current query. Success filtering and outcome-informed retrieval therefore address complementary sources of memory contamination. In open-ended deployment, the write filter depends on the reliability of the feedback used to determine task success.

W3

We would like to clarify that the GAIA leaderboard comparison assesses end-to-end system performance, including the planner–executor architecture and tools. The contributions of individual components are examined through the comprehensive ablation studies on the DeepResearcher benchmark (Table 3) and the multimodal benchmarks (Table 4). In particular, the memory ablations compare no case memory, similarity-based retrieval and learned retrieval whilst keeping the backbone models, tools and planner–executor configuration fixed. Learned retrieval further improves average text F1 from 52.4 to 53.6 and average multimodal accuracy from 56.6% to 57.4%. These controlled comparisons isolate the incremental contribution of learning the retrieval policy from the rest of the system.

W4 & Q4

Thank you for this insightful observation. We hypothesise that, with the executor fixed, stronger reasoning does not necessarily translate into a higher task success rate: a plan must also delegate executable subtasks in a form that the executor can follow. The observed degradation therefore motivates examining the compatibility between planning and execution, beyond the planner's standalone reasoning ability.

This is why execution feedback is important for case-based reasoning: it evaluates plans through their actual task outcomes. Memento retains successful trajectories as cases and uses execution feedback to train the retrieval scorer, so that the planner can draw on experience that is useful for the current query and executor. This provides a mechanism for adapting case selection to execution requirements.

We have evaluated Memento with diverse commercial and open-source backbones, including GPT-4.1, o3, Qwen3-32B, o4-mini, Qwen2.5-VL-7B, and Qwen2.5-7B, across the configurations reported in Tables 1–4, 8, and 12–13. These evaluations cover different model families, sizes, and reasoning modes within the Planner–Executor framework.

The reported memory ablations show consistent improvements in aggregate performance across backbone families: average F1 increases from 50.4 to 53.6 with Qwen2.5-7B (Table 3), average multimodal accuracy from 54.0 to 57.4 with Qwen2.5-VL-7B (Table 4), and UIS-QA accuracy from 25.5% to 36.4% with the GPT-4.1/o3 combination using non-parametric memory (Table 12). Table 13 further reports an average accuracy increase from 41.2% to 46.0% with Qwen2.5-VL-7B. Together, these results provide cross-backbone evidence for the effectiveness of our experience-based memory mechanism.

W5

We would like to clarify that SimCSE provides a strong semantic prior, but semantic similarity alone does not establish task logic equivalence. This is precisely why we further train the retrieval scorer using task outcomes. Whilst the SimCSE representations remain fixed, the scorer learns the utility of each case for a given query and reweights candidates according to outcome feedback. The objective is to prioritise planning experience that is useful for solving the current task, rather than simply retrieving the most similar wording. The learned-versus-similarity retrieval ablations in Tables 3–4 evaluate the contribution of this additional training.

Dear reviewer,

As the discussion period ends on 18th Nov, could you please let us know if our response and revisions above have resolved your concerns?

Best regards, Authors

Thank you for your detailed response and clarifications.

Your explanations have addressed my questions, and after considering the rebuttal, I will maintain my original score.

Thank you very much for your follow-up and for confirming that our responses have addressed your concerns. We are pleased that the clarifications were helpful and will incorporate the corresponding revisions into the manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We thank the reviewers and the Area Chairs for their careful reading and constructive discussion. During the rebuttal, we addressed the main technical and experimental questions and added a matched comparison with ReasoningBank and MemP. Under the same frozen Qwen2.5-7B models, tools, memory sources, and inference budgets, learned Memento achieves 53.6 average F1 on the seven DeepResearcher datasets, compared with 49.9 for ReasoningBank and 50.1 for MemP. Memento also uses fewer evaluation tokens per question, with 11,970 tokens compared with 20,667.8 for ReasoningBank and 18,034.4 for MemP.

We further clarified the Pass@N metric, the model configurations used across experiments, the memory-writing filter, and the scope of the theoretical assumptions. These clarifications addressed concerns about result interpretation, experimental consistency, and comparisons with closely related methods. Reviewer QJMo explicitly confirmed that our responses addressed the questions raised in the review. Reviewer qRgX also confirmed that the concerns regarding Pass@N, model configurations, and reported-result consistency had been resolved.

The remaining discussion with Reviewer qRgX concerns the scope of the term agentic continual learning. The reviewer expects this concept to include adaptation to non-stationary task distributions and explicit backward-retention or forgetting evaluations. Our paper follows recent training-free agent literature and uses a broader definition based on continual improvement through accumulated interaction experience, persistent memory, and transfer to later tasks. We acknowledge that evaluation under changing task distributions and explicit retention tests is important, and will state this scope and limitation more clearly in the revision.

Reviewer Nx1N’s latest comment indicates that the new comparisons with ReasoningBank and MemP substantially alleviate the empirical comparison concern. The remaining issue concerns the technical distinctiveness of the theoretical framework and the Parzen–KL-based selection mechanism beyond generic outcome-trained retrieval. We have clarified that state augmentation, absorbing a frozen LLM into the transition dynamics, and KL-regularised optimisation are classical ingredients. Our contribution is their integration for outcome-conditioned retrieval over an evolving episodic memory, together with separate analyses for fixed and evolving memory. We also clarify that the current top-K and binary-classification implementation is an approximate instantiation of the general framework, and that the theoretical guarantees are conditional on the stated assumptions rather than direct guarantees for every implementation detail.

Overall, the concrete technical, reporting, and comparison concerns have been resolved or substantially alleviated. The remaining discussion is limited to one theoretical-novelty clarification and one principled difference in the scope of agentic continual learning. We will incorporate the matched comparisons, theoretical boundaries, and evaluation limitations into the revised manuscript. We respectfully hope that the additional evidence and the more precise statement of our contribution will be considered in the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.