acceptodds
Under review as a conference paper at ICLR 2027

Toward Rigorous Evaluation of Continual Learning in Language Agents

Abstract

Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future episodes. Continual learning expects an agent to accumulate experience across a stream of tasks, improve over time, and avoid interference from irrelevant experiences. Unfortunately, existing benchmarks struggle to evaluate this problem rigorously. Most efforts focus on retrieval and reasoning over long-context conversations or documents, while recent lifelong-adaptation benchmarks often stream existing datasets whose cross-task dependencies are unknown, making it difficult to understand what an agent learns and reuses over time. This paper presents an evaluation framework AgentCL for continual learning in agents, centered on distinguishable task streams and metrics for CL properties. AgentCL distinguishes three cross-task relations by the availability of reusable knowledge. In dependent streams, later tasks can reuse knowledge from earlier ones. In conventional streams, whether such reuse exists is unknown. Besides, independent tasks are held out with little transferable knowledge. We use the benchmark to evaluate non-parametric memory designs for continual learning. To diagnose how memory design choices affect continual learning, we develop MemProbe, a probing method that stores interactions, insights, and skills, while filtering unreliable experiences during consolidation. Empirical analysis across coding, deep research, and language understanding/reasoning tasks shows that conventional streams offer limited ability to distinguish memory designs, whereas dependent streams more clearly distinguish their plasticity. Meanwhile, conventional streams and independent tasks often yield limited gains and can expose memory-induced degradation. These results highlight the need for stronger memory designs that balance plasticity and stable reuse.

Primary area:
datasets and benchmarks
Keywords:
continual learning, language agent, memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_616GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AgentCL evaluates continual learning in language agents by distinguishing dependent task streams, conventional streams with unknown reuse relations, and independent held-out tasks. A two-pass protocol measures plasticity, stability, and generalization gains. The paper also introduces MemProbe, which stores interactions, insights, and skills with quality filtering. Experiments across coding, information seeking, and reasoning environments show greater separation among memory methods on dependent streams, alongside limited transfer and occasional memory interference elsewhere.

Critical strengths
  1. Useful evaluation perspective. Explicitly distinguishing task streams by the availability of reusable experience helps clarify when memory should benefit an agent.

  2. Thorough empirical analysis. The paper goes beyond aggregate accuracy with component ablations, retrieval diagnostics, and analyses of retrieval depth, source–target distance, stream position, and different backbones. These analyses help explain when memory supports reuse and when it introduces interference.

Critical weaknesses
  1. Limited conceptual novelty. The framework mainly combines restructuring existing datasets, performance-difference metrics, and established memory extraction and retrieval techniques. The paper does not sufficiently establish a substantive advance in our understanding of continual learning.

  2. Task-bound skill retrieval. MemProbe retrieves historical task descriptions and loads their associated skills, rather than retrieving skills independently by capability or applicability. This design can miss useful skills from superficially different tasks and leaves the distinction between procedural knowledge and episodic memory underdeveloped.

  3. Case reuse is not sufficiently separated from general skill transfer. The dependent streams deliberately place related subtasks before their corresponding complex tasks. Although this creates controlled reuse opportunities, the gains may primarily reflect adapting closely related solutions. Stronger controls are needed to establish transfer beyond these constructed relationships.

  4. Limited coverage of realistic long-term memory demands. The tasks do not adequately represent evolving user preferences, persistent project state, conflicting information, or cross-context workflows. These are consequential challenges for personal agents, and their absence limits the practical scope of the conclusions.

  5. Missing comparisons with native-memory agent systems. Additional backbone experiments do not establish how the evaluated methods compare with complete agents such as Claude Code using their native memory mechanisms. Such comparisons would strengthen the connection between the benchmark findings and deployed systems.

Questions for authors
  1. Clarification: What evidence distinguishes abstract skill transfer from retrieving and adapting a closely related subtask solution?

  2. Evidence request: How does task-level retrieval compare with independently indexing skills by their capabilities and applicability conditions, particularly when source and target task descriptions differ substantially?

  3. Scope clarification: Which conclusions should extend to persistent personal-agent workflows? Comparisons with native-memory agent systems would be a valuable extension, although I recognize that these require additional evaluation.

5 comments
AuthorsOpenReview ↗

To Reviewer 616G Part 1

Dear Reviewer 616G,

Thank you for your valuable comments!

Q1: The contribution to the understanding of continual learning?

We respectfully clarify that our contribution is not limited to restructuring datasets. AgentCL addresses how CL should be evaluated, which determines what a CL study can conclude. Existing streaming benchmarks (§2.1) collect or stream data without discussing methodology, overlooking the role of task stream type and metrics.

  1. Modeling reuse opportunities. Prior streams leave cross-task relations unknown, so gains may reflect incidental task overlap rather than genuine reuse. We distinguish conventional, dependent, and independent relations. Dependent streams place sparse, implicit, and distant reuse opportunities among distractors, and keep the target tasks identical to the conventional streams (§4.1). This reveals entirely different learning processes on evaluated streams (Table 10).
  2. Redefining CL properties for agents. Agents learn from a few sequential episodes rather than i.i.d. batches. They receive no ground-truth feedback, and they must decide on their own what to store and reuse (§3.1, §3.2). They keep experience explicitly rather than absorbing it into weights, so the core challenge shifts from catastrophic forgetting to retrieving and using the right experience amid interference (§3.3). We define PG/SG/GG accordingly, and these metrics depend only on task performance and apply to any CL approach.
  3. Empirical Insights for agentic CL. Agents should actively accumulate relevant experience, since weakly related experience does not improve plasticity (App. B.12; §5.3). They should also stay robust to weakly relevant memory, which often makes them underperform the memoryless agent (Table 10; §5.3).

Q2: Limited coverage of realistic long-term memory demands. Which conclusions extend to persistent personal-agent workflows?

Our streams are designed to reflect the following demands of real-world workflows:

  • Evolving and diverse information. In our dependent streams, a later task can depend on an earlier session without being told which one. The information it may leverage is diverse: 1) User requirements: input/output formats in CodeEval-Pro and the user’s state constraints in BrowseComp+. 2) Knowledge sources: some entities may be shared, but not the inferential chain. 3) Procedures and skills: functions for coding tasks, or reusable skills in BabyAI, e.g., unlocking a door before picking up an object.
  • Persistent environments. All CodeEval-Pro tasks evaluate coding with a few common packages. All BrowseComp+ tasks search the same corpus of 100k documents. In ScienceWorld and BabyAI, the agent acts in the same simulator throughout the stream, with the same world dynamics and action space.
  • Past experience can conflict with the present. On CodeEval-Pro, a complex task reuses the logic of an earlier function, but its interface is different, e.g., a prior task defines task_func(s1, s2) for a single pair of Series, while the complex task requires combined_visualization(series_pairs) over many pairs. In ScienceWorld and BabyAI, tasks in the same family share a goal but take place in different environment configurations, so the steps for a prior task may not apply to a later one.
  • Workflows span contexts with sparse relevance. Each task runs in a fresh context, and memory is the only link between sessions. When a relevant episode is needed, it makes up about 1% of memory and lies roughly half a stream earlier.

Since the streams reflect these demands, we expect our main conclusions to extend to persistent personal-agent workflows.

AuthorsOpenReview ↗

To Reviewer 616G Part 2

Q3: What evidence distinguishes abstract skill transfer from retrieving and adapting a closely related subtask solution?

Both abstract skill transfer and concrete interaction records are important. We show evidence on both and clarify that the gain does not come from copying prior task solutions:

  1. Concrete interactions contribute the largest share. On CodeEval-Pro, MemProbe gains +21.9 points over ReAct (66.7 vs. 44.8). Removing interaction memory costs 16.0 points of this gain, versus 7.0 points for insights and 4.9 points for skills (Table R4).
  2. Abstract memory is useful without concrete interactions. When MemProbe keeps only skill and insight memory and drops all raw interactions, it still beats ReAct by +5.9 points (50.7 vs. 44.8, Table R4). On BrowseComp+, methods that store only abstractions also gain substantially (AWM workflows +19.0, DC-RS +14.0). Episode-based ExpRAG gains +32.0, and MemProbe, which combines both, gains the most (+40.0, Table 11).
  3. A retrieved prior task is not a ready answer. On CodeEval-Pro, adding the ground-truth subtask solution to the subtask description does not help (59.0 vs. 60.4), and both stay below MemProbe (66.7, Table 10). The gain comes from how earlier tasks were solved, not from their final code. On BrowseComp+, only 1.6% of subtasks share both the answer and the evidence documents with their parent task (Appendix D.2).
  4. Transfer to unrelated tasks is limited, and AgentCL measures it separately. On the held-out independent tasks, no memory method outperforms ReAct (GG of −0.8 for MemProbe, Table 10). We agree that current gains primarily reflect specialized reuse (§5.3).

Q4: How does task-level retrieval compare with indexing skills by capability, especially when task descriptions differ a lot?

1. Skill is one form of reusable experience, but not the only one. We view the experience an agent accumulates over a task stream as heterogeneous, so MemProbe stores interaction, insight, and skill memories for every solved task (Appendix B.6). MemProbe is a probe for these design choices rather than a proposed skill library, so it uses one shared task index for all three forms.

2. Experiment: task index vs. skill index. We add a MemProbe variant that differs only in how skills are found.

  • Task index (in the paper). The current task is matched against stored task descriptions, and the skills of the top-2 retrieved tasks are loaded.
  • Skill index (Table R8). Every skill is embedded on its own by its name and description. The current task retrieves the top-6 skills directly, regardless of which task produced them. Top-6 is chosen to cover the number of skills retrieved by the task index, since each of the top-2 tasks stores at most three skills.

Interaction and insight memories remain task-indexed in both. We run the comparison on CodeEval-Pro, since skills, as procedural knowledge, matter most in coding.

Table R8: Accuracy (%) and 95% CI on the 48 CodeEval-Pro complex tasks, averaged over three runs. Skill index results are new.

StreamIndexPass 1Pass 2
DependentTask66.7 -7.6/+7.664.6 -8.3/+7.6
Skill62.5 -7.6/+7.661.8 -7.6/+7.6
ConventionalTask43.1 -7.6/+7.644.5 -7.6/+8.3
Skill44.4 -7.6/+8.344.4 -8.3/+8.3
  • The skill index yields no measurable gain over the task index. On both streams and in both passes, the two indexes are close in accuracy and their CIs overlap.
  • The skill index does reach skills from superficially different tasks. 54% of the skills it retrieves on the dependent stream and 66% on the conventional stream come from tasks outside the top-2 of the task index, but they do not yield a measurable gain.
  • Retrieving skills directly is harder. Under the same similarity threshold, the skill index retrieves any skill for only 28.5% of the complex tasks on the dependent stream and 11.1% on the conventional stream. A new task description is much closer to an earlier task description than to a one-sentence skill description. The task index finds the relevant subtask reliably (MemProbe Hits@2 of 91.7%–100% in Table 17), and its skills come with it.

3. The two indexes correspond to two different ways of using memory in practice. The task index models recording sessions and retrieving them by task similarity, which grows naturally with use. A skill library usually requires dedicated development, such as writing skill files with explicit applicability descriptions. AgentCL studies what an agent learns from its own task stream, so the task index is the natural default, while the benchmark itself is agnostic to the form and index of memory.

AuthorsOpenReview ↗

To Reviewer 616G Part 3

Q5: Comparison with native-memory agent systems.

We run Claude Code (v2.1.321) with Sonnet 5, the same backbone as in Table 15, and use its built-in auto memory (Table R9). The only modification is to prompt the model to use its auto memory.

Table R9: Accuracy (%) and PG/SG on the 48 CodeEval-Pro complex tasks with Sonnet 5. The last four rows are taken from Table 15.

MethodDependent 1st (PG)Dependent 2nd (SG)Conventional 1st (PG)Conventional 2nd (SG)
Claude Code, auto-memory off45.8–45.8–
Claude Code, auto-memory on47.9 (+2.1)50.0 (+2.1)47.9 (+2.1)45.8 (−2.1)
ReAct50.050.050.050.0
ExpRAG62.5 (+12.5)64.6 (+2.1)47.9 (−2.1)47.9 (+0.0)
ReMem68.8 (+18.8)64.6 (−4.2)47.9 (−2.1)45.8 (−2.1)
MemProbe58.3 (+8.3)56.3 (−2.0)50.0 (+0.0)50.0 (+0.0)
  • Native memory gives almost no gain. The net gain is a single task, while the methods in Table 15 gain +8.3 to +18.8 on the dependent stream.
  • The reason is what native memory chooses to store. Claude Code wrote nothing to memory while solving the 48 subtasks, and its 7 notes were all written while solving the complex tasks.

The comparison shows that a strong agent harness with native memory does not automatically reuse earlier solutions across related tasks.

Changes in the revised PDF

New content in the PDF is marked in blue.

  • Q4. Appendix B.6 now includes the skill index (Table R8) and the discussion above.
  • Q5. The Claude Code rows of Table R9 are added to Table 15 and discussed in Appendix B.9.
Reviewer_YA3ROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AgentCL, a framework for evaluating continual learning in LLM agents that use non-parametric memory. The key idea is to control how tasks in a stream relate to each other. "Dependent" streams place simpler subtasks before complex tasks that can reuse their solutions. "Conventional" streams contain the same complex tasks without those subtasks. Held-out "independent" tasks test transfer. A two-pass protocol gives three metrics: plasticity (gain over a memoryless agent in the first pass), stability (change in a second pass with frozen memory) and generalization (gain on independent tasks). The streams are built from CodeEval-Pro, BrowseComp+ (with subtasks generated by GPT-5.2) and BabyAI, with MMLU-Pro and ScienceWorld used as conventional streams. The authors compare seven memory methods, including their own MemProbe, against a memoryless ReAct agent. They find that dependent streams separate the methods much more clearly than conventional streams, and that memory often gives little gain, or even hurts, on conventional and held-out tasks.

Critical strengths
  • The paired design is clean. The dependent and conventional streams contain exactly the same complex tasks and differ only in whether related subtasks come first (Section 4.1), and held-out independent tasks are added on top. I think this is the most reusable idea in the paper.
  • The comparison of seven memory methods against a memoryless agent gives useful negative results. On the HumanEval-Pro independent tasks, memoryless ReAct is strongest (72.5), and DC-RS has the largest conventional-stream SG (+4.2) but the largest GG drop (-11.7).
Critical weaknesses
  • The main "discriminative power" result is largely built in by construction. If later tasks are made solvable by reusing earlier ones, methods that store and retrieve past episodes will separate from those that do not. So the larger spread across methods in Section 5.3 is expected, and by itself it does not show the benchmark is more valid. This matters most on BrowseComp+.
  • SG does not measure stability in the usual sense, and the differences behind the stability claims are small. By Eq. 2, the second pass reruns the same task after its own trajectory was written to memory. So SG mixes reuse of the agent's own earlier attempt with interference from later updates.
Questions for authors
  • For the BrowseComp+ dependent stream, what fraction of each parent's intermediate entities appear as subtask answers, and what is PG if subtask answers are removed from memory? If most of the gain remains, my main concern would ease.
  • Can you add a control stream matched in length and topic, for example the 48 CodeEval-Pro complex tasks preceded by 48 similar but non-reusable problems? If the spread across methods then stays near conventional levels, that would show it comes from reuse.
  • How does SG change if task i's own memory entry is excluded in the second pass? Also, was ReAct re-run in the second pass? Its values are identical across passes in Tables 2 and 3, so I cannot judge run-to-run noise.
  • Which methods were reimplemented (lines 313-314), and is uniform top-2 retrieval a fair setting for each of them? MemProbe was developed alongside the benchmark (Section 5.2), so I would like to know whether the baselines got similar tuning effort.
2 comments
AuthorsOpenReview ↗

To Reviewer YA3R Part 1

Dear Reviewer YA3R,

Thank you for your valuable comments!

Q1: The benchmark validity: Is a larger spread across methods expected?

The construction provides reuse opportunities, but it does not determine the spread.

  • Every method sees the same opportunities. The dependent stream models the sparse and implicit reuse opportunities that arise in real work. If all methods exploited them equally well, or if none could, the spread would stay small.
  • The spread reflects how current memory methods differ. On CodeEval-Pro, PG ranges from −5.2 to +21.9, and three of the seven memory methods do not gain at all (Table 10).
  • Conventional streams cannot support this attribution. They leave cross-task relations unknown, so a gain there cannot be told apart from incidental task overlap. Dependent streams fix the reuse opportunity, so differences among methods can be attributed to reuse.

Q2: How does SG measure stability? What is SG when task i’s own memory is excluded in pass 2?

Excluding task i's own memory lowers SG on dependent streams. We ran the proposed experiment. On CodeEval-Pro, we ran pass 2 on the complex tasks with the frozen pass-1 memory in two new runs: (i) with the full memory, and (ii) with every memory written by task i removed from retrieval for task i. The gap between (i) and (ii) is the contribution of task i's own memory. We cover the three methods whose memories can be traced to their source task (ExpRAG, ReMem, MemProbe).

Table R5: Pass 2 with and without task i's own memory (CodeEval-Pro complex tasks, accuracy in %). Own-memory effect = 2nd (full memory) − 2nd (w/o own memory). 1st is the first-pass accuracy of the run that produced the frozen memory.

StreamMethod1st2nd (full memory)2nd (w/o own memory)SG (full memory)SG (w/o own memory)Own-memory effect
DependentExpRAG62.562.558.30.0−4.2+4.2
ReMem58.356.352.1−2.1−6.3+4.2
MemProbe62.564.656.3+2.1−6.3+8.3
ConventionalExpRAG39.643.845.8+4.2+6.3−2.1
ReMem41.745.845.8+4.2+4.20.0
MemProbe41.743.845.8+2.1+4.2−2.1

On dependent streams, excluding task i's own memory lowers second-pass accuracy by 4.2 to 8.3 points for all three methods. SG is within 2.1 points of zero with the full memory and becomes negative for every method without task i's own memory (−4.2 to −6.3). On conventional streams, excluding it changes second-pass accuracy by at most 2.1 points, which is one task.

SG measures whether the agent can reuse its own experience. Stability in CL concerns whether a task's own experience remains usable. For non-parametric memory, retrieving it poses little difficulty: MemProbe retrieves it for 98.8%/98.0% of complex tasks on CodeEval-Pro/BrowseComp+ in pass 2 (Table 14). The difficulty lies in using it, and this reuse can fail, e.g., ReMem's SG is −10.0 on the BrowseComp+ dependent stream (Table 11). To check for interference from later updates, we split tasks by whether any later-task memory enters their pass-2 context (Table R3 in our response to Reviewer B2zG, Q3). On dependent streams, such memories appear for at most 17% of tasks. Table R3 finds no significant effect of these memories on SG. Together, Tables R3 and R5 indicate that SG on dependent streams mainly reflects self-task reuse.

AuthorsOpenReview ↗

To Reviewer YA3R Part 2

Q3: BrowseComp+: What fraction of parent-task intermediate entities appear as subtask answers, and what is PG with those answers removed from memory?

BrowseComp+ does not annotate intermediate entities, so we use each complex task's evidence documents as a proxy for its intermediate reasoning and check whether subtask answers appear in them.

Table R6: Overlap between complex-task and subtask elements on BrowseComp+ (100 complex tasks). Each cell is the % of complex-task docs in that row that are also a subtask doc or contain a subtask answer string. Evidence docs are the non-gold documents that support intermediate reasoning. An answer string appearing in a doc does not mean the answer is inferred from it.

Complex task \ Prior subtasksGoldEvidenceAnswer string
Evidence docs (N=199)0.0%39.7%48.2%

By design, BC+ subtasks do not reveal the reasoning chain of the complex task (Appendix D.1). This mirrors a user whose earlier searches are related to a later, more complex need but do not spell out how to solve it.

  • Intermediate reasoning is not exposed. None of the documents used in a complex task’s intermediate reasoning (its non-gold evidence documents) serve as the gold document of any subtask (Table R6). Subtasks relate to certain properties of the final answer, not to the intermediate reasoning steps.
  • The gain remains without subtask answers in memory. We call a subtask answer exposed if it also appears in the evidence docs of its complex task. For 64 of the 100 complex tasks, the first-pass memory context of every method contains no exposed answer. On these 64 tasks, PG stays close to its value on all tasks (Table R7).

Table R7: Dependent-stream PG on the 64 complex tasks with no exposed subtask answer in memory, compared with all 100 complex tasks.

MethodPG, no exposed answerPG, all tasks
Mem0−1.6−1.0
LangMem−1.6−1.0
AWM+14.1+19.0
DC-RS+10.9+14.0
ExpRAG+29.7+32.0
ReMem+25.0+26.0
MemProbe+40.6+40.0

Q4: Add a control matched in length and topic (e.g., 48 non-reusable problems before the 48 CodeEval-Pro tasks).

We have run this control (Table R1 in our response to Reviewer B2zG, Q1). The 48 complex tasks are preceded by 48 non-reusable tasks from BigCodeBench, the same source as the original subtasks, so the stream matches the dependent stream in length and task domain.

The spread across methods stays at the conventional level. Its standard deviation is 2.5 and 1.7 in the two passes, compared with 3.0 and 1.9 on the conventional stream and 9.4 and 8.8 on the dependent stream. The spread therefore comes from reuse rather than stream length.

Q5: Was ReAct re-run in pass 2? How large is run-to-run noise relative to SG?

We did not re-run ReAct in pass 2 in our paper. ReAct keeps no memory, so its second pass would take exactly the same inputs as its first pass, and re-running it would only produce another sample of the same experiment.

Uncertainty is reported as 95% bootstrap CIs (Tables 7 and 8). On CodeEval-Pro, the CIs are computed over three runs and include run-to-run noise.

Our main stability conclusion is the gap between large PG and near-zero SG (Table 10), which does not depend on small SG differences.

Q6: Which baselines were reimplemented? Is uniform top-2 retrieval fair to every baseline?

Mem0 and LangMem use their official libraries, while AWM, DC-RS, ExpRAG, and ReMem were reimplemented following the original workflows and prompts.
These methods differ fundamentally in what they store and how memory enters the context, e.g., AWM injects induced workflows, DC-RS synthesizes a cheat sheet, and LangMem lets the agent issue its own search calls.
There is no common unit of memory budget across them, so we try to align the backbone model and the solve prompt, and use k=2k=2 wherever a method retrieves by similarity (Appendix B.3).

We do not tune the configuration of any method specifically for AgentCL, including MemProbe. All memory methods share k=2k=2, and we show that MemProbe is insensitive to kk (Table 16). For ExpRAG and ReMem, k=2k=2 already retrieves the reusable subtask for most complex tasks, with Hits@2 of 89–100% (Table 17), so a larger kk leaves little room for gain.

Changes in the revised PDF

New content in the PDF is marked in blue.

  • Q2. The analysis of SG is added to §5.3 (Table R5). Table R3 and the permutation test are in the new Appendix B.15.
  • Q3. Appendix D.2 now includes the answer-exposure analysis. The numbers in Table R6 are reported in the text, and Table R7 is added.
  • Q4. The matched conventional stream is described in Appendix B.13, its results in Table R1 are added to Table 7 as new columns, and §5.3 summarizes it.
  • Q5. Appendix B.2 states that ReAct is not re-run in the second pass.
  • Q6. §5.1 states which baselines are reimplemented.
Reviewer_B2zGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AGENTCL, an evaluation framework for continual learning in language agents that explicitly distinguishes cross-task relations. It constructs dependent streams, where later tasks can reuse knowledge from earlier subtasks, conventional streams, where such reuse relations are unknown, and independent tasks for evaluating generalization. The paper further proposes a two-pass evaluation protocol and three metrics to characterize different continual learning properties. Using AGENTCL, the authors evaluate several non-parametric memory methods and introduce MemProbe, a probing memory design that stores interaction, insight, and skill memories with experience filtering. Experiments across coding, deep research, and reasoning environments show that dependent streams produce substantially larger performance separation among memory methods than conventional streams, while also revealing a plasticity–stability trade-off and potential negative transfer from accumulated memory.

Critical strengths
  1. The paper addresses an important evaluation problem in continual learning for language agents: existing streaming benchmarks often do not explicitly control whether earlier experience is reusable for later tasks, making it difficult to determine whether a memory system genuinely learns and transfers experience. The distinction among dependent, conventional, and independent task relations is intuitive and potentially useful. The proposed PG/SG/GG metrics also provide a more informative view of learning dynamics than final accuracy alone.

  2. The paper evaluates a broad set of non-parametric memory methods across coding, deep-research, and reasoning environments, and the observation that dependent streams yield substantially larger method separation than conventional streams is consistent across several settings. The experiments also reveal meaningful plasticity–stability–generalization trade-offs and negative transfer from accumulated memory.

Critical weaknesses
  1. My main concern is whether the current experiments sufficiently establish the validity of AGENTCL as a rigorous benchmark, rather than simply showing that the constructed dependent streams amplify differences among memory methods. The paper primarily uses across-method performance dispersion as evidence of discriminative power, but larger variance does not necessarily imply better benchmark validity. Stronger evidence such as ranking stability, pairwise statistical significance, and consistency across stream realizations would be valuable.

  2. Moreover, dependent and conventional streams differ not only in task relations but also in the number and content of preceding tasks, introducing potential confounding factors related to stream length, memory size, and retrieval difficulty. A matched control containing the same number of irrelevant precursor tasks would better isolate the effect of reusable knowledge. Finally, PG and particularly SG aggregate several effects (including self-task memory, later-task transfer, retrieval interference, and consolidation), so their interpretation as clean measures of plasticity and stability requires further validation. The MemProbe ablation is also relatively limited and lacks uncertainty estimates and controlled memory-size comparisons.

Questions for authors
  1. Could the authors provide a matched control stream containing the similar complexity of precursor tasks as the dependent stream, but with no reusable relation to the target tasks? This would help isolate whether the observed gains and increased method separation are caused by reusable knowledge rather than differences in stream length, memory size, or retrieval corpus composition.

  2. The paper mainly uses across-method standard deviation to argue that dependent streams have stronger discriminative power. Could the authors provide a more rigorous quantitative analysis, such as pairwise significance sizes, ranking stability across stream seeds and backbones, and confidence intervals, to establish that the benchmark reliably distinguishes memory capabilities rather than merely increasing performance variance?

  3. SG compares second-pass performance with first-pass performance, but the second pass differs in multiple ways: it contains the task’s own stored experience as well as all subsequent experiences. How can SG disentangle stable retention from self-task memorization, positive backward transfer, and retrieval interference?

5 comments
AuthorsOpenReview ↗

To Reviewer B2zG Part 1

Dear Reviewer B2zG,

Thank you for your valuable comments!

Q1: A matched control containing the same number of irrelevant precursor tasks would better isolate the effect of reusable knowledge.

We have run the requested control. The matched conventional stream keeps the 48 complex tasks of the dependent stream in the same order and replaces each of the 48 prior tasks with a randomly selected BigCodeBench task that has no reusable relation to any later task. All other settings are identical to the dependent stream.

The gains disappear without reusable prior tasks. ExpRAG, ReMem, and MemProbe reach 62.5, 58.3, and 66.7 on the dependent stream, and drop to 41.7, 40.3, and 41.7 on the matched conventional stream (first pass, Table R1). No memory method outperforms ReAct (44.8) in the first pass.

The method separation disappears as well. The standard deviation of accuracy across methods is 2.5 and 1.7 in the two passes of the matched conventional stream. This is close to the 3.0 and 1.9 of the conventional stream and far below the 9.4 and 8.8 of the dependent stream (§5.3).

Hence both the gains and the separation come from reusable knowledge, not from stream length or memory size.

Table R1: Accuracy (%) and 95% CI on the 48 BigCodeBench-Lite-Pro complex tasks, using Qwen3.5-35B-A3B. The matched conventional stream consists of 48 random tasks plus 48 complex tasks from the dependent stream.

Dependent StreamConventional StreamMatched Conventional Stream
MethodPass 1Pass 2Pass 1Pass 2Pass 1Pass 2
ReAct44.8 -7.6/+8.344.8 -7.6/+8.344.8 -7.6/+8.344.8 -7.6/+8.344.8 -7.6/+8.344.8 -7.6/+8.3
LangMem44.4 -8.3/+8.345.8 -8.3/+8.339.6 -7.6/+8.342.4 -7.6/+8.342.4 -8.3/+8.342.4 -8.3/+8.3
Mem043.8 -8.3/+8.344.4 -8.3/+8.343.8 -7.7/+8.342.4 -7.6/+8.344.4 -8.3/+8.346.5 -8.3/+8.3
AWM39.6 -7.6/+7.638.2 -7.6/+8.344.5 -8.3/+8.341.0 -7.6/+8.342.4 -7.6/+8.342.4 -7.6/+8.3
DC-RS48.6 -8.3/+7.650.7 -8.3/+8.335.4 -7.6/+7.639.6 -7.6/+7.636.1 -7.6/+7.641.0 -7.6/+8.3
ExpRAG62.5 -7.6/+7.662.5 -7.6/+7.639.6 -7.6/+7.639.6 -7.8/+7.841.7 -7.6/+8.343.8 -8.3/+8.3
ReMem58.3 -8.3/+7.756.3 -7.6/+8.341.7 -7.7/+8.343.8 -8.3/+7.740.3 -7.6/+8.344.4 -7.6/+8.3
MemProbe66.7 -7.6/+7.664.6 -8.3/+7.643.1 -7.6/+7.644.5 -7.6/+8.341.7 -7.6/+8.341.7 -7.7/+8.3

Q2: The benchmark validity. Evidence such as ranking stability, pairwise statistical significance, and consistency across streams.

We summarize three kinds of validity evidence beyond the across-method standard deviation.

1. Pairwise statistical significance (Appendix B.7).

For every pair of methods in each of the three runs, we compute a 95% bootstrap CI of their accuracy difference. A CI that excludes zero means the two methods stay separated under resampling.

On CodeEval-Pro, the dependent stream separates 34.1% of these comparisons in the 1st pass and 31.9% in the 2nd, much larger than the conventional stream’s 7.1% and 6.0% (Table 9). On BrowseComp+ (new analysis), the dependent stream separates 78.6% of the method pairs in the 1st pass and 71.4% in the 2nd, versus 0.0% and 28.6% on the conventional stream.

If dependent streams only made the scores noisier, the pairwise CIs would widen accordingly and would still include zero. Instead, more pairs are separated, so the larger dispersion comes from real differences among methods that exceed the noise level.

2. Ranking stability (new analysis).

We test whether the method ranking is reproduced across repeated runs on CodeEval-Pro. A method pair is stable if one method scores higher than the other in every run.

Table R2: Percentage of stable method pairs among the 28 pairs of the eight methods on CodeEval-Pro.

Stream1st pass2nd pass
Dependent71.4%57.1%
Conventional21.4%14.3%
Matched conventional (Q1)25.0%0.0%

On the dependent stream most method pairs keep their order in every run. On the two conventional streams most pairs are tied or reversed between runs. Hence the separation among methods on dependent streams is a stable signal.

3. Consistency across stream realizations (in the paper).

The main conclusion holds across stream realizations. A different realization changes each complex task’s position in the stream and its distance from its subtask. We show that PG is positive in every source–target distance interval (Table 17) and non-negative in every stream-position bin (Table 18). The contrast with the conventional stream also holds across domains (Tables 10 and 11) and backbones (Table 15).

AuthorsOpenReview ↗

To Reviewer B2zG Part 2

Q3: Can SG separate retention from self-task memorization, backward transfer, and retrieval interference?

SG measures persistent reuse as a whole. SG is the net effect of writing task i's experience into memory, preserving it through later updates, and retrieving it from the frozen memory (§3.3), because stability in continual learning depends on all of them. Its main component is self-task reuse. With non-parametric memory, writing and retrieval pose little difficulty: MemProbe retrieves task i's own memory for 98.8%/98.0% of complex tasks on CodeEval-Pro/BrowseComp+ in pass 2 (Table 14), and excluding it lowers pass-2 accuracy for every method on the dependent stream (Table R5, Reviewer YA3R Q2). This is not memorization of answers, since memory is built without ground-truth labels (§3.1), so the reuse can help or hurt (Table 11). Retrieval interference from other tasks is measured separately on the (matched) conventional streams and independent tasks (Tables 10, 11, 19, and R1). We find no evidence that later tasks drive SG. Later tasks can affect task i (backward transfer) only if their memories enter its pass-2 context. We therefore split complex tasks by whether their pass-2 context contains any later-task memory, and compare SG between the two groups (Table R3). This covers ExpRAG, ReMem, and MemProbe, whose memory entries are traceable to individual tasks.

  • Later-task memories rarely reach a task on dependent streams. They appear for only 2.1% of CodeEval-Pro tasks and 8–17% of BrowseComp+ tasks.
  • Where they appear, they do not significantly change SG. On conventional streams, they appear for 12.5–50% of CodeEval-Pro tasks and 44–49% of BrowseComp+ tasks. We run a permutation test on the SG gap between tasks with and without later-task memories. We randomly reassign tasks to the two groups 20,000 times, and p is the fraction of reassignments whose SG gap is at least as large as the real one. p ≥ 0.12 in every case on both datasets (Table R3).

Table R3: SG (%) for tasks whose pass-2 context does or does not contain a later-task memory. p: p-value of the permutation test described above (p < 0.05 is conventionally significant). –: not tested, as only one task contains a later-task memory.

DatasetStreamMethodTasks with later-task memorySG of tasks without itSG of tasks with itp
CodeEval-ProDependentExpRAG2.1%0.00.0–
ReMem2.1%−2.10.0–
MemProbe2.1%−2.10.0–
ConventionalExpRAG50%0.00.01.00
ReMem12.5%0.0+16.70.12
MemProbe20.8%+1.80.01.00
BrowseComp+DependentExpRAG14%+2.30.01.00
ReMem17%−10.8−5.90.78
MemProbe8%−1.10.01.00
ConventionalExpRAG44%+5.4+4.51.00
ReMem47%+3.8+4.31.00
MemProbe49%+5.9+16.30.12
AuthorsOpenReview ↗

To Reviewer B2zG Part 3

Q4: Limited MemProbe ablations, including uncertainty estimates and controlled memory-size comparisons.

We re-ran the MemProbe ablation with three runs per variant (Table R4).

Uncertainty estimates. Table R4 reports 95% bootstrap CIs (the same procedure as Table 7, Appendix B.7) for every ablation result. On the dependent stream, MemProbe has the highest 1st-pass accuracy, and every variant that removes one memory type scores lower. On the conventional and the matched conventional streams, the ablations show no consistent direction, and no variant is clearly above ReAct (44.8).

Controlled memory-size comparisons. The matched conventional stream (Q1, Table R1) serves as this control, since only the reuse opportunity differs from the dependent stream. On this stream, the full MemProbe stores and retrieves the most memory, yet it does not clearly outperform its ablations and stays below ReAct (Tables R1 and R4). More memory alone does not raise accuracy. Retrieving more items per task (Table 16) does not improve accuracy either.

Table R4: Accuracy (%) and 95% CI of MemProbe ablations on the 48 BigCodeBench-Lite-Pro complex tasks with two passes, using Qwen3.5-35B-A3B, averaged over three runs. The ablation rows are new results.

Dependent StreamConventional StreamMatched Conventional Stream
VariantPass 1Pass 2Pass 1Pass 2Pass 1Pass 2
MemProbe66.7 -7.6/+7.664.6 -8.3/+7.643.1 -7.6/+7.644.5 -7.6/+8.341.7 -7.6/+8.341.7 -7.7/+8.3
w/o skill61.8 -7.6/+7.663.2 -7.6/+7.644.4 -8.3/+8.344.4 -7.6/+8.343.1 -7.6/+8.344.4 -8.3/+8.3
w/o insight59.7 -7.7/+8.361.1 -7.6/+7.639.6 -7.6/+8.341.7 -7.6/+8.341.0 -8.3/+8.340.3 -7.6/+8.3
w/o interaction50.7 -7.6/+8.351.4 -8.3/+8.345.1 -8.3/+8.345.8 -7.6/+8.343.1 -8.3/+8.341.7 -8.3/+8.3

Changes in the revised PDF

New content in the PDF is marked in blue.

  • Q1. The matched conventional stream is described in Appendix B.13, its results in Table R1 are added to Table 7 as two new columns, and §5.3 summarizes it.
  • Q2. In Appendix B.7, Table 9 now includes the pairwise analysis on BrowseComp+, and the ranking stability is added (Table R2).
  • Q3. The analysis of SG is added to §5.3 (Table R5). Table R3 and the permutation test are the new Appendix B.15.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.