- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AgentCL evaluates continual learning in language agents by distinguishing dependent task streams, conventional streams with unknown reuse relations, and independent held-out tasks. A two-pass protocol measures plasticity, stability, and generalization gains. The paper also introduces MemProbe, which stores interactions, insights, and skills with quality filtering. Experiments across coding, information seeking, and reasoning environments show greater separation among memory methods on dependent streams, alongside limited transfer and occasional memory interference elsewhere.
Critical strengths
-
Useful evaluation perspective. Explicitly distinguishing task streams by the availability of reusable experience helps clarify when memory should benefit an agent.
-
Thorough empirical analysis. The paper goes beyond aggregate accuracy with component ablations, retrieval diagnostics, and analyses of retrieval depth, source–target distance, stream position, and different backbones. These analyses help explain when memory supports reuse and when it introduces interference.
Critical weaknesses
-
Limited conceptual novelty. The framework mainly combines restructuring existing datasets, performance-difference metrics, and established memory extraction and retrieval techniques. The paper does not sufficiently establish a substantive advance in our understanding of continual learning.
-
Task-bound skill retrieval. MemProbe retrieves historical task descriptions and loads their associated skills, rather than retrieving skills independently by capability or applicability. This design can miss useful skills from superficially different tasks and leaves the distinction between procedural knowledge and episodic memory underdeveloped.
-
Case reuse is not sufficiently separated from general skill transfer. The dependent streams deliberately place related subtasks before their corresponding complex tasks. Although this creates controlled reuse opportunities, the gains may primarily reflect adapting closely related solutions. Stronger controls are needed to establish transfer beyond these constructed relationships.
-
Limited coverage of realistic long-term memory demands. The tasks do not adequately represent evolving user preferences, persistent project state, conflicting information, or cross-context workflows. These are consequential challenges for personal agents, and their absence limits the practical scope of the conclusions.
-
Missing comparisons with native-memory agent systems. Additional backbone experiments do not establish how the evaluated methods compare with complete agents such as Claude Code using their native memory mechanisms. Such comparisons would strengthen the connection between the benchmark findings and deployed systems.
Questions for authors
-
Clarification: What evidence distinguishes abstract skill transfer from retrieving and adapting a closely related subtask solution?
-
Evidence request: How does task-level retrieval compare with independently indexing skills by their capabilities and applicability conditions, particularly when source and target task descriptions differ substantially?
-
Scope clarification: Which conclusions should extend to persistent personal-agent workflows? Comparisons with native-memory agent systems would be a valuable extension, although I recognize that these require additional evaluation.
5 comments
To Reviewer 616G Part 2
Q3: What evidence distinguishes abstract skill transfer from retrieving and adapting a closely related subtask solution?
Both abstract skill transfer and concrete interaction records are important. We show evidence on both and clarify that the gain does not come from copying prior task solutions:
- Concrete interactions contribute the largest share. On CodeEval-Pro, MemProbe gains +21.9 points over ReAct (66.7 vs. 44.8). Removing interaction memory costs 16.0 points of this gain, versus 7.0 points for insights and 4.9 points for skills (Table R4).
- Abstract memory is useful without concrete interactions. When MemProbe keeps only skill and insight memory and drops all raw interactions, it still beats ReAct by +5.9 points (50.7 vs. 44.8, Table R4). On BrowseComp+, methods that store only abstractions also gain substantially (AWM workflows +19.0, DC-RS +14.0). Episode-based ExpRAG gains +32.0, and MemProbe, which combines both, gains the most (+40.0, Table 11).
- A retrieved prior task is not a ready answer. On CodeEval-Pro, adding the ground-truth subtask solution to the subtask description does not help (59.0 vs. 60.4), and both stay below MemProbe (66.7, Table 10). The gain comes from how earlier tasks were solved, not from their final code. On BrowseComp+, only 1.6% of subtasks share both the answer and the evidence documents with their parent task (Appendix D.2).
- Transfer to unrelated tasks is limited, and AgentCL measures it separately. On the held-out independent tasks, no memory method outperforms ReAct (GG of −0.8 for MemProbe, Table 10). We agree that current gains primarily reflect specialized reuse (§5.3).
Q4: How does task-level retrieval compare with indexing skills by capability, especially when task descriptions differ a lot?
1. Skill is one form of reusable experience, but not the only one. We view the experience an agent accumulates over a task stream as heterogeneous, so MemProbe stores interaction, insight, and skill memories for every solved task (Appendix B.6). MemProbe is a probe for these design choices rather than a proposed skill library, so it uses one shared task index for all three forms.
2. Experiment: task index vs. skill index. We add a MemProbe variant that differs only in how skills are found.
- Task index (in the paper). The current task is matched against stored task descriptions, and the skills of the top-2 retrieved tasks are loaded.
- Skill index (Table R8). Every skill is embedded on its own by its name and description. The current task retrieves the top-6 skills directly, regardless of which task produced them. Top-6 is chosen to cover the number of skills retrieved by the task index, since each of the top-2 tasks stores at most three skills.
Interaction and insight memories remain task-indexed in both. We run the comparison on CodeEval-Pro, since skills, as procedural knowledge, matter most in coding.
Table R8: Accuracy (%) and 95% CI on the 48 CodeEval-Pro complex tasks, averaged over three runs. Skill index results are new.
| Stream | Index | Pass 1 | Pass 2 |
|---|---|---|---|
| Dependent | Task | 66.7 -7.6/+7.6 | 64.6 -8.3/+7.6 |
| Skill | 62.5 -7.6/+7.6 | 61.8 -7.6/+7.6 | |
| Conventional | Task | 43.1 -7.6/+7.6 | 44.5 -7.6/+8.3 |
| Skill | 44.4 -7.6/+8.3 | 44.4 -8.3/+8.3 |
- The skill index yields no measurable gain over the task index. On both streams and in both passes, the two indexes are close in accuracy and their CIs overlap.
- The skill index does reach skills from superficially different tasks. 54% of the skills it retrieves on the dependent stream and 66% on the conventional stream come from tasks outside the top-2 of the task index, but they do not yield a measurable gain.
- Retrieving skills directly is harder. Under the same similarity threshold, the skill index retrieves any skill for only 28.5% of the complex tasks on the dependent stream and 11.1% on the conventional stream. A new task description is much closer to an earlier task description than to a one-sentence skill description. The task index finds the relevant subtask reliably (MemProbe Hits@2 of 91.7%–100% in Table 17), and its skills come with it.
3. The two indexes correspond to two different ways of using memory in practice. The task index models recording sessions and retrieving them by task similarity, which grows naturally with use. A skill library usually requires dedicated development, such as writing skill files with explicit applicability descriptions. AgentCL studies what an agent learns from its own task stream, so the task index is the natural default, while the benchmark itself is agnostic to the form and index of memory.
To Reviewer 616G Part 3
Q5: Comparison with native-memory agent systems.
We run Claude Code (v2.1.321) with Sonnet 5, the same backbone as in Table 15, and use its built-in auto memory (Table R9). The only modification is to prompt the model to use its auto memory.
Table R9: Accuracy (%) and PG/SG on the 48 CodeEval-Pro complex tasks with Sonnet 5. The last four rows are taken from Table 15.
| Method | Dependent 1st (PG) | Dependent 2nd (SG) | Conventional 1st (PG) | Conventional 2nd (SG) |
|---|---|---|---|---|
| Claude Code, auto-memory off | 45.8 | – | 45.8 | – |
| Claude Code, auto-memory on | 47.9 (+2.1) | 50.0 (+2.1) | 47.9 (+2.1) | 45.8 (−2.1) |
| ReAct | 50.0 | 50.0 | 50.0 | 50.0 |
| ExpRAG | 62.5 (+12.5) | 64.6 (+2.1) | 47.9 (−2.1) | 47.9 (+0.0) |
| ReMem | 68.8 (+18.8) | 64.6 (−4.2) | 47.9 (−2.1) | 45.8 (−2.1) |
| MemProbe | 58.3 (+8.3) | 56.3 (−2.0) | 50.0 (+0.0) | 50.0 (+0.0) |
- Native memory gives almost no gain. The net gain is a single task, while the methods in Table 15 gain +8.3 to +18.8 on the dependent stream.
- The reason is what native memory chooses to store. Claude Code wrote nothing to memory while solving the 48 subtasks, and its 7 notes were all written while solving the complex tasks.
The comparison shows that a strong agent harness with native memory does not automatically reuse earlier solutions across related tasks.
Changes in the revised PDF
New content in the PDF is marked in blue.
- Q4. Appendix B.6 now includes the skill index (Table R8) and the discussion above.
- Q5. The Claude Code rows of Table R9 are added to Table 15 and discussed in Appendix B.9.
To Reviewer 616G Part 1
Dear Reviewer 616G,
Thank you for your valuable comments!
Q1: The contribution to the understanding of continual learning?
We respectfully clarify that our contribution is not limited to restructuring datasets. AgentCL addresses how CL should be evaluated, which determines what a CL study can conclude. Existing streaming benchmarks (§2.1) collect or stream data without discussing methodology, overlooking the role of task stream type and metrics.
Q2: Limited coverage of realistic long-term memory demands. Which conclusions extend to persistent personal-agent workflows?
Our streams are designed to reflect the following demands of real-world workflows:
Since the streams reflect these demands, we expect our main conclusions to extend to persistent personal-agent workflows.