acceptodds
Under review as a conference paper at ICLR 2027

Unifying Text Memory and Code Memory for Self-Evolving Agents

Abstract

Self-evolving agents improve over time by distilling experience from past executions to facilitate future tasks. Existing systems represent such experience either as natural-language text injected into the agent context or as code that can be invoked via tools. However, the trade-offs between the two representations are poorly understood, and the choice between them is made a priori at the system level rather than based on the characteristics of each individual experience. We present the first controlled study that compares text memory and code memory under an identical task stream. The results show that the two memory forms are complementary in terms of construction cost, execution efficiency, and transfer robustness, such that neither representation alone is sufficient. Guided by these findings, we propose Metis, a self-evolving agent system that unifies text memory and code memory in a hierarchical manner. In particular, Metis organizes textual experience into execution plans, environment facts, and common pitfalls, and selectively crystallizes recurring plans into callable code tools. This design combines the good generality of text memory with the high execution efficiency of code memory and ensures that tool-generation cost is justified by repeated reuse. We evaluate Metis on three challenging benchmarks for interactive agents. Compared with a no-memory agent, SOTA baselines improve average accuracy by 9.5%–15.3%, while Metis improves accuracy by 26.9% and matches the most efficient baseline in execution cost.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Self-Evolving Agent, Agent Memory, Agentic Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_uALeOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

A well motivated work studying and proposing how to combine textual and code memory more effectively. This paper is very well written and easily to follow with insightful study upfront on the first 1-2 pages immediately attract readers. Then the authors propose an intuitive method (Metis) inspired by their study to consolidate both memory and show strong empirical evidence to demonstrate the effectiveness of Metis.

Overall, I think is a very solid work with minor improvement suggestions.

Critical strengths
  • I really enjoy the writing of this paper where authors clearly demonstrate the novel problem in the introduction.
  • The proposed method is intuitively and well motivated by the actual problem, and the writing does not over-complicate the approach with just enought math to define and clarify the method.
  • Experiments are comprehensive and clearly demonstrate the benefit of Metis with improved performance and reduced token usage.
Critical weaknesses
  • Some design choices can be further clarified or motivated. For example, why textual memory is defined with the 3 components between L203-204.
  • There are some hyper-parameters defined in the approach such as N in L222-223 and budget β\beta in L258. Provide supporting study on these hyper-parameters can make the paper stronger.
  • Some notations can be further improved. For example, L252-253, p is defined as a collection of functions with p as inputs. This is not very rigorous. Suggestion modify the notations.
  • The analysis should include the textual / coding memory ratio for all baselines compared with Metis - this will show better where the gains of Metic come from.
Questions for authors
  • L252-253, potential typo of "3"?
  • Do you think the design of the current memory can overfit the existing tasks? A justification of why it is generally applicable would be better besides showing empirical results on the 3 benchmarks (which can also be cherry picked).
5 comments

Thanks for your kind words and insightful comments! We really appreciate your time and detailed comments. We presents extra clarifications and conducted additional experiments, and sincerely hope they can address your concern.

[W1: Clarification for certain design choices]
Thank you for the constructive suggestion! We agree that some design choices deserve clearer motivation. The three text memory categories are designed to capture distinct types of reusable experience:

  1. Environment Facts: Agents interact with environments that may impose constraints not known in advance. Recording discovered constraints helps avoid repeated failures and exploration. For example, a project configured with uv may require uv run python instead of a bare python command.
  2. Execution Plans: Recurring workflows would otherwise require agents to derive similar procedures from scratch. Execution plans preserve these reusable patterns, as also motivated by prior work such as APC and Memento cited in our paper.
  3. Common Pitfalls: Some failure patterns are not tied to a specific plan but remain important across tasks. For example, proceeding with an operation without obtaining required user consent may lead to incorrect execution. Pitfall memories help agents recognize and avoid such mistakes.

As for other key design choices, such as using recent task queries to guide reflection and retaining previously encountered queries for subsequent codification, we explain their motivations and provide supporting studies in our response to W2, as these choices are directly supported by those additional experiments. We will revise the paper to include these clarifications.

[W2: Hyperparameter Analysis]
We have conducted additional studies on four key hyperparameters: recent task window N; related-task history budget β\beta; codification threshold θ\theta; retrieval budget top-k. Since DeepSeek-V4-Flash is no longer available through its official API, we use DeepSeek-V4.1-Flash for these studies. Given the limited rebuttal timeline and the need to retrain Metis for each configuration, we conduct these studies on the STATE-Bench Travel split (100 training and 50 test tasks). We present the results for each hyperparameter below. We also explain the motivation for their designs, as mentioned in W1.

W2.a Recent Task Window (N).

During text reflection, the reflector receives the recent NN task queries alongside the current trajectory. When task patterns exhibit temporal locality, this additional context helps the reflector identify recurring patterns rather than overfitting to the current task. We reconstruct Metis with different NN values and evaluate each configuration twice, fixing β=10\beta =10 and θ=3\theta =3. We report the average accuracy and the proportion of unused text memory entries.

02481014
Acc83.0%81%82%86%85%89%
Text Memory Entries716562646466
Unused Text Memory Ratio17(23.94%)13(20.00%)13(20.97%)12(18.75%)10(15.63%)11(16.67%)

Analysis. We observe that, for the same training task, reflectors with larger NN explicitly reference similar but distinct earlier queries when constructing text memories, producing more generalizable guidance rather than task-specific solutions. This is consistent with the quantitative results: larger windows generally improve accuracy (83% at N=0N=0 vs. 89% at N=14N=14) while reducing the proportion of unused text entries (23.94% vs. 16.67%). Though there are some fluctuations, we can see an increasing pattern here. These findings support the benefit of incorporating recent task context into memory reflection.

W2.a Related-Task History Budget (β\beta).

When codification is triggered, the codifier receives two sets of historical queries: candidate queries B(p)B(p), newly collected since the previous codification round; and related queries R(p)R(p), previously consumed by the codifier. Their sizes are governed by the codification threshold θ\theta and history budget β\beta, respectively. Similar to NN, the related-task history provides additional context to help the codifier generate more generalizable tools. We reconstruct Metis with different β\beta values and evaluate each configuration twice, fixing N=10N=10 and θ=3\theta =3. The averaged results are shown below.

06912
Acc79%84%87%82%
Code Memory Size19111520
Unused Code Memory Ratio5.5(28.95%)2(18.18%)5(33.33%)9.5(47.50%)

Analysis. The results suggest that both insufficient and excessive historical context can hurt codification. With β=0\beta =0, the codifier lacks previously observed task variations, making it more prone to generating narrowly scoped tools. Although its unused-tool ratio is lower than those at β=9\beta =9 and 1212, accuracy drops to 79%. Conversely, excessive history introduces greater task variation, making it harder to identify tools that are stable for shared routines. The increasing unused-tool ratio at larger budgets further reflects this issue. Overall, a moderate history budget provides a better balance between generalization and codification quality.

W2.c Codification Threshold (θ\theta).

The codification threshold θ\theta controls the size of candidate query buffer and how frequently tool generation is triggered, with larger values making codification more selective. We reconstruct Metis with different thresholds and evaluate each configuration twice, fixing N=β=10N=\beta =10. Here, θ=1\theta =1 represents Eager codification, while θ=100\theta =100 effectively corresponds to the Text-only variant. The averaged results are shown below.

1357100
Acc83%84%85%91%82%
Number of Tools5214870
Unused Code Entry Ratio27.5(52.88%)4(28.57%)00.5(7.14%)0
Code Reflection Cost27,639,9966,821,5032,879,8542,483,0420
Codification Count983110140
ReAct Turns7.37.667.887.918.45

Analysis. Increasing θ\theta from 1 to 7 reduces the number of generated tools from 52 to 7 and code construction cost by 91%, while improving accuracy from 83% to 91%. The unused-tool ratio also drops substantially (52.88% to 7.14%), suggesting that recurrence helps identify tools with greater reuse potential. Although more conservative codification settings slightly increase ReAct turns, they achieve a substantially better balance between accuracy and construction cost.

**

W2.d Retrieval Top-$k$

**

We evaluate different retrieval top-kk values on all three STATE-Bench domains using the same previously constructed memory. Top-kk limits the candidate pool passed to the Memory Manager, avoiding the cost of examining the entire memory library as it grows. The results are shown below.

Metrics251015203050
Acc76.0%80.0%81.3%80.0%79.3%80.0%79.3%
Manager tokens3,9997,98712,26114,71916,69819,49721,147
Steps7.037.077.247.227.377.257.19

Analysis. The results show that exposing more memories to the Manager does not necessarily improve performance. A small kk may exclude useful entries, reducing accuracy to 76.0% at k=2k=2. Conversely, increasing kk beyond 10 brings no further accuracy gains while steadily increasing Manager token consumption. Overall, k=10k=10 achieves the highest accuracy (81.3%) with moderate retrieval overhead, demonstrating the importance of balancing memory coverage and selection cost.

[W3: Improve the notation definitions.]
We agree that the original notation is unclear. Our intention at L252-253 was to emphasize that these fields are specific to each plan. We will instead define each plan directly as a tuple (e.g., BB,RR,DD) to denote its associated fields. We will revise the paper to improve our notation.

[W4: Textual / Coding Memory Ratio.]
Thanks for the suggestion! We actually found an interesting feature of Metis through this experiments: it automatically adjust its memory components ratios to suits different workloads. We provide a breakdown of the memory composition of Metis, SkillX, and ASI across all three benchmarks. ACE maintains a compound textual playbook and is therefore excluded from the entry-level ratio comparison. For SkillX, we separately report planning and atomic skills as textual entries and functional skills as code-bearing entries.

STATE-Bench

Text MemoriesCode MemoriesCode : Text Ratio
Metis266290.109
SkillX2114001.896
ASI-422-

OfficeBench

Text MemoriesCode MemoriesCode : Text Ratio
Metis45360.8
SkillX77380.494
ASI-56-

AppWorld

Text MemoriesCode MemoriesCode : Text Ratio
Metis52400.769
SkillX802523.15
ASI-162-

The results reveal two observations:

  1. Selective codification substantially reduces tool construction. Compared with ASI, Metis generates 93.1%, 35.7%, and 75.3% fewer callable tools on STATE-Bench, OfficeBench, and AppWorld, respectively. Compared with SkillX's code-bearing functional skills, Metis produces 92.8%, 5.3%, and 84.1% fewer code entries on the same benchmarks. This further proves the effectiveness of selective codification.
  2. Metis adaptively adjusts its memory composition to different workloads. On STATE-Bench, which emphasizes dynamic user interactions and policy-dependent decisions, Metis predominantly retains textual experience, with a code-to-text ratio of only 0.109. In contrast, on OfficeBench and AppWorld, where reusable tool-based procedures are more prevalent, the ratios increase to 0.800 and 0.769, respectively. These results demonstrate Metis' ability to adaptively balance textual guidance and executable memory according to workload characteristics.

[Q1: Typo]
Thanks for pointing this out! We agree that the wording is unclear. "Three" refers to the additional codification-related fields, B(p)B(p), R(p)R(p), and D(p)D(p), while body(p)body(p) and v(p)v(p) are shared with other text memories. We will revise the passage to make this distinction explicit.

[Q2: Discussion on the generalization of the proposed memory design.]

Frankly, current memory designs of Metis may not benefit all workloads equally. Its design primarily targets settings where (1) agents need to interact with stateful environments through tools and APIs, and (2) tasks exhibit reusable execution patterns. These conditions are common in personal assistants and potentially embodied agents, but may be less prevalent in chat-only applications or highly heterogeneous tasks. Within the targeting scopes, Metis explicitly mitigates task-level overfitting, as supported by our evaluations across benchmarks that span different task scenarios.

But broadly, the key principle behind Metis is to match memory representations to the characteristics of reusable experience. We do not claim that the current memory schema suits all scenarios. As agents become increasingly integrated into everyday life, we expect them to perform more tasks through tool-based interactions with external environments. We hope Metis's design principles can provide valuable guidance for developing more efficient self-evolving agents with memory components tailored to their specific needs.

Reviewer_sWXdOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Metis is a self-evolving agent system that unifies text memory and code memory in a hierarchical manner. In particular, Metis organizes textual experience into execution plans, environment facts, and common pitfalls, and selectively crystallizes recurring plans into callable code tools. Compared with a no-memory agent, SOTA baselines improve average accuracy by 9.5%–15.3%, while Metis improves accuracy by 26.9% and matches the most efficient baseline in execution cost.

Critical strengths
  1. Textual guidance and executable tools serve different functions. The paper proposes a novel mechanism for selectively promoting text memory into code memory, addressing the question of which experiences are worth turning into tools.

  2. The method is described in sufficient detail to support reproducibility, including experience categorization, deduplication and updates, joint retrieval, dependency completion, parameterization, and sandbox testing.

  3. The experiments report executor tokens, ReAct turns, and memory-construction cost across multiple benchmarks, models, and cross-model experience transfer settings. Metis performs consistently well across different task types, providing convincing evidence of its effectiveness.

Critical weaknesses
  1. Although memory-construction cost is reported, the paper does not establish how its upfront token cost compares with the gains in task success rate at different task volumes.

  2. The extent of actual memory reuse remains unclear. Separate statistics on text-memory selection and code-tool invocation are needed, including the proportion of entries never reused and the distribution of reuse counts across tasks.

  3. The recurrence threshold discourages one-off tool generation, but past recurrence does not necessarily imply continued future reuse. Under long-tailed or changing task distributions, Metis may accumulate text memories and code tools with low usage. The resulting growth in storage and retrieval overhead remains insufficiently examined.

Questions for authors
  1. Could the authors analyze the cost-effectiveness of the upfront memory-construction tokens relative to the improvement in overall task success rate as the number of tasks varies?

  2. Could the authors separately report the selection frequency of text memories and the actual invocation frequency of code tools, including the proportion of entries never reused and the distribution of reuse counts across tasks?

  3. Under long-tailed task distributions or distribution shifts, could Metis continue accumulating rarely used text memories and code tools? Please report how memory-store size, tool utilization, and storage and retrieval overhead change as tasks accumulate, and discuss whether an eviction mechanism based on usage frequency, recency, or utility is needed.

6 comments

Thank you for your insightful comments and kind suggestions. We conducted several extra experiments and hope they can answer the major concerns.

**

W1&Q1W1\&Q1

: Cost-effectiveness analysis for memory construction cost relative to the task success rate.**
We conducted additional experiments on STATE-Bench to examine how memory-construction cost translates into task success improvements. Specifically, we reconstruct memory stores from different portions of the original training logs (300 training tasks in total) and evaluate them on the same test set. Since DeepSeek-V4-Flash is no longer available through its official API, we use DeepSeek-V4.1-Flash for these experiments.

MetricNo MemTrain 20%Train 40%Train 60%Train 80%Full Train
Accuracy66.67%73.33%74.67%79.33%80.67%81.33%
Acc. Gains-+6.66+8.00+12.66+14.00+14.66
Construction cost (M)-4.9010.1815.9721.9326.77

Analysis. The results show that Metis achieves substantial gains even with a limited construction budget. With only 20% training tasks, it improves accuracy by 6.66 percentage points, capturing approximately 45% of the full-training improvement. As the training pool grows, accuracy continues to improve but eventually approaches saturation. These findings suggest that Metis can deliver meaningful benefits with relatively modest upfront investment, while further memory construction yields diminishing returns.

**

W2&Q2W2\&Q2

: Memory usage and tool utilization analysis**
We provide detailed memory utilization statistics across all three benchmarks, separately reporting text-memory selection and code-tool invocation frequencies. For text memory, we compare Metis with SkillX, as ACE maintains a monolithic playbook. For code memory, we compare Metis with ASI and count actual tool invocations rather than retrievals. Notably, ASI exposes its entire tool library to the executor without retrieval, whereas Metis selectively retrieves relevant tools. For each method, we report task accuracy, memory library size, and the distribution of reuse counts (0, 1, 2–5, 6–10, and >10). A reuse count of 0 indicates that an entry was never selected (text) or invoked (code).

  1. STATE-Bench
MethodAccTool Count012-56-10>10
ASI68.0%422253 (59.95%)56 (13.27%)83 (19.67%)17 (4.03%)13 (3.08%)
Metis81.3%2910 (34.48%)3 (10.34%)8 (27.59%)4 (13.79%)4 (13.79%)
MethodAccText Memory Count012-56-10>10
SkillX71.3%21162 (29.38%)54 (25.59%)81 (38.39%)12 (5.69%)2 (0.95%)
Metis81.3%26679 (29.70%)70 (26.32%)93 (34.96%)13 (4.89%)11 (4.14%)
  1. OfficeBench
MethodAccTool Count012-56-10>10
ASI41.5%5638 (67.86%)5 (8.93%)7 (12.50%)2 (3.57%)4 (7.14%)
Metis44.9%3619 (52.78%)6 (16.67%)8 (22.22%)1 (2.78%)2 (5.56%)
MethodAccText Memory Count012-56-10>10
SkillX34.3%7719 (24.68%)7 (9.09%)31 (40.26%)6 (7.79%)14 (18.18%)
Metis44.9%4514 (31.11%)3 (6.67%)12 (26.67%)5 (11.11%)11 (24.44%)
  1. AppWorld
MethodAccTool Count012-56-10>10
ASI91.1%16274 (45.68%)23 (14.20%)42 (25.93%)8 (4.94%)15 (9.26%)
Metis94.0%4016(40.00%)0(0.00%)7(17.50%)5(12.50%)12(30.00%)
MethodAccText Memory Count012-56-10>10
SkillX92.3%8037 (46.25%)7 (8.75%)17 (21.25%)8 (10.00%)11 (13.75%)
Metis94.0%5223(44.23%)4(7.69%)11(21.15%)6(11.54%)8(15.38%)

Analysis. Two observations emerge from these results. First, Metis constructs substantially fewer code tools than ASI, yet a larger proportion of them are actually invoked across all three benchmarks, suggesting that recurrence-guided codification effectively concentrates construction effort on procedures with greater reuse potential. Second, Metis maintains comparable text-memory selection coverage to SkillX, while consistently achieving a higher proportion of frequently selected entries (more than 10 uses). Together, these findings suggest that Metis effectively balances the two representations: retaining broadly reusable textual guidance while selectively crystallizing recurring procedures into executable tools.

**

W3&Q3a.Memorymaintenanceunderdistributionshifts.W3\&Q3 a. Memory maintenance under distribution shifts.

**

This is a very interesting question. We acknowledge that Metis may accumulate rarely used memories under long-tailed or shifting task distributions, as past recurrence does not guarantee future reuse. This is a general challenge for persistent-memory systems. But the overhead is manageable: Metis employs embedding-based top-k retrieval followed by LLM-based selection, so only a bounded number of candidates are examined by the Memory Manager. Moreover, modern vector search is highly efficient even at billion scales, keeping retrieval overhead manageable.

Nevertheless, we agree that an indefinitely growing memory store would eventually introduce storage and maintenance challenges. We believe long-term memory forgetting is an important research direction, especially as standardized benchmarks for evaluating memory aging under evolving task distributions remain limited. One promising approach is MemoryBank [1], which introduces an Ebbinghaus-inspired forgetting mechanism: R=e−t/SR={e}^{-t/S}, where tt is the time since last recall and SS represents memory strength. Successful recalls increase SS, slowing future decay, while rarely recalled memories are more likely to be forgotten. Such mechanisms could naturally complement Metis to manage stale memories.

[W3&Q3 b. Memory growth and retrieval overhead analysis]

Under our train-then-test protocol, memory construction occurs only during training, and the resulting memory is frozen during evaluation. To examine how memory growth affects storage and retrieval overhead, we track the evolution of Metis's memory store as training tasks accumulate across all three benchmarks. Specifically, at every 10 training tasks, we report the number of text memories and code tools, total storage size (KiB), average Memory Manager token cost, and code-tool utilization rate. We skipped the overhead brought by top-k retrieval because with advanced vector search systems like DiskANN[2], top-k retrieval should hardly become an overhead bottleneck. These vector search systems can scale to a billion vectors while achieving under 10 ms mean query latency. The results are summarized below.

Appworld

Task RangeTextCodeStorage Size (KiB)Manager Token CostTool Utilization
1-1091049.04,890.220.0%
11-20151689.516,650.431.3%
21-301823127.418,371.343.5%
31-402427163.916,037.837.0%
41-503130195.723,148.943.3%
51-603534222.717,488.541.2%
61-704137253.318,898.743.2%
71-804438270.216,964.842.1%
81-905240290.122,015.642.5%

OfficeBench

Task rangeTextCodeMemStorage Size(KiB)Manager Token CostTool Utilization
1-104534.51,826.320.0%
11-209744.73,019.414.3%
21-30131173.513,078.518.2%
31-40171487.210,044.321.4%
41-502117107.712,720.529.4%
51-602722132.913,612.122.7%
61-703326155.813,239.626.9%
71-804227165.514,500.429.6%
81-904434182.29,439.535.3%
91-934536189.28,730.338.9%

Statebench

Task rangeTextCodeMemStorage Size(KiB)Manager Token CostTool Accumulated Utilization
1-1054153.67,368.3100.0%
11-20945109.114,453.720.0%
21-301347164.916,880.714.3%
31-4016610215.916,950.240.0%
41-5019216282.319,128.831.2%
51-6020719325.918,805.342.1%
61-7022222369.517,408.240.9%
71-8024124420.316,590.245.8%
81-9025327467.119,951.055.6%
91-10026629499.421,218.558.6%

Analysis. As training tasks accumulate, memory stores grow steadily, but their storage footprints remain modest, reaching at most 499.4 KiB. Meanwhile, Manager token cost does not grow proportionally with memory size. For example, on STATE-Bench, storage increases from 164.9 to 499.4 KiB after task 30, while Manager token cost increases only from 16.9K to 21.2K. This is consistent with our bounded top-k retrieval design. These results demonstrate manageable storage and retrieval overhead at the evaluated task scales.

[1] W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang, “MemoryBank: Enhancing Large Language Models with Long-Term Memory,” 2023. [Online]. Available: http://arxiv.org/abs/2305.10250

[2] S. J. Subramanya, F. Devvrit, H. V. Simhadri, R. Krishnawamy, and R. Kadekodi, “DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node,” in NeurIPS, 2019.

Dear reviewer sWXd, Thank you for reviewing our paper and for your insightful feedback. With only about three days left in the discussion period, we would really appreciate it if you could review our responses and let us know whether any specific concerns remain. This will give us time to address them before the discussion closes. Best, Authors

Reviewer_QWnsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies experience reuse for agents with frozen language-model weights, distinguishing textual guidance from directly callable code. Metis maintains environment facts, pitfalls, and execution plans as text, and promotes repeatedly selected plans into executable helpers. A memory manager jointly selects text and tools, while a custom harness handles trajectory inspection, dependencies, and tool admission. The main evaluation constructs memory in one pass over training tasks, then freezes it for testing on AppWorld, OfficeBench, and STATE-Bench. A separate Oracle-versus-Streaming profiling study motivates the design. The paper also reports cross-model transfer and ablations of memory forms and the recurrence gate.

Critical strengths
  1. A coherent implementation of selective skill construction. Maintaining general guidance as text while materializing recurring routines as callable helpers is a sensible system design. The dependency-closure and trajectory-inspection mechanisms in Appendix C address concrete execution and reflection problems.

  2. The complete system improves on the no-memory agent in several settings. Tables 1–2 cover three application environments and two main executor families. The Eager ablation is informative: selective codification uses 7.90M construction tokens versus 11.60M for eager codification, with reported accuracy of 66.7% versus 63.2% (Figure 4). This supports studying when to construct tools, even though the broader conclusions require stronger evidence.

Critical weaknesses
  1. The incremental contribution over existing skill learning is limited, and the representation study does not establish the claimed general principle. Converting experience into reusable textual procedures or executable programs is already central to skill-learning systems. The included ASI baseline induces and verifies callable programmatic skills, and its original paper explicitly compares programmatic skills with textual workflow memory. SkillX already extracts hierarchical planning, functional, and atomic skills. Practical Agent Skills also package instructions with executable scripts, so a text/code combination alone is insufficient differentiation. Metis's more specific contribution is a staged construction rule based on repeated plan selection, but this contribution is limited. Moreover, the threshold counts selection, including on unsuccessful tasks, rather than verified successful reuse, semantic stability, or measured future benefit (§3.3). The paper does not establish that this signal is preferable to similarly simple alternatives.

  2. The evaluation is too limited and simplified to establish practical value for contemporary agents. The experiments use bounded, repeatable application workflows with benchmark-specific interfaces. AppWorld is evaluated only on test-normal, although the Grok no-memory agent already reaches 94.05% on that split; the harder test-challenge split is omitted. This gives insufficient evidence on tasks that still challenge strong agents. OfficeBench and STATE-Bench are not demonstrated to be saturated, but they also do not establish effectiveness on open-ended coding or heterogeneous long-running work. The connection to current foundation-model evaluation is weak: the recent foundation-model technical reports checked for this review do not report results on these three benchmarks. Their absence is context for the coverage concern; the concrete problems are the omitted difficult split and the untested deployment scope.

  3. The paper defines a ReAct executor and its own Metis harness (Appendix C), but does not show whether the gains survive within a mature harness with existing context management and skill support. A comparison within such a harness, holding the model, tools, and budget fixed, would be more informative about added practical value. Moreover, some wins over the strongest baseline are small: 81.3% versus ACE's 80.7% on STATE-Bench with DeepSeek, and 40.6% versus 40.1% on OfficeBench with Grok. No repeated memory-construction runs, task-order variation, or uncertainty are reported. The key ablations instead use GPT-4o and Claude Sonnet 4.6 on AppWorld dev, leaving their applicability to the main executors unresolved. Finally, equal single-pass task exposure does not establish equal interaction budgets: SkillX's exploratory expansion is disabled while Metis reflection can inspect and probe the environment (§3.2; Appendix B).

Questions for authors
  1. What evidence shows that repeated plan selection predicts successful tool reuse or future savings? Please provide existing selection/success/use counts and clarify the advantage over successful-use counting or other simple codification schedules.
  2. Why is AppWorld test-challenge omitted despite the high no-memory test-normal score? What existing evidence supports added value within a mature harness that already supports reusable skills? Please distinguish the application-environment results from broader deployment claims.
11 comments

Thank you for your insightful comments and kind suggestions. We conducted several extra experiments to answer your major concerns. We would be grateful if you would reconsider your rating in light of these results.

[W1.a: Contributions over existing works and Agent Skills]

Existing self-evolving systems have demonstrated the value of both textual guidance and executable tools. However, the choice of memory representation has largely remained a fixed design decision, with its underlying trade-offs insufficiently understood. The core contribution of Metis is not whether both representations can be supported, but why and when each should be used, and how the two forms interact with each other.

In terms of terminology, as highlighted in our introduction, we distinguish the two memory forms by how the agent consumes the stored experience rather than by the content of the experience. Text memory is injected into the agent's context and interpreted through reasoning, even when it contains code snippets. Code memory encapsulates experience as tools that the agent can directly invoke. Under this distinction, each baseline actually only covers one memory format, but not both:

  1. ACE: ACE maintains an evolving textual playbook. The playbook is injected into the agent's context to guide future executions, corresponding to our text-memory category.
  2. SkillX: SkillX constructs a hierarchical skill library consisting of planning, functional, and atomic skills. These skills are retrieved and injected into the agent's context rather than directly invoked as executable tools, thus falling into our text-memory category.
  3. ASI: ASI induces executable programs from successful execution trajectories, verifies them, and exposes them as callable tools, corresponding to our code-memory category.

Agent Skills indeed defines an open format for packaging reusable capabilities through SKILL.md instructions, optional executable scripts, and supporting resources. But being reusable does not mean it defines how agents autonomously acquire and evolve skills from their execution experience, which Metis tries to answer.

[W1.b, Q1.a: Analysis for the selective consolidation strategy]

We use repeated plan selection as a recurrence signal for two reasons. First, reliable task-level success feedback is not always available in real-world agent deployments. Second, task success does not necessarily reflect the reusability of an individual routine. A task may fail for reasons unrelated to the selected plan, while the underlying procedure remains useful across other tasks. Therefore, plan selection provides a more direct signal of recurring procedural demand. Following your suggestion, we conducted additional experiments on OfficeBench to compare Metis with the success-use counting codification strategy. We use DeepSeek-V4.1-Flash, as DeepSeek-V4-Flash is no longer available through its official API. The results are summarized below, and suggest that plan selections from unsuccessful tasks can also provide useful evidence of recurring procedures.

Codification Strategysuccessful-use countinguse counting (Metis)
Accuracy43.96%45.41%

[W2 a. Benchmark concerns]

1. Diverse Scope
Our evaluation covers three distinct agentic scenarios:

  • AppWorld: Multi-application workflows involving API interactions, cross-application coordination, and state-dependent execution.
  • OfficeBench: Office productivity workflows involving document processing, spreadsheet operations, and multi-step task automation.
  • STATE-Bench: Multi-turn enterprise workflows involving user interactions, policy constraints, and complex tool operations.

These benchmarks cover diverse application domains, interaction patterns, and task structures. Our selection is also consistent with recent research on self-evolving agent works. For example, ACE and SkillX are evaluated on AppWorld, while LEGOMem uses OfficeBench. Our evaluation follows the corresponding baseline settings to ensure direct comparability. STATE-Bench is a relatively recent benchmark introduced by Microsoft, which explicitly provides a dedicated Agent Learning Track for evaluating agents' ability to learn from past experiences, closely matching Metis' research objective.

2. Difficulty

While Grok 4.6 approaches saturation on AppWorld test-normal with 94.05% accuracy without memory, OfficeBench and STATE-Bench still leave substantial room for improvement. On OfficeBench, both DeepSeek-V4-Flash and Grok 4.6 achieve only 30.4% accuracy without memory, which increases to 44.9% and 40.6% with Metis, respectively. Similarly, on STATE-Bench, Metis improves accuracy from 66.0% to 81.3% for DeepSeek-V4-Flash and from 67.3% to 82.0% for Grok 4.6. These results show that Metis remains effective on tasks that are even challenging for strong foundation models.

[W2.b. & Q2.a: Experiments on AppWorld test-challenge split.]
AppWorld's test-challenge split introduces applications entirely unseen during training, creating a substantial distribution shift with limited opportunities to reuse previously accumulated experience. Since Metis focuses on learning and reusing experience across tasks, our main evaluation follows SkillX's protocol and uses test-normal.

To further examine Metis on test-challenge tasks and address your concern about repeated memory construction and task ordering, we conduct three independent training runs. Specifically, we combine the original training set (90 tasks) with test-challenge (417 tasks), shuffle the combined pool using three random seeds 42, 11, and 17, and split each into 90 training and 417 test tasks.

Since it requires several hours for each training and evaluation, it is impractical to retrain all baseline methods from scratch given the limited time. Thus, we select the second strongest baseline SkillX, for this evaluation. For each split, we independently reconstruct the memories of Metis and SkillX. For No Memory we conduct three independent evaluations on each held-out test. Across the three splits, Metis achieves an average accuracy of 89.13%, compared with 85.92% for No Memory and 80.88% for SkillX. Metis also reduces execution turns by 15.9% and 7.4%, respectively. The results are summarized below. Train counts and Test_challenge counts represent the resulting evaluation set after reshuffling and sampling.

Train countsTest_challenge countsNoMem Run1 AccNoMem Run2 AccNoMem Run3 AccNoMem Avg. Acc
Seed 427334485.61%85.85%86.81%86.06%
Seed 176934885.13%85.13%85.61%85.29%
Seed 117334485.37%86.33%87.53%86.41%

Seed 42:

AccExec Count
NoMem86.06%18.05
SkillX73.34%16.06
Metis90.17%15.40

Seed 17:

AccExec Count
NoMem85.29%17.81
SkillX82.49%16.35
Metis88.25%15.09

Seed 11:

AccExec Count
NoMem86.41%17.93
SkillX86.81%16.45
Metis88.97%14.74

[W3.a: Integration with Mature Agent Harnesses]
We would like to clarify that all methods use the benchmark-provided executors and share the same tool interfaces. Metis's harness primarily supports memory reflection, tool validation, and dependency management, rather than providing a more capable executor. ASI also has its own testing harness designed for its own tool generation. These harnesses are designed for memory components, and all baselines use the same executor.

As for mature agent harnesses, supporting reusable skills does not enable agents to autonomously acquire and evolve them. Even within mature harnesses, in order to enable agents to learn from past experiences, the system still needs to decide what experience to retain and how to represent the gained memories. These challenges remain relevant regardless of the underlying harness. We believe Metis's findings can provide useful guidance for integrating self-evolving memory into such systems.

[W3.b: Limited benefit over the strongest baseline]

Our main claim is not that Metis substantially outperforms every baseline in every setting, but that combining complementary memory representations provides more consistent improvements across different workloads. ACE performs well on STATE-Bench, SkillX on AppWorld, and ASI on OfficeBench. However, each baseline has a benchmark where its improvement over No Memory drops to only 2.0–3.9 percentage points, whereas Metis consistently achieves gains of at least 8.3 points across all three benchmarks. Averaged across benchmarks, Metis achieves 73.4% accuracy, compared with 68.5% for ACE, 66.9% for ASI, and 66.0% for SkillX.

[W3.c: Fair Comparison with SkillX]

Our evaluation controls training-task exposure rather than enforcing identical internal interaction methods across baselines. We preserve ASI's rewrite-and-test process, SkillX's plan rewriting for retrieval, and ACE's delta-based updates, as these mechanisms operate on the same provided training tasks. In contrast, SkillX's exploratory skill expansion generates additional tasks, introducing training exposure unavailable to other methods. Also, SkillX's own ablation study shows that one round of skill expansion provides only modest improvements, increasing accuracy from 62.35% to 64.58% on AppWorld while leaving BFCL-V3 performance unchanged. Across 3 benchmarks, Metis achieves 73.4% compared to 66% for SkillX, while the best effect of SkillX's own experiment shows only 2.53% improvement. Exploratory task expansion is orthogonal to our focus on memory representation and selective codification, and could benefit other self-evolving methods as well.

[Q1.b: Evidence for effectiveness of repeated plan selection]
To further examine the empirical evidence of the repeated plan selection signal, we retrained Metis with different codification thresholds ( θ\theta = 1, 3, 5, 7, 100) on the STATE-Bench Travel split, using DeepSeek-V4.1-Flash. Each configuration is evaluated twice, and we report the average accuracy. Notably, θ\theta = 1 and θ\theta = 100 correspond to the Eager and Text-only variants, respectively, extending our ablation study beyond GPT-4o. The results are summarized below.

Threshold1357100
Acc83%84%85%91%82%
Number of Tools5214870
Unused Code Entry Ratio27.5(52.88%)4(28.57%)00.5(7.14%)0
Codification Count983110140
ReAct Turns7.37.667.887.918.45

Compared with Eager, setting the threshold to 7 reduces codification attempts from 98 to 14 and generated tools from 52 to 7, while improving accuracy from 83% to 91%. The unused-tool ratio also drops substantially (52.88% to 7.14%), suggesting that repeated plan selection helps identify procedures with stronger future reuse potential. Also, these results reinforce our original ablation findings: recurrence-guided codification outperforms both Eager and Text-only variants.

[Q2. b: Added value within mature agent harnesses that supports reusable skills]

Thank you for the question! Mature agent harnesses already provide infrastructure for loading and executing reusable skills. However, once these systems aim to autonomously evolve their skills from execution experience, they face the same fundamental questions addressed by Metis: which experiences should be retained as textual guidance, which are worth crystallizing into executable scripts, and when the benefits of codification justify its construction cost. Existing skill infrastructure does not inherently resolve these decisions. Metis addresses them through differentiated text memory and selective codification, providing a complementary mechanism for experience-driven skill evolution. We believe these design principles are also applicable to mature agent harnesses seeking autonomous skill evolution.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their constructive feedback. We would like to briefly summarize the progress made during the rebuttal period:

Reviewers sWXd and uALe initially recommended acceptance, with suggestions primarily focusing on additional analysis, experimental statistics, design motivation, and presentation clarity. We responded with extensive new results together with further clarifications. We are glad that reviewer sWXd explicitly confirmed that our responses addressed the concerns and raised the score from 3 to 4, while reviewer uALe explicitly reaffirmed the recommendation for acceptance.

Reviewer QWns mainly raised concerns about baseline fairness, promotion signal alternatives, benchmark coverage, and practical value within mature agent harnesses. We conducted additional experiments, and the reviewer explicitly acknowledged that most concerns were resolved. The only remaining concern is the practical value within mature agent harnesses. To address the concern, we completed a Codex integration experiment as the reviewer suggested, and Metis still shows great benefits to Codex, even without careful integration optimization.

Overall, we believe Metis makes a meaningful contribution by turning memory representation from a fixed design choice into an empirically guided evolution strategy. We sincerely appreciate the reviewers' thoughtful engagement and hope these additional findings will inform the Area Chair's final assessment.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.