acceptodds
Withdrawn from ICLR 2027

ERSkill: Evolving for Skill-Guided Adaptive Memory Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

While Large Language Model (LLM) agents increasingly rely on long-term memory for persistent interactions, the retrieval mechanisms governing this memory are rarely treated as evolvable components. This static approach limits performance on heterogeneous memory queries, which often demand diverse evidence construction strategies. To address this, we introduce ERSkill, a retrieval-centric framework for evolving, skill-guided memory access. ERSkill compiles interaction histories into a structured memory store and represents retrieval behaviors as executable skills composed of fundamental primitives. At inference time, a trained router dynamically matches each query to a suitable retrieval skill to construct tailored evidence for answer generation. ERSkill co-evolves the skill set and the router during training. It employs an experience trie to efficiently record explored retrieval paths, alongside a double-frontier mechanism that separates oracle-side capability expansion from router-validated deployment. Experiments across multiple agent memory benchmarks demonstrate that ERSkill substantially outperforms strong non-evolving and evolving baselines. Notably, it improves the overall average across F1, BLEU-1, and LLM-judge scores by 31.3% with Qwen3-Next-80B-A3B-Instruct and by 21.4% with GPT-5.4-nano.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, Agent, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZQDQOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents ERSkill, a framework that improves how LLM agents retrieve information from long-term memory. Instead of using a fixed retrieval method, ERSkill builds retrieval skills by combining basic operations such as search, evidence expansion, and LLM-based processing. For each query, a learned router selects the most suitable skill to retrieve relevant evidence.

During training, ERSkill jointly improves the retrieval skills and the router. An experience trie records previously tested skills to avoid repeated exploration. A double-frontier mechanism maintains useful candidate skills while ensuring that only skills that work well with the router are deployed.

Experiments on LoCoMo, LongMemEval, and PerLTQA show that ERSkill outperforms several non-evolving, adaptive-retrieval, and evolving-agent baselines with two LLM backbones. The ablation studies and controlled experiments further suggest that the improvements come from evolving and selecting retrieval skills, rather than only from using a richer memory structure.

Critical strengths
  1. The proposed method is well designed. Retrieval skills are executable and interpretable, while the experience trie and double-frontier mechanism help control the evolution process.
  2. The experiments are comprehensive. The paper evaluates the method on three benchmarks and two LLM backbones, with useful ablation studies, controlled comparisons, and efficiency analysis.
Critical weaknesses
  1. The paper should discuss SelRoute, which also routes queries to specialized retrieval pipelines. Please clarify how ERSkill differs from this line of work and what its evolving skill compositions add beyond selecting among predefined retrieval strategies.

  2. The baseline comparison should include recent methods for adaptive, multi-step memory retrieval, such as MemR³, MAGMA, and MemRetriever. Please clarify how ERSkill differs from these approaches and, where feasible, compare them under a shared evaluation setup.

  3. Appendix D.1 explains the conversation-level split and the exclusion of adversarial LoCoMo questions, but does not clarify how the final split sizes were obtained. Please specify any additional filtering or sampling, what counts as an “example,” and provide the conversation IDs or split files and random seed to support reproducibility.

Questions for authors

The construction of the final LoCoMo subset remains unclear. After excluding adversarial questions, the public LoCoMo-10 benchmark is commonly evaluated on approximately 1,540 questions, whereas the reported train, validation, and test sets contain only 233, 152, and 314 examples, respectively, for a total of 699.

Could the authors clarify what additional filtering or sampling was applied, what “example” denotes in this section, and how many conversations are assigned to each split? Since LoCoMo contains only a small number of underlying conversations, it would also be useful to discuss the sensitivity of the results to the particular conversation-level split.

3 comments

We sincerely thank you for your positive assessment of our method design and experiments. Below, we address your concerns in detail.

W1 & W2 (Comparison with SelRoute, MemR³, MAGMA, and MemRetriever)

Thank you for pointing us to these recent works. We clarify both how ERSkill differs from them conceptually and how their core paradigms are already covered in our experimental comparisons:

  • 1. Conceptual differences from ERSkill:
    • Routing over predefined pipelines (SelRoute, W1): SelRoute routes queries among a fixed, hand-crafted set of retrieval pipelines. In contrast, ERSkill treats the pipelines themselves as evolvable multi-step programs and co-evolves new skill compositions alongside the router.
    • Online iterative control and RL policy training (MemR³, MemRetriever, W2): MemR³ runs an online closed-loop controller (retrieve-reflect-answer) at test time, and MemRetriever fine-tunes the LLM backbone via RL for multi-turn search. In contrast, ERSkill is training-free on the LLM backbone and compiles multi-step retrieval patterns offline into reusable programs, executing a single router-selected program at test time with low cost (1.95k tokens/QA, Table 3).
    • Multi-relational graph traversal (MAGMA, W2): MAGMA builds multi-relational graphs (temporal, causal, semantic, entity) and traverses them with a fixed policy. ERSkill's memory substrate already includes an equivalent multi-relational graph (Appendix A), but rather than using a fixed traversal heuristic, ERSkill evolves the multi-step graph-traversal programs themselves.
  • 2. Coverage in our shared-setup experiments: Each of these paradigms is already represented in our existing experiments:
    • Selecting among predefined pipelines/primitives (paralleling SelRoute): First, our w/o Skill Evolution ablation (Tables 4 & 5) directly trains the exact same router to select among predefined seed pipelines without evolving new compositions; evolving new multi-step skills lifts Gold Evidence Hit Rate from 66.99% to 80.13% and LoCoMo L-J from 63.38 to 70.06 (Qwen) and 61.78 to 72.93 (GPT). Second, in + Shared substrate (Tables 2 & 10), the three evolving baselines (Dynamic Cheatsheet, ReasoningBank, and GEPA) are also given our exact memory store and predefined retrieval primitives to make their own selection decisions, yet all still trail ERSkill by 5.28–13.22 Average L-J points.
    • Test-time adaptive multi-step retrieval (paralleling MemR³ / MemRetriever): Covered by Adaptive-RAG in Table 1, which ERSkill outperforms by over 35 L-J points on LoCoMo (and +17.82 to +22.25 Average L-J across all benchmarks).
    • Graph-structured memory (paralleling MAGMA): Covered by the graph-based memory baseline A-Mem in Table 1; moreover, our + Shared substrate control (Tables 2 & 10) further shows that providing an explicit multi-relational graph memory alone is insufficient without evolving the retrieval programs that traverse it.

W3 & Q (LoCoMo Subset Construction, Reproducibility, and Split Sensitivity)

Thank you for raising this question.

  • 1. What an "example" denotes: In Appendix D.1, each "example" refers to a single QA pair (one question and its ground-truth answer). Therefore, 233, 152, and 314 are the numbers of QA pairs in the train, validation, and test sets, respectively (699 QA pairs in total).
  • 2. How the conversation-level split was obtained: Similar to recent evolving-memory work (such as MemSkill), we partition LoCoMo at the conversation level so that no conversation history is shared across splits. To keep iterative rollout costs manageable, we use 5 complete conversations from LoCoMo-10 in a 2:1:2 split: conv-26 (152 QAs) and conv-30 (81 QAs) for training (233 examples), conv-41 for validation (152 examples), and conv-49 (156 QAs) and conv-50 (158 QAs) for testing (314 examples). Within these 5 conversations, no additional filtering is applied beyond excluding adversarial questions: all non-adversarial QA pairs across all four categories (single-hop, multi-hop, temporal, and open-domain) are kept in full (233 + 152 + 314 = 699).
  • 3. Sensitivity to the particular conversation-level split: Because train, validation, and test splits use completely separate conversations with different speakers and storylines, the evolved skills are already tested on unseen dialogues (conv-49 and conv-50). More importantly, in LongMemEval* (Table 1), we take the skills and router evolved on these exact LoCoMo conversations (conv-26, 30, 41) and apply them zero-shot to LongMemEval without any re-training, where ERSkill still outperforms all baselines trained directly on LongMemEval (59.39 vs. 55.83 on Qwen; 56.35 vs. 50.25 on GPT). This confirms that the evolved skills capture generalizable retrieval patterns rather than depending on a particular LoCoMo conversation split.

We hope our response has addressed your concerns, and we will gladly and promptly respond to any further questions you may have.

Really appreciate the authors' detailed response.

I acknowledge the methodological design of ERSkill and the additional experimental work provided. However, my core concern remains: the response has not sufficiently demonstrated that ERSkill offers a clear and substantive methodological innovation over existing approaches to adaptive retrieval and skill evolution. While the authors point out certain differences at the implementation level, these distinctions are not enough to fully address my concern.

  1. Regarding SelRoute: ERSkill organizes retrieval operations into reusable programs, selects these programs based on the query, and uses task feedback to refine both the candidate programs and the selection mechanism. The selection of retrieval strategies conditioned on a query has already been explored in works like SelRoute, and the broader framework of co-evolving a skill library and a selection policy based on task feedback bears a clear resemblance to MemSkill (which the paper itself discusses). While shifting the focus of evolution from "memory-construction skills" to "memory-retrieval programs" is practically meaningful, I do not view this as a methodological advancement beyond an incremental extensio, essentially applying an existing skill-evolution framework to the retrieval stage.

  2. Regarding MemR³ and MemRetriever: The authors highlight differences concerning online iterative control, model parameter training, and the offline evolution of retrieval programs. Yet, these primarily represent different optimization approaches; they do not, in themselves, establish that ERSkill provides a significant, distinct contribution in terms of methodological innovation or capability.

In summary, while the complete system proposed in ERSkill outperforms its own ablated variants, this does not sufficiently establish its innovation relative to existing literature. I appreciate the additional experiments and clarifications, which help demonstrate the effectiveness of the system's individual components. However, they do not adequately clarify how much of the paper's methodological contribution stems from genuinely novel mechanisms, as opposed to the combination of existing retrieval operations, skill selection strategies, and feedback-driven evolution processes.

Consequently, despite the clarifications provided by the authors, my assessment of the paper's novelty remains unchanged. The accumulation of several localized techniques does not, in itself, constitute a breakthrough in methodological innovation. Based on the current manuscript and the authors' response, I maintain that the methodological innovation and the evidence supporting its incremental contribution, does not meet the acceptance criteria for ICLR.

Reviewer_LphXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies the problem of memory systems in LLM agents. These models often employ retrieval mechanisms to retrieve memories in response to user queries. The authors argue that this approach is static and limits performance. As a result, the authors introduce ERSkill, a retrieval framework that allows for evolving retrieval strategies through skill-guided memory access. Instead of using a single retrieval mechanism, the method employs executable skills as programs composed of retrieval primitives. This requires training a router that selects which skill to use for a given query. To support this process, the authors propose an experience trie data structure to record explored programs and a double-frontier mechanism to manage exploration and deployment. This results in increased performance on memory benchmarks.

Critical strengths
  1. The motivation is clear and the problem is interesting.
  2. The approach is interesting: retrieval skills are executable programs composed of reusable primitives, and the skills and router co-evolve.
  3. The method shows promising results across three memory benchmarks and two LLMs, outperforming baselines.
Critical weaknesses

My main concern with this work is clarity and unclear impact of the validation set on the final performance.

  1. The dataset splits are insufficiently described. For LoCoMo, the authors report 699 examples across train/validation/test, whereas the standard non-adversarial benchmark contains 1,540. Interestingly, the difference matches the number of single-hop questions, although the authors report single-hop results in Figure 4. The authors should clarify which examples were used, how the splits were constructed, and provide the exact splits for reproducibility.
  2. The paper lacks a comparison with prior work on autonomous memory retrieval, eg TA-Mem (arXiv:2603.09297), which uses an LLM agent to dynamically select retrieval tools and is also evaluated on LoCoMo. A comparison using the same retrieval primitives would help. Right now comparing them isn't clear due to (1).
  3. The paper would benefit from qualitative examples illustrating how retrieval skills evolve and how the resulting changes improve evidence retrieval. Now, Figures 3 and 7 show skill structures but it doesn't fully show behavior
Questions for authors

See above.

4 comments

We sincerely thank you for your positive assessment of our motivation, executable skill-program design, co-evolution mechanism, and empirical results. Below, we address your concerns in detail.

W1 (Dataset Splits and Impact of the Validation Set)

Thank you for pointing this out. We clarify how the LoCoMo examples are split and how the validation set is used:

  • 1. Dataset Splits: In Appendix D.1, each example refers to one QA pair. Similar to recent evolving-memory work (such as MemSkill), we partition LoCoMo at the conversation level so that no conversation history is shared across splits. To keep iterative rollout costs manageable, we use 5 complete conversations from LoCoMo-10 in a 2:1:2 split: conv-26 (152 QAs) and conv-30 (81 QAs) for training (233 examples), conv-41 for validation (152 examples), and conv-49 (156 QAs) and conv-50 (158 QAs) for testing (314 examples), totaling 699 QAs. Within every selected conversation, all four non-adversarial question categories (single-hop, multi-hop, temporal, and open-domain) are kept in full.
  • 2. Role and impact of the validation set: During evolution, the training split is used to diagnose failures, propose candidate skills, and train the router, while the validation split acts only as a gatekeeper to check whether a new candidate skill improves routed validation performance before updating the deployed skill pool. Furthermore, to verify that ERSkill does not rely on overfitting to a specific validation set, we transfer the skills and router evolved on LoCoMo directly to LongMemEval without using any LongMemEval training or validation data, and still outperform baselines evolved and evaluated on LongMemEval (59.39 vs. 55.83 on Qwen; 56.35 vs. 50.25 on GPT).

W2 (Comparison with Autonomous Memory Retrieval such as TA-Mem and Same-Primitive Controls)

Thank you for pointing us to agentic memory retrieval. We clarify (i) how test-time adaptive/agentic retrieval is already represented in our baseline suite, (ii) the conceptual difference between TA-Mem and ERSkill, and (iii) our controlled comparisons under the exact same retrieval primitives:

  • 1. Coverage of test-time adaptive/multi-step retrieval in Table 1 (Adaptive-RAG): Our evaluation in Table 1 already includes a dedicated Adaptive Retrieval category (such as Adaptive-RAG), which similarly decides retrieval strategies and iterative multi-step search at test time based on each query, and ERSkill substantially outperforms it across all benchmarks.
  • 2. Conceptual difference (Test-time online selection over fixed tools vs. Offline program evolution): Both TA-Mem and Adaptive-RAG rely on an LLM at inference time to dynamically choose or iterate over a fixed, pre-defined set of retrieval tools/workflows for each query, which incurs multi-turn LLM overhead at test time while leaving the underlying retrieval tools static. In contrast, ERSkill evolves new multi-step retrieval programs offline from execution traces and pairs them with a lightweight co-evolved router, enabling single-pass execution at test time with low inference cost (1.95k tokens/QA, Table 3).
  • 3. Controlled comparison using the exact same retrieval primitives (+ Shared substrate in Tables 2 & 10): Furthermore, to directly test whether competing mechanisms can match ERSkill when given the exact same memory store and retrieval primitives, Tables 2 and 10 equip representative baselines (Dynamic Cheatsheet, ReasoningBank, and GEPA) with ERSkill's exact retrieval primitives. Even under the identical retrieval primitives, all competing methods still lag behind ERSkill by 5.28 to 13.22 Average L-J points (55.83 best vs. 61.11 on GPT; 53.20 best vs. 61.30 on Qwen).

W3 (Qualitative Example of Skill Evolution and Step-by-Step Retrieval Behavior)

Thank you for this helpful suggestion. We add a step-by-step qualitative case study in Appendix F.2 tracing how an initial seed skill evolves during training and how the evolved program recovers missing gold evidence on an unseen LoCoMo test query:

  • 1. From Training Failure Diagnosis to Structural Evolution: During training, our analyzer diagnosed that the seed skill entity-focus (a single entity_search call) retrieved broad entity matches in multi-session dialogues (often landing on follow-up turns with unresolved pronouns) without capturing the underlying relational context. Based on this diagnosis, ERSkill evolved the single-step seed into the multi-step skill entity-relational-inference: entity_search -> llm_process -> relation_expand -> llm_process.

  • 2. Step-by-Step Evidence Recovery on an Unseen Test Query (conv-49, QA 16):

    • Query: "What recurring issue frustrates Sam at the grocery store?" (Gold answer: "Malfunctioning self-checkout machines"; Gold evidence turn: D22:19).
    • Step 1 (entity_search, 0/2 gold turns): Retrieves turn D22:21 ("Apparently I attract that to me, if you ever want to be in that situation, call me at the store with you!"), which mentions the store but leaves "that situation" unresolved, so a single-step entity search fails to identify the issue.
    • Step 2 (First llm_process): Inspects the retrieved turns and explicitly identifies the information gap: "What specifically is Sam attracting or frustrated by at the store?"
    • Step 3 (relation_expand, recovers gold turn D22:19): Follows a temporal_neighbor edge in the memory graph from D22:21 back to the preceding turn D22:19: "I went to the store again and, unsurprisingly, had issues with the self-checkout. It's becoming a regular annoyance."
    • Step 4 (Second llm_process & Answer): Extracts the structured evidence (recurring_issue: self-checkout problems, frequency_indicator: regular annoyance), enabling the reader model to produce the correct answer "issues with the self-checkout" (L-J = 1).

Overall, this case demonstrates the advantage of ERSkill: instead of relying on a fixed retrieval pipeline, evolution yields effective skill templates that allow the system to identify missing context, expand across related memory turns to recover the evidence, and answer complex queries accurately.

We hope our response has addressed your concerns, and we will gladly and promptly respond to any further questions you may have.

Dear reviewer LphX,

Thanks for your time and effort in reviewing ICLR2027 submissions. The authors have submitted their responses to your review. Please take the time to read, raise any further comments, and discuss them with the authors. Thanks very much!

AC

Thank you for the clarifications. But I'm still concerned about the missing comparison with TA-Mem. I understand the methodological differences you explained, but TA-Mem allows an LLM to select and use retrieval tools as needed. Why not compare against it, ideally using the same retrieval primitives? It seems like a direct comparison point for establishing the benefits of evolving retrieval skills.

Regarding the splits, I now understand that the experiments use only 5 of the 10 LoCoMo conversations, with just 2 conversations for testing. This seems quite limited, especially since the results aren't evaluated across different conversation splits.

Because of these reasons I'll maintain my score

Reviewer_N8E5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ERSkill, which treats memory retrieval as something the agent learns. It builds retrieval skills by chaining fixed building blocks (entity, keyword and embedding search, ...), and trains a router that picks one skill per query. Skills and router evolve together during training. An experience trie records which skill programs have already been tried, and only accepts new skills if routed validation performance holds. On LoCoMo, LongMemEval, the paper reports gains over non-evolving, adaptive-retrieval and evolving baselines.

Critical strengths

1- Treating retrieval itself as something that evolves and is learned is an interesting framing and nice contribution. Skills written as short programs of shared steps are easy to read and reuse.

2- The analysis is careful within its own setup. It includes ablations of each component.

Critical weaknesses

1- Missing standard baselines. The paper does not compare against the systems most often reported on LoCoMo: Mem0/Mem0-graph, Zep, MemGPT, or recent graph and episodic memories such as Synapse and AriGraph.

2- Despite missing some important baseline, the paper compares against some irrelevant baseline which has nothing to do with memory use. I don't understand the significance of comparing against GEPA which is a prompt optimization technique in this work and missing simple obvious baseline like MemGPT

3- The gain over the comparing methods looks small. Published numbers from other papers are not directly comparable, since they use a different answer model but they are close. ERSkill’s LoCoMo judge score (70.1 and 72.9) roughly equals the full-context judge score reported in the Mem0 paper (72.9, GPT-4o-mini). The gains also smaller than reported gains by Synapse.

4- no code and supplementary material are provided to make it more difficult to access the validity of the claims beyond the reports in the paper.

4- The cost relative to cheaper alternatives isn’t justified. Specially comparing Evolution cost of 9.5M token for ERSkill vs .2-.3 for Reasoning Bank and Dynamic cheatsheet. It is unclear this cost buys a meaningful improvement.

5- The router’s contribution is minimal. On LoCoMo, always using Skill 1 reaches vary close to router's performance.

Questions for authors

Can you report the performance of the missing baselines mentioned in the weakness section?

3 comments

We sincerely thank you for your positive assessment of our framing and component analysis. Below, we address your concerns in detail.

W1, W2, & Q1 (Baseline Selection, Coverage of Memory Paradigms, and Role of GEPA)

Thank you for raising this question. We clarify both how the suggested memory baselines relate to our evaluated baselines and why GEPA is included as a controlled baseline:

  • 1. Coverage of the suggested memory baselines in Table 1: Rather than omitting standard memory families, we have evaluated recent representative systems covering the exact same architectural categories you suggested:
    • OS-inspired hierarchical memory (MemGPT): Directly represented by MemoryOS[1], which follows and extends the OS-inspired memory paging and management paradigm.
    • Graph-linked and episodic/semantic memory (Mem0-graph, Zep, Synapse, AriGraph): Directly represented by A-Mem[2] (dynamically linked agentic memory graphs).
    • Fact extraction and memory consolidation (Mem0): Directly represented by MemSkill[3] (skill-guided memory construction and organization) and LightMem[4] (streamlined extraction, compression, and offline consolidation).
  • 2. Core architectural difference (Memory Storage Construction vs. Retrieval Evolution): Systems such as Mem0, Zep, MemGPT, Synapse, and AriGraph focus on how memory is extracted, indexed, and stored (e.g., building better hierarchical tiers or temporal/episodic graphs), while using a fixed, hand-crafted retrieval procedure at query time. In contrast, ERSkill addresses a different problem: given a structured memory store, how to automatically evolve and route multi-step retrieval programs at query time.
  • 3. Role of GEPA in our evaluation (W2): GEPA is a representative trace-reflective agent evolution framework that iteratively mutates and selects instructions/prompts using execution feedback and Pareto candidate search. Because ERSkill's core story sits at the intersection of long-term memory and agent self-evolution, evaluating only static memory systems would leave a critical question unanswered: to improve memory utilization through evolution, do we really need to evolve retrieval programs and a router, or is reflective prompt evolution already sufficient? Along with ReasoningBank, Dynamic Cheatsheet, and MemSkill, GEPA serves as the primary Self-Evolving baseline to answer this question (Table 1). Furthermore, we also conduct a controlled comparison under the exact same memory store and retrieval primitives (+ Shared substrate in Table 2), where ERSkill still outperforms GEPA by +5.28 to +8.10 Average L-J at less than half the evolution token cost (9.488M vs. 24.256M, Table 3), confirming the necessity of evolving executable retrieval skills over prompt optimization.

[1] Kang, Jiazheng, et al. "Memory os of ai agent." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. [2] Xu, Wujiang, et al. "A-mem: Agentic memory for llm agents." Advances in Neural Information Processing Systems 38 (2026): 17577-17604. [3] Zhang, Haozhen, et al. "Memskill: Learning and evolving memory skills for self-evolving agents." arXiv preprint arXiv:2602.02474 (2026). [4] Fang, Jizhan, et al. "Lightmem: Lightweight and efficient memory-augmented generation." International Conference on Learning Representations. Vol. 2026. 2026.

W3 (Magnitude of Performance Gains and Unified Re-Evaluation)

As you rightly noted, published numbers from other papers (such as Mem0 or Synapse) cannot be compared directly because they use different backbone LLMs, different judge models/prompts, and different data splits. Precisely to avoid such confounded cross-paper comparisons, we reproduced all baseline methods under the exact same backbone models, data splits, retrieval budgets, and evaluation protocol in Table 1.

Under this controlled, apples-to-apples evaluation, ERSkill's gains over the comparing methods are substantial:

  • On LoCoMo, ERSkill outperforms the strongest baseline by +5.10 L-J points on Qwen (70.06 vs. 64.96) and +13.38 L-J points on GPT-5.4-nano (72.93 vs. 59.55).
  • Across all datasets on average, ERSkill outperforms the strongest baseline by +7.86 L-J points on Qwen (61.30 vs. 53.44) and +7.40 L-J points on GPT (61.11 vs. 53.71), alongside a +11.54% to +13.14% improvement in gold evidence retrieval recall and hit rate (Table 5).

W4a (Code and Supplementary Material)

Both the code and the supplementary material are already included in our submission:

  • Code repository: Provided in the Reproducibility Statement on Page 10 (https://anonymous.4open.science/r/ERSkill_anony-08A3).
  • Supplementary material: Included directly in the submitted PDF as Appendices A–G (Pages 12–30), covering memory construction and primitives (Appendix A), evolution algorithms and prompts (Appendix B), theoretical proof (Appendix C), experimental settings (Appendix D), and additional experiments and case studies (Appendices E–G).

W4b (Evolution Cost)

Thank you for raising this practical question. Comparing the token cost and accuracy trade-off in Tables 1 and 3 clarifies why the offline evolution cost is worthwhile:

  • 1. Why ReasoningBank and Dynamic Cheatsheet cost 0.2–0.3M, and what ERSkill's 9.5M buys: ReasoningBank and Dynamic Cheatsheet only summarize past trajectories into text notes in a single pass without evolving retrieval strategies, which caps their LoCoMo accuracy at 57.96 and 59.55 L-J. In contrast, ERSkill evolves retrieval skills and a query router to achieve a +13.38 to +14.97 L-J improvement (72.93 vs. 59.55 / 57.96), while still costing less than half the evolution tokens of comparable skill-evolution methods such as MemSkill (20.37M, L-J 58.59).
  • 2. One-time offline cost with zero-cost cross-dataset transfer (LongMemEval, Table 1):* Evolution is paid only once. Moreover, the skills and router evolved on LoCoMo can be directly transferred zero-shot to new benchmarks (LongMemEval* in Table 1) with 0 additional evolution tokens, while still outperforming baselines trained directly on LongMemEval (59.39 vs. 55.83 on Qwen; 56.35 vs. 50.25 on GPT).
  • 3. Nearly 5$\times$ lower online inference cost per query (Table 3): Because Dynamic Cheatsheet repeatedly stuffs accumulated text notes into the prompt at test time, it consumes 9.08k tokens on every user query. By contrast, ERSkill executes compact retrieval programs and requires only 1.95k tokens per query at inference time (nearly 5$\times$ cheaper than Dynamic Cheatsheet and 2.3–2.8$\times$ cheaper than A-Mem and MemoryOS), making it much more economical in ongoing deployment.

Overall, ERSkill's one-time evolution cost is modest compared to peer skill-evolution methods and pays off with substantial accuracy gains, zero-shot cross-dataset transferability, and much lower per-query inference cost.

W5 (Contribution of the Router vs. Always Using Skill 1)

Thank you for pointing this out. Looking at Table 7 alongside the router ablation in Table 4 clarifies the role and contribution of our router:

  • 1. Why Skill 1 alone already scores high (70.56), and what the router adds (72.93): Through evolution, Skill 1 emerges as a strong general-purpose retrieval program that covers the majority query pattern (55.73% routing share in Table 7). However, a single retrieval program cannot optimally handle all query types (e.g., temporal vs. multi-hop queries). By routing the remaining 44.27% of queries to specialized complementary skills (Skills 2–4, which alone score 57.64–68.15), the router further boosts performance from 70.56 to 72.93 (+2.37 L-J), outperforming every single-skill policy across all metrics (L-J, F1, and B1).
  • 2. The router actively drives skill evolution rather than only dispatching at test time (w/o Router in Table 4): In ERSkill, the router and skills co-evolve: by matching queries to skills according to each query's retrieval needs, the router helps the evolution loop pinpoint which query types are poorly served by existing skills and guides newly mutated skills to specialize on those underserved cases. Consequently, when we remove our learned router (w/o Router in Table 4), the system loses both need-aware guidance during evolution and accurate skill matching at inference, causing LoCoMo L-J to drop sharply from 72.93 to 60.19 (-12.74 points on GPT) and from 70.06 to 58.28 (-11.78 points on Qwen).

In short, the router not only pushes test-time accuracy beyond the best single skill by dispatching heterogeneous queries to specialized skills, but also guides the evolution process itself by aligning skill refinement with specific query needs.

We hope our response has addressed your concerns, and we will gladly and promptly respond to any further questions you may have.

Thank you for the response, and for pointing me to the code location and the appendices. I missed them, and that concern is resolved. I also accept the motivation for including GEPA as a self-evolving baseline. However, my main concerns about insufficient baselines and the small margin of improvement still remain. Using one method to stand in for a whole family (e.g., A-Mem for graph memory) is not the same as comparing against the strongest systems. A-Mem is among the weakest graph memories in published results, while Zep, Mem0-graph and Synapse report much higher numbers, and Mem0 is open source and could easily be run under your protocol. If storage and retrieval are really orthogonal as you argue, it would also help to show ERSkill on top of a stronger memory store such as Mem0 or Synapse.

The w/o Router ablation replaces the router with LLM-based selection, which scores below always using Skill 1, so reporting the best single-skill score in that run would help separate the router’s role during evolution from its role at inference. Given the small test sets, confidence intervals for the main judge-score differences would also be needed to show the gains are not noise. I am keeping my score as main concerns are not resolved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.