- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents ERSkill, a framework that improves how LLM agents retrieve information from long-term memory. Instead of using a fixed retrieval method, ERSkill builds retrieval skills by combining basic operations such as search, evidence expansion, and LLM-based processing. For each query, a learned router selects the most suitable skill to retrieve relevant evidence.
During training, ERSkill jointly improves the retrieval skills and the router. An experience trie records previously tested skills to avoid repeated exploration. A double-frontier mechanism maintains useful candidate skills while ensuring that only skills that work well with the router are deployed.
Experiments on LoCoMo, LongMemEval, and PerLTQA show that ERSkill outperforms several non-evolving, adaptive-retrieval, and evolving-agent baselines with two LLM backbones. The ablation studies and controlled experiments further suggest that the improvements come from evolving and selecting retrieval skills, rather than only from using a richer memory structure.
Critical strengths
- The proposed method is well designed. Retrieval skills are executable and interpretable, while the experience trie and double-frontier mechanism help control the evolution process.
- The experiments are comprehensive. The paper evaluates the method on three benchmarks and two LLM backbones, with useful ablation studies, controlled comparisons, and efficiency analysis.
Critical weaknesses
-
The paper should discuss SelRoute, which also routes queries to specialized retrieval pipelines. Please clarify how ERSkill differs from this line of work and what its evolving skill compositions add beyond selecting among predefined retrieval strategies.
-
The baseline comparison should include recent methods for adaptive, multi-step memory retrieval, such as MemR³, MAGMA, and MemRetriever. Please clarify how ERSkill differs from these approaches and, where feasible, compare them under a shared evaluation setup.
-
Appendix D.1 explains the conversation-level split and the exclusion of adversarial LoCoMo questions, but does not clarify how the final split sizes were obtained. Please specify any additional filtering or sampling, what counts as an “example,” and provide the conversation IDs or split files and random seed to support reproducibility.
Questions for authors
The construction of the final LoCoMo subset remains unclear. After excluding adversarial questions, the public LoCoMo-10 benchmark is commonly evaluated on approximately 1,540 questions, whereas the reported train, validation, and test sets contain only 233, 152, and 314 examples, respectively, for a total of 699.
Could the authors clarify what additional filtering or sampling was applied, what “example” denotes in this section, and how many conversations are assigned to each split? Since LoCoMo contains only a small number of underlying conversations, it would also be useful to discuss the sensitivity of the results to the particular conversation-level split.
3 comments
W3 & Q (LoCoMo Subset Construction, Reproducibility, and Split Sensitivity)
Thank you for raising this question.
- 1. What an "example" denotes: In Appendix D.1, each "example" refers to a single QA pair (one question and its ground-truth answer). Therefore, 233, 152, and 314 are the numbers of QA pairs in the train, validation, and test sets, respectively (699 QA pairs in total).
- 2. How the conversation-level split was obtained: Similar to recent evolving-memory work (such as MemSkill), we partition LoCoMo at the conversation level so that no conversation history is shared across splits. To keep iterative rollout costs manageable, we use 5 complete conversations from LoCoMo-10 in a 2:1:2 split: conv-26 (152 QAs) and conv-30 (81 QAs) for training (233 examples), conv-41 for validation (152 examples), and conv-49 (156 QAs) and conv-50 (158 QAs) for testing (314 examples). Within these 5 conversations, no additional filtering is applied beyond excluding adversarial questions: all non-adversarial QA pairs across all four categories (single-hop, multi-hop, temporal, and open-domain) are kept in full (233 + 152 + 314 = 699).
- 3. Sensitivity to the particular conversation-level split: Because train, validation, and test splits use completely separate conversations with different speakers and storylines, the evolved skills are already tested on unseen dialogues (conv-49 and conv-50). More importantly, in LongMemEval* (Table 1), we take the skills and router evolved on these exact LoCoMo conversations (conv-26, 30, 41) and apply them zero-shot to LongMemEval without any re-training, where ERSkill still outperforms all baselines trained directly on LongMemEval (59.39 vs. 55.83 on Qwen; 56.35 vs. 50.25 on GPT). This confirms that the evolved skills capture generalizable retrieval patterns rather than depending on a particular LoCoMo conversation split.
We hope our response has addressed your concerns, and we will gladly and promptly respond to any further questions you may have.
Really appreciate the authors' detailed response.
I acknowledge the methodological design of ERSkill and the additional experimental work provided. However, my core concern remains: the response has not sufficiently demonstrated that ERSkill offers a clear and substantive methodological innovation over existing approaches to adaptive retrieval and skill evolution. While the authors point out certain differences at the implementation level, these distinctions are not enough to fully address my concern.
-
Regarding SelRoute: ERSkill organizes retrieval operations into reusable programs, selects these programs based on the query, and uses task feedback to refine both the candidate programs and the selection mechanism. The selection of retrieval strategies conditioned on a query has already been explored in works like SelRoute, and the broader framework of co-evolving a skill library and a selection policy based on task feedback bears a clear resemblance to MemSkill (which the paper itself discusses). While shifting the focus of evolution from "memory-construction skills" to "memory-retrieval programs" is practically meaningful, I do not view this as a methodological advancement beyond an incremental extensio, essentially applying an existing skill-evolution framework to the retrieval stage.
-
Regarding MemR³ and MemRetriever: The authors highlight differences concerning online iterative control, model parameter training, and the offline evolution of retrieval programs. Yet, these primarily represent different optimization approaches; they do not, in themselves, establish that ERSkill provides a significant, distinct contribution in terms of methodological innovation or capability.
In summary, while the complete system proposed in ERSkill outperforms its own ablated variants, this does not sufficiently establish its innovation relative to existing literature. I appreciate the additional experiments and clarifications, which help demonstrate the effectiveness of the system's individual components. However, they do not adequately clarify how much of the paper's methodological contribution stems from genuinely novel mechanisms, as opposed to the combination of existing retrieval operations, skill selection strategies, and feedback-driven evolution processes.
Consequently, despite the clarifications provided by the authors, my assessment of the paper's novelty remains unchanged. The accumulation of several localized techniques does not, in itself, constitute a breakthrough in methodological innovation. Based on the current manuscript and the authors' response, I maintain that the methodological innovation and the evidence supporting its incremental contribution, does not meet the acceptance criteria for ICLR.
We sincerely thank you for your positive assessment of our method design and experiments. Below, we address your concerns in detail.
W1 & W2 (Comparison with SelRoute, MemR³, MAGMA, and MemRetriever)
Thank you for pointing us to these recent works. We clarify both how ERSkill differs from them conceptually and how their core paradigms are already covered in our experimental comparisons: