- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ExpVoyager proposes dynamic skill synthesis through active navigation of accumulated agent trajectories. Instead of constructing a fixed skill library or retrieving only top-ranked experiences, a curator navigates trajectory-level and step-level views, maintains structured knowledge and unresolved questions, and generates a task-conditioned skill for a frozen executor. The approach is evaluated on ALFWorld, WebShop, and ScienceWorld.
Critical strengths
-
The paper addresses an important information-loss problem in agent memory systems: fixed procedural abstractions can discard details needed by future tasks. The distinction between retrieving summaries and actively inspecting original experiences is meaningful.
-
The experiments demonstrate consistent gains across several environments and executor backbones. On ALFWorld with Qwen3.5-9B, ExpVoyager improves success to 63.7%, compared with 51.1% for SkillTTA. Component ablations and matched-budget agentic retrieval comparisons provide additional evidence for the proposed navigation mechanism.
Critical weaknesses
W1. Substantial test-time overhead relative to the marginal performance gains.
The default configuration performs 20 navigation rounds and consumes approximately 385K end-to-end tokens per ALFWorld task, compared with approximately 52K for the base agent. More importantly, the five-round configuration already achieves 60.4% success using approximately 159K tokens, while the 20-round configuration reaches 63.7%. Thus, more than doubling the token cost produces only a 3.3-percentage-point improvement. The practical value of the default navigation budget is therefore questionable, particularly for latency-sensitive or frequently repeated tasks.
W2. The distinction from general-purpose agentic retrieval remains insufficiently established.
The key mechanisms—iterative tool-based retrieval, maintaining accumulated knowledge, tracking unresolved questions, and adapting subsequent search queries—overlap considerably with established agentic search and structured reasoning workflows. The paper includes comparisons against iterative retrieval baselines under matched experience-access budgets, which is valuable. However, these baselines do not appear to receive the same fine-grained trajectory interface and structured navigation state simultaneously. Consequently, the relative contribution of the proposed navigation formulation remains incompletely isolated.
W3. Limited adaptation to new evidence encountered during task execution.
ExpVoyager synthesizes a task-conditioned skill before the executor begins interacting with the target environment. Once execution starts, the generated skill is effectively fixed. This creates a mismatch between experience navigation and execution feedback: newly observed failures or environmental conditions cannot directly trigger further evidence gathering and skill revision. The method may therefore be less effective in environments where critical uncertainties only become apparent during execution.
W4. The framework lacks explicit validation of synthesized procedural knowledge.
The curator derives procedural guidance from prior trajectories and maintains knowledge items and unresolved questions through language-model-based updates. However, identifying a pattern in previous experience does not guarantee that the pattern is causally valid or applicable to the current task. The paper does not systematically test robustness to contradictory, misleading, or low-quality trajectories, nor does it provide an explicit verification mechanism for newly synthesized guidance. This leaves the reliability of experience transfer uncertain.
W5. Cold-start behavior reveals a potentially important negative-transfer problem.
The online self-evolution experiments show that ExpVoyager can provide little benefit or even degrade performance when the available experience pool is small. This indicates that the method is not uniformly beneficial throughout continual deployment. Although its performance improves as experience accumulates, the paper does not establish a principled criterion for determining when the available evidence is sufficient to justify navigation. An adaptive fallback to the base agent or a confidence-aware navigation policy could be important for practical deployment.
W6. Scalability claims are stronger than the demonstrated retrieval setting.
The experience-scaling experiments use nested subsets of a pool containing up to approximately 1,000 trajectories. The navigation interface relies on field-specific regular-expression search and full-trajectory inspection. It is unclear how efficiently this design would scale to much larger, heterogeneous experience repositories. In particular, retrieval latency, indexing cost, and navigation quality under a substantially expanded corpus are not systematically evaluated.
W7. Dependence on rich recorded trajectories limits applicability.
The experience interface assumes access to detailed records containing observations, actions, outcomes, and often the executor's recorded reasoning. In realistic deployments, such internal reasoning traces may be unavailable, incomplete, inconsistent, or generated by different models. The method does not sufficiently evaluate whether its advantages persist with action-observation-only logs, highly compressed records, or trajectories collected by heterogeneous agents.
Questions for authors
-
Can the authors report a complete performance–cost Pareto frontier, including latency, curator calls, and token consumption? Is there an adaptive stopping rule that approaches the 20-round performance with substantially fewer navigation operations?
-
How much of the gain is preserved when a strong agentic retrieval baseline receives the same trajectory/step-level interface and maintains an explicit knowledge-and-open-questions state?
2 comments
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts, particularly the stronger retrieval baselines and additional evaluations on scalability and heterogeneous experience. However, I still have substantial concerns regarding the novelty and practical efficiency of the proposed method. The distinction from existing agentic retrieval approaches remains insufficiently clear, and the considerable test-time computational overhead is not fully justified by the performance gains. Furthermore, the lack of execution-time adaptation and the remaining issues with knowledge reliability limit the method's practical applicability. Overall, while the additional experiments strengthen the empirical evidence, they do not fully resolve my concerns about the methodological contribution and generality. Nevertheless, considering the authors' substantial efforts and the additional evidence provided, I am willing to raise my rating from 1 (Clear Reject) to 2 (Weak Reject).
Follow-up on our responses
Dear reviewer gWH7,
Thank you again for your detailed review and valuable feedback. Your comments have helped us strengthen our evaluation and clarify the scope and limitations of our approach. We hope our responses and additional analyses help address your concerns. If any concerns remain, we would appreciate the opportunity to discuss them further.
Best regards,
Authors of Paper #37719