- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes MetaSkill, a meta-reinforcement learning framework that trains LLM agents to adapt to unseen tool-using environments through limited interactions. The method consists of three stages involving environment exploration, extraction of reusable environment-specific skills, and skill-guided task execution. The authors introduce a stage-wise optimization strategy to train these components using downstream task rewards. Experiments on BFCL, τ-bench, τ²-bench, and ACEBench show improvements over the evaluated baselines, including on unseen environments and cross-benchmark transfer.
Critical strengths
-
The paper identifies a concrete limitation of existing agents in adapting to implicit tool conventions and operational rules in unseen environments.
-
The proposed stage-wise optimization offers a practical approach to credit assignment across exploration, skill extraction, and exploitation.
-
The evaluation covers multiple model sizes, unseen environments, and cross-benchmark transfer, with ablation studies supporting the effectiveness of the proposed training design.
Critical weaknesses
Although I like the work, several concerns remain, and I would appreciate the authors’ responses.
-
My main concern is the claim of generalizability. Without adaptation, MetaSkill shows limited additional gains on τ-bench, τ²-bench, and ACEBench and often performs similarly to RL and Experience-guided RL. Therefore, what does meta-skill training contribute beyond training-free approaches such as FAMA [1], which already use failure analysis to improve agent decisions? Please provide a comparison that establishes this benefit.
-
My second concern is reliability. In real service environments, a failed task can frustrate customers or affect the system, and repair may not be possible. Although MetaSkill is trained, its adaptation still uses interactions that may contain failures. How does it address settings where test-time trial and error is unacceptable? Does training improve first-attempt reliability, or mainly improve subsequent tasks after feedback?
-
I remain unconvinced that the results establish broad generalization learned during training. The largest improvements occur on BFCL, while two-shot gains over the strongest trained baseline on the transfer benchmarks are only 1.2 to 2.7 percentage points. Could the BFCL gains reflect shared tool-use logic between training and evaluation environments, while transfer improvements mainly depend on target-environment experience? Moreover, these small gaps may reflect evaluation variability. Please evaluate more than five runs, report confidence intervals, and compare decoding temperatures with equal tuning budgets across methods to establish whether the gains persist.
-
Several skill-based RL methods already exist. Therefore, the authors should compare MetaSkill with representative methods from this literature and clearly identify its algorithmic contribution. Without these comparisons, the paper does not sufficiently establish novelty at the algorithmic level.
-
Please specify which 17 BFCL environments were used for training and which three were held out, including task IDs and counts. Also clarify whether all training tasks come from public BFCL data or include additional constructed data. Since tasks can involve multiple tool classes, how are environments defined and separated? Without these details, the unseen-environment evaluation cannot be independently verified.
-
Please identify the user-simulator models and decoding settings for τ-bench, τ²-bench, and ACEBench. Although Appendix C.3 reports three evaluation runs, please clarify what varies between runs given the agent’s greedy decoding. For ACEBench’s agent portion, please also specify the evaluated subsets and scoring protocol.
Reference:
[1] FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments. https://arxiv.org/abs/2604.25135
Questions for authors
Please read the weaknesses.
Summary of Discussion and Manuscript Revisions
Dear Area Chair and Reviewers,
We thank Reviewers TJcG, wH8i, and YXTR for their careful reviews. We summarize our responses and additional evidence below. Manuscript revisions are marked in blue.
1. Core Contribution:
MetaSkill jointly trains exploration, skill extraction, and utilization through performance on new query tasks after learning from support tasks in the same environment. Its contribution is their coordinated ability to use environment experience for cross-task adaptation. We assess this ability through post-adaptation accuracy under the same support budget and inference pipeline, without test-time parameter updates.
2. Key Evidence and Additional Evaluations:
On BFCL unseen environments, Qwen3-8B MetaSkill, RL, and Experience-guided RL start at similar accuracy (42.9%, 42.8%, and 43.1%) but reach 50.2%, 44.2%, and 43.6% after three-shot adaptation (Table 1). Shared tool familiarity alone therefore does not explain MetaSkill's advantage.
We added five-run cross-benchmark evaluations with independently sampled support tasks and 90% confidence intervals. Without further parameter training, MetaSkill retains the highest mean post-adaptation accuracy across all transfer domains, both model sizes, and both adaptation budgets (Section 3.2, Tables 2–5). Representative Qwen3-8B two-shot results are:
The baseline is the better of RL and Experience-guided RL. Full tables report confidence intervals; Appendix C.3 specifies the evaluation protocol, including fixed agent and user-simulator temperatures of 0.0.
3. Point-by-Point Summary:
Reviewer TJcG
Reviewer wH8i
Reviewer YXTR
Thank you for your consideration. We welcome any further questions.