- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper introduces SkillPilot, a framework that reframes skill library adaptation for frozen LLM agents as a tree-search problem via Meta-Optimizer-Guided Monte Carlo Evolution Tree Search (MCETS). To overcome error accumulation in traditional single-path editing and resolve multi-skill failure attribution, it employs UCB1-based path exploration, a hierarchical meta-optimizer that clusters failure trajectories into local semantic gradients across target sections and code blocks, and depth-aware backpropagation. Evaluated across ALFWorld, WebShop, and SkillsBench on models like DeepSeek-V3.2, Qwen-3.5-Flash, and Doubao-Seed 2.0 Mini, SkillPilot significantly enhances task success rates and execution efficiency over greedy single-path baselines while ensuring cross-task skill reusability.
Critical strengths
- You release the anonymous code with a clear structure.
- You propose an innovative method to evolve skills.
- The paper structure is very clear, and there's no single word on a line by itself. The whole paper looks very comfortable to read.
- You evaluate on three benchmarks. I think the experiment is generally comprehensive.
Critical weaknesses
- The baselines you compared on SkillsBench are very limited. Can you compare more of the related baselines you mentioned in the related work section, eg EvoSkillBank, CoEvoSkills, SkillForge? Especially the baselines compared on ALFWorld and WebShop; it seems like the baselines are different.
- The reference format for SkillForge and SkillClaw seems like there's a little problem.
- It's very hard for LLMs to make the accurate failure attribution.
- MCETS requires running full multi-turn rollouts in the target environment for every candidate branch during simulation; as the tree depth and branching factor scale, evaluating these trajectories incurs severe execution latency and exponential token consumption.
- It requires deterministic, sandboxed reward functions or code-level evaluators to score rollouts, making it unfeasible for open-ended or subjective real-world tasks that lack programmatic verifiers.
Questions for authors
- What's the main points you want to express in Figure 1?
- How do you achieve the Trajectory Failure Attribution and Skill-to-Patch Attribution? For example, what attribution taxonomy do you design? And can the LLM judge accurately reflect the failure reasons?
7 comments
-
Weakness 3: "It's very hard for LLMs to make the accurate failure attribution."
a. The attribution in our framework is not an open-ended judgement but a constrained, hierarchical classification. We provide the attribution process below:
Level Problems to be solved Answer space Failure clustering which failed trajectories share the same root cause? merge/split over the failed trajectories; each cluster carries a root cause , the implicated skill indices and a fix direction Failure-to-skill attribution which invoked skill is responsible? , restricted to the skills actually invoked in the failing trajectory, decided by reading the skill description Skill-to-patch attribution which part of that skill is defective? , restricted to the pre-computed sections/code blocks of the targeted skill Gradient generation how should that part be changed? one concise edit direction per targeted patch; overlapping directions are merged (Eq. 5) The section partition itself is deterministic and model-independent: markdown
#-headings and code blocks, so the meta-optimizer selects among a small, enumerable set of candidate patches. In our experience this is where the reliability comes from: the hard judgement is the clustering, and the attribution of a cluster to a skill and a section is a selection over a handful of candidates that the LLM can verify against the trajectory text.b. The evidence that the attribution carries real signal is already in the paper. (i) Ablation: replacing the attributed proposals by unguided ones drops the ALFWorld Seen/Unseen success rate by 7.15/12.57 points — if the attribution were noise, removing it would not cost that much. (ii) Targeted, not diffuse, improvements: the reusability analysis (Sec. 3.4, Fig. 3) shows that the skills that are actually invoked most often are precisely the ones whose success rate improves (e.g. "object-placer" and "object-picker" on DeepSeek-V3.2, "object-placer" and "locate target object" on Doubao-Seed-2.0-Mini), and that the invocation pattern itself shifts in a model-specific way, which is the signature of diagnosed edits rather than random rewrites. (iii) We add attribution case studies to the appendix. Here we list two representative cases of failure attribution.
task_completion_detection (Failure due to suboptimal skills invocation)
- Count: 3
- Case indices: [1657, 1824, 2440]
- Suspected skill areas: ['task-verifier', 'alfworld-object-placer', 'alfworld-clean-object']
- Root cause: Agent fails to recognize task completion conditions and continues unnecessary actions.
- Impact: Wasted steps and inability to properly terminate successful trajectories.
- Fix directions:
- Add goal state verification checks
- Implement completion condition detection
- Improve termination logic
This example is generated by the base model DeepSeek V3.2, and demonstrates the effective failure attribution with the hierachical meta-optimizer. It clusters the failures and attributes them to the completion detection, and then raises some directions to fix for the subsequent skill patch attribution and generation. With this successful attribution, the verification capbility of the agent can be substantially enhanced, which is consistent with the results shown in Figure 3.(a), where the success rate and invocation rate of the skill s
task-verifierandobject-placerare improved substantially.object_grounding_failure (Failure due to skills understanding and task execution)
- Count: 5
- Case indices: [546, 1918, 2358, 2396, 3168]
- Suspected skill areas: ['alfworld-object-locator', 'alfworld-receptacle-preparer', 'alfworld-receptacle-navigator']
- Root cause: Agent fails to systematically locate target objects due to incomplete search strategies and inability to handle closed receptacles properly.
- Impact: Task stalls in initial search phase, preventing progression to core manipulation steps.
- Fix directions:
- Enhance systematic search algorithms for object location
- Improve handling of nested/closed containers
- Add object existence verification routines
This example is generated by the base model DeepSeek V3.2. The meta-optimizer clusters the failures and attributes them to the object grounding failures, and then raises some directions to fix for the subsequent skill patch attribution and generation. With this successful attribution, the capability of locating objects and navigation can be substantially enhanced, which is consistent with the results shown in Figure 3.(a), where the success rate and invocation rate of the skills
locate-target-objectandlocation-navigatorare improved substantially.We add this experiment to the case study in Appendix E.6.
-
Weakness 4: "MCETS requires running full multi-turn rollouts in the target environment for every candidate branch during simulation; as the tree depth and branching factor scale, evaluating these trajectories incurs severe execution latency and exponential token consumption."
a. We thank the reviewer for this careful observation. We believe this might be a misunderstanding of the evaluation protocol, so we will further explain it and make it explicit in Appendix C.1. The number of rollouts is linear, not exponential, in the size of the search: in every iteration the search expands one children and each child is simulated on a fixed support set of tasks ( for ALFWorld/WebShop, / for SkillsBench). The maximum number of children of each node is set to . The support set does not grow with the depth: a deeper node is scored on exactly the same tasks as a node at depth 1, and the support set of a round is fixed to the failures collected in that round. The total cost is therefore rollouts (for us: on ALFWorld/WebShop, on SkillsBench), which is linear in the number of iterations and independent of the tree depth; the depth is additionally capped () and the search stops when the budget is exhausted or no new valid edit can be generated.
b. The measured cost is favourable, and we will add the per-iteration accounting. On the six SkillsBench tasks of Table 5 that reach the same final reward, SkillPilot needs 8.83 rounds and 3.71M tokens against 10.67 rounds and 6.95M tokens for Skill-Evolver (about 47% fewer tokens); on ALFWorld and WebShop the token overhead of SkillPilot is lower than that of SkillOpt and Skill-Evolver for all three base models (Fig. 9), because the joint optimization re-uses one shared context instead of rewriting entire skill documents.
-
Weakness 5: "It requires deterministic, sandboxed reward functions or code-level evaluators to score rollouts, making it unfeasible for open-ended or subjective real-world tasks that lack programmatic verifiers."
a. We thank the reviewer for the insightful feedback. We fully agree, and we already state this in the Limitations section of the paper. MCETS is guided by the reward obtained in the simulation phase; in open-ended real-world settings (e.g. a personal assistant) constructing such a verifier is inherently subjective, the constructed verifier may deviate from the true objective, and the search may overfit it. We will strengthen this paragraph and make the limitation more prominent in the main text.
b. Two observations that qualify the severity of the limitation. (i) The requirement is a programmatic reward, not necessarily a sandboxed one: on WebShop we do not execute a code-level evaluator at all — the task metric (type / attribute / option / price matching) is provided to the agent in the prompt and the environment reports the corresponding reward, so the reward pipeline is identical in kind to the SkillsBench one but is not a pytest suite. (ii) The simulation reward is a pluggable component: an LLM-based verifier or a human preference signal can replace the deterministic verifier without touching the search algorithm, since MCETS only requires a scalar score per rollout. We list LLM-verifier-based simulation and human feedback explicitly as future work; these directions are outside the scope of the current paper, where we focus on the search structure.
-
Question 1: "What's the main points you want to express in Figure 1?"
Figure 1 carries the two motivating observations of the paper. (a) Where to evolve is model-specific. The same curated skill library induces very different invocation behaviours on different base models — the skill-invocation frequency and the error rate differ substantially per model (e.g. DeepSeek-V3.2 keeps the error rate below 50% and concentrates on reusing head skills) — so a static skill document leaves a model-specific gap that cannot be closed by a single best artifact. This is the motivation for adapting the library to the target model. (b) Which path to evolve on is an essential problem. On SkillsBench, randomly sampling five evolution paths per task shows that single-path evolution frequently lands on a suboptimal path, because errors accumulate and oscillate along a path; this is the motivation for searching over evolution paths rather than following one. Together the two panels motivate the two components of SkillPilot: a hierarchical meta-optimizer that decides where to adapt, and MCETS that decides which adaptation path to follow.
We are very grateful to the reviewer for the valuable feedbacks. Here we provide the detailed response:
Weakness 1: "The baselines you compared on SkillsBench are very limited. Can you compare more of the related baselines you mentioned in the related work section, eg EvoSkillBank, CoEvoSkills, SkillForge? Especially the baselines compared on ALFWorld and WebShop; it seems like the baselines are different."
a. We agree that the baseline sets should be unified, and we will extend the comparison. The ReAct baseline is equivalent to the agent execution with vanilla skills. We did not select Expel as the baseline since it needs to compare the successful and failed trajectories but it may help the agent to cheat by citing the successful records instead of refining the skills correctly. Due to the limited time of the rebuttal period, we add strong baselines EvoSkillBank and CoEvoSkills for skill evolution.
The results show that CoEvoSkills is capable of jointly optimizing multiple skills and leverages surrogate skill verifiers to strengthen skill verification; this alleviates, to some extent, the suboptimal-skill-evolution problem from a verification standpoint, but it still fails to fully escape suboptimal evolution paths. By contrast, SkillPilot steers the agent toward more promising evolution paths and delivers consistently stronger performance.
b. Why the baseline sets currently differ. The difference is dictated by what each method needs as input, and it is not a choice of favourable opponents. SkillOpt performs prompt-level adaptation on minibatches of trajectories (we use a reflection minibatch of 5 and a merge batch of 5); this requires a pool of trajectories of the same task distribution to reflect over. On ALFWorld and WebShop each task family provides such a pool (we sample 200 training tasks and collect K=20 failure trajectories), so the full baseline set (ReAct, Expel, Skills, SkillOpt, Skill-Evolver) is applicable. On SkillsBench, by contrast, every task is an isolated Docker environment with its own verifier and only T=5 (curated) / T=3 (self-generated) protocol trials, so there is no minibatch of trajectories for a given skill to reflect over, and pooling trajectories across tasks would violate the benchmark's per-task anti-leakage protocol. Table 2 therefore compares the benchmark's three protocols (Without Skills / Skills / Self-Generated Skills) with Skill-Evolver, the strongest reflective baseline that is applicable in that regime, and the newly added repository-level methods above.
Weakness 2: "The reference format for SkillForge and SkillClaw seems like there's a little problem."
Thank you for pointing this out. The author lists in the related-work paragraph were not consistently wrapped in parentheses (e.g., "SkillForge Liu et al. (2026c)" and "SkillClaw Ma et al. (2026)"). We will fix the formatting to "SkillForge (Liu et al., 2026c)" and "SkillClaw (Ma et al., 2026)".