- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes OpenClaw-Skill, a pipeline for building skill-augmented training data and training small agent models for OpenClaw-style tasks. The core component, Collective Skill Tree Search (CSTS), decomposes a task into ordered subtasks. For each subtask, several frontier models produce trajectories, and a separate synthesizer distills these into candidate skill documents. Candidates are scored by a panel of judges (quality) and by whether other models can succeed using them (transferability). The selected skill path and teacher executions are used for SFT. The authors then introduce Collective Skill RL (CSRL), which samples rollouts conditioned on the top-K skills per subtask, optimizes a GRPO-style objective, and feeds the cross-skill advantage back into the node scores to revise the skill path. Experiments on QwenClawBench, PinchBench and ALFWorld show improvements over the base Qwen models and several skill-based RL baselines.
Critical strengths
- Using cross-model execution as a filter for skills is a sensible idea. It grounds skill selection in whether a skill actually helps a different model, rather than relying only on judge opinions, and the ablation in Appendix E.2 suggests it contributes to the final result.
- The evaluation targets realistic OpenClaw-style environments rather than only toy benchmarks. The gains are consistent across several backbones, including Gemma and MiniCPM in the appendix.
Critical weaknesses
- The paper frames itself as a skill construction method, but every reported result comes from a model that was also fine-tuned on GPT-5.4 teacher trajectories. As a result, I can’t tell how much of the gain comes from the skills and how much from distilling a strong teacher. At minimum I’d expect three controls: SFT on teacher trajectories without skills, the frozen skill bank injected into the base model with no training, and the trained model evaluated without skill injection. Without these, the central claim that CSTS produces high-quality, transferable skills isn’t really tested.
- The novelty is more limited than presented. The overall recipe (multiple models jointly expand and evaluate a tree, the tree becomes SFT data, the model is trained on it) closely follows Mulberry’s Collective MCTS, which is cited only in passing and never discussed. The search itself is greedy layer-wise selection with no rollout-based value estimation or backpropagation, so calling it “tree search” overstates it. On the RL side, the authors themselves note that the within-skill and cross-skill advantages sum to (r − μ)/(σ + δ). That means the policy update is standard GRPO over the pooled group, and the only new part is the EMA score feedback.
- The three limitations used to motivate the work (fragmentation, single-model bias, poor cross-backbone transfer) are asserted but never demonstrated, either in this paper or for the baselines. Some of them sit uneasily with prior work. Trace2Skill, which is cited, already targets fragmented skills and reports that its skills transfer across model scales and families. Yet neither Trace2Skill nor CoEvoSkills nor EvoSkill is included as a baseline. It’s also unclear whether Skill-GRPO, Skill-SD and SkillRL start from the same SFT data and use the same skills as the proposed method.
- The evaluation is small and reported without any measure of variance. QwenClawBench has 100 tasks, several categories have fewer than ten, and much of the 9B model’s gain comes from two such small categories (CS and SVM). Some categories actually regress against the base model or baselines, e.g., FQT for the 4B model, and this isn’t discussed. Also, SkillRL was originally evaluated on ALFWorld, WebShop and search QA, but only ALFWorld is used here. The authors should explain why the other settings were left out.
Questions for authors
- During construction, skills are scored conditioned on a prefix context. At test time those scores are averaged into a single context-free value and skills are retrieved by embedding similarity. How much of the tree structure actually matters at inference? Could you report how often the retrieved skills are relevant and actually followed? In the Appendix H example, the retrieved skill says secondary sources shouldn’t be the final evidence, but the model ends up using values from a third-party summary page.
- What is the total compute and API cost of building CSTS-SFT-2K? This involves three generator models, three judges, cross-model execution for every candidate, and a separate synthesizer. How would the method compare against baselines given a matched budget, for example SFT on an equal amount of plain teacher trajectories?
- The training tasks are reverse-synthesized from public agent trajectories (ClawGym, Nex, SWE-agent), and the training and test examples in the appendix are structurally very similar (financial ratio from a filing, write to file, verify). Did you check for task-level overlap or near-duplicates between the synthesized training tasks and the evaluation benchmarks?
9 comments
W3: Evidence for the Three Limitations and Additional Baseline Comparisons
W3(a): Clarification and Evidence for the Three Limitations
The three limitations used to motivate the work (fragmentation, single-model bias, poor cross-backbone transfer) are asserted but never demonstrated, either in this paper or for the baselines.
Thank you for this suggestion. We clarify the meaning of these limitations and the supporting empirical evidence below.
1. Skill fragmentation. We would like to clarify that our work focuses on skill coordination and execution dependencies across subtasks in long-horizon tasks. As discussed in the Introduction, local skills need to be organized into coherent execution sequences. Table 3 in the main text provides relevant evidence: both settings use multi-model skill generation and SFT, while CSN-Gen decomposes tasks into ordered subtasks and generates candidate skills conditioned on preceding skills and their execution history, improving performance by 4.2 points. This result supports our motivation to model cross-subtask dependencies through structured skill construction.
| Setting | QwenClawBench |
|---|---|
| SFT with vanilla multi-model skill generation | 35.6 |
| SFT with CSN-Gen | 39.8 |
2. Single-model bias. Skills constructed from trajectories generated by one model may inherit its problem-solving preferences, restricting the diversity of strategies they capture. CSN-Gen therefore uses heterogeneous models to provide complementary execution experience. Our additional ablation shows that increasing the number of generation models from one to three improves performance by 2.3 points, supporting the effectiveness of incorporating multiple models into skill construction. Consistent with this finding, MultiEvoSkill [1] reports performance drops across all three ablation benchmarks when any proposer model is removed.
| Number of generation models in CSN-Gen | QwenClawBench |
|---|---|
| N=1 | 37.5 |
| N=2 | 38.2 |
| N=3 | 39.8 |
[1] Du, K., Liu, J., Shan, R., Zhang, Y., Li, K., Sun, X., & Zhang, W. (2026). MultiEvoSkill: Structure-Aware Multi-Agent Skill Evolution for Generalizable Skill Artifacts. arXiv preprint arXiv:2610.34260.
3. Cross-backbone transfer. CoEvoSkills [2] reports that GPT-5.2 achieves 65.0% with Opus-transferred skills, compared with 69.8% with its own evolved skills, revealing a 4.8-point gap relative to model-matched skills. This illustrates that skill utility can remain backbone-dependent despite successful transfer. In our paper, Table 8 shows that adding transferability scoring improves performance by 1.8 points (43.1 → 44.9). These findings support explicitly evaluating cross-model utility during skill construction.
| Scoring variant | QwenClawBench |
|---|---|
| Without transferability scoring | 43.1 |
| With transferability scoring | 44.9 |
[2] Hanrong Zhang et al. CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification. arXiv preprint arXiv:2604.01687, 2026.
W3(b): Comparison with Trace2Skill
Some of them sit uneasily with prior work. Trace2Skill, which is cited, already targets fragmented skills and reports that its skills transfer across model scales and families. Yet neither Trace2Skill nor CoEvoSkills nor EvoSkill is included as a baseline.
Thank you for the suggestion. We respectfully clarify that Trace2Skill and CSTS organize skills at different levels. Trace2Skill analyzes successful and failed trajectories to propose local patches to the same initial skill, then merges these patches into a single skill. CSTS decomposes long-horizon tasks into ordered subtasks, builds a candidate skill tree with one layer per subtask, and constructs coherent skill paths conditioned on preceding skills and their execution history, explicitly modeling cross-subtask execution dependencies.
Moreover, Trace2Skill, CoEvoSkills, and EvoSkill focus on training-free skill construction and optimization, whereas our full framework also includes skill-augmented SFT and RL.
To directly compare skill-construction methods, we additionally reproduced these three methods and evaluated their generated skills on QwenClawBench using the same frozen Qwen3.5-9B backbone. CSTS skills improve the base model’s score from 34.5 to 38.3, outperforming all three baselines and supporting the effectiveness of CSTS-generated skills.
| Setting | QwenClawBench |
|---|---|
| Qwen3.5-9B | 34.5 |
| + Trace2Skill | 36.3 |
| + CoEvoSkills | 35.7 |
| + EvoSkill | 35.9 |
| + CSTS skills | 38.3 |
W3(c): SFT Initialization and Skill Usage across Baselines
It’s also unclear whether Skill-GRPO, Skill-SD and SkillRL start from the same SFT data and use the same skills as the proposed method.
Thank you for raising this concern. We kindly clarify that constructing a tree of skills through CSTS is one of our key contributions. Skill-SD and SkillRL have their own skill-construction designs, which we preserve in the main comparisons. Meanwhile, Skill-GRPO, as our skill-conditioned counterpart to vanilla GRPO, uses the skills generated by CSTS.
Nevertheless, to control for differences in SFT initialization and skill construction, we additionally evaluated Skill-GRPO, Skill-SD, SkillRL, and CSRL starting from the same CSTS-SFT checkpoint and using the same CSTS-generated skills, as shown below. CSRL still outperforms all baselines under this setting, although the performance gaps narrow because these baselines also benefit from CSTS-generated skills and CSTS-SFT initialization.
| Method | QwenClawBench |
|---|---|
| CSTS-SFT | 42.8 |
| Skill-GRPO | 38.7 |
| Skill-SD | 40.3 |
| SkillRL | 41.5 |
| CSTS-SFT + Skill-GRPO | 43.2 |
| CSTS-SFT + Skill-SD | 43.1 |
| CSTS-SFT + SkillRL | 43.4 |
| CSTS-SFT + CSRL | 44.9 |
Dear reviewer kyfC,
Thank you for your detailed and constructive review. We appreciate your recognition of our cross-model skill assessment and our evaluation across realistic agent tasks. We are going to try our best to address your questions one by one as follows.
W1: Contributions of Skill Construction
Thank you for raising this concern. We conducted additional controls using Qwen3.5-9B. Teacher-only SFT uses the exact same teacher trajectories as CSTS-SFT, with only the skill documents removed from the student's training inputs. The base-model control retrieves and injects skills from the skill bank frozen after CSTS construction. We also evaluate teacher-only SFT with CSTS skills at test time and compare the same CSTS-SFT + CSRL checkpoint with and without test-time skill injection.
With the same teacher demonstrations and the same CSTS skills provided at test time, CSTS-SFT outperforms teacher-only SFT by 3.6 points (42.8 vs. 39.2), supporting the benefit of skill-conditioned training. Injecting frozen CSTS skills improves the base model by 3.8 points (34.5 → 38.3) without updating its parameters. For the same CSTS-SFT + CSRL checkpoint, removing test-time skill injection reduces performance by 4.7 points (44.9 → 40.2). Together, these controls show that CSTS produces reusable skills that benefit the base model without additional training, while skill-conditioned training provides further gains beyond matched teacher supervision alone.
W2: Methodological Differences from Mulberry
Thank you for this comment. Mulberry is cited as Yao et al. (2024) in Section 2.3, and we provide a more detailed comparison below to clarify the methodological differences. Our work differs from Mulberry in research topic (reasoning vs. agent skills), training framework (SFT vs. SFT&RL), and search strategy (MCTS vs. CSRL-guided search). We clarify these differences below.
1. Different research topics. Mulberry focuses on multimodal reasoning, searching reasoning paths to construct SFT data for single-turn reasoning and reflection. Our work focuses on searching, selecting, and composing reusable skills into skill trees, and training models to effectively use these hierarchical skills to solve OpenClaw-style multi-turn agent tasks.
2. Different methodological designs. These different research topics lead to two main differences in our method.
2.1 SFT data construction. Mulberry uses Collective MCTS to construct reasoning demonstrations, whereas CSTS performs layer-wise skill-tree search. Compared with reasoning-only settings, MCTS-style exploration in multi-turn agentic tasks can incur substantially higher costs, as switching to a different skill at an earlier layer requires regenerating the subsequent execution trajectory. Our design therefore prioritizes stability and efficiency in the costly process of generating teacher demonstrations.
2.2 CSRL with cross-skill advantage feedback. Mulberry does not include an RL stage, whereas we introduce CSRL to jointly support policy learning and skill-tree adaptation. CSRL explores multiple candidate skills within each rollout group and uses cross-skill advantage feedback to update skill-node scores and revise skill paths.
3. Comparison with Mulberry. We additionally construct SFT data using Mulberry’s CoMCTS with the same search-iteration limit and fine-tune the same base model. CSTS-SFT outperforms CoMCTS-SFT by 3.5 points on QwenClawBench, supporting the effectiveness of our skill-tree construction for long-horizon agent tasks. The results also suggest that CoMCTS may require more search iterations to achieve comparable performance on long-horizon agent tasks, but doing so would substantially increase search costs.