- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SkillGate, which trains in-policy skill selection through separate credit assignment for skill-selection and execution tokens with length-independent weighting, and evaluates its effectiveness on five agent benchmarks.
Critical strengths
-
The paper identifies weak and potentially conflicting learning signals for skill-selection tokens in long agent trajectories. Its trajectory analysis provides concrete motivation for treating selection and execution separately.
-
SkillGate separates selection and execution credit within a single policy and improves aggregate task success over outcome-only RL across five benchmarks.
Critical weaknesses
-
Table 3 pairs external selectors with a frozen SFT executor, whereas SkillGate benefits from RL training of both selection and execution. This confounds the selection architecture with executor training. Despite discussing related approaches such as daVinci-kernel and ToolOmni, the paper lacks controlled comparisons with their joint-training or credit-separation strategies. Consequently, the evidence is insufficient to establish the proposed design’s advantage over relevant alternatives.
-
SkillGate assigns zero selection utility to every read in a trajectory containing multiple reads, including a correct reselection after an initial mistake. This conflates unnecessary reading with potentially useful correction. Although the constraint helps in Table 2, removing it yields higher success in Table 5 (Action credit: 65.3%; SkillGate: 61.2%). Without recovery-specific analysis or repeated training runs, the paper does not establish when this restriction improves task performance or whether the reported differences are robust.
Questions for authors
-
Have SkillGate and the main baselines been independently trained with multiple training seeds? If so, please report the mean and variability of final performance, along with the consistency of the rankings across methods. If not, what additional evidence helps distinguish a reproducible benefit of the proposed design from differences specific to individual training runs, and how does this limitation constrain the paper’s conclusions?
-
Among the cited related work, daVinci-kernel jointly trains skill selection and execution using execution outcomes, while ToolOmni separates the rewards and advantages for retrieval and execution. Thus, explicitly training selection or separating selection and execution learning signals has already been explored. What additional benefits do SkillGate’s identity-token-level credit assignment and length-independent weighting provide over these approaches? In particular, are there comparative results demonstrating their advantages over existing credit-separation strategies, independently of the single-read utility? If the current ablations establish this distinction, please identify which comparisons do so.
-
In Table 3, external selectors are paired with a frozen SFT executor, whereas SkillGate receives additional RL training for both selection and execution. It is therefore difficult to determine whether the performance difference reflects an advantage of in-policy selection or the effect of additional executor training. Do you have comparisons pairing an external selector with an RL-trained executor under matched training budgets and supervision? Have you also evaluated a baseline trained with oracle skills provided during training, using RL solely to learn execution? Without these controls, what supports interpreting Table 3 as evidence of limitations in separate-selector approaches, beyond the benefit of additional executor training?
-
What criterion motivated adopting the single-read constraint, given that Action credit achieves higher success than SkillGate on the held-out Claw-Eval subset (65.3% vs. 61.2%)? Have you analyzed the existing trajectories to distinguish unnecessary additional reads from reselection that successfully corrects an initially wrong skill choice?
9 comments
Author Response (2/5)
2 (Weakness 2 and Question 4)
Single-read gives zero utility to every read in a multi-read trajectory, including a correct reselection after a wrong first read, and removing it gives higher success in Table 5 (Action credit 65.3% vs SkillGate 61.2%). What criterion motivated single-read, and can a recovery analysis separate unnecessary extra reads from useful correction?
Action credit in the paper (the version that allows multiple reads) is a variant of SkillGate. Our naming may have caused a misunderstanding, so we rename it SkillGate-MR. To address your concern about the single-read constraint, we ran a temporally ordered recovery analysis on the existing trajectories: single-read does not suppress correction after a wrong first read; it reduces wrong first reads and extra reads after the oracle, both of which lower task success. The gap on Claw147 is not significant and does not come from recovery. Overall, the two versions trade wins on task success, while SkillGate selects more cleanly and costs less at inference.
(1) SkillGate-MR (Action credit in the paper) is a variant of SkillGate. It applies the same credit-separation method and only drops the single-read requirement when assigning selection credit:
| Identity-token credit | Read call removed from the outcome loss | weight | Which selection receives credit | |
|---|---|---|---|---|
| SkillGate | Yes | Yes | Yes | The only read in the trajectory, when it reads the oracle |
| SkillGate-MR (Action credit in the paper) | Yes | Yes | Yes | The first read that reaches the oracle; other reads are unrestricted |
SkillGate-MR is therefore an experimental variant, not a different method, and its strength is itself evidence that SkillGate's core method works. We adopt SkillGate as the default because it reads more cleanly at lower cost.
(2) Criteria for adopting single-read.
- Credit cannot be earned by reading more. If each read is scored separately, a policy can collect selection credit simply by reading several skills.
- Cost and consistency. Without the constraint the model reads more skills and uses more tool calls and tokens, without a consistent gain in success:
| Metric | SkillGate | SkillGate-MR |
|---|---|---|
| Clean oracle read, eval70 / Claw147 | 75.4 / 76.9 | 64.6 / 73.5 |
| Misleading read, eval70 / Claw147 | 21.8 / 15.0 | 32.5 / 20.4 |
| Turns / tool calls / input tokens per trial, eval385 | 17.3 / 8.8 / 521k | 18.3 / 10.4 / 555k |
(3) Recovery analysis: unnecessary extra reads vs. successful correction. In the 280 eval70 trajectories of each model, skill reads are labelled in temporal order: whether the first read is wrong, whether a wrong first read is later followed by the oracle (correction), and whether other skills are read after the oracle (extra reads).
| First read wrong | Correction rate | Extra reads after the oracle | Exactly one read, of the oracle | |
|---|---|---|---|---|
| SkillGate | 52 (18.6%) | 23.1% | 14 (5.0%) | 183 (65.4%) |
| SkillGate-MR | 73 (26.1%) | 23.3% | 33 (11.8%) | 161 (57.5%) |
| Task-mask only (no selection credit) | 180 (64.3%) | 22.2% | 60 (21.4%) | 48 (17.1%) |
- Correction is not suppressed in SkillGate. After a wrong first read, the three models reach the oracle at the same rate, although only SkillGate-MR gives credit to a corrective read.
- Single-read reduces the other two behaviours: wrong first reads by 7.5 pp [−13.9, −1.4] and extra reads after the oracle by 6.8 pp [−12.1, −2.1].
(4) Effect of these behaviours on task success. Within each prompt group of the training rollouts, we compare the success of trajectories with different reading patterns:
| Comparison | SkillGate training rollouts | Outcome-only training rollouts |
|---|---|---|
| Extra reads after the oracle vs. oracle only | −4.9 pp [−9.0, −0.8] | −5.2 pp [−10.7, +0.1] |
| Wrong and uncorrected vs. oracle only | −19.9 pp [−28.6, −11.2] | −19.0 pp [−25.5, −12.6] |
| Wrong then oracle vs. wrong and uncorrected | +12.9 pp [+2.1, +23.9] | +8.4 pp [+1.9, +14.9] |
This answers when the constraint helps: extra reads and wrong first reads both lower success, and these are the two behaviours single-read targets. Correction is beneficial, and the single-read setting does not reduce it.
(5) The Claw147 gap does not come from recovery. The 147 trajectories of each model, sorted by the order of their reads:
| Claw147 (147 tasks, 1 trial each) | Oracle only | Oracle, then others | Wrong, then oracle | Read, never the oracle | No read | pass@1 |
|---|---|---|---|---|---|---|
| SkillGate | 113 | 2 | 2 | 20 | 10 | 90 (61.2%) |
| SkillGate-MR | 108 | 6 | 0 | 27 | 6 | 96 (65.3%) |
- SkillGate-MR has no wrong-then-oracle trajectory on Claw147, so its additional successes cannot come from correction.
Revision. We will add the recovery analysis to Appendix C.1, and rename Action credit to SkillGate-MR throughout the paper.
Author Response (3/5)
3
Were SkillGate and the main baselines trained with multiple seeds? If not, what evidence separates a reproducible benefit of the design from run-specific differences, and how does this limit the conclusions?
Response:
Each configuration is trained once: one run takes 100 steps of on-policy GRPO on 16 H800 GPUs and collects 12,800 long-horizon trajectories online, so repeating all eight RL configurations over multiple seeds is prohibitively expensive. Evaluation-side randomness is controlled by four trials per task, SkillGate is best or tied for best among the 9B methods on all five benchmarks, and three further pieces of evidence show that the gain comes from the credit design rather than from the chance of a single run.
(1) Reproducibility.
All configurations share the same SFT checkpoint, the same 491 training tasks and the same 100-step budget (Sec. 4.1, Appendix A). The three pieces of evidence are:
| # | Evidence | Result |
|---|---|---|
| 1 | Independent runs split consistently | Two runs with identity-token selection credit (SkillGate, SkillGate-MR): 53.2% and 51.9% on eval385. Six runs without it (Outcome-only, Task-mask only, Skill-free RL, Skill1, Group-level regret, Trajectory bonus): 44.7%–48.6% |
| 2 | Large effect sizes in selection behaviour, which transfer to unseen tasks and unseen skills | vs Outcome-only on eval70: clean oracle read +53.9 pp [+44.3, +63.2], misleading read −47.9 pp [−57.5, −37.9]. On Claw147: clean oracle read 76.9% vs 34.7% (+42.2 pp [+33.3, +51.0]), misleading read 15.0% vs 53.7% (−38.8 pp [−47.6, −29.9]) |
| 3 | Training curves show a sustained trend | SkillGate's clean oracle read rises 27.8 → 55.9 → 69.1 → 78.2 → 76.7%; under Outcome-only it falls from 13.9% to 9.1% |
Evidence 1 is the most direct: the eight RL runs are independent and fall into two non-overlapping groups according to whether identity-token selection credit is present. The effect is thus reproduced by two independent runs, and the six runs without it give a reference for run-to-run variation. Differences of 40–58 pp in selection behaviour, which persist on a benchmark never seen in training, are not explained by such variation.
(2) Mean, variability and ranking consistency. With one run per configuration there is no cross-seed mean or variance to report. The ranking is consistent: the two runs with identity-token selection credit rank above all six runs without it, and the two groups do not overlap.
(3) Scope of the conclusions. The paper's two conclusions rest on large effects: identity-token selection credit changes selection behaviour by 40–58 pp, and task success improves over Outcome-only by 6.2 pp on eval385 and by 7.9 pp on eval70 (p=0.020).
Author Response (1/5)
Thank you for dedicating your time and effort to our work.
1
To address your concerns: (1) with new-added the executor matched experiment, the conclusion is unchanged. External routers paired with the SkillGate executor remain largely below SkillGate. (2) The paper's ablations already compare, in the same setting, the credit strategies that daVinci-kernel and ToolOmni represent.
(1) Matched-executor analysis. We compare two executors on eval70: the frozen SFT model of Table 3 and the SkillGate checkpoint (identical RL training). The Oracle-only rows hand the oracle directly to the executor, which is equivalent to a perfect router. The first eight rows form four pairs: the same source of selection, with the SFT executor and the SkillGate executor comparison.
The RL-trained controls without identity-token credit (Outcome-only, Task-mask only, Skill-free RL) receive the same executor training as SkillGate and stay at 46.0%–47.0% on eval385, against 53.2% for SkillGate.
(2) Controlled comparison with the credit strategies of daVinci-kernel and ToolOmni. The two systems cannot be applied as-is to read calls inside the execution trajectory, so we compare the credit designs they represent, under the same initialisation, data and budget:
Both existing strategies stay within 44.7%–48.6% in our setting, while identity-token credit reaches 51.9%–53.2%; SkillGate-MR does not use single-read, so the gap is unrelated to it. Neither method performs in-policy selection: they select and execute in two separate stages, so they do not encounter selector credit starvation.
Revision. We will add the matched-executor analysis to Sec. 4.4 and mark in Sec. 4.3 how each ablation row corresponds to an existing credit strategy.