- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Skill-α learns to turn documents and execution traces into reusable textual skills. A Qwen3-8B editor applies CREATE, UPDATE, MERGE, PRUNE, or NOOP to an evolving skill. It is first trained on synthetic demonstrations, then optimized with GRPO on fixed local editing states. The rollback reward compares the worker's performance with the original and edited skills on the same training query. The query is used for reward evaluation and is hidden from the editor. After training, skill generation requires neither anchor queries nor parameter updates. Experiments cover CL-Bench, SpreadsheetBench, and tau2-bench, with transfer to BFCL and other worker models.
Critical strengths
-
The tau2-bench results are the strongest evidence for the method. With GPT-4o, Table 3 reports an average of 55.83, compared with 49.17 for SkillPro and 33.33 without a skill. Across three trials with different optimization seeds and independently sampled anchors, the corresponding means are 56.67 ± 0.83 for Skill-α and 48.61 ± 2.55 for SkillPro (Table A8). The same generated skills also improve Claude-Sonnet-4.5 from 63.33 to 70.83. This gives me confidence that the experience-derived skills are useful beyond the worker used in training.
-
Using one editing policy for documents and trajectories is a useful contribution. The SFT-only and action-space ablations in Table 5 support the roles of RL and skill maintenance. Table A10 also shows that rollback outperforms direct reward after NOOP is removed, so the result cannot be explained entirely by the abstention rule. Applying the trained editor to BFCL without further parameter updates extends the evaluation beyond the training domains.
Critical weaknesses
-
I would like a clearer explanation of what rollback changes during training. On SpreadsheetBench and tau2-bench, feedback is binary, so Eq. 5 gives
R_i = (1 - r_ctrl) * r_edit_ifor valid content-changing edits. If the control fails, this is direct binary reward; if it succeeds, every such edit receives zero. Without NOOP, successful-control groups therefore contribute no policy-gradient signal, while failed-control groups retain the same reward ordering as direct reward. Table A10 establishes a performance difference, but leaves open how much comes from changing which groups contribute updates. Appendix A's expected-reward analysis is consistent with this observation. Control success rates, counts of informative update groups, and worker execution costs would help explain the gain. This equivalence applies to the binary benchmarks; CL-Bench uses graded training feedback. -
The smaller generalization gains need uncertainty estimates. On CL-Bench with GPT-4o, Skill-α scores 10.38 versus 8.57 without a skill. The 3.1-point gain quoted in the paper compares against the strongest skill-generation baseline, which is below No Skill. On BFCL, gains over No Skill are 1.01 points with GPT-4o and 4.02 with Claude-Sonnet-4.5. Table A8 covers neither benchmark. Tasks sharing a CL-Bench context, and BFCL variants sharing a base task, should be treated as dependent when estimating uncertainty. These results are promising, but I place more weight on tau2-bench in my recommendation. I lean toward weak acceptance because of its repeated gains and the transfer across workers.
Questions for authors
-
For each binary-verifier benchmark, what fractions of training groups have a successful control or zero reward variance? How many groups contribute policy-gradient updates under rollback and direct reward, and what are their worker execution costs? These statistics would help assess whether rollback mainly improves training by concentrating updates on states where the current skill fails.
-
Can you provide uncertainty estimates for gains over No Skill on CL-Bench and BFCL under both workers? A paired bootstrap over source contexts for CL-Bench and base tasks for BFCL would help, preserving the reported macro averages and overlapping Sub-API assignments. Repeated runs would also help quantify generation and execution variability.
-
Appendix C gives 50, 8, 19, and 19 editing steps for the four experience source groups, averaging 24, while Table A13 reports 40 generator calls per skill. What accounts for the additional calls? Were the cost and accuracy measurements obtained with the same generation settings?
4 comments
Rebuttal by Authors (2/3)
Review (W1&Q1 - Part 2): What are their worker execution costs?
Response: With valid candidates per group and isolating the effect of skipping candidate evaluations when , the average worker executions per group under short-circuit rollback is , as shown in the table below.
| Benchmark | Control Success () | Direct Reward () | Standard Rollback () | Short-Circuit Rollback () | Savings vs. Direct Reward |
|---|---|---|---|---|---|
| SpreadsheetBench | 149 / 512 (29.1%) | 8.00 | 9.00 | 6.67 | 16.60% |
| -bench | 411 / 768 (53.5%) | 8.00 | 9.00 | 4.72 | 41.02% |
While standard synchronous rollback evaluates 1 control rollout and 8 candidate rollouts per group, binary rollback allows skipping all 8 candidate evaluations whenever , since for all non-NOOP edits and deterministically. This reduces average worker executions per group below direct reward by 16.60% on SpreadsheetBench and 41.02% on -bench. When within-group NOOP evaluation reuse is additionally enabled for all methods, the average calls per group for Direct, Standard Rollback, and Short-Circuit Rollback are 7.21, 7.86, and 5.59 on SpreadsheetBench and 7.51, 8.23, and 4.05 on -bench, corresponding to savings of 22.47% and 46.03% over direct reward.
Review (W2&Q2): The smaller generalization gains need uncertainty estimates. Can you provide uncertainty estimates for gains over No Skill on CL-Bench and BFCL under both workers? A paired bootstrap over source contexts for CL-Bench and base tasks for BFCL would help, preserving the reported macro averages and overlapping Sub-API assignments. Repeated runs would also help quantify generation and execution variability.
Response: Following your suggestion, we have computed 95% confidence intervals using a 10,000-replicate clustered paired bootstrap, resampling over source contexts for CL-Bench and unique base tasks for BFCL while preserving macro averages and overlapping Sub-API assignments. The results against both No-Skill and the strongest skill-generation baseline are shown in the table below.
| Benchmark | Backbone | Skill- Macro Avg. | vs. No-Skill [Clustered 95% CI] | Best Skill-Gen Baseline | vs. Best Skill-Gen Baseline [Clustered 95% CI] |
|---|---|---|---|---|---|
| CL-Bench | GPT-4o | 10.38% | +1.81 [-1.04, +4.75] | Ctx2Skill (7.30%) | +3.07 [+0.21, +5.99] |
| CL-Bench | Claude-Sonnet-4.5 | 9.55% | +1.85 [-0.70, +4.36] | AutoSkill (8.40%) | +1.15 [-1.53, +3.86] |
| BFCL Multi-Turn | GPT-4o | 57.13% | +1.01 [-5.24, +7.43] | SkillPro (51.50%) | +5.63 [+0.20, +10.85] |
| BFCL Multi-Turn | Claude-Sonnet-4.5 | 62.94% | +4.02 [-2.01, +10.34] | SkillPro (57.54%) | +5.41 [-0.82, +11.69] |
On CL-Bench and BFCL, while the single-run 95% intervals over No-Skill include zero due to the limited number of evaluation entries, Skill- achieves strictly positive 95% confidence intervals over the strongest skill-generation baselines on both benchmarks under GPT-4o, and remains the only method that consistently improves over No-Skill across both benchmarks and workers. We have added these bootstrap intervals to Appendix D.4 of the revised paper.
Rebuttal by Authors (3/3)
Review (Q3): Appendix C gives 50, 8, 19, and 19 editing steps for the four experience source groups, averaging 24, while Table A13 reports 40 generator calls per skill. What accounts for the additional calls? Were the cost and accuracy measurements obtained with the same generation settings?
Response: Thanks for pointing out this detail. We confirm that the cost in Table A16 and accuracy in Table 3 are measured from the exact same skill-generation runs under identical settings. In our setup, a batch size of 4 specifies an upper bound of up to 4 trajectories per editing step under the fixed 8,192-token prompt budget. Appendix C reports the nominal step count of 24 obtained by dividing the total source trajectories by the maximum batch size of 4, whereas Table A16 logs the actual LLM calls required during progressive generation.
Specifically, each generator call in Skill- must fit the system prompt, the evolving skill , and the trajectories within the 8,192-token prompt cap. As grows to an average of 2,968 tokens, packing 4 full trajectories alongside often exceeds the 8,192-token limit. Whenever this happens, the loader packs fewer trajectories into the current call and processes the remaining trajectories in subsequent steps so that no evidence is truncated, increasing the average number of progressive editing calls from 24 to 40.00. The same prompt-budget sub-batching also explains why Progressive Prompt Skill logs 30.75 calls instead of 24 without separate merge calls, while other baselines incur additional calls from recursive chunk merges. We have clarified this in Appendix C and Appendix D.7 of the revised paper.
Kind Request for Your Feedback on Our Rebuttal
Thank you very much for your thoughtful review, as well as the valuable questions and constructive suggestions you provided. In our rebuttal and revised paper, we have done our best to address all of your concerns and clarifications. As the author-reviewer discussion period will end in about 1 day, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. If you have any remaining questions or would like further clarification on any point, please feel free to let us know and we will respond promptly. Thank you very much for your time and consideration!
Rebuttal by Authors (1/3)
We thank Reviewer NnuH for the supportive comments. Below, we address all raised questions and will incorporate the feedback into the revised paper.
Response: Thanks for this insightful observation. Your analysis of the binary-verifier case is exact. When the observed control fails with rctrl=0, rollback reward preserves the same candidate reward ordering as direct binary reward among valid content-changing candidates, while the
NOOPfallback rule can provide an additional signal when no valid candidate edit improves over the failed control. Thus, on binary benchmarks, rollback reward acts by filtering which states and actions receive positive advantages based on the observed control outcome. Specifically, on successful-control groups where rctrl=1, removingNOOPcauses rollback to assign 0 to all content edits, yielding zero reward variance std(R)=0 and no reward-driven update. When validNOOPis present on rctrl=1 groups, rollback assigns 1 toNOOPand 0 to content edits, giving a positive advantage toNOOPand negative advantages to unnecessary edits.To quantify this effect across the 512 SpreadsheetBench and 768 τ2-bench editing states with G=8, valid
NOOPincluded, and informative update groups defined by std(R)>0, we report the training-group statistics logged from the rollback reward Rrb and direct reward Rdir training runs below.When rctrl=1, direct reward assigns 1 to any candidate that achieves riedit=1, giving it a positive advantage whenever at least one candidate in the group fails. Across the rctrl=1 groups in the logged training runs, direct reward assigns positive advantages to 918
CREATEandUPDATEcandidates that do not improve over the observed control score, whereas rollback reward assigns zero positive advantage to content edits on rctrl=1 states, as shown below.These logged training statistics confirm that direct reward assigns positive advantages to content edits that do not improve over the observed control score on rctrl=1 states, whereas rollback reward restricts positive content-editing advantages to states with an observed rctrl=0→riedit=1 transition. We have added these statistics to Section 5.3 of the revised paper.