- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SkillGrace, a skill optimization method that leverages same-task contrastive evidence from grouped rollouts to generate and evaluate local skill patches via trajectory replay. By employing advantage-weighted scoring, gain checks, and KL constraints, it updates skills without modifying model parameters or requiring additional environment interactions. Experiments on SearchQA, SpreadsheetBench, and ALFWorld show a 2.8% average improvement over SkillOpt with 77.6% fewer rollouts, validating the effectiveness of same-task contrastive learning and replay-based optimization.
Critical strengths
-
The motivation is clear, and the optimization framework is coherently designed. To address the limited use of trajectory information and costly candidate patch evaluation, this paper proposes a targeted overall optimization framework.
-
The replay-based candidate patch screening mechanism improves interaction efficiency. By reusing existing trajectories for patch evaluation, it reduces total optimization rollouts by 77.6%.
-
The method achieves positive results on multiple-agent tasks. Across three benchmarks, the average score improves by 2.8 percentage points over SkillOpt, and the ablation preliminarily supports the same-task contrast mechanism.
Critical weaknesses
-
The relationship between the replay objective and task performance is unclear. The paper uses an advantage-weighted token-likelihood objective to select patches, but its relationship to actual task rewards remains unclear.
-
The efficiency evaluation is incomplete. The paper mainly measures optimization rollout counts, excluding the inference costs of patch generation and trajectory replay. Thus, it does not capture end-to-end computational cost.
-
The ablation study is limited. It compares same-task and cross-task contrastive evidence but does not isolate the contributions of replay screening and other constraints (such as the KL constraint and ESS checks).
-
The experimental scope is limited. The evaluation uses one executor across three benchmarks, leaving cross-model generalization unclear. The initial skill's construction is also insufficiently specified, limiting reproducibility.
Questions for authors
See weaknesses. I am open to raising my score if the authors provide convincing responses to the concerns raised.
3 comments
Many thanks for addressing my comments. I will raise my rating to 3.
Thank you for your thoughtful feedback and for raising your rating to 3. Your questions and suggestions helped us clarify our contributions and strengthen the paper.
Thank you for the constructive feedback. We address each concern below.
W1. The relationship between the replay objective and task performance is unclear. The paper uses an advantage-weighted token-likelihood objective to select patches, but its relationship to actual task rewards remains unclear.
Higher replay objective were positively correlated with better task performance. To investigate the relationship between replay objective and task performance, we added Table 5 (line 444) to report how gains in the replay objective and validation reward correlate with reward gains on test set. Appendix D (line 1332) defines these gains and details the scoring protocol.
Replay had a higher mean correlation than validation across these iterations, although validation was slightly better at iteration 5. This supports replay as a useful selection signal without interactions with environment.
W2. The efficiency evaluation is incomplete. The paper mainly measures optimization rollout counts, excluding the inference costs of patch generation and trajectory replay. Thus, it does not capture end-to-end computational cost.
We added Appendix F (line 1404) to account for total usage beyond rollouts. The token totals include patch generation and trajectory replay. Across the three benchmarks, SkillGrace uses 77.8% fewer optimization rollouts and processes 32.3% fewer optimization tokens overall than SkillOpt.
We report both measures because teacher-forced replay still uses model inference, but scores stored trajectories without generating new actions, rerunning tools, or recomputing rewards. These measurements quantify token use and environment interactions separately.
W3. The ablation study is limited. It compares same-task and cross-task contrastive evidence but does not isolate the contributions of replay screening and other constraints (such as the KL constraint and ESS checks).
In light of your suggestion, we have conducted an additional replay-selection ablation study to test whether replay improves final task performance beyond what the patch generation procedure achieves on its own. We held the initial skill and patch generation procedure fixed and compared random, validation-gated, and replay-based patch selection. The results are shown in Table 4 (line 433) of the revised manuscript and detail the experimental protocol in Appendix D (line 1332). It can be observed that replay improves test success on SpreadsheetBench by 7.5 percentage points over random selection and 4.6 points over validation-gated selection. Replay thus outperforms both alternatives on SpreadsheetBench, showing that stored trajectories provide a useful signal for patch selection without requiring fresh rollouts.
W4. The experimental scope is limited. The evaluation uses one executor across three benchmarks, leaving cross-model generalization unclear. The initial skill's construction is also insufficiently specified, limiting reproducibility.
To address your concern, we evaluated another model Qwen3.6-27B, updated Table 1 (line 325)
With Qwen3.6–27B, SkillGrace uses the executor itself as the editor and achieves the highest observed score on all three benchmarks. It also outperforms SkillOpt, the strongest baseline, by 0.7, 3.5, and 2.3 points on SearchQA, SpreadsheetBench, and ALFWorld, respectively, while using Qwen3.6–27B itself as the editor rather than GPT‑5.5. Together with the original results, these findings show that SkillGrace’s skill optimization procedure is effective across models without requiring a separate stronger editor.
Initial Skill's Construction: We have added the initial skill for each benchmark to Appendix C (line 1198).