- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes SkillTrainBench, which aims to test the capabilities of LLM agents authoring skills for other LLM agents. With tasks on HealthBench, \tau^3-bench banking, HLEBench and QFBench, this paper provides some preliminary empirical results on this specific setting.
Critical strengths
- The investigated setting seems interesting.
Critical weaknesses
- This submission seems to be largely written by generative AI. The overall logic and presentation are not smooth for human readers. The findings are also not presented in a precise manner.
- The overall novelty of this paper is quite limited. This paper only introduces an interesting setting (while this setting is not novel and investigated by many prior papers) without constructing novel tasks (or providing enough justification on why we should choose these four tasks) or novel agent framework. Thus, I do not think this paper meets the basic bar of ICLR.
Questions for authors
None
2 comments
Thanks for the response. I would like to maintain my score as a clear reject.
Dear Reviewer 58eY,
We thank you for your time and thoughtful comments. We appreciate the opportunity to clarify the contribution and improve the presentation. We have expanded the benchmark rationale and added a human–AI curator ablation.
The contribution is a benchmark protocol and empirical study of skill authoring. A curator chooses development tasks, spends a metered learner-interaction budget, revises a package and declares its final submission. Private held-out evaluation measures the submitted package, saved revisions and transfer to other learners. The study covers 36 curator–learner–domain configurations. We do not claim that generating reusable instructions is itself new; Related Work now distinguishes our evaluation focus from SkillsBench, SkillLearnBench, SkillGenBench and Metacognitive Reuse.
The four benchmarks were chosen to cover distinct interaction and verification settings: clinical consultation with rubric grading, interactive banking, expert science answers, and quantitative-finance checks. This makes it possible to test whether an authoring method's gains extend beyond one type of task.
In the new §5.3, we report a human–AI curator ablation. We organized a hackathon in which human–AI teams replaced autonomous curators while performing the same skill-authoring task. We allocated approximately matched learner-token budgets and $50 in OpenAI credits per participant, with two participants per team. Teams could also use additional coding assistants; learner probes and evaluation used GLM 5.3 Flash through OpenHands. On 159 shared HealthBench tasks with GLM 5.3 Flash, 5, 7 and 9 of 18 teams exceed the respective Opus-, Fable- and GPT-curated packages. The cohort has 18 HealthBench entries and 16 per other domain. Human HealthBench evaluation allowed four learner iterations versus twelve for autonomous packages, and total curator resources differed. We therefore present a descriptive comparison, not a controlled ranking of authors.
We take responsibility for the writing and presentation. We corrected duplicated and miscopied appendix tables, repaired error counts, and made the domain and checkpoint-selection conclusions more precise. We hope the clarified contribution, additional evidence and presentation changes address your concerns. If so, we would be grateful if you would consider updating your assessment. We welcome any further questions and thank you again for your comments.