- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies how a Builder can learn meta-skills from a Target’s execution feedback and use them to construct harnesses for new tasks. With both models’ weights fixed, the Builder translates learned support principles into tools, memory, and execution controls. Comparisons with no-skill construction, direct skill delivery to the Target, and independently learned Target skills support the value of learning and implementing external support.
Critical strengths
- The research question is clear and valuable. Learning how to support an agent provides a meaningful complement to learning task-solving skills directly.
- The experimental design is solid. The baselines closely match the central research questions. Experiments across models, component ablations, and outcome-transition analyses provide complementary evidence for the main findings.
Critical weaknesses
The case studies leave some ambiguity about how support design produces the gains. In the NewtonBench example, the Builder provides not only memory and failure-recovery mechanisms, but also specific scientific guidance, such as jointly inferring energy loss and using cross-mass probes. Success may therefore reflect both execution support and task-solving guidance, whose contributions are not clearly separated. By comparison, the checker and submission controls in the Harness-Bench example more directly illustrate translating workflow requirements into executable support. These cases may demonstrate different sources of improvement, and their relationship to the paper’s interpretation of meta-skills could be clearer. This does not undermine the overall performance gains, but it limits our understanding of what the method has learned.
Questions for authors
- In the NewtonBench case, does the specific scientific guidance come from the learned meta-skills, or does the Builder derive it from the current task? How can the contributions of task-solving guidance and execution support be distinguished?
- Could the authors show the relevant meta-skills verbatim and map them to the generated instructions, tools, or control logic, explaining what changed relative to the no-skill Builder? This would clarify the connection between learned experience, support design, and improved execution.
2 comments
Thanks. All Concerns are solved. I think the paper meets the bar for ICLR. So I keep my score and support the authors.
Follow-Up on Our Rebuttal Responses
Dear Reviewer F7WL,
As the discussion period is coming to a close, we would like to kindly follow up and ask whether our responses have adequately addressed your concerns, or if you have any further questions.
If your major concerns have been resolved, we would greatly appreciate it if you could reflect this in your final score and confidence updates. Thank you very much for your time and thoughtful feedback!
Best regards, Authors of Paper 27157