acceptodds
Under review as a conference paper at ICLR 2027

Code-to-Harness: Learning Language Policies through Executable Practice

Abstract

Learning reusable policies for frozen large language models (LLMs) raises a representation question: must a policy be discovered in the same form in which it is deployed? We introduce Code-to-Harness, which uses executable programs as the discovery representation and compact instructions as the deployment representation. A practice agent proposes and revises optimizer programs from executable feedback; a development competence screen determines which completed runs are eligible for promotion; and a one-shot in-context synthesis compresses the accumulated executable practice into an approximately 200-word frozen policy. Only policies from promoted runs enter the primary transfer evaluation, and no source program is executed at deployment. Across ten completed discovery runs, four satisfy the recorded promotion criterion. On 30 genuinely fresh objective instances per task, promoted policies reduce mean regret relative to the unaided executor by 44.4–60.7% on quadratics and 32.5–50.0% on Rastrigin. One fixed policy attains lower mean regret than all five numerical baselines on eight of ten BBOB families, and all four promoted texts improve a second vendor's executor on both core tasks. On three LCBench/YAHPO-Gym HPO tasks, frozen policies attain the lowest reported mean on two datasets. A five-batch audit further shows that the recorded threshold behaves as a conservative competence screen rather than a calibrated predictor of downstream text quality. Together, the results show that useful search-strategy information can cross an executable-to-language representation boundary without requiring the discovery program at deployment.

Primary area:
optimization
Keywords:
Black-box optimization, LLM-based optimizers

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_MgRfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes to discover optimization strategies for a frozen LLM through executable practice. An agent first writes and refines optimizer programs on simple synthetic functions, and the practice is then distilled into a short text of principles. The text is given to the LLM at deployment without running any program, and the paper shows that it improves black-box optimization on new instances and several transfer settings.

Critical strengths
  1. The paper is well-written and easy to follow.
  2. The idea of learning strategies through executable practice and then turning them into plain text is interesting.
Critical weaknesses
  1. Unclear Motivation. It is not shown that executable practice is really necessary. A text written directly by the same LLM without any practice may work just as well, but this comparison is missing.
  2. Limited Baselines. The comparison with other prompt optimization methods is too thin, and the original programs are never tested on the transfer tasks, so it is unclear what is gained by turning them into text.
Questions for authors
  1. How would a strategy text written directly by the LLM without practice perform?
  2. Why is text a better way to carry the strategy than the program itself?
  3. Missing references. https://arxiv.org/abs/2405.20132, https://arxiv.org/abs/2401.02051 and https://arxiv.org/abs/2402.01145 should be discussed.
Reviewer_jCb6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper tackles the problem of learning reusable skills for frozen LLMs. They observe that existing works discover skills in language, where advice needn't commit to concrete actions and evaluation requires repeated LLM execution. Therefore, they propose Code-to-Harness, which iteratively revises optimizer programs using execution feedback, then summarizes the practice into a fixed six-principle skill for an LLM executor. The core idea of Code-to-Harness is to discover strategies in code and deploy them through approximately 200 words organized into six prescribed principles. They tested the learned skills on fresh numerical objectives, ten BBOB families, additional LLM executors, and three HPO tasks. The results show that the four skills from ten discovery runs improve the unaided executor on fresh core tasks, while transfer varies across objectives, dimensions, and executors.

Critical strengths
  1. The method separates executable strategy discovery from language-based deployment without updating model weights.
  2. The frozen skills consistently improve the unaided executor on fresh core-task instances.
  3. The paper reports all promoted skills and documents transfer failures and limitations.
Critical weaknesses
  1. Although the authors describe their method as a “discovery-deployment representation split,” the underlying idea is a skill-making workflow for a specific class of numerical optimization tasks. They evaluate four extracted skills on related tasks, but do not establish that the workflow can reliably discover skills more broadly. The paper reads more like a development report for these skills than a study of a general skill-learning method.
  2. The contribution of executable practice is not isolated. The distillation prompt already prescribes six strategy topics, including informative probing, incumbent-relative search, budget adaptation, and recovery from failed predictions. The paper lacks a matched control that generates a skill from the same instructions without executable practice. The gains could partly reflect this supplied structure and the model’s existing optimization knowledge.
  3. The six-principle format is imposed rather than discovered. The paper does not explain why six principles are appropriate or compare this format with unconstrained summaries, different principle counts, or a matched human-written skill. Consequently, recurring principles across outputs do not by themselves demonstrate knowledge acquired through practice.
  4. The scope of reuse is narrow and uneven. Most evaluations retain the same bounded continuous optimization interface. HPO is mapped into that interface, and some skills hard-code its bounds. Transfer weakens on Flash-Lite, all four skills worsen performance on Katsuura, and Harness B fails at higher dimension. The evidence supports reuse within this optimization setting, rather than broadly reusable agent skills.
Questions for authors

see weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.