acceptodds
Under review as a conference paper at ICLR 2027

Harness Learning Enables Generalizable Test-Time Adaptation

Abstract

A language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Different tasks call for different ways of organizing these operations, creating a need to adapt the harness using feedback from the task at hand. We introduce harness learning, which trains a proposer model to revise a solver’s harness using execution information. We formulate this process as meta-learning over executable programs, with harness revisions playing the role of weight updates in gradient-based adaptation. We train the proposer through reinforcement learning using the task performance of revised harnesses as the reward. At test time, the proposer uses feedback from successive executions on a new task to refine the harness, while both models remain frozen. Experiments across reasoning and multi-hop question answering show improved revision quality and transfer to unseen tasks. Policies trained on individual revisions can continue improving harnesses over multiple rounds, while the benefits of training on revision sequences vary across settings. These findings suggest a path towards continually learning agents that turn accumulated experience into generalizable improvements.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Harness, Meta Learning, RL, Self-evolving

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_napROpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes harness learning, which trains a proposer model to revise the executable workflow around a frozen solver using execution feedback. The method optionally initializes the proposer through supervised fine-tuning on teacher-generated revisions, followed by RL based on revised-harness performance. At test time, both models remain frozen, while the harness is iteratively modified using target-task development feedback. Experiments on Reasoning Gym and multi-hop QA investigate transfer to unseen tasks, successive revision, and single-step versus multistep training.

Critical strengths

The paper addresses a practically relevant problem: improving agent behavior through executable workflow changes without updating the solver’s parameters.

Critical weaknesses
  1. Limited novelty and insufficient comparison with closely related work The framework substantially overlaps with existing methods that learning agent-harness construction and revision. As described in the paper’s related work section, Harness-R1 already combines supervised initialization and reinforcement learning for an editor with a frozen solver, while JIT-Agent studies task-conditioned harness generation and revision. The paper does not clearly identify the contribution that distinguishes its approach from these methods. The meta-learning formulation primarily reframes the revision process, while optimization still uses immediate, per-revision rewards. Without direct empirical comparisons against closely related approaches, the incremental contribution remains unclear.

  2. The adaptation setting requires clearer qualification Conventional unsupervised TTA assumes access to unlabeled target-domain data. Here, adaptation uses target-task development examples and explicit supervision. QA feedback includes correctness, gold answers, and supporting-document information. Candidate harnesses are also selected using target-task scores. Although final evaluation examples are held out, the harness has already been optimized using supervision from the same target task. The results therefore establish supervised target-task adaptation more directly than conventional unsupervised TTA. This distinction should be prominent.

  3. Performance-dependent task selection weakens the generalization evidence The evaluation subsets are partly determined by observed performance. Appendix B.1 excludes five difficult OOD families from the main comparison. More concerningly, Appendix C.2 and Table 12 select additional multi-step evaluation families using earlier results, including criteria favoring the ordering RL > SFT > Base. Although these choices are disclosed, they introduce selection bias into the aggregate evaluation. The generalization claims should be supported by comprehensive task coverage or a subset defined independently of the compared methods’ results.

  4. The presentation is unnecessarily difficult to follow Table 1 compresses heterogeneous information into a crowded layout, making the experimental settings difficult to compare. Figures 3 and 4 contain overlapping information without clearly separating the purpose of each comparison. Across Figures 3–5, numerous colors, legends, markers, and line styles are used without a sufficiently consistent visual encoding. Readers must repeatedly distinguish training configurations, revision protocols, averages, and oracle statistics. The presentation would benefit from fewer redundant panels, standardized symbols, and a clearer separation between development-selected results and held-out oracle statistics.

Questions for authors

See above.

6 comments

W1. Novelty relative to harness-evolution work Harness-R1 and JIT-Agent, the two most related works, appeared on arXiv on 3 and 26 August 2026, a few weeks before the deadline, and we treat them as concurrent work. We cite and discuss both (Section 2 and Appendix E). Our work differs from them in four ways:

  1. Transfer to unseen tasks. We test the trained proposer on task families and benchmarks it never saw in training: 21 unseen Reasoning Gym families, and MuSiQue and 2WikiMultihopQA for a proposer trained only on HotpotQA. Harness-R1 tests on new instances of its three training benchmarks. JIT-Agent evaluates on other benchmarks but does not say whether their task types were kept out of training.
  2. Successive revision at test time that emerges from single-step training. Our frozen proposer trained with single-step RL keeps revising its own revised harnesses, and in QA the harness keeps improving for ten rounds on unseen benchmarks (Fig. 4). Harness-R1 applies one patch per instance, and JIT-Agent repairs a harness at most twice.
  3. Whole-program revision. Our proposer can rewrite any part of the harness, including the prompts, the answer parsing, and the steps that run as code. Harness-R1 adds hook functions at four fixed points of an existing agent loop, and JIT-Agent fills in four modules with fixed interfaces.
  4. Learning without a teacher. In QA, we train our proposer with RL directly from the base model, without teacher demonstrations, and it has only 4B parameters. Harness-R1 starts its 9B editor from about 1,000 edits written by GPT-5.5, and JIT-Agent trains its 27B generator on data from a stronger teacher before RL.

The meta-learning formulation is what makes the generalization across tasks claim precise. It specifies what is adapted (executable harness code), the adaptation rule (the proposer), and how transfer is measured (held-out tasks), so successive revision at test time becomes the inner loop of a learned adaptation procedure. Training uses immediate per-revision rewards, and successive improvement at test time already emerges from this single-revision training (Fig. 4). A faithful controlled reimplementation of the two concurrent methods is not feasible within the rebuttal period, because they use different harness interfaces, benchmarks, and target agents. We will extend Appendix E with the comparison above.

W2. The adaptation setting is supervised target-task adaptation

We agree that this distinction is important. Supervised adaptation from a small labeled development set is the setting our method is designed for. Section 4 defines adaptation with a feedback set and a scoring set of labeled development questions and keeps the evaluation questions disjoint from both, and the paper does not describe the setting as unsupervised.

Supervised adaptation from a small labeled set matches how adaptation is done in practice. A coding task, for example, usually comes with a few tests with known expected outputs, which are used to check a solution before it is deployed on inputs it has not seen. Our development set plays the role of those tests, and the held-out set plays the role of the unseen inputs. The same structure is the standard few-shot meta-learning setting: in MAML, the inner loop adapts from a labeled support set, and evaluation uses a disjoint query set. Search-based harness optimizers (AFlow, Meta-Harness, AutoHarness [1,2,3]) also optimize against labeled task scores, and they must run their search anew on every task.

Labels enter our results at three points:

  • The single-step means involve no selection: every proposal counts, failed ones at the seed's score. Labels enter only through the round-1 execution report.
  • Multistep adaptation on Reasoning Gym keeps a revision only if its development score improves, and QA picks the best of eight by development exact match.
  • Oracle best-of-N numbers select on held-out scores, and the paper labels them as oracle statistics.

Selection without labels. In our method, the development gate uses labeled questions to decide which proposed harness to keep. We tested a version of this decision that uses no labels, so that adaptation needs only unlabeled questions from the target task. Our proposer was never trained for this step, so some loss against the labeled gate is expected. The harness is selected in three steps:

  1. The frozen solver answers 25 unlabeled questions of the task with each harness: the seed and each proposal.
  2. Where the harnesses give different answers, the frozen proposer checks each answer on its own. It sees the task family, the question, and the proposed answer, with no reference answer.
  3. The harness with the most answers judged correct is kept. If the seed has the most, no proposal is kept.

The harness is selected on one half of the held-out questions and scored on the other half, and then the halves swap. Every rule in the table, including the labeled gate, is scored the same way, averaged over 17 unseen families with wins/ties/losses in parentheses.

proposerseedrandom candidateselection without labelsdevelopment gate (labels)
Base0.6210.3390.7230.760
Single-step RL0.6210.5960.7960.834

Without any labels, the Single-step RL proposer improves on the seed by +0.176 (14/2/1), leading over the Base proposer by +0.074 (11/0/6). We will update the numbers when all results are out. We will add this label-free variant to the appendix.

Updates on Nov 18th: we have finished all 21 tasks and the table is as below:

proposerseedrandom candidateselection without labelsdevelopment gate (labels)
Base0.5550.3320.6910.718
Single-step RL0.5550.6230.7920.821

Without any labels, the Single-step RL proposer improves on the seed by +0.237 ± 0.070 (18/2/1) and keeps 89% of the labeled gate's gain. Its lead over the Base proposer is +0.100 ± 0.043 (13/1/7), almost the same as under the labeled gate (+0.103 ± 0.041). The Single-step RL proposer without labels scores above the Base proposer with labels (0.792 vs 0.718).

W3. Performance-dependent task selection

Single-step (Appendix B.1). The paper already reports all 26 unseen families. With all 26, the scores are Base 0.282, SFT 0.415, Single-step RL 0.528, and teacher 0.501, in the same order as on the 21 families of the main comparison (0.32, 0.49, 0.62, and 0.56), and the 5 extra families are hard for every method, including the teacher and the fixed tool loop (Table 9). The main comparison excludes 5 unseen families: knight_swap, letter_jumble, modulo_grid, sokoban, and ab. The stated criterion is that both the 4B proposers and the teacher find them hard to address with an algorithmic harness.

Multistep (Appendix C.2, Table 12). The paper's 12 additional multistep families were chosen from a pool of 22 unseen families using earlier results, and we thank the reviewer for pointing this out. To remove this selection, we ran the other 10 families of the pool with the same protocol. In summary, the order of models does not change with single-step RL scoring much higher on the format-varied data. The changes in scores are within the confidence intervals of already reported numbers.

Canonical questions (22 families; seed 0.552):

methodround 1round 5round 5 − seedround 5 − Baseselected 12: round 5 − Base
Base0.5730.657+0.104 ± 0.053ref.ref.
Single-step RL0.6340.662+0.110 ± 0.048+0.005 ± 0.037 (10/5/7)+0.047 ± 0.061
Multistep RL0.6740.705+0.153 ± 0.056+0.048 ± 0.035 (11/4/7)+0.104 ± 0.058

Format-varied questions (21 families*; seed 0.519):

methodround 1round 5round 5 − seedround 5 − Baseselected 12: round 5 − Base
Base0.6120.617+0.098 ± 0.055ref.ref.
Single-step RL0.6720.714+0.195 ± 0.067+0.097 ± 0.035 (9/8/4)+0.152 ± 0.050
Multistep RL0.6520.652+0.133 ± 0.058+0.035 ± 0.033 (7/7/7)+0.050 ± 0.051
  • One family was excluded from the format-varied comparison since the Base proposer declined to edit in all five rounds.

  • Every trained proposer ends at or above Base on both question formats.

  • The methods rank in the same order as on the 12 selected families.

In the revision, the main text will report the unselected family numbers next to the selected ones, and the Fig. 3 caption will state how its families were chosen. The QA multistep results (Fig. 4) involve no task selection: each benchmark is evaluated in full on its held-out split.

Reviewer_oAEbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces harness learning task via SFT and RL to focus on training a policy (proposer) that focuses on revising current harness to help fixed solver to resolve a task.

The main contribution of the paper is that it introduces a training paradigm for learning a policy to improve agent's harness. The overall approach seems to be generalizable as the proposer takes input as task description, current harness, and execution feedback and produces a harness revision.

The method shows the following benefits:

  1. approach leads to more useful revisions (the scores on held-out problems are 0.32, 0.49, and 0.62 for base, SFT and RL ckpts, respectively)
  2. the paper also shows some adaptability or transferability. 2.a. proposer trained on HotpotQA shows strength on MuSiQue and 2WikiMultihopeQA, and performance continues improving over continuous revision 2.b transferrability is also observed on reasoning gyms across task families, but evidence is weaker.
Critical strengths
  1. the method shows clear evidence that the proposer's revision goes beyond training tasks.
  2. the single-step training seems to be simple yet effective on QA tasks.
  3. the analysis section provides detailed information about what exactly are revised in both tasks, which helps reader understand better of the output of the trained proposer.
Critical weaknesses

I have following doubts on this paper's novelty claims:

  1. Unclear how much value task-specific adaptation adds over a stronger harness. While the method shows benefits against base ckpts, the performance remains lower than tool-loop baseline (Table 9). Providing experiments on adaptation from such stronger baseline would be helpful.

  2. Novelty - The paper mentions the related work "JIT-Agent" learns task-conditioned harness generation, repair, and evolution. However, the method in this paper seems also to be task-dependent as the input includes task descriptions. Also, similarly, the jit-agent method generates executable and task-specific harness, learns to revise existing harness using execution feedback, training harness evolution policy from execution outcomes, shows improvements across domains such as shopping, travel, officeBench, etc.

  3. Novelty - I also see substantial overlap between this paper and Harness-R1 that also trains harness editor via outcome-based GRPO using frozen solver. Harness-r1 also shows generalization performance.

Questions for authors
  1. can you provide more details on how this papers distinguishes from the harness-r1 and jit-agent papers?
  2. is there any controlled experiments to demonstrates the benefits of the proposed approach against harness-r1 and jit-agent?
  3. does adaptation provides additional gains when baseline harness is stronger?
3 comments

Q1. Novelty relative to harness-evolution work

Harness-R1 and JIT-Agent, the two most related works, appeared on arXiv on 3 and 26 August 2026, a few weeks before the deadline, and we treat them as concurrent work. We cite and discuss both (Section 2 and Appendix E). Our work differs from them in four ways:

  1. Transfer to unseen tasks. We test the trained proposer on task families and benchmarks it never saw in training: 21 unseen Reasoning Gym families, and MuSiQue and 2WikiMultihopQA for a proposer trained only on HotpotQA. Harness-R1 tests on new instances of its three training benchmarks. JIT-Agent evaluates on other benchmarks but does not say whether their task types were kept out of training.
  2. Successive revision at test time that emerges from single-step training. Our frozen proposer trained with single-step RL keeps revising its own revised harnesses, and in QA the harness keeps improving for ten rounds on unseen benchmarks (Fig. 4). Harness-R1 applies one patch per instance, and JIT-Agent repairs a harness at most twice.
  3. Whole-program revision. Our proposer can rewrite any part of the harness, including the prompts, the answer parsing, and the steps that run as code. Harness-R1 adds hook functions at four fixed points of an existing agent loop, and JIT-Agent fills in four modules with fixed interfaces.
  4. Learning without a teacher. In QA, we train our proposer with RL directly from the base model, without teacher demonstrations, and it has only 4B parameters. Harness-R1 starts its 9B editor from about 1,000 edits written by GPT-5.5, and JIT-Agent trains its 27B generator on data from a stronger teacher before RL.

We will extend Appendix E with this comparison.

Q2. Controlled comparison against Harness-R1 and JIT-Agent

We agree that a controlled comparison would be informative. Both works are concurrent arXiv preprints and use different harness interfaces, benchmarks, and target agents (Harness-R1: hooks into WebShop, ALFWorld, and DBBench agents; JIT-Agent: a four-module protocol with a 27B generator), so a faithful controlled reimplementation does not fit in the rebuttal period. We will extend Appendix E with this comparison and list controlled comparisons as future work.

Q3. Does adaptation add gains when the baseline harness is stronger?

methodtool loopmean of 8 proposalsgate-promoted pickbest of 8 (oracle)mean of 8 − tool loopgate pick − tool loop
Base0.6860.620 ± 0.0650.740 ± 0.0540.776 ± 0.052−0.066 ± 0.021 (5/1/15)+0.053 ± 0.036 (11/2/8)
Single-step RL0.6860.600 ± 0.0580.754 ± 0.0580.794 ± 0.050−0.086 ± 0.030 (3/1/17)+0.067 ± 0.039 (12/2/7)

The tool loop is already a strong harness. On the 21 unseen families, it scores 0.69, above the average revision that the 35B teacher makes to the one-call seed (0.56), so there is less room to improve than from the one-call seed. Adaptation still improves it. The gate-promoted harness beats the tool loop by +0.067 (Single-step RL).

We also ran multistep adaptation from a self-consistency harness, which samples three solver answers and asks the solver to settle disagreements. On the 21 unseen families, it scores 0.599 on held-out questions, below the tool loop. After five rounds, the kept harness reaches 0.700 with Base (+0.101 ± 0.056) and 0.795 with Single-step RL (+0.196 ± 0.061, 13 wins, 1 loss), and Single-step RL ends above Base by +0.095 ± 0.049 (11/6/4). Results are listed in the table below.

methodseedround 1round 5round 5 − seedround 5 − Base
Base0.5990.615 ± 0.0710.700 ± 0.058+0.101 ± 0.056 (8/10/3)ref.
Single-step RL0.5990.696 ± 0.0480.795 ± 0.047+0.196 ± 0.061 (13/7/1)+0.095 ± 0.049 (11/6/4)

I thank the authors for addressing my questions and providing follow-up experiments. My concern on adaptation gain is partially addressed that adaptation can improve a strong harness when combined with candidate selection. The remaining uncertainty i have is how reliably the trained proposer improves over an untrained proposer given the same revision and selection budget. The benefit from training the proposer is less clear for the strongest starting harness.

The novelty concern remains. Based on authors' response, it seems that the main contribution is demonstrating transfer to unseen task families and successive test-time adaptation from single-step training, instead of a new harness-learning paradigm fro methodological perspective. This should be made explicit in the paper.

I would maintain my original score.

Reviewer_889COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focus on harness learning, which trains a proposer model to revise a solver’s harness using execution information. The authors formulate it as meta-learning over executable programs, with harness revisions playing the role of weight updates in gradient-based adaptation. They optimize the proposer with the revised harnesses through RL. And the proposer uses feedback from successive executions on a new task to refine the harness at test time. Experiments across reasoning and multi-hop question answering show some improvements.

Critical strengths
  1. The paper is well organized and easy to follow. The illustrations are vivid and easy to understand.
  2. The paper studies a timely and promising problem, harness learning. Given rapidly improving model capabilities and the evolving agentic ecosystem, a fixed, hand-engineered harness is unlikely to remain optimal. Learning the harness so that it co-evolves with the underlying model is a well-motivated direction for solving more complex tasks.
  3. The proposed method is conceptually simple, intuitive, and easy to follow. It first leverages a strong teacher model to improve the proposer, and then iteratively refines the harness at test time. These two stages complement each other and jointly improve overall performance.
Critical weaknesses
  1. The proposed method is relatively straightforward, and its novelty appears limited compared with existing work on harness evolution. Both core components, training the proposer via reinforcement learning and editing the harness code based on agentic feedback, are fairly common techniques in the current literature. As a result, the technical contribution of the paper is somewhat incremental.
  2. The experimental setup is rather limited in terms of both the models and the benchmarks used. Specifically, the proposer has only 4B parameters, whereas existing work on harness evolution typically builds on recent, more capable LLMs. In addition, the evaluation covers only a simple multi-hop QA task and a single reasoning dataset. It lacks more challenging agentic benchmarks, such as the Terminal-Bench series. As a result, it remains unclear whether the proposed method would still yield similar gains with stronger models and on more difficult tasks.
Questions for authors

N/A

3 comments

We thank the reviewer for finding the problem timely and the method simple and easy to follow. Below we explain what distinguishes our work from harness evolution and from concurrent learned editors (W1). We also extend our experiments to a larger solver model, and our trained proposer still clearly outperforms the untrained one there (W2).

W1. Novelty relative to harness-evolution work

We study a different problem from harness evolution. Search-based methods (AFlow, Meta-Harness, AutoHarness [1,2,3]) optimize the harness of one task and start over on every new task without any training on the proposer model. Instead, we train a proposer that improves harnesses for tasks it never saw in training for multiple rounds. We see the simplicity of the method as an advantage. Training uses standard SFT and RL with no hand-designed search or task-specific components, so it is easy to scale to larger systems and new settings.

Harness-R1 and JIT-Agent, the two most related works, appeared on arXiv on 3 and 26 August 2026, a few weeks before the deadline, and we treat them as concurrent work. We cite and discuss both (Section 2 and Appendix E). Our work differs from them in four ways:

  1. Transfer to unseen tasks. We test the trained proposer on task families and benchmarks it never saw in training: 21 unseen Reasoning Gym families, and MuSiQue and 2WikiMultihopQA for a proposer trained only on HotpotQA. Harness-R1 tests on new instances of its three training benchmarks. JIT-Agent evaluates on other benchmarks but does not say whether their task types were kept out of training.
  2. Successive revision at test time that emerges from single-step training. Our frozen proposer trained with single-step RL keeps revising its own revised harnesses, and in QA the harness keeps improving for ten rounds on unseen benchmarks (Fig. 4). Harness-R1 applies one patch per instance, and JIT-Agent repairs a harness at most twice.
  3. Whole-program revision. Our proposer can rewrite any part of the harness, including the prompts, the answer parsing, and the steps that run as code. Harness-R1 adds hook functions at four fixed points of an existing agent loop, and JIT-Agent fills in four modules with fixed interfaces.
  4. Learning without a teacher. In QA, we train our proposer with RL directly from the base model, without teacher demonstrations, and it has only 4B parameters. Harness-R1 starts its 9B editor from about 1,000 edits written by GPT-5.5, and JIT-Agent trains its 27B generator on data from a stronger teacher before RL.

We will extend Appendix E with this comparison.

W2. Model scale and benchmarks

We agree that performing a larger-scale experiment will better validate the benefit of the proposed method.

  • Compute: Our current experiment is already on the edge of what an academia project can afford (every RL update runs the proposed harnesses with the solver in the loop, so each RL run of our 4B proposer uses at least 8 H100 GPUs (4 train the proposer and 4 serve the frozen solver) for about 18 hours, due to the nature of RL training, not to mention the exploration of training recipes and different benchmarks.
  • Training matters more than size: Due to the simplicity of our proposed algorithm, we are confident that the results we observe on a smaller scale will generalize to a larger scale, as indicated by other RL studies of a similar flavor [4,5].
  • Outperforming the Teacher: The reviewer's concern about the 4B proposer is eased by one result in the paper. After training, our 4B proposer writes better revisions on average than its 35B teacher (0.62 vs 0.56 on the 21 unseen families), so training matters more than size here. Training a frontier-scale proposer is beyond our compute, since every RL update executes the proposed harnesses with the solver in the loop.

New experiment: a stronger frozen solver. We kept the trained 4B proposers unchanged and replaced the 4B solver with a 35B solver, to test whether learned revisions still help when the solver is stronger.

The experiment setup is as below:

  • using the same setting as the paper’s seed harness and tasks, and 25 development and 75 held-out questions per family;
  • Qwen3.6-35B-A3B as the frozen solver, in non-thinking mode with the same tool-call parser and sampling settings as the 4B solver;
  • the Base and Single-step RL proposers, unchanged, under the single-step protocol.
  • the families: of the 21 unseen families, those where the seed scores at most 0.70 on the development questions with the 35B solver, taking the lowest 6.

With the 35B solver, the harness from our RL-trained proposer reaches 0.82 on held-out questions, compared to 0.60 for the untrained proposer, and it improves on the seed in all six families.

The table below reports held-out scores, as the mean over the 6 families ± SE. Differences are paired over families, with wins/ties/losses in parentheses.

methodseedmean of 8 proposalsgate-promoted pickbest of 8 (oracle)mean of 8 − seedgate pick − seed
Base0.1990.295 ± 0.1050.600 ± 0.1550.616 ± 0.147+0.096 ± 0.079 (3/0/3)+0.401 ± 0.138 (4/1/1)
Single-step RL0.1990.527 ± 0.0780.823 ± 0.1320.827 ± 0.133+0.328 ± 0.085 (5/0/1)+0.624 ± 0.142 (6/0/0)
  • The gain over the seed is +0.62 with the RL-trained proposer and +0.4 with the untrained one. Single-step RL's gate-promoted harness improves on the seed in all 6 families. The gains are universal across the six tasks and not concentrated to one task.

We will include this experiment in the revised version of the paper.

Benchmarks. Reasoning Gym provides many structurally different task families with exact graders, which is a good setup for studying generalization: we train on some families and evaluate on families excluded from training. The QA setting is a multi-step retrieval workflow, with transfer from HotpotQA to MuSiQue and 2WikiMultihopQA. Terminal-Bench needs containerized environments and a solver strong enough to make progress on its tasks, which a 4B solver is not. We list harder agentic benchmarks such as Terminal-Bench as future work.

[1] AFlow: Automating Agentic Workflow Generation

[2] Meta-Harness: End-to-End Optimization of Model Harnesses

[3] AutoHarness: improving LLM agents by automatically synthesizing a code harness

[4] Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model

[5] SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Area Chairs and Reviewers

General Response: We thank the reviewers for their careful reviews. Below, we highlight the strength recognized by the reviewers, and our response to the general concerns.

Strengths identified by the reviewers

  • The problem is timely, since a hand-engineered harness is unlikely to stay optimal as models improve (889C). It is practically relevant, since it improves agents without updating the solver (napR).
  • The method is conceptually simple and intuitive, and its training and test-time stages complement each other (889C).
  • The proposer's revisions transfer beyond the training tasks, and single-step training is simple yet effective on QA (oAEb).
  • The analysis shows what the proposer revises (oAEb), and the paper is well organized with clear illustrations (889C).

Questions and our responses

1. Novelty relative to harness evolution, Harness-R1 and JIT-Agent (889C, oAEb, napR). Our focus is different from these works: we focus on how to train the harness proposer to generalize to new, unseen tasks, whereas search-based harness evolution (AFlow, Meta-Harness, AutoHarness [1,2,3]) optimizes the harness of one given task, which can not generalize to other tasks. Harness-R1 and JIT-Agent appeared on arXiv on 3 and 26 August 2026, weeks before the ICLR submission deadline, and we cite and discuss both. We believe these are contemporary works. That said, four properties that our experiments measure set our work apart from them:

  • transfer to whole task families and benchmarks held out of training, which neither paper reports testing
  • successive revision at test time that emerges from single-step training
  • revision of the whole harness program, where Harness-R1 adds hook functions at four fixed points of an existing agent loop, and JIT-Agent fills four modules with fixed interfaces
  • a 4B proposer that, in QA, learns by RL without any teacher. We will extend Appendix E with this comparison.

2. Controlled comparison with Harness-R1 and JIT-Agent (oAEb, napR). We agree that a controlled comparison would be valuable. Both works appeared shortly before the deadline, so we treat them as concurrent works, and they use different harness interfaces, benchmarks, and target agents, so a faithful reimplementation is rather impractical in the rebuttal period. We will discuss both in detail in Appendix E and list a controlled comparison as future work.

New experiments. We ran four new experiments for the reviewers' questions:

  1. A larger solver (889C). The trend from our smaller experiments holds with a larger 35B solver. Specifically, the harness from our RL-trained proposer reaches 0.82 on held-out questions, against 0.60 for the untrained proposer.
  2. A stronger starting harness (oAEb). Adaptation helps even when the starting harness is already strong. Specifically, starting from the tool loop, the RL-trained proposer improves it by +0.067, more than the untrained proposer (+0.053).
  3. Selection without labels (napR). Firstly, we study a different problem that focuses on generalization to different tasks under supervised harness selection. Secondly, when the frozen proposer judges answers in place of the labeled development gate, the Single-step RL proposer improves on the seed by +0.176.
  4. No task selection (napR). Our conclusions do not depend on which families we chose. Specifically, on all 22 families of the pool, every method improves on the seed, and every trained proposer ends at or above the untrained one. Presentation (napR). We will simplify Table 1, use one visual encoding for methods across Figures 3 to 5, state the purpose of each figure in its caption, and separate development-selected results from oracle statistics.

[1] AFlow: Automating Agentic Workflow Generation

[2] Meta-Harness: End-to-End Optimization of Model Harnesses

[3] AutoHarness: Improving LLM agents by automatically synthesizing a code harness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.