acceptodds
Under review as a conference paper at ICLR 2027

Mixture of Self-Improving Branches for Agent Harness Optimization

11 views

Abstract

Harness optimization provides a practical setting for recursive self-improvement (RSI), where agent-generated modifications inform subsequent changes through execution feedback. Recent work such as Meta-Harness implements this process through iterative code generation and evaluation, but retains a fixed development set and proposal policy. These constraints channel evolution along a single search trajectory, increasing the risk of converging to a local optimum. We make the improvement process itself adaptive by organizing search into branches with evolving development subsets and proposal policies. Each branch retains development cases solved by more of its leading harnesses than by those of other branches, drops cases solved by every leading harness across all branches, and revises its proposal policy using its own search history. To deploy the resulting complementary harnesses, we propose a router to select one development-selected branch head for each new input before execution. Across mathematical reasoning and agentic coding benchmarks, our system achieves relative improvements over Meta-Harness of 34.8% on Olympiad-level mathematical reasoning, 11.6% on Terminal-Bench 2.0, and 3.8% on SWE-bench Lite, with harness selection and router configuration based solely on development data. These results show that evolving branch objectives and proposal policies can discover complementary harnesses whose strengths a router combines without access to test outcomes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent harness optimization, Recursive self-improvement, Branching search

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZQyFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Mixture of Self-Improving Branches to address limited search diversity in automated agent harness optimization. The method dynamically adjusts branch-specific development subsets based on comparative frontier coverage and updates proposal guidance using each branch's search history. A router then selects between the final branch heads for new tasks.

Experiments on mathematical reasoning, Terminal-Bench 2.0, and SWE-bench Lite show improvements over Meta-Harness, with the largest gain of 16 percentage points on Math–Gemini.

Critical strengths

Meaningful problem and concrete mechanism. The paper addresses the limitations of optimizing harnesses against a fixed development set. Dynamic task assignment provides a potentially useful mechanism for encouraging complementary search trajectories.

Evidence of branch specialization. Figure 2 and Appendix C illustrate distinct harness mechanisms and evolving proposal guidance across branches. Table 3(b) provides encouraging evidence that combining development-set updates with guidance adaptation improves performance.

Held-out evaluation and complementarity analysis. The paper evaluates development-selected harnesses on held-out tasks and demonstrates complementary capabilities between the resulting branch heads.

Critical weaknesses

1.Insufficient explanation of how dynamic task assignment induces specialization.

Dynamic development-set assignment is the paper's main methodological contribution. While the trajectory and guidance analyses demonstrate that branches evolve differently, the causal role of task assignment remains unclear.

In particular, the final development subsets overlap substantially in some settings. For Math–Sonnet, the two branches retain 92 and 103 tasks, sharing 81; for SWE-bench Lite, they retain 27 and 28 tasks, sharing 24. The paper does not analyze whether the branch-specific tasks differ meaningfully in category, difficulty, or failure modes.

Moreover, independent searches can also discover different harness mechanisms. Although Table 3(b) provides a useful fixed-development baseline, it does not compare the resulting branch complementarity. Stronger evidence is needed to establish the specific contribution of dynamic task assignment.

  1. Large performance differences across action models remain unexplained.

The improvement over Meta-Harness is substantial on Math–Gemini (+16.0 points) but much smaller on Math–Sonnet (+1.5 points), despite using the same mathematical reasoning setting.

It is unclear whether the large Gemini gain reflects the general effectiveness of the proposed search mechanism or model-specific characteristics, such as baseline capability and sensitivity to harness design.

Both action models are also relatively old compared with models available before submission. Evaluation on a newer model, together with an analysis of the different search outcomes, would strengthen the generalizability claim.

  1. Baseline search configuration and computational fairness need clarification.

The proposed method uses two branches with 20 iterations each, but the Meta-Harness baseline's search iterations and candidate budget are not sufficiently specified.

Table 3(b) includes a useful comparison against two independent fixed-development searches, but only for Math–Gemini. Appendix A also reports substantially higher total token consumption for the proposed method in several settings.

This makes it difficult to separate the benefits of dynamic specialization from those of additional search resources. Matched-budget comparisons and repeated full search runs would strengthen the conclusions.

Questions for authors
  1. Dynamic development subsets and specialization.

Could the authors analyze how the final branch-specific development subsets differ in task categories, difficulty, or failure modes? Given their substantial overlap in some settings, what evidence connects dynamic task assignment to the distinct harness mechanisms and complementary test performance? How does this complementarity compare with two independent fixed-development searches?

  1. Differences across action models.

What explains the large discrepancy between Math–Gemini (+16.0 points) and Math–Sonnet (+1.5 points)? Have the authors examined whether these differences arise from model capabilities, search trajectories, or harness-specific effects? Does the method retain its benefits with newer action models?

  1. Search budget and baseline configuration.

Could the authors clarify the Meta-Harness baseline's search iterations, number of generated candidates, and total compute budget? How does the proposed method compare with a longer single Meta-Harness search or two independent searches under matched candidate or token budgets?

  1. Robustness and implementation.

How are missing coverage evaluations and historical candidate rankings handled after development cases are removed from a branch?

2 comments

We thank Reviewer ZQyF for the detailed questions about specialization, variation across action models, baseline comparisons, and implementation. We address each concern below.

Question 1: Development-subset differences and complementary mechanisms

Our analysis characterizes branch strengths by the mechanisms that solve different cases. Section 4.3 identifies concrete answer checks and proof-obligation routing in Branch 1, and free-form derivation, two-sided closure, and agreement challenges in Branch 2. The polynomial example shows how Branch 1’s counterexample check recovers a valid family on a case missed by Branch 2.

Development-subset overlap does not imply identical search objectives: branch-exclusive cases can change candidate rankings, selected heads, and subsequent proposals. Appendices B.1–B.2 document this process. Shared SWE-bench Lite cases decrease from 30 to 24 as exclusive cases increase from 1 to 7. At Math–Gemini iteration 13, Branch 2’s subset shrinks from 44 to 40 cases and a free-form derivation harness becomes the head, which later proposals extend through progressive refinement and two-sided closure. With proposal adaptation enabled, adding development pruning increases Math–Gemini accuracy from 50.0% to 62.0%.

Revision: Section 4.3 now explicitly connects the branch-exclusive cases to differing search objectives and points to the recorded examples in Appendices B.1–B.2.

Question 2: Model-dependent gains and newer-model transfer

We interpret the difference between the Gemini and Sonnet gains as reflecting the action models’ underlying capabilities: a harness can improve how a model approaches a task, while its benefit can remain constrained by the model’s task-solving capability.

We also evaluate transfer to Gemini 3.8 Flash using the best single harness generated with Gemini 3 Flash as the action model. The transferred harness achieves 75.0% accuracy, compared with 68.0% for BM25 retrieval, a gain of 7.0 percentage points. The transfer uses the existing evolved harness rather than rerunning evolution with the newer model.

Table 5: Test accuracy (%) across action models

MethodGemini 3 FlashGemini 3.8 Flash
BM25-all40.568.0
Branch 1 head56.075.0
Branch 2 head58.075.0

Question 3: Search budget and repeated baseline configuration

Section 4.5 includes two independent Meta-Harness runs on Math–Gemini, achieving 46.0% and 51.0% test accuracy. We report the better individual test result, 51.0%, compared with 62.0% for our method. This is an 11.0-percentage-point improvement even with favorable test-based selection for Meta-Harness.

Revision: Section 4.2 reports the development/test scores of both runs. Section 4.5 explicitly distinguishes the better individual baseline result from routed deployment. Appendix A reports full-run token usage.

Question 4: Missing coverage evaluations and historical rankings after pruning

All candidates are scored on the current development set using their stored per-case evaluations. This is feasible because we only remove cases: after pruning, we recompute each candidate’s mean reward on the retained cases and update the rankings.

Revision: Appendix B describes the rescoring procedure.

Thanks for clearing that up and running the extra tests. That adressed my major concerns. I'll increase my score up accordingly. Thanks for the effort!

Reviewer_zqioOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Mixture of Self-Improving Branches, a framework for agent harness optimization that addresses the limitations of fixed development sets and proposal policies in existing harness evolution methods. The approach maintains multiple search branches, dynamically adjusts their development subsets based on relative task-solving performance, and updates branch-specific proposal guidance using historical search experience. After evolution, an LLM-based router optimized with GEPA selects an appropriate harness for each new task. Experiments on mathematical reasoning, Terminal-Bench 2.0, and SWE-bench Lite demonstrate improvements over Meta-Harness, suggesting that adaptive multi-branch evolution can produce complementary harnesses and improve overall task performance.

Critical strengths
  1. Interesting and well-motivated research problem. The paper identifies a meaningful limitation of existing harness evolution methods: optimizing against a fixed development set may overlook complementary capabilities and lead to suboptimal search trajectories. The motivation for encouraging specialization through multiple evolving branches is interesting and reasonable.

  2. Effective and intuitive methodology. The proposed framework is relatively straightforward and easy to follow. Dynamically adjusting development subsets and updating branch-specific proposal guidance provide a coherent approach to encouraging complementary harnesses. The experimental results demonstrate the effectiveness of combining these mechanisms.

  3. Promising empirical results and useful analysis. The paper evaluates its approach across mathematical reasoning and agentic coding benchmarks, showing improvements over Meta-Harness. The ablation studies and search trajectory analyses further help illustrate how the proposed components contribute to the observed improvements.

Critical weaknesses
  1. Insufficient justification for routing rather than combining evolved harnesses. The ultimate goal of harness evolution is to improve task-solving performance. Although the proposed framework discovers complementary harnesses, the authors do not explain why they choose to route tasks to a single harness rather than combine their capabilities into a unified workflow or aggregate their outputs. Such alternatives may better exploit the improvements discovered during evolution. Further justification and empirical comparisons are needed to establish the necessity and effectiveness of the routing strategy.

  2. Potential overfitting and insufficient consideration of execution randomness. The dynamic reduction of development subsets raises concerns about overfitting, particularly in mathematical reasoning. For example, in Math–Gemini, the development subsets eventually shrink from 250 problems to only 39 and 31 for the two branches, respectively(page 14, figure 3). This may encourage harnesses to overfit a small number of specific cases rather than learn generalizable strategies. Moreover, the paper does not clearly specify how many independent rollouts are performed for each harness–task pair. Given the stochastic nature of LLM agents, individual successes or failures may be unreliable indicators of harness capabilities, potentially affecting development-set pruning and subsequent evolution. Repeated evaluations and variance analysis would help establish the robustness of the proposed method.

Questions for authors

I have no additional major questions beyond the concerns raised in the weaknesses. I would appreciate it if the authors could address the following:

  1. Could the authors further justify the choice of routing rather than combining evolved harnesses, ideally with empirical comparisons against harness integration or output aggregation strategies?

  2. Could the authors clarify how many independent rollouts are performed for each harness–task pair and provide evidence that development-set pruning and the resulting harness improvements are robust to execution randomness and potential overfitting?

3 comments

We thank Reviewer zqio for the constructive questions about deploying complementary harnesses, generalization after development-set pruning, and execution variability. We address the weaknesses and questions below.

Question 1: Empirical comparison against harness integration

We aim to create complementary harnesses and let a router select the appropriate one for each task. To compare with direct integration, the same proposer used during evolution, Claude Opus 4.6, is asked to combine the two validation-best Math–Gemini branch heads into one harness. The individual heads score 56.0% and 58.0%; their integration achieves 58.5%, below routing’s 62.0%.

Revision: The new paragraph at the end of Section 4.4 reports this direct integration comparison.

Question 2: Rollouts, generalization, and repeated evaluation

We use one rollout per harness–task pair during evolution to control evaluation cost and accelerate search. Final accuracy is measured on held-out test cases excluded from evolution, development pruning, and router optimization.

We repeat the evaluation of the validation-best Math–Gemini harness twice. The original and repeated accuracies are 58.0%, 58.5%, and 56.5%, respectively, giving a range of 2.0 percentage points. The original evaluation and the repeat achieving 58.5% have 81.5% solved-case overlap, versus approximately 70% between the two branch heads.

Revision: The single-rollout protocol is stated in Section 4.1. Appendix F reports the repeated evaluations and overlap definition; Section 4.3 summarizes the comparison.

Thank you for the additional experiments and clarifications. I appreciate the authors' efforts to address my concerns, particularly the direct comparison between harness integration and routing, which provides useful evidence supporting the proposed design.

Based on these additional results, I have decided to increase my score.

However, my concern regarding computational cost remains unresolved. The proposed method involves additional computation through multi-branch evolution, and it is still unclear to what extent the performance improvements come from the proposed design rather than the increased computational budget.

We thank the reviewer for the positive reassessment and for recognizing the integration-versus-routing comparison.

Our current experiments already examine allocating 40 configured search iterations, the budget used in our branch-based search, to two independent 20-iteration Meta-Harness runs on Math–Gemini. As reported in Section 4.5 and highlighted in the revised Section 4.2, even the better baseline test result, 51.0%, remains below our method’s 62.0%.

We further examine an alternative allocation that spends the search iterations on a single longer run. Extending Meta-Harness from 20 to 40 iterations gives the same 46.0% test accuracy for the development-selected harness. Favorable test-based selection increases this to 49.0%, still below our method’s 62.0%. The recorded proposals largely refine existing mechanisms rather than explore substantially different approaches. This suggests that additional iterations can continue refining a similar harness family, whereas our method adapts branch objectives and proposal guidance to encourage complementary search directions. We have updated Section 4.5 of the revised manuscript to report this longer-run comparison and discuss the observed concentration on related harness mechanisms.

Reviewer_iJb5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Agentic LLMs have the ability to interact with their environment through tools in order to take actions and receive feedback, but this is currently limited by the development of harnesses that offer these tools. Given the coding abilities of Agentic LLMs, recent work such as Meta-Harness proposes to have the agents design their own harnesses, but current methods mainly optimize for harnesses that cover general cases. The authors propose to instead specialize harnesses on specific scenarios while maintaining complementarity, and route requests to the most appropriate harness.

Critical strengths
  • Selection of harnesses is based on the development data and then validated on a held-out dataset, so the evaluation seems honest.
Critical weaknesses
  1. The harness candidate budget for the authors' method is 40, while Meta-Harness only generates 20. Section 4.5 matches them through 2 independent Meta-Harness runs, but this is not presented as the main comparison.
  2. There are several limitations of this work, which are not discussed. For example branching is fixed at B=2, so it is unknown how scale ownership scales as we increase the number of branches.
  3. The proposed pipeline combines existing techniques from literature for the specific case of evolving harnesses, reducing novelty.
  4. The abstract and stated contributions describe the results more strongly than what the evidence supports, only Math-Gemini shows encouraging results.
Questions for authors
  1. How often does a single harness disagree with itself across runs for Gemini-Math? This would clarify whether harnesses are truly complementary or the gains are partially caused by noise in the evaluation process.
  2. How did you choose the hyperparameters and the router configuration?
6 comments

Weakness 1: Baseline candidate budget and prominence of the repeated comparison

We thank the reviewer for pointing out that the repeated-baseline comparison should be more prominent. Section 4.5 includes two independent Meta-Harness runs on Math–Gemini, achieving 46.0% and 51.0% test accuracy. We report the better test result, 51.0%, giving the baseline favorable test-based selection. Our method achieves 62.0%, exceeding this reference by 11.0 percentage points.

Revision: We add a paragraph at the end of Section 4.2 reporting the development/test scores of both runs. Section 4.5 now explicitly identifies the 51.0% entry as the better individual test result, rather than a routed result. Appendix A reports full-run token usage.

Weakness 2: Scaling beyond two branches

Our goal is to evolve better individual harnesses with complementary strengths and then use a router to select the appropriate harness for each task. The search therefore improves the individual harnesses before combining their capabilities through routing. Extending this approach to more branches also calls for a router that can effectively use the additional harnesses. Our current experiments use B=2B=2 branches, and developing better routers to exploit the strengths of more branches is a direction for future work.

Revision: We state the two-branch scope and this future direction in the conclusion.

Weakness 3: Novelty relative to existing techniques

Adaptive case selection and specialist preservation have precedents in evolutionary search [1,2,3]. Our contribution focuses on coupling comparative-coverage task ownership with branch-specific proposal adaptation during agent-harness evolution. Comparative frontier coverage changes each branch’s retained development cases and hence which candidates are selected and extended. Branch-specific guidance then uses local search experience to adapt subsequent proposals. Together, these mechanisms direct branches toward complementary strengths that a router can use at deployment.

The Math–Gemini results support this combined design: the full method achieves 62.0% accuracy, compared with 54.0% for pruning alone, 50.0% for proposal adaptation alone, and 51.0% for the better of two fixed-development baseline repeats.

Revision: Section 2 now acknowledges adaptive case selection and specialist preservation [1,2,3] and explains the coupling studied in our work.

Weakness 4: Strength of the abstract and contribution statements

The manuscript reports numerical improvements across all four evaluated settings: +16.0 percentage points on Math–Gemini, +1.5 on Math–Sonnet, +5.2 on Terminal-Bench 2.0, and +2.4 on SWE-bench Lite. These results show gains of varying magnitudes, with the largest gains on Math–Gemini and Terminal-Bench 2.0.

References

[1] Christian W. G. Lasarczyk, Peter Dittrich, Wolfgang Banzhaf. Dynamic subset selection based on a fitness case topology. Evolutionary Computation, 12(2):223–242, 2004.

[2] Ryan Boldi, Martin Briesch, Dominik Sobania, Alexander Lalejini, Thomas Helmuth, Franz Rothlauf, Charles Ofria, Lee Spector. Informed Down-Sampled Lexicase Selection: Identifying Productive Training Cases for Efficient Problem Solving. Evolutionary Computation, 32(4):307–337, 2024.

[3] Thomas Helmuth, Edward Pantridge, Lee Spector. Lexicase Selection of Specialists. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), 2019.

Question 1: Single-harness disagreement across repeated evaluations

We evaluate the validation-best Math–Gemini harness again to measure how often the same harness changes its case-level outcomes. The original test accuracy is 58.0%, and the repeat achieves 58.5%. The ratio of cases solved in both evaluations to cases solved in either evaluation, i.e., the intersection over union of their solved-case sets, is 81.5%.

For the two branch heads, Figure 2(b) shows 47% shared success and 67% combined coverage, giving a corresponding overlap of 47/67≈47/67 \approx 70%. The lower overlap across branches indicates greater coverage differences between the two harnesses than between the specified repeated evaluations of a single harness. A second additional evaluation achieves 56.5%; the three accuracies are therefore 58.0%, 58.5%, and 56.5%, with a range of 2.0 percentage points.

Revision: Appendix F reports the repeat accuracies and defines the overlap measure. Section 4.3 briefly summarizes the within-harness versus between-branch comparison.

Question 2: Hyperparameters and router configuration

We set the number of iterations according to our computational budget. We update development subsets every iteration by design, expecting frequent updates to help search respond to emerging branch strengths. The ablation in Section 4.5 supports this choice: updates every 1, 3, and 5 iterations yield 62.0%, 56.0%, and 53.0% accuracy on Math–Gemini, respectively.

For routing, we compare three context configurations, each with and without GEPA optimization. The main results use the full-context configuration: expert source code, head-exclusive development examples, and both heads’ outputs on those examples. GEPA optimizes the routing instruction using development cases solved by exactly one final head, labeled by the successful head. For each new problem, the router selects a head before execution. Section 4.4 and Appendix D.2 report the ablations, which show small average gains from GEPA, with the largest benefit from full context.

I appreciate the authors' clarifications and the efforts to revise the manuscript. The writing in the current draft is still below the quality bar I hold for ICLR, most sentences are unnecessarily convoluted. Some of my other concerns were addressed, but I did not find a revision to the conclusion stating any future directions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.