acceptodds
Under review as a conference paper at ICLR 2027

Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

Abstract

Recent work on self-evolving LLM agents treats the agent harness as an optimization target that can be iteratively improved through execution feedback. Existing approaches, however, largely optimize a single objective and evolve the harness as a monolithic artifact, limiting their ability to handle competing goals and to separate transferable design from environment-specific adaptation. We introduce Harmony Harness, a framework for multi-objective, compositional harness evolution. Harmony Harness formulates harness evolution as a multi-objective optimization problem and uses decomposition-based evolutionary search to maintain solutions across different regions of the Pareto frontier, avoiding reliance on a hand-designed structural diversity metric for repository-scale harnesses. In parallel, we introduce factorized evolution operators that separately refine general-purpose harness mechanisms and benchmark-specific adaptations, together with a dynamic composition workflow that integrates them at inference time. Across SecRepoBench and BaxBench, Harmony Harness consistently outperforms strong harness-evolution baselines, with gains exceeding 10% on both benchmarks. Our results show that explicitly modeling both objective trade-offs and harness structure yields substantially stronger and more transferable agent evolution than existing single-objective, monolithic approaches.

Primary area:
optimization
Keywords:
Self-evolving agent, agent harness, multi-objective evolving, agent safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WSJFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for evolving multi-objective agents called Harmony Harness. This method integrates multi-objective evolutionary algorithms (MOEA/D), factorized evolution, and dynamic workflow composition. Experiments on SecRepoBench, BaxBench, and CWEval demonstrate that this method improves functionality, security, and transferability.

Critical strengths
  1. Optimizing both functionality and security is of great value. Both of these objectives are evaluated through executable tests.
  2. This method outperformed the baseline in two benchmark tests and demonstrated encouraging transfer performance. The results of the ablation experiments support the proposed components.
Critical weaknesses
  1. The Pareto front is narrow and eventually converges to a dominant solution (Appendix G.1). Compared to weighted sum optimization, it yields a gain of 6.7% (Table 2), and this does not include uncertainty estimates. Appendix D also erroneously claims that the TChebyshev substitution cannot reduce the value of a single objective function.
  2. Each partition contains only 28-30 tasks, 3 runs were conducted, and no confidence intervals are provided. The effective sample size also fluctuates (Appendix C). Furthermore, Appendix K describes an overfitting threshold but does not report its detection accuracy or the threshold value.
Questions for authors
  1. Can the authors provide the results, confidence intervals, and effective sample size for each run?
  2. Can the authors demonstrate that MOEA/D has a clear advantage when the objective function is conflicting?
  3. Was an overfitting threshold used in the main experiments? If so, how was it calibrated and evaluated?
1 comment

We thank the reviewer for their valuable time and constructive feedback.

Weakness 1, Q1 and Q2: The Pareto front is narrow and eventually converges to a dominant solution (Appendix G.1).

Compared to weighted sum optimization, it yields a gain of 6.7% (Table 2), and this does not include uncertainty estimates. Appendix D also erroneously claims that the TChebyshev substitution cannot reduce the value of a single objective function. Can the authors provide the results, confidence intervals, and effective sample size for each run? Can the authors demonstrate that MOEA/D has a clear advantage when the objective function is conflicting?

Response: We politely address it from two perspectives.

1. Functionality and security are not always aligned.

Although functionality and security can improve together during some stages of evolution, they are not universally aligned. The complete Pareto fronts in Figures 5 and 6 show solutions for which improving security reduces functionality, and vice versa. Thus, the search does encounter genuine objective conflicts rather than optimizing two equivalent metrics.

We agree that our original discussion did not make this distinction sufficiently clear. In the revised manuscript, we will explicitly discuss the trade-offs visible in Figures 5 and 6 and clarify that the purpose of MOEA/D is to preserve multiple solutions representing different functionality–security preferences.

2. Additional functionality–time-efficiency experiment

Evaluating an objective related to computational efficiency provides a stronger test of the proposed multi-objective framework. Therefore we conducted an additional experiment that jointly considers task performance and time efficiency.

All methods in the controlled comparison start from the same seed harness and use the same search budget. Results are reported on the 20-task search split. Higher values are better for both pass rate and time efficiency.

Table R4. Joint optimization of task performance and time efficiency.

MethodSelected childEdit chainPass rateTime efficiency
Seed harness (no evolution)tb_wf_seed–58.349.6
Meta-Harness, reported result with Haiku 4.5––37.4–
Meta-Harness, reported result with Opus 4.6––76.4–
Meta-Harness, our run with Gemini 3.6wf_s2_diffskill → diff69.263.9
Weighted-sum evolutionwf_s2_toolskill → tool58.561.0
HH (ours)wf_s2_gentool → skill72.065.2

Compared with the unevolved seed harness, HH improves the pass rate from 58.3 to 72.0 and time efficiency from 49.6 to 65.2. Under our matched experimental setting, HH also outperforms our Meta-Harness run on both pass rate (72.0 vs. 69.2) and time efficiency (65.2 vs. 63.9). Weighted-sum evolution improves time efficiency but provides almost no pass-rate improvement, illustrating the difficulty of balancing the two objectives through a single linear scalarization.

This experiment extends our evaluation beyond functionality and security and demonstrates that the proposed framework can also optimize task performance jointly with computational efficiency.

And yes we would correct the saying in Appendix D.

###Weakness 2 and Q3: Each partition contains only 28-30 tasks, 3 runs were conducted, and no confidence intervals are provided.

The effective sample size also fluctuates (Appendix C). Furthermore, Appendix K describes an overfitting threshold but does not report its detection accuracy or the threshold value. Was an overfitting threshold used in the main experiments? If so, how was it calibrated and evaluated?

Response: Evaluation size. Our setting is actually a fair amount of testing (30+30+30), compared to the 5 harness papers evolving on the TerminalBench2, which has 89 tasks. The difference is that they evolve and test on the same set, evolved on 89 and tested on 89. After a second thought, we authors decided to separate them into 3 sets: 30 for evolving, 30 for in-distribution test, and 30 for out-of-distribution test. So, the difference is that we evolve on a smaller set, but for evaluation, we are the same, at a ~90 level.

Gate. Appendix K is designed to prevent reward hacking by rejecting invalid modifications that exploit the evaluator rather than genuinely improving functionality or security. There's no threshold or calibrated related to the overfitting gate, it's applied on every harness child, and if rejected, this harness is considered to be hacking and invalid.

Reviewer_iKgkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes HARMONY HARNESS, a framework for multi-objective and compositional harness evolution in LLM agents. It uses decomposition-based evolutionary search to explore different objective trade-offs. The framework also separates general-purpose harness mechanisms from benchmark-specific adaptations and dynamically combines them at inference time.

Critical strengths

Framing harness self-evolution as multi-objective optimization is meaningful and timely. This paper treats security as an independent objective instead of folding it into the reward. That makes trade-offs explicit and puts direct search pressure on whichever objective lags, rather than leaving the compromise to the agent.

Critical weaknesses
  1. The multi-objective setting is too limited. The paper is framed as multi-objective, but it uses only two objectives (functionality and security), and the authors themselves report that these are largely aligned. As a result, the benefits of MOEA/D are never really tested, and objectives that genuinely conflict, such as cost or latency, are not considered. Related multi-objective prompt and agent optimization methods (e.g., GEPA, SoS) are only briefly mentioned and never compared experimentally.

  2. The evaluation is too small and the reported results are inconsistent. With 28–30 tasks per split, one task equals about 3.3 points, and many reported gains amount to only 1–2 tasks. All results are reported as 3-run means without std.

Questions for authors

All results are described as means over 3 runs, but no variance is reported. It is better to report mean ± std for every table. Table 1 reports only security gains (∆evolve, ∆OUT) and their ratio, how about the original results?

2 comments

We thank the reviewer for their valuable time and constructive feedback.

Weakness 1: The multi-objective setting is too limited.

The paper is framed as multi-objective, but it considers only two objectives—functionality and security—which the authors report are largely aligned. Consequently, the benefits of MOEA/D are not fully tested under strongly conflicting objectives, such as performance versus cost or latency. Related multi-objective prompt and agent optimization methods, such as GEPA and SoS, are discussed only briefly and are not evaluated experimentally.

Response: We politely address it from three perspectives.

1. Functionality and security are not always aligned.

Although functionality and security can improve together during some stages of evolution, they are not universally aligned. The complete Pareto fronts in Figures 5 and 6 show solutions for which improving security reduces functionality, and vice versa. Thus, the search does encounter genuine objective conflicts rather than optimizing two equivalent metrics.

We agree that our original discussion did not make this distinction sufficiently clear. In the revised manuscript, we will explicitly discuss the trade-offs visible in Figures 5 and 6 and clarify that the purpose of MOEA/D is to preserve multiple solutions representing different functionality–security preferences.

2. Additional functionality–time-efficiency experiment

Evaluating an objective related to computational efficiency provides a stronger test of the proposed multi-objective framework. Therefore we conducted an additional experiment that jointly considers task performance and time efficiency.

All methods in the controlled comparison start from the same seed harness and use the same search budget. Results are reported on the 20-task search split. Higher values are better for both pass rate and time efficiency.

Table R4. Joint optimization of task performance and time efficiency.

MethodSelected childEdit chainPass rateTime efficiency
Seed harness (no evolution)tb_wf_seed–58.349.6
Meta-Harness, reported result with Haiku 4.5––37.4–
Meta-Harness, reported result with Opus 4.6––76.4–
Meta-Harness, our run with Gemini 3.6wf_s2_diffskill → diff69.263.9
Weighted-sum evolutionwf_s2_toolskill → tool58.561.0
HH (ours)wf_s2_gentool → skill72.065.2

Compared with the unevolved seed harness, HH improves the pass rate from 58.3 to 72.0 and time efficiency from 49.6 to 65.2. Under our matched experimental setting, HH also outperforms our Meta-Harness run on both pass rate (72.0 vs. 69.2) and time efficiency (65.2 vs. 63.9). Weighted-sum evolution improves time efficiency but provides almost no pass-rate improvement, illustrating the difficulty of balancing the two objectives through a single linear scalarization.

This experiment extends our evaluation beyond functionality and security and demonstrates that the proposed framework can also optimize task performance jointly with computational efficiency.

3. Relationship to GEPA and SoS

Here's the comparison of these methods.

DimensionGEPASoSHH (ours)
Optimization targetPrompts associated with individual modulesA single instruction promptThe complete harness, including tools, skills, control logic, and orchestration
Multi-objective mechanismMaintains candidate diversity across task-level feedbackAlternates between objectives or combines them through a weighted sumUses MOEA/D with Tchebycheff scalarization and an explicit set of preference vectors
Search outputPrompt variantsInstruction-prompt variantsExecutable harness variants
Transfer structureNo explicit general/environment scope separationNo explicit general/environment scope separationSeparates general and environment-specific scopes

A direct implementation-level comparison is kinda hard, would require substantial redesign and would no longer constitute a faithful implementation of the original methods.

Nevertheless, we evaluate baselines that instantiate their central optimization strategies within our search space:

  • The SoS has two strategies, one is closest to our weighted-sum baseline , and the other is close to sequential baseline alternates optimization between objectives. Our implementation is a fairer and more honest comparison with other advanced designs (dynamic orchestration, decomposition, etc.).
  • The JEPA is actually close to Meta-Harness baseline; JEPA compresses task-level feedback into a single scalar objective. But again, the Meta-Harness implementation is a fairer and more honest comparison with other advanced designs.

Weakness 2: The evaluation is too small, and the reported results are inconsistent.

With approximately 28–30 tasks per split, one task corresponds to roughly 3.3 percentage points, so several reported improvements represent only one or two additional successful tasks. Moreover, the results are reported as means over three runs without standard deviations.

Response: Evaluation size. Our setting is actually a fair amount of testing (30+30+30), compared to the 5 harness papers evolving on the TerminalBench2, which has 89 tasks. The difference is that they evolve and test on the same set, evolved on 89 and tested on 89. After a second thought, we authors decided to separate them into 3 sets: 30 for evolving, 30 for in-distribution test, and 30 for out-of-distribution test. So, the difference is that we evolve on a smaller set, but for evaluation, we are the same, at a ~90 level.


Q1. All results are described as means over three runs, but no variance is reported. Can the authors report mean ± standard deviation for every table?

Response: We provide the run-level results and confidence intervals in the following supplementary audit: Run-level results and confidence intervals. And the absolute results underlying Table 1 are reported in Tables 9 and 10.

Reviewer_xLbGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies evolution of agent harnesses under separate functionality and security objectives. It combines MOEA/D-style weighted Tchebycheff subproblems with operators that edit tools, skills, and logic in general or environment-specific scopes. A composition agent builds a task-dependent execution DAG. Experiments on SecRepoBench and BaxBench report substantial secure-pass improvements, with zero-shot transfer to the other benchmark and CWEval.

Critical strengths
  1. Functional tests and exploit checks provide a useful setting for studying whether harness evolution preserves security. Reporting the conjunction, rather than only marginal scores, avoids equating two separately good rates with secure functional solutions.
  2. Table 9 reports held-out-OUT secure-pass of 66.7 versus 36.7 for the unevolved SecRepoBench harness. Table 10 reports BaxBench averages of 61.3 versus 42.9. Table 12 evaluates transfer in both directions; Table 11 adds 357 CWEval instances across five languages.
Critical weaknesses
  1. The multi-objective explanation is stronger than the demonstrated mechanism. Table 13 acknowledge narrow fronts, including a final search-set harness that dominates the others. Thus recovery of unsupported non-convex Pareto regions is not demonstrated. Table 2 does provide a useful matched comparison, and I do not regard the aggregation ablation as absent. However, Appendix G (p. 24) also notes that weighted-sum evolution reaches 70.0 secure-pass versus 63.3 for plain Tchebycheff evolution. The full system reaches 76.7 after adding other mechanisms. This supports a combined system contribution more directly than superiority of decomposition itself. Moreover, Appendix D's statement that a child improving one objective by sacrificing the other is not accepted does not generally follow from Tchebycheff replacement: an improved maximum weighted shortfall can coexist with a regression on another objective.

  2. Attribution of transfer to scope factorization remains incomplete. Table 7 provides substantial evidence for component factorization plus orchestration, but does not isolate the general/environment scope distinction. The direct generalize ablation in Table 6 uses six tasks and one-task improvements; its starting scores also differ between the compared operator-pool runs. The authors appropriately call this directional evidence. CWEval is useful transfer evidence, but its comparator is the unevolved one-shot Claude Code harness, rather than matched evolved alternatives at comparable inference cost (Table 11). Consequently, the strong transfer results do not yet uniquely identify scope factorization as the cause.

Questions for authors
  1. Can you provide per-run task manifests, exclusions, denominators, and paired uncertainty estimates, with either retries under controlled resources or a comparison on a common evaluable task set?
  2. Was the Appendix K gate applied identically to HH, Meta-harness, DGM, and all aggregation ablations? What changes if it is disabled under the same evaluator protections?
  3. With factorized operators and orchestration fixed, what are the run-level differences between Tchebycheff and weighted-sum search on held-out tasks, rather than just search scores?
  4. Can the existing transfer results be broken down with and without general/environment scope separation while preserving tools, orchestration, and inference budget?
2 comments

We thank the reviewer for their valuable time and constructive feedback.

Weakness 1: The multi-objective explanation is stronger than the demonstrated mechanism.

Response: We politely address the reviewer’s three points below.

1. Table 13 suggests narrow Pareto fronts, including a final search-set configuration that dominates the others.

We would like to emphasize that our method does produce Pareto fronts with meaningful trade-offs. Figures 5 and 6 (p. 23) present the complete fronts and show that improving code security can come at the cost of code functionality. Table 13 reports only a limited summary and therefore does not fully reflect the trade-offs shown by the complete fronts. We will clarify this distinction in the revised manuscript.

2. Weighted-sum evolution performs better than plain Tchebycheff evolution, while the full system reaches 76.7 only after adding the other components.

We emphasize that the aggregation rules should be compared under matched configurations. We therefore conducted additional controlled experiments comparing weighted-sum and Tchebycheff aggregation with and without factored evolution and dynamic orchestration. Within each comparison pair, only the aggregation rule differs; the model, random seeds, search budget, and preference weights are identical.

Table R1. Aggregation rule × system components on the SecRepoBench search split (%, mean over three runs).

For weighted-sum aggregation, the scalarized score is

αS+(1−α)F.\alpha S + (1-\alpha)F.

AggregationWeightsFactored evolutionDynamic orchestrationFunc.Sec.F&SSource
Weighted sumSame λ\lambda set (N=5N=5)✗✗63.360.053.3New
Tchebycheff (ours)Same λ\lambda set (N=5N=5)✗✗70.076.763.3Table 5
Weighted sumSame λ\lambda set✓✗63.360.060.0New
Tchebycheff (ours)Same λ\lambda set✓✗70.083.366.7Table 5
Weighted sumSame λ\lambda set✓✓70.073.370.0Table 2
Tchebycheff (ours)Same λ\lambda set✓✓76.793.376.7Table 2
Weighted sumα=0.25\alpha=0.25 only✓✓76.760.056.7New
Weighted sumα=0.50\alpha=0.50 only✓✓70.073.370.0New
Weighted sumα=0.75\alpha=0.75 only✓✓56.776.753.3New

These results support two conclusions.

  1. Tchebycheff aggregation consistently outperforms weighted-sum aggregation under matched configurations. The gap is already present when the aggregation rule is evaluated in isolation: Tchebycheff improves F&S from 53.3 to 63.3 without sacrificing either individual objective. Adding factored evolution and dynamic orchestration benefits both aggregation rules—weighted sum improves from 53.3 to 70.0, while Tchebycheff improves from 63.3 to 76.7—but does not eliminate the gap. Thus, the benefit of Tchebycheff aggregation is complementary to, rather than solely explained by, the other system components.

  2. A weighted sum with a single preference is sensitive to α\alpha. The settings α=0.25\alpha=0.25 and α=0.75\alpha=0.75 improve one objective at the expense of the other. The best single setting, α=0.50\alpha=0.50, achieves 70.0 F&S, still below the 76.7 achieved by our method. Moreover, covering multiple preferences using separate single-α\alpha weighted-sum searches requires multiple runs, whereas our decomposition-based method covers the preference set within a single run under the same per-run budget.

3. The claim that a child sacrificing one objective to improve another is not accepted does not generally follow from Tchebycheff replacement. It is an additional design choice rather than a property of vanilla Tchebycheff optimization.

This is more of a design choice. The motivation for our design is to prevent degradation in either objective, it is a property of our replacement rule, not of vanilla Tchebycheff replacement itself.

But we are curious about the reviewer’s suggestion, so we added the following exp: replacing our rule with the vanilla criterion, which accepts a child whenever its scalarized value improves, even if one of the individual objectives decreases.

Table R2. Effect of the replacement rule, with all other settings held fixed.

Replacement ruleGenerationsSecure-passSecurityUtility
Vanilla Tchebycheff: accept iff g(child)<g(parent)g(\text{child}) < g(\text{parent})467.770.070.0
Vanilla Tchebycheff: accept iff g(child)<g(parent)g(\text{child}) < g(\text{parent})570.080.073.3
Ours476.793.376.7

The results show that vanilla Tchebycheff replacement remains effective and improves when given an additional generation. However, even with five generations, it remains below our replacement rule using only four generations. This result suggests that our non-degradation constraint improves search efficiency.

Weakness 2: Attribution of transfer to scope factorization remains incomplete.

Response: We think this is a reasonable concern, so we conducted an additional ablation using the full pipeline with a single shared scope. We keep component factorization, dynamic orchestration, mutation operators, source environment, and search budget fixed; the only difference is whether the general and environment-specific scopes are separated.

All evolved variants are optimized on BaxBench and transferred zero-shot to CWEval. We evaluate all methods on the same 357 CWEval instances and report the mean over three runs, following the same protocol as Table 11.

Table R3. Isolating the effect of scope factorization on zero-shot transfer to CWEval.

HarnessEvolvedComponent factorizationOrchestrationGeneral/environment scope splitSRFunc.S&FSource
One-shot Claude Code✗–––73.991.973.1Table 11
Evolved Meta-Harness✓–––65.977.863.3New
Ours without scope split (single scope)✓✓✓✗81.687.280.2New
Ours✓✓✓✓89.489.987.4Table 11

These results support two conclusions.

  1. The general/environment scope split substantially improves transfer under a controlled comparison. When component factorization, orchestration, operators, and search budget are held fixed, introducing the scope split improves SR from 81.6 to 89.4, functionality from 87.2 to 89.9, and S&F from 80.2 to 87.4. This matched ablation provides direct evidence that scope separation contributes to zero-shot transfer.

  2. Evolution without an explicit transfer-oriented structure does not necessarily improve generalization. The Meta-Harness evolved on BaxBench achieves 63.3 S&F on CWEval, below the 73.1 of the unevolved one-shot Claude Code harness. This result is averaged over three independent runs. It suggests that source-environment evolution alone can overfit to the source benchmark. In our single-scope ablation, environment-specific changes can be absorbed into shared components and subsequently transferred to the target benchmark. Separating general and environment-specific scopes mitigates this issue and produces substantially stronger transfer.


Q1. Can you provide per-run task manifests, exclusions, denominators, and paired uncertainty estimates, with either retries under controlled resources or a comparison on a common evaluable task set?

Response: Yes. We provide the per-run task manifests, exclusions, denominators, and paired uncertainty estimates, together with details of the controlled-resource evaluation and the common evaluable task set, in the following audit document:

Per-run evaluation audit and reviewer response

Q2. Was the Appendix K gate applied identically to HH, Meta-Harness, DGM, and all aggregation ablations? What changes if it is disabled under the same evaluator protections?

Response: Yes. The Appendix K gate is applied identically to HH, Meta-Harness, DGM, and all aggregation-rule ablations. It is not a method-specific component and does not selectively benefit our approach.

The gate is designed to prevent reward hacking by rejecting invalid modifications that exploit the evaluator rather than genuinely improving functionality or security. As discussed in Appendix K, disabling the gate under the same evaluator protections allows reward-hacking behaviors to enter the search process, making the measured rewards unreliable. Therefore, the purpose of the gate is to preserve evaluation validity, not to improve the transfer performance of our method. We will clarify both its uniform application and its role in the revised manuscript.

Q3. With factorized operators and orchestration fixed, what are the run-level differences between Tchebycheff and weighted-sum search on held-out tasks, rather than only their search scores?

Response: We address this question in our response to Weakness 1. In particular, Table R1 reports matched comparisons between Tchebycheff and weighted-sum aggregation.

Q4. Can the existing transfer results be broken down with and without general/environment scope separation while preserving tools, orchestration, and inference budget?

Response: Yes. We address this question in our response to Weakness 2 and in Table R3 above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.