acceptodds
Under review as a conference paper at ICLR 2027

Evo²: Evolving the Evolver through Weight-Harness Co-Evolution

11 views

Abstract

Self-evolving agents are increasingly used to improve language model systems without human intervention through either weight updates or harness improvements. Existing approaches typically pursue these directions separately: some methods update model weights using self-generated experience, while others rewrite the harness that defines how an agent uses prompts, tools, context, and execution feedback. However, these approaches overlook the relationship between model weights and the agent harness. To connect these processes and ensure effective and smooth agent self-evolving, we propose Evo², a framework for weight-harness co-evolution. For harness updates, we design a unified dual-role pipeline that a single policy serves as both a Proposer, which writes harness edits, and a Reflector, which diagnoses their effects and guides revision. For weight updates, harness modifications and their execution outcomes guide the training of both the Proposer and the Reflector. We further develop EvoTIDE, a training pipeline that recovers usable trajectories through edit extraction and prefix continuation, then combines SFT with interleaved GRPO across the two roles. Extensive experiments across three coding benchmarks demonstrate that Evo² significantly outperforms existing self-evolution methods. It achieves 82.1% on Polyglot and 48.3% on SWE-bench Verified, compared with 62.6% and 41.0% for the strongest baseline, respectively. We also show that the evolved harnesses transfer to different tasks without further evolution and rank first in cross-benchmark evaluations. Our dual-role harness improvement framework is also able to extend to self-evolution with proprietary models, improving a GPT-5-mini agent's performance from 24.4% to 48.3%. In addition, Evo² generalizes to other tasks smoothly. We extend self-evolution to memory tasks, achieving the best performance on the LoCoMo and LongMemEval benchmarks. These results demonstrate the applicability of the framework Evo² across domains and its broad transferability across models and tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Self-evolving, LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZX7pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Evo², a framework that jointly evolves an agent's harness and the model responsible for improving that harness. A shared policy acts as both a Proposer, which generates harness modifications, and a Reflector, which analyzes execution feedback and suggests further improvements. The authors also introduce crossover searching to combine useful strategies from different branches of the harness evolution tree.

For weight evolution, EvoTIDE collects trajectories from both roles and trains the shared policy using SFT followed by interleaved GRPO. The experiments cover three coding benchmarks, proprietary models, and memory tasks. The results show improvements over existing self-evolving frameworks, along with cross-benchmark transfer of the evolved harnesses.

Critical strengths

S1. Interesting direction for self-evolving agents. The paper explores how experience from harness modifications can be used to improve the model responsible for generating those modifications. Connecting weight evolution with harness evolution is a reasonable direction for improving self-evolving systems.

S2. A unified framework for harness modification and feedback. Using a shared Proposer–Reflector policy allows the system to learn both how to modify a harness and how to interpret the results. The combination with crossover searching also provides a practical way to reuse improvements discovered in different branches.

S3. Encouraging results across several benchmarks. Evo² achieves strong performance on the three coding benchmarks, with particularly large improvements on Polyglot. The additional experiments on proprietary models and memory tasks suggest that parts of the framework can be applied beyond a single model or domain.

S4. Evidence of transferability. The evolved harnesses are evaluated on benchmarks different from those used during evolution, without additional updates. The results in Table 2 provide useful evidence that some discovered improvements generalize beyond their original tasks.

Critical weaknesses

W1. Limited methodological novelty. The main components of Evo², including reflection-guided modification, tree-based harness search, experience sharing, SFT, and GRPO, have been explored in previous work. While combining weight and harness evolution is interesting, the paper could better establish what fundamentally new mechanism makes their interaction more effective than existing self-improvement approaches.

W2. Insufficiently controlled comparisons with existing methods. The baselines use an untrained model as the evolver, whereas Evo² benefits from an evolver trained through EvoTIDE. This makes the main comparisons difficult to interpret, since the improvements may come from a stronger evolver rather than the proposed harness evolution procedure. Although Table 5 provides a trained-versus-untrained comparison within Evo², a more controlled comparison with baselines using the same trained evolver would be useful.

W3. The contributions of individual components are not sufficiently isolated. Evo² combines dual-role training, trajectory recovery, crossover searching, SFT, and interleaved GRPO. Table 5 examines weight evolution, but there is no comprehensive ablation separating these components.

W4. Potentially unfair harness selection in the proprietary-model experiments. Section 4.4 states that the baselines evaluate their top-ranked harness, while Evo² reports the better result of its two top-ranked harnesses.

Questions for authors

If the authors can carefully address the weaknesses, I can consider update my score

1 comment

We thank the reviewer for the helpful review. Below, we provide detailed responses to your comments.

W1. Novelty.

  • Weight–harness co-evolution. Outcomes of the harness search train one Proposer–Reflector policy, which then drives the next rounds of the search.
  • Crossover searching. Reflection and directed editing adapt useful strategies across branches of the harness tree.
  • EvoTIDE. Trajectory recovery, SFT and interleaved GRPO over both roles, trained on a parent pool that spans different depths and performance levels.

W2. Baselines with the trained evolver.

  • Controlled comparison. We ran HGM with the trained and with the untrained evolver, with the same search procedure, budget and evaluation window: trained evolver improved +14.1% on all 149 Polyglot tasks.
  • Revision. We will add DGM, GEA and SICA with the trained evolver.

W3. Component isolation. We will add a component-wise ablation in the revision: SFT, GRPO, shared vs. separate roles, trajectory recovery and crossover, under the same initial harness and search budget.

W4. Selection in the proprietary-model experiments. We re-scored every method under the same rule: the better of its two top-ranked harnesses.

SICADGMHGMGEAEvo²
GPT-5-mini30.627.825.021.748.3
Claude Haiku 4.543.943.345.650.047.8

Evo² remains first with GPT-5-mini and is second with Claude Haiku 4.5.

We sincerely thank you again for your thoughtful feedback and hope that these clarifications and additional results address your concerns; we would be happy to answer any follow-up questions.

Reviewer_mHzCOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Evo², a framework that jointly evolves an agent's harness and the model responsible for improving it. A shared Proposer–Reflector policy modifies harnesses, diagnoses execution feedback, and combines useful strategies across search branches, while EvoTIDE trains this policy using recovered trajectories, supervised fine-tuning, and interleaved GRPO. Experiments on coding and memory benchmarks demonstrate improvements over existing self-evolving agent frameworks, along with cross-benchmark transfer and applicability to proprietary models.

Critical strengths

S1. A meaningful framework for weight-harness co-evolution.

The paper connects two complementary directions of agent self-improvement by using harness modification experience to train the evolver itself. The shared Proposer–Reflector policy enables both editing and feedback interpretation to improve through experience. This provides a coherent approach to learning how to improve agents rather than only improving individual harnesses.

S2. Strong empirical results across multiple benchmarks.

Evo² achieves 82.1% on Polyglot and 48.3% on SWE-bench Verified, outperforming the strongest reported baselines by 19.5 and 7.3 percentage points, respectively. The method also demonstrates improvements on memory benchmarks and transfers evolved harnesses across coding tasks. These results suggest that the framework has practical value beyond a single benchmark.

Critical weaknesses

W1. Insufficient isolation of the contributions of individual components.

Evo² combines weight evolution, dual-role reflection, crossover search, trajectory recovery, and tree-based parent selection, making it difficult to identify which components drive the improvements. Although Table 5 isolates weight evolution and the paper reports a crossover-related improvement, a systematic component-wise ablation is missing. In particular, the contributions of shared-role training, reflection, and trajectory recovery remain unclear.

W2. Incomplete experimental controls and statistical reporting.

The paper does not clearly establish that all methods use comparable evolution budgets, including the number of candidate evaluations and training or inference costs during search. Furthermore, Section 4.4 states that Evo² selects the better of its two top-ranked harnesses using held-out evaluation, whereas the baselines evaluate only their top-ranked harness, potentially introducing a selection advantage. Statistical evidence is also limited, and the reported weight-evolution improvement on SWE-bench Verified is not significant at the conventional 0.05 level (Table 5, p=0.086p=0.086).

Questions for authors

Q1. Can author report the number of harness candidates, total evaluation runs, training tokens, training GPU-hours, and search costs for all Baseline and Evo² models, and provide a performance curve with an equal total computational budget?

Q2. Could author compare the performance of an untrained Evolver, an SFT-only Evolver, a GRPO-only Evolver, the full EvoTIDE, and a version without Crossover, all under the same initial harness and search budget?

Q3. Could author explain the discrepancy in SWE-bench scores between Table 1 and Table 5, the statistical testing methods used in Table 5, and whether the selection of the two candidate harnesses in the Proprietary Model experiments utilized held-out test feedback?

1 comment

We thank the reviewer for the helpful review. Below, we provide detailed responses to your comments.

W1 / Q2. Component ablation.

  • We will add a component-wise ablation under the same setting and search budget.

W2 / Q1. Budgets, cost and statistics.

  • Search. Every method's Polyglot search used 400 task evaluations on the same task search set.
  • Training (Polyglot). 194 SFT rows; 26 GRPO steps × 16 rollouts; ~35 h on three H200 GPUs.
  • Revision. We will report candidate counts, evaluation runs and training cost for every method.
  • Selection. Under the same rule for all methods, Evo² remains first with GPT-5-mini (48.3 vs. 30.6) and is second with Claude Haiku 4.5 (47.8 vs. 50.0). Full tables are in our response to FVvk W1.

Q3. Table 1 vs. Table 5. Table 1 is the trained-evolver tree's harness, averaged over five evaluation draws. Table 5 is a one-step child of a fixed parent, evaluated once on the 100 held-out tasks. These are different harnesses with different numbers of draws.

We sincerely thank you again for your thoughtful feedback and hope that these clarifications and additional results address your concerns; we would be happy to answer any follow-up questions.

Reviewer_FVvkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The central argument of this paper is that after an agent finds a better harness through search, it should also use that search experience to train a model that is better at modifying harnesses. The aim is to accumulate two kinds of progress: improved programs and reusable improvement skills encoded in the model's parameters.

The paper keeps the model that actually performs the tasks fixed and trains only the evolver responsible for modifying the harness.

The paper makes the following main points.

  1. Train modification and diagnosis together. The same model plays two roles: the Proposer suggests code changes, and the Reflector uses execution results to decide whether to revise further, borrow a strategy from another branch, or keep the current version. The training signal comes from the change in task performance before and after an edit.

  2. Use the search process to continually generate training experience. The authors maintain harnesses at different depths and performance levels, train both roles through SFT and interleaved GRPO, and then continue searching with the updated model. They also recover some editing trajectories that contain formatting errors or fail partway through, increasing the amount of usable training data.

  3. Reuse discoveries from other search branches through crossover. The Reflector selects a strategy from another branch, and the Proposer adapts it to the current program. This attempts to transfer a useful mechanism rather than simply splice code together.

  4. Report positive results for both the complete system and training itself.

  5. Report transfer of completed harnesses and applicability in other domains.

One distinction is particularly important: these transfer experiments mainly show that “an improved program can be used elsewhere”; they do not directly show that “a trained modifier becomes better at improving programs in a new environment.” The latter capability is closer to the paper's most compelling long-term promise.

I think the work has research value, but the depth of what it has established is currently limited.

Its most valuable perspective is to make the optimization process itself an object of learning. An expensive harness search could leave behind both a champion program and data for training the next generation of optimizers. If that improvement skill transfers, past search experience could reduce the cost of optimizing new agents in the future. This is a direction worth pursuing.

However, there is already closely related work on this broad direction. Harness-R1 trains a harness editor using outcomes from a frozen task-execution model; JIT-Agent also combines generator training with updates to a candidate archive. Evo²'s more specific new information should therefore come from shared dual-role training, continually evolving experience, and directed strategy transfer. These recent works do not negate independent discovery, but they narrow the scope of the distinctive contribution that this paper needs to establish.

The problem is that the paper currently mainly tells the reader: “This complete training and search recipe produces better results.” It does not yet adequately explain deeply:

  • Why do the modifier and the diagnostician need to be trained together?
  • What does continual co-evolution add beyond training a good modifier once?
  • Is the model learning more reliable edit formatting, familiar repair patterns, or transferable improvement skills?
  • Which strategies can be successfully combined across branches, and under what conditions does this fail?

But the paper has not yet turned its most distinctive designs into clean, controlled, and surprising empirical findings that readers feel highly valuable. The reader gets a promising approach and initial positive results; there is still a gap between that and coming away with “a new principle that is worth applying more broadly.”

Therefore I remain hesitant in backing up for acceptance. I would like to see the authors' clarifications on their deeper insights.

Critical strengths
  1. The learning target is meaningful and the high-level design is coherent. Training the evolver from actual changes in downstream task performance is a sensible way to accumulate editing experience. Keeping the executor fixed separates harness changes from improvements in the executor's weights. The frozen counterpart used within each training round also makes the shared-role procedure understandable (§§3.1–3.3).

  2. There is substantive positive evidence worth preserving. Table 5 compares trained and untrained evolvers while reportedly fixing search, executor, and parent harnesses; the Polyglot improvement from 61.8 to 78.0 is encouraging. The cross-benchmark results also go beyond reporting search-set champions: Evo² leads in five of six directions in Table 2, and the LoCoMo harness obtains 71.5 on LongMemEval, which the paper explicitly states was unseen during evolution and selection (§4.5).

Critical weaknesses
  1. Table 3 uses unequal selection on the reported held-out evaluation. Section 4.4, lines 396–401, states that each baseline evaluates its top-ranked harness, whereas Evo² reports the better result of its two top-ranked harnesses. Lines 455–460 explicitly describe an additional held-out evaluation as a way to recover development-set ranking errors. This allows the reported evaluation to choose Evo²'s final candidate, while the baselines receive no equivalent opportunity. Averaging over three searches does not remove this advantage. I cannot determine its magnitude or attribute the entire GPT-5-mini improvement to it, but it undermines the comparison, particularly the smaller Claude margin. Please report results under a common, development-only selection rule, provide both candidate results, and clarify which other tables use this procedure. An independent final test is needed if held-out selection remains part of the method.

  2. The experimental boundaries and optimization budgets are not specified well enough to audit the central results. Sections 3.3 and 4.1 do not identify the SFT/RL task sources and sizes, their relation to search/selection/test splits, the checkpoint used for each benchmark, or the candidate-evaluation budgets for the competing methods. The training rounds, update counts, and essential optimization settings are also missing from the supplied manuscript. These details determine whether a transfer target was unseen by the evolver and whether stronger champions reflect a better optimizer or more optimization opportunities. I am not alleging training contamination. A stage-by-stage data/checkpoint/budget specification would resolve much of this uncertainty. Offline training may be amortized, but its cost should be disclosed and search-stage budgets made comparable; Figure 3 measures deployment cost instead.

  3. The paper does not yet establish the distinct information added by its most characteristic design. Harness-R1, §3 already trains a harness editor through SFT and outcome-based GRPO with a frozen target; JIT-Agent, §4.3 trains a harness generator while updating an archive of evaluated designs. These recent works do not make Evo² equivalent or rule out independent development. They do, however, place the remaining contribution in the shared Proposer–Reflector training, evolving parent distribution, and directed strategy transfer. Table 5 compares the whole training package against no training. It therefore cannot show whether these distinctive choices add value beyond ordinary outcome-trained editing. A focused control, such as training the Proposer with a fixed Reflector under matched conditions, would be more informative than further aggregate benchmark gains. Likewise, Tables 2 and 4 test transfer of completed harnesses; they do not directly test whether the learned evolver becomes a reusable optimizer in unseen settings.

  4. The main analysis is difficult to interpret and overextends the available evidence. Section 4.6 attributes a further 4.1 Polyglot points to crossover, apparently comparing Table 5's 78.0 with Table 1's 82.1, but the tables do not specify the matched crossover conditions. Table 5's trained SWE score is 51.0 rather than Table 1's 48.3, and its memory scores also differ from Table 4; these may be valid differences in setup, but they need explanation. The meaning and independent sampling unit of P are unspecified. The claims about better average children, search reliability, and reduced evaluation budget are not accompanied by child-level results or search-progress evidence. Please connect the tables through explicit configurations and either provide the relevant existing logs or narrow these conclusions. One concrete donor-to-recipient edit would also help readers understand what the method actually learned.

Questions for authors

see above comments

2 comments

We thank the reviewer for the helpful review. Below, we provide detailed responses to your comments.

W1. Selection in Table 3. We re-scored every method under the same rule: the better of its two top-ranked harnesses, averaged over three searches.

GPT-5-miniSICADGMHGMGEAEvo²
Paper24.423.924.319.448.3
Same selection for all30.627.825.021.748.3
Claude Haiku 4.5SICADGMHGMGEAEvo²
Paper42.243.345.044.447.8
Same selection for all43.943.345.650.047.8

Under the same rule, Evo² remains first with GPT-5-mini and is second with Claude Haiku 4.5.

W2. Data, checkpoints and budgets. We will add a stage-by-stage specification in the revision. Polyglot, as an example:

  • Training. 194 SFT rows; 26 GRPO steps × 16 rollouts; ~35 h on three H200 GPUs.
  • Search. 400 task evaluations for every method on the same task search set.
  • Splits. The GRPO reward tasks and the search set are disjoint from all reported test tasks.

W3. Harness-R1 and JIT-Agent. We thank the reviewer for both pointers.

  • They are concurrent work. Harness-R1 appeared on August 3, 2026 and JIT-Agent on August 26, 2026, while the ICLR 2027 abstract deadline was September 18, 2026. Under the ICLR reviewer guidelines, both count as concurrent work, and Evo² was developed independently.
  • How Evo² differs. It trains one policy for both the Proposer and the Reflector. It also transfers strategies across branches through crossover.
  • Revision. We will cite and discuss both papers.

We sincerely thank you again for your thoughtful feedback and hope that these clarifications and additional results address your concerns; we would be happy to answer any follow-up questions.

Thank you for the rebuttal. The key hesitance stems from not understanding why the proposed mechanism works and under what premises. Specifically, the experiments suggest that training the evolver helps it discover stronger harnesses, but could you clarify what it learns from past evolution attempts and why that knowledge should remain useful beyond the training setting? Does it internalize reusable principles for diagnosing failures and selecting interventions, or primarily preferences for modifications that performed well under a particular task distribution and evaluator? For example, aggressive context pruning might improve performance on short, self-contained coding tasks but harm repository-level tasks that require information scattered across many files. In this hypothetical case, would the trained evolver learn when pruning is appropriate, or could it inherit a preference that becomes counterproductive after the task distribution changes? This distinction also seems important because transferring an already evolved harness does not directly establish that the trained evolver can improve unfamiliar harnesses. Under what assumptions about task similarity, tool interfaces, and feedback quality do you expect the learned improvement strategy to transfer, and what evidence could distinguish transferable diagnostic knowledge from specialization to the evolution environment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.