acceptodds
Under review as a conference paper at ICLR 2027

RelEvolve: Learning What to Evolve, Evolving What to Learn

Abstract

Automated harness optimization improves LLM agents by iteratively revising the programs that govern prompting, memory, tool use, and control flow. Existing harness optimizers use prior harnesses and evaluation feedback to decide how to modify the next candidate, but the unresolved improvement directions that should guide subsequent evolution typically remain implicit. We introduce RelEvolve, which represents these directions as inquiries and couples inquiry updating with harness evolution: inquiries guide which candidate to continue from and what change to make, while the resulting evaluation provides evidence to refine the inquiry state for later evolution rounds. RelEvolve realizes this coupling through a Harness Lineage that records executable inheritance, an Inquiry Graph that maintains inquiry state across branches, and a Relational Trace that links the inquiries guiding each evolution to the resulting harness change and evaluation evidence. Across five benchmarks and two LLM backbones, RelEvolve yields a 4.4% absolute improvement in held-out test performance over the strongest evaluated baseline in each setting and a 12.5% absolute improvement over the unoptimized harness, averaged across settings. On HotpotQA, it reaches 89.84 and 91.24 F1 with GPT-5.6 Luna and GLM-5.3-Flash, respectively. Three independent search seeds on HotpotQA and InterCode-SQL maintain this improvement in these two representative settings, and on HotpotQA, ablations further indicate that weakening the coupling between inquiry updating and harness evolution lowers performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, automated agent design, harness optimization, agent self-improvement, evolutionary search

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HcmVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes RelEvolve, a framework for automated LLM harness optimization that maintains a persistent Inquiry Graph alongside the harness evolution history. A Curator updates the inquiry state and selects a parent harness, while an Evolver generates new candidates based on the selected parent and active inquiries. The paper reports improvements over several prompt and harness optimization baselines across five benchmarks and two LLM backbones.

While the empirical results appear promising, I have substantial concerns regarding the clarity of the problem formulation, the technical novelty of the proposed approach, the fairness of the experimental comparisons, and the generalizability of the method.

Critical strengths

The paper tries to address an interesting problem in automated agent optimization.

Critical weaknesses

The paper is difficult to follow due to excessive, unclear definitions. What exactly are "unresolved improvement directions"? The motivation is also weak. What specific limitations of existing harness optimizers does RelEvolve address?

The technical novelty is unclear. How does RelEvolve fundamentally differ from existing feedback-driven optimization methods? The separation between Curator and Evolver also seems unnecessary. Why are two separate roles needed? The ablations do not fully justify the proposed complexity. Would a simpler memory structure achieve similar results?

The experimental comparison raises fairness concerns. What stopping criteria are used for each baseline, and are their total token budgets comparable? Finally, the generalizability of the method remains unclear. Can the same prompts and hyperparameters work on unseen benchmarks without task-specific tuning?

Questions for authors

See my weakness

Reviewer_tvpDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper studies the currently active topic of self-improving agent harnesses, also described as recursive self-improvement (RSI) of agent harnesses. The authors propose managing the Inquiry Graph separately from the Harness Lineage, allowing experience and proposed improvements (inquiries) accumulated during evolution to be shared across different branches. They conduct experiments using two base models across a relatively broad range of benchmarks and compare their approach with several baseline RSI methods. They also include experiments controlling token costs to demonstrate the method’s advantage on the performance–cost Pareto frontier, along with ablation studies to examine the contribution of each component.

Critical strengths
  1. The proposed design is easy to understand and appears straightforward to integrate into a variety of agent harnesses as an optional module.
Critical weaknesses
  1. My main concern is the novelty of the proposed method. As I understand it, the main differences from Meta-Harness are the separate management of an Inquiry Graph and the ability to use inquiries generated by other branches. I believe these ideas have already been discussed extensively in earlier work such as Arbor and HDSO. The authors should include a discussion of Arbor, HDSO, and similar work. In its current form, I find the novelty of the proposed method very limited.

  2. Regarding model selection, the experiments use only the least capable model in the GPT-5.6 family and the Flash variant of the GLM-5.3 family. The authors also do not report the reasoning-related parameters, particularly reasoning effort, which I consider especially important. Even with these model choices, scores on many benchmarks are already very high and close to saturation. This raises concerns about whether the current benchmarks are sufficiently challenging, and whether evaluation on more challenging benchmarks is needed to demonstrate the value of the work.

Questions for authors

Could the authors more clearly explain how their approach differs from prior work, including, at a minimum, work made public at least three months before the submission deadline, and clarify its novelty and distinctive value?

Reviewer_iiD5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RelEvolve, a framework for automatically optimizing agent harnesses while keeping model weights fixed. Its central idea is to maintain executable harness lineage separately from unresolved improvement directions, represented as inquiries. A Curator updates inquiry states using evaluation evidence and selects a parent harness, while an Evolver uses open inquiries to generate a child harness. A Relational Trace links the targeted inquiries, harness modifications, and evaluation outcomes.

Critical strengths
  • The Harness Lineage and Inquiry Graph serve distinct purposes. Keeping them separate while linking inquiries to harness changes and evaluation evidence through the Relational Trace allows unsuccessful attempts to inform subsequent search. This is a well-motivated and practically useful design.

  • The experiments show consistent improvements over the evaluated baselines across five benchmarks and two backbones, supported by targeted ablations and multi-seed results in two representative settings.

Critical weaknesses
  • Although Table 3 includes a Persistent Inquiry List ablation, it would be useful to compare against Meta-Harness augmented with a continuously updated structured inquiry list and access to the same evaluation evidence and search history. Would RelEvolve still maintain a consistent advantage over this stronger baseline?
  • The reliability of inquiry lifecycle decisions remains unclear; please provide an independent audit of their consistency and evidential support, distinguishing inquiries closed as resolved from those deemed no longer useful.
  • There is an extra space before “Validation scores support” on line 210.
Questions for authors

See the weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.