acceptodds
Under review as a conference paper at ICLR 2027

CompEvolve: Competitive Evolution of LLM Agent Harness Regularized by Behavioral Diversity and Generalization Judging

Abstract

Evolutionary search is a well-established paradigm for automatic design, and has recently been applied to evolve the harness around a large language model (LLM): its prompt, control flow, and multi-agent structure. Because it modifies the harness rather than the model weights, this paradigm extends even to closed-source models, at lower cost and with greater interpretability than fine-tuning. Within it, competition, keeping only the fittest of competing harnesses, is a natural mechanism for directed improvement. We find, however, that naive competition among LLM agents does not reliably surpass independent, non-competitive evolution: the population rapidly converges to a few near-identical architectures, and selection by training accuracy alone rewards designs that overfit the training problems. We present CompEvolve, which turns competitive evolution into a reliable optimizer through three components: information isolation prevents a capable LLM architect from exploiting the training signal, compelling it to improve genuinely transferable ability; selection regularized by behavioral diversity retains agents that are initially weak but complementary to the rest of the population, which are more likely to be recombined into stronger agents in later generations; and an LLM generalization judge evaluates the transferability of an agent's design at the architecture level, complementing training accuracy. CompEvolve consistently improves out-of-distribution generalization over independent evolution across multiple reasoning benchmarks and LLMs of varying capability.

Primary area:
optimization
Keywords:
Agent Evolution, Harness Evolution, Fitness of Genetic Algorithm

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AXEpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

An LLM-based evolutionary method is introduced for improving LLM harnesses. The approach includes behavioral diversity and generalization considerations and is shown to be superior to 'naive' competition-based evolution.

Critical strengths

Overall the algorithm is creative and the paper shows the progression from naive competition, identifying its failure points, and addressing them in the new algorithm. The ablations show the introduced modifications to the naive approach seem to be the right ones. The inclusion of failure points alongside the other three lineages is novel.

Critical weaknesses

None

Questions for authors

There is no mention of the type and size of LLM used for the various steps of the algorithm. Authors also do not mention the LLMs tested with the various harnesses. What prompt was used for the LLM judge? Have the authors applied this method on agentic benchmarks such as SWE bench or terminal bench?

Reviewer_BxkpOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article proposes CompEvolve, a competitive evolution framework for LLM agents that relies on behavior diversity regularization and generalization judges to screen candidate agents. The framework compares independent evolution and naive competition baselines in mathematical, coding, and visual reasoning tasks, achieves performance improvements on multiple OOD datasets, and is accompanied by ablation experiments, population convergence analysis, and correlation verification between Judge scores and OOD effects. The review mainly focuses on three points: there are inconsistencies between the algorithm steps in the main text and Figure 2 in the process; The diversity selection mechanism of this method lacks controlled comparison with existing niche methods such as EvoFlow, and the innovation boundary is not clear enough; The generalization judge relies on accuracy drop to define cheating labels, which has limited discriminative ability and lacks stability statistics under multiple random seeds, weakening the credibility of experimental conclusions.

Critical strengths

This work established a multi task competitive evolution evaluation system, covering mathematical, coding, and visual reasoning scenarios, and achieved stable improvement compared to the basic evolution baseline under OOD generalization settings. The article conducted component ablation, population evolution analysis, and correlation analysis to verify that the three modules of information isolation, behavioral diversity, and generalization judge each play a role; The experiment not only reports the final accuracy, but also provides evolutionary cases, which helps to understand group dynamics. The entire system has practical reference value, and the integrated effect cannot be denied just because the components have previous foundations.

Critical weaknesses

There are three core flaws in the paper that affect the reliability of the argument. Firstly, the algorithm description in the main text requires first screening the top 5 candidates based on accuracy, and then calculating the diversity and Judge comprehensive score. However, the candidate ranked 8th in Figure 2 is still retained and subsequently wins, which conflicts with the process and lacks reasonable explanation, damaging the soundness of the method. Secondly, its behavior diversity selection mechanism and mature niche diversity strategies such as EvoFlow lack controlled experiments within the same framework, which cannot prove the independent advantages of this selection mechanism and the boundary of innovation contribution is blurred. Thirdly, the generalization judge uses the huge difference in accuracy between the training set and OOD as cheating labels, which cannot distinguish between hard coded cheating and ordinary distribution bias; At the same time, the lack of statistical results on repeated evolution of multiple random seeds makes it difficult to prove the stability and reliability of generalization benefits.

Questions for authors

There is an apparent inconsistency between the algorithm description (filter top-5 by accuracy first) and Figure 2 (candidate ranked 8 is retained). Could you clarify the exact selection pipeline used to generate Figure 2, and reconcile this with Section 3.1?

Under the same evolutionary backbone, how does the proposed behavior-diversity selection compare against established niching baselines such as EvoFlow? What unique benefit does answer-level behavioral diversity bring?

Reviewer_tqLMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CompEvolve, a framework for evolving LLM agent harnesses through competition. It combines information isolation, behavioral diversity based on output agreement, and an LLM-based generalization judge to mitigate architectural convergence and overfitting. Experiments on mathematical reasoning, code generation, and visual reasoning compare the framework with independent evolution, naive competition, and existing agent optimization methods.

Critical strengths
  1. The motivation is clear: how to make competitive harness evolution produce useful, transferable improvements rather than converge prematurely or exploit training examples.
  2. The paper is clearly written and includes component-wise ablations across multiple domains, providing useful evidence about the incremental effects of the proposed components.
Critical weaknesses
  1. The practical value relative to computational cost is insufficiently established. Harness search incurs costs for architecture generation, candidate evaluation, and judging, while the resulting harnesses may also require multiple solver calls per query. The evaluation does not adequately establish the accuracy–cost trade-off against simpler alternatives, particularly direct CoT with a stronger model or straightforward sampling and verification. Search costs may be amortized through reuse, but the required deployment volume and reuse across task domains remain unclear. Evaluation primarily on static benchmarks, some approaching saturation, provides limited evidence that the additional complexity and cost are warranted in broader agentic settings.

  2. The evidence does not sufficiently establish what the generalization judge predicts. Its correlation with OOD accuracy is encouraging, but does not distinguish prediction of generalization from recognition of generally effective designs or detection of obvious specification gaming. It remains unclear whether the judge provides useful information beyond training accuracy among otherwise comparable, non-cheating candidates. A comparison with selection using held-out validation data is also missing. Consequently, the necessity of the judge and the interpretation of its score as a generalization signal are not sufficiently supported.

  3. Behavioral diversity via output agreements and stochastic selection draw on established ideas, and softmax sampling itself is a standard mechanism. Although EvoFlow and evolution controls are included, the experiments do not clearly establish the advantage of the proposed combination over simpler diversity-aware evolutionary alternatives. In particular, the cumulative ablations do not fully separate the benefit of behavioral diversity from that of less-greedy selection. Several incremental gains are modest, and the lack of independent repeated runs and uncertainty estimates makes their reliability difficult to assess.

Questions for authors
  1. Please report search and deployment costs separately, including architect and judge overhead, and compare accuracy versus total cost with stronger-model CoT and simple sampling or verification baselines. How much reuse is needed to justify the search cost? It is also unclear if the proposed method can work on stronger models and more complex evaluation setups.
  2. Does the judge predict OOD performance after controlling for training accuracy and excluding obvious cheating candidates? How does it compare with held-out validation-based selection under comparable resource constraints?
  3. Could the authors include an accuracy-only softmax selection control and a standard behavioral-diversity baseline? A comparison with an appropriately adapted approach such as GEPA would also help clarify the contribution.
  4. Please evaluate robustness across independent search runs and report sensitivity to diversity/judge weights, sampling temperature, and judge thresholds.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.