acceptodds
Under review as a conference paper at ICLR 2027

Where Autonomy Belongs: A Separation of Powers for Agentic Program Evolution

Abstract

LLM-driven program evolution rests on three decisions: govern directs the search from accumulated evidence, propose constructs the next candidate, and judge settles validity and fitness. All three appear in every system, and existing work splits into two lines over who makes each. Controller-centric systems strengthen govern and neglect the autonomy of propose, which caps what evolution can reach. Agent-centric systems let a coding agent take over both govern and propose, at a cognitive and computational cost. Neither line protects the independent authority of judge, so benchmark reliability suffers. These failures share one root: autonomy is misallocated across the three decisions. We address this problem with EvoTrias, a separation-of-powers architecture in which each decision has one owner: Structured Governance for what spans evolution rounds, Open Proposal for construction inside a task, and Protected Judgment behind a private evaluator. These owners exchange typed contracts only, and runtime hooks reinforce the boundaries. This separation also lets EvoTrias run as a plugin that turns an existing coding agent into an evolution system. EvoTrias sets a new state of the art on mathematical, systems, and algorithm-design benchmarks. Three further results show what each owner buys: separating Governance from Proposal cuts token cost by 64%, Open Proposal extends evolution to a 680K-line industrial repository, and Protected Judgment keeps benchmark scores faithful to transfer.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-driven program evolution, evolutionary search, open-ended discovery, algorithm discovery

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jtqwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EvoTrias, a framework for LLM-driven program evolution, where the main idea is to separate the responsibilities of governance, proposal, and judgment. Basically, the authors argue that existing approaches either give too much autonomy to the coding agent or too little, and that by separating these responsibilities, one can get better performance and efficiency. The proposed framework combines an evolutionary controller, coding agents, and a protected evaluator. the authors evaluate it on a number of benchmarks comparing mainly to the baseline of CORAL-4, with some results on optimizing a large code repository.

Critical strengths
  1. I think the problem that the paper is trying to address is relevant, specially with the recent progress in coding agents and LLM-based program evolution. Having a framework that can manage these agents and make them more efficient is certainly useful.

  2. The experiments are relatively extensive, including comparisons with multiple existing methods and across different benchmarks and LLM backbones. I also appreciate that the authors have included quite a few ablation studies and implementation details in the appendix.

  3. I found the experiment on evaluator exposure and how this can result in overfitting to be interesting, and I think this is one of the more convincing parts of the paper.

Critical weaknesses
  1. The main research contribution is not very clear to me. My main concern with this paper is that it reads more like a technical report of a software framework rather than a research paper. I understand that the authors are proposing to separate governance, proposal, and judgment, however, I am not really convinced what is fundamentally novel about this. To me it seems that the proposed method is mostly putting together a number of already existing ideas and components in a particular way. I think the paper needs to better explain what is the key research insight here, beyond the fact that this particular architecture happens to work well.

  2. The experiments do not really validate the main hypothesis of the paper. The authors argue that the allocation of autonomy is the main problem with existing approaches, however, I don't think the experiments really establish this. For instance, the authors argue that having agents manage the governance can lead to unreliable memory, premature narrowing of the search, and unnecessary computational costs. But I could not really find experiments that directly evaluate these claims.

Most of the comparisons are against other systems which differ in many different aspects, so it is difficult to know whether the improvements are actually because of the proposed separation of responsibilities or because of other design choices.

  1. The ablation studies are not sufficient to isolate the contribution. While the paper includes multiple ablations, most of them seem to be about removing individual components, such as Notes, Meta-guidance, or changing the initialization. I think these are useful, but they don't really answer the main question of the paper, which is whether separating governance from proposal is actually better than letting an agent manage both.

I think a much more interesting experiment would be to keep the same coding agent, evaluator, and budget, and only change how the governance and proposal responsibilities are allocated. Without something like this, I find it difficult to attribute the improvements to the central idea of the paper.

  1. The large repository experiment is interesting, but the comparison is not very convincing. The authors report improvements on OR-Tools, which is a large C++ repository. However, from what I understand, the actual editing surface is limited to four specified source files. Also, the main comparison seems to be against a single attempt of the coding agent.

I don't think this is a very strong baseline, since naturally running an evolutionary optimization for 50 iterations should have an advantage over a single attempt. I would expect comparisons against simpler approaches, such as running the coding agent multiple times independently or a basic iterative refinement loop, under the same computational budget. It would also be useful to evaluate the final solutions on held-out instances.

Questions for authors
  1. What exactly is the main novel research contribution of this paper? I understand the proposed separation of governance, proposal, and judgment, but why should this be considered a new research insight rather than an architectural or engineering design choice? I think this distinction is important and currently not very clear to me.

  2. Can the authors provide a more direct experiment to validate the autonomy allocation hypothesis? For instance, keeping the same agent, tools, and budget, and comparing a setup where the agent manages both governance and proposal against one where these responsibilities are separated. I think this would be much more convincing than comparisons between completely different frameworks.

6 comments
AuthorsOpenReview ↗

Response - 1

We sincerely thank Reviewer jtqw (R3) for the detailed and constructive review. We are greatly encouraged that you consider a framework that manages coding agents and makes them more efficient "certainly useful." We are also glad that you find both the evaluator-exposure study and the large-repository experiment "interesting." Below, we provide detailed responses to each of your comments.

Q1: The main research contribution is unclear. Why is the separation of governance, proposal, and judgment a research insight rather than a combination of existing components or an engineering design choice? (W1, Q1)

A1:

Thank you for this question. It gives us the chance to state our core insight and contribution more clearly. Dividing an agent system into roles by responsibility is indeed common practice. Our work, however, studies a fundamentally different problem that begins where this division ends, and to our knowledge it is the first to do so: once the proposer is a coding agent, do the powers stay divided, and what does it cost when they do not?

Before coding agents, the boundaries came for free. While propose was a single LLM call (OpenEvolve, AdaEvolve, EvoX), nobody had to ask. A call sees only its prompt: it cannot read the evaluator or the history or choose its parent, and equally cannot read a repository, build, or debug. A coding agent removes that limit, and its file and shell access removes the boundaries with it. Recent work embraces this and extends the agent's autonomy to the search itself: CORAL's agent also chooses what to build on, writes the shared memory, and finds the grader source in its working directory. Existing systems therefore come in two kinds: boundaries without autonomy, or autonomy without boundaries.

So what does autonomy without boundaries cost? As agents grow stronger, the field is heading this way, yet to our knowledge no prior work has measured the cost. Our paper does, one decision at a time:

Autonomy overOutcomeEvidence
propose: read the repository, edit files, build, debuggain: tasks a single LLM call cannot take onOR-Tools, 680K lines of C++: +13.2% / +16.9% over the unpatched solver (§6.1)
govern: read the full history, choose the parent, write shared notescost: lower score, more tokensvs. CORAL-4: ahead on six of seven Math tasks at 64% lower token cost (§6.2); isolated in new §6.3: Signal Processing 0.760 → 0.639
judge: read the evaluator sourcecost: inflated score§6.4 (§6.3 in the submission): +0.093 on the original score, +0.009 out of distribution

The govern row is the premise of the paper: §1 argues that an agent owning the search pays a cognitive cost and a computational cost, and the submission supports this through the comparison with CORAL. The new §6.3, added at your suggestion, isolates it with a single variable: with the agent, model, evaluator, and budget unchanged, handing the search decisions to the agent lowers Signal Processing from 0.760 to 0.639 and raises the token cost of a run by 19–28% (details in A2).

Where the boundary belongs. This had no answer before; the current trend is to draw none and hand more decisions to a stronger agent. Our results give one: the boundary belongs between propose and the other two decisions. EvoTrias keeps the agent's full tool capability inside a task and rebuilds the boundaries outside the agent. To our knowledge, it is the first program-evolution system to do both. Plugging an agentic proposer into an existing controller is not enough: AdaEvolve and EvoX then change by only −1.2% to +1.9% (Appendix C.5).

The next question is how to build these boundaries while keeping the agent's full tool capability. Prompts alone do not hold them: as Appendix A.7 shows, even with the prohibition written in its prompt, one task executor still tried to read evaluator.py, and another tried to read a sibling task's evaluation record. EvoTrias therefore defines a clear division of power among its three planes (Structured Governance, Open Proposal, and Protected Judgment), in which each decides alone inside its own boundary, and uses runtime hooks to keep these boundaries firm (§4, Appendix A.7).

This division has two direct consequences. First, parallelism comes at almost no extra cost. Structured Governance makes the search decisions outside the agent, so running more tasks in parallel only needs more executors, each building one candidate: EvoTrias's cost varies by less than 5% from 1 to 8 concurrent executors, while CORAL's grows 3.9× from one agent to four. Second, EvoTrias runs as a plugin. Govern and judge live outside the agent, and an executor only builds one candidate from an assigned context, so any coding agent that supports MCP, hooks, and subagents can host EvoTrias unmodified.

Revision. New §6.3 reports the controlled ablation (setup and results in A2).

AuthorsOpenReview ↗

Response - 2

Q2: The experiments and ablations do not directly validate the autonomy-allocation hypothesis. A comparison that keeps the same agent, tools, and budget and only changes how governance and proposal are allocated is needed. (W2, W3, Q2)

A2: We are truly grateful for this suggestion, which has meant a great deal to our paper. Q2 does more than raise a concern: it lays out the exact experiment that settles it. We ran it just as you describe (new §6.3), with the same agent, tools, and budget and a single difference, namely who makes the search decisions. It tests the core motivation of our paper directly, and the paper is substantially stronger for it.

Setup.

  • The single variable. The search decisions are which parent and inspirations each task starts from, which lessons enter the shared notes, and when to change direction. In configuration A (EvoTrias), Structured Governance makes them. In configuration B, the coding agent that writes the candidates makes them itself, with the full run history available (every candidate with its code, score, and lineage, every failed task, and every note). B's prompt describes its tools and leaves the strategy to the agent.
  • Held constant. The coding agent (Claude Code), the model (GLM-5.2), the instructions for constructing a candidate, the protected evaluator, the budget of 50 tasks, the number of initial directions (4) and of concurrent executors (4), and the code version.
  • Runs. Three runs per configuration on Circle Packing and Signal Processing. To count tokens in the same way for both configurations, we re-ran A alongside B, so A's scores here differ from Table 1.

Results (Table 19 in Appendix C.11 of the revised paper).

Owner of the searchScore ↑Tokens / runTokens / taskLineagesTop rootNotes / runSmall-Δ
Circle Packing
A: Structured Governance2.6340 ± 0.002625.9M321K → 383K3.70/34.00%
B: coding agent2.6253 ± 0.012633.2M364K → 675K1.73/33591%
Signal Processing
A: Structured Governance0.7597 ± 0.031727.6M368K → 418K4.00/316.30%
B: coding agent0.6391 ± 0.057932.8M355K → 605K1.03/344.789%

Score: final best, mean ± std over three runs. Tokens / task: iterations 5–15 → 40–50. Lineages: root lineages that receive a task after the root phase. Top root: runs whose final best descends from the root with the highest root-phase score. Notes / run: notes written in a run. Lineages and Notes / run are averaged over three runs. Small-Δ: share of notes from attempts within 2% of the parent's score.

Analysis. A scores higher on both tasks: 2.6340 against 2.6253 on Circle Packing, which is close to saturation, and 0.760 against 0.639 on Signal Processing, where A's lowest run (0.736) is above B's highest (0.689). The other columns measure the three costs that you found untested in W2:

  • Unnecessary computational cost. B spends 19–28% more tokens per run. Its agent rereads a history that lengthens every iteration, so its tokens per task grow by ×1.70–1.86 from iterations 5–15 to iterations 40–50, against ×1.13–1.20 for A (new Figure 7).
  • Premature narrowing. After the root phase, B's search collapses onto 1.0–1.7 of its four root lineages; on Signal Processing, all three runs take every later task from one lineage. In all six B runs the final best descends from the root that scored highest in the root phase, so a root that starts lower gets almost no chance. A keeps 3.7–4.0 lineages in use, and in all six A runs the final best comes from another root.
  • Unreliable memory. B writes 35–44.7 notes per run, 89–91% of them from attempts that moved the score by less than 2%. A admits 4.0–16.3 notes per run, none below 2%.

Relation to the evaluator-exposure study. The study you found convincing (§6.3 in the submission, §6.4 in the revised paper) is the same kind of experiment for judge: it leaves part of judge's authority, the evaluator source, open to the coding agent and changes nothing else. The original score rises by 0.093 while the out-of-distribution gain shrinks to 0.009. The new experiment does the same for govern. Together, they show that on top of a division of roles, the boundaries around govern and judge are each necessary.

Revision. New §6.3 with Figure 7; the full table (Table 19) and the protocol are in Appendix C.11.

AuthorsOpenReview ↗

Response - 3(1)

Q3: The OR-Tools comparison is not convincing: (1) the editing surface is limited to four files; (2) the baseline is a single attempt of the coding agent; (3) there is no evaluation on held-out instances. (W4)

A3:

We really appreciate how carefully you examined the OR-Tools experiment. Your three points are specific and constructive, and addressing them has made this part of the paper considerably more convincing. We answer them in order; (2) and (3) each come with a new experiment.

(1) Editing surface.

The four files limit where a patch may be written. What the proposer has to read and get right is far larger, and this is what places the task beyond a single LLM call.

ScopeSize
Four editable files698 + 1,213 + 807 + 2,355 = 5,073 lines; about 30 classes and structs and about 120 functions, none marked for editing
Code they depend on22 other OR-Tools headers included directly; a dependency closure of 39 files and about 24.9K lines
Code that drives themcp_model_solver.cc (3,081 lines) constructs and schedules this local search; 13 parameters in sat_parameters.proto configure it
Module / repositoryortools/sat/ 234K lines / about 680K lines

Edits inside the four files are also coupled. This local search maintains every constraint violation incrementally, so a change to jump scoring must be carried into the incremental updates and the weight updates; a patch that misses one of them fails to build or silently degrades the solver. Writing a patch that compiles and improves the solver therefore takes understanding code far outside the four files, then building the complete solver and repairing from its output. The edit surface is four files, but the code a working patch depends on spans about 25K lines in 39 files, well beyond what a single LLM call can read and get right without building or running anything. The allowlist itself only protects the build scripts, the evaluator, the instances, and the scoring code, and keeps every candidate a patch that can be reviewed directly.

(2) Baselines under the same budget.

We agree that 50 iterations should be compared with more than a single attempt. Following your suggestion, we added a basic iterative-refinement loop of the agent alone, and the revised submission now includes it (§6.1 and Figure 5; protocol in Appendix C.8).

Setup. The same host and model (Claude Code, DeepSeek-V4-Flash), task description, edit surface, and evaluator, and the same budget of 50 evaluated candidates, without EvoTrias. In each round the agent reads the evaluation result of its previous candidate, makes one change on top of that candidate, and evaluates the new patch; it keeps its own progress log across rounds.

MethodEvaluated candidatesAll 300 instances60-instance subset
Single pass of the agent (mean of 4 independent runs)1−0.1%+1.3%
Iterative refinement by the agent alone50+4.6%+2.2%
EvoTrias50+13.2%+16.9%

Gain over the unpatched solver. The two instance sets are separate experiments.

Analysis. Iterating does help the agent, as you expected: on the full set it moves from −0.1% to +4.6%. With the same 50 evaluated candidates, EvoTrias reaches +13.2% on the full set and +16.9% on the subset, against +4.6% and +2.2% for the agent alone. This loop is also the simplest form of an agent governing its own search: it decides what to try next from its own record. Its gap to EvoTrias on a repository-scale task is consistent with the controlled ablation in A2.

Reviewer_czScOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • In previous agent-centric systems, a single agent was responsible for both governance and proposal, and could sometimes access the evaluator's internals. EvoTrias assigns these authorities to different components and prevents them from interfering with one another's authority.
  • This improves search quality, reduces computational costs, and prevents overfitting to the evaluation code.
Critical strengths
  • The idea of separating powers is interesting. I think preventing access to the evaluator's source code is an important effort.
  • The experiments also include a comprehensive set of baselines and benchmarks. Using two models and three runs provides reliable results.
  • The proposed harness also appears to outperform existing methods.
Critical weaknesses
  • I have concerns about novelty. The proposed method essentially seems to separate two functions previously handled by a single agent, governing the search and proposing algorithmic improvements, into two distinct components.

  • The evidence is also not sufficiently convincing. It is unclear whether the improvements come from the separation of powers or from various other techniques. For example, progress annealing itself is a technique independent of the separation of powers. Including it makes it difficult to isolate the effect of separating powers. I think the authors should compare simpler settings in which everything except the presence or absence of separation of powers is held constant.

  • Since spending more computational resources can lead to better performance, it is preferable to compare performance under a fixed budget. In Table 1, the proposed harness appears to outperform existing methods, but it is unclear whether it is budget-efficient. Although the authors control the number of proposals or wall-clock time, they do not control the token budget. Since different methods may consume substantially different numbers of tokens per proposal, the current comparison does not establish which method performs best under a fixed computational budget.. The proposed method does appear to be more token-efficient than CORAL-4, but I think the authors should repeat the comparisons under a fixed budget.

Questions for authors

n/a

6 comments
AuthorsOpenReview ↗

Response - 1

We sincerely thank Reviewer czSc (R2) for the thoughtful and constructive review. We are encouraged that you find the idea of separating powers "interesting" and consider preventing access to the evaluator's source code "an important effort." We also appreciate your recognition of our "comprehensive set of baselines and benchmarks" and of the reliability that two models and three runs provide. Below, we provide detailed responses to each of your comments.

Q1: I have concerns about novelty. The proposed method essentially seems to separate two functions previously handled by a single agent, governing the search and proposing algorithmic improvements, into two distinct components.

A1:

Thank you for this comment. It gives us the chance to state our core insight and contribution more clearly. Dividing an agent system into roles by responsibility is indeed common practice. Our work, however, studies a fundamentally different problem that begins where this division ends, and to our knowledge it is the first to do so: once the proposer is a coding agent, do the powers stay divided, and what does it cost when they do not?

Before coding agents, the boundaries came for free. While propose was a single LLM call (OpenEvolve, AdaEvolve, EvoX), nobody had to ask. A call sees only its prompt: it cannot read the evaluator or the history or choose its parent, and equally cannot read a repository, build, or debug. A coding agent removes that limit, and its file and shell access removes the boundaries with it. Recent work embraces this and extends the agent's autonomy to the search itself: CORAL's agent also chooses what to build on, writes the shared memory, and finds the grader source in its working directory. Existing systems therefore come in two kinds: boundaries without autonomy, or autonomy without boundaries.

So what does autonomy without boundaries cost? As agents grow stronger, the field is heading this way, yet to our knowledge no prior work has measured the cost. Our paper does, one decision at a time:

Autonomy overOutcomeEvidence
propose: read the repository, edit files, build, debuggain: tasks a single LLM call cannot take onOR-Tools, 680K lines of C++: +13.2% / +16.9% over the unpatched solver (§6.1)
govern: read the full history, choose the parent, write shared notescost: lower score, more tokensvs. CORAL-4: ahead on six of seven Math tasks at 64% lower token cost (§6.2); isolated in new §6.3: Signal Processing 0.760 → 0.639
judge: read the evaluator sourcecost: inflated score§6.4 (§6.3 in the submission): +0.093 on the original score, +0.009 out of distribution

The govern row is the premise of the paper: §1 argues that an agent owning the search pays a cognitive cost and a computational cost, and the submission supports this through the comparison with CORAL. The new §6.3, added at your suggestion, isolates it with a single variable: with the agent, model, evaluator, and budget unchanged, handing the search decisions to the agent lowers Signal Processing from 0.760 to 0.639 and raises the token cost of a run by 19–28% (details in A2).

Where the boundary belongs. This had no answer before; the current trend is to draw none and hand more decisions to a stronger agent. Our results give one: the boundary belongs between propose and the other two decisions. EvoTrias keeps the agent's full tool capability inside a task and rebuilds the boundaries outside the agent. To our knowledge, it is the first program-evolution system to do both.

This division has two direct consequences. First, parallelism comes at almost no extra cost. Structured Governance makes the search decisions outside the agent, so running more tasks in parallel only needs more executors, each building one candidate: EvoTrias's cost varies by less than 5% from 1 to 8 concurrent executors, while CORAL's grows 3.9× from one agent to four. Second, EvoTrias runs as a plugin. Govern and judge live outside the agent, and an executor only builds one candidate from an assigned context, so any coding agent that supports MCP, hooks, and subagents can host EvoTrias unmodified.

Revision. New §6.3 reports the controlled ablation (setup and results in A2 ).

AuthorsOpenReview ↗

Response - 2

Q2: The evidence is also not sufficiently convincing. It is unclear whether the improvements come from the separation of powers or from various other techniques.

A2:

We are especially grateful for this suggestion. The experiment it led to tests the core motivation of our paper directly and makes the paper substantially stronger. We added a controlled experiment that holds everything constant except the separation of powers (new §6.3): the two configurations differ in a single variable, namely who makes the search decisions.

Setup.

  • The single variable. The search decisions are which parent and inspirations each task starts from, which lessons enter the shared notes, and when to change direction. In configuration A (EvoTrias), Structured Governance makes them. In configuration B, the coding agent that writes the candidates makes them itself, with the full run history available (every candidate with its code, score, and lineage, every failed task, and every note). B's prompt describes its tools and leaves the strategy to the agent.
  • Held constant. The coding agent (Claude Code), the model (GLM-5.2), the instructions for constructing a candidate, the protected evaluator, the budget of 50 tasks, the number of initial directions (4) and of concurrent executors (4), and the code version.
  • Runs. Three runs per configuration on Circle Packing and Signal Processing. To count tokens in the same way for both configurations, we re-ran A alongside B, so A's scores here differ from Table 1.

Results (Table 19 in Appendix C.11 of the revised paper).

Owner of the searchScore ↑Tokens / runTokens / taskLineagesTop rootNotes / runSmall-Δ
Circle Packing
A: Structured Governance2.6340 ± 0.002625.9M321K → 383K3.70/34.00%
B: coding agent2.6253 ± 0.012633.2M364K → 675K1.73/33591%
Signal Processing
A: Structured Governance0.7597 ± 0.031727.6M368K → 418K4.00/316.30%
B: coding agent0.6391 ± 0.057932.8M355K → 605K1.03/344.789%

Score: final best, mean ± std over three runs. Tokens / task: iterations 5–15 → 40–50. Lineages: root lineages that receive a task after the root phase, averaged over three runs. Top root: runs whose final best descends from the root with the highest root-phase score. Notes / run: notes written in a run, averaged over three runs (in B a task may write more than one). Small-Δ: share of notes from attempts within 2% of the parent's score.

Analysis.

  • Score. A is higher on both tasks. Circle Packing is close to saturation and the gap is small (2.6340 against 2.6253). On Signal Processing the gap is 0.760 against 0.639, and A's lowest run (0.736) is above B's highest (0.689).
  • Cost. B spends 19–28% more tokens per run, and its cost keeps growing within a run: the coding agent rereads a history that lengthens every iteration, so its tokens per task grow by ×1.70–1.86 from iterations 5–15 to iterations 40–50, against ×1.13–1.20 for A. The new Figure 7 compares score and cost in more detail.
  • Search breadth and shared notes. After the root phase, A uses 3.7–4.0 root lineages on average, while B uses 1.0–1.7. B writes 35–44.7 notes per run, 89–91% of them from attempts that moved the score by less than 2%; A admits 4.0–16.3 notes per run, none below 2%. B turns attempts with little measured fitness change into shared lessons and concentrates the search on fewer root lineages. Alongside its higher token cost, these patterns support the concern in §1 that agent-managed governance can lead to unreliable memory, premature search narrowing, and unnecessary computational costs.

Conclusion. With the agent, model, evaluator, and budget held constant, separating govern from propose raises the score and lowers the cost.

Relation to the evaluator-exposure study. §6.3 of the submission (§6.4 in the revised paper) is the same kind of experiment for judge: letting the search read the evaluator, with nothing else changed, raises the original score by 0.093 while the out-of-distribution gain shrinks to 0.009. Together, the two experiments show that on top of a division of roles, the boundaries around govern and judge are each necessary.

Revision. New §6.3 with Figure 7; the full table (Table 19) and the protocol are in Appendix C.11.

AuthorsOpenReview ↗

Response - 3

Q3: Since spending more computational resources can lead to better performance, it is preferable to compare performance under a fixed budget.

A3: Thank you for this suggestion. A comparison under a fixed token budget is indeed well worth adding, and the revised paper now includes it (new Table 3 in §6.2). EvoTrias keeps the highest mean on both tasks.

Setup. Each baseline receives the mean number of tokens that a 50-iteration EvoTrias run consumes on the same task, and stops once this budget is spent. For OpenEvolve, EvoX, and AdaEvolve the budget buys 102–158 iterations. For CORAL it fixes the wall-clock time at which the run stops. All methods use GLM-5.2, with three runs per cell, on Circle Packing and Signal Processing.

Results (Table 3 of the revised paper).

MethodCircle Packing: budgetCircle Packing: scoreCircle Packing: ΔSignal Processing: budgetSignal Processing: scoreSignal Processing: Δ
OpenEvolve158 iter.2.5940 ± 0.0265+0.0508153 iter.0.6638 ± 0.0223+0.0265
EvoX102 iter.2.6297 ± 0.0064+0.0201110 iter.0.7057 ± 0.0120−0.0041
AdaEvolve129 iter.2.6277 ± 0.0039−0.0038131 iter.0.7273 ± 0.0042+0.0312
CORAL-12.66 h2.6266 ± 0.0064+0.00333.04 h0.6781 ± 0.0593+0.0143
CORAL-40.68 h2.6216 ± 0.0204−0.01260.79 h0.6894 ± 0.0134−0.0189
EvoTrias50 iter.2.6348 ± 0.0019–50 iter.0.7329 ± 0.0058–

Budget is the number of iterations that the token budget buys, or wall-clock hours for CORAL. Score is mean ± std over three runs. Δ is the change from the same method's mean in Table 1. Best in bold, second best in italics.

Analysis.

  • Same tokens. EvoTrias has the highest mean on both tasks: 2.6348 on Circle Packing (second: EvoX, 2.6297) and 0.7329 on Signal Processing (second: AdaEvolve, 0.7273).
  • The baselines receive more evaluations. Under this protocol the controller-centric baselines run 102–158 iterations, 2.0–3.2 times the proposals and evaluator calls of EvoTrias. OpenEvolve gains the most from them (+0.051 on Circle Packing, +0.027 on Signal Processing); AdaEvolve and EvoX change by −0.004 to +0.031. Table 1 matches proposal slots and evaluator calls, Table 3 matches tokens, and EvoTrias has the highest mean under both.
  • CORAL. CORAL-4 runs four complete agents, so the same budget stops it after 0.68–0.79 hours, and its score falls on both tasks (Circle Packing 2.6342 → 2.6216, Signal Processing 0.7083 → 0.6894). CORAL-1 runs for 2.66–3.04 hours under this budget and stays below EvoTrias on both tasks.

Conclusion. With tokens matched, EvoTrias still has the highest mean on both tasks. Token efficiency is also a direct advantage of EvoTrias over the other systems built on coding agents: with the same tokens, CORAL-1 and CORAL-4 both stay below EvoTrias. In EvoTrias, Structured Governance makes the search decisions outside the agent, so the agent's tokens go to constructing candidates. The controlled ablation in A2 shows the same effect with everything else fixed: when the agent makes the search decisions itself, a run spends 19–28% more tokens and scores lower.

Revision. §6.2, new paragraph "Matched token budget", and Table 3.

Reviewer_zv2DOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EvoTrias, an LLM-driven program evolution framework that separates the evolutionary process into three components: Structured Governance, Open Proposal, and Protected Judgment. These components respectively handle search management, candidate generation, and evaluation, with typed interfaces and runtime hooks enforcing their boundaries. Experiments on Math, ADRS, and Frontier-CS benchmarks demonstrate competitive performance against existing evolutionary methods.

Critical strengths
  1. The paper is well-organized and provides detailed descriptions of the proposed framework and its implementation.
  2. The experiments cover multiple benchmarks and LLM backbones, with extensive comparisons and ablation studies.
Critical weaknesses
  1. This paper does not clearly list its contributions in the introduction section. The idea of separating planning, execution, and evaluation has already appeared in earlier works such as Reflexion, which introduced a similar division of responsibilities. Therefore, we believe that the main contribution of this paper lies in enforcing permission boundaries. However, CORAL (COLM 2026) already incorporates similar safety mechanisms, including evaluator separation and workspace isolation. Although this paper argues that CORAL's protection is less strict in its code implementation, stricter enforcement of permission boundaries alone does not seem sufficient to justify the novelty of a new paper. We encourage the authors to clarify the fundamental contributions and technical novelty of EvoTrias in the introduction section. This is the most important factor in determining whether we would reconsider our rating.
  2. Section 6.1 presents repository-scale optimization as an open-ended task in which the agent must identify relevant code across a 680K-line repository. However, Appendix C.8 reveals that modifications are restricted to only four predefined files. This substantially narrows the search space and weakens the claim that the method can autonomously handle open-ended repository-scale optimization.
  3. The main experiments compare EvoTrias, which uses a general-purpose Coding Agent, against controller-centric baselines that primarily rely on single LLM calls. A more convincing comparison would replace the single-call proposers in existing evolutionary methods with the same Coding Agent used by EvoTrias, thereby isolating the contribution of the proposed architecture from differences in Proposal capabilities.
Questions for authors

Due to time constraints, we would like the authors to prioritize addressing Weakness 1. This paper provides extensive experiments, and our primary concern is not the empirical evaluation but the unclear methodological contributions and novelty of the proposed framework. We apologize for the incorrectly copied review comments and any confusion or inconvenience this may have caused in preparing your response.

7 comments
AuthorsOpenReview ↗

Request to Correct a Submission Mismatch

Thank you for your time. We noticed a substantial mismatch between your review and our paper: several points discuss content that does not appear in our submission.

  • Weakness 2 quotes “Section 5.5” as stating, “The primary Raw- and Filtered-public controls do not instantiate the full L2 pathway,” and describes “staged Evidence-Gated Inheritance” as our central contribution. Our Section 5 ends at 5.3, and neither this sentence nor these terms appears anywhere in our paper or appendix.
  • Strength 2 describes our work as addressing “preventing unreliable improvements from being inherited by self-evolving agents,” which is not a problem studied in our paper.

These discrepancies concern the central contribution and experimental evidence discussed in the review, and raise a serious concern that the assessment and rating are based on a different paper.

Could you please check whether the review was intended for our submission? If a mix-up occurred, we would appreciate a reassessment of our paper and a corresponding update to the review and rating.

Thank you for looking into this. We would be happy to clarify any aspect of our work.

AuthorsOpenReview ↗

Response - 1(1)

We sincerely thank Reviewer zv2D (R1) for the careful new review. We are encouraged that you find the paper "well-organized," with "detailed descriptions of the proposed framework and its implementation," and that you credit it with "extensive experiments" across "multiple benchmarks and LLM backbones." Below, we provide detailed responses to each of your comments.

Q1: The contributions are not clearly listed in the introduction. Role separation already appears in Reflexion, and CORAL already includes evaluator separation and workspace isolation; stricter permission enforcement alone does not justify novelty. (W1)

A1:

Thank you for this comment, which goes to the core of our paper. An introduction should present a paper's core insight and contributions clearly, and we have revised ours to do so (part (4)). We first state the insight itself, which differs in kind from both Reflexion and CORAL.

(1) Relation to Reflexion: a different problem.

Dividing an agent system into roles by responsibility is indeed common practice, and Reflexion is a clear example. Our work, however, studies a fundamentally different problem that begins where this division ends, and to our knowledge it is the first to do so: once the proposer is a coding agent, do the powers stay divided, and what does it cost when they do not?

  • The problem arises only with coding agents. Reflexion's Actor and Self-Reflection model are LLM calls, and a call sees only the prompt assembled for it. A coding agent has file and shell access: it can read the evaluator and every earlier candidate, and write to whatever memory the system keeps. How to use this autonomy as coding agents grow stronger is our topic, and it had no counterpart while proposers were single LLM calls.
  • Self-Reflection and Structured Governance carry different responsibilities. Self-Reflection turns the outcome of one trial into verbal advice for the next trial on the same task. Structured Governance (§4.2) decides, for every new task, which parent and inspiration candidates it builds on, how the evaluation budget is divided among competing lineages, which lessons are admitted to the shared notes, and what context the task receives. The two have in common only that they affect the next attempt.
  • The settings differ. Reflexion serves one agent retrying one task. An evolutionary search maintains a population of candidates under one finite budget, which requires these decisions, and its score must come from an evaluator the proposer cannot shape.

(2) Relation to CORAL: a different design philosophy.

You are right that stricter enforcement of permission boundaries alone would not justify a new paper. What separates EvoTrias from CORAL is the design philosophy itself. CORAL holds that coding agents are capable enough to run the search with full autonomy: in its own words, it "lets the agent decide what to explore and what knowledge to carry forward." Its agents therefore own govern as well as propose: each one sees every attempt, chooses what to build on and what to try next, and writes notes and skills into a memory shared by all. This is a natural idea, and later systems such as SwarmResearch also leave the search decisions to an agent.

EvoTrias does not follow this line. It asks a question this line leaves open: what does autonomy without boundaries cost? As agents grow stronger, the field is heading this way, yet to our knowledge no prior work has measured the cost. Our paper does, one decision at a time:

Autonomy overOutcomeEvidence
propose: read the repository, edit files, build, debuggain: tasks a single LLM call cannot take onOR-Tools, 680K lines of C++: +13.2% / +16.9% (§6.1)
govern: read the full history, choose the parent, write shared notescost: the search narrows early, the score drops, tokens growvs. CORAL-4: 64% lower token cost (§6.2); isolated in new §6.3: Signal Processing 0.760 → 0.639
judge: read the evaluator sourcecost: the score overfits the evaluator's fixed cases§6.4: +0.093 on the original score, +0.009 out of distribution

Even a highly capable coding agent should therefore not hold every decision: its autonomy pays off in propose and costs the search in govern and judge.

Where evaluator protection fits. CORAL's paper lists evaluator isolation among its execution safeguards (its Appendix C.7); it is not developed into a method, its effect is not studied, and in the released code the grader source is readable by every agent (our Appendix A.8). In EvoTrias, protecting judge is one of three parts of a single principle, and §6.4 measures what it is worth. The two systems therefore differ in the principle itself, the allocation of powers, and evaluator protection follows from it as one component.

AuthorsOpenReview ↗

Response - 1(2)

(3) Evidence: comparison with CORAL and a controlled ablation.

The submission already compares the two systems on the same host (Claude Code), model, and tasks:

  • Score. With one top-level agent, EvoTrias leads CORAL-4, which reports the best of four complete agents, on six of seven Math tasks (Table 1, GLM-5.2) and on Frontier-CS (median 80.67 against 67.14, §5.2).
  • Token cost. A run costs 64% less than CORAL-4's. CORAL's cost grows 3.9× from one agent to four; EvoTrias's varies by less than 5% from 1 to 8 executors (§6.2).
  • Matched tokens. With the same tokens, CORAL-1 and CORAL-4 both stay below EvoTrias (new Table 3).

The evaluator is closed to CORAL's agents in all these runs (Appendix A.8), so none of this lead comes from stricter evaluator protection.

EvoTrias and CORAL also differ in mechanisms beyond who owns the search. To isolate the effect of this allocation, we added a controlled ablation (new §6.3), as Reviewer jtqw suggested. Configuration A is EvoTrias. In configuration B, the coding agent that writes the candidates also makes the search decisions: it sees the full run history, chooses its own parent and inspirations, and writes notes that every later task reads. Everything else is identical: Claude Code with GLM-5.2, the construction instructions, 50 tasks, 4 initial directions, 4 concurrent executors, and the same hooks protecting the evaluator.

Owner of the searchScore ↑Tokens / runTokens / taskLineagesSmall-Δ
Circle Packing
A: Structured Governance2.6340 ± 0.002625.9M321K → 383K3.70%
B: coding agent2.6253 ± 0.012633.2M364K → 675K1.791%
Signal Processing
A: Structured Governance0.7597 ± 0.031727.6M368K → 418K4.00%
B: coding agent0.6391 ± 0.057932.8M355K → 605K1.089%

Three runs per row. Tokens / task: iterations 5–15 → 40–50. Lineages: root lineages used after the root phase. Small-Δ: notes from attempts within 2% of the parent's score.

Handing govern to the agent lowers the score on both tasks; on Signal Processing, A's lowest run (0.736) is above B's highest (0.689). It also brings the two costs that §1 attributes to agent-owned search. The search narrows: B keeps 1.0–1.7 of its four root lineages in use, against 3.7–4.0 for A, and 89–91% of B's shared notes come from attempts that moved the score by less than 2%. The cost grows: B's agent rereads a history that lengthens every iteration, so a run spends 19–28% more tokens. With all else fixed, evaluator protection included, the allocation of govern alone accounts for this gap. Reviewer jtqw has since confirmed that these follow-up experiments "have answered my main concerns."

In summary, the fundamental contribution and novelty of EvoTrias are as follows:

  • Relative to role-division work such as Reflexion, EvoTrias is, to our knowledge, the first to study what follows the division of roles: how each role keeps full autonomy inside its boundary while none crosses into another's, in a different setting (program evolution with a coding-agent proposer).
  • Relative to agent-centric work such as CORAL, EvoTrias asks nearly the opposite question. Coding agents extend what program evolution can take on, yet in a multi-round search their autonomy works best inside a bounded scope: the search stays broader and spends fewer tokens. Evaluator protection is one natural component of this design.
  • The experiments support this: one EvoTrias agent leads four-agent CORAL on six of seven Math tasks at 64% lower token cost, and controlled ablations confirm each boundary separately (§6.3 govern, §6.4 judge).

(4) The revised introduction. We fully agree that the introduction did not present these contributions clearly enough. Following your suggestion, the introduction of the revised submission now closes with an explicit list of contributions.

Revision. §1, closing paragraph. New §6.3 with Figure 7; the full table (Table 19) and the protocol are in Appendix C.11.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

For clarity, we refer to Reviewers zv2D, czSc, and jtqw as R1, R2, and R3, as in our responses.

We sincerely thank the AC and all reviewers for their time and constructive feedback. We are encouraged that the reviewers found the idea of separating powers "interesting" (R2) and, given "the recent progress in coding agents," a framework that manages them and makes them more efficient "certainly useful" (R3). They considered preventing access to the evaluator's source code "an important effort" (R2) and the evaluator-exposure study "one of the more convincing parts of the paper" (R3). All three reviewers recognized the strength of the evaluation: "This paper provides extensive experiments," with "extensive comparisons and ablation studies" across "multiple benchmarks and LLM backbones" (R1); a "comprehensive set of baselines and benchmarks" whose two models and three runs give "reliable results" (R2).

During the discussion, we added new experiments, clarified the core contribution, and revised the paper accordingly. We summarize these updates below, together with the reviewers' feedback on them.

1. New experiments

  • Controlled ownership ablation (new §6.3, Figure 7, Appendix C.11). Following R2 and R3, two configurations differ only in who makes the search decisions, with the same coding agent, model, protected evaluator, and budget. On Signal Processing, Structured Governance reaches 0.760 against 0.639 when the agent governs, and every Governance run scores above every agent-governed run. The agent-governed configuration also spends 19–28% more tokens per run and narrows to 1.0–1.7 of four root lineages (against 3.7–4.0). With the same agent proposing on both sides, this experiment also serves R1's stated aim of isolating the architecture from proposal capability.
  • Token-matched comparison (new Table 3). Following R2, each baseline receives the token budget of a 50-iteration EvoTrias run, which buys the controller-centric baselines 102–158 iterations. EvoTrias keeps the highest mean on both tasks, and both CORAL configurations stay below it.
  • OR-Tools baselines and held-out evaluation. Following R3, we added a baseline in which the same agent refines its own patch for 50 rounds (+4.6% / +2.2%, against +13.2% / +16.9% for EvoTrias; §6.1, Figure 5), and evaluated the evolved solver on all 100 unseen instances of the MiniZinc Challenge 2026 (+6.3%). For R1 and R3, we also quantified the scope of the task: a working patch depends on about 25K lines of code in 39 files (Appendix C.8).

Reviewer feedback. R3, who proposed the controlled ablation and the OR-Tools baselines, thanked us "for their extensive followup experiments" and found that they "have answered my main concerns." R2 wrote that the concern "about the fairness of the computational budgets has been largely resolved."

2. A clearer statement of the core contribution

Following R1's suggestion, the introduction now closes with an explicit list of contributions. EvoTrias studies a question that arises once the proposer is a coding agent: do the powers stay divided, and what does it cost when they do not? The paper answers it decision by decision: the agent's autonomy pays off in propose (+13.2% / +16.9% on the 680K-line OR-Tools, §6.1), lowers the score and raises the cost in govern (§6.2, §6.3), and inflates the score in judge (§6.4). Our responses also clarify how EvoTrias differs from Reflexion, CORAL, and Vesper.

Reviewer feedback. R1: "We now have a clearer understanding of the differences between EvoTrias and previous works such as Reflexion and CORAL."

3. Revisions to the paper

  • §1: the introduction closes with an explicit list of contributions.
  • §6.2: a new paragraph, "Matched token budget," with Table 3.
  • §6.3 (new): the controlled ownership ablation with Figure 7; Appendix C.11 gives its protocol and full results (Table 19).
  • §6.1 and Figure 5: comparison with the agent refining its own patch; Appendix C.8 adds the protocol of this baseline and statistics on the edit surface.

We are grateful that the reviewers recognized these efforts. R3 wrote, "I think they've done a great job in the rebuttal period," and R1, "we appreciate the authors' efforts in addressing our concerns"; both raised their ratings to 3. R2 likewise thanked us "for their detailed responses and additional experiments."

We are deeply grateful to the AC and all three reviewers for the time and care they devoted to our work. Their suggestions have made the paper substantially stronger.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.