acceptodds
Under review as a conference paper at ICLR 2027

Gödel Ecology: Productivity-Guided Continuous Co-Evolution of Agents and Evaluators for Collective Recursive Self-Improvement

Abstract

Self-improving coding agents rely on evolutionary search over their own implementations, with evaluators determining which variants survive. Recent work has begun to co-evolve agents and evaluators, but evaluator updates remain periodic and anchored to fixed criteria. We introduce Gödel Ecology, a framework of collective recursive self-improvement via productivity-guided continuous co-evolution of self-improving agents and evaluators. Evaluators are scored jointly with Co-Clade Metaproductivity of the agent lineages they induce, and a continuously updated soft mixture of evaluator candidates forms an adaptive evaluation landscape. Delayed credit assignment and adversarial interventions are additionally adopted to counter self-reinforcing evaluator drift with reliable external evidence. Together, these interacting populations form an evolving ecology in which agents and evaluation criteria actively interplay and shape one another. Across experiments on coding, math, and writing domains, Gödel Ecology produces stronger evaluator signals, better downstream search decisions, and improvements that transfer across benchmarks and model backbones. These results suggest that continuously adapting the evaluation criteria that determine which directions of improvement are worth pursuing helps make better recursive self-improvement.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
recursive self-improvement, coding agent, co-evolution, adaptive evaluation, agentic system, meta learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UMkEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Gödel Ecology (GE), a framework that jointly evolves agents and evaluators for recursive self-improvement. Its main idea is to score evaluators not only by how well they match external judgments, but also by whether the agent lineages they influence become more productive, while continuously updating a mixture of evaluators.

Critical strengths

The paper identifies an important problem in self-improving agents, and the empirical evaluation is reasonably broad. The paper tests GE in closed-loop coding-agent evolution, offline evaluator comparisons, non-code revision tasks, and cross-model transfer.

Critical weaknesses
  1. About reproducibility of closed-loop results. The end-to-end coding co-evolution experiment includes only three search runs. This provides limited evidence that the observed gains are robust across search trajectories.

  2. About stress testing against evaluator exploitation. Although the paper introduces several safeguards against evaluator drift, it does not clearly report systematic adversarial tests in which agents deliberately exploit evaluator weaknesses. Their effectiveness against active reward hacking therefore remains uncertain. This is my biggest concern.

  3. About human auditing of evaluator evolution. The paper does not clearly describe systematic human inspection of intermediate evaluator judgments. Aggregate metrics may miss systematic misjudgments, evolving biases, or cases where agents exploit weaknesses in evaluation criteria.

Questions for authors
  1. The end-to-end coding co-evolution experiment uses only three search runs and evaluates the final improvement on 17 tasks. How sensitive are the reported gains to random seeds, search trajectories, and task selection? Can the authors provide results across more independent runs and a larger evaluation set?

  2. evaluation hacking: How do the authors verify that agents improve genuine task-solving capabilities rather than exploiting weaknesses in the evolving evaluators? Have systematic adversarial tests been conducted in which agents deliberately optimize for evaluator weaknesses, and how effective are the proposed safeguards under such conditions?

  3. Do human experts inspect intermediate evaluator judgments or changes in evaluation criteria during co-evolution? If not, how do the authors detect systematic misjudgments, evolving biases, or exploitable evaluation criteria that may not be reflected in aggregate metrics?

Reviewer_4iD3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors propose Gödel Ecology, a framework that co-evolves agents and evaluators in RSI. They assign credit to evaluators according to the downstream productivity of agent lineages. The goal is to help evaluators select agents with greater potential for future improvement. The authors also introduce a continuously updated mixture of evaluators to guide the co-evolution.

Critical strengths

The motivation of this paper is clear and interesting. The authors point out that even with similar accuracy, evaluators may lead agent evolution in different directions. For example, some evaluators may be better at selecting agents with greater potential for future improvement, even when their evaluation accuracy is not higher than that of others. This provides an interesting insight into RSI.

Critical weaknesses
  1. The experiments in this paper do not fully support the authors' main claim. The main claim of this paper is that the proposed Gödel Ecology method could evolve better evaluators that can select agents with greater potential for future improvement. However, the strongest experimental results in this paper are improvements in evaluator AUC. These experiments are conducted on a fixed set of code patches. The paper itself points out that evaluators with similar accuracy may lead to different evolutionary outcomes. Therefore, higher evaluator AUC does not necessarily mean better evolutionary search. The main claim of the paper should be supported by the results of the closed-loop evolution experiments. However, each method is only tested in one complete search run, and the final agents are evaluated on a very small set of only 17 tasks. According to the results, GE solves 14 out of 17 tasks, only one task more than the initial agent and RQGM (CRAVE). These results are not sufficient to support the main claim. Apart from that, GE's evaluators can access test files while RQGM's can't, and the authors show that providing test files could greatly improves evaluator AUC. This introduces another confounder when comparing the methods. I'm wondering whether the better performance is due to this setting difference. Could it also lead to data leakage?

  2. The authors propose a method with quite a lot of components, but they do not conduct sufficient ablation studies. The method has several components, including the continuous evaluator mixture, delayed credit assignment, adversarial interventions, and others. However, the authors do not conduct ablation studies on these components or explain how each component contributes to the performance. For example, why is the Co-CMP productivity term necessary? It is one of the most important components of the method but the authors do not ablate it. The lack of ablation studies may leave readers unsure about why the proposed method works and which mechanisms are responsible for its performance.

Questions for authors

See weaknesses.

Reviewer_HV4HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a framework for recursive self improvement called the Goedel Ecology (GE). In this framework, an agent and an evaluator scaffold evolve in parallel using a common archive. The GE framework uses a mixture of evaluators which are updated with new evaluators. The weights of the evaluators are based on both external anchor fidelity and Co-Clade Metaproductivity (Co-CMP), where the latter is a delayed lineage-based signal that rewards evaluators who have preserved lineages that turn out to be productive. To ensure the stability of the framework, they also propose a number of auxiliary components such as mixture regularization, significance gated updates, stale score handling, probationary interventions, a verifiable admission floor, and value of information scheduling. They evaluate the proposed framework against RQGM on Polyglot coding tasks. Additionally, they provide several additional evaluations including offline evaluation of evaluator update, writer revision loop on non-coding tasks, and cross model transfer experiments.

Critical strengths

The authors raise an interesting problem with the use of evaluators in evolutionary search, namely that the choices of the evaluator can impact which intermediate candidates are selected, making it difficult to assess how well a particular choice of evaluator will work in the future.
 I think the distinction between external anchor fidelity and delayed lineage productivity is interesting. A soft mixture of evaluators is a plausible alternative instead of replacing evaluators at checkpoints.

The authors make some good points about the potential problems with using a coupled agent-evaluator system (e.g., evaluator drift, stale scores, feedback loops), as well as the issues with immutable/verifiable constraints in coding scenarios.

This paper has quite an ambitious set of goals, ranging from offline updates to closed-loop coding agent search, to non-code judgement, late evaluator adoption, and cross-backbone transfer. It also includes quite detailed descriptions of the system, and seems to follow a clear story line.

Critical weaknesses

This work compares two methods (GE and RQGM) for evaluating the quality of code patches. But the experiments are confounded by the fact that GE has access to the test file while RQGM does not. It seems that GE performs better than RQGM because of this advantage, not just because of the proposed co-evolution method.

The core experiments are based on a very small number of complete search runs, with few verified improvements and evaluated on only 17 tasks. Uncertainty estimates over different search runs would help to evaluate the robustness of the reported improvements.

The proposed framework includes many techniques such as mixing evaluators, Co-CMP, delayed credit assignment, regularisation, intervention procedures etc., but there are no systematic ablation studies to identify which of these contributes most to the improvement.

The Co-CMP mechanism does not seem to clearly specify how to distribute the credit when multiple evaluators jointly influence the decision, leading to double counting and possibly rewarding some evaluators who contribute little (because their contribution is mostly due to stronger evaluators in the mixture). Only delayed reward does not guarantee causal contribution.

Some implementation details appear missing: threshold for influencing evaluator, credit distribution, handling of stale scores, prompt templates, code, evaluator pool and full search run.

Questions for authors

Can you share a complete, well-controlled comparison between GE and RQGM using the same sets of evaluator candidates, prompts, task context (including access to test files), anchor data, calibration budget, gate, scheduler, compute budget, and task order, but with different evaluator update mechanisms?

Could you run the main closed-loop experiments from the paper multiple times over independent end-to-end searches, accounting for stochastic evaluator generation, meta-agent behavior, and agent execution, and report the distribution of final performance, accepted candidates, evaluator quality, and compute cost? This would demonstrate the robustness of your results.

Finally, could you conduct systematic closed-loop ablation studies that explore the impact of single-evaluator replacement vs. continuous evaluator mixtures, anchor-only vs. anchor-plus-Co-CMP weights, delayed vs. non-delayed credit assignment, and the influence of interventions and jump gates? Could you conduct experiments where one judge is left out at random (or alternatively, you could leave out a subset of judges) and see if the resulting credits match up with the causal impact of each judge's judgement?

What are the full prompts used for each task? How did the authors construct the task contexts? Who were the judges considered in each case? What were the initial seeds in the search? How was hyperparameter selection performed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.