- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes a new approach for how to find a best-performing self-improving agents when expanding different possibilities and keeping promising agents in an archive. This new approach is called Wright–Gödel Machine (WGM) and innovates on existing approaches in two main ways. First, WGM determines which agent to expand based on a combination of the EXP3 algorithm and a uniform selection probability. Second, for determining the final agent, WGM applies best-arm identification.
The paper proves mathematically that, under certain assumptions, WGM prevents the lineage of the most promising candidate to take over the selection probability. Moreover, the benefit of WGM is shown empirically, testing it against two competing approaches. These experiments show that WGM outperforms the competition, and an additional ablation study shows the benefit of each of the two innovations of WGM.
Critical strengths
-
Ablation study. This paper investigates the benefit of its two main innovations via an ablation study. Each one of them individually already improves the overall performance, and the combination is even more beneficial. This suggests that each building block has already its own merit.
-
Simple design. WGM follows rather simple ideas, which are, for the most part, clearly presented. This makes it easy to reason about the benefits of this approach.
Honorable mention. The paper provides a well-documented code repository.
Critical weaknesses
-
Multiple parameters. WGM comes with a fair amount of parameters. The empirical setup fixes all of them without elaborating the choices deeply. In particular, the choice of the exploration share , which seems to be the most crucial parameter, at least according to the theoretical results, is not stated anywhere in the paper, to the best of my knowledge. This makes it hard to assess how robust WGM is with respect to its parameter choices. The theoretical guarantees mostly make statements about the limit behavior, which is not too useful for practical applications.
-
Partially challenging presentation. The paper works with global variables, that are introduced in one place and then used throughout the paper. This makes retrieving vital information challenging, as if one forgets the meaning of one of the several variables, one needs to start searching for where it was defined. Moreover, certain pieces of information, such as the importance of Assumption 2 for the main result (Theorem 1) or additional experiment setup are only mentioned in the appendix, making the paper less self-contained.
Honorable mention. The code repository contains a license file that seems to violate the double-blind submission guidelines, as it appears to contain author-revealing information. This point is not that important to me, but I wanted to point it out and advise the authors to be more careful in the future.
Questions for authors
-
How do the authors intend to address my points listed as critical weaknesses? Please ignore the honorable mention for this question.
-
How important is the uniform selection floor for WGM? To me, it seems crucial for obtaining a sufficient diversity. Given that is a constant value throughout the run, it appears as if it has a great impact. Can the authors elaborate on how important the exponential scaling in this context really is?
-
How realistic is Assumption 2? The discussion in the appendix does not answer this question, in my opinion. Similarly, since Section B.4 shows how Assumption 2 can, in some sense, be replaced by Assumption 3, how realistic is this assumption?
-
Can the authors elaborate on the choice of the parameters? For example, all experiments are repeated three times. This sounds like a small number. Is this sufficient in order to obtain conclusive evidence for the benefit of WGM?
Further comments. The following comments are just a suggestion in order to improve the quality of the paper. They do not need to be addressed, and they did not affect my rating.
-
The table captions should mention what bold highlights mean. Interestingly, in Table 1, not all columns feature a bold entry.
-
The references have issues with the capitalization. Several words that should be in uppercase are not, and the casing of the titles is mixed. I recommend polishing the references.
Update During the Rebuttal
The authors have addressed my concerns well, so I increased my rating.
4 comments
Further comments: bold highlights in table captions, reference capitalization, and license file.
A: Thank you for the careful reading and the helpful suggestions.
- Bold entries. We bold only the best accuracy results. We have clarified this in the table captions and made the corresponding description in Line 377 more precise.
- Reference capitalization. Thank you for the reminder. The current references were exported directly from Google Scholar, which explains the inconsistent capitalization.
- License file. The license file is inherited from the open-source DGM codebase [2] and names the DGM authors, not ours, so it does not reveal our identities. We will be more careful in the future, and we have also reviewed our code and updated the anonymous repository.
We sincerely appreciate the time and effort you devoted to reviewing our paper, and your comments have helped us improve it considerably. If our responses address your concerns, we would be grateful for your consideration of a higher score.
Best,
The Authors
1.Wang et al. Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine. ICLR 2026.
2.Zhang et al. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. ICLR 2026.
3.Auer et al. The Nonstochastic Multiarmed Bandit Problem. SIAM Journal on Computing, 2002.
I thank the authors for their succinct and informative feedback as well as for their clarifications. Their response addresses all of my questions and concerns satisfactorily. My only remaining complaint is that I would have preferred it if the authors had highlighted the changes to the previous PDF clearly, as this makes it easier to check where an update occurred.
In any case, I am happy to increase my score. I will go with a weak accept because I am not too familiar with this research area in general.
Dear Reviewer LU9G,
Thank you for your positive feedback and for increasing your score. We are delighted that our responses have addressed your concerns. Following your suggestion, we have uploaded a revised PDF in which all changes are highlighted in blue. We sincerely appreciate your time and constructive comments, which helped improve our paper.
Best,
The Authors
Dear Reviewer LU9G,
Thank you for your thoughtful review. We appreciate the opportunity to clarify several points and address the concerns raised.
W1 & Q4: WGM has multiple parameters, and the exploration share γ is not stated. The theory mostly concerns limit behavior. Are three seeds sufficient?
A: Thank you for the catch. We did not tune hyperparameters for WGM. We use γ=0.1 in all experiments and have added it to Section 4.1(Line 402). All methods share the same initial agent, prompts, and backbone, adopted from the official DGM/HGM implementation, and the remaining settings are standard choices (z=1.96; ηt=γ/Nt as in EXP3 [3]). Due to limited time and budget, we vary γ on Polyglot with Sonnet 5 as the backbone and a budget of 200 task evaluations:
WGM remains effective across different values of γ. We would also like to clarify that our guarantees are not only asymptotic. Eq. B.4 holds at every step, and Proposition 2 gives finite-T bounds. The theory explains the mechanism, namely polynomial versus O(logT) expansions under Thompson sampling, and Figure 1(b) confirms it at a finite horizon.
On seeds, each run is costly, so many repetitions are infeasible. Three seeds already exceed common practice in this line of work. HGM [1] does not report repeated runs, and DGM [2] reports its main SWE-bench results without repeated runs and uses three runs only as a stability check on Polyglot.
W2 & Q1: Variables are defined once and reused throughout the paper. The role of Assumption 2 and parts of the setup appear only in the appendix.
A: Thank you for pointing this out. Our intention was to define the notation upfront so that the subsequent exposition would not be interrupted by definitions. We agree that this can make retrieval harder. In the revised version, we have restated the meaning of key symbols where they are used, such as the archive size N (Line 276) and Assumption 2.
Due to the page limit, we originally placed the detailed discussion of Assumption 2 and the additional experimental setup in the appendix. The main text already states that Theorem 1 relies on Assumption 2. In the revised version, we have moved the hyperparameters into Section 4.1 (Line 402).
Q2: How important are the uniform floor and the exponential scaling?
A: Thank you for the question. We agree that the uniform floor is crucial for diversity: our theory relies only on the floor (Eq. B.4). The exponential weights play a complementary role, steering the remaining 1−γ probability toward agents whose expansions produced better children. The sensitivity results in W1 show that a moderate γ performs best: a very small floor reduces diversity, while a large floor leaves less probability for the exponential weights. Moreover, replacing the exponential weights with Thompson sampling (w/o EXP3, Table 3) lowers accuracy from 54.7% to 51.1% on Polyglot. Thus, both components contribute.
Q3: How realistic are Assumptions 2 and 3?
A: Thank you for the question. We would first like to clarify that some condition on the archive is unavoidable, because L(T)=1 under any selection rule if the archive holds a single lineage (Lines 849–852). Both assumptions are mild in practice.
Theorem 1 uses Assumption 2 to show that WGM avoids homogenization, while Proposition 2 uses only Assumption 3, which places no condition on the archive or the rewards, to bound how fast reproduction can concentrate. The two results are therefore complementary.