- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces the Red Queen Gödel Machine (RQGM), a framework in which task agents and their evaluators co-evolve. Evaluation criteria remain fixed within each epoch, while evaluators can be replaced at checkpoints using externally labeled evidence. Records dependent on displaced evaluators are then removed from the search statistics. The paper studies this approach across coding, paper writing, mathematical proofs, and reward design.
Critical strengths
The research question is important, and the method directly addresses it. The paper brings evaluators into the self-improvement loop and carefully handles evidence that becomes outdated when evaluation criteria change. External anchors, frozen evaluators within epochs, and selective erasure form a clear and well-motivated framework.
The empirical study is broad and includes concrete mechanism analysis. Beyond comparisons with fixed-evaluator baselines, the paper provides component ablations, cross-reviewer evaluation, and search-trajectory analysis. In particular, the DeepSWE analysis connects reviewer decisions to actual patch test outcomes, examining both the ability to reject failing patches and the cost of rejecting correct ones.
Critical weaknesses
- The DeepSWE case study does not yet clearly connect reviewer improvements to coder improvements. Appendix E.4 shows that the reviewer that best distinguishes passing from failing patches emerged after the winning coder had already been generated. Earlier reviewers were generally permissive even toward failing patches. Consequently, the stronger final reviewer does not directly explain how the winning coder was discovered. While I acknowledge the overall performance gains, it remains unclear which evaluator updates provided useful search signals before the winning coder emerged and how those signals contributed to its improvement.
Questions for authors
- Could the authors trace the winning coder’s lineage and explain how preceding reviewer updates affected candidate selection or code modifications? This would clarify the connection between evaluator co-evolution and the final performance gains.
- Could the authors clarify the discrepancies between the paper and the supplied implementation? The paper-writing experiments specify GPT-5.5 with low reasoning effort, whereas the supplied configuration uses GPT-5.6 Luna with high reasoning effort. The paper also specifies fixed weights for the adversarial evaluator objective, whereas the implementation pools successes and trials across objectives before computing the score. Which configuration and selection rule produced the reported results? Please provide the corresponding code revision, configurations, and run records.
6 comments
Rebuttal Comment 2
Dear Reviewer 4rVo (continued)
Q2. Differences between the paper and the supplied implementation
Initially, we released the codebase to provide the reference code that was used to produce the results, where all settings were stated in the paper. Nevertheless, we have revised the supplementary material to explicitly include these configurations as runnable code. App. C.4 lists the settings of every run, the table below maps each result to its configuration. The supplement also includes the winning lineage of each reported run, as the per-node code changes from the root agent to each winner, with a script that rebuilds every winner from them. Runs that predate the released code also ship their original run configurations.
- Adversarial objective. The paper defines the score a reviewer is promoted on as . Here is its accuracy on anchor papers labelled accept, its accuracy on anchor papers labelled reject, and the share of the adversarial pool it rejects, where the pool holds AI-written papers that the previous reviewer accepted. The code does not apply these weights as coefficients. The scheduler samples the three kinds of item in the proportions of these weights, and
composite_beliefinrqgm/src/rqgm/search/belief.pyadds up their successes and trials, so the pooled score converges in expectation to . To make this more explicit in our work, the new paragraph “Pooled implementation” in App. F.6, in blue, states this and extends Prop. 1 to this setting. App. C.5 now also states which promotion rule each run used.
| Result | Supplementary config |
|---|---|
| DeepSWE RQGM, Fig. 1 and App. D.1 | deepswe-rqgm |
| DeepSWE HGM-H | deepswe-hgm, coder only |
| Paper writer and reviewer, Tab. 2 and Fig. 7 | paper-rqgm and paper-hgm |
| Proof prover and grader, Tab. 3 and Fig. 6, and ablations, App. D.2.2 | imo-rqgm and imo-hgm |
| Reward design, Tab. 1 | reward-design-curriculum and reward-design-static |
| Polyglot, Fig. 3, and paper ablations, App. D.2 | original run configurations; lineages for Polyglot |
| Proof writing on Claude Haiku 5.5, Tab. 5 and Figs. 8 to 9 | campaign-imo-haiku; two-stage: campaign-imo-haiku-hgm then tools/two_stage/stage2.py |
| DeepSWE on Claude Haiku 5.5, Tab. 4 and 7 and Fig. 10 | campaign-deepswe-haiku, campaign-deepswe-haiku-s0 and campaign-deepswe-haiku-hgm |
| Evaluator influence and anchor perturbations, Tab. 10 and 11 | scripts in analysis/ |
Thank you for the careful reading of both the paper and the code. If you find our responses satisfactory in addressing your concerns, we kindly ask you to consider raising your score.
The Authors
Thanks for the additional comments. And looking forward to a clearer repository with settings.
Rebuttal Comment 1
Dear Reviewer 4rVo
Thank you for reviewing both the paper and the provided code, and for recognising that "external anchors, frozen evaluators within epochs, and selective erasure form a clear and well-motivated framework". We address your points below.
W1 and Q1. Reviewer influence on the search
The reviewer improved the search through its anchor utility and through its code, which runs inside the coder. To show this effect more clearly, we replayed the 45 parent selections the DeepSWE search made before the winning coder appeared, removed each reviewer signal in turn, and recorded which parent the search would have chosen instead. The table below traces the lineage of the winning coder; Tab. 23 (App. E.4) adds the replayed selection probability of each step, and the new App. A.3, in blue, defines every quantity below.
e28cf32f1bf78e50, thenf076c908beec9b7, then5f52214337755f25f52214904d5a2a, the winner5f52214self_reviewtool and revises after a rejection. A change to the reviewer is therefore a change to the coder, and a better reviewer gives the coder better feedback. Overall, 2 of the 5 nodes in the lineage edited this functionality.Proof writing shows the same behaviour, across 4 runs on two models. We repeated the replay on Olympiad proof writing: the Nemotron 3 Ultra run of the proof ablations (Tab. 20), and the 3 Claude Haiku 5.5 seeds of Tab. 5. See Tab. 10.
Continued in part 2.