- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper applies AlphaEvolve to discover game-playing search algorithms by co-evolving shared search mechanisms and game-specific heuristics across multiple games. The search mechanisms modify components such as tree selection, value backups, and root action selection. Experiments across training, held-out, procedurally synthesized, and neural-policy/value settings show improvements over established MCTS baselines in several settings, while performance with frozen PPO networks is statistically comparable to tuned PUCT.
Critical strengths
The paper provides extensive empirical evaluation of the discovered search mechanisms. It covers held-out human-designed games, synthesized games, and frozen neural policy/value networks, with comparisons against MCTS baselines under both time and simulation budgets. Evaluating five independent evolutionary runs also helps assess the consistency of the results. The transfer across games provides useful evidence for automated search-code discovery beyond a single domain.
Critical weaknesses
-
I find the methodological novelty limited. The paper uses AlphaEvolve to evolve MCTS-style search code. Separating shared search mechanisms from game-specific knowledge is a useful design, but the contribution appears primarily to be an application of an existing code-evolution framework to general game search.
-
The additional benefit with learned policy/value networks is unclear. In the PPO evaluation, the evolved mechanisms do not significantly improve over tuned PUCT. This limits the demonstrated practical advantage in an important setting for strong game-playing systems.
Questions for authors
Could the authors clarify what they consider the main methodological contribution beyond applying AlphaEvolve to MCTS-style search? In particular, what does the proposed factorized co-evolution add beyond evolving a shared search mechanism across multiple games?
Thank you for your review
We thank Reviewer Ze1M for the assessment, for noting the breadth of the empirical evaluation across 5 independent runs, and for two questions that go to the core of the paper.
W1 and Q1: what factorized co-evolution adds
We agree that the code-mutation engine is AlphaEvolve. Standard AlphaEvolve evolves a single program against a fixed evaluator; our contribution is structuring multi-game search discovery as a multi-agent system—one
MechanismCoordinatoragent evolving a shared mechanism archiveA_mechacross 50 games, and 50 per-gameKnowledgeWorkeragents evolving heuristic archivesA_know(g)(Algorithm 2).The coordinator and workers communicate through shared archives with decoupled credit: when a worker generates a verified heuristic
κ'(Appendix B.1), it samples a mechanismm ~ A_mech, evaluates<m, κ'>against fixed reference PUCT<m_ref, κ'>and the standalone priorπ_κ', and reports headroom-normalizedAmpandImpback to the coordinator's running cross-game score ofm, paced by a completion quorum (Section 3.3, Appendix B.2).On Q1 (what co-evolving knowledge adds over evolving a shared mechanism alone under fixed default heuristics):
sqrt(1 − π(a))in first-play urgency are constant across actions and receive no selection signal. Co-evolved priors and value estimators give the coordinator informative inputs during evolution; four of the five runs discovered prior-dependent FPU schedules (revised Table 20; supplementary code).Hpuct ≤ 0.05 ΔU), dropping those games fromAmp. Promoting the co-evolved champion(m*_e, κ*_{g,e})to opponent re-expands headroom and restores valid selection coverage inconnect_four,gomokuandhexfrom 0% to 85–91%.m_refwhile ranking mechanisms by marginal headroom gain (Amp,Imp) overm_refon the same heuristic prevents a strong heuristic from masking a weak mechanism.Generalization beyond MCTS and games. Because the coordinator–worker protocol only requires factoring an algorithm into a domain-agnostic procedural core
mand task-specific heuristicsκ_g, the same generalist–specialist multi-agent meta-learning architecture transfers directly to other algorithmic domains—such as combinatorial optimization and SAT/MILP solvers (co-evolving a general branching/restart policymalongside problem-family scoring heuristicsκ_g), automated theorem proving (co-evolving proof-search controlmwith domain tactic priorsκ_g), and classical or robotic planning (co-evolving graph/tree searchmwith task-specific cost-to-go heuristicsκ_g)—using the same invariant-referenceAmp/Impcredit assignment and quorum-paced shared archives.W2: benefit with learned policy and value networks
We agree with your reading. On the 17-game neural tier the evolved mechanisms are at statistical parity with tuned PUCT, and we have updated Section 4.3, Section 6, and Appendix K.4 in the revised PDF to state this directly.
What the tier does show: