acceptodds
Under review as a conference paper at ICLR 2027

GAEolve: Graph-Augmented Evolutionary LLMs for Scientific Discovery via Reinforcement Optimization

Abstract

Evolutionary program search guided by Large Language Models (LLMs) has emerged as a powerful paradigm for automated scientific discovery. However, current approaches are fundamentally constrained by three bottlenecks: structurally blind parent selection, sparse whole-program evaluation rewards, and static mutation operators that fail to adapt during search. We present GAEvolve (Graph-Augmented Evolution), a framework that resolves these limitations through a tightly coupled, three-pillar architecture. First, a relational graph neural network (GNN) parses programs into typed computation graphs, producing structure-aware embeddings. Second, an RL-optimized meta-controller leverages these embeddings to replace blind evolutionary sampling with a directed policy, dynamically selecting optimal parents and mutation directions based on reward history. Third, an online GRPO fine-tuning loop continuously updates the LLM mutation operator at test-time using group-normalized evaluation rewards, directly aligning the model's generation distribution with high-fitness structural edits. We evaluate GAEvolve on challenging scientific discovery tasks. By transforming stochastic search into a directed, self-improving trajectory, GAEvolve efficiently discovers closed-form physical equations, consistently matching or outperforming static LLM-driven baselines and achieving state-of-the-art out-of-distribution performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Test-Time Training, Evolutionary Algorithms, Reinforcement Learning, AI for Science

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_nsajOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper tries to make LLM-based evolutionary search less blind. Instead of just asking a fixed LLM to mutate programs, it adds a graph representation of the program, an RL controller to choose the type of mutation, and online fine-tuning so the LLM can adapt during the search.

Critical strengths
  • The high-level motivation is interesting: LLM-guided evolutionary search does suffer from weak parent/mutation selection and static mutation operators.

  • Introducing the GNN to compare the structural differences of two parent nodes with two similar scores seems reasonable.

Critical weaknesses
  • The comparison between Table 1 and Table 2 is not meaningful as presented. Table 1 uses the high-budget PACEvolve setting with 1000 iterations and 10 runs, whereas Table 2 uses only 150 iterations and 3 runs. The backbone models are also inconsistent: for example, PACEvolve is reported with Gemini, while OpenEvolve benefits substantially from Qwen.

  • I am not convinced about the usage of SAC here because an explicit probabilistic model could be composed by using the reward and the graph produced by the GNN. This would make the mutation decision less computationally demanding and more explainable.

  • Table 3 is not enough to justify the full method.The ablation in the table is incomplete because it does not the ablations of the reward-shaping terms, despite these being the central part of the proposed method.

Questions for authors
  • Table~3 says that it was done over few runs how many are a few ?
  • In Table~2 a comparison with PACEvolve single based on gemini was conducted ?
  • Have you compared with PACEvolve Multi with Qwen and what is the conclusion of these comparisons?
Reviewer_cPviOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper is on long-horizon search for automated scientific discovery. The proposed method has three components: (1) rational GNNs for structure-aware embeddings; (2) RL-optimized meta-controller for parent sampling; and (3) online GRPO fine-tuning loop for mutation operator. The evaluation is on sybolic regression task with LLM-SRBench.

Critical strengths

The paper aims to address or alleviate the code evolution challenges in reward sparsity, parent selection, and test-time training. The current experimental results demonstrate sota performance on SR-Synth subset of LLM-SR Bench.

Critical weaknesses
  • The evaluation is limited on synthetic symbolic regression task. There are a lot of open-ended discovery tasks in openEvolve and ShinkaEvolve that should be evaluated.
  • Also, current evaluation on symbolic regression lacks the qualitative analysis on the researched formula.
  • It is also important to clarify which baselines has the test-time learning module like GAEvolve. If no such baseline, please include some of then for fair comparison.
Questions for authors

please see the weakness part.

Reviewer_UK6EOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper augments LLM-driven program evolution with a relational GNN encoder of program graphs, a discrete SAC controller that chooses mutation-type hints for the LLM, and online GRPO fine-tuning of the LLM.

Critical strengths
  1. The paper raises interesting questions about LLM-driven program evolution.
Critical weaknesses
  1. The problems raised in the introduction are not solved, and the text contradicts the method. For example, the abstract says the controller selects parents, but parents are sampled uniformly (L245–246).

  2. The experiments are rough and the gains are not significant. Results cover only one single problem, and the gain over OpenEvolve with the same backbone is within run-to-run variation

  3. The method requires access to model weights. Online GRPO fine-tuning rules out API-based models, and only one open-weight model is evaluated.

  4. Missing citations: [1–8]. The paper uses SAC, PPO, GRPO and a relational GCN without citing them [1–4], and lists LLM-SR in Table 1 without a reference [5]. The experiments should also include a comparison with [6-8].

  5. The manuscript is incomplete. Section 4.3 still contains a note addressed to the authors (L460: "State the LLM, iteration budget and ..."), and none of this information is given anywhere in the paper,.

Overall, the manuscript is not well prepared. I recommend rejection and encourage the authors to revise the paper thoroughly before resubmitting it to a future venue.

[1] Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. ICML 2018.

[2] Proximal Policy Optimization Algorithms. arXiv 2017.

[3] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv 2024.

[4] Modeling Relational Data with Graph Convolutional Networks. ESWC 2018.

[5] LLM-SR: Scientific Equation Discovery via Programming with Large Language Models. ICLR 2025.

[6] DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution. ICML 2026.

[7] Symbolic Regression with a Learned Concept Library. NeurIPS 2024.

[8] LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery. ICML 2024.

Questions for authors
  1. In Eq. 9, every child’s reward is shifted by the same parent score RαR_\alpha before group normalization, so the shift cancels and the advantage equals that of standard GRPO. What role is the parent-relative baseline meant to play?
Reviewer_GwwtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GAEvolve combines a program-graph encoder, a discrete SAC mutation-hint controller, and online GRPO-style LLM updates. Experiments on nonlinear-oscillator symbolic regression and molecular-property architecture search report improvements over selected evolutionary baselines and include SAC/GRPO ablations.

Critical strengths
  1. Adapting both mutation guidance and the generating model is a promising direction for program search.

  2. The paper reports multi-run symbolic-regression results, candidate-validation procedures, and discovered expressions, providing concrete evidence beyond headline scores

Critical weaknesses
  1. Equal iterations do not establish equal compute: GAEvolve generates eight candidates per iteration, versus one for the described PACEvolve configuration, and additionally updates the LLM and GNN. Table 2 also lacks a matched-Qwen PACEvolve comparison. The claimed efficiency advantage therefore lacks a budget-matched comparison.

  2. The learning objective is internally inconsistent. Eq. (8) and the experimental description include KL regularization, but p. 6 and Table 6 say KL contributes no gradient. The old-policy/reference-policy distinction is unclear. Moreover, the common parent reward cancels exactly under Eq. (9)'s group normalization, undermining its stated baseline role.

Questions for authors
  1. Assessment-changing evidence: Does the gain persist with the same backbone and matched evaluation counts and total computation? Please report valid candidates, tokens, and GPU time.

  2. Clarification: What loss is actually implemented, how are old and reference policies updated, and does KL receive gradients? Please reconcile the parent-baseline claim with Eq. (9).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.