- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper tries to make LLM-based evolutionary search less blind. Instead of just asking a fixed LLM to mutate programs, it adds a graph representation of the program, an RL controller to choose the type of mutation, and online fine-tuning so the LLM can adapt during the search.
Critical strengths
-
The high-level motivation is interesting: LLM-guided evolutionary search does suffer from weak parent/mutation selection and static mutation operators.
-
Introducing the GNN to compare the structural differences of two parent nodes with two similar scores seems reasonable.
Critical weaknesses
-
The comparison between Table 1 and Table 2 is not meaningful as presented. Table 1 uses the high-budget PACEvolve setting with 1000 iterations and 10 runs, whereas Table 2 uses only 150 iterations and 3 runs. The backbone models are also inconsistent: for example, PACEvolve is reported with Gemini, while OpenEvolve benefits substantially from Qwen.
-
I am not convinced about the usage of SAC here because an explicit probabilistic model could be composed by using the reward and the graph produced by the GNN. This would make the mutation decision less computationally demanding and more explainable.
-
Table 3 is not enough to justify the full method.The ablation in the table is incomplete because it does not the ablations of the reward-shaping terms, despite these being the central part of the proposed method.
Questions for authors
- Table~3 says that it was done over few runs how many are a few ?
- In Table~2 a comparison with PACEvolve single based on gemini was conducted ?
- Have you compared with PACEvolve Multi with Qwen and what is the conclusion of these comparisons?