- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses offspring infeasibility in LLM-driven algorithm evolution. Built on MCTS-AHD, RevivEvo integrates Parent-Specific Offspring Generation, Type-Aware Bounded Repair, and Failure and Repair Memory. Experiments on 14 tasks from CO-Bench and HeuriGym show improvements in Test Avg. Score and the final candidate feasibility rate. Component ablations, analysis of parent-specific generation, and lineage analysis further examine the roles of repair and experience reuse.
Critical strengths
-
The paper is well motivated and addresses offspring infeasibility, a practical bottleneck in LLM-driven algorithm evolution.
-
The experiments across 14 tasks support the method’s effectiveness in improving candidate feasibility and final solver quality, while the ablation studies provide evidence for the contributions of its main components.
Critical weaknesses
Limited evaluation across candidate budgets. The main experiments use a 20-candidate budget, including initialization. Although Figure 6(a) extends selected baselines to 80 candidates, it covers only seven tasks with single runs and does not evaluate RevivEvo at larger budgets. Larger-budget comparisons across multiple random seeds would clarify whether its advantage persists.
Questions for authors
- Table 3 supports the usefulness of both memory components. What evidence specifically supports separating Lineage-Specific Failure Memory and Retrieved Repair-Skill Memory, rather than using a single memory for both generation and repair with the same historical information and similar input lengths?
- For Figure 6(b), how are input, output, and reasoning tokens counted, and are all LLM calls used to compare programs and update memories included?
- Could the authors report, for each method, the proportion of test instances on which the final selected solver is feasible, the proportion of final selected solvers that are strictly feasible on the test set, and how often no strictly feasible candidate is found on the development set, requiring selection of the candidate with the highest partial score?
- Unclear sensitivity to the LLM backbone. RevivEvo is evaluated only with DeepSeek V4 Flash. The direct LLM synthesis baselines do not assess its sensitivity to the LLM backbone. Evaluation with additional backbones would clarify whether the reported gains depend on the chosen model.
4 comments
4. Response to Q3
Q3: Could the authors report, for each method, the proportion of test instances on which the final selected solver is feasible, the proportion of final selected solvers that are strictly feasible on the test set, and how often no strictly feasible candidate is found on the development set, requiring selection of the candidate with the highest partial score?
Thank you for this suggestion. We computed the following statistics from the main-experiment records. The three columns correspond directly to your questions:
- Test-instance feasibility: for each task–seed run, we compute "the proportion of test instances on which the final selected solver is feasible", then average these proportions equally across the 14 tasks and three seeds.
- Strict test feasibility: "the proportion of final selected solvers that are strictly feasible on the test set," requiring each solver to pass every test instance.
- No feasible dev candidate: "how often no strictly feasible candidate is found on the development set." This is the proportion of runs that end without finding any candidate passing all development instances; lower is better. Such runs require selection by the highest partial score.
The latter two columns report counts and percentages over 42 runs per method (14 tasks × 3 random seeds).
| Method | Test-instance feasibility ↑ | Strict test feasibility ↑ | No feasible dev candidate ↓ |
|---|---|---|---|
| FunSearch | 86.59% | 21/42 (50.00%) | 15/42 (35.71%) |
| EoH | 84.02% | 21/42 (50.00%) | 16/42 (38.10%) |
| ReEvo | 85.33% | 19/42 (45.24%) | 10/42 (23.81%) |
| HSEvo | 82.91% | 20/42 (47.62%) | 17/42 (40.48%) |
| MCTS-AHD | 65.26% | 12/42 (28.57%) | 24/42 (57.14%) |
| HiFo-Prompt | 80.13% | 20/42 (47.62%) | 17/42 (40.48%) |
| RefineEvo | 81.24% | 17/42 (40.48%) | 14/42 (33.33%) |
| A2DEPT | 81.18% | 18/42 (42.86%) | 20/42 (47.62%) |
| RevivEvo | 99.11% | 34/42 (80.95%) | 0/42 (0.00%) |
RevivEvo achieves 99.11% test-instance feasibility, compared with at most 86.59% among the listed baselines. In addition, 34 of its 42 selected solvers (80.95%) pass every test instance, compared with at most 21 of 42 (50.00%) for the baselines. All 42 RevivEvo runs find a strictly feasible development candidate.
5. Response to Q4: Sensitivity to the LLM Backbone
Q4: Unclear sensitivity to the LLM backbone. RevivEvo is evaluated only with DeepSeek V4 Flash. The direct LLM synthesis baselines do not assess its sensitivity to the LLM backbone. Evaluation with additional backbones would clarify whether the reported gains depend on the chosen model.
Thank you for this suggestion. We added experiments using Qwen 3.7 Flash and Doubao Seed 2.1 Turbo. Each backbone was evaluated independently on all 14 tasks using seed 2024 only, without repeated runs. The table reports the macro-averaged Test Avg. Score, as reported in Appendix B.3:
| LLM configuration | Test Avg. Score ↑ |
|---|---|
| DeepSeek V4 Flash | 0.8238 |
| Qwen 3.7 Flash | 0.8024 |
| Doubao Seed 2.1 Turbo | 0.8503 |
RevivEvo achieves macro-averaged scores of 0.8238 with DeepSeek V4 Flash, 0.8024 with Qwen 3.7 Flash, and 0.8503 with Doubao Seed 2.1 Turbo. These results show that RevivEvo maintains strong performance across all three tested LLM backbones, supporting the effectiveness of our pipeline beyond DeepSeek V4 Flash.
We again thank the reviewer for the careful evaluation and constructive feedback. If our responses have addressed your concerns, we would greatly appreciate your reconsideration of the score.
Dear Reviewer j4m9,
Thank you again for your time and consideration. As the ICLR discussion period is approaching its conclusion, we would greatly appreciate any feedback on whether our rebuttal has addressed your concerns.
If any concerns remain, we would be happy to provide further clarification. If our responses have adequately addressed your concerns, we would sincerely appreciate your reconsideration of the score.
Thank you again for your time.
Best regards,
The Authors
Good reply
Thank you for the additional experiments and clarifications. These have addressed most of my concerns, and I will maintain my score
1. Response to W1
Thank you for this suggestion. We have so far added a single-run evaluation of RevivEvo with a target budget of 80 candidates on the same seven tasks: aircraft landing, capacitated warehouse location, open-shop scheduling, periodic vehicle routing, operator scheduling, crew pairing, and pedigree. We compare against the 80-candidate results of EoH, ReEvo, and RefineEvo reported in Figure 6(a). Following Appendix A.3, the final program is selected using development performance and evaluated separately on the test set; the reported Test Avg. Score gives equal weight to the seven tasks. Due to the computational cost, repeated-seed experiments at this larger budget have not yet been completed; we plan to add them in follow-up experiments.
RevivEvo achieves a Test Avg. Score of 0.8971 versus 0.8108 for ReEvo, the strongest listed baseline, a gain of 0.0863. Candidate feasibility is also higher (93.21% versus 52.14%). The advantage is also observed in this single-run larger-budget evaluation. These results are reported in Appendix B.2.
2. Response to Q1
Thank you for this question. We maintain separate memories because they capture experience from different stages of the process, are accessed at different stages, and retain different types of information. Failure memory is populated with errors revealed when an offspring fails evaluation and guides subsequent generation along its lineage. In contrast, repair-skill memory is built from completed repair attempts and their validation outcomes, and is retrieved when repairing a later offspring with a similar failure. The two memories also differ in content: failure memory focuses on what failed and what to avoid, whereas repair-skill memory focuses on how the failure was addressed and whether the repair succeeded, including corrective actions and validation outcomes (Section 2.4 and Appendix C.3).
For example, an e-graph extraction offspring introduced an SCC-based selection mechanism that produced cyclic dependencies in the extracted subgraph, causing it to fail the acyclicity check during development evaluation. Repair restored level-based acyclic selection to resolve this constraint violation. This episode provided two forms of experience: failure memory recorded a warning against cycle-blind selection, which informed a later proposal to incorporate explicit cycle checks; repair-skill memory recorded the corrective procedure and its successful validation on six development instances for retrieval during later repairs.
3. Response to Q2
For Figure 6(b), we count input, output, and reasoning tokens from all LLM calls, including candidate generation, program comparison and selection, feedback analysis, failure diagnosis, code repair, and memory updates. All these calls are included in the total token count.