- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MeEvo is an LLM-based automatic heuristic design framework that alternates population-based crossover/mutation of heuristic programs with reflection over recorded reasoning, objective values, execution errors, and the best program. Reflection produces a Meta Insight that guides generation of further heuristic programs. The method also uses problem analysis, error repair, and a linear crossover-probability schedule. The paper evaluates two LLM backbones on TSP-ACO, BPP-ACO, adaptive curriculum sequencing (ACS), wireless sensor network deployment (WSN), and constructive TSP. It reports favorable objective values and stability on the first four tasks, competitive constructive-TSP performance, and ablations of phase schedules and inherited support components.
Critical strengths
-
The framework is sound. Shared search records, reflection-to-code generation, and ordinary evolutionary operators form an implementable design, with detailed prompts, pseudocode, generated programs, accompanied by supplementary artifacts. This makes the work potentially useful to practitioners extending LLM-based heuristic search.
-
The evaluation is comprehensive. It spans five tasks and two backbones and includes single-phase, schedule, APA, and ED ablations. Table 1 reports improvements over MeLA on ACS and WSN with both backbones, and Appendix provides exact nonparametric tests rather than relying only on mean rankings. Several supplied result groups, including ACS under both backbones, agree with the reported means. These are promising signals, subject to the concerns below.
Critical weaknesses
-
The main evidence does not identify the claimed cyclic feedback mechanism. For BPP-ACO, ACS, and WSN, Table G4 gives N0=20, Npop=10, and L=50. With the default N=2, M=1, Algorithm 1 performs initialization (20), NE (10), NE (10), then ME (10) and terminates. No ME-generated heuristic participates in later NE crossover or mutation. The supplied artifacts likewise end after their first ME batch, with one additional expert seed evaluation. Consequently, these results support a terminal reflection stage but do not demonstrate the repeated NE->ME->NE reinforcement emphasized in the contribution. Table 3 changes phase allocation as well as ordering, while no control removes or shuffles trace history at matched resources. This matters because prior ReEvo already accumulates long-term reflection across iterations; the incremental benefit of full history and phase separation needs to be isolated, rather than inferred from the names of the phases.
-
The empirical comparison needs reconciliation before the reported advantages are reliable. In addition, the statement that traditional ACS/WSN baselines receive larger computational budgets compares one execution of a generated heuristic with one baseline run. Searching roughly 50 heuristics on the same task entails approximately 50,000 ACS or 250,000 WSN objective calls before repairs, versus the traditional budgets of 15,000 and 100,000. Design costs could be amortized over future instances, but that interpretation requires held-out deployment results. Finally, repeating the selected ACS/WSN heuristic 30 times measures restart stability, not transfer to unseen task instances. Together these limitations weaken the interpretation of the constrained-task results as a fair, generalizable AHD advantage.
Questions for authors
-
Mechanism clarification and targeted validation: Under the reported L=50 settings, where does any ME-generated candidate enter a subsequent NE round? Can you show phase-resolved results that distinguish repeated coupling from a single terminal reflection, with equal NE/ME candidate counts and comparable model-call/token budgets? TSP-ACO already has a longer trajectory that could help address this question.
-
Budget and generalization clarification: Do the ACS/WSN baseline rankings persist when all objective calls across heuristic search, seeds, and repairs are included, or when the learned heuristic is evaluated on genuinely held-out learners/materials and sensor layouts? Please distinguish search-time solution quality from deployment-time algorithm performance.
4 comments
Request for Reviewer Follow-up Before the Discussion Deadline
Dear Reviewer,
Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author–reviewer discussion period will end in about 1 day, we gently request that you read the rebuttal and check whether your concerns are addressed. Thanks again for your time and help in improving the quality of this paper.
Regards from the authors.
Thank you for the additional experiments. The revised schedule, allocation-matched control, trace ablation, and hybrid comparisons substantially address my main concern. Please clarify reflection-call matching and uncertainty in the matched control, and qualify the unequal-budget comparisons and limited ACS generalization. The new allocation-matched comparison provides evidence that this organization improves performance, and I am raising my initial rating accordingly.
However, I believe the technical novelty remains incremental: ReEvo already uses accumulated reflection to guide evolutionary search; MeEvo’s distinction is organizing these mechanisms into separate, cyclically coupled phases.
Response to Reviewer qLtP
Dear reviewer:
We thank you for taking the time to re-examine the paper and for raising the rating. The comparison with ReEvo is the right one to draw: both methods reflect, and the difference lies in what the reflection reads and what it produces.
1. MeEvo evolves the design prompt; ReEvo evolves code with prompt hints.
In ReEvo the evolutionary unit is heuristic code: short-term reflection is written for one parent pair and inserted into the crossover prompt, and long-term reflection compresses accumulated hints into a string inserted into the elitist-mutation prompt (Ye et al., 2024, §4). Reflection conditions an operator. Each generation it produces is anchored to an existing individual, a parent pair or the elite, and the only persistent artifact of reflection is the compressed hint string, which is never itself searched or evaluated. The search remains single-level, over code.
MeEvo adds a prompt level, and this is prompt evolution: what evolves is the design guidance behind the code, not the code itself. ME reads shared history (traces TH, fitness F, errors ERR, incumbent h*) and writes a Meta Insight stating what to change and why; that insight is the unit that improves from round to round. It is mapped to a batch of N_pop heuristics, generated from the insight rather than by recombining selected parents, and because the batch returns to the NE pool and its outcomes feed the next ME round, the prompt level accumulates across cycles.
ReEvo reflects on algorithms: its input is a pair of heuristics in code with their relative scores, and its accumulated object is the hint string those comparisons produce. ME reflects on how the algorithms were designed: its input is the shared history of reasoning traces, the fitness trajectory, error records and the incumbent, and its output is a Meta Insight mapped to a batch. Code is the object under comparison in ReEvo; in ME it is a reference point and the form the batch takes. ReEvo's accumulation therefore conditions an operator and is never evaluated in its own right, whereas the insight is what ME searches over, measured through the batch it produces. That batch enters NE recombination rather than remaining a variation of one parent. The matched control tests that return path, and the schedule comparison tests when it runs.
2. Matched control: call matching. The control pairs 4NE→4ME with cyclic (1,1) at N0=20, L=100: both use 20 initial + 40 NE + 40 ME evaluations and four ME reflection calls, with identical prompts, operators, backbone and five runs, so only the return path differs. One ME block is one reflector call followed by one batch of ten candidates; model calls are matched by construction.
| Arm (L=100, five runs) | TSP-ACO | BPP-ACO |
|---|---|---|
| 4NE→4ME (terminal) | 5.772 ± 0.04 | 204.44 ± 0.53 |
| Cyclic (1,1), matched | 5.747 ± 0.01 | 204.00 ± 0.92 |
Both means fall, as do the best values (5.738→5.733; 203.80→203.60). The gaps (0.025 and 0.44) do not exceed one SD in either arm (0.04/0.01; 0.53/0.92); the direction is the same on both tasks. NE-first also beats ME-first (5.772/204.44 vs 5.794/204.68), and the default (2,1) with a different allocation is lower still (5.716/203.28). The trace ablation isolates the shared-history input: removing traces while keeping code and fitness gives 5.873/205.28/625.83/62.43 vs 5.716/203.28/554.29/50.00.
3. Unequal budgets. Design and deployment costs are reported separately (§4.3): Table 6 gives tokens, program evaluations and wall-clock time, and Table G11 charges every objective call of five search runs, including repairs. Per algorithm, a generated heuristic is executed with 1,000 ACS or 5,000 WSN calls, whereas a classical baseline runs its full configuration, 15,000 or 100,000 calls (30×500; 100×1,000), so the per-algorithm evaluation budget is not larger for the heuristics we generate. In total, all AHD methods use L=100 primary evaluations; under the full count MeEvo uses 0.50M/2.60M calls versus 0.50M/2.50M for FunSearch, EoH and ReEvo and 0.54M/2.80M for MeLA. MeEvo attains the lowest objective on both tasks whether costs are counted per algorithm or in full, so the ranking is unchanged by the accounting choice.
4. ACS generalization. The ACS experiment uses the established 30-learner OULAD instance; its 30 executions measure search-time quality and restart stability, and the three 10-learner cohorts are disjoint subsets of the same learners. The ACS numbers are therefore search-time results. Held-out evaluation is reported for TSP-ACO and BPP-ACO (64 unseen instances at n=100/1000) and for WSN on five fresh layouts (25/25 at the lower bound).
References.
Ye, H., et al. (2024). ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution. NeurIPS 37, 43571–43608.
Qiu, Z., et al. (2026). MeLA: A metacognitive LLM-driven architecture for automatic heuristic design. Expert Systems with Applications 330, 133022.
Dear Reviewer:
We address the two points in turn.
1. Cyclic feedback and history isolation
ME is a separate prompt-level local-search step. It reads the shared history (traces, fitness, errors, best heuristic), forms a Meta Insight, and uses that insight to generate a batch of improved heuristics. It therefore improves the reasoning process that produces code. These heuristics then rejoin NE for crossover and mutation, bringing the local search gains back into global search.
Table G4 (P34) sets L=100 for all four AHD tasks; Table 3 (P9) uses the same budget. The default (2,1) schedule is:
Thus each main task completes two ME->NE cycles. The artifact contains complete logs for all five rounds, including every NE/ME batch, candidate, objective, error, and ME-to-NE handoff; it no longer stops after the first ME batch.
Updated Table 1 (P7) at L=100 over five runs:
MeEvo wins all eight cells; DeepSeek gains over MeLA are 1.92%/0.43%/2.47%/7.73% (TSP/BPP/ACS/WSN).
The controls below separate return timing from phase allocation (Tables 3-4, P9):
The two 40/40 rows use the same prompts, operators, N0, model calls, and token budget. At this fixed allocation, cyclic (1,1) lowers the TSP-ACO/BPP-ACO means from 5.772/204.44 to 5.747/204.00. The NE-heavy cyclic (2,1) lowers them further to 5.716/203.28. Thus, the best design is the 2:1 NE-heavy cyclic schedule: ME->NE returns help, and the larger NE share improves the balance between global search and local refinement.
Removing all accumulated traces raises the means to 5.873/205.28/625.83/62.43, i.e. 2.75%/0.98%/12.91%/24.86% above full MeEvo (Table 5, P10); the gap exceeds the SD on every task. MeEvo runs a separately budgeted ME batch and returns its output to NE. The two controls show distinct contributions from the ME->NE return and accumulated history.
2. Budget-inclusive ranking and deployment checks
At L=100, one ACS/WSN heuristic uses 1,000/5,000 objective calls. Objective columns: Table 1 (P7); call totals: Table G11a (P38), including all candidates and repairs across five runs. Table 6 (P10) gives per-run counts:
ReEvo (Ye et al., 2024) is the best other AHD method on ACS, and MCTS-AHD is the best on WSN. MeEvo uses fewer calls than MeLA and remains first. SCSO uses 6.7x/5.2x fewer calls, but its objective is 201.9%/734.0% above MeEvo. The ordering is unchanged when all search calls are counted (Table G11a, P38).
Table 1 is search-time quality. At deployment, the selected heuristic is fixed and run without retraining. On five fresh WSN layouts, all 25 runs reach the 50.00 lower bound (Table G11b, P38). Three disjoint 10-learner ACS cohorts give 145.26 ± 9.02, 148.47 ± 8.08, and 164.28 ± 8.71. ACS contains one 30-learner population, so this is a within-population robustness check. TSP/BPP-ACO provide external held-out evidence on 64 instances at n=100/1000 (Figure 3, P8). The 30 ACS/WSN runs execute the fixed best heuristic after search; they measure restart stability, not another search stage.
References.