acceptodds
Under review as a conference paper at ICLR 2027

SM2-AHD: Search Experience Memory and Meta-Cognitive Reflection for LLM-Based Automatic Heuristic Design

Abstract

Automatic Heuristic Design (AHD) aims to construct high-quality heuristics with as little human intervention as possible; LLM-based AHD, which uses LLMs as code generators, has already produced heuristics that surpass hand-crafted ones through iterative search. Existing evolutionary methods, however, rely on a relatively fixed evolutionary framework and remain sensitive to the initial population and to the hyperparameters. We therefore propose SM2-AHD, a meta-level AHD framework in which the code population and the outer framework co-evolve throughout the search with two independent components. At the code level, Search Experience Memory (SEM) encodes each trajectory into a memory unit that carries its code lineage and failure attribution, and injects retrieved historical cases into the mutation prompt. At the level of the outer framework, Meta-Reflection periodically diagnoses verifiable search statistics and, under safety constraints, adjusts the exploration strength and the operator weights online. Experiments show that SM2-AHD outperforms several existing state-of-the-art methods across a range of heuristic design problems.

Primary area:
optimization
Keywords:
Combinatorial Optimization, Automatic Heuristic Design, Large Language Models, Agent Memory, Meta-Cognitive Reflection, Evolutionary Search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_JjinOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies LLM-based automatic heuristic design (AHD) by adding two components to MCTS-AHD. Search Experience Memory (SEM) records code lineage, edit differences, performance changes, and failure attributions, and retrieves historical cases from other search paths during mutation. Meta-Reflection uses search statistics, including stagnation, branch distribution, operator performance, and failure categories, to adjust exploration parameters, the novelty weight, and operator allocation online. The framework is evaluated with constructive and ant-colony solvers, covering five combinatorial optimization problems, six problem–solver combinations, and nineteen size configurations.

According to the means in Tables 1–3, the full method outperforms the four evaluated AHD baselines on sixteen configurations and performs slightly worse than MCTS-AHD on the remaining three. However, its central design substantially overlaps with existing experience-driven and adaptive evolutionary methods. The experiments do not establish an advantage over the most closely related work or contemporary program-evolution frameworks. The implementation contains differences, but their methodological significance and the paper's broad superiority claims are not adequately supported.

Critical strengths
  1. Concrete implementation details. The appendices specify the retrieval function, prompts, memory capacity, triggering conditions, parameter constraints, and token usage, making the system's operation reasonably clear.
  2. Improvements on some configurations. Several TSP and CVRP configurations improve over the selected AHD baselines, suggesting that the implementation warrants further investigation.
Critical weaknesses

W1. Limited novelty relative to closely related work

The central combination of experience memory and adaptive search control has substantial precedents: HiFo-Prompt couples an insight pool with search-state guidance [1], EvoPH co-evolves prompts and heuristics using execution experience [2], and RefineEvo combines parent–offspring reflection, positive and negative experience retrieval, and adaptive operator management [3]. ShinkaEvolve, EvoX, and AdaEvolve further demonstrate experience reuse or adaptation of the search process [4–6]. The remaining distinctions—identifier-based retrieval, lineage exclusion, failure categories, and bounded parameter updates—are plausible implementation choices, but the paper does not establish that they yield a substantive methodological advance, a new capability, or a generalizable insight.

W2. Missing comparisons with the most relevant recent methods

Tables 1–3 compare only against ReEvo, HSEvo, PathWise, and MCTS-AHD, omitting HiFo-Prompt and EvoPH despite citing them, as well as the directly relevant RefineEvo [1–3]. Comparisons with these methods are essential to establish the value of the proposed memory and control mechanisms. The evaluation should also include ShinkaEvolve [4] and a compatible adaptive-search baseline, such as EvoX or AdaEvolve [5,6], under a common model, evaluator, and resource budget. Without these comparisons, the reported gains do not establish an advance over the current field.

W3. The contributions of the proposed components are not isolated

Despite the claim that the host search's selection logic is unchanged, Equation (9) adds a novelty bonus to UCT, while Appendix C.3 introduces difference-aware mutation guidance. The ablations lack a control that retains both modifications while disabling SEM and Meta-Reflection, preventing clean attribution of improvements over MCTS-AHD. The additional use of deepseek-v4-pro for reflection introduces a further confound: matching the code-generation model and evaluation count does not match model capability or total inference expenditure. A common execution-layer control and matched-resource comparisons are needed to isolate the contributions of the two proposed components.

W4. The reported gains lack sufficient statistical support

Recomputing improvements against the best AHD baseline in each configuration yields a median relative gain of approximately 0.279%; 11 of 19 configurations improve by less than 1%, including three regressions. These calculations summarize reported means and do not establish statistical significance. Although some CVRP gains are appreciable, differences such as 202.698 versus 202.695 on BPP-ACO at N=500 are difficult to interpret without uncertainty estimates. Reporting only three-run means, without standard deviations, confidence intervals, or individual results, is insufficient to establish the robustness of the claimed advantage.

W5. Generalization and the reliability of reflection remain unverified

The manuscript does not clearly specify the sizes and separation of search, validation, and test sets, leaving test independence unresolved. Separate searches at different sizes do not establish cross-size generalization, and transfer beyond MCTS-AHD is untested. Failure attribution is also insufficiently validated: parent–child code and objective values alone provide limited evidence for diagnoses such as overfitting, while substring-based category parsing may propagate misclassification into global control. Success-only memory, unattributed failure feedback, and random-retrieval controls, together with comparisons against simple rule-based or bandit controllers, are needed to establish whether the proposed reflection mechanisms are reliable and necessary.

W6. Cost accounting and several central claims require correction

The reported 9.03% cost premium compares the two-model setup with pricing the same calls entirely using flash; it does not measure overhead relative to the baselines. Unequal evaluation budgets and modified population settings further complicate fairness. The claim of matching or outperforming baselines on all 19 configurations contradicts three lower-performing means, while component indispensability is undermined by CVRP-ACO at N=100, where removing Meta improves the objective from 14.679 to 14.055. Appendix E.2 also acknowledges prompts requesting lower objectives for maximization tasks. Complete resource accounting, corrected-prompt controls, and claims consistent with the reported evidence are required.

Questions for authors
  1. Novelty: Relative to HiFo-Prompt, EvoPH, and RefineEvo [1–3], which central mechanism is not already covered, and why is the distinction more than an implementation choice?
  2. Essential comparisons: Under the same model, evaluator, and total cost, does the method still outperform the directly related experience-driven methods [1–3], ShinkaEvolve [4], and EvoX or AdaEvolve [5,6]?
  3. Attribution: How much robust improvement does each component contribute when the common control retains novelty-enhanced UCT and difference-aware prompting but disables both SEM and Meta-Reflection?
  4. Controller necessity: Does LLM meta-reflection outperform simple rules or bandit control based on stagnation and operator rewards? Does categorical failure attribution improve over unclassified execution feedback?
  5. Reliability and generalization: Can the authors specify the search, validation, and test splits and provide individual run results, uncertainty estimates, search curves, and cross-size or out-of-distribution evaluations?
  6. Interpretation: What explains the deterioration caused by Meta on some CVRP configurations and the very small SEM gains on some TSP-ACO configurations? Do the conclusions hold after correcting the maximization-task prompts?
  7. Cost and claims: Can the authors report complete costs and runtimes relative to each baseline and revise the claims of 19/19 non-inferiority, component indispensability, and approximately 9% additional cost?

References

[1] Chentong Chen, Mengyuan Zhong, Ye Fan, Jialong Shi, and Jianyong Sun. HiFo-Prompt: Prompting with Hindsight and Foresight for LLM-based Automatic Heuristic Design. ICLR, 2026. arXiv:2508.13333.

[2] Yihong Liu, Junyi Li, Hongyu Lu, Xin Zhao, and Ji-Rong Wen. Experience-Driven Reflective Co-Evolution of Prompts and Heuristics for Autonomous Algorithm Design. Findings of ACL, pp. 4981–4995, 2026. DOI: 10.18653/v1/2026.findings-acl.245. (EvoPH.)

[3] Yang Wu, Junran Pan, Yifan Zhang, Ning Xu, Fanshuo Zeng, and Jian Cheng. RefineEvo: Planning-Guided Heuristic Evolution with Bidirectional Experience. ICML, PMLR 306:136582–136624, 2026.

[4] Robert Tjarko Lange, Yuki Imajuku, and Edoardo Cetin. ShinkaEvolve: Towards Open-Ended and Sample-Efficient Program Evolution. ICLR, 2026. arXiv:2509.19349.

[5] Shu Liu, Shubham Agarwal, Monishwaran Maheswaran, et al. EvoX: Meta-Evolution for Automated Discovery. arXiv preprint arXiv:2602.23413, 2026.

[6] Mert Cemri, Shubham Agrawal, Akshat Gupta, et al. AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization. arXiv preprint arXiv:2602.20133, 2026.

Reviewer_inmSOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SM2-AHD, a LLM-based AHD framework. Search Experience Memory (SEM) stores each search trajectory as a memory unit carrying its code lineage, edit differences, performance trace and failure attribution. At mutation time, SEM retrieves similar historical cases from the pool. It then injects the design highlights of successful cases and the lessons and reflections of failed ones into the mutation prompt, so that generation no longer depends on the parent code alone. Meta-Reflection is triggered periodically or upon stagnation. It reads the convergence trend, the coverage of search directions, operator performance and the failure-attribution distribution. An LLM then adjusts the exploration constant, the novelty weight and the operator biases, subject to a safety filter. Experiments cover 19 configurations of six problem classes, with four LLM-based AHD frameworks as baselines.

Critical strengths
  1. The motivation is clear. Two questions drive the paper: how to use historical search experience to guide the next code generation, and how to reallocate the exploration budget according to the overall search state. Both are of real practical significance.

  2. The paper gives concrete implementation details. These include the retrieval scoring rule, the verbatim prompts and output-parsing rules, and the per-model token usage. This makes the method transparent and easier to reproduce.

Critical weaknesses
  1. Limited novelty, and the differences from recent work are not sufficiently validated. Several recent works already combine historical experience accumulation with search state regulation, including HiFo-Prompt[1] and RefineEvo[2]. RefineEvo uses a Reflector to extract success experiences and failure lessons from parent-to-child evolutionary trajectories. It retrieves them according to their applicable context and injects them into subsequent code generation. A Planner analyzes the population state and historical operator performance to dynamically select and allocate evolutionary operators. This closely parallels SEM and Meta-Reflection. The paper discusses HiFo-Prompt and argues that condensing algorithmic insights into natural language loses information. However, HiFo-Prompt is not included in the main experiments, so this claim remains untested.

  2. The ablation does not isolate the two claimed contributions. The main text presents SEM and Meta-Reflection as the core contributions. However, SM2-AHD also uses two lightweight techniques that are described only in Appendix C: the novelty-enhanced UCT and the difference-aware mutation guidance. Appendix C explicitly notes that the ablation in Section 5.2 does not vary them. As a result, the ablation cannot show whether the improvement over the original baseline comes from SEM and Meta-Reflection.

[1] C. Chen et al. HiFo-Prompt: Prompting with hindsight and foresight for LLM-based automatic heuristic design. In International Conference on Learning Representations, 2026.

[2] Y. Wu et al. RefineEvo: Planning-guided heuristic evolution with bidirectional experience. In International Conference on Machine Learning, 2026.

Questions for authors
  1. Could the authors add to the ablation an MCTS-AHD baseline that uses the novelty-enhanced UCT and the difference-aware mutation guidance?

  2. The paper states that all methods use deepseek-flash for code generation. SM2-AHD, however, additionally uses the stronger deepseek-v4-pro for failure attribution and for the meta-analysis of Meta-Reflection. Could the authors report: SM2-AHD with deepseek-flash used throughout, and a comparison in which baselines with reflection mechanisms (e.g., ReEvo and HSEvo) use the same reflection model?

1 comment

I thank the authors for the clarifications and additional experiments. However, the response does not sufficiently resolve my concerns about novelty and attribution of the reported gains. I therefore maintain my original score.

Reviewer_CeWuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Paper proposes SM2-AHD, an extension of LLM-based automatic heuristic design on MCTS-AHD. The extension includes Search Experience Memory stores search histories, code edits, outcomes, and LLM-generated failure diagnoses. Meta-Reflection periodically examines search statistics and adjusts exploration parameters and operator weights under bounded updates. Experiments on six task solver settings for combinatorial optimization problems show that means outperform the four compared AHD methods on 16/19 configurations.

Critical strengths
  • The reported improvements span several distinct optimization settings, including constructive and ant-colony solvers and compare four AHD baselines. Some gains are practically meaningful.
  • The framework provides a concrete mechanism for reusing search experience and adapting search behavior. Retrieved edits connect historical outcomes to code generation, while constrained parameter updates provide an implementable form of online search control.
Critical weaknesses
  • The paper does not demonstrate whether its different memory representations provide a meaningful advantage over HiFo-Prompt, which already combines accumulated design knowledge with adaptive search guidance. The use of a stronger model for reflection also makes it unclear how much of the improvement comes from the proposed method versus the model itself.
  • The evaluation does not establish how reliable the reported gains are. Despite author(s) report results are averaged over three runs but without standard deviations or individual-run results. The manuscript also leaves the search/test split, instance counts, and control of stochastic ACO evaluation unclear. These make it difficult to assess whether the improvements are consistent and generalize beyond the instances used during search.
Questions for authors
  1. How does SM2-AHD compare with HiFo-Prompt on compatible tasks under the same model and budget?
  2. How many instances are used during search and final testing, and are these are held-out test sets?
  3. Can you report solution quality alongside total tokens, monetary cost, and elapsed time for each method? Do the gains persist when reflection uses the same model as code generation?
  4. How should the “outperforms or matches on 19/19” claim be interpreted given the lower reported performance on KP-cons (N=50,100) and BPP-ACO (N=1000)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.