acceptodds
Under review as a conference paper at ICLR 2027

AlgoEvo: Self-Evolving Agentic Search for Automated Algorithm Discovery

Abstract

Large language models have advanced automated algorithm discovery by synthesizing executable code, but existing frameworks trap them in rigid search pipelines with pre-defined control flows. This limitation restricts adaptive reasoning, blocks cross-paradigm transfer, and overlooks richer execution feedback. We propose AlgoEvo, a unified agentic framework that transforms automated algorithm discovery into an interactive, knowledge-accumulating process. An autonomous agent dynamically inspects, diagnoses, and edits code based on runtime feedback. A design skill hub decouples paradigm-specific knowledge from the core discovery engine, allowing a unified workflow to seamlessly handle single-heuristic, multi-objective, and multi-component design. Meanwhile, a hierarchical experience bank organizes search trajectories into a task-level tree to guide exploration and consolidates cross-task patterns into reusable skills. Across six representative benchmark tasks, AlgoEvo reaches state-of-the-art performance with as little as 7% of the evaluation budget and reduced token consumption, demonstrating strong intra-task accumulation, cross-task transfer, and the ability to reproduce or exceed the strongest existing methods through flexible skill activation.

Primary area:
optimization
Keywords:
Automated Algorithm Discovery, Multi-Objective Optimization, Multi-component design, Combinatorial Optimization, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_94BBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AlgoEvo, a unified agentic framework for automated algorithm discovery that replaces predefined evolutionary search pipelines with an adaptive, LLM-driven search process. The framework integrates three components: (1) an agentic search loop that dynamically selects code inspection, diagnosis, editing, and evaluation actions; (2) a design skill hub that abstracts paradigm-specific knowledge and enables a shared discovery engine to handle single-heuristic, multi-objective, and multi-component optimization; and (3) a hierarchical experience bank that organizes search trajectories into experience cards and task-level trees, with cross-task consolidation into reusable skills.

Experiments across six combinatorial optimization tasks demonstrate competitive or superior solution quality compared with specialized baselines, including EoH, ReEvo, FunSearch, MEoH, MOTIF, and E2OC. The paper reports substantial reductions in candidate evaluations and token consumption, alongside analyses of progressive optimization, cross-task knowledge transfer, and methodological skill reuse.

Critical strengths
  1. Unified and flexible framework. AlgoEvo integrates agentic search, design skills, and hierarchical experience memory into a unified framework supporting three algorithm-design paradigms. The separation of paradigm-specific knowledge from search execution is a valuable architectural contribution.

  2. Strong empirical performance and sample efficiency. Table 1 demonstrates competitive or superior performance across six benchmarks, with substantial reductions in candidate evaluations (as few as 7% of the baseline budget) and LLM token consumption.

  3. Promising experience accumulation and transfer. Tables 2–4 provide preliminary evidence that accumulated experience improves successive optimization runs and supports cross-task transfer. Section 4.4 further demonstrates the flexibility of reusing methodological knowledge from existing algorithms.

Critical weaknesses
  1. Insufficient isolation of core contributions. Existing ablations do not directly compare adaptive agentic search with a fixed search loop under identical tools and resources. Moreover, removing the experience bank or skill hub causes relatively small performance degradation (Appendix F.3), leaving their individual contributions unclear.

  2. Incomplete computational efficiency evaluation. The reported evaluation savings exclude diagnostic operations, which may incur substantial computational overhead. Without end-to-end runtime or total execution cost comparisons, the practical efficiency gains remain uncertain.

  3. Limited evidence for self-evolution. The progressive optimization experiments (Tables 2–3) reuse both previous best-performing code and accumulated experience, making their respective contributions difficult to distinguish. Cross-task transfer is evaluated on only two related task pairs (Table 4).

Questions for authors
  1. Adaptive search: Can the authors compare AlgoEvo against a fixed search loop using the same LLM, design skills, diagnostic tools, and evaluation budget to isolate the benefits of autonomous action selection?

  2. Computational efficiency: Could the authors report end-to-end runtime and total computational cost, including diagnostic operations, to substantiate the efficiency claims?

  3. Experience accumulation: Can the authors separate the effects of code-only transfer, experience-only transfer, and their combination under matched budgets?

  4. Cross-task evolution: In Table 4, does transfer involve retrieved experience cards, evolved design skills, or both? How do their individual contributions affect performance?

8 comments
AuthorsOpenReview ↗

Response to Reviewer 94BB: Contribution Isolation, End-to-End Cost, Self-Evolution and Cross-Task Transfer

We sincerely thank the reviewer 94BB for the constructive and detailed assessment. Your comments on contribution isolation, efficiency accounting and self-evolution have directly improved the evidence we present, and we are grateful for the time and care behind them.

We added a fixed-schedule control sharing every resource with the adaptive loop (Table 14), an end-to-end cost study with diagnostic overhead (Table 15), a five-variant ablation on two tasks (Table 10, Fig. 12), and a third transfer pair (Table 4).

1. (W1, Q1) Isolation: Fixed Search Loop

The control layer is isolated in Table 14. The fixed-schedule control shares the backbone, design skills, experience bank, starting code, evaluation entry, budget of 500 and action set, and alternates diagnosis probes and edits; only the decision differs. The adaptive loop reaches 8.860 / 8.999 (train / held-out) with 271 evaluations, 7530 s and 139.2M tokens; the fixed schedule 9.180 / 9.136 with the full 500, 10759 s and 152.1M. The adaptive loop wins on quality and on cost at once: 0.137 better held-out, 229 fewer evaluations, 3,229 s less wall-clock and 12.9M fewer tokens.

The knowledge layer is isolated in Table 10. Five variants, one mechanism each, run on CVRP-DR and Bi-TSP; CVRP-DR converges within the allowance (variants within 0.09), while on the unsaturated Bi-TSP they separate: autonomous diagnosis −5.41%, experience bank −4.11%, design skill hub −2.50%, situation-triggered injection −1.08%, all three seeds in the same direction (Fig. 12).

2. (W2, Q2) End-to-End Cost

Table 15 splits one TSP design run into TevalT_{\mathrm{eval}} and TsearchT_{\mathrm{search}}, with diagnostic operations inside TsearchT_{\mathrm{search}} rather than excluded. A complete design takes 5.78 min against 21.57–34.25 min, with 1.80 min of evaluation and 3.98 min of search, 2.9M tokens against 3.2–4.3M.

3. (W3) Self-Evolution Evidence

Tables 2–3 do not reuse the code of a previous round. Every seed and every round starts from the same template, and improved code is distilled into a card as a rule and retrieved as context. Warm starting moves TSP from 4.961 to 4.808 at 48→40 evaluations, and multi-round accumulation moves 4.8928 to 4.7995 (−1.91%) with Bi-FJSP HV rising from 0.1549 to 0.1862 (+20.21%).

Cross-task transfer now covers three pairs: CVRP→CVRP-DR 10.314→9.816 (−4.8%), TSP→Bi-TSP 336.5→352.2 (+4.7%) and FJSP 4-Ops→Bi-FJSP 0.911→0.914 (+0.33%). The percentage varies with problem difficulty and with the scale the evaluator reports on: a target near its achievable HV has little headroom, and a coarse objective moves in larger steps. The direction is what this comparison establishes, and it holds in all three pairs, including the one that crosses a paradigm boundary (FJSP 4-Ops → Bi-FJSP), a transfer that methods designed for a single problem class cannot attempt.

4. (Q3) Code Lives in Experience: No Code-Only Transfer

The design decision is not code versus experience; it is which rule applies here. The experience bank stores the situation, the modification as a rule, the reference implementation that realizes it and the outcome. At the next run the agent reads the bank and rewrites the program from the template, so code always travels inside the bank and never as a starting program. That is what separates AlgoEvo from continual-evolution systems such as E2OC, which carry the previous program forward as the new starting point. With a single channel, the natural test is to close it: the without-bank variant of Table 10 costs −4.11% on Bi-TSP and +0.66% on CVRP-DR, and Tables 2–3 give the controlled readings of the same channel.

5. (Q4) Only Experience Cards Transfer

Only the experience bank transfers; the skills are identical in both variants. The bank stores the situation, the modification, the rule and the outcome, so the two variants of Table 4 differ only in the bank they can retrieve from, while the target keeps the same template, skill files and budget.

The bank is tested at three levels: successive runs (Table 2, 4.961→4.808 at 48→40 evaluations; Table 3, 4.8928→4.7995 with HV 0.1549→0.1862), across problems (Table 4), and by removing it (Table 10, −4.11% on Bi-TSP and +0.66% on CVRP-DR). The skill is tested by removing the hub (Table 10, −2.50% and +0.58%) and by agents without it: general coding agents without a skill hub or bank reach only 9.03–10.50 (Table 13), and the weakest backbones produce no valid submission (Table 11). The control layer is isolated in Table 14. Skill consolidation needs at least μ\mu experience trees and is not enabled here, so Table 4 measures the bank alone.

All revisions are marked in red, and we hope they address your concerns. We note with appreciation the time and care behind your comments. Should you have any further questions or suggestions, we would be glad to discuss them.

AuthorsOpenReview ↗

Overview of the detailed content for Reviewer 94BB

Thank you for your review. The three points you raise, namely whether the core contributions are isolated, whether the efficiency evaluation is complete, and whether the evidence for self-evolution is sufficient, are exactly the questions we needed to answer head-on. Your remark that no direct comparison with a fixed search loop was provided led directly to the resource-matched control in the revision. The three responses follow these three points, with the table data included. If you think other controls are needed, we are glad to add them.

We believe this work has practical value for automated algorithm design, and we hope its positioning can be understood accurately.

If these clarifications resolve your concerns, we would be very grateful if you would reconsider the rating.

The three points are answered in the three responses below.

AuthorsOpenReview ↗

Detailed Response to Reviewer 94BB on W1 and Q1 (1/3): Isolating the Control and Knowledge Layers

W1: isolation of the core contributions. Insufficient isolation of core contributions. Existing ablations do not directly compare adaptive agentic search with a fixed search loop under identical tools and resources. Moreover, removing the experience bank or skill hub causes relatively small performance degradation (Appendix F.3), leaving their individual contributions unclear.

Q1: comparison against a fixed search loop. Adaptive search: Can the authors compare AlgoEvo against a fixed search loop using the same LLM, design skills, diagnostic tools, and evaluation budget to isolate the benefits of autonomous action selection?

Response

Thank you for your feedback. This comment addresses the isolation of core contributions across two key levels: comparing the autonomous control layer against a fixed loop, and separating the component-wise contributions of the knowledge layer in ablation studies. We have validated both aspects through newly added control experiments in the revision, detailed as follows:

1. Control-Layer Isolation: Adaptive Search vs. Fixed-Schedule Loop (Table 14) To isolate the performance gains brought by autonomous action selection, we established a rigorous iso-resource control experiment in Table 14:

  • Alignment Conditions: The fixed-schedule controller and the adaptive search loop share the exact same backbones, skill files, experience bank, initial templates, evaluation entry points, 500-evaluation budget, and action set. The sole difference lies in the action scheduling mechanism: the fixed controller strictly alternates between "diagnostic probing" and "code editing" in a predetermined order and submits every edit, whereas the adaptive loop decides the next action from the runtime state.
  • Experimental Results: Adaptive search achieves a cost of 8.999 on the held-out test set, outperforming the fixed schedule's 9.136. Concurrently, adaptive search consumes only 271 evaluations, 7,530s of wall-clock time, and 139.2M tokens, whereas the fixed schedule exhausts the full 500 evaluations, taking 10,759s and consuming 152.1M tokens.

Adaptive control outperforms the fixed loop across quality, evaluation count, and computational resources simultaneously. This rules out the explanation that "adaptiveness merely equates to early stopping," proving the efficacy of dynamically selecting actions according to search states.

Table 14 (Appendix F.5)

ControlTrainHeld-outEvalsT_tot (s)Tokens
Adaptive8.8608.9992717,530139.2M
Fixed schedule9.1809.13650010,759152.1M

2. Knowledge-Layer Contribution Separation: Task Saturation and Unsaturated Ablations (Table 10) In Appendix F.3 of the original draft, the performance degradation upon removing the experience bank or skill hub was relatively small (e.g., 0.66% and 0.58% on CVRP-DR). This is because CVRP-DR fully converges within the 500-evaluation budget: final metrics for all five ablation variants cluster between 9.040 and 9.122 (a narrow range of 0.09), compressing inter-mechanism differences within the measurement precision of the convergence plateau.

To prevent saturation effects from masking mechanistic differences, we re-ran this ablation suite under identical starting points and budgets on the unsaturated Bi-TSP task (Table 10 and Figure 12):

  • Full framework reaches 0.8566;
  • w/o autonomous diagnosis drops to 0.8103 (−5.41%);
  • w/o experience bank drops to 0.8214 (−4.11%);
  • w/o design skill hub drops to 0.8352 (−2.50%);
  • w/o contextual trigger injection drops to 0.8473 (−1.08%).

All four removals yield consistent negative degradations, with identical directional trends across all three random seeds.

Table 10 (Appendix F.3)

VariantCVRP-DRBi-TSP (Δ)
Complete9.0400.8566
w/o diagnosis9.122 (+0.91%)0.8103 (−5.41%)
w/o bank9.100 (+0.66%)0.8214 (−4.11%)
w/o hub9.092 (+0.58%)0.8352 (−2.50%)
w/o injection9.059 (+0.21%)0.8473 (−1.08%)

These two sets of experiments clearly delineate the sources of our core contributions from the control and knowledge layers respectively: Table 14 quantifies the dual advantages in quality and budget conferred by autonomous action scheduling (reducing held-out cost by 0.137 and saving 229 evaluations); Table 10 reveals the contribution hierarchy among components on an unsaturated task, where removing autonomous diagnosis incurs the largest drop (5.41% > 4.11% > 2.50%).

This substantiates our methodological positioning that "core control resides in runtime state" and justifies presenting it alongside saturated tasks. We have added both controls (Table 14, Appendix F.5; Table 10, Appendix F.3) and marked them in red. We are grateful for these suggestions.

Reviewer_HNdKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AlgoEvo, which replaces the fixed evolutionary pipelines of LLM-based algorithm design with an autonomous agent that reads, edits, diagnoses and evaluates code using a design skill hub and a hierarchical experience bank composed of MCTS-style "experience card" trees. On six tasks across three settings: single-heuristic, multi-objective, and multi-component, AlgoEvo claims state-of-the-art quality with as little as 7% of the evaluation budget against paradigm-specific baselines.

Critical strengths
  • The central abstraction is clean: it's a single engine that covers single-heuristic, multi-objective and multi-component design. Moreover, the evaluation is competently executed with three seeds, a shared protocol with identical splits, and explicit precautions against reward hacking.
  • The implementation and evaluation cover a wide range of settings.
Critical weaknesses
  • The full framework scores 9.040 with and 9.100 ± 0.132 without the experience bank. Without the skill hub, the score is 9.092 ± 0.153. My concern is that these ablation effects are quite small relative to noise. These differences of 0.05 - 0.06 are below one standard deviation of the ablated variants with n=3.
  • In Table 11, the MEoH, NSGA-II and MOEA/D skills all reach 0.828–0.829 HV on Bi-TSP. Plain AlgoEvo already reaches 0.827 in Table 1. This, to, me seems like only weak evidence of transfer.
Questions for authors
  • Did the Table 1 runs begin with an empty bank and un-evolved skills, or with a set of initial skills? Can the authors show the exact skill files active for each Table 1 task?
  • Can the authors add a plain coding-agent baseline using the same backbone, evaluation entry point and budget, but without the skill hub or bank? Appendix A (Q3) asserts that generic coding agents fail on multi-objective and multi-component tasks, but no experiment currently supports this.
9 comments
AuthorsOpenReview ↗

Response to Reviewer HNdK: Ablation on an Unsaturated Task, General versus Method-Specific Skills, and a Coding-Agent Baseline

We sincerely thank the reviewer HNdK for the constructive and rigorous assessment. Your comments on the ablation design and on the transfer evidence have directly improved how we present both, and we are grateful for the time and care behind them.

We added a five-variant ablation on two tasks (Table 10, Fig. 12), a comparison of general and method-specific skills (Table 16), the initial state of Table 1 (Sec. 4.1), and a plain coding-agent baseline (Table 13).

1. (W1) Effect vs Noise

We extended the ablation to a second, unsaturated task, where the mechanisms can be separated. CVRP-DR converges inside the 500-evaluation allowance and its endpoint is saturated; Bi-TSP is not, and the same five variants were run there under the same starting template, the same allowance and the same removals. On Bi-TSP the mechanisms separate: autonomous diagnosis costs −5.41% HV, the experience bank −4.11% and the design skill hub −2.50%, with all three seeds in the same direction (Fig. 12). The seed spread also narrows from ±0.0215 without the bank and ±0.0339 without autonomous diagnosis to ±0.0032 for the complete framework.

2. (W2) Table 1 vs Table 16

The two numbers come from different skills. Table 1 activates the general paradigm skill, which conveys how to complete the design task in that paradigm (Appendix G.1). Table 16 builds on it and adds method-specific skills that carry the implementation detail of MEoH, NSGA-II and MOEA/D, and in several cases these are refinements of the general skill rather than a different procedure. With a skill written in natural language alone, the engine matches methods that their authors implemented as code. Comparable HV is therefore what a working skill interface produces: the general skill already reaches the level these methods attain with their own frameworks. Against 500 evaluations for the native frameworks, the skill versions need 19–22 on TSP and reach better quality (EoH 7.1529→7.0414, ReEvo 7.3478→7.1889, FunSearch 7.6111→7.1815); on Bi-TSP the count stays at 284–391 because archive maintenance itself consumes evaluations.

The point extends to scope: Bi-TSP is one of the design problems the same engine handles, and one skill interface lets both a general and an algorithm-specific skill drive it, whereas each of these methods targets a single problem class.

3. (Q1) Initial State

Each run starts with an initial skill and an empty experience bank and tree, so each row of Table 1 is an independent result. The skill is provided at activation and is what the loop operates on: with the weakest backbones no valid submission is produced at all (Table 11). The active skill per task is single-heuristic constructive for TSP, capacity-aware constructive for CVRP, multi-objective for Bi-TSP and Bi-FJSP, multi-component for CVRP-DR, and multi-component with the four-operator specification for FJSP 4-Ops. No skill is evolved during a Table 1 run; evolution is triggered only across tasks. All skills appear in the appendix and in the source (Sec. 3.2, Sec. 4.4, App. G.1), and their final versions will be uploaded with the public code release.

4. (Q2) Coding-Agent Baseline

Table 13 provides that baseline: Codex, Claude Code and OpenCode on CVRP-DR with the same backbone, evaluation entry and budget, capped at 60 evaluations with the guard relaxed, and without a skill hub or bank, covering nine cells, three agents on three backbones. AlgoEvo reaches the lowest cost in all nine (8.43, 9.04 and 9.13, against 9.03, 9.19 and 9.44 for the best agent; lower is better). Since the agents share the starting code, entry and budget and differ only by the absence of the hub and the bank, the comparison isolates where the gain comes from.

Part of a larger set: a five-variant ablation on two tasks with a variant that removes autonomous diagnosis (Table 10, Fig. 12), a comparison across six backbones on both tasks (Tables 11–12), and an end-to-end cost study (Table 15), where a complete design takes 5.78 min against 21.57–34.25 min for the baselines. App. A's statement about generic coding agents now has direct support. Across the nine cells no agent reaches 9.03–9.35, and two runs stop at 3 and 5 evaluations. Model capability is the binding constraint: on the weakest backbones no valid submission is produced (Table 11), so instruction-following is a precondition for any agentic baseline.

All revisions are marked in red, and we hope they address your concerns. We note with appreciation the time and care behind your comments. Should you have any further questions or suggestions, we would be glad to discuss them.

AuthorsOpenReview ↗

Overview of the detailed content for Reviewer HNdK

Thank you for taking the time to review this work, and for grounding your concerns in specific numbers: the relation between the 0.05–0.06 differences and the standard deviations, the transfer evidence in Table 11, the initial state of the Table 1 runs, and the missing plain coding-agent baseline. These are the questions that make a paper more solid. We have organized one response per concern and attached the relevant tables and per-seed readings so that the numbers can be checked directly. Where more detail would help, we are happy to add experiments or revise the wording.

We have put substantial effort into this work, and we hope it provides a reusable framework for automated algorithm design.

If these additions address the open questions, we would be grateful for your further consideration and remain happy to discuss any remaining point. Thank you again for your time.

The four concerns are answered in the four responses below.

AuthorsOpenReview ↗

Detailed Response to Reviewer HNdK on W1 (1/4): Ablation Effects Relative to Noise

W1: the ablation effects relative to noise. The full framework scores 9.040 with and 9.100 ± 0.132 without the experience bank. Without the skill hub, the score is 9.092 ± 0.153. My concern is that these ablation effects are quite small relative to noise. These differences of 0.05 - 0.06 are below one standard deviation of the ablated variants with n=3.

Response

Thank you for your astute observation regarding statistical significance and noise boundaries in our ablation studies. The arithmetic you pointed out holds true on CVRP-DR, primarily because the task fully converges within the 500-evaluation budget: final metrics across all five variants cluster tightly between 9.040 and 9.122 (a narrow range of 0.09). At this saturated plateau, minor differences of 0.05–0.06 indeed fall within measurement precision and convergence noise, independent of the underlying strength of individual mechanisms.

To evaluate mechanism utility in a setting with higher discriminative power, we re-ran the same ablation suite under identical starting templates and budgets on the unsaturated Bi-TSP task (Table 10):

  • Full system reaches 0.8566;
  • w/o autonomous diagnosis drops to 0.8103 (−5.41%);
  • w/o experience bank drops to 0.8214 (−4.11%);
  • w/o design skill hub drops to 0.8352 (−2.50%);
  • w/o contextual trigger injection drops to 0.8473 (−1.08%).

Table 10 (Appendix F.3)

VariantCVRP-DRBi-TSP (Δ)
Complete9.0400.8566
w/o diagnosis9.122 (+0.91%)0.8103 (−5.41%)
w/o bank9.100 (+0.66%)0.8214 (−4.11%)
w/o hub9.092 (+0.58%)0.8352 (−2.50%)
w/o injection9.059 (+0.21%)0.8473 (−1.08%)

All four ablations yield consistent negative degradations, with identical directional shifts across all three random seeds (Figure 12).

Furthermore, cross-seed standard deviations reveal an important stability effect: the full system exhibits a cross-seed standard deviation of only 0.0032, whereas removing the experience bank and autonomous diagnosis expands the variance to 0.0215 and 0.0339, respectively. This demonstrates that both mechanisms not only elevate mean performance but also suppress cross-seed fluctuations, substantially stabilizing the search process.

Regarding sample size and statistical claims, given n=3n=3, we explicitly restrict our claims to directional consistency without making overstated significance assertions. We have also removed the ambiguous term "primary drivers" from the revised manuscript, replacing it with an objective statement: on converged, saturated tasks, mechanistic differences fall below noise levels; on unsaturated tasks, mechanism utilities and their relative hierarchy are clearly separable.

We have rerun the ablation on Bi-TSP (Table 10, Appendix F.3) and sharpened the contribution statement in §1; both are marked in red. We are grateful for this suggestion, which led us to state where each mechanism is observable.

Reviewer_Ygt1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AlgoEvo combines interactive coding, reusable design skills, and hierarchical experience memory for algorithm discovery across six tasks and three design paradigms. It reports improved solution quality, evaluation efficiency, and knowledge reuse.

Critical strengths
  1. A single engine handles single-heuristic, multi-objective, and multi-component design. Table 1 demonstrates promising quality, including held-out improvements on single-heuristic and multi-component tasks.
  2. Method skills reuse established search knowledge outside its original framework (Section 4.4; Appendix G.1). Table 4 also provides a direct with/without transferred-experience comparison, supporting some cross-task utility.
Critical weaknesses
  1. Methodological novelty beyond reflective search and experience reuse is insufficiently established. The distinctive challenge is diagnosis-driven search control and component coordination, not memory accumulation alone. Tables 2–3 confound experience with inherited code and continued search. Table 10 tests one task, does not isolate the diagnostic mechanism, and shows only 0.66%/0.58% mean degradation without memory/the skill hub, leaving the claimed “primary drivers” insufficiently supported.
  2. Existing specialized baselines are relevant, but do not establish an advantage over a tool-using code-optimization agent with ordinary trajectory memory, or a fixed controller with the same diagnostic tools and memory. Backbone robustness is also untested: GPT-4o-mini and DeepSeek-V4-Flash are assigned to different tasks (Section 4.1), rather than compared on the same task with component ablations.
  3. The “7%” efficiency claim needs consistent accounting. Table 1 reports consumed evaluations, while Section 4.3 describes continued search after the best incumbent; Appendix C.2 uses uncharged counterfactual objective calls. Time-to-best, full-run submissions, and total objective calls must be distinguished before concluding comparable resource savings.
Questions for authors
  1. Under matched models, tools, starting code, and total budgets, how does AlgoEvo compare with an agent using ordinary trajectory memory and a fixed controller using the same diagnostics and memory?
  2. On one multi-component task, do the main gains and diagnosis/memory ablations persist with a second backbone?
  3. Please reconcile Table 1 with continued search and Appendix D.4, counting diagnostic objective calls and separating full-run costs from time-to-best.
7 comments
AuthorsOpenReview ↗

Response to Reviewer Ygt1: New Ablations, Matched Baselines, Six Backbones and Reconciled Cost Accounting

We sincerely thank the reviewer Ygt1 for the insightful, constructive and rigorous feedback. Your comments on novelty, baselines, backbone generalization and efficiency accounting have substantially improved the quality of this manuscript, and we are grateful for the care and effort behind them.

We added four suites in Appendix F.3–F.6: an extended ablation, two baselines under matched tools and budgets, a six-backbone comparison and an end-to-end cost study. The accounting is clarified in Sec. 4.1–4.2.

1. (W1) Novelty & Diagnosis Isolation

1) Novelty

To our knowledge, AlgoEvo is the first framework in which a single engine covers single-heuristic, multi-objective and multi-component design, whereas existing AAD frameworks (e.g. EoH, ReEvo) rely on rigid, predefined generation-selection loops within one paradigm. We formulate discovery as an adaptive search control problem over the state st=(Πt,Ct,ϵ,Fd)s_t = (\mathbf{\Pi}_t, \mathcal{C}_t, \epsilon, F_d) (§3.1), where the agent chooses among inspect, modify, diagnose and evaluate from runtime feedback. Control sits in the runtime state: diagnosis is an action the agent selects, not a fixed prompt step. Two components support it:

  • Design Skill Hub (§3.2): carries paradigm knowledge, so a new paradigm needs a new skill, not a new engine.
  • Hierarchical Experience Bank (§3.3): organises experience cards, task-level trees and cross-task consolidation.

2) Diagnosis Ablation (Table 10) Table 10 adds a variant that isolates diagnosis, "without autonomous diagnosis": one LLM call per step, so diagnostic calls are zero by construction. It covers CVRP-DR and Bi-TSP, three seeds each.

  • On the unsaturated Bi-TSP, removing autonomous diagnosis costs the most (−5.41%), more than the experience bank (−4.11%) or the skill hub (−2.50%), and it also tightens the seed spread from ±0.0339\pm 0.0339 to ±0.0032\pm 0.0032. On CVRP-DR all variants differ by under 1%, confirming saturation.
  • Tables 2–3: the description has been adjusted to avoid ambiguity; every round starts from the same template code, and the cards act as context rules that do not overwrite the starting program.

2. (W2, Q1, Q2) Controls & Backbone

To separate the framework from code editing and model strength, we ran two baselines (Q1) and a six-backbone comparison (Q2).

1) Coding Agents (Table 13) Codex, Claude Code and OpenCode ran on CVRP-DR with identical starting code, entry and budget, capped at 60 evaluations with the guard relaxed, each with its own trajectory memory and no skill hub or bank.

  • AlgoEvo reaches the lowest cost on every backbone (e.g. 8.43 vs 9.03–10.41). The agents use 3–78 evaluations.

2) Fixed-Schedule Control (Table 14) A fixed-schedule control shares the backbone, skill files, experience bank, starting code and budget (500), but alternates diagnosis probes and edits.

  • The adaptive loop wins on every metric: held-out cost 0.137 better, 229 fewer evaluations, 3,229 s less wall-clock and 12.9M fewer tokens. The gain comes from deciding when to diagnose.

3) Backbone Comparison (Tables 11–12) Six backbones (DeepSeek, Gemini, Claude, GPT, Qwen) ran both tasks.

  • All functional backbones improve on the heuristics and the ranking is invariant, so the gain comes from the orchestration rather than model scale. On most backbones AlgoEvo also beats the specialised AAD baselines. The two exceptions, gpt-4o-mini and qwen-flash, could not follow the task instructions reliably enough to produce a valid submission and stayed at the starting point.

3. (W3, Q3) Efficiency Accounting

Our main experiments follow the standard AAD protocol (EoH, ReEvo, FunSearch, MEoH): all methods share an evaluation budget of 500, and we report the evaluations each consumes. On Bi-FJSP AlgoEvo completes in 33 evaluations, or 6.6% of the allowance (§4.2); the shares are 6.6–7.8% on the single- and multi-objective tasks and 54–55% on the multi-component tasks. The evaluation column counts every call through the shared evaluation entry, including diagnostic probes (App. D.4).

Performance under an identical budget is documented in the ablation and the fixed-schedule comparison (Tables 10, 14), where every variant shares the same 500-evaluation allowance. In the ceiling runs the allowance is unchanged: the agent stops early once it judges that further search has no potential, and Fig. 3 shows that the remaining budget brings little further gain. Those runs report the total cost of a complete design (App. F.6, Table 15): 5.78 min against 21.57–34.25 min for the baselines, with diagnosis, tools and edits included.

All revisions are marked in red, and we hope they address your concerns. We also note with appreciation the time and care behind your comments. Should you have any further questions or suggestions, we would be glad to discuss them.

AuthorsOpenReview ↗

Overview of the detailed content for Reviewer Ygt1

Thank you very much for taking the time to review our work and for such a detailed set of comments. Your three concerns, namely the positioning of the method, the confound in Tables 2–3, and whether Table 10 isolates the diagnostic mechanism, all point to the same question: whether the boundary of our contribution is stated clearly enough. To make the changes easy to follow, we have organized one response for each concern, with the relevant tables and per-seed readings placed directly in the replies. If you would like additional experiments or different wording, we are glad to continue the discussion.

We care a great deal about this work, and we hope that, at a time when AI is advancing quickly, it offers a different kind of solution for automated algorithm design.

If these explanations address your concerns, we would be very grateful if you would reconsider the rating.

The four concerns are answered in the four responses below.

AuthorsOpenReview ↗

Detailed Response to Reviewer Ygt1 on W1 (1/4): Novelty, Tables 2–3, and Diagnosis Isolation

W1: Methodological novelty beyond reflective search and experience reuse is insufficiently established. The distinctive challenge is diagnosis-driven search control and component coordination, not memory accumulation alone. Tables 2–3 confound experience with inherited code and continued search. Table 10 tests one task, does not isolate the diagnostic mechanism, and shows only 0.66%/0.58% mean degradation without memory/the skill hub, leaving the claimed “primary drivers” insufficiently supported.

Response

Thank you for the constructive feedback. We address the core concerns regarding methodological novelty, the experimental setup of Tables 2–3, and the isolation of the diagnostic mechanism below.

1. Methodological Positioning and Novelty

Our framework departs from conventional fixed pipelines by formulating algorithm design as an adaptive search process (§3.1). Rather than executing predefined action sequences where LLMs merely generate code, our agent dynamically selects actions (inspect, modify, diagnose, evaluate) based on runtime states st=(Πt,Ct,ϵ,Fd)s_t = (\mathbf{\Pi}_t, \mathcal{C}_t, \epsilon, F_d). This enables on-demand diagnosis when bottlenecks arise and bypasses it when unnecessary. Furthermore, our unified cross-paradigm engine (§3.2) externalizes paradigm-specific knowledge (strategy roles, code interfaces, modification principles, and evaluation protocols) via design skills, leaving the underlying search mechanism invariant across all six tasks in Table 1. The experience bank (§3.3) sustains evolution by distilling evaluations into task-specific trees and consolidating stable patterns back into skills. These components reinforce one another: skill contracts define loop actions, and loop evaluations yield retrievable cards that are finally consolidated back into skills. We therefore describe the experience bank and skill hub as mechanisms supporting cross-paradigm scalability and stability, and have removed the "primary drivers" wording.

2. Autonomous Diagnosis vs. Conventional Reflection

Traditional reflection executes periodically at fixed pipeline stages [1]. Our autonomous diagnosis is triggered by runtime state and can be skipped when unnecessary. Moreover, while standard evaluation provides only scalar feedback, diagnosis outputs component-level attribution, interaction effects, and failure modes to explicitly indicate why a unit failed and how to modify it. Table 10 isolates this mechanism with a variant that issues one LLM call per step and no diagnosis.

3. Experimental Protocol in Tables 2–3

In our protocol, each (task,seed,round)(task, seed, round) strictly originates from the exact same default template; prior-round optimal code is never inherited as a starting point. Instead, the agent retrieves distilled card rules as context while rewriting programs from scratch. Thus, the performance gains of warm start and multi-round accumulation stem purely from the experience bank. Specifically, warm start reduces TSP tour length from 4.961 to 4.808 and evaluations from 48 to 40, while four-round accumulation progressively advances TSP from 4.8928 to 4.7995 (−1.91%) and increases Bi-FJSP HV from 0.1549 to 0.1862 (+20.21%).

Per-round numbers are in Tables 2–3 (§4.3).

4. Task Saturation and Mechanism Isolation in Table 10

In the converged CVRP-DR task, all five variants cluster between 9.040 and 9.122 (range 0.09), which explains the 0.66%/0.58% drops you noted: near convergence, differences are compressed within measurement noise. In contrast, on the unsaturated Bi-TSP task, component contributions are clearly delineated (Table 10, Figure 12): the full system achieves 0.8566, while removing autonomous diagnosis leads to the steepest drop to 0.8103 (−5.41%), followed by removing the experience bank to 0.8214 (−4.11%), the skill hub to 0.8352 (−2.50%), and contextual triggers to 0.8473 (−1.08%). All three seeds show consistent directional trends (cross-seed std: 0.0032, 0.0215, 0.0339). Given n=3n=3, we claim directional consistency rather than overstating significance. The ordering also settles the attribution question: if gains came mainly from passive accumulation, removing the bank would hurt most; instead removing diagnosis hurts most (5.41% > 4.11% > 2.50%), consistent with control residing in runtime state.

Table 10 (Appendix F.3)

VariantCVRP-DRBi-TSP (Δ)
Complete9.0400.8566
w/o diagnosis9.122 (+0.91%)0.8103 (−5.41%)
w/o bank9.100 (+0.66%)0.8214 (−4.11%)
w/o hub9.092 (+0.58%)0.8352 (−2.50%)
w/o injection9.059 (+0.21%)0.8473 (−1.08%)

We have further revised §1 and §3.1, added the Bi-TSP ablation (Table 10, Appendix F.3) and adjusted the protocol description; these changes are marked in red. We are grateful for this suggestion, which led us to support each claim with concrete numbers.

[1] H. Ye et al., ReEvo, NeurIPS 2024

Comments on the paper

1 comment
AuthorsOpenReview ↗

Request to the Area Chair: Kindly Encourage Reviewer Engagement

Dear Area Chair,

Thank you for managing the review process for our paper.

Although the discussion period has been open for several days, we have not yet received any responses from the reviewers. We are concerned that they may not yet have had a chance to review our rebuttal, which includes new experiments and analyses directly addressing their primary concerns.

Given the limited time remaining, we would greatly appreciate it if you could kindly remind the reviewers to review our rebuttal and participate in the discussion. We believe that further engagement is essential to ensure a fair and thorough review process.

Thank you very much for your time and support.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.