- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the LLM-based evolution of local-search operator sets for vehicle routing. The main idea is to replace free-form source code generation with a typed DSL and compiler. The search starts from an empty operator set. At each round, a newly proposed operator is tested by replacing each incumbent operator or by being appended to the current set, and an update is accepted only when the objective improves. The authors evaluate EoLS against a raw free-code arm and strategy layers adapted from EoH, FunSearch, ReEvo, and HiFo-Prompt under a common solver host and token budget. The paper calls three routing variants “unseen” because their constraint combinations are not covered by the existing hand-written operator library. The main claim is that the typed DSL transfers better to these variants and that representation is more important than the choice of search strategy. The paper also analyzes token consumption, static rejection, and operators that compile and run but fail to produce improving moves.
Critical strengths
-
I find the general empirical question useful. It is reasonable to ask whether the representation used for LLM-generated operators affects search efficiency and reliability, and the proposed typed DSL is a promising implementation of this idea.
-
I appreciate the attempt to go beyond final objective values and inspect failure modes. Measurements such as token cost per proposal, static rejection, and the number of accepted improving moves could be useful for understanding why an LLM-based search succeeds or fails. The empty-set setup is also an interesting attempt to make the contribution of evolved operators easier to inspect.
Critical weaknesses
-
The central claim is substantially stronger than the novelty and evidence support. The general idea of changing the search representation is not new in LLM-based algorithm discovery. Prior work has already searched over operator design thoughts and combinations [1], semantic concepts [2], knowledge as the primary search object [3], multiple representation spaces [4], and joint code/thought representations [5]. E2OC is particularly close because it already performs progressive replacement inside a multi-operator combination. The new contribution therefore appears much narrower, mainly a routing-specific typed DSL and compiler. More importantly, the empirical comparison used to support the title-level claim is seriously unbalanced. EoLS is evaluated with multiple independent trajectories, while EoH, FunSearch, ReEvo, and HiFo-Prompt are each reported from only a single trajectory. This is especially problematic because EoLS itself shows large stochastic variation, ranging from 10.6% to 25.5% on
sd_q_tw_o, while the single EoH run achieves 19.2%, well within that range. Comparing a multi-run mean for EoLS against single-run outcomes for stochastic LLM baselines is not a fair basis for concluding that representation dominates search strategy. In addition, EoLS receives many more proposals under the same token budget and benefits from compiler/backend support that removes many low-level validity failures. These results demonstrate useful engineering advantages, but they do not cleanly establish that the DSL itself produces better operators or that representation is the dominant factor. I am also not convinced by the transfer framing. EoLS appears to search directly on the support units of each “unseen” target variant. In that case, “unseen” seems to mean unsupported by the handwritten library, not unseen to the search process. This is different from the usual source-task to held-out-target notion of transfer. Finally, the empty-set argument is overstated. Once an incumbent set S_k contains evolved operators, a new operator can still have zero marginal improvement because the existing set has already captured the same moves. Replacement reduces this issue, but it does not remove it. The protocol measures contribution conditional on the current set, not the standalone usefulness of an operator. -
The manuscript is incomplete, and the experimental reporting is not reliable enough in its current form. The referenced appendix is missing even though the paper points to it for prompts, model details, and additional results. Section 5.1 says that three unseen variants are evaluated, but Table 1 reports only two, while Section 6 later states that the main table covers three. The abstract and conclusion report 4.8% for the free-form arm, whereas Table 1 gives a mean of 0.1%, and the listed EoLS replicate count is also inconsistent with the values shown. There is also a clear mismatch between the framework presented in Figure 1 and what is actually specified and evaluated. Figure 1 includes a population/reproduction process that is not sufficiently described in the methodology, and it presents an implementation-optimization channel with a “Correct and Faster” gate as an integral component, although Section 4.2 states that this channel is not exercised in any reported experiment. The supposedly held-out “selection units” are not reported, and the terminology itself is confusing because they are never used for selection. The “ablation” in Section 5.6 does not ablate any component of EoLS. The paper also contains several clarity and proofreading problems, including undefined terminology and a duplicated paragraph. The authors disclose AI assistance for drafting and editing, which is not an issue by itself, but it does not remove the need to carefully verify and polish the final manuscript. Given these critical issues, the overall manuscript quality is substantially below the standard I would expect for a conference submission.
References
[1] Qiu et al., Evolving Interdependent Operators with Large Language Models for Multi-Objective Combinatorial Optimization, 2026.
[2] Leleu et al., Contrastive Concept-Tree Search for LLM-Assisted Algorithm Discovery, 2026.
[3] Kiet et al., Back to the Beginning of Heuristic Design: Bridging Code and Knowledge with LLMs, 2026.
[4] Guo et al., CoEvo: Continual Evolution of Symbolic Solutions Using Large Language Models, AAAI 2026.
[5] Hu and Zhang, Partition to Evolve: Niching-enhanced Evolution with LLMs for Automated Algorithm Discovery, NeurIPS 2025.
Questions for authors
-
What exactly is meant by “transfer” and “unseen variant” in this paper? My reading is that EoLS searches directly on the support units of every target variant. If so, what artifact learned on one task is transferred to another task?
-
What is the complete search procedure used by the EoLS arm? In particular, how is o* generated, how is the operator population in Figure 1 maintained, how are parents selected, what reproduction or mutation operation is used, and what search history is included in the LLM prompt? These should be explained in the methodology and appendix of the manuscript; however, they are currently missing from it.
-
How is the proposed operator-set evolution different from E2OC’s operator-rotation mechanism? Relatedly, what prior work supports the statement that the “usual way” is to keep a fixed hand-written operator floor and only append one new operator at each round?
-
What exactly is generated by the LLM in the typed-DSL arm, and what is supplied by the compiler or backend? Could you give some examples of the evolved high-performing operators? Please also clarify how the comparison isolates representation when the DSL receives built-in support for feasibility and low-level execution details. Can the authors provide a matched-proposal-count comparison or statistics such as proposals and tokens per accepted improvement?
-
Please provide the missing pd_q_tw_o results and the final results on the selection units. Please also reconcile the 4.8% free-form result in the abstract and conclusion with the 0.1% mean in Table 1, and explain the entry when five sd_q_o values are listed.
-
What is the covered variant used in Section 5.6? Please report the per-method and per-replicate results, including the EoLS result. Can the authors also provide actual component ablations for the DSL constraints, compiler support, empty-set initialization, and replacement mechanism?