- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CoME, a framework that enables LLMs to automatically generate solver-integrated code for complex vehicle routing problems involving multiple interacting constraints. A major premise of the approach is that grounding experience in verified reference code improves the correctness and reliability of generated optimization programs. The authors report high success and accuracy rates across complex VRP variants and several solver backends.
Critical strengths
The paper identifies an important limitation of existing LLM-based VRP modeling methods, namely their insufficient understanding of constraint interactions. The focus on cross-constraint modeling is meaningful and more specific than simply improving LLM-based code generation.
Critical weaknesses
- The methodological novelty of CoME appears limited. The most challenging cross-constraint modeling knowledge is already provided through expert-written reference implementations, while the proposed framework mainly extracts, organizes, and retrieves this knowledge to guide code generation. This resembles a structured RAG pipeline rather than a fundamentally new approach to constraint reasoning Moreover, when a new problem involves constraints or interaction rules that are not covered by the knowledge library, CoME still relies on the LLM's own reasoning. This limits the scope of its claimed generalization capability.
2.The "w/o Interaction Hints" configuration appears to remove both interaction rules and search strategies. Consequently, the observed performance degradation cannot be uniquely attributed to the absence of cross-constraint modeling knowledge. Separate ablations are needed to distinguish improvements in modeling correctness from those arising from solver search configuration.
- The paper matches CoME's computational budget to AFL, but RF-POMO operates with a substantially different inference-cost profile. In Table 2, RF-POMO is often much faster than CoME. Therefore, claiming superior optimization performance without sufficiently emphasizing computational differences can be misleading. A more informative presentation would report solution quality, end-to-end wall-clock time, and inference cost separately, distinguishing solver execution from LLM code-generation overhead.
Questions for authors
See Weaknesses
2 comments
Thank you for the detailed response and the additional experiments. The new results do partially alleviate some of my concerns, but several issues remain.
W1. My main concern about the limited methodological novelty remains unresolved. The authors have not yet demonstrated that the proposed structured approach to experience extraction and reuse provides sufficient added value over a simple RAG approach using the same expert-written reference code.
W2. The additional ablation experiments largely address my concern by better isolating the roles of interaction hints and search guidance, which makes the issue clearer.
W3. I understand the explanation regarding runtime differences, but there is still a lack of quantitative evidence on LLM code-generation overhead and end-to-end computational cost.
Overall, I maintain my original rating of 2.
Response to W1: CoME's Methodology and generalization
Thank you for this comment. CoME builds on RAG and contributes an automated method for extracting, representing, and reusing cross-constraint integration knowledge. Per-constraint retrieval leaves the compostion of interacting implementations to the LLM. CoME makes these requirements explicit through rules describing which constraints interact and how their implementations must change when combined, then selects applicable rules for code generation. Expert reference programs supply the underlying modeling knowledge because solver documentation and isolated examples offer limited guidance on complex joint implementations. CoME extracts this knowledge from a small set of expert-curated variants and makes it reusable across many constraint compositions. Our generalization claim concerns new compositions of standard VRP constraints for which applicable atomic and interaction knowledge is available. When a new constraint or an uncovered interaction arises, the LLM attempts its implementation using its own reasoning and any relevant existing hints.
Response to W2: Interaction hints and search guidance ablations
Thank you for identifying this confound. Our original ablation removed search guidance together with interaction hints because search guidance is retrieved according to the identified constraint combination. We have added a separate “w/o Search Guidance” condition in Table 4 that retains interaction hints. Comparing this condition with full CoME isolates the contribution of search guidance, while comparing the two conditions without search guidance assesses the contribution of interaction hints. Removing only search guidance reduces AR while leaving SR unchanged for both LLMs. Removing interaction hints when search guidance is already absent substantially reduces SR. These results support a contribution from interaction knowledge beyond retrieved search configuration, while search guidance provides an additional improvement in BKS matching in this evaluation. PyVRP has no separate retrieved search guidance, so the additional search-guidance ablation applies only to OR-Tools.
Table: Ablation results on DRoC’s 48 VRP variants. AR and SR are reported (%) for each configuration on OR-Tools and PyVRP. N/A denotes that the search-guidance ablation does not apply to PyVRP, which has no separate retrieved search guidance.
Response to W3: Solution quality, runtime, and computational cost
Thank you for highlighting the computational differences between these methods. Table 2 reports both solution gaps and runtimes. We have revised the discussion in lines 409–410 to explicitly acknowledge that RF-POMO has substantially shorter runtimes than CoME, demonstrating its advantage in inference speed, while its gaps exceed 40% on several multi-depot variants. The time-budget matching described in the paper applies to CoME and AFL. We therefore interpret the comparison with RF-POMO as a solution-quality/runtime trade-off: RF-POMO provides faster solutions, while CoME obtains lower gaps under the reported solver budgets. Table 2 does not separately quantify LLM code-generation overhead, so these results do not establish equivalent end-to-end inference costs across the methods.