- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes FROM-ROUTE, an LLM-based framework for selecting mixed-integer programming (MIP) formulations from a predefined library of formulation experts. The framework first extracts instance-specific structural and numerical features, then uses deterministic heuristics to construct mechanism-oriented knowledge cards. An LLM subsequently selects an ordered portfolio of three candidate formulations, which are executed under a shared computational budget. The resulting solutions are independently validated against the original optimization problems.
The framework is evaluated on three classical operations research problems: multicommodity network design (MCND), vertex coloring (VCP), and resource-constrained project scheduling (RCPSP), covering 2,335 instances and 40 formulation configurations. The authors report reductions of 57.0%, 14.6%, and 3.6% in mean normalized primal integral relative to the best fixed formulation in the respective domains, along with improvements in validated model-completion counts. Additional experiments compare different knowledge configurations, investigate a restricted-completion relay strategy for MCND, and evaluate the framework against LLM-based modeling-kernel generation systems.
Critical strengths
- Interesting and Significant topic: The topic of this paper is to auto-selecting MIP formulation for solving efficiency by LLM, which have real impact and could be further explored.
- Comprehensive dataset and reasonable metrics: This paper has 3 different datasets and use multiple metrics, such as objective value, constraint violation and primal integral, which are all effecitve metrics in field of solver developing.
Critical weaknesses
- Paper writing is terrible: This paper is quite long and includes all details, which is good, but it fails to show the core idea and has a lot of defensive expressions. These make this paper more like a report instead of a mature paper. I think the reason is, as the authors said, this paper was largely finished by Gen AI. Although I agree with using Gen AI to draft papers, which allows authors to spend more time on research, I believe the paper should be modified by the authors to make the storyline clear, instead of putting everything in the paper and letting readers spend too much time.
- Lack of baselines: I think a strong baseline in this setting is traditional machine learning model and deep learning based method, as in this working, the feature of problem has already been extracted. However, this paper only compare their method with LLM-native method, which cannot convince me. If traditional method is enough, why we use expensive LLM.
- I think the role of deterministic heuristic is unclear, which is complex and designed manually. Also in different datasets, the heuristic are complete different, which let me doubt that these heuristic is overfitted on each dataset. Besides, I think the ablation of heuristics is also weak, the execution of FROM-ROUTE can execute multiple formulations but Rule Top-1 can only execute one formulation.
Questions for authors
Please see the weakness section.
2 comments
Thank you for clarifying. I decide to keep my score.
Response to Reviewer E7vN
W1
We have rewritten the Introduction's method overview around how instance data and modeling experience inform LLM formulation selection. Instance analysis supplies structural and construction information, mechanism cards store modeling experience, and the LLM uses both to order experts. The commodity-aggregation example explains why model size and relaxation strength must be considered together for an instance.
Figure 2 shows these relationships and the information or decision step lost when a component is removed. The new paragraphs replace the brief overview and consolidate adjacent repetition; detailed interfaces and execution protocols remain in the method section and appendices. The authors are responsible for the final argument, text, and experimental interpretation, with the tools' supporting roles recorded in the AI Use Statement.
W2
We compare against all 16, 8, and 16 fixed expert configurations for MCND, VCP, and RCPSP, respectively, plus the non-LLM MCND Rule Top-1 replay. Figure 4, Table 3, and Appendix E.6 report these results. Fixed experts test the value of instance-level selection; rule replay tests direct top-recommendation execution; RAW compares systems that generate modeling kernels.
The LLM interprets natural-language modeling experience alongside numerical features, without training a task-specific selection model. For example, it uses commodity structure and construction size to interpret a card's explanation of aggregation's size/relaxation trade-off. Table 4 tests whether this knowledge improves decisions through the same LLM interface.
MCND Full uses an average of 2,540.94 tokens per instance in retained decision calls (Appendix E.4). We have not compared its performance and cost with a trained selector, so these tokens do not establish a cost advantage over traditional selection.
W3
The Analyzer extracts instance facts, and the recommender uses them to select relevant mechanism cards; the LLM determines the final expert order. MCND presents cards in name order, hides recommendation scores and ranks, and leaves the entire eligible library selectable. First choices fall outside the card-covered set on 20 of 196 instances (Appendix D.10). VCP/RCPSP also provide ranking priors that the LLM can revise.
Domain-specific analyses reflect different modeling objects, such as commodity aggregation, color representations, and time or resource representations. Recommendation parameters were adjusted using the reported benchmark results (Appendix D.3); fixed-parameter performance on independent instances has not been evaluated.
Rule Top-1 runs one expert, while Full may continue with others, so their difference compares complete systems. MCND-clean (Appendix F.6) compares knowledge contexts with the same shortlist, LLM configuration, and single-expert execution, without fallback or relay. Against No RAG and Generic RAG, Full reduces NPI by 11.3% and 12.7%, with original-problem completions increasing from 141 and 140 to 143.