- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LEAP, an LLM-guided framework for instance-aware MILP solver configuration. It combines LLM-based initialization, adaptive search-space refinement, SMAC-based portfolio discovery, and GNN-based configuration selection. Experiments across 12 datasets and two MILP solvers demonstrate promising improvements in solving efficiency over default configurations and existing baselines.
Critical strengths
- The paper addresses a practically important problem in MILP solver configuration and presents a coherent framework integrating LLM guidance with numerical optimization.
- The evaluation covers 12 datasets and two widely used MILP solvers, providing encouraging evidence of practical effectiveness.
- The ablation studies support the contributions of LLM-guided initialization and adaptive search-space refinement.
Critical weaknesses
- The methodological novelty is moderate, as the framework largely combines established configuration and selection techniques. The unique benefits of LLM guidance over simpler heuristic alternatives remain insufficiently demonstrated.
- The computational comparisons do not fully account for additional initialization and portfolio evaluation costs, potentially affecting the fairness of the reported efficiency improvements.
- Some performance improvements are relatively small, and stronger statistical analysis would help establish their reliability.
Questions for authors
- How does LEAP compare with non-LLM heuristic initialization and search-space refinement methods?
- Can the authors provide comparisons under matched total computational budgets, including initialization and portfolio evaluation costs?
- Are the smaller performance improvements statistically significant across different random seeds?
3 comments
Response to Questions:
Q1: Non-LLM heuristic controls. Thank you for suggesting comparisons with non-LLM heuristic initialization and search-space refinement methods. We added LHS-Init and Staged-Refine, each replacing one LLM component. LEAP achieves lower mean test solving time with GNN selection than both controls across all 14 solver–dataset settings. Given the time and computational constraints during rebuttal, these experiments cover all Hard and BenLOC benchmarks in the paper. We will extend the evaluation to the remaining benchmarks and provide full reproducibility details in the final version.
LHS-Init uses SMAC’s Latin hypercube sampling to generate the same number of ten distinct initial candidates within the same master configuration space, replacing only the LLM Initializer while retaining the LLM Refiner.
Staged-Refine retains LLM initialization and adapts the alternating optimization of parameter blocks in BliStrTune (Jakubův & Urban, 2017) [1] to LEAP’s refinement between discovery rounds. Parameters are partitioned beforehand into two blocks, primarily covering cuts and presolving (A), and branching and heuristics (B). The first round searches the full space; subsequent rounds alternate between the two blocks, fixing inactive parameters to their values in the previous round’s SMAC incumbent. The portfolio is still updated after each round, and SMAC is reinitialized in the next round to optimize complementarity with the updated portfolio.
Both controls retain the same master configuration space, solver default, portfolio update mechanism, and SMAC settings. They follow the original protocol of five SMAC rounds with two hours per round; initialization and evaluations between rounds remain outside this SMAC search budget. The tables report test solving time with GNN selection, in seconds, as mean ± standard deviation over three random seeds; lower is better.
SCIP
| Dataset | LEAP | LHS-Init | Staged-Refine |
|---|---|---|---|
| rhs_3 | 290.7 ± 29.3 | 323.2 ± 20.4 | 347.0 ± 40.6 |
| obj_2 | 364.3 ± 11.5 | 388.3 ± 9.2 | 379.1 ± 19.6 |
| mix_1 | 125.3 ± 14.2 | 151.2 ± 12.8 | 132.4 ± 24.3 |
| mix_2 | 264.4 ± 3.1 | 276.4 ± 21.4 | 274.5 ± 7.1 |
| mix_3 | 86.0 ± 1.3 | 100.0 ± 28.2 | 103.0 ± 21.3 |
| nn | 52.9 ± 9.1 | 67.7 ± 7.6 | 53.7 ± 5.2 |
| load | 217.7 ± 9.2 | 225.1 ± 13.8 | 243.1 ± 21.7 |
Gurobi
| Dataset | LEAP | LHS-Init | Staged-Refine |
|---|---|---|---|
| rhs_3 | 22.9 ± 8.6 | 26.6 ± 6.4 | 39.8 ± 14.9 |
| obj_2 | 1.5 ± 0.6 | 1.9 ± 0.1 | 1.62 ± 0.10 |
| mix_1 | 12.0 ± 0.4 | 12.8 ± 1.2 | 12.5 ± 1.3 |
| mix_2 | 22.5 ± 11.3 | 52.9 ± 19.7 | 23.5 ± 2.3 |
| mix_3 | 6.6 ± 1.3 | 7.9 ± 0.4 | 8.5 ± 1.4 |
| nn | 4.6 ± 0.9 | 5.4 ± 1.9 | 5.9 ± 3.5 |
| load | 0.8 ± 0.1 | 0.9 ± 0.1 | 0.9 ± 0.2 |
The LHS-Init comparison supports the value of LLM-proposed initialization with the same candidate count, while Staged-Refine supports the benefit of LLM-guided refinement relative to the evaluated block-search rule, with LLM initialization retained. These replacement experiments complement the existing component-removal ablations and show that the benefits of both LLM components carry through to performance with the deployed selector. We would also welcome the reviewer’s suggestions for other relevant non-LLM heuristics to explore in future work.
[1] Jakubův, Jan, and Josef Urban. "BliStrTune: hierarchical invention of theorem proving strategies." Proceedings of the 6th ACM SIGPLAN Conference on Certified Programs and Proofs. 2017.
Q2: Discovery costs and budget comparison. Thank you for highlighting the importance of including initialization and full discovery-set evaluations when assessing discovery efficiency. We conducted additional experiments in which initialization evaluations, SMAC search, and inter-round evaluations jointly receive a 10-hour wall-clock discovery budget. LEAP achieves the lowest mean test solving time with GNN selection among the compared methods in 12 of 14 settings, including one tie, with improvements of up to 93.8% over Global-SMAC and 61.0% over w/o Both. In one further setting, SCIP obj_2, LEAP is within 0.6% of the strongest baseline. In the remaining setting, Gurobi rhs_3, LEAP takes 26.5s compared with 23.0s for the strongest baseline, DGCAC.
Given the time and computational resources available during rebuttal, these experiments cover all five Hard and both BenLOC datasets, using both solvers. We train GNN selectors for the portfolios discovered under this budget. Baselines retain their main-experiment results: Global-SMAC and w/o Both retain their full 10-hour SMAC search allocations, and w/o Both also retains its additional evaluations. Thus, LEAP achieves these results after deducting evaluation time and using less SMAC search time, while the baselines’ search and evaluation resources remain unchanged.
The tables report test solving time in seconds, as mean ± standard deviation over three repetitions. Lower is better; bold marks the lowest mean in each row.
SCIP
| Dataset | Global-SMAC | DGCAC | w/o Both | LEAP (10h) |
|---|---|---|---|---|
| rhs_3 | 317.4 ± 39.3 | 327.1 ± 27.8 | 313.1 ± 12.8 | 260.9 ± 32.1 |
| obj_2 | 364.5 ± 2.0 | 398.4 ± 20.6 | 383.7 ± 10.5 | 366.7 ± 23.9 |
| mix_1 | 146.3 ± 32.1 | 154.2 ± 36.4 | 155.7 ± 10.5 | 117.2 ± 17.8 |
| mix_2 | 259.2 ± 1.3 | 264.0 ± 5.5 | 285.8 ± 3.1 | 258.5 ± 11.1 |
| mix_3 | 95.3 ± 3.1 | 91.4 ± 33.2 | 111.8 ± 34.7 | 72.9 ± 17.2 |
| nn | 72.3 ± 8.5 | 65.6 ± 6.7 | 58.9 ± 2.7 | 54.0 ± 6.3 |
| load | 241.9 ± 9.0 | 348.6 ± 101.9 | 252.3 ± 19.3 | 229.8 ± 1.2 |
Gurobi
| Dataset | Global-SMAC | DGCAC | w/o Both | LEAP (10h) |
|---|---|---|---|---|
| rhs_3 | 25.8 ± 8.9 | 23.0 ± 13.5 | 25.1 ± 10.9 | 26.5 ± 2.3 |
| obj_2 | 25.7 ± 0.1 | 28.3 ± 1.8 | 4.1 ± 5.9 | 1.6 ± 0.0 |
| mix_1 | 12.2 ± 0.0 | 13.6 ± 0.0 | 12.3 ± 0.4 | 12.2 ± 4.5 |
| mix_2 | 45.7 ± 17.1 | 47.8 ± 9.5 | 26.6 ± 32.1 | 18.0 ± 10.5 |
| mix_3 | 8.6 ± 0.1 | 8.8 ± 2.3 | 13.1 ± 12.6 | 5.2 ± 0.9 |
| nn | 5.2 ± 0.7 | 5.1 ± 1.2 | 5.1 ± 1.3 | 3.9 ± 0.3 |
| load | 1.3 ± 0.4 | 3.5 ± 4.0 | 0.9 ± 0.4 | 0.5 ± 0.0 |
Note that SMAC search uses previous evaluation feedback to adaptively select subsequent candidates; in contrast, during initialization and inter-round evaluation, the candidate configurations are already fixed, and runs of a given configuration on different instances require no feedback from one another. These evaluations are therefore naturally amenable to batch parallelism, which we exploit using 20 parallel single-thread CPU workers, allowing the evaluations to proceed concurrently and shortening their elapsed time.
Q3: Variability and smaller performance improvements. Thank you for highlighting the reliability of smaller improvements. Appendix E.1 now reports the results corresponding to the main tables as mean ± standard deviation across three configuration-discovery repetitions. LEAP’s overall performance gains span both solvers and diverse tasks: it achieves lower mean test solving time than Default in all 24 settings, with improvements of up to 43.2% for SCIP and 94.7% for Gurobi.
The advantages over other configuration methods also extend across most settings. LEAP achieves lower mean solving time than DGCAC and Global-SMAC in 21/24 settings each. For example, on Gurobi obj_2, LEAP achieves 1.5 ± 0.6 seconds compared with 25.7 ± 0.1 seconds for Global-SMAC, an improvement of approximately 94.2%. Other comparisons are close: on SCIP obj_2, LEAP achieves 364.3 ± 11.5 seconds versus 364.5 ± 2.0 seconds for Global-SMAC. Across the ablations, LEAP achieves the lowest mean test solving time in 21/24 settings. On SCIP load, LEAP achieves 217.7 ± 9.2 seconds compared with 241.1 ± 15.7 seconds without refinement, an improvement of approximately 9.7%. On Gurobi load, both LEAP and w/o Refine achieve 0.8 ± 0.1 seconds, corresponding to improvements over Default of 94.7% and 94.6%, respectively. The variant retaining LLM initialization therefore already delivers substantial gains on this benchmark.
We thank the reviewer for the encouraging assessment and helpful suggestions. We appreciate the opportunity to strengthen the comparisons and presentation of our results, and respond to each point below.
Response to Weaknesses:
W1. Methodological novelty and benefits of LLM guidance. Thank you for this comment. LEAP’s methodological contribution centers on coupling problem-informed initialization with portfolio-aware search-space refinement throughout iterative configuration discovery. In related MIP configuration work, Hydra-MIP discovers complementary configurations through repeated solver evaluations, whose cost limits exploration of large parameter spaces under finite budgets. Lawless et al. (2025) provide effective cold-start separator configurations, but their family-level selection remains constrained by the generated candidates and cannot fully exploit differing configuration preferences across instances. LEAP addresses these limitations by using problem-family knowledge and parameter semantics to propose strategically distinct starting configurations across multiple solver components, then using portfolio performance and search history to focus or expand subsequent numerical search. Semantic guidance thus shapes both the initial portfolio and where subsequent searches seek additional complementary gains.
To quantify the benefits of LLM guidance over simpler heuristic alternatives, we added two controls that separately replace LLM initialization and refinement. LEAP achieves lower mean test solving time with GNN selection than both controls across all 14 evaluated solver–dataset settings. The gains are substantial in several settings: on Gurobi
mix_2, LEAP improves over LHS-Init from 52.9s to 22.5s (57.5% improvement) with the same number of initial candidates; on Gurobirhs_3, it improves over Staged-Refine from 39.8s to 22.9s (42.5% improvement) with LLM initialization retained. These comparisons quantify the additional benefits of the LLM components relative to the evaluated non-LLM alternatives. Our response to Question 1 provides the control definitions, experimental protocol, and complete results.W2: Discovery-cost accounting. Thank you for highlighting the fairness of the computational comparisons. We added experiments that include initialization and inter-round evaluations within LEAP’s 10-hour total wall-clock discovery budget, correspondingly reducing its SMAC search time. LEAP achieves the lowest mean test solving time among the compared methods in 12 of 14 settings, including one tie. Appendix A.3 also provides the stage-by-stage timing breakdown and supplementary CPU evaluation costs. Our response to Question 2 presents the experimental protocol, complete performance results, and cost-accounting details.
W3: Reliability of smaller improvements. Thank you for highlighting the reliability of smaller performance improvements. Appendix E.1 now reports mean ± standard deviation across three configuration-discovery repetitions for the main-table results. Our response to Question 3 discusses the improvement magnitudes and variability, including both substantial gains and settings with close mean performance.