- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper is motivated by the fact that progress in solving MILP is hard to judge when a) a combination of different methods is used (i.e., for a combination of Method A + search trick + Gurobi solver, it is difficult to predict whether the gain came from the neural network method, search trick, or solver), b) different data is used (i.e., different instance sizes, generators, and benchmark reference solutions differ or are not released), c) metrics are inconsistent (i.e., primal gap often calculated differently or infeasible outputs are handled differently), and d) comparison is based on old solvers. To benchmark the MILP-solving pipeline, the authors split each learning-based pipeline into three swappable stages: learner, enforcer, and refiner. They re-implemented everything with a shared GNN backbone, the same data (set cover, independent set, combinatorial auction, facility location), and current solvers (Gurobi 13, SCIP 10, COPT 8). They then tested many Learner × Enforcer × Refiner combinations.
Critical strengths
The major positive points about the paper are a) it addresses the real problem of fair comparison, b) uses up-to-date solvers, c) systematic experimentation, and d) reports even the negative results.
Critical weaknesses
The paper suffers from several limitations: a) the problems studied are easy (up to 100 instances), b) very few mixed-integer LP instances studied, c) forcing everything onto one backbone and one hyperparameter setup may not compare the results fairly for methods reported often with different optimized parameters, d) claims are not backed statistically as results are not generated with multiple seeds, e) checkpoints trained on set cover were applied directly to WA, BIP and MIPLIB which may not generalize well and f) several interesting extensions such as RL methods like RL-SPH, and solver-internal learning (branching, cuts) are left out.
Questions for authors
The detailed comments are given below:
- The main tests use 100 synthetic instances per task, and Gurobi solves several of them in seconds. On such easy instances, learned guidance has little room to help. The benchmark proposed needs to be demonstrated on medium and hard problems to prove its usefulness. Further, the conclusion that the solvers make the difference added by learners obscure may not hold on hard problems.
- Very few mixed-integer problems were covered, despite the paper’s title. FMIP’s continuous-variable component was disabled, and only pure-binary MIPLIB instances were used.
- Forcing everything onto one backbone and one hyperparameter setup may not compare the reported methods fairly. DDIM’s guidance was disabled, Apollo was simplified (no UEBO training), and Local Branching does only one solve. ConPaS and CoCo-MILP got near-zero validity or absurd gaps on several tasks. The differences in performance reported in the paper may be caused by implementation or tuning issues rather than by the actual limitations of the methods being compared, and the paper doesn’t check against the original papers’ reported numbers.
- The main tables in the paper report no confidence intervals. Results should be generated with multiple random seeds for statistical validity.
- Checkpoints trained on set cover were applied directly to WA, BIP, and MIPLIB. That’s a very weak test of transfer.
- Learned LNS, RL methods like RL-SPH, and solver-internal learning (branching, cuts) are left out. The paper justifies this, but it limits the generalizability of the conclusions.
8 comments
Author Response to Reviewer m7ob (page 2)
W2&5 / Q2&5: Mixed-integer coverage, FMIP's continuous-variable component, and SC-only zero-shot transfer versus in-domain adaptation.
A: Thank you for raising these concerns about mixed-integer coverage and in-domain adaptation. First and foremost, we have clarified in the new abstract that ML4MILP-Bench is a modular benchmark and analysis framework for neural primal heuristics, with a primary empirical focus on binary ILPs and support for mixed-integer settings, consistent with our statement in Sec. 2.1 in the initial submission.
In the past week, following your suggestion, we have conducted in-distribution training on WA and BIP, both of which contain integer and continuous variables. In particular, we implemented FMIP's joint continuous-integer flow in the benchmark and enabled its continuous-variable component in both experiments. The new FMIP models therefore learn integer and continuous variables jointly. For Base-SL, PS-SL, ConPaS, and CoCo-MILP, we train on integer-variable targets and let PS/Gurobi assign the continuous variables, following the original prediction setups. We trained each learner for 100 epochs per target task. Note that DiffILO’s loss penalizes expected constraint violations under a distribution over binary assignments, and guided DDIM likewise supports learning only for binary ILPs. So they are not trained on MILP data, but we retain the evaluation for both by transferring their SC-trained checkpoints directly to WA and BIP. The SC-transfer rows serve as a zero-shot baseline alongside the new target-trained models. All learner rows use PS with a 1000s Gurobi budget.
WA Results
| Learner | Setting | Gap@10s(%) | Gap@100s(%) | Gap@1000s(%) | Obj.@1000s |
|---|---|---|---|---|---|
| Gurobi | Solver only | 13.885 | 0.040 | 0.013 | 707.910 |
| Base-SL | SC transfer | 23.938 | 0.101 | 0.036 | 708.070 |
| Base-SL | i.i.d. | 1.656 | 0.096 | 0.037 | 708.080 |
| PS-SL | SC transfer | 23.847 | 0.102 | 0.036 | 708.070 |
| PS-SL | i.i.d. | 1.797 | 0.085 | 0.029 | 707.663 |
| ConPaS | SC transfer | 19.759 | 2.934 | 2.871 | 728.080 |
| ConPaS | i.i.d. | 1.314 | 0.091 | 0.033 | 708.050 |
| CoCo | SC transfer | 25.804 | 0.102 | 0.036 | 708.070 |
| CoCo | i.i.d. | 1.268 | 0.100 | 0.039 | 708.090 |
| FMIP | SC transfer | 24.924 | 0.099 | 0.037 | 708.080 |
| FMIP | i.i.d. | 1.215 | 0.120 | 0.033 | 708.050 |
| OPTFM | SC transfer | 22.978 | 0.176 | 0.126 | 708.700 |
| DiffILO | SC transfer | 24.821 | 0.099 | 0.036 | 708.070 |
| DDIM | SC transfer | 22.959 | 0.104 | 0.036 | 708.070 |
BIP Results
| Learner | Setting | Gap@10s(%) | Gap@100s(%) | Gap@1000s(%) | Obj.@1000s |
|---|---|---|---|---|---|
| Gurobi | Solver only | 61.856 | 26.023 | 6.881 | 13.294 |
| Base-SL | SC transfer | 28.710 | 8.653 | -0.730 | 12.532 |
| Base-SL | i.i.d. | 29.382 | 7.074 | -1.709 | 12.366 |
| PS-SL | SC transfer | 27.475 | 8.708 | -0.767 | 12.530 |
| PS-SL | i.i.d. | 33.166 | 9.842 | 1.154 | 12.704 |
| ConPaS | SC transfer | 34.889 | 13.154 | 0.306 | 12.579 |
| ConPaS | i.i.d. | 33.684 | 8.907 | -0.739 | 12.528 |
| CoCo | SC transfer | 27.237 | 8.624 | -0.751 | 12.534 |
| CoCo | i.i.d. | 37.476 | 14.785 | 1.898 | 12.642 |
| FMIP | SC transfer | 31.053 | 9.490 | -0.645 | 12.547 |
| FMIP | i.i.d. | 31.066 | 9.908 | -0.729 | 12.488 |
| OPTFM | SC transfer | 27.402 | 8.677 | -0.763 | 12.530 |
| DiffILO | SC transfer | 27.383 | 8.704 | -0.710 | 12.538 |
| DDIM | SC transfer | 29.868 | 7.425 | -1.829 | 12.427 |
Analysis. The clearest short-budget gains from in-distribution training occur on WA. At 100 s, four of the five learners have lower reported gaps, including ConPaS, whose gap falls from 2.934% to 0.091%. On BIP, three of the five matched learners improve at 1,000 s, and all eight SC-transfer pipelines reach lower final gaps than standalone Gurobi. For the jointly trained FMIP model on BIP, the gap falls from 31.066% at 10 s to −0.729% at 1,000 s within a single PS solve on the same 100 instances. Target-domain training can improve the guidance provided to PS, and the gains within the solve show how substantially final solution quality depends on solver-backed completion and search.
Remark. Considering current results, we note that raw learner results show considerable room to improve feasibility and objective quality even on our standard binary ILPs. As such, we believe the binary ILP study, the zero-shot transfer evaluations, and the new in-distribution WA/BIP results together properly support the revised scope now. Also, the framework provides common interfaces for MILP training and evaluation, including joint continuous–integer learners, and for incorporating additional learner and solver components. The scope clarification and the new WA/BIP experiments are also highlighted in red in our revised manuscript (Abstract, p. 1, lines 19–20 and Sec. 3.2, p. 9, lines 457–468 and Table 6). We sincerely hope these additions and discussions address your concerns about mixed-integer coverage and in-domain adaptation.
Author Response to Reviewer m7ob (page 3)
W3/Q3: Reproduction fidelity, simplified components, and anomalous baseline results.
A: Thank you for raising this concern. We appreciate the opportunity to clarify our standardization choices and the interpretation of the stage-wise results.
1. Backbone choice and preservation of method-specific components.
-
Variable–constraint bipartite graphs are widely used in learning-based MILP optimization. Most methods considered here (e.g. PS [1], ConPaS [2], DiffILO [3], CoCo-MILP [4], FMIP [5]) build on closely related GNN backbones, with method-specific extensions. Much of their innovation concerns training paradigms, learning objectives, and interactions with solver-backed search. This architectural commonality motivates a shared foundation for examining these differences while preserving each method’s specialized mechanisms. Related benchmarks, like RL4CO [6], ML4TSPBench [7], and ML4CO-Bench-101 [8], likewise standardize models and/or data to facilitate comparable evaluation.
-
As detailed in Appendix G, our backbone follows DiffILO, retaining alternating bipartite message passing, graph-wise normalization, and multi-layer readout. We also preserve CoCo-MILP’s competitive layer and contrastive learning, OPTFM’s separate pretraining and downstream prediction pipeline [9], and the newly added optional joint continuous–discrete modeling for FMIP. Appendix H.2 documents method-specific learning rates, objectives, etc., drawing on the original settings. Enforcer defaults similarly follow FMIP’s published configurations for PS, ND, Apollo, and PMVB.
-
For Apollo [10], we adopt the simplified downstream implementation used in FMIP. Our empirical focus places learning at the Learner stage: improving the feasibility and quality of initial variable predictions and examining their value under subsequent solver intervention. Accordingly, we evaluate representative post-processing procedures without separately comparing their additional learning components. This keeps the investigation focused and proportionate to a conference paper. Our guiding principle is to combine a shared architectural foundation with specialized training mechanisms and established method-specific parameter choices.
2. Interpretation of the seemingly anomalous baseline results.
-
ConPaS and CoCo-MILP were developed as prediction–search pipelines in which neural predictions interact closely with a powerful solver. Table 2 evaluates raw learner outputs before solver intervention, while the subsequent enforcer and refiner experiments assess their contribution to feasibility recovery and solution improvement. These stages together characterize each method’s capabilities.
-
For example, with Gurobi-based LB refinement on MIS (Table 4), ConPaS and CoCo-MILP achieve PIs of 0.89 and 0.83, respectively, outperforming DDIM [11] (1.19) and FMIP (1.40) under the same refinement configuration. With the PS enforcer on SC (Table 3), CoCo-MILP achieves a PI of 853.56, compared with 1035.82 for DDIM with the same enforcer and 973.68 for standalone Gurobi. These results demonstrate the value of their predictions when paired with compatible search procedures.
-
This distinction motivates our benchmark. Methods sharing a variable-logit interface can eventually reach similarly small gaps after substantial solver processing, while differing considerably in raw prediction quality, enforcer compatibility, and anytime performance. Our stage-wise evaluation makes these distinct contributions visible.
In general, we value all the contributions of the included methods, and we sincerely hope this explains how our evaluation supports fair comparison while recognizing their different strengths and intended roles.
[1] A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming. ICLR, 2023.
[2] Huang et al. Contrastive Predict-and-Search for Mixed Integer Linear Programs. ICML, 2024.
[3] Differentiable Integer Linear Programming. ICLR, 2025.
[4] CoCo-MILP: Inter-Variable Contrastive and Intra-Constraint Competitive MILP Solution Prediction. AAAI, 2026.
[5] FMIP: Joint Continuous-Integer Flow for Mixed-Integer Linear Programming. ICLR, 2026.
[6] Berto et al. RL4CO: An Extensive Reinforcement Learning for Combinatorial Optimization Benchmark. KDD, 2025.
[7] Unify ML4TSP: Drawing Methodological Principles for TSP and Beyond from Streamlined Design Space of Learning and Search. ICLR, 2025.
[8] ML4CO-Bench-101: Benchmark Machine Learning for Classic Combinatorial Problems on Graphs. NeurIPS, Datasets and Benchmarks Track, 2025.
[9] OPTFM: A Scalable Multi-View Graph Transformer for Hierarchical Pre-Training in Combinatorial Optimization. NeurIPS, 2025.
[10] Apollo-MILP: An Alternating Prediction-Correction Neural Solving Framework for Mixed-Integer Linear Programming. ICLR, 2025.
[11] Effective Generation of Feasible Solutions for Integer Programming via Guided Diffusion. KDD, 2024.
Author Response to Reviewer m7ob (page 1)
Dear Reviewer m7ob,
Thank you for the time and effort reviewing our paper. We appreciate your recognition of the importance of fair comparison, systematic experimentation, and transparent reporting of negative results. Below, we respond point by point to the weaknesses (W) and questions (Q) you have kindly raised, grouping closely related points for clarity.
A: Thank you for raising this important concern. We agree that instance difficulty matters when interpreting the benefits of learned guidance. We would appreciate the opportunity to clarify the benchmark settings and the scope of our conclusions.
1. The number of test instances is distinct from instance size and difficulty.
“100 instances” denotes the test-set size for each problem family, over which performance is aggregated. It does not indicate the number of variables or constraints within an instance (an analogy is the distinction between the number of test images and their resolution). Appendix F, Table 9 reports the actual dimensions:
The selected MIPLIB instances span 214–364,417 variables and 64–188,879 constraints. Moreover, our 1500-variable MIS instances match the graph size designated “Hard” in Gasse et al. [1], although difficulty also depends on instance structure, solver configuration, and computational budget.
2. The benchmark combines established problem families and evaluation conventions.
Gasse et al. [1] study SC/MIS/CA/CFL using 20 instances per difficulty level and five seeds. Guided Diffusion [2] evaluates 100 instances per family on SC/MIS/CA/CFL. Predict-and-Search [3] uses 100 test instances each for MIS/CA/BIP/WA, and DiffILO [4] uses 100 each for SC/MIS/CA. Our selection combines these established families to support comparable evaluation across methods developed under different protocols.
Regarding the Gurobi performance, our results show that the suite includes settings where optimization remains nontrivial under the evaluated budgets: Gurobi retains a mean BKS-relative gap of 0.14% on SC after 1000 seconds, and 26.023% and 6.881% on BIP after 100 and 1000 seconds, respectively.
3. Fast solver performance does not eliminate the challenge of reliable neural prediction.
We acknowledge that Gurobi handles some instances quickly. Our benchmark measures the feasibility and quality of neural predictions before solver intervention. With 1,000 samples, DDIM and FMIP find feasible solutions on only 36% and 50% of CFL instances, respectively, and several learners find none (Table 2). This leaves substantial room to improve neural primal heuristics on established benchmarks.
4. Our goal is to identify effective learned guidance and its interaction with search.
Our analyses explicitly examine both better raw predictions and better solver guidance. For example, Sec. 3.1 identifies the feasibility-quality strengths of generative learners and constraint-aware unsupervised learning, Sec. 3.2 examines why prediction calibration and Learner-Enforcer compatibility matter, and Sec. 3.3 identifies Refiner-dependent learner rankings. On BIP, the learned pipelines reduce primal integral by approximately up to 59% relative to Gurobi, demonstrating substantial benefits within our own evaluation.
Solver-backed search can narrow differences in final solution quality across learners for certain tasks, backends, and budgets. Meaningful differences in anytime performance can persist. Unify ML4TSP [5] reports a related finding. Pre-MCTS solution quality has limited predictive value for post-MCTS quality, and learned heatmaps improve search. These interactions motivate modular evaluation to identify effective learning–search combinations.
Remark. We value the contributions of existing learning-based methods and see considerable potential to improve learned guidance. Our framework supports reproduction and fair, transparent evaluation so researchers can identify each method’s strengths and build on complementary advances.
[1] Exact Combinatorial Optimization with Graph Convolutional Neural Networks. NeurIPS, 2019.
[2] Effective Generation of Feasible Solutions for Integer Programming via Guided Diffusion. KDD, 2024.
[3] A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming. ICLR, 2023.
[4] Differentiable Integer Linear Programming. ICLR, 2025.
[5] Unify ML4TSP: Drawing Methodological Principles for TSP and Beyond from Streamlined Design Space of Learning and Search. ICLR, 2025.