acceptodds
Under review as a conference paper at ICLR 2027

Learning Through the Hard Problem: Decision-Focused Learning with a Full MILP Combinatorial Layer

Abstract

While real-world applications frequently require solving problems of mixed-integer linear programs (MILPs) with shared structural properties, commercial solvers process each new instance independently from scratch. Graph neural networks (GNNs) offer a promising alternative by predicting optimal binary decisions to guide solvers through variable fixing or warm-starting. However, hard fixing based on thresholded predictions can render the restricted problem infeasible or substantial suboptimal when the GNN is wrong, and training the predictor with a standard classification loss does not account for how prediction errors propagate into decision quality. We investigate three ways of injecting GNN predictions into a MILP solver: hard fixing, big- penalization, and a new bounded linear soft-penalization surrogate. We show that the soft surrogate can be trained end-to-end with decision-focused losses (SPO+, Fenchel-Young) without introducing the discontinuities that make hard-thresholded restrictions unstable. We evaluate this framework on an energy-aware lot-sizing problem, both in deterministic and two-stage stochastic settings, and show that the proposed methodology achieves favorable and tunable trade-offs between computational time and solution quality.

Primary area:
optimization
Keywords:
Decision-focused learning, Graph Neural Network, Mixed-integer linear programming, Combinatorial optimization

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_7sTxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Existing prediction-guided heuristics (hard fixing, penalization) either cause infeasibility when predictions are wrong or respond discontinuously to small changes in predictions near the threshold, making them unsuitable for training the predictor end-to-end within the decision-focused learning (DFL) loop. To address this, the paper introduces a soft guidance term, M⋅∥p^−y∥1M \cdot \|\hat{p} - y\|_1, added to the objective while leaving the feasible region intact. The authors combine the proposed soft-guidance layer with existing DFL losses, including SPO+ and the Fenchel-Young loss, and evaluate cost gap and relative runtime on an energy-aware lot-sizing problem.

Critical strengths
  1. The proposed framework uses the trained model's predictions to guide the solver without modifying the constraints, so feasibility is preserved regardless of prediction quality.

  2. The guidance strength can be tuned at inference time through MM, without any retraining. This design decouples trade-off tuning from training, which could be particularly useful for large-scale problems where training is expensive.

Critical weaknesses
  1. The stated contributions are not sufficiently supported by the experiments. First, the problems and baselines are too limited to support the claim of a "generic, problem-independent" soft-guidance framework. The evaluation covers a single problem, energy-aware lot-sizing, which provides limited evidence about how the method behaves across different problem classes, and there is no validation on broader ML4CO benchmarks spanning diverse problem classes (e.g., Distributional MIPLIB [1]). Moreover, the soft-guidance formulation is closely related to a Lagrangian-relaxation view of the trust-region constraint used by PaS (Han et al., 2023) [2], cited in Appendix A. Yet there is no experimental comparison against trust-region-based ML-guided heuristics (PaS [2], Apollo-MILP [3], DiffILO [4]), so it remains unclear whether the proposed soft guidance offers an advantage over these methods and how it should be positioned in the literature. At the solver-guidance stage, the proposed method is structurally closer to DiffILO, which imposes a trust region on the entire solution, than to PaS, which does so on a subset of variables; this relationship is not discussed. Second, the benefit of integrating the proposed CO layer into the DFL pipeline is not evident, which makes it difficult to count as a contribution. In Section 4.1, SEQ BCE is not clearly distinguishable from the DFL variants: in Figure 1 (left), SEQ BCE and most DFL variants overlap within the reported error bars. The effect of DFL is not clearly demonstrated, and the DFL training cost (solver calls per step, total training time) is not reported, so it is difficult to assess whether the gain justifies the cost. More fundamentally, the main novelty appears to lie in the optimization formulation rather than in the learning methodology, given that the model architecture and DFL losses are largely standard. Therefore, the current evidence does not establish a substantial ML contribution.

  2. Some claims overstate the presented results. First, the claim of a "strictly dominant Pareto front" (Line 371) does not hold. On D1\mathcal{D}_1 in Figure 1(a), the triangle at approximately (0.48 relative run time, 0.1% gap) is not dominated by any proposed configuration (circles): every circle with a shorter run time has a larger gap, while the one with a smaller gap has a larger run time. Hence no circle beats this triangle on both axes. Second, the claim that MM is "interpretable" (Line 84, 236) is weak. MM is an unnormalized absolute quantity, so the same value can mean different things on different problems. Obs. 4 itself provides the evidence: on larger instances, the average cost gap grows much faster with MM than on the training distribution. The authors also acknowledge in Obs. 3 that no result establishes MM as calibrated to the predictor's reliability; the interpretability claim should either be modified or supported by evidence.

[1] Distributional MIPLIB: a Multi-Domain Library for Advancing ML-Guided MILP Methods, arXiv, 2024.

[2] A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming, ICLR, 2023.

[3] Apollo-milp: An alternating prediction-correction neural solving framework for mixed-integer linear programming, ICLR, 2025.

[4] Differentiable Integer Linear Programming, ICLR, 2025.

Questions for authors
  1. The current version omits information needed to interpret the results. Could you provide the following?

    • The DFL training cost (number of epochs, solver calls per step, total training time). Given the claim "cheap enough to solve at every training iteration" at Line 199, this should be reported.
    • The absolute solve time (in seconds or minutes) of all compared methods for each dataset, along with the solver settings (gap tolerance, time limit) and the instance sizes (the number of variables and constraints). Without these, it is hard to judge how difficult the instances are and whether the reported speedups are practically meaningful.
    • What do the error bars in Figure 1 represent (e.g., standard deviation, standard error, or range)?
  2. In Figure 1, what value of MM is used in the CO layer at inference time, and what value during training? Is MM fixed during training?

  3. Are there results comparing the proposed method against the unguided solver (e.g., CPLEX) with a relaxed gap tolerance of roughly 1–5%?

  4. Minor comments

    • Line 120, 180: These two diagrams are unnumbered and uncaptioned, so they cannot be referenced from the text; please add figure numbers and captions.
    • Line 174: The quotation marks around "confident" appear to be mis-typeset.
    • Line 183: F(p^)F(\hat{p}) and c(k,p^)c(k, \hat{p}) in the DFL diagram appear without definition.
    • Line 323: "circular markers" and "triangular markers" are introduced before Figure 1 is referenced, which makes the passage hard to follow.
    • Line 418, 420: The text refers to purple and yellow markers while describing Figure 2a; please check whether these references are correct.
2 comments

Thank you for the valuable feedback regarding baseline connections (PaS, Apollo-MILP, DiffILO) and claim precision.

Baseline Positioning (PaS, Apollo-MILP, DiffILO)

We will update Appendix A to explicitly position soft guidance relative to PaS, Apollo-MILP, and DiffILO:

  • PaS restricts trust regions to dynamically selected subsets of variables.
  • DiffILO and Apollo-MILP impose trust regions or penalty structures across the solution space.
  • Our approach applies a linear soft penalty globally over the entire, unrestricted feasible set, ∑i=1n[(1−p^i) yi+p^i (1−yi)]\sum_{i=1}^{n} \left[(1-\hat{p}_i)\,y_i + \hat{p}_i\,(1-y_i)\right]. This preserves feasibility while allowing complete compatibility with DFL loss surrogates (SPO+, FY).

Experiments on other problem families (e.g., capacitated facility location and unit commitment) and direct comparisons with trust-region methods are planned for a future version of the paper.

Clarification of Overstated Claims

  • "Strictly Dominant Pareto Front" (Line 371): We agree that on dataset D1\mathcal{D}_1, a single high-threshold triangle baseline (≈0.48\approx 0.48 run time, 0.1%0.1\% gap) is not strictly dominated by every circular configuration. We will rewrite this statement in the revision.
  • Interpretability of MM (Lines 84, 236): We agree that MM is unnormalized across instance scales. We will reframe MM strictly as a guidance-strength hyperparameter that controls the quality–speed trade-off at inference time.

Answers to Questions

  • Q1 (Training Cost & Solve Times):
    • Datasets D1\mathcal{D}_1 and DS1\mathcal{DS}_1: Solve times range from 2 to 400 seconds per instance.
    • Datasets D2\mathcal{D}_2 and DS2\mathcal{DS}_2: Evaluated with a time limit of 20 minutes (1200 seconds).
    • Training Duration: SEQ BCE takes ∼5\sim 5 minutes (no solver calls). DFL methods take ∼60\sim 60 minutes (∼50\sim 50 epochs).
  • Q2 (MM during Training vs. Inference): During training, M=100M = 100 is fixed. At inference, MM is swept in [20,400][20, 400] to control speed vs. gap.
  • Q3 (Relaxed Gap Tolerance Baseline): We agree this is an important baseline. In a future version, we will compare against unguided CPLEX with relaxed MIP gap tolerances (1–5%) and time limits, measuring the true gap on the original objective.
  • Minor Comments: Diagrams (lines 120, 180) will be given formal figure numbers/captions, quote marks fixed, and color references in Figure 2 verified.

Thank you for the clarifications. While several points are better explained, my main concerns remain unresolved without additional experimental evidence addressing the issues raised in my review. Therefore, my overall assessment remains unchanged.

Reviewer_StP3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper tackles mixed integer linear programs (MILPs) by injecting predictions from a graph neural network (GNN) into a MILP solver. The GNN predicts an approximate solution in the form of probabilities for each binary variable. These are then used to augment the MILP objective with a term that softly penalizes deviation from the approximate solution. The pipeline is trained end-to-end using decision-focused learning (DFL) techniques. The paper includes a mathematical analysis and an empirical evaluation on an energy-aware multi-product lot-sizing problem, including a two-stage stochastic extension.

Critical strengths
  • S1: Clear presentation and structure. The paper is easy to read and to understand.
  • S2: The proposed approach is simple and well-motivated.
  • S3: The authors provide a mathematical analysis of their approach, giving a better understanding of its properties.
Critical weaknesses

Major weakness (W1):

The experiments in section 4.1 do not demonstrate that the soft penalization approach performs better than the simple thresholding baseline (this also means that the claim made in observation 1 is not supported by the experiments). While figure 1 shows that the soft penalization variants are faster, the solutions they produce generally have a larger cost gap. In other words, they trade off solution quality for a faster run time. For example, on dataset D1\mathcal{D}_1, the "thresh 0.70" baseline already achieves a cost gap of 0 (or very close to 0). The soft penalization variants are only 30% to 70% faster with a larger cost gap. Whether it is worth worsening solution quality for a faster run time depends on how long one solve takes. The paper doesn't mention how long a relative run time of 1 is in absolute terms. If this run time is very long (e.g. minutes or hours), it could be argued that the tradeoff is worth it. On the other hand, if the run time is fast in absolute terms (e.g. seconds or milliseconds), it would be hard to argue that saving such a small amount of time is worth it if it leads to a worse solution. However, if the run time is slow, training would take prohibitively long, as the DFL variants are trained end-to-end. This is my main concern blocking acceptance. (See Q1 and Q2.)

Note: The proposed method already contains a parameter (MM) that allows adjusting the tradeoff between run time and cost gap. For this reason, an effective way to address this main weakness would be to adjust MM such that either the average run time or the average cost gap aligns with a particular baseline (ideally, this would be done for multiple baselines/threshold settings). If the proposed method variants find lower-cost solutions in the same run time or require a shorter run time to reach a solution of the same cost, this would convincingly demonstrate that they outperform the baselines in a fair comparison. (See Q3.) The authors vary MM in their experiments in section 4.2, but these omit the baselines.

Minor weaknesses:

  • W2: The paper claims that the proposed approach is generic, but only evaluates it on one problem family. While the method is theoretically applicable to other problem families, the experiments do not demonstrate that it works well outside of the single-machine energy-aware lot-sizing problem used in the paper.
  • W3: The paper does not discuss the limitations of the proposed method.

These additional, very minor points are intended to improve the paper and don't play a part in my decision assessment:

  • W4: In the figure in lines 120-124, φω\varphi_\omega is labeled as "MLP", whereas the text below describes it as a GNN.
  • W5: Presentation of results figures. Some of the figures have very small text, especially figures 4 and 5. In figure 1, the colors of some of the icons are so close together that they become difficult to distinguish (this might be even more difficult for readers with impaired color vision, which affects roughly 5% of the population).
  • W6: Please re-check the manuscript for typos. (In line 186, there is a space missing before the parentheses: "layer(CO layer)". In the AI use statement, there is a superfluous period at the end of the first paragraph: "results. and".)
  • W7: Appendix D details the MILP used in the experiments, but in my opinion the reader would benefit from at least an intuitive understanding of the problem going into the case study section. For example, this could be achieved by moving appendix D.1 to the main paper (a shorter summary may also suffice) and placing it near the beginning of section 4, alongside a brief explanation of the two-stage stochastic case.

I currently lean towards rejection as the experiments do not convincingly show that the proposed approach improves over the simple thresholding baseline at a comparable compute budget. However, I am open to increasing my score if W1 is fully addressed.

Questions for authors
  • Q1: In figures 1, 2, and 3, how long is a relative run time of 1 in absolute terms? Please also report the hardware on which the timings are taken and, if relevant, the number of CPU cores used.
  • Q2: How long does it take to train the DFL variants? How many training iterations/solver calls does it take? How does the training time compare to the non-DFL methods?
  • Q3: How do the soft penalization variants compare to the baselines in a fair comparison, where the methods are adjusted to achieve very similar average run times or very similar average cost gaps?
  • Q4: Line 336 states that "solid horizontal and vertical error bar lines depict variability across instances", referring to figure 1. What measure of variability is used here?
  • Q5: Lines 328-329 mention the variants DFL SPO+(MA3/MA5). How exactly do these work? The lines I mentioned only touch on this very briefly.
2 comments

Thank you for the detailed review, the actionable suggestion on comparing Pareto frontiers, and the clear questions.

Major Weakness (W1) & Q3: Fair Comparison at Matched Runtimes and Matched Cost Gaps

The reviewer raises an essential point: comparing fixed configurations without sweeping parameters across identical performance targets can mask true Pareto dominance. We agree that this is the right way to compare. In a future version, we will run a full matched comparison (equal average run time and equal average cost gap) by sweeping M∈[10,500]M \in [10, 500] for soft guidance and threshold cutoffs τ∈[0.50,0.99]\tau \in [0.50, 0.99] for the thresholding baselines.

Q1 & Q2: Absolute Solve Times, Hardware Setup, and DFL Training Overhead

Absolute Runtimes (Q1):

  • Datasets D1\mathcal{D}_1 and DS1\mathcal{DS}_1: Solve times range from 2 to 400 seconds per instance.
  • Datasets D2\mathcal{D}_2 and DS2\mathcal{DS}_2: Evaluated with a time limit of 20 minutes (1200 seconds).
  • Hardware & Thread Count: The implementation was done in Python using the mathematical solver CPLEX 20.10. The computational experiments were carried out on a laptop running Linux with an Intel(R) Core(TM) i7-1165G7 CPU @ 2.80GHz processor and 32 GB RAM.

DFL Training Time and Amortization (Q2):

  • Training Duration: SEQ BCE takes ∼5\sim 5 minutes (no solver calls). DFL methods take ∼60\sim 60 minutes (∼50\sim 50 epochs).
  • Amortization & Practical Value: Training is a one-time offline cost. Models trained on small instances (D1\mathcal{D}_1) transfer directly to hard instances (D2\mathcal{D}_2, DS2\mathcal{DS}_2), saving hundreds of seconds per online solve. On DS2\mathcal{DS}_2, saving 50%50\% runtime per instance saves ∼10\sim 10 minutes per call, fully amortizing training overhead after just a few dozen test solves.

Minor Weaknesses (W2–W7) and Questions Q4–Q5

  • W2 (Scope & Genericity): We will revise the abstract and introduction to clarify that empirical claims are validated on energy-aware lot-sizing MILPs and their stochastic extension, and we plan experiments on other problem families, such as the capacitated facility location problem and the unit commitment problem, in a future version.
  • W3 (Limitations Section): We will add a dedicated Limitations section detailing: (i) the offline training overhead of full-MILP DFL, (ii) the initial scale selection of MM, and (iii) reduced relative speedup on loosely coupled problems.
  • W4 (Architecture Diagram Label): We will fix the block label in lines 120–124 from "MLP" to "GNN".
  • W5 (Figure Presentation & Accessibility): Figures 1, 4, and 5 will be updated with enlarged text, colorblind-safe palettes, and distinct marker shapes (circles, squares, triangles, diamonds).
  • W6 (Typos): We will fix "layer (CO layer)" on line 186, remove the extraneous period in the AI statement, and re-proofread the entire text.
  • W7 (Intuitive Problem Overview): We will move the intuitive problem description from Appendix D.1 into the opening of Section 4, alongside a brief overview of the two-stage stochastic recourse formulation.
  • Q4 (Error Bar Definition): Solid error bars in Figure 1 represent the interquartile range (IQR, 25th to 75th percentiles) across test instances, chosen to capture right-skewed solver runtimes. This definition will be added to Section 4.1 and the caption.
  • Q5 (DFL SPO+(MA3/MA5) Explanation): MA3 and MA5 denote temporal moving-average target smoothing across adjacent time periods during training; e.g., MA3 corresponds to Y~j,r(3)=13(Yj,r−1+Yj,r+Yj,r+1).\tilde{Y}_{j,r}^{(3)} = \frac{1}{3}\left(Y_{j,r-1} + Y_{j,r} + Y_{j,r+1}\right). In multi-period lot-sizing, setup decisions often exhibit symmetry or minor timing shifts across adjacent periods with nearly identical objective costs. Smoothing the target labels across a 3- or 5-period window prevents the subgradient updates from over-penalizing slight temporal shifts (e.g., triggering a setup at r−1r-1 instead of rr), creating a smoother training signal and reducing prediction variance. We will add this explicit mathematical definition to Section 4.1.

Thank you for the reply. However, without seeing the results of the matched comparison at equal run time or cost gap I cannot justify raising the score, as my main concern remains unaddressed. I maintain my rating of 2.

Reviewer_SSchOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies decision-focused learning for mixed-integer linear programs using a soft-guidance combinatorial layer. A GNN predicts probabilities for binary decisions, and the solver minimizes the original objective plus M times the L1 disagreement between the prediction and those decisions. For binary variables, this penalty is a bounded, prediction-dependent linear objective perturbation. The formulation preserves the original feasible region, and the guidance strength M can be changed at inference time. The paper analyzes the quality-versus-guidance trade-off, trains predictors with SPO+ and perturbed Fenchel-Young surrogate gradients, and evaluates deterministic and two-stage stochastic energy-aware lot-sizing problems, including transfer to larger instances.

Critical strengths
  1. Preserving the complete feasible region is a useful practical property. Incorrect predictions cannot themselves make an otherwise feasible problem infeasible, as can happen with hard variable fixing. The bounded probability-dependent coefficients also give a simple, interpretable way to guide the solver without making irreversible decisions.

  2. The ability to vary guidance strength at inference time is useful for deployment. The lot-sizing experiments examine solution quality and runtime in both deterministic and stochastic settings, as well as transfer to larger instances. This provides a meaningful application-level study of the trade-off controlled by M.

Critical weaknesses
  1. The differentiability claims need more precise framing, and the stated Fenchel-Young loss does not appear to yield the claimed gradient. Although the objective coefficients depend continuously and linearly on the predicted probabilities, the MILP argmin mapping remains piecewise constant and generally non-differentiable, as Section 3.2 itself acknowledges. The contribution is therefore the removal of an additional threshold discontinuity and the construction of a cost parameterization usable with existing DFL surrogates. More substantively, Eq. (9) in Appendix C.2 subtracts the expected perturbed optimal value from z_Soft^(bar p; M), which is constant with respect to the prediction. Writing the value function as min_{x,y} f(x,y)+M 1^T p+M(1-2p)^T y, its derivative away from tie points is M 1-2M y_p^. Consequently, differentiating Eq. (9) does not produce the target term y^* in the cost-gradient y^*-hat y_epsilon stated in Eq. (10). A per-group softmax can remove constant all-ones directions during backpropagation, but it cannot create the missing target-dependent term. Since DFL-FY is evaluated experimentally, this is a material correctness and reproducibility concern rather than a cosmetic notation issue.

  2. The methodological advance and empirical scope are not yet strong enough to support the broader positioning. The penalty is exactly M||hat p-y||_1 and becomes a linear objective perturbation for binary y; the accompanying bounds and monotonicity properties largely follow from standard weighted-sum and parametric-optimization arguments. The paper's related-work discussion acknowledges prior use of full integer-program oracles in DFL and earlier learned guidance beyond irreversible fixing. The potentially useful contribution is the particular bounded probability-based perturbation, its integration with full-MILP DFL, and inference-time adjustment of M. Its value should be demonstrated more sharply against those alternatives. Moreover, all main experiments remain within one energy-aware lot-sizing family. The stochastic extension and larger-instance transfer are useful, but do not establish effectiveness across substantially different MILP structures. This limited evidence matters because the paper emphasizes generic, problem-independent applicability.

Questions for authors
  1. Is Eq. (9) missing the usual target-linear term of a Fenchel-Young loss, or does the implementation optimize a different objective? Please give the complete implemented scalar loss and derive its gradients with respect to both the predicted costs and probabilities, including the factor -2M and the effect of softmax. Would correcting the discrepancy change the DFL-FY results?

  2. Can you state explicitly which mapping is smooth and which remains non-differentiable, and revise the claim that soft guidance makes the full CO layer smooth enough to train through? This clarification is needed to distinguish the new parameterization from the existing surrogate-gradient machinery.

  3. Relative to full-integer-oracle DFL and learned guidance methods such as predict-and-search, local branching, and prediction-correction approaches, what is the principal new capability? Which controlled comparison most directly demonstrates its benefit?

  4. Do you have results on a structurally different MILP family that would support the broad effectiveness claim? If not, would you narrow the empirical claim to a generic formulation demonstrated on energy-aware lot sizing? Evaluation on another family would be valuable future work, but is not a substitute for resolving the loss/gradient discrepancy in the current submission.

2 comments

Thank you for the incisive theoretical critique and constructive feedback regarding differentiability framing and empirical scope.

W1 & Q1: Fenchel-Young Loss Scalar Form, Gradients, and Softmax Effect

The reviewer is right: Equation (9) misses the target-linear term, and the softmax cannot create it. We correct it in the revision. With predicted costs c^p=M(1−2p^)\hat{c}_p = M(1-2\hat{p}), define

\qquad \hat{y}_\epsilon(c) = \mathbb{E}_Z\left[ y^*(c + \epsilon Z) \right],$$ so that $z^*_{\mathrm{Soft}}(\hat p; M) = M\mathbf{1}^{\top}\hat p + v(\hat c_p)$ by Equation (5). The complete implemented scalar loss is $$\mathcal{L}_{\mathrm{FY}}(\hat{c}_p; y^*) = \langle \hat{c}_p, y^* \rangle - \mathbb{E}_Z\left[ v(\hat{c}_p + \epsilon Z) \right] + \Omega_\epsilon(y^*),$$ where $\Omega_\epsilon(y^*)$ is constant in the prediction. Its gradient with respect to the predicted costs is exactly Equation (10): $$\nabla_{\hat{c}_p} \mathcal{L}_{\mathrm{FY}} = y^* - \hat{y}_\epsilon(\hat{c}_p).$$ We write the loss with $v$ rather than $z^*_{\mathrm{Soft}}$: as the reviewer notes, the term $M\mathbf{1}^{\top}\hat p = \tfrac{1}{2}(Mn - \mathbf{1}^{\top}\hat c_p)$ would add a constant $\tfrac{1}{2}\mathbf{1}$ to the cost gradient. The perturbation $\hat p + \epsilon Z$ in the paper corresponds to perturbing the costs by $-2M\epsilon Z$. **Gradient with respect to the probabilities and the logits.** Since $\partial \hat{c}_p / \partial \hat{p} = -2MI$, $$\nabla_{\hat{p}} \mathcal{L}_{\mathrm{FY}} = -2M \left( y^* - \hat{y}_\epsilon(\hat{c}_p) \right).$$ In the case study, $\hat{p}_r = \sigma(h_r)$ is a per-period softmax of the GNN logits $h_r$, so $$\nabla_{h_r} \mathcal{L}_{\mathrm{FY}} = J_\sigma(h_r)^{\top} \left( -2M \left( y_r^* - \hat{y}_{\epsilon,r} \right) \right), \qquad J_\sigma(h_r) = \mathrm{diag}(\hat p_r) - \hat p_r \hat p_r^{\top}.$$ Since $J_\sigma(h_r)^{\top}\mathbf{1} = 0$, the softmax removes constant directions per period, so the extra $\tfrac{1}{2}\mathbf{1}$ would not have affected training; the corrected loss no longer relies on this. Our implementation used Equation (10) via a custom PyTorch gradient operator, so the reported DFL-FY results are unaffected. Appendix C.2 will be updated in the revision. ## W1 & Q2: Precise Framing of Differentiability and Smoothness Claims - The MILP argmin mapping $\hat{p} \mapsto y^*(\hat{c}_p)$, with $y^*(c) \in \arg\min_{(x,y)\in\mathcal{F}} f(x,y) + c^{\top}y$, remains piecewise constant: its derivative is zero almost everywhere and undefined where the optimal solution changes, so it provides no useful gradient by itself. - Soft guidance replaces non-differentiable hard-fixing thresholds with a continuous, linear cost parameterization $\hat{c}_p = M(1-2\hat{p})$. This parameterization allows the CO layer to interface directly with existing decision-focused surrogate gradient frameworks: through the subgradient-based convex surrogate loss in SPO+, or through expectation-based perturbation smoothing in Fenchel-Young (FY). We will revise Section 3.2 to make this explicit distinction clear. ## W2 & Q3: Principal New Capability vs. Local Heuristics (Proximity Search / Neural Diving) Our principal contribution lies in its global, non-restrictive architecture. While methods like weighted proximity search, local branching, and neural diving use distance penalties or hard restrictions, they typically operate sequentially or myopically on local neighborhoods or subsets of variables. Instead of cutting off portions of the search space, our framework differs by acting globally: it introduces a problem-independent linear soft penalization over the entire, unrestricted feasible set, $$\sum_{i=1}^{n} \left[(1-\hat{p}_i)\,y_i + \hat{p}_i\,(1-y_i)\right].$$ The original feasible region remains completely intact, ensuring that the solver can always violate soft preferences whenever required by complex coupling constraints (e.g., inventory and energy balances). ## W2 & Q4: Scope of Empirical Evaluation and Generalization Claims We acknowledge the reviewer's observation that evaluating on energy-aware lot-sizing (both deterministic and two-stage stochastic) does not automatically prove cross-domain transfer to all MILP classes. In the revision, we will narrow the empirical claim to a generic formulation demonstrated on energy-aware lot-sizing. Evaluations on structurally distinct MILP families (e.g., capacitated facility location and unit commitment) are planned for a future version of the paper.

The corrected loss derivation and clarification of the MILP solution map address this original concern. Nevertheless, I retain my score of 2 because the evidence for incremental benefit over existing guidance methods and simpler supervised training remains limited. The remaining issue concerns whether the implementation matches the corrected derivation and whether decision-focused training provides sufficient additional value to justify its computational cost.

Reviewer_RRpsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how GNN predictions of binary decisions can guide the solution of MILPs with recurring structure. Instead of fixing predicted variables or applying threshold-based penalties, the authors introduce bounded linear penalties determined by predicted probabilities while preserving the original feasible region, and train the predictor using surrogate losses such as SPO+ and perturbed FY. The paper provides bounds and monotonicity analysis relating guidance strength to degradation in the original objective. Experiments on deterministic and two-stage stochastic versions of energy-aware lot-sizing demonstrate adjustable trade-offs between solution time and solution quality, as well as transfer across problem sizes. The core soft-guidance mechanism has practical value.

Critical strengths

S1. The soft-guidance mechanism is simple, preserves the original feasible region, and has a clear interpretation in terms of solution quality.

The L1 penalty on binary variables can be expressed directly as a linear objective term without changing the original constraints. Compared with hard fixing, the solver retains the freedom to correct predictions; compared with threshold-based penalties, the coefficients vary continuously with the predicted probabilities. Pages 5–6 further relate guidance strength to the loss in the original objective and account for additive solution error in the augmented objective. This gives the method clear practical value and makes it straightforward to implement.

S2. The experiments do explore multiple ways of using the method and trade-offs between quality and time.

Figure 1 includes BCE, SPO+, Hybrid, FY, and label-smoothing variants; Figures 2–3 sweep MM; Appendix F compares linear and log-odds penalties; and Appendix G compares hard fixing and threshold-based penalties. Tests on deterministic, stochastic, and larger instances provide preliminary evidence of transfer. The paper therefore contains practical results worth discussing.

Critical weaknesses

W1. The FY loss definition is inconsistent with the stated gradient, affecting the correctness and reproducibility of the method.

Equation (9) in Appendix C.2 defines the FY loss as

LFY(p^,pˉ)=zSoft∗(pˉ;M)−EZ[zSoft∗(p^+ϵZ;M)],pˉ=y∗.L_{\mathrm{FY}}(\hat{p},\bar{p}) = z_{\mathrm{Soft}}^*(\bar{p};M) - \mathbb{E}_Z[z_{\mathrm{Soft}}^*(\hat{p}+\epsilon Z;M)], \bar{p}=y^*.

However, Equation (9) does not yield the gradient given in Equation (10). Fix M>0M>0, let c^p=M(1−2p^)\hat c_p=M(\mathbf1-2\hat p), and define

v(c)=min⁡(x,y)∈F{f(x,y)+c⊤y}.v(c)=\min_{(x,y)\in\mathcal F}\{f(x,y)+c^\top y\}.

By Equation (5), the optimal value can be written as

=M\mathbf1^\top\hat p+v(\hat c_p) =\frac{Mn}{2}-\frac12\mathbf1^\top\hat c_p+v(\hat c_p).$$ Differentiating with respect to the predicted costs, under conditions that allow differentiation and expectation to be interchanged, gives a gradient for Equation (9) that differs from Equation (10): $$\nabla_{\hat{c}_p}L_{\mathrm{FY}} = \frac{1}{2}\mathbf{1} - \mathbb{E}_Z[y_\epsilon] \ne y^* - \mathbb{E}_Z[y_\epsilon].$$ where $y_\epsilon$ is the optimal binary decision under the perturbed input $\hat p+\epsilon Z$. In Equation (9), the target label $y^*$ appears only in the first term. That term does not vary with the prediction, so differentiating it cannot produce the $y^*$ term in Equation (10). A grouped softmax ensures that the predicted probabilities within a group sum to 1, but it cannot supply the target-label term missing from the loss. FY is one of the training methods compared in the main text, so this error affects the interpretation of the experimental results. The authors need to make the loss and gradient consistent, state which loss or gradient was used in the experiments, and explain whether the correction changes the results. At present, it cannot simply be treated as a typographical error with no effect on the results. (p. 6, §3.2; p. 15, Appendix C.2) If the experiments used the correct gradient, correcting the manuscript may not change the results. If training used Equation (9), the FY results and related comparisons need to be re-examined. This error alone does not invalidate the independent SPO+ and BCE results or the main theoretical conclusions. Section 5.1 provides a detailed derivation and counterexample. **W2. The evaluation of computational efficiency is insufficient to support the central claim that a full MILP layer is suitable for practical DFL training.** The paper emphasizes that each training update requires solving a full MILP, but does not report the total training time. Readers therefore cannot assess how much additional time this training requires or whether the eventual benefits justify the cost. The inference-time comparison also raises concerns. The paper uses unguided exact optimization as the reference, whereas soft guidance allows degradation in the original objective, so the two methods operate under different solution requirements. The paper should compare solution quality under the same time budget, or runtime at the same solution quality. It should also compare against methods such as early termination of the original solver and warm starts based on network predictions. The current results demonstrate a speed–quality trade-off, but do not yet establish that the proposed method outperforms these simpler approaches. (p. 4; pp. 7–9; Appendix E) **W3. The contribution beyond prior work and the necessity of DFL training have not been sufficiently established.** Using full integer optimization problems in DFL training has precedents, as the paper itself acknowledges in the appendix. Methods such as proximity search have also used distance penalties to guide integer programming search. The paper needs to clarify what is added, and what benefits are obtained, by converting predicted probabilities into soft penalties and incorporating DFL training relative to these existing methods. [Prior work using full integer optimization problems in DFL training](https://cdn.aaai.org/ojs/5521/5521-13-8746-1-10-20200512.pdf); [Weighted proximity search](https://link.springer.com/article/10.1007/s10732-021-09466-0). (p. 13, Appendix A) In Figure 1, SEQ BCE with the same soft guidance is already competitive on some test sets, and DFL does not show a consistent advantage. It is therefore still unclear whether the main benefits come from the soft penalty itself or from training through the full MILP. (p. 7, Figure 1) Weighted proximity search also has variants with bounded weights that allow the improvement requirement to be relaxed. Thus, being “bounded” or “preserving the feasible region” is not sufficient on its own to establish novelty. There are still differences between the prior methods and this paper in their reference solutions, objectives, and search procedures, so they should not simply be considered equivalent. Nevertheless, the authors need to demonstrate what additional practical value GNN predictions and DFL training provide. **W4. The paper contains several spelling, grammatical, and equation-reference errors that require further proofreading.** - Page 1, abstract, line 18: “substantial suboptimal” → “substantially suboptimal”. - Page 6, lines 300–301: “the parameters of the GNN is updated” → “the parameters of the GNN are updated”. - Page 13, line 680: “Smart Predict-than-Optimize” → “Smart Predict-then-Optimize”. - Page 16, lines 858–859: “Equations equation 12” → “Constraints (12)”.
Questions for authors

Please refer primarily to the weaknesses above.

  1. Clarification of the implementation: Did the FY experiments use the loss in Equation (9), or directly use the manually specified gradient in Equation (10)? Please provide a consistent loss, derivation, and implementation, and explain whether the correction affects the results.
  2. Evidence of practical feasibility: Were the MILPs solved to optimality during training? Please report the total costs of label generation and training, the number of oracle calls per update, the number of FY samples, the stopping criteria, and the computational cost relative to BCE.
  3. The independent benefit of DFL and the reported statistics: With the soft-guidance formulation, architecture, data, and MM held fixed, what are the paired results across multiple random seeds for SPO+/FY versus BCE? Do these results support the paper's claims of strict Pareto superiority and tail stability?

I will reconsider my scores if these questions can be addressed.

1 comment

Thank you for the constructive feedback regarding theoretical guarantees and empirical evaluation metrics.

W1 & Q1: Mathematical Formulation and Implementation of Fenchel-Young (FY) Loss

Implementation Verification: We confirm that our code used only the gradient formula in Equation (10) (∇c^pLFY=y∗−y^ϵ\nabla_{\hat{c}_p}\mathcal{L}_{\mathrm{FY}} = y^* - \hat{y}_\epsilon), using custom gradient operators from PyTorch, rather than differentiating Equation (9) directly via automatic differentiation.

Theoretical Discrepancy & Correction: The reviewer is correct: Equation (9) as written omits the target-dependent linear term, and its first term zSoft∗(pˉ;M)z^*_{\mathrm{Soft}}(\bar p; M) is constant in the prediction, so differentiating it cannot produce y∗y^*. We correct it following the perturbed Fenchel-Young framework (Berthet et al., 2020), written in the space of the predicted costs c^p=M(1−2p^)\hat{c}_p = M(1-2\hat{p}). Define the value function and the perturbed solution

\qquad \hat{y}_\epsilon(c) = \mathbb{E}_Z\left[ y^*(c + \epsilon Z) \right],$$ where $y^*(c)$ is an optimal $y$ for cost $c$. By Equation (5), $z^*_{\mathrm{Soft}}(\hat p; M) = M\mathbf{1}^{\top}\hat p + v(\hat c_p)$. The corrected loss is $$\mathcal{L}_{\mathrm{FY}}(\hat{c}_p; y^*) = \langle \hat{c}_p, y^* \rangle - \mathbb{E}_Z\left[ v(\hat{c}_p + \epsilon Z) \right] + \Omega_\epsilon(y^*),$$ where $\Omega_\epsilon(y^*)$ does not depend on the prediction and only ensures $\mathcal{L}_{\mathrm{FY}} \ge 0$. Since $\nabla_c\, \mathbb{E}_Z[v(c+\epsilon Z)] = \hat{y}_\epsilon(c)$, this gives exactly Equation (10): $$\nabla_{\hat{c}_p} \mathcal{L}_{\mathrm{FY}} = y^* - \hat{y}_\epsilon(\hat{c}_p).$$ We use $v$ rather than $z^*_{\mathrm{Soft}}$ on purpose: the extra term $M\mathbf{1}^{\top}\hat p = \tfrac{1}{2}(Mn - \mathbf{1}^{\top}\hat c_p)$ would add $\tfrac{1}{2}\mathbf{1}$ to the gradient. In our case study this constant direction is removed by the per-period softmax, but with the formulation above Equation (10) holds for any output activation. The perturbation $\hat p + \epsilon Z$ written in the paper corresponds to perturbing the costs by $-2M\epsilon Z$; we will state the noise distribution, $\epsilon$, and the number of samples explicitly. **Impact on Results:** Because our experimental pipeline directly evaluated the gradient using the correct perturbation-based expectation step (Equation 10), the experimental results, figures, and comparative conclusions for DFL-FY reported in the paper are completely accurate and remain unaffected. We will update Appendix C.2 with this complete scalar loss and derivation in the revision. ## W2 & Q2: Practical Feasibility and Computational Training Costs During training, the combinatorial layer sub-problems were solved using CPLEX with a tight time limit per solve (or solved to exact optimality for small training instances). The time for label generation and training under different pipelines will be reported in the revision. While DFL training incurs more computational cost, this cost is offline. At inference time, the trained model generates guidance vectors in milliseconds, delivering $2\times$ to $10\times$ solver speedups per solve. **Warm-Starting Comparison:** We agree that a MIP start built from the same predictions is a natural baseline. In a future version of the paper, we will compare soft guidance with a CPLEX MIP start from the rounded predictions (full and partial assignments), at equal time budget and at equal cost gap. ## W3 & Q3: Novelty Beyond Proximity Search and Independent DFL Advantage While methods like weighted proximity search, local branching, and neural diving use distance penalties or hard restrictions, they typically operate sequentially or myopically on local neighborhoods or subsets of variables. Instead of cutting off portions of the search space, our framework differs by acting globally: it introduces a problem-independent linear soft penalization over the entire, unrestricted feasible set, $$\sum_{i=1}^{n} \left[(1-\hat{p}_i)\,y_i + \hat{p}_i\,(1-y_i)\right].$$ The original feasible region remains completely intact, ensuring that the solver can always violate soft preferences whenever required by complex coupling constraints (e.g., inventory and energy balances). The reviewer correctly notes that in Figure 1, SEQ BCE combined with the same soft-guidance mechanism achieves competitive performance on in-distribution test sets ($\mathcal{D}_1$). Our current results suggest that DFL mainly improves stability on the out-of-distribution sets ($\mathcal{D}_2$, $\mathcal{DS}_2$), but we agree that a single run per method does not establish this. In a future version, we will test this with paired multi-seed runs (BCE vs. SPO+/FY with the same architecture, data, and $M$). ## W4: Formatting and Syntax Corrections All noted corrections (Lines 18, 300–301, 680, 859) will be fixed in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.