acceptodds
Under review as a conference paper at ICLR 2027

Central Path SPO: An Interior-Projection Based Approach for Solver-Free Decision-Focused Learning

Abstract

Decision-focused learning trains predictive models for the quality of the decisions they induce, but repeated optimization during training limits its scalability. By leveraging concepts from path-following methods in convex programming, and recent advances in constrained machine learning, we formulate a novel framework for solver-free decision-focused learning for linear programs. The framework is based on training a neural network to directly learn the solution of a much easier, smoothed and regularized version of the linear program via first-order methods and an interior projection map. Via Fenchel conjugacy, the learned smooth solution is transformed into a cost prediction, which is used to solve the original linear program at inference. The framework is applicable with any suitably chosen interior projection map and legendre function regularizer. We focus on the case where the interior projection map is a soft-radial projection anchored to the analytic center of the constraint set, and the regularizer is a logarithmic barrier from the classical theory of interior point methods. The resulting method is both fast, and highly performant. We derive theoretical guarantees, and evaluate the method on large scale versions of standard benchmarks, achieving lower regret and order-of- magnitude speed-ups compared with standard approaches.

Primary area:
optimization
Keywords:
DFL, Solver-free, Log-barrier, radial projection, SPO, Decision-Focused Learning, Smart-predict-and-optimize, linear programming

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZyPzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a decision-focused learning framework that eliminates the need for solver calls during training. The method learns an interior policy by minimizing a barrier-regularized linear objective, recovers predicted costs via the gradient of the regularizer, and solves the original optimization problem at inference time. The paper establishes Fisher consistency and regret bounds for linear programs, along with an output-space PL guarantee under sufficient smoothing. Empirical evaluations span four benchmark families.

Critical strengths
  1. The integration of barrier smoothing, interior projection, and cost recovery is elegant and well motivated, resulting in a coherent methodological framework.

  2. The theoretical results provide meaningful justification and support for the proposed approach.

  3. The empirical results demonstrate competitive regret performance and substantial training speedups compared with solver-in-the-loop baselines.

Critical weaknesses
  1. The direct-policy ablation changes multiple components simultaneously, making it difficult to isolate the effects of the training objective, interior parameterization, and cost recovery. More controlled ablations would help clarify their individual contributions.

  2. The PL guarantee requires strong smoothing and applies only to output space, leaving its practical relevance unclear. Moreover, CP-SPO consistently underperforms WISE on random LPs, suggesting that its effectiveness may depend on problem structure.

Questions for authors
  1. Could the authors provide a controlled ablation comparing CP-SPO against cost regression using the same interior parameterization and cost-recovery architecture?

  2. Are the conditions of Theorem 3 satisfied under the experimentally selected smoothing parameters? How does this guarantee relate to the observed training behavior?

1 comment

Reply to Reviewer ZyPz

We would first like to thank the reviewer for their high quality feedback. We were particularly impressed by the suggestion to run CP-SPO with an MSE training objective, to identify the source of the method’s strength.

We have worked hard to address each point raised by the reviewer, and believe that our new results should be convincing.

Issue 1: The ablation studies are insufficient.

In our new experiments, we test running CP-SPO with an MSE objective, through exactly the same interior parameterization and cost recovery and tuned over the same grid. The result is significantly weaker performance across the board: higher regret on 16 of 16 instances and on all 160 paired seeds, with family means of 15.6–21.2% against 5.5–11.3% for CP-SPO.

A second controlled ablation isolates cost recovery. Deploying the learned interior point directly gives regret of 53–100% on the continuous instances; the same weights deployed through cost recovery give 20–41%; CP-SPO gives 7.6–9.0%.

The Hessian whitening and the output rescaling are both necessary for the method to be numerically stable. Without the output rescaling, numerical rounding brings outputs to the boundary of the polytope, causing log-barrier blow-up. Without the Hessian whitening, conditioning becomes very bad, and training times grow intractable.

The new experiments are found in Appendix E. We believe they successfully demonstrate what it is that makes our method perform well.

Issue 2: The practical benefit of the PL theorem is not clear.

This is a fair objection, and it is true that the selected hyperparameters do not satisfy the assumptions of the theorem. However, our tuning grid always includes parameters for which the assumptions actually are satisfied (Appendix C). This is what is important: a PL guarantee gives a shield against a specific form of vanishing-gradient pathology. Our theorem shows that our method can always select parameters for which this shield is in effect (even if in practice, it may prefer other values empirically, just as methods with no PL-guarantees can outperform methods that have them).

We agree that the guarantee holds only in output space, and the revised paper states this explicitly (Section 4.3). A PL guarantee in the network weights is in general not achievable, for our loss or for any other.

Guarantees in output space are nevertheless treated as important in the literature, and for a good reason: they ensure that whatever non-convexity remains in training comes entirely from the neural network. The non-convexity of neural networks is comparatively well understood, and optimizers such as Adam typically handle it well. A non-convex decision loss can behave far more pathologically. Mandi et al. (2025) show, for example, that with quadratic regularization the regret still has zero gradient across large regions of predictions where the regret is positive. No optimizer and no architecture can make progress there. A loss with a PL-guarantee in output space cannot fail in this way. The convexity of SPO+ is valued for the same reason, although it too is a statement about the predicted cost and not about the network weights.

We have also replaced the theorem with one that covers the loss exactly as implemented.

Issue 3: Perhaps there is no benefit compared with WiSE?

Our new, much more extensive experiments cover training sizes from 100 to 1000 pairs, and predictors from affine to three hidden layers, with every method tuned separately in each setting (see the general comment).

Against WiSE, and the other solver-free baselines, CP-SPO generally achieves lower regret. With validation-selected architectures it beats WiSE on 11, 12, 13 and 14 of the 16 instances at 100, 300, 500 and 1000 training pairs, and with an affine predictor for both on 16, 15, 16 and 16. Random LP with 100 training pairs is one of the very few cases where WiSE is able to compete, and there the two are close (mean regret 1.9% against 2.1%). With more data CP-SPO is ahead on random LP as well (0.16% against 0.35% at 1000 pairs).

Reviewer_bsnZOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CP-SPO, a method for decision-focused learning on LPs without solver calls during training. A neural network is trained to directly output the solution of a log-barrier-smoothed LP inside the feasible set through a soft-radial projection, and at inference the cost is recovered from the first-order condition of the barrier problem and used to solve the original problem. The paper provides a regret bound and a PL guarantee in the output space, and reports the lowest regret in 9 of 16 settings across shortest path, knapsack, facility location, and random LP, together with faster training than SPO+ and FY.

Critical strengths

S1. The idea of smoothing the LP with a log barrier, learning its solution directly, and recovering the cost is intuitive, and the motivation of removing the solver from training to gain scalability is clear.

S2. The regret bound, which bounds regret by the training loss without any smoothing bias, is clean.

S3. The experiments include large instances by the standard of the DFL literature, and training time is substantially reduced compared to SPO+ and FY.

Critical weaknesses

W1. In Mandi et al. (2025), from which the authors state they take their benchmark implementations, major DFL methods such as SPO+ and PFY achieve lower regret than the MSE-trained model on the same shortest path benchmark, including the 15x15 and 25x25 grids. In this paper, however, representative DFL baselines such as SPO+ and FY (the same method as PFY) have higher regret than MSE on every shortest path grid. This is very different from the results reported in prior work and raises doubts about whether the experiments were conducted properly. If this difference is due to the prediction model or the training data size, results under the same setting as the original paper should also be reported. In addition, unlike prior work, which uses the same prediction model for all methods, this paper selects the network architecture separately for each method, so it is not possible to tell whether the differences in regret come from the loss function or from the model architecture. The hyperparameter search space for each method is also not disclosed, which makes it difficult to judge whether the baselines were tuned fairly.

W2. The idea of directly predicting the solution of the log-barrier-smoothed problem is good, and the claim that this removes the solver from training and brings scalability is convincing. However, directly predicting the solution requires the prediction to lie inside the feasible set, and what makes this possible is the soft-radial projection from prior work (Schneider & Kuhn, 2026). The concept of using a log barrier in DFL has also already been studied in prior work such as IntOpt (Mandi & Guns, 2020). Recovering the cost from the learned solution is likewise a standard relation that follows directly from the first-order condition of the barrier problem. The proposed method is therefore close to a simple combination of existing components, and I am not convinced that the resulting performance gains or speed-ups are sufficiently attractive compared to existing methods. In fact, it trains more slowly than WISE, another solver-free method, and does not outperform it on every problem. In particular, given the concerns about the rigor of the experiments raised in W1, it is even harder to find the novelty of this paper.

W3. According to the barrier strength settings in the submitted code, the values used for shortest path, random LP, and knapsack appear unlikely to satisfy the sufficient condition of Theorem 3. It should be verified whether this condition holds in the experimental settings.

References

  • Mandi et al. (2025). Minimizing Surrogate Losses for Decision-Focused Learning using Differentiable Optimization. ECAI 2025.
  • Schneider & Kuhn (2026). Soft-Radial Projection for Constrained End-to-End Learning. arXiv:2602.03461.
  • Mandi & Guns (2020). Interior Point Solving for LP-based Prediction+Optimisation. NeurIPS 2020.
  • Elmachtoub & Grigas (2022). Smart "Predict, then Optimize". Management Science 68(1).
Questions for authors

Q1. The degree, noise level, and feature dimension used for data generation cannot be found in either the paper or the submitted code. What are these values? Can you also report results under the same setting as Mandi et al. (2025)?

Q2. Since MSE and SPO+ are also consistent for the conditional mean, can you provide analysis or experiments showing under which conditions (e.g., model misspecification or small training data) the proposed loss is advantageous?

Q3. How do regret and training time compare with IntOpt, which uses the same log-barrier smoothing?

Q4. The hyperparameter search space for each method is not reported. Can you disclose the search space for each baseline?

References

  • Mandi et al. (2025). Minimizing Surrogate Losses for Decision-Focused Learning using Differentiable Optimization. ECAI 2025.
2 comments

Reply to Reviewer bsnZ — part 1 of 2

We would first like to thank the reviewer for a review of very high quality. Though we would of course have hoped for a better assessment, and we disagree somewhat on the novelty objection, it is obvious that the reviewer has made significant effort, and understood the paper very well.

We mostly agree with the reviewer’s objections and have spent considerable effort trying to address them all fully. We hope that our new results will convince the reviewer to change their assessment.

Issue 1: The protocol is not properly disclosed.

We agree that our original submission should have done a much better job of reporting the exact experimental protocol. We now provide code for a complete reproduction, and disclose all tuning protocols, selected parameters, and more.

In short: all methods share the same data, seeds, optimizer (Adam, batch size 32, 300 epochs), learning-rate grid and checkpoint selection. Every method is tuned separately for an affine predictor and for networks with one to three hidden layers, and selects its architecture on validation regret. Results are over ten paired seeds, for 100, 300, 500 and 1000 training pairs. The full protocol and all search spaces are in the general comment and Appendix C.

Issue 2: The baselines seem surprisingly weak.

We assure the reviewer that we use standard implementations of the baselines. FY and NCE use the PyEPO implementation with a HiGHS solver, and our SPO+ implementation gives results identical to the released code of Mandi et al. (see below).

Our protocol also allows methods to select the depth of their neural net architectures as a hyperparameter. The non-method-specific hyperparameters are tuned identically for all baselines.

We additionally run tests across regimes of various training data sizes, and examine what happens when learning is restricted to an affine model for every method (as is common in DFL). This separates the effect of the loss from the effect of the architecture: with the same predictor for every method, CP-SPO has lower regret than SPO+ on 12, 16, 14 and 14 of the 16 instances with zero, one, two and three hidden layers at 100 training pairs (Appendix D).

The unexpectedly weak performance of the baselines compared with MSE in the original table is mostly due to the scarce data, and the deep learning architecture allowing MSE to perform well. Our new, more extensive experiments reveal that this is a regime that is particularly favorable for our method.

We see that with more data, classical DFL methods regain their expected superiority compared with MSE. On shortest path with 1000 training pairs, SPO+ and FY have mean regret 5.0% and 5.8%, against 7.6% for MSE.

To additionally validate the implementation, we also ran the requested experiments of Mandi et al. (2025), on the 15×15 and 25×25 grids, with their released generator, split and code. Our implementations of MSE, SPO+ and FY give identical regret to their released code on every seed. In their setting SPO+ and FY outperform MSE, as they report, and CP-SPO is competitive with both at a fraction of the training time:

GridMetricMSESPO+FYCP-SPO
15×15Realized relative regret (their metric)19.1515.4715.6416.77
Conditional-mean regret (our metric)7.765.796.365.23
Training time (s)0.457.357.61.2
25×25Realized relative regret (their metric)20.7317.7717.7118.06
Conditional-mean regret (our metric)8.416.767.875.72
Training time (s)0.8273.0269.94.6

Regret in %, mean over their five seeds. Details are in Appendix G.

Issue 3: The novelty seems limited.

We agree that both soft-radial projection, and log-barrier regularization, are established techniques. The novelty of our contribution comes from two factors:

Reply to Reviewer bsnZ — part 2 of 2

(1) We propose a framework, not just a specific method. Our trick using regularizer-gradient cost prediction can be combined with a variety of direct-policy learning methods and regularizers, and provides a way for the methods of optimizer learning to be applied to DFL problems. Even if the idea is simple, the regret guarantee we establish is non-trivial. Our hope is that our work will spearhead greater interaction between the two communities.

(2) Our method is more than just a trivial application of soft-radial projection to log-barrier regularized linear problems. Such an application is not numerically stable, and our adjustments serve to address this issue. We introduce an output rescaling and a whitening technique leveraging the Hessian of the log-barrier, to extend the applicability of soft-radial projection to problems with Legendre layers. Both adjustments are necessary for the method to run efficiently. The paper should have done a better job of communicating this. Our direct policy learning method might be called “barrier-adjusted soft-radial projection”, and is arguably an interesting contribution in its own right. Additionally, our theorem shows that it obtains a PL-guarantee, which was a theoretical guarantee that was explicitly noted as being out of reach for standard soft-radial by the original authors.

A new controlled ablation supports this: regression on costs through exactly the same parameterization and cost recovery, tuned over the same grid, has higher regret than CP-SPO on 16 of 16 instances and on all 160 paired seeds (Appendix E). Regarding WiSE: it trains faster, but CP-SPO has the lower regret on 11, 12, 13 and 14 of the 16 instances at 100, 300, 500 and 1000 training pairs.

Issue 4: The selected hyperparameters do not satisfy the assumptions of the theorem.

This is a fair objection. However, our tuning grid always includes parameters for which the assumptions actually are satisfied (Appendix C). This is what is important: a PL guarantee gives a shield against a specific form of vanishing-gradient pathology. Our theorem shows that our method can always select parameters for which this shield is in effect (even if in practice, it may prefer other values empirically, just as methods with no PL-guarantees can outperform methods that have them).

Q1. Shortest path, knapsack and facility location use polynomial degree 6, multiplicative noise of half-width 0.5 and 10 features. Random LPs use the released generator of Berden et al. (2025), with degree 8, 5 features and no noise (Appendix C). The results in the setting of Mandi et al. are given above.

Q2. We add an analysis under misspecification (Appendix B.4): an explicit example where both methods use the same model class, every minimizer of our loss has zero regret, and the MSE minimizer has strictly positive regret. Empirically, the advantage of our loss is largest with small training data (see the general comment).

Q3. We now compare with IntOpt (Appendix F). CP-SPO has lower regret on 15 of 16 instances. IntOpt additionally trains very slowly compared to all other methods, and reaches an imposed one-hour limit on the largest instances.

While we do not comprehensively benchmark IntOpt across all training sizes, we believe its slow training time alone to be a strong argument for the utility of CP-SPO. And the regret table shows that the success of CP-SPO is not merely attributable to its similarity to IntOpt.

Q4. The search space of every method is in Appendix C and in the general comment.

Reviewer_WXaVOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper contributes a new method to solver-free DFL: it combines learning to solve a proxy problem, namely learning to predict a solution which is then projected back into a the feasible space through soft-radial projection; together with a regularized objective/loss that measures the penalized/regularized cost of this proxy solution wrt the true cost vector. At inference time, the proxy solution is mapped back to a meaningful cost vector and a regular LP solve is used.

This is a novel, non-trivial combination that advances the state-of-the-art in solver-free DFL.

Experiments in the data-low regime show benefits; though it does leave questions open about less data-low regimes.

Critical strengths

novel and non-trivial combination of proxy (projection) and a regularized objective that is reversible, so the LP can still be called at inference time

theoretical guarantees and bounds worked out for the two components

Critical weaknesses

The empirical evaluation is only in the 'low data' regime: 100 training pairs. We can see that for many of the instances, SPO+ and FY, two robust methods that are strong baselines in DFL, actually often perform worse than MSE. This means that in these settings, its not clear that its beneficial to do DFL...

While on the one hand that makes it even more impressive that the method is able to outperform MSE on downstream loss; it also raises the question whether the results would still hold in settings where SPO+/FY outperform MSE: does CP-SPO then still offer benefit?

Line 328: the writing becomes difficulty dense "Work in the normalized coordinates... Fix..."

Questions for authors

Please comment on the non-low-data regime question.

Do methods also share the nr of epochs, or is that tuned? (only loss-specific parameter grids mentioned)

Other parts unclear:

  • Table 2: for LAVA, both 'mean' and 'common' have the same value? what does '11 common' mean, those that lava did not time out on?
  • line 450 describes an ablation of 'learning a smooth policy and recovering a cost; rather than direclty deploying the learned policy'; but your Fig 1 claims that learning a smooth policy and recovering a cost IS the main approach, not the direct use of learned policy?
  • line 305, what does 'additive barrier constant is ommitted' relate to?
  • line 322, same for 'unbounded coordinate'? you do clipping, then contraction...
1 comment

Reply to Reviewer WXaV

We thank the reviewer for the careful reading and the constructive questions. We of course also appreciate the reviwer's willingness to go against the grain, and award us such a high score.

The question about the data regime was something we have thought about as well, we were impressed that the reviewer had the same reflection. A great portion of our work since the submission has been focused on addressing it.

Q1. Beyond the low-data regime. We agree this was the main gap, and we have run the full comparison with 300, 500 and 1000 training pairs. In this regime SPO+ and FY do beat MSE, as the reviewer expected. With 1000 pairs and an affine predictor for every method, on shortest path:

Shortest path, N = 1000CP-SPOSPO+FYMSE
10×104.064.295.057.53
15×155.355.176.118.12
20×205.105.205.497.59
25×255.166.247.669.47

With validation-selected architectures, CP-SPO has lower regret than SPO+ on 16, 12, 12 and 11 of the 16 instances at 100, 300, 500 and 1000 pairs, and than FY on 16, 12, 12 and 10. So the results do hold where SPO+ and FY beat MSE, though the margin narrows with more data: on facility location with 300 pairs or more, SPO+ and FY are better. CP-SPO trains in 33 seconds where they take 2,775 and 2,899 (shortest path 25×25, 1000 pairs).

Q2. Epochs. They are shared, not tuned. Every method trains for 300 epochs, and each run selects its checkpoint on validation among epochs 0, 5, 10, 20, 40, 80, 120, 180, 240 and 300.

Q3. LAVA columns. “11 common” were the 11 instances on which every method, LAVA included, had a result. LAVA finished only on those, so its two means were the same number. The revised tables report LAVA on the instances it finishes, and the captions say so.

Q4. Recovery or direct deployment. Cost recovery is the method, as in Figure 1. The sentence at line 450 described the ablation, where the interior point is deployed directly. It is rewritten. A new experiment shows how much recovery matters: on shortest path 25×25, direct deployment gives 100.1% regret, the same weights through recovery 23.9%, and CP-SPO 9.0%.

Q5. The omitted constant. In normalized coordinates the barrier splits into a term that does not depend on the decision and a term that does. The first is the omitted constant. Dropping it changes neither the minimizers, the gradients nor the recovered cost. This is now stated where the normalized barrier is introduced.

Q6 and line 328. “Unbounded coordinate”. The network output is unconstrained. It then passes through three maps: a guard that bounds its norm, a radial clipping onto the feasible set, and a contraction into the interior. “Unbounded coordinate” referred to the raw output before these maps. The passage is rewritten to introduce them one at a time.

Comments on the paper

6 comments

General response (1/5): Summary of the revision — part 1 of 2

To begin with, we would like to express sincere gratitude for the unusually excellent quality of these reviews. Though the reviewers came to very different conclusions regarding the merits of our contribution, it is very evident that each one invested genuine effort in their critique, that they formed their own independent assessment, and that they've understood the work impressively well.

We have worked intensely to address each and every one of the concerns raised. We sincerely think that we have done so successfully, and believe that the skeptical reviewers will change their assessment when presented with our new results. However, even if they do not, we appreciate the privilege of having reviewers who we know will have good reasons for their final assessments.

We would have liked to provide a response sooner, to allow for more of a back-and-forth during the discussion period. But the work required to comprehensively address the critiques, along with exceptional external circumstances, unfortunately prevented this.

The new results we present would have been impossible to complete in the last two weeks alone. Luckily, we ourselves identified several of the areas of improvement raised by the reviewers shortly after the original submission, and could begin the work to address them in the time since.

In what follows, we summarize the new results.

Improved empirical rigor. We provide code required for a full replication of our experiments, along with a detailed tuning schedule, and selected hyperparameters for all methods. Baselines are generally implemented via standard libraries or original author code. We run replication checks on experiments from Mandi et al. (2025) to validate that our implementation is correct.

All shared parameters (e.g. learning rate, neural net depth, etc.) are selected via an identical protocol for all baselines. Method-specific hyperparameters are given generous tuning budgets by the standards of the literature, to ensure a fair comparison.

Extended core experiments. We have significantly extended coverage of the primary experiment. We redo our tests across various regimes, ranging from small to large training data, and simple to complex learning architectures, to give a nuanced picture of when solver-free methods generally, and our method in particular, perform well.

The investigation is very extensive by the standards of the literature. We run over 100,000 fits across tuning and ten-seed validation.

The results indicate that our method is a strong baseline across all regimes. It generally wins against each baseline more often than it loses within all regimes, with few exceptions (discussed below). It is fair to say that it is generally the best performing solver-free method in terms of minimizing regret.

Compared with classical baselines, it generally trains up to one order of magnitude faster on integer benchmarks and one to two orders of magnitude faster on non-integer benchmarks. Conversely, regression-based methods like WiSE and MSE generally train several times faster than our method (typically 2–3x; more on the larger facility location instances), in exchange for weaker performance.

Its relative performance is dominant when data is scarce and deep learning architectures are used (which is a regime where DFL methods have previously been known to struggle to outperform standard regression; Elmachtoub et al., 2023). Since real life applications often involve data scarcity (Chung et al., 2026), this is a valuable niche.

Restricting the depth of the architecture, and increasing training data size, reduces its relative performance to classical baselines, even if it remains competitive. In the least favorable regime, with an affine architecture and large data set, it still leads on three of four shortest-path instances and is close on knapsack, but is outperformed by SPO+ and FY on random LP and facility location. Facility location is the one benchmark where SPO+ and FY pull clearly ahead with more data whatever the architecture; WiSE and LAVA struggle there in the same way (see the results comment).

Improved theory. We strengthen our theorem for the PL-guarantee, and expand on its practical utility.

The benefit of the PL-guarantee is that it ensures our method can always avoid severe vanishing gradient pathologies. We do allow our method to choose parameter values for which the guarantee does not hold during validation, in line with the fact that methods with no vanishing gradient guarantees, such as QPTL, can outperform methods that have such guarantees (such as IntOpt) (Hornewall et al., 2026). As with all theoretical guarantees, the point is to exclude possibilities: our theorem says that everywhere-vanishing gradients will never be an issue that needs to be considered.

General response (1/5): Summary of the revision — part 2 of 2

More rigorous ablation studies. We confirm that both cost-recovery, and decision-focused training, are integral for the performance of our method.

The output rescaling, and Hessian whitening, are required for the method to at all be numerically stable. This together shows that all aspects of the method serve an important purpose.

References

  • Chung, Abdulai, Bayoh, Sandi, Smart, Bastani, Bastani. Improving access to essential medicines via decision-aware machine learning. Nature 653, 1178–1183, 2026.
  • Elmachtoub, Lam, Zhang, Zhao. Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective. arXiv:2304.06833, 2023.
  • Hornewall, Homem-de-Mello, Leclère. Decision-Focused Learning for Scenario Generation in Contextual Stochastic Programming. NeurIPS, 2026.

General response (2/5): Experimental protocol

Benchmarks. Four families, four sizes each: shortest path (10×10 to 25×25 grids), knapsack (50 to 500 items), capacitated facility location (50×5 to 250×10) and random LPs (50 to 400 variables, 50 constraints). Shortest path, knapsack and facility location use the generator of Mandi et al. (2025) with polynomial degree 6 and multiplicative noise of half-width 0.5. Random LPs use the released generator of Berden et al. (2025).

Baselines and sources. FY and NCE: PyEPO 1.0.2 with HiGHS. LAVA: the authors' released code. MSE, SPO+ and WISE: our implementations. In the shortest-path setting of Mandi et al. (2025), our MSE, SPO+ and FY give the same regret as their released code on every seed (Appendix G).

Data. N = 100, 300, 500 and 1000 training pairs, with 32, 96, 160 and 320 validation pairs and 100 test contexts. All methods share the same data and seeds. We report regret normalized by the optimal value, over ten paired seeds, with paired tests.

Architecture. Affine predictor, or a width-64 GELU network with one to three hidden layers. We report both the validation-selected architecture and the affine predictor for all methods.

Optimization. Adam, batch size 32, 300 epochs for every method. Each run selects its checkpoint on its own validation set, among epochs 0, 5, 10, 20, 40, 80, 120, 180, 240 and 300.

Tuning. Hyperparameters are tuned once per benchmark family, training-set size and predictor depth, and never per instance.

Reused across problem sizes. Each family has four problem sizes (for example, shortest path on 10×10 to 25×25 grids). All tuning is done on the smallest one, with a single development seed. The configuration with the lowest validation regret is then used unchanged on the three larger problems of that family. The ten seeds we report are separate from the development seed.

Retuned at every training-set size. The full grid is searched again at each of N = 100, 300, 500 and 1000, for every method.

Retuned at every depth. At N = 100, every method is tuned separately for the affine predictor and for one, two and three hidden layers, and selects the architecture with the lowest validation regret. At N = 300, 500 and 1000, every method is tuned for the affine predictor and for the nonlinear depth that was best for it at N = 100, and again selects the architecture with the lowest validation regret.

Grids. All methods share the learning rates {3e-4, 1e-3, 3e-3, 1e-2}. Each method's own hyperparameters are searched jointly with the learning rate:

  • MSE, SPO+, WISE: none.
  • FY: perturbation scale σ, 7 values from 0.01 to 10.
  • NCE: solve ratio, 7 values from 0.01 to 1.
  • LAVA: threshold, 4 values from 0 to −1.
  • CP-SPO: barrier multiplier (7 values) × guard factor (4) × contraction function (3) × contraction scale (5).

Ablations. N = 100, affine predictor, each arm tuned over the full CP-SPO grid, ten paired seeds.

Baselines with limited coverage. LAVA's pre-processing step proves computationally infeasible on the three larger facility location instances at N = 100 and on all four at larger N, and LAVA is reported on the instances it finishes.

Supplementary material. The supplementary material contains the code to reproduce all experiments, the data generators, the tuning grids and seeds, the selected hyperparameters of every method, per-seed results, training times, and the scripts that build the tables.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.