- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a decision-focused learning framework that eliminates the need for solver calls during training. The method learns an interior policy by minimizing a barrier-regularized linear objective, recovers predicted costs via the gradient of the regularizer, and solves the original optimization problem at inference time. The paper establishes Fisher consistency and regret bounds for linear programs, along with an output-space PL guarantee under sufficient smoothing. Empirical evaluations span four benchmark families.
Critical strengths
-
The integration of barrier smoothing, interior projection, and cost recovery is elegant and well motivated, resulting in a coherent methodological framework.
-
The theoretical results provide meaningful justification and support for the proposed approach.
-
The empirical results demonstrate competitive regret performance and substantial training speedups compared with solver-in-the-loop baselines.
Critical weaknesses
-
The direct-policy ablation changes multiple components simultaneously, making it difficult to isolate the effects of the training objective, interior parameterization, and cost recovery. More controlled ablations would help clarify their individual contributions.
-
The PL guarantee requires strong smoothing and applies only to output space, leaving its practical relevance unclear. Moreover, CP-SPO consistently underperforms WISE on random LPs, suggesting that its effectiveness may depend on problem structure.
Questions for authors
-
Could the authors provide a controlled ablation comparing CP-SPO against cost regression using the same interior parameterization and cost-recovery architecture?
-
Are the conditions of Theorem 3 satisfied under the experimentally selected smoothing parameters? How does this guarantee relate to the observed training behavior?
Reply to Reviewer ZyPz
We would first like to thank the reviewer for their high quality feedback. We were particularly impressed by the suggestion to run CP-SPO with an MSE training objective, to identify the source of the method’s strength.
We have worked hard to address each point raised by the reviewer, and believe that our new results should be convincing.
Issue 1: The ablation studies are insufficient.
In our new experiments, we test running CP-SPO with an MSE objective, through exactly the same interior parameterization and cost recovery and tuned over the same grid. The result is significantly weaker performance across the board: higher regret on 16 of 16 instances and on all 160 paired seeds, with family means of 15.6–21.2% against 5.5–11.3% for CP-SPO.
A second controlled ablation isolates cost recovery. Deploying the learned interior point directly gives regret of 53–100% on the continuous instances; the same weights deployed through cost recovery give 20–41%; CP-SPO gives 7.6–9.0%.
The Hessian whitening and the output rescaling are both necessary for the method to be numerically stable. Without the output rescaling, numerical rounding brings outputs to the boundary of the polytope, causing log-barrier blow-up. Without the Hessian whitening, conditioning becomes very bad, and training times grow intractable.
The new experiments are found in Appendix E. We believe they successfully demonstrate what it is that makes our method perform well.
Issue 2: The practical benefit of the PL theorem is not clear.
This is a fair objection, and it is true that the selected hyperparameters do not satisfy the assumptions of the theorem. However, our tuning grid always includes parameters for which the assumptions actually are satisfied (Appendix C). This is what is important: a PL guarantee gives a shield against a specific form of vanishing-gradient pathology. Our theorem shows that our method can always select parameters for which this shield is in effect (even if in practice, it may prefer other values empirically, just as methods with no PL-guarantees can outperform methods that have them).
We agree that the guarantee holds only in output space, and the revised paper states this explicitly (Section 4.3). A PL guarantee in the network weights is in general not achievable, for our loss or for any other.
Guarantees in output space are nevertheless treated as important in the literature, and for a good reason: they ensure that whatever non-convexity remains in training comes entirely from the neural network. The non-convexity of neural networks is comparatively well understood, and optimizers such as Adam typically handle it well. A non-convex decision loss can behave far more pathologically. Mandi et al. (2025) show, for example, that with quadratic regularization the regret still has zero gradient across large regions of predictions where the regret is positive. No optimizer and no architecture can make progress there. A loss with a PL-guarantee in output space cannot fail in this way. The convexity of SPO+ is valued for the same reason, although it too is a statement about the predicted cost and not about the network weights.
We have also replaced the theorem with one that covers the loss exactly as implemented.
Issue 3: Perhaps there is no benefit compared with WiSE?
Our new, much more extensive experiments cover training sizes from 100 to 1000 pairs, and predictors from affine to three hidden layers, with every method tuned separately in each setting (see the general comment).
Against WiSE, and the other solver-free baselines, CP-SPO generally achieves lower regret. With validation-selected architectures it beats WiSE on 11, 12, 13 and 14 of the 16 instances at 100, 300, 500 and 1000 training pairs, and with an affine predictor for both on 16, 15, 16 and 16. Random LP with 100 training pairs is one of the very few cases where WiSE is able to compete, and there the two are close (mean regret 1.9% against 2.1%). With more data CP-SPO is ahead on random LP as well (0.16% against 0.35% at 1000 pairs).