acceptodds
Under review as a conference paper at ICLR 2027

WSNet: Learning to Warm-Start Interior-Point Methods for Nonlinear Optimization

Abstract

Interior-point methods (IPMs) are widely used to solve constrained nonlinear programs, yet warm-starting an IPM is particularly challenging because an initial point must not only be close to a solution, but also preserve the primal-dual interiority and centrality required for reliable numerical progress. We propose the Warm-Start Network (WSNet), a learning-based framework that reuses the primal-dual trajectory of a reference problem to warm-start a perturbed instance. WSNet first employs an attention-based selection module to identify an informative trajectory iterate. It then adjusts the selected point by predicting a dual-slack correction, recovering the remaining primal-dual updates through finite-step residual minimization, and enforcing centrality with neighborhood-aware training losses. Under explicit optimization and regularity assumptions, we show that mini-batch training produces strictly interior primal-dual points with high probability. Combined with finite-step recovery, this result guarantees membership in a prescribed central-path neighborhood. Experiments on convex and nonconvex quadratic programs, quadratically constrained quadratic programs, and second-order cone programs demonstrate that WSNet effectively accelerates IPOPT, reducing its iteration count by up to 59.9% and total solution time by up to 60.3%.

Primary area:
optimization
Keywords:
Interior-Point Methods, Learning to Warm Start, Nonlinear Optimization, Primal-Dual Optimization

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_TpvtOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper warm-starts IPOPT on a parametric family of NLPs using the stored IPM trajectory of a previously solved reference instance. A selection module picks one iterate of the reference trajectory using a Gumbel-Softmax over scores computed from the two parameter vectors. An adjustment module then predicts a dual-slack correction, recovers the primal and equality-dual corrections with a few L-BFGS steps on the KKT residual, and is trained with a loss that pushes the complementarity products toward a central-path neighborhood ( N2\mathcal{N}_2 or N−∞\mathcal{N}_{-\infty} ). Under optimization assumptions on the training loss, the paper shows that the trained outputs are strictly interior and central on the training instances, and that enough recovery steps place them in the prescribed neighborhood. On synthetic convex and nonconvex QPs, QCQPs, and SOCPs with 1,000 variables, WSNet reduces IPOPT iterations by up to 60% and total time, including inference, by up to 60%.

Critical strengths
  1. A neat idea that respects how IPMs work. Primal-only learned warm starts, such as the DC3 and FSNet baselines here, do not control interiority or centrality. This paper builds a full primal-dual point and trains it directly for both, which is what an IPM needs. Reusing the reference trajectory is simple and effective: Fig. 3 and Fig. 6 show that an intermediate iterate is a better warm start than the final one, and the selection module learns to pick one.

  2. Gains hold end to end. The total time in Table 1 includes inference. WSNet improves on cold-start IPOPT in all six problem classes and has the lowest total time in five of them. The selection-only ablation (AttNet) shows that the adjustment step matters, and Fig. 4 shows on convex SOCPs that each loss produces the intended complementarity structure.

Critical weaknesses
  1. The benchmarks are synthetic and narrow.

    • All six classes use random data, and only the right-hand side of the equality constraints varies with the parameter (Ax=pAx = p, App. A). In the nonconvex classes, nonconvexity comes only from a q⊤sin⁡(x)q^\top \sin(x) term in the objective, and the constraints stay convex. These may be easy instances for trajectory reuse.
    • Problem families from applications, for example model predictive control, portfolio optimization, or optimal power flow with nonconvex constraints, would show whether the gains carry over to the settings that motivate the paper (l. 33–36).
    • The comparison misses the closest learned primal-dual warm start, IPM-LSTM (Gao et al., 2024), which the paper cites (l. 139). It also lacks a simple non-learned baseline that always picks a fixed trajectory iterate, tuned on the validation set.
    • The comparison also lacks a non-learned warm start that targets centrality, such as the smoothing approach of Chen et al. (2025), which the paper cites (l. 131) and which applies directly to the convex QP and SOCP classes.
  2. The theory assumes most of what it proves, and differs from the implementation.

    • Assumption (A2) requires zero attainable loss and a PL-type condition over the whole training region, and (A4) requires zero attainable KKT residual. Under these, Theorem 5.1 is the standard SGD convergence argument under PL plus Markov's inequality.
    • The guarantees hold only on training instances; the conclusion acknowledges that generalization is not covered (l. 483–484).
    • The analysis uses gradient descent with step 1/Lϕ1/L_\phi for recovery (App. D.2), while the method uses 50 L-BFGS steps (App. C.1).
Questions for authors
  1. Recovery steps in Corollary 5.2. The corollary chooses the number of recovery steps ss large enough after training. Theorem 5.1 bounds the complementarity of the output produced with the ss used during training. Is ss the same in both? If not, how does the bound of Theorem 5.1 carry over?

  2. IPOPT settings. How is the warm start passed to IPOPT (e.g., warm_start_init_point, mu_init, bound push options), and which linear solver is used? Is the cold start IPOPT's default initialization? Suri et al. (2026), which the paper cites, argue that the cold-start baseline strongly affects reported warm-start gains. Also, a cold-start time of 32 s for a 1,000-variable QP (Table 1) seems high.

  3. Reference instance and trajectory length. Is p0p_0 fixed, or sampled per instance (App. B.1 says it is sampled from U(−1,1)\mathcal{U}(-1,1) during training)? The selection matrix Wθ∈RK×2dW_\theta \in \mathbb{R}^{K \times 2d} assumes a fixed trajectory length KK. How are reference trajectories of different lengths handled?

  4. Interpretability. As written in Eq. (5), the selection scores are linear in (p,p0)(p, p_0). Can the learned selection rule be interpreted, for example as a function of ∥p−p0∥\|p - p_0\|? Fig. 6 suggests that it picks advanced but nonterminal iterates, and a simple description/interpretation/plot of this rule would be useful.

Minor:

  • The paper calls the instances "perturbed", which suggests small changes to a nominal problem. Here both pp and p0p_0 are drawn from U(−1,1)\mathcal{U}(-1,1) (App. B.1), so "parametric" describes the setting better.
  • The selection module is called "attention-based", but Eq. (5) is a linear layer followed by Gumbel-Softmax. Please clarify where attention enters.
4 comments

We sincerely appreciate the reviewer’s effort and positive feedback. Revisions in the manuscript are highlighted in blue. Below, we address the concerns and questions raised.

Critical Weaknesses

The benchmarks are synthetic and narrow.

W1. All six classes use random data, and only the right-hand side of the equality constraints varies with the parameter (Ax=pAx=p, App. A). In the nonconvex classes, nonconvexity comes only from a q⊤sin⁡(x)q^{\top}\sin(x) term in the objective, and the constraints stay convex. W2. These may be easy instances for trajectory reuse. Problem families from applications, for example model predictive control, portfolio optimization, or optimal power flow with nonconvex constraints, would show whether the gains carry over to the settings that motivate the paper (l. 33–36).

Thank you for this suggestion. We have added MOSARQP2, a standard convex QP from the Maros–Mészáros repository [1], together with two real-world benchmarks (as suggested): MPC for model predictive control from RLQP [2] and ACOPF for power-system generation dispatch used in DC3 [3].

  • We acknowledge the limited scope of the original synthetic benchmarks. Our expanded evaluation goes beyond perturbing equality right-hand sides: MOSARQP2 varies inequality right-hand sides, while MPC varies initial states and state/input bounds. ACOPF also introduces nonlinear, nonconvex power-balance equality constraints, extending the evaluation beyond problems whose nonconvexity comes solely from a sinusoidal objective.
  • The table below summarizes warm-start gains on MOSARQP2 and the real-world benchmarks as iteration / total-time reductions over cold-start IPOPT, including prediction cost. The best value for each metric is shown in bold. WSNet$_2$ achieves the largest total-time reductions on MOSARQP2 and MPC, while AttNet is fastest on ACOPF because of its lower initialization cost.
MethodMOSARQP2MPCACOPF
DC31.5% / 8.7%-46.5% / -41.6%20.6% / 0.9%
FSNet13.4% / -14.7%8.5% / -76.7%34.7% / 23.2%
AttNet31.9% / 43.9%-4.7% / -1.8%77.5% / 76.3%
WSNet$_{-\infty}$54.4% / 53.4%57.9% / 40.8%77.8% / 52.8%
WSNet$_2$57.9% / 56.6%57.8% / 41.3%77.8% / 54.6%

The new benchmarks are introduced in Section 6.1, with results in Table 1 and analysis in Section 6.2. Their formulations and data-generation procedures are detailed in Appendix A.2.

W3. The comparison misses the closest learned primal-dual warm start, IPM-LSTM (Gao et al., 2024), which the paper cites (l. 139).

Thanks for mentioning this.

  • IPM-LSTM’s nested IPM and LSTM iterations introduce substantial computational overhead, which may limit its scalability to larger problems. We therefore added a comparison with IPM-LSTM [4] on a convex QP benchmark with only 200 variables, 100 inequality constraints, and 100 equality constraints;
  • Despite its higher prediction cost, IPM-LSTM achieves a strong IPOPT iteration reduction. WSNet$_2$ attains a comparable iteration reduction (40.40% vs. 42.40%) while requiring substantially less prediction time. Consequently, WSNet$_2$ delivers a considerably larger end-to-end time reduction relative to cold-start IPOPT (48.60% vs. 15.11%).

The detailed results are reported in Appendix B.2 of the revised manuscript.

MethodEnd-to-EndIPOPT (warm start)Total Time (s) ↓Gain (Ite./Time) ↑
Obj. ↓Max ineq. ↓Mean ineq. ↓Max eq. ↓Mean eq. ↓Time (s) ↓Ite. ↓Time (s) ↓
IPOPT−29.1760.0000.0000.0000.0000.64212.50———
IPM-LSTM−29.0500.0000.0000.0020.0010.1757.200.3700.54542.40%/15.11%
WSNet$_{-\infty}$−26.4810.0040.0000.0160.0050.0017.560.3570.35839.52%/44.24%
WSNet$_2$−28.7840.0050.0000.0040.0010.0017.450.3290.33040.40%/48.60%

W4. It also lacks a simple non-learned baseline that always picks a fixed trajectory iterate, tuned on the validation set.

Thank you for the suggestion. This baseline is a special case of AttNet: choosing one fixed iterate corresponds to using a constant, one-hot attention vector for every target instance. AttNet instead selects an iterate adaptively for each instance, since the most suitable reference iterate may vary with the problem parameters.

W5. The comparison also lacks a non-learned warm start that targets centrality, such as the smoothing approach of Chen et al. (2025), which the paper cites (l. 131) and which applies directly to the convex QP and SOCP classes.

We have added a comparison with the smoothing-based warm-starting method of Chen et al. [6], denoted as Smoothing, on the convex QP, MOSARQP2, and MPC benchmarks. The results below report IPOPT iteration reduction / end-to-end total-time reduction relative to cold-start IPOPT, with total time including initialization.

MethodConvex QPMOSARQP2MPC
Smoothing17.2% / 33.8%39.7% / 46.1%23.5% / 33.3%
WSNet$_{-\infty}$49.8% / 52.0%54.4% / 53.4%57.9% / 40.8%
WSNet$_2$45.5% / 53.7%57.9% / 56.6%57.8% / 41.3%
  • WSNet consistently achieves substantially larger iteration reductions than Smoothing across all three benchmarks. More importantly, it provides the best total-time reduction on Convex QP, MOSARQP2, and MPC—up to 53.7%, 56.6%, and 41.3%, respectively—despite including its initialization cost.

Detailed results and further analysis are provided in Appendix B.3.

W6. Assumption (A2) requires zero attainable loss and a PL-type condition over the whole training region, and (A4) requires zero attainable KKT residual. Under these, Theorem 5.1 is the standard SGD convergence argument under PL plus Markov's inequality.

Thank you for this comment.

  • We acknowledge that these assumptions are a little bit restrictive. (A2) requires an arbitrarily small attainable training loss and a PL-type condition, which is not guaranteed for general neural networks. (A4) assumes that recovery residuals can be made arbitrarily small; it does not assert that finite-step recovery achieves zero residual.
  • We agree that the SGD convergence argument uses standard PL analysis and Markov’s inequality. The role of our analysis is to connect training-loss and recovery-residual bounds to sufficient conditions for strict interiority and membership in the prescribed central-path neighborhood.

These are conditional guarantees rather than a complete explanation of practical performance. We state the assumptions in Section 5 and acknowledge their limitations in the Conclusion of the revised manuscript.

W7. The guarantees hold only on training instances; the conclusion acknowledges that generalization is not covered (l. 483–484).

Thanks for raising this. Our theoretical guarantees apply only to training instances, while generalization is evaluated empirically on held-out instances and under broader parameter distributions (Appendix B.4). Establishing generalization guarantees remains an important direction for future work, as noted in the conclusion.

W8. The analysis uses gradient descent with step 1/Lϕ1/L_{\phi} for recovery (App. D.2), while the method uses 50 L-BFGS steps (App. C.1).

Thank you for pointing out this distinction. Our recovery convergence analysis considers GD under the stated assumptions. On the synthetic benchmarks, we use 50 L-BFGS steps to align the recovery method with FSNet. On the public benchmarks, we use five GD steps to reduce recovery overhead.

We additionally compare 50-step GD and 50-step L-BFGS recovery for WSNet$_2$ on the convex QCQP benchmark with 1,000 variables, 500 inequality constraints, and 500 equality constraints. As shown below, GD reduces prediction time from 1.643 s to 0.051 s and still achieves a 36.49% reduction in total solution time relative to cold-start IPOPT. L-BFGS provides stronger warm-start gains and a lower total solution time on this benchmark.

MethodEnd-to-EndIPOPT (warm start)Total Time (s) ↓Gain (Ite./Time) ↑
Obj. ↓Max ineq. ↓Mean ineq. ↓Max eq. ↓Mean eq. ↓Time (s) ↓Ite. ↓Time (s) ↓
IPOPT−186.9860.0000.0000.0000.00090.63213.18———
WSNet$_2$ (L-BFGS50)−185.8230.0060.0000.0020.0011.6436.9248.44150.08447.50%/44.74%
WSNet$_2$ (GD50)−185.7320.0050.0000.0070.0020.0517.7857.51157.56240.97%/36.49%

Questions For Authors

Q1. Recovery steps in Corollary 5.2. The corollary chooses the number of recovery steps ss large enough after training. Theorem 5.1 bounds the complementarity of the output produced with the ss used during training. Is ss the same in both? If not, how does the bound of Theorem 5.1 carry over?

Thank you for pointing out this issue. We use the same recovery budget ss during training and inference. The loss in Theorem 5.1 is evaluated on the output of this same finite-step recovery map Rec⁡s\operatorname{Rec}_s.

Q2. IPOPT settings. How is the warm start passed to IPOPT (e.g., warm_start_init_point, mu_init, bound push options), and which linear solver is used? Is the cold start IPOPT's default initialization? Suri et al. (2026), which the paper cites, argue that the cold-start baseline strongly affects reported warm-start gains. Also, a cold-start time of 32 s for a 1,000-variable QP (Table 1) seems high.

Thank you for raising this point.

  • We have added the detailed IPOPT settings to Appendix C.1, including the initialization procedure and warm-start parameters.
  • The relatively long solve time for the 1,000-variable QPs is largely due to the dense structure of these synthetic instances, which increases the computational cost of each IPOPT iteration.

Q3. Reference instance and trajectory length. Is p0p_0 fixed, or sampled per instance (App. B.1 says it is sampled from U(−1,1)\mathcal{U}(-1,1) during training)? The selection matrix Wθ∈RK×2dW_{\theta} \in \mathbb{R}^{K\times 2d} assumes a fixed trajectory length KK. How are reference trajectories of different lengths handled?

For each problem class, p0p_0 is sampled once and held fixed throughout training, validation, and testing. Its stored IPM trajectory of length KK is reused to warm-start many related instances. Both the reference trajectory and KK are therefore fixed before training. In our current setup, changing the reference instance requires training a corresponding model; if KK changes, the selector’s output dimension is adjusted accordingly.

Q4. Interpretability. As written in Eq. (5), the selection scores are linear in (p,p0)(p,p_0). Can the learned selection rule be interpreted, for example as a function of ∣p−p0∣|p-p_0|? Fig. 6 suggests that it picks advanced but nonterminal iterates, and a simple description/interpretation/plot of this rule would be useful.

Thank you for this suggestion. With p0p_0 fixed, the scores in Eq. (5) are affine functions of p−p0p-p_0, allowing the selector to distinguish parameter changes in different directions rather than relying solely on their magnitude. The selection rule is learned jointly with the adjustment module through the centrality loss evaluated after recovery.

Figure 6 suggests that the selector balances progress toward the reference solution against interiority, favoring advanced but nonterminal iterates. This interpretation is consistent with Appendix E.1: the contextual residual bound contains a term proportional to ∥p−p0∥/μ0k\|p-p_0\|/\mu_0^k, so later iterates with smaller barrier parameters can be more sensitive to parameter changes.

Minor:

Q5. The paper calls the instances "perturbed", which suggests small changes to a nominal problem. Here both pp and p0p_0 are drawn from U(−1,1)\mathcal{U}(-1,1) (App. B.1), so "parametric" describes the setting better.

Thanks for your suggestion. We agree that “parametric” more precisely describes the synthetic setting, where pp is sampled independently of the fixed reference p0p_0. We have revised the terminology throughout the manuscript, using “parametric problem families” for the overall setting and “target instance” for the instance to be solved. The newly added public benchmarks use independent perturbations of selected parameter coordinates around the nominal p0p_0, as detailed in Appendix A.2.

Q6. The selection module is called "attention-based", but Eq. (5) is a linear layer followed by Gumbel-Softmax. Please clarify where attention enters.

The selection module is inspired by the attention mechanism in GAT [5], which computes learned scores from concatenated features and uses normalized weights for aggregation. We adapt this idea to a fixed reference trajectory: Eq. (5) computes iterate-specific scores from [p;p0][p;p_0], and Eq. (6) applies the resulting weights to the stored primal–dual iterates and barrier parameters. Our simplified design omits the LeakyReLU activation and replaces softmax with Gumbel–Softmax to support discrete selection during gradient-based training.

References

[1] István Maros and Csaba Mészáros. A repository of convex quadratic programming problems. Optimization Methods and Software, 11(1–4):671–681, 1999. doi:10.1080/10556789908805768.

[2] Jeffrey Ichnowski, Paras Jain, Bartolomeo Stellato, Goran Banjac, Michael Luo, Francesco Borrelli, Joseph E. Gonzalez, Ion Stoica, and Ken Goldberg. Accelerating quadratic optimization with reinforcement learning. In Advances in Neural Information Processing Systems, 34:21043–21055, 2021.

[3] Priya L. Donti, David Rolnick, and J. Zico Kolter. DC3: A learning method for optimization with hard constraints. In International Conference on Learning Representations, 2021.

[4] Xi Gao, Jinxin Xiong, Akang Wang, Qihong Duan, Jiang Xue, and Qingjiang Shi. IPM-LSTM: A learning-based interior point method for solving nonlinear programs. In Advances in Neural Information Processing Systems, 37:122891–122916, 2024.

[5] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.

[6] Yuwen Chen, Paul Goulart, and Colin Jones. A warmstarting technique for general conic optimization in interior point methods. arXiv preprint arXiv:2512.00693, 2025.

Reviewer_E86aOpenReview ↗
Rating
3(+2)
Confidence
2
Informativeness
2(+1)
Presentation
2(+1)
Soundness
3(+2)
Read review
Paper summary

The paper proposes yet another method for warm-starting interior point methods by learning from previous IPM trajectories. Despite the problem being important in practice, the paper require a large amount of work before it can be published. My main concerns are about the readability of the paper - it is extremely hard to follow, and details that should be presented explicitly and clearly appear scattered throughout an need to be assembled from different pieces, for example:

  1. What exactly kind of models does the paper aim to solve? Is it a generic model for initializing IPMs in zero-shot manner by learning from a large corpus of trajectories? Is it a narrower, but still foundation models for a large family, like "here is an architecture, and we can train it on many QPs and it will be able to zero-shot QPs but not SDPs"? Or is it the narrowest sense - we train a given problem of a given dimension and test on perturbed data? The scope should be clear from the very beginning.
  2. How is the training data collected? What are the features, and what is the loss? It can be reconstructed from the details in the paper, but it needs to be explicit.
  3. How does inference work? What is exactly the input and the output? Can also be reconstructed from details, but I don't want to reconstruct it - it has to be explicit.
  4. How are hyperparameters tuned, if any? How are hyperparameters of the baseline tuned? Obviously, we can't compare a tuned model to an untuned baseline, for example. This is in the appendix about WSNet, but nothing about baselines.

A lot of things missing, unclear, or scattered.

Update

The authors clarified many things and updated the manuscript accordingly. See my reply. Raising my score to weak accept.

Critical strengths
  1. Saving time on IPMs is an important practical problem, especially for problems that are repeatedly solved with different data, namely, many similar problems but perturbed.
  2. Tables indeed shows that the authors' method improve performance on some problem types.
Critical weaknesses

My main concern is the presentation. If published, this paper will be an incomprehensible block of text and will contribute little to the community. See my paper summary.

My secondary concern is reproducibility - there is no code we can use, even though the method doesn't appear to be very complex.

Questions for authors

NA

5 comments

We sincerely thank the reviewer for the time and effort devoted to evaluating our work. Revisions in the manuscript are highlighted in blue. Below, we address the concerns and questions raised regarding presentation and reproducibility.

Paper Summary

... My main concerns are about the readability of the paper ...

Q1. What exactly kind of models does the paper aim to solve? Is it a generic model for initializing IPMs in zero-shot manner by learning from a large corpus of trajectories? Is it a narrower, but still foundation models for a large family, like "here is an architecture, and we can train it on many QPs and it will be able to zero-shot QPs but not SDPs"? Or is it the narrowest sense - we train a given problem of a given dimension and test on perturbed data? The scope should be clear from the very beginning.

Thank you for raising this important question. Our setting is the third, narrower case: WSNet is designed for a fixed-dimensional parametric optimization problem family.

  • For each problem family, we train a separate WSNet using instances generated by varying the problem parameters pp. The model leverages the solution trajectory of one fixed reference instance, p0p_0, and learns to produce warm starts for previously unseen parameter instances within the same family. At test time, it is applied without retraining to new instances drawn from that family. Thus, “zero-shot” in our paper refers to generalization to unseen parameter values within a trained problem family; it does not mean zero-shot transfer across problem classes (e.g., from QPs to SDPs) or across incompatible dimensions.
  • This setting is motivated by repeated-solve applications [1, 2], in which the optimization structure and dimensions remain fixed while data or operating conditions vary. MPC and ACOPF, as suggested by Reviewer Tpvt, are representative examples: related optimization problems must be solved repeatedly as the system state, forecast, or demand changes.

We agree that this scope should have been stated more explicitly. In the revision, we clarify it in the abstract and introduction, and expand the training-setting description in the paragraph preceding Algorithm 1.

Q2. How is the training data collected? What are the features, and what is the loss? ....

Thank you for raising these questions.

  • Training data. For each synthetic problem family, we sample pp from specified distributions, following the benchmark setups of DC3 [3] and FSNet [4]. A fixed reference trajectory is obtained by solving p0p_0 with IPOPT. Training is unsupervised and requires no optimal-solution labels. We have expanded Appendix A with detailed data-generation procedures. We are also extending the evaluation to public benchmarks, with additional instances generated by perturbing problem-specific parameters.
  • Input features. The selection network takes the concatenated vector [p⊤,p0⊤]⊤[p^\top,p_0^\top]^\top as input and produces weights for selecting an iterate from the stored reference trajectory, as defined in Eqs. (5)–(6). The dual-correction network takes pp as input and predicts Δz\Delta z, as defined in Eq. (7). Here, pp represents selected parameters of the optimization problem, such as the equality-constraint right-hand-side vector.
  • Training losses. We use the two centrality losses in Eqs. (10) and (11), corresponding to WSNet2\mathrm{WSNet}_2 and WSNet−∞\mathrm{WSNet}_{-\infty} in Table 1, respectively.

Q3. How does inference work? What is exactly the input and the output? ...

We have added Algorithm 2 and an explicit description of the inference procedure in Section 4.2. Given pp, p0p_0, and the stored reference trajectory, the trained WSNet selects a reference iterate, predicts the dual-slack correction Δz\Delta z, and recovers Δx\Delta x and Δλ\Delta\lambda. It adds these corrections to the selected primal-dual point and returns the resulting point together with the selected barrier parameter μ~0\tilde{\mu}_0. The output can be used directly as an end-to-end approximate solution or to warm-start an IPM.

Q4. How are hyperparameters tuned, if any? How are hyperparameters of the baseline tuned?...

Thank you for raising this concern.

  • Baseline configurations. For DC3 [3] and FSNet [4], we follow the recovery/correction procedures in their publicly available implementations. We have added their detailed hyperparameter settings in Appendix C.2.
  • IPOPT configuration. We have added detailed IPOPT settings for cold and warm starts in Appendix C.1.
  • Baseline performance. High-quality primal predictions do not necessarily yield effective IPM warm starts. DC3 and FSNet produce primal-only solutions, whereas WSNet constructs a primal-dual initialization with centrality considerations. This distinction helps explain the warm-start results and is consistent with the discussions in WARP [5] and IPM-LSTM [6].

Critical Weaknesses

W1. My main concern is the presentation. If published, this paper will be an incomprehensible block of text and will contribute little to the community. See my paper summary.

We acknowledge that the presentation needed clarification and have addressed the specific issues raised above. However, we respectfully disagree with the assessment that the paper “will contribute little to the community.”

  • Our main contribution is a two-stage framework that combines learned selection from a stored reference trajectory with centrality-aware primal-dual adjustment. This design directly addresses the interiority, feasibility, and centrality requirements of IPM warm starts.
  • Reviewer Tpvt described this design as “A neat idea that respects how IPMs work.” The theoretical analysis and end-to-end experiments provide complementary support for this approach. We believe that learning to reuse intermediate trajectory information offers a useful perspective for research on IPM warm-starting.

We have clarified the method’s scope, training procedure, and inference process in the revised manuscript. We welcome any further questions and would be happy to address them.

W2. My secondary concern is reproducibility - there is no code we can use, even though the method doesn't appear to be very complex.

Thank you for raising this concern. We are committed to making our code publicly available upon publication. We have added a code availability statement at the end of the first paragraph of Section 6.1.

References

[1] E. Alper Yildirim and Stephen J. Wright. Warm-start strategies in interior-point methods for linear programming. SIAM Journal on Optimization, 12(3):782–810, 2002. doi:10.1137/S1052623400369235.

[2] Elizabeth John and E. Alper Yildirim. Implementation of warm-start strategies in interior-point methods for linear programming in fixed dimension. Computational Optimization and Applications, 41:151–183, 2008. doi:10.1007/s10589-007-9096-y.

[3] Priya L. Donti, David Rolnick, and J. Zico Kolter. DC3: A learning method for optimization with hard constraints. In International Conference on Learning Representations, 2021.

[4] Hoang Nguyen and Priya Donti. FSNet: Feasibility-seeking neural network for constrained optimization with guarantees. In Advances in Neural Information Processing Systems, 38, 2025.

[5] Dhruv Suri, Helgi Hilmarsson, and Shourya Bose. WARP: A benchmark for primal-dual warm-starting of interior-point solvers. arXiv preprint arXiv:2605.05728, 2026.

[6] Xi Gao, Jinxin Xiong, Akang Wang, Qihong Duan, Jiang Xue, and Qingjiang Shi. IPM-LSTM: A learning-based interior point method for solving nonlinear programs. In Advances in Neural Information Processing Systems, 37, 2024.

Thank you again for your helpful comments. As the rebuttal deadline is approaching, we would appreciate any further feedback you may have and would be glad to address any remaining concerns.

Reviewer_3QTNOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to learn good warm-start solutions for primal-dual interior-point methods. It does so by learning primal-dual solutions that achieve low residual and good centrality, through a combination of previous iterates and correction mechanism.

Numerical experiments are reported on small instances that show promising reduction in interior-point iteration and end-to-end solving time.

Critical strengths

The paper jointly considers primal/dual feasibility and centrality to obtain high-quality starting points for interior-point methods. This is an area of interest for the optimization community, and has been a challenge since interior-point methods first gained popularity.

Critical weaknesses

The theoretical analysis is existential and hence of little practical value.

  • The validity of Assumption 5.1 deserves further discussion, especially given that ReLU functions have non-Lipschitz gradient
  • Theorem 5.1 and essentially say that SGD will eventually reach a region with small loss, provided that the step size is chosen according to a parameter λj\lambda_{j} whose existence is assumed and value unknown.

The numerical experiments are conducted on small, artificially-generated instances not representative of real-world instances. The "non-convex" variants only modify the objective term by adding a sin(x)sin(x) term, which is a relatively well-behaved function. It is unclear whether the proposed approach scales well to large instances or to more realistic types of problems.

Questions for authors
  • Previous work by Klamkin, Tanneau and Van Hentenryck (https://arxiv.org/abs/2402.02596) investigates the learning of primal-dual well-centered solutions using a different approach; while that reference does not investigate warm-start, it would be relevant to position the paper against it.
  • How are primal and dual residuals controlled in Algorithm 1? I understand that a recovery procedure helps reduce these residuals (Eq.s (8) and (9)), however, the residuals may be affected by the projection step
  • L-BFGS is a second-order method, how would this approach scale as the underlying problem (number of variables and constraints) increase?
  • Please comment on the validity of Assumption A1 given that ReLU functions do not have Lipschitz gradients
  • Please give more details as to the meaning of Assumptions A2 and A4 and how realistic they are
  • Please provide detailed instance generation procedure for the numerical experiments
7 comments

We sincerely appreciate the reviewer’s effort and positive feedback. Revisions in the manuscript are highlighted in blue. Below, we address the concerns and questions raised.

Critical Weaknesses

W1. The validity of Assumption 5.1 deserves further discussion, especially given that ReLU functions have non-Lipschitz gradient

Thank you for pointing this out. The loss construction also admits a continuously differentiable alternative: the inner ReLU functions can be replaced by ELU with α=1\alpha=1, whose derivative is Lipschitz continuous. We have added this option after Eqs. (10) and (11) in the revised manuscript.

W2. Theorem 5.1 and essentially say that SGD will eventually reach a region with small loss, provided that the step size is chosen according to a parameter λj\lambda_j whose existence is assumed and value unknown.

Thank you for this comment. Indeed, λJ\lambda_J is a key constant in Assumption (A2): it characterizes the nondegeneracy of the training loss and affects the theoretical step-size bound through the PL condition. Explicitly computing Jl(θ)J_l(\theta) and estimating λJ\lambda_J are computationally demanding. In our experiments, we use Adam with a fixed learning rate of 10−410^{-4}.

W3. The numerical experiments are conducted on small, artificially-generated instances not representative of real-world instances. The "non-convex" variants only modify the objective term by adding a sin(x)sin(x) term, which is a relatively well-behaved function. It is unclear whether the proposed approach scales well to large instances or to more realistic types of problems.

Thank you for this suggestion. We now add MOSARQP2, a standard convex QP from the Maros–Mészáros repository [1], together with two real-world benchmarks (suggested by Review Tpvt): MPC for model predictive control from RLQP [2] and ACOPF for power-system generation dispatch used in DC3 [3].

The table below reports percentage reductions in IPOPT iterations / total solution time, including prediction, relative to cold-start IPOPT. The best value for each metric is shown in bold. WSNet achieves the largest iteration reductions on all three benchmarks, reaching 77.8%, and the largest total-time reductions on MOSARQP2 and MPC.

MethodMOSARQP2MPCACOPF
DC31.5% / 8.7%-46.5% / -41.6%20.6% / 0.9%
FSNet13.4% / -14.7%8.5% / -76.7%34.7% / 23.2%
AttNet31.9% / 43.9%-4.7% / -1.8%77.5% / 76.3%
WSNet$_{-\infty}$54.4% / 53.4%57.9% / 40.8%77.8% / 52.8%
WSNet$_2$57.9% / 56.6%57.8% / 41.3%77.8% / 54.6%

We have added these benchmarks to Section 6.1 and Table 1, analyzed their results in Section 6.2, and detailed their formulations and perturbation schemes in Appendix A.2 of the revised manuscript.

Questions For Authors

Q1. Previous work by Klamkin, Tanneau and Van Hentenryck (https://arxiv.org/abs/2402.02596) investigates the learning of primal-dual well-centered solutions using a different approach; while that reference does not investigate warm-start, it would be relevant to position the paper against it.

Thank you for pointing us to this relevant work. The work of Klamkin et al. [4] learns dual-feasible solutions for bounded LPs through dual completion and barrier-based training. Their focus is on valid dual bounds, rather than constructing primal–dual IPM warm starts with explicit feasibility and central-neighborhood control. We now clarify this in the Introduction of the revised manuscript.

Q2. How are primal and dual residuals controlled in Algorithm 1? I understand that a recovery procedure helps reduce these residuals (Eq.s (8) and (9)), however, the residuals may be affected by the projection step

Thank you for pointing this out.

  • Actually, we realize that the projection is not essential for obtaining interior points. Under the stated assumptions and loss threshold, Theorem 5.1 guarantees strictly positive primal variables on the training instances.
  • In practice, the centrality losses in Eqs. (10) and (11), which incorporate ReLU, also encourage the primal variables to satisfy the interiority requirement.

We have modified the description of the post-recovery projection for enforcing interiority in the revised manuscript.

Q3. L-BFGS is a second-order method, how would this approach scale as the underlying problem (number of variables and constraints) increase?

Thank you for raising this concern.

  • We acknowledge that multi-step L-BFGS recovery may incur additional computational overhead as the problem size grows. However, the recovery procedure in WSNet is not restricted to L-BFGS. For larger-scale nonlinear programs, we instead use five steps of gradient descent (GD) on the real-world benchmarks and MOSARQP2, which substantially reduces the recovery cost.
  • We further compare 50-step GD recovery with L-BFGS recovery on the convex QCQP benchmark with 1,000 variables, 500 inequality constraints, and 500 equality constraints. Replacing L-BFGS with GD reduces the recovery/prediction time from 1.643 s to 0.051 s, while still achieving a 36.49% reduction in total solution time compared with cold-start IPOPT (see the table below). These results suggest that WSNet can scale to larger problems by using lightweight first-order recovery methods, while retaining a substantial end-to-end speedup.
MethodEnd-to-EndIPOPT (warm start)Total Time (s) ↓Gain (Ite./Time) ↑
Obj. ↓Max ineq. ↓Mean ineq. ↓Max eq. ↓Mean eq. ↓Time (s) ↓Ite. ↓Time (s) ↓
IPOPT−186.9860.0000.0000.0000.00090.63213.18———
WSNet$_2$ (L-BFGS50)−185.8230.0060.0000.0020.0011.6436.9248.44150.08447.50%/44.74%
WSNet$_2$ (GD50)−185.7320.0050.0000.0070.0020.0517.7857.51157.56240.97%/36.49%

Q4. Please comment on the validity of Assumption A1 given that ReLU functions do not have Lipschitz gradients

Thank you for this comment. ReLU itself does not have a Lipschitz-continuous derivative. As now noted after Eqs. (10) and (11), the inner ReLU functions can alternatively be replaced by ELU with α=1\alpha=1, which is continuously differentiable with a Lipschitz-continuous derivative. This removes the explicit loss-level kink, while (A1) remains an additional assumption on the full composite training objective.

Q5. Please give more details as to the meaning of Assumptions A2 and A4 and how realistic they are

Thank you for this question.

  • Assumption (A2) combines a realizability condition—the training loss can be made arbitrarily small—with a PL-type nondegeneracy condition on its Jacobian. The latter prevents the gradient from vanishing at a positive loss and supports the training convergence analysis. We acknowledge that this is a strong condition and is not guaranteed for general neural networks.
  • Assumption (A4) states that, for the predicted dual-slack correction, the primal and dual residuals can be made arbitrarily small by adjusting Δx\Delta x and Δλ\Delta\lambda. This is an existence condition, not a guarantee that finite-step recovery reaches zero residual. For QPs with linear equality constraints and a nonsingular KKT matrix, the corresponding recovery equations are solvable. For general nonlinear problems, this condition requires additional justification.

We now state these assumptions explicitly in Section 5 and acknowledge their limitations in the Conclusion, where relaxing them is identified as future work.

Q6. Please provide detailed instance generation procedure for the numerical experiments

Thanks for raising this.

  • For the synthetic benchmarks, we sample the objective and constraint coefficients once, keep them fixed, and independently sample the equality right-hand-side vector pp across instances. For MOSARQP2 and the real-world benchmarks, we independently perturb inequality right-hand sides in MOSARQP2, initial states and state/input bounds in MPC, and active/reactive demands in ACOPF.

We have added the detailed generation procedures and perturbation schemes to Appendices A.1 and A.2, respectively.

References

[1] István Maros and Csaba Mészáros. A repository of convex quadratic programming problems. Optimization Methods and Software, 11(1–4):671–681, 1999. doi:10.1080/10556789908805768.

[2] Jeffrey Ichnowski, Paras Jain, Bartolomeo Stellato, Goran Banjac, Michael Luo, Francesco Borrelli, Joseph E. Gonzalez, Ion Stoica, and Ken Goldberg. Accelerating quadratic optimization with reinforcement learning. In Advances in Neural Information Processing Systems, 34:21043–21055, 2021.

[3] Priya L. Donti, David Rolnick, and J. Zico Kolter. DC3: A learning method for optimization with hard constraints. In International Conference on Learning Representations, 2021.

[4] Michael Klamkin, Mathieu Tanneau, and Pascal Van Hentenryck. Dual interior point optimization learning. arXiv preprint arXiv:2402.02596, 2024.

Re-Theorem 5.1: My point is that SGD convergence under Polyak-Losajiewicz condition is a standard result. Combined with the strength of the assumptions and its existential nature, its overall novelty and value is therefore limited.

Re-experiments: thank you for conducting additional experiments; I should however note that the new instances are still small. I'd encourage the authors to consider larger instances (>100k or >1M variables) from convex optimization problems to strengthen the computational claims. (given the timeline of the discussion period, I don't expect this would be done in this round).

Re-Klamkin et al paper: I agree that that paper does not consider the warm-start aspect and consider a more restrictive setting (linear programming with bounded variables). However, looking at the training of the ML itself, there is a strong parallel in that both approaches aim to learn primal and dual solutions with good feasibility and centrality. The [Klamkin et al] paper does that via the loss function and an implicit layer that derives from KKT conditions, whereas the present paper does so via a correction mechanism.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.