- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces pdLIP, a learned preconditioning method for a primal-dual interior-point method (IPM) for smooth nonlinear programs called pdProj. pdLIP reduces the number of iterations compared to the cold-started pdProj by replacing the second-order Newton update with a learned update (obtained by learning a positive-diagonal scaling of the reduced right-hand side of the KKT system). Empirical validation includes portfolio optimization, support-vector machine, and quadrotor navigation examples, showing that a pdProj warm-started with pdLIP needs fewer refinement iterations and has lower total runtime. The results show promise for scaling to several hundred variables and constraints across these domains.
Critical strengths
- The premise of the paper is coherent. Replacing an expensive linear-system solve with an inexpensive learned update makes sense.
- Current empirical validation shows promise: the paper contains a good range of problems, and the results appear strong for many of them.
Critical weaknesses
- Despite showing great improvements on QPs, the paper provides more limited evidence of effectiveness on general nonlinear problems. For example, the quadrotor task shows much smaller improvements, leaving the method's general applicability less established.
- It is unclear what the practical cost-benefit trade-offs are. pdLIP is trained and evaluated on examples drawn from the same distribution, which makes sense. However, it is unclear whether the method’s end-to-end performance against widely used solvers is competitive in terms of efficiency / whether the inference speed-up benefits justify the training costs.
Questions for authors
- Could the authors include benchmarks against other methods? It would help to compare performance against other warm-start/preconditioning methods.
- Could the authors clarify why they used an LSTM+MLP, as opposed to any other recurrent architecture? To that end, it would be helpful to see ablations and performance on other NN architectures (e.g., how performance would change with a GRU/RNN/MLP-only/etc. instead of an LSTM).
- The abstract could be revisited and manually refined for clarity. A more direct, intuitive explanation of the method and its motivation would make the contribution easier to grasp on a first read and improve the presentation considerably.
2 comments
Thank you to the authors for addressing my questions and concerns, adding the ablations, and improving the presentation of the paper.
I remain supportive of accepting this work by maintaining a score of 3.
Nonlinear evidence, comparisons, and cost
Thank you for the careful review. We have added nonlearned baselines and a memoryless MLP comparison, and made the nonlinear evidence and practical costs clearer.
The quadrotor also includes nonlinear dynamics and obstacle constraints. There, pdLIP reduces refinement from 9.74 to 8.00 iterations and total online time from 19.05 to 15.81 s per problem, including warm-start generation (Table 5). This extends the evidence beyond synthetic problem families to a robotics-motivated nonlinear control problem. As in many learn-to-optimize approaches, we target repeated solution of related problems. \textbf{Section 1} now clarifies that models are trained separately for each problem family and dimension. Generalization to unseen dimensions and evaluation on larger, more complex nonlinear programs remain future work.
We match the warm-start generation time across methods at approximately 20 seconds for a batch of 1,000 problem instances (about 20 ms per instance, amortized), allowing the cheaper nonlearned methods to take many more warm-start steps. Even with this allowance, pdLIP roughly halves refinement iterations: 4.35 versus 8.42 for Jacobi-style scaling, the best nonlearned result by this measure. The mean warm-start KKT residual falls from 0.2309 to 0.0258, and total online time is 2.44 s per problem, compared with 5.02-5.79 s for the alternatives. Appendix A.3.5, Table 6 also gives an equal-iteration comparison with 300 warm-start steps.
On the same convex quadratic family, LSTM-MLP reduces refinement iterations from 6.34 to 4.35 (31.4%) and total online time from 3.61 to 2.44 s. Although recurrence makes warm-start generation more expensive, it saves more time in refinement. This supports using memory in the update; we further motivate the architectural choice in Section 4.1.
Appendix A.5.2 directly compares pdLIP with IPOPT for approximate solutions. On 200-variable box-constrained QPs, both methods solve 100% of the test instances to the same 10−2 KKT tolerance, with reported average times of 4.02 ms for pdLIP versus 85.27 ms for IPOPT. This demonstrates the computational benefit of pdLIP's GPU-friendly updates under the reported protocol: batched GPU inference for pdLIP and sequential CPU solves for IPOPT, with GPU input transfer excluded (Section 5.1).
For high-accuracy solutions, Table 1 shows that pdLIP initialization reduces pdProj's reported total online time from 7.41 to 2.44 s on the 200-variable convex QPs with all problem data varying, at the same 10−8 KKT tolerance. This approximately threefold speedup includes warm-start generation. The saving is 4.97 s per problem, equivalent to about 83 minutes over 1,000 solves under this protocol.
These savings accumulate as the trained model is reused, without retraining for each instance. Self-supervised training also avoids the expense of generating optimal solutions as training targets. This makes repeated-solve workloads the natural setting for pdLIP: the training investment is worthwhile once the accumulated online savings exceed the offline training and model-selection costs.