- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
NLPOpt-Net combines learned primal–dual predictions with an objective-aware projection scheme for parametric nonlinear optimization. The network warm-starts a sequence of local quadratic subproblems, each solved using a modified Chambolle–Pock algorithm to improve feasibility and solution quality. Unlike Euclidean projection, these subproblems incorporate a quadratic approximation of the original objective. Training uses a modified Lagrangian loss with primal–dual consistency regularization and implicit differentiation through the projection.
Critical strengths
- The paper is generally well organized, and the architecture and computational pipeline are clearly presented.
- The objective-aware projection, which combines a diagonal quadratic approximation of the objective with sequential constraint linearization, is an interesting and potentially novel approach to restoring feasibility while improving solution quality.
- The reported objective values are very close to those obtained by conventional solvers on the evaluated benchmarks.
Critical weaknesses
- Restoring feasibility requires repeated QP solves using a first-order method. Beyond QPs with affine constraints, limited local approximation accuracy may require more layers and higher computational cost.
- The single-layer exactness claim for quadratic objectives and affine constraints is generally incorrect: the scaled diagonal Hessian need not equal the original Hessian (p. 4; Appendix F).
- The baselines should include recent relevant methods such as FSNet [1] and Bisection Projection [2].
- Solver runtimes differ between Tables 1 and 2. Please clarify whether the instances, initialization, tolerances, and timing boundaries are identical, and distinguish batch time from per-instance time.
- Section 3.2 lacks objective values, numerical feasibility measures, and solver comparisons, making large-scale performance difficult to assess.
[1] Hoang T. Nguyen and Priya L. Donti. FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees. Advances in Neural Information Processing Systems (NeurIPS), 2025.
[2] Enming Liang and Minghua Chen. Efficient Bisection Projection to Ensure Neural-Network Solution Feasibility for Optimization over General Set. ICML 2025, PMLR 267:37071–37099.
Presentation Corrections
The following errors should be corrected; the authors should also carefully proofread the entire manuscript and check notation consistently.
Main text
- p. 1, lines 41–42: Inequality constraints are written as
g_j(x,y) = 0; this should beg_j(x,y) <= 0. - p. 3, Figure 1 caption: Delete the repeated word in “the the original objective.”
- p. 6, Algorithm 2: The loop computes
z^{k+1}, but the output is assignedz^k. Please correct the apparent off-by-one error. - p. 7, Eq. (4): The objective is missing
yafterc^T. - p. 8, line 380: Remove the repetition in “similar to similar to QP case.”
- p. 9, line 462: Correct “just-in-tim” to “just-in-time.”
- p. 9, lines 464–465: Replace “Results in Table 2 shows” with “The results in Table 2 show.”
- p. 9, line 481: Remove the extra period in “tolerance..”.
Appendices
- Appendix C, p. 15: Replace “for a fare comparison” with “for a fair comparison.”
- Appendix H, p. 29, Eq. (56): The original Lagrangian should contain
lambda^T h(y), notlambda^T grad h(y).
Questions for authors
- The multipliers of a locally approximated QP generally differ from those of the original NLP, although they can agree at an optimal fixed point. Is it therefore more accurate to interpret the network as learning a primal–dual warm start for the sequence of k approximate subproblems?
- Compared with FSNet's feasibility-seeking procedure and penalty-based training, what specific advantage does NLPOpt-Net offer in feasibility guarantees, achieved feasibility, or computational cost at matched accuracy?
- For larger problems and problems with second-order cone constraints, would the generic linearization approach require more projection layers and longer solve times? How does this change when the cone structure is handled directly, as discussed in Appendix G.4? Please report layer counts, actual CP iterations per layer, and runtime as problem size and constraint complexity increase.
2 comments
Response to Reviewer 3: Questions
Questions:
-
We thank the reviewer for this insightful observation and appreciate this interesting interpretation of our framework. Indeed, the backbone network can be viewed as learning a primal–dual warm start for the sequential quadratic subproblems. However, its role extends beyond initialization, as the network is trained jointly with the projection layers to produce predictions that facilitate constraint satisfaction and improve objective quality within a limited number of layers. Thus, the framework aims not only to provide effective warm starts but also to learn predictions that enable efficient feasibility restoration through objective-aware projection.
-
We have added FSNet to our baseline comparisons, evaluating objective quality, constraint feasibility, and computational performance (Table 1). The results indicate that NLPOpt-Net achieves better objective quality with lower computational time in the evaluated cases. Unlike FSNet's feasibility-seeking correction, NLPOpt-Net incorporates the original objective directly into the projection procedure, enabling objective-aware feasibility restoration. These results highlight the practical advantages of the proposed approach.
-
Our empirical studies suggest that using fewer projection layers during training encourages the backbone network to learn better initial predictions. Furthermore, as the problem size increases, we do not observe a significant increase in the number of inner-solver iterations. Each projection subproblem is formulated as a QP, and we employ equilibration to improve numerical conditioning and reduce the number of Chambolle–Pock iterations required for convergence. These results demonstrate the computational scalability of the proposed approach in the evaluated settings. The results can be found in Appendix J.
Response to Reviewer 3: Weaknesses
Comment:
We thank the reviewer for their valuable time. We believe to have addressed the comments (here and in the revised manuscript - relevant changes highlighted in blue). We hope these changes highlight the strength of our method, and that the reviewer will consider increasing the score accordingly.
Weakness:
We employ an inversion-free variant of the Chambolle–Pock primal-dual algorithm to improve computational efficiency. We acknowledge that repeated projections can become expensive for general nonlinear problems. However, NLPOpt-Net can provide a sufficiently accurate initialization for conventional solvers, significantly reducing their solution time. Our additional vehicle trajectory optimization experiments demonstrate this practical advantage, highlighting the potential of NLPOpt-Net as an efficient learned warm-start strategy.
We thank the reviewer for identifying this issue. We have corrected the claim in the revised manuscript and Appendix F. A single projection layer recovers the exact solution of the original QP only when the constraints are affine and the quadratic objective has a diagonal Hessian exactly represented by the surrogate, assuming an exact inner solve. For general non-diagonal Hessians, the projection solves an approximate QP and does not necessarily recover the original optimal solution.
We have expanded our baseline comparisons to include FSNet, Pi-Net, and the updated ENFORCE method. Bisection Projection was not included because its experimental solution-prediction network is trained using supervised learning with optimizer-generated solutions, whereas NLPOpt-Net employs fully unsupervised training without requiring precomputed optimal solutions. We acknowledge that Bisection Projection also incorporates an unsupervised interior-point prediction component. Given the scope of this study, we have prioritized comparisons with methods that do not require optimal solution labels for their primary solution-prediction networks.
The runtime consistency issue occurs in Table 1 and Table 2 (now Table 3) due to the use of different machines. The timings in Table 1 is done while training the model in cluster where as the Table 2 runs the trained model in a CPU with less resources.
We have added a new metric Table in Section 3.2 to include objective quality, constraint feasibility, and computational performance as problem size increases, demonstrating the scalability of NLPOpt-Net. In the vehicle trajectory optimization case study, IPOPT failed to solve 31 out of 37 sample instances within the 60-second time budget, whereas NLPOpt-Net generated solutions for all 37 instances. These results further highlight the practical advantages of NLPOpt-Net for computationally challenging optimization problems. Additionally, we have conducted an initialization study using the numerical solver IPOPT on the vehicle trajectory optimization case study.W ithout learned initialization, IPOPT successfully completed only 6 out of 37 instances within the 60-second time budget. In contrast, using the learned neural-network predictions as initialization, IPOPT successfully solved all 37 instances, with significantly reduced computational time. Furthermore, initializing IPOPT with the projected solutions reduced the mean solver time to 1.33 seconds, compared with 50.70 seconds for the six completed vanilla IPOPT runs. These results demonstrate the effectiveness of learned initialization and highlight the potential of NLPOpt-Net to accelerate conventional numerical optimization.
We have now included sparse computation kernels for the projection procedure. We train all NLPOptNet models with the sparse kernels.
Presentation Corrections:
We thank the reviewer for carefully going through the manuscript. We have made the mentioned corrections and gone through the manuscript for consistency check.