acceptodds
Under review as a conference paper at ICLR 2027

NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees

Abstract

We introduce the Nonlinear Parametric Optimization Network (NLPOpt-Net), an unsupervised learning architecture for solving parametric nonlinear programs (NLPs) across varying parameter instances while guaranteeing constraint feasibility. The architecture consists of a backbone neural network (NN) followed by a multilayer (-layered) projection to enforce nonlinear constraints. NLPOpt-Net employs an inversion-free, modified Chambolle–Pock algorithm for rapid projection during the forward pass, leveraging local quadratic approximations of the original NLP, and uses the implicit function theorem for efficient backpropagation. Numerical experiments suggest that the projection improves solution quality by driving the predictions closer to optimality while preserving feasibility. NLPOpt-Net solves convex multiparametric QP, QCQP, and NLP problems with near zero optimality gap and constraint violations reduced to machine precision, thereby enabling a scalable approach for multiparametric programming. While a first-order Taylor approximation provides a feasible projection for convex problems, we observe that, with appropriate linearization, NLPOpt-Net can also solve nonconvex problems. While the models are trained using JAX in Pyhton, the fixed structure of the projection further allows compiling a trained model in C. The compiled models provide significant improvement in inference time. Finally, we consider a nonconvex collision-free fleet trajectory optimization problem to demonstrate the ability of NLPOpt-Net to incorporate large number of constraints.

Primary area:
optimization
Keywords:
Physics constrained machine learning, parametric optimization, nonlinear optimization, trajectory optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eJucOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

NLPOpt-Net combines learned primal–dual predictions with an objective-aware projection scheme for parametric nonlinear optimization. The network warm-starts a sequence of local quadratic subproblems, each solved using a modified Chambolle–Pock algorithm to improve feasibility and solution quality. Unlike Euclidean projection, these subproblems incorporate a quadratic approximation of the original objective. Training uses a modified Lagrangian loss with primal–dual consistency regularization and implicit differentiation through the projection.

Critical strengths
  • The paper is generally well organized, and the architecture and computational pipeline are clearly presented.
  • The objective-aware projection, which combines a diagonal quadratic approximation of the objective with sequential constraint linearization, is an interesting and potentially novel approach to restoring feasibility while improving solution quality.
  • The reported objective values are very close to those obtained by conventional solvers on the evaluated benchmarks.
Critical weaknesses
  1. Restoring feasibility requires repeated QP solves using a first-order method. Beyond QPs with affine constraints, limited local approximation accuracy may require more layers and higher computational cost.
  2. The single-layer exactness claim for quadratic objectives and affine constraints is generally incorrect: the scaled diagonal Hessian need not equal the original Hessian (p. 4; Appendix F).
  3. The baselines should include recent relevant methods such as FSNet [1] and Bisection Projection [2].
  4. Solver runtimes differ between Tables 1 and 2. Please clarify whether the instances, initialization, tolerances, and timing boundaries are identical, and distinguish batch time from per-instance time.
  5. Section 3.2 lacks objective values, numerical feasibility measures, and solver comparisons, making large-scale performance difficult to assess.

[1] Hoang T. Nguyen and Priya L. Donti. FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees. Advances in Neural Information Processing Systems (NeurIPS), 2025.

[2] Enming Liang and Minghua Chen. Efficient Bisection Projection to Ensure Neural-Network Solution Feasibility for Optimization over General Set. ICML 2025, PMLR 267:37071–37099.

Presentation Corrections

The following errors should be corrected; the authors should also carefully proofread the entire manuscript and check notation consistently.

Main text

  • p. 1, lines 41–42: Inequality constraints are written as g_j(x,y) = 0; this should be g_j(x,y) <= 0.
  • p. 3, Figure 1 caption: Delete the repeated word in “the the original objective.”
  • p. 6, Algorithm 2: The loop computes z^{k+1}, but the output is assigned z^k. Please correct the apparent off-by-one error.
  • p. 7, Eq. (4): The objective is missing y after c^T.
  • p. 8, line 380: Remove the repetition in “similar to similar to QP case.”
  • p. 9, line 462: Correct “just-in-tim” to “just-in-time.”
  • p. 9, lines 464–465: Replace “Results in Table 2 shows” with “The results in Table 2 show.”
  • p. 9, line 481: Remove the extra period in “tolerance..”.

Appendices

  • Appendix C, p. 15: Replace “for a fare comparison” with “for a fair comparison.”
  • Appendix H, p. 29, Eq. (56): The original Lagrangian should contain lambda^T h(y), not lambda^T grad h(y).
Questions for authors
  1. The multipliers of a locally approximated QP generally differ from those of the original NLP, although they can agree at an optimal fixed point. Is it therefore more accurate to interpret the network as learning a primal–dual warm start for the sequence of k approximate subproblems?
  2. Compared with FSNet's feasibility-seeking procedure and penalty-based training, what specific advantage does NLPOpt-Net offer in feasibility guarantees, achieved feasibility, or computational cost at matched accuracy?
  3. For larger problems and problems with second-order cone constraints, would the generic linearization approach require more projection layers and longer solve times? How does this change when the cone structure is handled directly, as discussed in Appendix G.4? Please report layer counts, actual CP iterations per layer, and runtime as problem size and constraint complexity increase.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3: Weaknesses

Comment:

We thank the reviewer for their valuable time. We believe to have addressed the comments (here and in the revised manuscript - relevant changes highlighted in blue). We hope these changes highlight the strength of our method, and that the reviewer will consider increasing the score accordingly.

Weakness:

  1. We employ an inversion-free variant of the Chambolle–Pock primal-dual algorithm to improve computational efficiency. We acknowledge that repeated projections can become expensive for general nonlinear problems. However, NLPOpt-Net can provide a sufficiently accurate initialization for conventional solvers, significantly reducing their solution time. Our additional vehicle trajectory optimization experiments demonstrate this practical advantage, highlighting the potential of NLPOpt-Net as an efficient learned warm-start strategy.

  2. We thank the reviewer for identifying this issue. We have corrected the claim in the revised manuscript and Appendix F. A single projection layer recovers the exact solution of the original QP only when the constraints are affine and the quadratic objective has a diagonal Hessian exactly represented by the surrogate, assuming an exact inner solve. For general non-diagonal Hessians, the projection solves an approximate QP and does not necessarily recover the original optimal solution.

  3. We have expanded our baseline comparisons to include FSNet, Pi-Net, and the updated ENFORCE method. Bisection Projection was not included because its experimental solution-prediction network is trained using supervised learning with optimizer-generated solutions, whereas NLPOpt-Net employs fully unsupervised training without requiring precomputed optimal solutions. We acknowledge that Bisection Projection also incorporates an unsupervised interior-point prediction component. Given the scope of this study, we have prioritized comparisons with methods that do not require optimal solution labels for their primary solution-prediction networks.

  4. The runtime consistency issue occurs in Table 1 and Table 2 (now Table 3) due to the use of different machines. The timings in Table 1 is done while training the model in cluster where as the Table 2 runs the trained model in a CPU with less resources.

  5. We have added a new metric Table in Section 3.2 to include objective quality, constraint feasibility, and computational performance as problem size increases, demonstrating the scalability of NLPOpt-Net. In the vehicle trajectory optimization case study, IPOPT failed to solve 31 out of 37 sample instances within the 60-second time budget, whereas NLPOpt-Net generated solutions for all 37 instances. These results further highlight the practical advantages of NLPOpt-Net for computationally challenging optimization problems. Additionally, we have conducted an initialization study using the numerical solver IPOPT on the vehicle trajectory optimization case study.W ithout learned initialization, IPOPT successfully completed only 6 out of 37 instances within the 60-second time budget. In contrast, using the learned neural-network predictions as initialization, IPOPT successfully solved all 37 instances, with significantly reduced computational time. Furthermore, initializing IPOPT with the projected solutions reduced the mean solver time to 1.33 seconds, compared with 50.70 seconds for the six completed vanilla IPOPT runs. These results demonstrate the effectiveness of learned initialization and highlight the potential of NLPOpt-Net to accelerate conventional numerical optimization.

  6. We have now included sparse computation kernels for the projection procedure. We train all NLPOptNet models with the sparse kernels.

Presentation Corrections:

We thank the reviewer for carefully going through the manuscript. We have made the mentioned corrections and gone through the manuscript for consistency check.

AuthorsOpenReview ↗

Response to Reviewer 3: Questions

Questions:

  1. We thank the reviewer for this insightful observation and appreciate this interesting interpretation of our framework. Indeed, the backbone network can be viewed as learning a primal–dual warm start for the sequential quadratic subproblems. However, its role extends beyond initialization, as the network is trained jointly with the projection layers to produce predictions that facilitate constraint satisfaction and improve objective quality within a limited number of layers. Thus, the framework aims not only to provide effective warm starts but also to learn predictions that enable efficient feasibility restoration through objective-aware projection.

  2. We have added FSNet to our baseline comparisons, evaluating objective quality, constraint feasibility, and computational performance (Table 1). The results indicate that NLPOpt-Net achieves better objective quality with lower computational time in the evaluated cases. Unlike FSNet's feasibility-seeking correction, NLPOpt-Net incorporates the original objective directly into the projection procedure, enabling objective-aware feasibility restoration. These results highlight the practical advantages of the proposed approach.

  3. Our empirical studies suggest that using fewer projection layers during training encourages the backbone network to learn better initial predictions. Furthermore, as the problem size increases, we do not observe a significant increase in the number of inner-solver iterations. Each projection subproblem is formulated as a QP, and we employ equilibration to improve numerical conditioning and reduce the number of Chambolle–Pock iterations required for convergence. These results demonstrate the computational scalability of the proposed approach in the evaluated settings. The results can be found in Appendix J.

Reviewer_kANGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents NLPOpt-Net, a learning-to-optimize method that learns the solution map of a parametric constrained nonlinear programming problem. To address the main challenge of constraint satisfaction, the paper proposes a projection layer based on a modified Chambolle–Pock algorithm, which projects the solution predicted by the neural network onto a feasible solution. To train the network, the authors use a regularized Lagrangian loss function, enabling a self-supervised learning approach that bypasses the need to collect training data by solving the underlying optimization problem. The predicted solution is then projected using a (K)-layer projection procedure, where each layer solves a quadratic program (QP) that approximates the original nonlinear program. Experiments are conducted on synthetic convex QPs, QCQPs, convex NLPs, and a nonconvex benchmark from the DC3 paper. The proposed method is compared against a solver, a soft-constrained network, a predict-and-complete network, and DC3.

Critical strengths

The main strength of the paper is that it addresses an important challenge in the learning-to-optimize literature: learning the solution map of a constrained parametric optimization problem while ensuring feasibility without sacrificing solution quality or computational efficiency. To address this challenge, the paper proposes combining a backbone neural network, trained using a penalty-based loss function, with a projection layer that uses a KKT-based representation to formulate a QP and recover a feasible solution. To handle nonlinear constraints, the method employs (K) layers of QP approximations. This combination is novel, as most existing approaches rely on Euclidean projection. Furthermore, the use of a KKT-based representation that requires computing only a diagonal Hessian makes the projection computationally tractable, which is another strength of the proposed approach. The paper also evaluates the method across different problem classes, demonstrating its applicability to a range of optimization problems. The reported results show a near-zero optimality gap and no constraint violations.

Critical weaknesses

One major weakness concerns the interaction between the two main components of the proposed pipeline. While the backbone network learns approximate primal and dual solutions, the paper does not investigate how the quality of these predictions affects the quality of the final projected solution.This is closely related to the computational tradeoff between obtaining a less accurate but computationally inexpensive solution from the backbone network and using additional projection layers, which introduce higher computational costs.

It is unclear how much the projection modifies the solution predicted by the network, particularly since, unlike previous methods, the entire predicted primal solution is passed to the projection layers. Consequently, the role of training in determining the final projected solution quality remains unclear, as does the extent to which the backbone network serves primarily as a warm start for the projection procedure.

Furthermore, each projection layer requires solving a QP, which can be computationally expensive. Thus, the computational overhead increases with the number of projection layers. This tradeoff is not directly addressed in the paper, although it is particularly important given that one of the main motivations for learning-to-optimize methods is to achieve faster runtimes than standard optimization solvers. This can actually be observed in Table 1 where the reported runtimes raise concerns about the computational efficiency of the proposed method, as NLPOpt-Net is slower than the solver across all settings and approximately an order of magnitude slower than DC3. Given that computational efficiency is one of the main motivations behind learning-to-optimize methods, I believe that these results weaken one of the main motivations of the paper, namely achieving faster runtimes than standard solvers, and that the authors need to better justify the computational advantage of the proposed approach.

The experimental evaluation could also be strengthened in several aspects.

First, the results compare NLPOpt-Net, which undergoes hyperparameter tuning, against DC3 using its default settings. This difference in experimental settings weakens the strength of the comparison.

Second, DC3 is the only directly considered baseline of this type, while other methods, such as PDL (Self-Supervised Primal-Dual Learning for Constrained Optimization), could also be considered to better demonstrate the significance of the proposed approach.

Third, an ablation study examining the number of projection layers and the evolution of the solution as additional layers are introduced would be valuable. Since the projection mechanism is the key component responsible for ensuring feasibility, the number of projection layers may have a substantial impact, particularly for problems involving nonlinear or nonconvex constraints. Another weakness of the paper is its positioning relative to the existing literature. More emphasis should be placed on distinguishing the proposed contributions from existing approaches. I believe the paper would greatly benefit from including additional references throughout the presentation to clarify which components are novel and which are based on existing ideas. For example, in the Backbone Neural Network section, the authors do not discuss closely related work, most notably Self-Supervised Primal-Dual Learning for Constrained Optimization, which proposes a similar training loss and discuss the Self-Supervised Penalty Approach .

Questions for authors

1- Given that one of the main motivations behind learning-to-optimize methods is to reduce computational time compared to standard solvers, I am wondering what the main advantage of the proposed approach is, particularly since the reported runtimes are higher than those of standard solvers, while DC3 achieves a reduction in runtime compared to solvers. Could the authors better justify the novelty, motivation and practical advantage of the proposed method?

2- I am wondering how the quality of the primal and dual solutions predicted by the backbone network affects the quality of the final projected solution. Is the method more forgiving since the output of the NN is projected using k-layers and in particular, how much does the projection modify the original prediction of the network? And does the quality of its prediction have a significant impact on the final solution?

3- The backbone predicts both primal and dual variables, but the predicted duals only seem to serve as the initial point of the Chambolle–Pock algorithm, and the loss uses the duals returned by the projection. How sensitive is the method to the predicted dual variables, and how does it perform without dual prediction?

4- Since each projection layer requires solving a QP, I am wondering how the computational cost evolves with the number of projection layers. In particular, is there a tradeoff between using a less accurate but cheaper prediction from the backbone network and increasing the number of projection layers? This seems particularly important given that the main motivation behind learning-to-optimize methods is to reduce computational time compared to standard solvers.

5- Regarding the experimental comparison with DC3, could the authors explain why DC3 is evaluated using its default hyperparameters while NLPOpt-Net undergoes hyperparameter tuning?

6- Finally, I am wondering how the number of projection layers affects the final solution, particularly for problems with nonlinear or nonconvex constraints. Could the authors provide an ablation study showing how the solution evolves as additional projection layers are introduced?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 2: Weaknesses

Comment:

We thank the reviewer for the constructive comments and their valuable time. We believe to have addressed the comments (here and in the revised manuscript - relevant changes highlighted in blue). We hope these changes highlight the strength of our method, and that the reviewer will consider increasing the score accordingly.

Weaknesses:

We thank the reviewer for these constructive suggestions. We have added a comprehensive ablation study in Appendix J to investigate the effect of backbone prediction quality, the role of learned initialization, and the modifications introduced by successive projection layers. This analysis further clarifies the interaction between the neural network and the projection procedure.

We have also expanded our experimental comparisons by including additional baselines, such as Pi-Net, ENFORCE, and FSNet.

Regarding PDL (Self-Supervised Primal-Dual Learning), we acknowledge its relevance as a closely related self-supervised approach. However, PDL primarily employs an augmented-Lagrangian-based training strategy with jointly learned primal and dual networks, whereas NLPOpt-Net incorporates an explicit objective-aware projection procedure that iteratively improves constraint satisfaction and solution quality. Given the scope of our experimental study, we have prioritized comparisons with projection-based methods.

Finally, we have revised the Introduction and Conclusion to more clearly distinguish our methodological contributions from existing learning-to-optimize approaches and emphasize the novelty of the objective-aware projection framework.

We have now included sparse computation kernels for the projection procedure. We train all NLPOptNet models with the sparse kernels.

AuthorsOpenReview ↗

Response to Reviewer 2: Questions

Questions:

  1. We thank the reviewer for this important comment. The computational analyses reports timings for both batch settings and single instance settings. We have expanded our experimental evaluation to demonstrate the practical advantages of NLPOpt-Net (section 3.3), particularly for large-scale optimization problems and as a learned initialization for conventional numerical solvers. The additional experiments demonstrate substantial improvements in solver efficiency when using NLPOpt-Net predictions as warm starts. Furthermore, NLPOpt-Net achieves improved objective quality compared with competing learning-based methods in the evaluated settings. These results highlight that the primary advantage of NLPOpt-Net lies not only in computational speed but also in solution quality, scalability, and its ability to accelerate conventional optimization solvers.

  2. We have added an ablation study in Appendix J to investigate the impact of learned initialization on the final projected solution. The results demonstrate that without learned initialization, significantly more projection layers are required to achieve feasibility, with a substantial deterioration in objective quality. In contrast, learned predictions require fewer projection layers while producing higher-quality solutions. These findings highlight the importance of the backbone network in improving both projection efficiency and final solution quality.

  3. We have added an ablation study on dual initialization in Appendix J. The results show that learned dual predictions have a relatively small impact on the final objective quality. However, without learned dual initialization, the number of Chambolle–Pock iterations required for the inner solver to converge increases. This demonstrates that dual predictions primarily improve the computational efficiency of the projection procedure.

  4. Our ablation study demonstrates that better backbone predictions require fewer projection layers to achieve the same level of feasibility. While the computational cost increases approximately linearly with the number of projection layers for a comparable inner-solver cost, learned initialization significantly reduces the required projection depth. This highlights the tradeoff between prediction quality and projection cost, demonstrating how training improves the overall computational efficiency of NLPOpt-Net.

  5. We thank the reviewer for raising this concern. We acknowledge that differences in hyperparameter tuning may influence the comparison. DC3 was evaluated using its publicly available default configuration to provide a reproducible reference implementation, whereas NLPOpt-Net was tuned for the proposed architecture. We have clarified this distinction in the revised manuscript and acknowledge it as a limitation of the comparison. Furthermore, we have expanded our experimental evaluation to include additional baselines, including Pi-Net, ENFORCE, and FSNet, to provide a broader assessment of the proposed method beyond DC3.

  6. We have added a detailed ablation study in Appendix examining the effect of the number of projection layers on the final solution, including the evolution of objective values and constraint violations across successive layers for nonlinear and nonconvex optimization problems.

Reviewer_sQdJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NLPOpt-Net for learning solutions to parametric nonlinear optimization problems. A backbone network predicts primal and dual variables, followed by several correction layers that linearize the constraints and approximate the objective using a scaled diagonal Hessian. Each resulting structured QP is solved with an inversion-free modified Chambolle-Pock method, and gradients are computed using implicit differentiation. Training uses an objective-based loss with constraint and consistency terms. The paper presents a local convergence analysis, evaluates synthetic QP, QCQP, and NLP instances and a nonconvex fleet trajectory problem, and reports deployment timings for compiled models. Its central claim is that the correction layers improve optimality while enforcing feasibility.

Critical strengths
  1. Making the correction objective-aware is well motivated. A correction based only on distance to the network prediction can improve feasibility while worsening objective value. Combining a local objective model with a structured primal-dual solver provides a concrete way to address that trade-off, and the diagonal structure allows inexpensive inner updates.

  2. The implementation study considers practical aspects beyond average prediction error. It evaluates constraint residuals and objective quality, includes a trajectory problem with thousands of constraints, and reports single-instance inference with ahead-of-time compilation. These experiments make the proposed architecture relevant to repeated optimization in deployment.

Critical weaknesses
  1. The stated feasibility guarantee and convergence analysis do not yet justify the implemented fixed-depth architecture. A tangent approximation to a convex nonlinear inequality is an outer approximation: satisfying the linearized inequality does not imply satisfaction of the original constraint. Section 2.2 recognizes that a single layer may be insufficient, but the local convergence result requires a sufficiently close initialization and an approximation-error condition. It does not establish a prescribed original-constraint tolerance after a fixed number of layers for arbitrary network predictions, especially with inexact inner solves. There is also a mismatch between the analyzed and implemented maps. The main algorithm recomputes the objective model at successive iterates, whereas Appendix F formulates convergence for a fixed quadratic objective centered at the original prediction. An analysis for the original NLP should account for the objective-Hessian diagonalization error as well as constraint-curvature error. Finally, the claim that one layer exactly solves any problem with a quadratic objective and affine constraints does not hold for a general non-diagonal Hessian after it is replaced by a scaled diagonal approximation. One layer may enforce the affine constraints, but exact optimality for the original QP is a different claim. These distinctions are central to the paper's title and contribution.

  2. The experiments do not sufficiently isolate the new contribution or establish its scope on general nonlinear constraints. Comparisons with Pi-net, the updated ENFORCE method, and learned numerical-solver warm starts are important, as is applying the identical correction procedure to non-learned initializations. Without these controls, it is difficult to separate the contribution of learning and the proposed objective model from the work performed by the numerical backend. The QCQP generator also has a particularly restrictive structure: C_i=alpha_i r_i r_i^T and d_i=r_i, so each inequality is a convex scalar quadratic in r_i^T y. For each fixed parameter instance, its feasible interval can be represented by affine bounds on that scalar. This limits what the QCQP results establish about general nonlinear constraint handling, even though the paper also includes other NLP and trajectory examples. More general coupled nonlinear tests and consistent timing conventions, interfaces, and achieved accuracies are needed to assess the claimed advantage.

Questions for authors
  1. What exact feasibility statement holds after the implemented fixed number of layers? Please specify the required initialization neighborhood, the relation between layer count and the original-constraint tolerance, admissible inner-solver errors, and the procedure used when the target tolerance is not reached.

  2. Which iteration does Appendix F analyze: repeated constraint linearization with a fixed quadratic objective, or the algorithm that rebuilds the objective approximation at each layer? Please reconcile the two maps, include the objective-Hessian approximation error where needed, and specify when a single layer solves the original QP rather than only its surrogate.

  3. Can you compare against Pi-net, the updated ENFORCE method, learned solver warm starts, and the same correction method initialized without a learned network, under matched stopping tolerances and total online work? These controls would directly affect my assessment of the methodological contribution.

  4. Does the advantage persist for QCQPs with higher-rank, coupled quadratic constraints that cannot be reduced to affine bounds on individual projections? Please also clarify the timing units and batch-to-per-instance conversion, and separate compilation and interface effects from algorithmic speed at matched feasibility and objective accuracy.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1: Weaknesses

Comment:

We thank the reviewer for the constructive comments and the time spent in reviewing the manuscript. We believe to have addressed the comments (here and in the revised manuscript - relevant changes highlighted in blue). We hope these changes highlight the strength of our method, and that the reviewer will consider increasing the score accordingly.

Weakness:

  1. We have revised the convergence analysis to account for the original nonlinear objective, including the objective-Hessian diagonalization error and constraint-curvature error, and clarified the conditions for local convergence. We agree that a fixed number of projection layers does not universally guarantee feasibility. Therefore, we have modified the inference procedure to adaptively execute additional projection layers until the prescribed original-constraint tolerance is satisfied, subject to a computational budget, while retaining fixed-depth projection during training for computational efficiency. We have also corrected our exact-solution claim: a single layer recovers the exact solution for quadratic objectives with diagonal Hessians and affine constraints, assuming an exact inner solve.

  2. We have expanded our baseline comparisons by including Pi-Net, the updated ENFORCE method, and FSNet. Additionally, we have conducted an initialization study using the numerical solver IPOPT on the vehicle trajectory optimization case study.\ Without learned initialization, IPOPT successfully completed only 6 out of 37 instances within the 60-second time budget. In contrast, using the learned neural-network predictions as initialization, IPOPT successfully solved all 37 instances, with significantly reduced computational time. Furthermore, initializing IPOPT with the projected solutions reduced the mean solver time to 1.33 seconds, compared with 50.70 seconds for the six completed vanilla IPOPT runs. These results demonstrate the effectiveness of learned initialization and highlight the potential of NLPOpt-Net to accelerate conventional numerical optimization.

  3. We have now included sparse computation kernels for the projection procedure. We train all NLPOptNet models with the sparse kernels.

Vehicle optimization results over 37 instances. Warm-start timings exclude neural-network inference and projection. Maximum primal violations are rounded to three decimal places. All times are in seconds per instance.

MethodWarm startCompleted calls (n/37)Max primal violationMean objectiveMean timeMedian timeMax time
NLPOpt-Net—37/370.0076.2990.1930.1900.216
Vanilla IPOPT*—6/370.00018.99450.69554.52859.635
IPOPTNN primal37/370.0004.8693.5091.53118.523
IPOPTNN primal–dual37/370.0004.8745.9971.52054.801
IPOPTProjected primal37/370.0004.8481.3300.7116.609
IPOPTProjected primal–dual37/370.0004.8691.4260.6957.566

* Vanilla IPOPT violation, objective, and timing statistics are computed over the six completed runs. The remaining 31 runs reached the 60-second time budget.

AuthorsOpenReview ↗

Response to Reviewer 1: Questions

Questions:

  1. We do not claim that an arbitrary fixed number of projection layers guarantees feasibility for all network predictions. For convex problems, including the QP, QCQP, and NLP cases considered in this study, a single projection layer may be sufficient to achieve the prescribed feasibility tolerance when the subproblem is solved accurately. Our revised analysis establishes local linear convergence under LICQ, second-order sufficient conditions, a sufficiently close initialization, and bounded curvature-approximation error. Consequently, the number of layers required to achieve a prescribed original-constraint tolerance depends on the initialization error and local convergence rate. With inexact inner solves, an additional solver-error term must be accounted for, and the inner-solver accuracy must be sufficiently small relative to the target tolerance.

Additionally, we have modified the inference procedure to adaptively execute additional projection layers when the prescribed tolerance is not satisfied. After each layer, feasibility is evaluated using the original nonlinear constraints rather than their linearized approximations. Projection continues until the target tolerance is achieved or the computational budget is exhausted.

  1. We have revised Appendix F to analyze the implemented iterative projection procedure, where both the objective approximation and constraint linearizations are updated at each layer. The revised proof accounts for both objective-Hessian diagonalization and constraint-curvature errors. We have also corrected the exact-solution claim, clarifying that a single layer solves the original QP exactly only when the quadratic objective has an exactly represented diagonal Hessian and affine constraints, assuming an exact inner solve.

  2. We added comparisons with Pi-Net, the updated ENFORCE method, and FSNet (Table 1), along with an ablation that applies the same NLPOpt-Net correction procedure from random primal starts with zero duals (Appendix J). Without learned initialization, the linearized correction generally required more layers to reach the 10−610^{-6} constraint-violation tolerance, and the returned solutions had substantially worse objective values. These results show that learned initialization improves both the efficiency of feasibility correction and the quality of the returned solutions.

  3. We have added a new illustrative QCQP exaple with higher-rank, coupled quadratic constraints that cannot be reduced to affine bounds on individual projections (in Appendix K). We have also revised Table 1 to explicitly report both per-batch and per-instance execution times, clarifying the timing units and batch-size conversion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.