acceptodds
Under review as a conference paper at ICLR 2027

lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers

Abstract

Differentiable optimization brings the structural guarantees of mathematical optimization to network pipelines, allowing them to be trained end-to-end. However, its application remains challenging for nonconvex constrained problems, as existing differentiable solvers often suffer from limited modeling expressiveness due to their reliance on specialized problem structures, while also incurring substantial computation time and memory overhead in both the forward and backward passes. To address these challenges, we propose lapanda, a matrix-free differentiable solver for nonconvex optimization with general constraints. It reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization. We establish local well-posedness of the solution map and convergence of the outer iterations, and further derive a sensitivity alignment between the original problem and the final subproblem in the backward pass, demonstrating that the subproblem sensitivity, which can be computed efficiently in a matrix-free manner, provides a principled approximation to the exact optimizer sensitivity. We evaluate lapanda on nonconvex constrained Rosenbrock benchmarks, imitation learning with several representative constrained optimal control problems, and embedded robotic obstacle-avoidance tasks. Compared with state-of-the-art differentiable solvers, lapanda delivers substantial reductions in computation time and memory footprint while maintaining reliable constraint satisfaction and learning performance.

Primary area:
optimization
Keywords:
Differentiable Optimization, Matrix-free Solver, Model Predictive Control, Robot Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ge4qOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LAPANDA, a matrix-free differentiable solver for nonconvex constrained optimization. The forward pass combines ALM with the first-order PANDA solver, while the backward pass approximates optimizer sensitivities using those of the final ALM subproblem. The paper establishes local convergence guarantees and a sensitivity error bound. Experiments demonstrate promising improvements in runtime and memory efficiency.

Critical strengths
  1. The work addresses an important bottleneck in differentiable optimization and provides a practical, cross-platform implementation suitable for resource-constrained deployment.
  2. The matrix-free approach for general nonconvex constrained programs avoids explicit matrix factorization, relying on operator-based derivatives for efficient forward and backward computation.
  3. Experiments demonstrate substantial improvements in runtime and memory usage over the tested baselines.
Critical weaknesses
  1. The theoretical guarantees rely on strong regularity assumptions and do not cover degenerate cases. Moreover, finite-penalty sensitivity errors may persist even after forward convergence.
  2. The rectangular-obstacle formulation violates LICQ, raising concerns about the reliability of the computed sensitivities.
  3. Some comparisons use inconsistent convergence criteria. Additionally, the failure of acados may stem from the degenerate constraint formulation and initialization.
  4. The method largely integrates established ALM and PANDA frameworks, while the individual contributions of these components are not sufficiently investigated.
Questions for authors
  1. Could the authors provide ablations with alternative inner solvers to isolate PANDA's contribution?
  2. How sensitive are the gradients to penalty parameters, convergence tolerances, and problem conditioning? Does performance degrade on ill-conditioned instances?
  3. Could the authors compare solvers under matched accuracy criteria? How does acados perform on the rectangular-obstacle task with feasible warm starts or a nondegenerate constraint formulation?
4 comments
AuthorsOpenReview ↗

Reponse to reviewer ge4q (1/2)

We thank the reviewer for the feedback on our method and experiments, especially the concerns about sensitivity accuracy. We have expanded the error analysis and added experiments to address these concerns.

W1. Regularity assumptions and finite-penalty errors.

We have emphasized in the main text that the lapanda-ALM backward strategy offers a trade-off between sensitivity accuracy and computational efficiency, and discussed it in the limitations. Its finite-penalty bias can remain after forward convergence. We also provide lapanda-KKT, which differentiates the original KKT conditions without this additional approximation.

Such regularity assumptions are commonly required by nonconvex differentiable solvers to establish local differentiability of the optimizer mapping and justify sensitivity computation.

W2. The rectangular-obstacle formulation violates LICQ.

We introduced this problem to examine lapanda’s empirical performance outside the assumptions of our theory. Degenerate MPCCs are not the main problem class targeted by lapanda. We have clarified the purpose of this experiment and also report results for a smoothed rectangular-obstacle formulation.

W3. Convergence criteria.

Thank you for raising this valuable question. We had not discussed the relationship between stopping criteria and sensitivity accuracy in sufficient detail. Appendix B.6 now analyzes three sources of sensitivity error: forward-point error, numerical backward-solve error, and the finite-penalty bias of lapanda-ALM. Based on this analysis, we have revised the learning comparison to evaluate the computation required to reach the same loss target; see Q3.

W4. Contributions of the individual components.

The core of our work is the identification and exploitation of different problem structures in the backward pass. We analyze the relationship between original-problem and ALM-subproblem sensitivities and develop two matrix-free backward strategies with different accuracy–efficiency characteristics.

For the forward pass, our contribution is primarily the analysis establishing the applicability of PANDA to the ALM subproblems and the local convergence of the resulting method. The PANDA/PANOC ablation is reported under Q1.

Q1. Ablations with alternative inner solvers.

We compare PANDA and PANOC within the same lapanda ALM implementation. Appendix C.4.2 reports the following results; each cell gives mean forward time in milliseconds / total inner iterations.

OCPWarm: PANDAWarm: PANOCCold: PANDACold: PANOC
CartPole0.257 / 37.80.277 / 42.30.264 / 51.40.258 / 51.4
Quadrotor0.976 / 150.11.025 / 159.91.689 / 250.11.717 / 253.3
Robot arm0.952 / 163.91.272 / 230.91.971 / 282.33.641 / 546.7

Each setting contains 18 sampled training inputs. The gains are largest on Robot arm, where adaptive step-size enlargement substantially reduces the number of inner iterations. The differences are smaller on CartPole and Quadrotor.

Q2. Sensitivity to penalties, tolerances and conditioning.

Thank you for this valuable question. Appendix C.4.1 adds a controlled quadratic problem with analytical reference gradients to examine these factors separately.

At the exact forward solution, using exact backward solves:

Smallest constraint singular value δ\deltaPenalty ρ\rhoFinite-penalty gradient bias (%)
1110310^30.0735
10−210^{-2}10310^383.61
10−210^{-2}10710^70.06895

Increasing ρ\rho reduces the bias, but the required penalty depends strongly on conditioning. In the last row, the subproblem condition number increases to approximately 10710^7.

For an ill-conditioned instance with δ=μ=10−4\delta=\mu=10^{-4} and ρ=103\rho=10^3, tightening the backward tolerance gives:

Backward toleranceCG total gradient error (%)CG subproblem-solve error (%)MINRES gradient error (%)
10−310^{-3}60.3356.94×10−26.94\times10^{-2}60.357
10−610^{-6}60.3352.73×10−62.73\times10^{-6}0.155
10−910^{-9}60.3351.35×10−91.35\times10^{-9}1.72×10−71.72\times10^{-7}*

The 10−910^{-9} MINRES runs did not meet the requested residual target.

The CG subproblem-solve error is measured against its exact subproblem gradient. Tightening the tolerance reduces this error, while the total error remains limited by finite-penalty bias.

Finally, with the same forward KKT residual 10−310^{-3} and exact backward solves:

δ\deltaForward-induced gradient error (%)
110.0865
10−210^{-2}4.37
10−410^{-4}377.8

These results show how conditioning amplifies sensitivity errors and why tightening only one tolerance may be insufficient. We have added this observation to the limitations.

AuthorsOpenReview ↗

Reponse to reviewer ge4q (2/2)

Q3. Comparison criteria and the rectangular-obstacle experiment.

As discussed under W3, the overall sensitivity error has several sources. Matching it across all the compared solvers—CasADi/IPOPT, SafePDP, acados and TurboMPC—is difficult. We use common requested forward tolerances within each task. However, the factorization-based backward computations in CasADi/IPOPT, SafePDP and acados typically produce much smaller linear-solve errors than loosely stopped iterative methods such as lapanda and TurboMPC. lapanda-ALM additionally incurs finite-penalty bias. A common numerical backward tolerance therefore would not match gradient accuracy.

We instead compare cumulative complete solver-call wall time to the same recorded training-loss target, using appropriate backward settings for each method:

TaskTarget losslapanda-ALM (s)lapanda-KKT (s)CasADi/IPOPT (s)SafePDP (s)acados (s)TurboMPC-CPU (s)
CartPole/OL0.52.452.60138.63255.3599.01—
CartPole/CL0.0314.0114.04NRNR61.28—
Quadrotor/OL19018.6619.28130.88210.5647.05342.89
Quadrotor/CL2.312.7713.36187.70288.73113.27396.38
Robot arm/OL19.9910.19NRNR56.62564.92
Robot arm/CL0.035.545.70NRNR305.69209.47

NR denotes a target not reached within the prescribed training run; — denotes unsupported.

Under these settings, both lapanda strategies reach all six targets with low computation and memory costs. Their complete-solver memory is 3.5–5.0 MiB, compared with 56.2–277.4 MiB for these external baselines.

Appendix B.6, Step 3 also analyzes approximate stationarity under bounded absolute gradient errors for ordinary gradient descent. Under its stated assumptions, this provides theoretical support for using controlled inexact gradients in outer optimization.

The original manuscript already included a smoothed rectangular-obstacle experiment with feasible initialization and shifted warm starts in Appendix C.3, with a pointer from the main text. Both methods successfully avoid the obstacle in five 30-step MPC rollouts. The updated timings are:

Timed MPC calls after initializationlapanda mean wall time (ms)acados mean wall time (ms)
All 30 steps1.1279.705
Excluding the first step0.6899.982

Thank you for the additional experiments. However, my main concerns regarding sensitivity reliability under ill-conditioning and the limited methodological novelty are not fully addressed. In particular, the additional error analysis helps explain the limitations but does not fully resolve my concerns about the reliability of the computed sensitivities in practice. Therefore, I will maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer ge4q

We understand the reviewer's concerns about sensitivity reliability, ill-conditioning, and methodological novelty. Our latest revision strengthens the connection between the subproblem structure, gradient errors, and outer learning. We summarize the main additions below.

1. Reliability: connecting the approximation error to outer-learning convergence.

Building on standard inexact-gradient optimization analysis, we have expanded Appendix B.6 to study how the subproblem approximation affects learning. For the squared-residual imitation objective, we derive

\le b_{\rho,\tau_B}\|r_{\mathrm{out}}\|+\delta_F, \qquad b_{\rho,\tau_B}=O(\rho^{-1})+O(\tau_B),$$ under the stated regularity and uniform boundedness assumptions. Here, $r_{\mathrm{out}}$ is the imitation residual, $\tau_B$ is the backward residual tolerance, and $\delta_F$ accounts for additional forward-point errors. This reveals a useful structural property: the absolute gradient bias from subproblem differentiation is modulated by the learning residual. Consequently, at a finite penalty, this contribution can decrease as the imitation residual decreases. The analysis thus goes beyond a pointwise sensitivity-error bound and explains why approximate subproblem gradients can remain useful during learning. We further establish a finite-iteration approximate-stationarity guarantee for the projected normalized update used in our experiments, with a neighborhood controlled by the update step and total gradient error. For ordinary gradient descent, additional local gradient-dominance assumptions also yield local loss convergence for realizable imitation objectives. These results connect the method's error structure to outer optimization, while distinguishing the guarantees for the two update rules. **2. Conditioning: a shared challenge in constrained sensitivity computation.** Ill-conditioning can amplify not only the finite-penalty bias, but also forward-solve and backward-solve errors. This is a shared numerical challenge in constrained sensitivity computation. To provide additional context, Table 8 now also reports acados's final-batch learning-gradient discrepancies against independently re-solved high-accuracy NLP references: | Task | lapanda-ALM | lapanda-KKT | acados | |---|---:|---:|---:| | CartPole/OL | 0.015% | 0.007% | 0.003% | | CartPole/CL | 0.458% | 0.115% | 2.932% | | Quadrotor/OL | 0.065% | 0.402% | 1.168% | | Quadrotor/CL | 0.239% | 0.144% | 1.522% | | Robot arm/OL | 0.119% | 0.010% | 0.167% | | Robot arm/CL | 14.200% | 10.830% | 2.448% | Errors include both forward and backward effects, evaluated at each method's own final batch. † Some reference points do not pass all qualification checks; these full-batch entries are diagnostic rather than fully qualified accuracy estimates. The numerical difficulties in the Robot arm setting also affect other solvers; the discrepancies can additionally involve forward-point and solution-branch differences. Appendix B.6 explicitly identifies the amplification factors, while Appendix C.4.1 isolates these effects in controlled experiments. Preconditioning and related techniques are worthwhile directions for improving robustness to ill-conditioning. A general solution to this challenge is beyond the focus of this paper, and we identify it as a limitation. **3. Novelty: a matrix-free framework for general constraints, with a structure–error–learning analysis.** Our contribution is a matrix-free differentiable framework for general smooth equality and inequality constraints, together with a systematic study of its backward computation. Within the ALM–PANDA framework, we establish the sensitivity relationship between the original problem and the final subproblem, characterize structural and numerical errors, and connect those errors to outer learning. Original-KKT differentiation provides an accuracy-oriented option and a reference for the subproblem approximation; the latter exploits a smaller, locally positive-definite adjoint system for additional computational savings. The experiments evaluate this trade-off at both levels. Rosenbrock compares backward accuracy, time, and memory at shared forward points, including accuracy-calibrated KKT comparisons. The OCP experiments evaluate complete solver-call time to common loss targets and separately report backward costs. Across the six OCP protocols, the subproblem strategy reduces cumulative backward time by **15.5%** and sampled peak backward allocation by **7.5%** on average relative to our original-KKT implementation. These results demonstrate the practical value of exploiting the subproblem structure when its gradient accuracy is sufficient for the learning task.
Reviewer_64bAOpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper uses an augmented Lagrangian to incorporate arbitrary equality and inequality constraints with a matrix-free solver specialized for model predictive control problems.

Critical strengths

The proposed method can handle a set of constraints richer than the PANDA method, while benefiting from the matrix-free forward and backward passes.

Critical weaknesses

The paper proposes a marginal contribution compared to PANDA and the corresponding paper, which already contains most of the ingredients. LAPANDA adds the penalized and dualized constraints, passes the formulation to PANDA, and uses the last PANDA solve to compute sensitivities. The proposed experiments are outside of the theoretical guarantees provided by the paper (MPCC violates LICQ).

Questions for authors

What would be in essence the core advantages of the augmented Lagrangian method, what are important problems that cannot be solved with the original PANDA framework.

3 comments
AuthorsOpenReview ↗

Reponse to reviewer 64bA

We thank the reviewer for the comments on our contributions and theoretical scope. We address the weaknesses and question below.

W1. Novelty beyond combining ALM and PANDA.

ALM and PANDA are established methods. Our forward contribution is the theoretical connection between them: Theorem 1 establishes the compatibility of the ALM subproblems with PANDA’s assumptions and the local convergence of the resulting procedure.

Our main focus is the backward pass. We analyze the structural differences between the original problem and the final ALM subproblem, establish the relationship between their sensitivities, and characterize the resulting accuracy–efficiency trade-off (Theorem 2). In the revision, Appendix B.6 further analyzes the overall sensitivity error and its implications for outer optimization. Together, these results provide a matrix-free differentiable framework for general smooth constraints, with two backward strategies and an accompanying error analysis.

W2. Experiments outside the theoretical assumptions.

Our main numerical and learning experiments concern standard smooth-constrained problems. We introduced the MPCC experiment specifically to examine lapanda’s empirical performance outside the assumptions of our theory; degenerate MPCCs are not its main target problem class.

Degeneracy of the original KKT system does not necessarily prevent progress of the residual-based ALM/PANDA iterations. In this particular experiment, lapanda successfully completed the learning task. We have revised the main text to clarify that this is an empirical stress test, rather than a validation of the sensitivity guarantees at degenerate points.

Q. Advantages of ALM over the original PANDA framework.

The original PANDA framework considers composite objectives of the form f(x)+g(x)f(x)+g(x), where the nonsmooth term gg admits an efficiently computable proximal mapping. This can encode proximal-friendly constraints, particularly box constraints. General smooth constraints c(x)=0c(x)=0 and h(x)≤0h(x)\leq0, however, do not generally provide a readily computable projection onto their feasible set.

Such constraints arise in many control and optimization problems. For example, the Robot arm task includes nonlinear end-effector obstacle-avoidance constraints, while state constraints in single-shooting OCPs become functions of the control sequence through the dynamics. Handling these constraints without requiring their feasible-set projection is the motivation for introducing ALM.

More importantly, ALM provides an additional opportunity for backward computation. The original sensitivity system contains both primal variables and active-constraint multipliers and is generally indefinite. The final ALM subproblem gives a smaller primal system that is positive definite under the stated local assumptions and a sufficiently large penalty. We analyze the sensitivity relationship and finite-penalty bias in Theorem 2 and examine the accuracy–efficiency trade-off experimentally.

The resulting lapanda framework provides two matrix-free differentiation methods:

  • lapanda-KKT differentiates the original KKT conditions using MINRES and avoids the additional finite-penalty bias.
  • lapanda-ALM differentiates the final subproblem using CG, exploiting its smaller, locally positive-definite system at the cost of finite-penalty bias.

Appendix B.6 further separates this bias from forward and backward numerical errors. Step 3 establishes an approximate-stationarity result for ordinary gradient descent with bounded absolute gradient errors under its stated assumptions. The learning experiments complement this analysis by showing that useful learning progress can be obtained with inexact sensitivities.

Thus, ALM both extends the constraints that lapanda can handle and provides an additional backward strategy: differentiating a subproblem with favorable computational structure, with its approximation error explicitly analyzed.

Reviewer_64bAOpenReview ↗

Responses

I thank the authors for their responses, and the complementary experiments and changes to the paper. With the changes and clarifications, I think the paper is a valuable read to the ICLR audience and will raise my score to acceptance.

AuthorsOpenReview ↗

Reponse to reviewer 64bA

We sincerely thank you for recognizing our work. Your encouragement means a great deal to us and motivates us to continue pursuing this research.

Reviewer_eA6KOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes lapanda, a differentiable optimization method for nonconvex problems with general nonlinear equality and inequality constraints. In the forward pass, lapanda adopts the augmented Lagrangian method (ALM) framework of Alpaqa (Pas et al., 2022), reducing general constraints into a sequence of box-constrained subproblems, each solved by PANDA (Chen et al., 2026). In the backward pass, lapanda differentiates the final ALM subproblem rather than the original KKT system, solves the resulting adjoint system via matrix-free methods such as conjugate gradient (CG), and establishes an O(ρr−1)+O(ϵr+1)O(\rho_r^{-1}) + O(\epsilon_{r+1}) bound on the deviation from the exact KKT sensitivity (Theorem 2). The solver is implemented in C with Python/MATLAB bindings and evaluated on constrained Rosenbrock benchmarks, short-horizon optimal control problem (OCP) imitation learning, and embedded obstacle avoidance.

Critical strengths
  1. The full forward and backward pipeline is implemented in a self-contained C backend with Python and MATLAB interfaces, achieves a small memory footprint, and enables practical deployment on a resource-constrained embedded platform.
  2. Following the ALM penalty track provides natural regularization in both the forward and backward passes, allowing lapanda to handle ill-conditioned or degenerate constraints (e.g., the MPCC task in Section 5.3) where solvers relying on exact KKT linearization (e.g., acados) fail.
Critical weaknesses

1. Limited Novelty Beyond a Backward Pass for an Existing Solver, and Conflation of the ALM Reduction with Matrix-Free Differentiation

The forward pass is an existing method: the ALM framework is that of Alpaqa, with its inner solver replaced by PANDA. Apart from the forward convergence analysis (Theorem 1), the algorithmic contribution therefore amounts to a backward pass for this existing solver, namely differentiating the final ALM subproblem together with the error bound in Theorem 2.

Furthermore, the paper overstates the link between differentiating the ALM subproblem and being matrix-free: it motivates the ALM alignment by avoiding "explicitly forming and solving" the active-set KKT sensitivity system (Eq. 11), yet explicit formation is only one option; Eq. (11) can equally be solved matrix-free via MINRES. The actual role of differentiating the final ALM subproblem is that the −ρr−1I-\rho_r^{-1} I block (Eq. 13) allows the multiplier sensitivity to be eliminated, yielding the projected operator PU,r+1∇uu2ψρrPU,r+1P_{U,r+1}\nabla^2_{uu}\psi_{\rho_r}P_{U,r+1} (Eq. 76), which is SPD on the free subspace. This SPD reduction usefully enables more matrix-free methods such as CG, but at the cost of an O(ρr−1)O(\rho_r^{-1}) bias and conditioning that deteriorates as ρr\rho_r grows.

2. Flawed Baseline Configurations and Missing Comparisons in Sections 5.1 and 5.2

2.1. In Section 5.1, the baseline Explicit KKT combines the worst of both worlds: although the Rosenbrock chain (Eq. 15) is very sparse, it converts the KKT matrix into a dense matrix (79.65 MB) and runs unpreconditioned MINRES, which predictably converges slowly. Normally, this problem can be solved by a sparse direct solver with an assembled sparse KKT matrix or a preconditioned iterative method without assembling the KKT matrix. Two natural baselines should be compared instead:
(a) Matrix-Free KKT + MINRES: Solve Eq. (11) via preconditioned MINRES without matrix assembly.
(b) Explicit KKT + Sparse Direct Solve: Form the sparse KKT matrix and apply a sparse direct solve. Notably, this baseline is already implemented in the codebase (same_forward_kkt_reference in kkt_utils.py), yet its runtime and memory are unreported in Table 2.
2.2. In Section 5.1 (Table 2), a forward ablation against Alpaqa (PANOC) is missing. The paper attributes the forward speedup over IPOPT to the PANDA inner solver, yet Alpaqa solves the exact same outer ALM formulation using PANOC. Without comparing forward runtime against Alpaqa (PANOC), one cannot tell whether the speedup over IPOPT comes from PANDA itself or simply from Alpaqa's outer ALM framework.
2.3. In Section 5.2 (Table 3), although Table 1 lists acados for general constrained OCPs, it omits acados from the comparison on all three smooth OCP benchmarks.

3. Restriction to Short-Horizon OCPs and Tension with the Matrix-Free Motivation

For OCPs, lapanda is restricted to short horizons both architecturally (single shooting eliminates states by recursively unrolling NN-step dynamics, which compounds ill-conditioning as NN grows) and empirically (Appendix C.2). This also contradicts the matrix-free motivation: across Sections 5.2 and 5.3, single shooting over short horizons leaves a small decision dimension where forming the explicit Hessian and applying a direct solve are trivial, whereas increasing NN to reach high dimensions makes single shooting severely ill-conditioned.

4. Reliance on Additional Forward Optimization for High-Accuracy Sensitivities is Hidden in Appendix C.1

For Section 5.1 (Table 2), Appendix C.1 reveals that lapanda requires an extra "Aligned subproblem refinement" step (enlarging ρ\rho and re-solving the forward subproblem) to reduce gradient error from 7%–10% to <2%<2\%, accounting for ~90% of the reported backward time (Table 8). Because the forward ALM stops at a moderate ρr\rho_r with a large O(ρr−1)O(\rho_r^{-1}) bias, this extra forward solve (echoing FFOLayer criticized in Section 2) and its penalty-scaling hyperparameter are essential for accurate gradients, yet both are omitted from the main text and lumped into Table 2's backward time.

Questions for authors
  1. In Table 2, why does Explicit KKT convert a sparse KKT system into a dense matrix and solve it with unpreconditioned MINRES? In addition, what are the runtime and memory of (a) matrix-free preconditioned MINRES and (b) the sparse direct solve on the active-set KKT system (Eq. 11)?
  2. How does lapanda's forward time in Table 2 compare against Alpaqa (PANOC), to verify whether the speedup over IPOPT comes from PANDA or from Alpaqa's outer ALM framework?
  3. What are the forward/backward runtimes and memory usage of acados on the three OCP tasks in Section 5.2?
  4. Why does the backward pass run unpreconditioned CG rather than preconditioned CG (for instance, by reusing the L-BFGS curvature already collected by PANDA on the final ALM subproblem)?
  5. In the OCP sensitivity accuracy validation (Appendix C.2, Table 12), I notice from the diagnostic results (ocp_same_forward_kkt_summary.csv) that while the rollout-averaged gradient error is small, single-step gradient errors can be very large (e.g., on closed-loop Quadrotor). Could the authors explain the cause of these large single-step errors?
5 comments
AuthorsOpenReview ↗

Reponse to reviewer eA6k (1/3)

We thank the reviewer for the careful reading of our paper, including the inspection of our code and diagnostic results. These observations motivated revisions to the presentation, method positioning and experiments. In particular, Appendix B.6 now separates three sources of sensitivity error: forward-point error, numerical backward-solve error, and finite-penalty bias.

W1. Novelty and the relationship between subproblem differentiation and matrix-free computation.

As also clarified in our response to Reviewer 64bA, our contribution goes beyond combining ALM and PANDA. ALM allows PANDA to handle general smooth constraints, while Theorem 1 establishes the compatibility of the resulting subproblems with PANDA’s assumptions and the local convergence of the forward procedure.

Our main focus is the backward pass. We investigate the structural differences between the original problem and the final ALM subproblem, establish their sensitivity relationship, and examine the resulting accuracy–efficiency trade-off theoretically and experimentally.

We agree that the original presentation blurred the relationship between subproblem differentiation and matrix-free computation. The original KKT system can also be solved without assembling its matrix. The revision therefore presents two backward strategies:

  • lapanda-KKT: matrix-free MINRES on the original active-set KKT system, avoiding the additional finite-penalty approximation.
  • lapanda-ALM: CG on the smaller final-subproblem system, which is positive definite under the stated local assumptions and a sufficiently large penalty, but introduces finite-penalty bias.

Both strategies are matrix-free. Their differences concern the sensitivity target, system structure and approximation error.

W2.1. Explicit-KKT baseline configuration.

Please see the revised comparison under Q1.

W2.2. Forward attribution to PANDA.

Please see the PANDA/PANOC ablation under Q2.

W2.3. Missing acados comparison.

We have added acados to all three smooth OCP benchmarks, including both open-loop and closed-loop learning. Results are provided under Q3.

W3. Short-horizon OCPs and the matrix-free motivation.

The conditioning of single shooting depends on the stability and sensitivity propagation of the underlying dynamics; horizon length alone does not determine whether the problem becomes severely ill-conditioned. Moreover, the ALM framework is compatible with multiple shooting, where the dynamics are handled as equality constraints. The single-shooting transcription used in our experiments therefore does not restrict the underlying lapanda framework.

Our primary target is real-time computation on resource-constrained embedded platforms. Short-horizon problems still face tight memory and latency budgets, and reducing matrix construction and backward storage is valuable in this setting. The motivation for matrix-free computation is therefore broader than solving problems with very large decision dimensions.

W4. Additional forward optimization for accurate sensitivities.

The refinement procedure and its computational cost were already documented in the original paper. The main text explicitly referred to Appendix C.1, which “describes both the strategies used to control this error and the accuracy-matching protocol used in this benchmark to ensure a fair comparison.” The reported backward time included the refinement cost.

We acknowledge that space constraints limited the detail provided in the main text. In the revision, we make the distinction between the two backward strategies more explicit: lapanda-ALM exploits the subproblem structure while accepting finite-penalty bias, whereas lapanda-KKT avoids this additional approximation. Section 5.1 now compares subproblem CG and original-KKT MINRES at comparable low gradient accuracy, alongside direct methods, to show their respective accuracy, runtime and memory characteristics.

AuthorsOpenReview ↗

Reponse to reviewer eA6k (2/3)

Q1. Matrix-free KKT and sparse direct baselines.

We have rebuilt the Section 5.1 comparison to include final-subproblem CG, matrix-free original-KKT MINRES, sparse LU and dense LDLT. All methods receive identical saved forward points. MINRES calibration uses two separate starts, followed by evaluation on five held-out starts per problem size.

Table 3 reports all four sizes. Representative results at n=1000n=1000 are:

MethodGradient error (%)Complete backward time (μs)Peak backward allocation (KiB)
lapanda-ALM9.955320.0209.2
lapanda-KKT, calibrated9.979412.6284.6
lapanda-KKT, fixed tolerance0.332628.6284.6
Sparse LU0.00331723.34,977.2
Dense LDLT0.0033170,090.523,960.4

These results illustrate the respective advantages of the methods. At the tested low-accuracy operating points, the subproblem strategy reduces backward time and memory. Original-KKT differentiation avoids the additional finite-penalty bias. Direct factorization gives much smaller linear-solve errors, although forward-point error can remain.

Appendix B.6, Step 3 also establishes an O(1/K)+ε2O(1/K)+\varepsilon^2 bound on the best squared gradient norm for ordinary gradient descent with bounded absolute gradient errors, under its stated assumptions. This provides approximate-stationarity support for outer optimization using controlled inexact gradients.

Q2. PANDA versus PANOC in the forward pass.

The outer ALM constructions in lapanda and alpaqa are closely related. We therefore isolate the inner solver within the same lapanda ALM implementation, comparing PANDA and PANOC.

Appendix C.4.2 reports the following results; each cell gives mean forward time in milliseconds / total inner iterations:

OCPWarm: PANDAWarm: PANOCCold: PANDACold: PANOC
CartPole0.257 / 37.80.277 / 42.30.264 / 51.40.258 / 51.4
Quadrotor0.976 / 150.11.025 / 159.91.689 / 250.11.717 / 253.3
Robot arm0.952 / 163.91.272 / 230.91.971 / 282.33.641 / 546.7

Each setting contains 18 sampled training inputs. Means use inputs passing the common feasibility and projected-stationarity checks for both methods: 17/18 for warm-started Quadrotor and 18/18 otherwise.

The benefit is largest on Robot arm, where adaptive step-size enlargement substantially reduces the number of inner iterations. PANDA reduces warm-started forward time by approximately 7%, 5% and 25% across the three tasks, and cold-started Robot arm time by approximately 46%.

Q3. acados and the revised learning comparison.

We have added acados to all six learning protocols and revised the comparison metric based on the error analysis in Appendix B.6.

Requested forward tolerances are shared within each task: 10−410^{-4} for CartPole/Quadrotor and 10−510^{-5} for Robot arm. Both lapanda backward tolerances are 10−310^{-3}. However, a common numerical backward tolerance would not match gradient accuracy across factorization-based and iterative methods. The backward computations in CasADi/IPOPT, SafePDP and acados typically have much smaller linear-solve errors than loosely stopped iterative methods, while lapanda-ALM additionally incurs finite-penalty bias.

We therefore compare cumulative complete solver-call wall time to the same recorded training-loss target, using appropriate backward settings for each method. Each cell below reports time in seconds | complete-solver memory in MiB.

TaskTarget losslapanda-ALMlapanda-KKTCasADi/IPOPTSafePDPacadosTurboMPC-CPU
CartPole/OL0.52.45 | 3.72.60 | 3.6138.63 | 58.7255.35 | 56.799.01 | 60.3—
CartPole/CL0.0314.01 | 3.814.04 | 3.5NR | 59.1NR | 56.361.28 | 60.0—
Quadrotor/OL19018.66 | 5.019.28 | 4.8130.88 | 60.7210.56 | 57.547.05 | 61.0342.89 | 275.9
Quadrotor/CL2.312.77 | 3.613.36 | 3.6187.70 | 60.6288.73 | 56.2113.27 | 59.5396.38 | 276.5
Robot arm/OL19.99 | 4.710.19 | 4.8NR | 59.2NR | 56.856.62 | 59.8564.92 | 275.4
Robot arm/CL0.035.54 | 3.85.70 | 3.8NR | 59.2NR | 56.3305.69 | 60.1209.47 | 277.4

NR denotes a target not reached within the prescribed training run; — denotes unsupported.

Under these configurations, both lapanda strategies reach all six targets with low computation and memory costs. Their complete-solver memory is 3.5–5.0 MiB, compared with 56.2–277.4 MiB for these external baselines. This comparison evaluates task-level solver efficiency.

AuthorsOpenReview ↗

Reponse to reviewer eA6k (3/3)

Q4. Preconditioning CG with forward L-BFGS information.

We considered this approach and tested both Jacobi and forward-history L-BFGS preconditioners. Appendix C.4.3 uses 64 saved forward points per OCP and residual targets 10−3,10−6,10−810^{-3},10^{-6},10^{-8}. At 10−310^{-3}, median backward time in microseconds / iterations is:

OCPCGJacobi-PCGL-BFGS-PCG
CartPole39.05 / 457.95 / 1342.76 / 5
Quadrotor46.90 / 1265.79 / 2256.98 / 13.5
Robot arm50.91 / 1152.99 / 11.561.00 / 13

All tested solves satisfy their requested residual targets, but neither preconditioner reduces median backward time on these OCPs. Forward L-BFGS histories contain curvature information along selected optimization directions and need not provide a sufficiently accurate approximation of the backward operator. Preconditioner construction and application also add cost.

We therefore retain unpreconditioned CG in the main experiments. The results and discussion are included in Appendix C.4.3.

Q5. Large single-step gradient errors.

We thank the reviewer for the exceptionally detailed inspection of the pointwise diagnostic files in our code repository, including results beyond those presented in the manuscript. These observations prompted the additional sensitivity-error analysis and reference checks in the revision.

We now distinguish discrepancies caused by the returned forward point and active set from those introduced by the backward computation. Under the updated configuration, Appendix C.2.2 reports the following final-batch relative discrepancies. Each entry is lapanda-ALM / lapanda-KKT, in percent:

TaskIndependently resolved original-NLP referenceActive-KKT reference at the returned forward point
CartPole/OL0.015 / 0.0070.012 / 0.002
CartPole/CL0.458 / 0.1150.008 / 0.007
Quadrotor/OL0.065 / 0.4020.478 / 0.010
Quadrotor/CL0.239 / 0.1440.062 / 0.046
Robot arm/OL0.119 / 0.0100.070 / 0.001
Robot arm/CL14.200 / 10.8300.045 / 0.026

Gradients are averaged before computing the relative discrepancy: ∥g‾−g‾ref∥/∥g‾ref∥\|\overline g-\overline g_{\mathrm{ref}}\|/\|\overline g_{\mathrm{ref}}\|.

For Robot arm closed-loop, the discrepancy against the independently resolved NLP is much larger than against the same-point reference. This indicates that forward-point and active-set differences are important contributors, rather than finite-penalty bias or backward stopping error alone.

Ill-conditioning can amplify these perturbations, as demonstrated separately in Appendix C.4.1. A very small reference-gradient norm can additionally produce a large single-step relative error even when the absolute discrepancy is modest.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Area Chair and all reviewers

We thank all reviewers for their careful reading and constructive comments. Their feedback has helped us sharpen the contribution, strengthen the analysis, and improve the experiments. During rebuttal, we worked to address the concerns with additional analysis and evidence, rather than clarification alone. The main revisions are:

  • Overall sensitivity-error analysis: Appendix B.6 separates finite-penalty bias, forward-point error, and backward-solve error, and connects the resulting gradient errors to outer-learning behavior.
  • Original-problem versus subproblem differentiation: Section 4.2 and Appendix B.6 explain their sensitivity relationship, the smaller and locally positive-definite subproblem system, and the accuracy–efficiency trade-off.
  • Fairer experimental comparisons: Section 5.1 now includes matrix-free KKT MINRES, sparse and dense direct methods, and accuracy-calibrated comparisons at shared forward points. Section 5.2 adds acados and compares complete solver-call time to common learning-loss targets, with backward costs reported separately. Additional inner-solver and preconditioning ablations appear in Appendices C.4.2–C.4.3.
  • Conditioning and numerical accuracy: Appendix B.6 identifies the relevant error-amplification factors, Appendix C.4.1 isolates penalty, tolerance, and conditioning effects, and Table 8 reports learning-gradient discrepancies with reference-qualification information.

The remaining discussion concerns the methodological contribution, sensitivity reliability, and whether subproblem differentiation offers sufficient benefit. We would like to clarify our position on these points.

1. Contribution.

The contribution of lapanda is a matrix-free differentiable framework for nonconvex problems with general smooth constraints, together with an analysis of how to differentiate within that framework. ALM extends PANDA beyond constraints with readily computable proximal mappings; our analysis establishes the compatibility and local convergence of the resulting forward procedure.

The backward analysis then connects two different sensitivity targets: the original constrained problem and the final ALM subproblem. It explains what is gained computationally, what information is approximated, how the resulting error can be controlled, and how it affects learning. Original-KKT differentiation provides an accuracy-oriented option and a reference for this analysis; the subproblem structure and its learning implications remain central to the work.

2. Reliability of the gradients used for learning.

The expanded Appendix B.6 goes beyond explaining why errors occur. For squared-residual imitation objectives, under the stated regularity and boundedness assumptions, it establishes

\qquad b_{\rho,\tau_B}=O(\rho^{-1})+O(\tau_B).$$ **The absolute subproblem bias is therefore modulated by the learning residual** and can decrease as imitation improves, even at finite penalty. Building on inexact-gradient analysis, we also derive **a finite-iteration approximate-stationarity guarantee for the projected normalized update used in the experiments**. Stronger local loss-convergence results for ordinary gradient descent are stated under additional assumptions. This supplies a connection between the approximation and outer optimization, complementing the empirical learning results. ### 3. Value of the two backward strategies. We view the two strategies as an opportunity to exploit different operating regimes. Original-KKT differentiation is suitable for higher accuracy requirements or when increasing the subproblem penalty becomes too costly. Subproblem differentiation is useful when its accuracy meets the learning task's needs and the smaller system reduces computation and storage. **This benefit persists after accuracy calibration**: at Rosenbrock size $n=1000$, CG and calibrated MINRES achieve approximately 9.96% and 9.98% gradient error, while CG reduces backward time from 413 to 320 μs and peak allocation from 284.6 to 209.2 KiB. Across six OCP protocols, subproblem differentiation reduces cumulative backward time by 15.5% and sampled peak backward allocation by 7.5% on average relative to our KKT implementation. We appreciate the time the Area Chair and reviewers have devoted to this work, and hope this summary helps make the revised contribution and supporting evidence easier to assess.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.