acceptodds
Under review as a conference paper at ICLR 2027

EG-Opt: Efficient Learning for Nonlinear Programming via Explicit Gradients from Sample-Calibrated Linear Surrogates

Abstract

Large-scale nonlinear programs remain challenging, especially in repeated applications. Learning-based methods enable fast inference. However, feasibility-preserving training often requires repeated nonlinear solutions or iterative projections, while implicit differentiation or unrolling adds substantial computational overhead. We propose EG-Opt, an efficient framework based on explicit gradients from sample-calibrated linear surrogates. During training, EG-Opt replaces repeated nonlinear completion with a reusable linear surrogate enabling explicit gradient propagation and efficient batching, caching, and factorization reuse. In addition, we derive local error bounds for state and gradient, and provide sufficient conditions for surrogate gradients to be descent directions for the exact completion loss. At inference, the linear surrogate warm-starts the original nonlinear completion solver, reducing iterations and helping recover the reference solution branch associated with the labels when nonlinear completion admits multiple roots, while ensuring equality feasibility. Across ten benchmarks spanning alternate-current optimal power flow, water distribution, gas flow optimization, and a controlled nonlinear problem, EG-Opt achieves full equality and inequality feasibility with high solution quality. It speeds up end-to-end training by up to 128× versus implicit differentiation and 2,160× versus two-point zeroth-order training while accelerating inference by approximately 1.5× over nonlinear-completion baselines.

Primary area:
optimization
Keywords:
learning-based optimization, differentiable optimization, surrogate modeling, feasibility-preserving learning, explicit gradients

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3QMoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EG-Opt uses a sample-calibrated linear surrogate and explicit gradients to replace repeated nonlinear equality completion during training. At inference, a label-independent surrogate provides a warm start for the original nonlinear solver. The paper derives local bounds on state and gradient errors and conditions under which surrogate gradients guide descent. It evaluates ten benchmarks in AC-OPF, water and gas systems, and a controlled nonlinear problem, reporting feasibility, solution quality, training and inference time, gradient fidelity, and a calibration ablation.

Critical strengths

The method addresses the cost of repeated nonlinear completion and implicit differentiation during training. The experiments cover several application areas and measure both training and inference time. The paper also examines gradient accuracy, descent behavior, calibration, and cases where poor conditioning degrades the surrogate.

Critical weaknesses
  1. Limited methodological novelty. EG-Opt combines a frozen linear surrogate, calibration at labeled samples, and explicit gradients. The experiments show savings in training time, but the paper does not establish a methodological advance beyond combining these elements or explain how the construction differs in principle from existing surrogate and sensitivity methods.

  2. The operating range of one frozen surrogate is unclear. Calibration reproduces states at labeled points, but the paper does not establish how far one fixed matrix remains accurate as operating conditions change. Appendix B discusses multiple surrogates without testing when they are needed or what they cost.

  3. The effect of calibration on gradient fidelity is not isolated. The Cal/NoCal ablation reports objective quality and feasibility, while the gradient study compares EG-Opt with Exact-ID and ZO. It does not compare gradient error or descent behavior between Cal and NoCal, leaving calibration’s contribution to the training signal unclear.

  4. The theoretical results depend on local conditions. The error bounds and descent result require regularity, suitable conditioning, and proximity to the exact completion. The paper tests failure modes under poor conditioning, but does not show how often these conditions hold during the reported training runs.

  5. Inference still requires nonlinear completion. The surrogate provides a warm start, and the paper reports an inference speedup of about 1.5 times. The original solver remains in the deployment path. The practical benefit depends on how consistently the warm start reduces iterations while preserving solution quality and branch recovery.

Questions for authors
  1. What methodological advance does EG-Opt provide beyond combining a frozen linear surrogate, sample calibration, and explicit gradients? A comparison with the closest numerical approximation and sensitivity methods would make the claimed contribution more precise.

  2. Over what range of operating conditions does one frozen surrogate remain accurate? Please report how state and gradient errors change across that range and indicate when multiple surrogates are needed.

  3. Does calibration improve gradient accuracy or descent behavior compared with NoCal? A comparison at matched checkpoints, particularly early in training, would isolate this effect.

  4. When do the local error bounds and descent conditions apply during training? Please relate the assumptions to conditioning and approximation errors measured along the training runs.

  5. How does EG-Opt’s surrogate warm start differ from warm-start strategies in prior work? A comparison using the same nonlinear solver and predicted controls, with iterations, runtime, and branch recovery reported, would isolate its contribution at inference.

13 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1

Response to Weakness 1 and Question 1

We thank the reviewer for carefully examining the method and considering EG-Opt's methodological contribution and its distinction from related approaches. Its main methodological advances are as follows.

(1) Using constraint structure as a computational prior for learning. EG-Opt's results support the broader insight that constraint structure can serve as a computational prior to guide learning. Establishing reusable computational structures in advance enables states and gradients to be computed consistently from the same inexpensive surrogate during training. Across the tested problems, these approximate computations provide effective learning signals without requiring nonlinear completion and exact implicit differentiation at every update. Allocating computation according to the needs of training and inference makes efficiency a consequence of the learning design itself. The saved resources also allow more training or broader model exploration on the same hardware within a limited computational budget.

(2) Providing task learning signals without direct control supervision. After removing direct control supervision, EG-Opt still achieves a 100.0% joint feasibility rate and good objective accuracy in all four tested cases. This shows that surrogate gradients support effective learning and suggests potential extensions to unsupervised optimization learning. To provide empirical evidence, we retrain EG-Opt with the direct control supervision weight λs\lambda_s set to 00, 0.10.1, and 11. The results are reported in Table 6.

Table 6. Effect of direct control supervision weight on EG-Opt solution quality and joint feasibility

CaseMean objective gap (%) at λs=0\lambda_s=0Mean objective gap (%) at λs=0.1\lambda_s=0.1Mean objective gap (%) at λs=1\lambda_s=1Joint feasibility rate (%) at λs=0\lambda_s=0Joint feasibility rate (%) at λs=0.1\lambda_s=0.1Joint feasibility rate (%) at λs=1\lambda_s=1
CSCP0.06930.07030.0706100.0100.0100.0
Case574.60650.44450.0712100.0100.0100.0
C-Town0.12310.10860.0655100.0100.0100.0
GasLib1351.20900.19930.1413100.0100.0100.0

Table 6 shows that all cases retain good objective accuracy and 100% feasibility when direct supervision is reduced or removed. This suggests the potential of inexpensive gradient computation for unsupervised optimization learning.

Among related methods, modified Newton [1] reuses matrices and their factorizations but still requires iterative root finding. Zeroth order additive correction [2] matches the function value at a reference point without changing the derivative. First order model management (AMMF) [3] additionally matches gradients and updates the surrogate during optimization. JFB and phantom gradients retain the forward solve and primarily simplify backward computation. When nonlinear equalities require numerical solution, DC3 still performs completion using the original equations and implicit backpropagation, with differentiable correction for inequalities. EG-Opt instead precomputes sample calibration using existing labels and reuses a fixed operator across samples and training updates. The same explicit surrogate supplies both states and gradients, reducing repeated computation in both forward solves and backward differentiation.

References

[1] Alan C. Hindmarsh, Radu Serban, Cody J. Balos, David J. Gardner, Daniel R. Reynolds, and Carol S. Woodward. User documentation for KINSOL. Lawrence Livermore National Laboratory, 2026. URL https://sundials.readthedocs.io/en/latest/kinsol/Mathematics_link.html. Version 7.9.0, Sections 7.2.1–7.2.3. Accessed November 15, 2026.

[2] Sandia National Laboratories. Dakota 6.18 Documentation: Correction Approaches for Surrogate Models, 2023. URL https://snl-dakota.github.io/docs/6.18.0/users/usingdakota/reference/model-surrogate-global-correction.html. Accessed November 15, 2026.

[3] N. M. Alexandrov, E. J. Nielsen, R. M. Lewis, and W. K. Anderson. First-order model management with variable-fidelity physics applied to multi-element airfoil optimization. In 8th Symposium on Multidisciplinary Analysis and Optimization. American Institute of Aeronautics and Astronautics, 2000. doi: 10.2514/6.2000-4886. URL https://ntrs.nasa.gov/citations/20000097390. AIAA Paper 2000-4886.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2 (Part 1 of 3)

Response to Weakness 2 and Question 2 (Part 1 of 3)

We thank the reviewer for carefully examining the surrogate construction and asking specific questions about the applicability of one frozen surrogate and extensions using multiple surrogates. We discuss, in turn, the effects of linearization point selection and demand range, approximation accuracy under changing operating conditions, and the benefits and computational costs of multiple surrogates.

(1) Effects of linearization point selection and demand perturbation range

Part 1. Effects of reasonable anchor choices. Anchor selection is straightforward. In the current manuscript, an anchor is taken directly from an actual sample and its reference solution labels, namely (xˉ,uˉ,zˉ)=(xj,uj⋆,zj⋆)(\bar{x},\bar{u},\bar{z})=(x_j,u_j^\star,z_j^\star). The selected anchor is then validated following Appendix B, including checks of the surrogate matrix for nonsingularity and conditioning. This procedure requires no additional continuous optimization over a high dimensional anchor space. For example, the training anchor for GasLib135 is the first sample in the dataset used for surrogate construction. Section 4.4 of the original manuscript already examines the effects of anchor conditioning on gradient approximation and warm starts, while Appendix B discusses candidate surrogate construction, selection principles, and extensions using multiple anchors.

We select actual samples at different demand levels as linearization points, construct a linear surrogate at each point, and retrain the network to compare final solution quality. We also hold the predictions of a single network fixed to compare the surrogates' state and gradient approximation errors and the inference cost of warm starts. The results are reported in Table 7.

Table 7. Solution quality, gradient and state errors, and inference cost across anchors

CaseCandidate anchorMean objective gap (%)Joint feasibility rate (%)Mean relative gradient error (%)Normalized state RMS errorMean Newton iterationsMean time (ms per sample)
C-TownCentral demand0.0655100.010.910.00402.13402.7793
C-TownLow total demand0.0654100.015.900.00772.61003.3650
C-TownHigh total demand0.0640100.018.050.00492.43433.1553
GasLib135Central demand0.1418100.01.860.01071.99930.2981
GasLib135Low total demand0.1329100.04.180.02191.99900.2981
GasLib135High total demand0.1412100.01.570.01031.99000.2978

Table 7 shows similar final objective accuracy across linearization points for both the water and gas networks, with a joint feasibility rate of 100.0% in all cases. Linearization point selection affects state and gradient approximation accuracy and inference time, but these variations do not produce appreciable differences in final solution quality among the tested candidates. This indicates flexibility in selecting and validating linearization points from actual samples.

Part 2. Retraining and testing across demand perturbation ranges. To assess whether the same fixed surrogate accommodates changes in demand range, we retrain and test EG-Opt and Exact-ID separately over different demand perturbation ranges, comparing final solution quality and constraint feasibility. The results are reported in Table 8.

Table 8. Demand perturbation ranges, solution quality, and joint feasibility rates

CaseRangeGlobal demand perturbation amplitude (%)Regional demand perturbation amplitude (%)Node local demand perturbation amplitude (%)EG-Opt mean objective gap (%)Exact-ID mean objective gap (%)EG-Opt joint feasibility rate (%)Exact-ID joint feasibility rate (%)
GasLib135Narrow±1\pm1-±4\pm40.08390.0860100.0100.0
GasLib135Original±2\pm2-±8\pm80.14130.1473100.0100.0
GasLib135Wide±3\pm3-±12\pm120.19110.1948100.0100.0
C-TownNarrow±5\pm5±2.5\pm2.5±1\pm10.00050.0005100.0100.0
C-TownOriginal±10\pm10±5\pm5±2\pm20.06550.0632100.0100.0
C-TownWide±15\pm15±7.5\pm7.5±3\pm30.46320.4313100.0100.0

Table 8 shows that objective gaps increase for both methods as the demand perturbation range widens. Nevertheless, EG-Opt maintains objective accuracy close to Exact-ID and a 100.0% joint feasibility rate in every range. These results show that the same sensitivity matrix can be reused when retraining separately over the tested ranges, with sample calibration supporting effective learning.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2 (Part 2 of 3)

Response to Weakness 2 and Question 2 (Part 2 of 3)

(2) State and gradient approximation errors under changing demand

We compare the surrogate's state and gradient approximation errors under different operating conditions, as reported in Table 9. RMS is the mean of the root mean squared errors between approximate and exact states, computed for each sample. r‾\overline r is the mean relative error between approximate and exact gradient vectors. c‾\overline c is the mean cosine of the angle between these vectors, with values closer to 1 indicating stronger directional agreement. The positive inner product rate is the fraction of valid samples for which the two gradient vectors have a positive inner product.

Table 9. State and gradient approximation errors under different operating conditions

GasLib135

FactorRMSr‾\overline rc‾\overline cPositive inner product rate
0.900.20900.08101.0000100.00%
0.950.05910.03561.0000100.00%
1.000.01030.01481.0000100.00%
1.050.03050.07571.0000100.00%
1.100.02890.16851.0000100.00%

C-Town

FactorRMSr‾\overline rc‾\overline cPositive inner product rate
0.800.09870.15460.9898100.00%
0.900.01250.18130.9829100.00%
1.000.00400.09840.9940100.00%
1.100.04800.10590.9974100.00%
1.200.33120.15810.9945100.00%

CSCP

FactorRMSr‾\overline rc‾\overline cPositive inner product rate
0.800.00020.04000.9990100.00%
0.900.00020.04820.9985100.00%
1.000.00040.05480.9981100.00%
1.100.00070.06280.9974100.00%
1.200.00110.07300.9963100.00%

Table 9 shows that operating conditions affect state errors and relative gradient errors. Nevertheless, the mean cosine between approximate and exact gradients remains close to 1 in all three cases across the tested conditions, and the gradient inner product is positive for every valid sample. Thus, variations in state approximation error do not disrupt gradient directional agreement on these samples, and the fixed surrogate continues to provide useful directions for task optimization. The applicability of a single surrogate should therefore be assessed by both state accuracy and preservation of the exact gradient direction. These results provide empirical support for reusing the same surrogate across the tested operating range.

Reviewer_DiXdOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EG-Opt for learning solutions to parametric nonlinear programs. A neural network predicts control variables, while a fixed linear surrogate with sample-specific calibration estimates states during training. This avoids repeated nonlinear completion and implicit differentiation. At inference, the uncalibrated surrogate initializes nonlinear equality completion.

Critical strengths
  1. The method addresses a practical computational bottleneck through a simple design that supports reusable factorizations and efficient batched computation.

  2. Experiments cover power, water, and gas networks at several scales, reporting substantial training speedups and competitive solution quality. Calibration ablations and gradient diagnostics help explain the method’s behavior and limitations.

Critical weaknesses
  1. Although local error bounds and offline selection criteria are provided (Section 3.2 and Appendices A–B), the paper does not establish that errors induced by the fixed matrix AA remain controlled throughout training. The convergence analysis largely follows standard biased-SGD results, permits a persistent bias floor. Moreover, stationarity of the penalized exact-completion loss does not guarantee feasibility of the original state-dependent inequalities. The checkpoint diagnostics are useful but do not establish these properties along the training trajectory.

  2. Related-work positioning could be clearer. The paper does not discuss SurCo (Ferber et al., 2023), which learns linear surrogate objectives for nonlinear combinatorial optimization. Its mechanism differs from EG-Opt’s fixed surrogate for equality completion, but discussing this distinction would better position the contribution within surrogate-based optimization learning.

Questions for authors

How sensitive are deployment solution quality and inequality feasibility to reasonable anchor choices and changes in the operating range?

9 comments
AuthorsOpenReview ↗

Response to Weakness 1 (Part 1 of 2)

Response to Weakness 1 (Part 1 of 2)

We thank the reviewer for carefully considering errors throughout training and feasibility under the original constraints. We address three aspects. First, we present uniform error bounds for the fixed surrogate, state their conditions, and examine actual training trajectories through diagnostics at each update. Second, we clarify the biased SGD analysis and the residual bias term. Third, we explain the feasibility conditions for the original inequalities.

(1) Uniform error bounds for the fixed surrogate and training trajectory diagnostics

To address whether errors induced by the fixed matrix AA remain controlled throughout training, we provide uniform error bounds covering all training steps and state their applicability conditions. EG-Opt computes surrogate states directly from the current predictions and reference labels, without recursion across training steps. The state error expression therefore has no term that accumulates over successive steps.

For a fixed sample, let utu_t denote the predicted control at update tt, (u⋆,z⋆)(u^\star,z^\star) the reference labels, ze(u)z_e(u) the exact completion state of the original equalities, and Je(u)=∂ze(u)/∂uJ_e(u)=\partial z_e(u)/\partial u its sensitivity. The affine surrogate is z~(u)=z⋆+K(u−u⋆)\widetilde z(u)=z^\star+K(u-u^\star), where K=A−1BK=A^{-1}B, AA is a nonsingular fixed surrogate matrix, and BB is the coefficient matrix of the controls in the right hand side. Assume that every segment [u⋆,ut][u^\star,u_t] lies in a region containing the same regular solution branch, that JeJ_e has Lipschitz constant LL there, and that ∥ut−u⋆∥2≤R\Vert u_t-u^\star\Vert _2\le R at all training steps. Let e0=∥z⋆−ze(u⋆)∥2e_0=\Vert z^\star-z_e(u^\star)\Vert _2 be the label error and δ=∥K−Je(u⋆)∥2\delta=\Vert K-J_e(u^\star)\Vert _2 the sensitivity mismatch at the reference point. Integrating along each segment and applying the Lipschitz condition gives

sup⁡t∥z~(ut)−ze(ut)∥2≤E:=e0+δR+12LR2,sup⁡t∥K−Je(ut)∥2≤D:=δ+LR.\sup_t\Vert \widetilde z(u_t)-z_e(u_t)\Vert _2\le E:=e_0+\delta R+\tfrac12 LR^2,\qquad \sup_t\Vert K-J_e(u_t)\Vert _2\le D:=\delta+LR.

For the task loss ℓ(u,z)\ell(u,z), including constraint penalties, assume that the segment between the surrogate and exact states lies in a smooth region of the loss. Let the partial derivatives ℓu,ℓz\ell_u,\ell_z have Lipschitz constants Lu,LzL_u,L_z with respect to the state, and assume ∥ℓz∥2≤Mz\Vert \ell_z\Vert _2\le M_z, with all constants valid uniformly throughout training. By Equation (A.6) of the manuscript, the control gradients gc,geg_c,g_e computed through the surrogate and exact completion mappings satisfy

sup⁡t∥gc(ut)−ge(ut)∥2≤(Lu+∥K∥2Lz)E+MzD.\sup_t\Vert g_c(u_t)-g_e(u_t)\Vert _2\le (L_u+\Vert K\Vert _2L_z)E+M_zD.

If these conditions hold for every sample and the network Jacobian remains uniformly bounded throughout training, the chain rule also gives a uniform bound on the parameter gradient bias. These bounds do not grow with the number of training steps, although their magnitudes depend on sensitivity mismatch, prediction displacement, and numerical conditioning.

AuthorsOpenReview ↗

Response to Weakness 1 (Part 2 of 2)

Response to Weakness 1 (Part 2 of 2)

To address this concern empirically, we compare approximate and exact control gradients for the same inputs at initialization and after every training update, and check constraint feasibility after completion using the original nonlinear equalities. This evaluates the fixed surrogate throughout training. The results are reported in Table 3.

Table 3. Mean absolute control gradient error and joint feasibility rate across training stages

Training stageCSCP mean absolute gradient errorCSCP joint feasibility rate (%)GasLib135 mean absolute gradient errorGasLib135 joint feasibility rate (%)
Initialization1.45×10−41.45\times10^{-4}100.01.57×1061.57\times10^{6}0.0
0%–20%3.48×10−63.48\times10^{-6}100.05.31×1035.31\times10^{3}87.3
20%–40%1.92×10−61.92\times10^{-6}100.03.45×1003.45\times10^{0}99.9
40%–60%1.96×10−61.96\times10^{-6}100.06.78×10−16.78\times10^{-1}99.8
60%–80%1.97×10−61.97\times10^{-6}100.05.31×10−55.31\times10^{-5}100.0
80%–100%1.98×10−61.98\times10^{-6}100.05.40×10−55.40\times10^{-5}100.0
Final1.98×10−61.98\times10^{-6}100.05.38×10−55.38\times10^{-5}100.0
Training stageCase57 mean absolute gradient errorCase57 joint feasibility rate (%)C-Town mean absolute gradient errorC-Town joint feasibility rate (%)
Initialization5.38×1015.38\times10^{1}0.06.51×10−46.51\times10^{-4}100.0
0%–20%3.88×1013.88\times10^{1}92.19.38×10−49.38\times10^{-4}100.0
20%–40%1.86×1001.86\times10^{0}100.01.39×10−41.39\times10^{-4}100.0
40%–60%2.51×1002.51\times10^{0}100.09.46×10−59.46\times10^{-5}100.0
60%–80%2.56×1002.56\times10^{0}100.07.32×10−57.32\times10^{-5}100.0
80%–100%3.25×1003.25\times10^{0}100.07.28×10−57.28\times10^{-5}100.0
Final4.32×1004.32\times10^{0}100.06.35×10−56.35\times10^{-5}100.0

Table 3 shows an overall reduction in the mean absolute control gradient error under the fixed matrix AA along the measured trajectories. In all four cases, the final error is at least one order of magnitude below its initial value. Thus, gradient approximation can improve during training even when the same matrix is reused throughout.These diagnostics across successive epochs provide the full training trajectory evidence requested by the reviewer. Under the stated conditions, the uniform error bounds also ensure that errors introduced by the fixed matrix AA share a common upper bound at every training step.

(2) Biased SGD analysis

The biased SGD analysis concerns two distinct objectives. EG-Opt applies the exact chain rule to its explicitly constructed surrogate loss. The bias is defined relative to the loss with exact nonlinear completion. Equation (A.28) of the original manuscript quantifies its effect on the upper bound on first order optimality error. The residual term does not imply a positive lower bound that the actual error must attain, nor does it imply that a fixed surrogate necessarily causes substantial accuracy loss. Our contribution is to reduce state bias through sample calibration and propagate state and sensitivity mismatch into a gradient error bound. Standard SGD analysis connects these quantities to optimization accuracy.

(3) Feasibility of the original inequalities

EG-Opt can ensure feasibility of the original inequalities by reserving a feasibility margin. If the surrogate state error is bounded by EE and an inequality has Lipschitz constant LgL_g with respect to the state, replacing the surrogate state with the exact completion state changes the inequality value by at most LgEL_gE. A surrogate solution with a feasibility margin of at least LgEL_gE therefore remains feasible after exact completion. If inequalities are still violated after completion, DC3's inequality correction step can be added at inference to adjust the controls according to the actual violations and repeat equality completion after each correction. When local convergence conditions hold and equality completion is solvable, this process can restore inequality feasibility while maintaining equality feasibility. It requires no changes to EG-Opt's training surrogate or gradient computation and is therefore directly compatible with inexpensive training.

AuthorsOpenReview ↗

Response to Weakness 2

Response to Weakness 2

We thank the reviewer for the thoughtful suggestions on positioning our work. We clarify how SurCo and EG-Opt differ in the role of their surrogates and what they learn, and describe the addition to our related work discussion.

SurCo learns the coefficients of a linear surrogate objective, which a solver optimizes over the given feasible set to obtain candidate solutions to the original nonlinear problem. It evaluates their quality using the original nonlinear objective and updates the coefficients by differentiating through the surrogate solver. Its surrogate therefore guides the solver toward high quality decisions. The central distinction is the role of the surrogate. SurCo learns a surrogate objective to assist optimization, whereas EG-Opt reuses a fixed calibrated surrogate to replace nonlinear equality completion and implicit differentiation during training, accelerating the training of the control prediction network. We have added a citation to SurCo and a discussion of these differences in the manuscript's Related Work section.

Reviewer_4kB7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies learning-based methods for parametric nonlinear programs (NLPs) with nonlinear equality and inequality constraints. To avoid repeated nonlinear solves and implicit differentiation during training, the authors propose EG-Opt, which uses offline reference solutions (ui⋆,zi⋆)(u_i^\star, z_i^\star) to construct a sample-calibrated linear surrogate for explicit gradient propagation in training, and a label-independent linear surrogate to warm-start a Newton solver at inference. The paper provides local error bounds and evaluates the method across power, water, gas, and synthetic benchmarks.

Critical strengths
  1. The paper proposes a simple and pragmatic local linearization mechanism that eliminates zeroth-order state bias at labeled points via sample calibration and reuses a frozen anchor sensitivity matrix to turn forward and backward passes into lightweight matrix multiplications. This effectively propagates gradients for state inequality penalties and resolves the boundary-violation issue of pure control regression (Partial-NN).
  2. The experimental evaluation is comprehensive and solid across 10 benchmarks spanning power, water, gas, and synthetic problems. Moreover, Section 4.4 thoroughly explores the failure modes of the method, such as when the frozen matrix AA is nearly singular, or under high curvature and multiple solution branches.
Critical weaknesses
  1. Combining the first-order Taylor expansion in Eq. (3) with the sample calibration in Eq. (4) directly reduces the training surrogate to an xix_i-independent form, z~i(u)=zi⋆+K(u−ui⋆)\tilde z_i(u) = z_i^\star + K(u - u_i^\star) with K=−(∂h/∂z)−1(∂h/∂u)K = -(\partial h / \partial z)^{-1}(\partial h / \partial u) evaluated at the anchor. In essence, this is just a linear approximation around (ui⋆,zi⋆)(u_i^\star, z_i^\star) using a fixed implicit-function sensitivity matrix KK. However, the main text (Section 3.1 and Algorithm 1) does not explicitly point out this simplified form, and instead frames training as caching residuals rir_i and repeatedly solving linear systems Az~i=bi(u)+riA\tilde z_i = b_i(u) + r_i at each step. This presentation obscures the actual simplicity of the training mechanism and somewhat over-packages the method.
  2. Because both the frozen sensitivity matrix and the inference warm-start rely on a single anchor (xˉ,uˉ,zˉ)(\bar{x}, \bar{u}, \bar{z}), the method is sensitive to anchor conditioning (as shown in Section 4.4), yet the anchor itself is high-dimensional in large-scale networks, making its selection non-trivial. Although Appendix B devotes nearly five pages to anchor selection and multi-anchor extensions, the authors do not specify how the anchor is determined for the water and gas networks, and the paper reports no empirical results or ablations on these designs.
  3. Unlike DC3, EG-Opt has no inequality correction step at inference and evaluates inequalities only on the approximate surrogate state z~i\tilde z_i during training. As a result, balancing the inequality penalty against the control supervision term during training can be tricky, as reflected by the case-specific loss designs in Appendix D.2 (e.g., custom Softplus temperatures, safety margins, and tail penalties).
Questions for authors
  1. What is the exact definition of the task loss evaluated in Figure 2? The notation between the main text and Appendix A.3 is inconsistent regarding whether the task loss includes the control supervision term λs∥u−ui⋆∥2\lambda_s \Vert u - u_i^\star \Vert^2 and the auxiliary equality penalty λh∥h∥2\lambda_h \Vert h \Vert^2. Since including the supervision term can mask gradient errors on the original objective (as shown in Table D6), could the authors clarify the exact loss terms and weights used in Figure 2?
  2. All experiments include a direct control supervision term λs∥u−ui⋆∥2\lambda_s \Vert u - u_i^\star \Vert^2 with λs=1\lambda_s = 1, which keeps predictions near ui⋆u_i^\star where the first-order surrogate z~i=zi⋆+K(u−ui⋆)\tilde z_i = z_i^\star + K(u - u_i^\star) remains accurate (and on CSCP, EG-Opt achieves the exact same 0.0704% objective gap as Partial-NN, which trains only on ∥u−ui⋆∥2\Vert u - u_i^\star \Vert^2). Have the authors tested reducing or removing this supervision term (e.g., λs=0\lambda_s = 0, training solely with gradients backpropagated through the calibrated surrogate), and does EG-Opt still converge reliably in that setting?
5 comments
AuthorsOpenReview ↗

Response to Weakness 1

Response to Weakness 1

We thank the reviewer for carefully assessing our work and offering specific suggestions on the presentation of our method. We first explain the practical benefits of this concise design and then discuss its methodological implications.

Although EG-Opt has a simple construction, its computational benefits are substantial. Across the tested problems, it markedly reduces training cost while maintaining competitive solution quality, and its surrogate warm start further reduces inference cost. These gains arise from addressing a key computational bottleneck by replacing repeated nonlinear completion and exact implicit differentiation during training with reusable linear computations. Our emphasis is therefore on the efficient learning and inference enabled by this concise design.

The local linear construction is straightforward, but integrating it into a complete framework for solving nonlinear programming (NLP) problems is the central research challenge. EG-Opt uses the same surrogate for training and warm start inference, coupled with completion using the original equations and constraint checks. This integration improves efficiency in both stages while also addressing constraint feasibility.

The results also support a broader insight that constraint structure can serve as a "computational prior" to guide learning. By establishing reusable computational structures in advance, it enables states and gradients to be computed consistently from the same inexpensive surrogate during training. Across the tested problems, these approximate computations provide effective learning signals without requiring nonlinear completion and exact implicit differentiation at every training update. Allocating computation according to the needs of training and inference thus makes efficiency a consequence of the learning design itself. The saved resources also allow more training or broader model exploration on the same hardware within a limited computational budget.

AuthorsOpenReview ↗

Response to Weakness 2 and Weakness 3

Response to Weakness 2

We thank the reviewer for carefully examining our experimental setup and offering specific suggestions on anchor selection and validation. We first describe how anchors are selected and validated, and then assess their effects on solution quality, gradient approximation, and inference cost through supplementary experiments.

Anchor selection is straightforward. In the current manuscript, an anchor is taken directly from an actual sample and its reference solution labels, namely (xˉ,uˉ,zˉ)=(xj,uj⋆,zj⋆)(\bar{x},\bar{u},\bar{z})=(x_j,u_j^\star,z_j^\star). The selected anchor is then validated following Appendix B, including checks of the surrogate matrix for nonsingularity and conditioning. This procedure requires no additional continuous optimization over a high dimensional anchor space. For example, the training anchor for GasLib135 is the first sample in the dataset used for surrogate construction. Section 4.4 of the original manuscript already examines the effects of anchor conditioning on gradient approximation and warm starts, while Appendix B discusses candidate surrogate construction, selection principles, and extensions using multiple anchors.

We select actual samples at different demand levels as linearization points, construct a linear surrogate at each point, and retrain the network to compare final solution quality. We also hold the predictions of a single network fixed to compare the surrogates' state and gradient approximation errors and the inference cost of warm starts. The results are reported in Table 1.

Table 1. Solution quality, gradient and state errors, and inference cost across anchors

CaseCandidate anchorMean objective gap (%)Joint feasibility rate (%)Mean relative gradient error (%)Normalized state RMS errorMean Newton iterationsMean time (ms per sample)
C-TownCentral demand0.0655100.010.910.00402.13402.7793
C-TownLow total demand0.0654100.015.900.00772.61003.3650
C-TownHigh total demand0.0640100.018.050.00492.43433.1553
GasLib135Central demand0.1418100.01.860.01071.99930.2981
GasLib135Low total demand0.1329100.04.180.02191.99900.2981
GasLib135High total demand0.1412100.01.570.01031.99000.2978

Table 1 shows similar final objective accuracy across linearization points for both the water and gas networks, with a joint feasibility rate of 100.0% in all cases. The choice of linearization point affects state and gradient approximation accuracy and inference time, but these variations do not lead to appreciable differences in final solution quality. This indicates flexibility in selecting and validating linearization points from actual samples. The strong final performance reported in the manuscript therefore need not be attributed to one particular linearization point.

Response to Weakness 3

We thank the reviewer for carefully examining the method and loss design and for specific suggestions on inequality handling and the balance between loss terms.

The EG-Opt framework naturally accommodates DC3's inequality correction step. At inference, the original nonlinear equalities first recover the state corresponding to the predicted controls. If inequalities are violated, the controls are corrected, followed by equality completion and constraint checks. This module requires no changes to the training surrogate or gradient computation. Because DC3 already introduced this correction mechanism, we do not elaborate on it further. Our contribution focuses on efficient gradient computation and a unified design for training and inference, in which the same computational structure supports fast training, faster inference, and recovery of the target solution branch.

AuthorsOpenReview ↗

Response to Question 1 and Question 2

Response to Question 1

We thank the reviewer for carefully crosschecking the main text, appendices, and Figure 2, and for specific questions about the loss definition and the reliability of the gradient comparison.

The gradient comparison in Figure 2 excludes the direct control supervision term, so its shared gradient contribution cannot mask approximation errors. The auxiliary equality penalty remains included. Its gradient is zero under ideal exact completion, whereas surrogate states can yield a nonzero equality penalty gradient. This difference is also included in the comparison. The loss terms and their weights are specified below.

The task loss used for the gradient comparison in Figure 2 is

ℓdiag=λffˉ+λhH+λgG,\ell_{\mathrm{diag}}=\lambda_f\bar f+\lambda_h H+\lambda_g G,

where fˉ\bar f is the normalized objective term, HH is the mean squared equality residual penalty, and GG is a smooth penalty for inequality violations. The coefficients λf\lambda_f, λh\lambda_h, and λg\lambda_g weight the objective, equality penalty, and inequality penalty, respectively.

(1) Weights. In the order of objective, equality penalty, and inequality penalty, the weights are (0.001,1,20)(0.001,1,20) for the power and water networks, (1,1,10000)(1,1,10000) for the gas network, and (0.001,1,0)(0.001,1,0) for CSCP.

(2) Objective term. ff is the objective value at the current state, and fi⋆f_i^\star is the reference objective value for sample ii, where ii indexes the samples. sfs_f is the normalization scale. The power, water, and gas networks use fˉ=f/sf\bar f=f/s_f. CSCP uses the signed objective difference fˉ=(f−fi⋆)/sf\bar f=(f-f_i^\star)/s_f.

(3) Equality term. All cases use a mean squared equality residual penalty. The power and water networks normalize residuals using fixed scales for each case. CSCP uses the standard deviation of each coordinate of the equality right hand side over the training set. The gas network uses the raw residuals.

(4) Inequality term. The power and water networks apply Softplus to normalized constraint violations, take the maximum within each constraint class, and sum these maxima. The smoothing scale controls the transition width near constraint boundaries. It is 0.02 for the power networks and 0.001 for the water and gas networks.

The above specifies the gradient evaluation in Figure 2. Equation (5) gives the basic training objective, Appendix A.3 analyzes the gradient contribution of the auxiliary equality term, and Appendix D.2 describes the full training configuration.

Response to Question 2

We thank the reviewer for carefully considering the role of supervision. We address this question through baseline comparisons and ablation experiments.

CSCP has relatively simple constraints, and direct supervision combined with equality completion already achieves good objective accuracy, explaining the similar performance of EG-Opt and Partial-NN. For the more complex AC-OPF problems, Table 1 of the original manuscript shows that EG-Opt satisfies all constraints in all three cases, whereas Partial-NN does not consistently satisfy the inequalities. This demonstrates EG-Opt's advantage in feasibility.

As requested, we also ablate the direct control supervision weight by setting λs=0\lambda_s=0 and retraining EG-Opt, with λs=1\lambda_s=1 and 0.10.1 as comparisons. The results are reported in Table 2.

Table 2. Effect of direct control supervision weight on EG-Opt solution quality and joint feasibility

CaseMethodSupervision weightMean objective gap (%)Joint feasibility rate (%)
CSCPEG-Opt10.0706100.0
CSCPEG-Opt0.10.0703100.0
CSCPEG-Opt00.0693100.0
Case57EG-Opt10.0712100.0
Case57EG-Opt0.10.4445100.0
Case57EG-Opt04.6065100.0
C-TownEG-Opt10.0655100.0
C-TownEG-Opt0.10.1086100.0
C-TownEG-Opt00.1231100.0
GasLib135EG-Opt10.1413100.0
GasLib135EG-Opt0.10.1993100.0
GasLib135EG-Opt01.2090100.0

Table 2 shows that EG-Opt retains good objective accuracy and a 100.0% joint feasibility rate when direct supervision is substantially reduced or removed. These results indicate that surrogate task gradients support effective learning, while direct supervision provides additional accuracy benefits in some complex cases. The reviewer's suggestion also highlights the potential of this inexpensive gradient computation approach for unsupervised optimization learning. The current method still uses reference labels for calibration, motivating future exploration of calibration and surrogate construction without labels. We thank the reviewer again for this suggestion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.