- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EG-Opt uses a sample-calibrated linear surrogate and explicit gradients to replace repeated nonlinear equality completion during training. At inference, a label-independent surrogate provides a warm start for the original nonlinear solver. The paper derives local bounds on state and gradient errors and conditions under which surrogate gradients guide descent. It evaluates ten benchmarks in AC-OPF, water and gas systems, and a controlled nonlinear problem, reporting feasibility, solution quality, training and inference time, gradient fidelity, and a calibration ablation.
Critical strengths
The method addresses the cost of repeated nonlinear completion and implicit differentiation during training. The experiments cover several application areas and measure both training and inference time. The paper also examines gradient accuracy, descent behavior, calibration, and cases where poor conditioning degrades the surrogate.
Critical weaknesses
-
Limited methodological novelty. EG-Opt combines a frozen linear surrogate, calibration at labeled samples, and explicit gradients. The experiments show savings in training time, but the paper does not establish a methodological advance beyond combining these elements or explain how the construction differs in principle from existing surrogate and sensitivity methods.
-
The operating range of one frozen surrogate is unclear. Calibration reproduces states at labeled points, but the paper does not establish how far one fixed matrix remains accurate as operating conditions change. Appendix B discusses multiple surrogates without testing when they are needed or what they cost.
-
The effect of calibration on gradient fidelity is not isolated. The Cal/NoCal ablation reports objective quality and feasibility, while the gradient study compares EG-Opt with Exact-ID and ZO. It does not compare gradient error or descent behavior between Cal and NoCal, leaving calibration’s contribution to the training signal unclear.
-
The theoretical results depend on local conditions. The error bounds and descent result require regularity, suitable conditioning, and proximity to the exact completion. The paper tests failure modes under poor conditioning, but does not show how often these conditions hold during the reported training runs.
-
Inference still requires nonlinear completion. The surrogate provides a warm start, and the paper reports an inference speedup of about 1.5 times. The original solver remains in the deployment path. The practical benefit depends on how consistently the warm start reduces iterations while preserving solution quality and branch recovery.
Questions for authors
-
What methodological advance does EG-Opt provide beyond combining a frozen linear surrogate, sample calibration, and explicit gradients? A comparison with the closest numerical approximation and sensitivity methods would make the claimed contribution more precise.
-
Over what range of operating conditions does one frozen surrogate remain accurate? Please report how state and gradient errors change across that range and indicate when multiple surrogates are needed.
-
Does calibration improve gradient accuracy or descent behavior compared with NoCal? A comparison at matched checkpoints, particularly early in training, would isolate this effect.
-
When do the local error bounds and descent conditions apply during training? Please relate the assumptions to conditioning and approximation errors measured along the training runs.
-
How does EG-Opt’s surrogate warm start differ from warm-start strategies in prior work? A comparison using the same nonlinear solver and predicted controls, with iterations, runtime, and branch recovery reported, would isolate its contribution at inference.
13 comments
Response to Weakness 2 and Question 2 (Part 1 of 3)
Response to Weakness 2 and Question 2 (Part 1 of 3)
We thank the reviewer for carefully examining the surrogate construction and asking specific questions about the applicability of one frozen surrogate and extensions using multiple surrogates. We discuss, in turn, the effects of linearization point selection and demand range, approximation accuracy under changing operating conditions, and the benefits and computational costs of multiple surrogates.
(1) Effects of linearization point selection and demand perturbation range
Part 1. Effects of reasonable anchor choices. Anchor selection is straightforward. In the current manuscript, an anchor is taken directly from an actual sample and its reference solution labels, namely . The selected anchor is then validated following Appendix B, including checks of the surrogate matrix for nonsingularity and conditioning. This procedure requires no additional continuous optimization over a high dimensional anchor space. For example, the training anchor for GasLib135 is the first sample in the dataset used for surrogate construction. Section 4.4 of the original manuscript already examines the effects of anchor conditioning on gradient approximation and warm starts, while Appendix B discusses candidate surrogate construction, selection principles, and extensions using multiple anchors.
We select actual samples at different demand levels as linearization points, construct a linear surrogate at each point, and retrain the network to compare final solution quality. We also hold the predictions of a single network fixed to compare the surrogates' state and gradient approximation errors and the inference cost of warm starts. The results are reported in Table 7.
Table 7. Solution quality, gradient and state errors, and inference cost across anchors
| Case | Candidate anchor | Mean objective gap (%) | Joint feasibility rate (%) | Mean relative gradient error (%) | Normalized state RMS error | Mean Newton iterations | Mean time (ms per sample) |
|---|---|---|---|---|---|---|---|
| C-Town | Central demand | 0.0655 | 100.0 | 10.91 | 0.0040 | 2.1340 | 2.7793 |
| C-Town | Low total demand | 0.0654 | 100.0 | 15.90 | 0.0077 | 2.6100 | 3.3650 |
| C-Town | High total demand | 0.0640 | 100.0 | 18.05 | 0.0049 | 2.4343 | 3.1553 |
| GasLib135 | Central demand | 0.1418 | 100.0 | 1.86 | 0.0107 | 1.9993 | 0.2981 |
| GasLib135 | Low total demand | 0.1329 | 100.0 | 4.18 | 0.0219 | 1.9990 | 0.2981 |
| GasLib135 | High total demand | 0.1412 | 100.0 | 1.57 | 0.0103 | 1.9900 | 0.2978 |
Table 7 shows similar final objective accuracy across linearization points for both the water and gas networks, with a joint feasibility rate of 100.0% in all cases. Linearization point selection affects state and gradient approximation accuracy and inference time, but these variations do not produce appreciable differences in final solution quality among the tested candidates. This indicates flexibility in selecting and validating linearization points from actual samples.
Part 2. Retraining and testing across demand perturbation ranges. To assess whether the same fixed surrogate accommodates changes in demand range, we retrain and test EG-Opt and Exact-ID separately over different demand perturbation ranges, comparing final solution quality and constraint feasibility. The results are reported in Table 8.
Table 8. Demand perturbation ranges, solution quality, and joint feasibility rates
| Case | Range | Global demand perturbation amplitude (%) | Regional demand perturbation amplitude (%) | Node local demand perturbation amplitude (%) | EG-Opt mean objective gap (%) | Exact-ID mean objective gap (%) | EG-Opt joint feasibility rate (%) | Exact-ID joint feasibility rate (%) |
|---|---|---|---|---|---|---|---|---|
| GasLib135 | Narrow | - | 0.0839 | 0.0860 | 100.0 | 100.0 | ||
| GasLib135 | Original | - | 0.1413 | 0.1473 | 100.0 | 100.0 | ||
| GasLib135 | Wide | - | 0.1911 | 0.1948 | 100.0 | 100.0 | ||
| C-Town | Narrow | 0.0005 | 0.0005 | 100.0 | 100.0 | |||
| C-Town | Original | 0.0655 | 0.0632 | 100.0 | 100.0 | |||
| C-Town | Wide | 0.4632 | 0.4313 | 100.0 | 100.0 |
Table 8 shows that objective gaps increase for both methods as the demand perturbation range widens. Nevertheless, EG-Opt maintains objective accuracy close to Exact-ID and a 100.0% joint feasibility rate in every range. These results show that the same sensitivity matrix can be reused when retraining separately over the tested ranges, with sample calibration supporting effective learning.
Response to Weakness 2 and Question 2 (Part 2 of 3)
Response to Weakness 2 and Question 2 (Part 2 of 3)
(2) State and gradient approximation errors under changing demand
We compare the surrogate's state and gradient approximation errors under different operating conditions, as reported in Table 9. RMS is the mean of the root mean squared errors between approximate and exact states, computed for each sample. is the mean relative error between approximate and exact gradient vectors. is the mean cosine of the angle between these vectors, with values closer to 1 indicating stronger directional agreement. The positive inner product rate is the fraction of valid samples for which the two gradient vectors have a positive inner product.
Table 9. State and gradient approximation errors under different operating conditions
GasLib135
| Factor | RMS | Positive inner product rate | ||
|---|---|---|---|---|
| 0.90 | 0.2090 | 0.0810 | 1.0000 | 100.00% |
| 0.95 | 0.0591 | 0.0356 | 1.0000 | 100.00% |
| 1.00 | 0.0103 | 0.0148 | 1.0000 | 100.00% |
| 1.05 | 0.0305 | 0.0757 | 1.0000 | 100.00% |
| 1.10 | 0.0289 | 0.1685 | 1.0000 | 100.00% |
C-Town
| Factor | RMS | Positive inner product rate | ||
|---|---|---|---|---|
| 0.80 | 0.0987 | 0.1546 | 0.9898 | 100.00% |
| 0.90 | 0.0125 | 0.1813 | 0.9829 | 100.00% |
| 1.00 | 0.0040 | 0.0984 | 0.9940 | 100.00% |
| 1.10 | 0.0480 | 0.1059 | 0.9974 | 100.00% |
| 1.20 | 0.3312 | 0.1581 | 0.9945 | 100.00% |
CSCP
| Factor | RMS | Positive inner product rate | ||
|---|---|---|---|---|
| 0.80 | 0.0002 | 0.0400 | 0.9990 | 100.00% |
| 0.90 | 0.0002 | 0.0482 | 0.9985 | 100.00% |
| 1.00 | 0.0004 | 0.0548 | 0.9981 | 100.00% |
| 1.10 | 0.0007 | 0.0628 | 0.9974 | 100.00% |
| 1.20 | 0.0011 | 0.0730 | 0.9963 | 100.00% |
Table 9 shows that operating conditions affect state errors and relative gradient errors. Nevertheless, the mean cosine between approximate and exact gradients remains close to 1 in all three cases across the tested conditions, and the gradient inner product is positive for every valid sample. Thus, variations in state approximation error do not disrupt gradient directional agreement on these samples, and the fixed surrogate continues to provide useful directions for task optimization. The applicability of a single surrogate should therefore be assessed by both state accuracy and preservation of the exact gradient direction. These results provide empirical support for reusing the same surrogate across the tested operating range.
Response to Weakness 1 and Question 1
Response to Weakness 1 and Question 1
We thank the reviewer for carefully examining the method and considering EG-Opt's methodological contribution and its distinction from related approaches. Its main methodological advances are as follows.
(1) Using constraint structure as a computational prior for learning. EG-Opt's results support the broader insight that constraint structure can serve as a computational prior to guide learning. Establishing reusable computational structures in advance enables states and gradients to be computed consistently from the same inexpensive surrogate during training. Across the tested problems, these approximate computations provide effective learning signals without requiring nonlinear completion and exact implicit differentiation at every update. Allocating computation according to the needs of training and inference makes efficiency a consequence of the learning design itself. The saved resources also allow more training or broader model exploration on the same hardware within a limited computational budget.
(2) Providing task learning signals without direct control supervision. After removing direct control supervision, EG-Opt still achieves a 100.0% joint feasibility rate and good objective accuracy in all four tested cases. This shows that surrogate gradients support effective learning and suggests potential extensions to unsupervised optimization learning. To provide empirical evidence, we retrain EG-Opt with the direct control supervision weight λs set to 0, 0.1, and 1. The results are reported in Table 6.
Table 6. Effect of direct control supervision weight on EG-Opt solution quality and joint feasibility
Table 6 shows that all cases retain good objective accuracy and 100% feasibility when direct supervision is reduced or removed. This suggests the potential of inexpensive gradient computation for unsupervised optimization learning.
Among related methods, modified Newton [1] reuses matrices and their factorizations but still requires iterative root finding. Zeroth order additive correction [2] matches the function value at a reference point without changing the derivative. First order model management (AMMF) [3] additionally matches gradients and updates the surrogate during optimization. JFB and phantom gradients retain the forward solve and primarily simplify backward computation. When nonlinear equalities require numerical solution, DC3 still performs completion using the original equations and implicit backpropagation, with differentiable correction for inequalities. EG-Opt instead precomputes sample calibration using existing labels and reuses a fixed operator across samples and training updates. The same explicit surrogate supplies both states and gradients, reducing repeated computation in both forward solves and backward differentiation.
References
[1] Alan C. Hindmarsh, Radu Serban, Cody J. Balos, David J. Gardner, Daniel R. Reynolds, and Carol S. Woodward. User documentation for KINSOL. Lawrence Livermore National Laboratory, 2026. URL https://sundials.readthedocs.io/en/latest/kinsol/Mathematics_link.html. Version 7.9.0, Sections 7.2.1–7.2.3. Accessed November 15, 2026.
[2] Sandia National Laboratories. Dakota 6.18 Documentation: Correction Approaches for Surrogate Models, 2023. URL https://snl-dakota.github.io/docs/6.18.0/users/usingdakota/reference/model-surrogate-global-correction.html. Accessed November 15, 2026.
[3] N. M. Alexandrov, E. J. Nielsen, R. M. Lewis, and W. K. Anderson. First-order model management with variable-fidelity physics applied to multi-element airfoil optimization. In 8th Symposium on Multidisciplinary Analysis and Optimization. American Institute of Aeronautics and Astronautics, 2000. doi: 10.2514/6.2000-4886. URL https://ntrs.nasa.gov/citations/20000097390. AIAA Paper 2000-4886.