- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper addresses the difficulty of learning feasible and high-quality solutions for repeated constrained optimization problems. Neural predictors may produce infeasible outputs, and the subsequent numerical correction or refinement can substantially change their objective quality. Consequently, training the predictor without accounting for the deployed refinement procedure may produce starting points that are not effective after refinement. One way to address this issue is to train end-to-end through the refiner, but this can be challenging when the solver is family-specific, non-smooth, includes discrete decisions, or is computationally expensive to unroll. The paper instead uses a bootstrapping procedure: it periodically runs the deployed refiner without gradient tracking, retains the best feasible output found so far for each training instance, and uses that output as a gradient-detached target for training the proposer to generate more effective starting points.
Critical strengths
- The paper addresses an important and challenging problem: learning solutions that are both feasible and competitive in objective value.
- The high-level motivation is clear: the learned proposer should account for the numerical refinement procedure used at deployment.
- The method achieves strong feasibility results on the controlled benchmark, particularly at the stricter tolerance.
- The paper is fluent, well organized at the presentation level, and appropriately acknowledges several limitations.
Critical weaknesses
- Despite very fluent exposition, the method is difficult to distill. The paper combines self-elite bootstrapping, learned multistart proposals, solver-specific hints, family-specific refiners, feasibility-first selection, a shared-weight GNN, and scaling experiments without clearly distinguishing the central contribution from supporting design choices.
- The conceptual contribution and practical advantage over a simpler learned multistart-plus-refinement approach remain unclear.
- The experiments do not isolate the contribution of the proposed components. In particular, there is no no-self-elite ablation, proposal-count ablation, or equal-budget random/heuristic multistart baseline.
- The scaling results should be interpreted cautiously: each dimension is trained separately, the generated instances are weakly coupled, and only five test instances are used per scale. These results demonstrate operational scaling on a particular synthetic suite, but not transfer across dimensions or structures.
- The timing comparisons are difficult to interpret as general speedups because they include unmatched CPU/GPU comparisons, different objective accuracies, and no accounting for training cost.
Overall, the feasibility results are promising, but the paper does not yet make clear what the primary conceptual contribution is or whether the full framework provides a substantial advantage over simpler alternatives. Several design choices appear heuristic or greedy, including the proposal portfolio, diversity mechanism, self-elite updates, and feasibility-first selection, but their individual contributions are not isolated or sufficiently supported by experiments. I would therefore lean toward borderline reject, while viewing the work as potentially interesting if the authors clarify the method and better justify and isolate these choices.
Questions for authors
- What is the primary contribution, and which components are essential versus implementation-specific?
- Could “self-elite” be defined more plainly at first use?
- Why use stop-gradient rather than differentiating through the refiner when differentiation is feasible? Please separate mathematical limitations from practical considerations.
- How does the bootstrapping mechanism differ from existing self-training, self-imitation, and refinement-based methods?
- Can the authors provide ablations for the self-elite term and proposal count, along with an equal-budget random or heuristic multistart baseline?
- What value of is used in the scaling experiments, and how does the cost of refining all proposals scale with problem size?
- What do the scaling experiments establish given that each dimension is trained separately and the instances become weakly coupled?
7 comments
Response to Reviewer xbqm (2/3)
Q5 / W3 / Overall — Self-Elite and Multistart Controls
Weakness W3 (excerpt). The experiments do not isolate the contribution of the proposed components.
Question 5. Can the authors provide ablations for the self-elite term and proposal count, along with an equal-budget random or heuristic multistart baseline?
Response. Tables 1–2 isolate the self-elite benefit through matched retraining. Table 3 compares learned, random and heuristic starts with the same refiner and iteration caps. Pool size 96 and 28 refined starts give the lowest mean final objectives in their respective ablations at the 10⁻⁴ feasibility threshold (Tables 12–13).
Table 12. Candidate pool ablation with one refined start
| Task | Pool size | After restoration Objective ↓ | Feasible (%) ↑ 10⁻⁴ / 10⁻⁶ | After refinement Objective ↓ | Feasible (%) ↑ 10⁻⁴ / 10⁻⁶ |
|---|---|---|---|---|---|
| QP | 1 | 74.5699 ± 11.6889 | 20.27 / 20.07 | −18.8623 ± 4.8209 | 80.40 / 80.40 |
| QP | 4 | 46.1976 ± 13.2499 | 35.47 / 35.20 | −25.4187 ± 5.3931 | 82.80 / 82.73 |
| QP | 16 | 34.0763 ± 9.8212 | 44.07 / 43.80 | −27.3222 ± 4.9270 | 85.13 / 85.07 |
| QP | 32 | 19.7997 ± 5.6830 | 54.53 / 54.40 | −29.1702 ± 3.2240 | 87.20 / 87.20 |
| QP | 96 | −35.1683 ± 0.1993 | 98.33 / 98.33 | −35.8543 ± 0.1836 | 100.00 / 100.00 |
| QCQP | 1 | −6.5280 ± 23.8384 | 84.53 / 84.53 | −13.7948 ± 16.0474 | 86.00 / 6.33 |
| QCQP | 4 | −14.2369 ± 17.6463 | 97.13 / 97.13 | −17.6910 ± 14.6107 | 98.07 / 9.27 |
| QCQP | 16 | −19.0407 ± 10.7456 | 98.27 / 98.27 | −21.4498 ± 8.9634 | 99.60 / 10.27 |
| QCQP | 32 | −22.7885 ± 4.6470 | 98.20 / 98.20 | −24.6476 ± 3.5445 | 99.60 / 11.67 |
| QCQP | 96 | −25.7987 ± 0.1731 | 98.00 / 98.00 | −26.8101 ± 0.0623 | 100.00 / 13.67 |
Pool size varies the subset of 96 generated candidates available for selection. One start is refined with caps of 40 restoration and 40 objective steps. Objectives average all test outputs, including infeasible ones. Results aggregate three trained checkpoints; Appendix I.3 gives the protocol.
Table 13. Number of refined starts under matched total iteration limits
| Refined starts | Maximum steps per start Restoration / objective | Convex QP Final objective ↓ | Convex QCQP Final objective ↓ | Feasible at 10⁻⁶ (%) ↑ QP / QCQP |
|---|---|---|---|---|
| 1 | 1,120 / 1,120 | −35.8543 ± 0.1836 | −26.8169 ± 0.0574 | 100.00 / 13.67 |
| 4 | 280 / 280 | −35.9856 ± 0.0984 | −26.8774 ± 0.0740 | 100.00 / 3.67 |
| 8 | 140 / 140 | −35.9916 ± 0.0913 | −26.8811 ± 0.0784 | 100.00 / 3.00 |
| 16 | 70 / 70 | −35.9916 ± 0.0913 | −26.8807 ± 0.0782 | 100.00 / 3.00 |
| 28 | 40 / 40 | −36.2695 ± 0.0042 | −26.9292 ± 0.000011 | 100.00 / 0.00 |
All rows achieve 100% feasibility at 10⁻⁴. Each row has the same total caps per instance: 1,120 restoration and 1,120 objective steps. Actual executed work depends on early stopping.
Response to Reviewer xbqm (3/3)
Q6 — Candidate Count and Cost in the Scaling Study
Question 6. What value of K is used in the scaling experiments, and how does the cost of refining all proposals scale with problem size?
Response. We generate K = 96 candidates but refine only one selected start. Table 14 reports total online time. Refinement-only time and its dependence on problem size will be added in the camera-ready version.
Table 14. Total online time and start counts in the convex scaling suite
| Decision variables | Candidates K | Refined starts | QP Time (s) | QCQP Time (s) | SOCP Time (s) |
|---|---|---|---|---|---|
| 1,000 | 96 | 1 | 0.58 | 0.62 | 21.21 |
| 10,000 | 96 | 1 | 0.56 | 1.02 | 9.26 |
| 100,000 | 96 | 1 | 0.65 | 0.84 | 9.71 |
| 1,000,000 | 96 | 1 | 1.07 | 1.42 | 17.45 |
| 10,000,000 | 96 | 1 | 7.02 | 9.50 | 33.82 |
| 100,000,000 | 96 | 1 | 22.63 | 27.40 | 101.3 |
| 1,000,000,000 | 96 | 1 | 225.1 | 255.1 | 431.7 |
Original scaling study: 5/5 instances are feasible at 10⁻⁴ in each row (revised manuscript Table 12).
Q7 / W4 — Scope of the Scaling Results
Question 7. What do the scaling experiments establish given that each dimension is trained separately and the instances become weakly coupled?
Response. Our scaling claim concerns the largest problem size the framework can solve after training at that size. The sparse, weakly coupled benchmark reaches one billion variables. Appendix G and Table 5 provide complementary evidence on coupled AC-OPF, where learned initialization accelerates the same MadNLP backend.
W5 — Timing Comparisons and Training Cost
Weakness W5. The timing comparisons are difficult to interpret as general speedups because they include unmatched CPU/GPU comparisons, different objective accuracies, and no accounting for training cost.
Response. Table 7 compares end-to-end online time at the same feasibility and objective target, with GPU proposals and single-threaded CPU refinement and IPOPT. The paper also includes matched-GPU comparisons and AC-OPF with the same MadNLP backend. Per-scale training hardware and costs will be reported in the camera-ready version.
Response to Reviewer xbqm (1/3)
Thank you for your constructive feedback. We address the weaknesses and questions below. Additional experiments were run independently; their settings are given in Appendix I. Objectives are test-set means; ± denotes standard deviation across training runs.
Q1 / W1–W2 / Overall — Core Contribution and Design Choices
Response. Our central contribution is a learn-and-refine framework for large-scale constrained optimization. Self-elite trains the proposer on the deployment refiner’s best feasible solution found so far for each instance, without differentiating through the solver (Tables 1–2). Its sparse implementation scales to one billion variables. Structural hints and auxiliary losses support the implementation: the focused ablation below shows that removing structural hints and their losses, or the stationarity loss alone, worsens objectives on both tasks under the same refinement budget. Appendix I.4 reports the complete component ablations.
Table 11. Focused ablations of structural hints and stationarity loss
All shown variants achieve 100% feasibility at maximum constraint violation ≤ 10⁻⁴. Objectives are test-set means; bold marks the lowest displayed value in each column. One training run per variant per task.
Q2 — Definition of Self-Elite
Response. We now define self-elite at first use as the best feasible solution found so far by the refiner for each training instance.
Q3 — Why We Use Stop-Gradient
Response. We use stop-gradient to avoid the memory cost of backpropagating through refinement and to support backends with nonsmooth active-set and acceptance decisions. This interface also accommodates differentiable refiners.
Q4 — Relation to Prior Bootstrapping Methods
Response. Compared with prediction-based self-training [1] and replay-based self-imitation [2], our supervision comes from numerically improving the model’s proposals, rather than directly reusing its predictions or past outputs. Compared with differentiable correction [3] and learned refinement updates [4], we retain the numerical algorithm and train the initializer from its improved outputs, without differentiating through its iterations.
The closest connection is learning from improved targets. Expert iteration distills search policies [5], while Moving Targets adjusts supervised labels to satisfy constraints [6]. Our targets are solutions to the original constrained optimization problem: each instance retains its best feasible refined solution, updated only when the objective improves. The deployment solver generates these targets, so training transfers its feasible improvements into future initializations. Tables 1–2 demonstrate the benefit of this feedback under identical test refinement.
References
[1]. Van Engelen and Hoos (2020). A survey on semi-supervised learning.
[2]. Oh et al. (2018). Self-Imitation Learning.
[3]. Nguyen and Donti (2025). FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees.
[4]. Lüken and Lucia (2026). Self-Supervised Learning of Iterative Solvers for Constrained Optimization.
[5]. Anthony, Tian and Barber (2017). Thinking Fast and Slow with Deep Learning and Tree Search.
[6]. Detassis, Lombardi and Milano (2021). Teaching the Old Dog New Tricks: Supervised Learning with Constraints.