acceptodds
Under review as a conference paper at ICLR 2027

NEURAL-REFINE: A Learn-and-Refine Framework for Scalable Constrained Optimization

Abstract

Learning-based solvers can amortize the cost of repeatedly solving related constrained optimization problems, but a neural prediction is useful only if it is both feasible and competitive in objective value. Existing predict-then-correct approaches often train the predictor largely independently of the numerical procedure used later to repair or improve its output, so the learned component need not discover starting points that are especially useful to that downstream optimizer. We introduce \method, a closed-loop learn-and-refine framework in which the numerical backend used at deployment also generates training feedback, without differentiating through the solver. The proposer produces multiple candidate starting points and, when useful, compact auxiliary information used by the numerical refiner. Periodically during training, the backend refines these candidates; the best feasible refined solution found so far for each instance is retained as a stop-gradient self-elite target for later proposer updates. At deployment, the trained proposer, deterministic refiner, and a common feasibility-first selector are executed once. Across all nine benchmark tasks formed by QP, QCQP, and SOCP in convex, nonconvex, and nonsmooth settings, \method passes the feasibility criterion on all nine tasks, attains mean feasible rate versus for FSNet, and gives the best feasible objective among learning-based methods on six tasks. It also substantially reduces online solution time relative to CPU IPOPT on the tasks where IPOPT satisfies the same feasibility criterion. In a separate matched-GPU scaling study, numerical solvers are faster at small dimensions, while \method exhibits a crossover as problem size grows. With separate training at each target dimension, a sparse realization remains operational up to variables, extending the same learn-and-refine design from standard benchmark sizes to the billion-variable scale.

Primary area:
optimization
Keywords:
learning to optimize, constrained optimization, feasibility, large-scale optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xbqmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses the difficulty of learning feasible and high-quality solutions for repeated constrained optimization problems. Neural predictors may produce infeasible outputs, and the subsequent numerical correction or refinement can substantially change their objective quality. Consequently, training the predictor without accounting for the deployed refinement procedure may produce starting points that are not effective after refinement. One way to address this issue is to train end-to-end through the refiner, but this can be challenging when the solver is family-specific, non-smooth, includes discrete decisions, or is computationally expensive to unroll. The paper instead uses a bootstrapping procedure: it periodically runs the deployed refiner without gradient tracking, retains the best feasible output found so far for each training instance, and uses that output as a gradient-detached target for training the proposer to generate more effective starting points.

Critical strengths
  • The paper addresses an important and challenging problem: learning solutions that are both feasible and competitive in objective value.
  • The high-level motivation is clear: the learned proposer should account for the numerical refinement procedure used at deployment.
  • The method achieves strong feasibility results on the controlled benchmark, particularly at the stricter tolerance.
  • The paper is fluent, well organized at the presentation level, and appropriately acknowledges several limitations.
Critical weaknesses
  • Despite very fluent exposition, the method is difficult to distill. The paper combines self-elite bootstrapping, learned multistart proposals, solver-specific hints, family-specific refiners, feasibility-first selection, a shared-weight GNN, and scaling experiments without clearly distinguishing the central contribution from supporting design choices.
  • The conceptual contribution and practical advantage over a simpler learned multistart-plus-refinement approach remain unclear.
  • The experiments do not isolate the contribution of the proposed components. In particular, there is no no-self-elite ablation, proposal-count ablation, or equal-budget random/heuristic multistart baseline.
  • The scaling results should be interpreted cautiously: each dimension is trained separately, the generated instances are weakly coupled, and only five test instances are used per scale. These results demonstrate operational scaling on a particular synthetic suite, but not transfer across dimensions or structures.
  • The timing comparisons are difficult to interpret as general speedups because they include unmatched CPU/GPU comparisons, different objective accuracies, and no accounting for training cost.

Overall, the feasibility results are promising, but the paper does not yet make clear what the primary conceptual contribution is or whether the full framework provides a substantial advantage over simpler alternatives. Several design choices appear heuristic or greedy, including the proposal portfolio, diversity mechanism, self-elite updates, and feasibility-first selection, but their individual contributions are not isolated or sufficiently supported by experiments. I would therefore lean toward borderline reject, while viewing the work as potentially interesting if the authors clarify the method and better justify and isolate these choices.

Questions for authors
  • What is the primary contribution, and which components are essential versus implementation-specific?
  • Could “self-elite” be defined more plainly at first use?
  • Why use stop-gradient rather than differentiating through the refiner when differentiation is feasible? Please separate mathematical limitations from practical considerations.
  • How does the bootstrapping mechanism differ from existing self-training, self-imitation, and refinement-based methods?
  • Can the authors provide ablations for the self-elite term and proposal count, along with an equal-budget random or heuristic multistart baseline?
  • What value of KK is used in the scaling experiments, and how does the cost of refining all proposals scale with problem size?
  • What do the scaling experiments establish given that each dimension is trained separately and the instances become weakly coupled?
7 comments
AuthorsOpenReview ↗

Response to Reviewer xbqm (1/3)

Thank you for your constructive feedback. We address the weaknesses and questions below. Additional experiments were run independently; their settings are given in Appendix I. Objectives are test-set means; ± denotes standard deviation across training runs.

Q1 / W1–W2 / Overall — Core Contribution and Design Choices

Weakness W1 (excerpt). The paper combines self-elite bootstrapping, learned multistart proposals, solver-specific hints, family-specific refiners, feasibility-first selection, a shared-weight GNN, and scaling experiments without clearly distinguishing the central contribution from supporting design choices.

Weakness W2. The conceptual contribution and practical advantage over a simpler learned multistart-plus-refinement approach remain unclear.

Question 1. What is the primary contribution, and which components are essential versus implementation-specific?

Response. Our central contribution is a learn-and-refine framework for large-scale constrained optimization. Self-elite trains the proposer on the deployment refiner’s best feasible solution found so far for each instance, without differentiating through the solver (Tables 1–2). Its sparse implementation scales to one billion variables. Structural hints and auxiliary losses support the implementation: the focused ablation below shows that removing structural hints and their losses, or the stationarity loss alone, worsens objectives on both tasks under the same refinement budget. Appendix I.4 reports the complete component ablations.

Table 11. Focused ablations of structural hints and stationarity loss

VariantQP objective ↓QCQP objective ↓
Full-36.267686-26.929243
No structural hints or losses-35.493741-26.361344
No stationarity loss-35.513572-26.282492

All shown variants achieve 100% feasibility at maximum constraint violation ≤ 10⁻⁴. Objectives are test-set means; bold marks the lowest displayed value in each column. One training run per variant per task.

Q2 — Definition of Self-Elite

Question 2. Could “self-elite” be defined more plainly at first use?

Response. We now define self-elite at first use as the best feasible solution found so far by the refiner for each training instance.

Q3 — Why We Use Stop-Gradient

Question 3. Why use stop-gradient rather than differentiating through the refiner when differentiation is feasible? Please separate mathematical limitations from practical considerations.

Response. We use stop-gradient to avoid the memory cost of backpropagating through refinement and to support backends with nonsmooth active-set and acceptance decisions. This interface also accommodates differentiable refiners.

Q4 — Relation to Prior Bootstrapping Methods

Question 4. How does the bootstrapping mechanism differ from existing self-training, self-imitation, and refinement-based methods?

Response. Compared with prediction-based self-training [1] and replay-based self-imitation [2], our supervision comes from numerically improving the model’s proposals, rather than directly reusing its predictions or past outputs. Compared with differentiable correction [3] and learned refinement updates [4], we retain the numerical algorithm and train the initializer from its improved outputs, without differentiating through its iterations.

The closest connection is learning from improved targets. Expert iteration distills search policies [5], while Moving Targets adjusts supervised labels to satisfy constraints [6]. Our targets are solutions to the original constrained optimization problem: each instance retains its best feasible refined solution, updated only when the objective improves. The deployment solver generates these targets, so training transfers its feasible improvements into future initializations. Tables 1–2 demonstrate the benefit of this feedback under identical test refinement.

References

[1]. Van Engelen and Hoos (2020). A survey on semi-supervised learning.

[2]. Oh et al. (2018). Self-Imitation Learning.

[3]. Nguyen and Donti (2025). FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees.

[4]. Lüken and Lucia (2026). Self-Supervised Learning of Iterative Solvers for Constrained Optimization.

[5]. Anthony, Tian and Barber (2017). Thinking Fast and Slow with Deep Learning and Tree Search.

[6]. Detassis, Lombardi and Milano (2021). Teaching the Old Dog New Tricks: Supervised Learning with Constraints.

AuthorsOpenReview ↗

Response to Reviewer xbqm (2/3)

Q5 / W3 / Overall — Self-Elite and Multistart Controls

Weakness W3 (excerpt). The experiments do not isolate the contribution of the proposed components.

Question 5. Can the authors provide ablations for the self-elite term and proposal count, along with an equal-budget random or heuristic multistart baseline?

Response. Tables 1–2 isolate the self-elite benefit through matched retraining. Table 3 compares learned, random and heuristic starts with the same refiner and iteration caps. Pool size 96 and 28 refined starts give the lowest mean final objectives in their respective ablations at the 10⁻⁴ feasibility threshold (Tables 12–13).

Table 12. Candidate pool ablation with one refined start

TaskPool sizeAfter restoration Objective ↓Feasible (%) ↑ 10⁻⁴ / 10⁻⁶After refinement Objective ↓Feasible (%) ↑ 10⁻⁴ / 10⁻⁶
QP174.5699 ± 11.688920.27 / 20.07−18.8623 ± 4.820980.40 / 80.40
QP446.1976 ± 13.249935.47 / 35.20−25.4187 ± 5.393182.80 / 82.73
QP1634.0763 ± 9.821244.07 / 43.80−27.3222 ± 4.927085.13 / 85.07
QP3219.7997 ± 5.683054.53 / 54.40−29.1702 ± 3.224087.20 / 87.20
QP96−35.1683 ± 0.199398.33 / 98.33−35.8543 ± 0.1836100.00 / 100.00
QCQP1−6.5280 ± 23.838484.53 / 84.53−13.7948 ± 16.047486.00 / 6.33
QCQP4−14.2369 ± 17.646397.13 / 97.13−17.6910 ± 14.610798.07 / 9.27
QCQP16−19.0407 ± 10.745698.27 / 98.27−21.4498 ± 8.963499.60 / 10.27
QCQP32−22.7885 ± 4.647098.20 / 98.20−24.6476 ± 3.544599.60 / 11.67
QCQP96−25.7987 ± 0.173198.00 / 98.00−26.8101 ± 0.0623100.00 / 13.67

Pool size varies the subset of 96 generated candidates available for selection. One start is refined with caps of 40 restoration and 40 objective steps. Objectives average all test outputs, including infeasible ones. Results aggregate three trained checkpoints; Appendix I.3 gives the protocol.

Table 13. Number of refined starts under matched total iteration limits

Refined startsMaximum steps per start Restoration / objectiveConvex QP Final objective ↓Convex QCQP Final objective ↓Feasible at 10⁻⁶ (%) ↑ QP / QCQP
11,120 / 1,120−35.8543 ± 0.1836−26.8169 ± 0.0574100.00 / 13.67
4280 / 280−35.9856 ± 0.0984−26.8774 ± 0.0740100.00 / 3.67
8140 / 140−35.9916 ± 0.0913−26.8811 ± 0.0784100.00 / 3.00
1670 / 70−35.9916 ± 0.0913−26.8807 ± 0.0782100.00 / 3.00
2840 / 40−36.2695 ± 0.0042−26.9292 ± 0.000011100.00 / 0.00

All rows achieve 100% feasibility at 10⁻⁴. Each row has the same total caps per instance: 1,120 restoration and 1,120 objective steps. Actual executed work depends on early stopping.

AuthorsOpenReview ↗

Response to Reviewer xbqm (3/3)

Q6 — Candidate Count and Cost in the Scaling Study

Question 6. What value of K is used in the scaling experiments, and how does the cost of refining all proposals scale with problem size?

Response. We generate K = 96 candidates but refine only one selected start. Table 14 reports total online time. Refinement-only time and its dependence on problem size will be added in the camera-ready version.

Table 14. Total online time and start counts in the convex scaling suite

Decision variablesCandidates KRefined startsQP Time (s)QCQP Time (s)SOCP Time (s)
1,0009610.580.6221.21
10,0009610.561.029.26
100,0009610.650.849.71
1,000,0009611.071.4217.45
10,000,0009617.029.5033.82
100,000,00096122.6327.40101.3
1,000,000,000961225.1255.1431.7

Original scaling study: 5/5 instances are feasible at 10⁻⁴ in each row (revised manuscript Table 12).

Q7 / W4 — Scope of the Scaling Results

Question 7. What do the scaling experiments establish given that each dimension is trained separately and the instances become weakly coupled?

Response. Our scaling claim concerns the largest problem size the framework can solve after training at that size. The sparse, weakly coupled benchmark reaches one billion variables. Appendix G and Table 5 provide complementary evidence on coupled AC-OPF, where learned initialization accelerates the same MadNLP backend.

W5 — Timing Comparisons and Training Cost

Weakness W5. The timing comparisons are difficult to interpret as general speedups because they include unmatched CPU/GPU comparisons, different objective accuracies, and no accounting for training cost.

Response. Table 7 compares end-to-end online time at the same feasibility and objective target, with GPU proposals and single-threaded CPU refinement and IPOPT. The paper also includes matched-GPU comparisons and AC-OPF with the same MadNLP backend. Per-scale training hardware and costs will be reported in the camera-ready version.

Reviewer_7xYmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

NEURAL-REFINE proposes a framework for training neural surrogate models (PθP_\theta, or the proposer) that uses a numerical solver (RτR_\tau) to help guide the training process. The proposer is trained using a self-supervised loss and outputs N=96 candidate solutions (and optional information for the solver). Every R epochs, the candidate solutions are passed through a numerical solver and PθP_\theta gets an extra training signal from the solver. The paper also introduces the concept of “self-elite” targets, where the best feasible solution (found so far) is stored as a target for training. Results show improved feasibility and optimality across various problem types and sizes.

Critical strengths
  1. The underlying idea of guiding training with a solver but without differentiating through it makes sense and could be highly useful in domains where a solver is either not differentiable, or prohibitively expensive to differentiate through.
  2. The empirical results are promising (but there are caveats, as described below).
Critical weaknesses
  1. The motivation in its current form is not very convincing, specifically the claim that “Existing predict-then-correct approaches often train the predictor largely independently of the numerical procedure used later to repair or improve its output [...]”. Methods designing differentiable projection layers and feasibility-seeking steps (including the FSNet baseline in the paper) train the predictor end-to-end with the output feasibility correction. Clearer positioning relative to existing work would strengthen the paper.
  2. More extensive empirical testing and ablations are needed to properly establish scalability and other claims:
  • The testing is currently done on 5 test instances per family of problems, which isn’t enough.
  • The AC-OPF examples could also be moved to the main section to strengthen the paper.
  • While the paper acknowledges that the experiments haven’t isolated self-elite training, including that as an ablation would improve the paper.
Questions for authors

In addition to addressing the weaknesses:

  1. Could the authors justify and discuss certain design choices in more detail? For example, the design of the input feature vector is not standard; the assumption that only the right-hand side of equality constraints would change between problem instances is quite strong and may not be true.
  2. I strongly recommend replacing Figure 1 with a more structured (yet concise) diagram that shows the inputs and outputs. In its current form, the paper could benefit from clearer writing.
4 comments
AuthorsOpenReview ↗

Response to Reviewer 7xYm

Thank you for your constructive feedback. We address the weaknesses and questions below. Additional experiments were run independently; their settings are given in Appendix I. Objectives are test-set means; ± denotes standard deviation across training runs.

W1 — Positioning Relative to FSNet

Weakness W1 (excerpt). Methods designing differentiable projection layers and feasibility-seeking steps (including the FSNet baseline in the paper) train the predictor end-to-end with the output feasibility correction. Clearer positioning relative to existing work would strengthen the paper.

Response. FSNet trains through its feasibility correction. We instead learn from the deployment refiner’s solutions without differentiating through it. The revised abstract and introduction clarify this distinction.

W2a — Test Set Size in the Standard and Scaling Experiments

Weakness W2a (excerpt). The testing is currently done on 5 test instances per family of problems, which isn’t enough.

Response. The five-instance count applies only to the original scaling study because large-scale solves are costly. The standard benchmark and new timing experiments use 100 test instances per case (Table 10).

Table 10. Number of training, validation and test instances

Evaluation settingTrainingValidationTest
Standard suite (each of 9 tasks)800100100
Scaling (each setting and dimension)90105
Additional QP/QCQP timing (each task and size)9010100

W2b — AC-OPF Results in the Main Text

Weakness W2b (excerpt). The AC-OPF examples could also be moved to the main section to strengthen the paper.

Response. We have moved a representative same-backend AC-OPF result into the main text (revised manuscript Table 4).

W2c — Self-Elite Ablation

Weakness W2c (excerpt). While the paper acknowledges that the experiments haven’t isolated self-elite training, including that as an ablation would improve the paper.

Response. Paired retraining at 1,000 and 10,000 variables shows lower mean final objectives with self-elite for both tasks at both sizes under the same test refiner (Table 2). Both arms satisfy the same 10⁻⁴ feasibility threshold; Table 1 reports the preceding restoration endpoint.

Q1 — Input Features and Benchmark Parameterization

Question 1. Could the authors justify and discuss certain design choices in more detail? For example, the design of the input feature vector is not standard; the assumption that only the right-hand side of equality constraints would change between problem instances is quite strong and may not be true.

Response. We follow FSNet’s equality-RHS setting for a direct comparison. The four summary statistics—mean, standard deviation, minimum and maximum—are our additional features. The RHS-only ablation changes the objective little in these tests (Appendix I.4).

Q2 — Clarity of Figure 1

Question 2. I strongly recommend replacing Figure 1 with a more structured (yet concise) diagram that shows the inputs and outputs. In its current form, the paper could benefit from clearer writing.

Response. We will revise Figure 1 in the camera-ready version to clarify the inputs, outputs and training feedback.

Thank you for addressing some of the concerns and questions I raised in my review.

Unfortunately, I still don't think the paper is ready in its current state, even accounting for potential camera-ready improvements.

Some of my main concerns remain:

  • A test set of 100 is still quite small-scale. The initial results are promising, but further testing and experimentation against other methods are needed to convincingly demonstrate the method's efficacy.
  • There remain many unconventional design choices or components that are not fully explained in the paper, as other reviewers have also noted. The authors have partially addressed this with new ablations, but the underlying rationale for the method's design choices is still missing and unclear.
  • In my review, I gave the "Presentation" of the paper a "1". That assessment has not changed, as the paper remains difficult to follow, even with the new revisions. This makes it harder to understand and evaluate the proposed method.

Given these remaining concerns, I will maintain my score of 1.

Thank you for identifying the remaining concerns. We would like to address three specific points.

1. Test size and evidence. The standard benchmark uses 100 test instances for each of nine tasks. Revised Table 16 also reports finite-sample uncertainty. For example, on convex QP, feasibility at 10−610^{-6} is 100% with a 95% confidence interval of [96.3%, 100%], versus 1% [0.2%, 5.4%] for FSNet. Such interval separation occurs on eight of the nine tasks, supporting the feasibility advantage on the tested tasks. Appendix I.2 additionally compares against a DiOpt adaptation and random/heuristic starts under the same test refiner.

2. Design rationale. The core mechanism trains the network on good feasible solutions found by the deployment solver, feeding the solver’s improvements back into subsequent predictions. Keeping the best feasible solution prevents target quality from deteriorating with poorer new outputs; stop-gradient avoids backpropagation through solver iterations. Multiple starts address sensitivity to initialization, while auxiliary predictions initialize multipliers or active-set information. Varying only the equality RHS follows FSNet’s benchmark setting; the additional summary features are optional, and removing them changes results little in Appendix I.4. These components serve distinct purposes; the central contribution is the solver-feedback training mechanism.

3. Presentation. Thank you for your feedback. The revised Section 1 distinguishes the core contribution from supporting components, and Appendix B.2 explains the method through a two-dimensional walkthrough. We will further improve Figure 1 to illustrate the training feedback and deployment procedure more clearly.

Thank you again for your time and helpful feedback. We would appreciate another look at our follow-up clarifications and revised manuscript. If these address your remaining concerns, we would be grateful if you could consider a higher score. We are happy to clarify any specific point.

Reviewer_oMj8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes NEURAL-REFINE, a closed-loop framework for repeated constrained optimization. A neural proposer generates multiple initial states and, for some problem classes, auxiliary information for a deterministic numerical refiner. During training, the same backend used at deployment periodically refines the proposals, and the best feasible solution found for each training instance is cached as a stop-gradient self-elite target. The method therefore incorporates solver feedback without differentiating through the solver. The evaluation covers nine synthetic QP, QCQP, and SOCP tasks, a separate sparse scaling study reaching one billion variables with separately trained models, and an AC optimal power flow application. The main reported benefits are improved feasibility and competitive objective values after refinement.

Critical strengths
  1. Reusing the deployment backend to generate training targets is a practically useful design. It accommodates deterministic, family-specific, and potentially non-differentiable refinement routines while avoiding the memory cost of differentiating through their iterations. This makes the interface relevant to applications in which a neural prediction is valuable primarily as an initialization for a numerical solver.

  2. The feasibility-first evaluation provides useful evidence about the complete pipeline. The paper reports performance at both 10^{-4} and 10^{-6} tolerances, and the main benchmark reports 10^{-4} feasibility on all nine tasks. Evaluating several optimization families and adding an AC-OPF application strengthens the practical motivation, although these results do not by themselves isolate the contribution of closed-loop training.

Critical weaknesses
  1. The central benefit of self-elite training is not established by the current controls. The before/after refinement experiment holds the trained proposer fixed, so it demonstrates the value of numerical refinement rather than the incremental benefit of training with self-elites. Appendix F explicitly acknowledges that matched retraining without the elite term and equal-work random or heuristic multistart controls are missing. A static-target distillation baseline with a matched offline solution-generation budget is also needed to distinguish the feedback loop from ordinary learned warm starts. This matters because historical-best caching, self-training, solver initialization, and shared-parameter graph architectures are closely related to existing approaches. Although the paper discusses this overlap, its incremental contribution over methods such as DiOpt should be established through the specific deployment-generated targets and their measured benefit, rather than through the general learning-and-improvement narrative. Without these comparisons, I cannot attribute the reported gains to the paper's primary methodological contribution.

  2. The scaling and timing evidence supports a narrower conclusion than general large-scale optimization capability. At one billion variables, the structural constraint rows touch fewer than 0.1% of the coordinates, and applicable objectives use diagonal quadratic structure. The manuscript acknowledges this limitation, but the experiment still primarily establishes execution and memory scalability on weakly coupled instances. Eliminating independent variables and testing generators that preserve the proportion of coupled variables would make the optimization claim more informative. Timing comparisons also remain difficult to interpret: the standard benchmark uses one fixed, untuned CPU IPOPT configuration; some larger QP comparisons use general-purpose MadNLP; and several speed comparisons use unequal objective accuracy. The paper appropriately labels some of these as deployment references, but this does not replace problem-appropriate numerical baselines and time-to-common-accuracy comparisons. Directly applying IPOPT to nonsmooth formulations without additional treatment is especially limited as evidence about numerical methods more broadly. Reproduction details and the conflicting convex-QP feasibility rates in Tables 1 and 10 further limit confidence in the empirical conclusions.

Questions for authors
  1. Could you provide matched retraining without the self-elite term, equal-total-work random or heuristic multistart through the same refiner, and static-target distillation with the same offline solution-generation budget? Which improvements remain when architecture, candidate count, numerical work, and training budget are controlled?

  2. What is the precise difference from DiOpt's per-instance historical-best candidate mechanism and bootstrapping? Which experiment demonstrates the benefit of using targets generated by the deployment backend?

  3. How do the scaling results change after analytically eliminating independent variables, or when the generator maintains a substantial coupled-variable fraction as dimension increases? Please report objective error on the remaining coupled problem.

  4. Can you provide time-to-common-feasibility-and-objective-accuracy curves against problem-appropriate numerical solvers, with hardware and online work clearly specified? For nonsmooth tasks, what comparison supports conclusions beyond the particular unsmoothed IPOPT configuration?

  5. Could you give the complete refiner updates, acceptance and stopping rules, the number of starts actually retained, and per-scale hardware and training costs? Please also reconcile convex-QP feasibility of 1.00 in Table 1 with 0.99 after refinement in Table 10, and explain how the stationarity auxiliary loss in Eq. (23) accounts for active box constraints.

6 comments
AuthorsOpenReview ↗

Response to Reviewer oMj8 (1/3)

Thank you for your constructive feedback. We address the weaknesses and questions below. Additional experiments were run independently; their settings are given in Appendix I. Objectives are test-set means; ± denotes standard deviation across training runs.

Q1 / W1a–b — Matched Retraining and Self-Elite

Question 1. Could you provide matched retraining without the self-elite term, equal-total-work random or heuristic multistart through the same refiner, and static-target distillation with the same offline solution-generation budget? Which improvements remain when architecture, candidate count, numerical work, and training budget are controlled?

Weakness W1a (excerpt). “The before/after refinement experiment holds the trained proposer fixed, so it demonstrates the value of numerical refinement rather than the incremental benefit of training with self-elites.”

Response. We retrained the same network with and without self-elite at 1,000 and 10,000 variables. Under the same test refiner, self-elite achieves lower mean final objectives for both tasks at both sizes (Table 2); Table 1 shows the preceding restoration endpoint. This control isolates the self-elite term; it does not compare refreshed targets with fixed-target distillation.

Table 1. Mean test objective after feasibility restoration

TaskVariablesWithout self-elite ↓With self-elite ↓
QP1,000−849.6858 ± 1.7261−851.2900 ± 1.2982
QCQP1,000−828.8442 ± 0.0942−831.8280 ± 0.4564
QP10,000−8743.9672−8759.2710
QCQP10,000−8686.7128−8702.7625

Table 2. Mean test objective after objective refinement

TaskVariablesWithout self-elite ↓With self-elite ↓
QP1,000−856.2996 ± 0.0007−856.3447 ± 0.0250
QCQP1,000−840.4466 ± 0.0105−840.5379 ± 0.0342
QP10,000−8778.3924−8778.7791
QCQP10,000−8729.9218−8730.7373

At 1,000 variables: mean ± SD across two paired training runs. At 10,000: one paired run. Each run averages 100 test instances; both arms are 100% feasible at 10⁻⁴ in both tables.

Both tables evaluate the same retrained models with K = 8 and one refined start (Appendix I.1).

Q1 / W1b — Matched Multistart Comparison

Question 1 (excerpt). “equal-total-work random or heuristic multistart through the same refiner”

Response. Learned starts achieve better objectives than random or heuristic starts with the same refiner and iteration limits (Table 3). Learned solver hints are disabled. Early stopping may lead to different amounts of work.

Table 3. Final objective by multistart proposal source

Proposal sourceConvex QP Final objective ↓Convex QCQP Final objective ↓Feasible at 10⁻⁴ QP / QCQP
Uniform random10.18051.6971100% / 100%
Heuristic−16.2336−13.7127100% / 100%
Learned−35.4521−26.0671100% / 100%

Our heuristic combines the bounds midpoint, an equality-corrected zero point and Gaussian midpoint perturbations.

This is our initialization heuristic, following the general multistart approach reviewed by Martí (2003), Multi-Start Methods.


AuthorsOpenReview ↗

Response to Reviewer oMj8 (2/3)

Q2 / W1c — Relation to DiOpt

Question 2. What is the precise difference from DiOpt's per-instance historical-best candidate mechanism and bootstrapping? Which experiment demonstrates the benefit of using targets generated by the deployment backend?

Response. DiOpt reweights sampled and historical candidates; we train on solutions improved by the deployment refiner. Our method achieves better objectives than the DiOpt adaptation with the same test refiner (Table 4). Tables 1–2 separately isolate the self-elite term.

Table 4. DiOpt with the same deployment refiner

Training seedDiOpt Final objective ↓NEURAL-REFINE Final objective ↓Feasible at 10⁻⁴ DiOpt / ours
0−21.6717−35.5318100% / 100%
1−28.3284−35.4347100% / 100%
2−26.1870−34.8444100% / 100%
Mean ± SD−25.3957 ± 3.3982−35.2703 ± 0.3720100% / 100%

This is a full-method comparison. Training updates: DiOpt 40,000; ours 1,100 (Appendix I.2).

Q3 / W2a — Scaling on Coupled Problems

Question 3. How do the scaling results change after analytically eliminating independent variables, or when the generator maintains a substantial coupled-variable fraction as dimension increases? Please report objective error on the remaining coupled problem.

Response. Our scaling study measures the problem size the framework can handle. Appendix G adds coupled AC-OPF, where learned initialization accelerates the same MadNLP backend (Table 5; revised manuscript Table 4).

Few constraint rows also arise in applications. CUTEst [1] includes PORTFL6, a financial-return problem with 12 variables and one constraint, and the portfolio-type PORTSQP test with 100,000 variables and one global equality. MIPLIB's rail507 railway crew-scheduling instance [2] has 63,019 variables and 509 constraints. Counts exclude simple variable bounds.

A balanced transportation model with r sources and r destinations has n = r² variables and m = 2r = 2√n supply/demand rows (one redundant) [3]. These structural examples motivate the constraint-count regime; they are not additional experiments. Few rows do not imply weak coupling. The requested eliminated-variable and fixed-coupling controls remain untested.

Table 5. AC-OPF learned versus flat initialization

AC-OPF caseGPU backendSolver toleranceWall-time speedup ↑
13,659-busMadNLP10⁻⁶14.7×

Speedup = flat-start time / learned-start time with the same backend.

W2b–c / Q4 — Numerical Solver Baselines

Weakness W2b (excerpt). Timing comparisons also remain difficult to interpret: the standard benchmark uses one fixed, untuned CPU IPOPT configuration; some larger QP comparisons use general-purpose MadNLP; and several speed comparisons use unequal objective accuracy.

Response. IPOPT and GPU MadNLP handle the general nonlinear constraints in our QP, QCQP, SOCP and AC-OPF benchmarks. We also report the specialized QP solver MadIPM, which is faster at 1,000 variables (revised manuscript Table 14). Table 7 compares times at a common accuracy.

Weakness W2c / Question 4 (excerpt). Directly applying IPOPT to nonsmooth formulations without additional treatment is especially limited as evidence about numerical methods more broadly.

Response. The nonsmooth result applies only to the direct IPOPT setup in Appendix F. Smooth convex QP and QCQP provide separate timing evidence (Tables 6–7).

Table 6. Online time on smooth convex problems

TaskVariablesNEURAL-REFINE Time (s) ↓MadNLP Time (s) ↓NEURAL-REFINE Objective gap (%) ↓
Convex QP1,000,0001.072.630.59
Convex QCQP1,000,0001.427.170.49

Matched GPU; feasibility threshold 10⁻⁴. Gap is relative to MadNLP. Source: revised manuscript Table 14.

References

[1]. Gould, Orban and Toint (2015). CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization. Instance definitions: PORTFL6, PORTSQP.

[2]. Gleixner et al. (2021). MIPLIB 2017: Data-Driven Compilation of the 6th Mixed-Integer Programming Library. Instance statistics: rail507.

[3]. Mathematical Optimization, Transportation Problem.


AuthorsOpenReview ↗

Response to Reviewer oMj8 (3/3)

Q4 / W2b — Time to Common Accuracy with Retrained Models

Question 4 (excerpt). Can you provide time-to-common-feasibility-and-objective-accuracy curves against problem-appropriate numerical solvers, with hardware and online work clearly specified?

Response. With retrained models, our method has a lower median time to the same feasibility and objective target in all four cases (Table 7; Figure 3 in Appendix I.5 of the revised manuscript). Timing includes the full online pipeline.

Table 7. Time to common accuracy: separate experiments with retrained models

TaskVariablesNEURAL-REFINE Time (ms) ↓IPOPT Time (ms) ↓Time ratio IPOPT / ours ↑
QP1,0006.2014.892.40×
QCQP1,0007.7116.122.09×
QP10,00028.06110.163.93×
QCQP10,00025.98120.314.63×

Median first recorded time to violation ≤ 10⁻⁶ and relative objective excess ≤ 2% against converged IPOPT. Both methods succeed on 100/100 instances per setting.

RTX A5000 proposals and single-threaded CPU refinement; IPOPT uses one CPU thread. These retrained models use K = 8 and one refined start (Appendix I.5).

Q5 / W2d — Refinement and Reproducibility

Question 5 (excerpt). “Please also reconcile convex-QP feasibility of 1.00 in Table 1 with 0.99 after refinement in Table 10”

Response. The two evaluations use independently trained models, with 28 refined starts for the main result and one for the diagnostic. We have clarified these settings in the caption of Table 11 in the revised manuscript.

Question 5 (excerpt). “explain how the stationarity auxiliary loss in Eq. (23) accounts for active box constraints.”

Response. Equation (23) omits bound multipliers, so it is an auxiliary loss rather than a full KKT residual. The refiner enforces the bounds. Appendix B now clarifies this.

Question 5 (excerpt). Could you give the complete refiner updates, acceptance and stopping rules, the number of starts actually retained, and per-scale hardware and training costs?

Response. We added the update, acceptance and stopping rules to Appendix C.2. Table 9 gives the starts retained in the new controls; Table 14 covers the original scaling study.

Table 9. Refinement budget for the 100-variable convex controls

KSelectedPerturbedRefinedRestoration capObjective cap
961612284040

K counts generated candidates; 16 selected + 12 perturbed starts = 28 refined starts. Step caps are per refined start. Per-scale training hardware and costs will be reported in the camera-ready version.

Comments on the paper

1 comment
Area_Chair_8u6EOpenReview ↗

Reminder: Reviewer–Author Discussion

Thank you to the reviewers who have already engaged in the discussion and provided valuable feedback. As the discussion deadline is approaching, I would kindly encourage the remaining reviewer to review the authors' responses and share their follow-up comments, particularly on whether their concerns have been adequately addressed.

Authors are also encouraged to respond promptly to any further questions.

Thank you all for your time and contributions to a constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.