acceptodds
Under review as a conference paper at ICLR 2027

Feasible by Tolerance or by Geometry? Radial Retractions in Neural Solvers for Constrained Optimization

Abstract

Many neural solvers for parametric constrained programs correct the network's prediction with a differentiable projection or feasibility iteration, which approaches the feasible set from outside and stops at a small residual. Whether the corrected output passes a fixed feasibility check then depends on how this iteration was stopped. Evaluating all methods under one strict rule, we find that the strict feasibility rate of two recent solvers moves from 0% to 100% as the iteration budget, the floating-point precision or the stopping tolerance changes, while the trained network stays fixed. We propose RRA (radial retraction from an anchor), which reaches the boundary from inside instead. A differentiable Gauss–Newton interior map moves the network's proposal to a strictly feasible anchor in the basin the network selected, and a radial retraction stops at the first boundary contact on the ray from this anchor toward the proposal. The exit parameter of this ray is a closed-form root for affine, convex-quadratic, box and second-order-cone constraints, and a scan brackets it from below for trigonometric constraints. The returned point is then feasible for any strictly feasible anchor, whatever the tolerance, depth or convergence of the map that placed it. RRA passes the strict rule on every test instance and training seed of ten families of convex and nonconvex QP, QCQP and SOCP, and its optimality gap is lower than that of FSNet on all ten. It is on par with the most accurate correction baseline while training 5 to 14 times faster. Longer training, which still takes at most 55% of that baseline's training time, moves RRA ahead of it. RRA also solves the two nonconvex second-order-cone families on which that baseline returns almost no feasible point.

Primary area:
optimization
Keywords:
neural solvers, hard constraints, constrained optimization, differentiable optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PzYBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RRA, a neural solver architecture for parametric constrained optimization that enforces feasibility through a geometric construction rather than relying on iterative stopping tolerances. It first maps a network proposal to a strictly feasible anchor using a Gauss–Newton interior map, then retracts along the ray toward the proposal until reaching the first boundary contact. Closed-form exit calculations are provided for affine, convex-quadratic, box, and second-order-cone constraints. Across ten benchmark families, RRA achieves 100% feasibility under the paper's strict evaluation criterion, improves optimality over FSNet, and achieves competitive accuracy with substantially lower training costs than ENFORCE.

Critical strengths
  1. The paper presents an interesting geometric approach to enforcing feasibility by combining a differentiable interior map with radial retraction.
  2. The experiments cover various benchmark problems and multiple baselines, demonstrating strong performance in terms of feasibility, optimality, and training efficiency.
Critical weaknesses
  1. Dependence on feasible anchors. The method requires a strictly feasible anchor for each instance, which can be computationally expensive, especially for nonconvex SOCPs, where the inference time is substantially longer than that of FSNet, as shown in Table 16.
  2. Limitations of the feasibility guarantee. For nonconvex SOCPs, the scanning procedure does not guarantee that the first boundary crossing is found. Although this limitation is acknowledged, a quantitative analysis of potentially missed crossings would help clarify its practical significance.
  3. Insufficient ablation of the interior map. The experiments show that the Gauss–Newton interior map improves performance, but they do not clearly distinguish the benefits of better anchor locations from those of differentiating through the interior map.
Questions for authors
  1. How often does the interior map fail and fall back to the global anchor?
  2. Could the authors compare full differentiation with stop-gradient while keeping the same Gauss–Newton forward map?
  3. How sensitive are the results to the interior margin, damping parameter, scan resolution, and evaluator feasibility thresholds?
1 comment
AuthorsOpenReview ↗

Reply to Reviewer PzYB

We thank the reviewer for the thoughtful questions, which led to the analyses below, now in the revised paper (changes in blue).

W1. Cost of the anchor. On eight of the ten families RRA takes 0.2 to 3.0 times the inference time of FSNet and is faster on five of them (Section 4.4). On the two nonconvex SOC families the anchor search is costly, and it buys much lower gaps than FSNet and feasible outputs that ENFORCE does not return.

  • On nonconvex SOCP and SOCP$+$, RRA takes 114 to 127 times the time of FSNet on the L40S of Table 17 and 64 to 75 times on one A100 GPU, as Section 4.4 and the conclusion now state.
  • A lighter search with 8 starts and 100 Adam steps takes 22 to 25 times the time of FSNet, and on 2,000 fresh instances per family its mean objective is at most 0.36% above that of the reported search (Appendix J.3). A rule fixed before the validation runs chose it among nine settings that we defined after their test results were known.
  • On these two families RRA reaches gaps of 8.62% and 14.2%, against 24.8% and 56.7% for FSNet (Table 2), and ENFORCE returns almost no feasible point in 23 of 24 training runs.

W2. Missed crossings on nonconvex SOC. The quantitative analysis the reviewer asked for finds no missed crossing in the reported runs, and the solver now checks the first crossing itself.

  • The Taylor check of Lemma C.3 proves that no crossing lies inside a scanned cell, and a new curvature bound for squared cone rows (Lemma J.1) extends this check to the cone rows.
  • On every ray of the reported runs, both families and three seeds (Appendix J.4):
testapplied tooutcome
Taylor check with the new bound95,192 rays of positive lengthno missed crossing
Taylor check on the sliding stages286,645 moves286,635 proved
solver with the check built in (Algorithm 2)12,000 test instancessame outputs as Table 2
  • On the ten moves that the check leaves unproved, the largest row value is 2.2×10−142.2\times10^{-14}.
  • Every output also passes a row check at cx≤10−10c_x\le10^{-10} (Lemma C.5), the bound that Section 3.3 now states in place of the submitted cx≤0c_x\le0. With this threshold set to 0, the gap is unchanged and the largest row value falls to 1.5×10−141.5\times10^{-14} (Appendix I.1).

W3, Q2. Anchor location against the derivative of the map. With the same Gauss–Newton forward map, replacing only its derivative raises the gap on every family, so training through the map matters beyond where the anchor lies. The comparison the reviewer proposed is now Table 3 (Section 4.3).

On the nine families where RRA's gap is positive:

derivative of the interior mapgap relative to RRA
full (RRA)1
zero (stop-gradient)6.3 to 1,680
identity2.3 to 22
  • On nonconvex QCQP$+$ they raise RRA's gap of −0.0058%-0.0058\% to 12.6% and 0.64%.
  • All 30 paired 95% intervals for the zero derivative lie above zero.

Q1. Fallback to the global anchor. At the reported settings the returned output starts from the global anchor on at most one instance per seed of any family (Appendix H.1).

  • On the eight families without nonconvex cone rows, the output of the map passed its test on all 48,000 test instances.
  • On the two nonconvex SOC families, where the solver keeps four anchors per instance, the map fails in 0.4% to 9.4% of its first-step runs, yet the returned output starts from the map on 1,999 or 2,000 of 2,000 instances per seed.

Q3. Sensitivity. No setting we varied made a returned output fail the strict rule. The gap responds to the margin and hardly to the damping or the scan.

  • In 840 test-time runs that change one setting at a time on all ten families, all 1,665,509 returned outputs pass (Appendix I.1). The 14,491 instances without an anchor, on nonconvex QCQP and QCQP$+$ with the map off or weakened, are reported as unsolved.

Gap (%) for three values of the margin μ\mu:

μ\muconvex QPnonconvex QP
10−810^{-8}0.00170.397
10−410^{-4} (reported)0.00210.128
10−210^{-2}0.0320.142
  • At μ=10−8\mu=10^{-8} the output of the map misses the acceptance test on 102 of 6,000 nonconvex QP instances, which then start from the global anchor.

  • The damping ρ\rho of Eq. (1) changes the gap by at most 2.5% of its value from 10−1210^{-12} to 10−410^{-4}.

  • Scans with 8 to 128 cells and 5 to 50 bisection steps change the gap on the four trigonometric families by at most 0.00037 points.

  • On nonconvex SOC, the margin and the ridge move the gap by at most 0.45 points.

  • Every output passes at every common evaluator tolerance down to 10−1010^{-10} (Appendix I.2). At 10−1210^{-12}, 86.8% and 93.4% of the nonconvex SOC outputs pass, since their sliding stages accept rows up to 10−1010^{-10}.

We hope these analyses help, and we would be glad to discuss any point further.

Reviewer_UAcVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses feasibility guarantees for neural solvers. RRA obtains a strictly feasible interior anchor through a differentiable Gauss–Newton map, then retracts toward the network’s proposal. Boundary contact is computed in closed form for several constraint classes or located through a scan. The method is evaluated on convex and nonconvex QP, QCQP, and SOCP families.

Critical strengths
  • The paper addresses an important problem with a potentially compelling approach: separating anchor placement from final feasibility enforcement.
  • The evaluation covers multiple constraint families and training seeds, reporting feasibility, objective quality, and runtime. The fixed-network sweeps usefully illustrate how numerical settings affect reported feasibility.
Critical weaknesses
  1. Exposition.

    Exposition is a key concern. I found it difficult to understand what problem the paper addresses, what it proposes, and which gaps in existing methods it fills. The introduction discusses comparisons and results before adequately explaining the problem, method, or competing approaches.

    The organization compounds this difficulty. The central “strict rule” appears in the abstract and repeatedly in the introduction but is defined only in Section 2, which focuses on benchmark details rather than establishing the general problem. Conceptual explanations, technical definitions, implementation details, and results are frequently mixed without sufficient context. The nonconvex SOCP extension is particularly abrupt.

    The contribution list also reads more as a summary than an explanation of what is new, especially relative to radial constructions such as RAYEN. A clearer conceptual presentation and sharper positioning are needed.

  2. Claims and their support.

    Several claims appear stronger than the theory and experiments support and should be qualified:

    • Proposition 1 supports the ideal first-contact construction given a strictly feasible anchor. However, the nonconvex SOCP scan does not certify the first crossing, and the method depends on successfully finding an interior anchor. The paper should distinguish the ideal guarantee, implemented endpoint checks, and empirical feasibility rates.
    • The claim of feasibility independent of stopping rules needs qualification: the implementation still uses finite scans, bisection, and numerical acceptance thresholds. The “strict rule” is itself tolerance-based. Sensitivity to evaluation tolerances and constraint-specific residuals would strengthen the comparison.
    • The ablations change both anchor placement and differentiation, so they do not isolate the claimed benefit of the exact interior-map derivative. A controlled backward-only ablation is needed to support this attribution.
    • Practical advantages should be qualified by the roughly 114–127-fold inference cost relative to FSNet on the nonconvex SOCP families. The objective-quality trade-off introduced by radial retraction also needs clearer discussion.

    Please tone down the relevant claims or provide additional evidence, clearly separating conditional theoretical guarantees from observed benchmark performance.

Questions for authors
  • How is a strictly feasible anchor found and verified, and what happens when the search fails? Can the method handle feasible sets with no strict interior or insufficient slack for the required margin?
  • Which guarantees apply to the implemented nonconvex SOCP method when its scan does not exclude earlier crossings between grid points?
  • What precisely is the “basin selected by the network”? How is this interpreted for convex sets, and what preserves it when the global-anchor fallback is used?
  • How does RRA differ from RAYEN in obtaining and adapting the interior anchor? Can you provide a direct comparison on supported convex families?
  • Please explain “the solver works in pp” more directly and clarify Figure 1’s numerical settings and sources of feasibility failure.
5 comments
AuthorsOpenReview ↗

Reply to Reviewer UAcV (1/2): exposition and claims (W1, W2)

We thank the reviewer for the careful reading. Changes are marked in blue in the revised PDF.

W1. Exposition. The revision changes the text at each of the places the review names.

  • The introduction now states the problem, the correction methods, the constructions that are feasible by design and the gap they leave, and only then proposes RRA (pages 1 and 2).
  • The introduction defines the strict rule at its first use, and the abstract gives its tolerances.
  • Section 3 states the method for a general set Cx={p:cx(p)≤0}\mathcal C_x=\{p:c_x(p)\le0\}, for which Proposition 1 assumes only continuous rows and a strictly feasible anchor.
  • Section 3.1 states when an anchor is accepted and when an instance is unsolved, defines the basin, and leaves the global search to Appendix E.1.
  • The nonconvex SOC extension has its own Section 3.3, and contribution 2 states what RRA adds to RAYEN (Q4).

W2, first point and closing request. Guarantee, check and measured rate. Table 36 (Appendix K) now separates the three for each family group, and the abstract, the introduction and contribution 2 state the guarantee in its conditional form.

familiesguaranteerows checked atcoverage under strict rate
four convexProposition 1, closed-form exitthe anchor100%, 100%
nonconvex QP, QP$+,QCQP,QCQP, QCQP, QCQP+$Proposition 1, bracketed exit (Lemma C.3)the anchor and the scan100%, 100%
nonconvex SOCP, SOCP$+$cx≤10−10c_x\le10^{-10} (Lemma C.5)the anchor and every output100%, 100%
  • Proposition 1 holds for any strictly feasible anchor. An instance without one is reported as unsolved and counts as failing in the strict rate (Section 2, Algorithm 1).

W2, second point. Scans, bisection and thresholds. These settings decide where the output lands and whether an anchor exists, but no setting we tested made a returned output fail the strict rule.

  • In 840 test-time runs on all ten families that change one setting at a time, all 1,665,509 returned outputs pass (Appendix I.1).
  • The scan and the bisection return the lower end of a bracket at which every row was evaluated and found nonpositive, so their resolution sets only how close to the crossing the output stops.
  • A weakened map leaves 14,491 instances of nonconvex QCQP and QCQP$+$ without an anchor, and counted as failures they put those runs at 71.1% to 99.9% (Appendix I.1). The settings of the map thus decide whether an instance has an anchor, as the text after Proposition 1 now states, and the solver reports such an instance instead of returning a point.
  • With all evaluator tolerances set to a common τ\tau, RRA passes on every family and seed down to τ=10−10\tau=10^{-10} (Appendix I.2). ENFORCE does so on the five families where it runs with ε=10−10\varepsilon=10^{-10}, and FSNet passes on at most 12.4% of its outputs at 10−610^{-6} on nine families.

Largest residual over all reported outputs:

methodequalityinequalitybox
RRA8.5×10−148.5\times10^{-14}4.8×10−144.8\times10^{-14} (9.98×10−119.98\times10^{-11} on nonconvex SOC)0
FSNet2.5×10−42.5\times10^{-4}2.0×10−42.0\times10^{-4}2.6×10−42.6\times10^{-4}

W2, third point. Backward-only ablation. Table 3 (Section 4.3) now keeps the reported forward map and replaces only its derivative, on all ten families and three seeds. On the nine families where RRA's gap is positive:

derivative of the interior mapgap relative to RRA
full (RRA)1
zero (stop-gradient)6.3 to 1,680
identity2.3 to 22
  • From the same weights, the three variants return identical outputs on the first training batch, so only the gradient differs.
  • On nonconvex QCQP$+$, where RRA's gap is −0.0058%-0.0058\%, the zero and identity derivatives raise it to 12.6% and 0.64%, and all 30 paired 95% intervals for the zero derivative lie above zero.

W2, fourth point. Cost and objective quality. Section 4.4 and the conclusion now qualify the practical advantages by the inference cost, which is specific to the two nonconvex SOC families.

  • On the other eight families RRA takes 0.2 to 3.0 times the inference time of FSNet and is faster on five of them (Section 4.4).
  • On nonconvex SOCP and SOCP$+$, RRA takes 64 to 75 times the time of FSNet on one A100 GPU, against the 114 to 127 times measured on the L40S of Table 17.
  • The radial exit has a lower objective than the Euclidean projection of the same proposal on convex QP, QCQP and SOCP, by 0.0029 to 0.23 points with every 95% interval below zero, and a higher one by 0.0028 to 0.0029 points on QP with n=200n{=}200 (Appendix G.7).
  • From the global anchor, the same ray ends 0.27 to 1.85 points above the projection, so the objective cost grows when the anchor does not follow the proposal.
AuthorsOpenReview ↗

Reply to Reviewer UAcV (2/2): questions (Q1 to Q5)

Q1. Finding and verifying the anchor. The solver tests every anchor row by row in double precision and reports an instance without a passing anchor as unsolved, with no output.

  • The interior map, started at the proposal, yields the anchor when every inequality and box row is at most −10−9-10^{-9} there.
  • Otherwise the solver uses the global anchor, computed once per instance by up to 300 Adam steps from p=0p=0 (16 polished starts on nonconvex SOC). It is accepted only when every row's relative slack exceeds 10−610^{-6} (Appendix E.1).
  • At the reported settings all 60,000 test instances have an anchor, and on the 578 of 2,000 instances of nonconvex QCQP and QCQP$+$ where the global search finds none, the map supplies it (Appendix H.1).

Q1. No strict interior, or little slack. Such a set lies outside the condition of Proposition 1, and RRA reports its instances as unsolved.

  • The margin μ\mu is the target of the map, not a requirement, since an anchor needs only slack above 10−910^{-9}.
  • When all rows are tightened until the largest slack is a tenth of its value, every returned output still passes on the tightened rows, and coverage falls most on the nonconvex families (Appendix I.4).

Q2. Guarantees on nonconvex SOC. Every returned point satisfies cx≤10−10c_x\le10^{-10}, which the solver checks row by row, and a new curvature bound proves the first crossing on every ray of the reported runs.

  • Section 3.3 and Appendix C.4 now state the bound cx≤10−10c_x\le10^{-10} of the sliding stages, where the submission stated cx≤0c_x\le0.
  • With the bound of Lemma J.1, the Taylor check of Lemma C.3 proves all 95,192 rays of positive length and 286,635 of the 286,645 sliding moves free of an earlier crossing. Along the other ten, a fine grid finds row values of at most 2.2×10−142.2\times10^{-14} (Appendix J.4).
  • Algorithm 2 runs this check inside the solver and flags a ray that it does not prove, whose output then keeps the row bound cx≤10−10c_x\le10^{-10}. With the same global anchors it flags no ray and returns outputs identical to those of the reported solver on all 12,000 test instances (Appendix J.1).

Q3. The basin. Section 3.1 now defines it as the connected component of the strictly feasible set {p:cx(p)<0}\{p:c_x(p)<0\} that contains the anchor.

  • The network selects this component through the anchor that the Gauss–Newton map returns from its proposal.
  • A convex set has a single component, and there the anchor stays close to the proposal because each step is a minimum-norm correction.
  • When the global anchor replaces the output of the map, the network no longer selects the component. At the reported settings this happens only on nonconvex SOC, on at most one instance per seed.

Q4. RRA and RAYEN. RAYEN maps along rays from one interior point of a set that does not depend on the input. RRA takes the anchor of each instance from a Gauss–Newton map started at the proposal and trains through this map.

  • Since our constraints change with xx, we solved RAYEN's interior-point program per instance and gave RAYEN the network, input, training and evaluator of RRA (Appendix G.6).
  • Every row is at most −3.6-3.6 at this interior point, so the ray from it reaches the proposal on at most 1.85% of the test instances. On the others RAYEN returns the boundary point in the direction of the network output, whatever its length.
  • Both methods pass the strict rule on every output, and RAYEN's mean gap (%) over three seeds is 8.9 to 54 times that of RRA:
familyRRARAYENRAYEN with proximity term
convex QP0.00210.1160.062
convex QCQP0.00590.2950.274
convex SOCP0.00550.2480.221
convex QP, n=200n{=}2000.00690.1130.061

Q5. Working in pp, and Figure 1. The solver writes every point as y=A+x+Npy=A^+x+Np, so every point it evaluates satisfies Ay=xAy=x up to rounding (Lemma C.1), and only the inequality and box rows cx(p)c_x(p) remain.

  • Figure 1(b) shows one Π\Pinet network on convex QP, trained in double precision with 50 projection iterations and run with 10 to 5,000 at test time. Its outputs fail below 200 iterations because the projection has not converged, and all pass from 200 on.
  • The submitted panel also marked a second network trained and run in single precision. That marker did not hold the network fixed, so the revised panel drops it.
  • Figure 1(c) shows one ENFORCE network on nonconvex QP, trained with ε=10−10\varepsilon=10^{-10} and depth 12 and run with ε\varepsilon from 10−210^{-2} to 10−1010^{-10}. Its outputs stop outside the set by up to ε\varepsilon and pass from ε=10−4\varepsilon=10^{-4} on.

We would be grateful if the reviewer could tell us which point, if any, remains open.

As the discussion period ends on November 18, we would be grateful to know whether our reply resolves the concerns on the exposition, the guarantee and the cost, or which point remains open. We are glad to provide further detail on any of them!

Reviewer_XLNgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Iterative correction method in prior works moves the infeasible neural network output towards the feasible set, which is sensitive to the hyperparameters of the stopping rule. Existing methods that guarantee exact feasibility are restricted to certain class of constraints. This paper proposes a neural solver that outputs a feasible solution to a general constrained optimization problem, where feasibility is enforced by gradually moving an strictly feasible point towards the boundary. . The proposed method RRA is tested on ten families of convex and nonconvex optimization problems across three random seeds. Its optimality gap is comparable to SOTA neural solver FSNet.

Critical strengths
  1. Thorough experiments. The proposed method is compared to multiple baselines across multiple convex and nonconvex optimization problems. Strict feasibility rate, optimality gap, variance of the optimality gap across random seeds. Ablations on the interior map and training objective are also carried out.
  2. The overall idea of the proposed method is simple yet effective.
Critical weaknesses
  1. The motivation why we need feasibility that does not depend on stopping rule is not well-explained. Even if feasibility depends on stopping rule, Being ϵ\epsilon outside the feasible set seems acceptable, particularly when the feasible set is nonconvex.
  2. The dependence on stopping rule (e.g. tolerance, iteration budget) has moved into anchor search instead. Whether Gauss-Newton has converged to a strictly feasible point is still determined by some tolerance, so what is its difference from using a stopping rule?
Questions for authors
  1. To explain the motivation better, could you construct an example where it is necessary for the neural solver output to be exactly feasible (if not, then something really bad could happen)?
  2. How is the global anchor a0a_0 computed? Are there any cases where the global anchor is not available?
  3. Are there cases where Gauss-Newton diverge or does not return a strictly feasible point? When the global anchor is also not a strictly feasible point, is RRA not applicable?
  4. The paper did not explain why the optimality gap of RRA is better than baselines (e.g. FSNet). The benefits of RRA come from enforcing feasibility, and I do not understand why it also improves the optimality gap.
  5. Why are only 20 iterations of Gauss-Newton needed?
5 comments
AuthorsOpenReview ↗

Reply to Reviewer XLNg (1/2): motivation and the depth of the Gauss–Newton map (W1, Q1, W2, Q5)

We thank the reviewer for the careful review, whose questions on the motivation helped us sharpen the claim and add the requested example (manuscript's changes are marked in blue).

W1. Why feasibility should not depend on a stopping rule. A correction returns its point in every case, and its residual is known only after the constraints are evaluated again. In the paper's own baselines this residual is not always small, least of all on nonconvex sets.

  • An RRA output is feasible by construction from an anchor that the solver checks, and an instance without such an anchor is reported as unsolved (Algorithm 1, Table 36).
  • In the reported runs, the strict rate of a correction changes with its stopping rule, its training seed and the family (Appendix F.2 for FSNet):
method and familywhat changesstrict rate
FSNet, convex QP10, 20, 50 test iterations0%, 89%, 100%
Π\Pinet, convex QCQPtraining seed100%, 95.8%, 99.5%
ENFORCE, nonconvex SOCPtraining seedat most 0.05%
  • On nonconvex SOCP, the median inequality violation of ENFORCE's outputs is 44, 1,100 and 30 on the three seeds.
  • Small residuals also accumulate. On convex QP, where FSNet passes the strict rule on every output, its outputs miss the equality rows by a median of 5.3×10−65.3\times10^{-6} to 7.1×10−67.1\times10^{-6}, and 30% to 48% of them lie outside an inequality row.
  • Summed over NN outputs, as with hourly dispatch against a yearly budget, such one-sided violations add up to NεN\varepsilon. The largest inequality value of RRA's outputs on convex QP is 2.5×10−142.5\times10^{-14}.
  • We agree that a point ε\varepsilon outside the set is acceptable for many uses, and a correction stopped below the evaluator's tolerance passes it, as ENFORCE does on its five families run with ε=10−10\varepsilon=10^{-10}.

Q1. An example where the output must be feasible. An incumbent in branch-and-bound must be feasible, because only the objective of a feasible point bounds the optimal value from above. This example is now Appendix I.5.

The Π\Pinet network of Figure 1(b), trained with 50 projection iterations and run with 10, 25 and 50 at test time, returns infeasible outputs below the global optimum:

test iterationsinstances below the optimummedian amount belowmedian inequality violation
10100%5.65%2.1
25100%1.3%0.58
5096.8%0.11%0.092
  • Used as incumbents, these points would prune the node that contains the optimum.
  • At 100 iterations no output lies below the optimum, yet none passes the strict rule, so the objective alone does not reveal the infeasibility.

W2. The stopping rule inside the anchor search. The map does stop after a fixed number of steps, but its output serves only as an anchor, which the solver tests before use and never returns.

  • In 840 test-time runs that change the depth from no map to 40 steps, the acceptance threshold and the other settings, all 1,665,509 returned outputs pass the strict rule (Appendix I.1).
  • The map drives the selected rows to the margin −μ=−10−4-\mu=-10^{-4}, so a map stopped with a remaining error below μ\mu still yields a strictly feasible anchor (Section 3.1).
  • The reviewer is right that a weak map can leave an instance without an anchor. On nonconvex QCQP the depth then moves the strict rate as a stopping budget does, and the text after Proposition 1 now says so. With unsolved instances counted as failing (lowest of three seeds, 2,000 instances each):
depth of the mapunsolved instancesstrict rate
none or one stepup to 57871.1%
two stepsup to 37881.1%
five stepsat most 3798.15%
20 steps (reported)0100%
  • The solver reports each unsolved instance, whereas Π\Pinet and ENFORCE return their failing outputs with no flag (Figures 1(b) and 1(c)).
  • A correction that checked its outputs in the same way would keep only the share given by its strict rate. At the reported settings this share is at most 0.05% for ENFORCE on the two nonconvex SOC families and 87.8% for FSNet on convex QP with n=200n{=}200, against 100% for RRA (Table 1).

Q5. Why 20 steps. Feasibility does not require the map to converge, so we chose the depth for accuracy, where the gap stopped improving on seed 2025.

  • The residual falls at the quadratic rate of a Newton iteration (Ben-Israel, 1966), on nonconvex QP from a median of 0.42 to 1.0×10−31.0\times10^{-3}, 3.3×10−93.3\times10^{-9} and 1.1×10−151.1\times10^{-15} after one, two and three steps (Appendix H.2).
  • On nonconvex QP the gap falls from 13.1% without the map to 0.418% at depth 10 and 0.128% at depth 20, and depth 40 changes nothing on the families that use 20 steps (Appendix I.1).
AuthorsOpenReview ↗

Reply to Reviewer XLNg (2/2): the global anchor, failures of the map and the accuracy (Q2 to Q4)

Q2. The global anchor. A feasibility search computes it once per instance from the constraints alone. It is unavailable on 578 of the 2,000 test instances of nonconvex QCQP and of QCQP$+$, where the interior map supplies the anchor.

  • On eight families Adam starts at p=0p=0, the minimum-norm point that satisfies the equality constraints, and reduces the shortfall of the normalized slack for at most 300 steps.
  • Convex SOCP adds a 10-step Gauss–Newton polish, and the two nonconvex SOC families use 16 polished starts and keep the four anchors with the largest slack.
  • A result is accepted only when every row's slack, relative to its scale, exceeds 10−610^{-6} (Appendix E.1).
  • On the 578 instances of nonconvex QCQP without a global anchor, the outputs pass with a mean gap of 0.29%, below the 0.40% of the other instances (Appendices G.4 and H.1).

Q3. When the map fails. RRA then uses the global anchor, and when that fails too, it reports the instance as unsolved and returns no output (Algorithm 1).

  • At the reported settings no test instance is unsolved, and on the eight families without nonconvex cone rows the output of the map passed its test on all 48,000 test instances.
  • On the two nonconvex SOC families the map fails in 0.4% to 9.4% of its runs at the first ray step, each time because more than w=16w=16 rows are violated or near the boundary. The returned output still starts from the map on 1,999 or 2,000 of 2,000 instances per seed (Appendix H.1).
  • With untrained, random or amplified proposals in place of the network's, all 139,355 distinct outputs pass the strict rule at gaps of 37% to 913%, and the 645 instances without an anchor are reported as unsolved (Appendix I.3).

Q4. Why RRA's gap is lower than FSNet's. RRA obtains feasibility and accuracy from different operators. The retraction makes the output feasible, and the network, trained through the interior map, decides where the output lands.

  • RRA's gap is lower than FSNet's on all ten families, by factors of 1.1 to 133 where both gaps are positive, and 28 of the 30 paired intervals lie below zero (Table 2).
  • Each change below raises the gap of RRA (Table 3, Table 19, Appendix G.6):
change to RRAfamiliesgap relative to RRA
zero derivative of the mapnine6.3 to 1,680
identity derivative of the mapnine2.3 to 22
global anchor with 16 sliding stagesconvex QP, QCQP4.5, 5.9
RAYEN's interior point and lossfour convex16 to 54
  • The proximity term trains the proposal toward the output, so the network places the output directly instead of steering it through the direction of the ray. Without this term the gap grows 6.3 to 2,490 times on seven families (Table 24).
  • Projecting the raw output of FSNet's own networks exactly onto the set removes 12% to 50% of FSNet's gap and leaves 6.2 to 87 times the gap of RRA (Appendix G.7). At least half of FSNet's gap therefore comes from where its network places the prediction.

The example, the depth sweep, the stress test and the decomposition of FSNet's gap that these questions prompted are now Appendices I.5, I.1, I.3 and G.7. We hope that these additions address the reviewer's concerns, and we would be glad to continue the discussion on any point.

Thank you again for your review! As the discussion closes on November 18, we would like to check whether our reply answers your questions on the motivation and on the tolerance of the Gauss–Newton map. We would be glad to clarify anything that remains open!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision for all reviewers and the Area Chair

We thank the reviewers and the Area Chair for their time. This comment summarizes what the revision adds and where to find it. Changed text is marked in blue.

Questions shared by the reviews

  • Do the settings of RRA decide feasibility, as a stopping tolerance does? (XLNg W2, UAcV W2, PzYB Q3) No returned output fails the strict rule in 840 test-time runs that change one setting at a time (Appendix I.1). The settings do decide whether an anchor exists. On nonconvex QCQP and QCQP$+$ a weakened map leaves up to 578 of 2,000 instances without one, and the solver reports them instead of returning a point.
  • What does the derivative of the interior map contribute? (UAcV W2, PzYB W3 and Q2) With the forward map fixed, a zero derivative raises the gap 6.3 to 1,680 times and an identity derivative 2.3 to 22 times on the nine families with a positive gap (Table 3).
  • Which guarantee covers the nonconvex SOC families, and what do they cost? (UAcV W2 and Q2, PzYB W1 and W2) Every output satisfies cx≤10−10c_x\le10^{-10}, and a new curvature bound proves the first crossing on all 95,192 rays of positive length (Appendix J.4). RRA takes 0.2 to 3.0 times the inference time of FSNet on eight families, and 64 to 75 times on these two on one A100 GPU (114 to 127 on the L40S of Table 17), as Section 4.4 and the conclusion now state.

Presentation

The changes to the presentation are in five places of the main text.

point raised in the reviewschangewhere
problem and existing approaches before the resultsthe introduction states the problem, the existing approaches and the gap they leave, and then RRApages 1 and 2
strict rule used before its definitiondefined at its first use, with its tolerances in the abstractpage 2
definitions and implementation mixedSection 3.1 states when an anchor is accepted, when an instance is unsolved and what the basin ispages 3 and 4
abrupt nonconvex SOC extensionits own Section 3.3page 5
contribution list against RAYENcontribution 2 states what RRA addspage 2

Other new results

questionresultwhere
When must an output be feasible? (XLNg Q1)Unconverged Π\Pinet outputs lie below the global optimum on 96.8% to 100% of instances, so as incumbents in branch-and-bound they would prune the optimum.Appendix I.5
How large is the residual of a correction? (XLNg W1)The median inequality violation of ENFORCE on nonconvex SOCP ranges from 30 to 1,100 over the seeds.Appendix I.2
How often is the global anchor used? (XLNg Q2, UAcV Q1, PzYB Q1)At the reported settings, on at most one test instance per seed of any family.Appendix H.1
How does RRA compare with RAYEN? (UAcV Q4)With the network of RRA, RAYEN has 8.9 to 54 times the gap of RRA on four convex families.Appendix G.6
Where does the gap of FSNet come from? (XLNg Q4)Exact projection of FSNet's raw output removes 12% to 50% of its gap and leaves 6.2 to 87 times the gap of RRA.Appendix G.7

Corrections to the submitted text

The revision also corrects the statements below. None of them changes the method, the code behind the reported results or the numbers in Tables 1 and 2.

  • The nonconvex SOC outputs satisfy cx≤10−10c_x\le10^{-10}, where the submission stated cx≤0c_x\le0 (Section 3.3).
  • On nonconvex SOC the second ray step starts from the output of the interior map or from a point backed off from the first output, never from that output itself (Appendix E.1).
  • ENFORCE ran with ε=10−6\varepsilon=10^{-6} and depth 8 on convex SOCP, nonconvex QCQP$+$ and the two nonconvex SOC families, not with ε=10−10\varepsilon=10^{-10} and depth 12 (Appendix E.4). With 18 further runs, it passes the strict rule on at most 5.0% of the outputs in 23 of 24 training runs on nonconvex SOC, and the remaining run passes on every output at a gap of 34.6%, against 11.8% for RRA.
  • The one-start coverage of submitted Table 21 and the 13% of submitted Section 3.2 came from the best of eight starts (Appendix G.4).
  • Figure 1(b) now shows only the network whose test iterations vary, which was trained for 30 epochs apart from the seeds of Table 2 (Appendix F.2).
  • Stage counts include the retraction as the first sliding stage, and no number changes (Appendix C.6).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.