acceptodds
Under review as a conference paper at ICLR 2027

Only Project Once: Projection-Adaptive Loss for Exact Constraint Satisfaction

Abstract

Precise constraint satisfaction is a prerequisite to deploying learned models in many areas, motivating methods that repair raw neural predictions with a projection procedure. Current repair-based methods unroll multiple repair steps in training and softly penalize constraint violations that remain after the unroll. This is compute- and memory-intensive, lacks robustness when the repair fails to converge, and surrenders most of the constraint satisfaction work to the repair. Our central finding is that, contrary to common practice, a single detached projection step suffices in training. We accomplish this with a Projection-Adaptive Loss (PAL), which uses the constraint residual after this single step to adaptively weigh constraint penalties on the raw prediction. PAL is the only method that retains full feasibility on extremely nonlinear constraints, and matches or outperforms current methods on synthetic and engineering benchmarks. Because it only requires a single detached projection step, PAL trains faster than the canonical repair-based method DC3 on its own AC optimal power flow benchmark. PAL can also be trained when constraints are expensive to evaluate (e.g., via neural surrogates), a setting where current unrolled methods are memory-intractable.

Primary area:
optimization
Keywords:
constrained optimization, hard constraint satisfaction, amortized optimization, projection, differentiable simulation, learning with constraints, engineering design, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Dns1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to accelerate the repair procedure of the constraint satisfaction method. Previously, the SOTA methods iterated between the constraint projector and the evaluator, using soft penalties to design the training loss. This paper identifies three drawbacks of prior works: 1) computational overheads induced by iterative repairs, 2) hard constraint satisfaction not guaranteed by soft penalties, and 3) insufficient awareness of constraint satisfaction during training. Accordingly, the paper addresses them by using single-step repair in training and proposing a projection-adaptive loss to enhance feasibility awareness. The proposed method is evaluated on synthetic and engineering benchmarks, achieving acceleration and higher feasibility in nonlinear problems.

Critical strengths
  1. This work is well-motivated and easy to follow.
  2. The effectiveness of the proposed method is validated in extensive benchmarks.
  3. The code is provided, facilitating future work.
Critical weaknesses
  1. The overall idea is quite straightforward with limited novelty.
  2. Why can the projection-adaptive loss enhance feasibility awareness? I would expect a more in-depth understanding or justification.
  3. The tables are hard to follow. For example, Table 1 uses too many colors without explanation.
  4. The evaluation metrics are not sufficiently justified. Why can forward and backward passes represent computational overhead? The efficiency of these processes depends on network size and architecture, which differ across methods. Why use different metrics- memory and time- in different benchmarks?
  5. It would be better to provide an anytime comparison for different baselines.
Questions for authors

See Weaknesses

2 comments
AuthorsOpenReview ↗

Reply to Reviewer Dns1

Novelty

The overall idea is quite straightforward with limited novelty.

We disagree.

  1. Despite the idea's elegance it overturns a core assumption held by all previous repair papers that we are aware of. (Namely, the prevalent consensus that one must backpropagate through the full test-time repair mechanism to get optimal results that is held by all previous repair papers we are aware of: SnareNet, FSNet, DC3, ENFORCE, Hardnet, ....)
  2. Impact: The insights provided by the paper enable an entirely new class of applications. Repair-aware training can now be applied to problems with expensive constraint evaluations. This allows for online training on surrogates, simulators, geometry checking, etc.

In-depth Understanding and Justification

Why can the projection-adaptive loss enhance feasibility awareness? I would expect a more in-depth understanding or justification.

The paper answers this in several places:

  • Figure 2 (lines 261-271 in the original submission) provides an illustrative explanation of the improved load sharing between net and repair.
  • Ablation R (summarized in section 5.4, lines 415-417 and 432-434, details in appendix B, lines 701-705) provides extra evidence for the load share argument.
  • The underlying reason for the improved robustness under high nonlinearity is discussed in section 5.3 (lines 373-377), illustrated in figure 3c, and expanded on in more detail in appendix C (lines 737-751).
  • Summary in the introduction (lines 42-48 and 54-58) and conclusion (lines 457-459).

If something remains unclear, we kindly ask the reviewer to point to it more specifically, and we will gladly clarify!

Table readability

The tables are hard to follow. For example, Table 1 uses too many colors without explanation.

We have added a colorbar to Table 1. If you find any other table is hard to read, please let us know more specifically and we will gladly consider further improvements.

Evaluation Metrics

Why can forward and backward passes represent computational overhead?

We focus on applications where constraint evaluation is expensive. Measuring forward and backward passes on the constraint evaluation is the best non-polluting method for measuring compute. It is not dependent on controlling implementation details across 7-8 methods. Given same net size, this overhead is constant across all methods.

The efficiency of these processes depends on network size and architecture, which differ across methods.

We would like to clarify that all methods use the same network size and architecture on each benchmark (Appendix N.2).

Why use different metrics- memory and time- in different benchmarks?

In the engineering benchmarks, we focus on practical implications and highlight how, on different problem archetypes, PAL helps with core challenges. For Urban and UAV this is memory due to the very expensive evaluator. On the ACOPF benchmark, it is the extremely high number of constraints resulting in costly Jacobian builds. Peak memory and wall-clock time measure these challenges directly.

### Anytime comparison

We have added an anytime comparison in the new Appendix Q since it also helps answer some questions by other reviewers. This is not a new experiment, just a more detailed visualization. Thank you for this suggestion!

Further Points

We would be glad to hear whether these clarifications and improvements change your view of the paper. If there is anything else we could do to support a higher score in your view, please let us know!

AuthorsOpenReview ↗

Follow-up on our Rebuttal

Dear Reviewer Dns1,

Thank you again for your feedback and questions.

As the discussion period nears its end, we'd appreciate any final thoughts you may have on our responses, if time permits. We'd also be happy to clarify further.

If you feel that our responses have addressed your concerns, we would appreciate it if you could take them into account when finalizing your evaluation.

Reviewer_6ycgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Projection-Adaptive Loss (PAL) for repair-aware amortized constrained optimization. During training, PAL performs one detached repair step and uses its residuals and displacement to adapt penalty weights on the raw neural prediction. Training optimizes the pre-repair objective, raw constraint violations, and distance to the detached repaired point. At inference, the method performs multiple repair steps, typically up to ten with early stopping. The default repair operator is damped Levenberg–Marquardt, with additional SQP and interior-point variants. Evaluation includes synthetic benchmarks, engineering applications, high-curvature constraints, and component ablations. Appendix F.1 invokes an existing local convergence result for an idealized LM iteration.

Critical strengths

The method addresses a meaningful computational limitation of repair-aware training. Using a single detached repair step avoids differentiating through a long repair sequence. The reported memory results on expensive neural evaluators and the ACOPF training-time comparison provide useful evidence that this approach can make otherwise costly training more practical.

The experimental evaluation covers several relevant failure modes. The synthetic suite, high-curvature experiment, alternative repair operators, and inference-precision study provide evidence that single-step training need not restrict the accuracy achievable through subsequent multi-step repair. The paper also explicitly distinguishes its heuristic repairability proxy from a certified local convergence condition.

Critical weaknesses

The distinctive methodological contribution is not sufficiently isolated. PAL introduces a new feedback-based loss and weight update, but retains existing repair operators and combines familiar ingredients: constraint penalties, detached repair targets, and adaptive weighting. The key question is whether post-one-step residual feedback provides a substantial advantage beyond these simpler ingredients. Table 4 shows that disabling either the constraint or displacement penalty preserves similar performance on the synthetic suite, while disabling both still permits approximately 99% feasibility through the projector alone. This does not establish which proposed mechanism is essential. Comparisons against methods using different repair operators also confound the training-rule contribution with projector robustness. More controlled comparisons are needed to distinguish PAL from a fixed-weight detached-target method and an adaptive penalty method driven by raw residuals, using the same projector and matched inference settings.

The theoretical analysis does not establish guarantees for the proposed training procedure or the deployed pipeline. Appendix F.1 invokes an existing local quadratic-convergence theorem under a fixed active set, a smooth residual map with Lipschitz Jacobian, a local error bound, and sufficiently close initialization. These are substantive applicability conditions, not properties established for PAL’s trained predictions. The result does not show that the adaptive updates converge, that training places predictions inside the local convergence region, or that the finite-budget practical projector achieves feasibility. The authors explicitly acknowledge that the theorem does not automatically cover active-mask switching or a numerical damping floor. There is also no guarantee for post-training optimality. The detached training loss optimizes the raw objective and attraction to a one-step target, whereas evaluation uses the objective after multi-step repair; Appendix E provides a heuristic explanation and limited empirical evidence for this choice, rather than an analysis of the full mismatch. Lack of formal guarantees is not inherently disqualifying, but the existing projector theorem does not resolve the central questions about PAL’s distinctive mechanism or its “exact constraint satisfaction” framing.

Questions for authors

Assessment-changing mechanism comparison: With the same projector, damping, inference tolerance, and repair budget, how does PAL compare against (i) fixed-weight raw constraint and detached displacement penalties and (ii) adaptive constraint penalties driven by pre-repair residuals? Can these comparisons isolate the benefit of post-one-step residual feedback?

Theory clarification: Beyond the cited local LM theorem, is there any result connecting Eq. 2 to convergence of training or entry of the trained predictions into the projector’s local convergence region? If not, please distinguish the conditional projector guarantee from the empirical claims about PAL.

Training–inference alignment: Does controlling the first repair displacement also control the total displacement and objective change after multi-step inference repair? Please report one-step versus final repaired objectives and cumulative repair displacement, particularly on the high-curvature and engineering benchmarks.

Feasibility clarification: For the engineering benchmarks, particularly E3 where Table 2 reports violation magnitudes rather than feasibility percentages, what fraction of outputs satisfies all constraints at explicit tolerances? How often does the inference repair budget expire before feasibility is reached?

4 comments
AuthorsOpenReview ↗

Mechanism Ablation

Assessment-changing mechanism comparison: With the same projector, damping, inference tolerance, and repair budget, how does PAL compare against (i) fixed-weight raw constraint and detached displacement penalties and (ii) adaptive constraint penalties driven by pre-repair residuals? Can these comparisons isolate the benefit of post-one-step residual feedback?

We have now added this comparison as a new ablation. You can find this in Table Z, Section 5.4 in the revised version. It compares 0-step (raw residual training, i.e., classical penalty methods) with 1-step detached repair and 10-step unrolled repair with backprop (similar to all the baselines) in both adaptive and fixed-weight variants. All methods use the same LM projector and inference protocol. Here is an excerpt from this Table:

Training variantFeas. at High Curvature (%)Obj. Gap S1-S6Memory E2
PAL (1 detached step, adaptive)98.30.0318.7 GB
Fixed w=0.01w=0.0176.10.0118.7 GB
Fixed w=0.1w=0.177.40.0018.7 GB
Fixed w=1w=189.80.0618.7 GB
Fixed w=10w=1099.92.1018.7 GB
Fixed w=100w=10099.93.3418.7 GB
ALM+Bolt-On (adaptive, pre-repair)75.919.4717.2 GB
10-step unroll (adaptive)46.00.03OOM
  • (i) Of the fixed-weight methods, no choice of weights simultaneously reaches PAL's feasibility at high curvature and low objective on the synthetic benchmarks.
  • (ii) Adaptive penalty on pre-repair residuals underperforms. This is the ALM+Bolt-On baseline that has previously been included in other experiments.
  • Fixed-weight 1-step detached repair corresponds to training on pre-projection residuals (classical penalty method) since without PAL's adaptive weight update, the repair step's residuals effectively do not inform the training loss in any way.
  • We also include unrolled training with backpropagation through 10 steps. Like other unrolled methods, it falls short on feasibility at high curvature and runs out of memory on E2.
  • For fairness, the fixed weights are not tuned per problem just like all other methods, which share hyperparameters across the full problem suite. We sweep weights w∈0.01,0.1,1,10,100w \in \\{0.01, 0.1, 1, 10, 100\\}.
  • In the paper, we also relate the result to Appendix P (theoretical bounds, see comment right below) and talk about the trade-off between small weights (well-conditioned optimization problem) and large weights (guaranteed projector convergence) and how an adaptive method can help resolve this trade-off better than fixed weights.

We think that adding this ablation really helps isolate the benefits of our method more explicitly and would like to thank you for this suggestion.

If this and the other replies below resolve your concerns, we would greatly appreciate a reassessment of your score.

AuthorsOpenReview ↗

Theoretical Bounds and Update Rule

We would like to separate two questions here. First, under which conditions does the projector converge? Second, how can these conditions be created, i.e., how can projector convergence be guaranteed at the system level? As you point out, Appendix F covers almost only the first question, and only for our default projector. For the second question, we think a complete convergence proof that also holds in practice is near-impossible. Such proofs about training convergence rely on assumptions that rarely hold in practice or cannot be diagnosed. The literature in this field reflects this. Out of all the baselines, there is only a single one (FSNet) that attempts to provide guarantees beyond the projector itself, and we feel that the assumptions there are also somewhat disconnected from real-world scenarios (e.g., a network that outputs the global minimizer of the penalized problem). This is why we were somewhat skeptical towards these types of guarantees.

Nevertheless, we have now added a new Appendix P with notes on conditions for projector convergence at the system level (problem, projector, network, and loss). The approach is, roughly, as follows:

  • We assume a generic projector P\mathcal{P} that converges at least linearly near the feasible set.
  • We assume the backbone NθN_\theta is an imperfect but global optimizer of the training loss.
  • The loss is f(y)+∑kwc,k⋅ck(y)+wd∣Δy∣f(y) + \sum_k w_{c,k} \cdot c_k(y) + w_d \\|\Delta y\\| (equivalent to Eq. 1 in the paper), with weights at steady state in late training.
  • The main part of the appendix derives two sufficient conditions (lower bounds) on wc,kw_{c,k} that guarantee projector convergence.
  • We also derive an upper bound for ∣f(y^)−f(y~(N))∣|f(\hat y) - f(\tilde y^{(N)})|, which, for a projector of second-order convergence, can be reduced primarily via wdw_d.
  • We then discuss how the lower bounds for wc,kw_{c,k} and wdw_d are not derivable in practice, because the constants involved (e.g., the network suboptimality ξ\xi) are not directly measurable. The weights also cannot be set arbitrarily large, especially in early training, because this makes the training problem ill-conditioned. This is standard optimization theory, and we also supply empirical evidence in our response to your first question (Table Z in the revised version).
  • The "bridge" to the actual weight update rule (Eq. 2) does not follow from the proof. Eq. 2 is a heuristic method to find wc,kw_{c,k} and wdw_d above these bounds, based on quantities we can easily measure (R1,kR_{1,k} and ∣Δy(1)∣\\|\Delta y^{(1)}\\|). We want to mention here that our method is not the only possible solution, and the appendix also discusses other approaches one could explore in the future to find appropriate weights wc,kw_{c,k} and wdw_d.
  • We discuss how the projectors we use in the paper (LM, SQP, IP) relate to our idealized projector. LM and SQP converge locally at least linearly, and usually faster, which is all the idealized projector assumes.
  • We also discuss the assumptions behind our derivations and why they usually do not hold in practice.

About the notes on the projector in Appendix F, we agree that the guarantees for LM are limited. However, this does not mean that the projector fails in cases the guarantees do not cover. For example, our experiments show that LM performs roughly as well as SQP and IP. Additionally, since PAL works with other projectors, one can use any other Gauss-Newton-style projector with specialized guarantees, especially if a specific projector type can exploit known problem structure.

AuthorsOpenReview ↗

Other Questions

Training-Inference Alignment

Does controlling the first repair displacement also control the total displacement and objective change after multi-step inference repair?

Yes, the displacement in the first repair step makes up the majority of overall displacement and barely changes the objective.

We have added detailed visualization of test-time metrics over repair steps in Appendix Q. Figure 13c shows cumulative displacement. The first step's share of overall displacement is annotated. For almost all repair methods, the first repair step makes up virtually all of the cumulative displacement, which is how Newton-style projections perform when the constraint's local linearization holds. After this first step, subsequent steps have very little displacement or objective change. (This is PAL's underlying idea). The only exceptions are FSNet (not projection-based, gradient descent tends to take many similarly-sized steps) and SnareNet (due to high damping which leads to conservative step sizes).

Specifically, for PAL the ratio of overall displacement to first-step displacement is between 1.0x (on most benchmarks), 1.15x for the highly nonlinear problems, and 1.37x for the highly coupled, non-linear E3. E2 exhibits a large ratio but this is a spurious artefact due to very low overall displacement.

Displacements are also small overall so that objectives barely change during repair as shown in Figure 13d. The largest relative change can be observed in S5, but this is at a numerically insignificant level (<1e-3).

Note that non-repair aware training (pre-projection penalty) sees significant degradation of objective.

In Appendix P, we have now also added an upper bound for f(y~)f(\tilde y) vs f(y^)f(\hat y) in Eq. 12 (Appendix P.5).

Feasibility on the Engineering Benchmarks

Feasibility is provided below.

Step BudgetFeas. at Budget †^\daggerFeas. at step 50
E3, PAL1084.2 ±\pm 6.1%88.8 ±\pm 5.6%
E3, DC3570.8 ±\pm 4.7%71.1 ±\pm 4.6%
E3, ALM + Bolt-On100.0%0.0%
E3, FSNet500.0%0.0%
E3, ALM00.0%--
E3, IPOPT (10 starts)--84.4 ±\pm 5.3%--

†^\dagger corresponds to the constraint magnitudes provided in Table 2.

On some queries in the test set, it seems that no feasible point (within our tolerance) exists, which we tested with elastic IPOPT at 10 starts per query. Based on these samples, we estimate that no method can exceed 92.3% feasibility on this benchmark. We discuss this in detail in the new Appendix R.

Why did we not provide feasibility in Table 2? One, to match the reporting of DC3's original publication. Two, because DC3 uses plain gradient descent to correct inequality constraints (linear convergence). At small numerical values near the tolerances (10−410^{-4}) this is extremely slow and unfairly disadvantages DC3.

AuthorsOpenReview ↗

Follow-up on our Rebuttal

Dear Reviewer 6ycg,

Thank you again for your feedback and questions.

As the discussion period nears its end, we'd appreciate any final thoughts you may have on our responses, if time permits. We'd also be happy to clarify further.

If you feel that our responses have addressed your concerns, we would appreciate it if you could take them into account when finalizing your evaluation.

Reviewer_pLimOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Projection-Adaptive Loss (PAL) for training neural optimization proxies with hard constraints. A single detached repair step provides residual and displacement signals that adapt penalties on the raw prediction. Training avoids differentiation through an unrolled repair sequence, while inference can use multiple repair steps. Experiments on synthetic and engineering problems show competitive solution quality and substantial memory savings, particularly with expensive differentiable evaluators.

Critical strengths
  1. The design is simple and useful. Using repair diagnostics to guide training connects the penalties to the difficulty of correcting a prediction. Detaching the repair also allows flexibility in the repair operator, as demonstrated with LM, SQP, and interior-point variants.

  2. The memory savings enable demanding applications. The aircraft and urban-planning examples show that PAL can train with expensive neural evaluators under memory limits that prevent most evaluated unrolled baselines from training. On the ACOPF variant, PAL also improves objective quality and maximum violation over DC3 while reducing training time from 9h52m to 3h50m.

  3. The empirical analysis is thorough. The synthetic experiments use ten reporting seeds separate from tuning seeds. The paper examines loss placement, weight updates, repair operators, inference precision, and high curvature, and reports baseline failures and limitations openly.

Critical weaknesses
  1. The benefit of using post-repair residuals needs a more direct comparison. The experiments support the overall training scheme, but most baselines also differ in their repair or loss formulation. Table 4 shows that the displacement-only variant performs similarly on the synthetic suite, a redundancy the paper acknowledges. A matched comparison would clarify what the post-repair signal adds over simpler penalty updates.

  2. Some baseline failures support narrower conclusions than the paper draws. In ACOPF, FSNet uses generic L-BFGS rather than the LM variant described in its original paper; Appendix L explains that the LM variant is absent from the public code. SnareNet also uses a reduced repair budget. These are understandable practical choices, but failures under these configurations do not establish a general limitation of the competing approaches.

  3. Aggregate feasibility does not establish pointwise feasibility. The aircraft strain and urban wind constraints average positive local violations. At a nonzero tolerance, a small average can still hide a larger violation at one station or cell. The reported feasibility rates therefore do not by themselves establish satisfaction of the stated pointwise limits.

  4. The remaining repair failures are not well characterized. PAL reaches 98% feasibility on urban planning and remains below 100% at the highest curvature. Appendix G shows excellent numerical precision on the six synthetic tasks, but does not explain whether these harder failures reflect an insufficient repair budget or stalled convergence. This limits the scope of the broad precision claims, even with the qualifications in Appendix F.1.

Questions for authors
  1. With the projector, detached gradient path, displacement term, and objective placement held fixed, how does PAL's post-repair weight update compare with tuned fixed weights or an update based on pre-repair residuals?
  2. How much does the ACOPF comparison depend on FSNet's repair variant? If an LM comparison is unavailable, could you limit the interpretation of this failure to the evaluated L-BFGS configuration?
  3. Among designs classified as feasible, what are the maximum station-wise strain excess and cell-wise wind-speed excess in their original units?
  4. Do additional inference steps resolve the urban-planning and high-curvature failures, or do their residuals plateau?
4 comments
AuthorsOpenReview ↗

1. Mechanism Ablation

With the projector, detached gradient path, displacement term, and objective placement held fixed, how does PAL's post-repair weight update compare with tuned fixed weights or an update based on pre-repair residuals?

We have now added this comparison as a new ablation. You can find this in Table Z, Section 5.4 in the revised version. It compares 0-step (raw residual training, i.e., classical penalty methods) with 1-step detached repair and 10-step unrolled repair with backprop (similar to all the baselines) in both adaptive and fixed-weight variants. All methods use the same LM projector and inference protocol. Excerpt from Table Z:

Training variantFeas. at High CurvatureObj. Gap S1-S6Memory on E2
PAL (1 detached step, adaptive)98.30.0318.7 GB
Fixed w=0.01w=0.0176.10.0118.7 GB
Fixed w=0.1w=0.177.40.0018.7 GB
Fixed w=1w=189.80.0618.7 GB
Fixed w=10w=1099.92.1018.7 GB
Fixed w=100w=10099.93.3418.7 GB
ALM+Bolt-On (adaptive, pre-repair)75.919.4717.2 GB
10-step unroll (adaptive)46.00.03OOM

Main observations:

  • Fixed-weight 1-step detached repair corresponds to training on pre-projection residuals (classical penalty method) since without PAL's adaptive weight update, the repair step's residuals do not inform the training loss for detached repair steps.
  • Of the fixed-weight methods, no choice of weights simultaneously reaches PAL's feasibility at high curvature and low objective on the synthetic benchmarks.
  • Fixed weights are not tuned per problem. We sweep one suite-wide w∈0.01,0.1,1,10,100w \in \\{0.01, 0.1, 1, 10, 100\\} for all constraint and displacement terms.
  • The adaptive penalty variant on pre-repair residuals underperforms. This is the ALM+Bolt-On baseline that has previously been included in other experiments.
  • We also include unrolled training with backpropagation through 10 steps. Like other unrolled methods, it falls short on feasibility at high curvature and runs out of memory on E2.
  • In the paper, we also relate the result to Appendix P (theoretical bounds, see comment right below) and talk about the trade-off between small weights (well-conditioned optimization problem) and large weights (guaranteed projector convergence) and how an adaptive method can help resolve this trade-off better than fixed weights.

We would like to thank the reviewer for this suggestion. We feel that adding this ablation really helps isolate the benefits of our method.

AuthorsOpenReview ↗

Engineering Benchmarks

2. Additional FSNet-LM Baseline for ACOPF Benchmark

We have added FSNet-LM and it performs much better than vanilla FSNet. PAL still trains much faster than both FSNet-LM and DC3 (3h50m vs. 22h06m and 9h52m).

We did not include FSNet's LM version because FSNet's selling point, in our interpretation, is Jacobian-free feasibility seeking. Their ACOPF experiment replaces it with Jacobian-based LM projection, which undermines this selling point. Additionally, we did not use it because it is not in the public code. (We contacted the authors about this to no avail). We agree the LM variant is a relevant comparison, so we reimplemented it as best as we could based on the details they provide throughout the paper and especially in Appendix B.6.

FSNet-LM reaches somewhat higher feasibility than PAL (89.8% vs. 84.2%), but also has 3× higher constraint violations. We therefore have changed the claim about PAL being the best-performing method on this benchmark and now write that it is "competitive in feasibility and objective" rather than best, and now only highlight its compute advantage (at least 2.5× faster training than DC3 and FSNet-LM). We updated the text and table accordingly, and added a new figure in Appendix L (Figure 12) where we show the constraint violation distributions for FSNet-LM, PAL and DC3 since this provides richer information than feasibility and maximum constraint violation (which give conflicting information about which is better).

Lastly, we have also added details in the new Appendix R where we discuss the underlying reason no method reaches full feasibility on this benchmark.

3. Constraint Formulation in Benchmarks

Aggregate feasibility does not establish pointwise feasibility. The aircraft strain and urban wind constraints average positive local violations.

Just like with any other optimization algorithm and all the baselines, PAL uses constraints c(y) as given, so correct constraint formulation is always user-side responsibility. How a constraint is formulated is only a property of the problem the method is applied to.

Among designs classified as feasible, what are the maximum station-wise strain excess and cell-wise wind-speed excess in their original units?

We have analyzed the repair trajectories at test time. For all 10 seeds on both E1 and E2, all designs reported feasible are individually feasible. In fact, both these constraints are slack with these margins in native units:

PALALMALM + Bolt-On
E14.0e-41.8e-41.8e-4
E20.12 m/s0.13 m/s0.13 m/s

However, we agree that, even though no issue occurs here in practice, in applications where upper bound guarantees are needed one should ideally use a maximum operator over samples or multiple per-station/pixel constraints. Additionally, how a constraint is sampled (e.g. resolution of stations / pixels) is critical.

AuthorsOpenReview ↗

4. Remaining Repair Failures for All Methods

Do additional inference steps resolve the urban-planning and high-curvature failures, or do their residuals plateau?

Additional inference steps improve urban planning (E2) to 98.9% (at 50 steps) (albeit with slow convergence), while the residuals in high-curvature κ=106\kappa = 10^6 plateau for PAL and all baselines.

Feas. step 10 (paper)Feas. step 50Med. c(y~(10))c(\tilde y^{(10)})Med. c(y~(50))c(\tilde y^{(50)})
E2, PAL97.8%98.9%3.6e-41.3e-4
E2, ALM + Bolt-On69.1%83.6%9.1e-41.9e-4
κ=106\kappa=10^6, PAL98.3%98.4%0.150.12
κ=106\kappa=10^6, DC392.2%92.2%0.220.22

Detailed Reasoning for why this happens:

  • E2: Additional steps reduce the mean violation and increase feasibility, but convergence is slow on some of these samples. After 50 repair steps, the remaining failures are limited by the site-boundary constraint (see Appendix K.2), which is a mean of squared softplus terms. Its Jacobian vanishes as c(y) approaches zero, so the repair converges only linearly near the solution, at about 1.3% residual reduction per step. Extrapolating this rate, these samples reach the tolerance within about 100 total repair steps. PAL's default limit is 10 steps.
  • High curvature: κ=106\kappa = 10^6 is very extreme and stalls for both PAL and the best baselines (SnareNet, DC3). The reason here is that the local convergence basin of projection-based repair scales inversely with κ\kappa (due to the Lipschitz constant of the constraint Jacobian, see Dennis and Schnabel, 1996). We suspect that this basin is narrower than one period of the periodical constraint c(y)=a⊤y−sin⁡(ω,b⊤y)c(y) = a^\top y - \sin(\omega\\, b^\top y), so linearized repair steps can "jump" between periods without converging.

We want to stress that both of these cases represent very difficult edge cases and PAL performs at least as well as or better than all baselines on both.

This limits the scope of the broad precision claims, even with the qualifications in Appendix F.1.

We never intended to make a claim to this effect. The experiment in Appendix G is included to illustrate that the first-step residual target τc=10−4\tau_c = 10^{-4} and the actual precision that can be reached for N >1 (at test time) are not to be confused.

This is not to be mistaken with "PAL's projector unconditionally converges in a finite number of steps". We never claim this, and in fact know that this is not true for any of the repair methods that can deal with constraints of general form.

Changes in Paper:

  • In Appendix F.1, we now relate the LM projector to the new Appendix P. We removed "consistently reach arbitrary precision" from Appendix G to avoid creating reader confusion.
  • We also added a new Appendix P in response to reviewer 6ycg's inputs where we discuss conditions for projector convergence.
  • Discussion of benchmarks where full feasibility is not reached have been added in Appendix R.

Please let us know if the wording in F.1 triggered this concern, and if the removal and the new additions resolve this issue from your point of view.

If any other concern still keeps you from a higher score, please let us know and we will do our best to address it before the discussion closes.

AuthorsOpenReview ↗

Follow-up on our Rebuttal

Dear Reviewer pLim,

Thank you again for your feedback and questions.

As the discussion period nears its end, we'd appreciate any final thoughts you may have on our responses, if time permits. We'd also be happy to clarify further.

If you feel that our responses have addressed your concerns, we would appreciate it if you could take them into account when finalizing your evaluation.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overview of Changes to the Revised Version

Below is a summary of the additions made for the revised version. We feel that these last refinements have helped sharpen some final details, and we want to thank all the reviewers for their valuable inputs.

The main claims of the paper remain unaffected. We have not made any changes to the introduction or conclusion.

Major Additions:

  • Additional ablation in Section 5.4 that more explicitly isolates the effect of post-one-step residual and adaptive weight updates.
  • The new Appendix P derives conditions for projector convergence and suboptimality, and relates them to the weights wdw_d and wc,kw_{c,k} and to our weight update scheme.
  • We've also added Appendix Q with a visualization of repair iterates at test time, where we show feasibility, constraint violation, optimality and cumulative displacement over repair step ii and discuss these.

Minor Additions:

  • One additional baseline variant, FSNet-LM, for the ACOPF benchmark, and a slightly narrower claim about PAL outperforming all baselines in Section 5.2.
  • A new Figure 12 that shows the ACOPF constraint violation density for PAL, DC3, and FSNet-LM.
  • Our own IPOPT reference for E3 ACOPF in Table 2, which replaces the citation to Donti et al.
  • Appendix R discusses why no method reaches perfect feasibility on some benchmarks (E3 ACOPF, high curvature, and E2 urban design). In this appendix, we also relate the non-convergence on these three benchmarks to the conditions for repair convergence in Appendix P.
  • Slightly revised wording of the precision claims in Appendix G, because the previous wording could be misread as a claim of unconditional projector convergence.
  • In Appendix F.1, we now relate the LM projector to the new Appendix P.
  • Corrected a minor error in the lift constraint equation in Appendix K.
  • Slightly changed E2 memory numbers, mainly for PAL. We found a mistake in our code that loaded a surrogate model twice for some methods. We now report 18.7 GB instead of 25.7 GB for PAL.

We placed the new appendices at the end of the paper and named the new table "Table Z" so that all existing appendix letters and table numbers stay unchanged. If the paper were to be accepted, we would rearrange the appendices for better flow in the camera-ready version.

Area_Chair_h1HBOpenReview ↗

Reminder: Reviewer–Author Discussion

Thank you to the reviewers for your valuable feedback on this submission. As the discussion deadline is approaching, I would kindly encourage all reviewers to review the authors’ responses and share your follow-up comments, particularly on whether your concerns have been adequately addressed.

Authors are also encouraged to respond promptly to any further questions. Thank you all for your time and contributions to a constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.