- Rating
- 3
- Confidence
- 2(+1)
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper studies a limitation of learning through orthogonal projection: although projected outputs satisfy the constraints, projection can remove gradient directions needed to leave a suboptimal boundary face. The proposed Optimality-Aware (OA) projection mixes orthogonal projection with an interior soft-radial map during training. A lagged KKT residual controls the mixing strength, and a decaying schedule eventually recovers orthogonal projection. The paper establishes convergence to surrogate stationary points and conditional KKT consistency results, and evaluates the approach on quadratic and nonconvex programs, trajectory optimization, DC optimal power flow, and graph flow.
Critical strengths
- A well-motivated distinction between feasibility and optimality. The geometric explanation and the two-dimensional example in Appendix A.1 make the failure mechanism concrete. The paper identifies a meaningful limitation of a commonly used architectural component.
- A principled, feasibility-preserving construction. The convex combination of two feasible maps preserves feasibility, while the lagged residual gives a clear mechanism for activating the correction. The analysis distinguishes convergence to projection-surrogate stationarity from the stronger, conditional KKT conclusions.
- Consistent evidence across several optimization settings. The appendix reports multiple seeds, objective gaps, constraint violations, and natural residuals. For example, the mean minimum-cost-flow gap improves from 0.2260 for OP to 0.0810 for OA (Table 10), with very small reported violations. These are useful controls beyond objective value alone.
Critical weaknesses
-
The practical scope of the KKT guarantee needs clearer qualification. Theorem 5 establishes that cluster points are stationary for the OP-composed objective; it does not itself rule out non-KKT limits. Theorems 6-7 additionally assume convergence to a point and particular face geometry. Theorem 7 also requires the augmented feature matrix to have full row rank, which implies that the feature dimension plus one must be at least the number of training samples. In the DC-OPF experiment, however, Appendix D.3.1 specifies 4,000 fit samples and a width-128 feature backbone, so this condition cannot hold for the described head. That experiment also uses Adam rather than the analyzed SGD updates. The empirical improvements remain valuable, but the paper should distinguish the settings covered by the guarantee from those supported experimentally.
-
The experiments do not isolate the benefit of KKT feedback from a simpler scheduled mixture. Comparisons against OP and SR show that the full method is useful, but do not establish whether residual regulation is necessary. A fixed mixture and a decaying mixture without residual feedback would be informative controls using the existing benchmarks. Furthermore, the per-sample residual in Eq. (12) requires an additional constrained least-squares calculation. Results under equal iteration budgets do not establish the practical trade-off between solution quality and this training cost; a representative comparison at equal wall-clock time would strengthen the evidence.
-
The description of the operator is occasionally confusing. The abstract emphasizes a correction to backward gradient flow without modifying the OP layer, whereas Eq. (9) and Algorithm 1 also change the training-time forward output through a convex mixture. Please state this distinction explicitly and clarify that standard OP is used at deployment, as described in Appendix D.3.1.
Questions for authors
- Can you explain how Theorem 7 relates to the 4,000-sample, width-128 DC-OPF setup, and identify which guarantees apply to the actual Adam training protocol?
- On a representative existing benchmark, how does OA compare with a fixed OP/SR mixture and a decaying mixture without KKT feedback, both at equal iterations and equal wall-clock time? What fraction of training time is spent computing the residual?
- Is the forward mixture in Eq. (9) used in every experiment, and is it replaced by OP for validation and deployment in all three application benchmarks?