- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a second-order approximation of convex conic problems to accelerate their differentiation in differentiable optimization frameworks.
There is use of vague language in the intro that could be improved. "the structure needed to solve...", what structure? An algorithm? A representation of the problem?
Several sections are similarly obfuscated, 3.2 uses vague terms and "sells" some mathematical concepts, e.g "this restricts attention to the perturbation direction", what attention? Of the reader? Restricting the feasible set? What is a "cone-spaced object"?
Critical strengths
Computationally, the proposed method appears to accelerate differentiable optimization layers of SDPs, thus showing the effectiveness of the implementation.
Critical weaknesses
The proposed method solves (sparse) linear systems, exactly like other methods from the literature. It is unclear why this would be faster than other methods. Replacing matrix inversions by linear system solves for backward differentiation is already mentioned in the literature. The experiments should provide a finer analysis of the amount of work performed by iteration (size of the linear systems, sparsity) compared to the baselines.
Questions for authors
Can the authors explain the difference in performance? If fundamentally, the alternative baselines are also solving sparse linear systems, what is the main effect making dopt faster
A minor thing: although the github repository respects anonymity, it is not a good practice to submit temporary repos on github using a temporary account, this is unsustainable for the platform. Use the ZIP file submission of openreview and publish a github repo along public versions of the paper.
We thank the reviewer for the constructive feedback and suggestions for improving the presentation and computational analysis.
Computational performance. A main advantage of dOPT is that it reduces the size and changes the structure of the backward system. Using linear solves in place of matrix inversions is already standard practice in backward differentiation.
dOPT exploits the local cone geometry to retain only perturbation directions relevant to first-order sensitivity, incorporating cone curvature into the reduced Hessian. Inactive conic constraints are eliminated, while SDP constraints are reduced to conditions on the zero eigenspace. The resulting backward problem is a smaller, symmetric KKT system that can be solved directly.
The baselines use different backward formulations. FFOLayer constructs an active-set Lagrangian surrogate and approximates hypergradients through finite differences involving an additional perturbed optimization solve of the same problem class as the forward solve (e.g., SOCP or SDP). CVXPYLayers differentiates the homogeneous self-dual embedding of the full conic program, retaining both active and inactive components and auxiliary variables. This yields a larger, generally nonsymmetric system, for which DiffCP uses iterative LSQR by default, with cost depending on the number of operator evaluations and the convergence rate.
Two examples illustrate the difference in backward system dimension. In the synthetic SOCP benchmark (n=500), dOPT's reduced KKT system has dimension 570 versus 42,263 for CVXPYLayers. In the switched linear system SDP task (50×50 matrix, 1,276 variables), dOPT's system has an average dimension of 2,050 over the first 10 training steps versus 5,103 for CVXPYLayers.
Runtime also depends on sparsity, conditioning, solver choice, and implementation details, so system dimension alone does not fully explain the observed speedups. We find it encouraging that substantial speedups were achieved with a prototype that has not been heavily optimized.
Presentation. We appreciate the reviewer identifying imprecise language in Sections 1 and 3.2 and will revise these passages. Specifically, "the structure needed to solve" refers to the full conic constraints of the forward problem; "restricts attention" means imposing the critical-cone restriction dy∗∈CK(y∗) on admissible first-order perturbations; and "cone-space objects" refers to quantities y=G(z)∈K, as distinct from the decision variables z.