- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the inference for SOC-ICNNs through an attractive optimization-based step. It derives an explicit characterization of the optimal dual multipliers and shows that their input-space readout recovers the complete convex subdifferential. Using this geometry, the proposed dual-certified inference method jointly optimizes over objective subgradients and constraint normals to obtain a stationarity residual and a tangent descent direction. The method combines curvature-based local steps with a proximal safeguard. The analysis establishes global convergence for the exact algorithm and local quadratic convergence near structurally nondegenerate interior minimizers under additional regularity assumptions. Experiments validate the geometric formulas, isolate the roles of curvature and constraint geometry, and evaluate inference on learned battery-scheduling objectives.
Critical strengths
-
The explicit subdifferential construction is useful and well motivated. Proposition 3 and Theorem 4 turn the network's internal structure into a representation of all supporting slopes, including the recursive coupling between ReLU multipliers. This addresses a real limitation of relying on one AD-selected derivative at a nonsmooth point.
-
The theory and component studies form a coherent argument for the method. Theorem 9 states the exact-solve and regularity assumptions explicitly, and the convergence argument connects the local decrease test to a proximal fallback. The empirical results support the value of the proposed combination.
Critical weaknesses
The efficiency advantage is dependent on the problem regime instead of universal: DCI-C is fast on the frozen battery objectives, but Lifted-PD is faster on both controlled boundary suites, with both methods achieving 100% success. On smooth branches, DCI-C and Canonical-Newton have identical iteration counts, so that comparison primarily demonstrates the benefit of the structured Hessian implementation. So when the inference efficiency is superior desires more discussions.
Questions for authors
- See weakness.
- Even if DCI efficiently minimizes the learned objectives, it still shows high operational regret. How should we interpret its practical benefit for downstream decision-making? This is the question I care about most. But even if this is not addressed, it still serves as a non-trivial result and progress for ICNNs.
3 comments
Response to the rebuttal
Thanks for the detailed comment. I think it's indeed a broadly applicable method and brings people new insights about the inference stage of (non-smooth) ICNNs. But it still remains unclear whether the inference stage is the dominating bottleneck of SOC-ICNN, so enhancing its inference itself is worth 3, but whether it is worth a 4 depends on the true bottleneck practically. And I am uncertain about this. Based on this, I recommend a score bigger than 3 but lower than 4.
Thank you for the follow-up
Thank you for the follow-up and for recognizing the value of the geometric readout. We will add a short paragraph to the Introduction explaining when inference cost matters in practice, particularly for repeated queries and online control, citing some representative ICNN application works. This will clarify the application context while acknowledging that the dominant bottleneck depends on the deployment setting. Thanks you again.
Response to Reviewer jJqm
Thank you for your careful reading and for recognizing the value of the subdifferential construction and its connection to the algorithm and theory. Your comments helped us clarify what the experiments demonstrate, especially the distinction between inference accuracy and operational performance. Revisions primarily addressing your comments are marked in blue.
Weakness / Question 1: Where does the efficiency advantage come from?
Our original description did not clearly distinguish Lifted-PD from a full SOCP reformulation. In the controlled tests, Lifted-PD used an explicitly available, single-layer structured objective and a specialized primal-dual formulation. That benchmark did not measure the complete process of expanding a deep SOC-ICNN into a full SOCP and solving it with a conic solver.
To evaluate that route, we added a full-SOCP baseline that constructs the auxiliary variables, constraints and coefficient matrices from the network, then solves the model with Clarabel. We report construction and solve/validation time separately. DCI timings likewise include setup, geometric queries, local updates and all proximal safeguards.
The new benchmark covers 60 objectives across three network scales, smooth or mixed-boundary minimizers, and interior or active-box minimizers. DCI reaches the target accuracy on all 60. Median times for the mixed groups are:
Full SOCP remains faster on the small mixed problems, while DCI is faster at the two larger scales. These results give a more direct comparison of inference costs starting from a network.
Your interpretation of the smooth-branch results is correct: the benefit there is cheaper curvature evaluation, with the same Newton iteration counts. We now measure this cost separately. Across 38 depth/width configurations at input dimension 20, explicit Hessians are 9.11–37.27 times faster than the fastest tested AD implementation.
We also added a matched ablation to isolate the role of complete geometry at nonsmooth boundaries. On ten boundary initializations, both DCI and a single-subgradient variant accept two local updates, but the latter additionally needs one proximal safeguard. DCI reaches the target using local updates alone. This identifies a concrete benefit of the full subdifferential beyond faster Hessian computation.
Question 2: What is the downstream benefit despite high operational regret?
We appreciate your emphasis on this question. We agree that the original operational-cost comparison was poorly matched to the inference contribution. Our pipeline first learned a cost model and then optimized it; Point Forecast and Historical SAA optimized explicit engineering models built from forecasts or historical scenarios. The resulting regret therefore reflected differences in modeling and information as well as numerical solution quality. Although learning is outside our contribution, its error affected the reported operational outcome.
We checked this by independently computing high-accuracy reference solutions of the same learned objectives. These solutions also retain high operational regret. The mean absolute realized-cost difference between DCI and the reference is only 1.54×10−5 in normalized operating units. Thus, solving the learned objective more accurately does not remove the operational gap: the main discrepancy in this case lies between the learned objective and realized cost.
The revised battery experiment focuses on inference for fixed trained networks. Every method solves the same learned objective, and we report model-fit errors separately from solver errors. Across three trained models and 363 test days, DCI reaches the target on all 1,089 objectives. On the separate 90-objective timing set, its median runtime is 0.34 ms, compared with 1.68 ms for full SOCP and 1.14 ms for Bundle (τ=0.1); all three solve the entire timing set. The realized-cost deviation above also shows that DCI closely matches the downstream outcome of high-accuracy reference inference.
The demonstrated benefit is therefore accurate inference for trained models at lower computational cost. Improving the learned objective remains necessary to improve this model's operational decisions. The revised evaluation makes these two roles explicit.
The new experimental code is included in the supplementary material. Thank you again for the constructive comments. We hope these additions address your questions about both the efficiency advantage and its downstream significance.