acceptodds
Under review as a conference paper at ICLR 2027

Dual Certified White-Box Inference for Input Convex Neural Networks

Abstract

Input convex neural networks (ICNNs) are used to learn convex objectives whose minimizers define decisions, making efficient and reliable optimization central to inference. At nonsmooth inputs, automatic differentiation returns a single derivative rather than the full subdifferential governing optimality and descent. Second-order cone ICNNs (SOC-ICNNs) admit an exact representation as value functions of parametric second-order cone programs, providing a white-box approach to recovering their full subdifferentials from optimal dual multipliers and deriving explicit Hessians on smooth regions. Building on this representation, we develop dual-certified inference (DCI), which combines the network and feasible set geometries to obtain exact stationarity certificates and tangent common descent directions. DCI uses local curvature for Newton acceleration and an exact proximal safeguard. We establish global convergence and, under standard regularity conditions, local quadratic convergence near structurally nondegenerate interior minimizers. Numerical experiments validate the recovered geometry and demonstrate the reliability and efficiency of DCI.

Primary area:
optimization
Keywords:
White-box inference, Subdifferencial information, Input convex neural networks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jJqmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the inference for SOC-ICNNs through an attractive optimization-based step. It derives an explicit characterization of the optimal dual multipliers and shows that their input-space readout recovers the complete convex subdifferential. Using this geometry, the proposed dual-certified inference method jointly optimizes over objective subgradients and constraint normals to obtain a stationarity residual and a tangent descent direction. The method combines curvature-based local steps with a proximal safeguard. The analysis establishes global convergence for the exact algorithm and local quadratic convergence near structurally nondegenerate interior minimizers under additional regularity assumptions. Experiments validate the geometric formulas, isolate the roles of curvature and constraint geometry, and evaluate inference on learned battery-scheduling objectives.

Critical strengths
  1. The explicit subdifferential construction is useful and well motivated. Proposition 3 and Theorem 4 turn the network's internal structure into a representation of all supporting slopes, including the recursive coupling between ReLU multipliers. This addresses a real limitation of relying on one AD-selected derivative at a nonsmooth point.

  2. The theory and component studies form a coherent argument for the method. Theorem 9 states the exact-solve and regularity assumptions explicitly, and the convergence argument connects the local decrease test to a proximal fallback. The empirical results support the value of the proposed combination.

Critical weaknesses

The efficiency advantage is dependent on the problem regime instead of universal: DCI-C is fast on the frozen battery objectives, but Lifted-PD is faster on both controlled boundary suites, with both methods achieving 100% success. On smooth branches, DCI-C and Canonical-Newton have identical iteration counts, so that comparison primarily demonstrates the benefit of the structured Hessian implementation. So when the inference efficiency is superior desires more discussions.

Questions for authors
  1. See weakness.
  2. Even if DCI efficiently minimizes the learned objectives, it still shows high operational regret. How should we interpret its practical benefit for downstream decision-making? This is the question I care about most. But even if this is not addressed, it still serves as a non-trivial result and progress for ICNNs.
3 comments
AuthorsOpenReview ↗

Response to Reviewer jJqm

Thank you for your careful reading and for recognizing the value of the subdifferential construction and its connection to the algorithm and theory. Your comments helped us clarify what the experiments demonstrate, especially the distinction between inference accuracy and operational performance. Revisions primarily addressing your comments are marked in blue.

Weakness / Question 1: Where does the efficiency advantage come from?

Our original description did not clearly distinguish Lifted-PD from a full SOCP reformulation. In the controlled tests, Lifted-PD used an explicitly available, single-layer structured objective and a specialized primal-dual formulation. That benchmark did not measure the complete process of expanding a deep SOC-ICNN into a full SOCP and solving it with a conic solver.

To evaluate that route, we added a full-SOCP baseline that constructs the auxiliary variables, constraints and coefficient matrices from the network, then solves the model with Clarabel. We report construction and solve/validation time separately. DCI timings likewise include setup, geometric queries, local updates and all proximal safeguards.

The new benchmark covers 60 objectives across three network scales, smooth or mixed-boundary minimizers, and interior or active-box minimizers. DCI reaches the target accuracy on all 60. Median times for the mixed groups are:

ScaleDCI (ms)Full SOCP: construction + solve/validation (ms)
Small4.640.99 + 1.28
Medium8.702.69 + 42.33
Large13.517.73 + 623.10

Full SOCP remains faster on the small mixed problems, while DCI is faster at the two larger scales. These results give a more direct comparison of inference costs starting from a network.

Your interpretation of the smooth-branch results is correct: the benefit there is cheaper curvature evaluation, with the same Newton iteration counts. We now measure this cost separately. Across 38 depth/width configurations at input dimension 20, explicit Hessians are 9.11–37.27 times faster than the fastest tested AD implementation.

We also added a matched ablation to isolate the role of complete geometry at nonsmooth boundaries. On ten boundary initializations, both DCI and a single-subgradient variant accept two local updates, but the latter additionally needs one proximal safeguard. DCI reaches the target using local updates alone. This identifies a concrete benefit of the full subdifferential beyond faster Hessian computation.

Question 2: What is the downstream benefit despite high operational regret?

We appreciate your emphasis on this question. We agree that the original operational-cost comparison was poorly matched to the inference contribution. Our pipeline first learned a cost model and then optimized it; Point Forecast and Historical SAA optimized explicit engineering models built from forecasts or historical scenarios. The resulting regret therefore reflected differences in modeling and information as well as numerical solution quality. Although learning is outside our contribution, its error affected the reported operational outcome.

We checked this by independently computing high-accuracy reference solutions of the same learned objectives. These solutions also retain high operational regret. The mean absolute realized-cost difference between DCI and the reference is only 1.54×10−51.54\times10^{-5} in normalized operating units. Thus, solving the learned objective more accurately does not remove the operational gap: the main discrepancy in this case lies between the learned objective and realized cost.

The revised battery experiment focuses on inference for fixed trained networks. Every method solves the same learned objective, and we report model-fit errors separately from solver errors. Across three trained models and 363 test days, DCI reaches the target on all 1,089 objectives. On the separate 90-objective timing set, its median runtime is 0.34 ms, compared with 1.68 ms for full SOCP and 1.14 ms for Bundle (τ=0.1\tau=0.1); all three solve the entire timing set. The realized-cost deviation above also shows that DCI closely matches the downstream outcome of high-accuracy reference inference.

The demonstrated benefit is therefore accurate inference for trained models at lower computational cost. Improving the learned objective remains necessary to improve this model's operational decisions. The revised evaluation makes these two roles explicit.

The new experimental code is included in the supplementary material. Thank you again for the constructive comments. We hope these additions address your questions about both the efficiency advantage and its downstream significance.

Reviewer_jJqmOpenReview ↗

Response to the rebuttal

Thanks for the detailed comment. I think it's indeed a broadly applicable method and brings people new insights about the inference stage of (non-smooth) ICNNs. But it still remains unclear whether the inference stage is the dominating bottleneck of SOC-ICNN, so enhancing its inference itself is worth 3, but whether it is worth a 4 depends on the true bottleneck practically. And I am uncertain about this. Based on this, I recommend a score bigger than 3 but lower than 4.

AuthorsOpenReview ↗

Thank you for the follow-up

Thank you for the follow-up and for recognizing the value of the geometric readout. We will add a short paragraph to the Introduction explaining when inference cost matters in practice, particularly for repeated queries and online control, citing some representative ICNN application works. This will clarify the application context while acknowledging that the dominant bottleneck depends on the deployment setting. Thanks you again.

Reviewer_PiVYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies the problem of getting valid subgradients from automatic differentiation when using input-convex neural networks (ICNNs). To address this problem, the authors propose dual-certified inference for a generalization called second-order cone ICNNs. They characterize the full set of dual multipliers at a given point xx and establish that a simple linear mapping gives the full subdifferential. Moreover, at points where the function is differentiable, the authors present closed-form expressions for the second-order Hessian information. Convergence guarantees are presented along with local quadratic convergence in cases where the minimizer is not structurally degenerate.

Critical strengths
  • The dual certified inference algorithm is able to fully recover the full set of subdifferentials, which seems novel to me. This seems to overcome a classic limitation of automatic differentiation when working with nonsmooth functions

  • The theoretical convergence results seem sound to me

Critical weaknesses
  • The battery-scheduling experiment is weak. The regret incurred by the proposed approach is about double the regret incurred by using no storage and is far worse than other reasonable baselines. Including this experiment in the main text seriously undermines the case for utility of this approach

  • The local quadratic convergence result in Theorem 9 seems to rely on some restrictive assumptions under which the problem is equivalent to an unconstrained smooth Newton method. Is there anything new on the local quadratic convergence result? Are there other interesting things that can be said about convergence to boundary minimizers?

Questions for authors
  • How much is lost if you just smooth the model so that we can avoid the lack of differentiability? In practice, we often replace ReLU with a smooth approximation (softplus =1βlog⁡(1+exp⁡(βx))= \frac{1}{\beta}\log(1+\exp(\beta x)) with large β\beta) and a non-differentiable Euclidean norm with a smoothed norm. With these substitutions, the network remains convex and is smooth so that we can get exact gradients and Hessians using automatic differentiation. I wonder how such a baseline would compare in the numerical experiments.

  • Maybe it is my naivete, but I am not very familiar with when people use ICNNs or ICNN-SOCs within deep learning frameworks. How relevant a problem is it that automatic differentiation doesn't give exact subdifferentials? Can't a user use a different convex optimization solver routine and circumvent this issue?

  • What is the scalability of this approach? As the network grows in size, how challenging is it to use DCI?

4 comments
AuthorsOpenReview ↗

Response to Reviewer PiVY

Thank you for recognizing the full subdifferential construction and the convergence analysis. Reviewer-specific revisions addressing your comments are marked in green.

W1. Battery-scheduling regret

We agree that the original operational-cost comparison was mismatched with our inference focus. Our pipeline learned a cost model before optimizing it, while Point Forecast and Historical SAA optimized explicit models using forecasts or scenarios. Learning is outside our contribution, but its error affected the final regret.

We recomputed independent high-accuracy solutions of the same learned objectives. They retain similarly high operational regret; their mean absolute operating-cost difference from DCI is only 1.54×10−51.54\times10^{-5}. Thus, the high regret mainly reflects the mismatch between the learned objective and operating cost, rather than inaccurate inference.

The revision centers the comparison on solving the same fixed networks and reports model-fit and solver errors separately. Learning ICNN models for control already has experimental support, e.g., Bünning et al. (2021). Our contribution concerns inference after learning.

W2. Local quadratic convergence and boundary minimizers

You are right that the local quadratic rate is inherited from smooth Newton theory. The result shows how DCI, including its geometric queries, feasible line search and proximal safeguard, eventually enters this regime and accepts unit Newton steps near a structurally nondegenerate interior minimizer under the stated regularity assumptions.

Boundary minimizers remain covered by global convergence under the stated compact-level-set, bounded-metric and exact-solve assumptions. The constrained residual certifies optimality and, under strong convexity, bounds the objective gap. We do not claim a quadratic rate at boundary minimizers.

Q1. Smoothing as an alternative

Smooth ICNNs are a useful modeling choice, as illustrated by Huang et al. (2021). For an already trained nonsmooth network, however, replacing ReLU with Softplus changes the objective and does not preserve the exact SOCP representation used by our readout. This structural trade-off is discussed in SOC-ICNN, Liu et al., NeurIPS 2026. Our SOCP-based multiplier readout therefore does not directly carry over.

We added Softplus/norm-smoothing continuation with AD-based Newton and L-BFGS-B, retaining the weights and constraints and evaluating all solutions on the original nonsmooth objective. They reach the target on 58/60 and 57/60 problems, versus 60/60 for DCI. On large mixed problems, DCI takes 13.51 ms (10/10 successes), versus 196.53 ms (9/10) and 370.61 ms (8/10). These are median times over successful instances. Full settings and results are in the appendix.

Q2. Why complete geometry rather than AD or a convex solver?

ICNN inference optimizes network inputs to obtain decisions, as in control and convex-potential models. An AD-selected vector can be a valid subgradient, but one vector does not describe all directional derivatives and may miss constrained stationarity. Our readout retains cross-layer multiplier coupling and combines the full subdifferential with constraint normals to certify optimality or construct descent directions. This use of internal structure is what makes DCI white-box; merely avoiding a lifted model does not.

A conic solver is also a valid option, and we now compare against full SOCP. This route assembles auxiliary variables, constraints and coefficients before solving the lifted model. Both costs are included. For medium/large mixed problems, lifting plus solve/validation takes 2.69+42.332.69+42.33 / 7.73+623.107.73+623.10 ms, versus 8.70 / 13.51 ms for DCI. Full SOCP is faster on small mixed problems. The growing cost is dominated by solving the lifted model. Such costs matter when inference is called repeatedly, including in decision-focused learning. DCI uses local geometric queries and input-space updates while retaining a proximal safeguard.

Q3. Scalability

We distinguish geometric readout from full inference. Readout tests cover 38 depth/width configurations, reaching depth 64 or width 4096, plus input-dimension and structural-boundary sweeps. Explicit Hessians are 9.11–37.27 times faster than the fastest tested AD implementation in the depth/width sweep at input dimension 20.

The full inference benchmark covers 60 objectives at (L,w,d)=(2,32,12),(8,64,20),(16,128,32)(L,w,d)=(2,32,12),(8,64,20),(16,128,32). DCI solves all 60; mixed-group median times are 4.64, 8.70 and 13.51 ms. Costs depend on input dimension, the number of boundary variables and safeguard use, as well as network size.

The new experimental code is included in the supplementary material. We hope these revisions address your concerns.

As the rebuttal period draws to a close, we wanted to briefly follow up on our response.

The revision separates learning and solver errors in the battery experiment, adds the requested smoothing and full-SOCP comparisons, and clarifies convergence guarantees and scalability. The relevant changes are marked in green.

We would be grateful if you could revisit our response and let us know whether these additions address your concerns or whether further clarification would help. Thanks for your time.

Reviewer_PiVYOpenReview ↗

Response

I would like to thank the authors for their comprehensive revision and response to my previous comments. While I appreciate the changes, I still think this is a relatively minor contribution that can be completely circumvented if a designer used a smooth ICNN. While an existing nonsmooth one may benefit from the methodology proposed in this paper, I do not know of many pretrained ICNNs that require this sophistication. While it is an interesting theoretical contribution, I am unclear of the practical implications. For that reason, I will keep my score the same but am open to further discussions with the other reviewers and the AC.

AuthorsOpenReview ↗

Ackwnoledgement

Thank you for the thoughtful follow-up. We agree that, when modeling a largely unknown system from data, a smooth ICNN is a natural choice. It avoids the nonsmoothness issue considered here and allows the use of standard smooth optimization methods.

SOC-ICNNs nevertheless offer computational advantages through explicit geometric readout. In Experiment 1 (Section 6.1), explicit Hessian evaluation is 9.11–37.27 times faster than the fastest tested AD implementation across the depth–width configurations. This makes second-order inference particularly attractive for repeated decisions and real-time deployment. For applications without such latency requirements, inference speed is naturally less pressing.

More importantly, our motivation is system identification with structural priors. When a target model has known LP, QP, or SOCP structure, retaining relevant piecewise-linear, quadratic, and norm components can reduce the burden of learning them through a generic smooth approximation. SOC-ICNNs provide a way to preserve these components while learning unknown convex contributions. This is a useful direction alongside established ICNN applications in optimal transport and other widely used control systems. Our battery analysis also highlights why this matters: the main operational gap came from the learned objective, rather than insufficient accuracy in solving it.

Your comments have made us realize that these modeling motivations deserve a clearer explanation in the Introduction. We will strengthen this discussion with representative references and continue developing applications that make fuller use of structural priors and efficient inference. Regardless of the ICLR outcome, we will keep improving the work in this direction. Thank you again for your time and helpful comments.

Reviewer_1PS1OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies minimization of a convex, piece-wise continuously differentiable function represented by a SOC-ICNN model; the latter is the sum of an input convex neural network (that is a ReLU-network with non-negativity restrictions on the weights) and quadratic terms as well as terms represented by the norm of affine functions, recently introduced by Liu et al. As the sum of a convex continuous piecewise-linear function (with a potentially non-trivial representation as a neural network) and several simple convex quadratic and norm terms, it is possible to describe a lot of the structure explicitly: the authors give exact expressions for dual multipliers, subdifferentials and directional derivatives. This is used to solve constraint minimization problems over closed convex feasible sets, adapting Newton iterations. This is complemented by computational experiments minimizing some functions of unclear origin (Section 5.2) and from battery scheduling (Section 5.3).

Critical strengths
  • The paper provides explicit expressions for various key structures of the minimization problem including subdifferentials and conditions for stationarity. This may lead to further insights building on these expressions.
Critical weaknesses
  • Related work is barely discussed in the main text, the relevant discussion is given in Appendix A. However, also the discussion there is not entirely satisfactory: the comparison with work on (optimizing) input convex neural networks makes sense, and in particular the overview of nonsmooth optimization techniques; however, I do not really see what is the substantial additional value of this paper compared to the related work. The Related Work section emphasizes that earlier work does not give explicit expressions of the "geometric structure" (I guess this means subdifferentials and directional derivatives), however, the substantial additional value is not explained. The minimized function is anyway very structured, it is even a convex function.

  • The results seem to be a minor adaptation on existing tools from non-smooth optimization and duality. The structure of the considered objective functions is pretty simple, so it is not surprising that one can write out explicit expressions. The additional value for these expressions is not sufficiently clear.

  • Throughout, it is difficult to follow the technical details, as there are various subtleties which are crucial for understanding if the considered structure is benign or not. For example, in the description of SOC-ICNN models in the introduction, the crucial property of non-negativity of weights is omitted.

Questions for authors
  • Please elaborate on the choice of examples for demonstrating that your algorithm is particularly well-behaved. Furthermore, as far as I know, there are various algorithms that one could apply to this problem (see also the related work section on non-smooth objectives), though you don't seem to compare your algorithm with state of the art algorithms for your problem; please justify this.
6 comments
AuthorsOpenReview ↗

Response to Reviewer 1PS1

Thank you for your comments. The original paper did not explain the purpose of the geometric readout clearly enough, and the experiments needed stronger comparisons. We have revised both. Changes addressing your comments are marked in red.

W1. Relation to existing work.

Related Work is now in Section 2. We have expanded the discussion of ICNN inference, including LP and bundle-entropy methods, and its connections to neural-network geometry and nonsmooth optimization.

We have also rewritten the Introduction and Problem Statement around white-box inference. The SOC-ICNN–SOCP equivalence is our starting point: it gives a way to solve the inference problem, but doing so requires constructing and solving a lifted model. We ask what information can instead be obtained from the network itself and used to update its input under additional constraints.

Here, “white-box” refers to the information used by the optimizer. An optimizer that receives only function values and AD-selected derivatives still treats the network as an oracle, even if no SOCP is constructed. DCI uses the internal module states and cross-layer multiplier relations to recover the full subdifferential and query it for certification and descent. On smooth branches, it also uses explicit curvature.

W2. What the explicit geometry adds.

The duality tools are classical. What we obtain from them is a representation of the full subdifferential that can be built from network parameters and forward states, with free network multipliers retained only at structural boundaries after elimination.

At a nonsmooth point, taking the negative of a valid subgradient can increase the objective, while a nonzero subgradient does not show that the point is nonoptimal. The full set lets us evaluate the directional derivative and, together with constraint normals, find a common descent direction or certify optimality. In the new two-layer example in Section 4.2, dropping the cross-layer coupling introduces slopes that are not subgradients. The same example shows why minimizing the multiplier norm does not give the minimum-norm input-space subgradient.

We have added measurements of the cost and use of this representation. In one boundary configuration, elimination reduces 540 multiplier coordinates to 40, and the same minimum-norm-subgradient query takes 1.57 ms rather than 25.46 ms. Both times include forward evaluation, construction, and solution. In ten matched boundary-start experiments, full geometry reaches the target in two local updates; selecting one subgradient requires an additional proximal step. For smooth curvature, explicit Hessians are 9.11–37.27 times faster than the fastest tested AD implementation across 38 depth–width configurations at fixed input dimension. These results are reported in Sections 6.1–6.2.

W3. Presentation and missing conditions.

We have added the nonnegativity requirement to the Introduction and specified all convexity conditions and module definitions in Section 3. We also clarify which formulas apply at structural boundaries and which require a smooth branch. Section 5 now includes a short algorithm showing how the readout, direction computation, feasible search, and proximal safeguard fit together.

Q1(a). Choice of examples.

The new synthetic comparison uses 60 strongly convex objectives: three network scales, four combinations of smooth/mixed-boundary and interior/active-box minimizers, and five instances per combination. We choose the affine term so that a prescribed point satisfies the optimality conditions. Strong convexity makes this minimizer unique, giving us a known reference for measuring solution accuracy.

This construction lets us examine nonsmoothness, constraint activity, and network scale separately. Iterative methods start at the same point and use normalized objective-error and feasibility targets of 10−810^{-8}. The construction and parameters are given in Appendix C.3. Ablations test the components; the battery experiment tests inference on trained networks.

Q1(b). Choice of baselines.

The revised comparison includes AD–L-BFGS-B, a full-memory fixed-proximity proximal Bundle implementation, smoothing continuation with Newton or L-BFGS-B, and full SOCP solved by Clarabel. These test direct oracle-based optimization, smoothing, and explicit conic modeling. We evaluate every method on the original objective and include setup and inner-solve costs in the timings.

DCI reaches the target on all 60 objectives. Its median runtime is lowest in the three smooth groups and the medium and large mixed groups; full SOCP is faster in the small mixed group. Section 6.2 reports the results by group, and the appendix gives the implementations and settings.

Code for the new experiments has been uploaded as supplementary material. We hope these revisions and additional experiments address your concerns.

As the rebuttal period draws to a close, we wanted to briefly follow up on our response.

The revision clarifies the value of complete geometry through a coupled-network example, timing measurements, and matched ablations, alongside expanded related work and baseline comparisons. The relevant changes are marked in red.

We would appreciate your feedback on whether these revisions address your concerns, thank you for your time.

I appreciate that the authors have spent more effort on computational experiments, related work and more care with details. However, I am still not fully convinced of the additional value of the `geometry', whatever that is supposed to mean in this context; I find your explanation not sufficiently clear. Could you try to address this point again in a brief response? Maybe you could also point out on a conceptual level, how your insights differ from other established non-smooth optimization?

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revision summary

Revision Summary

We thank the reviewers for their careful evaluations. Their comments highlighted two aspects requiring clearer explanation: the computational value of geometric readout and its relationship to conventional optimization methods and solvers; and the purpose of the synthetic and battery-scheduling experiments. We have reorganized the manuscript around white-box inference for structured ICNNs.

The revision articulates two connected questions: How can we recover inference geometry directly from network parameters and forward states without constructing the full equivalent optimization model? How can this geometry support DCI with optimality certification and convergence guarantees? The established SOC-ICNN–SOCP equivalence provides the foundation, while explicitly constructing and solving the lifted model entails additional variables, constraints, and computation. Our focus is on preserving complete multiplier coupling, enabling input-space geometric queries, and quantifying their computational benefits.

We have rewritten the Introduction and Problem Statement, moved Related Work into the main text, and expanded its discussion of neural-network geometry, ICNN inference, and nonsmooth optimization. A new two-layer example illustrates why cross-layer coupling matters for exact subdifferential recovery and why minimum-norm multipliers need not yield minimum-norm input-space subgradients. We explain two uses of the representation: evaluating candidate directions through directional derivatives, and combining the readout with feasible-set normals to construct optimality certificates and common descent directions.

The experiments address three questions:

  1. Is geometric readout accurate and efficient? We validate smooth gradients, Hessians, and nonsmooth directional derivatives, and add direct timing comparisons with automatic differentiation. Measurements cover 38 depth–width configurations, with depths up to 64 and widths up to 4096, plus a separate input-dimension study. Across 27 boundary configurations, we compare full and recursively eliminated multiplier representations on identical geometric queries, measuring accuracy and cost.

  2. Does the geometry improve constrained inference? We construct 60 strongly convex SOC-ICNN objectives with known minimizers, varying network scale, structural state, and input-constraint activity. Comparisons include AD–L-BFGS-B, proximal bundle methods, smoothing continuation using Newton or L-BFGS-B, and explicit full-SOCP solution. All methods are evaluated on the original objective; full-SOCP time is split into construction and solve/validation costs. Matched ablations examine complete geometry and safeguarding, complemented by curvature and general-constraint studies. We improve the implementation of the same metric-proximal safeguard and include all construction, validation, and fallback costs in its runtime.

  3. Do these benefits extend to trained application models? The battery experiment evaluates three trained SOC-ICNNs over 363 test days, giving 1,089 fixed objectives. All inference methods optimize the same learned objective and are compared with an independent high-accuracy reference. We report numerical accuracy, runtime, and realized operating-cost differences from that reference decision. DCI reaches the prescribed accuracy on all 1,089 objectives, with a mean absolute operating-cost difference of 1.54×10−51.54\times10^{-5} in normalized operating-cost units.

Clarification of the battery results. The SOC-ICNN pipeline included learning-based identification of a context-dependent cost model before optimization. In the original operational comparison, Point Forecast and Historical SAA instead optimized analytically specified engineering objectives constructed from forecasts or historical scenarios. That comparison therefore reflected differences in model construction and information use as well as numerical optimization.

Our additional high-accuracy comparison isolates the numerical inference contribution. Accurately minimizing the same learned objective produces essentially the same operational regret as DCI, while the operating-cost difference between their decisions is very small. This supports attributing the dominant regret in this experiment to the mismatch between the learned objective and realized operating cost, rather than insufficient solution accuracy. We now report model-fit diagnostics separately and focus the solver comparison on fixed learned objectives, directly assessing how efficiently and accurately network geometry supports constrained inference.

Code for the new experiments has been uploaded as supplementary material.

AuthorsOpenReview ↗

Rebuttal Summary

We thank the three reviewers and the AC for their time and discussion. As the discussion concludes, we would like to briefly summarize the positioning of our contribution and the reviewers’ latest feedback.

The discussion helped us explain our central contribution more directly: recovering complete, computable optimization geometry from network parameters and forward states, enabling established optimization methods to exploit the internal structure of ICNNs. The representation preserves cross-layer multiplier coupling and eliminates determined multipliers, leaving free network multipliers only at structural boundaries. DCI demonstrates how this representation supports inference with additional input constraints. The geometric readout supplies upstream information to existing optimization frameworks; its value lies in recovering, reducing, and computationally exploiting that structure.

In the final exchanges:

  • 1PS1 stated that our clarification substantially improved their understanding of the work. Although retaining their original score, they expressed willingness to support a more positive judgment from the AC. They also emphasized that this positioning should be communicated more clearly in the paper.
  • jJqm explicitly recognized the method’s broad applicability and the new insights it offers into nonsmooth ICNN inference, recommending an assessment between 3 and 4 in their written response. Their main reservation concerns whether inference is the dominant bottleneck in practical applications.
  • PiVY appreciated the comprehensive revision but regarded smooth ICNNs as a viable alternative and remained unconvinced about the practical demand and magnitude of the contribution. They expressed openness to further discussion with the other reviewers and the AC.

Regarding practical relevance, we further clarified the intended settings. When learning models with known structural priors, retaining piecewise-linear, quadratic, and norm components supports precise modeling. Preserving this structure can improve model accuracy and help avoid drift in control systems. Efficient inference also has direct computational value in deployment settings involving repeated queries and online control. Smooth ICNNs offer another modeling choice, trading interpretability for greater approximation flexibility, whereas our work provides white-box inference for structured models. The battery analysis further distinguishes the quality of the learned objective from the ability to optimize it, clarifying the inference component addressed by this work.

We will incorporate the concise explanations and application motivations developed during the discussion more explicitly into the Introduction. We hope the AC will consider the final discussion, revised manuscript, and experimental evidence together when assessing our contribution to geometric readout and constrained inference for structured ICNNs. We appreciate the time and constructive suggestions from everyone involved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.