- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper theoretically analyzes the limitations of analytical Lipschitz certificates and proposes two complementary approaches to improve verification: an exact reformulation of GroupSort2 that preserves dependencies in the computational graph, and a hybrid certification framework that decomposes the network into a 1-Lipschitz backbone and a shallow head verified using linear bound propagation. These approaches improve certification tightness and scalability in selected architectural and training regimes. The paper’s central finding is that more sophisticated verifiers do not necessarily yield tighter robustness certificates: for tightly constrained 1-Lipschitz networks, inexpensive analytical bounds can match or outperform LiRPA, whereas LiRPA becomes more beneficial when architectural constraints are relaxed.
Critical strengths
Strong points:
- The geometric analysis of Lipschitz-certified radii motivates the proposed framework and provides a clear, useful comparison between analytical Lipschitz certification and formal verification.
- The computational graph reformulation substantially improves certified robust accuracy in the evaluated settings.
- Ablation studies, repeated training runs, and evaluations on an additional test subset strengthen the empirical evidence.
- The experiments identify regimes in which computationally expensive verification offers little or no improvement over inexpensive analytical certificates.
Critical weaknesses
Weaknesses:
-
The interpretation of Proposition 3.1 is overstated. The proposition establishes conditional gradient attenuation, rather than proving that every expanding architecture must violate the Eikonal condition. A more precise conclusion is that orthogonal expansion does not guarantee global gradient-norm preservation.
-
The temperature explanation requires clarification. The appendix defines the loss as (p. 19), meaning that acts as an inverse temperature. Increasing reduces the unscaled logit margin required to achieve a given confidence. Therefore, the claim that larger requires larger logit margins (p. 5) does not follow from the stated formulation. Although the empirical trend may hold, the proposed causal explanation needs further justification.
-
The soundness of some analytical certificates is unclear. The paper uses power iteration to estimate certain Lipschitz constants (p. 7 and Appendix E.4). Finite power iteration alone does not guarantee an upper bound on the spectral norm. The authors should clarify how these estimates are converted into sound bounds, since underestimating the Lipschitz constant can invalidate the corresponding CRA values. This concern does not, by itself, invalidate the LiRPA certificates.
-
Claims of broad superiority require qualification. Table 10 reports an Imagenette VGG13 baseline with 85.0% clean accuracy and 84.0% CRA, both higher than the highlighted hybrid VGG16 results of 84.5% clean accuracy and 72.5% VRA. Although this comparison involves different architectures, it shows that hybridization does not uniformly dominate the reported constrained baselines. The paper should distinguish improvements in certification for a fixed model from improvements in the overall accuracy–robustness trade-off.
-
The empirical evaluation could be strengthened. Five-run experiments support the smaller-model results, but comparable variability estimates are not reported for the main hybrid results. With only 200 test examples, a 0.5-percentage-point difference corresponds to a single example and should be interpreted cautiously. More complete runtime and peak-memory measurements, together with ablations on the backbone–head split, would better substantiate the scalability claims.
-
Practical applicability remains insufficiently demonstrated. The experiments address image classification under bounded perturbations; in particular, the reported radius of should not be interpreted as the commonly used threat model. This evaluation supports a specific robustness guarantee, but provides limited evidence for the broader safety-critical applications invoked in the motivation.
Questions for authors
No further questions
5 comments
(W5) (Note: As this concern was raised by all reviewers, we provide a common response below.) We initially restricted our evaluation to the 200 test points and perturbation radii established by SDP-CROWN. This was done to ensure a fair comparison with the baseline. However, we fully agree with all the reviewers that 200 samples can be limited, and that drawing conclusions from a 1-to-3 point difference (representing just a few images) requires nuance. To address this, we ran evaluations on 1000 test points for the CIFAR-10 architectures (CNN-A, ConvLarge, VGG13 and hybrid VGG13). These extended benchmarks confirm the trends with higher statistical confidence and will be reported in the revised appendix. In light of this, we will qualify the claims in the main text. Where the margins are thin, we will modify statements like 'Lipschitz-certified radii consistently surpass formal verification' to state that they 'consistently match or outperform' formal verification on fully constrained models. In such cases, it is preferable to use the CRA as it is immediate to compute. We provide the results for Convlarge in the following tab, the complete results will be added to the final version.
Table A
| Model (Setting) | Architecture | Clean Acc (%) | ERA (%) | CRA (%) | SDP-CROWN VRA (%) |
|---|---|---|---|---|---|
| ConvLarge (Accurate) | RKO (Björck) | 82.9 | 79.1 | 18.6 | 26.7 |
| ConvLarge (Accurate) | AOC | 73.7 | 69.7 | 56.1 | 58.0 |
| ConvLarge (Robust) | RKO (Björck) | 68.1 | 66.8 | 64.8 | 65.3 |
| ConvLarge (Robust) | AOC | 67.0 | 65.2 | 64.2 | 64.1 |
To substantiate our scalability claims, we profiled the peak GPU memory and runtime on our deepest model (VGG16 on Imagenette). Standard bound propagation physically cannot scale to this depth, instantly triggering an Out-Of-Memory (OOM) error by attempting to allocate over 2,048 GB of VRAM on an 80GB GPU. This confirms the fundamental memory bottleneck described in our theoretical analysis. In contrast, our hybrid framework successfully bypasses this barrier, completing the verification with a peak memory of only 0.62 GB and an average time of 121.59 seconds per sample. We will include these results in the appendix of the final version.
We agree that ablating the backbone-head split provides important context for the scalability of our hybrid framework. We evaluated varying depths for the unconstrained classification head and present the results for VGG13 architecture (CIFAR-10). If we restrict the head to only 1 linear layer, the lack of non-linearities means SDP-CROWN provides no verification advantage over standard last-layer normalization (e.g., as seen in LiResNet). Conversely, if we detach the Lipschitz constraint earlier (3 or 4 free layers), the robustness property seems to vanish. As shown in the table below, even with high weight decay regularization (wd0.5), the bounds become too loose for the verifier to recover robustness, causing both analytical CRA and formal VRA to collapse to near zero. Therefore, the 2-layer unconstrained head configuration in our main experiments (Section 6.4) represents a good structural balance: it contains sufficient non-linearity to leverage SDP-CROWN's localized precision, while maintaining the global Lipschitz constraint deep enough to preserve verifiable robustness. We observed the same phenomenon on Imagenette models, and we will include the ablation tables for both datasets in the final version.
Table D
| Head Depth (Unconstrained Layers) | Weight Decay (wd) | Clean Accuracy (%) | Analytical CRA (%) | Formal VRA (%) |
|---|---|---|---|---|
| 3 Layers | 1.0 | 84.5 | 0.0 | 11.5 |
| 3 Layers | 0.5 | 85.0 | 0.0 | 4.5 |
| 4 Layers | 1.0 | 85.5 | 0.0 | 0.0 |
| 4 Layers | 0.5 | 86.5 | 0.0 | 0.0 |
To further demonstrate the stability and reliability of our hybrid certification approach, we present the statistical results for 3 VGG13 models on Imagenette below. The variance results for the remaining hybrid evaluations, specifically VGG13 on CIFAR-10 and VGG16 on Imagenette, will be added in the appendix of the final version.
Table E
| Model (Setting) | Clean Acc (%) | CRA (%) | Hybrid VRA (%) |
|---|---|---|---|
| VGG13 (Hybrid, wd=0.5) | 87.5 ± 1.0 | 69.8 ± 4.9 | 73.2 ± 4.0 |
| VGG13 (Hybrid, wd=0.1) | 88.5 ± 0.5 | 37.5 ± 1.3 | 63.0 ± 3.0 |
(W6) We agree that an radius of 8/255 represents a different radius than the conventional 36/255 in the Lipschitz community. We selected this specific radius to ensure a direct comparison with the experimental setup used in the SDP-CROWN baseline (Chiu et al., 2025). We also evaluated the models at a larger radius of 36/255, please refer to Table C in the answer to Reviewer 7SYd. However, the shallow architectures standard to formal verification benchmarks lack the capacity to maintain a meaningful accuracy trade-off at this larger perturbation level. We will revise the introduction to clarify that our 8/255 evaluation establishes a standardized verification baseline. We will reframe our current methods as a necessary algorithmic foundation for scaling toward future safety-critical deployments.
[1] Delattre et al., Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration
[2] Leino et al., Globally-Robust Neural Networks
[3] Béthune et al., Pay Attention to your Loss
[4] Tsuzuku et al., Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
[5] Boissin et al., An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures
[6] Li et al., Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networks
We thank Reviewer geKM for their careful evaluation of our paper and for the thoughtful comments that have helped us clarify our contributions.
(W1) The Proposition 3.1 conclusion can indeed be clarified. We will apply this modification in the final version, clearly stating that “orthogonal expansion does not guarantee global gradient-norm preservation”. As Reviewer 7SYd mentioned, we will also acknowledge the prior result in [6].
(W2) We thank the reviewer for pointing out this incorrect statement. The sentence should have read "higher temperatures allow smaller logit margins" and we will revise accordingly. More precisely, in case the surrounding paragraph was not sufficiently clear, at low temperatures, achieving high confidence requires larger unscaled logit margins, encouraging the network to push decision boundaries away from the training data. Conversely, higher temperatures allow smaller unscaled logit margins, reducing this margin-maximizing pressure and potentially leading to more complex decision boundaries that weave and fold around local data clusters (Figure 1b), particularly when the gradient norm is capped at one (Figure 5). We will also clarify that this interpretation is intended as an intuitive explanation of the empirical observations rather than a formal theoretical consequence.
(W3) We agree that, formally, Power Iteration (PI) does not provide a guaranteed upper bound on the spectral norm. However, since the final Lipschitz constant estimation is performed only once, we can afford a large number of iterations to obtain an accurate approximation, as is common practice in the literature [2]. We have also cross-checked the estimates using SVD or Gram iteration (which provide an upper bound) [1] to assess their numerical accuracy. We will add a clarification in the revised manuscript to explain this validation procedure and support the reliability of the reported results.
(W4) We agree that it is not stated clearly in the paper that we consider the Pareto front associated with a given architecture. This aspect will be clarified in the final version.