- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates formal verification of Kolmogorov-Arnold Networks (KANs), whose learned spline-based edge functions are not directly supported by mainstream neural network verification tools. The authors adapt several established verification techniques, including IBP, CROWN, α/β-CROWN, MILP, and star-set reachability, to KANs.
The paper establishes exact reachability for degree-1 KANs without base branches and sound over-approximation for higher-degree KANs. Experiments cover 26 trained networks across adversarial robustness, output ranges, monotone ordering, reachability, and control-barrier certification.
Critical strengths
-
Comprehensive verification framework. The paper systematically adapts multiple verification techniques to KANs, with useful theoretical results on knot-cell partitioning, exact reachability, and sound spline relaxations.
-
Diverse applications. The evaluation goes beyond adversarial robustness to include scientific and control applications. In particular, verification of all three barrier conditions for trained KAN-based barrier and controller networks is compelling.
Critical weaknesses
-
Limited methodological novelty: Much of the proposed methodology appears to adapt established neural network verification techniques to KANs. In particular, CROWN-style linear relaxations, optimized slopes, and branch-and-bound have already been extended to general nonlinear operators, as in auto_LiRPA and GenBaB. Similarly, exact verification of degree-1 KANs follows naturally from their piecewise-affine structure. While knot-aware partitioning and spline-tail handling are useful, it is unclear whether they constitute a sufficiently substantial algorithmic advance beyond existing general nonlinear verification frameworks.
-
Missing comparison with general nonlinear verifiers. The paper compares its native KAN-specific α/β-CROWN implementations against released CROWN tools applied to ReLU-converted networks, but not against a general nonlinear verifier equipped with B-spline relaxations. Given that frameworks such as GenBaB already handle general nonlinear operators, this comparison is important for establishing whether the proposed KAN-specific methods provide benefits beyond adding support for a new operator. The current experiments do not clearly establish this distinction.
-
Limited scalability evidence: The high-dimensional experiments primarily use shallow MNIST and Fashion-MNIST networks. While these demonstrate scalability in input dimension, they provide limited evidence of scalability to deeper nonlinear compositions. Given the paper's motivation from scientific and control applications, experiments on deeper, more realistic KANs would be more convincing.
Rebuttal
The rebuttal has addressed most of my concerns and therefore I am increasing my score.
Questions for authors
-
What are the key algorithmic differences between the proposed KAN-specific β-CROWN and existing general nonlinear verification methods such as GenBaB?
-
Why is there no direct comparison against a general nonlinear verifier equipped with certified B-spline relaxations?
-
Can the authors provide additional depth-scaling experiments, preferably on more challenging scientific or control applications?
-
In which regimes does native KAN verification consistently outperform ReLU-conversion-based approaches?
-
How are formal soundness guarantees maintained under floating-point arithmetic and numerical solver tolerances?
8 comments
W2 and Q2, comparison with a general nonlinear verifier.
We have added the comparison (App. C.15, Table 15): our native -CROWN and -CROWN, the KAN implementations of Sec. 5.3 that use our structure directly, against the released -CROWN with GenBaB, in double precision, on 538 degree-3 queries of Table 2 under the same 60 s end-to-end budget, once on an exact rewriting of the KAN in operators the tool supports (truncated-power form) and once with a B-spline operator that carries our certified parallel relaxation and leaves the branching to GenBaB (with timeouts in parentheses):
| Group (#, ceiling) | native -CROWN | native -CROWN, default / anytime | GenBaB, truncated-power form | GenBaB + B-spline operator |
|---|---|---|---|---|
| rob., 2–30 inputs (240, 224) | 223 | 224 / 224 | 136 (81) | 224 |
| rob., 784 inputs (48, 37) | 21 (16) | 21 (16) / 37 | 22 (15) | 37 |
| CS, boxes (250, 250) | 214 | 250 / 250 | 196 (50) | 229 [227] |
| all (538, 511) | 458 (16) | 495 (16) / 511 | 354 (146) | 490 [488] |
Group: # is the number of queries or boxes, and the ceiling the number without a re-executed counterexample, the most any method can prove. Square Brackets: without the rounding allowance of the operator's offsets.
On the truncated-power form GenBaB proves 354 of 511 (146 budget hits and 11 errors of the tool, counted as not proved); with the B-spline operator it proves 490 (which carries our certified relaxation, the contribution of this paper), matching native -CROWN on every row except the pykan enclosure (15 of 36 boxes against 36). The anytime configuration computes the CROWN bound first and spends the remaining budget on branch and bound; it proves all 16 depth-2 MNIST queries, where the default root bound exceeds the budget.
W3 and Q3, depth and control or scientific KANs.
The native -CROWN and -CROWN members now support any depth (Sec. 5.3, App. C.14), and Sec. 6 adds a "Depth" paragraph with Table 3 (details per member in App. C.14, Table 13) on 18 networks trained by us: MNIST classifiers, a cart-pole controller imitating a saturated LQR controller (actuator bound and one-step closed-loop reach on 162 boxes) and a Lorenz-63 KAN-ODE vector field (one-step reach on 64 boxes) at depth 2, 3 and 4 in both the degree-1 and the degree-3 regime, against the released -CROWN on the ReLU conversion, under the same 60 s end-to-end budget and at twice that budget (queries proved):
Degree-3+SiLU regime
| Network | depth | architecture | ceiling | best native member, 60 s / 120 s | best converted (released -CROWN), 60 s / 120 s |
|---|---|---|---|---|---|
| MNIST classifier | 2 | [784,32,10] | 100 | 100 / 100 | 99 / 99 |
| 3 | [784,32,16,10] | 95 | 92 / 92 | 55 / 55 | |
| 4 | [784,32,32,16,10] | 94 | 89 / 90 | 10 / 10 | |
| cart-pole controller | 2 | [4,16,1] | 161 | 161 / 161 | 155 / 155 |
| 3 | [4,8,8,1] | 157 | 124 / 129 | 140 / 140 | |
| 4 | [4,8,8,8,1] | 160 | 57 / 66 | 127 / 133 | |
| Lorenz-63 KAN-ODE | 2 | [3,16,3] | 64 | 64 / 64 | 22 / 22 |
| 3 | [3,8,8,3] | 63 | 58 / 63 | 4 / 4 | |
| 4 | [3,8,8,8,3] | 63 | 2 / 2 | 0 / 0 |
Degree-1 no-base regime
| Network | depth | architecture | ceiling | best native member, 60 s / 120 s | best converted (released -CROWN), 60 s / 120 s |
|---|---|---|---|---|---|
| MNIST classifier | 2 | [784,32,10] | 65 | 63 / 64 | 63 / 65 |
| 3 | [784,32,16,10] | 77 | 38 / 42 | 42 / 60 | |
| 4 | [784,32,32,16,10] | 65 | 0 / 0 | 26 / 41 | |
| cart-pole controller | 2 | [4,16,1] | 161 | 161 / 161 | 154 / 156 |
| 3 | [4,8,8,1] | 162 | 131 / 137 | 102 / 108 | |
| 4 | [4,8,8,8,1] | 158 | 9 / 9 | 0 / 0 | |
| Lorenz-63 KAN-ODE | 2 | [3,16,3] | 61 | 61 / 61 | 61 / 61 |
| 3 | [3,8,8,3] | 56 | 56 / 56 | 53 / 53 | |
| 4 | [3,8,8,8,3] | 47 | 28 / 47 | 40 / 43 |
Bold: the larger of each pair at each budget.
The released -CROWN, the overall winner of VNN-COMP 2024 and 2025, is a mature and highly optimized verifier, so this comparison favors the conversion; even so, at 60 s (Table 3) the best native member proves at least as many queries on 13 of the 18 networks and more on 11. For the 120 s entries (App. C.14, Table 14), every query left undecided at 60 s is run again with twice the budget. Where the conversion leads, the native member gains more on the degree-3 cart-pole networks, which narrows the lead from 16 to 11 queries at depth 3 and from 70 to 67 at depth 4, and on the degree-1 Lorenz-63 network at depth 4, where it overtakes the conversion and reaches the ceiling (47 against 43); on the degree-1 MNIST networks the conversion gains more (35 queries against 5). At 120 s the best native member proves at least as many queries on 13 networks and more on 12.
W1 and Q1, novelty and the differences from GenBaB.
Thank you for the detailed review, particularly highlighting the comparison with general nonlinear verifiers and the need for deeper networks. We answer each question with new experiments; added text is blue in the revised PDF.
We agree that the bounding and branching algorithms are those of α,β-CROWN and GenBaB and we now mention this in Sec. 5.3, "Relation to branch and bound for general nonlinearities". The contribution is the structure that these algorithms need in order to work on a KAN, together with the proofs that it is sound and, at degree 1, exact and complete. We believe there is also novelty as it can be used as a future VNN-COMP benchmark.
The new experiment E24 measures that structure inside α,β-CROWN, by giving GenBaB our relaxation as a B-spline operator and changing aspects:
Our contribution therefore goes beyond running GenBaB on a KAN: adding our certified B-spline relaxation to GenBaB proves 136 more of the 511 provable queries (354 → 490), whereas changing GenBaB's branching choices moves the count by at most 4. The other rows add guarantees rather than speed: exact and complete verification at degree 1, soundness outside the spline grid, and the barrier condition through the derivative network.
Sec. 5.3 ("Relation to branch and bound for general nonlinearities") states this comparison in the main text, and App. C.15 (Table 15) gives the full results.