- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The present manuscript explores the extent to which the curse of dimensionality (CoD) applies to the newly developed Kolmogorov-Arnold networks (KANs). In particular, the authors ask whether the previously reported favorable approximation scaling laws of KANs come from the architecture itself or from the structure of the example datasets used to obtain them. The authors provably obtain a worst-case CoD lower bound for spline-based KANs by transforming KANs into equivalent MLPs using a known construction. Additionally, they conduct numerical experiments comparing KANs and MLPs on structured target functions and random-label classification, for demonstrating that their performance depends on the target structure.
Critical strengths
The controlled ridge-profile experiments are an interesting aspect of the manuscript. By independently varying the dimension, number of ridge components, nonlinear profile, interaction support, the authors provide a systematic way to investigate how architectural inductive biases interact with the structure of target functions. I believe that this experimental methodology could be broadly useful for evaluating the inductive biases of several other neural network architectures.
Critical weaknesses
The primary theoretical contribution combines established facts (KAN-MLP construction, known bounds, Sobolev construction) to obtain a rigorous but limited-in-scope result. The authors themselves note that the result does not apply outside of standard spline-based KANs. In the past year, KANs have been generalized extensively (Fourier KANs, wavelet KANs etc.), including many that are modest extensions of the original architecture itself (e.g., alternate activation functions (SiLU) and multiplication nodes). Due to this, the direct implications of this result are limited and not sufficiently general. A discussion of which generalizations might admit analogous theoretical guarantees would strengthen the paper.
Moreover, the worst-case Sobolev lower bound does not directly challenge the previously observed favorable approximation rates for targets with sufficiently smooth KAN representations, since the two results concern different function classes. As a result, the extent to which the theoretical result changes our understanding of KANs' advantages on scientific problems remains somewhat unclear.
Questions for authors
- Could the authors clarify what makes the extension to arbitrary-depth KANs and dimension-uniform constants nontrivial?
- Could the authors point out whether the obstacles to extending the argument to more general KAN architectures are fundamental, or primarily limitations of the particular KAN-to-MLP technique used in the proof?
3 comments
Response to Reviewer xvTB (2/3)
W1 / Q2: Possible extensions to other KAN variants
Although our theorem is established for spline-only KANs, its proof suggests two steps that may extend to other variants:
- KAN-to-network representation. Convert the variant into a tractable network class while explicitly controlling the full parameter overhead. For spline-KANs, the exact -MLP conversion [5], together with our parameter accounting, yields , uniformly over depth and edge-wise knots. For other variants, the representing activation class and conversion cost may differ.
- Theoretical lower-bound transfer. Combine a suitable capacity bound (such as a VC-dimension upper bound) for the representing class with a dimension-uniform Sobolev shattering argument. The approximation threshold and constants must be controlled independently of .
This separates limitations of our particular conversion from potentially more fundamental capacity differences.
KAN variants.
- Spline and polynomial variants. Spline-only EfficientKAN is already covered by our analysis. P1-KAN admits an explicit ReLU-network representation [9], while ReLUKAN and fixed-order HRKAN may admit exact constructions through truncated-power representations. Pure polynomial-basis variants such as ChebyKAN and LegendreKAN can be represented using polynomial identities and exact multiplication constructions. Thus, our proof strategy may extend to these variants, provided the conversion overhead and dimension-uniform capacity bounds are appropriately controlled.
- Pfaffian-based extensions. Gaussian RBF and tanh-based variants, such as FastKAN/RBFKAN and FasterKAN, as well as Gaussian-based wavelets in WavKAN, may be analyzed using Pfaffian-network capacity bounds [8]. Extending our CoD analysis along this direction requires suitable dimension-uniform bounds and consideration of additional architectural components.
- Other variants requiring different tools. FourierKAN and SineKAN pose challenges due to potentially unbounded oscillatory complexity, even with fixed basis frequencies under unrestricted compositions. Other wavelet formulations require analysis depending on the mother wavelet, while rational variants such as rKAN may be amenable to semialgebraic VC-dimension methods. These extensions require further theoretical investigation.
Architectural modifications.
-
SiLU base branch. KANs with SiLU branches can be represented exactly by an enlarged sparse MLP with mixed /SiLU activations. However, our existing CoD proof does not directly apply, as its VC-dimension argument relies on piecewise-polynomial activations. Pfaffian-network techniques [8] offer a possible extension, but establishing the required dimension-uniform capacity bounds remains future work.
-
Multiplication nodes (MultKAN). For any fixed integer , binary multiplication admits an exact constant-size representation. Extending this construction to arbitrary MultKANs still requires controlling the whole-network conversion overhead and establishing a compatible capacity bound.
Thus, the proof strategy is not inherently restricted to spline-only KANs, but extending it requires specific constructions and suitable dimension-uniform complexity bounds.
Empirical evaluation with a SiLU branch. Our additional experiments show that an edge-wise SiLU branch can improve spline-KAN performance without changing our main conclusions from the comparison with MLP. On MNIST, adding this branch improves random-label fitting and slightly reduces standard-label test accuracy, while MLP retains higher accuracy on both tasks within . Within the new fixed-grid square-ridge scan (), SiLU lowers test MSE in four of five support settings at relative to spline-only KANs in the same scan. KANs still have an advantage at , while MLPs have lower error at tight budgets when . Detailed experimental settings and full results are provided in the newly added Appendices C.4 and C.5.1 (Figures 12 and 14) of the revised manuscript.
References are listed at the end of Part 3.
Response to Reviewer xvTB (3/3)
W2: Contribution and significance
As discussed in Q1, our Theorem 1 gives an explicit CoD lower bound for spline-only KANs on isotropic Sobolev classes, while earlier MLP results [3,4] establish favorable dimensional scaling on suitable compositional classes. These facts are consistent with the original conditional KAN theorem [1].
Our interpretation differs from the approximation-based motivation emphasized in [1,11]. These studies connect favorable KAN approximation rates to potential advantages in scientific applications. We argue that the favorable dimensional dependence reflects target-function structure and does not, by itself, explain an advantage of KANs over MLPs. We therefore propose investigating how target-function structure and optimization affect the relative performance of KANs and MLPs. Our controlled ridge experiments pursue this direction by examining how coordinate interactions affect the KAN–MLP comparison under finite parameter budgets.
Quantitatively ruling out a general escape from CoD for spline-only KANs is itself a contribution. It eliminates this general architectural explanation and helps focus subsequent research on the conditions and mechanisms that actually favor KANs in scientific applications.
If any questions remain unresolved, please feel free to raise them. We would be glad to address them.
References
[1] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, and Max Tegmark. KAN: Kolmogorov–Arnold Networks. arXiv:2404.19756v5, 2025. https://arxiv.org/abs/2404.19756v5
[2] Namjoon Suh and Guang Cheng. A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models. Annual Review of Statistics and Its Application, 12:177–207, 2025. https://doi.org/10.1146/annurev-statistics-040522-013920
[3] Hrushikesh N. Mhaskar and Tomaso Poggio. Deep vs. Shallow Networks: An Approximation Theory Perspective. Analysis and Applications, 14(6):829–848, 2016. https://doi.org/10.1142/S0219530516400042
[4] Hrushikesh Mhaskar, Qianli Liao, and Tomaso Poggio. When and Why Are Deep Networks Better Than Shallow Ones? AAAI, 2017. https://doi.org/10.1609/aaai.v31i1.10913
[5] Yixuan Wang, Jonathan W. Siegel, Ziming Liu, and Thomas Y. Hou. On the Expressiveness and Spectral Bias of KANs. ICLR, 2025. https://arxiv.org/abs/2410.01803
[6] Silviu-Marian Udrescu, Andrew Tan, Jiahai Feng, Orisvaldo Neto, Tailin Wu, and Max Tegmark. AI Feynman 2.0: Pareto-Optimal Symbolic Regression Exploiting Graph Modularity. NeurIPS, 2020. https://proceedings.neurips.cc/paper/2020/hash/33a854e247155d590883b93bca53848a-Abstract.html
[7] Ziming Liu, Max Tegmark, Pingchuan Ma, Wojciech Matusik, and Yixuan Wang. Kolmogorov–Arnold Networks Meet Science. Physical Review X, 15:041051, 2025. https://doi.org/10.1103/4t7t-v19l
[8] Marek Karpinski and Angus Macintyre. Polynomial Bounds for VC Dimension of Sigmoidal and General Pfaffian Neural Networks. Journal of Computer and System Sciences, 54(1):169–176, 1997. https://doi.org/10.1006/jcss.1997.1477
[9] Nandi Schoots, Mattia Jacopo Villani, and Niels uit de Bos. Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks. AISTATS (PMLR 258:199–207), 2025. https://proceedings.mlr.press/v258/schoots25a.html
[10] Dmitry Yarotsky. Error Bounds for Approximations with Deep ReLU Networks. Neural Networks, 94:103–114, 2017. https://doi.org/10.1016/j.neunet.2017.07.002. Preprint first posted in October 2016: https://arxiv.org/abs/1610.01145.
[11] Salah A. Faroughi, Farinaz Mostajeran, Amin Hamed Mashhadzadeh, and Shirko Faroughi. Kolmogorov-Arnold Networks for Data-Driven, Physics-Informed, and Deep-Operator Learning: A Review, Synthesis, and New Analysis. Neural Networks, 200:108791, 2026. https://doi.org/10.1016/j.neunet.2026.108791
Response to Reviewer xvTB (1/3)
Dear Reviewer xvTB,
Thank you for your careful review and constructive comments. We appreciate your recognition of the controlled ridge-profile experiments as a systematic way to study how target-function structure affects the relative performance of KANs and MLPs. Below, we first clarify the difficulties in establishing the CoD result, then discuss possible extensions to other KAN variants and the significance of our findings.
Q1: Difficulties in establishing the KAN CoD result
We address three aspects: connecting existing results, comparing parameter counts, and examining the interpretation of favorable KAN approximation rates.
Connecting results across the literature. The first difficulty is connecting Wang et al.'s KAN–MLP representation [5] with Yarotsky's lower-bound argument [10]. KAN and Wang et al.'s work appeared in 2024 [1,5], whereas Yarotsky's paper was published in 2017 [10]. Recognizing this connection requires theoretical knowledge drawn from a broad span of time, as well as the ability to connect related research areas. We adapt Yarotsky's argument to ReLUk-MLPs and transfer the bound to spline-only KANs.
Technical difficulty. The additional technical difficulty is minor, as our proof mainly combines existing results. The main additional step is to compare KAN and MLP parameter counts (Lemma 3) and verify that the quadratic overhead preserves the exponential dependence on dimension in the lower bound.
Questioning a potentially misleading narrative. The original KAN paper [1] derives a dimension-independent approximation exponent under sufficiently smooth compositional representations and connects this result to KANs' potential advantages in scientific applications. Related motivations are also discussed in the review [11]. This line of argument can be misleading because it may suggest that the favorable dimensional dependence is an advantage specific to the KAN architecture. Comparing the target-function classes and structural assumptions underlying the KAN and MLP approximation results leads us to question this implication. Our view, stated in the Introduction, is:
Our view is supported by two complementary mathematical facts:
Together, these facts support our interpretation while remaining consistent with the original conditional KAN theorem. The controlled ridge experiments then examine which structures favor each architecture at finite budgets, helping interpret potential advantages on scientific problems with separability or sparse compositional structure [6,7].
References are listed at the end of Part 3.