acceptodds
Under review as a conference paper at ICLR 2027

Kolmogorov–Arnold Networks: Curse of Dimensionality and Structural Bias

Abstract

Kolmogorov–Arnold networks (KANs) have a dimension-independent approximation exponent when the target admits a sufficiently smooth representation by univariate edge functions. We ask whether this favorable scaling belongs to the architecture or to the assumed target class. For arbitrary-depth standard spline-KANs, we prove a worst-case lower bound on isotropic Sobolev balls: below a dimension-independent accuracy threshold, the required number of spline coefficients grows at least exponentially with input dimension. Existing composition-matched MLP results, by contrast, show that favorable exponents on low-arity targets are not exclusive to KANs. We also revisit empirical comparisons. On four scaling-law targets, spline-KAN approximants constructed from known decompositions transfer exactly to MLPs with the cubic ReLU activation , although their full dense parameter counts are larger. In controlled ridge-profile scans, KANs have a strong advantage when each nonlinear component depends on one input coordinate. When a component couples multiple input coordinates, this advantage may narrow sharply. At tight budgets, interacting targets often favor MLPs. In the random-label test on MNIST, ReLU-MLPs fit the randomized training labels more efficiently than spline-only KANs. Together, these results locate favorable scaling in target structure and a compatible inductive bias, not a general KAN-specific escape from the curse of dimensionality.

Primary area:
learning theory
Keywords:
Kolmogorov--Arnold networks (KANs) have a dimension-independent approximation exponent when the target admits a sufficiently smooth representation by univariate edge functions. We ask whether this favorable scaling belongs to the architecture or to the assumed target class. For arbitrary-depth standard spline-KANs, we prove a worst-case lower bound on isotropic Sobolev balls: below a dimension-independent accuracy threshold, the required number of spline coefficients grows at least exponentially with input dimension. Existing composition-matched MLP results, by contrast, show that favorable exponents on low-arity targets are not exclusive to KANs. We also revisit empirical comparisons. On four scaling-law targets, spline-KAN approximants constructed from known decompositions transfer exactly to MLPs with the cubic ReLU activation $\sigma(x)=(\max\{0, x\})^3$, although their full dense parameter counts are larger. In controlled ridge-profile scans, KANs have a strong advantage when each nonlinear component depends on one input coordinate. When a component couples multiple input coordinates, this advantage may narrow sharply. At tight budgets, interacting targets often favor MLPs. In the random-label test on MNIST, ReLU-MLPs fit the randomized training labels more efficiently than spline-only KANs. Together, these results locate favorable scaling in target structure and a compatible inductive bias, not a general KAN-specific escape from the curse of dimensionality.

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_xvTBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The present manuscript explores the extent to which the curse of dimensionality (CoD) applies to the newly developed Kolmogorov-Arnold networks (KANs). In particular, the authors ask whether the previously reported favorable approximation scaling laws of KANs come from the architecture itself or from the structure of the example datasets used to obtain them. The authors provably obtain a worst-case CoD lower bound for spline-based KANs by transforming KANs into equivalent MLPs using a known construction. Additionally, they conduct numerical experiments comparing KANs and MLPs on structured target functions and random-label classification, for demonstrating that their performance depends on the target structure.

Critical strengths

The controlled ridge-profile experiments are an interesting aspect of the manuscript. By independently varying the dimension, number of ridge components, nonlinear profile, interaction support, the authors provide a systematic way to investigate how architectural inductive biases interact with the structure of target functions. I believe that this experimental methodology could be broadly useful for evaluating the inductive biases of several other neural network architectures.

Critical weaknesses

The primary theoretical contribution combines established facts (KAN-MLP construction, known bounds, Sobolev construction) to obtain a rigorous but limited-in-scope result. The authors themselves note that the result does not apply outside of standard spline-based KANs. In the past year, KANs have been generalized extensively (Fourier KANs, wavelet KANs etc.), including many that are modest extensions of the original architecture itself (e.g., alternate activation functions (SiLU) and multiplication nodes). Due to this, the direct implications of this result are limited and not sufficiently general. A discussion of which generalizations might admit analogous theoretical guarantees would strengthen the paper.

Moreover, the worst-case Sobolev lower bound does not directly challenge the previously observed favorable approximation rates for targets with sufficiently smooth KAN representations, since the two results concern different function classes. As a result, the extent to which the theoretical result changes our understanding of KANs' advantages on scientific problems remains somewhat unclear.

Questions for authors
  1. Could the authors clarify what makes the extension to arbitrary-depth KANs and dimension-uniform constants nontrivial?
  2. Could the authors point out whether the obstacles to extending the argument to more general KAN architectures are fundamental, or primarily limitations of the particular KAN-to-MLP technique used in the proof?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xvTB (1/3)

Dear Reviewer xvTB,

Thank you for your careful review and constructive comments. We appreciate your recognition of the controlled ridge-profile experiments as a systematic way to study how target-function structure affects the relative performance of KANs and MLPs. Below, we first clarify the difficulties in establishing the CoD result, then discuss possible extensions to other KAN variants and the significance of our findings.

Q1: Difficulties in establishing the KAN CoD result

We address three aspects: connecting existing results, comparing parameter counts, and examining the interpretation of favorable KAN approximation rates.

Connecting results across the literature. The first difficulty is connecting Wang et al.'s KAN–MLP representation [5] with Yarotsky's lower-bound argument [10]. KAN and Wang et al.'s work appeared in 2024 [1,5], whereas Yarotsky's paper was published in 2017 [10]. Recognizing this connection requires theoretical knowledge drawn from a broad span of time, as well as the ability to connect related research areas. We adapt Yarotsky's argument to ReLU⁡k\operatorname{ReLU}^{k}-MLPs and transfer the bound to spline-only KANs.

Technical difficulty. The additional technical difficulty is minor, as our proof mainly combines existing results. The main additional step is to compare KAN and MLP parameter counts (Lemma 3) and verify that the quadratic overhead preserves the exponential dependence on dimension in the lower bound.

Questioning a potentially misleading narrative. The original KAN paper [1] derives a dimension-independent approximation exponent under sufficiently smooth compositional representations and connects this result to KANs' potential advantages in scientific applications. Related motivations are also discussed in the review [11]. This line of argument can be misleading because it may suggest that the favorable dimensional dependence is an advantage specific to the KAN architecture. Comparing the target-function classes and structural assumptions underlying the KAN and MLP approximation results leads us to question this implication. Our view, stated in the Introduction, is:

Together, these observations suggest that the favorable dimension dependence reflects the structure of the target class rather than a general approximation advantage of KANs.

Our view is supported by two complementary mathematical facts:

  1. On broad function classes, spline-only KANs still suffer from CoD. Our Theorem 1 gives an explicit parameter lower bound for uniform approximation of isotropic Sobolev balls, quantifying the general obstruction acknowledged in [1] and complementing the broader approximation-theoretic discussion in [2].
  2. On suitable compositional classes, favorable dimensional scaling is also attainable by MLPs. Section 2.1 discusses earlier MLP results [3,4] whose approximation exponents depend on local arity rather than ambient dimension. These are existing results that we bring into the KAN–MLP comparison. Section 5.1 further shows that our constructed spline-KAN approximants transfer exactly to MLPs, preserving their represented functions, although the dense parameter counts can be larger.

Together, these facts support our interpretation while remaining consistent with the original conditional KAN theorem. The controlled ridge experiments then examine which structures favor each architecture at finite budgets, helping interpret potential advantages on scientific problems with separability or sparse compositional structure [6,7].

References are listed at the end of Part 3.

AuthorsOpenReview ↗

Response to Reviewer xvTB (2/3)

W1 / Q2: Possible extensions to other KAN variants

Although our theorem is established for spline-only KANs, its proof suggests two steps that may extend to other variants:

  1. KAN-to-network representation. Convert the variant into a tractable network class while explicitly controlling the full parameter overhead. For spline-KANs, the exact ReLU⁡k\operatorname{ReLU}^{k}-MLP conversion [5], together with our parameter accounting, yields PMLP≤CkPKAN2P_{\mathrm{MLP}}\leq C_kP_{\mathrm{KAN}}^2, uniformly over depth and edge-wise knots. For other variants, the representing activation class and conversion cost may differ.
  2. Theoretical lower-bound transfer. Combine a suitable capacity bound (such as a VC-dimension upper bound) for the representing class with a dimension-uniform Sobolev shattering argument. The approximation threshold and constants must be controlled independently of dd.

This separates limitations of our particular ReLU⁡k\operatorname{ReLU}^{k} conversion from potentially more fundamental capacity differences.

KAN variants.

  • Spline and polynomial variants. Spline-only EfficientKAN is already covered by our analysis. P1-KAN admits an explicit ReLU-network representation [9], while ReLUKAN and fixed-order HRKAN may admit exact constructions through truncated-power representations. Pure polynomial-basis variants such as ChebyKAN and LegendreKAN can be represented using polynomial identities and exact multiplication constructions. Thus, our proof strategy may extend to these variants, provided the conversion overhead and dimension-uniform capacity bounds are appropriately controlled.
  • Pfaffian-based extensions. Gaussian RBF and tanh-based variants, such as FastKAN/RBFKAN and FasterKAN, as well as Gaussian-based wavelets in WavKAN, may be analyzed using Pfaffian-network capacity bounds [8]. Extending our CoD analysis along this direction requires suitable dimension-uniform bounds and consideration of additional architectural components.
  • Other variants requiring different tools. FourierKAN and SineKAN pose challenges due to potentially unbounded oscillatory complexity, even with fixed basis frequencies under unrestricted compositions. Other wavelet formulations require analysis depending on the mother wavelet, while rational variants such as rKAN may be amenable to semialgebraic VC-dimension methods. These extensions require further theoretical investigation.

Architectural modifications.

  • SiLU base branch. KANs with SiLU branches can be represented exactly by an enlarged sparse MLP with mixed ReLU⁡k\operatorname{ReLU}^{k}/SiLU activations. However, our existing CoD proof does not directly apply, as its VC-dimension argument relies on piecewise-polynomial activations. Pfaffian-network techniques [8] offer a possible extension, but establishing the required dimension-uniform capacity bounds remains future work.

  • Multiplication nodes (MultKAN). For any fixed integer k≥2k\geq 2, binary multiplication admits an exact constant-size ReLU⁡k\operatorname{ReLU}^{k} representation. Extending this construction to arbitrary MultKANs still requires controlling the whole-network conversion overhead and establishing a compatible capacity bound.

Thus, the proof strategy is not inherently restricted to spline-only KANs, but extending it requires specific constructions and suitable dimension-uniform complexity bounds.

Empirical evaluation with a SiLU branch. Our additional experiments show that an edge-wise SiLU branch can improve spline-KAN performance without changing our main conclusions from the comparison with MLP. On MNIST, adding this branch improves random-label fitting and slightly reduces standard-label test accuracy, while MLP retains higher accuracy on both tasks within P≤500,000P\le500{,}000. Within the new fixed-grid square-ridge scan (d=R=16d=R=16), SiLU lowers test MSE in four of five support settings at P=105P=10^5 relative to spline-only KANs in the same scan. KANs still have an advantage at s=1s=1, while MLPs have lower error at tight budgets when s>1s>1. Detailed experimental settings and full results are provided in the newly added Appendices C.4 and C.5.1 (Figures 12 and 14) of the revised manuscript.

References are listed at the end of Part 3.

AuthorsOpenReview ↗

Response to Reviewer xvTB (3/3)

W2: Contribution and significance

As discussed in Q1, our Theorem 1 gives an explicit CoD lower bound for spline-only KANs on isotropic Sobolev classes, while earlier MLP results [3,4] establish favorable dimensional scaling on suitable compositional classes. These facts are consistent with the original conditional KAN theorem [1].

Our interpretation differs from the approximation-based motivation emphasized in [1,11]. These studies connect favorable KAN approximation rates to potential advantages in scientific applications. We argue that the favorable dimensional dependence reflects target-function structure and does not, by itself, explain an advantage of KANs over MLPs. We therefore propose investigating how target-function structure and optimization affect the relative performance of KANs and MLPs. Our controlled ridge experiments pursue this direction by examining how coordinate interactions affect the KAN–MLP comparison under finite parameter budgets.

Quantitatively ruling out a general escape from CoD for spline-only KANs is itself a contribution. It eliminates this general architectural explanation and helps focus subsequent research on the conditions and mechanisms that actually favor KANs in scientific applications.

If any questions remain unresolved, please feel free to raise them. We would be glad to address them.


References

[1] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, and Max Tegmark. KAN: Kolmogorov–Arnold Networks. arXiv:2404.19756v5, 2025. https://arxiv.org/abs/2404.19756v5

[2] Namjoon Suh and Guang Cheng. A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models. Annual Review of Statistics and Its Application, 12:177–207, 2025. https://doi.org/10.1146/annurev-statistics-040522-013920

[3] Hrushikesh N. Mhaskar and Tomaso Poggio. Deep vs. Shallow Networks: An Approximation Theory Perspective. Analysis and Applications, 14(6):829–848, 2016. https://doi.org/10.1142/S0219530516400042

[4] Hrushikesh Mhaskar, Qianli Liao, and Tomaso Poggio. When and Why Are Deep Networks Better Than Shallow Ones? AAAI, 2017. https://doi.org/10.1609/aaai.v31i1.10913

[5] Yixuan Wang, Jonathan W. Siegel, Ziming Liu, and Thomas Y. Hou. On the Expressiveness and Spectral Bias of KANs. ICLR, 2025. https://arxiv.org/abs/2410.01803

[6] Silviu-Marian Udrescu, Andrew Tan, Jiahai Feng, Orisvaldo Neto, Tailin Wu, and Max Tegmark. AI Feynman 2.0: Pareto-Optimal Symbolic Regression Exploiting Graph Modularity. NeurIPS, 2020. https://proceedings.neurips.cc/paper/2020/hash/33a854e247155d590883b93bca53848a-Abstract.html

[7] Ziming Liu, Max Tegmark, Pingchuan Ma, Wojciech Matusik, and Yixuan Wang. Kolmogorov–Arnold Networks Meet Science. Physical Review X, 15:041051, 2025. https://doi.org/10.1103/4t7t-v19l

[8] Marek Karpinski and Angus Macintyre. Polynomial Bounds for VC Dimension of Sigmoidal and General Pfaffian Neural Networks. Journal of Computer and System Sciences, 54(1):169–176, 1997. https://doi.org/10.1006/jcss.1997.1477

[9] Nandi Schoots, Mattia Jacopo Villani, and Niels uit de Bos. Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks. AISTATS (PMLR 258:199–207), 2025. https://proceedings.mlr.press/v258/schoots25a.html

[10] Dmitry Yarotsky. Error Bounds for Approximations with Deep ReLU Networks. Neural Networks, 94:103–114, 2017. https://doi.org/10.1016/j.neunet.2017.07.002. Preprint first posted in October 2016: https://arxiv.org/abs/1610.01145.

[11] Salah A. Faroughi, Farinaz Mostajeran, Amin Hamed Mashhadzadeh, and Shirko Faroughi. Kolmogorov-Arnold Networks for Data-Driven, Physics-Informed, and Deep-Operator Learning: A Review, Synthesis, and New Analysis. Neural Networks, 200:108791, 2026. https://doi.org/10.1016/j.neunet.2026.108791

Reviewer_U9ZtOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper critiques the KAN architecture in high-dimensional settings. Its central argument is compelling: the dimension-independent approximation rate stems from specific structural assumptions about the target function. When these assumptions fail to hold, the KAN architecture cannot, in general, offer such guarantees.

Critical strengths
  1. The strongest contribution is not simply showing that a lower bound contains the dimension dd, but establishing a fixed-accuracy threshold independent of dd. Theorem 1 gives

PKAN≥c(ε0ε)d/(4n),P_{\mathrm{KAN}}\geq c\left(\frac{\varepsilon_0}{\varepsilon}\right)^{d/(4n)}, where ε0\varepsilon_0 is independent of dimension.

  1. The argument combines exact spline-to-ReLUk\mathrm{ReLU}^k realization, a parameter-count bound, and a shattering-based Sobolev lower bound.

  2. The paper appropriately distinguishes what a model can represent from what a training procedure actually discovers.

Critical weaknesses

The theoretical advance is narrower than the broad motivation suggests

The paper acknowledges that the original KAN work already recognized general Sobolev impossibility results. Its contribution is therefore a quantitative extension to a particular spline-only architecture, rather than a refutation of a previously established unconditional KAN theorem.

Questions for authors

How would including the SiLU base branch and adaptive grid refinement affect the reported KAN–MLP performance comparisons?

4 comments
AuthorsOpenReview ↗

Response to Reviewer U9Zt (1/2)

Dear Reviewer U9Zt,

Thank you for your careful review, thoughtful questions, and support for our work. We appreciate your recognition of the dimension-independent fixed-accuracy threshold and our distinction between what a model can represent and what training discovers. Below, we address your question about the effects of the SiLU branch and grid refinement, and clarify the scope and significance of our contribution.

Q1: Effects of the SiLU branch and grid refinement

The added experiments show that SiLU and grid refinement can improve KAN performance without changing our main conclusions from the comparison with MLP. Our primary comparison uses a common optimization protocol under controlled parameter budgets. For this revision, we additionally compare spline-only KANs, KANs with SiLU, KANs with grid refinement, and KANs with both, alongside the original MLP baseline. We progressively refine the grid by inserting new knots while preserving the existing knot locations.

1. MNIST classification and random-label fitting. SiLU and grid refinement improve random-label fitting, while MLP retains higher accuracy on both tasks within P≤500,000P\le500{,}000. For standard labels, SiLU alone slightly reduces test accuracy, whereas the combined variant performs close to the spline-only baseline. Within this budget, the table reports the highest mean accuracy across architectures. Each mean is computed over three seeds, after selecting the optimizer and learning rate by final training accuracy for each architecture and seed.

ModelStandard labels: test accuracy ↑Random labels: training accuracy ↑
Spline-only KAN97.27%20.06%
KAN + SiLU96.94%34.77%
KAN + refinement85.46%63.62%
KAN + SiLU + refinement97.33%89.55%
ReLU-MLP98.54%100.00%

Architecture search ranges differ across variants. Detailed experimental settings and the full budget curves are provided in the newly added Appendix C.5.1 and Figure 14 of the revised manuscript.

2. Square-ridge fitting. SiLU and grid refinement can improve KAN fitting, but the main KAN--MLP comparison remains unchanged. In our added square-ridge experiments (φ(t)=t2\varphi(t)=t^2, d=R=16d=R=16), KANs retain a strong advantage on decoupled targets, while interacting targets still exhibit a budget-dependent crossover, with MLPs favored at tight budgets. The gains from SiLU, refinement, or their combination depend on support size and parameter budget. Detailed experimental settings and the full results are provided in the newly added Appendix C.4 and Figure 12 of the revised manuscript.

References are listed at the end of Part 2.

AuthorsOpenReview ↗

Response to Reviewer U9Zt (2/2)

W1: Scope and significance of our results

We agree with the reviewer that the original KAN paper recognizes general Sobolev approximation limits and that our theorem concerns spline-only KANs. We would like to clarify the scope and significance of our results through the following three points.

1. We provide an explicit lower bound for a previously recognized limitation. The original paper [1, Section 2.3] acknowledges that nonlinear nn-width theory prevents a general escape from CoD on Sobolev spaces. Our Theorem 1 establishes an explicit parameter-complexity lower bound for arbitrary-depth standard spline-only KANs on isotropic Sobolev balls, proving exponential growth with the input dimension at any fixed accuracy below a dimension-independent threshold. The proof combines the exact spline-KAN-to-ReLU⁡k\operatorname{ReLU}^{k}-MLP representation [4] with an explicit bound on the full dense parameter overhead and a dimension-uniform Sobolev shattering argument.

2. We would like to note that the original KAN approximation theorem is not unconditional. Specifically, Theorem 2.1 of [1, Section 2.3] requires the target function to admit a Kolmogorov–Arnold (KA) representation in which every univariate component is (k+1)(k+1)-times continuously differentiable. Its favorable approximation exponent therefore applies to targets satisfying this structural and smoothness assumption. The original paper connects these rates to potential advantages in scientific applications, with related motivations also discussed in the review [5]. Our Sobolev lower bound is consistent with these conditional guarantees.

3. Our argument concerns the source of favorable dimensional scaling and its implications for comparing KANs with MLPs. Our analysis supports a different interpretation: favorable rates on these structured target classes do not by themselves establish a general approximation advantage of KANs over MLPs. Our Introduction explicitly states (excerpts):

First, KANs suffer from CoD on harder classes. […]

Second, low-arity compositional classes are easy enough for MLPs to beat CoD as well. […]

Together, these observations suggest that the favorable dimension dependence reflects the structure of the target class rather than a general approximation advantage of KANs.

The explicit lower bound above establishes the first point. For the second, Section 2.1 reviews earlier MLP results [2,3] whose approximation exponents depend on local arity rather than ambient dimension. Section 5.1 further constructs spline-KAN approximants and transfers them exactly to ReLU⁡3\operatorname{ReLU}^3 MLPs, preserving the represented functions, although the dense MLP parameter counts can be larger.

As discussed in Q1, the added SiLU and refinement experiments show that these extensions can improve KAN performance without changing the main conclusions of our comparison with MLP. Detailed settings and results are provided in the newly added Appendices C.4 and C.5.1 of the revised manuscript.

If any questions remain unresolved, please feel free to raise them. We would be glad to address them.


References

[1] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, and Max Tegmark. KAN: Kolmogorov–Arnold Networks. arXiv:2404.19756v5, 2025. https://arxiv.org/abs/2404.19756v5

[2] Hrushikesh N. Mhaskar and Tomaso Poggio. Deep vs. Shallow Networks: An Approximation Theory Perspective. Analysis and Applications, 14(6):829–848, 2016. https://doi.org/10.1142/S0219530516400042

[3] Hrushikesh Mhaskar, Qianli Liao, and Tomaso Poggio. When and Why Are Deep Networks Better Than Shallow Ones? Proceedings of the AAAI Conference on Artificial Intelligence, 31(1):2343–2349, 2017. https://doi.org/10.1609/aaai.v31i1.10913

[4] Yixuan Wang, Jonathan W. Siegel, Ziming Liu, and Thomas Y. Hou. On the Expressiveness and Spectral Bias of KANs. ICLR, 2025. https://arxiv.org/abs/2410.01803

[5] Salah A. Faroughi, Farinaz Mostajeran, Amin Hamed Mashhadzadeh, and Shirko Faroughi. Kolmogorov-Arnold Networks for Data-Driven, Physics-Informed, and Deep-Operator Learning: A Review, Synthesis, and New Analysis. Neural Networks, 200:108791, 2026. https://doi.org/10.1016/j.neunet.2026.108791

I thank the authors for their thorough response. I remain in support of this paper and keep my score.

Thank you very much for your continued support and for taking the time to review our responses. We greatly appreciate your thoughtful feedback.

Reviewer_YTnSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The text discusses the Curse of Dimensionality with respects to Kolmogorov-Arnold networks (KANs). The authors summarize, that earlier reported favorable parameter scaling with dimensions in KANs is due to the structure of used test functions. The text first discusses that on generic smooth functions, KANs need exponentially many parameters but a target built from smooth univariate pieces leads to favorable scaling; not only for KANs but for simple MLPs as well. The authors derive a formal lower and upper bound for a worst case parameter scaling. Alongside other empirical evaluation, experiments show that entangled feature dimensions are harder to learn for KANs than for MLPs.

Critical strengths

Finding of formal bounds and controlled experiments

The paper contributes an important formal finding regarding the limits of KANs, specifically in their required parameters with data dimensionality. While it has been acknowledged before, that KANs do not escape the Curse of Dimensionality, actual bounds were, to my knowledge, not established. Moreover, in their controlled ridge experiment which isolates specific structural properties, the authors highlight the origin of a possible KAN advantage. This makes the paper a solid contribution to the community serving as a backbone for future work.

Presentation

The paper is very comprehensively written and to a great degree self-containing. The theory developed in the main text is enough to follow the contribution's idea, is not overstraining and can be checked for further detail in the appendix. Thereby, the paper's findings are easily accessible to the community.

Critical weaknesses

Simplification of default KANs

As the authors mention as limitations, their theorem covers spline-only KANs and a SiLU\mathrm{SiLU} contribution is excluded from the paper's treatment. The latter, as far as I can tell, is default in the implementation of KANs. The bridge translates KAN edge functions to ReLUk\mathrm{ReLU}^k units but as SiLU\mathrm{SiLU} is not piecewise polynomial, the developed theory does not natively hold for the default KANs that do have a SiLU\mathrm{SiLU} contribution.

Recommendation

I would rate this submission with a weak accept (3). Most important for my recommendation is the theoretically sound and empirically backed finding of the limits of KANs required parameter scaling with dimensionality, and their benign scaling's dependence on target structure. Apart from my stated weakness (the seeming departure from the default KAN formulation), I have not rated the paper higher due to my difficulty to assess the paper result's importance. More specifically, and as I am neither an expert in the usage of KANs nor approximation theory of neural networks, it is hard for me to gauge how meaningful the bounds are on the theory side, but also numerically and if applied to actual network architectures.

Questions for authors
  • Alluding to my stated weakness: To what degree do both theory and experiment findings extend to the default KAN implementation with a SiLU\mathrm{SiLU} term?

  • As far as I understand, the theory also covers grid point updates. In the empirical evaluation, however, I do not see the usage of grid extension and refinements mentioned. The original KAN paper (Liu et al., ICLR 2025) seemed to make use of a grid refinement schedule for their scaling experiments. In what way, do the authors' experiments make use of grid adaptations?

  • Referring to my recommendation, can the authors discuss the non-vacuousness of their bounds and, thereby, their practical relevance?

7 comments
AuthorsOpenReview ↗

Response to Reviewer YTnS (1/3)

Dear Reviewer YTnS,

Thank you for your careful review, thoughtful questions, and support for our work. We appreciate your positive assessment of our theoretical bounds, controlled experiments, and presentation. Below, we address your questions about the SiLU branch, grid refinement, and the non-vacuousness and practical relevance of our bounds.

W1 / Q1: SiLU in the default KAN implementation

We would like to clarify the role of SiLU in the original implementation first. The original KAN paper [1] states (here we refer to the arXiv v5 version, Section 2.2, p. 6):

Implementation details. Although a KAN layer Eq. (2.5) looks extremely simple, it is non-trivial to make it well optimizable. The key tricks are:

They then describe three implementation choices (paraphrased here):

  1. Add a residual base branch to each spline edge, ϕ(x)=wbb(x)+wsspline⁡(x)\phi(x)=w_b b(x)+w_s\operatorname{spline}(x), choosing b(x)=SiLU⁡(x)b(x)=\operatorname{SiLU}(x) in most cases.
  2. Initialize the spline contribution near zero, set ws=1w_s=1, and use Xavier initialization for wbw_b.
  3. Adjust spline grids during training according to the input activations so that the grids track their changing range.

The original paper [1] thus explicitly motivates the residual SiLU branch as part of the design for improving KAN optimization. This motivates restricting our theoretical analysis to spline-only KANs, whose edge functions contain only spline terms and no SiLU residual branch. We next discuss KANs with SiLU residual connections from both theoretical and empirical perspectives.

Theoretical scope. Our theorem concerns the spline-only KAN class, with the SiLU branch disabled. The mathematical distinction is consequential: fixed-degree spline edges are piecewise polynomial and admit exact ReLU⁡k\operatorname{ReLU}^{k} representations, whereas SiLU is not piecewise polynomial. Retaining SiLU therefore requires a capacity analysis beyond the piecewise-polynomial argument used in our proof. Accordingly, our CoD lower bound is established for arbitrary-depth spline-only KANs. Extending it to SiLU-augmented KANs requires separate analysis.

Empirical evaluation with SiLU. Adding SiLU can improve KAN performance, but our main conclusions from the comparison with MLP remain unchanged. On MNIST, it improves random-label fitting and slightly reduces standard-label test accuracy, while MLP retains higher accuracy on both tasks within P≤500,000P\le500{,}000. On square-ridge targets (φ(t)=t2\varphi(t)=t^2, d=R=16d=R=16), it lowers test MSE in four of the five support settings at P=105P=10^5 relative to spline-only KANs in the same new fixed-grid scan. KANs still have an advantage at s=1s=1, while MLPs have lower error at tight budgets when s>1s>1. Detailed experimental settings and full results are provided in the newly added Appendices C.4 and C.5.1 (Figures 12 and 14) of the revised manuscript.

References are listed at the end of Part 3.

AuthorsOpenReview ↗

Response to Reviewer YTnS (2/3)

Q2: Grid extension and refinement

Theoretical coverage. Grid extension increases the number of intervals GG, while grid refinement progressively transfers learned functions to finer grids [1, Sections 2.2 and 2.4]. The lower bound in our theorem applies to the final spline-only KAN obtained after grid extension or refinement, with complexity measured by its final parameter count. Although these procedures may change initialization and the optimization trajectory, the resulting network remains within the spline-only class covered by our theorem, which allows arbitrary edge-wise knot locations.

Why we omitted grid refinement. We omit progressive grid refinement from the primary comparison for three reasons: 1) Our goal is to compare the performance of KANs and MLPs under controlled parameter budgets. 2) We use a common optimization protocol, whereas spline-grid refinement has no direct counterpart in standard MLPs. 3) A multi-stage refinement schedule introduces additional choices, including grid sizes, stage lengths, and potentially stage-specific learning rates, that require a separate tuning protocol. Accordingly, we used a common optimization protocol for both MLPs and KANs in the original experiments: Adam for ridge fitting and the same SGD/Adam search space for MNIST.

Empirical evaluation with SiLU and grid refinement. Grid refinement, with or without SiLU, can improve KAN performance, but our main conclusions from the comparison with MLP remain unchanged. In the added MNIST experiments, refinement raises random-label training accuracy from 20.06%20.06\% to 63.62%63.62\% within P≤500,000P\le500{,}000, and combining it with SiLU reaches 89.55%89.55\%. MLP retains higher accuracy on both tasks within this budget, reaching 100%100\% on random labels. On square-ridge targets (φ(t)=t2\varphi(t)=t^2, d=R=16d=R=16), refinement improves fitting in some settings, with gains depending on support size and budget. KANs retain an advantage on decoupled targets, while MLPs have lower error on interacting targets at tight budgets. Detailed experimental settings and full results are provided in the newly added Appendices C.4 and C.5.1 (Figures 12 and 14) of the revised manuscript.

W2 / Q3: Non-vacuousness and practical relevance

1. Mathematical non-vacuousness. For the standard spline-KAN class in Theorem 1, uniformly approximating the isotropic Sobolev ball to accuracy 0<ϵ<ϵ00<\epsilon<\epsilon_0 requires

PKAN≥cˉ′(ϵ0ϵ)d/(4n),P_{\mathrm{KAN}}\geq \bar c'\left(\frac{\epsilon_0}{\epsilon}\right)^{d/(4n)},

where ϵ0,cˉ′>0\epsilon_0,\bar c'>0 are independent of the input dimension dd under the stated assumptions. For fixed smoothness nn and fixed ϵ<ϵ0\epsilon<\epsilon_0, the lower bound grows exponentially with dd. Thus, its dimension-independent constants establish an asymptotically non-vacuous worst-case CoD result.

2. Significance of our findings.

Our argument differs from the view that favorable KAN rates on compositional classes establish a general approximation advantage over MLPs. The original KAN paper [1] derives a dimension-independent approximation exponent under sufficiently smooth compositional representations and connects this result to KANs' potential advantages in scientific applications. Related motivations are also discussed in the review [3]. These conditional results motivate examining whether the favorable dimensional dependence is specific to the KAN architecture.

Our Introduction explicitly states (excerpts):

First, KANs suffer from CoD on harder classes. […]

Second, low-arity compositional classes are easy enough for MLPs to beat CoD as well. […]

Together, these observations suggest that the favorable dimension dependence reflects the structure of the target class rather than a general approximation advantage of KANs.

References are listed at the end of Part 3.

AuthorsOpenReview ↗

Response to Reviewer YTnS (3/3)

W2 / Q3: Non-vacuousness and practical relevance (continued)

We develop these two points as follows.

First, KANs still suffer from CoD on harder target classes. The original KAN paper [1] acknowledges a general obstruction on Sobolev spaces through nonlinear nn-width theory. Our Theorem 1 quantifies the worst-case parameter cost for arbitrary-depth spline-only KANs on isotropic Sobolev balls: below a dimension-independent accuracy threshold, the required parameter count grows at least exponentially with the input dimension. This provides a quantitative characterization of their limitations, complementing the broader discussion in [4].

Second, favorable approximation rates on suitable compositional classes are also attainable by MLPs. Section 2.1 reviews earlier results [5,6] in which approximation exponents depend on local arity rather than ambient dimension. Section 5.1 further shows that our constructed spline-KAN approximants can be transferred exactly to ReLU⁡3\operatorname{ReLU}^3 MLPs, preserving their represented functions, although the dense MLP parameter counts can be larger. Thus, the favorable dimensional scaling on these structured targets does not by itself establish a KAN-specific advantage.

Our controlled ridge experiments then examine the remaining question of which target structures favor each architecture at finite budgets. KANs perform particularly well when nonlinear components depend on individual input coordinates. Coupling multiple coordinates narrows this advantage, with the ranking also depending on the budget and target nonlinearity. These results help interpret potential advantages on scientific problems with separability or sparse compositional structure [7,8].

If any questions remain unresolved, please feel free to raise them. We would be glad to address them.


References

[1] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, and Max Tegmark. KAN: Kolmogorov–Arnold Networks. arXiv:2404.19756v5, February 2025. https://arxiv.org/abs/2404.19756v5. All section, theorem, and equation references to [1] follow arXiv v5.

[2] Marek Karpinski and Angus Macintyre. Polynomial Bounds for VC Dimension of Sigmoidal and General Pfaffian Neural Networks. Journal of Computer and System Sciences, 54(1):169–176, 1997. https://doi.org/10.1006/jcss.1997.1477

[3] Salah A. Faroughi, Farinaz Mostajeran, Amin Hamed Mashhadzadeh, and Shirko Faroughi. Kolmogorov-Arnold Networks for Data-Driven, Physics-Informed, and Deep-Operator Learning: A Review, Synthesis, and New Analysis. Neural Networks, 200:108791, 2026. https://doi.org/10.1016/j.neunet.2026.108791

[4] Namjoon Suh and Guang Cheng. A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models. Annual Review of Statistics and Its Application, 12:177–207, 2025. https://doi.org/10.1146/annurev-statistics-040522-013920

[5] Hrushikesh N. Mhaskar and Tomaso Poggio. Deep vs. Shallow Networks: An Approximation Theory Perspective. Analysis and Applications, 14(6):829–848, 2016. https://doi.org/10.1142/S0219530516400042

[6] Hrushikesh Mhaskar, Qianli Liao, and Tomaso Poggio. When and Why Are Deep Networks Better Than Shallow Ones? Proceedings of the AAAI Conference on Artificial Intelligence, 31(1):2343–2349, 2017. https://doi.org/10.1609/aaai.v31i1.10913

[7] Silviu-Marian Udrescu, Andrew Tan, Jiahai Feng, Orisvaldo Neto, Tailin Wu, and Max Tegmark. AI Feynman 2.0: Pareto-Optimal Symbolic Regression Exploiting Graph Modularity. NeurIPS, 2020. https://proceedings.neurips.cc/paper/2020/hash/33a854e247155d590883b93bca53848a-Abstract.html

[8] Wolfgang Dahmen. Compositional Sparsity, Approximation Classes, and Parametric Transport Equations. Constructive Approximation, 61:219–283, 2025. https://doi.org/10.1007/s00365-024-09699-9

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.