acceptodds
Under review as a conference paper at ICLR 2027

Autocorrelation Reintroduces Spectral Bias in KANs for Time Series Forecasting

Abstract

Existing theory suggests that Kolmogorov-Arnold Networks (KANs) can overcome the spectral bias commonly observed in neural networks under the assumption that inputs are statistically independent. However, this assumption does not hold in time series forecasting (TSF), where inputs are lagged observations with strong temporal autocorrelation. Through theoretical analysis and empirical validation, we obtain an unexpected finding: temporal autocorrelation reintroduces spectral bias in KANs, and the bias becomes increasingly pronounced as the degree of autocorrelation increases. This suggests that standard KANs may face substantial difficulties in TSF with strongly autocorrelated inputs. To address this problem, we introduce the Discrete Cosine Transform (DCT) to reduce the correlations among the network inputs. As expected, experimental results reveal that DCT preprocessing substantially reduces the observed low-frequency preference in TSF. These results highlight input autocorrelation as a key factor shaping spectral bias in KANs for time series forecasting.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Spectral Bias, KAN, DCT

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PmwLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This article investigates how temporal autocorrelation affects the spectral bias of Kolmogorov Arnold Networks (KANs) in time series forecasting. By analyzing the dominant order of the Hessian matrix, the input autocorrelation, optimization conditions, and difficulties in learning frequency dependence were linked. The author further proposed DCT-KAN, which combines discrete cosine transform preprocessing and coordinate standardization. Extensive synthesis experiments have shown that under correlated inputs, this method can better recover challenging frequency components.

Critical strengths
  1. The paper investigates how temporal autocorrelation affects the optimization behavior of KANs, extending existing spectral-bias analysis beyond the independent-input setting.

  2. The authors provide extensive synthetic experiments, including frequency-direction comparisons, multiple random seeds, and explicit Hessian calculations, which offer useful evidence for the proposed mechanism.

Critical weaknesses
  1. All experiments utilize synthetic datasets. Without benchmarking against real-world predictions, the generalizability is questionable.

  2. The main analysis relies on a transfer approximation of covariance, and the accuracy of this approximation decreases with finer spline resolutions.

  3. The images and text in the paper heavily rely on AI, and should be carefully revised to enhance readability.

  4. There are multiple 1-2 words occupying a line in the article (such as line 100, 107, 168, 215, 290, 421, 431, etc.), which need to be adjusted and modified.

Questions for authors

See weaknesses

Reviewer_X5n9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how temporal autocorrelation affects spectral bias in Kolmogorov–Arnold Networks for time series forecasting. Under a covariance-transfer approximation, it derives bounds connecting the input correlation matrix to the conditioning of a shallow spline model’s Hessian. It then proposes DCT preprocessing followed by coordinate-wise standardization to improve learning. Synthetic experiments demonstrate better recovery of difficult oscillatory components, supplemented by analyses of target direction, frequency scaling, and Hessian approximation accuracy.

Critical strengths
  1. The paper addresses a relevant gap between independent-input analyses of KANs and the correlated lag vectors encountered in forecasting. This gives the investigation a clear motivation beyond proposing another architectural modification.

  2. The crossed frequency–direction experiments, swapped targets, and direct spline-Hessian calculations provide useful checks of the proposed explanation. The appendix also examines limitations of the approximation and validates the component-recovery metric.

  3. DCT followed by training-fitted standardization is simple to implement and preserves all input coefficients. The improvements on the tested difficult components suggest that input representation deserves more attention in KAN forecasting.

Critical weaknesses
  1. Poor Hessian conditioning alone does not imply a low-frequency preference; the required connection between target frequencies and slow eigenfunctions remains incomplete. The crossed experiments also show high-frequency difficulty with independent inputs, which calls for a more qualified interpretation of the claim that autocorrelation “reintroduces” spectral bias.

  2. The theory concerns a single-layer model with fixed B-spline bases, while the learning experiments use multilayer FastKAN with RBF bases, a SiLU branch, and Adam. The exact-Hessian calculations do not establish that the same mechanism explains the training dynamics of the evaluated network.

  3. All predictive evaluations use constructed sinusoidal labels on synthetic autoregressive inputs. These tasks are useful for controlled analysis, but evidence from actual forecasting datasets is needed to establish how often the identified effect matters in practice.

Questions for authors

Under what conditions is the covariance-transfer residual small enough to preserve the smallest positive Hessian eigenvalues, and can the resulting predictions be verified in the FastKAN model used experimentally?

Reviewer_dBU2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that autocorrelated inputs reintroduce spectral bias in KANs. It studies an approximate Hessian and proposes DCT preprocessing followed by standardization. Synthetic experiments examine learning errors across correlation levels, target frequencies, and directions in the lag space.

Critical strengths

The appendices provide useful controls, including crossed frequency–direction experiments and comparisons with exact Hessians. Reporting where the approximation and preprocessing fail makes the evidence easier to assess.

Critical weaknesses
  1. The claimed mechanism is not established. The Hessian analysis concerns conditioning across lag directions. It does not directly establish how target frequency determines learning speed. These are different notions of frequency. The analysis also relies on Assumption 1: only 116 of 240 exact-Hessian cases satisfy the interval derived for the approximate model (Appendix E). This limits its applicability; it does not invalidate the theorem under its approximation.

  2. The experiments do not isolate autocorrelation. At zero autocorrelation, DCT still reduces the high-frequency errors by about 100 times (Table 1). The chosen target directions align with DCT basis vectors, so coordinate alignment is a plausible contributor. Target normalization also changes frequency in the original input coordinates. Under fixed physical frequency, DCT can worsen error: 0.4443 to 1.9656 for the easy direction at frequency 6 and correlation 0.9 (Table 3). These results do not justify attributing the gains mainly to decorrelation.

Questions for authors
  1. What connects the Hessian condition-number result to preferential learning of low-frequency target functions, rather than difficulty along particular input directions?

  2. Does the DCT advantage persist for target directions that are not DCT basis vectors, compared with a random orthogonal rotation plus standardization?

  3. With target frequency fixed in the original lag coordinates, what consistent effect of autocorrelation remains across the tested directions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.