acceptodds
Withdrawn from ICLR 2027

Spatio-Temporal Spectral Backpropagation for Training Spiking Neural Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spiking Neural Networks (SNNs) offer significant advantages in time-series modeling due to their event-driven nature and biological plausibility, yet their training remains challenging. In existing backpropagation algorithms, parameters are deeply coupled within the temporal dimension, hindering the quantification of individual influences and limiting the analytical depth of gradient dynamics and theoretical interpretability. This paper proposes the Spatio-Temporal Spectral Backpropagation (STSBP) framework, which decomposes SNN error propagation in the spatio-temporal frequency domain into a product of four terms: (1) a spatial frequency kernel dependent on network structure and surrogate gradients; (2) a temporal frequency kernel governed by the leakage factor ; (3) an error spatio-temporal spectrum derived from task errors; and (4) an input signal spectrum. The framework reveals that SNNs prioritize low-frequency components in the spatial domain. In the temporal domain, the learning order is jointly determined by the temporal frequency kernel, the spatial-frequency integral of the error spectrum, and the input signal spectrum. Based on this theory, this paper presents the Frequency-LIF (Frequency Leaky Integrate-and-Fire) model, which comprises two core modules: the Spatial Frequency Dynamic Adjustment Mechanism (SFDAM) and the Temporal Frequency Dynamic Adjustment Mechanism (TFDAM). In the spatial domain, SFDAM detects low-frequency dominance, then attenuates low-frequency weights and enhances high-frequency learning, guiding the model from global trends to local details. In the temporal domain, TFDAM exponentially decays the leakage factor over timesteps, steering the model from initial low-frequency focus toward high-frequency learning. Experimental results on eight time-series prediction benchmark datasets demonstrate that Frequency-LIF outperforms current state-of-the-art methods. The relevant code is available at: https://anonymous.4open.science/r/Frequency-LIF-86F4.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Spiking Neural Networks, Backpropagation and Time Series

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BYW4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

As far as we know, existing backpropagation algorithms, parameters are tightly coupled across the time dimension, hindering the quantification of individual impacts and limiting the depth and theoretical interpretability of gradient dynamics analysis. In this manuscript, the author proposes a spatio-temporal spectral backpropagation (STSBP) framework that decomposes error propagation in pulse-coupled neural networks into the spatio-temporal frequency domain.

Critical strengths
  1. In the field of time series forecasting, developing efficient algorithms for training light-weight SNNs is a very meaningful research topic.
  2. The integration training from the perspective of time and space is more conducive to the SNN model obtaining valuable information.
Critical weaknesses
  1. The current title and abstract seem to need further improvement. After all, the experiment focuses on the task of time series prediction. Looking at the title, it seems that this is a training method designed for classification tasks, similar to most SNN models.
  2. By observing Table 1, it is intuitively seen that the performance improvement effect is rather limited, and further significant verification is required. Furthermore, how significant is the efficiency advantage when compared to other complex full-precision Transformer models?
  3. During the formal analysis process, several lines of integral expressions were added. Some details of the binary integrals are suggested to be placed in the appendix.
  4. Could some visualized results of the gradients be provided to verify the correctness of the proposed theory?
Questions for authors

Please see weaknesses!

Reviewer_mwzpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

STSBP aims to factor SNN weight gradients into spatial, temporal, error-spectrum, and input-spectrum terms. It motivates Frequency-LIF: SFDAM uses gradient spectra to switch trend/seasonal loss weights, while TFDAM decays the leakage coefficient across simulation steps. Experiments address multivariate time-series forecasting.

Critical strengths
  1. The research question is useful. Understanding how surrogate gradients and membrane dynamics affect frequency learning could inform lightweight training mechanisms.

  2. The evaluation is broad. Table 1 compares neuron models within iTransformer across 8 datasets and 4 prediction horizons; for example, average ETTh2 MSE is 0.378 versus AMP2's 0.392. Table 2 extends evaluation to 5 backbones.

  3. The appendix provides supporting analyses. Results over 5 random seeds, component ablations, hyperparameter studies, and frequency visualizations support the combined method's practical potential.

Critical weaknesses
  1. The central spectral derivation contains verifiable errors. Substituting Eq. (42) into Eq. (38) does not yield Eq. (43), p. 16. For one neuron with a=w=α=1,b=0a=w=\alpha=1,b=0 and residual e(x)=xe−x2e(x)=xe^{-x^2}, the paper's Cauchy surrogate-gradient rule gives

    Gw=∫Rx2e−x2π(1+x2) dx≈0.136606,G_w=\int_{\mathbb R}\frac{x^2e^{-x^2}}{\pi(1+x^2)}\,dx\approx0.136606,

    whereas Eq. (43) gives 0. Moreover, Eq. (61), p. 17, replaces the local membrane-error signal dj[t]d_j[t] from Eq. (50) with a spatial integral using a static weight-gradient kernel. These are different quantities. The resulting Sj(k)S_j(k) and four-factor decomposition in Eq. (11) therefore lack a valid derivation, and the dependent theorems require rechecking.

  2. SFDAM does not connect its detector to the theoretical frequency variable. Theoretical kk is Fourier-dual to input xx, whereas Eq. (21), p. 6, transforms parameter gradients ∇θL\nabla_\theta L, without specifying axes or establishing this correspondence. Also,

    \frac{\mathbb E[|\mathcal F\{\nabla_\theta L\}|_{k>k_l}]} {\mathbb E[|\mathcal F\{\nabla_\theta L\}|_{k<k_h}]}.$$ With default $k_l=0.1,k_h=0.5$, the bands overlap, and increasing high-frequency amplitude can increase the ratio. Thus $R_{\mathrm{grad}}>1$ does not imply the claimed low-frequency dominance, undermining the trigger's interpretation.
  3. The constant-leak theory does not establish the actual schedule's behavior. The temporal kernel T(ω)=1/(1−βeiω)T(\omega)=1/(1-\beta e^{i\omega}) assumes constant β\beta, but Eq. (25) uses β[t]\beta[t]. Backpropagation then involves ∏q=t+1sβ[q]\prod_{q=t+1}^{s}\beta[q], which depends on the starting time and cannot directly use that single transfer function. The mapping between the 4 SNN simulation steps and the length-96 observation sequence on p. 24 also needs clarification to interpret temporal-frequency control.

  4. Ablation labels conflict, and the detector's benefit remains unisolated. Table 3, p. 9, reports ETTm1 MSE of 0.417/0.412 for TFDAM-only/SFDAM-only, while Table 8, p. 27, labels these values w/o TFDAM/w/o SFDAM. The assignments also reverse on the other 3 datasets. The stated removal penalties, 0.012/0.009, match neither table. After correcting this attribution, a fixed-switch control with the same seasonal/trend loss would isolate the spectral detector's benefit. Applying that loss to ordinary LIF/GELU would further separate loss reweighting from neuron design.

Questions for authors
  1. How do you resolve the Eq. (43) counterexample and the substitution from Eq. (50) to Eq. (61)? Do corrected gradients for aj,wj,bja_j,w_j,b_j match direct surrogate-gradient calculations, and which theorems remain valid?
  2. Which axes and frequency bands does Eq. (21) use, and why does its trigger indicate low-frequency dominance? How do these quantities and the 4 simulation steps correspond to the theoretical spatial/temporal frequencies?
  3. Which ablation labels in Tables 3 and 8 are correct? Are fixed-switch results available under the same weighted loss to isolate the adaptive detector's contribution?
Reviewer_fEE7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates frequency-dependent learning within the context of surrogate-gradient Spiking Neural Network (SNN) training. The proposed STSBP method claims to factorize gradients into spatial, temporal, input, and error spectral terms. Additionally, Frequency-LIF combines leakage scheduling across simulation steps with a dynamically triggered mechanism for reweighting trend and seasonal losses during training. The experiments cover eight forecasting datasets and evaluate five additional backbones on four of these datasets. While the reported results suggest practical promise, the theoretical derivation and the attribution of the underlying mechanisms remain unresolved.

Critical strengths

Frequency-LIF offers a practical training modification that yields broadly favorable average results across various datasets and backbones (Tables 1–2, 6–7). This is further supported by results averaged over five seeds for the proposed method (Table 5).

Critical weaknesses
  1. Algebraic Inconsistencies in Derivation: The central derivation contains a notable algebraic inconsistency: Eq. (43) does not logically follow from Eqs. (38) and (42), and its coefficient reappears unexpectedly in the spatial kernel. Furthermore, Eq. (61) lacks sufficient justification for connecting the static weight-gradient kernel to the instantaneous membrane-potential gradient. These issues undermine the claimed STSBP decomposition.
  2. Contradictions in Interpretation and Reporting: The spectral ratio in Eq. (21) conflicts with its interpretation of low-frequency dominance. Additionally, Tables 3 and 8 appear to have reversed labels for the single-module ablation studies. Although Figure 11 provides relevant evidence regarding training dynamics, it fails to isolate the effects of adaptive triggering from the subsequent loss reweighting.
Questions for authors
  1. Could you please reconcile Eqs. (38), (42), and (43), and provide a justification for Eq. (61) based on Eq. (50)? It would be helpful to identify which theoretical results remain valid after correction. A direct surrogate-gradient consistency check would also help delimit the scope of these issues.
  2. Could you clarify the specific FFT axis, frequency bands, and inequality used in Eq. (21)? Additionally, please confirm which ablation labels are correct. Providing implementation excerpts and corrected tables would help clarify whether these are merely reporting errors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.