acceptodds
Withdrawn from ICLR 2027

Advanced Burstprop: an online energy-efficient local learning paradigm for deep spiking neural networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Backpropagation through time with surrogate gradients (SG-BPTT) is the state of the art for training deep spiking neural networks (SNNs), but it is offline, with an unrolled graph costing activation memory, and its backward phase executes dense multiply-accumulate operations. Recent online local learning rules (E-prop, OSTL, OTTT, DECOLLE, ETLP) remove the unrolled graph, yet their weight updates remain dense floating-point multiply-accumulates at every timestep (or, for the event-triggered ETLP, at every triggered timestep), so the backward phase remains the energy sink. This raises a critical question for the field: if training an SNN costs orders of magnitude more energy than its forward pass could ever save, in what sense is the SNN energy-efficient at all? We answer this question with Advanced Burstprop (ABP), an online local learning rule whose backward phase is as event-driven as the forward phase: signed ternary burst events , emitted at a rate, carry error transport and weight updates strictly in memory, multiply-free and accumulate-only. On DVS-Gesture, under the 45 nm CMOS energy model, ABP matches the dense-backward baselines ( vs. for the strongest baseline) at lower backward energy, and the deterministic variant pushes the reduction to . Widening the burst alphabet beyond ternary to multiplies the backward energy without accuracy gain, confirming ternary as the optimal code. In a controlled depth sweep, ABP's accuracy decays with depth, but widening the layers, rather than raising the burst rate, recovers it while maintaining the energy-efficiency advantage. On SHD, ABP attains the smallest gap to SG-BPTT among online methods on the temporal-convolutional LSTM architecture while being more backward-energy-efficient. We further discuss the feasibility of deploying ABP on existing neuromorphic hardware, including Intel Loihi 2 and SpiNNaker 2. To our knowledge, ABP is the first online learning rule to scale multiply-free, event-sparse burst learning to deep SNN architectures, and the first to quantify the resulting energy advantage, making SNN training, not merely SNN inference, energy-efficient.

Primary area:
learning theory
Keywords:
spiking neural networks, online learning, energy-efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_woAhOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper aims to reduce the cost of online training for spiking neural networks. The method maintains incrementally updated activity traces and replaces continuous error signals with sparse signed ternary events, allowing parts of error propagation and weight updates to use additions and subtractions without storing the full temporal sequence. The authors apply the method to CNNs, Transformers, LSTMs, and Mamba models, evaluate accuracy and estimated computational energy on DVS-Gesture and SHD, and analyze the effects of error encoding precision and network depth. The main contribution is an event-based learning rule and its adaptation to different network architectures.

Critical strengths

The paper addresses the practically relevant problem of SNN training efficiency. It evaluates the method across several architectures and demonstrates a reasonably competitive trade-off between accuracy and estimated computational cost on CNNs, providing useful empirical results for event-based online learning.

Critical weaknesses
  1. The incremental contribution is unclear. Positive and negative error events have precedents; the main additions are architecture adaptation, a layer-averaged surrogate derivative, and multiple-timescale traces. Please identify which changes are essential for deep learning and demonstrate their value through comparisons with closely related methods using the same architecture.

  2. The effects of key components are not isolated. The error-alphabet and depth studies are useful, but do not establish the individual contributions of the averaged surrogate derivative, multiple-timescale traces, and event control. Please provide key ablations to distinguish benefits of the new mechanisms from those of sparsification, quantization, or training changes.

  3. Performance and applicability are limited. The stochastic CNN variant is 2.22 percentage points below OSTL in the same group, while the Transformer is approximately 12 points below strong baselines. Absolute SHD accuracies are also low across methods. Although the authors acknowledge this limitation, practical competitiveness on challenging temporal tasks remains unestablished.

  4. Widening confounds the depth analysis with increased capacity. Widening increases parameters, capacity, and computation, so recovered accuracy cannot be attributed solely to the error channel. Please provide baselines with the same width or matched parameter/computational budgets to test depth-scaling and efficiency advantages under comparable resources.

  5. Ternary optimality lacks controlled evidence. The study covers a single CNN, and stochastic versus deterministic encoding is not clearly held consistent across alphabet sizes. Please fix other key conditions to separate encoding precision from sampling and event-budget effects, and limit the conclusion if it only applies to this setting.

Questions for authors

Please see the weaknesses above.

Reviewer_PMuGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce a local learning rule (advanced burstprop) for spiking neural networks, which is based on ternary backpropagation feedback signals. The advantage claimed over the previous local learning rules using floating-point backward tensors is that it can avoid such floating-point arithmetic operations that are costly. To this end, ABP deploys a basal dendritic tensor and apical dendrite per neuron, which cannot be shared with any other neurons. The algorithm was tested on spiking CNNs and spiking transformer on DVSGesture and various networks on SHD.

Critical strengths

None

Critical weaknesses
  1. Low accuracy: Although accuracy is not the only critical metric, the proposed algorithm considerably falls behind previous local algorithms.
  2. Not local rule: This algorithm is not a local rule since weight update for a given layer depends on the error signal backpropagating from the output layer. The proposed algorithm is simply dependent on ternarized backpropagation error signal.
  3. Unclear evaluation of energy: I strongly recommend the authors to count the number of FLOPs (multiplications and additions) rather than implicit energy of 45nm CMOS. The authors should honestly report the FLOPs for the basal dendritic tensor per neuron, which is not used in previous local rules. For a linear layer, the total dendritic tensor for a layer is the same size as its weight matrix. For a convolutional layer, it is C\times H\times W \times C_{in}\times K_H\times K_W.
  4. Additional memory overhead: As stated above, the use of basal dendritic tensors largely increase the space complexity of the algorithm.
Questions for authors

Can the authors report all FLOPs per backward step including the basal dendritic tensor update (decay)?

Reviewer_TBnPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  • This paper proposes a learning rule for training deep spiking neural networks. Specifically, this learning rules looks to replace the MAC operations incur in the backpropagation of errors across layers, by a mechanism that encode the learning signal into a burst of ternary signals, having both forward and backward passes as matmul-free processes. Moreover, this learning signal propagation mechanism is integrated in the three-factor learning rule framework, so the resulting learning rule is temporally local with memory cost independent of timesteps. The idea is indeed interesting, as it looks for making both forward and backward passes energy efficient. The experiments shown that this learning rule is effective to train SNN with a competitive performance.
Critical strengths
  • The idea of making both forward and backward passes completely event-driven and matmul-free is highly interesting and can potentially be leverage in custom neuromorphic processor to achieve energy efficient training of SNN on resource constrained devices. Therefore, this is an interesting work for the neuromorphic community that can spark some ideas.
Critical weaknesses
  • The main weakness is the claim regarding energy-efficiency. The methodology to report energy assumes that only synaptic operations matter. However, the model introduces a second state to produce the burst-event rate, therefore introducing additional data movements. Since data movement is what consumes more energy in a compute system the comparison and claims regarding energy-efficiency with respect to other methods are unfair.
  • The second main weakness is that the evaluation is focused mainly on a single dataset. DVS-Gesture recognition is a small and simple dataset so there is no clarity on how these results generalize beyond this setting, or if the results are mainly due to overparameterization.
Questions for authors
  • Could you provide energy comparisons including data movement on the neuron states?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.