- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SRSC, a training framework for improving the energy efficiency of spiking neural networks (SNNs) through spike regularization and curriculum-based training. The method introduces a time-weighted spike penalty whose strength gradually increases after the validation accuracy reaches a predefined threshold. Experiments on several image classification benchmarks demonstrate reduced firing rates and synaptic operations with limited accuracy degradation. The authors also evaluate the potential energy savings using a hardware accelerator model.
Critical strengths
-
The experimental results are promising. The paper demonstrate reductions of firing rates and SOPs, while maintaining model performance across various classification datasets.
-
The paper goes beyond software evaluation and energy estimate. The authors evaluate the energy efficiency of the proposed method on a hardware accelerator.
Critical weaknesses
- The major limitation of this paper is novelty and substantial overlap with existing literature. The core idea of reducing neuronal activity in SNN through explicit spike regularization has been extensively studied in previous works, including Allred et al. (2020), Pellegrini et al. (2021), Deng et al. (2021), Yan et al. (2022), and Suetake et al. (2023) [1-5]. For example, Allred et al. (2020)[1] already explored progressively increasing the spike regularization strength during training, which is similar to the core algorithmic contribution of this paper. The authors should provide specific clarification.
[1] Allred et al. (2020). Explicitly Trained Spiking Sparsity in Spiking Neural Networks with Backpropagation. [2] Pellegrini et al. (2021). Low-activity Supervised Convolutional Spiking Neural Networks Applied to Speech Commands Recognition. IEEE SLT. [3] Deng et al. (2021). Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization. IEEE TNNLS. [4] Yan et al. (2022). Backpropagation With Sparsity Regularization for Spiking Neural Network Learning. Frontiers in Neuroscience. [5] Suetake et al. (2023). Synaptic Interaction Penalty: Appropriate Penalty Term for Energy-Efficient Spiking Neural Networks. TMLR.
-
Another weakness of this paper is that the motivation for penalizing spikes at later timesteps has not been explained. Since the computational cost of a spike does not inherently depend on when it occurs, the proposed temporal weighting appears heuristic rather than directly motivated by energy efficiency.
-
The results show subtle improvements compare to exisiting sparse SNN training methods. For example, SRSC even exhibits higher estimated energy consumption and lower accuracy than UniSparse in DVS-CIFAR10. Overall, the result table shows that the paper seems to trade more accuracy for less energy and fail to demonstrate an improved accuracy–energy Pareto frontier.
Questions for authors
See in weaknesses
Thanks for the reviewer's reply. I remain concerned that the novelty of penalty scheduling (the penalty's starting point) is insufficient. Therefore, I maintain my score.