Spike Regularization and Curriculum Based Direct Training of Spiking Neural Networks for Energy-Efficient Inference

Spiking neural networks emit binary events that can be exploited for higher energy efficiency on neuromorphic hardware. A spiking neuron that does not fire costs no energy, and hence, spike count is an important objective to optimize. We propose SRSC (Spike Regularization with Spike Curriculum) that applies a time-weighted penalty on every layer's spike tensor, held negligible until the validation accuracy crosses a threshold, and then ramped geometrically to a ceiling. This does not incur additional parameters or architectural changes, and only requires the sparsity ceiling to be tuned to obtain a desirable accuracy-sparsity trade-off. %We also study a per-layer variant of the penalty, allocated in proportion to each layer's gradient sensitivity, against the uniform coefficient used above. We validate the proposed method on VGG, ResNet, and transformer-based architectures across CIFAR-10, CIFAR-100, DVS-CIFAR10, and ImageNet datasets. SRSC matches state-of-the-art accuracy ($\pm 0.5\%$ for ResNet-19 on CIFAR-10, $\pm 0.75\%$ for VGGSNN on DVS-CIFAR10), while reducing the firing rate by 49% and 44%, and synaptic operations (SOPs) by 24% and 16%, respectively, relative to state-of-the-art methods.