Structural Regularization for Spiking Neural Networks

Spiking Neural Networks have gained increasing popularity as a biologically plausible alternative to traditional Artificial Neural Networks (ANNs). SNNs are both cost-efficient and deployment-friendly, as they process input in a spatiotemporal manner using binary spikes. It is widely believed that increasing the number of timesteps enables SNNs to capture more features, thereby improving accuracy. However, we observe that as the number of timesteps grows, the risk of overfitting also increases, leading to diminishing returns in accuracy improvement. To handle this issue, we propose a structural regularization technique for SNNs, called DropTime. Specifically, during training, we randomly remove information from certain timesteps. We also provide a theoretical justification showing that this simple operation effectively mitigates overfitting. Experimental results demonstrate that our method consistently improves SNN accuracy on both widely used static non-spiking and neuromorphic datasets. We offer a straightforward yet effective approach for training high-accuracy SNNs.