Toward General-Purpose Spiking Neurons for Vision and Language Modeling

Widely regarded as the third generation of neural networks, Spiking Neural Networks (SNNs) have attracted considerable attention due to their biological plausibility and energy efficiency. Recent advances in large models call for spiking neurons that combine high performance, adaptability, and training efficiency. In this work, we first present a functional perspective that provides general guidance for designing the next generation of spiking neurons. Guided by these principles, we propose the Adaptive Spiking Neuron (ASN), which incorporates trainable parameters to learn membrane-potential dynamics and enable adaptive firing. ASN adopts an integer-training and spike-inference paradigm that facilitates efficient SNN training. To further improve training stability, we introduce a normalized variant, the Normalized Adaptive Spiking Neuron (NASN). We evaluate the ASN family on 20 datasets spanning five task groups across vision and language, demonstrating its effectiveness and versatility. In particular, NASN exhibits promising scalability in spike-based language modeling, with increasingly pronounced advantages as model and data scales grow. These results highlight the potential of the ASN family to serve as a next-generation general-purpose spiking neuron for large-scale SNNs.