NeuroProp: Bio-Inspired Local STBP for Memory-Efficient Spiking Transformers

Spatiotemporal backpropagation (STBP) is among the strongest methods for directly training deep spiking neural networks (SNNs), yet its globally coupled graph retains neuronal states across both depth and time. We introduce NeuroProp, a bio-inspired framework that reduces this memory burden by localizing, rather than replacing, STBP. NeuroProp partitions a spiking Transformer into consecutive units, stops gradients at unit boundaries, and performs full surrogate-gradient STBP over the blocks and time steps within each unit. This converts the depth-wise activation graph from a whole-network dependency to one governed by the largest unit. To coordinate gradient-isolated units, we propose adaptive intrinsic plasticity (AdaIP), inspired by contextual neuromodulation of neuronal excitability. AdaIP broadcasts a semantic condition and transforms it into channel-wise synaptic gain and membrane bias, equivalently adjusting the effective firing threshold and equilibrium potential of LIF neurons. Detached inter-unit interfaces also admit pipeline-parallel execution. NeuroProp reduces peak memory by $48.29\%$ on SpikFormer and $79.41\%$ on SDTv3 in controlled CIFAR-100 experiments, while AdaIP recovers part of the accuracy lost by naive localization. Without global-STBP fine-tuning, NeuroProp reaches $71.30\%$ ImageNet-1K top-1 accuracy. These results establish local STBP with adaptive neuronal modulation as a practical accuracy–memory trade-off for scaling spiking Transformers.