LRTB: Low-Rank Temporal Batch Normalization in spiking neural networks

Batch normalization (BN) facilitates the training of deep spiking neural networks (SNNs), but temporally evolving neuronal activations pose additional challenges for statistical estimation and parameter adaptation. Existing temporal normalization methods may overlook between-step mean drift when estimating cumulative variance, while independently learned time-specific affine parameters do not explicitly exploit temporal structure. To address these limitations, we propose Low-Rank Temporal Batch Normalization (LRTB), which combines exact cumulative statistical estimation with dynamics-inspired low-rank temporal parameterization. LRTB accounts for both within-step variability and between-step mean drift to accurately estimate the variance of accumulated activations. It further represents time-varying affine parameters using a compact set of exponential basis functions inspired by leaky integrate-and-fire dynamics, enabling structured temporal parameter sharing with fewer learnable affine parameters. Experiments on CIFAR-10, CIFAR-100, DVS-CIFAR10, and ImageNet demonstrate improvements over reported temporal normalization baselines.