FGKDSNN: Learning from Firing Activity via Firing-Guided Knowledge Distillation for Spiking Neural Networks

Spiking neural networks (SNNs) provide an energy-efficient alternative to artificial neural networks (ANNs) through sparse spike-based computation, yet a substantial performance gap remains under ultra-low time-steps. Knowledge distillation (KD) from pretrained ANNs has emerged as an effective approach to improve SNN training. Nevertheless, existing SNN distillation methods uniformly transfer feature- and logits-level knowledge, without considering the firing activity of the student SNN. In this work, we reveal that student firing activity is closely related to knowledge transfer at both feature- and logits-levels: the firing activity is highly concentrated across feature channels, while higher sample-wise firing ratios are associated with larger differences between teacher and student non-target distributions. Based on the observations above, we propose firing-guided knowledge distillation for SNNs (FGKDSNN), consisting of firing-guided feature-based knowledge distillation (FGFKD) and firing-guided logits-based knowledge distillation (FGLKD). At the feature level, FGFKD leverages channel-wise firing activity to guide feature masking and restoration, strengthening teacher feature transfer to more active student channels. At the logits level, FGLKD reweights non-target distillation according to sample-wise firing activity. Extensive experiments across CNN- and Transformer-based SNNs demonstrate that FGKDSNN consistently outperforms existing SNN distillation methods (e.g., +1.04% on ImageNet), achieving state-of-the-art performance on static and neuromorphic image classification, object detection, and semantic segmentation.