- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a post-training compression framework for spiking neural networks (SNNs), named FastSNN. FastSNN collects two types of information for compression. First, it uses the local reconstruction Hessian matrix to describe the impact of weight changes on the reconstruction of the membrane potential. Then, it uses the Fisher matrix to estimate the impact of weight changes on the final task loss by gradients. Experimental results show that FastSNN outperforms state-of-the-art methods under high sparsity.
Critical strengths
- This paper provides on-device evaluation, including CPUs and neuromorphic hardware. Experimental results demonstrate its potential for acceleration in deployments.
- FastSNN outperforms state-of-the-art methods under high sparsity, including 2:8 structured sparsity and 87.5% unstructured sparsity.
Critical weaknesses
- Although this paper claims to be motivated by flip balance, the two metrics actually used in the proposed method do not directly measure flip balance but are indirectly reflected through the Fisher term in the final loss. The proposed method more closely resembles the Hessian term from existing SBC [1] and a Fisher term that uses the gradient of the final loss to guide layer-by-layer compression, similar to GuidedQuant [2].
- This paper uses temporal flip balance to measure the flip balance. It is defined as the ratio of the balanced spikes to the total flipped spikes. However, this ratio does not directly reflect the total number of unbalanced spikes. I suggest using either the total number of unbalanced spikes or the unbalanced spike firing rate to measure flip balance.
- The proposed FastSNN does not always outperform state-of-the-art methods on low-bit quantization of spiking transformers.
[1] Shi, Lianfeng, et al. "Spiking brain compression: exploring one-shot post-training pruning and quantization for spiking neural networks." Optimization for Machine Learning. 2025.
[2] Kim, Jinuk, et al. "GuidedQuant: Large language model quantization via exploiting end loss guidance." International Conference on Machine Learning. 2025.
Questions for authors
- Please compare FastSNN with SBC on the total number of unbalanced spikes or the unbalanced spike firing rate.
- Please explain in detail how FastSNN improves flip balance.
2 comments
Sorry for my late reply. The rebuttal has fully addressed my concerns. I believe this paper is worthy of acceptance.
To Reviewer Y8TU
We sincerely thank the reviewer for the detailed and expert comments, which helped us state our contribution more precisely.
W1: novelty over SBC and GuidedQuant
FastSNN builds on both: H is the SMP Hessian of SBC, and guiding layerwise compression by the end loss follows GuidedQuant. It adds two things.
The first is a Fisher term that measures flip balance. GuidedQuant squares the loss-weighted error of each row and then sums, ∑i,j(∂ℓ/∂Zij⋅ΔZij)2. In a spiking layer the rows of one sample are its timesteps, so opposite flips cannot cancel. We sum over timesteps before squaring (Eq. 12). At the last spiking layer the term then equals the squared net spike-count change weighted by the loss sensitivity (Appendix A.2). It is thus a first-order measure of count imbalance itself, which answers the concern that our metrics reflect flip balance only indirectly.
The second is the adaptive mixture, because neither part is sufficient alone. For MS-ResNet-56 on CIFAR-100 under 2:8 pruning (Fig. 2(b)), SBC reaches 33.5%, our Fisher term alone 48.4%, and the mixture 51.7%. The split-half weight that sets the mixture without a search is in neither prior method.
W2 & Q4: Bl versus the number of unbalanced spikes
We agree that Bl, as a ratio, does not give the number of unbalanced spikes, so we measured both suggested quantities. At the final spiking layer FastSNN has fewer unbalanced spikes than SBC in every setting (6%, 21% and 17% fewer), although it produces more flips.
SEW-ResNet on CIFAR10-DVS, all 1,000 test samples, final spiking layer, SBC / FastSNN:
Ul=∑i∣si+−si−∣ counts, per sample, the flips that cannot be paired with an opposite flip in the same neuron (Eq. 10), and rU,l=Ul/(NlT) is the unbalanced spike firing rate.
Both follow from statistics we already report. Layer l has RlKl flips, where Kl is the number of dense spikes, and a fraction Bl of them is balanced, so Ul=(1−Bl)RlKl. The reviewer's metrics and ours therefore carry the same information and rank the methods identically. We keep Bl in the paper because it isolates the unbalanced fraction from the flip count and has a count-matched chance level (dashed lines in Fig. 1(b) and 2(a)).
W3: low-bit quantization of spiking transformers
On Transformer-series models FastSNN trails the best baseline by about 1% on average at 2 bits (Appendix A.6). However, the best baseline changes across settings. Over the 48 transformer quantization settings, SBC leads at 3 and 4 bits but falls up to 18.4% behind at 2 bits, and GPTQ leads at 2 bits but falls up to 5.6% behind at 3 bits. FastSNN has the smallest mean gap to the best method (0.75%, against 0.98% for GPTQ and 1.79% for SBC) and the smallest worst case (2.4%). Its gain grows with the damage caused by compression, and at 2 bits the best baseline loses 5.2% on transformers against 8.5% on ResNets.
Q5: how FastSNN improves flip balance
For a weight perturbation δr, the two metrics penalize different things: δrHδr⊤∝∑n,t∣δVn,r[t]∣2,δrFrδr⊤=∑n(∑tan,r[t])2, where δVn,r[t] is the deviation of the surrogate membrane potential and an,r[t] is its signed first-order effect on the loss at timestep t (Appendix A.2). The Hessian of SBC is a sum of squares over timesteps, so deviations of opposite sign both add to the cost. The Fisher term is a square of a sum, so contributions of opposite sign cancel before squaring.
ExactOBS (Eq. 8) uses the metric to choose which weight to remove and to compensate the rest. With H(α), a weight whose removal causes a one-signed count drift tends to be kept, and the compensation may add opposite deviations that restore the net count (Fig. 1(c)).
Thank you again for the careful and constructive review.