Compression order in spiking neural networks: normalization and model construction
Abstract
Compression-order comparisons depend on how models are constructed and calibrated. We study channel pruning, unfolded weight-only quantization and threshold scaling in spiking neural networks (SNNs), without intervening training. Matched-state controls distinguish batch-normalization (BN) history from model construction: aggressive clipping without rounding reproduces large history-dependent gaps, including in ReLU networks, and common recalibration yields identical logits. Separately, a prespecified twelve-run CIFAR-100 ResNet-SNN comparison at two time steps gives a mean absolute accuracy gap of 0.17 percentage points (pp) on reused test images, despite 6.0% prediction disagreement under the specified construction. Both orders satisfy the prespecified operational 5-pp budget on evaluated validation and test sets, with 10.08% fewer convolution multiply–accumulate operations. Descriptive extensions include matched T=4 retraining, additional image and event-frame datasets, and Transformer MLPs. Calibration improves both-order budget passage in the complete ImageNet-100 comparison; full reconstruction changes one finite-split qualification decision that final-only recalibration preserves. In a descriptive comparison of twelve T=2 ResNets at =1.02, the mean between-order relative synaptic-operation range contracts from 12.48% to 0.031% after common recalibration, with all orders remaining budget-feasible. Retrospective candidate policies on reused validation splits quantify compression choices and test-budget violations. These observations distinguish normalization history, model behavior, budget feasibility and activity-dependent work in compression-order evaluation.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.