acceptodds
Withdrawn from ICLR 2027

HUS-SNN: Homeostatic Unbiased Sparsity for N:M Sparse Spiking Neural Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spiking neural networks offer high energy efficiency, but their real-world deployment is limited by the mismatch between theoretical sparsity and dense weight implementations. N:M structured sparsity can improve hardware efficiency, but applying it to SNNs leads to performance issues: gradient expectation bias from the non-commutativity of pruning masks and surrogate gradients, and irreversible neuronal silencing caused by reduced synaptic in-degree. We propose the Homeostatic Unbiased Sparse (HUS) framework to address these issues via a two-path optimization design. It combines gradient-aware mask updates with shadow gradient accumulation and multiplicative updates to enable unbiased topological search under N:M constraints. At the same time, a homeostatic weight update path regulates neuron excitability to prevent silent neurons. Experiments on multiple datasets show that our method maintains competitive accuracy while significantly reducing FLOPs, latency time and energy. Codes are available at https://anonymous.4open.science/r/HUS-SNN-FDE0/.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Spiking Neural Networks, N:M sparsity, Network pruning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_799HOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies training spiking neural networks under N:MN:M structured sparsity constraints and proposes the HUS framework. It uses gradient-guided dynamic masks, a historical gradient buffer, and firing-rate homeostasis to improve connectivity topology search and reduce neuron silence. The authors evaluate it on multiple datasets, network architectures, and surrogate gradient functions, and report accuracy, neuron activity, and inference latency and energy results. The research problem is practically important, but in the current manuscript, the correspondence between the theoretical analysis and the algorithm update rules, the key claim about neuron silence, and the definition of hardware acceleration measurements all have major issues, which limits confidence in the core conclusions.

Critical strengths
  1. Combining hardware-friendly structured weight sparsity with SNN temporal dynamics is an important direction for closing the gap between theoretical sparsity and real deployment gains.

  2. Topology search and activity regulation separately target connection selection and neuron function maintenance, and their combination provides a sparse training framework worth exploring.

  3. In addition to dense and unstructured sparsity baselines, the authors compare SR-STE, STEP, and SpikeNM, and provide component ablations, results under different surrogate gradients, and training traces, which help reveal the optimization behavior of the method.

Critical weaknesses
  1. Algorithm 1 applies the mask to the entire corrected gradient, while the convergence analysis in Appendices D–E adds the historical gradient term directly outside the masked gradient. Ignoring constant scaling, these correspond to M⊙(g+G)M\odot(g+G) and M⊙g+GM\odot g+G, respectively, producing different updates for inactive connections. Therefore, the analysis that restores the dense gradient flow cannot be directly applied to the given algorithm. In addition, Theorem F.1 mainly compares gradients at different computation points before and after pruning, which is not enough to attribute the difference to the surrogate-gradient problem specific to SNNs; Theorem F.2 mainly proves that historical signals are preserved, and does not fully establish the unbiased search emphasized in the title.

  2. Section 3.2 and Theorem G.1 directly infer from "no spike produced" that "the surrogate gradient of all input weights is zero." But the sigmoid surrogate derivative in Eq. (3) is usually nonzero at finite inputs, so this inference cannot hold in general. On the other hand, the main text uses layer-average firing rate for regulation, while the survival guarantee proof relies on per-neuron firing rate and uses a non-negative weight condition; how these conditions correspond to the actual algorithm is not explained.

  3. Theorem 4.3 in the main text does not include the complexity term related to the number of support patterns ∣S∣|\mathcal S| in Appendix H.1, which is non-negligible for dynamic topology search. The proof also analyzes the feedforward network as a Lipschitz composition of smooth surrogate activations, but the actual SNN forward pass uses hard spike functions, so it needs to be explained how this analysis applies to the actual model.

  4. Table 5 describes HW Speedup as end-to-end measured speedup, but Appendix I defines it as the ratio of summed per-layer linear and convolution microbenchmark times; these are not equivalent. The appendix does not specify the sparse storage format, computation kernels, or how they are invoked, and giving only y=(W⊙M)xy=(W\odot M)x does not show how actual sparse speedup is obtained. The relationship between the vGPU configuration and the stated T4 architecture also needs clarification. In addition, the main accuracy experiments use 1:4 sparsity, while the hardware experiments use 2:4 sparsity, so a complete accuracy-efficiency trade-off under the same configuration is still missing.

  5. In the five comparisons in Table 4, HUS has lower accuracy than SR-STE in all cases, and the comparison reports only a single random seed. Whether reducing dead neurons leads to better task performance or energy efficiency has not been directly established. In the ablation, removing firing-rate homeostasis only lowers accuracy from 97.52% to 97.36%, with no error bars, so it is impossible to tell whether the gain is stable. At the same time, increasing firing rates may raise event-driven computation cost, so its benefits and costs need to be evaluated jointly.

Questions for authors
  1. What is the complete weight update rule used in the actual experiments? Please unify the main text, Algorithm 1, and Appendices D–E, and clarify which theoretical conclusions apply to the actual implementation.

  2. "Unbiased" with respect to what target or reference gradient exactly? Can you give the corresponding estimator, the random variables involved in the expectation, and the basis for saying the actual masked update satisfies these conditions?

  3. How is the topology gradient for inactive connections computed? Does homeostasis regulation use layer-wise or per-neuron firing rate? Does actual training constrain weights to be non-negative, and how do these choices affect the survival guarantee?

  4. What measurement scope do the inference time and HW Speedup in Table 5 correspond to? What are the actual GPU model, data precision, batch size, sparse format, and computation kernels? Can you provide reproducible end-to-end latency and energy results under the same sparsity configuration?

  5. Can you add multi-seed baseline comparisons and core ablations, and jointly report accuracy, dead neuron ratio, firing rate, and actual energy, so we can judge whether homeostasis regulation provides a stable overall benefit?

Reviewer_a689OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors address the gap between the theoretical and practical efficiency of sparsity in SNNs by applying hardware friendly N:M structured weight sparsity. They identify two problems with static magnitude based sparsity methods in SNNs. The first is gradient expectation bias where masking and the surrogate gradient operator do not commute, which starves pruned weights of learning signal. The second is irreversible neuronal silencing caused by reduced synaptic in degree. They propose HUS, which addresses the first problem with gradient informed masking, a shadow gradient for masked weights, and a multiplicative mask update. It addresses the second with a firing rate homeostasis mechanism that scales weights toward a target firing rate. The experiments cover six architectures, six datasets, and four surrogate gradients at 1:4 sparsity. They also compare against 3 other structured sparsity methods on accuracy and dead neuron count, run an ablation and a hyperparameter sweep, and measure latency and energy at 2:4 sparsity, reporting roughly 1.8–1.9 speedups.

Critical strengths

• Table 5 reports real measured latency and energy, with roughly 1.8–1.9 speedups across five architectures. • The evaluation covers six architectures and datasets and four surrogate gradients. Accuracy often stays close to dense despite big FLOPs reductions in Table 2.

Critical weaknesses

The paper argues that neuronal silencing limits learning, but the results don’t seem to back this up. SR-STE has more dead neurons than HUS (7 vs 1 in Table 4b), yet beats it on accuracy in every setting (Table 4a), and removing homeostasis from HUS only costs 0.16 points. It’s not clear that dead neurons are a bottleneck here. Several key results look like single runs (Table 4 uses one seed, the ablation has no error bars, and the CIFAR/ImageNet results have no errors). Section 5 says HUS is compared against “static magnitude-based N:M pruning under identical training budgets”, but this baseline doesn’t appear in the main results (Table 2), only in a single ablation setting (Figure 5, 97.26% vs. 97.52%). Since the limitations of static N:M pruning are the paper’s main motivation, it would help to show this comparison across the main datasets and models, especially as SR-STE outperforms HUS in Table 4. The authors present gradient informed masking with λ as a fix for gradient bias, but Table 6 suggests it makes little consistent difference. λ=0 , which removes the gradient term from mask selection, gives the best peak accuracy in the table (96.57%) and is otherwise comparable to the other settings. Combined with the 0.18 point drop in the ablation, it’s not clear this component meaningfully helps. Removing the multiplicative update in Figure 5 makes HUS worse than static pruning.

Questions for authors

Since many of the differences are small, could you report errors? Could you add static magnitude based N:M as a column in Table 2, and clarify the setting used in Figure 5? Since λ=0 performs about as well as other values there, can you show the gradient term helps at higher sparsity (e.g., 1:8) or on a harder dataset, with multiple seeds? Removing the multiplicative update makes HUS worse than static pruning. Doesn’t this suggest that the other components which are meant to improve on static pruning may hurt performance on their own? Why are the fast_sigmoid and piecewise columns identical in Table 3 for four of the five MNIST models?

Reviewer_ZmqSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes HUS, a method for training spiking neural networks with N:M structured sparsity. It addresses two problems caused by pruning: inactive connections lose learning signals, and some neurons stop firing. HUS updates connectivity using weight and gradient information, stores past gradients for inactive connections, and rescales weights based on average firing rates. These mechanisms allow connections to re-enter the network while regulating neuronal activity.

Critical strengths

The method addresses both connection selection and neuronal activity under a fixed N:M constraint. This is a sensible design for SNNs, where preserving useful connections may not be enough if the remaining input cannot drive neurons to fire. The reduction from 7–8 inactive neurons in competing sparse methods to 1 with HUS provides some evidence that the activity controller helps. The ImageNet result is promising: the 1:4 sparse ResNet achieves 60.05% accuracy against 60.23% for the dense model. Retaining accuracy while removing 75% of weights in the pruned groups supports the practical potential of the approach beyond small datasets.

Critical weaknesses
  1. I am not convinced by the definition of gradient bias. Theorem 3.1 compares dense and pruned networks, whose membrane trajectories and spikes differ. Why should their gradients agree? Also, Appendix E adds shadow corrections to inactive weights, whereas Eq. 8 masks their updates out. The claimed correction appears to be proved for a different algorithm.

  2. The shadow-buffer rules are inconsistent. Eq. 10 decays old signals, but Algorithm 1 gives an inactive connection \xi_new=clip(2\xi_old+g). An existing signal can therefore double without any fresh gradient. Which rule produced the results? This changes whether the buffer preserves useful history or amplifies stale information.

  3. Layer-average homeostasis cannot guarantee individual neuron survival. A layer can meet its firing-rate target while some neurons remain silent, leaving Eq. 14 with no correction. The proof instead uses per-neuron feedback and nonnegative weights. These assumptions do not match the stated method.

  4. Does reducing neuron death preserve useful computation, or simply increase firing? HUS has fewer inactive neurons but lower accuracy than SR-STE in every matched setting. Comparing against SR-STE or SpikeNM with the same homeostatic controller, and at matched spike activity, would clarify what HUS’s topology mechanism actually contributes.

  5. Finally, the source of the hardware speedup is unclear. Appendix I gives (W o M)x without specifying sparse kernels, and defines speedup through isolated layer timings rather than the end-to-end measurement claimed in Table 5. Accuracy is mainly reported at 1:4 sparsity, while hardware measurements use 2:4, so the claimed trade-off needs results from the same deployed models.

Questions for authors
  1. Which shadow-buffer rule was used in the experiments: Eq. 10 or Algorithm 1? Also, how is G_topo computed for inactive connections without the mask zeroing their gradients?

  2. Which objective’s gradient is HUS claimed to estimate without bias? How does Appendix E’s update, which applies shadow corrections to inactive weights, establish this property for the masked update in Eq. 8?

  3. Was homeostasis implemented per neuron or per layer, and were weights constrained to be nonnegative? If the implementation uses layer averages and signed weights, which part of the survival guarantee still applies?

  4. Do any existing results separate the benefit of HUS’s topology updates from the effect of increasing firing activity - for example, by applying the same homeostatic controller to a competing method?

  5. Which GPU, sparse representation, and kernels produced Table 5? Are its speedups end-to-end or aggregated layer timings, and what accuracies did those same 2:4 models achieve?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.