- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies training spiking neural networks under structured sparsity constraints and proposes the HUS framework. It uses gradient-guided dynamic masks, a historical gradient buffer, and firing-rate homeostasis to improve connectivity topology search and reduce neuron silence. The authors evaluate it on multiple datasets, network architectures, and surrogate gradient functions, and report accuracy, neuron activity, and inference latency and energy results. The research problem is practically important, but in the current manuscript, the correspondence between the theoretical analysis and the algorithm update rules, the key claim about neuron silence, and the definition of hardware acceleration measurements all have major issues, which limits confidence in the core conclusions.
Critical strengths
-
Combining hardware-friendly structured weight sparsity with SNN temporal dynamics is an important direction for closing the gap between theoretical sparsity and real deployment gains.
-
Topology search and activity regulation separately target connection selection and neuron function maintenance, and their combination provides a sparse training framework worth exploring.
-
In addition to dense and unstructured sparsity baselines, the authors compare SR-STE, STEP, and SpikeNM, and provide component ablations, results under different surrogate gradients, and training traces, which help reveal the optimization behavior of the method.
Critical weaknesses
-
Algorithm 1 applies the mask to the entire corrected gradient, while the convergence analysis in Appendices D–E adds the historical gradient term directly outside the masked gradient. Ignoring constant scaling, these correspond to and , respectively, producing different updates for inactive connections. Therefore, the analysis that restores the dense gradient flow cannot be directly applied to the given algorithm. In addition, Theorem F.1 mainly compares gradients at different computation points before and after pruning, which is not enough to attribute the difference to the surrogate-gradient problem specific to SNNs; Theorem F.2 mainly proves that historical signals are preserved, and does not fully establish the unbiased search emphasized in the title.
-
Section 3.2 and Theorem G.1 directly infer from "no spike produced" that "the surrogate gradient of all input weights is zero." But the sigmoid surrogate derivative in Eq. (3) is usually nonzero at finite inputs, so this inference cannot hold in general. On the other hand, the main text uses layer-average firing rate for regulation, while the survival guarantee proof relies on per-neuron firing rate and uses a non-negative weight condition; how these conditions correspond to the actual algorithm is not explained.
-
Theorem 4.3 in the main text does not include the complexity term related to the number of support patterns in Appendix H.1, which is non-negligible for dynamic topology search. The proof also analyzes the feedforward network as a Lipschitz composition of smooth surrogate activations, but the actual SNN forward pass uses hard spike functions, so it needs to be explained how this analysis applies to the actual model.
-
Table 5 describes HW Speedup as end-to-end measured speedup, but Appendix I defines it as the ratio of summed per-layer linear and convolution microbenchmark times; these are not equivalent. The appendix does not specify the sparse storage format, computation kernels, or how they are invoked, and giving only does not show how actual sparse speedup is obtained. The relationship between the vGPU configuration and the stated T4 architecture also needs clarification. In addition, the main accuracy experiments use 1:4 sparsity, while the hardware experiments use 2:4 sparsity, so a complete accuracy-efficiency trade-off under the same configuration is still missing.
-
In the five comparisons in Table 4, HUS has lower accuracy than SR-STE in all cases, and the comparison reports only a single random seed. Whether reducing dead neurons leads to better task performance or energy efficiency has not been directly established. In the ablation, removing firing-rate homeostasis only lowers accuracy from 97.52% to 97.36%, with no error bars, so it is impossible to tell whether the gain is stable. At the same time, increasing firing rates may raise event-driven computation cost, so its benefits and costs need to be evaluated jointly.
Questions for authors
-
What is the complete weight update rule used in the actual experiments? Please unify the main text, Algorithm 1, and Appendices D–E, and clarify which theoretical conclusions apply to the actual implementation.
-
"Unbiased" with respect to what target or reference gradient exactly? Can you give the corresponding estimator, the random variables involved in the expectation, and the basis for saying the actual masked update satisfies these conditions?
-
How is the topology gradient for inactive connections computed? Does homeostasis regulation use layer-wise or per-neuron firing rate? Does actual training constrain weights to be non-negative, and how do these choices affect the survival guarantee?
-
What measurement scope do the inference time and HW Speedup in Table 5 correspond to? What are the actual GPU model, data precision, batch size, sparse format, and computation kernels? Can you provide reproducible end-to-end latency and energy results under the same sparsity configuration?
-
Can you add multi-seed baseline comparisons and core ablations, and jointly report accuracy, dead neuron ratio, firing rate, and actual energy, so we can judge whether homeostasis regulation provides a stable overall benefit?