- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Async-SNN, a framework for LLMs that supports asynchronous execution by replacing normalization and attention operators—which typically rely on global statistics—with components compatible with quantization-aware training and spiking neuron conversion. It employs a greedy adaptive firing strategy to dynamically allocate the neuron-firing budget. Experiments demonstrate that the model maintains performance levels comparable to unconstrained settings even when limiting active neurons to 10% per timestep. Hardware simulations indicate a reduction in energy consumption of approximately 33% for a 1.7B-parameter model compared to a quantized ANN. While this approach offers a viable pathway for low-power LLM inference, its performance under real-world hardware constraints and fluctuating power supply conditions remains to be validated.
Critical strengths
This paper reports hardware simulation results obtained using Eyeriss for ANN models and SANA-FE/Loihi for SNN models.
Critical weaknesses
-
SpikeZIP-TF [1] already addresses an exact mapping from quantized ANNs to SNNs, and Dynamic Tanh [2] already replaces normalization with element-wise functions. The paper merely combines existing techniques and has limited incremental novelty.
-
The authors' method employs pre-training on 100 billion tokens and SFT on 10 billion tokens, whereas some competing spiking LLM methods [3] used for comparison rely on training-free approaches. Such a comparison is unfair. Furthermore, the extensive training involved runs counter to the original intent behind ANN-to-SNN conversion.
[1] SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN.
[2] Transformers without Normalization.
[3] SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking.
Questions for authors
-
The paper provides simulations comparing Eyeriss with SANA-FE/Loihi, which strengthens the efficiency evaluation. However, the two systems employ different hardware architectures, and the current description is insufficient to reproduce the model mapping, weight storage, communication, and scheduling overheads. Could the authors further provide the simulation configurations and clarify the accounting assumptions?
-
The main text emphasizes that freezing only alters the timing of output generation without changing the cumulative output. However, the specific operation in Algorithm 1 involves setting unselected spikes to zero, without illustrating how the membrane potential and spike-tracer states are updated. Could the authors explain this part in more detail?
Sincerely looking forward to the further discussions
Dear Reviewer qsYe,
As the discussion deadline is approaching, we kindly remind that we have answered your remaining concerns carefully with additional experiments and clarification.
Could you please check our response and consider improving the rating if we successfully addressed you concerns?
Any feedback will be greatly appreciated and we are glad to provide any additional clarifications that you may need to bring a fruitful discussion.
Warm regards, The Authors