- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
SpikeHENA is a training-free framework for converting pretrained Transformers into SNNs. It combines activity-constrained BinarySpike encoding with SingleSpike residual encoding and introduces spiking arithmetic units for Softmax, RMSNorm, and SiLU. Experiments on BERT and four LLMs show small accuracy and perplexity changes after conversion. The paper also provides encoding and nonlinear-component ablations, local error analyses, and theoretical energy estimates.
Critical strengths
-
The paper tackles the two real bottlenecks in LLM-to-SNN conversion jointly, encoding and nonlinear operators, instead of improving only one side. The framework preserves near-ANN performance across several architectures and model scales supported by both task accuracy and perplexity.
-
The hybrid encoding design is well motivated, and the ablations and local error analyses provide useful support for its fidelity–spike-activity trade-off.
Critical weaknesses
-
Table 1 provides a useful comparison with LAS and other conversion methods on BERT. The BERT comparisons are useful, but the LLM experiments primarily compare SpikeHENA with its source ANNs. Consequently, extending this comparison to representative LLMs would help assess whether the relative benefits also hold for larger generative models.
-
The scope of the LLM efficiency evaluation is limited. The reported energy ratios cover only synaptic computation, excluding encoding, nonlinear operations, and memory and communication costs. The authors clearly acknowledge this limitation and provide a broader analysis for BERT. Nevertheless, a similar breakdown for one LLM would help determine whether the estimated savings remain substantial under more complete accounting.
Questions for authors
-
Could you clarify whether other conversion methods could also be evaluated on LLMs under matched conditions? Such a comparison would help assess whether the relative benefits observed on BERT extend to larger generative models.
-
Table 2 reports synaptic-only energy estimates, whereas Table 4 provide broader accounting for BERT. Do you have an analysis of how the LLM energy ratios would change after including neuron updates, encoding/control, and nonlinear operations under the same cost model?
-
How frequently does exponentiation clipping occur in the evaluated models, and how sensitive are the results to the clipping range?
Low time-step results and method comparisons
W2.
Q2.
A: Among the time steps reported by LAS, SpikeHENA scores higher at T=10 and T=11 by 29.85 and 4.33 percentage points respectively; at T=4 and T=8, our exploratory evaluations yield mean scores of 45.68 and 66.02 respectively, substantially below the source ANN's 85.55. LAS does not report results at T=4 and T=8. SpikeHENA's T counts both stages of hybrid encoding; the configuration for each exploratory setting was selected on a separate subset of 64 SST-2 training samples.
Table R3: BERT results across seven GLUE tasks at different time steps.
We recomputed the LAS [1] mean scores at T=10, 11 and 13, first averaging the two metrics within each of MRPC and STS-B, then taking the equally weighted mean across all seven tasks in LAS Table 6.
At T=13 the published LAS score is 1.44 percentage points higher, and at T=16 SpikeHENA scores 0.23 percentage points higher in our evaluation. These comparisons align only the time steps, so they do not establish equal computational budgets, nor do they isolate the contribution of the encoder. To isolate the effect of encoding, we performed an ablation experiment on 24 tensors from the Qwen1.5-MoE-A2.7B SNN: at T=44, under comparable spike activity budgets, compared with BinarySpike-only encoding, hybrid encoding reduces aggregate relative L2 reconstruction error by 78.04%, with 0.78% fewer spikes.
In comparing with FAS, we also need to consider differences in the training pipeline. FAS first replaces the activation functions, performs full-parameter fine-tuning, then transfers the weights, and applies layer-wise, neuron-wise calibration; SpikeHENA directly reuses pretrained ANN parameters, without retraining, fine-tuning or additional post-conversion optimization. Therefore, even when the number of time steps is matched, the comparison still reflects differences arising from fine-tuning and calibration.
On OPT-2.7B, at T=16 SpikeHENA achieves 61.39% mean accuracy across seven zero-shot tasks, compared with 61.55% reported for LAS; SpikeHENA uses N=10, M=6, α=0.9.
Table R4: OPT-2.7B zero-shot accuracy (%).
† The ANN and LAS values are from LAS Table 1, and the SpikeHENA values are from our evaluation. We will add the BERT and OPT-2.7B comparisons in Section 3.2, and provide the evaluation settings and encoding ablation in the appendix.
Reference
[1] Long Chen, Xiaotian Song, and Yanan Sun. LAS: Loss-less ANN-SNN Conversion for Fully Spike-Driven Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(3), 1730–1738, 2026.