- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SpikCTU is a compact hybrid spiking architecture for speech command recognition. It constructs a fixed temporal memory containing spike-derived keys and continuous values, then repeatedly accesses this memory through a shared recurrent unit with binary states and lightweight stage adapters. Intermediate predictions combine fixed base evidence with accumulated corrections, supporting either full-depth aggregation or confidence-based early exit.
Experiments on SHD, SSC, and GSC demonstrate competitive accuracy with approximately 0.16–0.17M parameters. The paper includes component ablations, five-seed results, recurrent-trajectory analyses, temporal occlusion experiments, and operation-based energy estimates.
Critical strengths
Compact architecture with competitive performance. The parameter counts are substantially lower than several strong SSC and GSC baselines. Separating fixed memory construction from shared recurrent computation provides a coherent approach to controlling model size.
Useful evidence for adaptive recurrent computation. At matched average recurrent depth, the selected early-exit policies outperform interpolated truncation baselines by 0.81 and 1.21 percentage points on SSC and GSC. The paper appropriately distinguishes truncating one trained model from independently training models at different depths.
Substantial diagnostic analysis and transparent limitations. The paper examines both corrections and degradations across recurrent steps. Temporal occlusion provides stronger evidence of input relevance than attention visualization alone. The appendix also reports seed variability and clearly separates recurrent-step savings from total proxy-energy savings.
Critical weaknesses
W1. The practical efficiency benefit of adaptive recurrence is limited by the fixed computation. Section 5.4; Appendix D, Tables 11–12.
The reported 59% and 67% reductions concern recurrent evaluations. Under the paper’s own proxy, reducing inference to one recurrent step lowers total energy by only 0.76%, 4.90%, and 1.50% on SHD, SSC, and GSC, respectively. Thus, substantial savings in recurrent depth do not establish substantial savings in total inference cost.
The appendix acknowledges this limitation, but the selected early-exit policies are not accompanied by their own total-operation or energy results. Moreover, cross-model energy comparisons use different accounting conventions, and the proxy excludes memory movement, normalization, nonlinearities, and control overhead. This limits conclusions about deployment efficiency.
The paper should report total costs for the actual selected policies and use consistent accounting for the most relevant comparisons. A measured latency result on an available platform would further clarify whether adaptive execution provides a practical benefit; specialized neuromorphic hardware is not essential.
W2. The experiments do not sufficiently isolate the contribution of the spiking recurrent design. Sections 3 and 5.2; Tables 2 and 7.
The component ablations and independently trained depth sweep establish that several design choices matter within SpikCTU. However, they do not establish whether binary recurrent dynamics provide an advantage over a comparably sized continuous recurrent model using the same encoder, memory, supervision, and stopping rule. The “continuous K/V memory” ablation changes the memory representation rather than testing this alternative.
A base-evidence-only control is also missing, although every intermediate prediction is anchored to that predictor and most computation occurs before recurrence. These controls would help separate the contributions of the encoder, shared parameterization, recurrent refinement, and spiking dynamics.
Confidence-based early exit is already established in SNNs, including SEENN. The paper recognizes this literature and distinguishes recurrent depth from sensory timesteps, but the empirical case for the specific proposed combination remains incomplete.
W3. Important implementation details require clarification for reproducibility. Section 3.1; Appendix A.1.
The GSC configuration specifies an 8-kHz sample rate together with (f_{\max}=14{,}000) Hz. The paper explicitly retains this inherited setting, but 14 kHz exceeds the 4-kHz Nyquist frequency. The effective filterbank behavior—including clipping, empty filters, or any difference between the actual and stated sample rate—needs to be documented because it affects the representation and comparison with other preprocessing pipelines.
There is also an apparent ambiguity between the token-averaging head used to construct base evidence in Section 3.1 and the statement in Appendix A.1 that an auxiliary head averaging token predictions is disabled. These may be different modules, but their distinction and active configuration should be explicit.
Questions for authors
How do the selected early-exit policies perform across seeds in accuracy, total operations, and total proxy energy?
What performance is obtained from the base predictor alone and from a parameter-matched continuous recurrent counterpart under the same training and evaluation protocol?
What is the effective GSC sampling/filterbank configuration, and how does the disabled auxiliary head differ from the base-evidence head?
4 comments
Thank you for the detailed response and the additional experiments. The results across five seeds, measured latency, continuous counterpart, and preprocessing clarification substantially address several of my original concerns. I have three remaining points.
- Efficiency claims and timing protocol. The new measurements provide useful evidence of reduced batch-1 model-forward latency on the evaluated GPU. I encourage the authors to emphasize this measured benefit while clearly distinguishing it from the modest total proxy-energy savings and reporting the associated accuracy changes. Please clarify the timing protocol, including CUDA synchronization, whether confidence computation and early-exit decisions are included, and how the 236 examples were selected.
- An independently trained non-recurrent baseline. The base-branch results show that the jointly trained model relies on recurrent corrections. Since this branch receives no independent classification loss, its low standalone accuracy leaves open how well a non-recurrent classifier could perform when trained directly. A baseline using the same encoder and training budget, with an independently supervised classification head and no recurrent refinement, would help establish the benefit of recurrence.
- Stability of the binary-versus-continuous comparison. The parameter-matched continuous counterpart is a useful addition. Paired multi-seed comparisons on SSC and GSC would further clarify the consistency of the relatively small accuracy differences. Please also distinguish the combined effect of binary outputs and persistent neuronal dynamics from the contribution of binarization alone.
I recognize the limited time remaining in the discussion period. Additional experiments would strengthen the evidence, but I would also consider a careful analysis of the existing results, a clear explanation of the remaining limitations, and appropriately scoped claims sufficient grounds to reconsider my current rating upward. My reassessment will reflect the overall contribution and how convincingly these remaining points are addressed.
We sincerely thank you for your follow-up. We tried to address your remaining questions with additional experiments.
1. Efficiency claims and timing protocol.
- Efficiency claims. As shown in the following table, early exit substantially reduces measured batch-1 model-forward latency, with the largest gains on SSC and GSC, while preserving accuracy close to full aggregation. In the revised version, we will clarify this point.
| Task | Accuracy (%) | SOPs (M) | MACs (M) | Proxy energy (mJ) | Mean latency (ms), B=1 |
|---|---|---|---|---|---|
| SHD | 96.60 → 95.63 | 1.360 → 1.341 | 2.786 → 2.767 | 0.0140 → 0.0139 | 6.10 → 4.62 |
| SSC | 81.74 → 81.41 | 1.606 → 1.505 | 3.122 → 3.028 | 0.0158 → 0.0153 | 15.39 → 7.08 |
| GSC | 95.96 → 95.99 | 0.616 → 0.548 | 4.895 → 4.841 | 0.0231 → 0.0228 | 9.00 → 4.83 |
- Timing protocol. Latency was measured with synchronized CUDA events and includes confidence computation and the exit decision. We used the first 256 test examples from the data loader, discarded 20 warm-up examples, and used the remaining 236 examples to compute the average latency.
2. An independently trained non-recurrent baseline.
Following your comment, we additionally conducted an experiment to evaluate the effectiveness of recurrent refinement. We trained a non-recurrent classifier with the same encoder, a 300-epoch budget, an independently supervised classification head, and a similar parameter count, as shown in the following table.
| Task | SpikCTU accuracy / parameters | Non-recurrent accuracy / parameters |
|---|---|---|
| SSC | 81.74% / 0.174 M | 73.13% / 0.167 M |
| GSC | 95.96% / 0.169 M | 90.54% / 0.167 M |
At nearly the same parameter count, SpikCTU improves accuracy by 8.61 points on SSC and 5.42 points on GSC. These results demonstrate the effectiveness of recurrent CTU refinement beyond the encoder and classification head.
3. Paired multi-seed comparison with the continuous counterpart.
We added a multi-seed evaluation with five seeds on the SSC benchmark to compare SpikCTU with a continuous counterpart. The results are provided in the following table and show comparable performance between the two models. The continuous counterpart only replaces the three recurrent PLIF activations with scalar PReLU activations. However, the SNN model has higher energy efficiency.
| Evaluation | SNN | Continuous |
|---|---|---|
| SSC accuracy (%) | 81.39 ± 0.22 | 81.46 ± 0.12 |
| Energy cost (mJ) | 0.015807 | 0.018205 |
The multi-seed evaluation on GSC is ongoing.
4. Claims and limitations.
From these results, SpikCTU achieves competitive accuracy and energy efficiency compared with strong existing baselines, while early exit further reduces inference latency. However, despite the relatively small model size of 0.16–0.17 M parameters, the energy efficiency remains constrained by the hybrid spiking embedding module, namely the hSEM module. Consequently, the energy savings brought by early exit are relatively modest. We will clarify these limitations in the revised manuscript.
Once again, we appreciate the reviewer’s helpful feedback.
We have completed the GSC comparison with five paired seeds. The continuous counterpart replaces the three recurrent PLIF activations with scalar PReLU activations. The table reports mean ± standard deviation at full recurrent depth.
| Evaluation on GSC | SpikCTU (hybrid) | Continuous |
|---|---|---|
| Accuracy (%) | 95.78 ± 0.13 | 95.83 ± 0.18 |
| Proxy energy (mJ/sample) | 0.023075 ± 0.000008 | 0.024283 ± 0.000012 |
The mean accuracy differs by 0.04 percentage points, while the continuous counterpart uses 5.24% more proxy energy. Thank you for suggesting this comparison.
We thank you for the detailed review and for recognizing the compact architecture, adaptive computation, and diagnostic analysis. We are glad to address these questions you raised.
Actual policies and variation across seeds. We evaluated all 15 original paper checkpoints using the Table 3 thresholds unchanged across the five seeds, h = 0.01/0.84/0.91 for SHD/SSC/GSC. New CUDA FP32 evaluations on an NVIDIA A800 preserve original input order and padding. Values are mean ± sample standard deviation.
Arrows below denote full → exit. SOPs/MACs are millions per sample, averaged across seeds.
Exit-energy standard deviations are 0.00001069/0.00001278/0.00000775 mJ. We count fixed encoding and memory work plus each sample's recurrent prefix, verified with online single-sample probes. The counts represent independent stopping with the original padding. Fixed computation dominates total energy, explaining the smaller savings relative to recurrent-step reductions.
The shared proxy uses 0.9 pJ/SOP and 4.6 pJ/MAC and excludes memory traffic, normalization, nonlinearities, and control overhead. The binary and continuous controls below use this same counter, providing a consistent cost comparison.
Separate A800-SXM4-40GB batch-1 measurements give mean model-forward latency of 6.10 → 4.62, 15.39 → 7.08, and 9.00 → 4.83 ms on SHD/SSC/GSC. They cover 236 examples after warm-up and exclude feature extraction, transfer, and external reset. Thus, this implementation shows a larger latency reduction than the arithmetic-energy proxy reduction.
Continuous counterpart and base prediction. Historical controls use the same encoder, fixed hybrid memory, supervision, task training recipe, and exact parameter count. The three recurrent PLIF activations are replaced by scalar PReLUs. We evaluate both versions using the same GPU protocol and stopping thresholds. For seed 312:
The binary design uses less proxy energy with higher SHD/SSC accuracy and similar GSC accuracy in these paired runs. Across five SHD seeds, full accuracy is 95.51 ± 0.71% for binary and 95.14 ± 0.64% for continuous. The replacement changes both output domain and activation persistence, so it evaluates that combined design. The base-only values are branch readouts from the jointly trained model, with recurrent corrections removed at evaluation. This branch receives supervision through the combined predictions and has no separate classification loss.
Relation to SEENN. Confidence stopping is established in SNNs. Here it controls updates after sensory encoding and fixed memory construction. The contribution is the compact memory-reading recurrent architecture and its intermediate outputs. In the submitted paper, adaptive allocation improves accuracy over interpolated truncation of the same trained model by 0.81/1.21 points on SSC/GSC at equal average recurrent depth.
GSC preprocessing and heads. Audio is resampled from 16 to 8 kHz before a 256-point FFT, giving 129 bins over 0–4 kHz. The inherited 14-kHz upper mel setting is applied to the 0–4-kHz spectrum and produces 53 empty filters out of 140. Setting the upper frequency to 4 kHz leaves 7 empty filters because of finite FFT resolution. Separate follow-up training runs with the same recipe reach 95.75% and 95.87%, respectively, one run per setting.
The active base head classifies composed tokens, averages their class probabilities over valid tokens, and takes the logarithm. The disabled auxiliary token head is a separate optional branch. Its removal does not disable base evidence or intermediate recurrent supervision.