acceptodds
Withdrawn from ICLR 2027

Bio-Inspired Auditory Processing: Coupling Parallel Resonate-and-Fire Dynamics with Ternary Recurrent Networks for Speech Recognition

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spiking Neural Networks (SNNs) promise energy-efficient inference through sparse, event-driven computation, yet remain constrained on continuous, context-dependent tasks such as automatic speech recognition (ASR). A key bottleneck is the Leaky Integrate-and-Fire (LIF) neuron, whose membrane decay erases information across pauses and phonetic boundaries while it must both extract features and retain long-range context. Inspired by cortical separation of nonlinear filtering and somatic integration, we introduce the Spiking Ternary GRU (stGRU): a ternary-weight gated recurrent unit performs temporal filtering, while downstream Resonate-and-Fire (RF) neurons generate the binary spike stream driving the next block's event pathway. This makes binary event transmission a native training constraint rather than a separate deployment-time activation quantization step; together with ternary weights, learned projections avoid dense floating-point matrix MACs. Trained end-to-end on LibriSpeech-960, Small, Medium, and Large stGRUs reach , , and WER on test-clean without an external language model. To our knowledge, these are the first matrix-MAC-free spiking networks without a learned ANN sequence decoder to achieve single-digit WER on LibriSpeech. The architecture also achieves test CER on Aishell-1 ( on dev) when trained from scratch. Internally, trained networks develop a depth-graded hierarchy: shallow layers act as fast acoustic filters, while deep layers form slower semantic integrators whose latent representations segregate into part-of-speech manifolds and word-level spike events. Deep-layer firing rates follow lognormal distributions with a sparsity gradient, mirroring hierarchical processing observed in human cortex in vivo. Our biologically grounded architecture combines competitive ASR with favorable modeled compute and memory cost while exposing interpretable internal dynamics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Automatic Speech Recognition, Spiking Neural Networks, Ternary, Efficient, GRU, Resonant and Fire, Binary, Leaky Integrate and Fire

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1VrnOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes stGRU, combining ternary MLGRU filtering, parallel resonate-and-fire neurons, and CTC decoding for continuous speech recognition. Experiments on LibriSpeech-960 and Aishell-1 report competitive recognition performance, reduced modeled arithmetic costs, and faster training than matched LIF variants. The paper also interprets depth-dependent activity as an error-monitoring and confidence-processing hierarchy.

Critical strengths

The architecture extends low-precision spiking computation to large-vocabulary ASR. The evaluation includes matched continuous ternary and LIF baselines, six-seed results for the Large RF model, and a streaming variant. Measured training speedups of approximately 3.1–3.4× over LIF are a useful practical result.

Critical weaknesses

W1. The added value of the RF interface is not sufficiently isolated.

MLGRU and PRF are established components, making their combination and demonstrated benefits central to the contribution. However, the Large RF model’s six-seed mean WER of 6.44±0.17% is compared with a single-seed continuous ternary result of 6.49%. The stronger 6.14% result emphasized in Section 4 is the best RF seed reported. Moreover, LIF achieves lower Aishell-1 test CER at every scale, including 10.05% versus 10.88% for Large.

Equation 10 also retains a continuous inter-block residual pathway, so binary spikes are not the sole communication channel. Matched-seed comparisons and controls for the continuous residual and resonant dynamics would clarify the independent contribution of the proposed interface.

W2. The efficiency comparisons do not establish deployment-level benefits.

The energy estimates exclude normalization, nonlinearities, residual scaling, memory traffic, and decoding/search; parameter-storage estimates exclude runtime states and activations. Appendix D.2 also models continuous ANN baselines with replacement CTC readouts, while Table 1 uses literature recognition results. The paper should verify that each accuracy–energy pair describes the same model and decoding configuration. Measured inference latency and runtime memory would strengthen the deployment claims.

W3. The biological interpretation exceeds the evidence.

Outcome selectivity and confidence correlations support descriptive observations, but do not establish ACC/PFC-like functions or an autonomous handoff. The attempted-emission control is useful, yet removing MLGRU changes capacity and increases WER from 6.14% to 11.64%, confounding the interpretation. Cross-seed replication, performance-matched controls, and interventions on the identified layers are needed. Part-of-speech UMAP separation should likewise be supported by held-out probes controlling for lexical identity and acoustic differences.

Questions for authors

Do RF’s benefits persist across matched seeds, and how much do they depend on resonance versus the continuous residual pathway?

Do baseline accuracy and energy estimates correspond to identical systems, and what are the measured inference costs?

Are the proposed functional patterns reproducible and supported by performance-matched controls or causal interventions?

Reviewer_gSPiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a Spiking Ternary GRU, or stGRU, which uses ternary weights for temporal filtering and Resonate-and-Fire neurons to generate binary spike streams. The spike constraint is included during training, rather than being added only at deployment. The model also uses parallel scanning and direct CTC decoding, so it does not need a dense ANN sequence decoder. Experiments on LibriSpeech and AISHELL-1 show competitive speech-recognition performance, and the internal activity patterns appear to form error-monitoring and decision-confidence hierarchies.

Critical strengths

The method is evaluated on both LibriSpeech and AISHELL-1, with comparisons of accuracy, parameter storage, and energy consumption.
By combining ternary weights with binary spike transmission, the method avoids treating spiking as a purely post-training deployment constraint.

Critical weaknesses
  1. The energy results are mainly based on theoretical estimates using a 45 nm CMOS model rather than measurements on real hardware. The model still uses element-wise multiplications, normalization, and residual scaling.
  2. The claimed ACC/PFC-like functional organization is mainly based on similarities in the network’s firing patterns. It should not yet be considered a genuine reproduction of the corresponding functions in neuroscience.
Questions for authors
  1. The energy results are mainly based on theoretical estimates using a 45 nm CMOS model rather than measurements on real hardware. The model still uses element-wise multiplications, normalization, and residual scaling.
  2. The claimed ACC/PFC-like functional organization is mainly based on similarities in the network’s firing patterns. It should not yet be considered a genuine reproduction of the corresponding functions in neuroscience.
Reviewer_mgxrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes stGRU, a spiking architecture for end-to-end automatic speech recognition that combines ternary-weight convolutions, gated linear recurrences, SwiGLU channel mixing, and Parallel Resonate-and-Fire (PRF) neurons. All learnable projections use ternary weights, while binary spikes are used for communication between blocks. The model is trained with CTC without a learned ANN sequence decoder. Experiments on LibriSpeech-960 and Aishell-1 report competitive recognition accuracy with fewer energy consumption.

Critical strengths
  1. The paper demonstrates that a ternary-weight spiking architecture can be trained end-to-end on large-scale speech recognition tasks without relying on a learned ANN sequence decoder.

  2. The paper provides a broad set of ablations, including comparisons between full-precision and ternary weights, different activation and neuron types, scaling strategies, and training throughput. These experiments help characterize the trade-offs between recognition accuracy, computational cost, and architectural choices.

Critical weaknesses
  1. The claimed energy-efficiency advantage is not established under a fair comparison with low-precision ANN baselines. The energy analysis charges ANN models for FP32 MAC operations while evaluating stGRU using low-cost ternary arithmetic. Under the same 45nm energy model, INT8 MACs should be substantially cheaper than FP32 MACs, potentially reversing the reported advantage. Moreover, the estimates exclude memory access, normalization, and other operations that may contribute significantly to deployment cost, particularly for the 329M-parameter model. The ablations also suggest that most of the estimated savings come from ternary weights rather than spiking dynamics.

  2. In Table S7, the LIF variant outperforms RF at all three model sizes On Aishell-1, contradicting the motivation that LIF dynamics are a primary limitation.

  3. The architectural novelty and neuroscience interpretations are somewhat overstated. The main block closely resembles existing combinations of gated linear recurrences, local convolutions, and channel-mixing layers, particularly the MLGRU-based architecture in MatMul-free LM, with PRF neurons added for spike generation.

Questions for authors
  1. Can RF, LIF, ternary ReLU, and a stateless binary activation be compared across multiple seeds under identical training conditions?

  2. How does stGRU compare with parameter- or storage-matched low-precision ANN architectures, particularly quantized Conformers and ternary gated recurrent networks, under the same accuracy and energy evaluation protocol?

1 comment
Reviewer_mgxrOpenReview ↗

No Rebuttal So Far.

I see no rebuttal so far and remain concerned regarding the energy model used, so I am keeping my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.