acceptodds
Under review as a conference paper at ICLR 2027

STFSFormer: Integrating Neuronal Dynamics and Full-Spike Attention for Energy-Efficient EEG Modeling

Abstract

Efficient electroencephalography (EEG) modeling requires representations that retain rapid signal variations while capturing temporal dependencies under limited energy budgets. Spiking neural networks provide an energy-efficient computing substrate, but conventional leaky integrate-and-fire dynamics can attenuate rapid changes through temporal smoothing, while global attention incurs quadratic computational cost in sequence length. We introduce STFSFormer, a spiking Transformer that integrates transient-sensitive neuronal dynamics with local full-spike attention for energy-efficient EEG modeling. A high-frequency-enhanced leaky integrate-and-fire neuron combines a differential synaptic pathway with refractory feedback to emphasize input changes and regulate repeated firing. Parallel encoding branches with distinct decay profiles capture complementary temporal responses. Spatial-temporal full-spike attention integrates these representations using binary queries, keys, and values within local temporal blocks. Its softmax-free formulation selects the cheaper of two equivalent multiplication orders based on block length and head dimension, with attention cost scaling linearly in sequence length for fixed block size and feature dimension. Experiments on SEED, SEED-V, and DREAMER demonstrate competitive emotion recognition performance. Under the evaluated input configuration, STFSFormer uses 1.80 million parameters and 0.46 billion multiply–accumulate operations per segment. A consistent synthesis-based FPGA evaluation estimates whole-model inference power at 1.2 W and energy consumption at 1.5 mJ per segment, corresponding to 50.0% and 44.4% lower energy than the implemented SpikingGCN and SCNN baselines, respectively. These results demonstrate a favorable balance between recognition performance, computational cost, and energy-efficient inference.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Electroencephalography, Emotion recognition, Spiking neural networks, Transformer, Energy-efficient inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_37y8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes STFSFormer, a spiking Transformer for EEG emotion recognition. It combines HELIF neurons with differential drive and refractory feedback, a multi-decay-branch encoder, and local spiking attention, and selects the attention multiplication order based on matrix dimensions. The authors conduct within-subject evaluation, cross-subject evaluation, and component ablations on SEED, SEED-V, and DREAMER, and report model complexity and FPGA synthesis energy estimates.

Critical strengths
  1. The paper considers both the representation ability for time-varying EEG and inference cost, and combines neuron dynamics with local attention into a fairly complete model design.

  2. The authors first split complete trials and then generate non-overlapping windows, avoiding windows from the same trial entering different data sets; the LOSO evaluation also clearly isolates the test subject and all their sessions.

  3. Besides recognition performance, it also includes component ablations, hyperparameter sensitivity, firing rate, and complexity analysis. The HELIF and attention replacement experiments show that both components contribute under the current configuration.

Critical weaknesses
  1. The main text says it uses window-level Accuracy and Macro-F1, but in Table 1, Brain-GRL on DREAMER-Arousal reports 85.27% and 87.05% for the two metrics. For the same set of binary predictions, same sample weights, and a consistent aggregation method, standard Macro-F1 should not exceed Accuracy. Similar cases appear in other baselines, so the metric definitions, aggregation method, or table values need clarification.

  2. Choosing between two equivalent multiplication orders based on dimension is essentially a deterministic computation-order optimization. The default configuration and the reported block-length sweep both satisfy L<dhL<d_h, so the same branch is always used, and there is no empirical validation of the benefit of switching branches. The existing ablations are also not enough to separate the contributions of the local window, spike representation, and multiplication-order selection.

  3. SEED LOSO accuracy improves over Spikformer by only 0.59 percentage points, and some DREAMER metrics are lower than EEGNet. The paper does not report per-subject paired analysis or confidence intervals, so it is impossible to tell whether these gains are stable.

  4. The HELIF ablation removes both the differential path and refractory feedback at the same time, so the contribution of each part cannot be determined; the lack of frequency-response and noise-robustness analysis also makes it hard to tell whether the enhanced variations correspond to useful EEG information. In addition, there remains a gap between offline zero-phase filtering and whole-trial normalization and continuous online deployment.

Questions for authors
  1. In Table 1, how are Accuracy and Macro-F1 aggregated across windows, subjects, and random seeds? Can you provide the evaluation code and corresponding confusion matrices to explain the cases where binary Macro-F1 is higher than Accuracy?

  2. Can you add attention controls under the same local window, similar parameter count, and comparable compute budget, and separately measure the actual latency and memory cost of fixed token-first, fixed feature-first, and the dimension-selection strategy?

  3. Can you report per-subject LOSO results, paired differences, and confidence intervals, and add stronger EEG emotion recognition baselines to verify the generalization advantage?

  4. What device, frequency, numerical precision, and power analysis flow are used for the FPGA estimates? Which memory and data movement costs are included in the energy accounting, and do all baselines use comparable implementation optimizations?

  5. Can you separately ablate differential drive and refractory feedback, and validate performance under causal filtering, online normalization, and noise perturbation, to support the neuron mechanism and continuous deployment motivation?

1 comment

We thank the reviewer for recognizing the trial-before-window split, strict subject isolation, and the combined evaluation of representation and efficiency. We address the reporting inconsistency first, followed by the attention controls and online preprocessing.

Corrected Table 1 and metric aggregation

We thank the reviewer for identifying the inconsistency between Accuracy and Macro-F1. The cause was a logging error in our result-saving script: during an earlier hyperparameter-tuning run, the script wrote the Accuracy value into the Macro-F1 field, and these stale entries were carried into Table 1. We have fixed the script, regenerated the affected entries from the correct evaluation logs, and updated the PDF. The corrected values are computed from the same pooled predictions, so no experimental results or conclusions change.

For each seed, we pool the test windows from all subjects and compute Accuracy and Macro-F1 once from that pooled prediction set, without trial voting or an equal-subject average. Accuracy is the fraction of correctly classified windows, and Macro-F1 is the unweighted mean of class-wise F1 scores. Table 1 reports the mean and standard deviation of the five seed-level values. This aggregate is window-weighted: subjects with more test windows contribute more observations.

Attention controls and measured execution cost

We compare fixed token-first, fixed feature-first, and dimension selection using the same checkpoint, inputs, and batch. The supplementary measurements use one A10 with batch=128, T′=100T'=100, L=3L=3, and dh=64d_h=64:

ImplementationComplete batch forward (ms)Incremental peak memory (MiB)
Fixed token-first176.68178.0
Fixed feature-first177.48401.6
Dimension selection176.80178.0

The isolated token-first and feature-first cores take 0.145 and 1.589 ms, respectively. At the equal-core-MAC boundary L=dh=64L=d_h=64, the two orders have similar core latency and both use 32 MiB. These measurements show that dimension selection primarily reduces core computation and memory requirements, while complete-model forward latency remains similar in the tested configuration. The benefit is an execution-cost optimization, rather than an accuracy improvement from reordering mathematically equivalent products.

Separate neuron mechanisms and online preprocessing

The additional ablations separately remove differential drive and refractory feedback, reducing average accuracy by approximately 3.22 and 2.35 percentage points, respectively. These controls address the contribution of each mechanism beyond the original combined HELIF/LIF replacement.

The original pipeline uses offline zero-phase filtering, within-trial normalization, and state resets at each EEG window. Replacing only the zero-phase filter with a causal filter reduces accuracy by approximately 0.95 percentage points. Introducing 1-second EMA online normalization gives a drop of approximately 13.72 points. Lengthening the EMA time constant to 10 seconds improves accuracy by approximately 4.08 points relative to the 1-second configuration. These results identify online normalization as an important remaining gap between the offline evaluation and continuous deployment.

On trial-isolated SEED, 20-45 Hz narrow-band noise at 0 dB reduces accuracy by approximately 0.30 percentage points, while EMG-like bursts cause a drop of 1.29 points. EOG-like and step artifacts produce larger drops of 5.93 and 8.67 points, respectively. These results indicate greater sensitivity to the tested EOG-like and transient disturbances than to the narrow-band noise condition. If accepted, we will incorporate these controls and distinguish efficient model inference from a validated continuous EEG acquisition and preprocessing system.

The FPGA device, clock configuration, numerical precision, power-analysis flow, and memory-accounting scope are addressed in the shared FPGA response under General Response.

Reviewer_VAvsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents STFSFormer for EEG-based emotion recognition. The framework combines multi-scale temporal encoding, a differential-driven spiking neuron (HELIF) with refractory feedback, and parallel branches with different decay profiles. The paper is well organized and clearly written. However, its novelty over existing methods, the evidence for frequency complementarity, and the overall efficiency benefits require further clarification and validation.

Critical strengths

The paper is well organized and clearly written.

Critical weaknesses
  1. Binary QKV, softmax-free attention, and multiplication reordering have been explored previously, while Max-Former[1] has addressed high-frequency preservation. The methodological novelty and empirical advantages of HELIF require further clarification.

  2. Different decay factors do not necessarily yield complementary frequency responses, and the selection of branch-specific time constants remains unclear. Whether the performance gains arise from frequency complementarity therefore requires further evidence.

  3. Local partitioning restricts direct interactions, while real-valued projections, multiple encoding branches, and additional neuronal states introduce overhead. These costs should be included in the overall efficiency evaluation.

[1] Fang Y, Zhou D, Wang Z, et al. Spiking neural networks need high-frequency information[J]. Advances in Neural Information Processing Systems, 2026, 38: 103404-103425.

Questions for authors
  1. What EEG time interval does each HELIF update represent after temporal downsampling, and is this interval consistent across datasets?

  2. Are the decay coefficients (\gamma,\eta,\lambda) fixed or trainable? How are they initialized, and are they shared across channels or layers?

  3. How sensitive is recognition accuracy to the local block length (L) and the starting positions of the temporal partitions?

  4. How robust is HELIF to high-frequency noise and EEG artifacts? Could the authors evaluate whether the differential pathway amplifies these disturbances?

2 comments

We thank the reviewer for the clear assessment and the questions about temporal interpretation, frequency complementarity, and robustness. These questions help distinguish the neuron mechanism from the surrounding architecture.

Relation to prior work and interpretation of the branches

We agree that high-frequency preservation, binary Q/K/V, and softmax-free attention are established ideas. Max-Former addresses high-frequency preservation through Max-Pool in patch embedding and depth-wise convolution replacing attention. HELIF instead modifies the temporal neuron recurrence, combining differential drive with a decaying refractory trace:

f[t] &= \gamma f[t-1] + x[t] - x[t-1],\\ v[t] &= \eta v[t-1] + f[t] - V_{\mathrm{th}}r[t-1],\\ r[t] &= \lambda r[t-1] + s[t]. \end{aligned}$$ The distinction lies in the recurrent neuron dynamics. We do not claim to introduce frequency preservation or attention associativity. Different decay values alone do not prove task-relevant spectral complementarity. For the linear synaptic recurrence with zero initial state, $$H_\gamma(z)=\frac{1-z^{-1}}{1-\gamma z^{-1}}.$$ This describes different temporal responses and suppression of constant input; it is not the transfer function of the complete nonlinear spiking neuron. Appendix A states that the branches do not impose predefined EEG bands. The five branches use the specified decay values to provide distinct temporal responses. We will clarify this interpretation and the relation to Max-Former in the final manuscript. **Update interval and fixed coefficients** With temporal downsampling by AvgPool1d(2,2), consecutive HELIF updates are separated by $\Delta t=2/f_s$. The intervals are therefore 10 ms on SEED (200 Hz), 8 ms on SEED-V (250 Hz), and 15.625 ms on DREAMER (128 Hz). The configuration fixes a two-sample step; the physical update interval differs across datasets. The decay coefficients are fixed hyperparameters. The five branch values are {0.81, 0.85, 0.89, 0.93, 0.97}, with $\eta=0.89$ and downstream $\gamma_{\mathrm{shared}}=0.89$. The refractory coefficient $\lambda$ is also fixed during training. Channels within a branch share its $\gamma$; downstream attention and MLP neurons share $\gamma_{\mathrm{shared}}$. These coefficients are not separately learned per channel or layer. Neuronal membrane and refractory states remain separate. **Partition sensitivity and noise robustness** In the additional block-length experiment over L in {1,2,3,4,8}, the maximum accuracy spread is approximately 3.57 percentage points, with L=3 performing best. This is a supplementary sweep, distinct from the original Figure 3. For a fixed trained model at L=3, changing only the inference partition offset among 0, 1, and 2 tokens gives a maximum spread of approximately 0.23 percentage points. On the trial-isolated SEED evaluation, 20–45 Hz narrow-band noise at 0 dB and EMG-like bursts reduce accuracy by approximately 0.30 and 1.29 percentage points. EOG-like and step artifacts cause larger drops of 5.93 and 8.67 points. The no-differential control outperforms the full model under the tested white-noise, EOG-like, and step perturbations by 4.16, 3.04, and 5.31 points, respectively. Thus, the clean-data benefit does not imply uniform robustness to noise. Accuracy changes also do not directly measure disturbance-amplitude amplification inside HELIF. We will report these outcomes and qualify the frequency-enhancement claim accordingly. The FPGA implementation and power-accounting details are provided in General Response. If accepted, we will incorporate the supplementary sensitivity and robustness results and clarify the temporal and frequency interpretation in the final manuscript.

Thank you for the response. However, I remain concerned about how the fixed decay coefficient was selected. The authors have not provided an ablation study comparing different values, leaving this choice insufficiently justified. I therefore maintain my current score.

Reviewer_gntpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

STFSFormer combines a differential-input and refractory-feedback spiking neuron, parallel temporal-response branches, and block-local softmax-free spiking attention for EEG emotion recognition. It selects between two associative attention multiplication orders from tensor dimensions. Experiments cover subject-dependent recognition on SEED, SEED-V, and DREAMER, a strict LOSO evaluation on two datasets, component ablations, model complexity, and synthesis-based FPGA power and energy estimates.

Critical strengths
  1. The trial-before-window split and the subject-disjoint LOSO protocol are explicitly described, including keeping all sessions of a held-out SEED subject together. These are consequential safeguards against correlated EEG windows crossing evaluation partitions. The three-dataset evaluation and the neuron/attention substitution study provide useful application evidence.

  2. The paper clearly defines which activations are binary and which operations remain real-valued. It also distinguishes the operation-level energy model from the whole-model FPGA estimate and acknowledges that EEGNet is smaller and faster in the GPU comparison. This makes the intended accuracy–cost trade-off reasonably clear.

Critical weaknesses
  1. The central FPGA efficiency result is not reproducible from the reported hardware protocol. Section 4.5 and Table 5 give 1.2 W and 1.5 mJ per segment, but the manuscript and appendices do not identify the FPGA part, synthesis/tool configuration, actual clock, numeric representation, achieved segment throughput/latency, or activity assumptions used for power estimation. Resource percentages cannot be interpreted without the device capacity. It is also unclear how memory traffic and the real-valued projections, normalization, residual operations, and neuronal state updates are covered. These details are necessary to assess the headline 50.0% and 44.4% energy reductions against the two implemented spiking baselines; the separate arithmetic model in Appendix F does not validate a whole-model FPGA estimate.

  2. The attention contribution is not separated from ordinary locality and existing softmax-free SSA. Equation 4 uses matrix associativity, with the lower-cost order determined by block length versus head width. All reported defaults use L=3 and d_h=64, so the token-first branch is always selected. The resulting small local attention operator has linear sequence scaling for fixed block size, a property also available to conventional block-local attention. The paper appropriately acknowledges this in Appendix C, but Table 3 replaces STFSAtten with spatial-only SSA and therefore does not isolate the proposed multiplication rule or the benefit of the particular local operator against a matched local SSA baseline. Similarly, the HELIF/LIF substitution changes differential processing and refractory history together. The current evidence leaves the methodological advance over combining established components unclear.

The application results are encouraging, but the main efficiency claim needs a checkable hardware protocol and the methodological contribution needs a more discriminating comparison. Those are the reasons for the recommendation, rather than a requirement to lead every recognition metric.

Questions for authors
  1. Can you provide the FPGA device and tool versions, precision/quantization configuration, clock, achieved latency/throughput, power activity methodology, and included memory/operator scope for every Table 5 model? Please include the accuracy of the implemented numerical model, so that the energy comparison can be evaluated at the reported recognition quality.

  2. With width, branch count, training protocol, and temporal neighborhood held fixed, how does STFSAtten compare with ordinary block-local SSA? What changes when the differential pathway and refractory trace are removed separately? These controls would distinguish a new neuronal/attention effect from locality and additional state.

1 comment

We thank the reviewer for the detailed assessment and for recognizing the trial-isolated evaluation, strict LOSO protocol, and distinction between binary activations and real-valued operations. We address the hardware evidence and the attribution of the attention/neuron effects separately.

Matched local attention and the role of multiplication order

We agree that binary Q/K/V, softmax-free attention, and matrix associativity are established ingredients. At the reported default L=3 and dₕ=64, the rule always selects token-first evaluation. Its purpose is to choose the lower-cost execution order, not to change the attention output or improve accuracy through reordering.

Under a common spatial backbone, the new block-local SSA and the Spatial-only SSA used in the paper both use width 256, five encoding branches, two network layers, and the same training protocol. STFSAtten and block-local SSA use the same local spatiotemporal neighborhood (L=3). The training budget is 100 epochs.

The block-local SSA and spatial-only SSA controls have similar average accuracy, and both are below STFSAtten in this evaluation. This comparison addresses locality more directly than the original Table 3. The architectural comparison is distinct from an algebraically equivalent multiplication-order comparison; an accuracy difference cannot be attributed to reordering alone.

Separate differential and refractory ablations

In the additional single-component ablations, removing differential drive reduces average accuracy by 3.22 percentage points, while removing refractory feedback reduces it by 2.35 points.

Deep Q/K/V firing rates increase from approximately 13–17.5% to 41–43% and 47–48%, respectively. These controls support contributions from both mechanisms to recognition and sparse activity under the tested configuration, beyond replacing HELIF with LIF as a single combined change.

If accepted, we will incorporate the matched-local controls and separate neuron ablations into the final manuscript. The FPGA device, tools, precision, clocks, and memory-accounting scope are detailed in the shared FPGA response.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.