acceptodds
Under review as a conference paper at ICLR 2027

Exploiting Temporal High Frequencies in Spiking Transformers

Abstract

Existing spiking Transformers flatten the temporal dimension into the batch axis and average over time, which is equivalent to low-pass filtering each pixel's temporal signal. We find that the temporal high-frequency components discarded by this paradigm carry discriminative information: although they account for only 18.1% of the signal energy, masking them at the input layer reduces accuracy by up to 6.2%. We further reveal a compensatory amplification mechanism: as spiking dynamics attenuate these components with depth, learnable frequency gates partially offset this decay, raising their effective utilization from 17.9% to 45.7% on CIFAR10-DVS. We propose a Spiking Spatial-Temporal Wavelet Transformer (SSTWformer), which embeds spatial-temporal Haar transforms into spiking Transformers to explicitly decompose features into eight spatial-temporal subbands for structured learning across multiple frequency bands.Experiments on six datasets spanning object recognition (CIFAR10-DVS, N-Caltech101, NCARS), gesture recognition (DVS128),and speech recognition (SHD, SSC) demonstrate that SSTWformer consistently outperforms spatial-only baselines and the attention-based Spikformer, achieving a 1.81% average accuracy gain and state-of-the-art (SOTA) on multiple tasks.The code is available in the supplementary material.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Spiking Transformer, Spatial-Temporal Haar Transform, Frequency Modeling, Multi-Scale Temporal Dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XS1LOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SSTWformer, which combines spatial and temporal Haar transforms into eight subbands and learns frequency-specific gating and processing for event-based Spiking Transformers. It reports comparisons on four neuromorphic vision datasets and two spiking audio datasets, plus NoTime/component ablations, temporal-window tests, subband perturbations, and gate/energy profiles. These findings support the usefulness of adding explicit temporal-frequency processing, but not all proposed mechanistic and efficiency interpretations are equally established.

Critical strengths
  1. A focused temporal-frequency extension is evaluated on several datasets and with a relevant spatial-wavelet comparison. The best reported T=16 accuracies exceed SWFormer on CIFAR10-DVS (84.90% vs 82.20%) and N-Caltech101 (87.48% vs 84.93%), and the paper includes existing temporal-module baselines as well as the six-dataset summary (Tables 1–2).
  2. The NoTime and input-subband interventions provide evidence beyond headline accuracy. Removing temporal decomposition reduces CIFAR10-DVS accuracy from 84.90% to 83.60%, and masking temporal-high subbands reduces it to 78.70%. Analogous N-Caltech101 results are reported. These checks support a functional role for the added temporal-frequency path in the trained system (Tables 3 and 6).
Critical weaknesses
  1. Method-specific attribution and novelty relative to existing wavelet/temporal models remain incomplete. Spatial and temporal Haar transforms are standard separable operators, while SWFormer already introduces wavelet-based spiking frequency processing. More importantly, the NoTime variant already improves over SWFormer by 1.40 points on CIFAR10-DVS (83.60 vs 82.20) and 1.42 points on N-Caltech101 (86.35 vs 84.93). The incremental temporal-decomposition contributions are 1.30 and 1.13 points, respectively—substantial, but not the whole headline improvement. Model capacity also differs (2.59M vs 2.10M parameters for the listed models). These comparisons do not isolate temporal Haar decomposition from changes in the processor/gating architecture or compare with a parameter-matched temporal convolution/mixer.
  2. Strong mechanistic claims exceed the interventions. Average pooling of time-indexed network outputs does not by itself prove that all previously encoded high-frequency information is irreversibly lost, because temporal dynamics and nonlinear feature transformations occur upstream. Larger degradation under masked/noisy temporal-high inputs measures learned sensitivity, not uniquely structural information waste; energy-matched noise may also produce off-distribution corruption. The claimed 'effective utilization' in Table 7 is calculated from subband energy and gate magnitudes, not a direct downstream information or attribution measure. A larger deep-layer gate, by itself, does not causally establish compensation for information lost through spiking dynamics.
  3. The temporal Haar transform operates on adjacent time steps and is not shift-invariant. Consequently, small temporal shifts or changes in event binning may significantly alter the extracted high-frequency components. Although the paper evaluates different temporal window lengths, it does not establish whether the proposed method is robust to such temporal variations.
Questions for authors
  1. Under the same backbone, time window, and comparable parameter/operation budgets, how does 1D temporal Haar processing compare with a simple temporal convolution or temporal mixer? Please isolate the incremental gain due to the temporal transform from the changed frequency processor.
  2. How do the authors distinguish lower sensitivity of SWFormer to high-frequency masking from genuinely weaker use of discriminative high-frequency information? Would a distribution-controlled perturbation or selective high-frequency reconstruction clarify the interpretation?
  3. Can the compensation claim be tested by freezing or selectively suppressing deep-layer temporal-high gates after training and measuring accuracy and feature dependence, rather than relying on the energy-weighted proxy alone? Also distinguish kernel/software speed differences in Table 8 from intrinsic operation savings.
  4. How robust is SSTWformer to small temporal shifts or changes in event-binning alignment? Evaluating performance under one-step temporal shifts or mild temporal jitter, compared with SWFormer, would help determine whether the improvements generalize beyond a particular temporal alignment.
5 comments

Q1.

We thank the reviewer for the valuable comments. Under the same backbone and training settings, we replaced only the temporal modeling module and compared Temporal Conv with Temporal Attention. The results are as follows:

Table 2 Performance of SSTWFormer under different temporal modeling approaches

ModelFLOPsParams (M)CIFAR10-DVS (%)N-Caltech101 (%)
Notime-2.5983.686.35
HaarO(T)O(T)2.5984.9(+1.3)87.48(+1.13)
ConvO(C2T)O(C^2T)2.7383.8(+0.2)86.39(+0.04)
Attention$O(T^2C+TC^2) $3.183.2(-0.4)85.9(-0.45)

Based on the above results, we elaborate from three points:

(1) Best accuracy with the fewest parameters: Haar outperforms Conv and Attention on both datasets by 1.10/1.70 and 1.09/1.58 percentage points, respectively, while having the smallest parameter count.

(2) Lowest complexity: the temporal Haar transform is a fixed, invertible transform using only addition and subtraction, with O(T) complexity and no learnable parameters, whereas Conv is O(C²T) and Attention is O(T²C+TC²). Haar thus achieves higher accuracy at lower additional cost.

(3) Learnable temporal operators bring no benefit: compared with NoTime, Conv improves by only 0.20%/0.04%, while Attention decreases by 0.40%/0.45%. This indicates that simply stacking learnable temporal operators does not stably improve event data modeling, whereas Haar provides an effective temporal-frequency prior through fixed low/high-frequency decomposition.

In summary, temporal Haar shows advantages in accuracy, parameter count, and computational complexity. The relevant comparative experiments and discussion will be added to the revised manuscript.

Q2.

We thank the reviewer for pointing out this issue. We have added distribution-controlled Pair-Swap perturbations and selective high-frequency restoration experiments to examine, respectively, the model's dependence on temporal high-frequency structure and whether restoring genuine high-frequency information can bring positive gains.

First, we clarify the experimental setup: subbands such as LL_high are all defined by the spatio-temporal Haar decomposition at the input level, rather than being explicit modules inside the model. For both models, we adopt exactly the same pipeline: first decompose the original event input, then intervene on the specified subband, then reconstruct an ordinary event tensor via the inverse Haar transform, and finally feed the same reconstructed input to SWFormer and SSTWformer, respectively. Therefore, this experiment compares the functional dependence of the two models on the same input-level temporal high-frequency information.

(1).Distribution-controlled Pair-Swap perturbation Pair-Swap keeps temporal-low unchanged by swapping each pair of adjacent frames, turning temporal-high into its negation while keeping the total spike count and total energy unchanged. Mask T-High instead zeroes the four temporal-high subbands and reconstructs the remaining subbands. The results are shown in Table1.

Table 1 Results of the model under distribution-controlled Pair-Swap perturbation

DatasetModelCleanMask T-HighPair-Swap
CIFAR10-DVSSWFormer82.278.30 (-3.90)80.80 (-1.40)
SSTWformer84.978.70 (-6.20)82.20 (-2.70)
N-Caltech101SWFormer84.9384.20 (-0.73)84.20 (-0.73)
SSTWformer87.4885.05 (-2.43)86.03 (-1.46)

Since Pair-Swap fixes temporal-low, the total spike count, and the total energy, and only changes the direction and order structure of temporal-high, the more pronounced performance drop of SSTWformer indicates that its predictions rely more on these specific structures rather than merely being sensitive to changes in high-frequency energy. Meanwhile, after Mask T-High, SSTWformer's absolute accuracy remains higher than that of SWFormer, so the experimental results also do not support simply attributing this to worse overall robustness.

(2).Selective high-frequency restoration experiment

Furthermore, on CIFAR10-DVS, starting from a Low-only input that retains only temporal-low, we restored one genuine temporal high-frequency subband at a time and then reconstructed the input via the inverse transform. Table 2 shows the accuracy changes relative to Low-only:

Table 2 Results of the model under Selective high-frequency restoration experiment.

Restoration conditionSWFormerSSTWformer
Low-only78.378.7
LLhighLL_{\mathrm{high}}81.00 (+2.70)82.20 (+3.50)
LHhighLH_{\mathrm{high}}76.80 (-1.50)79.50 (+0.80)
HLhighHL_{\mathrm{high}}76.90 (-1.40)79.00 (+0.30)
HHhighHH_{\mathrm{high}}77.90 (-0.40)79.30 (+0.60)
Full82.284.9

Restoring LL_high compensates for the Mask performance loss of both SWFormer and SSTWformer, indicating that SWFormer can also exploit this dominant high-frequency channel; our conclusion is therefore not that "SWFormer does not use high-frequency information." The difference lies mainly in the scope of utilization: under this restoration protocol, SWFormer gains only from LL_high restoration, whereas SSTWformer gains on all four temporal high-frequency subbands, consistent with its broader exploitation of input temporal high-frequency components.

We also note that the gain from restoring each subband individually is not equal to the total gain when all are restored, and some conditions even show negative restoration rates, indicating that subband contributions are non-additive and context-dependent. Restoration rates only describe performance changes under specific interventions and do not represent each subband's independent information proportion; a negative individual gain does not mean the subband lacks discriminative information.

In summary, Pair-Swap rules out alternative explanations based on low-frequency content, total spike count, and total energy, while selective restoration shows that genuine high-frequency information can positively improve performance. Together, the two experiments support that SSTWformer exhibits broader functional utilization of various input temporal high-frequency structures, rather than merely being more sensitive to high-frequency perturbations. The relevant results will be added to the revised manuscript.

Q3.

(1)

Thank you for the reviewer's suggestion. we added two types of gating intervention experiments.

The first group directly performed gating intervention on the trained model. The experimental results are shown in Table 3. Zeroing the deep and shallow temporal high-frequency gates reduced accuracy from 84.9% to 84.0% and 84.4%, respectively, i.e., drops of 0.9 and 0.5 percentage points. Furthermore, replacing the deep high-frequency gate with the shallow high-frequency gate or the same-layer low-frequency gate yielded accuracies of 84.5% and 84.3%, respectively. This shows that the deep high-frequency gate can neither be removed arbitrarily nor be fully replaced by the shallow high-frequency gate or the same-layer low-frequency gate, further supporting its depth and frequency specificity.

Table 3 Gating intervention experiments in the trained model.

InterventionAccuracy (%)
original84.9
deep-high-zero84.0(-0.9)
Shallow-high-zero84.4(-0.5)
deep-high-to-shallow84.5(-0.4)
deep-high-to-low84.3(-0.6)

The second group was trained from scratch with gating fixed. The experimental results are shown in Table 4. We fixed the same number of gates at their initial values so that the model could not learn the corresponding frequency regulation during training. Fixing the deep and shallow high-frequency gates reduced accuracy to 82.5% and 83.8%, respectively, i.e., drops of 2.4 and 1.1 percentage points. In contrast, fixing the same number of shallow and deep low-frequency gates yielded accuracies of 84.7% and 84.3%, respectively, with drops of only 0.2 and 0.6 percentage points. Therefore, the performance degradation cannot be simply explained as a general side effect of freezing parameters; instead, it exhibits a clear frequency selectivity, in which the learnable high-frequency gates, especially the deep high-frequency gates, play a more important role.

Table 4 Retrained gating intervention experiments

SettingAccuracy (%)
original84.9
deep_high82.5(-2.4)
Shallow_high83.5(-1.5)
shallow_low84.7(-0.2)
deep_low84.3(-0.6)

Combined with the original observation that "temporal high-frequency energy decreases with depth while the amplitude of the corresponding high-frequency gates increases," these intervention results provide stronger functional evidence for a depth-related compensatory amplification mechanism. The relevant experiments will be added to the revised manuscript.

(2)

We thank the reviewer for the concern regarding efficiency fairness. In the original paper, SSTWformer used its compatible CuPy backend and was significantly faster than SWFormer in both training and inference. To rule out backend differences, we added a controlled experiment under a common PyTorch backend; in this setting, SSTWformer was slower, indicating that the additional spatio-temporal decomposition and eight-subband processing do incur computational overhead.

The two sets of results are not contradictory: the common-backend comparison reflects the model's own computational overhead, whereas the original experiments reflect the actual end-to-end efficiency under currently available implementations. Since SWFormer's current negative-neuron implementation cannot use CuPy, while SSTWformer can use CuPy without structural modification, the latter can achieve acceleration in actual runs. We will report both sets of results in the revised manuscript and limit the conclusion as follows: SSTWformer incurs additional overhead under a common PyTorch backend, but has a significant practical speed advantage under its compatible CuPy backend.

Table 5 Comparison of inference time (IT), training time (TT) per batch, along with maximum GPU memory usage (MM) on N-Caltech101 at T=16.

ModelITTTMM
SWFormer (torch)0.2771.244235
SSTWformer (torch)0.3841.824064
SSTWformer (cupy)0.0730.514064

Q4.

We thank the reviewer for the suggestion. To examine whether SSTWformer's performance gain depends on a specific temporal alignment, we added one-step temporal shift and bucket phase variation experiments on CIFAR10-DVS.

1.One-step temporal shift

Zero-padding simulates "temporal shift + observation-window boundary change"; Circular Shift keeps the total event count unchanged and is used to rule out the effect of boundary event truncation, but it introduces head-tail concatenation. Table 6 shows the results of one-step temporal shift, with the values in parentheses indicating the drop relative to each model's Clean accuracy. Under Zero-padding and Circular Shift, the average drops of SSTWformer are 1.70% and 1.15%, respectively, while those of SWFormer are 1.45% and 1.00%, with the gaps between the two models being only 0.25% and 0.15%. Although SSTWformer is slightly more sensitive to one-step temporal shift than SWFormer, it still maintains higher absolute accuracy under all shift conditions, with advantages of 2.00%–2.70%. Therefore, SSTWformer remains robust within the tested one-frame shift range, and its performance advantage is not limited to the original temporal alignment.

Table 6 Effect of a one-step temporal shift on the accuracy of SSTWformer and SWFormer on CIFAR10-DVS.

InputSSTWformer(%)SWFormer(%)Absolute accuracy advantage(%)
Clean84.9082.20+2.70
Zero padding, -1 time step83.80 (-1.10)81.30 (-0.90)+2.00
Zero padding, +1 time step82.60 (-2.30)80.20 (-2.00)+2.40
Circular shift, -1 time step83.60 (-1.30)81.20 (-1.00)+2.40
Circular shift, +1 time step83.90 (-1.00)81.20 (-1.00)+2.70
  1. Bucket phase variation experiment

To examine whether changes in the positions of bucket boundaries affect model performance, we kept the number of buckets T=16 and the equal-time bucketing scheme unchanged, and only shifted all bucket boundaries forward or backward by half a time bucket. Table 7 shows the experimental results, with each model using its accuracy when the bucket boundaries are not shifted as its own baseline.

Table 7 Effect of temporal bin-boundary shifts on model accuracy with a fixed number of bins

Bin-boundary offsetSSTWformer(%)SWFormer(%)
-0.5 bin width83.20 (+0.80)78.50 (-1.90)
082.40 (0)80.40 (0)
+0.5 bin width82.30 (-0.10)79.50 (-0.90)

SSTWformer's maximum phase fluctuation is 0.90%, lower than SWFormer's 1.90 %, and it maintains higher absolute accuracy under all phases. This indicates that, with the number of buckets and the temporal resolution fixed, SSTWformer is stably robust to bucket phase changes.

In summary, SSTWformer's performance advantage is maintained under both one-frame shift and bucket phase variation, and does not depend on any particular fixed temporal alignment, demonstrating good robustness.

Reviewer_uyp2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2(−1)
Soundness
2
Read review
Paper summary

This paper proposes SSTWformer, which explicitly processes spatial and temporal frequencies in spiking Transformers. A Haar decomposition produces eight subbands, which are modulated by learned channel-wise gates and processed by spatially specialized feature extractors. Experiments cover neuromorphic visual datasets and spiking audio datasets.

Critical strengths
  1. The performance improvement is consistent across the six evaluated datasets, averaging approximately 1.81 percentage points over SWFormer. Removing temporal decomposition from SSTWformer reduces accuracy from 84.9% to 83.6% on CIFAR10-DVS and from 87.48% to 86.35% on N-Caltech101. This ablation supports the usefulness of the temporal component within the proposed architecture.

  2. The authors make an effort to understand the method beyond the final accuracy. The masking and noise-replacement experiments probe sensitivity to temporal high frequencies, and the runtime study includes measured training time, inference time, and memory.

Critical weaknesses
  1. The methodological novelty is limited. Frequency-decomposition strategy, subband weighting, and wavelet-domain convolution have precedents in ANN methods such as WF-VAE and WTConv, while SWFormer already introduces wavelet processing into spiking Transformers. The contribution beyond extending these designs to temporal subbands remains unclear.

  2. The motivation oversimplifies temporal processing in existing models. Flattening time into the batch dimension does not eliminate the temporal dynamics of spiking neurons. Likewise, averaging temporally processed features is not completely equivalent to low-pass filtering the raw input, weakening the claim that existing architectures structurally discard temporal high frequencies.

Questions for authors

see weaknesses

2 comments

Q1.

We thank the reviewer for the valuable comments. Our core contribution is not simply combining existing components, but identifying and systematically studying a new question: can temporal high-frequency information in event sequences be explicitly separated into independently representable and adjustable pathways for selective use by Spiking Transformers? SSTWformer transforms temporal high-frequency information from an implicit component of spike features into a representable, addressable, adjustable, and measurable object. Thus, our work provides both a concrete model and an experimental framework for studying temporal frequency information.

We agree that 1D Haar, frequency gating, and the spatial processor are individually simple. However, architectural novelty depends not only on component complexity, but also on the problem addressed and the resulting insights. Our work addresses a structural gap in mainstream Spiking Transformers: the lack of explicit temporal high-frequency modeling. Therefore, we hope this work is understood not as an incremental "adding temporal Haar to a spatial wavelet model" architecture, but as a systematic exploration and initial framework for temporal-frequency modeling in Spiking Transformers.

We added three sets of experiments to validate this contribution.

1.Independent contribution of the temporal modeling module

As shown in Table 1, SSTWformer outperforms NoTime by 1.30 percentage points on CIFAR10-DVS and 1.13 percentage points on N-Caltech101. These gains confirm the independent contribution of temporal-frequency modeling.

Table 1. Accuracy comparison between NoTime and the full model (%)

DatasetNoTimeSSTWformer
CIFAR10-DVS83.6084.90 (+1.30)
N-Caltech10186.3587.48 (+1.13)

2.The temporal high-frequency pathway contributes to prediction

Mask T-High alone cannot distinguish high-frequency usage from perturbation sensitivity. We therefore designed Pair-Swap, which changes only the sign of the temporal high-frequency component while preserving temporal-low, the total spike count, and total energy. SSTWformer drops by 2.7 percentage points, compared with 1.4 points for SWFormer. Thus, SSTWformer relies more on the specific structure of temporal high frequencies, rather than merely responding to changes in their energy.

Table 2. Accuracy under temporal high-frequency interventions (%)

ModelCleanMask T-HighPair-Swap
SWFormer82.278.3 (-3.9)80.8 (-1.4)
SSTWformer84.978.7 (-6.2)82.2 (-2.7)

3.Learned temporal high-frequency gates affect prediction

These results(Table3) show that the learned high-frequency gates actively affect prediction rather than being redundant. Deep high-frequency gates make a larger contribution.

Table 3 High-frequency gate interventions after training (%)

Gate interventionAccuracy
Original model84.9
Deep high-frequency gate zeroed84.0 (-0.9)
Shallow high-frequency gate zeroed84.4 (-0.5)
Deep high-frequency → shallow high-frequency84.5 (-0.4)
Deep high-frequency → same-layer low-frequency84.3 (-0.6)

As shown in Table4, with the same number of gates frozen, accuracy drops by 1.5–2.4 percentage points for the high-frequency group, but only 0.2–0.6 percentage points for the low-frequency group. The loss therefore cannot be attributed to a generic effect of freezing parameters and shows clear frequency selectivity. Together with the observation that temporal high-frequency energy decreases with depth while the corresponding gates strengthen, these interventions support depth-dependent compensatory modulation.

Table 4. Training from scratch with fixed gates (%)

Fixed Gate TypeAccuracy
Unfixed gates84.9
Deep high-frequency82.5(−2.4)
Shallow high-frequency83.5(−1.5)
Shallow low-frequency84.7(−0.2)
Deep low-frequency84.3(−0.6)
  1. To help the reviewers understand, we added a comparative experiment. Following the SWFormer's approach, we directly introduced temporal Haar into SWFormer via a cascade of "spatial transform—temporal transform—temporal inverse transform—spatial inverse transform." The results are as follows:
ModelParm (M)N-Caltech101CIFAR10-DVS
SWFormer2.0584.93%82.25
+ time Haar2.4785.25% (+0.32%)82.5% (+0.3%)

This experiment shows that simply introducing the Haar operator not only incurs a parameter increase but also yields limited gains—which precisely indicates that our gains do not stem from "simply embedding a temporal Haar operator into SWFormer," but from the complete framework of explicit decoupling + frequency addressing + differentiated processing.

Overall, our work asks not simply how existing wavelet components can improve performance, but whether temporal high frequencies can become independently learned, adjusted, and validated objects in Spiking Transformers. To our knowledge, this is the first systematic exploration in event-based vision Spiking Transformers to explicitly decompose temporal-low/temporal-high subbands, learn them independently, and apply selective intervention—providing an extensible framework for future work on multi-scale temporal wavelets, learnable time-frequency bases, and adaptive frequency gating.

Q2.

We thank the reviewer for the correction. Existing studies have shown that Spiking Transformer architectures such as Spikformer exhibit a low-pass bias[1][2]; our layer-by-layer measurements also show that the proportion of temporal high-frequency energy decreases as the network deepens, which is consistent with the trend that high-frequency components gradually attenuate during propagation. Our original intention was to emphasize that the commonly used paradigm represented by Spikformer mainly performs spatial processing at each time step and adopts temporal averaging as the classification readout, lacking explicit separation and independent regulation of temporal high-frequency components, and thus may fail to fully exploit such information. We agree that the original statement is ambiguous, and will therefore revise it in the revised version as follows: "Existing spiking Transformers typically fold the temporal dimension into the batch dimension and average the per-time-step outputs for classification, lacking explicit modeling of temporal high-frequency components. We find that the temporal high-frequency components that are underutilized by this paradigm contain discriminative information."

[1] Fang Y, Wang Z, Zhang L, et al. Spiking wavelet transformer[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2024: 19-37.

[2]Fang Y, Zhou D, Wang Z, et al. Spiking neural networks need high-frequency information[J]. Advances in Neural Information Processing Systems, 2026, 38: 103404-103425.

Reviewer_UcL8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SSTWformer, a Spiking Transformer architecture that incorporates spatial-temporal Haar wavelet decomposition to explicitly model spatial and temporal frequency components. The proposed framework decomposes spike features into eight frequency subbands and employs learnable frequency modulation and subband-specific feature processing. Experiments on six neuromorphic vision and audio datasets demonstrate consistent accuracy improvements over Spikformer and SWFormer. The authors also provide frequency-domain analyses to investigate the importance of temporal high-frequency information.

Critical strengths
  1. The paper addresses an interesting problem in temporal frequency modeling for Spiking Transformers and proposes a reasonable architectural extension for explicitly processing spatial-temporal frequency components.

  2. The method achieves consistent improvements across six vision and audio datasets, supported by component ablations and additional analyses of temporal high-frequency information.

Critical weaknesses

W1. Limited methodological novelty and insufficient comparisons.

The importance of high-frequency information in SNNs has already been extensively investigated, particularly by SWFormer and MaxFormer, which are also discussed in the paper. The proposed method mainly extends spatial Haar wavelet decomposition to the temporal dimension with learnable frequency gating. However, MaxFormer is notably absent from the experimental comparisons despite reporting competitive results on overlapping benchmarks. The paper also lacks comparisons with simpler temporal modeling alternatives, such as temporal convolution or temporal attention. Furthermore, evaluations are limited to event-based datasets, without experiments on standard static benchmarks such as CIFAR-100 or ImageNet. These limitations make it difficult to assess the novelty, necessity, and generalizability of the proposed architecture.

W2. Insufficient evidence for the claimed mechanisms and confounded efficiency comparisons.

The claim that existing Spiking Transformers structurally discard temporal high-frequency information is not sufficiently justified, since temporal averaging of learned features does not necessarily eliminate information captured through spiking dynamics. Similarly, the proposed compensatory amplification mechanism is primarily supported by frequency-gating statistics rather than direct causal evidence. Although component ablations are provided, they do not adequately isolate the benefits of frequency decomposition from general temporal modeling. Moreover, the reported 3.8× inference speedup over SWFormer is confounded by different neuron implementations and acceleration backends, making it difficult to attribute the efficiency gains to the proposed architecture.

Questions for authors
  1. Why is MaxFormer (NeurIPS 2025), which addresses high-frequency information preservation and reports competitive results on CIFAR10-DVS and DVS128, excluded from the experimental comparisons? Can the authors provide a controlled comparison under matched model sizes, timesteps, and training settings?

  2. Can the authors compare the proposed temporal Haar decomposition with simpler temporal modeling alternatives, such as temporal convolution or temporal attention, under comparable parameter and computational budgets?

  3. Given that related work has already investigated high-frequency information in SNNs, what are the fundamental methodological contributions of SSTWformer beyond extending spatial wavelet decomposition to the temporal dimension? How do the authors distinguish their findings from existing frequency-enhancement approaches?

  4. Why are standard static image classification benchmarks such as CIFAR-100 and ImageNet not included, particularly given the evaluation of related architectures on these datasets? Can the authors clarify the generalizability of SSTWformer beyond event-based inputs?

  5. Can the authors provide stronger causal evidence for the claimed compensatory amplification mechanism and clarify why temporal averaging necessarily discards high-frequency information? Additionally, can they provide efficiency comparisons using matched neuron implementations and acceleration backends?

3 comments

Q1.

Thank you for your suggestion. We reproduced MaxFormer under the same configuration. SSTWformer outperforms MaxFormer on all four datasets, with improvements of 1.50, 0.35, 4.37, and 1.40 percentage points, respectively, verifying the effectiveness of the proposed method in this paper under controlled conditions.

Table 1 Performance(%) comparison between MaxFormer and SSTWFormer

ModelCIFAR10-DVSDVS128N-Caltech101NCARS
Max-former83.498.2683.1194.34
SSTformer84.9(+1.5)98.61(+0.35)87.48(+4.37)95.74(+1.40)

Q2.

We thank the reviewer for the valuable comments. Under the same backbone and training settings, we replaced only the temporal modeling module and compared Temporal Conv with Temporal Attention. The results are as follows:

Table 2 Performance of SSTWFormer under different temporal modeling approaches

ModelFLOPsParams (M)CIFAR10-DVS (%)N-Caltech101 (%)
NotimeO(T)O(T)2.5983.686.35
HaarO(T)O(T)2.5984.9(+1.3)87.48(+1.13)
ConvO(C2T)O(C^2T)2.7383.8(+0.2)86.39(+0.04)
Attention$O(T^2C+TC^2) $3.183.2(-0.4)85.9(-0.45)

Based on the above results, we elaborate from three points:

(1) Best accuracy with the fewest parameters: Haar outperforms Conv and Attention on both datasets by 1.10/1.70 and 1.09/1.58 percentage points, respectively, while having the smallest parameter count.

(2) Lowest complexity: the temporal Haar transform is a fixed, invertible transform using only addition and subtraction, with O(T) complexity and no learnable parameters, whereas Conv is O(C²T) and Attention is O(T²C+TC²). Haar thus achieves higher accuracy at lower additional cost.

In summary, temporal Haar shows advantages in accuracy, parameter count, and computational complexity. The relevant comparative experiments and discussion will be added to the revised manuscript.

Q3.

We further emphasize this work's pioneering value for temporal-frequency research. It proposes and systematically studies a previously underexplored question: can high-frequency information along the temporal dimension of event sequences be explicitly represented and selectively used by a Spiking Transformer as an independent computational resource?

To our knowledge, this is the first work to make temporal-low/temporal-high subbands representable, addressable, tunable, and intervenable in an event-vision Spiking Transformer, establishing a "explicit decomposition—independent learning—selective intervention" framework.

We therefore hope this paper is seen not as an incremental "temporal Haar added to a spatial wavelet model" architecture, but as a systematic exploration and initial framework for temporal-frequency modeling in Spiking Transformers. By making temporal high frequency learnable, analyzable, and verifiable, and experimentally validating its role, it provides a foundation for multi-scale temporal wavelets, learnable time-frequency bases, and temporal-frequency-aware SNNs.

To help the reviewers understand, we added a comparative experiment. Following the SWFormer's approach, we directly introduced temporal Haar into SWFormer via a cascade of "spatial transform—temporal transform—temporal inverse transform—spatial inverse transform." The results are as follows:

ModelParm (M)N-Caltech101CIFAR10-DVS
SWFormer2.0584.93%82.25
+ time Haar2.4785.25% (+0.32%)82.5% (+0.3%)

This experiment shows that simply introducing the Haar operator not only increases the parameter count but also yields only limited gains—which precisely indicates that our benefit stems not from "extending spatial wavelet decomposition to the temporal dimension," but from the complete framework: explicit decoupling + frequency addressing + differentiated processing.

Unlike existing frequency-enhancement methods, the difference lies in research questions and findings, not just structure. SWFormer exploits spatial wavelet frequencies without explicitly separating or regulating temporal high/low frequencies; MaxFormer and related methods enhance high-frequency responses via pooling/convolution to improve overall representation. This paper does not broadly amplify high frequency but investigates what temporal high frequency contains and how the network selectively uses it. Existing methods ask "how to enhance high-frequency responses"; this paper asks "how temporal high frequency is independently represented, selectively regulated, and actually involved in prediction."

Q4.

Thank you for the reviewer's suggestion. We agree that CIFAR-100 and ImageNet are useful for evaluating the model's general visual representation capability, but they do not directly correspond to the core research question of this paper. This paper focuses on high-frequency modeling along the temporal dimension of event sequences, whereas static images themselves have no natural event time axis; if the same image is replicated into T frames, the temporal-high at the input level will be identically zero, making it difficult for such experiments to verify the core claim of this paper regarding the value of temporal high frequency.To examine the generalization of the method beyond event vision, we additionally selected SHD and SSC, two spiking speech datasets with genuine temporal structure. Speech data differ markedly from event vision in both data modality and task type, yet their temporal high frequency is likewise meaningful in practice. SSTWformer achieves an accuracy of 91.17% on SHD and outperforms the corresponding comparison methods on both SHD and SSC, indicating that the method is not limited to specific visual datasets but has the potential to model general temporal spike sequences.

Q5.

(1)

Thank you for the reviewer's suggestion. we added two types of gating intervention experiments.

The first group directly performed gating intervention on the trained model. The experimental results are shown in Table 3. Zeroing the deep and shallow temporal high-frequency gates reduced accuracy from 84.9% to 84.0% and 84.4%, respectively, i.e., drops of 0.9 and 0.5 percentage points. Furthermore, replacing the deep high-frequency gate with the shallow high-frequency gate or the same-layer low-frequency gate yielded accuracies of 84.5% and 84.3%, respectively. This shows that the deep high-frequency gate can neither be removed arbitrarily nor be fully replaced by the shallow high-frequency gate or the same-layer low-frequency gate, further supporting its depth and frequency specificity.

Table 3 Gating intervention experiments in the trained model.

InterventionAccuracy (%)
original84.9
deep-high-zero84.0(-0.9)
Shallow-high-zero84.4(-0.5)
deep-high-to-shallow84.5(-0.4)
deep-high-to-low84.3(-0.6)

The second group was trained from scratch with gating fixed. The experimental results are shown in Table 4. We fixed the same number of gates at their initial values so that the model could not learn the corresponding frequency regulation during training. Fixing the deep and shallow high-frequency gates reduced accuracy to 82.5% and 83.8%, respectively, i.e., drops of 2.4 and 1.1 percentage points. In contrast, fixing the same number of shallow and deep low-frequency gates yielded accuracies of 84.7% and 84.3%, respectively, with drops of only 0.2 and 0.6 percentage points. Therefore, the performance degradation cannot be simply explained as a general side effect of freezing parameters; instead, it exhibits a clear frequency selectivity, in which the learnable high-frequency gates, especially the deep high-frequency gates, play a more important role.

Table 4 Retrained gating intervention experiments

SettingAccuracy (%)
original84.9
deep_high82.5(-2.4)
Shallow_high83.5(-1.5)
shallow_low84.7(-0.2)
deep_low84.3(-0.6)

Combined with the original observation that "temporal high-frequency energy decreases with depth while the amplitude of the corresponding high-frequency gates increases," these intervention results provide stronger functional evidence for a depth-related compensatory amplification mechanism. The relevant experiments will be added to the revised manuscript.

(2)

We thank the reviewer for the correction. Existing studies have shown that Spiking Transformer architectures such as Spikformer exhibit a low-pass bias[1][2]; our layer-by-layer measurements also show that the proportion of temporal high-frequency energy decreases as the network deepens, which is consistent with the trend that high-frequency components gradually attenuate during propagation. Our original intention was to emphasize that the commonly used paradigm represented by Spikformer mainly performs spatial processing at each time step and adopts temporal averaging as the classification readout, lacking explicit separation and independent regulation of temporal high-frequency components, and thus may fail to fully exploit such information. We agree that the original statement is ambiguous, and will therefore revise it in the revised version as follows: "Existing spiking Transformers typically fold the temporal dimension into the batch dimension and average the per-time-step outputs for classification, lacking explicit modeling of temporal high-frequency components. We find that the temporal high-frequency components that are underutilized by this paradigm contain discriminative information."

[1] Fang Y, Wang Z, Zhang L, et al. Spiking wavelet transformer[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2024: 19-37.

[2]Fang Y, Zhou D, Wang Z, et al. Spiking neural networks need high-frequency information[J]. Advances in Neural Information Processing Systems, 2026, 38: 103404-103425.

(3)

We thank the reviewer for the concern regarding efficiency fairness. In the original paper, SSTWformer used its compatible CuPy backend and was significantly faster than SWFormer in both training and inference. To rule out backend differences, we added a controlled experiment under a common PyTorch backend; in this setting, SSTWformer was slower, indicating that the additional spatio-temporal decomposition and eight-subband processing do incur computational overhead.

The two sets of results are not contradictory: the common-backend comparison reflects the model's own computational overhead, whereas the original experiments reflect the actual end-to-end efficiency under currently available implementations. Since SWFormer's current negative-neuron implementation cannot use CuPy, while SSTWformer can use CuPy without structural modification, the latter can achieve acceleration in actual runs. We will report both sets of results in the revised manuscript and limit the conclusion as follows: SSTWformer incurs additional overhead under a common PyTorch backend, but has a significant practical speed advantage under its compatible CuPy backend.

Table 5 Comparison of inference time (IT), training time (TT) per batch, along with maximum GPU memory usage (MM) on N-Caltech101 at T=16.

ModelITTTMM
SWFormer (torch)0.2771.244235
SSTWformer (torch)0.3841.824064
SSTWformer (cupy)0.0730.514064

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.