- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SSTWformer, which combines spatial and temporal Haar transforms into eight subbands and learns frequency-specific gating and processing for event-based Spiking Transformers. It reports comparisons on four neuromorphic vision datasets and two spiking audio datasets, plus NoTime/component ablations, temporal-window tests, subband perturbations, and gate/energy profiles. These findings support the usefulness of adding explicit temporal-frequency processing, but not all proposed mechanistic and efficiency interpretations are equally established.
Critical strengths
- A focused temporal-frequency extension is evaluated on several datasets and with a relevant spatial-wavelet comparison. The best reported T=16 accuracies exceed SWFormer on CIFAR10-DVS (84.90% vs 82.20%) and N-Caltech101 (87.48% vs 84.93%), and the paper includes existing temporal-module baselines as well as the six-dataset summary (Tables 1–2).
- The NoTime and input-subband interventions provide evidence beyond headline accuracy. Removing temporal decomposition reduces CIFAR10-DVS accuracy from 84.90% to 83.60%, and masking temporal-high subbands reduces it to 78.70%. Analogous N-Caltech101 results are reported. These checks support a functional role for the added temporal-frequency path in the trained system (Tables 3 and 6).
Critical weaknesses
- Method-specific attribution and novelty relative to existing wavelet/temporal models remain incomplete. Spatial and temporal Haar transforms are standard separable operators, while SWFormer already introduces wavelet-based spiking frequency processing. More importantly, the NoTime variant already improves over SWFormer by 1.40 points on CIFAR10-DVS (83.60 vs 82.20) and 1.42 points on N-Caltech101 (86.35 vs 84.93). The incremental temporal-decomposition contributions are 1.30 and 1.13 points, respectively—substantial, but not the whole headline improvement. Model capacity also differs (2.59M vs 2.10M parameters for the listed models). These comparisons do not isolate temporal Haar decomposition from changes in the processor/gating architecture or compare with a parameter-matched temporal convolution/mixer.
- Strong mechanistic claims exceed the interventions. Average pooling of time-indexed network outputs does not by itself prove that all previously encoded high-frequency information is irreversibly lost, because temporal dynamics and nonlinear feature transformations occur upstream. Larger degradation under masked/noisy temporal-high inputs measures learned sensitivity, not uniquely structural information waste; energy-matched noise may also produce off-distribution corruption. The claimed 'effective utilization' in Table 7 is calculated from subband energy and gate magnitudes, not a direct downstream information or attribution measure. A larger deep-layer gate, by itself, does not causally establish compensation for information lost through spiking dynamics.
- The temporal Haar transform operates on adjacent time steps and is not shift-invariant. Consequently, small temporal shifts or changes in event binning may significantly alter the extracted high-frequency components. Although the paper evaluates different temporal window lengths, it does not establish whether the proposed method is robust to such temporal variations.
Questions for authors
- Under the same backbone, time window, and comparable parameter/operation budgets, how does 1D temporal Haar processing compare with a simple temporal convolution or temporal mixer? Please isolate the incremental gain due to the temporal transform from the changed frequency processor.
- How do the authors distinguish lower sensitivity of SWFormer to high-frequency masking from genuinely weaker use of discriminative high-frequency information? Would a distribution-controlled perturbation or selective high-frequency reconstruction clarify the interpretation?
- Can the compensation claim be tested by freezing or selectively suppressing deep-layer temporal-high gates after training and measuring accuracy and feature dependence, rather than relying on the energy-weighted proxy alone? Also distinguish kernel/software speed differences in Table 8 from intrinsic operation savings.
- How robust is SSTWformer to small temporal shifts or changes in event-binning alignment? Evaluating performance under one-step temporal shifts or mild temporal jitter, compared with SWFormer, would help determine whether the improvements generalize beyond a particular temporal alignment.
5 comments
(2).Selective high-frequency restoration experiment
Furthermore, on CIFAR10-DVS, starting from a Low-only input that retains only temporal-low, we restored one genuine temporal high-frequency subband at a time and then reconstructed the input via the inverse transform. Table 2 shows the accuracy changes relative to Low-only:
Table 2 Results of the model under Selective high-frequency restoration experiment.
| Restoration condition | SWFormer | SSTWformer |
|---|---|---|
| Low-only | 78.3 | 78.7 |
| 81.00 (+2.70) | 82.20 (+3.50) | |
| 76.80 (-1.50) | 79.50 (+0.80) | |
| 76.90 (-1.40) | 79.00 (+0.30) | |
| 77.90 (-0.40) | 79.30 (+0.60) | |
| Full | 82.2 | 84.9 |
Restoring LL_high compensates for the Mask performance loss of both SWFormer and SSTWformer, indicating that SWFormer can also exploit this dominant high-frequency channel; our conclusion is therefore not that "SWFormer does not use high-frequency information." The difference lies mainly in the scope of utilization: under this restoration protocol, SWFormer gains only from LL_high restoration, whereas SSTWformer gains on all four temporal high-frequency subbands, consistent with its broader exploitation of input temporal high-frequency components.
We also note that the gain from restoring each subband individually is not equal to the total gain when all are restored, and some conditions even show negative restoration rates, indicating that subband contributions are non-additive and context-dependent. Restoration rates only describe performance changes under specific interventions and do not represent each subband's independent information proportion; a negative individual gain does not mean the subband lacks discriminative information.
In summary, Pair-Swap rules out alternative explanations based on low-frequency content, total spike count, and total energy, while selective restoration shows that genuine high-frequency information can positively improve performance. Together, the two experiments support that SSTWformer exhibits broader functional utilization of various input temporal high-frequency structures, rather than merely being more sensitive to high-frequency perturbations. The relevant results will be added to the revised manuscript.
Q3.
(1)
Thank you for the reviewer's suggestion. we added two types of gating intervention experiments.
The first group directly performed gating intervention on the trained model. The experimental results are shown in Table 3. Zeroing the deep and shallow temporal high-frequency gates reduced accuracy from 84.9% to 84.0% and 84.4%, respectively, i.e., drops of 0.9 and 0.5 percentage points. Furthermore, replacing the deep high-frequency gate with the shallow high-frequency gate or the same-layer low-frequency gate yielded accuracies of 84.5% and 84.3%, respectively. This shows that the deep high-frequency gate can neither be removed arbitrarily nor be fully replaced by the shallow high-frequency gate or the same-layer low-frequency gate, further supporting its depth and frequency specificity.
Table 3 Gating intervention experiments in the trained model.
| Intervention | Accuracy (%) |
|---|---|
| original | 84.9 |
| deep-high-zero | 84.0(-0.9) |
| Shallow-high-zero | 84.4(-0.5) |
| deep-high-to-shallow | 84.5(-0.4) |
| deep-high-to-low | 84.3(-0.6) |
The second group was trained from scratch with gating fixed. The experimental results are shown in Table 4. We fixed the same number of gates at their initial values so that the model could not learn the corresponding frequency regulation during training. Fixing the deep and shallow high-frequency gates reduced accuracy to 82.5% and 83.8%, respectively, i.e., drops of 2.4 and 1.1 percentage points. In contrast, fixing the same number of shallow and deep low-frequency gates yielded accuracies of 84.7% and 84.3%, respectively, with drops of only 0.2 and 0.6 percentage points. Therefore, the performance degradation cannot be simply explained as a general side effect of freezing parameters; instead, it exhibits a clear frequency selectivity, in which the learnable high-frequency gates, especially the deep high-frequency gates, play a more important role.
Table 4 Retrained gating intervention experiments
| Setting | Accuracy (%) |
|---|---|
| original | 84.9 |
| deep_high | 82.5(-2.4) |
| Shallow_high | 83.5(-1.5) |
| shallow_low | 84.7(-0.2) |
| deep_low | 84.3(-0.6) |
Combined with the original observation that "temporal high-frequency energy decreases with depth while the amplitude of the corresponding high-frequency gates increases," these intervention results provide stronger functional evidence for a depth-related compensatory amplification mechanism. The relevant experiments will be added to the revised manuscript.
(2)
We thank the reviewer for the concern regarding efficiency fairness. In the original paper, SSTWformer used its compatible CuPy backend and was significantly faster than SWFormer in both training and inference. To rule out backend differences, we added a controlled experiment under a common PyTorch backend; in this setting, SSTWformer was slower, indicating that the additional spatio-temporal decomposition and eight-subband processing do incur computational overhead.
The two sets of results are not contradictory: the common-backend comparison reflects the model's own computational overhead, whereas the original experiments reflect the actual end-to-end efficiency under currently available implementations. Since SWFormer's current negative-neuron implementation cannot use CuPy, while SSTWformer can use CuPy without structural modification, the latter can achieve acceleration in actual runs. We will report both sets of results in the revised manuscript and limit the conclusion as follows: SSTWformer incurs additional overhead under a common PyTorch backend, but has a significant practical speed advantage under its compatible CuPy backend.
Table 5 Comparison of inference time (IT), training time (TT) per batch, along with maximum GPU memory usage (MM) on N-Caltech101 at T=16.
| Model | IT | TT | MM |
|---|---|---|---|
| SWFormer (torch) | 0.277 | 1.24 | 4235 |
| SSTWformer (torch) | 0.384 | 1.82 | 4064 |
| SSTWformer (cupy) | 0.073 | 0.51 | 4064 |
Q4.
We thank the reviewer for the suggestion. To examine whether SSTWformer's performance gain depends on a specific temporal alignment, we added one-step temporal shift and bucket phase variation experiments on CIFAR10-DVS.
1.One-step temporal shift
Zero-padding simulates "temporal shift + observation-window boundary change"; Circular Shift keeps the total event count unchanged and is used to rule out the effect of boundary event truncation, but it introduces head-tail concatenation. Table 6 shows the results of one-step temporal shift, with the values in parentheses indicating the drop relative to each model's Clean accuracy. Under Zero-padding and Circular Shift, the average drops of SSTWformer are 1.70% and 1.15%, respectively, while those of SWFormer are 1.45% and 1.00%, with the gaps between the two models being only 0.25% and 0.15%. Although SSTWformer is slightly more sensitive to one-step temporal shift than SWFormer, it still maintains higher absolute accuracy under all shift conditions, with advantages of 2.00%–2.70%. Therefore, SSTWformer remains robust within the tested one-frame shift range, and its performance advantage is not limited to the original temporal alignment.
Table 6 Effect of a one-step temporal shift on the accuracy of SSTWformer and SWFormer on CIFAR10-DVS.
| Input | SSTWformer(%) | SWFormer(%) | Absolute accuracy advantage(%) |
|---|---|---|---|
| Clean | 84.90 | 82.20 | +2.70 |
| Zero padding, -1 time step | 83.80 (-1.10) | 81.30 (-0.90) | +2.00 |
| Zero padding, +1 time step | 82.60 (-2.30) | 80.20 (-2.00) | +2.40 |
| Circular shift, -1 time step | 83.60 (-1.30) | 81.20 (-1.00) | +2.40 |
| Circular shift, +1 time step | 83.90 (-1.00) | 81.20 (-1.00) | +2.70 |
- Bucket phase variation experiment
To examine whether changes in the positions of bucket boundaries affect model performance, we kept the number of buckets T=16 and the equal-time bucketing scheme unchanged, and only shifted all bucket boundaries forward or backward by half a time bucket. Table 7 shows the experimental results, with each model using its accuracy when the bucket boundaries are not shifted as its own baseline.
Table 7 Effect of temporal bin-boundary shifts on model accuracy with a fixed number of bins
| Bin-boundary offset | SSTWformer(%) | SWFormer(%) |
|---|---|---|
| -0.5 bin width | 83.20 (+0.80) | 78.50 (-1.90) |
| 0 | 82.40 (0) | 80.40 (0) |
| +0.5 bin width | 82.30 (-0.10) | 79.50 (-0.90) |
SSTWformer's maximum phase fluctuation is 0.90%, lower than SWFormer's 1.90 %, and it maintains higher absolute accuracy under all phases. This indicates that, with the number of buckets and the temporal resolution fixed, SSTWformer is stably robust to bucket phase changes.
In summary, SSTWformer's performance advantage is maintained under both one-frame shift and bucket phase variation, and does not depend on any particular fixed temporal alignment, demonstrating good robustness.
Q1.
We thank the reviewer for the valuable comments. Under the same backbone and training settings, we replaced only the temporal modeling module and compared Temporal Conv with Temporal Attention. The results are as follows:
Table 2 Performance of SSTWFormer under different temporal modeling approaches
Based on the above results, we elaborate from three points:
(1) Best accuracy with the fewest parameters: Haar outperforms Conv and Attention on both datasets by 1.10/1.70 and 1.09/1.58 percentage points, respectively, while having the smallest parameter count.
(2) Lowest complexity: the temporal Haar transform is a fixed, invertible transform using only addition and subtraction, with O(T) complexity and no learnable parameters, whereas Conv is O(C²T) and Attention is O(T²C+TC²). Haar thus achieves higher accuracy at lower additional cost.
(3) Learnable temporal operators bring no benefit: compared with NoTime, Conv improves by only 0.20%/0.04%, while Attention decreases by 0.40%/0.45%. This indicates that simply stacking learnable temporal operators does not stably improve event data modeling, whereas Haar provides an effective temporal-frequency prior through fixed low/high-frequency decomposition.
In summary, temporal Haar shows advantages in accuracy, parameter count, and computational complexity. The relevant comparative experiments and discussion will be added to the revised manuscript.
Q2.
We thank the reviewer for pointing out this issue. We have added distribution-controlled Pair-Swap perturbations and selective high-frequency restoration experiments to examine, respectively, the model's dependence on temporal high-frequency structure and whether restoring genuine high-frequency information can bring positive gains.
First, we clarify the experimental setup: subbands such as LL_high are all defined by the spatio-temporal Haar decomposition at the input level, rather than being explicit modules inside the model. For both models, we adopt exactly the same pipeline: first decompose the original event input, then intervene on the specified subband, then reconstruct an ordinary event tensor via the inverse Haar transform, and finally feed the same reconstructed input to SWFormer and SSTWformer, respectively. Therefore, this experiment compares the functional dependence of the two models on the same input-level temporal high-frequency information.
(1).Distribution-controlled Pair-Swap perturbation Pair-Swap keeps temporal-low unchanged by swapping each pair of adjacent frames, turning temporal-high into its negation while keeping the total spike count and total energy unchanged. Mask T-High instead zeroes the four temporal-high subbands and reconstructs the remaining subbands. The results are shown in Table1.
Table 1 Results of the model under distribution-controlled Pair-Swap perturbation
Since Pair-Swap fixes temporal-low, the total spike count, and the total energy, and only changes the direction and order structure of temporal-high, the more pronounced performance drop of SSTWformer indicates that its predictions rely more on these specific structures rather than merely being sensitive to changes in high-frequency energy. Meanwhile, after Mask T-High, SSTWformer's absolute accuracy remains higher than that of SWFormer, so the experimental results also do not support simply attributing this to worse overall robustness.