- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Temporal-Aware Learning (TAL), a training objective designed to better coordinate supervision across different timesteps in SNNs. The authors observe that early temporal states exhibit unstable spike activity, while later states generally produce more accurate predictions and more consistent gradients. Based on these observations, TAL combines three components: quadratic temporal reweighting (QTR), later-step consistency regularization (LSCR), and temporal target smoothing (TTS). Experiments on static and neuromorphic datasets demonstrate improved classification accuracy and faster convergence without introducing additional inference overhead.
Critical strengths
1.The paper provides observations about the temporal evolution of SNNs, particularly the differences in gradient consistency between early and later timesteps. These observations offer a reasonable motivation for the proposed training strategy. 2.The proposed method achieves competitive performance across multiple static and neuromorphic datasets, including ImageNet-1K, and can be applied without modifying the inference architecture. 3.The paper includes extensive ablation studies on different temporal weighting strategies and hyperparameter settings, which help demonstrate the effectiveness of emphasizing later temporal states.
Critical weaknesses
1.According to Table 3, QTR alone already achieves most of the performance improvements. Adding LSCR and TTS only improves accuracy by another 0.30 and 0.18 percentage points on CIFAR10-DVS and N-Caltech101, respectively. This raises concerns about whether all three components are necessary. 2.The experimental evaluation could be strengthened by considering a wider range of network architectures, as only one architecture is evaluated on each dataset. 3.The discussion of related work could be more comprehensive. For example, TS-LIF [1] also investigates temporal gradient optimization, albeit from a different perspective. In addition, ETC [2] explores temporal consistency, which is closely related to the proposed LSCR component, but is not discussed. [1] Resolving the Timestep Scaling Paradox in Spiking Neural Networks with a Timestep-Scalable Neuron Model. ICML, 2026. [2] Improving Stability and Performance of Spiking Neural Networks through Enhancing Temporal Consistency. PR, 2025.
Questions for authors
1.Can the authors provide controlled comparisons with recent temporal optimization methods, particularly ETC, TMC, and DeepTAGE, under identical architectures and training settings? 2.Since QTR accounts for most of the performance gains, could the authors provide additional experiments to demonstrate the necessity of combining LSCR and TTS with QTR? 3.Why should more consistent gradients at later timesteps necessarily be more beneficial for optimization? 4.I would be willing to raise my score if the authors can adequately address these weaknesses and questions.
6 comments
Response to W2: Generalization across network architectures
We thank the reviewer for this constructive suggestion. We agree that evaluating multiple architectures, particularly on the same dataset, is important for assessing the generalizability of TAL. Our original experiments primarily adopted standard dataset-specific backbones, which facilitate comparisons with existing methods but provide limited evidence of cross-architecture transferability. To address this concern, we conduct additional evaluations using different convolutional SNN architectures and an attention-based Spiking Transformer.
(1) Cross-architecture validation on visual benchmarks
We additionally evaluate TAL on ResNet20 for CIFAR10DVS, ResNet-18 for CIFAR10, and ResNet-19 for CIFAR100. Within each configuration, the baseline and TAL are evaluated under matched experimental settings, with only the supervision objective changed. TAL consistently improves accuracy across all three additional configurations.
表格
| Dataset | Architecture | Baseline | TAL |
|---|---|---|---|
| CIFAR10DVS | ResNet20 | 83.3% | 84.5% |
| CIFAR10 | ResNet-18 | 95.72% | 96.53% |
| CIFAR100 | ResNet-19 | 81.48% | 82.71% |
(2) Extension to an attention-based Spiking Transformer
To further examine whether TAL generalizes beyond convolutional SNNs, we evaluate it on the Spiking Heidelberg Digits (SHD) dataset using TIM (IJCAI 2024), a Spikformer-based architecture equipped with a temporal interaction module. Under matched experimental settings, TAL achieves 86.9% accuracy, outperforming both the original TIM training objective (86.1%) and TET-style uniform temporal supervision (85.6%) by 0.8% and 1.3%, respectively. This result extends our evaluation to an attention-based Spiking Transformer and a non-visual auditory classification task. Importantly, TIM enhances temporal feature interactions within the network, whereas TAL coordinates timestep-specific supervision at the training-objective level. Their combination achieves improved performance without modifying the original attention mechanism, suggesting that architectural temporal modeling and temporally differentiated supervision can provide complementary benefits.
(3) Architectural applicability and generalization
TAL operates on timestep-specific predictions and losses without requiring backbone-specific architectural modifications or introducing additional inference overhead. The additional experiments, together with our original results, demonstrate consistent performance improvements across the evaluated VGG-based, ResNet-based, and Spiking Transformer architectures. In particular, the CIFAR10DVS experiments provide validation across different backbones on the same dataset, while the SHD experiment further extends the architectural and task coverage. Although these evaluations do not exhaustively cover all SNN architectures, they provide stronger empirical evidence that TAL's benefits are not confined to a particular network design.
Response to W3: Relationship to TS-LIF and ETC.
We thank the reviewer for highlighting these closely related studies. We agree that both TS-LIF and ETC are important to the discussion of temporal optimization in SNNs. In particular, ETC has already recognized that inconsistent temporal outputs can lead to conflicting optimization directions. We will explicitly acknowledge these contributions and clarify how TAL differs in its analysis and optimization formulation.
(1) Comparison with TS-LIF (ICML 2026)
TS-LIF investigates the timestep scaling paradox, attributing performance degradation at longer simulation horizons to vanishing temporal gradients and weakened cross-timestep dependencies. It addresses this problem by modifying neuronal dynamics through long-term memory reconsolidation and periodic temporal forgetting. The key distinction lies in the gradients being investigated. TS-LIF primarily studies temporal gradient propagation through neuronal states during BPTT, such as the derivatives of later membrane potentials with respect to earlier states. In contrast, TAL analyzes the directional relationships among timestep-specific supervision gradients with respect to shared network parameters. Accordingly, TS-LIF improves how temporal information and gradients propagate through neuronal dynamics, whereas TAL coordinates how supervision gradients from different timesteps contribute to parameter optimization. TAL does not modify neuronal state transitions or introduce additional temporal propagation paths. These approaches therefore address different aspects of temporal learning, although their potential complementarity remains to be evaluated empirically.
(2) Comparison with ETC (Pattern Recognition 2025)
We acknowledge that ETC is particularly relevant to our work. ETC identifies temporal output inconsistency as a source of optimization difficulties and introduces a consistency-based training objective. Our contribution is not the general observation that temporal gradients may conflict, nor the use of temporal consistency regularization itself. Instead, TAL further characterizes the stage-dependent structure of pairwise supervision-gradient relationships. As shown in Fig. 4, early–late gradients exhibit weak alignment, whereas gradients among later timesteps show substantially stronger consistency. These observations motivate differentiated treatment of temporal states rather than imposing uniform supervision or consistency constraints throughout the sequence. The two methods also differ in their overall training formulations. ETC combines sequence-level cross-entropy on temporally averaged outputs with an all-pair consistency objective based on temperature-scaled KL divergence between timestep-wise probability distributions, using stop-gradient targets. In contrast, TAL directly coordinates timestep-specific classification losses through quadratic temporal reweighting (QTR) and temporal target smoothing (TTS), while employing later-step consistency regularization (LSCR) as an auxiliary constraint. Thus, TAL explicitly differentiates both the relative importance and target confidence of temporal states. Regarding consistency regularization, ETC encourages agreement across different timestep pairs, whereas LSCR applies an penalty only to adjacent late-step logits, leaving early states unconstrained by this regularizer. This choice is motivated by the observed temporal asymmetry: early states undergo more substantial evolution, while later states exhibit stronger local consistency. Our boundary-sensitivity analysis in Table 8 further supports this design choice. On CIFAR10DVS, applying consistency regularization from timestep 2 yields 80.5% accuracy, whereas restricting it to timestep 5 or later achieves 86.8%. This suggests that the temporal placement of consistency constraints is important and that imposing them too early can interfere with useful temporal evolution.
(3) Empirical comparison and contribution
The additional controlled comparisons reported in our response to Q1 provide further empirical evidence. Under the evaluated matched configurations, the complete TAL objective achieves 86.8% versus 85.6% for ETC on CIFAR10DVS. These results support the effectiveness of the overall TAL objective, although they do not independently establish the superiority of LSCR over ETC's consistency regularizer. Overall, we position TAL as a temporally differentiated supervision framework motivated by the observed stage-dependent structure of supervision gradients, rather than as a new temporal consistency mechanism or a solution to temporal gradient vanishing. Its main contribution lies in coordinating temporal loss weighting, target confidence, and selective late-stage regularization according to the evolving characteristics of SNN states.
Response to W1 & Q2: Necessity and complementary roles of TAL components.
We thank the reviewer for this important comment. We agree that QTR accounts for the majority of the accuracy improvement, and acknowledge that the additional gains from LSCR and TTS are relatively modest. However, the three components address different aspects of temporal supervision rather than serving as interchangeable mechanisms. Specifically, QTR adjusts the relative importance of temporal losses, TTS relaxes supervision on less mature early states, and LSCR constrains local prediction variations among later states. We therefore further examine whether LSCR and TTS provide meaningful refinements beyond QTR.
(1) Additional controlled component ablations
We conduct additional ablations on CIFAR10DVS under matched training settings, including the previously missing QTR+LSCR and QTR+TTS configurations. The results show that both components provide incremental improvements beyond QTR. Adding LSCR to QTR improves accuracy by 0.2%, while adding TTS improves it by 0.1%. Importantly, the component-removal comparisons further show that introducing LSCR into QTR+TTS improves accuracy from 86.6% to 86.8%, whereas introducing TTS into QTR+LSCR improves accuracy from 86.7% to 86.8%.
(2) Distinct effects on temporal behavior
Beyond accuracy, we examine whether the additional components affect the temporal characteristics they are designed to address. For LSCR, adding it to QTR+TTS reduces late-step logit variation from 0.0023 to 0.0005 while improving accuracy from 86.6% to 86.8%. Although suppressing logit variation is consistent with the explicit regularization objective, the accompanying accuracy improvement indicates that this constraint does not merely reduce output variation at the expense of classification performance. For TTS, adding it to QTR+LSCR increases early–late gradient cosine similarity from 0.065 to 0.079, alongside an accuracy improvement from 86.7% to 86.8%. A similar trend is observed when adding TTS to QTR alone, where gradient similarity increases from 0.052 to 0.076. These results are consistent with TTS contributing to the coordination of early- and late-step supervision. Together, these observations indicate that LSCR and TTS provide distinct refinements, with the full TAL objective achieving the highest accuracy and gradient similarity and the lowest late-step logit variation among the evaluated configurations.
(3) Additional evidence supporting temporally differentiated designs
The temporal-boundary analyses in the original manuscript provide further evidence for the design choices underlying LSCR and TTS. For LSCR, Table 8 shows that starting consistency regularization too early substantially degrades accuracy. On CIFAR10DVS, accuracy increases from 80.5% at tlater=2 to 86.8% at tlater=5, remaining at 86.8% for later evaluated boundaries. This supports restricting consistency regularization to relatively mature temporal states rather than imposing it indiscriminately across the entire sequence. For TTS, Table 7 shows that extending target smoothing to later timesteps is not necessarily beneficial. On CIFAR10DVS, accuracy decreases from 86.8% at tearly=3 to 84.9% at tearly=4. This supports the motivation for relaxing supervision primarily at early timesteps while preserving stronger supervision at later stages. These sensitivity analyses complement the component ablations by showing that the temporal placement of the two mechanisms matters, rather than treating consistency regularization and target smoothing as uniformly applied auxiliary objectives.
(4) Clarification of the contributions
We agree that the results do not establish that all three components are strictly indispensable. Our intended claim is more specific: QTR provides the dominant performance gain, while LSCR and TTS offer complementary, temporally targeted refinements that can be combined to improve both accuracy and the corresponding temporal behavior metrics.