- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Spatio-Temporal Margin Regularization (STMR) to mitigate temporal error accumulation in spiking neural networks. The authors argue that incorrectly fired spikes at early timesteps may propagate through neuronal dynamics and degrade prediction performance. To address this issue, STMR introduces two components: Temporal Margin Regularization (TMR), which encourages target-related neurons to fire earlier, and Spatial Margin Regularization (SMR), which reduces confidence margins favoring incorrect classes across timesteps. Experiments on various datasets and architectures demonstrate improvements in classification accuracy without additional inference overhead.
Critical strengths
1.The paper addresses temporal error accumulation in SNNs from both spike dynamics and output representations, providing a reasonable motivation for the proposed regularization framework. 2.The method is evaluated on diverse tasks and datasets, including static images, neuromorphic vision, speech recognition, and 3D point cloud classification. The experiments also cover different architectures, such as VGG, ResNet, and Transformer.
Critical weaknesses
1.The authors use the absolute values of classifier weights to estimate class-specific spike latency. However, this formulation does not distinguish between positive and negative contributions to the target class. 2.The theoretical analysis is not sufficiently rigorous. In particular, the inequality in Eq. (13) does not appear to hold in general, since the softmax of temporally averaged logits is not equivalent to averaging the per-timestep softmax probabilities. 3.Although the experimental results are competitive, some improvements over existing methods are relatively small.
Questions for authors
1.Why are the absolute values of classifier weights used to estimate class-specific spike latency? Neurons with negative weights may decrease the target-class logit. 2.Could the authors provide an ablation study comparing the adaptive timestep weighting in SMR with uniform weighting to demonstrate its necessity?
5 comments
Thank you for the response and the additional experiments. However, I believe my main concerns have not been fully addressed.
-
My main concern is that using absolute weights may encourage not only neurons that positively contribute to the target-class logit to fire earlier, but also those that negatively contribute to it. I believe this issue has not been sufficiently explained. Using absolute weights still seems to be an empirical design choice.
-
My point was that some improvements over existing methods are relatively small, not that the improvements are small on all datasets.
Therefore, I will keep my original rating unchanged.
Thank you very much for your further response.
We greatly appreciate your response, which is crucial to clarifying the core attributes of the proposed method.
-
The effect of the absolute value of classifier weights. TMR measures the strength of connections between neurons and categories based on the absolute values of the classifier weights. Optimizing the TMR loss encourages neurons strongly correlated with a specific category to fire preferentially, which indeed includes both positive and negative correlations. However, the proposed method does not rely solely on TMR. The primary loss is the cross-entropy loss, which promotes correct classification. SMR further encourages the target category to have a more significant predictive margin relative to competing error categories. When TMR triggers spikes from negatively correlated neurons, it decreases the SNN’s logit output for the target category. The CE and SMR losses then work to suppress and offset the impact of these category-negative correlated spikes. Thus, the proposed STMR promotes preferential firing of positively correlated neurons, achieving an overall gain. In the future, we will delve deeper into metrics that can more comprehensively reflect the contributions of spiking neurons to categories, mitigating the impact of negatively correlated neurons even further.
-
Improvements over existing methods. The proposed method is independent of specific architectures, neuron types, and training methods, and can therefore be applied to a variety of existing SNNs. The extent to which the proposed method improves the performance of various existing methods depends on the characteristics of those methods. Nevertheless, we would like to emphasize that even if the improvement in performance on a specific existing method is not particularly significant, the experiments in Tables 1 and 2 have fully validated the effectiveness and generality of the proposed method. Accuracy improvements range from 0.70% to 2.43% across the four methods: PLIF, CLIF, TAB, and SLT. Furthermore, experiments also demonstrate that the proposed method is fully compatible with mainstream spiking VGG, ResNet, and Transformer architectures. Taken together, these results demonstrate that STMR is a competitive, plug-and-play regularization method for enhancing other methods because it preserves the network architecture and inference workflow of existing methods. Given the increasing complexity of current SNN architectures and neuron types, we believe such a seamless performance enhancer is highly necessary.
We hope these responses address your concerns fully by clarifying the mechanism by which the proposed method works and positioning our work within the SNN community. We look forward to discussing any concerns you may still have.
We sincerely appreciate your insightful comments
We sincerely appreciate your insightful comments. Below is our point-by-point response:
Signs of classifier weights. Using only positive or negative weights results in one type of weight being ignored, whereas using the absolute value of the weights takes into account the magnitude of contributions from both types. To verify the advantages of using absolute weight values in the proposed method, we compared it with methods that use only positive or negative weight values. As shown in the table below, using absolute values of weights consistently yields better performance.
Revision of Eq. (13). We have revised Eq. (13) using logit margins, thereby resolving the mismatch between the softmax function corresponding to the time-averaged logits and the mean of the softmax probabilities at each timestep. We emphasize that the revised analysis preserves our conclusion that SMR can improve the final probability margin. Please refer to Section B.1 of the revised submission.
Advantages of the proposed method. We would like to emphasize that, unlike other methods that modify the architecture or neurons, the proposed method regularizes the spatio-temporal impulse margins of SNNs and requires no changes to existing models. It offers superior generalizability, which can further enhance the performance of these methods. Furthermore, our method can boost the accuracy of SNNs by up to 4.52% with the simple plug-and-play setup. We hope this illustrates the full significance and strength of this work.
Adaptive timestep weighting versus uniform weighting. We compared our proposed adaptive timestep weighting method with the vanilla uniform weighting method on CIFAR10-DVS and DVS-Gesture. The results in the table below once again demonstrate the advantages of our proposed adaptive weighting method.
We have made revisions to the submitted manuscript and highlighted the changes in blue. We hope these responses address your concerns, and we look forward to discussing any further questions you may have.