- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the effects of different learning rules on the training dynamics, loss landscape, and generalization performance of spiking neural networks (SNNs). The authors compare three learning approaches, GP, LP, and HP, and analyze their optimization behaviors using time-averaged mean squared displacement (TAMSD), diffusion exponents, loss landscape visualization, and Hessian curvature analysis. The experimental results show that all three methods exhibit a transition from superdiffusive behavior in the early stages of training to subdiffusive behavior in later stages. In the main experiments, HP exhibits a late-stage diffusion exponent between those of GP and LP, while also achieving lower Hessian curvature and better noise robustness. The authors suggest that the Hebbian updates in HP may act as an implicit regularizer, encouraging convergence toward flatter regions of the loss landscape and thereby improving generalization performance.
Critical strengths
- The research question addressed in this paper is meaningful. Rather than focusing solely on classification accuracy, the authors further investigate weight dynamics and loss landscapes under different learning rules, providing a different perspective on understanding the optimization process of SNNs.
- The paper employs multiple analytical techniques, including TAMSD, diffusion exponents, PCA visualization, and Hessian curvature analysis. These methods examine the training process from different perspectives, making the experimental analysis relatively comprehensive.
- Beyond comparing classification performance, the authors also evaluate noise robustness and provide additional experiments in the appendix involving different network architectures, EventProp, and weight norm constraints. These supplementary analyses make the overall empirical investigation more complete.
Critical weaknesses
- The current experiments primarily focus on Fashion-MNIST and fully connected SNNs. Including additional datasets or network architectures would help strengthen the generality of the findings and make the conclusions more convincing.
- Although the paper demonstrates connections among diffusion behavior, loss landscape properties, and generalization performance, the relationships among these phenomena could be investigated more thoroughly to provide stronger support for the authors' proposed interpretation.
- Some experimental results are based on single runs. Conducting experiments with multiple random seeds and reporting the variability of the results would help establish the stability and reliability of the conclusions.
- Some results in the appendix differ from those reported in the main experiments. For example, the diffusion characteristics of HP exhibit certain variations across different network architectures. A more detailed discussion of these differences and their potential causes would provide a more complete understanding of the findings and help clarify the extent to which the main conclusions generalize across experimental settings.
Questions for authors
see weakness
Dear Reviewer,
This is a kind reminder that the discussion period will end in less than 24 hours. We would appreciate it if you could take a moment to review their responses and provide any further comments before the discussion period closes.
Thank you for your time and effort!