- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates the superiority of the Quadratic Integrate-and-Fire (QIF) model over the vanilla Leaky Integrate-and-Fire (LIF) model from the perspective of loss and gradient landscape continuity. The proposed approach is empirically validated on the SHD classification task under the Time-To-First-Spike (TTFS) coding scheme.
Critical strengths
- The authors effectively analyze the advantages of the QIF model through the lens of gradient continuity, and provide comprehensive, well-designed visualization-based analytical experimental results to support their claims.
Critical weaknesses
-
The justification for the superiority of the QIF model is currently constrained to a highly specific setup: the TTFS coding scheme, relatively simplistic downstream tasks, and basic SNN architectures. To robustly establish its efficacy, further empirical validation is warranted in more complex scenarios. Specifically, the authors should evaluate the QIF model using alternative spike coding schemes, on large-scale datasets (e.g., ImageNet-1k), and with advanced architectures such as Spiking CNNs and Spiking ViTs.
-
Additionally, the paper lacks comparisons with other recently proposed advanced spiking neuron models. Benchmarking the QIF model against these state-of-the-art alternatives is crucial to explicitly demonstrate its relative advantages.
Questions for authors
-
Under the Spatio-Temporal Back-propagation (STBP) framework, which relies on surrogate gradients and discrete time-steps, does the QIF model still maintain its superiority over the vanilla LIF model?
-
Regarding the intra-layer computations of spiking neurons, does the QIF model introduce higher computational overhead compared to the vanilla LIF model during the SNN inference phase?
4 comments
Dear Reviewer,
This is a kind reminder that the discussion period will end in less than 24 hours. We would appreciate it if you could take a moment to review authors' responses and provide any further comments before the discussion period closes.
Thank you for your time and effort!
I acknowledge the authors' response to the reviews. After thoroughly evaluating the rebuttal and the perspectives shared by other reviewers, I will maintain my original recommendation.
We thank the reviewer for their response and kindly ask them to consider the changes made in our revision document and specifically the extended Discussion section.
We thank the reviewer for their feedback. In the following, we address the critical weaknesses stated by the reviewer.
Our focus on time-to-first-spike coding is intentional since it relies on the precise timing of spikes. Spike count coding methods rely much more on spike rates than on precise timings. Thus, we expect the theoretical findings of [1] to prove particularly useful when a spike time-based coding scheme is considered. Apart from that, time-to-first-spike coding allows inference as soon as the first output spike appears and does not require counting spikes over a full trial, leading to a reduced latency. Another advantage of spike time-based coding schemes is that individual spikes carry more information and thus, to store the same information, less spikes are required, at least in principle.
In addition, we deem a thorough hyperparameter search necessary for a fair comparison between models. For our existing results presented in the paper, we needed 70 single GPU-days (not including experimental and ineffective runs). Performing a similarly thorough hyperparameter search on more complex structures or datasets would likely require even more resources. While considering such settings would certainly be interesting, performing the necessary simulations is unfortunately presently not possible for us for practical reasons. We are aware of this limitation of our study and note that we mention it in the Discussion section. Finally, we also note that besides a simple performance comparison, a focus of our study is a detailed analysis of the training process. We will emphasize the latter by introducing a new paragraph to the Discussion section.
We chose the LIF neuron for our comparisons since it is the most widely used spiking neuron model. Our approach is to focus on a single setting, carefully optimize both QIF and LIF networks on it and analyze them in detail. Unfortunately, performing a similarly careful optimization and analysis on a broad class of models is practically hard because of the conceptual difficulties of a fair model comparison and because of the high computational cost. We are aware of this limitation of our study and note that we mention it in the Discussion section.
Next, we answer the reviewers questions.
We agree that a comparison between exact spike-based gradient descent and surrogate gradient descent is an important, largely unaddressed problem. Here, the goal is a comparison of QIF and LIF neurons in a relevant practical application. For this, we choose to use exact spike-based gradient descent. Our goal is not a comparison between exact spike-based gradient descent and surrogate gradient descent. It may be that QIF neurons also outperform LIF neurons when both are trained with surrogate gradient descent. To keep its focus, our study should stick to a single gradient-based training method. Performing an equally thorough comparison using a different method or dataset would be a further study, due to the conceptual difficulty of performing a fair comparison (surrogate gradient descent has, for example, as an additional hyperparameter, the shape of the threshold function surrogate) and due to the large computational demand for the thorough hyperparameter tuning that will be necessary for a fair comparison between methods.
We will add a paragraph to the Discussion in which we will emphasize the focus of our study and the importance of a comparison between exact spike-based gradient descent and surrogate gradient descent.
Processing a single spike requires in our experience practically the same amount of time for QIF and LIF neurons. The main contributor to the training and inference time is the number of spikes that have to be simulated. This highly depends on the hyperparameters (e.g. higher input drives I_0 lead to more spikes). Since we did not focus on minimizing the number of spikes, but considered the accuracy to be our only target, we cannot make reliable claims about how many spikes are actually needed to achieve a good accuracy. For example, there might be hyperparameter configurations with a slightly smaller mean test accuracy that require significantly fewer spikes. A comparison of the number of spikes and training times between our best LIF and QIF networks would thus not be fair. Hence, we cannot make any claims on which model is more computationally efficient in practice.
We agree that improving efficiency, especially when applied in a neuromorphic device, will be an important subject for further studies and we will emphasize this point in the revised Discussion.