- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors introduce an adaptive spiking neuron that learns a layer-wise shift for its activation window, alongside a normalized variant designed to keep training stable. The main goal of this formulation is to marry efficient integer-based training with adaptive firing dynamics. On the empirical side, the paper evaluates the approach across image classification, event-based recognition, and language modeling benchmarks, and further examines how the neuron behaves as both model size and pre-training data scale up.
Critical strengths
The breadth of the experimental section is a clear plus, as it covers static images, event-based datasets, and multiple language benchmarks. It is also helpful that the authors benchmark their approach against recent spiking neuron models using identical backbone architectures, which makes it much easier to isolate and judge the actual impact of the proposed changes across different tasks.
Critical weaknesses
- There is a mathematical contradiction in the spike-driven inference claim, alongside limited technical novelty. If is not an integer, the quantizer in Eq. (7) can produce non-integer boundary values. That means taking its output and subtracting will not necessarily give a non-negative integer, which contradicts the binary-spike decomposition needed for Eq. (10), and NASN runs into the same problem in Eq. (12). Even setting this aside, the proposed mechanism mostly just introduces a learnable layer-wise offset. How is this meaningfully different from simply learning a bias in the integrated input current? Because this distinction is not established, the methodological contribution looks quite thin and does not justify calling it a next-generation ``general-purpose'' neuron.
- The experimental results do not convincingly prove stronger scalability. First, the improvements over NILIF on several ImageNet and CIFAR-100 runs are quite small. Second, the scaling comparison conflates model size with pre-training data size, and Table 6 is missing the NILIF results across the different token budgets. Because of these missing controls, I cannot tell if the proposed neurons genuinely scale better or if the gains just come from extra pre-training.
Questions for authors
- My main concern lies with the methodological novelty. How does the firing mechanism in Eq. (7) meaningfully differ from simply adding a learnable layer-wise bias to the integrated input current, or just learning a layer-wise firing threshold? Is there a real functional difference here, or does it boil down to a trivial reparameterization? Given how close these formulations are, what justifies pitching this as a next-generation ``general-purpose'' neuron rather than a straightforward tweak to an existing integer-based neuron?
- When takes a non-integer value, how can NASN still satisfy the binary-spike decomposition in Eq. (10)? Please write out the exact quantizer formulation used in your actual implementation.
- Do you have the matching NILIF baseline numbers for all the pre-training token budgets listed in Table 6? Without those side-by-side numbers, how can we separate the actual architectural benefit of the proposed neuron from the obvious gains of simply training on more tokens?