- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this work, the authors tackle runtime adaptation of temporal integration in spiking neural networks. Their design centers on a LIF neuron whose learned reference timescale is locally modulated by both input currents and recent firing activity. The paper also provides a theoretical analysis detailing when this formulation reduces spike counts under matched inputs. Evaluated across neuromorphic vision, physiological data, and event-based speech benchmarks, the proposed neuron improves both accuracy and firing sparsity while offering clear potential for energy savings.
Critical strengths
Careful theoretical treatment of firing sparsity under runtime modulation. The authors provide a thorough mathematical analysis of how dynamic temporal modulation shapes spike sparsity, working out an exact finite-horizon firing-rate balance alongside sufficient conditions for firing suppression. I especially like that the paper clearly separates what is guaranteed at the single matched-neuron level from what actually carries over to trained multi-layer networks. (As a side note, the AI Use Statement mentions using AI tools to help develop and verify the mathematical proofs, though it leaves the exact scope of that assistance somewhat vague.)
Critical weaknesses
- My main concern is that the proposed neuron appears to offer an incremental modification of existing neuronal dynamics. In RATI-LIF, recurrent input and output-spike traces are used to modulate and the membrane retention factor, which is really just another way of doing history-dependent gating of membrane memory. We already have several closely related mechanisms in the SNN literature, including compartment coupling in LM-H [1] and TC-LIF [2], dynamic feedback modulation in STC-LIF [3], and state-dependent auxiliary dynamics in CLIF [4]. I realize these models are not identical on paper, but the authors do not adequately explain what their specific formulation achieves that these prior neurons cannot. Highlighting firing sparsity is a nice angle, though it does not by itself make the neuron design novel.
- The practical advantages of RATI-LIF are not sufficiently established. First, Appendix A.6 already shows quite a bit of sensitivity to and on a single task, and there is no ablation on the clipping bounds in Eq. (9). In fact, I could not even find the values of those bounds in the experimental setup, which makes me wonder how much the model relies on hand-tuned clipping to stay stable. Second, RATI-LIF carries extra state variables and has to recompute its decay factor at every timestep. Without seeing actual training time and inference latency numbers measured on the same GPU against existing neuron models, I am skeptical of the claim that this is a lightweight practical realization.
[1] Hao, Z., et al. A progressive training framework for spiking neural networks with learnable multi-hierarchical model. ICLR 2024.
[2] Zhang, S., et al. Tc-lif: A two-compartment spiking neuron model for long-term sequential modelling. AAAI 2024.
[3] Lihao Wang, Zhaofei Yu. Autaptic synaptic circuit enhances spatio-temporal predictive learning of spiking neural networks. ICML 2024.
[4] Huang, Y. et al. CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Network. ICML 2024.
Questions for authors
- What actually sets RATI-LIF apart from the mechanisms already used in LM-H, TC-LIF, STC-LIF, and CLIF? A side-by-side comparison would really help clarify whether the core contribution comes from a genuinely different class of neuronal dynamics, a cleaner implementation, or simply a better empirical trade-off between accuracy and firing sparsity.
- What exact values did you use for and , and how were they chosen? Also, how often does actually hit or saturate at either bound during training and inference? Please show whether the gains in accuracy and spike sparsity still hold up when you vary the modulation coefficients and clipping bounds across more than just one dataset.
- Could you provide the per-iteration training time and inference latency measured on the same hardware against LIF, PLIF, and representative adaptive neuron baselines? To keep the comparison fair, all runs should share the exact same backbone, number of timesteps, and level of kernel optimization. Reporting peak GPU memory usage would also be very helpful for quantifying the overhead of tracking those extra neuronal states.
2 comments
Q3. Training and inference efficiency
To assess the practical computational overhead of RATI-LIF, we have conducted additional experiments comparing standard LIF and RATI-LIF on the SHD dataset under controlled experimental conditions. Both models employ the same two-hidden-layer recurrent fully connected network with 128 neurons per hidden layer, trained for 150 epochs with a batch size of 128 and 100 simulation timesteps. All experiments are conducted on the same NVIDIA RTX PRO 6000 GPU using identical training configurations and the same procedure for measuring peak GPU memory usage.
Table R7: Comparison of total training time, test time, and peak GPU memory usage on SHD.
| Neuron | Total Training Time (s) | Test Time (s) | Peak GPU Memory (MB) |
|---|---|---|---|
| LIF | 3970.41 | 0.913 | 1026 |
| RATI-LIF | 4687.23 | 1.197 | 1126 |
As shown in Table R7, RATI-LIF increases total training time, test time, and peak GPU memory usage compared with standard LIF. This additional overhead primarily arises from maintaining the two recursive modulation states and dynamically updating the membrane time constant. These results suggest that RATI-LIF can realize runtime-adaptive temporal integration with relatively limited computational and memory overhead under the evaluated configuration.
Due to the limited time available during the rebuttal period, we have not completed per-iteration training-time measurements, finer-grained inference latency measurements, or comparisons with PLIF and other adaptive neuron baselines. We therefore provide the controlled SHD comparison as an initial assessment of the practical overhead introduced by RATI-LIF. We will incorporate these results and clarify the scope of the efficiency evaluation in the revised manuscript.
We thank the reviewer again for the valuable suggestions. We will incorporate the additional results and clarifications into the revised manuscript and explicitly acknowledge the remaining limitations of our evaluation.
We sincerely thank the reviewer for the insightful comments on the methodological distinction, clipping behavior, and practical efficiency of RATI-LIF.
Q1. Distinction from existing adaptive neuron models
Our primary motivation is to investigate how runtime-adaptive temporal integration affects firing sparsity, rather than to introduce a fundamentally new class of neuronal dynamics. RATI-LIF serves as a lightweight instantiation of this research perspective, with theoretical characterization provided in Sec. 2.3 and Appendix A.2.
Mechanistically, LM-H and TC-LIF employ coupled membrane potentials, STC-LIF modulates membrane retention and input through learnable feedback connections, and CLIF regulates post-spike reset through a complementary potential. RATI-LIF instead directly modulates the membrane time constant through local input and output-spiking histories, providing a simple formulation for investigating the relationship between runtime-adaptive integration and firing sparsity.
Q2. Clipping bounds and hyperparameter sensitivity
As described in Appendix A.3, the clipping bounds are introduced primarily to ensure numerical stability and maintain a physically meaningful membrane decay factor. Specifically, we set τmin=0.01 and τmax=10.1 across all experiments. The lower bound ensures that the effective membrane time constant remains strictly positive, preventing zero or negative values, while the upper bound prevents excessively large time constants that would cause the membrane decay factor ρm(t)=exp(−Δt/τ(t)) to approach 1. Together, these bounds constrain the decay factor within (0,1), maintaining stable membrane integration. The theoretical effects of clipping are explicitly considered in Appendix A.2.5 (Eqs. 84–89).
To further investigate the practical impact of clipping, we additionally measured the frequency at which the adaptive membrane time constant reaches the lower and upper bounds during training and inference across different datasets. The results are summarized in Table R6.
Table R6: Frequency (%) of the adaptive membrane time constant reaching the clipping bounds during training and inference.
As shown in Table R6, no upper-bound saturation is observed in the evaluated datasets. Lower-bound saturation is below 1% on CIFAR10-DVS and SHD, while it occurs in approximately 14% of updates on DVS-Gesture during both training and inference.
The absence of upper-bound saturation is consistent with the fact that the learnable reference time constant τ0 is initialized below the upper bound and remains below it throughout training, while the observed runtime modulation does not push the effective time constant beyond τmax. These results indicate that the adaptive time constant operates within the unclipped range for the majority of neuronal updates, although the extent of lower-bound saturation varies across datasets.
Regarding the modulation coefficients, the values of λ and η used across different datasets are reported in Table 5 of the manuscript, while their sensitivity is analyzed on DEAP in Appendix A.6 (Table 7). The results demonstrate that RATI-LIF maintains competitive performance across multiple coefficient configurations within a reasonable range, although certain extreme settings can lead to performance degradation. This indicates that the model does not depend exclusively on a single hyperparameter configuration, while appropriate parameter selection remains important.
Due to the limited time available during the rebuttal period, a comprehensive sensitivity analysis of the modulation coefficients and clipping bounds across all datasets and architectures could not be completed. Nevertheless, the existing hyperparameter ablation and the additional clipping-frequency statistics provide further evidence that the proposed runtime adaptation mechanism operates predominantly within the unclipped range in the evaluated settings. We will incorporate these clarifications and additional results into the revised manuscript.