acceptodds
Withdrawn from ICLR 2027

Probabilistic Neuronal Dynamics: Expanding LIF Temporal Dynamics for Probabilistic Spike Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The temporal modeling capability of a spiking neuron depends on both how its dynamics weight historical inputs and how the resulting membrane state is converted into binary spikes. In a primitive reset-free leaky integrate-and-fire (LIF) neuron, these two stages are respectively constrained by a single geometric temporal response and a deterministic threshold that maps membrane values on the same side of the threshold to the same instantaneous event. These coupled restrictions motivate a neuronal mechanism that enriches the temporal response space before firing while allowing the magnitude of the resulting membrane trajectory to modulate spike generation without abandoning binary communication. We propose Probabilistic Neuronal Dynamics (PND), which expands this response space through multiple learnable temporal modes. A learned readout combines these modes into a scalar membrane trajectory, allowing the neuron to shape its response across delays beyond a single geometric decay. An exponential event model maps the readout to a firing probability, so membrane magnitude controls firing tendency while each emitted spike remains hard and binary. We characterize the response space through basis-rank analysis and non-geometric mode composition. Learned-mode measurements reveal diverse response shapes with low redundancy, and a capacity sweep links state expansion to improved task performance before saturation. Component comparisons identify the task-level role of expanded dynamics and examine probabilistic firing with its associated gradient rule. Repeated inference shows stable aggregate predictions under stochastic sampling. Across temporal modeling benchmarks, PND achieves competitive performance while retaining parallelizable pre-spike computation and sparse binary communication.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Spiking Neural Networks, LIF Neurons, Neuronal Dynamics, Temporal Representation, Probabilistic Spike Generation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_28RwOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The standard spiking neural network architecture with leaky-integrate and fire neurons is modified in two ways. The first modification replaces the scalar LIF temporal dynamics with an expansion to dstated_{\text{state}} dimensions that is followed by a linear transformation back to R\mathbb{R}. The second modification replaces the threshold firing rule with the sampling of a Bernoulli random variable whose parameter is determined by the membrane potential. A number of experiments are carried out to test the performance of these two modifications.

Critical strengths

The paper presents an understandable treatment of two interesting modifications.

Critical weaknesses

I suggest reject primarily due to issues with the experiments. The paper would be strengthened if it can isolate the effects of the modifications. It is likely that the modifications are not always desirable, and so identifying the situations where the modifications are and are not beneficial would be valuable. If comparing to prior works, their methods should be implemented, which gives the opportunity for confounds such as architecture size to be controlled and will allow for a shared training and testing infrastructure.

1. Experiments do not isolate the proposed changes (e.g. parameter count confound)

The two proposed modifications that come under the paper's PND label can be dropped in to any architecture that supports it. Section 4.3 does some work to isolating their effect, but it is insufficient. Section 4.3 tests a PND applied to a single base architecture compared to PND w/o probabilistic firing and PND w/o temporal expansion. The natural fourth condition (LIF dynamics with fixed firing threshold) is missing. Furthermore, a baseline that has its parameter counts match those of PND (which has increased learned parameters due to the temporal expasnion) should be included, as currently there is no control for parameter count. Lastly, an ablation that covers various base architecture sizes would be valuable. Without these comparisons, it is very hard to understand the effect of the modifications.

2. Weakness of comparison to other models (e.g. training and testing is not controlled)

Table 1 compares models from other works with the base architecture (from appendix E.1) modified with the PND modifications (the latter I'll refer to as the PND model). It appears that many hyperparameters of the PND model are tuned per-dataset, including the expansion dimension dstated_{\text{state}}. The baseline model performances look to be copied from other papers (also per-dataset tuned). A weakness of such a comparison is that one can "beat" another model by simply performing a more thorough hparam search. The response to this criticism may be: "this is what is done in the literature", and that may be true, but that does not change the fact that the comparison does not give the reader much understanding of the two proposals presented by this paper.

Furthermore, by copying the results from other papers, rather than re-running the baseline models, it is possible that the training and eval setup differences may be the causes of score differences. Consider the AAN dataset: the paper describes concatenation along the batch dimension (presumably for efficient GPU utilization), but the code (from the accompanying zip) concatenates them along the feature dimension. This suggests that the PND model is likely trained and evaluated differently to the other models and the results cannot be compared apples-to-apples to the other baselines' scores. A stronger comparison would have the authors implement the models and train and test them together.

Related to the tuning for Table 1, results in Figure 2 are also created using a base architecture specific to those tests (with different layer counts and widths). There is nothing wrong with this per se, but there is no justification given for it.

3. Test set used for checkpoint selection

You say: "IMDB has no separate validation split and reports the best test accuracy recorded during training" (L957). Performance on the test set should not be used for choosing the model checkpoint. A lack of a designated validation set is common, and practitioners should create their own validation set from the training set. By not specifying a fixed validation set, the dataset supports things such as k-fold cross-validation. Again, it is not a justification that prior work has also made this mistake. This is another reason why all baselines should be implemented and trained-tested on the same setup: so that you do not inherit flaws of previous works.

4. Variability quantification

It looks like there are just single runs for the results in Table 1. Some of the accuracies are quite similar, and it's not clear if stochasticity in training or evaluation can be ruled out as explanations for some of the differences.

Some issues/suggestions

The following are more minor concerns.

  1. Terms used in the paper should be defined on first use, even when standard. Mention that I1:tI_{1:t} is the input history, and specify its type (sequence of reals) (L157). In the appendix, terms like R,Iphys,u˙R, I_{\text{phys}}, \dot{u}, \tau_m, \tau $ are not defined, but are assumed to be known.

  2. "temporal mode" is a fundamental concept to the paper's argument and is used multiple times before any definition.

  3. Appendix A1 uses the same u(t) symbols to refer to both the continuous and discrete functions.

  4. The dd in dI(t)dI(t) is not defined. It appears to be a real, but it is easily confused with an attempt to represent a small change in I(t)I(t).

  5. LIF uses symbol $ u_t $ but also the (undefined) utLIFu_t^{\text{LIF}}.

  6. For consistency with later used indexing, equation 3 could be written as:

    \qquad K_\tau=\alpha^\tau.$$ Which puts K in the LHS and uses the same notation that is used elsewhere.
  7. Equation 6 depends on 7, so swap the order, or combine onto one line as one numbered equation.

  8. The fact that the paper concerns spiking neural networks should be mentioned in the abstract or title

  9. The differences to related work are mentioned in Section 2; however, statements like "These mechanisms enrich temporal behavior, but state dimension alone does not establish the independence of the resulting responses or how they combine into a scalar membrane trajectory" (L134-136) and "Their conditional event distributions alone do not establish prediction consistency under repeated sampling, which requires empirical evaluation" (L141-142) are stated without any supporting argument or evidence.

Questions for authors

Why present the two quite separate modifications together? I imagine that characterizing one of these in detail would create enough material for a single paper. Is the interaction between both important?

Reviewer_4pU5OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces "Probabilistic Neuronal Dynamics" (PND)—a spiking neuron model. The model integrates various learnable temporal patterns into a scalar membrane potential readout, which is then mapped to a spike-firing probability via an exponential function. It produces discrete, binary spikes and is trained using a "probability-path straight-through gradient estimator." Furthermore, the model supports parallel evaluation of its linear temporal dynamics. The authors characterize the response space expanded by the model through theoretical rank analysis, diagnostics of learned patterns, and a scan of state dimensions. Empirical validation includes five tasks from the "Long Range Arena" benchmark, component ablation studies, repeated stochastic inference, and other sequence-processing benchmarks. Experimental results demonstrate that the model achieves highly competitive performance in sequence modeling.

Critical strengths
  1. The idea of ​​expanding the temporal response space to enable neurons to respond to a richer set of temporal information is a promising design direction. However, the authors relegate a substantial portion of their response analysis to the appendix, with only minimal discussion in the main text. (importance: 7/10)
  2. PND achieves competitive performance across the five evaluated LRA tasks, with the highest reported average accuracy among the listed SNN baselines (83.87%) and particularly strong performance on Pathfinder (96.84%). These results support the practical relevance of the proposed neuronal mechanism across different sequence-modeling tasks. (importance: 7.5/10)
Critical weaknesses
  1. The manuscript is poorly organized, and its visual presentation is strikingly ineffective. The figures occupy substantial space while communicating remarkably little, squandering the opportunity to explain the method and establish the significance of its claimed innovations. The core contribution is buried in a diffuse narrative that urgently needs tighter organization and more purposeful visual exposition. These are basic failures of scientific communication that should have been addressed before submission. The presentation falls well below the standard expected of an ICLR submission. (importance 10/10)
  2. The paper’s appendix notes that the IMDb test set was used for both checkpoint selection and hyperparameter tuning. This approach carries the risk of test set leakage and does not align with standard practices for data augmentation or hyperparameter tuning. (importance 9.5/10)
Questions for authors

Clarification requests:

  1. Section 2 indicates that P-SpikeSSM already incorporates both extended neuronal states and a probabilistic spiking mechanism. Given this, what are the fundamental differences between PND and P-SpikeSSM in terms of their mathematical principles and architectural design? Please elaborate on the respective contributions of each approach regarding temporal dynamics, the mapping of spiking probabilities, and gradient estimation.
  2. Appendix E.3 indicates that the IMDb test set was used for both checkpoint selection and hyperparameter tuning. Could the authors provide experimental results using a validation set derived exclusively from the training data, while reserving the test set solely for final evaluation? How would this affect the benchmark results and the conclusions drawn from the comparative analysis of the various components?
Reviewer_PrekOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper combines an expanded, stable linear state-space recurrence with a scalar membrane readout and an exponential membrane-to-probability link. Hard binary events are sampled with a probability-path straight-through estimator. It analyzes temporal basis rank and learned response diversity, evaluates long-sequence classification, and separately reports stochastic-inference variation, parallel execution rates, and sparse-projection arithmetic estimates.

Critical strengths
  1. The response-space analysis connects an explicit neuronal parameterization to measurable learned behavior. The capacity sweep and the distinction between raw and column-normalized effective rank are useful diagnostics, and the assumptions behind the algebraic rank argument are stated clearly.

  2. The evaluation distinguishes several quantities that are often conflated: training a different event mechanism, resampling a fixed checkpoint, execution speed, and estimated projection energy. Table 2 also reports cases where the deterministic or single-mode alternative performs better, which makes the component analysis more informative.

Critical weaknesses
  1. The IMDB result is selected and tuned on the test set. Appendix E.3, p. 18, explicitly states that IMDB reports the best test accuracy during training and that the same evaluation is used to adjust hyperparameters. Consequently, its Table 1 entry and its contribution to the five-task average are not an independent held-out evaluation. This does not invalidate the other four tasks, but it prevents treating the reported average as a clean benchmark comparison and also affects interpretation of the IMDB component results. The appropriate repair is a training/validation selection protocol followed by one frozen test evaluation.

  2. The contribution beyond existing probabilistic spiking state-space models is not sufficiently isolated. Expanded linear states, a scalar readout, probabilistic binary outputs, and parallel kernel evaluation already occur in P-SpikeSSM (Bal and Sengupta, ICLR 2025; https://openreview.net/forum?id=Sf4ep9Udjf). The Vandermonde argument establishes a property of distinct linear modes, rather than a separation from that family. Table 2 compares against a single mode and changes firing and backward rules together; it does not isolate the exponential link from the state-space parameterization and gradient estimator against a matched multistate probabilistic baseline. Moreover, the ten repeated inference passes use earlier checkpoints (Appendix G.1), so they do not establish training-seed robustness or sampling stability of the actual Table 1 checkpoints. The evidence supports a useful implementation, but does not yet establish the significance of its particular design choices.

The recommendation is driven by the test-set selection issue and the insufficiently isolated advance over probabilistic spiking SSMs. A clean evaluation and a matched comparison could change the assessment.

Questions for authors
  1. Can you provide IMDB results for PND and its component controls using a validation split drawn only from training data, frozen hyperparameters/checkpoints, and an untouched test evaluation, and update the corresponding five-task average?

  2. Under the same backbone, state size, optimization budget, and data protocol, how does the exponential link compare with a clipped-affine or logistic probability link when the same backward estimator is used? Can you also report training-seed variation and repeated-inference statistics for the actual benchmark checkpoints, so that gains over a matched probabilistic SSM can be assessed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.