acceptodds
Under review as a conference paper at ICLR 2027

A Time-Variant Spiking Neuron from a Unified State-Dynamics Perspective

12 views

Abstract

Spiking neural networks (SNNs) have increasingly incorporated rich neuronal dynamics to enhance temporal processing. However, richer dynamics do not necessarily enable neurons to adapt how information is processed over time. To examine this issue systematically, we introduce a unified state-dynamics framework that characterizes diverse spiking neurons through a common formulation and enables direct comparison of their dynamical designs. The framework reveals a common limitation of existing neurons: although they enhance temporal processing through richer internal states and state evolution, the dynamical rules determining how inputs are integrated, how states evolve, and how they are read out generally remain unchanged throughout temporal processing. Such time-invariant rules limit the ability of neurons to adjust information processing as information relevance changes over time. Inspired by time-varying regulation in biological neurons, we propose the Time-Variant Multi-Compartment Neuron (TV-MCN). TV-MCN builds on multiscale compartment dynamics and makes the rules for input integration, state evolution, and state readout time-varying according to the current input. Specifically, an input-dependent internal time regulates the rate of state evolution, while the current input dynamically modulates input integration and state readout. Extensive experiments show that TV-MCN achieves strong overall performance, with particularly large gains on tasks requiring selective retention and retrieval of information, while ablation studies further support the effectiveness of its time-varying mechanisms.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spiking Neural Networks, Multi-Compartment Neurons, Time-Varying Dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jJXXOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce a unified state-space framework to compare different spiking neuron models, which leads to the design of a time-variant multi-compartment neuron. This new neuron model pairs multiscale oscillatory states with input-driven gates that control state transitions, input integration, and readout. Empirically, the approach delivers competitive accuracy on temporal classification benchmarks and shows clear advantages on synthetic memory tasks, with additional ablation and generalization studies backing up the design of the modulation mechanisms.

Critical strengths
  1. The results on selective copying and associative recall really stand out. TV-MCN achieves large gains here, and seeing this advantage hold consistently across several tasks under a unified network architecture makes the empirical contribution hard to ignore.
  2. The evaluation goes beyond simple test accuracy with well-designed memory probes. The delayed-recall and sequence-length experiments are particularly helpful. By padding the input with distractors while preserving the underlying target sequence, the authors directly test the exact state-retention capability that motivates their model.
Critical weaknesses
  1. It is not entirely clear what sets this model apart from selective SSMs such as Mamba. Having input-driven Δt\Delta_t, BtB_t, and CtC_t parameters is already the defining feature of Mamba. The authors need to do a much better job of explaining what the spiking realization specifically adds on top of standard selective state spaces.
  2. The math behind the reset dynamics is not spelled out clearly enough. Since the somatic membrane potential is just a readout ut=Cthtu_t=C_t h_t, saying it undergoes a soft reset only tells us what happens to utu_t, without specifying how the persistent internal state hth_t is updated upon firing.
Questions for authors
  1. Could you write out the exact reset function used in TV-MCN and clarify which post-reset states actually carry over into the next time step? Also, please specify the exact dimensions of xtx_t, BtB_t, and CtC_t.
  2. Conceptually and mathematically, what sets TV-MCN apart from a complex-valued selective SSM with a spiking readout bolted on at the end? Including a direct comparison or discussion of the model with and without the spiking and reset mechanisms would really help clarify where the core novelty lies.
2 comments
AuthorsOpenReview ↗

1. Spiking Neuron Reset Mechanism and Parameter Dimensions

We sincerely thank the reviewer for the careful evaluation of our work and the constructive suggestions. We address each concern below.

We thank the reviewer for pointing out that the descriptions of the reset mechanism and parameter dimensions were not sufficiently clear. We provide further clarification below.

In TV-MCN, the complex-valued multi-compartment state hth_t evolves continuously and is read out through CtC_t to provide input to the soma. The soma accumulates this input in a separate membrane potential state and performs a soft reset after spike generation:

ut=Re⁡(Ctht),vt−=vt−1++ut,st=H(vt−−ϑ),vt+=vt−−ϑst.\begin{aligned} u_t &= \operatorname{Re}(C_t h_t),\\\\ v_t^- &= v_{t-1}^+ + u_t,\\\\ s_t &= H(v_t^- - \vartheta),\\\\ v_t^+ &= v_t^- - \vartheta s_t. \end{aligned}

The reset only affects the somatic membrane potential and does not alter the multi-compartment state. Therefore, the reset mapping for the state is Rh(ht,st)=htR_h(h_t,s_t)=h_t. At the next time step, both hth_t and the reset membrane potential vt+v_t^+ are retained and used for subsequent state evolution and somatic integration, respectively.

Regarding parameter dimensions, for a single neuron containing NN compartments:

xt∈R,ht∈CN,Bt∈CN×1,Ct∈C1×N.x_t\in\mathbb R,\quad h_t\in\mathbb C^N,\quad B_t\in\mathbb C^{N\times1},\quad C_t\in\mathbb C^{1\times N}.

Here, xtx_t is the scalar input current received by the neuron, while BtB_t and CtC_t are generated from the complete input vector xt∈RD\mathbf{x}_t\in\mathbb R^D of the current layer.

AuthorsOpenReview ↗

2. Relationship between our method and Selective SSM & Mamba

We thank the reviewer for pointing out the connection between TV-MCN and Selective SSM. We acknowledge that our method draws on Selective SSM in terms of parameterization, and we would like to take this opportunity to clarify the motivation and core contributions of our work.

The main contribution of this paper is to summarize existing spiking neuron designs within a unified state-dynamics framework and, based on this framework, identify an overlooked issue: the state evolution, information writing, and state readout of existing neurons typically still follow fixed rules. Therefore, we aim to investigate whether introducing time-varying dynamics can further improve the temporal information processing capabilities of individual spiking neurons. Within this framework, a time-varying spiking neuron can be expressed as:

ht=Atht−1+Btxt,ut=Ctht,st=H(ut−ϑ),ht+=R(ht−,st).\begin{aligned} h_t &= A_t h_{t-1}+B_t x_t,\\\\ u_t &= C_t h_t,\\\\ s_t &= H(u_t-\vartheta),\qquad h_{t^+}=R(h_{t^-},s_t). \end{aligned}

The key idea behind this research direction is to allow the neuron's dynamical rules to change over time, and dynamically adjusting these rules according to the current input is one particular approach explored in this paper (other possible approaches include adjusting these rules based on the neuron's internal state itself rather than its input). To this end, TV-MCN adopts the input-dependent parameterization of Δt\Delta_t, BtB_t, and CtC_t previously studied in Selective SSM. However, our research consistently focuses on the design of the internal dynamics of individual spiking neurons, and does not adopt Mamba's input and output projections, local causal convolutions, nonlinear activations, gating branches, residual connections, normalization, or other such architectural designs.

Therefore, the goal of this paper is not to improve performance by adding spike generation and reset mechanisms to Mamba; these mechanisms are already integral components of spiking neurons. What we aim to investigate is how modifying a neuron's internal dynamical rules can improve its temporal processing capabilities. The input-dependent parameterization used in Selective SSM is simply an established approach that we can draw upon.

To further verify that this idea does not depend on the specific parameterization of Selective SSM, we introduce an additional input-dependent design that directly generates AtA_t, BtB_t, and CtC_t from the current input, allowing the neuron's state evolution, input integration, and state readout to vary over time without introducing a dynamic Δt\Delta_t. The specific parameterization is as follows:

at=fA(xt),ωt=fω(xt),At=diag⁡(exp⁡(−eat+jωt)),Bt=fB(xt),Ct=fC(xt).\begin{aligned} a_t &= f_A(\mathbf{x}_t),\qquad \omega_t=f_\omega(\mathbf{x}_t),\\\\ A_t &= \operatorname{diag}\left( \exp\left(-e^{a_t}+j\omega_t\right)\right),\\\\ B_t &= f_B(\mathbf{x}_t),\\\\ C_t &= f_C(\mathbf{x}_t). \end{aligned}

With all other experimental settings unchanged, this design also achieves competitive results across tasks. On MAD ICR and MAD Noisy ICR, it achieves 98.88% (-1.12 pp) and 99.22% (-0.77 pp), respectively, and reaches 100% (+1.60 pp) word accuracy on STORE--RECALL. On GSC, SHD, SSC, and DVS128 Gesture, it achieves 90.59% (+0.03 pp), 89.05% (+0.75 pp), 73.22% (+2.24 pp), and 91.67% (-2.08 pp), respectively. The values in parentheses indicate the differences in accuracy relative to the original TV-MCN with input-dependent Δt\Delta_t, BtB_t, and CtC_t.

These additional experiments further demonstrate that the effectiveness of time-varying dynamics in the input-dependent setting investigated in this paper is not limited to the parameterization used in Selective SSM. We thank the reviewer for raising this important question and for the opportunity to clarify the motivation and contributions of our work.

Reviewer_bvK9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that existing spiking neurons, despite incorporating multiple compartments or heterogeneous time constants, lack input-dependent control over state evolution, writing, and readout. To address this, the authors propose TV-MCN, which dynamically modulates the time step Δt\Delta_t, input matrix BtB_t, and readout vector CtC_t within multi-timescale oscillatory compartments. TV-MCN achieves competitive performance on classification tasks and substantially outperforms existing spiking models on long-range selective memory tasks. However, mathematically, TV-MCN is essentially a spiking variant of an SSM, while its multi-compartment design offers only a loose connection to biological neurons.

Critical strengths
  1. The authors provide a detailed description of TV-MCN’s mathematical formulation and implementation, making the proposed method easy to follow.

  2. The experiments cover temporal classification and synthetic sequence tasks. The results demonstrate TV-MCN’s advantages across multiple benchmarks, particularly in memory and retrieval tasks.

Critical weaknesses
  1. The fundamental mathematical mechanism of TV-MCN - introducing input-dependent modulation over the discretization step size Δt\Delta_t, input projection BtB_t, and state readout CtC_t - is almost identical to the selection mechanism proposed in Mamba (Gu & Dao, 2024). However, the authors cite Gu & Dao (2024) merely as the dataset source for synthetic tasks and Gu et al. (2022) for ZOH discretization, completely failing to discuss or acknowledge the direct methodological heritage from Selective State Space Models. Instead, the paper packages this selective mechanism entirely as an original concept inspired by biological time-varying regulation and multi-compartment dynamics. The lack of an explicit conceptual comparison with Selective SSMs obscures the true incremental novelty of this work.

  2. The paper places considerable emphasis on TV-MCN's substantial performance gains on synthetic long-sequence tasks, including Selective Copying, Induction, and MAD. However, these results are not particularly surprising, as such benchmarks are specifically designed to evaluate the noise-filtering and content-dependent retrieval capabilities of selective SSMs and Mamba. Thus, the observed improvements may primarily stem from introducing a Mamba-like selective mechanism rather than the proposed multi-compartment spiking neuron design itself. Without appropriate controls to disentangle these contributions, attributing the gains to enhanced temporal processing capabilities of multi-compartment spiking neurons is not sufficiently justified.

Questions for authors
  1. How are gradients backpropagated through the non-differentiable threshold function s(t)=H(u(t)−ϑ)s(t) = H(u(t) - \vartheta)?

  2. How does a standard, non-spiking Mamba model perform on these tasks under the same parameter budget (~0.06M)?

  3. The input-dependent parameterization of Δt\Delta_t, BtB_t, and CtC_t closely mirrors the selection mechanism in Mamba. Beyond applying complex-diagonal oscillatory states and an output spiking threshold with reset, what mathematically or formulationally distinguishes TV-MCN from a Selective SSM? Please explicitly clarify the novel mathematical contributions over existing Selective SSM formulations.

  4. In Eq. (10), all compartments appear to receive the same input x(t)x(t), with BtB_t determining their individual contributions.If so, how does it correspond to biological dendritic branches, which may receive distinct inputs?

6 comments
AuthorsOpenReview ↗

1. The non-differentiability of the threshold function

We thank the reviewer for this question. We directly adopt the Surrogate Gradient method widely used in the SNN literature (Wu et al., 2018, STBP). Specifically, the forward pass uses a hard threshold function to generate binary spikes, while the backward pass approximates its derivative using a triangular surrogate gradient:

s(t)=H(u(t)−ϑ),∂s(t)∂u(t)≈max⁡(0,1−∣u(t)−ϑ∣).s(t)=H(u(t)-\vartheta),\qquad \\\\ \frac{\partial s(t)}{\partial u(t)} \approx \max(0,1-|u(t)-\vartheta|).

This allows gradients to propagate through the non-differentiable spike generation function and update the neuron's dynamical parameters.

Reference:

Wu, Y., Deng, L., Li, G., Zhu, J., and Shi, L. (2018). Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks. Frontiers in Neuroscience, 12, 331. https://doi.org/10.3389/fnins.2018.00331

AuthorsOpenReview ↗

2. Comparison with a standard, non-spiking Mamba model

We thank the reviewer for suggesting a comparison with standard non-spiking Mamba under the same parameter budget.

First, it is important to clarify that Mamba is a complete sequence modeling architecture. In addition to Selective SSM, it includes input and output projections, local causal convolutions, nonlinear activations, gating branches, residual connections, and normalization. In contrast, our research focuses on the internal dynamics of individual spiking neurons and does not adopt the aforementioned Mamba network architecture.

The goal of this paper is not to improve performance by adding spike generation and reset mechanisms to Mamba. We focus on how the internal dynamics of spiking neurons should be designed, while spike generation and reset are inherent components of the spiking neurons under study, rather than additional mechanisms that this paper seeks to demonstrate are superior to Mamba.

We thank the reviewer for raising this question and for the opportunity to further clarify this misunderstanding.

AuthorsOpenReview ↗

3. Relationship between our method and Selective SSM

We thank the reviewer for highlighting the connection between TV-MCN and Selective SSM. We acknowledge that the input-dependent parameterization of Δt\Delta_t, BtB_t, and CtC_t in TV-MCN draws on the selective mechanism introduced in Mamba, and we will explicitly clarify this relationship.

We would like to emphasize that the main contribution of this paper is to establish a unified state-dynamics framework based on existing spiking neuron research and, through this framework, identify a long-overlooked issue: many existing neurons, despite incorporating richer internal states and temporal dynamics, still rely on largely fixed rules for state evolution, input integration, and state readout. Motivated by this observation, we investigate whether introducing time-varying dynamics can improve the temporal information processing capabilities of individual spiking neurons. The time-varying spiking neuron can be expressed as:

ht=Atht−1+Btxt,ut=Ctht,st=H(ut−ϑ),ht+=R(ht−,st).\begin{aligned} h_t &= A_t h_{t-1}+B_t x_t,\\\\ u_t &= C_t h_t,\\\\ s_t &= H(u_t-\vartheta),\qquad h_{t^+}=R(h_{t^-},s_t). \end{aligned}

Within this framework, input-dependent modulation is one possible realization of time-varying neuronal dynamics, for which TV-MCN adopts the established parameterization from Selective SSMs. Therefore, we do not claim the selective mechanism of Selective SSM as a novel mathematical contribution of this paper. Instead, we focus on how to redesign the internal computation of spiking neurons using time-varying dynamics.

To further verify that this core idea does not depend on the specific parameterization of Selective SSM, we introduce an additional input-dependent design that directly generates AtA_t, BtB_t, and CtC_t from the current input, allowing the neuron's state evolution, input integration, and state readout to vary with the input without introducing a dynamic Δt\Delta_t. The specific parameterization is as follows:

at=fA(xt),ωt=fω(xt),At=diag⁡(exp⁡(−eat+jωt)),Bt=fB(xt),Ct=fC(xt).\begin{aligned} a_t &= f_A(\mathbf{x}_t),\qquad \omega_t=f_\omega(\mathbf{x}_t),\\\\ A_t &= \operatorname{diag}\left( \exp\left(-e^{a_t}+j\omega_t\right)\right),\\\\ B_t &= f_B(\mathbf{x}_t),\\\\ C_t &= f_C(\mathbf{x}_t). \end{aligned}

With other experimental settings unchanged, this design also achieves competitive results across tasks. On MAD ICR and MAD Noisy ICR, it achieves 98.88% (-1.12 pp) and 99.22% (-0.77 pp), respectively, and reaches 100% (+1.60 pp) word accuracy on STORE--RECALL. On GSC, SHD, SSC, and DVS128 Gesture, it achieves 90.59% (+0.03 pp), 89.05% (+0.75 pp), 73.22% (+2.24 pp), and 91.67% (-2.08 pp), respectively. The values in parentheses indicate the differences in accuracy relative to the original TV-MCN with input-dependent Δt\Delta_t, BtB_t, and CtC_t.

These additional experiments further demonstrate that the effectiveness of time-varying dynamics in the input-dependent setting investigated in this paper is not limited to the parameterization used in Selective SSM. We thank the reviewer for raising this important question and for the opportunity to clarify the motivation and contributions of our work.

Reviewer_FiCjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a state-space perspective on spiking neuron models and proposes TV-MCN, which combines multiple complex-valued modes with input-dependent internal time, input integration, and state readout. It evaluates the model on four temporal classification datasets and six synthetic sequence tasks. The reported gains are particularly large on tasks requiring selective retention and retrieval.

Critical strengths
  1. Selective-memory performance and evidence for the mechanisms

    The evaluation covers several distinct information-processing requirements. Table 3 reports strong results, including 93.46% token accuracy on Selective Copying and 99.99% on MAD Noisy ICR, under a shared network depth and width.

    The mechanism ablations are informative: on Selective Copying, removing internal-time modulation or dynamic B/C reduces token accuracy to 7.20% or 61.24%, respectively (Table 5). The delayed-recall and instruction-dependent analyses add useful evidence.

  2. Detailed reporting that helps interpret the gains

    The appendix reports parameter counts, cost-estimation conventions, and stricter exact-sequence metrics. These help qualify the claims: for example, the shared-network Selective Copying result is 37.72% exact accuracy rather than near-perfect sequence recovery.

Critical weaknesses
  1. The robustness claim is inconsistent with the reported figure

    The high-noise robustness claim is inconsistent with the plotted evidence. On p. 7, lines 369–372 report TV-MCN accuracies of 99.99%, 99.99%, 99.97%, and 99.90% as the noise fraction increases, and 79.33% for HetSyn at 0.8. However, Figure 3 (right) visibly places TV-MCN around 70–75% at the two highest noise fractions and HetSyn around the mid-40% range at 0.8. These are approximate visual readings, but the discrepancy is much larger than rounding. It directly affects a central robustness claim. Appendix B.4 describes one protocol without explaining why the figure and text would differ.

  2. The contribution and experimental controls need clarification

    The novelty relative to selective SSMs is insufficiently established. Equations (6), (9), and (11) closely follow the input-dependent Delta/B/C and ZOH construction used in Mamba (Gu and Dao, 2024). That work is cited for benchmarks, but the methodological overlap is not adequately discussed. Complex modes and a spiking readout may provide a worthwhile adaptation; the present evidence does not isolate what the spiking/reset mechanism adds beyond a selective SSM. The broad fixed-dynamics framing also needs qualification in light of the state-dependent DGN formulation already discussed in Appendix A.2.

    The spiking implementation could be useful, but its contribution relative to selective state-space models needs a substantially sharper explanation.

Questions for authors
  1. Which data and protocol generated Figure 3 (right)? Please reconcile the plotted curves with the numerical claims on p. 7 and provide the corresponding values.
  2. What is the precise methodological difference from a selective SSM with complex modes and a spiking output? Can a small matched control, or existing results, identify the contribution of spike generation/reset, and can the internal-time ablation retain a learned static Delta?
  3. Please specify the reset map on the persistent complex state, including how it interacts with C_t, and report the number of independent training runs and their variability for the main results.
5 comments
AuthorsOpenReview ↗

1. **Correction of the Inconsistency in Figure 3 (Right)**

We sincerely thank the reviewer for the careful evaluation of our work and the constructive suggestions. We address each concern below.

  1. Inconsistency between Figure 3 (right) and the reported results:

    We thank the reviewer for pointing out the inconsistency between Figure 3 (right) and the numerical results reported in the main text. After carefully checking the experimental records and the manuscript, we confirmed that an outdated version of Figure 3 was inadvertently included in the submitted manuscript. We apologize for this oversight.

    Specifically, we conducted two sets of experiments on MAD Noisy ICR using different evaluation protocols. In the original protocol, models were trained separately at each noise ratio ρ\rho and tested at the corresponding ratio. To more directly evaluate the models' ability to generalize across different levels of interference, we adopted a revised protocol: all models were trained at ρtrain=0.2\rho_{\mathrm{train}}=0.2 and then evaluated using their respective fixed checkpoints at ρtest∈{0.2,0.4,0.6,0.8}\rho_{\mathrm{test}}\in\{0.2,0.4,0.6,0.8\}, without any retraining or additional checkpoint selection.

    The results reported in the main text and Appendix B.4 are all from the revised protocol. However, Figure 3 (right) in the submitted manuscript still contained the curves from the original protocol, resulting in the inconsistency between the figure and the reported results. We have now corrected Figure 3 (right) using the corresponding results from the revised experiments, ensuring consistency with both the numerical results in the main text and the experimental protocol described in the appendix.

    We appreciate the reviewer’s careful attention to this issue, which has helped us improve the accuracy and consistency of our presentation.

AuthorsOpenReview ↗

2. **Clarification of the Relationship with Selective SSM and Additional Ablation Studies**

  • Relationship between our method and Selective SSM:

    We thank the reviewer for highlighting the connection between TV-MCN and Selective SSM. We acknowledge that the input-dependent parameterization of Δt\Delta_t, BtB_t, and CtC_t in TV-MCN draws on the selective mechanism introduced in Mamba, and we will explicitly clarify this relationship.

    We would like to emphasize that our primary contribution lies in investigating time-varying dynamics at the level of individual spiking neurons, rather than proposing a new selective SSM mechanism. Specifically, we first establish a unified state-dynamics framework to systematically characterize existing spiking neuron designs. This framework reveals that many existing neurons, despite incorporating richer internal states and temporal dynamics, still rely on largely fixed rules for state evolution, input integration, and state readout. Motivated by this observation, we investigate whether introducing time-varying dynamics can improve the temporal information processing capabilities of individual spiking neurons. The time-varying spiking neuron can be expressed as:

    ht=Atht−1+Btxt,ut=Ctht,st=H(ut−ϑ),ht+=R(ht−,st).\begin{aligned} h_t &= A_t h_{t-1}+B_t x_t, \\\\ u_t &= C_t h_t, \\\\ s_t &= H(u_t-\vartheta),\qquad h_{t^+}=R(h_{t^-},s_t). \end{aligned}

    Within this framework, input-dependent modulation is one possible realization of time-varying neuronal dynamics, for which TV-MCN adopts the established parameterization from Selective SSMs. Our focus is not on adding spike generation and reset to selective SSM to improve its performance, but on improving the internal dynamics of spiking neurons, where spike generation and reset are already inherent components.

    To further verify that this idea does not depend on the specific parameterization of Selective SSM, we introduce an additional input-dependent design that directly generates AtA_t, BtB_t, and CtC_t from the current input, allowing the neuron's state evolution, input integration, and state readout to vary over time without introducing a dynamic Δt\Delta_t. The specific parameterization is as follows:

    at=fA(xt),ωt=fω(xt),At=diag⁡(exp⁡(−eat+jωt)),Bt=fB(xt),Ct=fC(xt).\begin{aligned} a_t &= f_A(\mathbf{x}_t),\qquad \omega_t=f_{\omega}(\mathbf{x}_t), \\\\ A_t &= \operatorname{diag}\left(\exp\left(-e^{a_t}+j\omega_t\right)\right), \\\\ B_t &= f_B(\mathbf{x}_t), \\\\ C_t &= f_C(\mathbf{x}_t). \end{aligned}

    With other experimental settings unchanged, this design also achieves competitive results across tasks. On MAD ICR and MAD Noisy ICR, it achieves 98.88% (-1.12 pp) and 99.22% (-0.77 pp), respectively, and reaches 100% (+1.60 pp) word accuracy on STORE--RECALL. On GSC, SHD, SSC, and DVS128 Gesture, it achieves 90.59% (+0.03 pp), 89.05% (+0.75 pp), 73.22% (+2.24 pp), and 91.67% (-2.08 pp), respectively. The values in parentheses indicate the differences in accuracy relative to the original TV-MCN with input-dependent Δt\Delta_t, BtB_t, and CtC_t.

    These results support our central claim that time-varying neuronal dynamics can be effective beyond the specific parameterization adopted in Selective SSM. We thank the reviewer for raising this important question and for the opportunity to clarify the motivation and contributions of our work.

  • Ablation with a learnable static Δ\Delta:

    We thank the reviewer for this suggestion. We conducted an additional ablation experiment with a learnable static Δ\Delta to distinguish the effect of learning an internal timescale from that of dynamically adjusting it based on the input. The results are shown below:

    Model VariantToken Acc. (%)Exact Acc. (%)
    TV-MCN93.4637.72
    w/o internal time modulation (Δ=1\Delta=1)7.200.00
    w/ learnable static Δ\Delta37.850.00
    w/o dynamic B/C coupling61.240.02

    When the internal timescale is fixed at Δ=1\Delta=1, Token Accuracy drops to 7.20%. Allowing Δ\Delta to be learned as a static parameter improves accuracy to 37.85%, but this remains substantially below the 93.46% achieved by the full TV-MCN. These results indicate that learning an appropriate timescale is beneficial in itself, while dynamically adjusting the timescale according to the current input provides substantially greater improvements.

AuthorsOpenReview ↗

3. Clarification of the Reset Mechanism and Additional Results on Training Variability

  • Clarification of the reset mechanism and persistent complex-valued states:

    We thank the reviewer for pointing out that the reset mechanism was not described clearly enough. We provide further clarification below.

    In TV-MCN, the complex-valued multi-compartment state hth_t evolves continuously and is read out through CtC_t to provide input to the soma. The soma accumulates this input in a separate membrane potential state and performs a soft reset after spike generation:

    ut=Re⁡(Ctht),vt−=vt−1++ut,st=H(vt−−ϑ),vt+=vt−−ϑst.\begin{aligned} u_t &= \operatorname{Re}(C_t h_t),\\\\ v_t^- &= v_{t-1}^+ + u_t,\\\\ s_t &= H(v_t^- - \vartheta),\\\\ v_t^+ &= v_t^- - \vartheta s_t. \end{aligned}

    Importantly, the reset only affects the somatic membrane potential and does not alter the complex-valued multi-compartment state. Therefore, the reset mapping for the complex-valued state is Rh(ht,st)=htR_h(h_t,s_t)=h_t. At the next time step, hth_t continues to evolve and is read out through the updated Ct+1C_{t+1}.

  • Variability across independent training runs:

    We thank the reviewer for this suggestion. To assess the stability of our results, we conducted 3 independent training runs for the main experiments, each with a different random seed, and evaluated the test results across runs. TV-MCN achieved 89.70±0.7489.70 \pm 0.74\\% on GSC, 70.36±0.6470.36 \pm 0.64\\% on SSC, 87.16±1.1187.16 \pm 1.11\\% on SHD, 89.93±3.6689.93 \pm 3.66\\% on DVS128 Gesture, (0.0022±0.0006)×10−1(0.0022 \pm 0.0006)\times 10^{-1} MSE on Adding, 90.67±3.3090.67 \pm 3.30\\% on Selective Copying, 99.99±0.0199.99 \pm 0.01\\% on MAD ICR, 99.98±0.0199.98 \pm 0.01\\% on MAD Noisy ICR, 99.92±0.0599.92 \pm 0.05\\% on STORE–RECALL, and 94.27±8.0294.27 \pm 8.02\\% on Induction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.