acceptodds
Under review as a conference paper at ICLR 2027

Spiking Neural Networks Incrementally Compute Continuous-valued Neural Networks through Attractor Dynamics

Abstract

Understanding how spiking neural networks (SNN) efficiently process data streams is a key to (i) study how biological brains conserve energy for lifelong predictions and (ii) enable an efficient continuous AI inference. In this work, we propose a theory on how an SNN incrementally computes its corresponding continuous-valued neural network (ANN), namely by reusing prior activation and updating only the temporal difference between ANN and the SNN. Specifically, we prove that discrete SNN output continuously attracts toward the instantaneous ANN output implicitly determined by the SNN input. Based on this principle, we develop LIF-Stream, a efficient streaming SNN inference framework which controls the SNN to continuously adapt to the reference ANN output which computes the high-accuracy frame-wise prediction online. Evaluation on three large-scale streaming inference benchmarks with distinct modalities show that LIF-Stream achieves the best efficiency-accuracy trade-off compared to high-accuracy SNN techniques.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spiking Neural Network, Stream Processing, ANN-to-SNN Conversion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DCjYOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper is quite difficult for me to follow; it took a long time to decipher what the authors were actually doing. As I understand it, the authors aim to address the conversion of Artificial Neural Networks (ANNs) to Spiking Neural Networks (SNNs) for processing time-series datasets, proposing a conversion model that they claim offers better equivalence. They argue that the SNN output tracks the ANN output, and their analysis touches upon certain "attractor" properties. They validate their approach on time-series datasets, demonstrating its effectiveness.

Note after reply: The authors' response has addressed my concerns, and I am willing to increase my score by one point in recognition of their efforts to explore this relatively underexplored research direction.

Critical strengths
  1. The shift toward applying SNNs to time-series data is a significant trend in the field, and I strongly support this direction.
  2. Investigating SNN conversion within the context of Recurrent Neural Networks (RNNs) is also highly relevant.
  3. Instead of relying on the traditional Integrate-and-Fire (IF) neuron model, the authors utilize Leaky Integrate-and-Fire (LIF) neurons.
Critical weaknesses
  1. A major technical limitation is the reliance on a setup where each input frame corresponds to multiple time steps; this forces the SNN to work significantly harder to keep pace with the input frequency.
  2. If the paper follows the standard conventions for ANN-to-SNN conversion literature, it remains remarkably obscure.
  3. There is a lack of experimental data.
Questions for authors
  1. Regarding experimental data: Why are the results for the original ANNs (prior to conversion) not reported in Table 1? Additionally, what was the rationale behind the selection of specific neurons and parameters?
  2. Regarding line 43: From the perspective of ANN-to-SNN conversion literature, the conclusion that the SNN output tracks the ANN output is not particularly surprising. How do the authors view this?
  3. The term "streaming SNN inference framework" is confusing. It should be clarified in the abstract and other prominent sections. I interpret "streaming" as referring to the processing of time-series data, "inference" as implying a conversion-based approach, and "framework" as suggesting a configurable system—which implies that the experimental setup and parameters should be explicitly detailed.
  4. Regarding line 58: The phrase "temporally correlated input streams" is unnecessarily convoluted.
  5. Upon my first reading, the statement "SNNs cost few ticks per frame on average to maintain high frame-wise accuracy" was unclear—what exactly is a "tick"?
  6. From line 159 onwards, why are coefficients like 1/τ1/\tau referred to as "step size"?
  7. Regarding line 215: Does the network's outer loop require a reset at the beginning of each iteration? 8. Looking further into your work, it seems to me that the "transient dynamics" you analyze might actually be the result of smoothing the neuronal activity (i.e., the presence or absence of spikes) over time. Is my understanding correct?
  8. A practical challenge arises with tasks like "streaming object detection," which require processing 30 FPS input. Given that your model—including the time spent on the inner loop—would have a high computational latency, what kind of hardware do you envision being used to run it?
3 comments
AuthorsOpenReview ↗

Response to Reviewer DCjY (1/2)

W1. Each input frame corresponds to multiple time steps; this forces the SNN to work significantly harder to keep pace with the input frequency.

Reducing ticks per frame is exactly what LIF-Stream improves over prior conversion methods, and the cost per frame drops as the input frequency rises. Prior conversions reset every frame and wait many ticks for convergence. LIF-Stream keeps its state across frames and pays only for the change between frames (Thm. 2.2). Most conversion baselines need 32 to 64 ticks or more per frame, while LIF-Stream is already usable at 4–8 (Table 1). Empirically, total ticks per second grow with frame rate ff as O(f0.56)O(f^{0.56}) for LIF-Stream versus O(f)O(f) for reset-based SNNs (Fig. 4), because consecutive frames become more similar at higher rates.

W2, Q2. Positioning over ANN-to-SNN conversion literature.

Our results on transient dynamics and tracking are stronger than the steady-state result of conversion. Conversion theory (Rueckauer et al.) says where an SNN ends up for a fixed input. We show that at every tick and from any state the SNN moves toward the ANN output of the current input with step size 1/τ1/\tau, and its tracking error stays bounded on a moving stream (Cor. 2.4., and our reply to Reviewer yzoD - W2). Steady-state conversion follows from this as a corollary (Cor. 2.3), but steady-state equivalence alone does not imply tracking. The IF neuron satisfies it exactly (App. C.5) yet cannot track a changing input in general (Fig. 1(d) and General response A). So whether tracking holds depends on the neuron. Tracking is what allows keeping the state across frames and paying only for the change, while prior IF conversions reset every frame.

W3. There is a lack of experimental data.

Q1. Why are the results for the original ANNs not reported in Table 1? What was the rationale behind the selection of neurons and parameters?

Table 1 now has an ANN row per task. These are 60.7 AP50 on COCO (32.2 AP50 zero-shot on Argoverse-HD) and 7.62% WER on LibriSpeech and 91.1% on DVS-Gesture. They were only in the text before. Beyond Table 1, the revision adds decoder and membrane-bound controls, synaptic operations at matched accuracy, a τ\tau sweep, and scene cuts, frame drops and noise (App. I.1 and I.5–I.7).

On neurons and parameters (App. E), each choice follows from the theory and none was tuned per task. We use LIF because a constant step size is what keeps tracking alive on a long stream (Thm. 2.2, Cor. 2.4., and our reply to Reviewer yzoD - W2). θ=1\theta=1 is fixed by the range of ReLU1, so no threshold is fitted from data, unlike the calibration-based baselines. urest=0u_{\text{rest}}=0 is standard. We use τ=8\tau=8 for all tasks and for budgets up to 64 ticks. It was fixed in advance, independently of the evaluation sets. The new τ\tau sweep (App. I.6) confirms it is the best choice on Argoverse-HD (18.4 AP50 at T=8T=8, against 14.0 at τ=16\tau=16 and 3.2 at τ=4\tau=4).

Q3. The term "streaming SNN inference framework" is confusing and should be clarified in the abstract.

The abstract and introduction now clarify it. Streaming means inputs arrive as an unbounded sequence of frames and each frame needs a prediction. Inference implies the SNN is converted from a trained ANN with no SNN training. Framework means a general recipe that any frame-wise ReLU1 ANN can be plugged into. Its configurable parts are the leak τ\tau, the ticks per frame TT, the input encoder (App. D.1 compares several), and an optional warm-up for the first frame.

Q4. In line 58, "temporally correlated input streams" is unnecessarily convoluted.

Agreed. It now reads "streams where consecutive frames are similar".

AuthorsOpenReview ↗

Response to Reviewer DCjY (2/2)

Q5. What "tick" means in "SNNs cost few ticks per frame on average to maintain high frame-wise accuracy"?

One discrete SNN time step. In one tick every neuron updates its membrane once and may emit one spike. It is now defined at first use.

Q6. From line 159, why are coefficients like 1/τ1/\tau referred to as "step size"?

Because Lemma 2.1 is a relaxed fixed-point iteration: y(t)=y(t−1)+1τ(H(⋅)−y(t−1))y(t) = y(t-1) + \frac{1}{\tau}\big(\mathbb{H}(\cdot) - y(t-1)\big). Its coefficient sets how far the output moves toward its current target per tick, and the same coefficient is called the step size in stochastic approximation (Robbins & Monro, 1951). We added this explanation after Lemma 2.1.

Q7. In line 215, does the outer loop require a reset at each iteration? Are the "transient dynamics" actually the result of smoothing the neuronal activity over time?

The outer loop never resets. The state is initialized once per stream and kept to the end (Alg. 1). The transient dynamics are a continuous pursuit of the ANN output, which is more than smoothing. Lemma 2.1 shows that the SNN acts as a damped (smoothing) fixed-point iteration. Importantly, what the theorem adds is the direction: at every tick the smoothed output moves toward the ANN output of the current input, at rate 1/τ1/\tau. Smoothing alone would not say where the output goes.

Q8. Streaming object detection requires 30 FPS. Given the computational latency including the inner loop, what hardware do you envision?

We envision digital neuromorphic chips with LIF and soft reset, such as Loihi 2 and SpiNNaker 2. No special neuron or message format is needed. At 30 FPS and 8 ticks per frame each tick may take up to about 4 ms. SpiNNaker, for example, runs a 1 ms time step in its real-time mode (Furber et al., 2014), so the tick budget itself is not the bottleneck. In contrast, a 64-tick budget, typical of reset-based conversions, leaves 0.5 ms per tick. Whether a detector of this size fits on one chip depends on its capacity, which we did not evaluate. At 8 ticks per frame we reach 21.5 AP50 on Argoverse-HD while per-frame-reset conversions are near 0 AP50. Hardware deployment is future work and stated as a limitation.

AuthorsOpenReview ↗

Gentle reminder

Dear Reviewer DCjY, thank you again for your review and your support of this direction. As the discussion period ends in a few days, we would like to check whether our response answers your questions, in particular the source ANN accuracies now in Table 1 and the clarified terms in the abstract and introduction. We would be glad to clarify anything that remains.

Reviewer_yzoDOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the transient behavior of SNNs and relates LIF dynamics to the fixed points of continuous-valued ANNs. Building on this connection, the authors introduce LIF-Stream, an ANN-to-SNN conversion method that preserves neuronal states across consecutive frames to track changing ANN outputs. The method is evaluated on streaming object detection, speech recognition, and event-based gesture recognition, showing promising accuracy-efficiency trade-offs.

Critical strengths
  1. The theoretical perspective of this paper is interesting. Viewing LIF dynamics as a process that tracks changing ANN attractors offers a useful way to understand how SNNs operate on streaming inputs.

  2. The method is simple and practical. Reusing neuronal states makes good use of temporal redundancy, without additional SNN training or data-driven calibration. The discussion of why LIF behaves differently from IF is also helpful.

  3. The experiments cover three different modalities and include a reasonable range of baselines. In particular, the state-reset ablations help support the argument that preserving neuronal states is beneficial for streaming inference.

Critical weaknesses
  1. The novelty compared with existing incremental computing methods is not entirely clear. Previous work, including Sigma-Delta Networks, Delta Networks, and streaming ANN-to-SNN conversion, has already explored temporal difference computation and the relationship between LIF dynamics and continuous-valued representations. While the attractor-based interpretation is interesting, I am not fully convinced that the paper establishes a sufficiently distinct theoretical or computational contribution beyond these existing approaches.

  2. The theoretical results do not seem strong enough to fully support the claims. Theorem 2.2 mainly describes how the updates move toward instantaneous attractors, but does not guarantee accurate tracking when inputs change over time. Stronger convergence results rely on additional assumptions, such as contractivity and input convergence. It would be helpful if the authors could clarify how these results translate to reliable long-term streaming inference.

  3. It is unclear whether the advantages of LIF over IF actually come from the leakage mechanism. The theoretical comparison uses EMA coding for LIF but rate coding for IF, making it difficult to separate the effects of neuronal dynamics from those of the decoding strategy. Comparing IF with EMA or sliding-window decoding would help clarify whether leakage itself is responsible for the improved tracking performance.

  4. The practical efficiency improvements need more convincing evidence. Most of the reported gains are measured in simulation ticks rather than actual inference time or energy consumption. In addition, the adaptive tick requirements in Figures 3–4 are estimated after inference, while the implemented method still uses a fixed tick budget for each frame. Actual latency, computational overhead, and hardware energy measurements would provide a clearer picture of the efficiency benefits.

  5. The applicability of the method and the experimental comparisons are limited. The framework relies on specially trained ReLU1-based networks with restricted operators, which makes it difficult to apply directly to general pretrained ANN models. Also, most conversion baselines reset their states between frames. Comparisons with existing stateful or incremental inference methods would be important for understanding how much of the improvement comes specifically from LIF-Stream.

Questions for authors

See weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer yzoD

W1. The novelty compared with existing incremental computing methods (Sigma-Delta Networks, Delta Networks, streaming ANN-to-SNN conversion) is not entirely clear. It is not fully convincing how the paper establishes a sufficiently distinct theoretical or computational contribution beyond these approaches.

See General response (B).

W2. Theorem 2.2 does not guarantee accurate tracking when inputs change over time, and stronger convergence results rely on contractivity and input convergence. How do these results translate to reliable long-term streaming inference?

We now state a tracking bound for moving inputs that needs no input convergence (new Cor. 2.4 in the main text):

If γ<1\gamma<1 and the target moves at most δ\delta per tick, ∥yt∗−yt−1∗∥∞≤δ\Vert y_t^{\ast} - y_{t-1}^{\ast}\Vert_\infty \le \delta, then from any state the error after kk ticks is at most R∗+ρkR^{\ast} + \rho^k, with ρ=max⁡{α,γ}\rho=\max\lbrace \alpha,\gamma\rbrace and R∗=max⁡{1/τ+γδ1−γ, (τ−1)δ}.R^{\ast} = \max\Big\lbrace \frac{1/\tau + \gamma\delta}{1-\gamma},\ (\tau-1)\delta\Big\rbrace.

The first term is the 1/τ1/\tau jitter of Cor. 2.3. The second is the lag that any exponential average pays when its target moves δ\delta per tick. Neither grows with time, so tracking holds for the whole stream. δ\delta itself scales with the frame-to-frame change of the input, which matches Fig. 3: frames with small ANN drift settle in fewer ticks. On 120 random LIF networks simulated spike by spike, the bound held with a largest ratio of 0.99, so it is nearly tight. For feedforward networks the contraction condition can be dropped (new Remark C.3). For IF see General response (A).

W3. The theory uses EMA coding for LIF but rate coding for IF, making it difficult to separate neuronal dynamics from decoding. Comparing IF with EMA or sliding-window decoding would clarify whether leakage itself is responsible.

See General response (A). It gives a result that holds for any decoder and adds IF with EMA and sliding-window decoders in Table R1 (rows 2–3), App. I.1, and the updated Fig. 6.

W4. Most gains are measured in simulation ticks rather than inference time or energy. The adaptive tick requirements in Figures 3–4 are estimated after inference, while the method uses a fixed tick budget per frame.

Every number in Table 1 uses a fixed per-frame tick budget, and the tick savings already show in such setup. At 8 ticks per frame LIF-Stream reaches 21.5 AP50 on Argoverse-HD while per-frame-reset conversions are near 0 AP50. Experiments in Figs. 3–4 are designed to show why: most frames need only a few ticks once the state is reused. An adaptive tick rule is a natural next step that we do not claim.

For cost beyond ticks we now report synaptic operations and a standard energy estimate per frame (App. I.5). At matched or higher accuracy on Argoverse-HD, LIF-Stream uses 6.1x less estimated energy than AdaFire, the strongest baseline (T=16 vs. T=64), 6.4x less than QCFS (T=16 vs. T=128), 6.6x less than SlipReLU (T=16 vs. T=128), and 2.1x less than the same LIF network with a per-frame reset (T=32 vs. T=64). Wall-clock time on a GPU does not reflect SNN cost, and we have no hardware energy measurements. This remains a stated limitation.

W5. The framework relies on specially trained ReLU1-based networks with restricted operators, which makes it difficult to apply directly to general pretrained ANN models.

ReLU1 comes from using the standard LIF neuron and not from our theory. Neurons that can approximate general multivariate nonlinearities with a constant step size would remove the ReLU1 restriction. The sign-gradient neurons of Oh et al. (2024) with constant step size are one example. The cost is the implementation of custom neuronal dynamics on hardware. We describe this trade-off in the Discussion. Even within ReLU1, modern architectures remain usable, as our three backbones show: a YOLOv8-style detector, a SummaryMixing speech recognizer, and a TCN for gestures.

W6. Most conversion baselines reset their states between frames. Comparisons with existing stateful or incremental inference methods would be important for understanding how much of the improvement comes specifically from LIF-Stream.

Table R1 now compares against the stateful binary-spike methods that apply to a frame-wise pretrained ANN without retraining. Table R1 adds IF conversion without reset under an EMA or sliding-window decoder. App. I.1 extends this to four IF conversions and adds the mechanism of Kugele et al. (2020), IF with no reset and a membrane lower bound, on the plain IF conversion they use. Kugele et al. itself needs a multi-frame ANN trained with BPTT, so we port its mechanism rather than its model. Sigma-delta and delta networks send graded, multi-bit messages, which is a different hardware setting from the binary-spike SNNs of Table 1 (General response B).

AuthorsOpenReview ↗

Gentle reminder

Dear Reviewer yzoD, thank you again for your review. As the discussion period ends in a few days, we would like to check whether our response addresses your concerns. In particular, the new Cor. 2.4 bounds the tracking error on a moving stream without input convergence, and General response (A) separates leak from decoding with a decoder-independent result and IF decoder control experiments (Table R1, App. I.1). If anything remains unclear, we would be glad to clarify before the deadline. If our response resolves your concerns, we would appreciate it if you would consider updating your evaluation.

Thank you for the detailed rebuttal and additional experiments, which have addressed several of my concerns. However, I remain concerned about the limited generality of the method due to its reliance on ReLU1-based ANNs, as well as the lack of actual hardware evaluation. I also think the claim of having implemented an end-to-end SNN sensing system on a neuromorphic chip in Appendix A (p. 18) is inappropriate without supporting hardware results. Overall, I appreciate the authors' efforts, but I will maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer yzoD

Thank you for your reply and for acknowledging that several of your concerns are addressed.

Appendix A. You are right. The sentence claiming an end-to-end implementation on a neuromorphic chip was incorrect, as we have no hardware results. We have removed it in the updated PDF, along with the statement that the gains "will readily translate" to silicon. App. A now states that the energy figures are estimates and that a chip demonstration is future work.

ReLU1. The restriction comes from using the standard LIF neuron, which keeps the method compatible with existing neuromorphic platforms without custom neurons. Within it, modern architectures remain usable, as our three backbones show: a YOLOv8-style detector, a SummaryMixing speech recognizer and a TCN. Lifting the restriction with constant-step-size neurons beyond the standard LIF is discussed in App. A as future work.

Hardware. We agree that a measurement on a chip would strengthen the efficiency claim. We follow the energy analysis convention of prior SNN literature (App. I.5) and state hardware evaluation as a limitation.

Reviewer_KsgBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LIF-Stream, a post-conversion framework designed for stream inference in SNNs. The authors theoretically demonstrate that an LIF-SNN can function as a damped fixed-point iteration that continuously tracks the attractor of a time-varying ANN. LIF-Stream preserves the membrane potential state across frames, eliminating the need for global resets and leveraging inter-frame temporal redundancy to reduce the number of simulation ticks per frame. Through evaluations on object detection, speech recognition, and DVS gesture recognition, the authors demonstrate the effectiveness of their method.

Critical strengths
  1. This paper compares the dynamics of discrete-time IF and LIF and formally derives their differences as fixed-point iterations.
  2. This paper conducts experiments on object detection, speech recognition, and event gestures to align with streaming inference scenarios.
Critical weaknesses
  1. According to Theorem 2.2, the LIF-SNN converges to an instantaneous time-varying attractor. However, this attractor depends on the EMA-filtered input, the network weights, and the residual term of the initial membrane potential. Therefore, when the initial conditions have not sufficiently decayed, the attractor is contaminated by the initial state. This paper does not analyze how residual initial conditions affect tracking accuracy over the long term.
  2. This paper attributes the shortcomings of the IF-SNN to its time-varying step size. It concludes that the IF-SNN must undergo a global reset to perform streaming inference. However, IF-SNNs are not limited to arithmetic mean (AM) rate coding and can easily use exponential moving average (EMA) coding instead. Nevertheless, the authors conducted their IF analysis using only AM rate coding and did not explore other decoding schemes. Consequently, their conclusions are scheme-dependent and do not represent an inherent, low-level limitation of IF neurons themselves.
  3. The existing literature includes IF-SNN streaming processing schemes that maintain local state and do not require a global reset. However, the main text inadequately discusses these works and portrays the claim that "IF must be reset frame by frame" as a universal truth.
  4. The set of DEQ fixed points corresponding to the LIF-SNN, denoted by E(t)\mathcal{E}(t) , is not equal to the output of the original, pre-trained ReLU1 ANN. The two only asymptotically converge under the assumptions of static input and a compressed mapping. Although the abstract and introduction of the main text state that "the SNN incrementally computes its corresponding continuous-valued neural network," there is actually a deviation introduced by the DEQ transformation.
  5. The core idea of the paper is to preserve the LIF membrane potential across frames and eliminate the global reset that occurs every frame, but this idea is not novel.
  6. Saving all membrane potentials continuously across frames results in significant storage overhead. However, the paper does not provide a quantitative assessment of this. Furthermore, the proposed method also entails hidden pre-charging overhead.
  7. The vast majority of the conversion baselines (ReLUNorm, QCFS, Spike-Calib) compared in the paper follow the settings outlined in their original papers and perform a global reset at every frame. However, LIF-Stream does not allow resets. The difference between the two is not merely a matter of neurons or conversion algorithms; rather, it is a completely different inference execution paradigm. Direct comparisons are therefore somewhat unfair. Furthermore, real-world streaming scenarios inevitably involve noise, frame corruption, and sudden cuts. This paper was tested exclusively on ideal, clean streams, and it lacks any discussion or experiments addressing these issues.
  8. This paper uses "tick/frame" as a measure of efficiency. However, a tick is not equivalent to a real hardware clock cycle.
Questions for authors

See Weakness.

5 comments
AuthorsOpenReview ↗

Response to Reviewer KsgB (1/2)

W1. The attractor depends on the EMA-filtered input, the network weights, and the residual initial membrane potential, so it is contaminated by the initial state until it decays. Its long-term effect on tracking accuracy is not analyzed.

The initial-condition term vanishes within the first few frames. The dependence on the input and the weights is intended, since the attractor is the ANN output. The only unwanted term is αtupre(0)\alpha^t u_{\text{pre}}(0) in b(t)b(t) (Lemma 2.1). With τ=8\tau=8 this term decays as 0.875t0.875^t and falls below 1% after 35 ticks. Fig. 1(c) shows this directly: LIF-SNN trajectories started from random initial states converge onto the same ANN trajectory in phase space once they receive the same inputs. We moved this from the appendix to the main text (after Cor. 2.3).

W2. IF-SNNs may easily use EMA coding instead of AM rate coding, but the IF analysis uses only AM rate coding.

See General response (A).

W3. The existing literature includes IF-SNN streaming schemes that maintain local state without a global reset, but the paper portrays "IF must be reset frame by frame" as a universal truth.

To our knowledge, the only IF-based conversion that keeps states across a stream without any reset is Kugele et al. (2020). General response (B) details how it differs from ours. In short, it uses state reuse only as a warm start, notes that this can slow convergence, and neither analyzes state reuse nor pitches it as a contribution. Diehl et al. (2016) reset between sentences, and Pérez-Carrasco et al. (2013) rely on a hand-tuned leak rather than IF. We now discuss all three in Related Work, and revised the claim in Sec. 3 to: "in practice the IF conversions we compare against reset every frame."

W4. The DEQ fixed points E(t)\mathcal{E}(t) are not equal to the output of the pre-trained ReLU1 ANN; the two only asymptotically converge under static input and a compressed mapping.

E(t)\mathcal{E}(t) is the pretrained ReLU1 ANN output for the EMA-filtered input x~(t)\tilde{x}(t). The asymptotic convergence concerns two different quantities, E∗\mathcal{E}^{\ast} and y(t)y(t).

  • E(t)\mathcal{E}(t): the ANN output for x~(t)\tilde{x}(t). Besides using x~(t)\tilde{x}(t) for x\mathbf{x}, its form differs from E∗\mathcal{E}^{\ast} only through the term αtupre(0)\alpha^t u_{\text{pre}}(0) in b(t)b(t) (with urest=0u_{\text{rest}}=0 as in our conversion). As in W1 this term decays exponentially and vanishes within the first few frames.
  • E∗\mathcal{E}^{\ast}: the ANN output for a fixed input x\mathbf{x}. E(t)\mathcal{E}(t) equals it only once x~(t)→x\tilde{x}(t)\to\mathbf{x}, which is the static-input condition of Cor. 2.3.
  • y(t)y(t): the SNN output. It tracks E(t)\mathcal{E}(t) within an O(1/τ)O(1/\tau) band and does not equal it exactly.

So the deviation lies between y(t)y(t) and the ANN output, not between E(t)\mathcal{E}(t) and the ANN. We now clarify the three quantities right after Cor. 2.3.

AuthorsOpenReview ↗

Response to Reviewer KsgB (2/2)

W5. Preserving the LIF membrane potential across frames and eliminating the per-frame global reset is not novel.

See General response (B).

W6. Saving all membrane potentials across frames results in significant storage overhead, which is not quantified. The method also entails hidden pre-charging overhead.

No extra state is stored, and the warm-up is a small one-time cost. A per-frame-reset SNN holds the same potentials during a frame; we only skip clearing them. The optional warm-up adds 4T4T ticks to the first frame only (e.g. 64 ticks at T=16T=16). Over an Argoverse-HD sequence of about 630 frames this is under 1% of all ticks. The conversion-time pre-charge (App. E) runs once offline: with firing disabled, a single forward pass sets the initial potentials.

W7. Most conversion baselines perform a global reset at every frame while LIF-Stream does not, so direct comparisons are somewhat unfair. The paper was tested exclusively on ideal, clean streams, without noise, frame corruption, or sudden cuts.

For a fair comparison, two control experiments separate the global reset from the method. The LIF-with-reset row in Table 1 uses our own network with only the reset added. Fig. 6 runs each published IF conversion without its reset. In both, removing the reset alone does not give the gain.

For robustness we conducted additional experiments with scene cuts, frame drops, and noise on Argoverse-HD. Argoverse-HD already consists of in-the-wild driving scenes with real sensor noise and fast motion, and Fig. 4 already widens the frame interval up to 10× by striding. Still, we streamed three sequences as one with no reset at the two cuts, dropped 30% of frames at random, and added Gaussian noise (App. I.7, T=8T=8). Across cuts the overall AP50 is unchanged (18.3 vs. 18.4). Right after a cut, starting from the previous scene's state is even slightly better than starting from scratch: AP50 over the first 10 frames of the new scene is 5.9 versus 4.3. Frame drops cost 15% at T=8T=8 and 6% at T=16T=16. Noise costs 7%, close to the 5% it costs the dense ANN.

W8. A tick is not equivalent to a real hardware clock cycle.

A tick is indeed not a clock cycle, but latency scales with ticks on any given chip. A tick is one SNN time step. Neuromorphic chips advance all neurons together one time step at a time, so the latency of a frame is roughly its number of ticks times that chip's time-step duration, which itself can grow with spike traffic. This is why ticks per frame is the standard hardware-agnostic unit in the conversion literature. Energy also depends on spike activity, so we now report synaptic operations per frame next to ticks (App. I.5). Actual measurement on a specific chip remains a stated limitation.

AuthorsOpenReview ↗

Gentle reminder

Dear Reviewer KsgB, thank you again for your review. As the discussion period ends in a few days, we would like to check whether our response addresses your concerns. In particular, General response (A) now shows that no decoder, including EMA, lets an IF neuron track a stream (App. C.5.4), with EMA and sliding-window IF control experiments in Table R1 and App. I.1. We also added experiments with scene cuts, frame drops and noise (App. I.7). If anything remains unclear, we would be glad to clarify before the deadline. If our response resolves your concerns, we would appreciate it if you would consider updating your evaluation.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

We uploaded a revised PDF, with changes marked in blue. We answer two concerns shared by several reviewers.

(A) No decoder lets an IF neuron track a stream. The obstacle is the charge debt that piles up in the IF membrane, and leak removes it (Reviewer KsgB - W2, Reviewer yzoD - W3).

What we claim. Tracking a stream means the output stays close to the ANN output of the recent input, with a lag that does not grow with how long the stream has run. LIF-SNNs have this property for any input (Cor. 2.4). IF-SNNs do not, whatever decoder reads their spikes: there are inputs for which every decoder fails for an arbitrarily long time.

Proof by counterexample (App. C.5.4). We demonstrate the IF neuron's infeasibility to continuously track the streaming ANN output with a counterexample using a single neuron. In short, negative input cannot leave an IF neuron as spikes, so it piles up as a debt and the neuron stays silent until new input repays it. In contrast, an LIF neuron's memory fades: it discounts old debt by α\alpha per tick, which leaves room to track the new ANN output.

  1. For input I≤θI\le\theta the IF spike count is exactly N(t)=max⁡k≤t⌊(u(0)+S(k))/θ⌋N(t)=\max_{k\le t}\lfloor (u(0)+S(k))/\theta\rfloor with S(k)=∑i≤kI(i)S(k)=\sum_{i\le k} I(i). It rises only when the cumulative input exceeds its earlier peak.
  2. Suppose we feed neuron A with I=−cI=-c for LL ticks and then I=aθI=a\theta, and neuron B with I=−cI=-c throughout. By step 1 both stay silent for the first T=L+⌊cL/(aθ)⌋T = L+\lfloor cL/(a\theta)\rfloor ticks, so their spike trains are identical for the first TT ticks.
  3. Since LL is arbitrary, we can make neuron A fail to track the new ReLU network output determined by aθa\theta for arbitrarily long, no matter what decoder is selected: neuron B, whose correct output stays 0, emits exactly the same (empty) spike train, so no decoder can tell A from B.
  4. Hence, no decoder gives IF a lag bound that holds over a stream. An LIF neuron in the same situation fires again within a time set by τ\tau and independent of LL (Remark C.11).

The obvious fix does not work. A bound on the membrane, as in Kugele et al., limits the debt but breaks the conversion. It discards charge whenever the input dips below it, so the firing rate is no longer ReLU1 of the mean input: input alternating between −θ-\theta and 1.5θ1.5\theta fires at rate 0.5 instead of 0.25 (Remark C.12).

We also validate this in a full network (Table R1, Fig. 6, App. I.1). Without reset, every IF conversion we tried (QCFS, SlipReLU, ReLU-Norm, AdaFire) stays below 1 AP50 at T=4T=4–1616 with every decoder, against 18.4 for LIF-Stream at T=8T=8.

(B) Difference from prior incremental and stateful SNN processing (Reviewer KsgB - [W3,W5], Reviewer yzoD - W1).

Sigma-delta / delta networksKugele et al. (2020)LIF-Stream (ours)
Source ANNtrained with delta quantization, or converted to dedicated neurons (SDNN)multi-frame ANN trained with BPTT (streaming rollout)any frame-wise ReLU1 ANN as is
Messagemulti-bit activation change (graded spike)binary spikebinary spike
Neurondedicated sigma and delta stagesIF with a lower bound on the membranestandard LIF with soft reset
Role of state kept across framesdelta encoding built in by designwarm start that can also slow convergence to the ANN activationsthe mechanism itself
Why state reuse worksby constructionnot analyzed, not claimed as a contributionThm. 2.2 plus fading memory from leak (A)

Compared with these works, our contribution is not a new neuron that computes deltas or a model built for stream processing. We reveal that a plain binary LIF neuron already computes incrementally, and explain why (Thm. 2.2).

Kugele et al. is the closest work on IF conversion, but it uses state reuse only as a side effect of streaming and does not exploit or analyze it. It also points out that the state carried across frames can slow convergence (Sec. 3.1). Ported to the plain IF conversion they use, the mechanism stays at 0 AP50 at 4–16 ticks per frame (App. I.1), where that conversion fails even with a per-frame reset. Kugele et al. use 15–60 steps per frame.

Table R1. Rows 2–3 answer (A). All rows are on Argoverse-HD, the setting of Fig. 6: sequences 5, 19, 10 (1,374 frames), so absolute values differ from the full-validation Table 1. IF rows use a QCFS-trained detector. App. I.1 repeats them for SlipReLU, ReLU-Norm and AdaFire, all below 1 AP50. The LIF rows use our ReLU1 detector, whose dense ANN reaches 32.5 AP50 on these frames. TT is ticks per frame.

MethodPer-frame resetAP50 TT=4AP50 TT=8AP50 TT=16
IF conversion (QCFS)yes0.71.74.6
IF conversion (QCFS), EMA decoderno0.60.70.7
IF conversion (QCFS), sliding-window decoderno0.60.60.7
LIF conversionyes0.00.00.0
LIF-Stream (ours)no12.118.422.9
AuthorsOpenReview ↗

Summary of contributions, responses, and revisions.

Dear Area Chairs and Reviewers,

Thank you for handling our submission, and we thank all reviewers for their constructive feedback. We provide a concise summary to assist your evaluation. All revisions are highlighted in blue in the revised PDF.

[Contributions]

  • Theory of streaming SNN dynamics. We prove that the output of an LIF-SNN tracks the instantaneous output of its corresponding ANN with a constant step size, and bound its tracking error on a moving stream (Thm. 2.2, Cor. 2.4). We also show why IF neurons cannot track a stream under any decoder (App. C.5.4). To our knowledge, prior stateful SNN conversions keep states without analyzing when this works.
  • A conversion framework that needs no training or calibration. Any frame-wise ReLU1 ANN is converted to standard LIF neurons that keep their state across frames. In contrast, high-accuracy conversion methods reset every frame and need calibration, custom neurons or multi-bit spikes.
  • Few ticks per frame across three modalities. At 8 ticks per frame, LIF-Stream reaches 21.5 AP50 on Argoverse-HD, where every reset-based conversion stays at or below 1.5, and 19.3% WER on LibriSpeech, against 42.7% for the best conversion baseline (Table 1).

[Strengths acknowledged by the reviewers]

  • yzoD: "The theoretical perspective of this paper is interesting … a useful way to understand how SNNs operate on streaming inputs"; the method is "simple and practical … without additional SNN training or data-driven calibration"; "the state-reset ablations help support the argument that preserving neuronal states is beneficial for streaming inference".
  • KsgB: the paper "compares the dynamics of discrete-time IF and LIF and formally derives their differences as fixed-point iterations", with experiments on detection, speech and gestures "to align with streaming inference scenarios".
  • DCjY: "I strongly support this direction" of applying SNNs to time-series data.

[Responses to the main concerns]

  • Reviewer KsgB. The main concern (W2) was that the IF limitation might stem from rate decoding. We prove that no decoder, including EMA, gives an IF neuron a lag bound over a stream (App. C.5.4). We also show that IF conversions streamed without reset stay near 0 AP50 with EMA and sliding-window decoders (Table R1, App. I.1). Even at 64 ticks per frame, where QCFS and AdaFire with reset reach 15.7 and 22.2 AP50 (App. I.5), the same networks streamed without reset stay near 0, and so does QCFS with EMA or sliding-window decoders (Fig. 6). We also addressed W1/W4 (initial state and DEQ fixed points, after Cor. 2.3), W3 (related work), W6 (no extra state; warm-up under 1% of ticks) and W7 (robustness, App. I.7).
  • Reviewer yzoD. For the main concern (W2), we added Cor. 2.4, a tracking-error bound on a moving stream that holds from any state without input convergence, checked spike by spike on 120 random networks (largest ratio 0.99). W3 is answered as for KsgB-W2, and W1 by General response (B).
  • Reviewer DCjY. We added the source ANN accuracy of every conversion to Table 1, defined "tick" and the framework, explained the step size, and added a τ sweep (App. I.6).

[Revisions and additional experiments]

  • As KsgB-W2, yzoD-W3 and yzoD-W6 suggested, we added IF baselines with EMA and sliding-window decoders for four IF conversions (Table R1, App. I.1), and a charge-debt analysis showing why leak, not decoding, enables tracking (App. C.5.4).
  • As yzoD-W2 suggested, we added the tracking bound for moving inputs (Cor. 2.4, App. C.4) and its extension to feedforward networks with a depth-dependent constant (Remark C.3).
  • As KsgB-W3, W5 and yzoD-W1 suggested, we added Kugele et al. (2020), Diehl et al. (2016) and Pérez-Carrasco et al. (2013) to Related Work, contrasted our work with them and with sigma-delta networks (General response B), and narrowed the IF reset claim in Sec. 3.
  • As KsgB-W7 suggested, we added experiments with scene cuts, frame drops and noise. AP50 across scene cuts is unchanged (18.3 vs 18.4), and frame drops and noise reduce it by at most 2.7 and 1.7 points (App. I.7).
  • As KsgB-W8 and yzoD-W4 suggested, we report synaptic operations and estimated energy per frame. At matched or higher accuracy, LIF-Stream uses 6.1–6.6× less estimated energy than AdaFire, QCFS and SlipReLU (App. I.5).
  • As DCjY-Q1 suggested, we added the source ANN accuracies to Table 1 and a τ sweep (App. I.6). As DCjY-Q3 to Q5 suggested, we revised the abstract and introduction, and as Q6 suggested, we explain the step size after Lemma 2.1.
  • As yzoD-W5 suggested, we discuss how neurons with a constant step size beyond the standard LIF could lift the ReLU1 restriction (App. A).

We are grateful for the reviewers' suggestions, which made both the theory and the evaluation more complete.

Regards, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.