- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper is quite difficult for me to follow; it took a long time to decipher what the authors were actually doing. As I understand it, the authors aim to address the conversion of Artificial Neural Networks (ANNs) to Spiking Neural Networks (SNNs) for processing time-series datasets, proposing a conversion model that they claim offers better equivalence. They argue that the SNN output tracks the ANN output, and their analysis touches upon certain "attractor" properties. They validate their approach on time-series datasets, demonstrating its effectiveness.
Note after reply: The authors' response has addressed my concerns, and I am willing to increase my score by one point in recognition of their efforts to explore this relatively underexplored research direction.
Critical strengths
- The shift toward applying SNNs to time-series data is a significant trend in the field, and I strongly support this direction.
- Investigating SNN conversion within the context of Recurrent Neural Networks (RNNs) is also highly relevant.
- Instead of relying on the traditional Integrate-and-Fire (IF) neuron model, the authors utilize Leaky Integrate-and-Fire (LIF) neurons.
Critical weaknesses
- A major technical limitation is the reliance on a setup where each input frame corresponds to multiple time steps; this forces the SNN to work significantly harder to keep pace with the input frequency.
- If the paper follows the standard conventions for ANN-to-SNN conversion literature, it remains remarkably obscure.
- There is a lack of experimental data.
Questions for authors
- Regarding experimental data: Why are the results for the original ANNs (prior to conversion) not reported in Table 1? Additionally, what was the rationale behind the selection of specific neurons and parameters?
- Regarding line 43: From the perspective of ANN-to-SNN conversion literature, the conclusion that the SNN output tracks the ANN output is not particularly surprising. How do the authors view this?
- The term "streaming SNN inference framework" is confusing. It should be clarified in the abstract and other prominent sections. I interpret "streaming" as referring to the processing of time-series data, "inference" as implying a conversion-based approach, and "framework" as suggesting a configurable system—which implies that the experimental setup and parameters should be explicitly detailed.
- Regarding line 58: The phrase "temporally correlated input streams" is unnecessarily convoluted.
- Upon my first reading, the statement "SNNs cost few ticks per frame on average to maintain high frame-wise accuracy" was unclear—what exactly is a "tick"?
- From line 159 onwards, why are coefficients like referred to as "step size"?
- Regarding line 215: Does the network's outer loop require a reset at the beginning of each iteration? 8. Looking further into your work, it seems to me that the "transient dynamics" you analyze might actually be the result of smoothing the neuronal activity (i.e., the presence or absence of spikes) over time. Is my understanding correct?
- A practical challenge arises with tasks like "streaming object detection," which require processing 30 FPS input. Given that your model—including the time spent on the inner loop—would have a high computational latency, what kind of hardware do you envision being used to run it?
3 comments
Response to Reviewer DCjY (2/2)
Q5. What "tick" means in "SNNs cost few ticks per frame on average to maintain high frame-wise accuracy"?
One discrete SNN time step. In one tick every neuron updates its membrane once and may emit one spike. It is now defined at first use.
Q6. From line 159, why are coefficients like referred to as "step size"?
Because Lemma 2.1 is a relaxed fixed-point iteration: . Its coefficient sets how far the output moves toward its current target per tick, and the same coefficient is called the step size in stochastic approximation (Robbins & Monro, 1951). We added this explanation after Lemma 2.1.
Q7. In line 215, does the outer loop require a reset at each iteration? Are the "transient dynamics" actually the result of smoothing the neuronal activity over time?
The outer loop never resets. The state is initialized once per stream and kept to the end (Alg. 1). The transient dynamics are a continuous pursuit of the ANN output, which is more than smoothing. Lemma 2.1 shows that the SNN acts as a damped (smoothing) fixed-point iteration. Importantly, what the theorem adds is the direction: at every tick the smoothed output moves toward the ANN output of the current input, at rate . Smoothing alone would not say where the output goes.
Q8. Streaming object detection requires 30 FPS. Given the computational latency including the inner loop, what hardware do you envision?
We envision digital neuromorphic chips with LIF and soft reset, such as Loihi 2 and SpiNNaker 2. No special neuron or message format is needed. At 30 FPS and 8 ticks per frame each tick may take up to about 4 ms. SpiNNaker, for example, runs a 1 ms time step in its real-time mode (Furber et al., 2014), so the tick budget itself is not the bottleneck. In contrast, a 64-tick budget, typical of reset-based conversions, leaves 0.5 ms per tick. Whether a detector of this size fits on one chip depends on its capacity, which we did not evaluate. At 8 ticks per frame we reach 21.5 AP50 on Argoverse-HD while per-frame-reset conversions are near 0 AP50. Hardware deployment is future work and stated as a limitation.
Gentle reminder
Dear Reviewer DCjY, thank you again for your review and your support of this direction. As the discussion period ends in a few days, we would like to check whether our response answers your questions, in particular the source ANN accuracies now in Table 1 and the clarified terms in the abstract and introduction. We would be glad to clarify anything that remains.
Response to Reviewer DCjY (1/2)
Reducing ticks per frame is exactly what LIF-Stream improves over prior conversion methods, and the cost per frame drops as the input frequency rises. Prior conversions reset every frame and wait many ticks for convergence. LIF-Stream keeps its state across frames and pays only for the change between frames (Thm. 2.2). Most conversion baselines need 32 to 64 ticks or more per frame, while LIF-Stream is already usable at 4–8 (Table 1). Empirically, total ticks per second grow with frame rate f as O(f0.56) for LIF-Stream versus O(f) for reset-based SNNs (Fig. 4), because consecutive frames become more similar at higher rates.
Our results on transient dynamics and tracking are stronger than the steady-state result of conversion. Conversion theory (Rueckauer et al.) says where an SNN ends up for a fixed input. We show that at every tick and from any state the SNN moves toward the ANN output of the current input with step size 1/τ, and its tracking error stays bounded on a moving stream (Cor. 2.4., and our reply to Reviewer yzoD - W2). Steady-state conversion follows from this as a corollary (Cor. 2.3), but steady-state equivalence alone does not imply tracking. The IF neuron satisfies it exactly (App. C.5) yet cannot track a changing input in general (Fig. 1(d) and General response A). So whether tracking holds depends on the neuron. Tracking is what allows keeping the state across frames and paying only for the change, while prior IF conversions reset every frame.
Table 1 now has an ANN row per task. These are 60.7 AP50 on COCO (32.2 AP50 zero-shot on Argoverse-HD) and 7.62% WER on LibriSpeech and 91.1% on DVS-Gesture. They were only in the text before. Beyond Table 1, the revision adds decoder and membrane-bound controls, synaptic operations at matched accuracy, a τ sweep, and scene cuts, frame drops and noise (App. I.1 and I.5–I.7).
On neurons and parameters (App. E), each choice follows from the theory and none was tuned per task. We use LIF because a constant step size is what keeps tracking alive on a long stream (Thm. 2.2, Cor. 2.4., and our reply to Reviewer yzoD - W2). θ=1 is fixed by the range of ReLU1, so no threshold is fitted from data, unlike the calibration-based baselines. urest=0 is standard. We use τ=8 for all tasks and for budgets up to 64 ticks. It was fixed in advance, independently of the evaluation sets. The new τ sweep (App. I.6) confirms it is the best choice on Argoverse-HD (18.4 AP50 at T=8, against 14.0 at τ=16 and 3.2 at τ=4).
The abstract and introduction now clarify it. Streaming means inputs arrive as an unbounded sequence of frames and each frame needs a prediction. Inference implies the SNN is converted from a trained ANN with no SNN training. Framework means a general recipe that any frame-wise ReLU1 ANN can be plugged into. Its configurable parts are the leak τ, the ticks per frame T, the input encoder (App. D.1 compares several), and an optional warm-up for the first frame.
Agreed. It now reads "streams where consecutive frames are similar".