acceptodds
Under review as a conference paper at ICLR 2027

PicardSNN: Breaking the Sequential Barrier of Spiking Neural Networks with Picard-Parallel-Training

Abstract

Many important intelligence problems are inherently spatiotemporal: their solutions depend on how signals evolve and retain information over time, as in neuromorphic perception, event-based processing, and long-context sequence modeling. Such temporal dynamics require recurrent state updates, which purely feedforward networks cannot represent. Spiking Neural Networks (SNNs) are an attractive solution, combining nonlinear recurrent state dynamics, sparse event-driven communication, and energy-efficient inference. Their training, however, is hard: backpropagation through time (BPTT) unrolls the recurrence over T steps, resulting in inherently sequential computation over long temporal dependencies, limiting scalability and training efficiency on long sequences. We introduce Picard-Parallel-Training (PPT), a parallel formulation of temporal training for SNNs, and use it to train PicardSNN. Instead of solving for the membrane potential, whose threshold-and-reset makes the recurrence a hard hybrid system, PPT solves for the pre-firing membrane potential, turning the dynamics into a linear recurrence with an explicit spike forcing term. PPT retains subtractive reset and isolates the spike-dependent nonlinearity as an explicit forcing term. Each Picard sweep freezes this term using the preceding iterate’s spikes and solves the resulting affine recurrence through a logarithmic-depth parallel scan. A fused CUDA implementation combines forcing assembly and scan execution without materialising full-length intermediate forcing tensors. The reformulation is algebraically exact, while finite-sweep evaluation approximates the sequential trajectory unless a fixed point is reached. PPT supports sequences of up to 65,535 time steps and achieves up to 763× forward and 348× end-to-end speed-ups over sequential BPTT, with finite-sweep spike agreement and activation sparsity evaluated separately. Across four task families—neuromorphic audio, event-based vision, pixel-serial images, and the Long Range Arena—PicardSNN substantially outperforms Transformer baselines, remains competitive with specialised State-Space Models, and ranks among the strongest recurrent and spiking neural network methods.

Primary area:
learning on time series and dynamical systems
Keywords:
Spiking Neural Networks, Parallel Training, Picard Iteration, Long-Range Sequence Modeling, Neuromorphic Computing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_APDXOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Picard-Parallel-Training (PPT) to address the sequential computation bottleneck in training spiking neural networks. By reformulating the membrane dynamics using pre-threshold potentials, the authors enable parallel evaluation through Picard iterations and affine scans. A fused CUDA implementation further improves computational efficiency. The reported speedups are substantial, and PicardSNN achieves competitive results across neuromorphic and long-sequence benchmarks.

Critical strengths
  1. The paper addresses an important challenge in SNN training. The pre-threshold reformulation is an interesting approach to enabling parallel computation while retaining the original spiking dynamics.
  2. The combination of theoretical analysis and CUDA implementation is a notable strength. The reported acceleration is impressive, and the experiments cover a diverse range of sequence-learning tasks.
Critical weaknesses
  1. The gap between theoretical convergence and practical approximation deserves further clarification. The convergence guarantee applies to a smoothed threshold under specific conditions, whereas the implementation uses hard thresholds with a fixed number of Picard iterations. With four iterations, spike agreement is approximately 85–90%. Although the authors report strong downstream accuracy, it remains unclear how these trajectory differences affect training gradients and convergence.
  2. The reported speedups would benefit from stronger baseline comparisons. The substantial acceleration over sequential BPTT is encouraging, but a comparison with an optimized sequential implementation or existing parallel SNN training methods would help distinguish the benefits of the proposed formulation from those of CUDA kernel optimization.
  3. The general applicability of PPT remains somewhat uncertain. The main experiments focus on neuron-wise recurrent dynamics, while extensions to more general recurrent connections are primarily discussed theoretically. Additional evidence would strengthen the broader applicability claims.
Questions for authors
  1. Could the authors provide further evidence that the spike mismatches observed with K=4 have limited effects on training convergence or final accuracy? A comparison with sequential BPTT under identical model configurations would be particularly helpful.
  2. How does PPT compare with existing parallel SNN training approaches, particularly FPT, in terms of runtime and numerical fidelity? Could the authors clarify how much of the reported speedup comes from the parallel algorithm versus kernel-level optimization?
  3. Have the authors examined whether the proposed approach maintains similar efficiency and convergence behavior with more general recurrent connections?
Reviewer_AidZOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Picard-Parallel-Training (PPT) for accelerating temporal training of spiking neural networks. The key idea is to reformulate subtractive-reset LIF dynamics using the pre-threshold membrane potential, which separates the affine temporal recurrence from an explicit spike-dependent forcing term. At each Picard iteration, the forcing term is frozen using the spike trajectory from the previous iterate, allowing the resulting recurrence to be evaluated with a logarithmic-depth parallel scan. The authors further develop a fused CUDA implementation and instantiate the method as PicardSNN. The paper provides convergence and fixed-point analyses and evaluates the approach on neuromorphic audio, event-based vision, pixel-serial image classification, and Long Range Arena tasks. The reported implementation achieves substantial speedups over sequential BPTT while maintaining competitive task accuracy.

Critical strengths
  1. A technically interesting reformulation of recurrent SNN dynamics. The pre-threshold formulation provides a clean way to separate spike-dependent nonlinear events from the affine temporal recurrence while retaining subtractive-reset LIF dynamics at the fixed point. This makes the connection between SNN recurrence and parallel scan algorithms conceptually clear.

  2. Strong systems implementation and broad empirical evaluation. The fused CUDA implementation yields very large reported forward and training speedups and supports sequences up to 65,535 steps. The evaluation also covers several substantially different sequence-modeling settings rather than a single benchmark family.

Critical weaknesses
  1. The novelty over prior fixed-point parallel SNN training is not sufficiently established. FPT already proposed fixed-point parallel training for LIF SNNs while preserving the reset dynamics and provided convergence analysis. The present paper argues that its pre-threshold formulation avoids ambiguity associated with the post-reset state and enables an efficient affine scan, but this distinction is not validated through a direct controlled comparison with FPT. In particular, there is no matched comparison of convergence behavior, spike fidelity, runtime, memory, or downstream accuracy. Since FPT appears to be the closest prior work conceptually, the absence of such evidence makes it difficult to assess whether PPT represents a substantial algorithmic advance or mainly a different formulation plus a stronger implementation.

  2. The theoretical guarantee and the finite-sweep hard-threshold implementation have different scopes, and the empirical evidence does not fully bridge the gap. The sequence-length-independent geometric convergence result applies to the smooth threshold operator under a contraction condition. In contrast, the actual implementation uses a hard threshold, for which the appendix establishes fixed-point consistency and prefix recovery but does not provide a sequence-length-independent global contraction guarantee. The practical choice K=4K=4 is therefore empirical and yields only approximately 85–90% spike agreement with the sequential trajectory. Moreover, the claim that these mismatches do not affect downstream accuracy is not directly supported by a matched comparison between the same architecture trained/evaluated with sequential BPTT and with finite-sweep PPT.

Questions for authors

Could the authors provide a direct comparison against FPT using the same LIF architecture, hardware, datasets, and optimization settings? In particular, how do FPT and PPT differ in convergence/failure cases, spike fidelity, runtime, memory consumption, and task accuracy?

For the hard-threshold implementation used in all experiments, what justifies treating a fixed K=4 as effectively independent of sequence length beyond the empirical results reported here? Can the authors characterize cases in which the number of required sweeps grows with T?

Could the authors report a matched sequential-BPTT control using exactly the same PicardSNN architecture and parameters? This would directly quantify the effect of the 10–15% spike mismatch introduced by finite-sweep PPT on downstream accuracy.

What exact GPU model was used for the runtime experiments, and how much of the reported speedup comes from the parallel algorithm versus the custom fused CUDA kernel? A comparison with an optimized sequential implementation and with FPT would help clarify the source of the speedup.

Reviewer_rqiBOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper rewrites a subtractive-reset LIF with neuron-wise self-feedback in terms of the pre-threshold potential, which turns the dynamics into a linear recurrence plus a spike forcing term κ⊙H(pret−1−θ)\kappa \odot H(\text{pre}_{t-1} - \theta). Training then uses Picard iteration: the forcing is frozen from the previous iterate, the resulting affine recurrence is solved with a parallel scan, and this is repeated K=4K = 4 times. The forward sweep is a fused CUDA kernel, and the backward pass is a reverse affine scan with surrogate gradients. On the theory side, the authors prove contraction for a smooth threshold when α+ρ<1\alpha + \rho < 1. For the hard threshold, they show fixed-point consistency plus exact recovery of the first KK steps after KK sweeps.

Critical strengths
  • The pre-threshold reformulation is simple and clean. The point that solving for vv is ambiguous while solving for pre\text{pre} gives a single-valued spike map is a nice insight.
  • It keeps the standard subtractive-reset LIF rather than modifying the neuron. The appendix shows how the scheme extends to adaptive thresholds, delays, and dense recurrent mixing via the forcing term.
  • The evaluation is broad and the numbers are competitive: the best SSC among listed methods, and strong LRA results for a spiking model.
Critical weaknesses

I have some concerns:

  • The fidelity numbers are a red flag, and they don't show convergence:
    • With K=4K = 4, only 85–90% of (b,n,t)(b, n, t) spike positions match the sequential rollout. SNN firing rates are often ~10% or lower, so an all-zero output could already score around 90% on this metric. Firing rates and spike-level precision/recall/F1 (or mismatches normalized by spike count) are needed.
    • Agreement doesn't improve with KK (90.8 → 87.6 → 87.3 → 89.8 → 92.6 for K=2K = 2 to 3232), which suggests the hard-threshold iteration isn't converging. That contradicts the claim that Table 1 "supports the length-independent convergence predicted by Theorem 1": Thm 1 needs a bounded slope and doesn't cover the hard threshold, and Thm 3 only guarantees the first KK steps.
    • Blaming 10–15% mismatches on "floating-point metastable spikes" isn't convincing; rounding should affect a tiny fraction of spikes, not one in ten. A plot of residuals and spike flips per iteration would help.
  • So the trained model is effectively a "K=4K = 4 truncated Picard SNN", not the sequential LIF network. The paper says the mismatches have "negligible impact on end-to-end accuracy (§4.3)", but §4.3 contains no such experiment. I'd want test accuracy for PPT-trained models run with exact sequential rollout (which is what neuromorphic hardware would do), and a comparison with sequential-BPTT training.
Questions for authors
  1. What are the firing rates, and what are spike precision/recall/F1 vs the sequential rollout across KK and TT?
  2. Why doesn't spike agreement increase with KK? Does the hard-threshold iteration oscillate? Can you plot residuals and spike flips per iteration?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.