PicardSNN: Breaking the Sequential Barrier of Spiking Neural Networks with Picard-Parallel-Training

Many important intelligence problems are inherently spatiotemporal: their solutions depend on how signals evolve and retain information over time, as in neuromorphic perception, event-based processing, and long-context sequence modeling. Such temporal dynamics require recurrent state updates, which purely feedforward networks cannot represent. Spiking Neural Networks (SNNs) are an attractive solution, combining nonlinear recurrent state dynamics, sparse event-driven communication, and energy-efficient inference. Their training, however, is hard: backpropagation through time (BPTT) unrolls the recurrence over T steps, resulting in inherently sequential computation over long temporal dependencies, limiting scalability and training efficiency on long sequences. We introduce Picard-Parallel-Training (PPT), a parallel formulation of temporal training for SNNs, and use it to train PicardSNN. Instead of solving for the membrane potential, whose threshold-and-reset makes the recurrence a hard hybrid system, PPT solves for the pre-firing membrane potential, turning the dynamics into a linear recurrence with an explicit spike forcing term. PPT retains subtractive reset and isolates the spike-dependent nonlinearity as an explicit forcing term. Each Picard sweep freezes this term using the preceding iterate’s spikes and solves the resulting affine recurrence through a logarithmic-depth parallel scan. A fused CUDA implementation combines forcing assembly and scan execution without materialising full-length intermediate forcing tensors. The reformulation is algebraically exact, while finite-sweep evaluation approximates the sequential trajectory unless a fixed point is reached. PPT supports sequences of up to 65,535 time steps and achieves up to 763× forward and 348× end-to-end speed-ups over sequential BPTT, with finite-sweep spike agreement and activation sparsity evaluated separately. Across four task families—neuromorphic audio, event-based vision, pixel-serial images, and the Long Range Arena—PicardSNN substantially outperforms Transformer baselines, remains competitive with specialised State-Space Models, and ranks among the strongest recurrent and spiking neural network methods.