SpikingPhoenix: Semantics-Preserving Execution Optimization for Direct SNN Training on GPUs
Abstract
Direct multi-step Spiking Neural Network (SNN) training remains costly on graphics processing units (GPUs) after temporal fusion because backward propagation spans recurrent state, backward-required forward quantities, and persistent minibatch updates. Existing systems optimize these costs separately, leaving their composition under a fixed training update unresolved. We present SpikingPhoenix, which reorganizes nested recurrent and minibatch execution boundaries without changing that update. SEQUENCE pairs forward and reverse scans, SEGMENT-K rematerializes discarded membrane state within recurrent backward, and GRAPH-V2 replays the device-side state transitions required by each minibatch update. The transformed paths match the reference exactly on discrete observables and within data-type-specific tolerances on floating-point observables. Across representative Spiking Transformers, including QKFormer and MaxFormer, SpikingPhoenix achieves 1.41-2.73× end-to-end training speedups over matched SpikingJelly runs on complete CIFAR-10 schedules. On ImageNet-1K, SEQUENCE further achieves 1.37-1.45× geometric-mean speedups across T=2,4,8.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.