How Reliable Is Online Training Through Time? A Convergence and Diagnostic Framework for Spiking Neural Networks

Spiking neural networks (SNNs) offer remarkable energy efficiency, yet their training typically relies on backpropagation through time (BPTT) with surrogate gradients, whose memory consumption grows linearly with the number of time steps. Online Training Through Time (OTTT) avoids full temporal backpropagation by maintaining presynaptic activity traces and performing same-time spatial backpropagation, enabling direct training with memory cost independent of the number of time steps. However, whether its biased updates can still provide reliable optimization guarantees for the original discrete-spike objective remains an open theoretical question. In this work, we establish a non-asymptotic convergence framework for fixed-within-sequence OTTTA in feedforward LIF SNNs. We introduce a randomized-smoothed spiking model as an analytical reference and use its full soft-BPTT gradient as the differentiable benchmark. The systematic OTTTA bias separates into two components: a hard–soft mismatch controlled by a threshold-tail functional measuring near-threshold membrane activity, and a temporal-decoupling bias quantified by the gradient discrepancy between soft OTTT and full soft-BPTT. Under explicit smoothness, stochastic-variance, trajectory-stability, and restricted-PL assumptions, we derive a finite-step bound directly on the hard-spike risk, relative to a restricted minimizer of the smooth reference risk, with explicit optimization, variance, hard–soft, temporal-decoupling, and risk-transfer terms. Across different static and event-based benchmarks, the two approximation discrepancies exhibit distinct task-dependent behavior, while mechanism-specific checkpoint diagnostics show pronounced seed-level associations with accuracy on several tasks. Together, these results connect a finite-step convergence guarantee with practical diagnostics for characterizing the approximation behavior of OTTT.