Event Actions Predict Loss Jumps at Spike Deletion Boundaries
Abstract
Spiking neural networks combine continuous dynamics with discrete events triggered by threshold crossings. A finite parameter change can remove a spike, switching the network to a different event sequence and producing a loss change that is not represented by the gradient evaluated before the spike disappears. We show that this loss jump can be predicted from the EventProp backward pass while the spike is still present. Deleting a spike removes both its membrane reset and its downstream synaptic transmission. We sum the corresponding backward terms and call the result the event action. Across 120 deletions of single spikes in a learned LIF network trained on the Spiking Heidelberg Digits (SHD) dataset, the event action computed before deletion closely matches the signed loss jump. The result replicates in an independently trained network and on examples from the speaker held out from training. Replacing the target spike with other spikes reduces prediction accuracy, and spike time, boundary location, spike position, and source neuron activity do not explain the relation. The prediction magnitude also separates deletions with small and large effects: the highest predicted impact quintile has a median realized loss change about 803 times larger than the lowest. For 20 parameter directions that cross a deletion boundary, adding the predicted jump raises the rank correlation with the realized loss change from 0.281 to 0.997 and selects the candidate with the lowest loss. The backward pass contains information about the loss change caused by losing an event that the parameter gradient before deletion does not capture.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.