Morphing with Spikes

Net2net techniques are used to increase the capacity of ANNs without any loss in accuracy during training. This paper extends those techniques to spiking neural networks. We show that net2widernet is exact on SNNs but net2deepernet is not when the inserted layers receive non-binary inputs. We also provide a method of mitigating the accuracy loss introduced by net2deepernet on spiking networks that optimizes the inserted layer’s gain via gridsearch. We demonstrate both afore- mentioned claims with experiments on image classification datasets CIFAR-10, CIFAR-100, CIFAR-10-DVS, and the RL task Cartpole, where our gain-searched spiking morphism techniques achieve substantially higher prediction agreement than raw insertion, allowing future work to build stronger and more efficient neu- ral architecture search algorithms for spiking neural networks