- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper extends Net2Net growth operators to spiking neural networks with LIF and ALIF neurons. Widenning duplicates channels with their BN, weight, and neuron states, and deepening inserts and identity conv and an LIF layer. The paper scales the identity kernel by a gain chosen per layer by grid search. Experiments use a 5-layer conv SNN on CIFAR-10, CIFAR-100 and CIFAR-10-DVS and an R-STDP on CartPole, reporting agreement at the growth point and accuracy curves afterwards.
Critical strengths
Problems are clear, and proofs are correct. Negative results are reported openly. Both BP and R-STDP training are covered.
Critical weaknesses
The technical step beyond net2net is small. The widening proof holds for any deterministic stateful unit and uses nothing specific to SNN.
Copied channels get identical gradients. The introduction says so itself, but the method didn't solve this problem.
On CartPole, agreement is measured with neuron noise turned off, so it does not show that the stochastic policy is preserved.
The Experiments do not show that growth helps training. The base stops training at the growth point, and the from-scratch model gets only half the budget.
Questions for authors
How do the methods compare with a base trained for 30 epochs or a grown model trained from scratch for 30 epochs?