- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes HomeoLayer, a homeostatic regulation layer designed for spiking neural networks operating on long event streams. The key idea is to maintain a target firing rate by dynamically adjusting neuronal firing thresholds, thereby replacing conventional normalization operations. The method derives a threshold update rule based on a probabilistic formulation of the LIF neuron model and combines it with weight normalization, per-feature affine transformations, and a spike-statistics-based threshold controller. The target firing rate can be learned end-to-end, while threshold updates during inference require only simple additions and multiplications. The paper further provides a theoretical interpretation of adaptive thresholds as a form of normalization and extends the proposed controller to other adaptive spiking neuron models. The authors also conduct detailed ablation studies on the individual components of HomeoLayer and provide reasonable empirical evidence supporting the effectiveness of each component.
Critical strengths
-
The motivation is clear and well justified. The paper provides a thoughtful analysis of why BatchNorm is not well suited to SNNs and clearly motivates the need for normalization-free alternatives in this setting.
-
The threshold update rule is theoretically well grounded rather than being purely heuristic. Starting from a probabilistic LIF formulation, the authors cast firing-rate control as an explicit optimization objective and derive the threshold update rule through gradient descent.
-
The overall method is technically complete, and the paper also provides a principled explanation of why the proposed mechanism can be interpreted as a form of normalization.
-
The experimental evaluation includes several well-targeted ablation studies. In particular, the authors separately examine the effects of the spike-flip approximation, closed-form gradient, batch pooling, and layer pooling. They also evaluate the transferability of the controller to C-SiLIF and RadLIF. These experiments align well with the main design choices of the proposed method and provide useful evidence for their effectiveness.
Critical weaknesses
-
The applicability of the method appears somewhat limited. The authors explicitly note that the proposed controller relies on sufficiently long temporal windows to obtain reliable firing statistics, which makes it less suitable for SNNs operating over short temporal horizons.
-
The empirical evaluation is largely concentrated on long event streams in the audio domain. It would strengthen the paper to include validation on additional modalities or task settings, which could better demonstrate the generality of the proposed approach.
-
Although inference is relatively simple, training still relies on a more involved probabilistic formulation. In addition, I wonder whether the assumption of a shared firing-rate target may be overly restrictive in some settings. It would be helpful for the authors to further discuss when this assumption is expected to hold and whether more flexible, neuron- or feature-specific targets could provide additional benefits.
-
The claimed hardware friendliness is currently supported mainly by theoretical analysis rather than end-to-end hardware experiments. That said, I fully appreciate the practical difficulty of providing such validation, and I do not consider this limitation to materially affect my overall assessment of the paper.
Questions for authors
Regarding Weaknesses 2 and 3, I would like to see stronger evidence that the proposed threshold adaptation mechanism can be effective in broader and more general settings. To the best of my knowledge, many online learning algorithms for SNNs are also incompatible with BatchNorm. If HomeoLayer could be adapted to such settings, I believe this would be a particularly compelling extension of the current work. This is only a suggestion rather than a strict requirement; demonstrating the method on additional benchmarks or tasks beyond the current setting would already provide valuable evidence of its general applicability.
In addition, I would appreciate further discussion of the assumption regarding firing-rate consistency. For example, would it be possible or beneficial to impose different firing-rate targets or constraints for shallow and deep layers, rather than sharing the same target across layers? A discussion or preliminary analysis along this direction could help clarify how restrictive the current assumption is.
Overall, I find this to be a valuable piece of work and would be very happy to engage in further discussion with the authors. If the authors can satisfactorily address some of the concerns above, I would be willing to increase my score.
5 comments
Reply to Reviewer 3_2
W4 (hardware claims). Agreed. The claims are rescoped (abstract, introduction, Sec. 5.3, conclusion): against folded time-invariant BatchNorm HomeoLayer's inference cost is not lower; what it removes is the per-timestep state of BNTT, the per-step square root and division of LayerNorm, the batch statistics of training and the threshold hyperparameter. A new appendix counts the controller's operations in hardware primitives: per layer and timestep, two spike counts over the layer (event-driven, one event per spike and one per flip), one scalar update and one broadcast of the threshold. Measured on the trained models, per timestep and summed over the hidden layers, against the dense bias additions of a folded normalizer:
| Dataset | Model | spikes/step | flips/step | controller events | ratio to | |
|---|---|---|---|---|---|---|
| SHD | AttnSNN | 3500 | 345 | 249 | 594 | 0.17 |
| SSC | AttnSNN | 3500 | 255 | 238 | 494 | 0.14 |
| GSC | AttnSNN | 3500 | 332 | 287 | 620 | 0.18 |
| SHD | FFSNN | 1400 | 243 | 195 | 437 | 0.31 |
| NMNIST | FFSNN | 1400 | 91 | 86 | 177 | 0.13 |
We have not run on neuromorphic hardware and now say so.
Q (online learning incompatible with BatchNorm). This was the extension we most wanted to try, and we report a preliminary result (new appendix). We trained the feedforward HomeoLayer model on SHD with no backward pass through the network by a three-factor rule: each hidden neuron receives a random per-sample offset to its drive held over the sequence, the per-sample loss change across two antithetic passes is broadcast as a scalar, and each synapse updates from that scalar, its neuron's offset sign and a local eligibility (the input trace weighted by the neuron's own density at threshold). is the number of offset draws per batch; three seeds:
| closed-form gradient | perturbation, | |
|---|---|---|
| HomeoLayer | ||
| fixed threshold | ||
| fixed threshold (best fixed value) |
With four draws per batch the rule comes within three points of the closed-form gradient. The same rule on the same model with the controller replaced by a fixed threshold loses fifteen points at the conventional and nineteen at the best fixed value. The rule uses no batch statistics and can run with no per-timestep state, and the controller is what makes it work: it keeps every layer in the regime where a drive perturbation changes the output. We would be glad to discuss this direction further.
I am pleased to see the authors' detailed response, and I appreciate their efforts to address my concerns. Most of my concerns have been adequately addressed. However, I would still like to clarify two points:
-
Regarding Weakness 1: The authors conducted additional experiments on DVS-Gesture with and revised the manuscript to state that "when the number of timesteps is too small, the controller has limited opportunities to take effect." However, I believe the authors should provide a more concrete explanation of what constitutes "too small." Specifically, I would appreciate a sensitivity analysis showing how the performance of HomeoLayer varies with the number of timesteps . Such an analysis would help identify the temporal range in which HomeoLayer is most effective. I believe that a relatively simple experiment sweeping would be feasible within the rebuttal period and would provide valuable evidence regarding the applicability of the proposed method.
-
Regarding Weakness 3: I would like to emphasize that online learning and backpropagation-free (BP-free) learning are not equivalent concepts. In fact, many online learning algorithms still rely on backpropagation-based gradient updates. The key distinction is that online learning does not accumulate gradients across the entire temporal dimension ; instead, gradient updates are performed incrementally at individual timesteps. Therefore, while the additional BP-free learning experiments provide useful evidence of the broader applicability of HomeoLayer, they do not directly establish whether HomeoLayer is compatible with online learning. In other words, these experiments neither demonstrate nor rule out its applicability to online learning settings.
Overall, I appreciate the authors' substantial efforts and the additional experimental results. Nevertheless, I would still welcome further clarification and responses from the authors regarding these two points.
Reply to Reviewer 3_1
W1 (short temporal horizons). The controller needs enough steps for its spike statistics to be informative, and we scoped the paper to long streams for that reason. Within the rebuttal period we tested the shortest horizon in common use for event vision, DVS-Gesture at T=16, on SpikingJelly's published convolutional network (five Conv-BN-LIF blocks of 128 channels and two spiking FC layers), trained under the library's recipe, with every BatchNorm replaced by the HomeoLayer controller in the second experiment; three seeds each:
Both experiments result in accuracy below the 97.6 reported for this network with learnable time constants and T=20; the comparison is between the two experiments under one recipe. Sixteen steps suffice because the controller's window is one step and its gain acts from the first spike; the Scope paragraph now says that a few steps leave the controller little to act on, rather than calling short tasks unsuitable.
W2 (modalities). Beyond the four benchmarks of the submission, the revision adds the convolutional DVS-Gesture result above (frame-binned event vision, a different architecture family) and, in response to your question below, a training regime without backpropagation.
W3a (training relies on the probabilistic formulation). At training time the formulation costs two extra state variables per neuron (the propagated mean and variance) and replaces the surrogate gradient; it is comparable in cost to a surrogate and is what makes the threshold and the rate target differentiable. Nothing of it is needed at inference.
W3b (shared firing-rate target). The target is shared within a layer but not across layers: each layer learns its own r∗=σ(ρ). The new appendix tabulates the learned values (initialized at 0.10 in hidden blocks and 0.01 in the input block; mean ± s.d. over three seeds):
The targets of one network span two orders of magnitude: the input blocks of SSC and GSC learn targets near zero, while the second output block learns 0.3 on SSC and 0.5 on GSC, so shallow and deep layers end up with different targets without being told to. How restrictive one shared value is we measured directly: the same experiment with the target held at its initialization in every layer (0.10 hidden, 0.01 input) and not trained, three seeds per dataset, now in the appendix:
The shared hand-set value loses nothing on any of the three datasets, including SSC and GSC where the learned targets diverge most; it is slightly ahead. So the assumption is not restrictive in our setting: the learned target is not what sets the accuracy, and what it removes is the choice of the value, which the sensitivity sweep shows to be forgiving.
On neuron-specific targets: under layer pooling the shared target constrains the layer mean only, not each neuron, because one threshold serves the layer while each neuron's operating point is set by its own learned affine scale and shift and its weights. We measured this on the trained SHD AttnSNN (two test batches): in the first attention layer, whose target is 0.082, the per-neuron mean rates run from 0.06 to 0.25 between the 10th and 90th percentiles (coefficient of variation 0.57, maximum 0.54), and in the two projection layers 16 to 19% of the neurons are silent while others fire above 0.4. The controller leaves this heterogeneity in place, so the assumption it makes is the one a normalizer makes too: that the layer's mean drive, here its mean rate, is the right quantity to regulate. A per-neuron target would add N parameters per layer and is identifiable only under batch pooling, where each neuron has its own threshold; under layer pooling only its mean would be. We have not tested per-neuron targets and say so.