acceptodds
Under review as a conference paper at ICLR 2027

Thresholds, Not Normalization: A Homeostatic Spiking Layer at Parity Without the Overhead

Abstract

Spiking neural networks processing long event streams frequently rely on normalization to match input currents to a fixed firing threshold. However, this reliance introduces batch reductions, floating-point operations, and per-timestep statistics that neuromorphic hardware struggles to accommodate efficiently. We propose adapting the threshold itself instead of the incoming current. We introduce HomeoLayer, a leaky integrate-and-fire layer featuring -normalized weights, a per-feature affine transform, closed-form spike-probability gradients, and an end-to-end learned threshold controller. Pooling this controller over the batch or layer recovers the classic BatchNorm/LayerNorm dichotomy. At inference, HomeoLayer utilizes only additions and multiplications, stores no running statistics, and remains completely batch-independent. Across two architectures and four datasets (SHD, SSC, GSC, and N-MNIST), HomeoLayer achieves parity with per-timestep batch normalization while maintaining a fraction of the per-neuron state. Furthermore, learning the firing rate target eliminates the need for manual threshold tuning. Transplanting the controller into existing adaptive models (C-SiLIF and RadLIF) successfully replaces native normalization with a marginal accuracy cost. This work is open-sourced at https://anonymous.4open.science/r/snn_homeolayer-E003.

Primary area:
learning on time series and dynamical systems
Keywords:
spiking neural network, homeostasis, threshold, normalization

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_66PQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes HomeoLayer, a homeostatic regulation layer designed for spiking neural networks operating on long event streams. The key idea is to maintain a target firing rate by dynamically adjusting neuronal firing thresholds, thereby replacing conventional normalization operations. The method derives a threshold update rule based on a probabilistic formulation of the LIF neuron model and combines it with ℓ2\ell_2 weight normalization, per-feature affine transformations, and a spike-statistics-based threshold controller. The target firing rate can be learned end-to-end, while threshold updates during inference require only simple additions and multiplications. The paper further provides a theoretical interpretation of adaptive thresholds as a form of normalization and extends the proposed controller to other adaptive spiking neuron models. The authors also conduct detailed ablation studies on the individual components of HomeoLayer and provide reasonable empirical evidence supporting the effectiveness of each component.

Critical strengths
  1. The motivation is clear and well justified. The paper provides a thoughtful analysis of why BatchNorm is not well suited to SNNs and clearly motivates the need for normalization-free alternatives in this setting.

  2. The threshold update rule is theoretically well grounded rather than being purely heuristic. Starting from a probabilistic LIF formulation, the authors cast firing-rate control as an explicit optimization objective and derive the threshold update rule through gradient descent.

  3. The overall method is technically complete, and the paper also provides a principled explanation of why the proposed mechanism can be interpreted as a form of normalization.

  4. The experimental evaluation includes several well-targeted ablation studies. In particular, the authors separately examine the effects of the spike-flip approximation, closed-form gradient, batch pooling, and layer pooling. They also evaluate the transferability of the controller to C-SiLIF and RadLIF. These experiments align well with the main design choices of the proposed method and provide useful evidence for their effectiveness.

Critical weaknesses
  1. The applicability of the method appears somewhat limited. The authors explicitly note that the proposed controller relies on sufficiently long temporal windows to obtain reliable firing statistics, which makes it less suitable for SNNs operating over short temporal horizons.

  2. The empirical evaluation is largely concentrated on long event streams in the audio domain. It would strengthen the paper to include validation on additional modalities or task settings, which could better demonstrate the generality of the proposed approach.

  3. Although inference is relatively simple, training still relies on a more involved probabilistic formulation. In addition, I wonder whether the assumption of a shared firing-rate target may be overly restrictive in some settings. It would be helpful for the authors to further discuss when this assumption is expected to hold and whether more flexible, neuron- or feature-specific targets could provide additional benefits.

  4. The claimed hardware friendliness is currently supported mainly by theoretical analysis rather than end-to-end hardware experiments. That said, I fully appreciate the practical difficulty of providing such validation, and I do not consider this limitation to materially affect my overall assessment of the paper.

Questions for authors

Regarding Weaknesses 2 and 3, I would like to see stronger evidence that the proposed threshold adaptation mechanism can be effective in broader and more general settings. To the best of my knowledge, many online learning algorithms for SNNs are also incompatible with BatchNorm. If HomeoLayer could be adapted to such settings, I believe this would be a particularly compelling extension of the current work. This is only a suggestion rather than a strict requirement; demonstrating the method on additional benchmarks or tasks beyond the current setting would already provide valuable evidence of its general applicability.

In addition, I would appreciate further discussion of the assumption regarding firing-rate consistency. For example, would it be possible or beneficial to impose different firing-rate targets or constraints for shallow and deep layers, rather than sharing the same target across layers? A discussion or preliminary analysis along this direction could help clarify how restrictive the current assumption is.

Overall, I find this to be a valuable piece of work and would be very happy to engage in further discussion with the authors. If the authors can satisfactorily address some of the concerns above, I would be willing to increase my score.

5 comments
AuthorsOpenReview ↗

Reply to Reviewer 3_1

W1 (short temporal horizons). The controller needs enough steps for its spike statistics to be informative, and we scoped the paper to long streams for that reason. Within the rebuttal period we tested the shortest horizon in common use for event vision, DVS-Gesture at T=16T = 16, on SpikingJelly's published convolutional network (five Conv-BN-LIF blocks of 128 channels and two spiking FC layers), trained under the library's recipe, with every BatchNorm replaced by the HomeoLayer controller in the second experiment; three seeds each:

Data splitBatchNorm (library)HomeoLayer controllerΔ\Delta
library loader, 1,176 / 28894.10±0.6094.10 \pm 0.6095.14±0.9295.14 \pm 0.92+1.04+1.04
tonic loader, 1,077 / 26493.75±1.0393.75 \pm 1.0394.01±1.2594.01 \pm 1.25+0.26+0.26

Both experiments result in accuracy below the 97.6 reported for this network with learnable time constants and T=20T = 20; the comparison is between the two experiments under one recipe. Sixteen steps suffice because the controller's window is one step and its gain acts from the first spike; the Scope paragraph now says that a few steps leave the controller little to act on, rather than calling short tasks unsuitable.

W2 (modalities). Beyond the four benchmarks of the submission, the revision adds the convolutional DVS-Gesture result above (frame-binned event vision, a different architecture family) and, in response to your question below, a training regime without backpropagation.

W3a (training relies on the probabilistic formulation). At training time the formulation costs two extra state variables per neuron (the propagated mean and variance) and replaces the surrogate gradient; it is comparable in cost to a surrogate and is what makes the threshold and the rate target differentiable. Nothing of it is needed at inference.

W3b (shared firing-rate target). The target is shared within a layer but not across layers: each layer learns its own r∗=σ(ρ)r^* = \sigma(\rho). The new appendix tabulates the learned values (initialized at 0.10 in hidden blocks and 0.01 in the input block; mean ±\pm s.d. over three seeds):

DatasetModelInputAttn 1 / HiddenOut 1Attn 2Out 2
SHDAttnSNN0.0087±0.00000.0087 \pm 0.00000.082±0.0000.082 \pm 0.0000.090±0.0010.090 \pm 0.0010.075±0.0020.075 \pm 0.0020.112±0.0050.112 \pm 0.005
SSCAttnSNN0.0033±0.00020.0033 \pm 0.00020.101±0.0080.101 \pm 0.0080.133±0.0130.133 \pm 0.0130.073±0.0050.073 \pm 0.0050.307±0.0320.307 \pm 0.032
GSCAttnSNN0.0000±0.00010.0000 \pm 0.00010.098±0.0040.098 \pm 0.0040.080±0.0030.080 \pm 0.0030.115±0.0030.115 \pm 0.0030.536±0.0800.536 \pm 0.080
SHDFFSNN0.074±0.0200.074 \pm 0.0200.168±0.0020.168 \pm 0.002------
NMNISTFFSNN0.0057±0.00030.0057 \pm 0.00030.141±0.0060.141 \pm 0.006------

The targets of one network span two orders of magnitude: the input blocks of SSC and GSC learn targets near zero, while the second output block learns 0.3 on SSC and 0.5 on GSC, so shallow and deep layers end up with different targets without being told to. How restrictive one shared value is we measured directly: the same experiment with the target held at its initialization in every layer (0.10 hidden, 0.01 input) and not trained, three seeds per dataset, now in the appendix:

learned per-layer targetsone shared targetΔ\Delta
SHD93.95±0.3893.95 \pm 0.3894.15±0.1494.15 \pm 0.14+0.20+0.20
SSC74.52±0.4274.52 \pm 0.4275.01±0.2675.01 \pm 0.26+0.49+0.49
GSC86.88±0.0986.88 \pm 0.0987.53±0.1687.53 \pm 0.16+0.65+0.65

The shared hand-set value loses nothing on any of the three datasets, including SSC and GSC where the learned targets diverge most; it is slightly ahead. So the assumption is not restrictive in our setting: the learned target is not what sets the accuracy, and what it removes is the choice of the value, which the sensitivity sweep shows to be forgiving.

On neuron-specific targets: under layer pooling the shared target constrains the layer mean only, not each neuron, because one threshold serves the layer while each neuron's operating point is set by its own learned affine scale and shift and its weights. We measured this on the trained SHD AttnSNN (two test batches): in the first attention layer, whose target is 0.082, the per-neuron mean rates run from 0.06 to 0.25 between the 10th and 90th percentiles (coefficient of variation 0.57, maximum 0.54), and in the two projection layers 16 to 19% of the neurons are silent while others fire above 0.4. The controller leaves this heterogeneity in place, so the assumption it makes is the one a normalizer makes too: that the layer's mean drive, here its mean rate, is the right quantity to regulate. A per-neuron target would add NN parameters per layer and is identifiable only under batch pooling, where each neuron has its own threshold; under layer pooling only its mean would be. We have not tested per-neuron targets and say so.

AuthorsOpenReview ↗

Reply to Reviewer 3_2

W4 (hardware claims). Agreed. The claims are rescoped (abstract, introduction, Sec. 5.3, conclusion): against folded time-invariant BatchNorm HomeoLayer's inference cost is not lower; what it removes is the per-timestep state of BNTT, the per-step square root and division of LayerNorm, the batch statistics of training and the threshold hyperparameter. A new appendix counts the controller's operations in hardware primitives: per layer and timestep, two spike counts over the layer (event-driven, one event per spike and one per flip), one scalar update and one broadcast of the threshold. Measured on the trained models, per timestep and summed over the hidden layers, against the NN dense bias additions of a folded normalizer:

DatasetModelNNspikes/stepflips/stepcontroller eventsratio to NN
SHDAttnSNN35003452495940.17
SSCAttnSNN35002552384940.14
GSCAttnSNN35003322876200.18
SHDFFSNN14002431954370.31
NMNISTFFSNN140091861770.13

We have not run on neuromorphic hardware and now say so.

Q (online learning incompatible with BatchNorm). This was the extension we most wanted to try, and we report a preliminary result (new appendix). We trained the feedforward HomeoLayer model on SHD with no backward pass through the network by a three-factor rule: each hidden neuron receives a random per-sample offset to its drive held over the sequence, the per-sample loss change across two antithetic passes is broadcast as a scalar, and each synapse updates from that scalar, its neuron's offset sign and a local eligibility (the input trace weighted by the neuron's own density at threshold). KK is the number of offset draws per batch; three seeds:

closed-form gradientperturbation, K=4K=4
HomeoLayer90.39±0.3490.39 \pm 0.3487.12±0.9787.12 \pm 0.97
fixed threshold θ=1\theta = 189.53±0.0889.53 \pm 0.0874.46±1.9674.46 \pm 1.96
fixed threshold θ=0.5\theta = 0.5 (best fixed value)90.18±0.8790.18 \pm 0.8771.39±2.1971.39 \pm 2.19

With four draws per batch the rule comes within three points of the closed-form gradient. The same rule on the same model with the controller replaced by a fixed threshold loses fifteen points at the conventional θ=1\theta = 1 and nineteen at the best fixed value. The rule uses no batch statistics and can run with no per-timestep state, and the controller is what makes it work: it keeps every layer in the regime where a drive perturbation changes the output. We would be glad to discuss this direction further.

I am pleased to see the authors' detailed response, and I appreciate their efforts to address my concerns. Most of my concerns have been adequately addressed. However, I would still like to clarify two points:

  1. Regarding Weakness 1: The authors conducted additional experiments on DVS-Gesture with T=16T=16 and revised the manuscript to state that "when the number of timesteps is too small, the controller has limited opportunities to take effect." However, I believe the authors should provide a more concrete explanation of what constitutes "too small." Specifically, I would appreciate a sensitivity analysis showing how the performance of HomeoLayer varies with the number of timesteps TT. Such an analysis would help identify the temporal range in which HomeoLayer is most effective. I believe that a relatively simple experiment sweeping TT would be feasible within the rebuttal period and would provide valuable evidence regarding the applicability of the proposed method.

  2. Regarding Weakness 3: I would like to emphasize that online learning and backpropagation-free (BP-free) learning are not equivalent concepts. In fact, many online learning algorithms still rely on backpropagation-based gradient updates. The key distinction is that online learning does not accumulate gradients across the entire temporal dimension TT; instead, gradient updates are performed incrementally at individual timesteps. Therefore, while the additional BP-free learning experiments provide useful evidence of the broader applicability of HomeoLayer, they do not directly establish whether HomeoLayer is compatible with online learning. In other words, these experiments neither demonstrate nor rule out its applicability to online learning settings.

Overall, I appreciate the authors' substantial efforts and the additional experimental results. Nevertheless, I would still welcome further clarification and responses from the authors regarding these two points.

Reviewer_vANROpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HomeoLayer, a spiking neural network (SNN) layer that replaces conventional normalization with online firing-threshold adaptation. The method combines ℓ2\ell_2-normalized weights, a learnable per-feature affine transformation, and a threshold controller that adjusts firing thresholds toward a learnable target firing rate. To reduce inference overhead, the controller estimates firing-rate errors and gradient-related gains using binary spike counts and inter-timestep spike-flip statistics, avoiding explicit probability-density calculations and normalization-related division and square-root operations. The authors evaluate HomeoLayer on four datasets (SHD, SSC, GSC, and N-MNIST) using feedforward and attention-based SNN architectures, comparing it with fixed thresholds, Batch Normalization Through Time (BNTT), and LayerNorm. They also transplant the threshold controller into C-SiLIF and RadLIF to investigate its applicability to existing adaptive-neuron models.

Critical strengths
  1. The paper addresses an important practical limitation of normalization in SNNs: temporal normalization methods can introduce timestep-dependent parameters and statistics, while LayerNorm requires runtime statistical computations. These operations can increase memory requirements and inference overhead, complicating deployment on resource-constrained neuromorphic hardware. Replacing normalization with threshold adaptation is a well-motivated alternative that directly targets these deployment constraints.

  2. The authors evaluate HomeoLayer on four datasets using both feedforward and attention-based SNN architectures, and further investigate its transferability to C-SiLIF and RadLIF. The experiments include comparisons with conventional normalization baselines and several ablations of the proposed mechanism. This provides a useful initial assessment of the method and helps characterize the settings in which it can serve as a normalization replacement.

Critical weaknesses
  1. Insufficient evidence for the claimed hardware benefits and limited comparison with normalization-free alternatives. Although HomeoLayer avoids some expensive operations required by complex normalization, it introduces online threshold updates during inference that require per-timestep firing-rate estimation, spike-flip statistics, and dynamic multiplications. These operations may incur non-negligible computational, memory-access, and communication overhead on neuromorphic hardware. The paper quantifies state counts but provides neither hardware measurements nor a quantitative evaluation of the controller's actual execution cost. Without evidence that these additional costs are sufficiently small to justify online adaptation, the practical significance of the method remains unclear, particularly because the accuracy gains are modest on most benchmarks. Furthermore, the paper does not adequately compare its approach with existing methods for eliminating normalization in SNNs, such as [1, 2], which explore alternative approaches to normalization-free inference.

  2. Insufficient hyperparameter analysis and limited experimental coverage. HomeoLayer relies on several hand-designed heuristics and hyperparameters, including the threshold update rate ηθ\eta_\theta, the gain floor ϕmin⁡\phi_{\min}, and the lower bound rmin⁡r_{\min} in the target firing-rate parameterization. The paper does not sufficiently justify these design choices or provide systematic sensitivity analyses, raising concerns about training stability, convergence, and reproducibility. Experimental validation is also limited to two relatively simple architectures and four datasets, with visual event-based evaluation restricted to N-MNIST. Further evaluation on larger and more challenging neuromorphic benchmarks, such as N-ImageNet or an appropriate subset, would help establish the method's robustness, scalability, and broader applicability.

  • [1]. Training Deep Spiking Neural Networks without Normalization, ICML 2026.
  • [2]. Intrinsically Stable Spiking Neural Networks: Overcoming the Performance Barrier in the Absence of Batch Normalization, ECCV 2026.
Questions for authors
  1. In Section 3.2, Eq. (5) appears to contain a sign error. Since ∂ps/∂θ<0\partial p_s/\partial\theta < 0, the update as written would decrease the threshold when the firing rate exceeds its target, contrary to the intended homeostatic behavior.

  2. Inconsistent target firing-rate parameterization. Section 4.2 defines the target firing rate as r∗=rmin+(1−rmin)σ(ρ)r^* = r_{min} + (1-r_{min})\sigma(\rho), whereas Algorithm A1 and Table A2 use σ(ρ)\sigma(\rho). The paper should clarify whether rminr_{min} is used in the actual implementation and, if so, specify its value.

  3. Some symbols are insufficiently defined or used for multiple purposes, making the method harder to follow and reproduce. For example, Algorithm A1 uses β\beta for both the synaptic decay coefficient and the affine shift parameter. Clear and consistent definitions of these symbols throughout the derivation and implementation would improve the paper's clarity and reproducibility.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer 2_1

W1a (hardware benefit not quantified). We agree that state counts alone do not establish a hardware benefit, and we have added an operation count (new appendix) and rescoped the claims. Per layer and timestep the layer-pooled controller needs two spike counts over the layer, one scalar update and one broadcast of the threshold. The counts are what one accumulator neuron with unit weights computes from the layer's spike bus, so they are event-driven: one synaptic event per spike and one per flip. Measured on the trained models, per timestep and summed over the hidden layers, against the NN dense bias additions a folded normalizer performs at every step:

DatasetModelNNspikes/stepflips/stepcontroller eventsratio to NN
SHDAttnSNN35003452495940.17
SSCAttnSNN35002552384940.14
GSCAttnSNN35003322876200.18
SHDFFSNN14002431954370.31
NMNISTFFSNN140091861770.13

The flip statistic needs one bit per neuron, and at batch size one the threshold is one register per layer. We now state explicitly that against folded time-invariant BatchNorm HomeoLayer's inference cost is not lower (Sec. 5.3); its advantages there are the removed threshold hyperparameter and training without batch statistics, while against BNTT it removes the 4T4T per-neuron state and against LayerNorm the per-step square root and division. We have not run on neuromorphic hardware and say so; a measured energy comparison is future work.

W1b (comparison with normalization-free SNNs [1, 2]). Thank you for the pointers; both are now discussed in Sec. 2 and added to Table 1. SpikeInit [1] initializes weights and surrogate widths so that firing rates and gradient magnitudes are stable at the start of training; IS-SNN [2] standardizes weights analytically and scales residual branches so that rates do not decay with depth. Both fix the threshold (0.5 and 1.0), operate at T=4T = 4 to 16 on static or frame-binned image benchmarks, and leave every quantity fixed after training: the operating point is set open-loop and must hold for whatever the input does. Our setting is long event streams, where the drive is non-stationary within a sequence, and HomeoLayer closes the loop at inference around a learned rate. The approaches are complementary, since an initialization such as SpikeInit's could seed HomeoLayer's weights; a direct comparison on SHD/SSC would require porting their deep-image recipes to long-stream models, which we could not complete within the rebuttal period.

W1c (modest accuracy gains). The paper claims parity, not a win (Sec. 5.2, "We delimit our claims"). The contribution is a normalizer with no per-timestep state, no square root or division, no batch dependence at inference and no threshold to tune, at the same accuracy as BNTT.

AuthorsOpenReview ↗

Reply to Reviewer 2_2

W2a (hyperparameters ηθ\eta_\theta, φmin⁡\varphi_{\min}, rmin⁡r_{\min}). Two of the three are not used in HomeoLayer, and the text now says so: φmin⁡\varphi_{\min} is 0 in every HomeoLayer arm (it is on only in the transplanted controller), and the target is r∗=σ(ρ)r^* = \sigma(\rho) with no lower bound; the rmin⁡r_{\min} formula in Sec. 4.2 was an error carried over from the transplant code, now corrected. For the hyperparameters HomeoLayer does introduce, the new appendix varies each one over one to two decades, one factor at a time, on SHD AttnSNN at three seeds (canonical arm 93.95±0.3893.95 \pm 0.38):

HyperparameterPaperSweptAccuracy (%)Δ\Delta
Surrogate variance σξ,surr2\sigma^2_{\xi,\mathrm{surr}}0.10.0193.54±0.5193.54 \pm 0.51−0.41-0.41
194.51±0.3594.51 \pm 0.35+0.56+0.56
Controller gain ηθ\eta_\theta10194.60±0.5094.60 \pm 0.50+0.65+0.65
10089.47±0.4789.47 \pm 0.47−4.48-4.48
Initial threshold θ0\theta_010.594.13±0.1894.13 \pm 0.18+0.18+0.18
293.63±0.5593.63 \pm 0.55−0.32-0.32
Window TwT_w1 step5 steps94.05±0.4194.05 \pm 0.41+0.10+0.10
Target init (hidden / input)0.10 / 0.010.05 / 0.00593.11±0.7393.11 \pm 0.73−0.84-0.84
0.20 / 0.0293.70±0.4893.70 \pm 0.48−0.25-0.25

Every setting but the controller gain stays within a point of the canonical arm; a ten-fold smaller gain is slightly better than the paper's value on SHD, a ten-fold larger one costs 4.5 points. On stability and convergence: all 27 sweep runs and every HomeoLayer run reported in the paper trained to completion without divergence, and the seed standard deviations across the sweep are 0.2 to 0.7 points; the update is a bounded first-order step whose size is capped by the gain times the flip statistic, and the gain is the one setting that needs care. On reproducibility, the code and the launch configuration of every arm in the paper are in the anonymous repository linked in the abstract. We note that the baselines' boxcar half-width (0.5) is a hyperparameter of the same kind that we, like the field, did not tune.

W2b (larger visual benchmarks). Within the rebuttal period we added a convolutional visual benchmark: SpikingJelly's published DVS-Gesture network (five Conv-BN-LIF blocks of 128 channels and two spiking FC layers, T=16T = 16), trained under the library's recipe, with every BatchNorm replaced by the HomeoLayer controller in the second arm; three seeds each:

Data splitBatchNorm (library)HomeoLayer controllerΔ\Delta
library loader, 1,176 / 28894.10±0.6094.10 \pm 0.6095.14±0.9295.14 \pm 0.92+1.04+1.04
tonic loader, 1,077 / 26493.75±1.0393.75 \pm 1.0394.01±1.2594.01 \pm 1.25+0.26+0.26

Both arms sit below the 97.6 reported for this network with learnable time constants and T=20T = 20; the comparison is between the two arms under one recipe, as everywhere in the paper. This also tests a short horizon, T=16T = 16. N-ImageNet is beyond what we can train in the rebuttal window; the limitations now say so.

Q1 (sign of Eq. 5). Correct, thank you. Eq. 5 omitted the minus sign of gradient descent. We now write Δθ=−ηθ(rˉ−r∗) ∂rˉ/∂θ=ηθ(rˉ−r∗) φˉ\Delta\theta = -\eta_\theta(\bar r - r^*)\,\partial \bar r/\partial\theta = \eta_\theta(\bar r - r^*)\,\bar\varphi with φ≜−∂ps/∂θ>0\varphi \triangleq -\partial p_s/\partial\theta > 0 defined explicitly, so a rate above target raises the threshold. The implementation always had this sign (controller pseudocode).

Q2 (rmin⁡r_{\min} inconsistency). The implementation uses r∗=σ(ρ)r^* = \sigma(\rho) with no floor; the rmin⁡r_{\min} form was the transplanted controller's parameterization (rmin⁡=5×10−3r_{\min} = 5 \times 10^{-3}) and appeared in Sec. 4.2 by mistake. Corrected; the pseudocode and the hyperparameter table were right.

Q3 (symbols). The affine shift is now bb throughout (Sec. 4.1, the controller pseudocode and the architecture appendix), leaving β\beta to the synaptic decay; α\alpha is the membrane decay only, and φ\varphi, previously used before it was defined, is now defined where it first appears (Eq. 5). We would be glad to fix any other symbol the reviewer found unclear.

Thank you for the clarifications. The rebuttal has addressed my main concerns, and I have increased my overall rating to 3.

My remaining suggestion concerns the framing of the paper, particularly the phrase "Without the Overhead" in the title. Strictly speaking, although HomeoLayer avoids some of the computational costs associated with advanced normalization, it introduces additional inference-time computations involving firing-rate estimation and spike-flip statistics. Therefore, the method does not eliminate extra overhead, but rather trades advanced normalization overhead for the overhead of threshold adaptation. I encourage the authors to make their claims more precise and consider qualifying this statement in the title and throughout the paper, so that the claimed hardware benefits are presented more accurately.

Additionally, I encourage the authors to carefully review the mathematical expressions throughout the paper and ensure their correctness and consistency with the corresponding algorithmic descriptions and implementation. In particular, inconsistent definitions and discrepancies between the main text, pseudocode, and hyperparameter tables should be checked to improve the clarity and reproducibility of the work.

Reviewer_FkGgOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method that dynamically adapts the firing threshold instead of normalizing the input current. The authors introduce HomeoLayer, which features L2 weight normalization followed by an affine transformation of the input current and an LIF neuron with an adaptive threshold. The threshold is updated at each timestep based on the difference between the observed average firing rate and a learned target firing rate. The authors claim that, during inference, the proposed method avoids storing running normalization statistics and requires only addition and multiplication operations.

Critical strengths
  1. The paper presents an interesting alternative to conventional normalization in SNNs by dynamically adapting the firing threshold based on the observed firing rate.

  2. The proposed threshold controller is computationally simple during inference, requiring only additions and multiplications.

  3. The authors provide a theoretical formulation for the threshold adaptation mechanism and introduce a learnable target firing rate, which could reduce the need for manual threshold tuning.

  4. The method is evaluated on four datasets using two SNN architectures. The experiments show that HomeoLayer achieves performance comparable to conventional normalization methods in most evaluated settings.

Critical weaknesses
  1. The authors argue that BatchNorm introduces batch reductions and floating-point operations during inference. However, conventional BatchNorm uses running means and variances computed during training, making inference independent of the current batch. Additionally, these statistics can be absorbed into the weights or precomputed affine transformations, avoiding expensive operations. Therefore, the authors should clarify which computational overheads remain problematic after applying these standard inference optimizations.
  2. The authors argue that their approach eliminates the need for manual threshold tuning. However, in practice, the threshold is often set to a fixed value (usually 1), while the weight parameters are optimized during training. Additionally, their method introduces additional hyperparameters, such as the surrogate variance. A sensitivity analysis is needed to understand how the model's performance varies with the choice of these hyperparameters.
  3. This method introduces additional computations at every timestep to estimate the average firing rate and spike-flip statistics. These operations require collecting information across neurons within a layer. Could the authors comment on its energy efficiency compared to simple batchnorm when deployed on neuromorphic hardware?
Questions for authors

Check the weaknesses section.

2 comments
AuthorsOpenReview ↗

Reply to Reviewer 1

W1 (folded BatchNorm has no batch reduction, square root or division at inference). We agree, and we have rewritten the abstract, the introduction and Sec. 5.3 to scope the costs precisely. Time-invariant BatchNorm and tdBN fold into a per-neuron scale and shift at inference, with no batch reduction, square root or division. Table 3 already counted BNTT in that folded form, and its caption now states that. What survives folding is (i) per-timestep normalization (BNTT), which folds into TT separate scale-and-shift pairs, 4T4T stored values per neuron (700 at T=175T=175), fetched by timestep; (ii) LayerNorm, which cannot fold and evaluates a square root and a division per sample at every step; and (iii) the training-time batch reduction, square root and division of every normalizer, which on-chip or online learning must pay. Against folded time-invariant BatchNorm, HomeoLayer's inference cost is not lower: its affine is the same multiply-add, and the controller adds two spike counts per layer and step. We now state this in Sec. 5.3. Its advantages in that comparison are the removed threshold hyperparameter, training with no batch statistics, and the normalizer fixed point of Proposition 1. tdBN, the foldable time-invariant form, did not train in our setting (Sec. 5.1), which is why the long-stream literature uses BNTT.

W2 (the threshold is usually fixed at 1; sensitivity to the new hyperparameters). Fixing the threshold at 1 and training the weights works when a normalizer rescales the drive to that threshold; that is the premise we start from, and BNTT at θ=1\theta = 1 is our main baseline. Without normalization, θ=1\theta = 1 with trained weights trails HomeoLayer by 0.2 (SHD AttnSNN), 3.5 (SSC), 4.0 (GSC), 0.9 (SHD FFSNN) and 0.1 (N-MNIST) points (fixed-threshold appendix, which also shows that the best fixed value differs by task and that the spread across fixed values reaches 6 points on GSC). On the hyperparameters HomeoLayer introduces we agree a sensitivity analysis was missing. The new appendix varies each one over one to two decades, one factor at a time, on SHD AttnSNN at three seeds (canonical model/setting 93.95±0.3893.95 \pm 0.38):

HyperparameterPaperSweptAccuracy (%)Δ\Delta
Surrogate variance σξ,surr2\sigma^2_{\xi,\mathrm{surr}}0.10.0193.54±0.5193.54 \pm 0.51−0.41-0.41
194.51±0.3594.51 \pm 0.35+0.56+0.56
Controller gain ηθ\eta_\theta10194.60±0.5094.60 \pm 0.50+0.65+0.65
10089.47±0.4789.47 \pm 0.47−4.48-4.48
Initial threshold θ0\theta_010.594.13±0.1894.13 \pm 0.18+0.18+0.18
293.63±0.5593.63 \pm 0.55−0.32-0.32
Window TwT_w1 step5 steps94.05±0.4194.05 \pm 0.41+0.10+0.10
Target init (hidden / input)0.10 / 0.010.05 / 0.00593.11±0.7393.11 \pm 0.73−0.84-0.84
0.20 / 0.0293.70±0.4893.70 \pm 0.48−0.25-0.25

Every setting but the gain stays within a point of the canonical model, while the gain has leverage: ηθ=1\eta_\theta = 1 is above the paper's setting and ηθ=100\eta_\theta = 100 costs 4.5 points. We report this as found and keep the paper's value elsewhere for consistency. No gain floor and no target floor are used in any HomeoLayer models. The baselines' boxcar half-width (0.5) is a hyperparameter of the same kind that we, like the field, did not tune.

W3 (energy on neuromorphic hardware). We have not run HomeoLayer on neuromorphic hardware; we answer with an operation count in the primitives such hardware provides (new appendix). Under layer pooling the controller needs, per layer and timestep, two spike counts over the layer, one scalar update and one broadcast of the threshold. The counts are what a single accumulator neuron with unit weights computes from the layer's spike bus, so they are event-driven, one synaptic event per spike and one per flip; the flip statistic needs one bit of memory per neuron, and at batch size one the layer-pooled threshold is a single register per layer. Measured on the trained models, per timestep and summed over the hidden layers, against the NN dense bias additions a folded normalizer performs at every step:

DatasetModelNNspikes/stepflips/stepcontroller eventsratio to NN
SHDAttnSNN35003452495940.17
SSCAttnSNN35002552384940.14
GSCAttnSNN35003322876200.18
SHDFFSNN14002431954370.31
NMNISTFFSNN140091861770.13

Folded BNTT adds a timestep-indexed fetch of its 2N2N scale-and-shift values per step; LayerNorm evaluates a square root and a division per step. The controller is therefore of the same order as a folded bias, event-driven rather than dense and scaling with the firing rate, and far below any unfolded normalizer; a measured energy comparison requires an on-chip implementation we leave to future work.

I thank the authors for their detailed response. After reading their discussions with the other reviewers, I have decided to raise my score accordingly.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the three reviewers for careful readings that converge on the same points: the inference-cost claims were stated too broadly, the hyperparameters HomeoLayer introduces were not swept, and the hardware claim rested on state counts alone. The revised PDF rescopes the claims (abstract, introduction, Sec. 5.3, conclusion), corrects the three errors the reviewers caught (sign of Eq. 5, the rmin⁡r_{\min} formula, the double use of β\beta), and adds five appendices with new experiments, all at three seeds:

  1. sensitivity sweep over every hyperparameter the controller introduces (SHD AttnSNN, 9 settings ×\times 3 seeds);
  2. published convolutional network on DVS-Gesture at T=16T = 16 with its BatchNorm replaced by the controller;
  3. preliminary training regime without backpropagation, with a fixed-threshold control;
  4. operation count of the controller in hardware primitives, with measured event counts on the trained models;
  5. learned per-layer rate targets.

Inserted and modified text is set in blue in the revised PDF. The new tables are reproduced below each reviewer point they answer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.