acceptodds
Under review as a conference paper at ICLR 2027

Membrane-Aware Dynamic Sparse Training for Federated Spiking Neural Networks

Abstract

Federated dynamic sparse training has emerged as a promising paradigm for distributed learning with reduced communication overhead. However, existing approaches inherit connection importance estimation criteria from artificial neural networks and overlook the membrane state of postsynaptic neurons, which determines spike emission in spiking neural networks (SNNs). This mismatch leads to inaccurate importance estimation and suboptimal sparse rewiring in federated SNNs. To address this challenge, we propose Membrane-Aware Dynamic Sparse Training (MADST), which introduces three mechanisms: (1) membrane-gated pruning that scores active connections by postsynaptic threshold proximity; (2) activity-guided growth that couples presynaptic spike activity with postsynaptic firing readiness; and (3) support-first server alignment that reconciles heterogeneous client masks under a fixed per-layer density budget. Experiments on CIFAR-10 and CIFAR-100 with Spiking ResNet-18 show that MADST attains the highest accuracy in 10 of the 12 matched-density sparse settings under IID and non-IID conditions, with accuracy gains of up to 1.23 percentage points over the strongest baseline.

Primary area:
optimization
Keywords:
Federated Learning, Spiking Neural Networks, Dynamic Sparse Training, Sparse Neural Networks, Neural Network Pruning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_goQ1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Membrane-Aware Dynamic Sparse Training (MADST) for federated spiking neural networks. It uses a neuron’s proximity to its firing threshold to adjust connection-pruning scores. New connections are selected based on presynaptic spikes occurring when the target neuron is near its threshold. The server combines client masks by prioritizing connections supported by more clients while maintaining a fixed density in each layer. Experiments on CIFAR-10 and CIFAR-100 with Spiking ResNet-18 show the highest accuracy in 10 of 12 matched-density settings.

Critical strengths

1. Incorporating membrane state into connection selection is well motivated for SNNs. The growth rule considers whether a source spike occurs when the target neuron is sensitive to additional input, providing a more specific criterion than firing activity alone.

2. The matched-density comparisons show promising improvements over existing sparse-training methods. The component ablations also suggest that server-side mask reconciliation matters, particularly on CIFAR-100 under non-IID data. Keeping membrane statistics local avoids introducing a separate communication requirement for these signals.

Critical weaknesses

1. The evidence for the proposed activity-coupling mechanism is not sufficiently convincing. The growth ablation compares against random growth rather than simpler activity-based scores, so it does not isolate the value of same-time spike–membrane coincidence. Tables 1–3 also lack variability estimates. Together, these limitations make it difficult to determine whether the central mechanism provides a reliable improvement.

2. The practical efficiency and federated robustness remain insufficiently established. The communication analysis counts weight values without quantifying mask or index overhead, and the additional training time and memory required for growth scoring are not measured. Moreover, evaluation uses only ten fully participating clients, leaving unclear whether support-first alignment remains effective with a larger population or partial participation. These gaps limit the evidence for deployment on resource-constrained federated systems.

Questions for authors

1. How do the membrane time constant τm\tau_m and surrogate slope α\alpha affect the importance scores? Does a larger time constant make the omitted lagged contributions important enough to change which connections should be grown?

2. Can you compare the joint growth score with the product of average presynaptic activity and average postsynaptic membrane sensitivity? How much improvement comes specifically from their same-time coincidence?

3. How does support-first alignment behave under partial client participation and stronger data heterogeneity? Does prioritizing majority support remove connections that are useful mainly to clients with less common classes?

4. Does the communication analysis account for the bit overhead of transmitting masks or indices? Since growth scoring requires additional computation and a weight-shaped buffer, how much local training time and peak memory does it add?

5. What motivates the choice of ten fully participating clients? Does MADST retain its advantage with a larger client population, such as 100 clients, particularly when only a subset participates in each round?

6. The heading formatting in Section 2 is inconsistent. “Dynamic Sparse Training” appears on a separate line, while the other headings run into the following text.

Reviewer_tbpGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MADST, which uses the degree of membrane potential approaching the threshold for sparse connection updating of federated SNNs. Adjust Taylor's importance using the average surrogate derivative during the pruning stage; The synchronous activity score of presynaptic discharge and postsynaptic threshold sensitivity during the growth phase; The server prioritizes selecting connections based on the number of supported clients, and then performs mask weighted aggregation to maintain a fixed density for each layer. The experiment used Spiking ResNet-18 and compared it on two CIFAR datasets, two data partitions, and three densities.

Critical strengths
  1. The motivation for the method is specific, and the explanation is relatively restrained. At page 5, it is explicitly acknowledged that the surrogate grade already contains threshold information, and the role of the newly added gate is positioned as budget allocation across target neurons; It was not mistakenly referred to as restoring gradient completely lost information.

  2. Fixed density and clear data transmission boundaries. Server Top-K ensures precise density, while membrane potential statistics are kept on the client side. The "10 wins out of 12 settings and maximum improvement of 1.23 percentage points" in Table 1-2 is consistent with the table, and the paper also discusses two failed settings.

Critical weaknesses
  1. The statistical credibility of accuracy returns is insufficient, and the selection of peak values in the test set further increases uncertainty. Section 4.1 clearly reports the highest test accuracy in 500/800 rounds, but does not provide the mean, variance, or confidence interval for multiple random seeds. Multiple leading margins of only 0.08, 0.22, and 0.27 percentage points make it difficult to determine whether they are stable.

  2. The closest existing mechanism lacks direct control, and the contribution of membrane potential is not sufficiently isolated. There is criticality based pruning [1] for measuring neuronal criticality using surrogate derivative, and FedDST [2] for mask aggregation and client-side voting. The author acknowledges that these precedents are advantageous, but the experiment still needs to prove the independent increment after combination.

  3. Avoiding task gradients that calculate inactive weights does not mean that training calculations are cheap At present, there is a lack of training time per round, scoring overhead, peak memory, and total cost required to achieve the same accuracy. The inference energy consumption estimate provided in the paper cannot replace these training efficiency evidences.

ref: [1] Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networks [2] Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better

Questions for authors
  1. After selecting rounds using the validation set and repeating the experiment, is the 10/12 lead still stable?

  2. Is MADST superior to existing critical pruning and gradient growth after fixing SFA? Is synchronous joint activities really better than marginal statistics?

  3. What is the total cost of achieving the same accuracy rate, including score calculation, buffer and mask transmission, under partial client participation conditions?

Reviewer_4gB2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Membrane-Aware Dynamic Sparse Training (MADST), a framework that integrates dynamic sparse training with federated SNNs. The method introduces (1) membrane-gated pruning (2) activity-guided growth and (3) support-first server alignment. Experiments on CIFAR-10 and CIFAR-100 using Spiking ResNet-18 show that the proposed method achieves improvements over existing baselines under both IID and non-IID settings.

Critical strengths

The authors consider both membrane potential dynamics and average spike activity for sparse connection selection, rather than directly adopting existing sparse federated learning methods.

The paper introduce comprehensive evaluations based on software simulations of SNN's performacne and efficiency. Metrics inlcudes classification accuracy, communication costs and estimates energy consumption, offering additional perspectives on the efficiency of the proposed framework.

Critical weaknesses

One of the major problem is that the general motivation combining federated learning, dynamic sparse training, and SNNs is unconvincing, while also the evaluation ceritera does not match the general setting of federate learning. One of the primary advantages of SNNs is energy-efficient computation on resource-constrained edge devices. Therefore, the authors should clarify the practical system architecture of the evaluation. For example, how are sparse mask and gradient communicated between edge devices and the central server. How to deal with the potential numerical precision problem between edge devices and serve. How are training and model updates synchronized across edge devices (or even async neuromorphic chips).

Moreoever, the overall evaluation is insufficient and the ablation results are unconvincing. Specifically, the experimental results do not convincingly demonstrate the effectiveness of the proposed membrane-aware mechanisms. For example, in table 3, removing both MGP and AGG results in accuracy drops of only 0.19% (on CIFAR-10), which seems to be marginal in absence of the report standard deviations or confidence intervals. There are also counterintuitive ablation results on CIFAR-10, non-IID, as removing both MGP and AGG leads to a smaller accuracy drop than removing AGG or MGP individually.

Questions for authors

See in weaknesses

Comments on the paper

1 comment

Thank authors for the response and the clarifications. However, the rebuttal provides no new experimental evidence for its claims. Most of the works are described as planned work. The communication savings are analytical estimates. The only reported training-time and memory profiling, comes from a separate implementation and does not use the same setting as paper. Since the core concerns remain unresolved, I will maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.