acceptodds
Under review as a conference paper at ICLR 2027

TDAP: Temporal and Dependency Aware Pruning for Spiking Neural Networks

Abstract

Spiking Neural Networks (SNNs) provide an energy-efficient alternative to tradi tional Artificial Neural Networks for event-based vision tasks. However, due to substantial parameter redundancy, their deployment on resource-constrained neu romorphic hardware remains challenging. Existing post-training pruning meth ods for SNNs primarily treat each channel as an independent feature extractor, overlooking two critical characteristics of spike dynamics: (1) the temporal infor mativeness of information propagation across time steps, and (2) the predictive dependencies reflecting inter-channel relationships. To address these limitations, we propose TDAP (Temporal Dependency-Aware Pruning), a post-training pruning framework that discovers channel dependencies directly from calibration data without requiring fine-tuning. TDAP comprises two modules: (i) a Temporal Firing Rate module that encodes information arrival through learnable temporal weightings, and (ii) a Channel Cross-Correlation Dependency module that captures predictive relationships between channels via lagged correlation analysis with spike-specific statistics. We further introduce a geometric mean fusion strategy to aggregate the temporal and dependency mod ules. On the challenging CIFAR-100 dataset, TDAP attains 54.23% at 60% spar sity, surpassing pure temporal methods by up to 8.63pp. For event-based DVS CIFAR10, TDAP reaches 37.00% at 70% sparsity, exceeding Wanda by 13.30pp. These results validate that modeling temporal dependency and channel depen dency is essential for aggressive SNN compression.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spiking Neural Networks, Post-training Pruning, Model Compression, Event-based Vision, Neuromorphic Computing, Temporal Dependency

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Wq59OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TDAP, a post-training pruning framework for spiking neural networks. It evaluates weight importance by jointly modeling the temporal information of spikes and the predictive dependencies between channels. The method employs a learnable Temporal Firing Rate (TFR) module and a Channel Cross-Correlation Dependency (CCD) module to derive scores that are then fused via a geometric mean. This approach requires only a small amount of unlabeled calibration data and eliminates the need for fine-tuning the backbone network. Experiments were conducted on CIFAR-10, CIFAR-100, and DVS-CIFAR10.

Critical strengths
  1. By simultaneously accounting for the importance of time steps and the lagged predictive relationships between channels, the method provides a valuable complement to traditional metrics based on weight magnitude or static activation.

  2. The calibration cost is low, requiring only 128 unlabeled samples without the need for backbone fine-tuning.

Critical weaknesses
  1. The experiments are limited to three relatively small-scale datasets and the Spiking ResNet-19 architecture. The study lacks evaluations on large-scale datasets (such as ImageNet) or spiking Transformer architectures.

  2. The paper does not provide assessments of inference latency, memory footprint, or energy consumption, making it difficult to fully substantiate the value of deploying this method on existing computational hardware and frameworks.

  3. There is existing research in this field [1][2][3], yet the paper lacks a systematic discussion of these works and appropriate experimental comparisons.

  4. The proposed framework largely follows the "weight magnitude multiplied by channel-wise statistics" scoring paradigm established by Wanda. Consequently, its innovation is limited, and the necessity for the newly introduced design components is not sufficiently justified.

  5. The source of the 128 calibration samples, their specific data partition, and the sampling strategy used are not clearly defined. Furthermore, it is not specified whether the calibration data is strictly independent of the evaluation data.

[1] Dynamic Spatio-Temporal Pruning for Efficient Spiking Neural Networks.

[2] QP-SNNs: Quantized and Pruned Spiking Neural Networks.

[3] Towards Energy Efficient Spiking Neural Networks: An Unstructured Pruning Framework.

Questions for authors
  1. Please clarify the key differences between this work and existing SNN pruning studies.

  2. Additionally, provide relevant experimental and theoretical comparisons and explain the advantages of your method.

Reviewer_1ctyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes TDAP (Temporal and Dependency-Aware Pruning), a post-training pruning framework for spiking neural networks (SNNs). The method consists of two main modules: Temporal Firing Rate (TFR), which learns the importance of different time steps from calibration data, and Channel Cross-Correlation Dependency (CCD), which estimates predictive dependencies between channels using lagged cross-correlation, firing density, and temporal stability. The two importance scores are combined using geometric mean fusion to determine which weights to prune, without fine-tuning the backbone network. The authors evaluate TDAP on CIFAR-10, CIFAR-100, and DVS-CIFAR10, comparing it with several pruning baselines and conducting ablation studies. The paper also provides theoretical analyses of dependency-aware pruning and geometric mean fusion.

Critical strengths

The paper addresses a relevant problem in SNN compression by considering both temporal information and inter-channel dependencies. The proposed combination of temporal weighting and lagged cross-correlation provides a reasonable approach to developing pruning criteria tailored to SNNs.

Critical weaknesses

The experimental results contain significant inconsistencies. In particular, the unpruned accuracy and the Magnitude baseline results reported in Tables 2 and 6 differ substantially under apparently comparable settings. These discrepancies undermine confidence in the ablation results and require clarification before the conclusions can be reliably assessed.

Questions for authors

What explains the discrepancies between Tables 2 and 6? Specifically, why does the unpruned CIFAR-100 model achieve 65.76% accuracy in Table 2 but 62.52% in Table 6, and why does the Magnitude baseline achieve 4.95% accuracy at 70% sparsity in Table 6 compared with 36.71% in Table 2? Were the same model checkpoints and experimental configurations used?

Reviewer_vHbAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a post-training pruning method for Spiking Neural Networks (SNNs) that models temporal and inter-channel dependencies to achieve high-sparsity SNN compression. The effectiveness of the proposed method is further validated through experimental results.

Critical strengths
  1. This paper proposes TDAP, a practical post-training pruning framework for SNNs that requires only 128 unlabeled calibration samples and does not require backbone fine-tuning.

  2. By integrating the Temporal Firing Rate (TFR) and Channel Cross-Correlation Dependency (CCD) modules through geometric mean fusion, TDAP provides a unified criterion for weight importance estimation that jointly considers temporal dynamics and inter-channel dependencies.

Critical weaknesses

1.Limited effectiveness of temporal-spatial fusion. On CIFAR-100, TDAP-Geo consistently underperforms the standalone CCD module, with a performance gap of 6.20 percentage points at 70% sparsity. This raises concerns about the effectiveness of incorporating temporal information into the proposed fusion strategy.

2.Insufficient comparisons with recent SNN compression methods. Although the introduction claims that TDAP outperforms recent SNN-specific compression methods, the experimental baselines are limited to magnitude pruning, WandaFixed, and variants of the proposed framework. The lack of direct comparisons with recent methods weakens the empirical support for this claim.

3.Inconsistency in the theoretical formulation. The entropy regularization terms in Eq. 8 and Eq. 26 appear to have inconsistent signs, raising concerns about the correctness of the optimization objective and its theoretical derivation.

Questions for authors

1.Why does incorporating temporal information degrade performance on CIFAR-100 compared with the standalone CCD module? Under what conditions is the proposed fusion strategy expected to provide improvements?

2.Could the authors provide comparisons with recent SNN-specific compression methods under matched evaluation settings to substantiate the performance claims made in the introduction?

3.Could the authors clarify the sign inconsistency between Eq. 8 and Eq. 26, specify the actual optimization objective used in the experiments, and explain whether this inconsistency affects the theoretical derivation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.