acceptodds
Under review as a conference paper at ICLR 2027

TRIM: Temporal-subspace Reconstruction Induced by Moments for Post-Training Structured Pruning of Spiking Neural Networks

Abstract

Spiking neural networks (SNNs) offer energy-efficient computation through sparse and event-driven processing, but their growing scale imposes substantial storage and computational burdens. Existing structured pruning methods for SNNs typically rely on sparsity-aware training or post-pruning fine-tuning, making model compression expensive. To our knowledge, we propose the first post-training structured pruning framework for SNNs that requires neither retraining nor fine-tuning. We begin by establishing a surrogate membrane-potential reconstruction baseline that uses a small calibration set to guide structure selection and weight compensation. However, under tight structural budgets and given the distinctive dynamics of SNNs, accurately reconstructing full temporal trajectories would require the retained structures to capture both low-order temporal components and fine-grained temporal variations. To address this challenge, we propose TRIM, namely Temporal-subspace Reconstruction Induced by Moments, which projects surrogate membrane potential trajectories onto a fixed temporal subspace that captures their mean level and linear temporal trend. This shifts the reconstruction objective toward preserving these low-order temporal components rather than matching entire trajectories. Experiments across diverse visual SNNs and spiking language models demonstrate competitive accuracy–compression trade-offs without any fine-tuning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Model compression, Post-training structured pruning, SNN

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_j2aDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TRIM, a structured pruning method for SNNs that requires no fine-tuning. It selects channels and compensates the remaining weights by matching the mean and linear trend of surrogate membrane potentials.

Critical strengths

The paper addresses a practical problem, compressing pretrained SNNs without expensive retraining. Considering channel selection together with weight compensation is sensible. The experiments cover both vision and language models, with clear improvements over the full-trajectory baseline in several VGG settings.

Critical weaknesses

1.Table 7 lists two time steps for ResNet20. In this case, the two temporal bases retain the full temporal information, so TRIM and the baseline should be equivalent under identical settings. Their different results in Table 1 need explanation. 2.The paper should specify whether methods use identical calibration samples and report repeated runs, especially where the improvements are small. 3.Changing the objective affects both channel selection and weight compensation. The current analysis does not separate these effects. 4.The comparison with the full-trajectory baseline is useful, but it does not sufficiently justify the choice of mean and linear trend. Comparisons with mean-only reconstruction, alternative two-dimensional bases, and higher-dimensional subspaces would clarify the contribution of each design choice.

Questions for authors

See Weakness

6 comments
AuthorsOpenReview ↗

Response to W1

W1: With T=2, the two temporal bases preserve all temporal information. Why do TRIM and the full-trajectory baseline produce different results in Table 1?

We thank the reviewer for identifying this special case. The reviewer is correct: at T=2, the normalized constant and linear components form a complete orthonormal basis. Projecting onto both components is therefore an orthogonal change of coordinates, not dimensionality reduction. With identical inputs, normalization, and regularization, TRIM and the full-trajectory baseline optimize equivalent objectives in exact arithmetic. The temporal-subspace distinction applies only when T>2.

The two implementations nevertheless follow different numerical paths. Both accumulate calibration statistics and apply compensating weight updates in FP32, but the baseline uses the full surrogate membrane-potential trajectories, whereas TRIM first projects them onto the temporal components. These algebraically equivalent operations need not produce identical floating-point results because their operation orders and rounding differ.

Our implementation checks found different FP32 statistics for identical inputs and different compensated weights even when early-layer channel selections agreed. These weight differences can change whether neurons near the firing threshold emit spikes, affecting the inputs used to calibrate subsequent layers. Meanwhile, small differences between closely ranked candidate scores can change the greedy deletion order. We observed subsequent differences in spikes and channel selections, consistent with numerical differences propagating through layer-wise pruning. Thus, identical precision and seeds do not guarantee identical pruned networks across the two implementations.

To assess the resulting variation, we repeated ResNet20 experiments on CIFAR-10 and CIFAR-100 with 10 calibration seeds and 4,096 samples. For each dataset, the pretrained checkpoint was fixed. Within each seed, both methods used identical calibration samples and ordering, with pruning and evaluation settings aligned. All results below are top-1 accuracy (%) reported as mean ± standard deviation over 10 seeds.

Table R1. ResNet20 at T=2.

Pruning ratioCIFAR-10CIFAR-100
BaselineTRIMBaselineTRIM
10%95.90 ± 0.1295.92 ± 0.1377.98 ± 0.1678.01 ± 0.17
20%95.57 ± 0.1195.56 ± 0.1276.28 ± 0.1576.25 ± 0.16
30%95.12 ± 0.1395.14 ± 0.1173.85 ± 0.1873.90 ± 0.16
40%93.92 ± 0.1393.89 ± 0.1469.98 ± 0.1669.95 ± 0.16
50%91.87 ± 0.1291.85 ± 0.1363.30 ± 0.1663.90 ± 0.17

TRIM does not achieve higher mean accuracy in every setting. More importantly, regardless of which implementation performs better, differences at T=2 cannot demonstrate benefits from temporal-subspace selection because no temporal component is discarded.

We therefore additionally evaluated ResNet19 at T=4, where the two retained components span a strictly lower-dimensional subspace. This comparison tests the method outside the equivalence case raised by the reviewer. For each dataset, OBC, the baseline, and TRIM used the same pretrained checkpoint, calibration budget, pruning ratios, and 10 calibration seeds.

Table R2. Additional ResNet19 results at T=4.

Pruning ratioCIFAR-10CIFAR-100
OBCBaselineTRIMOBCBaselineTRIM
10%94.27 ± 0.0894.34 ± 0.0794.41 ± 0.0774.28 ± 0.1074.40 ± 0.1174.53 ± 0.09
20%93.55 ± 0.0693.75 ± 0.0794.05 ± 0.0673.22 ± 0.1573.53 ± 0.1573.75 ± 0.15
30%92.71 ± 0.0793.07 ± 0.0693.45 ± 0.0772.01 ± 0.1572.34 ± 0.1372.70 ± 0.15
40%91.67 ± 0.0792.01 ± 0.0792.71 ± 0.0769.15 ± 0.1670.16 ± 0.1571.09 ± 0.17
50%88.89 ± 0.0789.73 ± 0.0990.68 ± 0.0866.76 ± 0.1567.63 ± 0.1468.92 ± 0.13

At T=4, TRIM achieves higher mean accuracy than both alternatives at every evaluated pruning ratio on both datasets. At 50% pruning, it improves over the baseline by 0.95 and 1.29 percentage points on CIFAR-10 and CIFAR-100, respectively, and over OBC by 1.79 and 2.16 points. These results support TRIM in a setting where its temporal-subspace objective genuinely differs from full-trajectory reconstruction.

We will explicitly state the T=2 equivalence and the T>2 dimensionality reduction in Section 3.3, and revise the Table 1 discussion so that T=2 differences are not presented as evidence of subspace benefits. We will also include the repeated ResNet20 results and the additional ResNet19 comparison, with numerical precision, statistics computation, calibration sampling, and evaluation settings documented in the appendix.

AuthorsOpenReview ↗

Response to W2

W2: Do the methods use identical calibration samples, and are the reported improvements supported by repeated runs, especially when the differences are small?

We thank the reviewer for requesting this clarification. Calibration-set sizes were selected using the validation set, not the test set, and the same selection procedure was applied to TRIM and all competing methods. In the ResNet20 comparison described in W1, both methods used identical calibration samples and ordering within each seed. The additional ResNet19 comparison used the same calibration budget and seed set across methods.

We repeated pruning over 10 calibration seeds and report the available results below as top-1 accuracy (%) with mean ± standard deviation. These results cover VGG16 on three datasets, SEW-ResNet18 on ImageNet, and ResNet19 and ResNet20 on both CIFAR datasets. These runs vary calibration seeds, rather than retraining the dense models.

Table R3. Repeated-run results on VGG16 and SEW-ResNet18.

ModelDatasetPruning ratioOBCBaselineTRIM
VGG16CIFAR-1010%92.47 ± 0.0792.67 ± 0.0693.13 ± 0.04
20%90.81 ± 0.0890.96 ± 0.0791.36 ± 0.06
30%88.60 ± 0.0787.55 ± 0.0688.91 ± 0.07
40%81.97 ± 0.0681.13 ± 0.0584.10 ± 0.06
50%69.41 ± 0.1368.47 ± 0.0974.42 ± 0.14
VGG16CIFAR-10010%69.07 ± 0.0869.40 ± 0.0769.61 ± 0.06
20%64.81 ± 0.0964.30 ± 0.0865.81 ± 0.08
30%58.46 ± 0.0755.33 ± 0.0858.54 ± 0.06
40%45.76 ± 0.0839.08 ± 0.0748.52 ± 0.07
VGG16TinyImageNet10%53.91 ± 0.0655.16 ± 0.0455.57 ± 0.05
20%44.95 ± 0.0547.52 ± 0.0649.91 ± 0.07
30%30.61 ± 0.0834.83 ± 0.0741.42 ± 0.08
SEW-ResNet18ImageNet10%59.97 ± 0.2159.77 ± 0.1659.92 ± 0.19
20%54.86 ± 0.1354.68 ± 0.1455.52 ± 0.15
30%47.92 ± 0.1448.51 ± 0.1549.82 ± 0.13
40%36.79 ± 0.1840.47 ± 0.1642.34 ± 0.17
50%23.61 ± 0.1929.22 ± 0.1531.87 ± 0.16

The larger improvements remain well above the observed seed-to-seed variation. On VGG16/CIFAR-10 at 10% pruning, TRIM exceeds OBC and the baseline by 0.66 and 0.46 percentage points, respectively. At 20% pruning on TinyImageNet, the corresponding gains are 4.96 and 2.39 points; at 50% pruning on ImageNet, they are 8.26 and 2.65 points. Thus, the improvements are not limited to small differences in individual runs.

We also distinguish these gains from close comparisons. On ImageNet at 10% pruning, OBC and TRIM achieve 59.97 ± 0.21% and 59.92 ± 0.19%, respectively. Similarly, TRIM's 0.08-point lead over OBC on VGG16/CIFAR-100 at 30% pruning is small. We do not interpret these cases as clear advantages for TRIM.

Table R4. Additional ResNet19 results, T=4.

Pruning ratioCIFAR-10CIFAR-100
OBCBaselineTRIMOBCBaselineTRIM
10%94.27 ± 0.0894.34 ± 0.0794.41 ± 0.0774.28 ± 0.1074.40 ± 0.1174.53 ± 0.09
20%93.55 ± 0.0693.75 ± 0.0794.05 ± 0.0673.22 ± 0.1573.53 ± 0.1573.75 ± 0.15
30%92.71 ± 0.0793.07 ± 0.0693.45 ± 0.0772.01 ± 0.1572.34 ± 0.1372.70 ± 0.15
40%91.67 ± 0.0792.01 ± 0.0792.71 ± 0.0769.15 ± 0.1670.16 ± 0.1571.09 ± 0.17
50%88.89 ± 0.0789.73 ± 0.0990.68 ± 0.0866.76 ± 0.1567.63 ± 0.1468.92 ± 0.13

TRIM has higher mean accuracy at every evaluated ratio on both datasets. At 50% pruning, its gains over OBC are 1.79 and 2.16 points on CIFAR-10 and CIFAR-100, respectively, and its gains over the baseline are 0.95 and 1.29 points. These repeated experiments extend the evidence to an additional residual architecture.

Table R5. ResNet20 results, T=2.

Pruning ratioCIFAR-10CIFAR-100
BaselineTRIMBaselineTRIM
10%95.90 ± 0.1295.92 ± 0.1377.98 ± 0.1678.01 ± 0.17
20%95.57 ± 0.1195.56 ± 0.1276.28 ± 0.1576.25 ± 0.16
30%95.12 ± 0.1395.14 ± 0.1173.85 ± 0.1873.90 ± 0.16
40%93.92 ± 0.1393.89 ± 0.1469.98 ± 0.1669.95 ± 0.16
50%91.87 ± 0.1291.85 ± 0.1363.30 ± 0.1663.90 ± 0.17

As discussed in W1, the T=2 objectives are mathematically equivalent, so these accuracy differences are not evidence of benefits from temporal-subspace selection. We include the repeated results for completeness and keep this implementation-consistency check separate from the T>2 comparisons.

We will include these repeated-run results in the revised manuscript and clarify the calibration-set selection and sampling protocols. Reporting means and standard deviations will make the substantial gains and close comparisons explicit.

AuthorsOpenReview ↗

Response to W3

W3: Changing the reconstruction objective affects both channel selection and weight compensation. The current analysis does not separate their contributions.

We thank the reviewer for suggesting this analysis. We evaluated all four combinations of channel selection and weight compensation using the full-trajectory baseline and TRIM. This allows us to compare compensation objectives with the selected channels fixed, and channel selections under the same compensation objective. The experiments cover VGG16 and ResNet19 on CIFAR-100, both with T=4.

For each cross-combination, we fixed the channel sets selected by the source method and recomputed compensation from the original pretrained checkpoint, without reselecting channels or fine-tuning.

Table R6. Channel-selection and weight-compensation ablation on CIFAR-100. Top-1 accuracy (%) at different pruning ratios.

ModelChannel selectionWeight compensation10%20%30%40%50%
VGG16BaselineBaseline69.4164.2655.1739.03—
TRIMBaseline69.4764.3555.4841.26—
BaselineTRIM69.5265.5557.3245.49—
TRIMTRIM69.5765.7858.5048.53—
ResNet19BaselineBaseline74.4073.5372.3470.1667.63
TRIMBaseline74.3373.5772.3670.2867.97
BaselineTRIM74.4173.6772.5170.6468.31
TRIMTRIM74.5373.7572.7071.0968.92

The cross-combinations are additional ablation results; the complete baseline and TRIM entries are the previously reported references. “—” denotes a setting not reported here.

The compensation comparison shows that TRIM's gains do not depend solely on its channel choices. On VGG16 at 40% pruning, replacing baseline compensation with TRIM compensation improves accuracy from 39.03% to 45.49% with baseline-selected channels fixed, a gain of 6.46 percentage points. With TRIM-selected channels fixed, the same replacement improves accuracy from 41.26% to 48.53%, a gain of 7.27 points. Temporal-subspace compensation therefore improves the reported accuracy with either channel selection.

Channel selection also contributes. With TRIM compensation fixed, using TRIM-selected rather than baseline-selected channels improves VGG16 accuracy from 45.49% to 48.53%, a further 3.04 points. The ResNet19 comparison shows a similar pattern: at 50% pruning, replacing baseline compensation with TRIM compensation on baseline-selected channels improves accuracy from 67.63% to 68.31%; using TRIM-selected channels with the same compensation objective further improves it to 68.92%.

Across both architectures and all reported pruning ratios, baseline selection with TRIM compensation outperforms TRIM selection with baseline compensation, while complete TRIM achieves the highest reported accuracy. These results identify a substantial contribution from compensation and an additional benefit from channel selection, rather than attributing the improvement to either component alone. We will add this selection–compensation ablation to the experimental section of the revised manuscript.

Reviewer_uPJnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper the authors propose TRIM a method for pruning SNNs post training without fine tuning. It removes channels using a small calibration set and compensates by adjusting the remaining weights, focusing on preserving the mean level and linear trend of the membrane potential over time rather than the full trajectory. Across 34 vision settings, TRIM is the best method in 29, with big gains in harder cases (e.g., +8.10 points over OBC on ImageNet at 50% pruning). On a spiking language model at 70% pruning it keeps a mean score of 72.2 compared with 77.2 unpruned.

Critical strengths

• TRIM only uses a small calibration set and requires no fine tuning. To prune a 110M parameter model it only takes between 3.87s-8.60s. This matters as re-training can become very expensive for language models. • TRIM is tested on a wide range of models. It outperforms MP, Wanda-sp, QP-SNN and FLAP in all settings, and beats the strongest competitor OBC by large margins in harder settings ( TinyImageNet at 30% and on ImageNet-1K at 50% pruning).

Critical weaknesses
  1. At light pruning OBC is very close to TRIM and sometimes better. Without error bars it’s hard to tell whether these differences are real.
  2. TRIM performs better than OBC at heavy pruning, but that’s where both methods lose substantial accuracy. It’s unclear how often anyone would deploy a model pruned that heavily.
  3. The introduction motivates structured pruning by its wall-clock speedups, but no latency is reported.
  4. One of the paper’s main points is avoiding fine tuning, but it never shows how much accuracy a short fine tune would recover or what it would cost.
Questions for authors

• How much accuracy does a short fine tune recover after pruning, and how much does it cost? If it recovers most of the accuracy cheaply, the benefit of avoiding fine tuning is smaller. • Could TRIM give a better starting point for fine tuning than other pruning methods like OBC? • Can you report error bars across to show whether the differences between TRIM, OBC and the Baseline are significant? • Does the error breakdown in Figure 2a hold for other models and datasets? For example, on Spikingformer on CIFAR-100, the Baseline beats TRIM at 20% and 30% pruning. Could you show the same breakdown for Spikingformer, to explain this result? • Can you report measured latency for the pruned models?

6 comments
AuthorsOpenReview ↗

Response to W1 and Q3

W1 & Q3: Are the differences between TRIM, OBC, and the baseline reliable, particularly at low pruning ratios?

We thank the reviewer for requesting this comparison. We repeated the experiments over 10 calibration seeds and report top-1 accuracy (%) as mean ± standard deviation below. Table R1 presents results for VGG16 on CIFAR-10, CIFAR-100, and TinyImageNet, and ResNet18 on ImageNet. We additionally evaluated ResNet19 on both CIFAR datasets to examine whether the findings extend to another residual architecture.

Table R1. Multi-seed results on the original vision benchmarks.

ModelDatasetPruning ratioOBCBaselineTRIM
VGG16CIFAR-1010%92.47 ± 0.0792.67 ± 0.0693.13 ± 0.04
20%90.81 ± 0.0890.96 ± 0.0791.36 ± 0.06
30%88.60 ± 0.0787.55 ± 0.0688.91 ± 0.07
40%81.97 ± 0.0681.13 ± 0.0584.10 ± 0.06
50%69.41 ± 0.1368.47 ± 0.0974.42 ± 0.14
VGG16CIFAR-10010%69.07 ± 0.0869.40 ± 0.0769.61 ± 0.06
20%64.81 ± 0.0964.30 ± 0.0865.81 ± 0.08
30%58.46 ± 0.0755.33 ± 0.0858.54 ± 0.06
40%45.76 ± 0.0839.08 ± 0.0748.52 ± 0.07
VGG16TinyImageNet10%53.91 ± 0.0655.16 ± 0.0455.57 ± 0.05
20%44.95 ± 0.0547.52 ± 0.0649.91 ± 0.07
30%30.61 ± 0.0834.83 ± 0.0741.42 ± 0.08
ResNet18ImageNet10%59.97 ± 0.2159.77 ± 0.1659.92 ± 0.19
20%54.86 ± 0.1354.68 ± 0.1455.52 ± 0.15
30%47.92 ± 0.1448.51 ± 0.1549.82 ± 0.13
40%36.79 ± 0.1840.47 ± 0.1642.34 ± 0.17
50%23.61 ± 0.1929.22 ± 0.1531.87 ± 0.16

The repeated experiments confirm that some comparisons are close. At 10% pruning on ImageNet, OBC and TRIM achieve 59.97 ± 0.21% and 59.92 ± 0.19%, respectively; this is comparable performance rather than a clear advantage for either method. Likewise, TRIM's 0.08-point lead over OBC on VGG16/CIFAR-100 at 30% pruning is small. We therefore do not claim a clear improvement in every setting.

However, the larger gains remain well above the observed seed-to-seed variation. At 20% pruning on VGG16/CIFAR-100, TRIM exceeds OBC and the baseline by 1.00 and 1.51 percentage points, respectively. On TinyImageNet at the same ratio, the gains are 4.96 and 2.39 points. At 30% pruning on TinyImageNet and 50% on ImageNet, the gains over OBC reach 10.81 and 8.26 points, respectively. Thus, the results distinguish near-ties from substantial improvements, rather than relying on small single-run differences.

Table R2. Additional ResNet19 results, T=4. Mean ± standard deviation over 10 calibration seeds.

Pruning ratioCIFAR-10CIFAR-100
OBCBaselineTRIMOBCBaselineTRIM
10%94.27 ± 0.0894.34 ± 0.0794.41 ± 0.0774.28 ± 0.1074.40 ± 0.1174.53 ± 0.09
20%93.55 ± 0.0693.75 ± 0.0794.05 ± 0.0673.22 ± 0.1573.53 ± 0.1573.75 ± 0.15
30%92.71 ± 0.0793.07 ± 0.0693.45 ± 0.0772.01 ± 0.1572.34 ± 0.1372.70 ± 0.15
40%91.67 ± 0.0792.01 ± 0.0792.71 ± 0.0769.15 ± 0.1670.16 ± 0.1571.09 ± 0.17
50%88.89 ± 0.0789.73 ± 0.0990.68 ± 0.0866.76 ± 0.1567.63 ± 0.1468.92 ± 0.13

The additional ResNet19 experiments show higher mean accuracy for TRIM across all five pruning ratios on both datasets, with increasing margins at higher pruning ratios. At 50% pruning, TRIM improves over OBC by 1.79/2.16 points and over the baseline by 0.95/1.29 points on CIFAR-10/100. These results extend the evidence beyond the architectures in the original comparison.

We will include these 10-seed results and the additional ResNet19 experiments in the revised manuscript, reporting means and standard deviations and distinguishing comparable performance from clear improvements.

AuthorsOpenReview ↗

Response to W2, W4, and Q1

W2, W4, and Q1: Are the gains at high pruning ratios practically useful, and how much accuracy can post-pruning fine-tuning recover at what cost?

We thank the reviewer for raising these related questions. We evaluated post-pruning fine-tuning on both vision and language models to examine accuracy recovery alongside its computational cost. The results show that TRIM provides a strong training-free option while also remaining effective when additional fine-tuning is affordable. All times below are wall-clock fine-tuning times using eight GPUs, reported per method.

Table R3. SpikingLM-110M on GLUE. Entries show the mean score before → after fine-tuning. The unpruned model scores 77.2.

Pruning ratioWanda-spTRIMFine-tuning time
10%74.9 → 75.676.0 → 76.415.0 h
30%68.3 → 71.075.1 → 75.713.6 h
50%60.3 → 65.474.4 → 75.212.1 h
70%52.7 → 60.172.2 → 73.710.3 h

TRIM without fine-tuning outperforms fine-tuned Wanda-sp at every reported pruning ratio, by 0.4, 4.1, 9.0, and 12.1 points, respectively. At 70% pruning, Wanda-sp recovers 7.4 points after 10.3 hours of fine-tuning on eight GPUs, but its resulting score of 60.1 remains well below TRIM's 72.2 without any fine-tuning. TRIM can further improve to 73.7 when fine-tuning is available. Thus, although fine-tuning helps, it does not eliminate the advantage of preserving model quality during pruning. The additional training cost is already substantial at 110M parameters, motivating training-free compression as spiking language models grow larger.

For vision models, we evaluated a longer recovery budget of 50 epochs for all three methods.

Table R4. Vision results before → after 50 epochs of fine-tuning. Top-1 accuracy (%). Unpruned accuracies are 59.08% for VGG16/TinyImageNet, 63.27% for SEW-ResNet18/ImageNet, and 74.47% for ResNet19/CIFAR-100.

Model / DatasetPruning ratioOBCBaselineTRIMFine-tuning time
VGG16 / TinyImageNet10%53.91 → 57.055.16 → 57.555.57 → 57.72.0 h
20%44.95 → 53.447.52 → 54.549.91 → 55.41.8 h
30%30.61 → 47.734.83 → 49.441.42 → 52.01.5 h
SEW-ResNet18 / ImageNet10%59.97 → 61.659.77 → 61.859.92 → 62.08.0 h
20%54.86 → 59.554.68 → 59.855.52 → 60.27.5 h
30%47.92 → 57.048.51 → 57.449.82 → 57.97.0 h
40%36.79 → 52.740.47 → 54.142.34 → 54.96.5 h
50%23.61 → 47.429.22 → 49.731.87 → 50.76.0 h
ResNet19 / CIFAR-10010%74.28 → 74.474.40 → 74.474.53 → 74.51.2 h
20%73.22 → 73.873.53 → 74.073.75 → 74.21.0 h
30%72.01 → 73.372.34 → 73.572.70 → 73.80.9 h
40%69.15 → 72.370.16 → 72.771.09 → 73.10.8 h
50%66.76 → 71.167.63 → 71.668.92 → 72.30.7 h

Fine-tuning recovers substantial accuracy on vision models, and TRIM achieves the highest post-fine-tuning accuracy in all 13 reported settings under the same epoch budget. At 30% pruning on TinyImageNet, TRIM reaches 52.0%, exceeding OBC and the baseline by 4.3 and 2.6 percentage points. At 50% pruning on ImageNet, its corresponding advantages are 3.3 and 1.0 points. These results also support TRIM as an effective starting point for further fine-tuning.

Regarding practical pruning ratios, TRIM's usefulness is not restricted to aggressive compression. Without fine-tuning, ResNet19 retains 73.75% accuracy at 20% pruning, only 0.72 points below the unpruned model. SpikingLM retains a GLUE mean of 75.1 at 30% pruning, compared with 77.2 before pruning. These settings offer more conservative accuracy–compression trade-offs without additional training.

Overall, TRIM supports both training-free deployment and further accuracy recovery when a training budget is available. Most notably, on the language model, it avoids hours of eight-GPU fine-tuning while outperforming a fine-tuned alternative. We will add these fine-tuning comparisons and their computational costs to the experimental section of the revised manuscript.

AuthorsOpenReview ↗

Response to W3 and Q5

Q5: Can you report measured latency for the pruned models?

We thank the reviewer for requesting direct evidence of inference acceleration. We measured inference speedups for the pruned vision models, including the additional ResNet19 experiments. Table R5 reports the ratio of unpruned to pruned inference latency; a larger value indicates faster inference. These are measured wall-clock speedups, rather than estimates from operation counts.

Table R5. Measured inference speedups of TRIM-pruned models relative to their unpruned counterparts.

ModelDataset10% pruning20% pruning30% pruning40% pruning50% pruning
VGG16CIFAR-101.24×1.55×2.01×2.72×3.86×
VGG16CIFAR-1001.24×1.55×2.01×2.72×—
VGG16TinyImageNet1.22×1.54×2.00×——
ResNet20CIFAR-101.12×1.25×1.43×1.66×2.00×
ResNet20CIFAR-1001.12×1.25×1.43×1.66×2.00×
ResNet19CIFAR-1001.12×1.25×1.42×1.65×1.98×
SEW-ResNet18ImageNet1.13×1.24×1.41×1.64×1.94×
SpikingformerCIFAR-1001.14×1.34×1.61×——

“—” denotes a setting not reported here.

The measurements show inference acceleration across all reported settings. At 30% pruning, VGG16 achieves approximately 2× speedup on CIFAR-10, CIFAR-100, and TinyImageNet, while Spikingformer achieves 1.61× on CIFAR-100. At 50% pruning, ResNet20, ResNet19, and SEW-ResNet18 achieve 2.00×, 1.98×, and 1.94× speedups, respectively. Gains are also present at lighter pruning ratios, so acceleration is not restricted to the most aggressive compression settings.

TRIM physically removes channels, and its compensation is absorbed into the retained weights without additional inference modules. These measurements therefore complement the operation-count and energy analyses by demonstrating reductions in actual inference latency. We will include these measured speedups in the experimental section of the revised manuscript.

Reviewer_iwCTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TRIM, a post-training structured pruning method for spiking neural networks (SNNs) that does not require retraining or fine-tuning. The main challenge is to remove channels while preserving the temporal dynamics of the original network.

The authors first develop a reconstruction-based pruning method that uses surrogate membrane potentials to account for temporal integration. Channels are removed greedily, and the remaining weights are adjusted using closed-form compensation. TRIM then replaces full membrane-potential trajectory reconstruction with a simpler objective that preserves two temporal components: the mean membrane potential and its linear trend over time.

Critical strengths

The paper presents a practical way to remove entire channels from pretrained SNNs without additional training. The combination of greedy channel selection and closed-form weight compensation is useful, especially for larger models where fine-tuning can be expensive. The results show that this approach preserves accuracy much better than several existing pruning baselines. Instead of matching the full membrane-potential trajectory, TRIM focuses on its mean and linear trend. I find this an interesting choice because it improves accuracy without adding much complexity to the pruning procedure. The gains are particularly noticeable at higher pruning ratios, such as the 9.5 percentage point improvement over full-trajectory reconstruction on VGG16/CIFAR-100 at 40% pruning.

Critical weaknesses
  1. The novelty over existing methods is not very clear. OSBC already uses surrogate membrane-potential reconstruction, while OSSCAR uses quadratic reconstruction for structured pruning. TRIM seems to combine these with a two-moment projection. I think the authors should better establish what is actually new beyond existing reconstruction methods.

  2. Appendix C says ResNet20 uses T = 2. Since TRIM uses two orthonormal temporal components, shouldn't this be equivalent to full-trajectory reconstruction? Then why are the accuracies different in Table 1? This raises some questions about the implementation and reported gains.

  3. Its unclear whether the improvement comes from better channel selection or reconstruction. TRIM changes both, and Figure 2 shows lower full-trajectory error than the baseline directly optimizing it. So is the gain mainly from choosing better channels? I think these two effects should be separated.

  4. Does preserving mean and linear trend actually preserve useful temporal information? Two trajectories can have similar moments but very different spike timings due to threshold crossings and resets. The paper doesn't show whether TRIM preserves actual spiking behavior better than the baseline.

  5. Finally, how much of the gain is specific to these two moments? Would PCA, mean-only, or random projections work similarly? Also, most models use very few time steps, so its unclear whether TRIM captures meaningful temporal dynamics or simply benefits from reducing the reconstruction dimension.

Questions for authors
  1. For ResNet20, T = 2, so shouldn't TRIM and full-trajectory reconstruction be mathematically equivalent? Then why are the results different in Table 1? I think this is important to clarify since it directly questions the claimed gains.

  2. In Figure 2, TRIM actually gets lower full-trajectory reconstruction error than the baseline which directly optimizes it. So is the improvement mainly coming from selecting better channels? What happens if we use TRIM-selected channels but compensate them using the full-trajectory objective?

  3. Why is the weight compensation restricted to WR+WDB⊤W_R+W_DB^\top? Since the inputs have already changed from pruning earlier layers, wouldn't directly optimizing the retained weights potentially give better reconstruction?

  4. How much of the 97.3% error concentration in the two moments is just because of the smoothing from leaky integration? Also, would randomly shuffled spike sequences show similar behavior? I think this matters for understanding if these moments actually capture useful temporal information.

  5. Finally, were the calibration-set sizes selected using a separate validation set or the test set? And were the other methods given the same selection procedure? This matters especially when some of the reported improvements are quite small.

9 comments
AuthorsOpenReview ↗

Response to W2 and Q1

Q1: For ResNet20 with T=2, shouldn't TRIM and full-trajectory reconstruction be mathematically equivalent? Why do their results differ?

We thank the reviewer for identifying this special case. At T=2, the constant and linear components form a complete orthonormal basis, so retaining both is an orthogonal change of coordinates without information loss. With identical inputs, normalization, and regularization, TRIM and the full-trajectory baseline have equivalent objectives in exact arithmetic. Thus, differences at T=2 are not evidence of benefits from temporal-subspace selection; a strictly lower-dimensional subspace exists only when T>2.

Both implementations accumulate calibration statistics and apply compensating weight updates in FP32. However, the baseline accumulates statistics from full surrogate membrane-potential trajectories, whereas TRIM first projects them onto the two temporal components. Finite-precision representations and different operation orders can produce different statistics and compensated weights even with identical inputs, precision, and seeds.

These differences can propagate through layer-wise pruning. Even with identical retained channels, weight perturbations can change spikes near the firing threshold; resets and subsequent layers then change the activations used for later calibration. Close candidate scores can also change the greedy deletion order. Our checks found different FP32 statistics for the same input and different compensated weights despite identical early-layer channel selections, followed by differences in spikes and later selections. This supports numerical propagation rather than an additional modeling advantage at T=2.

We repeated ResNet20 experiments on CIFAR-10 and CIFAR-100 with 10 calibration seeds and 4,096 samples. For each dataset, the pretrained checkpoint was fixed; within each seed, both methods used identical calibration samples and ordering, with pruning and evaluation settings aligned. Both tables report top-1 accuracy (%) as mean ± standard deviation over 10 seeds.

Table R1. ResNet20, T=2.

DatasetCIFAR-10CIFAR-100
Pruning ratioBaselineTRIMBaselineTRIM
10%95.90 ± 0.1295.92 ± 0.1377.98 ± 0.1678.01 ± 0.17
20%95.57 ± 0.1195.56 ± 0.1276.28 ± 0.1576.25 ± 0.16
30%95.12 ± 0.1395.14 ± 0.1173.85 ± 0.1873.90 ± 0.16
40%93.92 ± 0.1393.89 ± 0.1469.98 ± 0.1669.95 ± 0.16
50%91.87 ± 0.1291.85 ± 0.1363.30 ± 0.1663.90 ± 0.17

Relative performance varies with the calibration seed, and TRIM does not consistently outperform the baseline. We therefore do not interpret the original single-run improvements as evidence of a methodological advantage at T=2.

To evaluate TRIM beyond the T=2 equivalence case, we additionally tested ResNet19 at T=4, where the two temporal components span a strictly lower-dimensional subspace. For each dataset, OBC, the baseline, and TRIM used the same pretrained checkpoint, calibration budget, pruning ratios, and 10 calibration seeds.

Table R2. ResNet19 (74.47%), T=4.

Pruning ratioCIFAR-10CIFAR-100
OBCBaselineTRIMOBCBaselineTRIM
10%94.27 ± 0.0894.34 ± 0.0794.41 ± 0.0774.28 ± 0.1074.40 ± 0.1174.53 ± 0.09
20%93.55 ± 0.0693.75 ± 0.0794.05 ± 0.0673.22 ± 0.1573.53 ± 0.1573.75 ± 0.15
30%92.71 ± 0.0793.07 ± 0.0693.45 ± 0.0772.01 ± 0.1572.34 ± 0.1372.70 ± 0.15
40%91.67 ± 0.0792.01 ± 0.0792.71 ± 0.0769.15 ± 0.1670.16 ± 0.1571.09 ± 0.17
50%88.89 ± 0.0789.73 ± 0.0990.68 ± 0.0866.76 ± 0.1567.63 ± 0.1468.92 ± 0.13

At T=4, TRIM achieves higher mean accuracy than both alternatives across all five ratios on both datasets, with larger gains at higher pruning ratios. At 50% pruning, gains over the baseline are 0.95/1.29 percentage points on CIFAR-10/100, and gains over OBC are 1.79/2.16 points.

In the revised manuscript, we will explicitly clarify in the main text (Section 3.3) that TRIM reduces to full-trajectory reconstruction at T=2 and uses a strictly lower-dimensional temporal subspace only when T>2. We will revise the Table 1 discussion so that T=2 accuracy differences are not presented as evidence of subspace benefits. We will report the 10-seed ResNet20 consistency check and the additional ResNet19 comparison with means and standard deviations. The appendix will document statistics computation, numerical precision, calibration sampling, and evaluation settings to distinguish implementation-dependent differences from methodological gains.

AuthorsOpenReview ↗

Response to W3 and Q2

Q2 & W3: Do the improvements mainly come from channel selection? What happens when TRIM-selected channels are compensated using the full-trajectory objective?

We thank the reviewer for suggesting this ablation. Figure 2 compares complete pruning pipelines, which can differ in both channel sets and layer inputs, rather than compensation solutions for the same fixed reconstruction problem. To separate the contributions of selection and compensation, we evaluated both cross-combinations: baseline selection with TRIM compensation, and TRIM selection with baseline compensation. For each combination, we fixed the selected channels and recomputed compensation from the original pretrained checkpoint, without reselecting channels or fine-tuning. We evaluated VGG16 and additionally tested ResNet19 to examine whether the findings extend to a residual architecture. Both models use T=4.

Table R3. Selection–compensation ablation on CIFAR-100. Top-1 accuracy (%) at different pruning ratios.

ModelChannel selectionWeight compensation10%20%30%40%50%
VGG16BaselineBaseline69.4164.2655.1739.03—
TRIMBaseline69.4764.3555.4841.26—
BaselineTRIM69.5265.5557.3245.49—
TRIMTRIM69.5765.7858.5048.53—
ResNet19BaselineBaseline74.4073.5372.3470.1667.63
TRIMBaseline74.3373.5772.3670.2867.97
BaselineTRIM74.4173.6772.5170.6468.31
TRIMTRIM74.5373.7572.7071.0968.92

The cross-combinations are additional experimental results. “—” denotes a result not reported here.

For VGG16 at 40% pruning, TRIM-selected channels with baseline compensation achieve 41.26%, compared with 48.53% for complete TRIM. Thus, retaining TRIM's channel choices while reverting to full-trajectory compensation loses 7.27 percentage points. Conversely, baseline-selected channels with TRIM compensation achieve 45.49%, improving on the complete baseline by 6.46 points without requiring TRIM's channel choices. Combining TRIM selection with TRIM compensation further improves accuracy by 3.04 points, reaching 48.53%. These comparisons show that compensation contributes substantially, while channel selection provides an additional benefit.

The ResNet19 results support a similar pattern. At 50% pruning, TRIM selection with baseline compensation achieves 67.97%, whereas baseline selection with TRIM compensation achieves 68.31%; complete TRIM has a reference accuracy of 68.92%. Across both architectures and all reported pruning ratios, baseline selection with TRIM compensation outperforms the reverse combination, and complete TRIM has the highest reported accuracy.

These results indicate that TRIM's gains are not explained by better channel selection alone. Temporal-subspace compensation is an important component, and using the temporal-subspace objective for both selection and compensation produces the strongest overall results in these comparisons. We will add this selection–compensation ablation to the experimental section of the revised manuscript.

AuthorsOpenReview ↗

Response to Q3

Q3: Why is weight compensation restricted to WR+WDB⊤W_R + W_D B^\top? Since the inputs have already changed from pruning earlier layers, wouldn't directly optimizing the retained weights potentially give better reconstruction?

We thank the reviewer for this question. We restrict compensation to combinations of the dropped weights because these weights encode the contributions that pruning removes. In the pretrained model, WDW_D specifies how the deleted channels affect each output neuron. We keep these learned output connections and use BB to determine which retained channels take over their contributions and by how much. Thus, the update directions come from the connections removed by pruning, rather than from an arbitrarily chosen subspace.

The reason for retaining this restriction is to use the pretrained model's connection structure when fitting compensation on the calibration set. Unrestricted optimization allows the retained weights to be freely re-estimated to reduce the local reconstruction loss. For a fixed channel set and the same objective, its optimal loss can indeed be no higher. However, the calibration loss is only a surrogate for the final task: fitting it more closely does not guarantee better classification accuracy. We therefore use the dropped weights as a structural prior, so that calibration determines how to redistribute their contributions rather than freely refitting the retained layer. This is a deliberate choice about how to compensate for pruning, not a claim that unrestricted optimization cannot improve reconstruction.

This restriction does not assume that inputs remain unchanged after earlier-layer pruning. We solve for BB using the current inputs from the progressively pruned network and the fixed target from the unpruned network. Upstream input discrepancies therefore enter the compensation objective. What we restrict is the form of the weight update, not whether those discrepancies are considered.

To directly test this choice, we evaluated an unrestricted variant on VGG16 and ResNet19 using CIFAR-100, with T=4.

Table R4. TRIM versus unrestricted optimization on CIFAR-100. Top-1 accuracy (%) at different pruning ratios.

ModelVariant10%20%30%40%50%
VGG16Unrestricted68.9164.3156.6746.00—
TRIM69.5765.7858.5048.53—
ResNet19Unrestricted73.8173.2372.0170.1267.79
TRIM74.5373.7572.7071.0968.92

The unrestricted results are averaged over 10 seeds. “—” denotes a result not reported here.

TRIM achieves higher reported accuracy across both architectures and all evaluated pruning ratios. At 40% pruning on VGG16, TRIM achieves 48.53%, compared with 46.00% for unrestricted optimization, a difference of 2.53 percentage points. At 50% pruning on ResNet19, the corresponding accuracies are 68.92% and 67.79%, a difference of 1.13 points. These results support the proposed compensation parameterization: allowing unrestricted weight adjustment does not improve classification accuracy in these experiments, even when channel selection is also performed under unrestricted optimization.

We will clarify why we use this compensation form in the revised manuscript and add this comparison to the experimental section.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response Summary and Updated Manuscript to All Reviewers

We sincerely thank all reviewers for their thoughtful and constructive feedback, which has substantially strengthened our work. In particular, we greatly appreciate the acknowledgements from:

  • Reviewers uPJn and j2aD highlighted the practical value of pruning pretrained spiking models without fine-tuning and the evaluation across both vision and language models. uPJn also noted the gains over OBC in more demanding pruning settings.
  • Reviewers j2aD and iwCT recognized the value of considering channel selection together with weight compensation. iwCT specifically highlighted the combination of greedy selection and closed-form compensation for models where fine-tuning can be expensive.
  • Reviewer iwCT noted in the follow-up that the selection–compensation ablation and the comparisons with alternative temporal projections addressed most of their original concerns, and that the spike-preservation results were useful.

We have incorporated the additional experiments and clarifications into the revised manuscript. Additions are marked in blue.

ConcernReviewer(s)Location in the revised manuscript
Temporal reconstruction criterion and rationale for restricting weight compensationiwCTSections 3.2–3.3; Appendices A.1 and B.5 (Table 14)
Contributions of channel selection and weight compensationiwCT, j2aDSection 4.4, Table 5
Choice of temporal subspace, including PCA under matched calibration seedsiwCT, j2aDSection 4.4, Table 6
Calibration-set selection and variability across calibration seedsiwCT, uPJn, j2aDSection 4.1; Appendix B.4, Table 12
Equivalence at T=2 and the FP64 consistency checkiwCT, j2aDSection 3.3; Appendix B.4, Table 13
Error decomposition across architectures, temporal shuffling, and agreement with actual spikesiwCT, uPJnAppendix B.6, Tables 15–16
Measured inference speedupuPJnAppendix B.2, Table 8
Accuracy recovery and cost of post-pruning fine-tuninguPJnAppendix B.3, Tables 10–11

The main findings are:

  1. Selection and compensation (iwCT, j2aD). At 40% pruning on VGG16/CIFAR-100, the full-trajectory baseline reached 39.08% accuracy. Changing only channel selection raised accuracy to 41.26%; changing only compensation raised it to 45.49%; using TRIM for both reached 48.52%. TRIM’s compensation also outperformed unrestricted weight refitting in all nine settings tested.

  2. Choice of temporal components (iwCT, j2aD). Across the same 10 calibration seeds, TRIM outperformed mean-only reconstruction, both random-subspace alternatives, two-dimensional PCA, and full-trajectory reconstruction at all four pruning ratios on VGG16 and ResNet19. At 40% pruning, its gains over PCA were 1.56 and 1.26 percentage points, respectively.

  3. Numerical consistency at T=2 (iwCT, j2aD). At two time steps, the mean and linear components span the full temporal space, so the objectives are equivalent in exact arithmetic. On ResNet20/CIFAR-100 at 50% pruning, computing calibration statistics and weight updates in FP64 reduced the 10-seed mean accuracy gap from 0.60 to 0.06 percentage points. We do not interpret this difference as a benefit of temporal projection.

  4. Temporal error and spike preservation (iwCT, uPJn). The two moments accounted for 97.3% and 94.0% of the baseline’s error increase on VGG16 and ResNet19 (10–40% pruning), versus 81.2% on Spikingformer (10–30%). The larger share outside the two moments may help explain why the full-trajectory baseline performed better on Spikingformer at 20% and 30%. In a direct comparison of VGG16 spike trains at 40% pruning, TRIM improved spike F1 against the unpruned model from 45.4% to 48.9%.

  5. Accuracy and inference speed (uPJn). The 10-seed comparison on SEW-ResNet18/ImageNet showed nearly identical accuracy at 10% pruning (59.92% for TRIM vs. 59.97% for OBC), but an 8.26-point advantage for TRIM at 50%. At 30% pruning, measured VGG16 inference latency was approximately halved on CIFAR-10, CIFAR-100, and TinyImageNet.

  6. Fine-tuning recovery and cost (uPJn). Under the same 50-epoch budget, TRIM achieved the highest final accuracy in all 13 vision settings tested. On SpikingLM-110M at 70% pruning, TRIM achieved a mean GLUE score of 72.2 without fine-tuning, whereas Wanda-sp reached 60.1 after 10.3 hours of fine-tuning on eight GPUs.

We hope that these updates address the reviewers' concerns and further clarify the value of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.