acceptodds
Under review as a conference paper at ICLR 2027

Class-Specific Pruning of Vision Transformers via Activation Calibration

Abstract

Existing class-specific pruning methods compress models for target classes but primarily evaluate neuron importance using activation magnitudes, treating all tokens equally. Our analysis reveals that tokens with high activation responses are not concentrated on target foreground objects; instead, many lie in background regions unrelated to the target classes, interfering with the identification of category-critical neurons. Further analysis shows that using frequency stability and attention weights to calibrate the activation contributions of different tokens can mitigate interference from irrelevant background information in neuron importance evaluation. Based on these findings, we propose Activation-Calibrated Pruning (AC-Prune), which identifies category-critical neurons based on the calibrated activation contributions. The selected neurons then serve as anchors to guide feed-forward network (FFN) neuron consolidation, while low-rankness is encouraged in multi-head attention (MHA) weights. This mitigates irreversible knowledge loss caused by direct pruning, yielding compact and efficient class-specific Vision Transformers (ViTs). Comprehensive experiments with five base ViTs covering three representative visual tasks on four datasets demonstrate that AC-Prune-derived ViTs retain approximately 20%–40% of the base models' parameters while outperforming them on class-specific tasks by up to 15.72% in accuracy.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Class-specific pruning, Vision Transformers, Activation calibration, Structured pruning, Edge intelligence

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DAJKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AC-Prune, a class-specific pruning method for vision transformers. It addresses background-token interference in activation-based neuron selection. The method combines attention weights and a frequency-based signal to calibrate token contributions. It consolidates neuron knowledge and applies low-rank compression before structured pruning. The evaluation spans multiple architectures, datasets and visual tasks. Validation covers five base models, four datasets and three visual tasks. Under aggressive pruning, AC-Prune retains more accuracy than the compared pruning baselines. A gap remains relative to the unpruned fine-tuned model. The compressed model also runs faster on the edge device.

Critical strengths
  1. The calibration has component-level support. Attention weighting improves classification accuracy in the ablation. Removing attention lowers mean accuracy. The frequency component also contributes, with a smaller classification gain. Both components contribute under this controlled setting. This supports the scoring design empirically, although it does not prove the frequency interpretation
  2. Aggressive pruning preserves an accuracy advantage over the compared baselines. Measured edge-device latency also decreases substantially. The accuracy comparison and measured device speed jointly support practical utility. This is stronger evidence than parameter or FLOP reductions alone.
Critical weaknesses
  1. The frequency rationale needs justification. The Fourier operation runs along hidden-channel indices. The method interprets the resulting low-frequency signal as useful stability. A hidden-channel permutation can be compensated in the adjacent FFN projection. That preserves the FFN function but can change its Fourier representation. Channel adjacency therefore needs a rationale.
  2. The evaluation describes clustering, consolidation and factorization in Vulcan. The reported comparisons do not clearly isolate a scoring/allocation-only replacement within that pipeline. Table 5 documents seed 42; repeated-seed uncertainty is not reported with the main accuracy tables. The advantage is much smaller at moderate compression. At pruning rate 0.40, the 50-class mean advantage over Nuwa is only 0.17 percentage points. Without uncertainty estimates, the reliability of small gains is unclear.
Questions for authors

What property of channel order justifies the low-pass filter? Are function-preserving permutation checks available? Do they change selection or final accuracy? Which components differ from Vulcan? Do multiple-seed estimates show that smaller gains exceed run-to-run variation?

4 comments
AuthorsOpenReview ↗

Response to Reviewer DAJK (1/2)

We thank the reviewer for the careful evaluation of our work and the constructive comments and suggestions.

Response to Weakness 1: Frequency Rationale and Channel Permutation

We thank the reviewer for raising this important concern. We agree that hidden-channel indices do not have predefined spatial adjacency semantics, and therefore adjacent channels cannot be directly interpreted as neighboring visual regions. Our method does not assume a fixed spatial arrangement of hidden channels. Instead, it analyzes the frequency characteristics of channel responses under the parameterization of the current pretrained model to assess feature stability.

Specifically, our design is motivated by the observation that low-frequency components tend to preserve relatively stable feature responses associated with object-related information, whereas high-frequency components may capture more local variations and background interference. Based on this observation, we measure feature stability through the differences between the original features and their low-pass reconstructions, and incorporate this information into neuron importance scoring to better identify neurons relevant to the target categories.

To evaluate this design, we compared the proportions of the top 20% selected tokens falling within the target bounding boxes under three selection strategies: random selection, activation-only scoring, and frequency-only scoring. The results are shown below.

LayerRandomActivation-onlyFrequency-only
158.17%67.38%67.42%
258.22%63.74%68.73%
358.20%58.64%67.82%
458.18%62.64%65.65%
558.21%59.81%64.59%
658.19%56.34%60.52%
758.19%56.28%60.72%
858.20%55.60%65.47%
958.17%58.84%68.64%
1058.21%67.58%68.29%
1158.19%60.91%62.16%
1258.17%45.76%51.59%
12-layer Average58.19%59.46%64.30%

The results show that frequency-only scoring achieves a higher proportion of selected tokens within the target bounding boxes than activation-only scoring across all 12 layers, with an average improvement of 4.84 percentage points. It also consistently outperforms random selection.

These findings suggest that, under the current model parameterization, the frequency-stability metric derived from low-pass reconstruction differences can more effectively identify feature responses associated with target objects, providing empirical support for our frequency-based scoring design.

Furthermore, to address the reviewer's concern that reordering hidden channels may alter their Fourier representation, we conducted additional function-preserving channel permutation experiments. Specifically, we permuted the hidden channels of the FFN and simultaneously adjusted the corresponding projection matrices to preserve the original FFN function. We then compared the selected neuron sets before and after permutation. The results are shown below.

Pruning RateT4-50T5-50T6-50
0.20 / 88,3920 / 88,3680 / 88,368
0.42 / 66,2401 / 66,2881 / 66,240
0.60 / 44,0880 / 44,0880 / 44,112
0.80 / 21,9840 / 21,9600 / 21,984

The numerator represents the number of selected neurons that changed after permutation across three random seeds, while the denominator represents the total number of selected neurons across the three seeds.

The results show that the selected neuron sets remain nearly identical before and after channel permutation across the evaluated tasks and pruning rates. Only a few neurons differ at a pruning rate of 0.4, while no changes are observed at the other evaluated pruning rates. These results indicate that, under our experimental settings, channel permutation has a negligible effect on neuron selection, despite its potential influence on the Fourier representation.

AuthorsOpenReview ↗

Response to Reviewer DAJK (2/2)

Response to Weakness 2: Component Differences and Multi-Seed Evaluation

We thank the reviewer for raising concerns regarding the component differences between AC-Prune and Vulcan.

The key difference between AC-Prune and Vulcan lies in the scoring mechanism used to select FFN neurons. AC-Prune incorporates frequency stability and attention information to refine neuron importance scoring, whereas Vulcan relies on its original activation-based scoring mechanism. The clustering, consolidation, and attention low-rank factorization procedures are maintained within the same overall framework.

To evaluate the reliability of the performance improvements, we conducted additional experiments with three random seeds.

First, we compared AC-Prune with Vulcan on tasks T7, T8, and T9 at a pruning rate of 0.2. The results are reported as mean ± standard deviation.

MethodT7T8T9T7–T9 Average
Vulcan91.58 ± 0.1392.96 ± 0.0591.71 ± 0.1392.08
AC-Prune92.02 ± 0.0293.62 ± 0.1492.48 ± 0.0992.71

AC-Prune improves accuracy by 0.44, 0.66, and 0.77 percentage points on T7, T8, and T9, respectively, with an average improvement of approximately 0.63 percentage points across the three tasks. Paired t-tests with Holm correction indicate statistically significant improvements on all three tasks, with adjusted p-values of 0.0377, 0.0377, and 0.0111, respectively.

Second, to address the reviewer's specific concern that the average improvement over NuWa at a pruning rate of 0.4 is relatively small, we conducted additional multi-seed experiments on tasks T4, T5, and T6.

MethodT4T5T6T4–T6 Average
NuWa93.96 ± 0.1293.51 ± 0.1294.73 ± 0.1094.07
AC-Prune94.12 ± 0.0893.60 ± 0.1494.94 ± 0.1394.22

AC-Prune improves accuracy by 0.16, 0.09, and 0.20 percentage points on T4, T5, and T6, respectively, yielding an average improvement of 0.15 percentage points. Paired t-tests with Holm correction indicate statistically significant improvements on all three tasks, with adjusted p-values of 0.0396, 0.0396, and 0.0387, respectively.

Overall, the additional multi-seed experiments demonstrate that AC-Prune achieves statistically significant performance improvements over the corresponding baselines at both pruning rates of 0.2 and 0.4. This indicates that, although the improvements are relatively modest on some tasks, the performance advantages of AC-Prune are still supported by statistical significance tests.

I thank the authors for the additional experiments, which have addressed my concern. Therefore, I have raised my rate to 3.

We sincerely thank the reviewer for raising the score. We are pleased that the additional experiments have addressed your previous concerns. We greatly appreciate your time, constructive comments, and recognition of our work.

Reviewer_HX8QOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The task is class-specific pruning: deriving a compact edge model from a pre-trained general ViT that retains only the knowledge of a target class subset S. The paper first gives an analysis showing that when neuron importance is measured by activation magnitude, high-response tokens are not concentrated on foreground objects but often fall in background regions, interfering with the selection of category-critical neurons. On this basis, it proposes AC-Prune, which calibrates token activation contributions with frequency stability (1D FFT along the channel dimension, Gaussian low-pass, inverse; stability S = X-hat/(|X-hat - X|+epsilon), then voting to obtain token stability v_i) and attention weights (CLS attention, or token-wise attention omega_n), yielding a neuron importance score I_c (Eqs. 6/7). The highest-importance neurons serve as anchors; FFNs undergo k-means clustering plus Anchor-Centric Neuron Consolidation (ACNC, Eq. 9), while MHAs are pushed toward low rank by Truncated Nuclear Norm Regularization (TNNR, Eqs. 10-12); the two are jointly enforced by an augmented Lagrangian (Eq. 13), after which FFNs are merged under a function-preserving rule, and MHAs are pruned by SVD (Eqs. 14-16). Experiments span DeiT-Base/Small/Tiny and Swin-T, three tasks (recognition/detection/segmentation), four datasets, and pruning rates 0.20-0.80, against five class-agnostic and two class-specific baselines (Vulcan, NuWa). The main tables show AC-Prune best in most columns, but its average gain over the strongest class-specific baseline is below 2%.

Critical strengths
  • The comparison set is broad, and the protocol gives compute away to the baselines. The method is compared against five class-agnostic and two class-specific pruning methods, and Appendix B states that class-agnostic baselines are retrained "until convergence", i.e., they receive convergence-level recovery training, which works against the submission. The consistency of the end-to-end gain is also worth crediting: at R=0.80, AC-Prune leads Vulcan on all nine sub-tasks, 9/9 in the same direction, which single-seed noise does not easily explain; and the evaluation is not restricted to classification, with COCO detection and segmentation in Fig. 7.

  • Efficiency is reported as measured, not only as FLOPs. Table 2 gives latency, throughput, and memory on Jetson Orin NX (bz=1) and RTX 4090 (bz=256), and the paper aligns pruned dimensions to multiples of 8 so the reduction is realizable.

Critical weaknesses
  • The newly added component is not isolated. By the submission's own Appendix B, Vulcan's [1] CCNC plus TNNR plus train-then-prune corresponds item by item to AC-Prune's ACNC plus TNNR, and the only explicit difference is that the anchor criterion changes from activation magnitude to calibrated activation. Table 3 reports only removing attention (T4: 89.24) and only removing frequency (T4: 89.96), with no row for both removed — yet that row is the prior criterion and the only control that can attribute the gain to calibration. By magnitude, the two branches account for at most 1.2–1.3 points against a 2.08-point gap to Vulcan on T4 at R=0.80; with single-seed training (Seed 42) and training budgets unmatched to the prior work, the attribution behind the second contribution does not hold.

  • The motivating diagnostic measures a different quantity from the one the method criticizes. The paper faults baselines for "treating all tokens equally", but Fig. 2, 3, 5, 9, and 10 all measure the inside-box proportion of the top-20% of tokens selected by activation magnitude, whereas Appendix B describes NuWa's criterion as an equal-weight mean over all tokens. The decisive missing control is downstream accuracy with the token weights replaced by random weights, since Fig. 3's own claim that activation selection is worse than random implies random weighting should do better. If that chain does not hold, activation calibration is an effective empirical reweighting rather than the claimed removal of background interference for accurate identification of category-critical neurons.

Questions for authors
  • The controlled comparison inside AC-Prune's fixed framework: replace only the anchor selection with plain activation magnitude (no frequency, no attention), keeping ACNC, TNNR, the constrained-training budget, and D_S identical, over at least three seeds. Does the small AC-Prune-vs-Vulcan gap reproduce and remain significant?

  • For "w/o ACNC = 7.80%" in Table 4, state exactly what was disabled (only the consolidation regularization of Eq. 9, or also the function-preserving merge of Eq. 14?), and add a fair ACNC ablation that keeps a principled FFN-pruning alternative (e.g., retain top-K importance neurons without merging) rather than exhibiting collapse from direct pruning.

  • Report average gains over the strongest class-specific baseline (not single-sub-task peaks over the non-fine-tuned base) in the abstract and body, and clarify that 15.72% is relative to the general base while AC-Prune retains 98.54%/94.97% of the fine-tuned base.

  • Provide means and standard deviations over at least three seeds for Tables 1 and 6; are sub-1% margins (e.g., +0.66% over the best class-specific baseline at R=0.20) significant above seed variance?

  • Are the constrained-training steps, calibration-data usage, and per-step overhead (FFT, attention extraction) matched across AC-Prune, Vulcan, and NuWa? Report training steps and token/forward-call counts for each method to rule out a supervision or compute mismatch.

3 comments
AuthorsOpenReview ↗

Response to Reviewer HX8Q (1/3)

We sincerely thank the reviewers for their careful evaluation of our manuscript and for their constructive comments.

W1/Q1. Contribution of the New Scoring Mechanism and Controlled Comparison.

Thank you for this insightful comment. We agree that, to verify the actual contribution of the newly introduced scoring mechanism, a controlled comparison is necessary in which only the anchor selection criterion is changed while all other components and experimental settings remain identical.

To this end, we conducted a controlled comparison within the fixed AC-Prune framework. At a pruning rate of 0.6, we replaced the original anchor selection criterion with one based solely on activation magnitude, without using frequency or attention information, while keeping all other settings, including ACNC, TNNR, the constrained training budget, and DSD_S, exactly the same. We then compared this variant with the full AC-Prune method. The experiments were conducted on three 50-class tasks, T4, T5, and T6, with three random seeds for each method. The results are reported below:

MethodT4T5T6T4–T6 Avg.
Activation Only92.29 ± 0.1791.83 ± 0.1092.57 ± 0.1792.23
AC-Prune92.99 ± 0.1592.39 ± 0.1093.56 ± 0.0892.98

The results show that, compared with anchor selection based solely on activation magnitude, AC-Prune improves accuracy by 0.70, 0.56, and 0.99 percentage points on T4, T5, and T6, respectively, with an average improvement of 0.75 percentage points across the three tasks. Furthermore, paired-sample t-tests with Holm correction for multiple comparisons yield adjusted p-values of 0.0458, 0.0299, and 0.0281, respectively, all of which are statistically significant.

These results demonstrate that, while keeping all other components unchanged, the calibrated anchor scoring mechanism provides additional performance gains, further validating the effectiveness of the proposed scoring mechanism.

Q2. ACNC Ablation and Top-K Retention Comparison.

Thank you for this suggestion. Regarding the experimental setting reported as "w/o ACNC = 7.80%" in Table 4, we further clarify its pruning procedure. In this setting, the attention module is pruned first, followed by pruning of the FFN module. At high pruning rates, a substantial portion of the attention module is removed, which significantly affects model performance.

Furthermore, to avoid the performance degradation caused by direct pruning from confounding the evaluation of the FFN pruning strategy, we conducted an additional FFN Top-K retention experiment, following the reviewer's suggestion. In this experiment, the Top-K neurons with the highest importance scores are directly retained without clustering or merging, and the resulting model is compared with the full AC-Prune method. The results at a pruning rate of 0.6 are presented below:

MethodT4T5T6T4–T6 Avg.
Top-K Retention without Merging92.56 ± 0.2291.56 ± 0.3292.81 ± 0.1692.31
AC-Prune92.99 ± 0.1592.39 ± 0.1093.56 ± 0.0892.98

The results show that, compared with directly retaining the Top-K most important neurons, AC-Prune improves accuracy by 0.43, 0.83, and 0.75 percentage points on T4, T5, and T6, respectively, with an average improvement of 0.67 percentage points. This demonstrates that combining neuron clustering and merging during FFN pruning can better preserve model performance after compression.

Q3. Performance Improvement Magnitudes in the Abstract and Main Text.

Thank you for this suggestion. We agree with this comment and will revise the performance comparison statements in the abstract and main text to clarify the reference baselines for different performance improvement metrics.

Specifically, the 15.72% improvement is measured relative to the general-purpose base model, whereas 98.54% and 94.97% represent the performance retention rates of AC-Prune relative to the fine-tuned base model. For comparisons between AC-Prune and class-specific methods such as Vulcan and NuWa, we will report the average performance improvement over the strongest class-specific baseline under the same tasks and pruning rates, rather than using the maximum improvement on a single subtask relative to the unfine-tuned base model to characterize overall performance.

AuthorsOpenReview ↗

Response to Reviewer HX8Q (2/3)

W1/Q4. Multiple-Seed Experiments and Statistical Significance.

Thank you for your concern regarding experimental stability. In response to the fact that some experiments in the original manuscript used only a single random seed, we conducted additional experiments with multiple random seeds and report the means and standard deviations. We also performed paired-sample t-tests to evaluate the performance differences between AC-Prune and class-specific baselines, applying the Holm method for multiple-comparison correction.

At a pruning rate of 0.2, we compared AC-Prune with Vulcan on three tasks, T7, T8, and T9. The results are presented below:

MethodT7T8T9T7–T9 Avg.
Vulcan91.58 ± 0.1392.96 ± 0.0591.71 ± 0.1392.08
AC-Prune92.02 ± 0.0293.62 ± 0.1492.48 ± 0.0992.71

AC-Prune improves accuracy by 0.44, 0.66, and 0.77 percentage points on T7, T8, and T9, respectively, with an average improvement of approximately 0.63 percentage points across the three tasks. Paired-sample t-tests with Holm correction show statistically significant improvements on all three tasks, with adjusted p-values of 0.0377, 0.0377, and 0.0111, respectively.

These results demonstrate that, even at a pruning rate of 0.2, AC-Prune consistently outperforms Vulcan on T7, T8, and T9, with all improvements reaching statistical significance. This further confirms that AC-Prune maintains a performance advantage even at relatively low pruning rates.

Q5. Training Budget and Additional Computational Overhead.

Thank you for pointing out the importance of ensuring comparability in training budgets and computational overhead across different methods.

Regarding the training budget, AC-Prune, Vulcan, and NuWa all determine their training budgets according to the training-step formula specified in the paper:

Steps=6250R2+1250R,Steps = 6250R^2 + 1250R,

where RR denotes the pruning rate. Under the same task and pruning rate, all three methods use the same number of training steps, with a batch size of 256. The amounts of training data and calibration data used are also kept consistent. Therefore, the total number of tokens processed and the number of model forward passes during training are identical across the three methods.

Regarding computational overhead, we further measured the average time per training step and peak GPU memory consumption of Vulcan and AC-Prune. The results are shown below:

MethodAverage Time per Step (ms)Peak GPU Memory (MB)
Vulcan2921.8330908.35
AC-Prune2957.5030924.94

The results show that the average time per step of AC-Prune increases by only 35.67 ms compared with Vulcan, corresponding to approximately 1.22%, while peak GPU memory consumption increases by only 16.59 MB. The two methods exhibit comparable per-step execution times and memory consumption, indicating that the additional computational overhead introduced by the proposed components is relatively small.

AuthorsOpenReview ↗

Response to Reviewer HX8Q (3/3)

W2. Diagnostic Experiments and Random Token Weights.

Thank you for pointing out the potential discrepancy between the diagnostic experiments and the problem addressed by our method. To further validate the effectiveness of the calibrated scoring mechanism, we conducted additional comparisons of the proportion of selected tokens located inside ground-truth bounding boxes under different scoring strategies, as well as downstream performance experiments using random Token weights.

First, regarding the interpretation of Fig. 3, we would like to clarify that we do not claim that random selection generally outperforms activation-based selection. Fig. 3 illustrates variations across different network layers. In particular, at the 8th and 12th layers, the proportion of tokens selected by pure activation-based scoring that fall inside the bounding boxes is lower than that of random selection. However, averaged across all 12 layers, pure activation-based selection still outperforms random selection. Furthermore, we have added the results of AC-Prune and compared the proportions of selected tokens falling inside the bounding boxes under three strategies: random selection, activation-only scoring, and AC-Prune. The results are shown below:

ScopeRandom SelectionActivation OnlyAC-Prune
Layer 858.20%55.60%71.41%
Layer 1258.17%45.76%62.53%
Average over 12 Layers58.19%59.46%68.29%

The results show that the proportions of tokens falling inside the bounding boxes for AC-Prune reach 71.41% and 62.53% at the 8th and 12th layers, respectively, substantially outperforming both random selection and activation-only scoring. Averaged across all 12 layers, AC-Prune achieves an inside-box token proportion of 68.29%, representing improvements of 8.83 percentage points over activation-only scoring and 10.10 percentage points over random selection. These findings indicate that the calibrated scoring mechanism, which incorporates frequency stability and attention information, can more effectively identify tokens associated with foreground objects.

Second, following the reviewer's suggestion, we conducted additional experiments using random Token weights to investigate whether random weighting could achieve better downstream performance. Specifically, we evaluated downstream accuracy under different scoring strategies on three 50-class tasks, T4, T5, and T6, at a pruning rate of 0.6. The results are presented below:

MethodT4T5T6T4–T6 Avg.
Random Weights91.68 ± 0.3291.23 ± 0.3292.29 ± 0.4291.73
Activation Only92.29 ± 0.1791.83 ± 0.1092.57 ± 0.1792.23
AC-Prune92.99 ± 0.1592.39 ± 0.1093.56 ± 0.0892.98

The results indicate that random Token weights do not yield better downstream performance. Activation-only scoring outperforms random weighting, while the full AC-Prune method achieves further improvements.

Taken together, these two sets of experiments demonstrate that the advantages of AC-Prune are reflected not only in improved downstream task performance but also in its ability to select tokens that are more concentrated within foreground object regions. This suggests that the proposed calibrated scoring mechanism is not merely an empirical reweighting strategy. Instead, by integrating frequency stability and attention information, it can more effectively identify category-relevant feature responses and reduce the influence of background interference when identifying category-critical neurons.

Reviewer_hxTyOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses class-specific pruning. Existing approaches mainly evaluate neuron importance with respect to a target class based on activation magnitude. The authors observe that high activations do not necessarily concentrate on foreground objects but also fall on background regions, and further find that leveraging frequency stability and attention weights can mitigate interference from such irrelevant background information. Building on these findings, they propose Activation-Calibrated Pruning (AC-Prune). The selected neurons are used to guide FFN pruning, while a low-rank constraint is imposed on the multi-head attention (MHA) layers, enabling effective class-specific pruning. Experiments on ImageNet-1K with DeiT-Base as the baseline show that the proposed method outperforms existing class-agnostic and class-specific pruning approaches.

Critical strengths
  • The paper empirically identifies an important limitation of existing class-specific pruning methods: when neuron importance is estimated solely from activation magnitudes, high activations triggered by irrelevant background tokens can interfere with the accurate identification of category-critical neurons. This observation is well supported by the experiments and constitutes a meaningful finding that could benefit future research on class-specific pruning.
  • The proposed AC-Prune is clearly and rigorously defined in mathematical terms, which makes it easy to follow how each component of the algorithm operates and how the selected neurons are used in pruning. In addition, the method consistently outperforms recent class-specific pruning approaches, demonstrating not only its conceptual soundness but also its practical value.
Critical weaknesses
  • The comparison with class-agnostic pruning methods does not appear to be very meaningful, since these methods are designed for a different objective. Instead, the evaluation would be more convincing if the proposed method were compared against a broader range of existing class-specific pruning approaches. Compared with recent methods such as Vulcan and Nuwa, the performance gains appear marginal at low pruning rates, and a meaningful improvement is observed only at a high pruning rate (e.g., 0.8). The authors are encouraged to discuss why the advantage of AC-Prune becomes apparent only under aggressive pruning, and whether the gains at lower pruning rates are statistically significant.
  • Although the paper states that experiments were conducted on CIFAR-100/10 and COCO, comparisons with existing methods on these datasets are insufficient. To demonstrate the generality of the proposed method, the authors should provide comparative results showing that AC-Prune remains effective across diverse datasets and downstream tasks such as object detection and semantic segmentation.
Questions for authors
  • Gains at different pruning rates: The improvements over Vulcan and Nuwa appear marginal at low pruning rates but become significant at a pruning rate of 0.8. Could the authors explain why the advantage of AC-Prune emerges mainly under aggressive pruning, and whether the gains at lower pruning rates are statistically significant across multiple runs?
  • Generalization to other datasets and tasks: The paper mentions experiments on CIFAR-100/10 and COCO. Could the authors provide comparative results against existing class-specific methods on these datasets, as well as on downstream tasks such as object detection and semantic segmentation?
2 comments
AuthorsOpenReview ↗

Response to Reviewer hxTy(1/2)

We sincerely thank the reviewer for the careful evaluation of our work and for the constructive comments.

W1/Q1. Performance Gains at Different Pruning Rates and Statistical Significance

First, we conducted additional experiments with multiple random seeds on three tasks, T7, T8, and T9, at a pruning rate of 0.2, and report the mean accuracy and standard deviation. The results are presented below:

MethodT7T8T9T7–T9 Avg.
Vulcan91.58 ± 0.1392.96 ± 0.0591.71 ± 0.1392.08
AC-Prune92.02 ± 0.0293.62 ± 0.1492.48 ± 0.0992.71

The results show that AC-Prune improves accuracy by 0.44, 0.66, and 0.77 percentage points on T7, T8, and T9, respectively, yielding an average improvement of 0.63 percentage points across the three tasks. Furthermore, paired-sample t-tests with Holm correction for multiple comparisons show that the improvements are statistically significant on all three tasks, with adjusted p-values of 0.0377, 0.0377, and 0.0111, respectively.

Second, to investigate the reasons for the performance differences at different pruning rates, we analyzed the overlap between the FFN neuron sets selected using activation-magnitude-based scoring and frequency-stability-based scoring. The results, averaged across tasks, are presented below:

Pruning RateNumber of Identical NeuronsNumber of Different NeuronsPercentage of Different Neurons
0.222,4227,04723.91%
0.411,55310,52547.67%
0.64,7079,99867.99%
0.81,7425,58976.24%

As shown above, the difference between the neuron sets selected by the two scoring strategies increases with the pruning rate. At lower pruning rates, the two methods select highly overlapping neuron sets, resulting in relatively small differences in the resulting model architectures and, consequently, smaller performance gains. As the pruning rate increases, the two scoring strategies have a progressively greater impact on the resulting model architecture, making the performance differences between the selection strategies more pronounced. This also helps explain why AC-Prune demonstrates more substantial advantages at higher pruning rates, such as 0.6 and 0.8.

AuthorsOpenReview ↗

Response to Reviewer hxTy(2/2)

W2/Q2. Experimental Comparisons Across Different Datasets and Downstream Tasks.

We thank the reviewer for this valuable suggestion. Following the reviewer's recommendation, we conducted additional experiments on the CIFAR-100, CIFAR-10, and COCO datasets and compared AC-Prune with class-specific pruning methods, including Vulcan and NuWa. In addition, for downstream vision tasks, we supplemented the experiments with object detection and instance segmentation on the COCO dataset, as well as semantic segmentation on the ADE20K dataset, to further evaluate the applicability of AC-Prune across different datasets and visual tasks.

(1) Instance Segmentation on the COCO Dataset

We adopt Mask AP (%) as the evaluation metric for instance segmentation. We compare the performance of Vulcan, NuWa, and AC-Prune at different pruning rates on the COCO dataset. The results are presented below:

Method00.40.60.8
Vulcan66.465.662.561.2
NuWa66.466.162.060.6
AC-Prune66.467.063.962.6

(2) Object Detection on the COCO Dataset

We adopt Box AP (%) as the evaluation metric for object detection. We compare the performance of Vulcan, NuWa, and AC-Prune at different pruning rates on the COCO dataset. The results are presented below:

Method00.40.60.8
Vulcan68.867.465.963.8
NuWa68.868.564.962.6
AC-Prune68.869.466.666.1

(3) Classification on the CIFAR-100 Dataset

We conduct experiments using two model configurations, DeiT-Tiny and DeiT-Small, with classification accuracy (%) as the evaluation metric.

DeiT-Tiny:

Method00.40.60.80.9
Vulcan81.890.787.684.982.6
NuWa81.891.088.279.565.7
AC-Prune81.891.990.687.383.4

DeiT-Small:

Method00.40.60.80.9
Vulcan84.393.590.986.083.3
NuWa84.394.192.085.669.4
AC-Prune84.395.293.187.886.6

(4) Classification on the CIFAR-10 Dataset

Similarly, we conduct experiments using DeiT-Tiny and DeiT-Small, with classification accuracy (%) as the evaluation metric.

DeiT-Tiny:

Method00.40.60.80.9
Vulcan95.998.6598.0597.7597.1
NuWa95.999.098.5597.2594.6
AC-Prune95.999.4599.0598.7598.05

DeiT-Small:

Method00.40.60.80.9
Vulcan97.4599.699.198.6598.2
NuWa97.4599.499.0598.2595.0
AC-Prune97.4599.699.499.398.7

(5) Semantic Segmentation on the ADE20K Dataset

We adopt mIoU (%) as the evaluation metric for semantic segmentation. We compare the performance of Vulcan, NuWa, and AC-Prune at different pruning rates on the ADE20K dataset. The results are presented below:

Method00.40.60.8
Vulcan73.4681.1973.3272.29
NuWa73.4681.8173.6471.55
AC-Prune73.4681.6474.0672.67

The above experiments cover a range of visual tasks, including image classification, object detection, instance segmentation, and semantic segmentation. The results demonstrate that AC-Prune can be applied not only to image classification across different datasets but also to object detection, instance segmentation, and semantic segmentation. These findings highlight the applicability of AC-Prune across diverse visual tasks and its generalization capability across different tasks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.