- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AC-Prune, a class-specific pruning method for vision transformers. It addresses background-token interference in activation-based neuron selection. The method combines attention weights and a frequency-based signal to calibrate token contributions. It consolidates neuron knowledge and applies low-rank compression before structured pruning. The evaluation spans multiple architectures, datasets and visual tasks. Validation covers five base models, four datasets and three visual tasks. Under aggressive pruning, AC-Prune retains more accuracy than the compared pruning baselines. A gap remains relative to the unpruned fine-tuned model. The compressed model also runs faster on the edge device.
Critical strengths
- The calibration has component-level support. Attention weighting improves classification accuracy in the ablation. Removing attention lowers mean accuracy. The frequency component also contributes, with a smaller classification gain. Both components contribute under this controlled setting. This supports the scoring design empirically, although it does not prove the frequency interpretation
- Aggressive pruning preserves an accuracy advantage over the compared baselines. Measured edge-device latency also decreases substantially. The accuracy comparison and measured device speed jointly support practical utility. This is stronger evidence than parameter or FLOP reductions alone.
Critical weaknesses
- The frequency rationale needs justification. The Fourier operation runs along hidden-channel indices. The method interprets the resulting low-frequency signal as useful stability. A hidden-channel permutation can be compensated in the adjacent FFN projection. That preserves the FFN function but can change its Fourier representation. Channel adjacency therefore needs a rationale.
- The evaluation describes clustering, consolidation and factorization in Vulcan. The reported comparisons do not clearly isolate a scoring/allocation-only replacement within that pipeline. Table 5 documents seed 42; repeated-seed uncertainty is not reported with the main accuracy tables. The advantage is much smaller at moderate compression. At pruning rate 0.40, the 50-class mean advantage over Nuwa is only 0.17 percentage points. Without uncertainty estimates, the reliability of small gains is unclear.
Questions for authors
What property of channel order justifies the low-pass filter? Are function-preserving permutation checks available? Do they change selection or final accuracy? Which components differ from Vulcan? Do multiple-seed estimates show that smaller gains exceed run-to-run variation?
4 comments
Response to Reviewer DAJK (2/2)
Response to Weakness 2: Component Differences and Multi-Seed Evaluation
We thank the reviewer for raising concerns regarding the component differences between AC-Prune and Vulcan.
The key difference between AC-Prune and Vulcan lies in the scoring mechanism used to select FFN neurons. AC-Prune incorporates frequency stability and attention information to refine neuron importance scoring, whereas Vulcan relies on its original activation-based scoring mechanism. The clustering, consolidation, and attention low-rank factorization procedures are maintained within the same overall framework.
To evaluate the reliability of the performance improvements, we conducted additional experiments with three random seeds.
First, we compared AC-Prune with Vulcan on tasks T7, T8, and T9 at a pruning rate of 0.2. The results are reported as mean ± standard deviation.
| Method | T7 | T8 | T9 | T7–T9 Average |
|---|---|---|---|---|
| Vulcan | 91.58 ± 0.13 | 92.96 ± 0.05 | 91.71 ± 0.13 | 92.08 |
| AC-Prune | 92.02 ± 0.02 | 93.62 ± 0.14 | 92.48 ± 0.09 | 92.71 |
AC-Prune improves accuracy by 0.44, 0.66, and 0.77 percentage points on T7, T8, and T9, respectively, with an average improvement of approximately 0.63 percentage points across the three tasks. Paired t-tests with Holm correction indicate statistically significant improvements on all three tasks, with adjusted p-values of 0.0377, 0.0377, and 0.0111, respectively.
Second, to address the reviewer's specific concern that the average improvement over NuWa at a pruning rate of 0.4 is relatively small, we conducted additional multi-seed experiments on tasks T4, T5, and T6.
| Method | T4 | T5 | T6 | T4–T6 Average |
|---|---|---|---|---|
| NuWa | 93.96 ± 0.12 | 93.51 ± 0.12 | 94.73 ± 0.10 | 94.07 |
| AC-Prune | 94.12 ± 0.08 | 93.60 ± 0.14 | 94.94 ± 0.13 | 94.22 |
AC-Prune improves accuracy by 0.16, 0.09, and 0.20 percentage points on T4, T5, and T6, respectively, yielding an average improvement of 0.15 percentage points. Paired t-tests with Holm correction indicate statistically significant improvements on all three tasks, with adjusted p-values of 0.0396, 0.0396, and 0.0387, respectively.
Overall, the additional multi-seed experiments demonstrate that AC-Prune achieves statistically significant performance improvements over the corresponding baselines at both pruning rates of 0.2 and 0.4. This indicates that, although the improvements are relatively modest on some tasks, the performance advantages of AC-Prune are still supported by statistical significance tests.
I thank the authors for the additional experiments, which have addressed my concern. Therefore, I have raised my rate to 3.
We sincerely thank the reviewer for raising the score. We are pleased that the additional experiments have addressed your previous concerns. We greatly appreciate your time, constructive comments, and recognition of our work.
Response to Reviewer DAJK (1/2)
We thank the reviewer for the careful evaluation of our work and the constructive comments and suggestions.
Response to Weakness 1: Frequency Rationale and Channel Permutation
We thank the reviewer for raising this important concern. We agree that hidden-channel indices do not have predefined spatial adjacency semantics, and therefore adjacent channels cannot be directly interpreted as neighboring visual regions. Our method does not assume a fixed spatial arrangement of hidden channels. Instead, it analyzes the frequency characteristics of channel responses under the parameterization of the current pretrained model to assess feature stability.
Specifically, our design is motivated by the observation that low-frequency components tend to preserve relatively stable feature responses associated with object-related information, whereas high-frequency components may capture more local variations and background interference. Based on this observation, we measure feature stability through the differences between the original features and their low-pass reconstructions, and incorporate this information into neuron importance scoring to better identify neurons relevant to the target categories.
To evaluate this design, we compared the proportions of the top 20% selected tokens falling within the target bounding boxes under three selection strategies: random selection, activation-only scoring, and frequency-only scoring. The results are shown below.
The results show that frequency-only scoring achieves a higher proportion of selected tokens within the target bounding boxes than activation-only scoring across all 12 layers, with an average improvement of 4.84 percentage points. It also consistently outperforms random selection.
These findings suggest that, under the current model parameterization, the frequency-stability metric derived from low-pass reconstruction differences can more effectively identify feature responses associated with target objects, providing empirical support for our frequency-based scoring design.
Furthermore, to address the reviewer's concern that reordering hidden channels may alter their Fourier representation, we conducted additional function-preserving channel permutation experiments. Specifically, we permuted the hidden channels of the FFN and simultaneously adjusted the corresponding projection matrices to preserve the original FFN function. We then compared the selected neuron sets before and after permutation. The results are shown below.
The numerator represents the number of selected neurons that changed after permutation across three random seeds, while the denominator represents the total number of selected neurons across the three seeds.
The results show that the selected neuron sets remain nearly identical before and after channel permutation across the evaluated tasks and pruning rates. Only a few neurons differ at a pruning rate of 0.4, while no changes are observed at the other evaluated pruning rates. These results indicate that, under our experimental settings, channel permutation has a negligible effect on neuron selection, despite its potential influence on the Fourier representation.