acceptodds
Under review as a conference paper at ICLR 2027

BMIPrune: Class- and Layer-aware One-shot Average Activation-based Pruning for Convolutional Neural Networks via Mutual Information

Abstract

Activation-based pruning of convolutional networks has been pursued from two directions: class-agnostic criteria rank filters by the magnitude of their responses, while class-aware criteria rank filters by how strongly a response discriminates between classes. Both are applied across the network without regard to the depth of a filter’s layer, even though a filter’s role changes with depth. Early convolutions respond to generic low-level structure that is present regardless of the class, while later convolutions compose that structure into class-specific parts. A criterion that asks only what a filter reveals about the label therefore has little to mea- sure where filters are not class-selective, and a purely magnitude-based criterion may discard class information carried by a less active filter. This work proposes BMIPrune, a hybrid, one-shot criterion in which each layer’s own profile sets the balance between the two signals. Spatially averaged filter responses are modeled as class-conditional Gaussians, from which a Bayes mutual information (BMI) is computed for each filter. Weighting the posterior entropy by activation mag- nitude decomposes the resulting score exactly into a magnitude component and a class-information component, and their ratio defines each layer’s profile. On VGG16, the magnitude component accounts for about 98% of the profile in the first convolutions and as little as 68% in the last block, so the criterion falls back to magnitude where discriminative signal is absent and becomes class-aware where it is present. Before pruning, each layer is narrowed in isolation and its error curve recorded, giving a threshold-free redundancy score that is combined with the parameter cost of removing one filter from that layer. This sets each layer’s budget without reordering filters within the layer, so criterion and budget compose rather than interfere. On VGG19 with CIFAR-10, BMIPrune loses only 1.2 percentage points of accuracy at 60% parameter removal without fine-tuning, in line with other activation-based methods and more than 24 points ahead of uniform baselines. At 80% removal, where the baselines collapse, the loss is 7.7 points, 13 points less than the best competing activation-based method.

Primary area:
optimization
Keywords:
Convolutional Neural Networks, Activation-based Pruning, Discriminative-aware Pruning, Artificial Intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_64piOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a layer-wise profile, which weights whether filters in a layer are removed based on magnitude or on class-information criteria. This profile is based on an activation-weighted functional of the Bayes mutual information that decomposes into a magnitude component and a class-information component. They find that earlier layers should be pruned mainly based on magnitude and later layers should be pruned based on class information. Based on this weighting filters in each layer are removed within a per-layer budget, which determines how many parameters can be removed in each layer. The budget is determined by testing how much accuracy is lost when pruning each layer individually, weighed against the parameter savings.

Critical strengths

The paper quantifies the known intuition that earlier layers encode general feature information and later layers are class-discriminative. Together with the layer-wise pruning budget this gives an informed pruning method that does not rely on hyperparamter tuning.

Critical weaknesses

The usefulness of layer profiles and layer budgeting is not sufficiently shown, neither for understanding networks, nor for pruning them. The evaluation scope is very limited (e.g. single model checkpoints, few datasets, limited ResNet handling, very few re-training experiments, no ablations on the impact of layer profiles and layer budgeting individually), which weakens the empirical results, which in any case are not overly convincing.

While the theoretical part seems to be sound, the novelty is limited and as the authors mention, the underlying assumption that the class-conditionals are Gaussian is not necessarily given in ReLU models and not tested empirically.

These weaknesses are mostly acknowledged in the paper, but in my opinion a broader evaluation of the method showing a practical usefulness or new understanding of neural networks would be needed to accept the paper.

Questions for authors
  • Could you clarify how many evaluations it takes to compute the layer-budget allocation? How does the wall-clock time of your method compare to other methods?
  • How dependent is the theory on the Gaussian assumption?
  • Is it possible to derive layer profiles using more realistic distributional assumptions that are based on empirical measurements?
1 comment

Dear reviewer,

Thank you for the review and for recognizing that the method needs no hyperparameter tuning. Your central point, that the usefulness of the layer profiles was not shown either for understanding networks or for pruning, shaped the revision most. The paper now demonstrates both and positions itself accordingly. All overall changes to the paper can be found in the general response, along with all the new experiments R1-R9.

With regard to your addressed weaknesses:

Usefulness of the layer profiles for understanding networks:
 The revised paper presents the profile as an interpretable, per-layer measurement of how class evidence is encoded, and validates it against an estimate that does not use the Gaussian model (R5, new §4.5, App. F). Held-out probes trained on each layer's pooled responses measure class information directly. GlobalBMI's information component D agrees with them across 74 layers (Spearman rho = 0.85). When the same layer moves from CIFAR-10 to CIFAR-100, the change in D tracks the change in probe information (rho = 0.98 over 37 layer pairs, with depth held fixed), so D is not merely a proxy for depth. The probes also explain a finding that goes beyond "early general, late discriminative": alpha rises again in the final layer(s) of every network, although probe information there is maximal, because class evidence is carried by large, class-specific activations. This is consistent with neural collapse, which the revised paper discusses. Three independently trained networks per setting reproduce the profile to within 0.02 (R8). We also state the limits of this reading: within a single network D adds little beyond depth, and alpha does not predict a layer's prunability beyond depth (App. F).

Usefulness for pruning - profile and budget ablated separately:
R1, R2, added to the paper in §4.4, on five partitions. The budget is where most of the gain comes from. Other criteria placed in BMIPrune's allocation gain 6–29 points on average at 70–85% removal (mean activation +17.6, GFI-AP +6.2, L1 and random +24–29), and with BMIPrune's widths even random channels come within one point of the full method up to 85% on average. The profile and criterion decide at the highest compression: at 90–95% random channels fall 6.3 points behind, LocalBMI has the best mean rank of seven criteria, and removing either signal costs 1.5–10 points at ≥ 70%. A constant alpha = 0.5 is a competitive default, which the paper now says. Across all 66 VGG settings, BMIPrune has the best mean rank of six methods and the smallest worst-case shortfall to the best method (7.8 vs. 17.6–62.0 points; new Table in §4.3).

Evaluation scope:
The revised paper adds ImageNet-1k with two pretrained networks (R7), three independently trained checkpoints for each of two settings (R8), a native Taylor baseline (R9), training-set calibration with 10–75 images per class (R3), latency and matched FLOPs (R6), and the block->stage ResNet hierarchy on ImageNet ResNet-18. On retraining, the paper now frames BMIPrune as a fine-tuning-free method: after fine-tuning, methods with trainable masks recover to similar accuracy, and we report it that way.

Novelty:
As you note, the intuition that early layers are general and late layers discriminative is known. The revised paper's contribution is to measure it per layer without training or tuning, to show that the measurement is faithful and reproducible (R5, R8), to reveal where it departs from the intuition (the final-layer rise), and to turn it directly into a one-shot pruning method that is the most consistent across networks.

Q1: Number of evaluations and wall-clock time:
L*|Q|+1 BN-recalibrated forward passes over the calibration set (49 for VGG11, 79 for VGG16, 97 for VGG19), computed once for all targets: 87–289 s per network on a laptop GPU with 1.2–1.3 GB of GPU memory, versus 3–8 s for the one-shot baselines, and no gradients or training. With 10 images per class (R3) the cost falls further at unchanged accuracy. Added to App. A.

Q2: How dependent is the method on the Gaussian assumption?
Weakly, in practice (R4, App. I). Replacing the Gaussian by a zero-inflated Gaussian or a 16-bin histogram leaves the LocalBMI ranking essentially unchanged (median ρ ≥ 0.998, worst layer ≥ 0.96). The Gaussian overstates information only in the sparsest layers (≥ 30% exact zeros), where the ranking is dominated by magnitude anyway; the revised limitations section quantifies this.

Q3: Profiles under more realistic distributions:
Yes, and the revised appendix shows them (R4, App. I). Under the zero-inflated Gaussian and the histogram model, the first-layer alpha is unchanged (≈ 0.98); alpha shifts by at most ~0.1, towards magnitude, and only in the sparsest CIFAR-100 layers, while elsewhere the profiles stay close to the Gaussian one.

Reviewer_g8FJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an activation-based approach to pruning convolutional neural networks. The key innovation over prior art is that this work proposes (heuristically) combining two measures of filter importance - activation magnitude and discriminative capacity (measured by bayes' mutual information). This is then combined with a layerwise sparsity allocation methodology to obtain better sparsity-accuracy tradeoffs. Experiments are conducted using CIFAR10/CIFAR100 datasets on VGG and ResNet models.

Critical strengths
  • This paper has an interesting approach to identifying filter importance in CNNs. In particular, combining activation magnitude with discriminative capacity is interesting, particularly as it addresses the known problem of identifying important components in early layers (close to input) in CNNs.
  • The paper is generally quite well written and easy to follow, with a few minor nitpicks (would have like Rlj(⋅)R_{lj}(\cdot) to have been defined more clearly, for instance).
  • The key derivations, while many being heuristic (such as eq 11), are technically correct to the best of my knowledge, and are generally easy to follow.
Critical weaknesses

The key crucial weaknesses of this work are in it's experimental slate.

  • First, the datasets used in this paper appear to be quite old and quite simple. At the very least, experiments on Imagenet need to be provided in order to show the effectiveness of this approach at scale.
  • Additionally, to further support the value of this work in modern settings, it would be good to see these techniques adapted to vision transformers at the very least. Ideally, experiments would also be done on small generation tasks as well, though this may be out of scope.
  • I think additional ablations to clarify where the improvement is coming from would be useful as well. For instance, for a fixed sparsity allocation mechanism (both fixing sparsity a priori or using the method proposed in this work) it would be interesting to see how the different saliency choices (only activations, only discrimination, different combinations of the two) would affect the final performance. This is especially true since the key saliency SljS_{lj} is essentially a heuristic combination of the activation magnitude and the mutual information.
  • The sparsity allocation section (section 3.4) could be elucidated with algorithm pseudocode.
Questions for authors
  • I think that in Equation 6, IljI_{lj} is exactly equivalent to ∑cπcDKL(p(A∣c)∣∣p(A))\sum_c \pi_c D_{KL}(p(A|c)||p(A)) - that is, you're exactly measuring how similar the pooled class-specific activation is to that of the full mixture, correct? could the authors comment on how this differs from prior discrimination aware CNN pruning?
  • Why is the ranking (in terms of activation magnitude or BMI) computed first, and then combined into the saliency SljS_{lj} in equation 11? Would it not make more sense to use BljB_{lj}, or even combine the two measures of importance first (maybe with some sort of normalization to ensure scales match), and then rank?
  • Since this work relies on pooling activations, does that not remove significant correlation structure? After all, two activations could be [1,0][1,0] and [.5,.5][.5,.5], which both pool to the same number, and yet are significantly different.
1 comment

Dear reviewer,

Thank you very much for the positive assessment of the approach and for the precise technical questions. Each pointed out weakness has been addressed in the revised paper. All overall changes to the paper can be found in the general response, along with all the new experiments R1-R9.

ImageNet:
 R7, added to the paper in §4.6, App. J. We pruned pretrained torchvision ResNet-18 and VGG16-BN, calibrated on 10 images per class and evaluated on 40,000 disjoint validation images, one-shot and without fine-tuning.
On ResNet-18 (unpruned: 69.6%), BMIPrune is ahead of GFI-AP, L1 and random at every target. Mean activation is ahead at 30–50% because it concentrates removals in the parameter-heavy last stage and leaves FLOPs largely untouched (−22% vs. −38% FLOPs at 50% parameter removal); at matched FLOPs BMIPrune leads (42.5% vs. 29.8% at 20% FLOPs removed). On VGG16-BN (unpruned 73.3%), BMIPrune keeps 52.5% at 30% removal against at most 22.3% for every baseline. Because the classifier holds 75% of VGG16's parameters, its parameter cost concentrates removal on the last convolutional layer; the revised paper states this explicitly.

Vision transformers and generative tasks:
We agree these matter, and the revised limitations section now explains concretely why they remain future work, as we believe, as you mentioned, that this is out of scope for this paper. The method carries over naturally to ViT MLP neurons (token-pooled responses, with the block-level allocation already used for ResNets), but two components need replacing: BN recalibration, which has no LayerNorm counterpart, and the ReLU-based magnitude weight under GELU.


Ablations under a fixed allocation:
Added in both forms you suggest (§4.4, App. D–E), on five calibration partitions. (i) With BMIPrune's allocation: magnitude only, information only, the measured alpha, a constant alpha = 0.5 and depth schedules (R1), plus mean activation, GFI-AP, L1, random and Taylor, each run through the full allocator (R2), and BMIPrune's own widths filled with random channels. (ii) The same criteria under their own global or uniform allocations. The comparison answers your question directly: the allocation contributes most. Under BMIPrune's allocation, mean activation, L1 and random selection gain 18–29 points at 70–85% removal, and GFI-AP 6 points. The saliency matters only at high compression: up to 60% all saliencies are within 0.6 points (median), and with BMIPrune's widths even random channels come within one point of LocalBMI on average up to 85%. At 90–95% random channels fall 6.3 points behind and LocalBMI has the best mean rank of all saliencies. Either signal alone is worse (−1.5 points for magnitude only, −10.0 for information only).

Pseudocode for Section 3.4:
This was added as Algorithm 1 in the appendix, referenced in §3.4. Thank you for the suggestion!

Q1: Eq. 6 and discrimination-aware pruning: Yes, the mutual information in Eq. 6 equals Σ_c π_c KL(p(a|c) ‖ p(a)), so it measures how far each class-conditional pooled response departs from the full mixture (zero when they coincide). The revised text states this identity after Eq. 6. It is therefore a discrimination measure of the same family that prior discrimination-aware pruning relies on: class-separability scores in GFI-AP and TVSPrune, mutual information with the labels in HRel, and discrimination losses in DCP. The difference lies in how it is used. Those methods rank filters by class discrimination alone and apply the same criterion to every layer (DCP and TVSPrune also add optimization or iterative pruning). BMIPrune never ranks by information alone. It weights the information by activation magnitude and splits the score exactly into M and D, whose ratio sets, per layer and without tuning, how much discrimination should count. The data support this design: information alone is 10.0 points worse at ≥ 70% removal (App. E), and the M/D split is a faithful description of each layer, validated against held-out probes (§4.5).

Q2: Why rank first, then combine:
Mean activation and mutual information have different units and heavy-tailed, layer-dependent scales. Combining ranks is scale-free, so alpha directly sets each signal's share of the within-layer ordering; with raw-score fusion, alpha's meaning would depend on an arbitrary normalization. The revised §3.3 adds this sentence.

Q3: Pooling discards spatial structure:
You are correct, and the revised limitations section lists it. Pooling matches the decision being made (a filter is kept or removed as a whole) and is standard for activation-based criteria such as DropNet and GFI-AP. It also keeps the class-conditional estimates stable with very few calibration images: 10 per class suffice (R3). Spatially resolved statistics are named as an extension.

Reviewer_cmpDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BMIPrune, a structured CNN pruning method that combines activation magnitude and class information. It fits class-conditional Gaussian distributions to pooled channel responses, estimates mutual information, and uses a layer-specific weight to combine magnitude and information rankings. Layer-wise pruning probes then determine how the pruning budget is distributed. Experiments on VGG networks and ResNet56 show promising results, particularly under aggressive parameter reduction on CIFAR-10. However, the evaluation does not clearly separate the benefits of the ranking criterion from those of budget allocation, and broader validation is needed.

Critical strengths
  1. Motivation is clear. Combining generally useful early-layer features with more class-specific later-layer features is a reasonable approach to channel ranking.

  2. Promising results at high compression. On VGG19/CIFAR-10, BMIPrune retains 86.3% accuracy at 80% parameter removal and 59.6% at 90%, substantially outperforming the listed alternatives.

  3. Transparent discussion of limitations. The paper acknowledges the Gaussian approximation, limited architectural coverage, overlapping calibration partitions, and weaker ResNet56/CIFAR-10 results.

Critical weaknesses
  1. Generalization evidence is limited to CIFAR and a few architectures. Please include additional checkpoints, broader benchmarks, and stronger retraining-free baselines.

  2. The benefit of the new ranking is unclear because budget allocation also changes. Please compare different ranking criteria using the same allocation procedure.

  3. The adaptive mixing weights lack direct validation. Please compare them with fixed weights and a simple depth-based schedule.

  4. The calibration set is large and drawn from the original test set. Please test smaller calibration sets drawn from training or validation data and evaluate on the full test set.

  5. Parameter savings do not establish practical speedups. Please report accuracy at matched FLOPs or latency, together with total pruning time and memory usage.

  6. Validate the Gaussian information estimates against a zero-inflated or nonparametric alternative, at least on representative sparse channels.

  7. Report uncertainty across independently trained checkpoints; calibration splits alone do not measure training variability.

  8. Clarify channel ordering when the allocation weight bℓ=0b_\ell=0, since Eq. (14) then assigns identical global scores within that layer.

  9. After fine-tuning, BMIPrune and GFI-AP reach nearly identical VGG16/CIFAR-100 accuracy in Table 1. Distinguish the advantage before retraining from the advantage after retraining.

Questions for authors

See Weaknesses.

1 comment

Dear reviewer,

Thank you for the concrete and actionable list. We ran every requested experiment. All overall changes to the paper can be found in the general response, along with all the new experiments R1-R9.

  1. Generalization: Added ImageNet-1k with ResNet-18 and VGG16-BN (R7, App. J), three independently trained checkpoints each for VGG19/CIFAR-10 and VGG16/CIFAR-100 (R8, App. H), and first-order Taylor importance, both natively (R9) and under BMIPrune's allocation (R2). Taylor is the strongest baseline on CIFAR-100 up to 60–70% removal, which the paper now reports; on CIFAR-10 it degrades early (VGG11 at 70%: 64.1% vs. 81.0%), and at 80–95% BMIPrune is ahead on all six models. The baselines now cover magnitude (mean activation), class information (GFI-AP), gradient (Taylor), data-free (L1) and random criteria. HRank sets per-layer rates by hand and relies on fine-tuning, so it defines no allocation that can be evaluated one-shot.

  2. Ranking vs. allocation: Every criterion now runs through BMIPrune's full pipeline on five partitions (R2, §4.4, App. D); per-layer widths agree within 3% of channels, isolating the criterion. The allocation carries most of the gain and transfers: at 70–85% removal it lifts mean activation by 17.6 points, GFI-AP by 6.2, and L1 and random by 24–29. Up to 60% the criterion barely matters (median spread 0.6 points); at 90–95% it decides, e.g. random falls to 33.9% vs. 59.6% for LocalBMI on VGG19/CIFAR-10 at 90%. LocalBMI has the best mean rank of seven criteria at 70–95%, with GFI-AP close behind. The claim that metric and allocation "cannot be separated" is removed.

  3. Adaptive weights: Added (R1, App. E). One signal alone costs 1.5 (magnitude) to 10.0 points (information) at ≥ 70% removal. The measured profile beats a schedule between its own endpoints (+0.85), but a constant alpha = 0.5 (+1.1) and a 1→0.5 depth schedule (+0.6) are slightly better, so the paper presents alpha as a tuning-free, readable weight, not an optimal one (§4.5).

  4. Calibration: Added (R3, App. G). Calibrating on training images changes BMIPrune by ≤ 0.4 points up to 70% removal and improves it by 1.3–2.0 points at 80–90%. With 10–75 training images per class, accuracy on the full test set changes by ≤ 1.3 points, the method ranking is unchanged and redundancies move by ≤ 0.004. Ten images per class suffice.

  5. Speed, time and memory: Added (R6, App. A). Pruning needs L·|Q|+1 BN-recalibrated forward passes (49–97), i.e. 87–289 s per network on a laptop GPU for all 11 targets, 1.2–1.3 GB of memory and no gradients (baselines: 3–8 s). Compacted networks run 1.7–2.0× faster at 90% removal. At matched FLOPs, uniform L1 and random lead because they thin the high-resolution early layers; a FLOPs-based cost is named as an extension.

  6. Gaussian estimates: Added (R4, App. I). Under a zero-inflated Gaussian or a histogram, the LocalBMI ranking is unchanged (median Spearman ρ ≥ 0.998, worst layer ≥ 0.96). In the sparsest layers (≥ 30% zeros) the Gaussian overstates information without materially changing the selected filters, now a stated limitation.

  7. Independent checkpoints: Added (R8, App. H), trained with the original recipe, differing only in the seed. On VGG19/CIFAR-10, BMIPrune keeps 87.8 ± 0.5% at 80% and 57.9 ± 2.7% at 90% (original: 86.3%, 59.6%), with seed SD ≤ 0.5 points up to 80% vs. up to 5.8 for the baselines. Rankings and the alpha profile (within 0.02) are reproduced.

  8. b_l = 0: Thank you for spotting this. Such a layer's filters all get G = 1, so it is reached only after every other layer; in all 330 saved masks it is never pruned. §3.4 now states this and that ties in G are broken by the LocalBMI quantile.

  9. Before vs. after retraining: Agreed. BMIPrune's contribution is the fine-tuning-free regime; after fine-tuning, masks with a trainable path recover similarly (72.2% vs. 72.1% for GFI-AP, VGG16/CIFAR-100 at 80%). The paper now reports this as recovery and moves fine-tuning to App. K.

Comments on the paper

1 comment

We would like to thank all three reviewers for their careful and constructive reviews. We have run every requested experiment within the paper's scope, added each result to the revised paper, and, most importantly, revised how the paper positions its contribution. We summarize both here; the individual replies point to the exact sections.

Revised positioning: The reviews made clear that the submitted version claimed too broadly. The revised paper now places BMIPrune where the evidence puts it: A one-shot, fine-tuning-free structured pruning method whose claim is consistency, not dominance. Across all 66 VGG model–target settings and six methods, BMIPrune has the best mean rank (2.02 vs. 2.29 for the next best, first-order Taylor) and by far the smallest worst-case shortfall to the best method in any setting (7.8 points vs. 17.6–62.0 for every baseline). It is best or tied in 25 of 30 settings at ≥ 70% removal. Every baseline wins somewhere and fails badly elsewhere; we now say so explicitly, with the reason for each failure (new Table in §4.3). The five-partition ablation shows where the consistency comes from: mainly the layer allocation, which also lifts other criteria by 6–29 points when they adopt it (R2). An interpretable per-layer decomposition: GlobalBMI's magnitude/class-information split is now presented as a measurement of how each layer encodes class evidence, validated against independent held-out probes and reproduced across independently trained networks (new §4.5). We deliberately frame this as layer-wise representation analysis, alongside linear probes and the information plane, rather than as a method for explaining individual predictions.

The abstract, contributions, discussion and conclusion have been rewritten accordingly. Claims about α being optimal, about the metric "driving" the allocation, and about fine-tuned advantages have been removed or narrowed.

New experiments, all added to the revised paper:

R1 / Layer-profile ablation (6 models x 5 partitions) / §4.4, App. E Main result: Information only −10.0, magnitude only −1.5 points at ≥ 70% removal; the measured profile beats a linear schedule between its own endpoints (+0.85), while a constant alpha = 0.5 (+1.1) and a 1→0.5 depth schedule (+0.6) are slightly better, so alpha is reported as a tuning-free default, not an optimum.

R2 / Every criterion through BMIPrune's full pipeline (mean activation, Bayes MI, GFI-AP, L1, random, Taylor) / §4.4, App. D Main result: Five partitions. Widths agree within 3% of channels for every criterion. The allocation carries most of the gain and transfers: at 70–85% removal it lifts mean activation by 17.6, GFI-AP by 6.2, L1 and random by 24–29 points. The criterion is irrelevant up to 60%; at 90–95% random channels fail (−7.7) and LocalBMI has the best mean rank, GFI-AP close behind.

R3 / Calibration on training images (5 draws) and with 10/25/50/75 images per class, full test set / §4.6, App. G Main result: BMIPrune changes by ≤ 1.3 points across calibration sizes and ≤ 2 points vs. test-image calibration; method ranking unchanged across sizes; layer redundancies change by ≤ 0.004.

R4 / Zero-inflated Gaussian, histogram and kNN estimators / §4.6, App. I Main result: LocalBMI ranking unchanged (median Spearman rho ≥ 0.998, worst layer ≥ 0.96).

R5 / Independent held-out probes of class information (74 layers) / §4.5, App. F Main result: D agrees with probe information (rho = 0.85) and tracks CIFAR-10 → CIFAR-100 changes of the same layer (rho = 0.98 over 37 same-layer pairs, so depth is held fixed); 0.65 pooled after controlling for depth.

R6 / Cost and practical speed / §4.6, App. A Main result: 49–97 BN-recalibrated forward passes, 1.5–5 min per network on a laptop GPU, 1.2–1.3 GB GPU memory; compacted networks 1.7–2.0x faster at 90% removal.

R7 / ImageNet-1k, pretrained ResNet-18 and VGG16-BN / §4.6, App. J Main result: ResNet-18: ahead of GFI-AP, L1 and random at every target; VGG16: 52.5% vs. ≤ 22.3% top-1 at 30% removal.

R8 / Independently trained checkpoints, 3 seeds each (VGG19/CIFAR-10, VGG16/CIFAR-100) / §4.6, App. H Main result: Ranking, accuracy and alpha profile reproduced; BMIPrune's seed SD ≤ 0.5 points up to 80% removal on VGG19.

R9 / Native first-order Taylor baseline, all six VGG models / §4.3, App. B Main result: Strong on CIFAR-100 up to 60–70% removal, weak on CIFAR-10; BMIPrune ahead on all six models at 80–95%.

Scope: Vision transformers and generative tasks remain outside the paper's scope. The method extends naturally to ViT MLP neurons, but two components would need replacing: BN recalibration (LayerNorm has no running statistics) and the ReLU-based magnitude weight under GELU. The revised limitations section says so.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.