acceptodds
Under review as a conference paper at ICLR 2027

Don’t Just Prune-and-Fine-Tune: Optimal Information Migration

Abstract

Structured pruning typically follows a simple recipe: rank, remove, and fine-tune. We argue that this conflates structural removal with functional discard. A structural unit may no longer warrant an independent place in the compressed network, while its learned contribution need not be discarded with it. Two compressed models can therefore have the same final architecture yet retain very different amounts of pretrained functionality, depending on how these contributions are handled before deletion. We reformulate structured pruning as a migration-before-deletion process and propose Optimal Information Migration (OIM). Information Preservation Priority (IPP) selects retained targets, while OIM uses optimal transport to migrate pretrained contributions onto those targets before deletion. We further introduce DeepInversion-based BatchNorm recalibration (DI-BN) to correct compression-induced statistical mismatch without real training data or updates to learnable parameters. Across CNNs and a vision Transformer on CIFAR-10/100 and ImageNet-1K, OIM consistently preserves substantially more pre-fine-tuning accuracy than direct pruning at the same compression budget and final architecture. On ResNet-50/ImageNet-1K at 30.26% FLOPs reduction, Top-1 accuracy improves from 15.79% after direct pruning to 42.15% with OIM and further to 52.55% with DI-BN.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Optimal Transport, Information Migration, BN Recalibration, Structured Pruning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_c4QCOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studied the structured compression for vision models without access to the original training data or fine-tuning. The authors takes Information Preservation Priority (IPP) to select retained channels using the maximum of their incoming and outgoing squared weight norms. Optimal Information Migration (OIM) constructs descriptors from structurally coupled parameters, solves an optimal transport problem between the original channels and the retained subset, and adds weighted downstream contributions from removed channels to retained ones. A subsequent DeepInversion-based procedure, DI-BN, recalibrates BatchNorm statistics using 256 synthetic images.

Critical strengths
  1. The research question is interesting.
  2. The experiments cover several relevant architectures and datasets.
  3. The downstream update is concrete and interpretable. Equation (6) clearly identifies which weights are changed and which are preserved. Using dependency groups to coordinate structural removal is appropriate, and Figure 1 helps explain the procedure.
Critical weaknesses
  1. The contribution relative to the closest prior work is not sufficiently established. There are some highly relative works like Model Folding[2], IFM[1]. The comparison to these methods is missing.
  2. Similarly, the DI-BN procedure described in Section 3.3 closely follows the published Fold-DIR procedure in Model Folding paper[2]: synthesize inputs using the dense model’s BN statistics, then update the compressed model’s BN statistics without weight fine-tuning. The submission cites Model Folding, but does not explain what, if anything, is technically new in DI-BN.
  3. The absence of Direct + DI-BN leaves a central experimental question unanswered. Direct versus OIM provides useful evidence for migration when the dense model’s BN statistics are retained. OIM versus OIM + DI-BN then measures the benefit of recalibration for the migrated model. However, this does not show how much OIM helps when direct pruning receives the same recalibration.
  4. One thing I’m confusing is that the transport objective is not connected closely enough to the functional preservation claimed for the update.

For example, consider a local compression step with fixed hidden activations hi(x)h_i(x), followed by a linear layer with outgoing weights GiG_i. Equation (6) gives the local preactivation discrepancy

\left( h_p(x)-\sum_j M_{pj}h_{q_j}(x) \right).$$ Thus, functional preservation depends on how well the retained activations reconstruct each deleted activation, weighted by its downstream contribution. The OT objective instead uses distances between normalized, concatenated parameter descriptors. The submission provides neither a bound relating these quantities nor a quantitative evaluation of this reconstruction error. 5.The paper does not adequately specify how a global compression budget is converted into per-layer or per-dependency-group retained widths. It also leaves unclear the group-processing order, whether descriptors and scores are recomputed after earlier updates, and the precise parameter slices included in each descriptor. [1] Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory https://arxiv.org/abs/2310.06756 [2] Forget the Data and Fine-Tuning! Just Fold the Network to Compress https://arxiv.org/abs/2502.10216
Questions for authors

1.What is the precise methodological advance over Intra-Fusion’s OT-based aggregation and Fold-DIR’s statistics repair? Can the benefit of downstream-only writeback be demonstrated under matched compression and calibration settings?

2.How does Direct + DI-BN compare with OIM + DI-BN when both use exactly the same synthetic images and recalibration protocol? This is the most important missing control for interpreting the final pipeline.

Reviewer_tTn5OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a reframe of pruning as migration before deletion. IPP, DI-BN, and OIM are presented. IPP selects retained channels by the maximum of the squared filter norm and the squared norm of the corresponding next-layer input. OIM uses balanced optimal transport. DI-BN recomputes BatchNorm statistics with DeepInversion images. The evaluation covers pre-fine-tuning accuracy against direct deletion, IPP against L1, L2, and FPGM, and results after 100 epochs of KD fine-tuning. the evalaution use cifar10 cifar 100 and imagenet 1k.

Critical strengths

S1: The framing and the "rethinking" are clear, and Figure 1 explains the mechanism well. S2: The method is simple and data-free. It consistently improves over direct deletion across CNNs and DeiT-T.

Critical weaknesses

W1: The value of the Optimal Transport matching is not demonstrated or shown. The paper compares only against direct deletion, never against simple redistribution rules (uniform or random with the same totals). Without these controls, it is unclear what the contribution of the mechanism. W2: The no-fine-tuning claim conflicts with Tab. 1. The title and abstract emphasize avoiding fine-tuning, but all competitive results use 100 epochs of KD on real data. Data-free accuracy remains far below dense; the narrative is a bit unfair. W3: Tab. 1 reports no standard deviations. Most margins are within CIFAR seed variance: +0.08 over DepGraph on ResNet-56 and +0.07 over LIAP on ResNet-110. W4: Before deleting a channel, OIM adds its next-layer weights onto a kept channel. This only works if the two channels produce the same output. The method never checks that. It compares weights, not outputs, and it ignores size, so if one channel’s output is twice as big as the other’s, the result is wrong. So "optimal" just means "optimal by the paper’s similarity score" and not "keep the network working the same." W5: IPP decides which channels to keep using two numbers per channel: X size of the channel’s own weights) and Y (size of the weights the next layer uses to read that channel) The bound the paper cites says the damage from deleting a channel depends on X * Y. But IPP uses the bigger of the two instead. That can go wrong. If X is large and Y is almost zero, the product is almost zero, so deleting the channel is harmless. IPP still ranks it as important, because X is large. Two more issues.. With BatchNorm, X barely matters, because BatchNorm rescales the output anyway. And IPP beats simple L2 only by small margins, with no error bars. It is also never tested without migration, so we don’t know whether it helps migration...

MW1: Sec. 4.4 contains only qualitative CAMs on selected examples, no counterexamples. MW2: The BatchNorm-statistics analysis promised in the introduction is absent. MW3: In Tab. 1, results are bolded by absolute accuracy despite differing base accuracies, which overstates the advantage of the method (example: ResNet-110 at about 60%).

Questions for authors

Basically, to be accepted, all the main weaknesses should be fixed. I'll rephrase what I already pointed out in weaknesses: (W1): Prove that Optimal Transport matching contributes to the results; the bare minimum is to compare against simple redistribution rules (uniform or random with the same totals). (W2): Fix the narrative of the papers (W3): Claim only what the data support (W4): Define optimal or drop that claim (W5): Justify using max(X, Y) instead of the product X * Y from the cited bound, or switch to the product; explain why X is meaningful in BatchNorm networks, where BatchNorm rescales the output; show that IPP helps migration specifically by comparing it with L2 under direct deletion, with error bars.

Reviewer_SDMROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed an neural network pruning approach called Optimal Information Migration (OIM), trying to preserve (they called it migrate) a structure unit's information before removing it. This is achieved by selecting a target support in the network and transfer the knowledge/information in the unit to be removed. The proposed approach was tested with traditional CNNs and image classification benchmarks (CIFAR and ImageNet).

Critical strengths
  • The motivation to use information migration for network pruning is clear.
  • The method is presented clearly and the target selection is studied promisingly.
  • The paper is well organized and easy to follow.
Critical weaknesses
  • Network pruning on CNNs has been substantially developed and there's little room for traditional methods. The proposed work confines itself into a relatively baseline framework.
  • The proposed method is somehow similar to the existing work Intra-Fusion [1]. It is unclear what the difference is between the two, and therefore the novelty is somehow limited. Performance is also not compared to it.
  • The baselines in the experiments are somehow outdated. Literatures in recent years should be involved for evaluation. But I believe the numbers presented can hardly outperform recent SOTA works.
  • The scope of the proposed OIM+DI-BN is confined to the structures with BNs. And there is no ablation studies to show the effectiveness of DI-BN.
  • It is hard to functionally tell the difference between the proposed migration before deletion scheme and traditional finetuning. Finetuning can more or less work as what the "migration before deletion" did. In this case, the proposed method becomes much less useful.

[1] Towards Meta-Pruning via Optimal Transport. ICLR'24

Questions for authors

Please see the Weaknesses part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.