acceptodds
Under review as a conference paper at ICLR 2027

TROPICS: Tropical Geometry for Structured Neural Network Compression through Zonotope Approximation

Abstract

Structured compression of neural networks remains challenging when high accuracy must be preserved under severe parameter reduction. We introduce Tropical Representation for Optimized Pruning, Informed Clustering, and Scaling (TROPICS), a data-free, non-uniform, and single-pass framework based on the tropical geometry of ReLU neural networks. We compress layers in two phases. First, neurons are mapped to zonotope generators and clustered using K-means, merging inputs by mean and outputs by sum, while scale factors recover the signal energy lost during clustering. Second, to avoid forcing dissimilar generators to merge, we magnitude-prune the weakest remaining units. TROPICS requires neither training data nor fine-tuning, and it substantially outperforms existing baselines on VGG16-BN (CIFAR-10): our scaled Mixed strategy achieves an 84.7% parameter reduction with only a 0.44% accuracy drop, while at 92.5% parameter reduction the scale factors preserve up to 29% more accuracy than unscaled variants. On ResNet18 (ImageNet), TROPICS surpasses strong methods such as TropNNC, CUP, and Model Folding at comparable compression.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Neural Network Compression, Tropical Geometry, Structured Pruning, Zonotopes, ReLU Networks, Support Function

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7ptaOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes TROPICS, a post-training, data-free, one-shot, structured compression method for ReLU based simple networks. Each neuron or filter is mapped to a “zonotope generator which scales its input weights and bias by the norm of its output weights. Generators are standardized, L2-normalized and clustered with K-means. Each cluster is merged by averaging input weights and summing output weights, followed by two scale factors that restore variance (inputs) and L2 energy (outputs) which were lost in the merging process. An optional second phase magnitude-prunes the weakest merged units, and batch-norm statistics are merged with the law of total variance. The paper provides a theoretical error bound, based on the Hausdorff-distance between the original and compressed layer zonotopes, and reports that this bound correlates with measured accuracy drop (using both Pearson and Spearman metrics). Experiments cover relatively small models, such as VGG16-BN on CIFAR-10/100 and ResNet18 on ImageNet, with comparisons to TropNNC, CUP, Model Folding and three classic filter-pruning methods. The results show ~85% parameter reduction on VGG16-BN/CIFAR-10 with under 1% drop and no fine-tuning, and wins over other methods on ResNet18/ImageNet.

Critical strengths

S1. A Practical and desireable settings which is: Data-free, fine-tuning-free, with one-pass method which is simple to implement. Anonymous code is provided.

S2. The reporting is responsible and mostly careful. Many tables and graphs report mean ± std/error-bars over five K-means seeds, give parameters and FLOPs together, and include both scaled and unscaled variants. The paper also honestly states where it loses (aggressive FLOPs reduction on ImageNet against Model Folding).

S3. The appendix presents a variety of measurements and results to backup the claims of the paper

S4. The scale factors matter, and the ablation shows it. On VGG16-BN/CIFAR-10, the extreme setting (92.5% reduction) retains 71.3% accuracy with scaling versus 40.7% without (Table 2). Similar gaps appear for clustering at 87–88% reduction and on CIFAR-100. This is a clear, reproducible effect, and the explanation (signal decay through deep non-residual stacks) is plausible.

S5. A principled motivation is attempted. A weight-only error bound, derived in the appendix, is more than most data-free heuristics offer.

Critical weaknesses

W1. Limited scope and practical evidence. Architectures are small or old: custom MLP/CNN, VGG16-BN on CIFAR, and ResNet18 on ImageNet only. At most about 35% parameter reduction is shown on ImageNet (8% drop), which is not a strong practical result. VGG specifically is known as a compressible model. The method does not compress residual connections and mostly targets early layers on ResNet, so it does not handle the architectures in common use. No modern CNNs (MobileNet/EfficientNet), transformers or non-ReLU activations are tested, though the conclusion suggests extension. No wall-clock latency or memory measurements are given for the “structured” speedup claims, and FLOPs are reported analytically only. Hardware speedup for irregular per-layer ratios is not guaranteed. There is no comparison with a short fine-tuning budget, which would show how much headroom the data-free setting leaves, and whether TROPICS’ compressed models are a better starting point than baselines’.

W2. The bound is presented as a “reliable estimator” of accuracy drop, but the evidence does not support that. The Hausdorff distance is between layer zonotopes in the weight space. Not on the network output. No result relates this distance to the change in activations or logits. There is no propagation through depth, batch norm, pooling or residual connections, so how per-layer bounds combine into a network-level number is unclear. It should be stated precisely how the bound in Tables 14–17 is aggregated across layers. The correlation claim is cherry-picked and weak. The abstract and contributions quote “peaking at 0.97 and 0.98”. Table 14 shows values from 0.84 to 0.97 for Pearson depending on the setting. Each cell uses only n = 15 hand-chosen configurations from a single network, and the configurations span a narrow range, with many drops below 0.4% (Table 15). A few large points can dominate Pearson’s r. Rank correlation over many near-zero drops is weak evidence. Please report all cells, confidence intervals, and a pooled correlation across configurations. The paper shows the bound increases along with the measured accuracy drop within each network, but that does not make it a validated predictor.

W3. Novelty relative to TropNNC, Misiakos et al., and non-tropical neuron-merging work. TropNNC and Misiakos et al. already use zonotope approximation and Hausdorff distance for compression. The incremental pieces for this work appear to be K-means with output-norm scaling, the two scale factors, a magnitude-pruning phase, and BN merging. The related-work section is two short paragraphs and does not clearly state what is new versus TropNNC, which the paper describes only as having “an iterative optimization phase.” Closely related data-free merging and compensation methods are not discussed. The most obvious is Neuron Merging (Kim et al., 2020), which merges neurons data-free with compensation, and the neuron-merging/model-fusion line more broadly (please verify the exact references). Data-free pruning baselines such as DFPC, or plain L1/L2 filter-norm pruning and FPGM’s data-free variant on the same networks, are also missing as controls.

W4. Hyperparameters are hand-tuned on the test set, and the headline numbers are the best of many. Section 5.1 states that per-layer ratios p and r were set manually (“a manual selection of p and r was set regarding our scope”), with different strategies for parameter versus FLOPs reduction. All results are evaluated on the test sets, with no validation split. Tables 2–3 contain 20+ configurations per model. The method is advertised as “non-uniform” and “single-pass”, but the ratios come, as far as I understand, from trial and error, which is a search cost the comparison does not count. Baselines such as TropNNC and Model Folding appear to use uniform or automatically set ratios, so the comparison is not exactly apples tp apples. Please describe how layer ratios were chosen, give a rule or automatic procedure, and report results using a validation split.

W5. inconsistent reporting A. Abstract: "on VGG16-BN (CIFAR-10): our scaled Mixed strategy achieves an 84.7% parameter reduction with only a 0.44% accuracy drop" Section 5.2: "On VGG16-BN, the extensive compression finds our Mixed strategy achieves only a 0.58% drop at a massive 84.7% parameter reduction" The later is reported also in Table 2.

B. The abstract says scale factors preserve “up to 29% more accuracy” at 92.5% reduction, while Section 5.2 and Table 2 reports “over 30%" and 30.6%, respectively..

C. The abstract says TROPICS “surpasses ... Model Folding” on ResNet18, but Section 5.3 says it outperforms only at mild FLOPs reduction (below about 23%) and loses at aggressive reduction

D. Table 1’s comparison with classic pruning methods uses different baseline networks. The “TROPICS (baseline)” accuracies (94.27, 93.91, 94.02) are higher than the original papers’ baselines (93.25, 93.58, 93.66), so these are not matched models. The text asserts “we can reasonably assume” the competitors do not reach zero drop without retraining. This should be measured or removed. Only the FPGM row is a like-for-like data-free comparison, and there it needs the same network.

Questions for authors

Q1. How are per-layer ratios chosen, and can this be automated? How do results change if they are selected on a validation split? Q2. How is the network-level bound computed from per-layer Hausdorff distances, and is there any theoretical link from layer-wise Hausdorff distance to output error?

Q3. What does the tropical view contribute to the algorithm, beyond the bound, compared with merging neurons by cosine similarity plus norm-restoring scaling? Please include an ablation without standardization and with plain filter clustering. Q4. What exactly is new relative to TropNNC? Can you compare directly with Neuron Merging and data-free norm pruning at matched ratios? Q5. Can you evaluate on a modern CNN (e.g. MobileNetV2 or ResNet50) and measure real latency? Q6. Please fix or explain the discrepancy between the 0.44% (abstract) and 0.58% (Table 2) figures. Q7. Why does the bound increase with scale factors in Table 15 even though accuracy improves? How should this be interpreted?

4 comments

We thank the reviewer for the detailed review.

W4 / Q1 — How per-layer ratios are chosen; automation; test-set tuning

We accept that a manual search over configurations is not a fixed rule, and we have replaced it with one: Given a single global FLOP or parameter target τ, each compressible layer keeps a fraction r_l = clip(1 − α·w_l, 0.05, 1) of its filters. Here w is a fixed, hand-set intensity vector per stage (front-loaded for FLOPs, back-loaded for parameters). Brent's method solves for the unique α at which the exact linear cost model reaches (1 − τ) of the original cost. The 75/25 split fo cluestering/pruning is an empirical choice, fixed across all experiments. Automatic rule, VGG16 on CIFAR-10 vs FPGM, oiginal acc 93.33%

Param. red.40%50%60%70%80%
FPGM (data-free, same net)72.7654.5738.9314.1514.68
L1-norm pruning after de-dup89.3787.1979.9663.2034.15
TROPICS-auto (scaled)91.6390.7389.6387.7985.10

Automatic rule, ResNet-18 on ImageNet (single shared checkpoint at 69.32%, full 50k validation set, 3 seeds), compared with TropNNC and CUP at matched FLOPs:

FLOPsBaselineTROPICS-auto (scaled)
3.43 GTropNNC: 54.8364.71 ± 0.02
3.22–3.34 GCUP: 38.66 / TropNNC: 56.5761.38 ± 0.29
3.06 GCUP: 12.9651.04 ± 1.24
2.89 GTropNNC: 39.1141.81 ± 2.16

W3 / Q3 / Q4 — tropical geometry contribution; Neuron Merging, norm pruning, FPGM

**Ablation requested in Q3.

Dataset(param. red.)Raw filter weightsGenerators, no standardisationGenerators (TROPICS)
CIFAR-10(84.7%)90.8591.3191.61
CIFAR-10(87.6%)83.9188.6789.93
CIFAR-10(88.2%)84.7987.4989.28
CIFAR-10(86.1%)36.5682.9383.75
CIFAR-10(91.7%)37.9353.7384.94
CIFAR-10(92.1%)35.8247.6684.23
CIFAR-100(67.0%)52.1957.5362.80
CIFAR-100(72.9%)24.2728.0953.71
CIFAR-100(75.0%)28.5212.1554.18
CIFAR-100(80.6%)12.4427.0241.81
CIFAR-100(80.1%)10.8711.3940.34

(Top-1 %, VGG16-BN, scale correction on, identical per-layer budgets. Originals: 92.09% on CIFAR-10 and 73.21% on CIFAR-100.) . Neuron Merging (Kim et al., 2020) and data-free norm pruning, at identical per-layer ratios:

Dataset(param. red.)TROPICSNeuron MergingNorm pruning
CIFAR-10(84.7%)91.6191.5890.96
CIFAR-10(87.6%)89.9386.8383.33
CIFAR-10(85.0%)90.8891.7189.67
CIFAR-10(86.1%)83.7587.5328.41
CIFAR-10(91.7%)84.9464.3039.63
CIFAR-10(92.1%)84.2359.9428.82
CIFAR-100(55.9%)56.5724.794.81
CIFAR-100(72.9%)53.7126.979.71
CIFAR-100(75.0%)54.1833.6618.90
CIFAR-100(80.6%)41.8111.004.06
CIFAR-100(80.1%)40.3417.6111.64

(Top-1 %, VGG16-BN, identical per-layer ratios for all three methods.)

W2 / Q2 / Q7 — What the bound is and is not

Bound as accuracy predictors is a promising direction. It seems to have high corellations in some experiments. However, in this work, we present it as a per-layer worst-case certificate on how much each layer's zonotope changes, and we soften the estimator claims. The scale factor restores the norm (energy) of the merged generators. This repairs the typical activation magnitude seen by the next layer, but it can move the zonotope further from the original in the worst-case direction, which is what Hausdorff distance measures.

W1 / Q5 — A modern architecture, non-ReLU activations, latency

We evaluated ViT-B/16 on ImageNet, (GELU, LayerNorm and multi-head attention). The per-layer ratios are set automatically.

Param. red. (ViT-B/16, original 80.25%)7.5%14.9%22.4%29.8%37.3%44.8%
TROPICS (cluster generators)78.9577.9075.7073.0069.2062.95
Same pipeline, cluster raw filters78.8574.8011.0521.4511.358.75
Wall-clock for the evaluation pass (s; original 16.81)16.3315.3915.0714.1513.3912.72

How each component is handled: MLP hidden neurons and whole attention heads are compressed. LayerNorm, residual streams and the QKᵀ softmax interaction are kept intact.

Real speedup. Whole neurons and heads are removed, so the pruned model is a smaller dense network and needs no sparse kernels. The evaluation pass becomes up to 1.32× faster end-to-end (44.8% parameter reduction).

W5 / Q6 — Inconsistencies

We apologise for these; all will be fixed.

We thank the reviewer once more. These changes make the paper stronger.

ResNet-50 on ImageNet, with measured latency (Q5). Ratios are set by the same automatic Brent's-method rule, there is no fine-tuning, and results are mean ± std over 3 seeds. Speedup is measured wall-clock GPU time for a full inference pass over the validation set, compared with the uncompressed model:

FLOPs red.5.1%11.1%16.9%20.9%24.5%30.5%
Param. red.2.9%6.9%10.5%14.3%16.9%20.6%
TROPICS, no scale75.01 ± 0.6367.75 ± 0.4946.43 ± 0.8919.26 ± 7.413.15 ± 0.960.66 ± 0.20
TROPICS, scaled75.79 ± 0.5667.68 ± 0.3946.29 ± 3.0528.17 ± 13.654.20 ± 2.231.62 ± 0.66
Measured GPU speedup (scaled)1.02×1.05×1.07×1.08×1.10×1.14×

(Original model: 79.72% top-1, 8.27 GFLOPs, 60.3 s for the GPU inference pass.)

Thank you for the detailed response and the additional experiments. The ablation, the Neuron Merging and norm-pruning comparisons, and the ViT and ResNet-50 runs address several of my concerns. Below I note what is resolved and what remains. Many of the new results appear only in the reply, so I would like to see them integrated into a revised PDF.

Resolved to some extent Q3: The generator/standardization ablation is informative. So the standardization carries much of the gain. Please mention it clearly in the paper, since standardization has no tropical-geometric justification (seeds and STD will be also welcome). Q5: ResNet-50 and ViT results are welcome, as is the acknowledgment of W5 and Q6.

Other concerns: Q5. evaluate on a modern CNN: reported times cover a full validation pass which is typically dominated by data loading rather than GPU computation. Please report model-only GPU latency (batch size, hardware, warmup) so that data loading does not dominate.

Q4. Neuron Merging comparison. In your new table, Neuron Merging matches TROPICS at some points and is even better at other points. TROPICS is clearly ahead only at ≥87.6% and on CIFAR-100. Please revise the abstract and the “substantially outperforms” claim. Because the specific ratios in the table were tuned for TROPICS, please also report each baseline at its own best ratios. Is this row correct or is the data noisy? CIFAR-100 (75.0%) 28.52 12.15 54.18 it seems non-monotone with the other rows. Also, I could not find a direct statement of what is new relative to TropNNC beyond the numerical comparison. A short paragraph in related work on TropNNC, Misiakos et al. and Neuron Merging would help.

Q1. you note that "The 75/25 split fo cluestering/pruning is an empirical choice, fixed across all experiments." How do you choose it. The headline numbers therefore depend on manual search, please say it also in the abstract.

Q2. I appreciate the move to a “per-layer worst-case certificate”, but my questions were not answered. (a) How is the network-level bound in Tables 14–17 aggregated across layers? (b) Please report the pooled correlation across all configurations, with confidence intervals, and compare against a trivial predictor (e.g. fraction of parameters removed, or total removed generator norm). (c) Since scaling can increase the bound while improving accuracy, please state in the abstract and contributions that the bound does not capture the effect of scaling. The revised paper should remove the “data-free performance estimator” claim, not only soften it.

Assessment. The empirical evidence is stronger than at submission. However, the theoretical contribution is now much narrower, the headline results depend on manual configuration, and the ResNet-50 and ImageNet results show limited practical reach. I will potentially raise my score by 1, conditional on a revised PDF that includes these changes.

We sincerely appreciate your detailed review and insightful suggestions, which have helped strengthen our paper. We have revised the manuscript to the best of our ability to address all your comments within the constraints of the rebuttal period.

Thank you once again for your time.

Reviewer_F2rxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TROPICS, a data-free, one-shot layer-wise compression framework for ReLU networks, motivated by the zonotopic/tropical geometry view of neural layers. Each layer is represented as a zonotope whose generators encode neuron input/output directions and magnitudes. Compression proceeds in two phases: (1) zonotope-guided K-means clustering of normalized generators, followed by cluster merging with input-variance and output-energy scale factors; (2) magnitude-based pruning of the remaining compressed neurons. BatchNorm layers are merged analytically using the law of total variance. The paper also derives Hausdorff-distance-based error bounds for clustering, pruning, and the mixed strategy, and argues that these bounds correlate with observed accuracy drop. Experiments cover FC/CNN models on MNIST/FashionMNIST/CIFAR-10, VGG16-BN on CIFAR-10/100, and ResNet18 on ImageNet, without fine-tuning. TROPICS is compared with TropNNC, CUP, Model Folding, and classic filter-pruning baselines, and reports strong parameter/FLOP reductions at low accuracy drop in several settings.

Critical strengths
  1. The method is easy to implement: cluster normalized generators, merge with scale correction, then prune by magnitude. It is data-free and one-shot, requiring no retraining or fine-tuning, which is attractive for deployment.

  2. The input-variance and output-energy scale factors clearly prevent signal collapse, especially for VGG-style sequential networks under aggressive compression. Tables 2–4 show large gains over unscaled variants, and Figure 2/8 illustrate the zonotope volume recovery intuition.

  3. The law-of-total-variance update for merged running variance is a sensible detail that many compression pipelines overlook. This helps make the one-shot pipeline practical.

Critical weaknesses
  1. The actual algorithm is primarily normalized K-means clustering plus magnitude pruning and scale correction. Zonotopes/tropical geometry enter mainly through the generator construction and the Hausdorff bound. The paper would be stronger if it clarified which design choices are uniquely enabled by tropical geometry versus standard clustering/pruning intuition.

  2. The Hausdorff distance between zonotopes is a geometric quantity, not a task loss. Empirical correlations are encouraging but vary by setting, and the bound is not used to automatically choose per-layer ratios or predict absolute accuracy. It is also unclear whether the correlation holds across different methods, not just within TROPICS sweeps.

  3. Experiments are confined to small/medium vision models (FC, custom CNN, VGG16, ResNet18). There is no evaluation on Transformers, modern ConvNeXt-style models, or LLMs. The conclusion acknowledges this, but the paper’s claims would be more impactful if at least one transformer/LLM structured-pruning setting were included.

  4. TROPICS only compresses the first convolution of each residual block and leaves residual connections, downsampling projections, and the stem unchanged. This is safe but limits aggressive FLOPs reduction; the paper admits degradation beyond a certain scope. Comparisons at matched FLOPs should clearly reflect this restricted scope.

  5. Table 1 compares TROPICS to published filter-pruning methods that use retraining and different training setups. The claim of “strictly smaller budget at zero drop” is suggestive but not a controlled comparison under identical training/compression protocols.

Questions for authors
  1. Does the Hausdorff bound correlate with accuracy drop across different methods (TropNNC, CUP, Model Folding), or only within TROPICS configurations? Can it predict catastrophic collapse before accuracy evaluation?

  2. The method relies on ReLU and zonotope generators. How would TROPICS handle GELU/SwiGLU, LayerNorm, attention, and residual streams? Is the generator construction still meaningful for non-ReLU activations?

  3. For classic filter-pruning baselines, were the baselines reproduced under the same training and compression protocol, or are the numbers taken from the original papers? A controlled comparison would clarify the practical advantage

1 comment

We thank the reviewer for a careful review.

W1

Ablation where everything (per-layer budgets, pruning rule, scale correction) is fixed and only the space in which filters are clustered changes:

DatasetCfg (param. red.)Raw filter weightsGenerators, no standardisationGenerators (TROPICS)
CIFAR-10(84.7%)90.8591.3191.61
CIFAR-10(87.6%)83.9188.6789.93
CIFAR-10(88.2%)84.7987.4989.28
CIFAR-10(86.1%)36.5682.9383.75
CIFAR-10(91.7%)37.9353.7384.94
CIFAR-10(92.1%)35.8247.6684.23
CIFAR-100(67.0%)52.1957.5362.80
CIFAR-100(72.9%)24.2728.0953.71
CIFAR-100(75.0%)28.5212.1554.18
CIFAR-100(80.6%)12.4427.0241.81
CIFAR-100(80.1%)10.8711.3940.34

(Top-1 %, VGG16-BN, scale correction on, identical per-layer budgets. Originals: 92.09% on CIFAR-10 and 73.21% on CIFAR-100.)

What tropical geometry contributes, concretely, is:

  1. The representation being clustered. Each neuron becomes a zonotope generator that folds in its outgoing weights and its BN affine parameters. Two filters that look different in weight space but contribute the same generator are merged; raw-filter clustering cannot see this.
  2. The merge and scale correction. These are derived so the merged zonotope approximates the original one, not from a heuristic.
  3. The certificate (Hausdorff bound). We now present this as a worst-case guarantee, not a predictor (see W2/Q1).

W2

Bound as accuracy predictors is a promising direction. It seems to have high corellations in some experiments. However, in this work, we present it as a per-layer worst-case certificate on how much each layer's zonotope changes, and we soften the estimator claims. The original ratios were set by hand to match each baseline's parameter or FLOP budget, not selected by test accuracy. We accept that a manual search over configurations is not a fixed rule, and we have replaced it with one: -Given a single global FLOP or parameter target τ, each compressible layer keeps a fraction r_l = clip(1 − α·w_l, 0.05, 1) of its filters. Here w is a fixed, hand-set intensity vector per stage (front-loaded for FLOPs, back-loaded for parameters). Brent's method solves for the unique α at which the exact linear cost model reaches (1 − τ) of the original cost. The 75/25 split fo cluestering/pruning is an empirical choice, fixed across all experiments.

W3 / Q2

We added ViT-B/16 on ImageNet (GELU, LayerNorm, multi-head attention) (Original: 80.25%.):

Param. target5%10%20%30%40%
TROPICS (generators)78.9577.9073.0062.9531.20
Raw-filter clustering78.8574.8021.458.750.20

How each component is handled: -GELU. The generator construction is exact only for positively homogeneous activations (ReLU, Leaky-ReLU). GELU equals ReLU up to a bounded, smooth correction, so we use the ReLU generators as an approximation. The certificate then no longer holds formally, and we say so explicitly.

  • Attention. We compress the MLP hidden neurons and attention heads (a head is treated as a unit). The QKᵀ softmax interaction is not piecewise linear and is left uncompressed.
  • LayerNorm and residual streams are kept intact. Unlike BN, LayerNorm normalises per token and cannot be folded into the generators.

W4

The reviewer is right. Because TROPICS compresses only the first convolution of each residual block, at matched parameters it removes fewer FLOPs than CUP or TropNNC. We now report a FLOP-matched comparison (ResNet-18, same checkpoint, 69.32%, full 50k validation set, 3 seeds):

FLOPsBaselineTROPICS (scaled)
3.60 GCUP: 66.5467.43
3.43 GTropNNC v1: 54.8364.71 ± 0.02
3.22–3.34 GCUP: 38.66 / TropNNC v2: 56.5761.38 ± 0.29
3.06 GCUP: 12.9651.04 ± 1.24
2.89 GTropNNC v2: 39.1141.81 ± 2.16

By scope, we mean that under aggressive data-free compression, the degradation would be severe, making fine-tuning necessary. For example, achieving under 30% accuracy is effectively broken, and increasing complexity to achieve a higher threshold is beyond the scope of this work.

W5 FPGM comparison under the same protocol

We acknowledge the inconsistency of applying our method under a slightly different protocol, and we have fixed this for the FPGM comparison, as their code is publicly available. Original accuracy: (93.33% on CIFAR-10)

Param. red.40%50%60%70%80%
FPGM (data-free, same network)72.7654.5738.9314.1514.68
TROPICS (scaled)91.6390.7389.6387.7985.10
Once more, we thank the reviewer. The analyses above have made the paper's claims more precise.
Reviewer_L6gNOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission presents TROPICS: Tropical Geometry for Structured Neural Network Compression through Zonotope Approximation, which is a one-pass structured compression method for ReLU networks that needs no data and no fine-tuning. This method consists of four steps: clustering, pruning, BatchNorm analysis and error estimation. Experimental evaluations cover MLP/CNN on MNIST and FashionMNIST, VGG16-BN on CIFAR-10/100, and ResNet18 on ImageNet. The paper compares against TropNNC, CUP, Model Folding and classic filter-pruning methods.

Critical strengths

This submission is easy to follow and also the chosen topic here is somehow practical: i.e. Compression with no data and no fine-tuning is useful, and the method is simple and cheap. Beside the main strength, it also shows strengths at:

  1. The scale factors clearly help. The scaled vs unscaled comparisons are consistent across settings. For example, on CIFAR-10 at about 89% reduction, clustering scores 77.85 vs 54.52 (Table 2), and the Mixed strategy scores 89.95 vs 32.49 at 1.67M parameters (Table 5).
  2. Strong results against data-free Model Folding on ResNet18 at matched parameters (Table 11), often matching its data-dependent REPAIR variant.
  3. Results are repeatable: the authors reported seeds and their results average five K-means seeds with standard deviations, with code released.
Critical weaknesses

However, the weakness is more striking compared to its strengths:

  1. Most of the headline CIFAR compression comes from removing redundant filters, which is just simple. VGG16-BN trained on CIFAR is known to have many dead or redundant filters in its late layers. As stated in Table 2, "Baseline merge" (folding 2,382 duplicate or near-identical filters) already removes 81.1% of parameters with 0.01% accuracy drop. Thus, the headline "84.7% reduction with a 0.44% drop" is only about 3–4 points beyond trivial redundancy removal. Table 1 is somehow misleading: Li et al. and Network Slimming report retrained models at a fixed prune ratio, without first removing redundant channels. Needed: a baseline that removes zero or duplicate filters and then applies plain magnitude pruning, with no geometry involved. Results on models with less redundancy, such as ResNets on CIFAR or a well-regularized VGG, would also help.

  2. Per-layer ratios were manually tuned, likely using test accuracy. That makes the "data-free" claim debatable and the comparisons with baselines that use fixed rules unequal. Please give the exact selection procedure or use a fixed rule such as uniform ratios or ratios derived from the bound.

  3. The theory is loose, and its predictive value is not established. The layer zonotope with λ ∈ [0, 1] is a single-output tropical construction. Scaling each generator by ‖w_out‖₂ to handle many outputs is a heuristic, not a derivation. The Hausdorff bound is per layer and does not compose through nonlinear layers, so it does not bound network output error. The correlations (Table 14) use n = 15 hand-picked configurations each. They should be compared with trivial predictors such as the fraction of parameters removed or the total norm of pruned weights. The bound also gets the scaling question backwards. The scaled bound is larger than the unscaled one (Table 15), yet the scaled models lose much less accuracy at high compression. Therefore, the bound cannot explain the paper's main empirical finding.

  4. There are a lot of inconsistencies in the paper, for example: The abstract says "0.44% drop at 84.7%." The text says 0.58%. Table 2 shows 0.33 at 84.5%. The VGG16-BN CIFAR-10 baseline is 92.09% in Table 2, 93.64% in Table 5, and 94.27 / 93.91% in Table 1. These appear to be different checkpoints across comparisons. Table 1 says the baseline checkpoints were "reproduced," yet compares against numbers from the original papers. The ImageNet comparison with TropNNC and CUP (Table 6) is evaluated on "different unseeded random 2,000-image subsets" with different baselines (69.40 vs 69.75). Please use the full 50k validation set.

  5. The scope of this method is so limited: Only ReLU CNNs and MLPs are tested.

Questions for authors

How does a "de-dup + magnitude prune" baseline perform at the same parameter counts on VGG16-BN, CIFAR-10/100? How were the per-layer p and r chosen manually? Were test accuracies used during this chosen? Why is the scaled bound larger than the unscaled one, when scaling clearly reduces the accuracy drop? How does the bound's correlation compare with that of simple predictors? What are the ImageNet comparisons on the full validation set with a single shared baseline?

2 comments

We sincerely thank the reviewer for the useful comments on our work.

W1 / Q1 — "De-dup + magnitude prune" baseline at the same parameter counts

We aggre that the VGG architecture on CIFAR10 has redundant parameters. However, findinng them it is not always a trivial procedure. The deudplication phase is the first phase of our algorithm with generators and scaling. In figure 5 we point out that TROPICS achieves much higher reduction at the same compression ratios compared to filters only clustering.

We implemented exactly the requested baseline, deduplication + magnitude.

Cfg (param. red.)De-dup + magnitudeTROPICS (scaled)
15 (84.7%)90.8391.61
19 (87.6%)85.7689.93
29 (86.1%)25.2383.75
28 (91.7%)48.7584.94
26 (92.1%)25.5984.23

(Original accuracy: 92.09% on CIFAR-10.)

A model with little redundancy. FPGM We used a second VGG16 checkpoint (93.33% on CIFAR-10) in which de-duplication finds only 3 duplicate filters. Ratios were set by the automatic rule described under W2:

Param. red.40%50%60%70%80%
FPGM (data-free, same network)72.7654.5738.9314.1514.68
De-dup + magnitude89.3787.1979.9663.2034.15
TROPICS (scaled)91.6390.7389.6387.7985.10

A clustering-space ablation isolates the geometry. Everything is fixed except the space in which filters are clustered:

DatasetCfg (param. red.)Raw filter weightsGenerators, no standardisationGenerators (TROPICS)
CIFAR-10(84.7%)90.8591.3191.61
CIFAR-10(87.6%)83.9188.6789.93
CIFAR-10(88.2%)84.7987.4989.28
CIFAR-10(86.1%)36.5682.9383.75
CIFAR-10(91.7%)37.9353.7384.94
CIFAR-10(92.1%)35.8247.6684.23
CIFAR-100(67.0%)52.1957.5362.80
CIFAR-100(72.9%)24.2728.0953.71
CIFAR-100(75.0%)28.5212.1554.18
CIFAR-100(80.6%)12.4427.0241.81
CIFAR-100(80.1%)10.8711.3940.34

(Top-1 %, VGG16-BN, scale correction on, identical per-layer budgets. Originals: 92.09% on CIFAR-10 and 73.21% on CIFAR-100.)

W2 / Q2 — How per-layer ratios are chosen

The original ratios were set by hand to match each baseline's parameter or FLOP budget, not selected by test accuracy. We accept that a manual search over configurations is not a fixed rule, and we have replaced it with one:

  • Given a single global FLOP or parameter target τ, each compressible layer keeps a fraction r_l = clip(1 − α·w_l, 0.05, 1) of its filters. Here w is a fixed, hand-set intensity vector per stage (front-loaded for FLOPs, back-loaded for parameters). Brent's method solves for the unique α at which the exact linear cost model reaches (1 − τ) of the original cost. The 75/25 split fo cluestering/pruning is an empirical choice, fixed across all experiments.

W3 / Q3 / Q4 — Theory, the bound, and simple predictors

  • What the per-layer bound guarantees. The per-layer zonotope is the Newton polytope of a single-output tropical polynomial. The Hausdorff distance between two such polytopes equals the largest difference of their support functions over unit-norm inputs, so it bounds the change in that function per unit input norm.

  • What is heuristic. Scaling each generator by ‖w_out‖ to handle many outputs is a heuristic, not a derivation, and we will label it so. The per-layer bounds do not compose through nonlinear layers into a bound on network output error, and we will say so explicitly. Bound as accuracy predictors is a promising direction. It seems to have high corellations in some experiments. However, in this work, we present it as a per-layer worst-case certificate on how much each layer's zonotope changes, and we soften the estimator claims.

  • ImageNet (Q5). full 50k validation set**

FLOPsBaselineTROPICS (scaled)
3.60 GCUP: 66.5467.43
3.43 GTropNNC: 54.8364.71 ± 0.02
3.22–3.34 GCUP: 38.66 / TropNNC: 56.5761.38 ± 0.29
3.06 GCUP: 12.9651.04 ± 1.24
2.89 GTropNNC: 39.1141.81 ± 2.16

W5 — Scope beyond ReLU CNNs and MLPs

We added ViT-B/16 on ImageNet, which uses GELU, LayerNorm and attention. The method is exact only for positively homogeneous activations, so under GELU it is applied as an approximation; we state this in the paper.

Param. red.7.5%14.9%29.8%44.8%
TROPICS (cluster generators)78.9577.9073.0062.95
Same pipeline, cluster raw filters78.8574.8021.458.75

(Original: 80.25%.)

W4 / Q5 — Inconsistencies

Will be fixed in the revision. We thank the reviewer again. These changes help our work to bemore precise.

Thank you for providing the baseline comparisons, I will raise my score.

Comments on the paper

1 comment

Dear Reviewers, please take a moment to read the author responses and engage in the discussion. In particular, please comment on whether your questions and concerns have been adequately addressed and follow up on any points that remain unclear. Thank you! AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.