acceptodds
Under review as a conference paper at ICLR 2027

Pruning Then Perforating: Dendrites Recover What Structured Pruning Removes

Abstract

Deploying a vision model under a parameter budget pits two techniques against each other: structured pruning cuts parameters by removing capacity, while Perforated Backpropagation adds capacity back through artificial dendrite nodes grown during training. Each is a good mechanism to reach for when searching for a method to optimize accuracy and parameter count, but whether they compose has not been measured. We measure it on half-width ResNet-18 over Oxford-IIIT Pets performing structured pruning based on L2 magnitude and Taylor importance, as well as on a DeepLabV3 segmentation model deployed on an industrial robotics platform performing structured pruning on L2 magnitude only. Against matched zero-dendrite baselines at equal parameter count, perforated models come out ahead at every pruning rate. Composing these methods therefore is optimal compared to either method independently.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Structured pruning, Neural network pruning, Perforated Backpropagation, Artificial Dendrites, model compression, parameter efficiency, computer vision

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_9yS7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper provides a computational evaluation of alternating between network pruning (for reducing parameters) and perforated backpropagation (for growing parameters) in order to obtain a more parameter-efficient neural network in terms of accuracy. The study evaluates the results of three experimental settings: two involving a same neural network from a 2012 paper but with different pruning criteria, and another one involving an industrial application. The study concludes that perforation is more beneficial under heavy pruning.

Critical strengths

Although not new, the idea of growing and reducing parameters is an interesting one. According to the authors, this has not been studied for perforated backpropagation.

Critical weaknesses

For a paper like this, I would expect one of the two possibilities below:

  1. An extensive study involving several neural networks based on one or more configurable architectures with varying sizes and trained on multiple datasets, where a statistically significant pattern is observed and may inform future developments related to network pruning and perforated backpropagation. However, the study is limited to two neural networks and does not provide actionable results. For example, consider the breadth of the experiments carried out in these two network pruning studies:
  1. Alternatively, a study like this could originate a method based on combining network pruning and perforated backpropagation that produces competitive results in terms of parameter-efficiency. Such a study would require evaluation on several neural networks again, but perhaps not as many as in the case above. However, there would be an expectation of using more recent architectures in this case.

This paper may potentially evolve into either, but as it stands it is not sufficiently developed to warrant publication.

Questions for authors

I do not have questions, but I welcome the authors to discuss the points above.

Reviewer_FteQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates whether combining structured pruning with Perforated Backpropagation can improve model accuracy under a fixed parameter budget. It presents three studies: a controlled benchmark with half-width ResNet-18 on Oxford-IIIT Pets, repeating the benchmark with Taylor importance in place of L2 magnitude, and a production study on a segmentation model with an Atrous Spatial Pyramid Pooling head and a ResNet-18 backbone. Every headline comparison of the experiments is made against a predetermined zero-dendrite baseline. The perforated models achieve an average accuracy improvement of 1.15 percentage points across 54 runs over the baseline.

Critical strengths

None.

Critical weaknesses
  1. The paper does not clearly explain its main contribution or provide a sufficiently clear presentation of the proposed approach, making it difficult to understand the methodology and assess its novelty. The overall investigation also appears limited in depth, leaving the significance of the findings insufficiently established.

  2. The experimental validation is also very limited. The paper reported experiments only on one dataset and only on the ResNet-18 backbone with an additional segmentation model. With this limited set of experiments, it is difficult to assess the method. More extensive experiments and comparisons are needed to establish the robustness and broader explainability and applicability of the proposed approach.

Questions for authors
  1. Can the author(s) clearly explain the paper's core contribution, main focus, and novelty?
  2. Can the author(s) provide more methodological details with a clear presentation to make the proposed approach easier to understand and reproduce?
  3. Why was the evaluation limited to a single classification dataset with ResNet-18 and an additional DeepLabV3 segmentation model? Can the author(s) validate the approach on more datasets and architectures to establish the robustness and practical benefits of the proposed approach?
Reviewer_BB9zOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper tries to use two strategies together to see if it can help with model compression with less performance drop.

Critical strengths
  1. Model compression is an important topic.
  2. How to balance the performance drop by pruning and performance gain by adding more parameters is critical for practical model training and deployment.
Critical weaknesses
  1. The draft is hard to read. It is hard to follow the abstract. The whole draft is not ready with table and figure not informative. Overall, the writing is like a technical report instead of research paper.
  2. What is the motivation to do such an empirical analysis, why does it matter?
  3. What is the final conclusion is not clear. After the empirical analysis and exploration, what kind of practical conclusion of this draft? how it can be used for following research?
Questions for authors

Please check the section above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.