Pruning Then Perforating: Dendrites Recover What Structured Pruning Removes

Deploying a vision model under a parameter budget pits two techniques against each other: structured pruning cuts parameters by removing capacity, while Perforated Backpropagation adds capacity back through artificial dendrite nodes grown during training. Each is a good mechanism to reach for when searching for a method to optimize accuracy and parameter count, but whether they compose has not been measured. We measure it on half-width ResNet-18 over Oxford-IIIT Pets performing structured pruning based on L2 magnitude and Taylor importance, as well as on a DeepLabV3 segmentation model deployed on an industrial robotics platform performing structured pruning on L2 magnitude only. Against matched zero-dendrite baselines at equal parameter count, perforated models come out ahead at every pruning rate. Composing these methods therefore is optimal compared to either method independently.