Bandstop: Emergent Compression via Residual Flow Throttling

Residual connections turn network depth into a sequence of refinements to a shared representation. However, treating residual branches uniformly provides no structured way to exploit redundancy among these transformations. We introduce Bandstop, a gated residual architecture that imposes an inductive bias toward non-uniform residual branch scaling across depth. Without an auxiliary compression objective or prescribed pruning budget, gradient-based training learns two scalar cutoffs that delimit a contiguous intermediate region in which residual contributions are attenuated. Across architectures, this structure yields emergent compression: the attenuated region can be removed with limited loss in task performance. This learned depth structure improves robustness to post-hoc pruning by up to 8.6 percentage points in mean accuracy over the matched Residual network, with the advantage persisting even after layer replacement. In vision, Bandstop learns lower-rank representations while improving classification accuracy and robustness to input corruption. In language models, Bandstop counters the curse of depth reported for standard residual networks and shows the reasoning-primitive gains associated with depth-grown models. Our results demonstrate that non-uniform layer-wise throttling of residual flow is a simple mechanism for inducing compression during standard training, and establish layer-wise information flow as an important architectural degree of freedom for building efficient and adaptable deep networks.