Routing the Lottery: Adaptive Subnetworks for Heterogeneous Data

The Lottery Ticket Hypothesis (LTH) posits that large neural networks contain sparse subnetworks, or winning tickets, that can be trained in isolation to match the performance of their dense counterparts. The mask defining such a ticket is extracted from data but static at inference: a single subnetwork serves the entire data distribution, and input-conditional computation arises through activations rather than parameters. We ask whether the structure of the data distribution is also reflected at the parameter level: what family of tickets emerges when ticket extraction is conditioned on partitions of the data, and how is that family organized? To study this, we propose Routing the Lottery (RTL), an adaptive pruning framework that reveals such data-conditional subnetworks, which we term adaptive tickets. Rather than focusing on compression, we use pruning as a probe to study how computation is organized within a network under controlled data partitions. We analyze these subnetworks through mask overlap and structural decomposition calibrated against chance-level null models, uncovering systematic variation in parameter sharing across data-dependent pathways. Our results show that classical universal winning tickets can be interpreted as structured superpositions of adaptive pathways within a bounded sparsity regime. At extreme sparsity levels, this relationship largely breaks down as universal pruning converges to structurally degenerate solutions. Overall, RTL reframes pruning as an analytical tool for studying data-dependent neural pathways and provides a new lens for understanding how modular computational structure emerges in sparse deep networks.