When Equipartition Corrupts Relational Learning: Capacity–Geometry Mismatch under Long Tails
Abstract
Balanced optimal-transport assignments are widely used in representation learning to prevent collapse by enforcing equal mass across latent components. Under long-tailed data, however, this constraint can conflict with the geometry supported by the evolving representation. We identify this capacity–geometry mismatch as a source of systematic distortion in relational learning: mismatched component capacities displace samples across representation-supported basins, perturbing transport assignments and, in turn, contaminating the relational teacher and distillation signal. We theoretically characterize this mechanism and establish a finite-batch connection between full-bank transport relations and the minibatch teacher used during optimization. Guided by this analysis, we introduce PACI, which estimates representation-supported pseudo-basin capacities and applies them consistently to both transport assignment and relation construction. PACI consistently outperforms a broad range of state-of-the-art deep clustering methods across long-tailed benchmarks. Controlled capacity perturbations trace the predicted progression from capacity mismatch to assignment displacement and teacher contamination, while intervention experiments isolate assignment displacement as the dominant pathway to degraded representation quality, distinct from relation-normalization effects. These findings generalize across training regimes and long-tailed settings, supporting geometry-matched capacity as a principled design rule for long-tailed relational representation learning.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.