Declared Geometry: A Geometric Prior Bottleneck
Abstract
Representations for prediction should discard target-irrelevant input variation while preserving target-relevant information in a predictive geometry. Information-Bottleneck (IB) methods make this compression explicit but do not condition it on the target, a gap the Conditional Entropy Bottleneck (CEB) closes with a label-conditional KL criterion. Yet per-label translations leave the conditional KL unchanged, so collapsed and arbitrarily separated configurations coexist, and under a strict strong data-processing condition strong compression makes an input-independent encoder globally optimal for idealized CEB. We introduce the Geometric Prior Bottleneck (GPB), which declares the geometry in the label-conditional reference and ties the prediction head, making center collapse strictly KL-expensive and, whenever a feasible comparator has lower KL, excluding center-collapsed optima above an explicit compression threshold: OPB classifies from an equal-radius orthogonal frame through anchor energies; EPB regresses through a projection tied to an isometric label axis. We then analyze what the modified objective learns: strong regularization targets the conditional-expectation anchor rather than the input-independent collapse, and under the stated conditions the tied mean readouts remain Bayes-optimal at every compression strength. Across twelve dataset–backbone settings with IB and non-IB baselines, GPB is best or tied-best in eight and never significantly worse than CEB.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.