One-Hot Is Already a Geometry: What a Label Space Buys, in Bits

One-hot targets are a choice of geometry: $n$ equidistant points, the regular simplex of neural collapse. Nothing forces that equidistance, and since labels a classifier already confuses could sit close together, placing classes at informative distances is common. But no unit says what that buys, or whether the gain comes from where the points lie or from which label sits where. We define one, in bits. With a frozen encoder and a ridge map onto the label points, scored by distance, the held-out information gain of a label geometry splits exactly into a reference level, an arrangement gain and an assignment gain, each against its own null. Three things follow. Averaging any geometry over all its labellings returns the regular simplex, so one-hot is the non-informative prior over assignment: at fixed temperature it arranges as well as any geometry of the same scale, and assigns nothing at all. An informative geometry trades arrangement for the power to assign. Near the simplex the arrangement gain depends on just two numbers read off the label Gram matrix, its distance from the simplex, and how much of that distance lies on the radii. A second-order expansion weighs them, and although it misses the level, only the ratio of its two weights fixes the order. Both weights come in closed form from the ridge, so screening a candidate geometry is a matrix product, not a training run. Applied to affective and topic classification, those two numbers order candidate geometries where the log-determinant, blind to the radii, fails.