Intrinsic Dimension Reads Within-Class Structure Below the Class Size and Class-Mean Geometry Above It

Intrinsic dimension is a popular one-number summary of a neural network’s representation, used to argue that networks compress their representations during training and that lower dimension means better generalisation. We show that on labelled data this number measures one of two different things, and a rarely reported choice decides which: how many neighbours the estimator looks at, compared with how many images each class contributes. With few neighbours the number describes the shape of each class; with many, how the classes are laid out. The two geometries relate to accuracy in opposite ways. On 178 ImageNet classifiers, changing only the neighbour count turns the well-known correlation between dimension and error from +0.80 to −0.66 on the same networks and images, and in training runs the same checkpoints read as compression at one count and expansion at another. We prove two short theorems: one derives the switch from how a query’s neighbours fill with classmates, the other shows that class means, class covariances and even the network’s own logits do not determine what the estimator reports. The switch holds across architecture families and network scale, on four out-of-distribution test sets, in self-supervised encoders and in 26 language models. We close with a two-count check that any study can run: read at half and at four times the class size, and say which geometry the conclusion rests on.