A Geometric Diagnostic for Frozen Foundation Features

Classification on domain-specific data often pairs frozen foundation-model features with a lightweight classifier, yet uneven per-class performance is usually attributed to class imbalance rather than representation geometry. On PVEL-AD, a balanced linear probe on frozen DINOv2 features recalls 94% of one defect class from 56 labeled examples but only 26% of another from 96, showing that class frequency alone does not explain the failure pattern. We use Neural Collapse as a geometric reference and show that anisotropy imposes a strictly positive lower bound on deviation from the simplex equiangular tight frame (ETF), with a median 99% of the measured deviation across our benchmark accounted for by this floor. We introduce the Geometric Collapse Score (GCS), the mean pairwise cosine of L2-normalized class centroids, as a measure of how tightly an encoder compresses a domain. Across eleven datasets and four encoders, dataset ordering by GCS is nearly invariant, with pairwise Spearman correlations from 0.96 to 1.00. Across 55 classes from seven specialized datasets, per-class GCS strongly correlates with recall ($\rho =-0.785$), while class frequency does not ($\rho =-0.162$). Frozen GCS also separates two empirical regimes of adaptation response. LoRA improves balanced accuracy on eight of nine specialized datasets and reduces it on both natural-image controls, although the boundary is an ordering rather than a universal threshold. An ETF-alignment objective improves PVEL-AD balanced accuracy by 3.3 points over an identical-pipeline control while reducing cone alignment, with class-level gains remaining within seed noise and no meaningful transfer to ResNet-50. Together, these results identify anisotropic feature geometry as a measurable failure mode of frozen representations and position GCS as a practical pre-adaptation diagnostic for when the representation itself warrants intervention.