When Feature Rank Misses Model Collapse: Class Geometry in Recursive Classifiers
Abstract
Global feature rank can rise even when within- and between-class covariance components both become spectrally more concentrated. We study this behavior in image classifiers recursively retrained on generated images and inherited pseudo-labels. In a three-seed CIFAR-10 study with the original ResNet stem, four replacements increase centered-covariance effective rank by 14.5% and within-to-between-class scatter 3.22-fold, while accuracy falls from 88.4% to 62.2%. Exact entropy accounting shows how variance reweighting conceals concentration of both components. Later generations lose another 9.28 accuracy points while rank rises 3.30%; fixing the initial teacher avoids this additional loss on identical image pools. A standard CIFAR stem raises baseline accuracy to 95.4%, yet retains the full-trajectory mismatch: accuracy reaches 56.2% while rank increases 2.6%. Its later rank changes vary by seed. Alternative spectral definitions also give different directions, and GAN and ImageNet-100 diffusion workflows lose rank as geometry deteriorates. Disjoint subspace and frozen-feature readout analyses corroborate degraded class structure. Low-label audits estimate geometry but do not establish better warning or intervention decisions. These findings identify what increasing covariance rank can conceal, while showing why its interpretation depends on class-conditioned spectra, training stage and predictive performance.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.