acceptodds
Under review as a conference paper at ICLR 2027

Hidden Label Distinctions Change the Gain from Representation Rate

Abstract

How much accuracy does a fixed increase in representation rate buy under ambiguous supervision? We study whether this marginal gain changes when candidate-label channels have equal ambiguity and spectrum but hide different class distinctions. Using frozen ImageNet features, we compare a fixed increase from 32 to 64 nominal bits per feature vector. Separate DESIGN measurements select the class pairs with the largest and smallest discrimination gains, fixing two matched hiding interventions and their expected gain ordering before downstream fitting and TEST. Across 68 parent groups and 340 classes, hiding the largest-gain pairs yields 0.702 percentage points less mean fine-class gain (95% interval [0.088, 1.345]); the fine-minus-coarse contrast is 0.752 points. The interaction persists with each high-rate-trained classifier held fixed; fixed-low persistence remains unresolved. The higher final-accuracy arm has the smaller marginal gain. Difficulty and headroom remain possible explanations; the contrast does not isolate a unique mechanism. Concept-preserving EEG correspondence controls provide supporting discovery. In the tested system, matched channel ambiguity and spectrum do not equate the marginal value of the same representation increment.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.