Which Tasks Identify a Representation? Specificity Bounds and Task Acquisition
Abstract
Task structure can rule out latent mixing without making a representation easy to estimate. We study this distinction for finitely many outcomes with known overlapping supports and injective conditional signatures. Building on existing support-separation and partial-view principles, we characterize allowed mixing and derive a fractional-cover bound on block specificity. The bound combines task conditioning with signature error, avoids repeatedly charging the same excluded directions, and is optimal given its local variance constraints. A smooth signature example attains its constant and improves the coordinatewise bound by a factor 2(B-1) for (B) blocks. We then evaluate acquisition at a fixed sampled-label budget. In a Gaussian-signature comparison, a pilot-conditioned policy improves shifted accuracy over uniform candidate acquisition, and an information-matched inner-Lasso adaptation nearly matches the proposed spectral estimator. An invertible nonlinear model gives a more cautious result: a frozen 20-seed follow-up estimates a 2.52-point shifted-accuracy gain, with a 95% interval spanning zero. A separate 240-run composite-image benchmark exposes further failures outside the theorem. Together, the results distinguish structural separation, conditioning, and finite-budget utility, with released implementations, complete negative results, and explicit limits on practical certification.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.