Toward Spectral Kinematics:Transformation Laws and Their Limits
Abstract
Pretrained encoders map samples to feature vectors that can be stored and reused across tasks. Pooling, compression, and reweighting change their spectral diversity, the effective number of directions carrying feature energy. We study what information determines pooled diversity and what partial observations can guarantee. Building on classical transformation identities, we construct sources with identical spectra, energies, and principal angles but different pooled effective ranks. These summaries miss which energetic directions overlap. For second-order effective rank, which emphasizes high-energy directions, source spectra and energies plus one energy-weighted overlap total determine the answer exactly. Bounding unmeasured contributions to this total yields computable relative-error guarantees. Compatible completions, which preserve all observations while varying unmeasured overlaps, distinguish conservative bounds from information that permits opposite diversity-gain decisions. In a four-encoder vision cohort, raw features require a median of 1% of direction-pair measurements to guarantee at most 1% relative error; centering raises the median to full coupling on the tested query schedule. Fewer measurements do not always reduce runtime: gains depend on cached source factors and bound-tightening costs. These results connect the information needed for pooled spectral diversity to reliable prediction and spectral-diversity batch admission, while delimiting their computational benefits.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.