Rank Frontiers for Shared Linear Encoders: Paired Task Filters
Abstract
For paired task filters, we derive the exact information-loss frontier of a rank-limited shared linear encoder. At equal overlap, the global optimizer changes exactly from pairing rank units within output coordinates to spreading them across coordinates at . This allocation law is fragile to unequal overlap: an explicit three-output projector beats every block-aligned allocation, while the complete two-output unequal-overlap frontier remains solvable at all five ranks. A held-out learned-subspace study recovers the predicted transition within nats. These are results for a paired-filter linear–Gaussian model, not for nonlinear encoders or practical architecture selection. Common whitening covers one shared nonspherical covariance; distinct covariances receive only the stated residual results. Capacity-comparison signs are accounting-dependent, and all public-data selectors abstain, so neither supplies an architecture recommendation. Finite temporal histories are one covariance-structured application of the static Gaussian-vector result; we make no new claim about temporal dynamics.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.