The Cost of Equivariance in Noisy Linear Representations

Equivariance is a natural inductive bias for symmetric prediction tasks. Its benefit depends on where the symmetry is imposed. We ask what it costs to impose symmetry on a shared low-dimensional representation when receivers also have their own local features. In a paired Gaussian model, each receiver predicts its partner's coordinate from its own possibly noisy coordinate plus one common energy-constrained noisy linear broadcast; we minimize average risk over all fixed linear encoders and all measurable decoders. For up to half as many channels as input coordinates, we derive the exact minimum and characterize every optimal encoder Gram: equal-energy codes on selected coordinate pairs. On specified cycles and Boolean cubes, we also derive the exact optimum for translation-equivariant linear encoders with orthogonal latent actions. Their risk is strictly higher below the half-dimension threshold whenever local information is informative, so no decoder can recover the shortfall. Averaging optimal Grams over translations can exceed the rank budget; shared randomization instead restores symmetry in distribution. These exact floors separate information loss at the representation from approximation error at the decoder. In a prospectively specified 25-replicate CPU extension following five original replicates, frozen TabPFN-3 and TabICLv2 approach both floors; a separate ten-replicate cube study gives a second implementation check. An invertible post-noise transform preserves the floors while increasing ridge error, and both pretrained decoders retain low excess risk. The result is an exact cost of fixed encoder equivariance under matched budgets within this linear-Gaussian paired family.