Minimax Price of Approximate Equivariance in Linear Regression

Equivariance is an important source of data efficiency in physical prediction and geometric perception. However, practical symmetries are often only approximate. Exact enforcement can remove predictive departures, while an unconstrained fit sacrifices parameter sharing and increases variance with limited labels. The challenge is to determine how strongly to enforce a known symmetry and what prediction error is unavoidable as the departure grows. In this paper, we formulate this choice as a minimax prediction problem in multivariate linear regression and propose projected shrinkage between exact equivariance and an unconstrained fit. We derive an exact finite-sample risk identity for this family and prove matching upper and lower bounds that characterize the global minimax prediction rate up to universal constants. The result shows that retention is governed by the balance between the possible departure and the variance required to estimate it. We extend this principle to anisotropic covariance, restricted high-dimensional designs, and sampled group averaging. Simulations recover the predicted retention transition, demonstrate the value of residual information in underdetermined designs, and connect projector accuracy to downstream prediction. These results provide a finite-sample answer to how much structure to share through an approximate symmetry when data are limited.