Symmetry Slippage in Equivariant ML: Creation, Computation, and Corrections

Symmetry-based inductive biases are prevalent throughout machine learning for science and beyond. Often, the symmetries realized by an architecture differ from those prescribed. This mismatch can occur in two ways: when prescribed symmetries are not realized (*under-equivariance*) and when *non*-prescribed symmetries *are* realized; we call this *symmetry slippage*. Although famous examples of symmetry slippage abound, a systematic study of its origin, estimation, and corrections has yet to exist. This work aims to initiate such a study. We formulate symmetry slippage in terms of subarchitectural decompositions, and use this framing to identify three common mechanisms by which contemporary techniques can unintentionally create it. These creation mechanisms recover the familiar examples, theoretically ground popular heuristics, and identify symmetry slippage across nine active research areas. While we find that guaranteeing an absence of symmetry slippage is hard in general, we prove conditions under which our decomposition framing is exact. We use this result to exactly compute, for the first time, the realized symmetries of a classical family subsuming e.g. convolutional architectures. Finally, we study *grippage encodings* (GEs), simple symmetry-correcting features that restore expressivity. We find that GEs improve empirical performance across our nine case studies.