SEIS: Subspace-based Equivariance and Invariance Scores for Neural Representations

Understanding how neural representations respond to geometric transformations is important for diagnosing robustness, comparing architectures, and selecting models for spatial tasks. Existing measures typically examine output predictions, aggregate individual units independently, or fit an explicit transformation between representations. We introduce SEIS (Subspace-based Equivariance and Invariance Scores), a label-free subspace measure that disentangles equivariance and invariance directly from paired activations of original and transformed inputs, without knowledge of the transformation. SEIS organizes activations into spatial positions and observations, applies a continuous spectral filter that softly downweights weak components, then factorizes the comparison into an observation-space alignment for equivariance and a stricter spatial alignment for invariance, with both scores chance-corrected against null expectations. Controlled experiments show that SEIS cleanly separates the two properties and is stable given sufficient observations. Applied to trained networks, SEIS reveals how representations evolve during training, how augmentation strengthens deeper layers, how decoder architecture and multi-task supervision shape transformation geometry, and how positional encodings affect translation structure in vision transformers. Across a diverse pool of ImageNet-pretrained classifiers, SEIS predicts accuracy retention under affine transformation better than validation accuracy, without labels.