Beyond Covariance: How Shallow Autoencoders Learn Normal Relations for Multivariate Time Series Anomaly Detection

A simple shallow autoencoder (ShallowAE) is highly competitive on the TSB-AD-M benchmark for multivariate time series anomaly detection, yet what it learns from normal data and how this information shapes anomaly scores remain unclear. We characterize the statistical sufficiency boundary of squared-error reconstruction objectives, proving that the mean and covariance determine the entire risk function for affine reconstruction but are not universally sufficient for nonlinear reconstruction objectives. Moment-matched interventions on real time series show that training information beyond second-order statistics can change ShallowAE's learning and detection behavior. Controls within a fixed PCA subspace further show that the network can assign different scores to different combinations of variables even when all inputs lie in the same subspace and have zero PCA residual. An analytically tractable relation-switching experiment reveals that, between normal environments with identical coordinate-wise marginals and covariance, reversing the conditional relationship changes both the network's low-residual regions and its scoring preferences on fixed inputs. These findings identify the learning of normal relationships beyond covariance as a mechanism through which ShallowAE can reshape reconstruction geometry and score inputs differently despite identical PCA reconstruction scores.