Rethinking Structural Anomaly Detection: From Decision Boundaries to Projection Operators

Many anomaly detection methods either rely on estimating a probability density or learning an enclosing decision boundary, implicitly assuming that normal data occupies a region of non-zero volume in the ambient space. In contrast, structural anomaly detection deals with perceptual data such as images, where normal samples concentrate near a low-dimensional manifold and anomalies correspond to deviations outside its support. This view reveals a mismatch between the volumetric bias of existing methods and the geometric structure of the data, often resulting in degraded performance in practice. To address this mismatch, we formalize the task as learning a projection operator onto the manifold of normal samples and define a sample as anomalous if it is altered by this projection. This formulation naturally integrates the inductive bias of manifold-supported data and reframes anomaly detection in terms of a projection residual, thereby resolving issues arising from modeling degenerate distributions. Notably, it provides a unifying interpretation of reconstruction-based methods by explaining their success and failure in terms of projection quality. In particular, it explains the strong generalization ability of projection-aligned models to unseen anomalies as a consequence of their contraction behavior that suppresses off-manifold variation. Empirically, we demonstrate that such models substantially outperform a range of boundary-based methods and achieve state-of-the-art performance while producing segmentation masks with fine, accurate boundaries.