Back to Memory: Local Normal Support and Reference Routing for 3D Anomaly Detection

Memory-bank methods for 3D anomaly detection must select appropriate normal references and distinguish defects from allowable local variation. Feature-only retrieval can match defects to similar geometry in incompatible regions, while nearest-neighbor distances do not explicitly describe the variation permitted by each normal pattern. We propose BTM, a training-free framework that separates normal memory construction, reference routing, and anomaly scoring. Manifold-Aware Memory builds local normal patterns from graph-geodesic neighborhoods of fixed FPFH features, storing local subspaces, finite tangential supports, and orthogonal thicknesses. Spatial-Context Gated Routing restricts candidates by registered position and selectively reranks low-similarity queries using multiscale context. Corrected Tangent-Normal Scoring measures only deviations beyond the selected patterns' support bounds. BTM requires only normal training point clouds, without learned encoders or synthetic anomalies. It achieves P-AUROC of 95.19 ± 0.08% on Real3D-AD and 94.03 ± 0.06% on the original 40-category Anomaly-ShapeNet benchmark, leading the reported point-level comparisons on both datasets. Object-level results rank first and second, respectively, among the compared methods. Module comparisons and mechanism analyses examine the contributions and limitations of reference routing and local-support correction.