Match Every Patch Once: A Logical-Structural Tradeoff in Anomaly Scoring

Normal-only image anomaly detection often scores an image by matching each test patch independently to its closest patch in a normal reference. This allows many test patches to match the same reference patch, making changes in the number or composition of otherwise normal parts difficult to detect. We instead require a one-to-one patch assignment between a test image and a complete normal reference, keeping the features, patch costs and aggregation fixed. We define the assignment excess as the additional cost introduced by one-to-one matching and show that it exactly accounts for the resulting changes in anomaly ranking. The constraint improves sensitivity to logical anomalies but can reduce sensitivity to structural defects, a tradeoff that depends strongly on how normal references are constructed. Used as an auxiliary signal in a trained EfficientAD-M detector, a calibrated form of the assignment excess improves logical anomaly detection in all five training runs. Patch correspondence and reference construction must therefore be specified together in anomaly scoring.