acceptodds
Under review as a conference paper at ICLR 2027

Similarity Is Not Normality: Reasoning over Retrieved Evidence for Anomaly Detection

Abstract

Retrieval-based anomaly detection retains normal exemplars at inference, yet most existing methods fuse retrieval discrepancies without preserving which descriptor produced each of them. Similarity alone does not explain normality. A cracked but visually plausible component may retrieve close appearance neighbors while violating part layout, whereas a normal object observed from an unseen viewpoint may exhibit the opposite pattern. The key question is not only how large a discrepancy is, but also which property causes it and whether the remaining properties agree. Once an aggregation removes that distinction, a predictor observing only the aggregation cannot recover it. We refer to this limitation as the retrieval-to-reasoning gap and show that it is measurable, since permutation-invariant summaries of retrieved evidence are markedly less separable than the evidence itself on real data. R2AD retrieves from four factorized partitions of a frozen prototype bank corresponding to appearance, geometry, structure, and semantics, selecting the partition of the category predicted from the class token without using ground-truth category labels. It calibrates each discrepancy against an empirical distribution of normal costs, placing all factors on a common rank scale while preserving their identities. A conflict-aware mixture of experts then reasons over the resulting evidence spectrum, retaining isolated violations while attenuating the coherent elevations that benign variation tends to produce. R2AD reaches 75.1 pixel AP and 96.6 AUPRO on MVTec-AD and 46.5 pixel AP and 94.1 AUPRO on VisA, and ablation studies demonstrate the contribution of the evidence operators, the category conditioning, and the rank calibration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Anomaly Detection, Representation Learning, Evidence Spectrum Learning, Retrieval-to-Reasoning, Unsupervised Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RJtHOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Paper deals with fusing retrieval scores for anomaly detection when multiples factors. It first normalizes them by rank. It then executes a rather complex procedure to arrive at decision rules. The results on standard AD benchmarks exceeds previous methods.

Critical strengths

Anomaly detection is a challenging problem whose precise objective is still being worked out. Papers challenging the standard practices are highly encourage. The empirical results of the approach are quite good.

Critical weaknesses

While the writing is grammatical, it is dense and hard to parse. It is unclear to me why the proof is particularly informative or useful. I also think the claim that different factor ought to be treated differently is not actually controversial even if some approaches neglected it. Rank normalization is of features in AD is by no means new e.g., HBOS/LODA. The proposed features seem somewhat adhoc. At the end, the objective here is not to propose new features and it would have been better to use standard features from the literature. The approach relies heavily on pseudo supervision (synthetic anomalies). While this is fair game for unsupervised defect detection, it does not address the more interesting challenges of AD (not having positive data). It becomes a supervised learning task (although with low quality data). As the goal is not actually defect detection but AD, this limits the contribution. I am personally not so excited by empirical gains on the current, highly saturated defect detection benchmarks. Actually, it does not appear to me this method is too tied up to images. Indeed, it might be more natural for tabular AD problems. There are many datasets, and it would be interesting to see if the MoE approach would improve results there. An added bonus is that it is not easy to overfit those benchmarks as there are many of them.

Questions for authors

See cons

2 comments

We sincerely thank the reviewer for the candid and thoughtful assessment. Several of your points are well taken, and they have changed how we position the work. Our responses follow the order of your comments.

W1. Writing. We are sorry that the paper was hard to follow. We will open the paper with a three-step description of the method: (i) four descriptors each retrieve a distance to normal references; (ii) each distance is converted into a tail rank among normal distances, which makes the values comparable across descriptors while keeping each one labelled by its source; (iii) a small network scores these labelled values together with image context, so that a value can be interpreted in light of its source and of the other values.

W2. Propositions and novelty. We agree that the principle itself is not controversial, and we do not present it as our contribution. The propositions are elementary and serve a single purpose. They separate two restrictions, losing source identity and imposing a monotone rule, and each one corresponds to a matched control in Table 4. Removing identity costs 3.0 / 4.7 P-AP on MVTec-AD / Real-IAD, and imposing a monotone rule costs 1.4 / 2.5. What we contribute is a measurement of these costs under matched conditions, together with evidence that current retrieval pipelines incur them.

W3. Rank normalization (HBOS, LODA). Thank you for pointing us to this line of work; we agree. Per-dimension rank or density normalization is well established (HBOS, LODA, and also ECOD and COPOD). We will cite these methods and revise contribution (ii) so that calibration is presented as an adopted tool. Its role in our method is to make costs comparable across sources while keeping their labels. The connection is also informative for our question. These methods go on to sum the per-dimension scores, which is a permutation-invariant and monotone rule, and this is the class of rules that our two controls examine. On the same calibrated evidence, removing the labels costs 3.0 / 4.7 P-AP and a monotone rule costs 1.4 / 2.5.

W4. Ad hoc features. We agree that the operators are instances and should not be read as the contribution. Table 4 already includes standard features, namely plain DINOv2 tokens from four depths. Reasoning still improves them from 66.1 / 32.2 to 68.7 / 34.6 P-AP, so the benefit does not require our operators, although it is larger with them (+5.3 / +4.9). In addition, a new backbone study (please see our response to Reviewer oTJf, W2) shows that the gain holds across encoders, at +3.9 to +5.3 P-AP on DINOv2-S, DINOv2-B and DINOv3.

W5. Pseudo-supervision. We follow the standard industrial setting, which uses normal-only training images with synthetic self-supervision and no real defects or masks. We agree with you that fitting the reasoning head on synthetic positives is a form of supervised learning, and that this is different from AD without any positives. Two facts limit how much depends on it. The descriptors, banks and calibration use normal data only, and even without a learned rule our retrieval cost already reaches 69.8 P-AP on MVTec-AD (Table 4, row 1). The head also does not need to be trained on target data: in Table 3 it is trained on MPDD and applied unchanged to MVTec-AD and VisA. We will state clearly that our scope is industrial visual anomaly localization, and we will not present the method as a solution for AD in general.

W6. Saturated benchmarks. We agree that image-level AUROC is saturated, and we make no claim there. Localization still has considerable headroom: pixel AP is 75.1 / 46.5 / 47.3 / 40.6 on the four datasets. Our main evidence is the set of matched controls in Table 4, which does not rely on leaderboard margins.

W7. Tabular AD. Thank you for this suggestion. We agree that the motivation is not specific to images, and that evaluating on many datasets would guard against overfitting. The current implementation, however, relies on region construction, patch retrieval and texture synthesis. A tabular version would need new evidence sources and a validated pseudo-anomaly scheme. We see it as a natural direction for future work and will say so in the paper.

We hope these clarifications are helpful, and we would be happy to discuss any remaining concerns.

Thank you for your response. I actually reinforces my concerns regarding the writing and novelty, as well as the soundness of evaluation. I will keep my rejection score

Reviewer_favQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on the unsupervised anomaly detection task using only anomaly-free training data. A new framework, i.e., R2AD, is proposed to address the retrieval-to-reasoning gap by separating evidence acquisition from evidence reasoning. In particular, calibrated evidence spectrum retrieval and conflict-aware evidence-specialized reasoning are introduced. The experimental results show the effectiveness of the proposed method on several datasets.

Critical strengths
  1. The key idea of separating evidence acquisition from evidence reasoning is interesting and well-motivated for unsupervised anomaly detection.
  2. The experimental results under multi-class and few-shot settings show superior performance of the proposed method against several strong baselines.
Critical weaknesses
  1. The central motivation of R2AD is to address the retrieval-to-reasoning gap. However, for self-supervised training, this paper simply follows existing work (Cai et al., 2026) by generating synthetic image and mask pairs via blending textures. I am not sure whether MoE can learn reliable information from such supervision.
  2. Insufficient justification for cue selection and construction. Four cues (i.e., appearance, geometry, structure, and semantic) are extracted from specific blocks in a frozen DINOv2-B encoder. Why can these specific blocks correspond to those four cues? In addition, it remains unclear whether these descriptors can address illumination variations.
  3. The ablation studies are not thorough. The effect of different backbone architectures is unclear. The individual contribution and incremental combinations of the four cues have not been investigated.
  4. Following existing works, it is better to validate the generalization performance of the proposed method by merging multiple public datasets into a single large-scale benchmark.
  5. Minor issues. Fig.2 described in L45-L52 should be Fig. 1.
Questions for authors
  1. For the results in Table 4, why does the capacity-matched filter lead to a performance drop on Real-IAD?
  2. How does the number of experts affect training stability, computational cost, and inference time?
2 comments

We sincerely thank the reviewer for the constructive comments. Our point-by-point responses follow.

Q1. Capacity-matched filter on Real-IAD. Thank you for this question. The filter sees only the collapsed scalar cost, so extra capacity cannot recover the source identity that was discarded; this explains why it does not help. The drop has a further cause. On synthetic defects every high-cost region is a positive, so the filter learns to amplify such regions. On Real-IAD these regions also arise from benign viewpoint change, and false positives are amplified as a result (-3.9 P-AP). On aligned MVTec-AD objects the same behaviour helps (+1.3). We did not tune this baseline beyond the shared schedule, so optimization may also play a part.

Q2. Number of experts. We ran this study on MVTec-AD (256×256, batch size 1, one RTX 4090; latency excludes the one-time bank construction):

JParams (M)Latency (ms)I-AUROC / P-AUROC
1119.97.0198.9 / 98.1
2124.57.5699.2 / 98.5
3129.28.1099.5 / 98.8
4133.98.7699.4 / 98.7
5138.59.3899.3 / 98.7

Each expert adds about 4.7 M parameters and 0.6 ms. Accuracy improves up to J = 3 and not beyond, so we use J = 3 (32.8 GFLOPs). On stability, a balance loss (weight 0.005) prevents routing collapse. We did not run multiple seeds per J, so the 0.1 differences between J = 3 and J = 4 should not be over-interpreted.

W1. Reliability of synthetic supervision. We understand this concern. We keep RAID's generator fixed on purpose, so that only the representation and the decision rule vary; all variants in Table 4 use identical synthetic pairs. The synthetic masks train only the reasoning head, while the encoder, banks and calibration rely on normal data alone. Two observations suggest that the head learns a usable rule. First, under this fixed supervision, Real-IAD P-AP rises from 35.9 (identity removed) to 38.1 (monotone fusion) and 40.6 (full model), so the same supervision yields more as the evidence becomes more informative. Second, the head never sees pixels or features, only four rank values per patch, so it cannot fit the appearance of pasted textures. In line with this, a head trained on MPDD transfers, frozen, to MVTec-AD and VisA (Table 3).

W2. Cue construction and illumination. The cues are defined by the operators, not by the blocks: the patch token (appearance), a 3×3 neighbourhood residual (geometry), the region layout (structure), and the patch token with the class token (semantics). For example, structure and semantics both read block 12, yet behave differently (65.5 vs. 58.0 P-AP alone, App. E.2). Conversely, replacing the operators with plain tokens from the same depths lowers P-AP from 75.1 / 40.6 to 68.7 / 34.6 (Table 4), so the gain does not come from using multiple depths. We do not claim that blocks 8/10/12 are optimal. On illumination, you raise a fair point: the descriptors are not photometrically invariant by construction, and we did not test illumination in isolation. We will qualify the related claims in the abstract and introduction.

W3. Ablations. For backbones, please see our response to Reviewer oTJf (W2): reasoning adds 3.9 to 5.3 P-AP on DINOv2-S, DINOv2-B and DINOv3. For cues, App. E.2 reported each cue alone, and we agree that leave-one-out and incremental combinations were missing. We have now run them (MVTec-AD, P-AP; full model 75.1):

CueAlone (App. E.2)Full model without itDrop
Appearance58.673.71.4
Geometry62.872.82.3
Structure65.571.53.6
Semantics58.073.91.2

Adding cues incrementally, in order of single-cue strength, gives structure 65.5; + geometry 69.6; + appearance 73.9; + semantics 75.1. Removing structure causes the largest drop (3.6 P-AP) and removing semantics the smallest (1.2), and every cue contributes. We do not claim that the four-cue set is optimal or minimal.

W4. Merged benchmark. We agree that this is a useful test, and we regret that we could not complete it within the discussion period. In R2AD the banks and calibration stay per category and the reasoning module receives no category label, so merging datasets mainly tests category selection among more classes; within one dataset, predicted routing is within 0.4 to 0.6 P-AP of oracle routing (Table S2). The few-shot experiments also apply a module trained on MPDD to MVTec-AD and VisA unchanged. We plan to add the merged setting to the final version and will state the current scope explicitly.

W5. Thank you; corrected to Fig. 1a/1b.

We hope these responses address your concerns, and we would be glad to clarify anything further.

I appreciate the author’s clarifications. However, my main concerns about the motivation and evaluation are not fully addressed. Thus, I would like to keep my original rating.

Reviewer_oTJfOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

While retrieval- and reconstruction-based methods dominate industrial anomaly detection, existing approaches typically fuse multiple descriptors without keeping track of which descriptor caused each discrepancy. The authors formalize this limitation through two propositions: one showing that permutation-invariant summaries lose source information, and the other demonstrating that monotone fusion rules fail on complex evidence configurations. To address this, the authors propose R2AD. Using a single forward pass of a frozen DINOv2-B encoder, R2AD extracts four types of evidence,appearance, geometry, structure, and semantics, and performs category-conditioned retrieval. It then applies empirical rank calibration to align the retrieval costs into a unified evidence spectrum, followed by a conflict-aware Mixture-of-Experts (MoE) to reason over the joint evidence. This framework effectively shifts retrieval-based anomaly detection from simple score aggregation to multi-factor diagnosis. R2AD achieves strong localization results (75.1 pixel AP / 96.6 AUPRO on MVTec-AD, and 46.5 pixel AP / 94.1 AUPRO on VisA), with thorough ablations verifying the contribution of each component.

Critical strengths

1.This paper provides a very solid theoretical motivation. Proposition 1 proves that permutation-invariant aggregation loses information, making separable classes inseparable. Proposition 2 shows that monotone fusion rules cannot handle non-monotone logic, where consistent increases actually indicate normality. These two constructive proofs offer strong mathematical support for the subsequent architecture design.

2.The authors evaluate their model on four datasets with different characteristics. This demonstrates its competitive performance. The ablation studies are also thorough. They clearly show that each module is useful.

Critical weaknesses

1.The few-shot evaluation in Table 3 appears unfair. R2AD’s reasoning model is trained on synthetic anomalies from the full normal set. Baselines like WinCLIP never see target categories during training. Restricting only the memory bank to K-shots at inference does not constitute true few-shot learning. I have noticed similar protocol mismatches in other recent works, which has been confusing regarding what “few-shot” actually means in this field. Could the authors clarify this distinction?

2.The evaluation relies exclusively on a single frozen backbone, DINOv2-B/14 with registers. The authors don’t evaluate their framework on newer-generation vision foundation models such as DINOv3, and also don’t provide basic backbone ablations .

Questions for authors

1.The structure descriptor sorts regions by polar angle and concatenates them into a vector. However, a slight in-plane rotation causes a wrap-around effect at the angular boundary. Regions crossing this boundary jump from the end to the start of the sequence. This misaligns the concatenated vector and artificially inflates cosine distances. It contradicts the claimed robustness to pose variations.

2.See weakness 1.

3.The training data is made by pasting textures onto normal images. This only creates fake defects (Y=1). It cannot create normal images taken from new angles (Y=0). So, the model never sees cases where all four factors go up together but the image is still normal. This causes a big problem. The model cannot learn to ignore these all factors go up cases as safe changes. Because large fake defects also make all four factors go up at the same time (labeled Y=1), the model likely learns the opposite rule: when all factors go up together, it must be a bad defect. Could you please clarify this?

1 comment

We sincerely thank the reviewer for the careful reading and the constructive questions. They helped us find several places where the paper was unclear or claimed more than it showed. Our point-by-point responses follow.

Q1. Rotation sensitivity of the structure descriptor. Thank you for catching this; you are right. Angular ordering removes the dependence on cluster indices and on absolute position, but the ordering is discontinuous at the angular cut, so a small in-plane rotation can move a region across the cut and shift the entire layout block. We never meant to claim robustness to rotation, and we will remove the wording that gave this impression. In practice, two properties keep the effect limited. First, the reference costs are computed with the same operator on held-out normal images, so wherever the normal pose varies, this instability is already reflected in the reference distribution. Second, the costs are averaged over k = 5 neighbours. This is consistent with the rotation-varying categories screw, hazelnut and metal nut, which still reach 97.5 / 100 / 100 I-AUROC (Table S7). Structure is also only one of four sources (65.5 P-AP alone vs. 75.1 for the full model, App. E.2). We will state this limitation clearly in Sec. 3.1 and mention a circular soft-binned layout as a continuous alternative for future work.

Q2 / W1. Few-shot protocol. We apologise for the confusion here: the submission did not say how the reasoning module was trained for Table 3. Table 3 follows the auxiliary-data protocol of RAID. The module is trained only on MPDD and then applied, frozen, to MVTec-AD and VisA, so no image from the target dataset is used for training. For each target category, only the K normal shots are used to build the banks, the category centroids and the calibration references. The setting is therefore auxiliary-trained few-shot transfer. We agree with you that this is comparable to other auxiliary-trained methods such as RAID, but it is not a like-for-like comparison with training-free methods such as WinCLIP, which use no additional training data. To make this transparent, Table 3 and App. H.1 will state, for every method, which data its trainable parameters see.

Q3. Coherent elevation under synthetic training. This is a fair and important point. You are right that the synthesizer creates only positives and provides no label-0 examples of new viewpoints; the only label-0 supervision comes from the real normal pixels outside the pasted regions. We would like to clarify that we do not assume that "all sources up" means normal. Large synthetic defects keep label 1, and Prop. 2 is an existence example rather than a labelling rule. What the architecture contributes is context. The router (Eq. 4) combines local evidence with the global average of the evidence field, so a local all-up region on an otherwise quiet background can be treated differently from an elevation spread over the whole image. The gate (Eq. 5) scales only the preservation path, so coherently elevated defects are not forced to be suppressed. Both components matter in practice (App. E.3, P-AP / AUPRO):

VariantMVTec-ADReal-IAD
Full R2AD75.1 / 96.640.6 / 92.1
w/o global context73.8 / 96.038.3 / 90.8
Gate fixed to 171.7 / 95.135.4 / 88.6

The drops are larger on Real-IAD, where viewpoint variation is stronger. We should be clear about what this does and does not show: the components improve accuracy, but these results do not prove that the model has learned to ignore unseen viewpoints. We will revise the abstract and Sec. 3.4 to describe the mechanism as context-dependent modulation, and we will no longer present it as robustness to viewpoint change.

W2. Backbones. Thank you for this suggestion. For each encoder, we rebuilt the banks and the calibration and retrained the reasoning module. The table reports P-AP on MVTec-AD / Real-IAD:

EncoderScalar costFull R2ADGain
DINOv2-S68.0 / 29.572.3 / 33.4+4.3 / +3.9
DINOv2-B (submitted)69.8 / 35.775.1 / 40.6+5.3 / +4.9
DINOv3 ViT-B71.4 / 37.176.4 / 41.5+5.0 / +4.4

Reasoning adds 3.9 to 5.3 P-AP on every encoder, and moving to DINOv3 improves the full model further. We will include this study in the revision.

We hope these responses address your concerns, and we would be glad to clarify anything further.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.