- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Paper deals with fusing retrieval scores for anomaly detection when multiples factors. It first normalizes them by rank. It then executes a rather complex procedure to arrive at decision rules. The results on standard AD benchmarks exceeds previous methods.
Critical strengths
Anomaly detection is a challenging problem whose precise objective is still being worked out. Papers challenging the standard practices are highly encourage. The empirical results of the approach are quite good.
Critical weaknesses
While the writing is grammatical, it is dense and hard to parse. It is unclear to me why the proof is particularly informative or useful. I also think the claim that different factor ought to be treated differently is not actually controversial even if some approaches neglected it. Rank normalization is of features in AD is by no means new e.g., HBOS/LODA. The proposed features seem somewhat adhoc. At the end, the objective here is not to propose new features and it would have been better to use standard features from the literature. The approach relies heavily on pseudo supervision (synthetic anomalies). While this is fair game for unsupervised defect detection, it does not address the more interesting challenges of AD (not having positive data). It becomes a supervised learning task (although with low quality data). As the goal is not actually defect detection but AD, this limits the contribution. I am personally not so excited by empirical gains on the current, highly saturated defect detection benchmarks. Actually, it does not appear to me this method is too tied up to images. Indeed, it might be more natural for tabular AD problems. There are many datasets, and it would be interesting to see if the MoE approach would improve results there. An added bonus is that it is not easy to overfit those benchmarks as there are many of them.
Questions for authors
See cons
2 comments
Thank you for your response. I actually reinforces my concerns regarding the writing and novelty, as well as the soundness of evaluation. I will keep my rejection score
We sincerely thank the reviewer for the candid and thoughtful assessment. Several of your points are well taken, and they have changed how we position the work. Our responses follow the order of your comments.
W1. Writing. We are sorry that the paper was hard to follow. We will open the paper with a three-step description of the method: (i) four descriptors each retrieve a distance to normal references; (ii) each distance is converted into a tail rank among normal distances, which makes the values comparable across descriptors while keeping each one labelled by its source; (iii) a small network scores these labelled values together with image context, so that a value can be interpreted in light of its source and of the other values.
W2. Propositions and novelty. We agree that the principle itself is not controversial, and we do not present it as our contribution. The propositions are elementary and serve a single purpose. They separate two restrictions, losing source identity and imposing a monotone rule, and each one corresponds to a matched control in Table 4. Removing identity costs 3.0 / 4.7 P-AP on MVTec-AD / Real-IAD, and imposing a monotone rule costs 1.4 / 2.5. What we contribute is a measurement of these costs under matched conditions, together with evidence that current retrieval pipelines incur them.
W3. Rank normalization (HBOS, LODA). Thank you for pointing us to this line of work; we agree. Per-dimension rank or density normalization is well established (HBOS, LODA, and also ECOD and COPOD). We will cite these methods and revise contribution (ii) so that calibration is presented as an adopted tool. Its role in our method is to make costs comparable across sources while keeping their labels. The connection is also informative for our question. These methods go on to sum the per-dimension scores, which is a permutation-invariant and monotone rule, and this is the class of rules that our two controls examine. On the same calibrated evidence, removing the labels costs 3.0 / 4.7 P-AP and a monotone rule costs 1.4 / 2.5.
W4. Ad hoc features. We agree that the operators are instances and should not be read as the contribution. Table 4 already includes standard features, namely plain DINOv2 tokens from four depths. Reasoning still improves them from 66.1 / 32.2 to 68.7 / 34.6 P-AP, so the benefit does not require our operators, although it is larger with them (+5.3 / +4.9). In addition, a new backbone study (please see our response to Reviewer oTJf, W2) shows that the gain holds across encoders, at +3.9 to +5.3 P-AP on DINOv2-S, DINOv2-B and DINOv3.
W5. Pseudo-supervision. We follow the standard industrial setting, which uses normal-only training images with synthetic self-supervision and no real defects or masks. We agree with you that fitting the reasoning head on synthetic positives is a form of supervised learning, and that this is different from AD without any positives. Two facts limit how much depends on it. The descriptors, banks and calibration use normal data only, and even without a learned rule our retrieval cost already reaches 69.8 P-AP on MVTec-AD (Table 4, row 1). The head also does not need to be trained on target data: in Table 3 it is trained on MPDD and applied unchanged to MVTec-AD and VisA. We will state clearly that our scope is industrial visual anomaly localization, and we will not present the method as a solution for AD in general.
W6. Saturated benchmarks. We agree that image-level AUROC is saturated, and we make no claim there. Localization still has considerable headroom: pixel AP is 75.1 / 46.5 / 47.3 / 40.6 on the four datasets. Our main evidence is the set of matched controls in Table 4, which does not rely on leaderboard margins.
W7. Tabular AD. Thank you for this suggestion. We agree that the motivation is not specific to images, and that evaluating on many datasets would guard against overfitting. The current implementation, however, relies on region construction, patch retrieval and texture synthesis. A tabular version would need new evidence sources and a validated pseudo-anomaly scheme. We see it as a natural direction for future work and will say so in the paper.
We hope these clarifications are helpful, and we would be happy to discuss any remaining concerns.