acceptodds
Under review as a conference paper at ICLR 2027

Reference Saturation: Normal Exemplars Buy Structural Anomaly Detection Twice as Fast as Logical

Abstract

Zero- and few-shot anomaly detectors are almost always reported on MVTec LOCO as a single pooled AUROC, even though the benchmark separates structural anomalies, whose local appearance is atypical, from logical anomalies, whose every local region looks normal and only the composition is wrong. We re-measure the reference-based family under one protocol, split by subset, and find a stable asymmetry: going from zero to thirty-two normal exemplars closes 41.9% of the remaining headroom on structural anomalies but only 20.7% on logical ones, a ratio of 2.02 (95% CI [1.67, 2.51]), which an independent redraw of the thirty-two exemplars reproduces at 2.02 [1.69, 2.51]; it is 2.00 at sixteen exemplars and, for a second method with a different mechanism measured from its one-exemplar baseline, 1.75, and every interval excludes one. Reference diversity then saturates, and reference depth recovers little of what it leaves. Adding a third reference branch to a two-branch combination moves logical AUROC by at most 0.2 points while roughly doubling the exemplar budget, and spending the entire budget on one method's exemplar diversity buys nothing measurable: three independently seeded WinCLIP+ branches at 46.4 exemplars reach 66.4, against 66.1 for a single branch at 16 and 69.6 for the two-branch combination at that same 16; pooling the same budget into one 32-exemplar memory per branch instead, the strongest reference-only configuration we could build, reaches 70.5, a gain of 0.7 over the best diverse one that we cannot resolve. Replacing that third branch with a compositional part-count score — whose per-image scores on logical anomalies agree with the reference branches far more weakly than they agree with each other — raises logical AUROC from 69.6 to 73.3±1.2 over three independent draws of the composition branch's own references, at no more than 32 images per category. Against that deeper control the paired bootstrap over images gives +2.85, 95% CI [0.97, 4.79], and against the best three-branch reference configuration +3.58, 95% CI [1.77, 5.40]; structural AUROC does not move resolvably against either. With all three branches reading the same sixteen images, half the control's budget, the gain over it is +3.74, 95% CI [1.88, 5.66]. We report a pre-specified control that falsified our own initial two-branch claim, four protocol anchors against published numbers, and a spread of up to 17.8 points across three published tables for the same baseline at the same shot count. All per-image scores, exemplar manifests and per-category budgets are released. We make no state-of-the-art claim.

Primary area:
datasets and benchmarks
Keywords:
anomaly detection, logical anomaly detection, few-shot learning, vision-language models, evaluation and benchmarking

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qojEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates reference exemplar saturation for structural and logical anomaly detection on MVTec LOCO. It evaluates WinCLIP+ and PatchCore under a unified protocol. The results show approximately twice the normalized AUROC gain for structural anomalies. An OWLv2-based composition branch improves logical AUROC from 70.5% to 73.3% over the strongest reference-only control.

Critical strengths
  1. The paper is motivated by the importance of distinguishing between structural and logical anomalies, and its systematic evaluation across different reference budgets provides valuable empirical insights.

  2. The authors demonstrate careful experimental control by comparing reference diversity, memory depth, and compositional information under matched exemplar budgets.

Critical weaknesses
  1. ComAD [1] and SA-PatchCore [2] already address the limitations of local appearance matching through component-aware modeling. LogSAD [3] and UniVAD [4] also demonstrate the benefits of compositional information.

  2. The generalization of the proposed method is insufficient, as the experiments cover only five MVTec LOCO categories and two reference detectors. Section 7 incorrectly suggests that no other suitable industrial benchmark exists. CAD-SD [2] distinguishes local defects from co-occurrence anomalies and provides a relevant evaluation setting.

  3. The information-budget comparison is unfair because the Composition branch uses pretrained OWLv2 and manually specified part names, whereas the reference-only controls lack equivalent semantic priors. Therefore, matched exemplar counts do not imply matched information budgets. Moreover, the claimed half-budget configuration uses 32 versus 48 total normal images after including calibration images.

References:

[1] Liu et al. “Component-aware Anomaly Detection Framework for Adjustable and Logical Industrial Visual Inspection.” Advanced Engineering Informatics, 2023.

[2] Ishida et al. “SA-PatchCore: Anomaly Detection in Dataset With Co-Occurrence Relationships Using Self-Attention.” IEEE Access, 2023.

[3] Zhang et al. “Towards Training-free Anomaly Detection with Vision and Language Foundation Models.” CVPR, 2025.

[4] Gu et al. “UniVAD: A Training-free Unified Model for Few-shot Visual Anomaly Detection.” CVPR, 2025.

Questions for authors

Q1. How does the paper differ fundamentally from ComAD [1], SA-PatchCore [2], and LogSAD [3]? Is the main contribution the controlled reference-budget analysis rather than a new compositional detection mechanism?

Q2. Does the main conclusion remain statistically reliable without breakfast box?

Q3. Does component counting improve the final fusion after controlling for generic ensemble diversity? Can the authors report the missing no-count and lambda = 1 experiments? How much additional computation does OWLv2 require?

Reviewer_2hGHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper separates structural and logical anomaly detection on MVTec LOCO and measures how increasing normal exemplars affects WinCLIP+ and PatchCore. It reports that exemplar gains absorb roughly twice as much remaining AUROC headroom for structural as for logical anomalies, while reference-only combinations saturate. A budget-matched composition branch improves logical AUROC. The work provides bootstrap analyses, protocol audits, ablations, and a reproducibility artifact, while explicitly limiting its claims to LOCO.

Critical strengths
  • The study reports structural and logical results separately, documents its protocol and exemplar budgets, and supplies uncertainty analyses and an auditable artifact. This makes the reported effect substantially easier to inspect than pooled benchmark results.
  • The authors report a pre-specified control that falsified their initial two-branch claim and narrow the final conclusions accordingly. This strengthens the credibility of the empirical account.
Critical weaknesses
  • The main finding is benchmark- and category-specific. All primary claims come from the five categories of MVTec LOCO. The authors report that between-category variation is much larger than exemplar-draw variation (Sec. 4), so image-level bootstrap intervals do not establish that the roughly 2× asymmetry holds across categories or other settings. The title’s “twice as fast” framing also summarizes cumulative headroom ratios; the incremental ratios vary and the 16-to-32 interval is not measurable (Sec. 4; Appendix I). This limits the generality and rate interpretation of the central result.
  • Evidence across reference-based methods is limited, and the PatchCore result is not anchored to a validated implementation. The asymmetry is tested with WinCLIP+ and one in-house PatchCore implementation. The paper reports that its PatchCore AUROC is below the cited published range at four exemplars and below the cited one-exemplar value (Sec. 3), while the WinCLIP+ anchors reproduce published values more closely. Thus the second method provides weaker evidence that the effect extends beyond WinCLIP+.
  • The composition comparison matches exemplar count, but not the information supplied to each branch. The composition branch also receives a hand-written, category-specific part vocabulary and an external OWLv2 detector (Sec. 3; Sec. 7), whereas reference branches use normal exemplars alone. The reported gain supports the usefulness of this additional compositional signal, but does not isolate the effect of composition from those extra priors or establish a matched-information comparison.
Questions for authors
  1. Can the “twice as fast” conclusion be stated and tested as a cumulative headroom effect rather than a rate, given that the incremental ratios vary and the 16-to-32 interval is unresolved?
  2. Can you validate the PatchCore trend with an implementation that matches a published LOCO result, or otherwise show that the lower baseline does not drive the cross-method conclusion?
  3. Is there a control that removes the category-specific part vocabulary or the OWLv2 component while retaining the composition branch, to clarify which additional signal accounts for its gain?
Reviewer_yGaUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how the number of normal exemplars affects structural and logical anomaly detection, and whether methods based on normal-feature matching exhibit diminishing returns. The authors evaluate WinCLIP/WinCLIP+ and PatchCore across different exemplar counts on five MVTec LOCO AD categories, using the headroom absorbed metric to compare performance gains. They find larger cumulative gains for structural anomalies, with ratios of 2.02 for WinCLIP/WinCLIP+ (0–32 shots) and 1.75 for PatchCore (1–16 shots). Further experiments compare larger reference sets and additional WinCLIP+/PatchCore branches using different normal samples with a composition branch that uses OWLv2 to detect and count parts. The reference-only variants provide limited improvements in logical anomaly detection, while adding the composition branch increases the logical AUROC of the WinCLIP+–PatchCore ensemble from 69.6 to 73.3.

Critical strengths
  1. The paper separately analyzes structural and logical anomaly detection across normal-exemplar budgets and observes the same qualitative asymmetry with WinCLIP+ and PatchCore, offering a more informative evaluation than pooled AUROC alone.
  2. The evaluation uses clear controls across exemplar counts, additional reference branches, and the composition branch, while releasing per-image scores and exemplar manifests for independent verification.
Critical weaknesses
  1. The reference-saturation claim is not sufficiently supported by the current experiments. PatchCore does not show a clear saturation trend at the largest tested reference sizes, while the full scaling analysis is limited to PatchCore (2022) and WinCLIP+ (2023).
  2. The composition gains are uneven across categories. Against the strongest reference-only baseline, most improvement comes from breakfast box, while within-category image bootstrapping provides limited evidence of consistent benefits across categories.
Questions for authors
  1. Could the authors extend PatchCore to k=64 to clarify whether the curve reaches saturation? It would also be useful to repeat the scaling analysis on other anomaly detection methods proposed in the past two years.
  2. A comparison with the strongest reference-only baseline on another dataset covering different types of logical anomalies would help further validate the reported gains from the composition branch.
  3. The phrases "twice as fast" in the title and "twice the rate" in the conclusion suggest marginal scaling rates, whereas the reported ratios measure cumulative headroom gains. Clarifying this distinction would be helpful.
  4. Section 3 states that full command lines are released, but the supplement only reproduces results from saved scores. This could be addressed by revising the statement or supplementing the released materials.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.