acceptodds
Under review as a conference paper at ICLR 2027

Reading Anomalies as Types: Adapting Pretrained Models for Industrial Anomaly Diagnosis

Abstract

Scarce and diverse defect examples make product-specific inspection models costly to train. Detecting a deviation from normal is only part of the task: diagnosis also requires identifying which product-specific inspection criterion it violates. We study defect diagnosis with frozen visual and vision-language models, combining normal-reference localization, targeted views, and textual inspection criteria. The target setting requires no defect-image exemplars at diagnosis time. Our experiments use descriptions derived from held-out defect images as a proxy for inspector knowledge; direct image access is tested only in separate controls. On public industrial benchmarks and semiconductor data, descriptions improve defect typing with model weights and visual inputs fixed. Correctly aligned descriptions outperform descriptions assigned to the wrong labels, supporting the importance of matching inspection criteria to defect types. Across separate controlled comparisons, visual marking primarily recovers missed defects, while descriptions can also correct type errors. Detector misses and uneven per-type performance limit deployment claims. These results identify textual inspection criteria as a means of adapting frozen models to product-specific diagnoses.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Defect Detection, Industrial Inspection, Pretrained Models, Vision-Language Models, Prior Knowledge

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_U3arOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Contents In industrial environments, collecting sufficient defect datasets is costly and difficult. Therefore, this paper reports experimental results on whether frozen vision-language models (VLMs) can diagnose specific defect types by relying solely on textual descriptions and normal reference images, without using defect-image exemplars at diagnosis time.

Claimed Contributions

  • Evaluation of textual inspection knowledge: The paper empirically demonstrates that providing concrete textual descriptions of defects improves defect-type classification accuracy over providing bare class names, and that matching descriptions to the correct labels outperforms randomly permuted (deranged) descriptions.
  • Analysis of error patterns across text and spatial evidence: The authors report how different inputs change error types—specifically, that visual anomaly marking (via DINOv2) primarily recovers missed defects predicted as NORMAL, whereas textual descriptions also correct confusion between different defect types, while detector gating trades defect recall for lower false alarms.
  • Normal-reference diagnosis pipeline: The paper presents a training-free configuration that combines DINOv2-based normal-reference patch comparison, cropped/enlarged visual views, and textual criteria to predict either a specific defect type or NORMAL.

Validation

  • Instead of comparing against existing defect analysis or anomaly classification baselines, the authors validate their pipeline exclusively through internal ablations using two frozen VLMs (Gemma-4-26B-A4B and Qwen3.6-35B-A3B) and a frozen DINOv2 encoder.
  • Experiments are conducted on selected subsets of two public benchmarks (MVTec AD with 5 categories and Real-IAD with 5 categories) and a private semiconductor BGA dataset (90 evaluation defects across 11 types and 12 normal images).
  • To simulate human inspector knowledge, the authors use another VLM (gemma-4-31B-it) to generate textual descriptions from held-out defect images, and then evaluate type accuracy and false positive rates (using exact McNemar tests) by varying prompt conditions (e.g., class names only, generic definitions, region-derived descriptions, deranged labels, and direct image exemplars), visual markings, and detector gating thresholds.
Critical strengths

This paper provides a detailed empirical breakdown of how spatial anomaly marking and textual descriptions differently affect specific error types (recovering missed defects vs. resolving defect-type confusions), alongside an explicit analysis of the trade-off between defect recall and normal false positive rates.

Critical weaknesses

Lack of Comparisons

  • The authors frame their setup around diagnosing defects without defect-image exemplars at diagnosis time. Since this is a very common scenario in industrial visual inspection, numerous existing works already address this kind of problem. Simply observing and reporting phenomena using two VLMs (Gemma4 and Qwen3.6) without comparing against existing defect analysis or classification baselines is insufficient. Furthermore, although Real-IAD contains many defect instances and categories, the authors evaluate on only MVTec AD and Real-IAD. Authors should have conducted broader experiments across additional public defect datasets beyond the two datasets presented in this paper (e.g., the Magnetic-tile defect dataset, NEU-DET, etc.).

Rambling & Unstructured Writing

  • There are abrupt logical leaps throughout the text. For example, in the first paragraph of the Introduction, the authors write: "...NaturalBench shows that strong VLMs can fail when answers depend on visual differences rather than language priors (Li et al., 2024). Industrial diagnosis therefore requires direct evaluation under the information constraints of inspection." Jumping directly from VLM failures on natural image pairs to the necessity of evaluating under specific industrial information constraints lacks intermediate logical grounding.

  • Overall, this manuscript reads like a fragmented collection of experimental logs rather than a cohesive scientific paper. The experimental setups in Sections 4 and 5 shift arbitrarily across different visual views, prompt conditions, and gating rules, making the narrative rambling and very difficult to follow.

Unclear Problem Formulation

  • This paper primarily presents an empirical observation of specific model behaviors and failure modes, but it fails to clearly formulate a scientific problem or establish a concrete research objective beyond reporting these phenomena. Furthermore, this paper makes little effort to investigate how the identified failure modes (such as detector misses, high false-positive rates, or per-type failures) could be methodologically resolved or how model performance could be systematically improved. As a result, the contribution remains largely observational, leaving both the scientific motivation and the practical significance of this study unclear.

Unclear and Potentially Misleading Terminology

  • In Section 3.3, this paper introduces how the frozen VLM VV ingests the visual views, textual inspection knowledge KpK_p, and defect taxonomy TpT_p using only a high-level symbolic expression: y^=V(views(x,Rp),Kp,Tp)\hat{y} = V(\text{views}(x, R_p), K_p, T_p). Even though the exact way these multi-modal signals and prompt structures are combined is critical to VLM performance, this paper omits concrete formulation details and exact prompt structures from both the main text and the Appendix.

  • Several terms are introduced without clear definitions or context, such as "General benchmark" and "NaturalBench" in the Introduction. Granted, a reader might infer their rough meaning from context, but academic writing still requires at least a one-line explanation or appropriate citations when introducing such benchmarks.

  • In Section 4, the term "single-image factorial" is introduced without definition and is potentially misleading. In experimental design, "factorial" refers to systematically evaluating all combinations of multiple discrete factors. However, the main text does not clearly specify which factors and levels constitute this design or how the setup qualifies as a factorial experiment when the term first appears. The authors should clarify the intended meaning of this term and justify its usage in main page.

  • The term "generic definition" (or "generic criteria") is also overly ambiguous. This paper does not clearly illustrate in the main text what level of information is included in a "generic definition" versus "concrete descriptions," leaving the distinction vague to the reader.

Description Mismatch

  • In Section 5.1, while referring to Figure 2(a), the authors state that "Improvements extend across most product/model comparisons, but are not universal." However, Figure 2(a) only plots the aggregate gains across the four dataset/model pairs—all four of which are positive (+19.8, +16.9, +3.7, and +11.7 pp)—visually showing universal improvement at the plotted level. Making a claim about non-universal product-level behavior while citing a figure that only displays positive aggregate bars creates a confusing mismatch between the text and the referenced figure.
Questions for authors

Q1. Why did you not conduct experiments on MMAD, even though you explicitly cite and refer to this benchmark in the very early page ?

Reviewer_Gzh2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies defect typing with frozen DINOv2 and vision-language models. Normal-reference patch comparisons select regions, and Gemma4 or Qwen3.6 receives query/reference views and textual defect criteria. Experiments use 649 public-benchmark defects and 90 semiconductor defects, with descriptions derived from held-out defect images as a proxy for inspector knowledge. Separate comparisons examine description content, anomaly marking, and detector gating.

Critical strengths
  1. The fixed-view comparisons establish a substantial effect of exemplar-derived descriptions in some settings. On MVTec, combined descriptions raise Gemma4 accuracy from 46.70 to 66.48 percent and Qwen3.6 from 38.68 to 55.59 (Table 3).

  2. Label permutations, authoring-model substitutions, and shortcut controls help delimit what these gains demonstrate. Reporting failures and false alarms makes the empirical evidence more informative than headline accuracy alone.

Critical weaknesses
  1. The central contribution is a limited extension of established context adaptation to an industrial task. Section 2 already identifies normal-reference patch matching, textual descriptions, and visual evidence selection as existing capabilities. Their connection here does not introduce a new adaptation mechanism or solve a previously unsupported task formulation: a frozen VLM receives more informative class descriptions and localized views within a supplied taxonomy. An empirical paper need not propose a new algorithm, but it must then establish a consequential new finding. The strongest headline result, that correctly aligned exemplar-derived descriptions outperform class names or permuted descriptions, establishes that the model uses task-relevant information; it does not establish a distinctive principle beyond familiar context conditioning. The paper leaves unclear what researchers should now understand or do differently beyond supplying better descriptions and visual evidence.

  2. The proposed distinction between semantic criteria and spatial evidence remains descriptive rather than explanatory. Appendix B.5 reports that descriptions correct 75 wrong types and recover 6 misses for Gemma4 on MVTec, versus 9 and 54 for Qwen3.6. Thus, the same intervention predominantly changes different errors across models. The paper acknowledges dependence on initial errors, but does not establish which properties of the criteria or models determine the outcome. Table 7 crosses generic criteria with marking, while the main description gains use exemplar-derived text in another configuration. These results do not yield a tested rule for choosing or combining the information sources. The error accounting is useful, but its scientific value is narrower than an explanation of how to adapt frozen models for diagnosis.

  3. The practical advantage of the text-mediated approach is not established against simpler uses of the available supervision. The strongest descriptions are derived from labeled defect exemplars, whereas generic definitions yield no Holm-significant public gains (Table 5). In Table 12, changing the joint-description author from the primary model to Qwen3.6 reduces correct semiconductor diagnoses from 50 to 29 for Gemma4 and from 41 to 14 for Qwen3.6. This makes description construction central to the result, yet the study does not identify a reliable way to obtain effective criteria. Moreover, Gemma4 joint descriptions achieve macro-F1 of 34.1 versus 34.4 for DINO 1-NN, with 58.3 percent normal FPR; Qwen3.6 matches 1-NN's 45.6 percent type accuracy while missing 39 of 90 defects (Table 8). These are different operating points because retrieval always assigns a defect type, so they do not establish retrieval superiority either. They leave the claimed practical value unresolved: why prefer exemplar-to-text conversion and VLM inference over feature retrieval using comparable support information and matched rejection?

Questions for authors

What finding changes existing understanding of context-based diagnosis beyond demonstrating gains from informative inputs? What explains the authoring-model sensitivity, and can effective criteria be identified independently of downstream evaluation?

1 comment

Thank you for the additional analyses, which make the empirical study more careful. However, my central concern about the paper’s positioning remains. The revised response increasingly presents the work as a set of conditional observations: the advantage over retrieval depends on spatial evidence, the effect of information sources varies across models, and effective descriptions can only be identified through downstream evaluation. These are useful caveats, but they do not yet combine into a transferable finding or design principle that would justify the text-mediated approach as a contribution in its own right. I therefore maintain my original recommendation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.