- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Contents In industrial environments, collecting sufficient defect datasets is costly and difficult. Therefore, this paper reports experimental results on whether frozen vision-language models (VLMs) can diagnose specific defect types by relying solely on textual descriptions and normal reference images, without using defect-image exemplars at diagnosis time.
Claimed Contributions
- Evaluation of textual inspection knowledge: The paper empirically demonstrates that providing concrete textual descriptions of defects improves defect-type classification accuracy over providing bare class names, and that matching descriptions to the correct labels outperforms randomly permuted (deranged) descriptions.
- Analysis of error patterns across text and spatial evidence: The authors report how different inputs change error types—specifically, that visual anomaly marking (via DINOv2) primarily recovers missed defects predicted as NORMAL, whereas textual descriptions also correct confusion between different defect types, while detector gating trades defect recall for lower false alarms.
- Normal-reference diagnosis pipeline: The paper presents a training-free configuration that combines DINOv2-based normal-reference patch comparison, cropped/enlarged visual views, and textual criteria to predict either a specific defect type or NORMAL.
Validation
- Instead of comparing against existing defect analysis or anomaly classification baselines, the authors validate their pipeline exclusively through internal ablations using two frozen VLMs (Gemma-4-26B-A4B and Qwen3.6-35B-A3B) and a frozen DINOv2 encoder.
- Experiments are conducted on selected subsets of two public benchmarks (MVTec AD with 5 categories and Real-IAD with 5 categories) and a private semiconductor BGA dataset (90 evaluation defects across 11 types and 12 normal images).
- To simulate human inspector knowledge, the authors use another VLM (gemma-4-31B-it) to generate textual descriptions from held-out defect images, and then evaluate type accuracy and false positive rates (using exact McNemar tests) by varying prompt conditions (e.g., class names only, generic definitions, region-derived descriptions, deranged labels, and direct image exemplars), visual markings, and detector gating thresholds.
Critical strengths
This paper provides a detailed empirical breakdown of how spatial anomaly marking and textual descriptions differently affect specific error types (recovering missed defects vs. resolving defect-type confusions), alongside an explicit analysis of the trade-off between defect recall and normal false positive rates.
Critical weaknesses
Lack of Comparisons
- The authors frame their setup around diagnosing defects without defect-image exemplars at diagnosis time. Since this is a very common scenario in industrial visual inspection, numerous existing works already address this kind of problem. Simply observing and reporting phenomena using two VLMs (Gemma4 and Qwen3.6) without comparing against existing defect analysis or classification baselines is insufficient. Furthermore, although Real-IAD contains many defect instances and categories, the authors evaluate on only MVTec AD and Real-IAD. Authors should have conducted broader experiments across additional public defect datasets beyond the two datasets presented in this paper (e.g., the Magnetic-tile defect dataset, NEU-DET, etc.).
Rambling & Unstructured Writing
-
There are abrupt logical leaps throughout the text. For example, in the first paragraph of the Introduction, the authors write: "...NaturalBench shows that strong VLMs can fail when answers depend on visual differences rather than language priors (Li et al., 2024). Industrial diagnosis therefore requires direct evaluation under the information constraints of inspection." Jumping directly from VLM failures on natural image pairs to the necessity of evaluating under specific industrial information constraints lacks intermediate logical grounding.
-
Overall, this manuscript reads like a fragmented collection of experimental logs rather than a cohesive scientific paper. The experimental setups in Sections 4 and 5 shift arbitrarily across different visual views, prompt conditions, and gating rules, making the narrative rambling and very difficult to follow.
Unclear Problem Formulation
- This paper primarily presents an empirical observation of specific model behaviors and failure modes, but it fails to clearly formulate a scientific problem or establish a concrete research objective beyond reporting these phenomena. Furthermore, this paper makes little effort to investigate how the identified failure modes (such as detector misses, high false-positive rates, or per-type failures) could be methodologically resolved or how model performance could be systematically improved. As a result, the contribution remains largely observational, leaving both the scientific motivation and the practical significance of this study unclear.
Unclear and Potentially Misleading Terminology
-
In Section 3.3, this paper introduces how the frozen VLM ingests the visual views, textual inspection knowledge , and defect taxonomy using only a high-level symbolic expression: . Even though the exact way these multi-modal signals and prompt structures are combined is critical to VLM performance, this paper omits concrete formulation details and exact prompt structures from both the main text and the Appendix.
-
Several terms are introduced without clear definitions or context, such as "General benchmark" and "NaturalBench" in the Introduction. Granted, a reader might infer their rough meaning from context, but academic writing still requires at least a one-line explanation or appropriate citations when introducing such benchmarks.
-
In Section 4, the term "single-image factorial" is introduced without definition and is potentially misleading. In experimental design, "factorial" refers to systematically evaluating all combinations of multiple discrete factors. However, the main text does not clearly specify which factors and levels constitute this design or how the setup qualifies as a factorial experiment when the term first appears. The authors should clarify the intended meaning of this term and justify its usage in main page.
-
The term "generic definition" (or "generic criteria") is also overly ambiguous. This paper does not clearly illustrate in the main text what level of information is included in a "generic definition" versus "concrete descriptions," leaving the distinction vague to the reader.
Description Mismatch
- In Section 5.1, while referring to Figure 2(a), the authors state that "Improvements extend across most product/model comparisons, but are not universal." However, Figure 2(a) only plots the aggregate gains across the four dataset/model pairs—all four of which are positive (+19.8, +16.9, +3.7, and +11.7 pp)—visually showing universal improvement at the plotted level. Making a claim about non-universal product-level behavior while citing a figure that only displays positive aggregate bars creates a confusing mismatch between the text and the referenced figure.
Questions for authors
Q1. Why did you not conduct experiments on MMAD, even though you explicitly cite and refer to this benchmark in the very early page ?
Thank you for the additional analyses, which make the empirical study more careful. However, my central concern about the paper’s positioning remains. The revised response increasingly presents the work as a set of conditional observations: the advantage over retrieval depends on spatial evidence, the effect of information sources varies across models, and effective descriptions can only be identified through downstream evaluation. These are useful caveats, but they do not yet combine into a transferable finding or design principle that would justify the text-mediated approach as a contribution in its own right. I therefore maintain my original recommendation.