- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper asks an interesting problem to better analyze what leads to good performance of CLIP-based anomaly detection methods. Specifically, it analyze how the text branch affects the final performance by manipulating the text prompt input. Their finding is that the geometry cue matters more than semantics.
Critical strengths
I think this paper opens an interesting direction to better understand what matters and what works for image anomaly detection methods, specifically CLIP-based anomaly detection methods.
Build upon the previous paper's empirical finding that text branch in the CLIP-based anomaly detection is not a must to pursue strong performance for image anomaly detection, this work basically further makes more in-depth analyses to study the effect of semantics and geometry.
Critical weaknesses
-
After careful reading of this work, I think the definition of the "geometry" is not very clear to me in this work. To the best of my understanding the "geometry" here is potentially related to the defects' shape or structure in the visual representation rather than the semantics contained in the text input. Thus, I would suggest better define the geometry here. Otherwise, it would lead to some confusions especially for people working in 3D anomaly detection domain or broader geometric/3D learning community.
-
While I think this work somehow convinces me that the semantics is not a must for image anomaly detection in the studied setup. I have some concerns on the sensitivity of the result to the input prompt. For example, the CLIP model itself might be sensitive or not generalizable to certain text input considering the domain gap of texts it used for training and the targeted anomaly detection task.
Questions for authors
-
Could you please clarify the definition of geometry in this work? I would recommend not to make it as an opposite term to semantics.
-
How do you ensure that the design of 5k prompts is sufficient to span the input space of CLIP text input to disentangle the effect of semantics and geometry?
-
Any further insights on the applicability of the results to zero-shot anomaly detection methods that are not CLIP based but VLM based? Would the result still hold? It would be great to discuss on that research line to consolidate the whole paper claims.
2 comments
Response to Reviewer 1Uqr (2/2)
Concern 3 (Q2): Are 5k prompts sufficient to span the input space of CLIP text input to disentangle the effect of semantics and geometry?
The coverage of 5k prompts is sufficient to support our conclusions. We do not claim to exhaust CLIP’s entire text space, which is infeasible theoretically and practically. However, our 5k prompts span manual templates, learned contexts, and randomly generated sources. Semantically, they range from explicit defect naming to empty strings and meaningless text. Geometrically, continuous interpolation varies the direction from the original text direction to its orthogonal direction. Therefore, although our 5k prompts cannot exhaust CLIP’s text space, they provide sufficient coverage to support our conclusions.
The 5k prompts are sufficient for semantic–geometric disentanglement. We disentangle semantics and geometry through conditional analysis rather than by increasing the number of prompts. Therefore, 5k prompts are enough for disentanglement. Specifically, we first fit performance using directional alignment and then add an explicit defect-semantics variable. If semantics contributed beyond direction, the model’s explanatory power should increase substantially. However, directional alignment alone explains 0.8736–0.9162 of the variance, whereas adding semantics yields a of no more than 0.0012. Thus, once geometry is determined, the additional contribution of defect semantics to performance is negligible, thereby achieving semantic–geometric disentanglement.
Concern 4 (Q3): Applicability of the results to zero-shot anomaly detection methods that are not CLIP-based but VLM-based?
Our conclusions also generalize to other non-CLIP VLMs. We conducted the full set of additional experiments on two architecturally compatible non-CLIP VLMs, SigLIP and BLIP-ITC, and added the results to Tables 14 and 15 in the appendix of the manuscript. Results from both pipelines support our conclusions. Reduction: the median per-class exceeds 0.9983. Replacement: for BLIP-ITC on VisA, even a single image yields better performance. Even in the worst case, BLIP-ITC on MVTec AD, 16 images are sufficient to achieve comparable performance. Erasure: erasing the text plane reduces performance to near-random levels, whereas erasing its orthogonal complement has little effect. Exhaustion: directional alignment explains 0.8183–0.8965 of the performance variance, while explicit defect semantics contributes only 0.0004–0.0039 additionally. Prediction: on 10k unseen prompts, the Spearman correlation between predicted and measured performance reaches 0.7829–0.8874, and the calibration slope is within 0.0111 of unity. Therefore, our core conclusion that text prompts carry geometry rather than semantics also holds for other architecturally compatible non-CLIP VLMs. Note that we do not include generation- and diffusion-based VLMs, as these models generally do not natively provide normal–anomalous text similarity scoring based on a shared image–text embedding space and therefore fall outside the scope of our setting.
Response to Reviewer 1Uqr (1/2)
We sincerely appreciate the reviewer's encouraging assessment and insightful comments. We have responded point by point, revised the manuscript, and added experiments.
Concern 1 (W1, Q1): The definition of the "geometry" is not very clear in this work.
Clarifying the definition of geometry. We apologize for any potential misunderstanding and have clarified the definition of geometry. Geometry refers to a single direction in feature space from normal to anomalous representations. It is a feature axis determined by the difference between normal and anomalous representations, characterizing the transition from a normal to an anomalous state, rather than defect shape or 3D geometry information. To avoid ambiguity, we have revised the introduction (the third paragraph of Sec. 1).
Clarifying the distinction between geometry and semantics. We formulate geometry and semantics as two competing hypotheses. The geometry-load hypothesis holds that text serves only to define a geometric direction, namely, a single direction from normal to anomalous representations in feature space. The semantic-load hypothesis holds that text works because it provides explicit defect semantics conveyed by words such as “damaged” and “flawed.” This formulation reduces the two competing hypotheses to a testable question, and our results support the geometry-load hypothesis.
Concern 2 (W2): Domain gap between CLIP and anomaly detection task.
Domain gap does not affect our conclusions. First, to minimize domain gap, our prompt pool includes official prompt templates from WinCLIP/ZUMA, which have been extensively validated in prior work and are well suited to CLIP. Second, we constructed a diverse prompt pool to avoid reliance on a single distribution. Our analysis covers approximately 5k prompts from 6 sources, including manual templates, learned contexts, and language-free vector interpolations, so that our conclusions are grounded in multiple prompt distributions rather than a single one. Finally, even if a domain gap exists, it does not affect our conclusions: rather than directly comparing the absolute performance of different prompts, we control for directional alignment across the same prompt pool and separately test whether explicit defect semantics provides any additional contribution.