acceptodds
Under review as a conference paper at ICLR 2027

What Do Text Prompts Actually Carry in Zero-Shot Anomaly Detection? Geometry, Not Semantics

Abstract

Zero-shot anomaly detection with CLIP is standardly credited to the defect semantics of text prompts. This assumption is central for two reasons. The text branch is the only part that ever mentions defects, and prompt design is the field's main axis of progress. We argue instead that the text branch contributes only geometry, not semantics: a single direction from normal to anomalous in feature space. Vision alone supplies a functionally equivalent one, to the same effect. (1) Reduction. The text branch of both representative pipelines provably reduces to a monotone function of a single projection onto one direction (rank correlation ), which turns the question of what text contributes into the question of what makes this direction work: geometry or semantics. (2) Substitution. A direction computed from a few normal images, swapped in for the text pair with no other change, matches the complete pipeline on all combinations of pipeline and benchmark. The direction proves not only sufficient but exhaustive: across 5k diverse prompts, directional alignment alone determines performance () and semantics add nothing (). (3) Prediction. For 10k entirely unseen prompts, we first forecast each prompt's performance, one forecast per prompt, from a single geometric quantity, without running the pipeline. We then run the pipeline on all 10k prompts and measure the actual performance of each. The per-prompt forecasts match the per-prompt measurements closely, ranking unseen prompts at correlation to () with no systematic bias (calibration slope within of unity). Across these analyses, text prompts in zero-shot anomaly detection carry geometry, not semantics. The geometry is a direction from normal to anomalous, and vision alone supplies an equivalent one, to the same effect. We call this the modal load-bearing of text.

Primary area:
interpretability and explainable AI
Keywords:
Zero-shot anomaly detection, CLIP, Text prompts, Modal load-bearing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1UqrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks an interesting problem to better analyze what leads to good performance of CLIP-based anomaly detection methods. Specifically, it analyze how the text branch affects the final performance by manipulating the text prompt input. Their finding is that the geometry cue matters more than semantics.

Critical strengths

I think this paper opens an interesting direction to better understand what matters and what works for image anomaly detection methods, specifically CLIP-based anomaly detection methods.

Build upon the previous paper's empirical finding that text branch in the CLIP-based anomaly detection is not a must to pursue strong performance for image anomaly detection, this work basically further makes more in-depth analyses to study the effect of semantics and geometry.

Critical weaknesses
  1. After careful reading of this work, I think the definition of the "geometry" is not very clear to me in this work. To the best of my understanding the "geometry" here is potentially related to the defects' shape or structure in the visual representation rather than the semantics contained in the text input. Thus, I would suggest better define the geometry here. Otherwise, it would lead to some confusions especially for people working in 3D anomaly detection domain or broader geometric/3D learning community.

  2. While I think this work somehow convinces me that the semantics is not a must for image anomaly detection in the studied setup. I have some concerns on the sensitivity of the result to the input prompt. For example, the CLIP model itself might be sensitive or not generalizable to certain text input considering the domain gap of texts it used for training and the targeted anomaly detection task.

Questions for authors
  1. Could you please clarify the definition of geometry in this work? I would recommend not to make it as an opposite term to semantics.

  2. How do you ensure that the design of 5k prompts is sufficient to span the input space of CLIP text input to disentangle the effect of semantics and geometry?

  3. Any further insights on the applicability of the results to zero-shot anomaly detection methods that are not CLIP based but VLM based? Would the result still hold? It would be great to discuss on that research line to consolidate the whole paper claims.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1Uqr (1/2)

We sincerely appreciate the reviewer's encouraging assessment and insightful comments. We have responded point by point, revised the manuscript, and added experiments.

Concern 1 (W1, Q1): The definition of the "geometry" is not very clear in this work.

Clarifying the definition of geometry. We apologize for any potential misunderstanding and have clarified the definition of geometry. Geometry refers to a single direction in feature space from normal to anomalous representations. It is a feature axis determined by the difference between normal and anomalous representations, characterizing the transition from a normal to an anomalous state, rather than defect shape or 3D geometry information. To avoid ambiguity, we have revised the introduction (the third paragraph of Sec. 1).

Clarifying the distinction between geometry and semantics. We formulate geometry and semantics as two competing hypotheses. The geometry-load hypothesis holds that text serves only to define a geometric direction, namely, a single direction from normal to anomalous representations in feature space. The semantic-load hypothesis holds that text works because it provides explicit defect semantics conveyed by words such as “damaged” and “flawed.” This formulation reduces the two competing hypotheses to a testable question, and our results support the geometry-load hypothesis.


Concern 2 (W2): Domain gap between CLIP and anomaly detection task.

Domain gap does not affect our conclusions. First, to minimize domain gap, our prompt pool includes official prompt templates from WinCLIP/ZUMA, which have been extensively validated in prior work and are well suited to CLIP. Second, we constructed a diverse prompt pool to avoid reliance on a single distribution. Our analysis covers approximately 5k prompts from 6 sources, including manual templates, learned contexts, and language-free vector interpolations, so that our conclusions are grounded in multiple prompt distributions rather than a single one. Finally, even if a domain gap exists, it does not affect our conclusions: rather than directly comparing the absolute performance of different prompts, we control for directional alignment across the same prompt pool and separately test whether explicit defect semantics provides any additional contribution.

AuthorsOpenReview ↗

Response to Reviewer 1Uqr (2/2)

Concern 3 (Q2): Are 5k prompts sufficient to span the input space of CLIP text input to disentangle the effect of semantics and geometry?

The coverage of 5k prompts is sufficient to support our conclusions. We do not claim to exhaust CLIP’s entire text space, which is infeasible theoretically and practically. However, our 5k prompts span manual templates, learned contexts, and randomly generated sources. Semantically, they range from explicit defect naming to empty strings and meaningless text. Geometrically, continuous interpolation varies the direction from the original text direction to its orthogonal direction. Therefore, although our 5k prompts cannot exhaust CLIP’s text space, they provide sufficient coverage to support our conclusions.

The 5k prompts are sufficient for semantic–geometric disentanglement. We disentangle semantics and geometry through conditional analysis rather than by increasing the number of prompts. Therefore, 5k prompts are enough for disentanglement. Specifically, we first fit performance using directional alignment and then add an explicit defect-semantics variable. If semantics contributed beyond direction, the model’s explanatory power should increase substantially. However, directional alignment alone explains 0.8736–0.9162 of the variance, whereas adding semantics yields a ΔR2\Delta R^2 of no more than 0.0012. Thus, once geometry is determined, the additional contribution of defect semantics to performance is negligible, thereby achieving semantic–geometric disentanglement.


Concern 4 (Q3): Applicability of the results to zero-shot anomaly detection methods that are not CLIP-based but VLM-based?

Our conclusions also generalize to other non-CLIP VLMs. We conducted the full set of additional experiments on two architecturally compatible non-CLIP VLMs, SigLIP and BLIP-ITC, and added the results to Tables 14 and 15 in the appendix of the manuscript. Results from both pipelines support our conclusions. Reduction:​ the median per-class ρ\rho exceeds 0.9983. Replacement:​ for BLIP-ITC on VisA, even a single image yields better performance. Even in the worst case, BLIP-ITC on MVTec AD, 16 images are sufficient to achieve comparable performance. Erasure:​ erasing the text plane reduces performance to near-random levels, whereas erasing its orthogonal complement has little effect. Exhaustion:​ directional alignment explains 0.8183–0.8965 of the performance variance, while explicit defect semantics contributes only 0.0004–0.0039 additionally. Prediction:​ on 10k unseen prompts, the Spearman correlation between predicted and measured performance reaches 0.7829–0.8874, and the calibration slope is within 0.0111 of unity. Therefore, our core conclusion that text prompts carry geometry rather than semantics also holds for other architecturally compatible non-CLIP VLMs. Note that we do not include generation- and diffusion-based VLMs, as these models generally do not natively provide normal–anomalous text similarity scoring based on a shared image–text embedding space and therefore fall outside the scope of our setting.

Reviewer_BwePOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the contribution of text prompts in ZSAD. The paper proposes and verifies an counterintuitive conclusion: in representative pipelines such as WinCRIP and AnomalyCRIP, the impact of text branches on detection scores can be reduced to a single directional projection in the feature space. Text prompts mainly provide geometric direction rather than defect semantics. The paper further proves that replacing text pairs with visual directions extracted from a small number of normal images can match the detection performance of the original pipeline.

Critical strengths
  1. The structure of the paper is clear, with three steps (Reduction, Substitution, Prediction) advancing layer by layer, complete charts and appendices, and sufficient reproducibility statements.
  2. The problem is important and the conclusion is impressive. The paper directly challenges the common narrative in ZSAD that text prompts provide defect semantics and proposes an alternative explanation of text only provides geometric direction.
Critical weaknesses
  1. The paper claims that textual prompt provides geometry, not semantics, in VLM. Then, can this conclusion be extended to other VLM-based methods for non-AD tasks? If possible, I am curious why this work only focuses on VLM-based AD methods. If not, why it is only suitable for ZSAD task? This requires more explanations. Besides, I want to know whether the textual prompt only provides geometry in VLM-based methods? What about other methods that use textual prompts, like diffusion-based methods.

  2. In Eq. (1), what is "L", which is not explained. Besides, Eq. (1) in fact belongs to batched zero-shot (transductive batch setting). However, in real-world deployment, the whole test batches are not accessible. It is recommended that the paper clearly distinguish real zero-shot, few-shot normal references, and transductive batch setting.

Questions for authors

See weakness.

6 comments
AuthorsOpenReview ↗

Response to Reviewer BweP

We sincerely thank the reviewer for describing our work as important and impressive, and for recognizing the clear structure and complete charts and appendices. In response to the reviewer's concerns, we provide our point-by-point replies as follows:

Concern 1 (W1): Can this conclusion be extended to other VLM-based methods for non-AD tasks?

First, our conclusion cannot be directly generalized to all VLM-based non-AD tasks, because it stems from a specific structure, namely binary contrastive scoring, rather than from a universal property of VLMs. In short, WinCLIP/AnomalyCLIP compute similarities between an image and a pair of "normal" and "anomalous" texts and perform two-class scoring, so the score depends only on the difference direction between the two texts. However, for other VLM-based non-AD tasks that do not adopt this binary contrastive structure, the conclusion does not transfer. We have not experimentally verified them, and therefore make no claim of universality.

Second, our focus on anomaly detection (AD) is justified, and the conclusion is robust across models. AD is a mature task that naturally adopts this binary contrastive scoring, where the text branch is the only component that mentions defects. Therefore, it is the most suitable carrier for testing what text actually carries. Meanwhile, replacing the model with architecturally compatible SigLIP/BLIP-ITC leaves the conclusion intact, with detailed results in Tables 14 and 15 of the appendix.


Concern 2 (W1): Whether text prompts only provide geometry in VLM-based methods, and what about other methods that use text prompts, like diffusion-based methods?

First, in VLMs with binary contrastive scoring, text indeed provides only geometry. This is jointly established by Reduction, Substitution, Prediction, and intervention experiments that fix the direction while changing semantics or fix the semantics while changing the direction, and this conclusion also holds for SigLIP and BLIP-ITC.

Second, the conclusion does not apply to generative VLMs without binary contrastive scoring. Taking diffusion-based methods as an example, diffusion models are generative: text does not provide normal/anomalous similarity scoring, nor is there a projectable binary "normal-vs-anomalous" difference direction. Therefore, whether text provides only geometry in diffusion is a separate question requiring its own study, and we make no claim about it.


Concern 3 (W2): On the meaning of LL in Eq. (1).

We thank the reviewer for pointing out this omission. LL is the lower-triangular Cholesky factor of the covariance matrix Σ\Sigma, i.e., it satisfies Σ=LL⊤\Sigma = L L^{\top}. It is obtained by the Cholesky decomposition of the Σ\Sigma estimated from kk normal images. We have added this definition in the revised manuscript.


Concern 4 (W2): On the transductive batch setting.

We indeed adopted a transductive setting, i.e., using the test set to construct the direction. To address this concern, we added experiments under a non-transductive setting, where the direction is constructed without using the test set. The detailed setup and results are supplemented in Appendix E.

Experimental setup. In the few-shot setting, kk normal training images are used to estimate the normal mean and a PCA normal subspace (retaining 9999% of the variance). The direction is obtained by averaging the top-11% residuals of the current test image's features after projection onto the orthogonal complement of the normal subspace. Note that the current test image itself is the required input for prediction and no information from other test images is used. Therefore, using the current test image does not violate the non-transductive protocol. The zero-shot setting uses no training images.

Results under the non-transductive setting. All core conclusions still hold. Substitution holds: the few-shot (k=16k=16) visual direction matches the full text pipeline on all four pipeline-dataset combinations. The 5k exhaustion analysis holds: directional alignment alone explains 0.93070.9307–0.94440.9444 of the variance, with a semantic increment ΔR2≤0.0003\Delta R^2 \le 0.0003. The 10k performance prediction holds: a single geometric scalar attains Spearman's ρ=0.8713\rho = 0.8713–0.90460.9046 on the 10k unseen prompts.

In summary, the conclusions of the paper remain valid even under the non-transductive setting.

Thank you for your response. However, I am still concerned about the "geometry". Why the geometry only exists in binary contrastive scoring, but not others. This may affect the final rating. The authors should provide more experiments, visualizations, theoretical analyses to demonstrate this.

AuthorsOpenReview ↗

Response to Reviewer BweP (1/2)

We sincerely thank the reviewer for the timely and patient response. To address your concerns, we conducted a theoretical analysis and performed additional experiments. We organize our reply along three lines: Theory→Why the geometry only exists in binary contrastive scoring→Why the geometry does not exist in others.


1. Theory: the geometric reduction of binary contrastive scoring originates from the translation invariance of softmax

The geometric reduction of binary contrastive scoring originates from the translation invariance of the softmax, i.e.,

softmax([a,b])=softmax([a−c, b−c])\mathrm{softmax}([a,b])=\mathrm{softmax}([a-c,\,b-c])

Concretely:

S(f)&=\mathrm{softmax}\!\left(\frac{\hat{f}\cdot\hat{t}_n}{\tau},\ \frac{\hat{f}\cdot\hat{t}_a}{\tau}\right)\\ &=\frac{e^{\hat{f}\cdot\hat{t}_a/\tau}}{e^{\hat{f}\cdot\hat{t}_n/\tau}+e^{\hat{f}\cdot\hat{t}_a/\tau}}\\ &=\frac{e^{(\hat{f}\cdot\hat{t}_a-\hat{f}\cdot\hat{t}_n)/\tau}}{1+e^{(\hat{f}\cdot\hat{t}_a-\hat{f}\cdot\hat{t}_n)/\tau}} \qquad\text{dividing by } e^{\hat{f}\cdot\hat{t}_n/\tau}\\ &=\sigma\!\left(\frac{\hat{f}\cdot\hat{t}_a-\hat{f}\cdot\hat{t}_n}{\tau}\right),\qquad \sigma(u)=\frac{1}{1+e^{-u}}\\ &=\sigma\!\left(\frac{\hat{f}\cdot(\hat{t}_a-\hat{t}_n)}{\tau}\right)\qquad\text{linearity of the dot product}\\ &=\sigma\!\left(\frac{|\hat{t}_a-\hat{t}_n|}{\tau}\,\hat{f}\cdot d\right),\qquad d=\frac{\hat{t}_a-\hat{t}_n}{|\hat{t}_a-\hat{t}_n|} \end{aligned}$$

Dividing by ef^⋅t^n/τe^{\hat{f}\cdot\hat{t}_n/\tau} is precisely the translation invariance: it cancels the common component of the two texts, leaving only the difference direction dd, so that the score becomes a strictly monotonic function of f^⋅d\hat{f}\cdot d. As rank-based metrics are invariant to monotonic transformations, the entire effect of the text reduces to dd, and semantics acts only indirectly through the direction it induces. This reduction arises from two structural conditions, namely binarity (the softmax compares only a pair of texts, so that translation invariance cancels the common component and leaves a single difference direction dd) and linearity (the logit is a linear projection f^⋅t^\hat{f}\cdot\hat{t}, so that the score can be written as a monotonic function of f^⋅d\hat{f}\cdot d), rather than from any intrinsic property of the text. Whether these two conditions hold determines whether the geometric reduction exists, which is precisely the root of the reviewer's concern. Sections 2 and 3 verify each in turn.


2. Why the geometry only exists in binary contrastive scoring

The reduction of Section 1 relies on the following two conditions:

Binarity: the softmax compares only a pair of texts, so that translation invariance cancels the common component and leaves a single difference direction dd.

Linearity: the logit is a linear projection f^⋅t^\hat{f}\cdot\hat{t}, so that the score can be written as a monotonic function of f^⋅d\hat{f}\cdot d.

Binary contrastive scoring satisfies both binarity and linearity simultaneously. Consequently, its ρ\rho is approximately 1 and its semantic ΔR2\Delta R^2 is approximately 0.

Pipeline·Datasetρ\rho (closer to 1 is better)ΔR2\Delta R^2 (closer to 0 is better)
WinCLIP·MVTec0.99860.0009
WinCLIP·VisA0.99920.0001
AnomalyCLIP·MVTec1.00000.0008
AnomalyCLIP·VisA1.00000.0012
Reviewer_dkorOpenReview ↗
Rating
1
Confidence
4
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates the information provided by text prompts in CLIP-based zero-shot anomaly detection. It formulates binary cosine-similarity scoring as the projection of visual features onto the difference direction between normal and anomalous text prototypes. Through visual direction replacement, prompt interventions, and large-scale prompt analysis, the paper examines the relationship between directional geometry and detection performance. Experiments cover WinCLIP, AnomalyCLIP, and multiple industrial and medical datasets, using approximately 5k prompts for interpretability analysis.

Critical strengths
  1. The paper raises an important and falsifiable research question and combines algebraic simplification, component replacement, erasure interventions, and large-scale prompt-based performance prediction, demonstrating a systematic experimental design.

  2. The analysis covers WinCLIP/AnomalyCLIP, multiple industrial and medical benchmarks, and various prompt sources, providing a large-scale evaluation with extensive experimental results.

Critical weaknesses
  1. The fact that binary cosine softmax depends only on the difference between two text embeddings is an algebraic property, but semantic information can itself be encoded in this direction. The fact that another visual direction achieves similar AUROC only demonstrates that the decision direction is not unique, rather than proving that the original text direction contains no semantic information. The paper lacks identifiable interventions that change semantics while preserving the direction, or change the direction while preserving semantics. Therefore, the exclusive conclusion of “geometry, not semantics” is not logically justified.

  2. Both the few-shot and zero-shot visual directions use the entire unlabeled test set and select the top 1% of test features with the highest Mahalanobis norms. This procedure exploits the evaluation distribution and its anomalous samples in a transductive manner, making it fundamentally different from text prompts that do not require access to the test set. Moreover, reducing semantic variables to a binary indicator of whether defect-related words appear cannot adequately capture linguistic semantics.

Questions for authors
  1. If visual directions are constructed using only normal training images, with access to the evaluation test batch completely prohibited, do the conclusions from direction replacement, the 5k-prompt interpretability analysis, and the 10k-prompt performance prediction still hold?

  2. Could you introduce matched interventions that change semantic content while keeping the direction/embedding geometry approximately fixed, and rotate the direction while keeping semantic content fixed, and accordingly revise the conclusion that semantics makes no contribution?

4 comments
AuthorsOpenReview ↗

Response to Reviewer dkor

We sincerely thank the reviewer for recognizing the importance of our research, the systematic experimental design, and the large-scale evaluation. In response to the concerns raised, we address them point by point below.

Concern 1 (W1, W2, Q2): "geometry, not semantics" is not logically justified.

First, we would like to clarify that this claim does not deny that texts contain semantics. We aim to emphasize that although texts contain semantics, the decisive contribution of texts to performance comes from the geometry they induce, rather than from the semantics. Second, in response to this concern, we conducted two controlled intervention experiments and one continuous-semantics analysis, all of which show that geometry, not semantics, determines performance. Detailed settings and results are in Appendices F and G.

Experiment 1: fixed direction, changed semantics. We evaluated text pairs with nearly identical directions (cos⁡≥0.99\cos\ge0.99) but opposite semantics, fed them into CLIP to obtain an I-AUROC for each, and took their difference (Δ\Delta = defect −- non-defect). With the direction nearly fixed, reversing semantics changes I-AUROC by only 0.0049 on average (max ∣Δ∣=0.0095|\Delta|=0.0095). Therefore, as long as the direction is fixed, semantic changes have almost no effect on performance.

Experiment 2: fixed semantics, changed direction. We evaluated text pairs with nearly identical semantics but substantially different directions: a mean direction cosine of only 0.3451, yet an average I-AUROC difference of 0.4984. With semantics fixed and only the direction varying, performance changes substantially, showing that the direction, not semantics, determines performance. We also emphasize that our claim is that what texts carry and can determine performance is geometry, not semantics, and we never claim the direction is unique, so non-uniqueness does not conflict with our conclusion.

Experiment 3: replacing the binary indicator with continuous semantic representations. We design three configurations to replace the original binary defect-word indicator with continuous semantic representations. Across the four pipeline-dataset combinations, their semantic increments ΔR2\Delta R^2 average only 0.0042/0.0046/0.0070. Richer semantic representations still bring no substantial additional explanatory capacity, further confirming that the semantic contribution is minimal.

In summary, fixing the direction while changing semantics barely changes performance, fixing semantics while changing direction substantially changes performance, and continuous semantic representations still yield a negligible increment. Together, these confirm that geometry, not semantics, determines performance.


Concern 2 (W2, Q1): the "transductive" setting, i.e., that our direction construction relies on the unlabeled test set.

In response to this concern, we added non-transductive experiments, i.e., we re-ran all core experiments without using the test set to construct the direction. Detailed settings and results are in Appendix E.

Experimental setup. We adopt a non-transductive setting that does not use the test set to construct the direction. In the few-shot setting, we use k normal training images to estimate the normal mean and a PCA normal subspace (retaining 99% of the variance). The direction is the average of the top-1% residuals of the current test image's features, after projection onto the orthogonal complement of the normal subspace. (We emphasize that the current test image itself is the required input for prediction, and no information from other test images is used. Therefore, using the current test image does not violate the non-transductive protocol.) In the zero-shot setting, no training images are used. All other settings follow the main text.

Results. All findings still hold. Reduction holds, as it does not involve visual-direction construction. Replacement holds: the few-shot (k=16k=16) visual direction matches the full text pipeline on all four pipeline-dataset combinations, and some pipelines still perform well with fewer few-shot images. Erasure holds: the visual direction is not a shadow of the text direction. The 5k exhaustion analysis holds: directional alignment alone explains 0.9307-0.9444 of the variance, with ΔR2≤0.0003\Delta R^2\le0.0003. The 10k prediction holds: a single geometric scalar yields a Spearman ρ\rho of 0.8713-0.9046 on the 10k unseen prompts.

In summary, even under the non-transductive setting, all our core conclusions still hold. Several metrics exceed those obtained under the transductive setting in the main text, driven by the residual construction on the normal subspace, which measures the deviation from the normal more directly and does not rely on the test-batch distribution, so the directional alignment becomes more accurate.

Thank you for the response and the additional experiments. Appendix E resolves my concern about batch-level transduction, since the revised construction no longer uses other test images. Appendices F and G also make the empirical analysis more complete.

I understand the authors’ clarification that texts may contain semantics, while their contribution to performance is claimed to arise from the geometry they induce. My concern is that the current experiments do not separate these two explanations. Semantic content may affect performance precisely by inducing a particular embedding direction. Experiment 1 fixes this direction and therefore measures only whether semantic differences provide additional effects after the direction has been controlled. Experiment 2 confirms that performance changes with direction among paraphrases. Together, these experiments establish direction as the immediate variable used by the scoring function, but they do not establish that the semantic content responsible for inducing that direction has no decisive contribution. A conclusion supported by the experiments would be that semantic differences provide little residual explanatory power once the CLIP direction is fixed.

A protocol issue also remains. The revised dc,id_{c,i} is computed from the top residual patches of the image being evaluated, making it a query-adaptive direction rather than a fixed visual direction derived from normal training images. The paper also does not clearly explain how these image-specific directions produce a single prompt-level alignment value in the 5k and 10k prompt analyses.

The response resolves my concern about batch-level transduction, but the interpretation underlying the paper’s main claim remains insufficiently supported. I therefore maintain my original score.

AuthorsOpenReview ↗

Response to Reviewer dkor

We sincerely thank the reviewer for recognizing our response to Concern 2 ("transductive setting", W2, Q1), and for the timely reply and patient clarification. We address the comments point by point below.

Response 1: Detailed clarification on "geometry, not semantics"

First, we fully agree with the reviewer's reasoning:

A1. Semantics can induce geometry, and through the induced geometry influence performance, that is, semantics can itself influence performance through the induced geometry.

A2. The fact that visual features can supply a functionally equivalent geometry only shows that the effective geometry is not unique, and does not rule out that text semantics influence performance indirectly through the induced geometry.

Second, we would like to clarify that our core view is consistent with the reviewer's. However, the phrasing "geometry, not semantics" in our paper can easily be misread as saying that geometry determines performance and semantics contributes nothing. The intended meaning is as follows:

B1. We aim to state that performance (I-AUROC) is determined directly by geometry, rather than directly by semantics. We do not deny that semantics can influence performance indirectly, since we acknowledge that semantics can induce the geometry and influence performance through the induced geometry. We merely claim that semantics cannot directly determine performance. What directly determines performance is the geometry induced by semantics. Once this geometry is fixed, the measured residual semantic effect is small. (Consistent with A1).

Evidence: the algebraic derivation of reduction and the single-direction scoring reduction verified in Table 1. The controlled interventions in Appendix F. The exhaustion and prediction experiments in Section 3.4, Section 4 and Appendix E. The continuous semantic representation analysis in Appendix G.

B2. Moreover, this geometry is not only inducible by semantics. Vision can also supply a functionally equivalent geometry. And we have never claimed the uniqueness of the geometry (Consistent with A2).

Evidence: the existence and replacement experiments in Sections 3.1–3.2, the direction-alignment and erasure experiments in Section 3.3, and the non-transductive results in Appendix E.

Third, it follows that the reviewer's points A1 and A2 are consistent with our points B1 and B2. However, the phrasing "geometry, not semantics" in the paper can easily be misread as saying that geometry determines performance and semantics contributes nothing. We acknowledge this phrasing error and sincerely apologize for it. To avoid any potential misunderstanding, we have provided a detailed elaboration and clarification in the Introduction, and correspondingly revised the relevant statements in the Abstract, the Contributions, Sections 3 and 4, the Conclusion, and Appendix F.


A remark on the scope of our claim. Our experiments do not attempt to quantify the contribution of semantics through the induced geometry, nor can they prove that this contribution is non-decisive. Doing so would require a large amount of very detailed, specialized proof. The aim of this paper is to identify the direct carrier by which text prompts influence anomaly-detection scores, rather than to estimate the upstream causal contribution of semantics in forming the effective geometry.


Response 2: On the query-adaptive direction and the prompt-level alignment value

First, we agree that dc,id_{c,i} is a query-adaptive direction, rather than a fixed class-level visual direction. It is constructed from the normal reference distribution estimated from normal training images and the residual of the current image, and it does not use other test images. We would like to clarify that this setting takes only the current image as input and involves no other test images, and therefore does not violate the non-transductive protocol. Moreover, normal images contain no anomalies, so reading the anomaly direction from the residual of the current image itself is a natural choice.

Second, for a prompt pp and a class cc, we first compute the mean of the per-image alignment cosines xˉp,c=1Nc∑i=1Nccos⁡(dp,dc,i)\bar{x}_{p,c}=\frac{1}{N_c}\sum_{i=1}^{N_c}\cos(d_p,d_{c,i}), and then take the macro average across classes xˉp=1C∑c=1Cxˉp,c\bar{x}_{p}=\frac{1}{C}\sum_{c=1}^{C}\bar{x}_{p,c}. This xˉp\bar{x}_p is the single alignment value for each prompt in the exhaustion and prediction analyses. What is averaged here is the per-image cosine scalar, not the image-level direction. The cross-image aggregation is used only for the offline prompt-level analysis and is not fed back into the image-scoring process, so it does not change the non-transductive setting of the per-image scoring.

Thank you for the detailed clarification and the revisions to the manuscript. The new discussion resolves my concern about batch-level transduction and makes clear that semantics may influence performance indirectly by inducing the embedding direction.

This clarification leaves a substantially narrower central claim. The text-embedding direction is the computational variable directly used by the cosine-based scoring function. This result largely follows from the algebraic form of the scoring rule. The additional experiments show that directional alignment predicts prompt performance and that a query-adaptive visual direction can sometimes achieve comparable results. The broader distinction implied by the title remains unsupported.

The manuscript also continues to use stronger language in several places. The phrase “only geometry, not semantics” appears in the Abstract (p. 1), the contribution summary (p. 2), and the Conclusion (p. 10). “Semantics add nothing” appears in the Abstract (p. 1), the title and discussion of Section 3.4 (pp. 6–7), and the Conclusion (p. 10). The claim that the text branch contributes “a single direction ... and nothing else” appears in Section 2 (p. 3) and the concluding paragraph of Section 3.4 (p. 7). These statements go beyond the qualified interpretation given in the response.

The “semantic representations” in Appendix G are derived from CLIP embeddings and cannot independently quantify the contribution of linguistic semantics. The non-transductive replacement uses a query-adaptive direction, while the 5k- and 10k-prompt analyses aggregate information over the target evaluation images. The replacement result therefore concerns an image-specific visual procedure, and the prompt prediction is an offline, dataset-specific analysis.

The revisions clarify the authors’ intended interpretation, but the broader framing remains insufficiently supported. The narrower conclusion supported by the experiments largely follows from an algebraic property and does not constitute a sufficiently substantial conceptual contribution. I therefore maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.