acceptodds
Under review as a conference paper at ICLR 2027

Do Words Matter? What Text Contributes to Few and Full-Shot Vision–Language Anomaly Detection

Abstract

Vision–language anomaly detectors describe normal and abnormal appearance with text, and their gains are commonly credited to the semantic content of these descriptions. This attribution is plausible in the zero-shot setting, but few and full-shot detectors interpose a trained visual adapter, and in that setting it has not been isolated. Changing a description changes both its meaning and its numerical geometry, so accuracy comparisons cannot separate the two. We ask what a fixed pair of text embeddings, or anchors, contributes to such a detector: semantic content, effective scale, or joint anchor geometry. For a single anchor pair, the answer is effective scale alone. The score reads the anchors only through their difference, and a trained adapter can rotate to follow any replacement direction, leaving that direction, and with it the linguistic content, unidentifiable. We prove that replacement anchors with the same difference-vector length permit exactly the same probability maps and attainable losses. Across seven medical datasets, three vision backbones, both supervision regimes, and three adapter families, scale-matched random and orthogonal anchors track correct prompts; breaking the scale match opens a clear gap that restoring the scale closes without text. The same pattern holds empirically in MVFA's residual, multi-level detector and when MadCLIP's learned prompts are replaced by a text-free learned scale. When one adapter serves many categories, the answer expands to their joint arrangement. Rotating all category axes together does not change anything, as predicted. For the tested CLIP prompt banks, these axes are nearly collinear, and meaning-free rotations that spread them apart improve performance on both industrial benchmarks. Even under frozen zero-shot transfer, a random anchor frame transfers as well as the prompt frame: coordinate consistency, not meaning. What these detectors read from their prompts is scale and arrangement, not semantic content.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
anomaly detection, vision-language models, identifiability, prompt engineering, medical image analysis, industrial anomaly detection, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_MwGjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies what fixed text anchors contribute to few- and full-shot VLM anomaly detectors with trainable adapters. It theoretically demonstrates that binary predictions depend only on the anchor difference vector. If the adapter is retrained and rotation-closed, the direction of this vector can be injected by an adapter rotation. For a shared multi-category adapter, the invariant extends to the Gram matrix of category-based difference vectors. Extensive experiments show a consistent pattern: once the anchor scale is kept the same, random or orthogonal anchors work about as well as meaningful text prompts. Performance drops when the scale changes, but recovers when the correct scale is restored.

Critical strengths
  1. The paper asks a simple but important question: are these models actually using the meaning of the prompts, or just their geometry? The controls are carefully designed to separate the two.
  2. The main theoretical result is clean and easy to follow. It clearly explains why a retrained adapter can absorb changes in anchor direction, while changes in effective scale still matter.
  3. The experiments are broad and consistent. The same pattern appears across datasets, backbones, training settings, and adapter architectures, which makes the main finding more convincing.
Critical weaknesses
  1. I could not find enough detail about how the medical datasets are split. In particular, it is important to confirm that images or slices from the same patient do not appear in both the support and test sets.
  2. Although the paper discusses the limitations of its theory, the title and overall message can still sound broader than the evidence supports. The strongest conclusions apply to fixed binary anchors with a retrained, rotation-closed adapter, rather than vision-language anomaly detection in general.
  3. In the multi-category experiments, the geometry seed is tied to the training seed. It would be cleaner to vary these separately so that the gain can be attributed more confidently to the anchor geometry.
Questions for authors
  1. Could you run the multi-category experiments with geometry seeds and training seeds varied independently? This would help separate the effect of anchor geometry from ordinary training randomness.
  2. Could you clarify exactly how the medical datasets were split? In particular, are the support and test sets fully separated at the patient level?
4 comments
AuthorsOpenReview ↗

Response to Reviewer MwGj (1/2)

Thank you for the review. We answer each weakness and each question separately, in your order.

Weakness 1: dataset splits and patient-level separation (also Question 2)

"it is important to confirm that images or slices from the same patient do not appear in both the support and test sets"

In short. We did not make our own splits. We use the standard public benchmark splits exactly as released: BMAD (Benchmarks for Medical Anomaly Detection, Bao et al., CVPR Workshops 2024) for four datasets and the MediCLIP release for the other three. These are the splits on which MVFA, MadCLIP and MediCLIP report their results, so our numbers are directly comparable with theirs. Your question led us to audit these benchmark splits, and the answer is mixed: four of the seven are not fully separated by patient. We now document the splits and the audit in the paper.

How the splits are made. BMAD provides the splits for BraTS2021, LiverCT, RESC and OCT2017, and the MediCLIP release those for BUSI, CheXpert and Kaggle BrainMRI. We change nothing in them. In every run, the training seed draws the K support images at random from the benchmark's support pool, as MVFA does; the test split is used as released.

Audit (Table 21, worst case over all support sets used in our paper):

DatasetSplit bySupport images from patients also testedTest images of those patients
BraTS2021patient, normal images only16 of 3245 of 3,715
LiverCTslice32 of 321,493 of 1,493
RESCslice32 of 321,684 of 1,805
OCT2017image5 of 3213 of 968

BUSI, CheXpert and Kaggle BrainMRI carry no patient identifiers. Comparing images directly, we found two exact copies of test images in Kaggle BrainMRI, four near-identical images in BUSI, and none in CheXpert.

What it means for our claim. Every anchor condition uses the same support sets and test images, so the overlap is identical on both sides of every paired comparison and cannot by itself create the equivalence. It could hide a difference only by pushing both sides to the ceiling. It does not on the most affected dataset: LiverCT, split by slice, is the least saturated real-anomaly dataset, and there the scale-only positive control loses 3.9 points while the exact controls stay within ±0.5 (Table 33).

What it means for absolute numbers. Scores on BraTS2021, LiverCT and RESC are optimistic, for our detector and for every published method evaluated on these splits. We did not re-evaluate on patient-disjoint splits; we kept the published ones so that the comparison with published methods stays like for like.

In the revision. §6.1 (one sentence); §7 Limitations; Appendix B.2 (procedure, audit, Table 21); captions of Tables 44–45.

Weakness 2: the title and message sound broader than the evidence

"The strongest conclusions apply to fixed binary anchors with a retrained, rotation-closed adapter"

In short. We agree: the guaranteed conclusions apply to fixed binary anchors with a retrained, rotation-closed adapter, and the message can be read more broadly than that. Beyond that setting we present our results as empirical evidence only.

What the theory covers. Our research question is posed for trained fixed-anchor detectors, and the title is restricted to few- and full-shot detection, where an adapter is trained on support images (not zero-shot). Proposition 1 states where the guarantee stops: adapters that cannot absorb the rotation, frozen adapters, and multi-category replacements that change the joint geometry. The Limitations repeat these requirements.

Inside the theorem, we now test two published detectors, AnomalyGPT and APRIL-GAN, with the same result (see Reviewer sX9b).

Beyond the theorem, the settings we tested show the same pattern. Three lie outside its premise:

  • MVFA, whose residual, multi-level adapters are not rotation-closed: unmatched orthogonal anchors drop to 73.7 image AUROC, and restoring only the scale brings it back to 94.5, against 94.8 for its prompts (§6.3, Table 3).
  • MadCLIP, which learns its prompts: a text-free learned scale gives 96.3, against 96.2 for its learned prompts (§6.3, Table 3).
  • New in the revision: anchors optimized jointly with our adapter, starting from the prompts or from random vectors, do not beat fixed correct prompts, and their effect is the scale they select (Appendix F.1).

Where the claim stops, we say so. With a frozen adapter, as in zero-shot transfer, the anchors it was trained with need to be kept (§6.5).

Changed in the revision.

  • We changed the abstract's final sentence to name trained fixed-anchor detectors instead of "these detectors".
  • Every equivalence statement (abstract, contributions, §6.2, Conclusion) now gives its margin (±0.5).

The title. If the AC and reviewers prefer a narrower title than one restricted to few- and full-shot detection, we will adapt it for the camera-ready version.

AuthorsOpenReview ↗

Response to Reviewer MwGj (2/2)

Weakness 3: the geometry seed is tied to the training seed (also Question 1)

"It would be cleaner to vary these separately so that the gain can be attributed more confidently to the anchor geometry"

In short. We varied them separately. The gain holds for every draw of rotations, and the draw matters far less than the gain. Your question also exposed an issue we now disclose: in the original runs, category t used seed s + t, so consecutive geometry seeds shared all but one per-category rotation.

What we ran. Three new geometry seeds whose draws share no per-category rotation, crossed with the same three training seeds (3 × 3 grid). The training seed fixes the support set, initialization and data order; the geometry seed fixes only the rotations. Both banks (shared and independent rotations), MVTec-AD and VisA, all three backbones. A shared-rotation run and an independent-rotation run with the same two seeds form a pair, giving 108 paired comparisons.

What we found (Appendix G.1, Table 40):

  • Independent rotations beat the shared rotation in all 108 paired comparisons. Mean gains range from +1.14 to +8.94; the smallest single gain is +1.01, and on DINOv3 VisA all nine pairs gain +8.68 to +9.09 image AUROC.
  • The geometry seed moves the means by at most 0.27.
  • Its effect is statistically detectable (p < 0.05) only for DINOv2 on VisA, and neither of those two tests survives a correction for the twelve tests.
  • The separated means differ from the originally reported ones by at most 0.16.

What this shows. The gain comes from the geometry, not from a lucky draw of rotations: every draw wins, and changing the draw moves the results by less than a tenth of the gain in every setting. The overlap of the original draws did not affect the numbers reported in our paper.

In the revision. §6.4 (one sentence); Appendix A.4 (disclosure); Appendix G.1 (design, Table 40).

Question 1: Could you run the multi-category experiments with geometry seeds and training seeds varied independently? This would help separate the effect of anchor geometry from ordinary training randomness.

Yes, we ran these experiments, and we describe them in our answer to Weakness 3 above. In brief, independent rotations win all 108 paired comparisons, and the geometry seed moves the means by at most 0.27 (Appendix G.1).

Question 2: Could you clarify exactly how the medical datasets were split? In particular, are the support and test sets fully separated at the patient level?

We describe the splits and our audit in our answer to Weakness 1 above. In brief, we use the standard BMAD and MediCLIP benchmark splits unchanged, and the support images for each training seed are drawn from the benchmark's support pool. The splits are not separated by patient for BraTS2021 (abnormal images), LiverCT, RESC and OCT2017, and BUSI, CheXpert and Kaggle BrainMRI carry no patient identifiers. Because the overlap is the same for every anchor condition, it cannot create the equivalence; it does make absolute scores optimistic for all methods, ours and the published ones (Appendix B.2, Table 21).

AuthorsOpenReview ↗

Follow-up on our responses

Thank you again for your review. As the discussion period ends soon, we would be grateful to know whether our responses address your concerns. If any point remains open, we would be glad to clarify it.

Thanks for the detailed explanation. Most of my concerns have been addressed, and I believe that even though the scope of this paper is limited compared to what the authors claimed, from the theoretical perspective, this is interesting investigation. Therefore, I will keep my current rating.

Reviewer_ckm8OpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether text semantics or embedding geometry drives performance in vision–language anomaly detection with trainable visual adapters. This paper theoretically shows that, for a single fixed anchor pair, matching the effective scale is sufficient to preserve representational capacity under rotation-closed adapters. It extend the analysis to multi-category settings, where joint anchor geometry matters, and examine the importance of anchor–adapter consistency in frozen zero-shot transfer. The findings are validated through experiments across seven medical datasets, multiple vision backbones and adapter architectures, and existing detectors (MVFA and MadCLIP). Additional experiments on MVTec-AD and VisA demonstrate that modifying anchor geometry can improve detection performance without relying on text semantics.

Critical strengths
  1. The paper provides a mathematical characterization of the role of text anchors in vision–language anomaly detection. By establishing equivalence under matched effective scales and extending the analysis to multi-category geometry, it offers a principled explanation of why text-free anchors can be effective. This theoretical insight strengthens the understanding of existing anomaly detection methods.

  2. The paper supports its theoretical findings through experiments across multiple datasets, vision backbones, and adapter architectures. The controlled comparisons of anchor semantics and geometry, along with evaluations on existing detectors, provide convincing empirical support for the main findings and demonstrate their practical relevance.

Critical weaknesses

While the paper provides useful theoretical insights into existing anomaly detection methods, it offers limited guidance on how to systematically leverage these findings to improve detection performance. Although independent anchor rotations improve multi-category performance, the paper does not sufficiently investigate which geometric configurations are most effective or explain the underlying mechanism. Similarly, despite identifying effective scale as a key factor, the fixed-anchor experiments rely on prompt-derived scales without systematically exploring optimal scale selection or text-free alternatives. These limitations weaken the practical significance of the theoretical contributions, as the analysis does not yet provide generalizable principles for designing more effective anomaly detectors.

Questions for authors
  1. How does multi-category performance vary across different anchor geometries? For example, would intermediate effective ranks or approximately orthogonal anchors yield similar improvements?

  2. Could the authors evaluate a text-free scale selection strategy in the fixed-anchor setting? How would its performance compare with prompt-derived scales?

  3. Is there empirical evidence that reduced cross-category interference explains the gains from independent anchor rotations?

5 comments
AuthorsOpenReview ↗

Response to Reviewer ckm8 (1/2)

Thank you for the review. Your weakness paragraph raises three points, which we answer separately, followed by your three questions. Results are written as mean [90% confidence interval] of the paired difference, correct prompts minus control, in image AUROC points; for example, +0.04 [−0.03, +0.11]. ✓ marks equivalence (the whole interval lies within ±0.5) and ✗ its absence.

Weakness, part 1: limited guidance on how to use the findings

"it offers limited guidance on how to systematically leverage these findings to improve detection performance"

In short. We agree that our submission stated the guidance without enough evidence. We have now added that evidence, and the guidance for this detector family is:

  • Set one number, not a prompt bank. For a single anchor pair, only the effective scale matters. A fixed text-free choice works: an orthogonal pair at similarity scale τ = 20 is equivalent to correct prompts on CLIP and DINOv3 (Question 2). A wrong scale is what hurts: the protocol default τ = 100 costs 1.87 points on DINOv3 and up to 11.8 in a single cell.
  • With many categories, spread their axes. Independent rotations improve VisA by 5.6 image and 4.4 pixel AUROC on CLIP and up to 8.8 and 8.3 on other backbones, by changing only the anchor bank; the further the axes are spread, the better (Question 1).
  • With a frozen adapter, keep its anchors (§6.5, unchanged).

In the revision. The new evidence is in §6.2, §6.4 and Appendices F–G. The guidance itself is stated in the Conclusion ("Practical implications"), whose wording is unchanged.

Weakness, part 2: which geometries work best, and why

We answer this point together with Questions 1 and 3 below.

Weakness, part 3: no scale selection or text-free alternative

We answer this point together with Question 2 below.

Question 1: How does multi-category performance vary across different anchor geometries? For example, would intermediate effective ranks or approximately orthogonal anchors yield similar improvements?

In short. We ran exactly this sweep. Performance rises monotonically with the spread of the axes, and near-orthogonal anchors (independent rotations) are the best geometry we found.

What we ran. We rotated each category's anchor frame part of the way from the shared rotation (the prompts' geometry, effective rank 1.8) toward independent rotations, so that the effective rank of the difference vectors lands 25, 50 and 75% of the way, while every category's anchor cosine, and so its scale, stays fixed. MVTec-AD and VisA, CLIP and DINOv3, three seeds.

What we found (Table 4; Appendix G.3, Table 42), image / pixel AUROC along the path from shared rotation to independent rotations:

Shared25%50%75%Independent
VisA, CLIP86.3 / 91.689.2 / 93.890.1 / 95.090.8 / 95.691.8 / 96.1
VisA, DINOv383.2 / 88.487.0 / 91.488.8 / 94.490.1 / 95.691.9 / 96.8
MVTec-AD, CLIP96.2 / 93.697.0 / 95.497.1 / 95.997.2 / 96.397.4 / 96.6
MVTec-AD, DINOv395.0 / 92.896.0 / 94.796.5 / 95.296.8 / 95.797.3 / 96.3

Both metrics rise at every step, and every step is larger than the seed standard deviation. The curve is concave: the first quarter of the path gives roughly 35–65% of the total gain.

What this shows. The more spread out the scoring axes, the better, with no interior optimum on this path; the decisive step is breaking the near-collinearity of the prompt axes.

AuthorsOpenReview ↗

Response to Reviewer ckm8 (2/2)

Question 2: Could the authors evaluate a text-free scale selection strategy in the fixed-anchor setting? How would its performance compare with prompt-derived scales?

In short. Yes. One can take two orthogonal vectors, so that no text encoder is used, and set the similarity scale τ (the logit scale that multiplies the cosine); a fixed round value performs as well as the prompts' scale. Choosing τ without any labels is the part we could not do with K shots only (limits below).

Why one number suffices. A patch's binary logit is τ⟨u, a_A − a_N⟩, and a rotation-closed adapter absorbs the direction of a_A − a_N (Theorem 1). The anchors therefore contribute only τ‖a_A − a_N‖ = τ√(2 − 2ρ), where ρ is their cosine. Correct prompts are nearly collinear (ρ = 0.92–0.98), so at the protocol similarity scale τ = 100 they act exactly like an orthogonal pair at τ_prompt = 100√(1 − ρ), which is 15–28 for our seven prompt banks.

What we ran. One orthogonal pair, τ ∈ {5, 10, 15, 20, 25, 40, 100}, CLIP and DINOv3, all seven datasets, every K, three seeds. Compared with correct prompts and with the orthogonal pair at τ_prompt (computed from the prompts).

What we found (Appendix F.2, Table 39, pooled over all 28 dataset × K cells of each backbone; mean [90% CI], as in our first response):

Similarity scaleCLIPDINOv3
τ = 15+0.10 [−0.05, +0.25] ✓+0.21 [−0.02, +0.43] ✓
τ = 20+0.12 [−0.05, +0.28] ✓+0.01 [−0.15, +0.17] ✓
τ = 25+0.30 [+0.11, +0.49] ✓−0.06 [−0.28, +0.16] ✓
τ = 100 (default)+0.77 [+0.23, +1.31] ✗+1.87 [+1.12, +2.63] ✗
τ_prompt (needs text)+0.00 [−0.09, +0.10] ✓+0.07 [−0.04, +0.18] ✓

Comparison with prompt-derived scales. Figure 2 plots AUROC against τ for each dataset and marks correct prompts at τ_prompt: the prompts sit on the text-free curve (within 0.6 in every dataset), in the flat part of it. Prompts work because their geometry places the scale in this broad range, and a text-free sweep finds the range directly.

Limits. With only K support images there is no validation set, so the curves are read on the test set, like every grid in our paper. τ = 20 shows that a text-free value as good as the prompts exists; it is not a label-free selection rule.

In the revision. §6.2 (one paragraph); Appendix F.2 (Figures 2–3, Tables 37–39).

Question 3: Is there empirical evidence that reduced cross-category interference explains the gains from independent anchor rotations?

In short. Yes. We measured the interference directly, and it drops sharply under independent rotations; when the prompts are spread gradually, the interference falls and AUROC rises step by step.

Why we measured gradients. All categories are trained through one shared adapter, so interference means that a training step for one category also changes the scores of the others. A training step follows the gradient of the loss. If the gradients of two categories point in similar directions, a step for one also moves the other; if they are close to orthogonal, it barely does. The similarity between the categories' gradients is therefore a direct measure of the interference you ask about.

What we ran. On saved training snapshots of correct prompts, the shared rotation and independent rotations (three seeds each, three backbones), we computed each category's gradient with respect to the shared adapter on a fixed batch of its training images (the same images for every bank), and the mean absolute cosine between the gradients of different categories.

What we found (Appendix G.2, Table 41). With correct prompts the cosine is 0.10–0.21. The shared rotation gives the same values within 0.01, as the theory predicts, since rotating all axes together keeps their angles. Independent rotations reduce it 3.8–6.6× on every dataset and backbone, at every saved epoch and seed. Along the partial-spread path (Question 1), the interference falls step by step (CLIP VisA 0.125 → 0.065 → 0.047 → 0.032 → 0.026) while AUROC rises, and the training loss does not fall.

What this shows. With the prompts, a training step for one category also moves the others. Independent rotations remove most of this interference, and the more of it is removed, the higher the AUROC. Since the training loss does not improve along the way, the gain does not come from fitting the training data better; it comes from categories that no longer disturb each other's training.

In the revision. §6.4, Table 4; Appendix G.2–G.3.

Thanks for the detailed rebuttal. The additional experiments address most of my original concerns. However, I share the concerns raised by other reviewers regarding the overall significance and generality of the contribution. The theoretical result is relatively straightforward and applies to a restricted class of detectors. While the new experiments strengthen the empirical evidence, they do not extend the theoretical guarantee to modern prompt-learning approaches. Moreover, text-free scale selection still relies on test-set evaluation, limiting its practical applicability. Given these remaining limitations, I change my rating from 3 to 2.

Reviewer_E31TOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks what a fixed pair of text embeddings (anchors) actually contributes to a few- or full-shot vision–language anomaly detector that trains a visual adapter against them: changing a prompt changes both its meaning and its numerical geometry at once, so ordinary prompt comparisons cannot separate the two.

The detector's binary score sees the anchors only through their difference — one direction plus one magnitude. A trained adapter from a rotation-closed family can rotate its output to follow any replacement direction, so the direction, and with it the linguistic content, is unidentifiable. What survives is a single scalar called the effective scale, combining anchor separation with logit temperature: anchors with the same effective scale admit exactly the same abnormal-probability maps, losses and metrics.

Empirical design: five anchor conditions (correct medical prompts, a mismatched-dataset bank, scale-matched random, unmatched orthogonal, matched orthogonal) across seven medical datasets, three backbones, three adapter families and two supervision regimes. Scale-matched text-free anchors track correct prompts to within ~0.2% image AUROC; breaking the scale opens a gap up to ~11.8%; restoring only the scale closes it with no text; the same drop-and-recovery appears in MVFA and MadCLIP.

Two boundary settings extend the argument. With one adapter serving many categories, prompt axes are nearly collinear, and rotating them independently (preserving scale, adding no meaning) improves VisA by ~5.6% image and ~4.4% pixel AUROC. With the adapter frozen for transfer, the anchor frame is load-bearing but need not be linguistic.

Critical strengths

(1) An identifiability result that is actually proved and then separately tested. The paper does not just argue informally that prompt direction is absorbable; it states a theorem with an explicit condition (rotation closure) and then verifies that its own three adapter families satisfy that condition, and constructs controls that are exact at the probability-map level rather than merely "similar." The Gram-matched random and matched orthogonal conditions are the kind of control most papers in this literature skip. This is consequential because it converts a familiar suspicion ("maybe prompts don't matter much") into a precise claim with a stated scope — and the paper is explicit, in Proposition 1, about where the claim stops applying.

(2) The controls replicate outside the theorem's scope, in two real published systems. MVFA and MadCLIP both sit outside the fixed-anchor/rotation-closed premise, yet the same pattern appears: unmatched orthogonal anchors collapse MVFA from 94.8% to 73.7% image AUROC and restoring only the scale recovers 94.5%, and MadCLIP's learned prompts (96.2%) are matched by a text-free learned scale (96.3%). This matters because it means the finding is not an artifact of the toy architecture the theorem was written for, which is the single most predictable objection a reviewer would raise.

Critical weaknesses

(1) The headline claim rests on a null result with no formal equivalence test, and the benchmarks may not have the headroom to detect a semantic contribution.

  • The paper's own reading rule is that gaps are "compared descriptively without treating closeness as a formal equivalence test." But the conclusion — that prompt meaning contributes nothing beyond scale — is a non-inferiority claim, and non-inferiority requires a pre-specified equivalence margin and a test (e.g. TOST), not three seeds and a mean absolute gap. As written, "0.16%–0.26% mean gap, no consistent advantage for correct prompts" is consistent with both the paper's interpretation and with a small real advantage that the design cannot resolve.
  • More seriously, the image AUROCs in Table 1 are near ceiling: OCT2017 is 99.6, BraTS2021 96.9, RESC 97.2. At that level the metric has no room to register a semantic benefit, so a null result is the expected outcome regardless of whether words matter. The one dataset with genuine headroom is CheXpert at 69.3, and there the unmatched orthogonal control actually scores higher than correct prompts (69.8 vs 69.3) — which is either a real effect or noise, but is not discussed. The paper would be much stronger if the main-text table reported a regime where AUROC is not saturated.
  • The reasoning would also be well served by an upper-bound condition: an anchor pair chosen adversarially or optimised for the target task. If such anchors beat scale-matched random by a wide margin, then the representational space does contain semantic information that the paper's random controls simply fail to hit; if they do not, the claim is much better supported. This is a cheap experiment and its absence is the main missing control.

(2) The strongest-sounding evidence is the least well designed, and the positive result is under-developed relative to the negative framing.

  • The mismatched-dataset bank (e.g. scoring Kaggle BrainMRI with LiverCT description banks) is the result most likely to be quoted, and it is the one with the weakest design: replacing the complete bank changes the anchor cosine as well as the semantics, so it is explicitly not an exact probability-map control. A difference cannot isolate semantics (the paper says so) but neither can a null result support it, and the paper nonetheless uses it as "complementary evidence" for a semantics claim. The ≤1.7% difference under a completely wrong organ and modality is surprising enough that a reader will over-read it.
  • Conversely, the industrial independent-rotation result is the paper's most actionable finding — spreading near-collinear prompt axes yields +5.6% image and +4.4% pixel AUROC on VisA, and up to +8.8%/+8.3% across backbones, with no added meaning — but it occupies half a page and the proposed mechanism (reduced cross-category interference) is explicitly unverified. This is a free accuracy gain that requires only changing a bank, and the paper frames it as a consistency check rather than as a contribution.
  • The framing also creates an internal tension worth resolving: if meaning-free rotated anchors beat the correct prompt bank on VisA (91.8 vs 86.2 image AUROC), then the "correct prompts" condition is not a strong reference in that setting, which cuts both ways for the paper's argument.

(3) No comparison to the state of the art, so the significance of the detector family is unestablished. All three main tables are internal controls. MVFA and MadCLIP are evaluated only under the paper's own anchor conditions, never against other methods on the same benchmarks. A reader cannot tell whether the fixed-anchor family this paper analyses is competitive, or whether the conclusions extend to families that are (e.g. zero-shot prompt ensembles with frozen encoders, which are outside the theorem). The paper is scoped as a controlled study rather than a method paper, which is defensible, but then the practical import of the negative result — the stated motivation that practitioners should stop curating prompt banks — is argued in the conclusion without a demonstration that switching to a text-free scale-matched anchor actually helps anyone's pipeline end-to-end.

Questions for authors
  1. Optimised anchors. What happens if the anchor pair is optimised for the target task rather than drawn randomly at matched scale? If optimised anchors clearly beat correct prompts, does that not show the representational space does carry task-relevant semantic information?
  2. Mismatched bank. Since the mismatched bank changes the anchor cosine, does its ≤1.7% difference come from semantics or from the scale change? Please re-run the mismatched bank with the scale matched to correct prompts.
  3. Independent rotations. Is the VisA gain from spreading the axes reproducible when the rotated bank is also used at inference against a published baseline detector, not just your own? And can you test the interference mechanism directly by measuring gradient alignment across categories?
5 comments
AuthorsOpenReview ↗

Response to Reviewer E31T (1/4)

Thank you for the careful review. We answer each weakness bullet and each question separately, in your order. Results are written as mean [90% confidence interval] of the paired difference, correct prompts minus control, in image AUROC points; for example, +0.04 [−0.03, +0.11]. ✓ marks equivalence (the whole interval lies within ±0.5) and ✗ its absence.

Weakness 1, first bullet: no formal equivalence test

"non-inferiority requires a pre-specified equivalence margin and a test (e.g. TOST)"

In short. We agree, and we replaced our descriptive reading rule with a paired TOST.

What we ran. A cell is one dataset × vision backbone × K (48 real-anomaly cells, 36 normal-only). Within a cell, correct prompts and a control are paired by training seed, which fixes the support set, and the difference is averaged over the three seeds. Cells are pooled per regime with a t interval. Following standard practice for equivalence tests, we set the margin to the smallest effect that would matter: ±0.5 AUROC points. This is about a third of the average cost of a wrong scale (1.4 points with real anomalies, measured by the positive control below) and smaller than the seed-to-seed standard deviation of many single settings (up to 2.3 points in Table 1), so a smaller effect would be within the seed-to-seed variation of many single detectors. The positive control, unmatched orthogonal anchors, differs from matched orthogonal anchors only in scale, so it tests whether the design can detect a difference.

What we found (Table 2):

ControlReal anomalies (48)Normal-only (36)
Gram-matched random+0.04 [−0.03, +0.11] ✓+0.01 [−0.09, +0.10] ✓
Matched orthogonal+0.05 [−0.02, +0.12] ✓+0.01 [−0.09, +0.10] ✓
Mismatched bank, matched scale+0.01 [−0.05, +0.07] ✓+0.05 [−0.08, +0.17] ✓
Unmatched orthogonal (positive control)+1.42 [+0.88, +1.97] ✗+0.58 [+0.26, +0.91] ✗

What this shows. The exact controls stay equivalent down to margins of ±0.11 to ±0.17, so the conclusion does not hinge on the choice of ±0.5. Effects smaller than these intervals cannot be excluded by this test. The result also holds per backbone, on image average precision (AP), and under a random-effects model.

In the revision. §6.1 (test), §6.2 and Table 2 (result), Appendix B.3 (procedure, margin), Appendix E (Tables 31–34, every cell).

Weakness 1, second bullet: saturated benchmarks; CheXpert 69.8 vs 69.3

"At that level the metric has no room to register a semantic benefit"

In short. We repeated the test on every cell where correct prompts score below 90% image AUROC; the conclusion is unchanged, and these cells do register a difference when one exists.

What we ran. The 32 cells are LiverCT at K ≤ 8 on all three backbones (9 real-anomaly cells) and all CheXpert cells plus all but one BUSI cell (23 normal-only cells).

What we found (Table 33). Gram-matched random +0.04 [−0.31, +0.38] and −0.03 [−0.15, +0.09]; matched orthogonal +0.13 [−0.09, +0.34] and +0.01 [−0.12, +0.14]; mismatched bank at matched scale −0.01 [−0.17, +0.15] and +0.10 [−0.07, +0.27]: all equivalent. On the same real-anomaly cells, the positive control loses 3.89 [2.76, 5.02] AUROC and 4.88 [3.17, 6.59] AP points.

The CheXpert cell (CLIP, K = 16). You are right that it needed discussion. The unmatched orthogonal control is higher in all three seeds (69.9, 68.5, 71.1 against 69.5, 67.6, 70.8; mean −0.54). In the new similarity-scale sweep, CheXpert's AUROC is non-monotone in the scale (CLIP, K = 32: 72.0 at τ = 5, 66.6 at τ = 10, 70.1 at τ = 100, against 69.7 for correct prompts).

What this shows. These cells are far from the ceiling and can register a difference: a wrong scale costs almost 4 points there. Yet every control matched in scale stays within ±0.5 of correct prompts. A benefit of the words as large as the margin would have been visible in these cells, and none appears. The CheXpert cell does not contradict this: the control that scores higher carries no meaning, so its advantage cannot be semantic. Because CheXpert's AUROC rises and falls with the scale, a wrong scale can land on a favorable point.

In the revision. One clause in §6.2; Appendix E (Table 33, paragraphs "Unsaturated cells" and "The CheXpert cell"); Appendix F.2.

AuthorsOpenReview ↗

Response to Reviewer E31T (2/4)

Weakness 1, third bullet: no upper-bound condition (optimized anchors)

"an anchor pair chosen adversarially or optimised for the target task"

In short. We ran it on all seven datasets, every K, and all three backbones. Optimized anchors do not beat fixed correct prompts, and what optimization changes is the scale.

What we ran. Five conditions, each compared with fixed correct prompts:

  • Learned scale, no text: an orthogonal-start pair whose cosine is the only trainable anchor parameter.
  • Optimized anchors, from prompts: both anchor vectors trained with the adapter, starting from the correct prompts.
  • Optimized anchors, from a random pair: the same, starting from a random pair with the prompts' cosine.
  • Correct prompts at the learned scale and random pair at the learned scale: fixed anchors whose similarity scale is rescaled, run by run, to the scale the optimized anchors reached. These two separate the optimized anchors' scale from their direction.

The first three leave the fixed-anchor premise of our theorem, so they are empirical evidence; the last two are fixed again and covered by it.

What we found (Table 36, pooled over three backbones; mean [90% CI], as in our first response):

ComparisonReal anomalies (48)Normal-only (36)
From prompts − from a random pair−0.03 [−0.12, +0.06] ✓−0.16 [−0.33, +0.01] ✓
Correct prompts − optimized+1.61 [+1.12, +2.09] ✗+0.41 [+0.18, +0.64] ✗
Optimized − random pair at the same scale−0.18 [−0.45, +0.10] ✓+0.02 [−0.28, +0.32] ✓
Correct − random pair, both at learned scale+0.23 [+0.02, +0.44] ✓+0.20 [+0.00, +0.39] ✓

What this shows.

  • The starting words do not matter: from prompts and from a random pair end equivalent (row 1).
  • Optimized anchors are not an upper bound: correct prompts are better (row 2). Optimization moves the anchor cosine from about 0.95 to as low as −0.8, a scale like that of an orthogonal pair at τ ≈ 134 instead of the prompts' 15–28, and overfits the K observed anomalies. Fixed correct prompts placed at that scale lose most of the same ground (1.20 with real anomalies).
  • At the same scale, optimized anchors and a fixed random pair are equivalent (row 3): optimizing the anchors adds nothing beyond the scale it selects.
  • The text-free learned scale stays closer to the prompts' scale than the optimized anchors, but remains below correct prompts, by 0.74–1.96 points with real anomalies and 0.16–1.53 normal-only, depending on the backbone (Appendix F.1, Table 35). With real anomalies it overshoots the scale on OCT2017 and LiverCT; a fixed round scale is the safer text-free choice and is equivalent to the prompts (our answer to Weakness 3).

In the revision. §6.2 (one paragraph); Appendix F.1 (Tables 35–36).

AuthorsOpenReview ↗

Response to Reviewer E31T (3/4)

Weakness 2, first bullet: the mismatched-dataset bank is not an exact control

"replacing the complete bank changes the anchor cosine as well as the semantics"

In short. We agree. We ran the scale-matched version (also your Question 2), and the original bank is now described only as a stress test.

What we ran. For each target dataset, the donor bank (wrong organ, modality and findings) is kept, and the similarity scale in training and evaluation is multiplied by √((1 − ρ_correct)/(1 − ρ_donor)), where ρ is the cosine between the normal and abnormal anchors. This restores the target's effective scale exactly while keeping the wrong words, which makes the bank an exact probability-map-level control. Seven datasets, every K, three seeds, three backbones.

What we found (mean [90% CI], as in our first response). Equivalent to correct prompts: +0.01 [−0.05, +0.07] with real anomalies and +0.05 [−0.08, +0.17] normal-only, also per backbone and in the unsaturated cells. The largest per-dataset gap of the original bank, +0.84 on LiverCT, whose donor has a 1.43× larger effective scale, becomes −0.01 at matched scale; every per-dataset gap at matched scale is within 0.15.

What this shows. The original ≤1.7% differences came from the scale, not from the words.

In the revision. §5 (definition; "provides complementary evidence" removed); §6.2, Table 1 (new row) and Table 2; Appendix A.5 (Table 5); Appendices D and E.

Weakness 2, second bullet: the industrial result is under-developed; mechanism unverified

"the proposed mechanism (reduced cross-category interference) is explicitly unverified"

In short. We agree. We had listed it as a contribution, but the evidence was thin, so we added three analyses (§6.4, Appendix G).

What we ran.

  • Robustness to the geometry draw. Three geometry seeds, separate from the three training seeds, crossed in a 3 × 3 grid for both banks, both datasets and three backbones.
  • The interference, measured. With one shared adapter, interference means that a training step for one category also changes the scores of the others. A training step follows the gradient: if two categories' gradients point in similar directions, a step for one also moves the other. We therefore computed each category's training-loss gradient with respect to the shared adapter, on a fixed batch of its images, and measured how similar the gradients of different categories are.
  • Dose-response. Comparing only the two ends shows that interference and AUROC change together, not that they follow the arrangement step by step. If spreading the axes is what helps, spreading them part of the way should help part of the way. We built banks rotated 25, 50 and 75% of the way from the shared to independent rotations, keeping every category's scale fixed so that only the arrangement changes.

What we found.

  • Independent rotations beat the shared rotation in all 108 paired comparisons (smallest gain +1.01; up to +9.09 on DINOv3 VisA). The geometry seed moves the means by at most 0.27.
  • Under correct prompts or the shared rotation, the gradients are coupled (mean |cos| 0.10–0.21); independent rotations make them 3.8–6.6× less aligned on every dataset and backbone, at every saved epoch and seed.
  • Along the partial-spread path, coupling falls and AUROC rises at every step on both datasets, on CLIP and DINOv3 (VisA image AUROC 86.3 → 89.2 → 90.1 → 90.8 → 91.8 on CLIP), while the training loss does not fall.

What this shows. The gain comes from the arrangement of the anchors, not from a lucky draw. Spreading the axes stops a training step for one category from moving the others, and the further they are spread, the less interference and the higher the AUROC. Since the training loss does not improve, the gain comes from categories no longer disturbing each other, not from a better fit to the training data.

In the revision. §6.4 and Table 4 (partial-spread rows, |cos|); Appendix G.1–G.3 (Tables 40–42).

Weakness 2, third bullet: meaning-free rotations beat correct prompts on VisA

"the 'correct prompts' condition is not a strong reference in that setting"

In short. We agree for accuracy, and we now say in §6.4 that correct prompts are the reference for meaning there, not an upper bound on accuracy.

Why the comparison still isolates meaning. The shared rotation keeps the prompts' joint geometry and removes their meaning; it matches them on VisA (86.3 vs 86.2 image, 91.6 vs 91.7 pixel AUROC). What independent rotations change is the arrangement across categories: the prompt bank's per-category difference vectors are nearly collinear (mean |cos| about 0.72), the rotated ones nearly orthogonal (about 0.03). The gain therefore comes from geometry.

In the revision. §6.4; Table 4; Appendix G.2.

Reviewer_sX9bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates what information fixed text anchors contribute to few- and full-shot vision-language anomaly detectors when a trainable visual adapter is present. For a single normal--abnormal anchor pair, the authors show that binary cosine-based scoring depends on the anchors through their difference vector. If the visual adapter family is rotation-closed, the direction of this difference vector can be absorbed by reparameterizing the adapter, leaving the effective scale as the identifiable quantity. The paper further extends the analysis to a shared multi-category setting, where the joint Gram geometry of the category-specific difference vectors becomes relevant.

The theoretical claims are evaluated using seven medical datasets, three vision backbones, several adapter families, and multiple anchor controls, including correct prompts, random anchors, and orthogonal anchors with or without scale matching. The authors additionally study MVFA and MadCLIP as external anomaly detectors and perform multi-category experiments on MVTec-AD and VisA. The experiments generally show that scale-matched text-free anchors can approach the performance of prompt-derived anchors under the studied settings, while changing the effective scale can substantially affect performance.

Critical strengths
  1. The paper asks an important and underexplored attribution question. Rather than simply comparing different prompt wordings, the authors explicitly separate linguistic semantics from the numerical geometry of the text anchors. This provides a useful perspective for interpreting performance gains in vision-language anomaly detection.

  2. The theoretical analysis and controlled anchor experiments are carefully designed. The distinction between anchor direction and effective scale is clearly formulated, and the use of scale-matched random and orthogonal controls provides stronger evidence than conventional prompt ablations. The multi-category extension to the joint Gram geometry is also conceptually interesting.

Critical weaknesses
  1. The theoretical scope is substantially narrower than the current landscape of text-based anomaly detection. The main equivalence result requires prescribed fixed text anchors, binary cosine-based scoring, and a trainable visual adapter whose output space is closed under arbitrary orthogonal rotations. However, a substantial portion of recent CLIP-based anomaly detection methods employ learnable prompts, dynamic prompts, or visually conditioned text representations. Such methods do not satisfy the fixed-anchor assumption, and the proposed equivalence result therefore does not directly apply to them. This substantially limits the generality of the paper's central conclusion regarding the role of linguistic semantics in modern vision-language anomaly detection.

  2. The external-model validation does not adequately resolve this limitation. The paper directly evaluates only MVFA and MadCLIP as existing external anomaly detectors. Importantly, the paper itself states that neither of these methods falls within the exact theoretical coverage of Theorem 1, due to their residual visual paths and/or learned prompts. In contrast, APRIL-GAN is explicitly identified by the authors as satisfying the relevant rotation-closure condition for its trained linear-projection branch, yet no corresponding anchor-substitution experiment is provided. Consequently, the paper currently lacks a direct validation on a representative existing detector that actually satisfies the assumptions of its main theorem. This makes it difficult to determine whether the result transfers beyond the authors' own controlled architecture.

These two issues are consequential to my assessment because the paper's main contribution is not a new anomaly detector but a general statement about what text contributes to vision-language anomaly detection. If the statement applies only to a relatively narrow architectural subclass and is not directly validated on existing methods belonging to that subclass, then its broader significance is considerably reduced.

Questions for authors
  1. The paper's main theorem assumes prescribed fixed text anchors and a rotation-closed trainable visual adapter. How broadly do the authors believe this setting represents current CLIP-based anomaly detection, given that many recent approaches learn, dynamically generate, or visually condition their text prompts?

  2. The paper's own scope analysis identifies APRIL-GAN as satisfying the relevant rotation-closure condition for its trained linear-projection branch. Why is APRIL-GAN not included in the external-model anchor-substitution experiments? Results on APRIL-GAN appear particularly important because it is more directly covered by Theorem 1 than MVFA or MadCLIP.

  3. Can the authors add experiments on at least one or two existing CLIP-based anomaly detectors that actually satisfy the theorem's assumptions? In particular, an APRIL-GAN experiment using correct prompts, Gram-matched random anchors, unmatched orthogonal anchors, and matched orthogonal anchors would provide much stronger evidence for external validity.

  4. Since MVFA and MadCLIP are explicitly outside the theorem's exact scope, what should readers conclude from their empirical results? Are these intended merely as suggestive boundary experiments, or are they being used to support a broader claim that text semantics are generally unnecessary?

  5. Would the authors consider narrowing the central conclusion from a broad statement about vision-language anomaly detection to a more precise claim about fixed-anchor detectors with rotation-closed trainable visual mappings?

  6. More broadly, what is the practical relevance of the proposed result for recent prompt-learning approaches such as methods with learnable or image-conditioned normal/abnormal prompts, where the text embeddings themselves are part of the learned model rather than fixed readout coordinates?

5 comments
AuthorsOpenReview ↗

Response to Reviewer sX9b (1/4)

Thank you for the review. We answer each weakness and each question separately, in your order. Results are written as mean [90% confidence interval] of the paired difference in image AUROC points, real anomalies / normal-only; ✓ marks equivalence at ±0.5 and ✗ its absence.

Weakness 1: the theorem covers only part of current text-based detectors

"Such methods do not satisfy the fixed-anchor assumption, and the proposed equivalence result therefore does not directly apply to them"

In short. We agree: the guarantee covers fixed anchors with a retrained, rotation-closed adapter, which is a subset of current detectors; learned, dynamically generated and image-conditioned prompts fall outside it. We tested learned prompts and detectors outside the premise directly, and the same pattern held; image-conditioned prompts remain untested (Question 6).

Optimized anchors, with the same controls (Appendix F.1, Table 36; anchors trained jointly with our adapter, pooled over every dataset, backbone and K):

  • Optimized from the prompts − optimized from a random pair: −0.03 [−0.12, +0.06] ✓ / −0.16 [−0.33, +0.01] ✓. The starting words make no detectable difference.
  • Optimized anchors − a fixed random pair at the scale they reach: −0.18 [−0.45, +0.10] ✓ / +0.02 [−0.28, +0.32] ✓. Beyond that scale, optimizing the anchors adds no detectable gain.
  • Fixed correct prompts − optimized anchors: +1.61 [+1.12, +2.09] / +0.41 [+0.18, +0.64]. Optimized anchors fall below fixed prompts, because training moves them to a larger scale.
  • At that scale, correct words keep a small edge over a random pair (+0.23 / +0.20), inside the margin.

Two published detectors outside the premise (§6.3, Table 3; image AUROC, mean over four datasets at K = 16):

  • MVFA, whose residual adapters fall outside the closure result (Table 18): unmatched orthogonal anchors drop it from 94.8 to 73.7, and restoring only the scale brings it back to 94.5.
  • MadCLIP, which learns its prompts: replacing them with a learned scale and no text gives 96.3 against 96.2.

Where each part stands. Theorem 1 guarantees that scale-matched text-free anchors admit exactly the prompts' probability maps and losses; Proposition 1 and Appendix A.8 (Table 18) state where this stops. We label the results above as empirical evidence beyond the guarantee (§7).

Weakness 2: no published detector inside the theorem's premise

"the paper currently lacks a direct validation on a representative existing detector that actually satisfies the assumptions of its main theorem"

In short. You are right that the submission did not test a published detector inside the premise. We now report two: AnomalyGPT in its own unsupervised setting, and APRIL-GAN's own code both in its published zero-shot protocol and trained within each real-anomaly dataset under the few-shot protocol of our Table 1.

Which protocol applies where. MVTec-AD and VisA have no anomalous training images, so detectors train within the dataset on synthetic anomalies or transfer zero-shot from the other dataset; the real-anomaly medical datasets have abnormal support images. The theorem applies when the adapter is trained against the anchors it is tested with; in zero-shot transfer the projection is frozen, so it gives no guarantee.

ProtocolDatasetsTheoremDetectors
Few-shot, real anomalies, within the datasetBraTS2021, LiverCT, RESC, OCT2017appliesAPRIL-GAN (new); ours (Tables 1–2); MVFA, MadCLIP (outside the premise, Table 3)
Full-shot, synthetic anomalies, within the datasetMVTec-AD, VisAapplies to the shared rotation (Corollary 1)AnomalyGPT (new); ours (§6.4)
Zero-shot, trained on the other datasetMVTec-AD, VisAno guaranteeAPRIL-GAN (new)

The results of the three new experiments follow in our next response.

AuthorsOpenReview ↗

Response to Reviewer sX9b (2/4)

Weakness 2, continued: the three new rows of the protocol map

APRIL-GAN, few-shot with real anomalies. We ran APRIL-GAN's own code (model, training step and optimizer) within each real-anomaly dataset, with K = 16 normal and 16 abnormal support images drawn as for our Table 1, the same training masks and number of epochs, our clinical prompt banks, and the four anchor conditions. Image / pixel AUROC at K = 16, mean ± sd over training seeds (OCT2017 has no test masks):

AnchorsBraTS2021LiverCTRESCOCT2017
Correct prompts92.3±1.3 / 90.3±3.391.8±1.0 / 99.9±0.195.4±1.1 / 97.6±0.196.3±0.3 / –
Gram-matched random91.9±2.4 / 91.8±4.192.2±1.1 / 99.9±0.195.2±1.3 / 95.9±2.094.0±2.1 / –
Unmatched orthogonal70.4±22.5 / 91.8±4.087.9±4.3 / 99.4±0.774.4±20.2 / 88.4±9.553.5±3.1 / –
Matched orthogonal91.9±1.9 / 91.4±1.092.0±1.9 / 99.9±0.194.8±2.1 / 96.3±1.695.2±1.8 / –

Pooled over the four datasets (correct prompts minus control, mean [90% CI]), the exact controls stay close to the prompts: Gram-matched random +0.6 [−0.8, +2.0] and matched orthogonal +0.5 [−0.2, +1.1] image AUROC, within 1.1 points on every dataset except Gram-matched random on OCT2017 (2.4), and within 1.7 points in pixel AUROC. Changing only the scale costs +22.4 [+3.7, +41.2]: unmatched orthogonal anchors collapse on three of the four datasets.

Exactness in APRIL-GAN's code. Each trained correct-prompt model, with its weights rotated onto the Gram-matched or matched orthogonal pair, gives the same image and pixel AUROC on every dataset and seed (largest difference 0.003 points). Independently trained runs differ more, in every condition including correct prompts: with 32 fixed support images and a constant learning rate, some runs stop at a higher training loss. With our detector's training recipe, the same linear design is equivalent to the prompts, with intervals within [−0.06, +0.12] (Table 32).

AnomalyGPT, full-shot with synthetic anomalies (AAAI 2024). Its anchors are a fixed prompt ensemble, and its head is one free linear map per encoder stage, retrained within each dataset. We changed only the anchor bank. The shared rotation keeps the prompts' joint geometry, so the theorem predicts the same maps (Corollary 1); independent rotations give each category a random pair with its prompts' cosine. AUROC, mean ± sd over three training seeds:

Anchor bankVisA imageVisA pixelMVTec-AD imageMVTec-AD pixel
Correct prompts81.3±0.181.4±0.197.5±0.093.1±0.1
Shared rotation81.2±0.180.9±0.197.5±0.192.9±0.1
Independent rotations83.8±0.489.1±0.198.4±0.095.5±0.0

The shared rotation stays within 0.5 AUROC of the prompts; independent rotations gain 2.5 image and 7.7 pixel AUROC on VisA and 0.9 and 2.4 on MVTec-AD, as in our detector (Appendix G.4, §6.4).

APRIL-GAN, zero-shot transfer. The projection is trained on one dataset and applied, frozen, to the other; columns name the test dataset. The image score is the maximum of the trained map, since the published score uses the untrained CLIP image embedding. The matched pair has the training prompts' mean effective scale; the unmatched pair keeps the scale of 100. AUROC, mean ± sd over training seeds:

AnchorsVisA imageVisA pixelMVTec-AD imageMVTec-AD pixel
Correct prompts78.3±1.494.1±0.484.6±0.387.8±0.6
Orthogonal pair, matched scale77.8±2.993.7±0.985.0±2.486.2±1.2
Orthogonal pair, unmatched scale72.1±2.593.5±0.467.8±10.170.0±13.2

What we found. With APRIL-GAN's published image score, our correct-prompt runs reproduce its published results (77.5 / 94.1 on VisA against 78.0 / 94.2; 86.3 / 87.8 on MVTec-AD against 86.1 / 87.6). Although the theorem gives no guarantee for a frozen projection, the matched text-free pair stays close to the prompts. Paired by seed, correct prompts minus the matched pair: +0.6 [−1.3, +2.5] and −0.4 [−2.2, +1.4] image AUROC on VisA and MVTec-AD, and +0.4 [−0.1, +0.9] and +1.7 [+0.5, +2.8] pixel AUROC. At unmatched scale, the pair loses 6.2 and 16.9 image points.

AuthorsOpenReview ↗

Response to Reviewer sX9b (3/4)

Results are written as in our first response.

Question 1: How broadly does the setting represent current CLIP-based anomaly detection?

In short. We agree that many recent methods learn or condition their prompts. We see the scope at two levels: for fixed prompts with a trained, rotation-closed adapter, the theorem guarantees that scale-matched text-free anchors admit exactly the prompts' maps, and our experiments test how far the pattern extends beyond that.

Guaranteed. Fixed prompts with a trained, rotation-closed adapter: APRIL-GAN's linear projection when trained within the dataset, AnomalyGPT, and the linear, MLP and Transformer adapters of our paper. Prompts generated once, for example by a language model, and then held fixed also count as fixed anchors.

Tested beyond the guarantee. Anchors optimized jointly with our adapter, MadCLIP's learned prompts and MVFA's residual adapters all follow the same pattern (Weakness 1, Question 3).

Not tested. Prompts conditioned on each image (Question 6).

Question 2: Why is APRIL-GAN not included in the external experiments?

In short. APRIL-GAN is a natural test case, and we now report it in two protocols (Weakness 2): within each real-anomaly dataset, where the theorem applies, with all four anchor conditions; and in its published zero-shot protocol, where the projection is frozen at test and the theorem gives no guarantee.

Question 3: Can the authors add experiments on existing detectors with the four controls?

In short. Yes, both inside the guarantee and beyond it.

Inside the guarantee.

  • APRIL-GAN, its own code within each real-anomaly dataset: all four conditions (Weakness 2, continued).
  • Our linear adapter, which follows the same design (per-layer linear maps into the text space, 100 · cosine logits, focal plus Dice loss), on CLIP ViT-L/14 with real anomalies (Table 32): Gram-matched random +0.05 [−0.03, +0.12] ✓, matched orthogonal +0.03 [−0.06, +0.11] ✓, unmatched orthogonal +1.30 [+0.43, +2.18] ✗.
  • AnomalyGPT, in its own protocol: the exact multi-category control (Weakness 2, continued).

Beyond the guarantee.

  • MVFA (Table 3, image AUROC): correct prompts 94.8, Gram-matched random 93.9, unmatched orthogonal 73.7, matched orthogonal 94.5.
  • MadCLIP learns its prompts, so it has no fixed pair to control; we compare it with a learned scale and no text (Weakness 1).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their careful reviews and helpful suggestions. Below we summarize what we did during the rebuttal. The detailed responses, with the numbers and their limits, are posted under each review.

New experiments and analyses

  • An equivalence test with a margin fixed before the analysis, applied to the main medical results (every dataset, backbone and support size), together with a control that differs from the matched controls only in scale.
  • The wrong-dataset prompt banks rerun with their scale set to that of the correct prompts, so that only the words differ.
  • Anchors optimized jointly with the adapter, starting from the prompts or from random vectors, and fixed anchors retrained at the scale the optimized anchors reached.
  • A sweep of the similarity scale (the logit scale τ) for two orthogonal anchor vectors, with no text encoder, on two backbones and at every support size.
  • For the multi-category setting: rotations drawn with seeds separate from the training seeds; a measurement of how similar the training updates of different categories are; and anchor banks spread partway between the prompt arrangement and fully independent rotations.
  • The same anchor banks (correct prompts, shared rotation, independent rotations) in a published detector, AnomalyGPT, retrained on MVTec-AD and VisA (Appendix G.4).
  • APRIL-GAN's own code, trained few-shot within each real-anomaly medical dataset with the four anchor conditions, and in its published zero-shot protocol.
  • A comparison with published methods on the same datasets.
  • An audit of whether support and test images share patients in the published splits.

Changes to our paper

  • We added Appendices E (equivalence tests), F (scale controls and the similarity-scale sweep), G (multi-category experiments, with AnomalyGPT in G.4) and H (comparison with published methods). Appendix B now also describes the dataset splits, their audit, and the statistical analysis, Appendix A.5 the wrong-dataset banks at matched scale, and the scope table of Appendix A.8 (Table 18) includes AnomalyGPT.
  • In the main text, we describe the equivalence test in §6.1 and report it in §6.2 (new Table 2). The new results are summarized in §5, §6.2 and §6.4, with a new row in Table 1 and new rows and columns in Table 4.
  • The abstract now refers to trained fixed-anchor detectors, and the abstract, contributions and conclusion state the equivalence margin. The Limitations now mention the shared patients in four published splits.
  • All new or changed text is in blue. We did not change our claims or the research question.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.