acceptodds
Withdrawn from ICLR 2027

CRAG-CLIP: Cross-Layer Relational Abnormality Geometry for Zero-Shot Image Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

CLIP-based zero-shot image anomaly detection has achieved encouraging progress by leveraging transferable vision–language representations to identify anomalies in previously unseen datasets. Recent source-only approaches further improve cross-dataset transfer by learning anomaly-aware text prompts from auxiliary source data. However, existing prompt-based approaches typically represent the abnormal concept with a single normal-abnormal opposition and fuse multi-layer visual evidence through fixed heuristics. This paradigm inevitably under-utilizes the multi-faceted nature of real-world anomalies and the complementary semantic representations across different transformer layers. To address these limitations, we propose CRAG-CLIP, a source-only zero-shot anomaly detection framework that models abnormality as a cross-layer relational geometry in the CLIP text space. The framework represents abnormality through multiple object-agnostic normal–anomaly semantic relations, constrains the learned directions with frozen language priors to mitigate source-specific semantic drift, and promotes complementary semantic coverage through diversity regularization. It further estimates the reliability of layer-wise patch evidence from posterior uncertainty and cross-layer disagreement, enabling data-dependent fusion across visual layers. For image-level prediction, global semantic evidence and risk-sensitive local anomaly evidence are jointly aggregated to capture both holistic context and localized defects. Extensive evaluations on five industrial and nine medical datasets establish the effectiveness and competitive performance of CRAG-CLIP.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
zero-shot anomaly detection, vision-language models, anomaly localization, cross-layer fusion

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_C3uhOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a source-only zero-shot anomaly detection framework that aims to enhance the diversity of abnormality representations in the text embedding space and improve the aggregation of visual evidence. It introduces multiple normal–abnormal semantic pairs, reliability-aware cross-layer fusion, and global-local evidence aggregation. While the overall design is intuitive, it remains unclear whether the proposed semantic axes effectively capture real-world anomaly diversity and whether the reliability estimation is well justified.

Critical strengths

The paper is motivated by two clearly identified limitations of existing methods: limited diversity in abnormality representations and heuristic aggregation of multi-layer visual evidence. The proposed approach provides an intuitive way to address these limitations.

Critical weaknesses
  1. Unclear main figure. The main figure is difficult to follow and does not clearly illustrate the information flow and interactions among the proposed modules, making the overall framework harder to understand.
  2. Insufficient justification for the proposed designs. The connection between the proposed methods and their motivations is not sufficiently established. In particular, it remains unclear whether multiple semantic axes truly capture real-world anomaly diversity and whether posterior uncertainty and cross-layer agreement can reliably measure the quality of anomaly evidence. The designs appear largely heuristic, lacking sufficient theoretical justification or direct empirical validation of these assumptions.
  3. Insufficient experimental validation and limited performance gains. Although the paper evaluates the method on multiple benchmarks, the results in Tab.1 do not consistently outperform strong baselines. Moreover, several ablation studies show only marginal improvements, raising questions about the necessity and effectiveness of the proposed components. More comprehensive analyses are needed to support the claimed contributions.
Questions for authors
  1. In Table 3, the proposed CLPC module provides only marginal improvements over uniform layer fusion, and removing the disagreement penalty has little impact on performance. Could the authors further justify the necessity of this design, potentially through comparisons with simpler fusion strategies?

  2. Given the diversity of real-world anomaly types, how do the authors justify that the predefined normal–abnormal semantic pairs effectively capture the diversity of real-world anomalies?

  3. What is the rationale behind using semantic-axis posterior entropy and cross-layer agreement to estimate the reliability of anomaly evidence? Low entropy or strong agreement does not necessarily indicate accurate anomaly predictions. Could the authors provide theoretical insights or empirical evidence to validate this assumption?

  4. Although the proposed method achieves competitive image-level results, it underperforms FAPrompt on several pixel-level anomaly detection benchmarks, particularly in medical domains. Could the authors provide a more detailed analysis of these performance limitations and explain under which conditions the proposed method is most effective?

Reviewer_AHN6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CRAG-CLIP, which casts zero-shot anomaly detection as a cross-layer relational geometry in CLIP text space via three coupled modules (LAPA, CLPC, GLEA). The design is clear, ablations are thorough, medical image-level detection outperforms FAPrompt, and the strict source-only protocol with solid reproducibility is commendable. However, industrial benchmarks are only second-best overall, gains concentrate on medical image-level tasks, and medical pixel-level results are uneven; the eight semantic axes are hand-specified, so coverage is a design assumption; and hyperparameter sensitivity is analyzed on only two datasets.

Critical strengths

This work focuses on cross-domain anomaly detection problems with practical value. In this method, multiple sets of pairwise semantic relationships simultaneously serve anomaly scoring and cross-layer reliability estimation, allowing the representation and fusion mechanism to form a more coherent design. The medical image-level detection results reported in the article also show certain application potential. But, these advantages are not sufficient to eliminate doubts about novelty. Its main components overlap greatly with existing work, and the current contribution is closer to the incremental combination of specific mechanisms.

Critical weaknesses
  1. Multiple abnormality prompts, diversity regularization, and language anchoring overlap substantially with FAPrompt and DAPO. The substantive differences remain insufficiently established.

  2. Low entropy over semantic axes and cross-layer agreement do not necessarily indicate reliable predictions, particularly when multiple anomaly attributes coexist or layers share errors.

3.Ablations show marginal improvements from the disagreement penalty and smooth-max fusion, while language anchoring reduces PRO, providing limited support for their necessity.

4.Gains are concentrated in medical image-level detection, with no consistent improvements in industrial detection or medical pixel-level localization. Meanwhile, the experiments do not adequately establish the contribution of the proposed mechanisms or their robustness under domain shifts.

Questions for authors
  1. Compared with FAPrompt and DAPO, what capabilities do you provide that existing fine-grained prompts cannot achieve?

  2. Why does low entropy over semantic axes indicate more reliable layer-wise evidence? Does this assumption hold when multiple anomaly attributes coexist in the same region?

  3. Removing the cross-layer disagreement penalty barely changes performance. How do the authors justify its practical contribution?

  4. Are the small performance gains consistent across multiple random seeds? What is the variance?

  5. Why does the method improve medical image-level detection while remaining substantially behind FAPrompt in medical pixel-level localization?

  6. How sensitive is performance to descriptor choices? Are the results stable when descriptors are replaced with synonyms or semantically similar pairs?

Reviewer_2dNEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CRAG-CLIP for zero-shot anomaly detection. It models abnormality through multiple paired normal–anomaly semantic directions. The proposed Language-Anchored Paired Axes (LAPA) module learns complementary semantic relations with language-anchor and diversity regularization. Cross-Layer Posterior Consensus (CLPC) estimates the reliability of layer-wise patch evidence using semantic posterior entropy and cross-layer disagreement, while Global-Local Evidence Aggregation (GLEA) combines global image-level evidence with high-scoring local patches. Experiments on five industrial and nine medical datasets demonstrate that CRAG-CLIP achieves competitive performance compared to state-of-the-art methods.

Critical strengths

1.The method is reasonably well presented, with explicit formulations and implementation details. The appendix provides the semantic descriptor pairs, training configuration, checkpoint-selection protocol, hyperparameter sensitivity analysis, and category-level results.

2.The method achieves competitive results on several benchmarks, particularly medical image-level anomaly detection, where it outperforms the compared baselines on average.

Critical weaknesses

1.The proposed CLPC module assumes that concentrated semantic-axis posteriors and cross-layer agreement indicate reliable anomaly evidence. However, the semantic axes are not mutually exclusive, and a patch may simultaneously exhibit multiple abnormality attributes. Therefore, high posterior entropy does not necessarily imply unreliable evidence, while a concentrated posterior does not guarantee correctness. More importantly, Table 3 shows only marginal improvements over uniform layer fusion (+0.3 pixel AUROC and +0.1 PRO), and removing the disagreement penalty produces almost identical results.

2.Although CRAG-CLIP introduces paired normal–abnormal axes and language anchoring, it remains unclear whether these design choices provide meaningful advantages over existing multi-prompt representations. Moreover, the Gram-based diversity constraint encourages geometric separation but does not necessarily ensure complementary abnormality semantics. The current single-axis ablation also changes the number of learnable prompt parameters.

3.As shown in Table 1, CRAG-CLIP underperforms FAPrompt on all four averaged industrial metrics and both averaged medical pixel-level metrics, with particularly noticeable gaps in medical anomaly localization. While the method achieves competitive medical image-level detection results, these findings raise concerns about whether the proposed multi-axis representation and reliability-aware fusion consistently improve cross-domain anomaly localization, which is a central motivation of the paper.

Questions for authors

1.Table 3 indicates that removing the disagreement penalty has almost no effect on the results. What evidence supports its inclusion as a central component of the proposed cross-layer consensus mechanism?

2.Table 4 shows that smooth-max fusion provides only a marginal image-level improvement over arithmetic mean fusion. Could the authors clarify the practical advantage of GLEA and evaluate whether these small differences are statistically reliable?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.