acceptodds
Withdrawn from ICLR 2027

ERP: Empirical Representation Prototyping for Language-Free Zero-Shot Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent language-free zero-shot anomaly detection (ZSAD) methods have shown that textual prompts can be replaced by learnable normal and abnormal visual representations. However, these representations are typically optimized as free parameters through classification and segmentation objectives, serving primarily as discriminative anchors without being explicitly constrained to represent the visual patterns observed in training data. In this work, we revisit how normality and abnormality should be represented in language-free ZSAD and propose Empirical Representation Prototyping (ERP), which replaces optimization-based decision vectors with data-derived visual prototypes. Specifically, we introduce an EMA-based representation prototyping mechanism that continuously aggregates normal and abnormal features from training samples to construct representative visual references. For anomaly segmentation, we further develop an importance-aware prototype construction strategy that emphasizes ambiguous normal patches while preserving spatially diverse normal patterns, preventing prototypes from being dominated by trivial regions. In addition, a cosine-based multi-scale feature aggregation strategy enhances local representations for fine-grained anomaly localization. Without relying on textual prompts, ERP achieves strong performance across 12 industrial and medical anomaly detection benchmarks, demonstrating that empirical visual prototypes provide an effective alternative to freely optimized decision representations for language-free ZSAD.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Zero Shot Anomaly Detection, Representation Prototypes, CLIP, EMA, Language Free.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MGvVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ERP replaces freely learned normal/abnormal reference vectors with prototypes updated by exponential moving averages of source-domain visual features. It uses a frozen CLIP encoder, trainable adapters, importance-based normal-patch selection, and multi-scale feature aggregation. Evaluation covers 12 industrial and medical datasets under cross-dataset transfer, with the clearest gains.

Critical strengths
  1. Updating visual references from observed features is a simple, useful alternative to freely optimized decision vectors. The method requires no target-domain training data.

  2. Evaluation spans industrial and medical domains. Relative to UniADet, mean pixel AUPR improves from 42.3 to 45.4 on industrial datasets and from 53.0 to 54.4 on medical datasets.

Critical weaknesses

Major Weaknesses

  1. Limited novelty. The method retains the normal/abnormal reference-vector paradigm exemplified by AnomalyCLIP, with the main change being how these vectors are obtained and updated. Avoiding text-based initialization is a reasonable design choice, but text-free visual references have already been explored in VisualAD and UniADet. Relative to these works, the conceptual advance appears incremental.

  2. Limited evidence for the central claim. The controlled comparison in Table 4 covers only two datasets and shows modest gains, while VisA image AUROC decreases from 88.6 to 88.2. No run-to-run variability is reported. The broader experiments combine prototype updates with other changes, so they do not establish that empirical prototypes themselves provide a consistent transfer advantage.

Minor Weaknesses

  1. Limited industrial benchmark coverage. Compared with broader evaluations in prior ZSAD work, the industrial evaluation is relatively narrow. Additional datasets covering different inspection scenarios would strengthen the evidence for cross-domain generalization.
Questions for authors

Please address the concerns raised in the weaknesses above.

Reviewer_6xdLOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Empirical Representation Prototyping (ERP) for language-free zero-shot anomaly detection, replacing learnable visual prompts with EMA-based prototypes constructed from training features. It further introduces importance-aware prototype construction and cosine-based multi-scale feature aggregation to improve anomaly localization. While the method achieves competitive performance across multiple benchmarks, its contributions largely build on established techniques, and the experimental evidence is insufficient to substantiate the claimed advantages of empirical prototypes over learnable representations in terms of representativeness and cross-domain generalization.

Critical strengths
  1. This paper presented a practical framework for zero-shot anomaly detection. The EMA-based prototype construction provides a lightweight mechanism for maintaining normal and abnormal references.
  2. This work has an extensive ecperimental evaluation. The method is evaluated on 12 industrial and medical benchmarks and achieves competitive performance in both image- and pixel-level metrics.
Critical weaknesses
  1. Motivation is not well justified:

The paper emphasizes the limitations of text-prompt-based anomaly detection, but language-free ZSAD has already been explored by VisualAD and UniADet. The actual contribution is replacing learnable visual prompts with EMA-based prototypes. However, the paper does not provide sufficient evidence that empirical prototypes yield more representative visual features or better cross-domain generalization. Although the prototypes are not directly optimized through gradients, they are constructed from adapter features trained with source-domain supervision, so reduced source-domain bias is not guaranteed (claimed in line70, line 80-83, ...). In Table 4, replacing learnable prompts with ERP prototypes improves P-AUPR by only 1.0 and 0.6 percentage points on MVTec and VisA, respectively, while some other metrics deteriorate. More solid results for this hypothesis need to be presented.

  1. Methodological novelty is limited:

This work is composed by different components. However, most components rely on well-established techniques: language-free memory-retrieval approaches [1], EMA-based prototype updates [2, 3], hard-normal example selection [4], multi-scale feature aggregation [5, 6], and local neighborhood aggregation [5]. The overall framework appears to be an incremental engineering improvement rather than a substantial methodological innovation.

[1] Zero-Shot Anomaly Detection with Memory-Driven Retrieval

[2] Multi-Prototype Compactness and Boundary-Aware Synthesis for Unsupervised Anomaly Detection

[3] Learning with Mixture of Prototypes for Out-of-Distribution Detection

[4] Hard-Normal Example-aware Template Mutual Matching for Industrial Anomaly Detection

[5] Towards Total Recall in Industrial Anomaly Detection

[6] FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localization

Questions for authors
  1. Could the authors provide a computational efficiency comparison with existing ZSAD methods in terms of training/inference time, FLOPs, and GPU memory consumption?

  2. Since ERP does not use the CLIP text encoder, why is CLIP preferred over vision-only foundation models such as DINO or MAE?

  3. How does ERP address the limitations of CLIP in fine-grained anomaly localization, particularly for small defects, and could the authors report performance across different anomaly sizes?

Reviewer_xXh5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ERP for cross-domain zero-shot anomaly detection and segmentation. Using a frozen CLIP visual encoder and trainable adapters, it replaces directly learned normal/abnormal reference vectors with source-feature prototypes updated through exponential moving averages. IPC selects informative normal patches for prototype construction, while CMFA aggregates local features across multiple window sizes at inference. The method is trained with labeled source images and anomaly masks and evaluated on 12 industrial and medical datasets without target-domain adaptation.

Critical strengths
  • 1.ERP provides a simple, compact way to construct data-derived anomaly references, maintaining a fixed number of prototypes independent of the source dataset size.

  • 2.The evaluation covers 12 industrial and medical datasets. Compared with the reported UniADet baseline, mean pixel-level AUPR improves by 3.1 percentage points on industrial datasets and 1.4 on medical datasets.

  • 3.The main idea is clearly motivated. The direct comparison with learnable references and the component ablations provide useful evidence about the proposed design.

Critical weaknesses
  • 1.MRAD(ICLR 2026) is cited, but the paper does not explain how ERP differs from it. MRAD also uses normal and abnormal features from source data as references for detecting anomalies in unseen categories. The authors should explain the benefit of replacing multiple stored references with EMA prototypes and compare their accuracy.

  • 2.The paper does not clearly show why EMA prototypes work better on unseen datasets. Section 4.3 suggests that directly learned references may overfit the source data, but the cosine similarity between two reference vectors does not demonstrate overfitting. It remains unclear whether ERP helps because its prototypes capture more transferable visual patterns or because averaging past features makes the updates more stable.

  • 3.The feature-matching procedure needs clarification. The prototypes are built from adapted features, but Eq. (3) uses the original features, while the implementation paragraph specifies adapted features for medical datasets. The authors should explain which features are used in each setting and where CMFA is applied. Prototype initialization, update order, and training epochs should also be specified for reproducibility.

Questions for authors
  • 1.The motivation for using source features as references is reasonable. However, why does ERP summarize them into EMA prototypes? Could the authors explain the benefits of this choice?

  • 2.In Section 4.3, the authors suggest that ERP provides more representative prototypes and may reduce source-domain overfitting. How do the normal–abnormal reference cosine similarities and the results in Fig. 5 support this explanation? Could more stable EMA updates also contribute to the gains?

  • 3.In Table 4, do both methods use the same adapters, losses, and training settings? Is IPC used in the ERP row? Could the authors report the mean and standard deviation across multiple random seeds on MVTec and VisA?

  • 4.Does Eq. (3) use original or adapted features for prototype matching, and is this choice the same for industrial and medical datasets? Please clarify where CMFA is applied, when prototypes are updated, and how the training epochs and final checkpoint are chosen.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.