acceptodds
Withdrawn from ICLR 2027

Aligning Deviations, Not States, in CLIP-based Few-Shot Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Few-shot Generalist Anomaly Detection (GAD) detects and localizes anomalies from novel categories using only a few normal references without target-class retraining. Existing CLIP-based methods typically align query features directly with normal and anomalous textual semantics, while using normal references separately for visual matching. However, anomalies are inherently deviations from an expected normal state rather than absolute visual states, making state-based alignment sensitive to normal variations across regions and categories. We therefore exploit normal references in few-shot GAD to shift CLIP alignment from states to deviations. Specifically, we calibrate the query's semantic response with that of a matched normal reference, measuring how the query deviates from its matched normal state and relating this visual deviation to the semantic shift from normal to anomalous text. This naturally yields Residual-Residual Alignment. Based on this insight, we propose ResCLIP, which combines visual deviation magnitude with its semantic alignment for training-free anomaly scoring and further performs progressive spatial calibration followed by semantic refinement when auxiliary supervision is available. Experiments on multiple benchmarks demonstrate the effectiveness of ResCLIP in both training-free and fine-tune settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Anomaly Detection, Few-shot Anomaly Detection, Generalist Anomaly Detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_M4YcOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Res$^2$CLIP for few-shot generalist anomaly detection. It subtracts the response of matched normal visual references from query features and aligns this visual residual with a normal-to-anomalous text residual. A magnitude-plus-projection score supports training-free detection; with labeled auxiliary categories, two residual adapters perform spatial calibration and semantic refinement. Experiments on MVTec AD, VisA, BTAD, MPDD, and DTD-Synthetic report strong average pixel-level AP and PRO across 1-, 2-, and 4-shot settings, along with extensive ablations.

Critical strengths
  • The residual formulation is simple and interpretable, and separates deviation magnitude from alignment with anomaly semantics.
  • Evaluation spans five benchmarks and three shot counts, compares several CLIP-based methods, and includes training-free and fine-tuned variants with detailed component ablations.
Critical weaknesses
  • The evaluation does not ensure the stated class-disjoint generalist setting. The problem formulation requires Cseen∩Cunseen=∅C_{seen} \cap C_{unseen}=\varnothing (Sec. 3), but the protocol trains on the MVTec AD test set for VisA and uses VisA for MVTec AD (Sec. 4.1; Table 5). Both datasets include the capsule(s) category, so the source training set contains labeled examples from a category also evaluated as unseen in the target dataset. This is a category-level overlap even though the images come from different datasets. It weakens the central claim of generalization to unseen classes; the split should exclude overlapping categories or report results separately. The category overlap is documented in the MVTec AD paper and VisA dataset paper.
  • The shared text residual may be too narrow for the claimed open-set anomaly semantics. The method uses a fixed generic-object prompt with a single anomalous state word (“damaged”; Appendix C), and Stage II encourages anomalous residuals across categories to align with that same direction. Magnitude scoring partly mitigates this limitation, but the semantic-refinement claim is not validated across distinct anomaly semantics, and the paper itself notes that one fixed text residual may not capture anomaly diversity (Appendix I).
  • The likelihood argument does not derive the score used by the method. Appendix B assumes isotropic Gaussian normal and anomalous residuals with a mean shift along the text direction. Under those assumptions, the log-likelihood ratio is proportional to ∥Rv∥22\|R_v\|_2^2 plus a text projection (Eq. 28), whereas the proposed score uses ∥Rv∥2\|R_v\|_2 plus a weighted positive projection (Eq. 5). The assumptions are not empirically checked, and the actual score is therefore not justified as the derived likelihood ratio. The paper should present this as heuristic motivation or validate the assumptions and score form.
Questions for authors
  1. Can you report the results after removing the overlapping capsule(s) category from the auxiliary source data, so that training and target classes satisfy the disjointness in Sec. 3?
  2. What evidence supports using one fixed “damaged object” text residual across the evaluated object and texture categories? Does it remain effective when the anomaly prompt or semantic direction changes?
  3. Is the likelihood derivation intended only as motivation? If it is intended as justification, how do the Gaussian assumptions and squared-magnitude likelihood ratio support the norm-based score in Eq. 5?
Reviewer_vyLtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Res²CLIP to address the lack of a unified reference between visual queries and textual semantics in few-shot anomaly detection. The method takes the visual residual between query features and matched normal references as the detection target, while defining the semantic direction through the difference between normal and anomalous text prototypes. The training-free version combines residual magnitude and directional projection for anomaly scoring, while the supervised version further introduces spatial residual calibration and radial and tangential adaptation.

Critical strengths
  1. The paper explicitly formulates the reference-calibrated text-image response as the inner product between visual and textual residuals, providing a unified and interpretable formulation for training-free scoring and subsequent optimization.

  2. The experiments cover both training-free and fine-tuning settings, and provide relatively systematic quantitative and visual analyses of calibration, magnitude/projection scoring, the two-stage loss, and adapter types.

  3. The paper is evaluated on five industrial anomaly detection benchmarks and achieves competitive results.

Critical weaknesses
  1. The paper claims that the residual space reduces category dependence and suppresses variations in normal regions. However, the existing t-SNE visualizations and response distributions provide mainly descriptive evidence, lacking direct quantitative evaluations of inter-category variance, normal matching errors, background false positives, and cross-category directional consistency.

  2. In few-shot methods, the selection of support samples often has a significant impact on final performance. The paper does not sufficiently report performance variations caused by different support set samplings, making it difficult to demonstrate the method's effectiveness across different support sets.

  3. The image-level AUROC does not consistently outperform competing methods across all datasets. For example, performance degradation on certain texture datasets is not analyzed, making the claim of general effectiveness overstated.

Questions for authors
  1. Please report the mean and standard deviation across multiple support set samplings for all few-shot results, and clarify whether all methods use exactly the same support images.

  2. Could you quantify the inter-category variance, false positives in normal regions, and reference matching errors before and after calibration to directly validate the claim of reduced category dependence?

Reviewer_8xzGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Res2CLIP, few-shot anomaly detection method, where a model detects and localizes anomalies in unseen categories using only a few normal reference images without target-class retraining. The authors observe that existing CLIP-based methods (e.g., WinCLIP) compare query features directly with normal and anomalous text embeddings, so the response is affected by normal variations across regions and categories. They propose to calibrate the query’s text–image response by subtracting that of a matched normal reference. Based on this, Res2CLIP scores each patch by combining the magnitude of the visual residual with its positive projection onto the text residual. When auxiliary supervision from seen categories is available, it further refines the residuals with lightweight adapters. Experiments on MVTec-AD, VisA, BTAD, MPDD, and DTD-Synthetic under 1/2/4-shot settings show the best average pixel-level AP and PRO among the compared methods, while image-level AUROC remains comparable rather than the best.

Critical strengths
  • The proposed method is simple and intuitive: relating the visual deviation from a matched normal reference to the semantic shift from normal to anomalous text. It achieves strong localization performance across various benchmarks and shot settings.

  • The figures and tables are well organized and easy to follow. The qualitative comparisons, such as the anomaly maps in Fig. 4, help in understanding the behavior of the method.

Critical weaknesses
  • The contribution of the residual-residual alignment seems limited. Without training, the projection term alone performs worse than the residual magnitude alone, and adding it barely changes P-AP. The magnitude itself is essentially the distance to the matched normal features, as in memory-bank methods. The projection becomes much more effective after fine-tuning, where stage 2 explicitly trains anomalous residuals toward the text residual, so it is unclear whether the text semantics actually matter or the text residual just serves as a fixed target direction.

  • The effect of the train–test distribution is not studied. All fine-tuned results use a single auxiliary-training setup, and the gains over the training-free mode vary a lot across datasets: they are large on DTD-Synthetic, which resembles the texture categories in MVTec AD, but image-level AUROC drops on MPDD. Given the claim that the residual space reduces category dependence, it would be helpful to see how the results change with different auxiliary data.

Questions for authors
  • Could the authors clarify whether the semantics of the text residual matter after fine-tuning? Since stage 2 explicitly trains anomalous residuals toward the text residual, it seems possible that any fixed direction could serve a similar role.

  • Could the authors clarify how much of the improvement comes from the calibration itself?

  • How do the fine-tuning results change with a different auxiliary training set (e.g., training on VisA and evaluating on BTAD)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.