acceptodds
Withdrawn from ICLR 2027

ReCal-CLIP: Patch-Token Recalibration for Cross-Domain Zero-Shot Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Cross-domain zero-shot anomaly detection aims to identify and localize abnormal regions in unseen target domains without using target-domain anomalous samples, labels, or masks for optimization. Existing CLIP-based methods benefit from vision-language priors, but their patch-level representations are often not calibrated for context-dependent defects: a patch may look ambiguous in isolation, while its abnormality becomes clear only when compared with surrounding normal patterns. This limitation is particularly problematic for industrial inspection, where defects can be small, low-contrast, spatially fragmented, or visually defined by deviations from normal texture. We propose ReCal-CLIP, a patch-token recalibration framework that addresses this issue before anomaly scoring. The core idea is to recalibrate image-side CLIP tokens by routing anomaly-suspect evidence, contrasting it with normal-context cues, and injecting the resulting update through confidence-gated residual fusion. With the recalibrated tokens, ReCal-CLIP uses a lightweight token-space consistency step and a training-only Sampled Discriminative Patch Alignment (SDPA) objective that applies a mask-guided similarity-margin constraint during source-domain training. Experiments on industrial benchmarks, with supplementary medical-domain evaluation, show competitive image-level detection and pixel-level localization performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Zero-Shot Anomaly Detection, Patch-Token Recalibration, Cross-Domain Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xQ4aOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ReCal-CLIP for cross-domain zero-shot anomaly detection. With a frozen CLIP backbone, the method combines Hybrid Recalibration for local visual feature refinement, SCPA for spatial aggregation, and a training-only SDPA similarity-margin objective. The model is trained on labeled source-domain data and evaluated on unseen industrial and medical target datasets for anomaly detection and localization.

Critical strengths
  1. The paper presents a clear framework for local feature recalibration in cross-domain zero-shot anomaly detection, with extensive evaluation on industrial and medical datasets.
  2. The ablation studies support the effectiveness of the main components, and the paper clearly describes its evaluation protocol and methodological limitations.
Critical weaknesses
  1. The proposed method exhibits substantial structural overlap with FB-CLIP in its routing mechanism, three-branch visual processing, and context-residual transformation. Although the appendix acknowledges that some components are inherited, the current contribution statements do not clearly distinguish inherited components from newly introduced designs. Moreover, a direct comparison against the complete FB-CLIP framework under the same training and evaluation protocol is missing, making it difficult to assess the incremental contribution of the proposed method beyond FB-CLIP.
  2. Appendix B.2 also introduces an additional transformation that is similar to FB-CLIP's Background Suppression. If the performance gain mainly comes from this extra transformation, while the main text attributes the improvement primarily to the three-branch recalibration, then the explanation of the method's contribution is incomplete.
  3. Appendix B.2 constructs the context reference using a fixed selection of the first half of the token sequence, but does not explain why these positions should better represent normal background context. Although FB-CLIP compares using all tokens with using half of them, this experiment does not disentangle the effects of token count from those of token selection position.
  4. In Section 3.5, the sampling rule K=min⁡(K_0,|I_n |,|I_a |) implies that small defects may limit the effective sample count K to values far below K_0. Consequently, increasing K_0 may leave the actual sample count unchanged for many images. For each K_0, please report the distribution of effective K and the proportion of images that contribute zero SDPA loss.
  5. The interpretation of normal context in SUS requires further clarification. According to Eqs. (6)–(8), highly suspicious keys are not excluded from the normal-context aggregation, and keys within the same routing group receive identical attention weights. Moreover, when a query’s routing value is 1, the output coefficient of the separately computed normal-context aggregate becomes zero. It therefore remains unclear how this module provides a targeted normal reference for the most suspicious patches.
Questions for authors

See the above weakness.

Reviewer_ft5COpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents ReCal-CLIP for cross-domain zero-shot anomaly detection and localization. The model is trained using labeled source-domain images and anomaly masks, then applied to target datasets without target-domain optimization. Its central idea is to refine CLIP’s visual tokens before comparing them with learned normal and anomalous text prototypes. The framework combines discrepancy-based patch routing, three visual processing branches, confidence-gated residual adaptation, and spatial patch aggregation. A training-only Sampled Discriminative Patch Alignment (SDPA) loss encourages separation between sampled normal and anomalous patch groups. Predictions aggregate information from multiple visual layers and branches. Experiments on industrial and medical benchmarks evaluate the framework’s cross-domain detection and localization performance.

Critical strengths
  • 1.This paper addresses a practically problem: for local defects that require the surrounding context for recognition, a single CLIP patch feature may not be effective for detection.

  • 2.The experiments cover multiple industrial and medical datasets, report image-level and pixel-level metrics, and provide module ablation and qualitative comparisons.

Critical weaknesses
  • 1.The methodological novelty relative to FB-CLIP is not sufficiently clear. Anomaly-suspect routing and the IDE/SUS/STR branches closely resemble the foreground routing and ID/SEM/SPA views in FB-CLIP (CVPR 2026). ReCal-CLIP introduces a different residual update, SCPA, and SDPA, but the paper does not clearly distinguish its new contributions from closely related visual processing components in FB-CLIP.

  • 2.The fine-grained benefit of SDPA needs stronger justification. Based on Eqs. (14)–(16), SDPA can be rewritten as a margin-based loss on the squared distance between the normal and anomalous group means. This does not directly enforce separation between individual difficult patch pairs, raising questions about how well it captures diverse and subtle anomalies in unseen domains.

  • 3.The experiments do not sufficiently isolate the added contributions. FB-CLIP is missing from the main comparison despite its close methodological relationship, and computational overhead is also not reported, despite the multiple layers and branches used at inference.

  • 4.The main methodological description does not clearly present the complete prediction pipeline. Important operations, including the context-residual transformation and multi-layer, multi-branch logit aggregation,are mainly described in Appendix. Their relationship to patch-token refinement and CLS-based image-level prediction should be made explicit in Section 3 and Figure 2.

Questions for authors
  • 1.What are the main methodological differences between Hybrid Recalibration and FB-CLIP's visual feature enhancement? A controlled comparison with FB-CLIP, including incremental ablations of the new components, would help clarify the contribution. Please also report trainable parameter counts and inference latency.

  • 2.In Eq. (8), why does the explicit normal-context term receive zero weight for the most suspicious patches mi=1m_i=1? How do these patches benefit from normal-context calibration under this design?

  • 3.Since SDPA can be reduced to a separation constraint between sampled group means, how does it help distinguish challenging individual patches and generalize to unseen anomaly patterns?

  • 4.Could the authors clarify in the main text how the outputs from multiple visual layers and branches are aggregated into the final predictions? In particular, through which operations does patch-token recalibration affect the CLS-based image-level score?

Reviewer_JYeWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a patch-token recalibration framework, ReCal-CLIP, for zero-shot anomaly detection. It introduces anomaly-suspect routing to identify patches requiring context-aware refinement, followed by a three-branch Hybrid Recalibration module that integrates original patch features, suspect evidence, and local structural information through confidence-gated residual fusion. This work further incorporates spatially coherent patch aggregation and a training-only Sampled Discriminative Patch Alignment objective. Experiments on industrial and medical benchmarks demonstrate that ReCal-CLIP gains competitive performance in image-level anomaly detection and pixel-level localization.

Critical strengths

1.The proposed framework provides a potentially useful direction for enhancing context-aware anomaly localization by improving image-side patch representations rather than relying solely on text prompt optimization.

2.The experimental evaluation covers multiple industrial and medical benchmarks and reports competitive results.

Critical weaknesses

1.The routing coefficients are derived from several heuristic feature discrepancies with fixed weights and thresholds, but the paper does not directly validate whether they reliably identify anomaly-suspect regions. Moreover, according to Eqs. (6)–(8), when the routing coefficient reaches its maximum value, the normal-context attention becomes uniform and its contribution to the SUS output vanishes. It should be clarify how the proposed mechanism explicitly compares highly suspicious patches against normal context, as motivated in the paper.

2.The experiments do not establish whether the specific routing, three-branch recalibration, and gated residual fusion designs are necessary. The author should consider to add comparision with uniform or random routing, standard visual adapters, and simpler spatial attention or aggregation mechanisms.

3.The current ablations do not fully disentangle the contribution of these components from that of the proposed visual recalibration mechanism.

Questions for authors

See comments above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.