acceptodds
Under review as a conference paper at ICLR 2027

DeltaAD: Visual Delta Modeling for Zero-Shot Anomaly Detection

11 views

Abstract

Zero-shot anomaly detection (ZSAD) aims to identify anomalies in unseen domains without access to target-specific training data. To alleviate cross-domain visual discrepancies, existing ZSAD methods typically leverage pretrained vision or vision-language models to characterize visual distributions or learn text prompts from auxiliary data. However, their cross-domain generalization remains limited under substantial visual distribution shifts. To address this challenge, we propose DeltaAD, a novel framework that detects anomalies by amplifying anomaly-related differences while suppressing visual variations shared by normal and anomalous responses through differential similarity. Our train-free variant, DeltaAD-TF, freezes the CLIP image encoder and constructs separate memory banks of normal and anomalous region-pooled features from auxiliary data. During inference, it contrasts retrieval evidence from the two memory banks under shared visual contexts, highlighting relative differences between normal and anomalous responses while suppressing their shared variations. Building on this principle, our training-based variant, DeltaAD-LE, learns a residual relation that characterizes anomalies relative to retrieved normal visual explanations. Extensive experiments on 15 industrial and medical benchmarks demonstrate the strong zero-shot generalization of DeltaAD for both image-level anomaly detection and pixel-level anomaly localization under train-free and training-based settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Anomaly detection, Zero-shot anomaly detection, Differential similarity, CLIP

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Rb8gOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a new method for zero-shot anomaly detection named DeltaAD. It uses a memory bank of features gathered from a labelled auxiliary dataset. Based on the features in the memory bank, it calculates the patch features. The main idea is that the anomaly score should be calculated as the difference between the normal evidence and the anomalous evidence. The model is then tested on standard datasets.

Critical strengths
  • The usage of residuals is well motivated
  • I like the analysis of the method's performance based on the auxiliary dataset used for training. It also shows the importance of using the right one.
  • The training-free results are good
Critical weaknesses

Major Weaknesses:

  • The method is not compared to the current state of the art, AnomalyVFM [1] (with the RADIO backbone). AnomalyVFM outperforms it on almost all industrial datasets they test when paired with the correct backbone. I do not see a reason to compare it only to methods that use CLIP as a backbone; the main goal is to improve zero-shot anomaly detection overall, not just CLIP-based models. This holds even more for this method, as it is not tied down to using CLIP as a backbone.
  • Only CLIP is examined as a backbone, raising the question of whether the findings about the residuals are general or specific only to CLIP. Stronger backbones, such as RADIO [2] and DINOv3 [3], should be explored (as they show the best performance in the related literature).

Minor Weaknesses:

  • The U in the image-level detection differs from the U in the starting motivation, if I understood correctly? During the reading, this confused me a lot.
  • What are the results if we use the maximum function in Equation 2?
  • Similarly, what are the results if we take the largest patch residual (or the mean of top % residuals) in Equation 3?
  • The residual learning module is complex and seems quite hand-crafted

[1] Fučka, M., et al. (2026). AnomalyVFM--Transforming Vision Foundation Models into Zero-Shot Anomaly Detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 35555-35566).

[2] Ranzinger, M., Heinrich, G., Kautz, J., & Molchanov, P. (2024, June). Am-radio: Agglomerative vision foundation model reduce all domains into one. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 12490-12500). IEEE.

[3] Oriane Siméoni et al. (2026). DINOv3. In Transactions on Machine Learning Research.

Questions for authors
  • How does DeltaAD perform when using other backbones, such as RADIO and DINOv3? Does the motivation also hold for other backbones?
  • How does the construction of Equation 2 and Equation 3 affect the final downstream performance?
7 comments

We sincerely appreciate your thoughtful and constructive feedback. Thank you for taking the time to carefully review our work and share your valuable comments and suggestions.

Q1: How does DeltaAD compare with AnomalyVFM under matched RADIO and CLIP backbones, particularly on industrial benchmarks?

A1: Following the reviewer's suggestion, we have compared DeltaAD with AnomalyVFM using matched RADIO and CLIP backbones on the same 15 industrial and medical benchmarks evaluated in our paper. The results challenge the reviewer's assessment that AnomalyVFM generally outperforms DeltaAD on industrial datasets.

Table R3-A. Backbone-matched comparison across the 15 benchmarks of this paper (average metrics, %; bold indicates the best result within each backbone).

MethodP-AUROCPROI-AUROCI-AP
AnomalyVFM-RADIO93.086.391.889.2
DeltaAD-TF-RADIO94.088.989.287.6
DeltaAD-LE-RADIO95.184.193.291.8
AnomalyVFM-CLIP88.981.188.686.7
DeltaAD-LE-CLIP93.782.391.990.4

DeltaAD-LE-RADIO achieves higher P-AUROC on five of six industrial benchmarks, while DeltaAD-TF-RADIO also achieves higher PRO. Under CLIP, DeltaAD-LE outperforms AnomalyVFM in P-AUROC across all 12 pixel-level benchmarks.

Additionally, using AnomalyVFM's released synthetic auxiliary dataset improves DeltaAD-TF's average P-AUROC from 87.5% to 90.3% without modifying the differential similarity formulation. We further validate DeltaAD's generalization across visual backbones in Q2.

AnomalyVFM results are primarily obtained from its official repository, supplemented by our evaluations on CVC-300 and HIS. Detailed DeltaAD results are provided in Sec. 4.3 (Table 2) and Appendix E.11 (Tables 29–30).

Q2: Does DeltaAD's differential modeling, particularly its learned residual relation, generalize beyond CLIP to stronger visual backbones such as RADIO and DINOv3?

A2: Following the reviewer's suggestion, we have extended both DeltaAD-TF and DeltaAD-LE to DINOv3 ViT-L/16 and RADIO v2.5-L, evaluating them on the same 15 benchmarks as our original experiments. The results demonstrate that DeltaAD's differential modeling generalizes beyond CLIP, with RADIO improving all four evaluation metrics for both variants.

Table R3-B. Backbone generalization of DeltaAD-TF and DeltaAD-LE on the same 15 benchmarks (average metrics, %; bold indicates the best result within each variant).

VariantBackboneP-AUROCPROI-AUROCI-AP
DeltaAD-TFCLIP87.571.385.384.4
DINOv388.075.455.557.2
RADIO94.088.989.287.6
DeltaAD-LECLIP93.782.391.990.4
DINOv395.084.591.289.3
RADIO95.184.193.291.8

Notably, DeltaAD-LE maintains strong performance across all three backbones, demonstrating that its learned normal-explanation residual relation is not specific to CLIP representations. RADIO further improves both variants, while DINOv3 also improves their pixel-level performance.

These experiments retain the original differential modeling mechanisms with frozen visual encoders and further demonstrate the strong generalizability of DeltaAD across different pretrained visual backbones, supporting the transferability of its differential modeling framework beyond CLIP. Complete results are provided in Sec. 4.3 (Table 2) and Appendix E.11 (Tables 29–30).

Q3: Clarification of the notation ambiguity between the common evidence component and the image-level aggregation in DeltaAD-TF.

A3: We have resolved this notation ambiguity in the revised manuscript. In Sec. 3.1.1, Ui=(EiA+EiN)/2U_i=(E_i^A+E_i^N)/2 denotes the common component of normal and anomalous evidence, whereas the original Uxs(T)U_x^s(T) in Eq. (3) denotes image-level evidence aggregation.

We have renamed the latter as Gxs(T)G_x^s(T) and updated Eq. (4) accordingly:

SxTF=GxA(T)−GxN(T).S_x^{\mathrm{TF}}=G_x^A(T)-G_x^N(T).

This is purely a notational change and does not affect the method or experimental results.

Q4: How does replacing the aggregation operations in Eq. (2) with maximum pooling affect DeltaAD-TF's performance?

A4: Following the reviewer's suggestion, we evaluated two Max-based alternatives to Eq. (2): Support-level Max and Context-level Max. The results demonstrate that the proposed aggregation outperforms both alternatives, highlighting the importance of integrating retrieval evidence across multiple shared visual contexts.

Table R3-C. Support- and context-level Max ablations of DeltaAD-TF on MVTec-AD and HeadCT (I-AUROC, %).

AggregationMVTec-AD I-AUROCHeadCT I-AUROC
DeltaAD (Eq. 2)88.589.9
Support-level Max82.876.4
Context-level Max65.565.2

Support-level Max reduces performance on both datasets, while Context-level Max causes substantially larger degradation. These results indicate that aggregating complementary evidence across multiple query-relevant visual contexts is particularly important for reliable differential response estimation, supporting our aggregation design in Eq. (2).

Q5: How does replacing the image-level aggregation in Eq. (3) with Max or Top-k% pooling of patch-level differential responses affect performance?

A5: Following the reviewer's suggestion, we compared the proposed two-state LogMeanExp aggregation with Max, Top-1%, and Top-10% pooling. The results demonstrate that our formulation achieves the best average image-level detection performance across all nine benchmarks, validating the effectiveness of aggregating normal and anomalous evidence separately before computing their difference.

Table R3-D. Image-level pooling ablation of DeltaAD-TF across nine image-level benchmarks (average metrics, %).

AggregationI-AUROC (%)I-AP (%)
DeltaAD (Eq. 3)85.384.4
Max84.682.3
Top-1%83.883.5
Top-10%79.880.6

Unlike the alternatives, which directly pool the largest patch-level differential responses, our method separately aggregates normal and anomalous evidence using LogMeanExp (T=0.15T=0.15) before computing their difference. This avoids relying solely on extreme patch responses and incorporates evidence from multiple patches, supporting the proposed image-level aggregation design.

Complete results are provided in Appendix E.6 (Table 23).

Q6: Is DeltaAD-LE's residual relation learning module unnecessarily complex or hand-crafted?

A6: DeltaAD-LE employs a lightweight, learnable relation module rather than hand-crafted residual scoring. Controlled ablations demonstrate its effectiveness over simpler alternatives, with only 6.99M trainable parameters.

Relation modeling. The module combines query features q\mathbf q, normal explanations q^N\hat{\mathbf q}_N, signed and absolute residuals (r,∣r∣)(\mathbf r,|\mathbf r|), and retrieval statistics ρ\boldsymbol{\rho}. Their contributions are learned by the relation head rather than assigned fixed weights.

Table R3-E. Relation-input ablation of DeltaAD-LE on MVTec-AD (AUROC/PRO/AP, %).

Relation InputP-AUROCPROI-AUROCI-AP
Query-only91.682.292.496.1
Residual ∥r∥2\|\mathbf r\|_291.784.192.596.7
Full Relation93.187.294.397.8

The full representation leads on all four MVTec-AD metrics. Learnable ablations zero-fill removed inputs to preserve the relation-head architecture.

Model complexity. We compare DeltaAD-LE with training-based methods using the same frozen CLIP backbone:

Table R3-F. Accuracy and trainable-model size of CLIP-based methods across the 15 benchmarks.

MethodI-AUROCP-AUROCTrainable ParamsModel Size (MB)
MRAD-CLIP89.592.39.49M54
VisualAD-CLIP88.389.531.52M126.12
DeltaAD-LE (Ours)91.993.76.99M27.99

DeltaAD-LE achieves the highest average detection performance with fewer trainable parameters than MRAD-CLIP and VisualAD-CLIP. Only the retrieval projections and relation head are optimized, while the visual encoder remains frozen. These results support the effectiveness and parameter efficiency of our design (Sec. 4.3, Fig. 5; Appendix E.2, Table 15; Appendix D, Table 11).

Dear Reviewer Rb8g,

Thank you again for your time and constructive feedback. We sincerely appreciate your valuable suggestions.

We would like to briefly summarize the key findings from our response:

  • Comparison with AnomalyVFM: Under matched CLIP and RADIO backbones, DeltaAD achieves competitive and, on several evaluation metrics, superior performance compared with AnomalyVFM across industrial and medical benchmarks. These results provide evidence that DeltaAD remains competitive with recent state-of-the-art approaches under comparable backbone settings.
  • Backbone generalization: Evaluations with RADIO and DINOv3 demonstrate that our differential modeling framework generalizes beyond CLIP, with DeltaAD-LE maintaining strong performance across different pretrained visual representations.
  • Aggregation and relation modeling: Controlled ablations validate the effectiveness of our aggregation strategies over Max/Top-k alternatives and demonstrate the benefits of learnable residual relation modeling over simpler scoring approaches. We have also clarified the notation ambiguity in the revised manuscript.

We hope these additional experiments and clarifications help address your concerns. If any questions remain, we would be more than happy to provide further clarification.

Thank you again for your time and consideration.

Best regards,
Authors

Reviewer_QsnAOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DeltaAD studies cross-domain zero-shot anomaly detection using auxiliary visual memories. Its train-free variant contrasts normal and anomalous retrieval evidence under shared visual contexts. Its learned variant retrieves a normal reference and predicts anomalies from query–reference relations. Experiments on 15 industrial and medical datasets show improvements over the reported retrieval and CLIP-based baselines.

Critical strengths
  1. The paper presents a clear line of reasoning. The identified problem and the proposed solutions are intuitive, and the overall experimental findings are consistent with the design rationale.

  2. Both variants achieve meaningful improvements. Mean image/pixel AUROC increases from 79.2/85.3 to 85.3/87.5 over MRAD-TF, and from 89.5/92.3 to 91.9/93.7 over MRAD-CLIP.

Critical weaknesses
  1. The distinction from MRAD requires clarification, particularly for the train-free variants. Both methods perform anomaly classification through memory retrieval, and MRAD-TF already compares normal and anomalous entries through softmax-normalized label retrieval. What is the fundamental difference in DeltaAD-TF beyond the routing, normalization, and aggregation choices? An explicit equation-level comparison and a worked retrieval example would help establish whether differential scoring introduces a substantive conceptual advance. This distinction is important for assessing novelty.

  2. The paper presents DeltaAD-TF and DeltaAD-LE as a unified differential similarity modeling framework, but their mechanisms differ substantially. DeltaAD-TF explicitly contrasts normal and anomalous retrieval evidence, whereas DeltaAD-LE models anomalies through the residual between a query and its retrieved normal reference. The connection between these mechanisms, and what makes them instances of the same framework, needs further explanation.

  3. The normal-only memory in DeltaAD-LE is motivated by the difficulty of covering unseen anomaly patterns with an anomalous memory. However, this motivation lacks direct experimental validation. A controlled comparison with a DeltaAD-LE variant using both normal and anomalous memories, under matched training and model capacity, would clarify whether excluding anomalous references actually improves cross-domain generalization.

  4. The key component ablations are limited to MVTec-AD, ISIC, and HeadCT. Given the evaluation on 15 datasets, it is unclear whether these findings generalize beyond the selected benchmarks. The authors should explain the selection criteria and extend the main ablations to additional datasets.

Questions for authors

Please address the questions and clarification requests raised in the Critical Weaknesses above.

4 comments

We sincerely appreciate your thoughtful and constructive feedback. Thank you for taking the time to carefully review our work and share your valuable comments and suggestions.

Q1: What fundamentally distinguishes DeltaAD-TF from MRAD-TF's softmax-based normal–anomalous evidence comparison beyond routing and aggregation choices?

A1: The key contribution of DeltaAD-TF is constructing context-matched differential evidence to mitigate interference from cross-domain visual variations.

MRAD-TF directly retrieves normal and anomalous label evidence from the global auxiliary memory (MRAD, Eq. 8):

\left[\mathrm{softmax}(Q_{\mathrm{pat}}K_{\mathrm{pat}}^\top/\tau)V_{\mathrm{pat}}\right]_{n/a}.$$ In contrast, DeltaAD-TF constructs both responses under shared query-relevant visual contexts and identical routing weights (our Eq. 2): $$E_i^s=\log\sum_{c\in\mathcal C_i}w_{ic} \exp\left( \log\left( \frac{1}{|\mathcal M_c^s|} \sum_{\mathbf{k}\in\mathcal M_c^s} \exp\left(\frac{\mathbf{x}_i^\top\mathbf{k}}{\tau_p}\right) \right)\right),\quad s\in\{N,A\}.$$ $$\Delta_i=E_i^A-E_i^N.$$ **Illustrative example.** For a normal query with retrieval logits $C_1:(N=4,A=3)$ and $C_2:(N=0,A=4)$, and routing weights $w=(0.815,0.185)$, MRAD-TF produces an anomaly probability of 0.573, whereas DeltaAD-TF yields 0.372. This illustrates how context-matched retrieval reduces interference from irrelevant anomalous matches. **Experimental evidence.** **Table R2-A. MRAD-TF comparison and fixed-routing evidence ablation on MVTec-AD (AUROC, %).** | Comparison | Method / Evidence | P-AUROC (%) | I-AUROC (%) | | :--------------------- | :---------------- | ----------: | ----------: | | MRAD comparison | MRAD-TF | 86.7 | 79.0 | | MRAD comparison | **DeltaAD-TF** | **88.0** | **88.5** | | Fixed-routing ablation | $E^A$ | 82.9 | 86.8 | | Fixed-routing ablation | **$E^A-E^N$** | **88.0** | **88.5** | The fixed-routing ablation confirms that differential evidence improves performance even without changing the routing mechanism (Tables 1 and 3). We have clarified this distinction in the revised Introduction and Sec. 3.1.2. **Q2: What unifies DeltaAD-TF's normal–anomalous evidence comparison and DeltaAD-LE's query–normal residual learning within the same differential similarity framework?** **A2:** The central motivation of DeltaAD is to **mitigate interference from cross-domain visual variations through differential modeling of query-relevant visual representations**, thereby highlighting anomaly-related differences while suppressing irrelevant visual variations. DeltaAD-TF and DeltaAD-LE share this motivation but implement it through different differential modeling mechanisms. Specifically, **DeltaAD-TF** uses query features to identify relevant visual contexts and constructs normal and anomalous retrieval responses under the same selected contexts and shared routing weights. It then computes the difference between these two responses to capture anomaly-related variations (Eq. 2). Building upon this principle, **DeltaAD-LE** introduces learnable query and key projectors in the training-based setting. The query projector maps query features into a learned representation, while the key projector maps normal memory features into the same retrieval space. It then retrieves a query-matched normal visual explanation and learns the residual relation between the query representation and its normal explanation to characterize anomalies (Eqs. 5–8). The normal-only memory design of DeltaAD-LE is motivated by the diversity of anomalous patterns, which are difficult to cover with auxiliary anomalous references. Controlled experiments further demonstrate that normal-only retrieval outperforms dual-memory retrieval under matched model capacity and training conditions (see Q3 and Appendix E.9, Table 27). We have clarified this unified motivation and the two implementations in the revised Introduction and Sec. 3.1.2 (Approach Overview).

Q3: Does DeltaAD-LE's normal-only memory improve cross-domain generalization compared with dual-memory retrieval under matched training and model capacity?

A3: Following the reviewer's suggestion, we have conducted controlled experiments comparing normal-only and dual-memory retrieval. The results demonstrate that normal-only retrieval outperforms dual-memory retrieval across all six evaluated metrics on four unseen datasets, even under matched model capacity and training conditions.

Specifically, Dual-memory additionally retrieves 931 anomalous region prototypes from VisA. To ensure a fair comparison, the capacity-matched Normal-only variant replaces the anomalous-reference features with zeros, retaining the same 5,384-dimensional relation-head input, architecture, and training protocol. The original Normal-only model uses a 3,076-dimensional input.

Table R2-B. Normal-only versus dual-memory retrieval across four unseen datasets (%, Appendix E.9). Bold compares the capacity-matched variants.

DatasetMetricOriginal Normal-onlyMatched Normal-onlyDual-memory
MVTec-ADPRO87.286.885.6
MVTec-ADI-AUROC94.394.193.2
DTD-SyntheticPRO94.593.291.4
DTD-SyntheticI-AUROC96.596.194.1
ISICPRO87.687.186.2
HeadCTI-AUROC98.798.497.8

Since anomalous patterns vary substantially across unseen categories and domains, auxiliary anomalous references may introduce mismatched evidence. These results directly support the normal-only memory design of DeltaAD-LE (Appendix E.9, Table 27).

Q4: Do the key component ablations generalize beyond the originally selected benchmarks, and what motivates the dataset selection?

A4: Following the reviewer's suggestion, we have extended all three key component ablations to four additional datasets: DTD-Synthetic, BTAD, HIS, and Endo. The original benchmarks were selected to represent industrial anomaly localization (MVTec-AD), medical anomaly localization (ISIC), and medical image-level detection (HeadCT). All additional evaluations follow the same cross-dataset zero-shot protocol without target-domain adaptation.

Table R2-C. Differential-response ablation on DTD-Synthetic, BTAD, HIS, and Endo (AUROC/PRO, %; — indicates undefined PRO).

Dataset / MetricEAE^AEA−ENE^A-E^N
DTD-Synthetic / I-AUROC91.993.5
DTD-Synthetic / PRO79.382.2
BTAD / I-AUROC88.981.0
BTAD / PRO58.435.5
HIS / I-AUROC64.865.0
Endo / PRO—71.2

Table R2-D. Local context routing ablation on DTD-Synthetic, BTAD, HIS, and Endo (AUROC/PRO, %).

Dataset / MetricIndependentShared weighted
DTD-Synthetic / I-AUROC64.993.5
DTD-Synthetic / PRO80.882.2
BTAD / I-AUROC58.081.0
BTAD / PRO34.035.5
HIS / I-AUROC62.465.0
Endo / PRO67.271.2

Table R2-E. Normal-explanation relation ablation on DTD-Synthetic, BTAD, HIS, and Endo (AUROC/PRO, %).

Dataset / MetricResidual ∥r∥2\|\mathbf r\|_2Full
DTD-Synthetic / I-AUROC94.096.5
DTD-Synthetic / PRO88.094.5
BTAD / I-AUROC91.395.3
BTAD / PRO57.481.3
HIS / I-AUROC65.264.9
Endo / PRO74.075.2

The extended ablations support the generalizability of the proposed components. Shared weighted routing achieves the best performance on all six additional metrics, while the full relation representation leads on five of six. Differential response also improves performance on DTD-Synthetic and HIS, although anomaly-only evidence performs better on BTAD. Complete results are provided in Appendix E.1 (Tables 12–14).

Thank you for the detailed rebuttal and additional experiments, but DeltaAD remains closely based on MRAD’s memory-retrieval framework. I will therefore keep my rating unchanged.

Dear Reviewer QsnA,

Thank you for your time and careful consideration of our response and additional experiments.

We would like to briefly clarify that, although DeltaAD shares the auxiliary memory retrieval paradigm with MRAD, our core contribution lies in differential modeling rather than memory retrieval itself. Specifically, DeltaAD-TF constructs context-matched normal and anomalous evidence under shared visual contexts, while DeltaAD-LE learns query–normal residual relations using normal-only memory. These mechanisms aim to mitigate interference from cross-domain visual variations, as further supported by our controlled ablations.

We sincerely appreciate your constructive feedback and respect your assessment. Thank you again for your time and consideration. If you have any remaining questions or concerns, we would be more than happy to provide further clarification.

Best regards,
Authors

Reviewer_1TwJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DeltaAD, a vision-only framework for cross-domain zero-shot anomaly detection. The authors first show that cross-dataset normal and anomalous responses share common visual variations. Two variants are proposed. First, DeltaAD-TF constructs shared visual contexts with K-means and computes the differential response between normal and anomalous evidence. Second, DeltaAD-LE retrieves a normal visual explanation from the normal memory bank and trains a lightweight relation head to model the residual relationship between the query and its normal explanation. Extensive experiments across 15 industrial and medical benchmarks show that DeltaAD achieves SOTA performance.

Critical strengths

1.The variance decomposition analysis in Section 3.1.1 is very convincing. It provides strong statistical evidence for the proposed differential modeling. The t-SNE visualization in Figure 6 is very clear. It visually supports that relation learning effectively separates normal and anomalous features.

2.The experiments and ablation studies are comprehensive and clearly demonstrate the effectiveness of the proposed method. Extensive experiments across 15 datasets demonstrate strong cross-domain generalization, especially from industrial to medical domains. The model is also highly efficient.

Critical weaknesses

1.Under the training-based setting, all compared baselines in Table 1 are strictly limited to CLIP-based variants. Since training on the auxiliary dataset is allowed, the authors should also discuss and compare against recent Diffusion-based methods like DiAD and Mamba-based methods like MambaAD under the same cross-dataset setting.

2.The paper's core motivation is contrasting normal and anomalous responses under shared visual contexts. However, this narrative only fits DeltaAD-TF. DeltaAD-LE does not use anomalous memory entries or anomalous retrieval evidence during inference. Instead, it performs normal feature reconstruction and residual learning. The authors should refine the introduction to clarify this conceptual gap.

3.The experimental results reveal two limitations. First, when the auxiliary dataset has fewer categories, the image-level AUROC of DeltaAD-TF drops sharply. Second, the performance remains relatively low on complex structural objects and histopathology images. The authors should provide a failure case analysis for these scenarios.

Questions for authors

In Eq. 1, all normal patches in an image are averaged into a single vector for the memory bank. For complex objects with different local parts, doesn't this global pooling smooth out fine-grained details? Moreover, using these region-level keys to reconstruct a patch-level normal explanation causes a granularity mismatch. Why not build a patch-level memory bank instead?

8 comments

We sincerely appreciate your thoughtful and constructive feedback. Thank you for taking the time to carefully review our work and share your valuable comments and suggestions.

Q1: Comparison with diffusion- and Mamba-based anomaly detection methods under the same cross-dataset zero-shot setting, beyond CLIP-based baselines.

A1: Following the reviewer's suggestion, we have included DiAD and MambaAD in our comparative experiments and expanded their discussion in Sec. 2.1 (Related Work). The results demonstrate that DeltaAD-LE substantially outperforms both methods under the same cross-dataset zero-shot protocol.

Specifically, we train both baselines using 962 normal images from VisA, retain their original reconstruction frameworks, and directly evaluate their epoch-500 checkpoints on unseen industrial and medical datasets without target-domain training or adaptation. VisA is excluded to avoid training–test overlap.

Table R1-A. Comparison on common unseen industrial and medical benchmarks, excluding VisA (average AUROC, %).

MethodP-AUROC (%)I-AUROC (%)
DiAD73.853.2
MambaAD71.362.2
DeltaAD-LE93.592.7

Unlike reconstruction-based methods that primarily model normality within the auxiliary training distribution, DeltaAD-LE learns anomaly-related deviations relative to query-conditioned normal visual explanations, supporting cross-domain generalization.

Together with our additional comparisons against AnomalyVFM and evaluations using RADIO and DINOv3 (see Q1 and Q2 of Reviewer Rb8g), these results broaden the experimental assessment beyond CLIP-based baselines.

Q2: How do DeltaAD-TF and DeltaAD-LE share a unified differential modeling motivation despite their different retrieval and residual learning mechanisms?

A2: The central motivation of DeltaAD is to mitigate interference from cross-domain visual variations through differential modeling of query-relevant visual representations, thereby highlighting anomaly-related discrepancies while suppressing irrelevant visual variations.

Specifically, DeltaAD-TF uses query features to identify relevant visual contexts and constructs normal and anomalous retrieval responses under the same selected contexts and shared routing weights. It then computes the difference between these two responses to capture anomaly-related variations (Eq. 2). Building upon this principle, DeltaAD-LE introduces learnable query and key projectors in the training-based setting. The query projector maps query features into a learned representation, while the key projector maps normal memory features into the same retrieval space. It then retrieves a query-matched normal visual explanation and learns the residual relation between the query representation and its normal explanation to characterize anomalies (Eqs. 5–8).

Both variants therefore follow the same differential modeling principle, with DeltaAD-TF contrasting context-matched retrieval evidence and DeltaAD-LE learning query–normal residual relations. The normal-only design of DeltaAD-LE is further supported by controlled experiments under matched model capacity and training conditions (see Reviewer QsnA, Q3, and Appendix E.9, Table 27). We have clarified this unified motivation in the revised Introduction and Sec. 3.1.2.

Q3: Robustness to reduced auxiliary category coverage and failure-case analysis for complex structural objects and histopathology images.

A3: Following the reviewer's suggestion, we have conducted additional experiments with reduced auxiliary category coverage and analyzed challenging industrial and medical cases. The results demonstrate that DeltaAD-TF maintains competitive cross-domain performance even with substantially fewer auxiliary categories.

We evaluate three VisA subsets, each containing three categories: A (candle, macaroni2, cashew), B (capsules, pcb1, pipe_fryum), and C (macaroni1, cashew, candle). VisA-12 contains all 12 VisA categories, while BTAD-3 uses all three BTAD categories. MVTec-AD is the target dataset throughout, with inference settings unchanged.

Table R1-B. Auxiliary category coverage on MVTec-AD (P-AUROC and I-AUROC, %).

Method / MetricVisA-12VisA-3 (A)VisA-3 (B)VisA-3 (C)BTAD-3
DeltaAD-TF / P-AUROC88.089.088.188.885.5
DeltaAD-TF / I-AUROC88.590.883.789.772.4
MRAD-TF / P-AUROC86.7————
MRAD-TF / I-AUROC79.0————

Across the three VisA-3 subsets, DeltaAD-TF averages 88.6% P-AUROC and 88.1% I-AUROC, comparable to 88.0% and 88.5% with VisA-12. The MRAD-TF reference is available only for VisA-12; “—” denotes settings not evaluated for MRAD-TF, not performance results. The weaker BTAD-3 result also suggests that auxiliary visual composition matters, not category count alone (Appendix E.3, Tables 16–17 and Fig. 6). For challenging structural defects, comparative visualizations show that localization difficulties also occur in MRAD-TF and anomaly-only retrieval, rather than being specific to DeltaAD-TF (Appendix F, Fig. 8). Additional industrial category-level results are provided in Appendix C (Tables 7–8), with HIS comparisons in Table 1.

Q4: Why does DeltaAD-LE use region-pooled normal memory instead of patch-level memory, and how is fine-grained anomaly information preserved?

A4: Following the reviewer's suggestion, we compared Region Mean with three patch-level memory alternatives under the same model architecture and evaluation protocol. The results demonstrate that region-pooled normal memory achieves better anomaly localization with equal or fewer memory entries, supporting its effectiveness for cross-domain anomaly detection. Representative-1 selects the patch nearest to the region mean, Random-1 samples one patch, and Representative-4 retains one representative and three diverse patches per region.

Table R1-C. Normal-memory granularity ablation on MVTec-AD and DTD-Synthetic (PRO, %).

MemoryEntriesMVTec PRODTD PRO
Region Mean (Ours)2,16287.294.5
Representative-12,16284.693.1
Random-12,16284.593.0
Representative-48,64884.393.2

Fine-grained anomaly information is preserved because region pooling applies only to memory references, not query features. Each query location retains its projected feature qi\mathbf q_i, retrieves a corresponding normal explanation q^N,i\hat{\mathbf q}_{N,i}, and learns their residual relation to predict patch-level anomaly scores (Sec. 3.3). Thus, region-pooled references do not limit the spatial granularity of anomaly localization.

These results show that even quadrupling the patch-level memory size does not improve localization on these benchmarks. Region pooling provides compact normal references while preserving patch-specific anomaly information through relation learning. Complete results are provided in Appendix E.8 (Table 26).

Dear Reviewer 1TwJ,

Thank you again for your time and thoughtful feedback. We sincerely appreciate your valuable suggestions.

We would like to briefly highlight the key findings from our response:

  • Broader comparisons and generalization: Under the same cross-dataset zero-shot setting, DeltaAD-LE substantially outperforms DiAD and MambaAD, demonstrating stronger generalization to unseen domains. Furthermore, experiments with RADIO and DINOv3 show that our differential modeling framework extends beyond CLIP, with DeltaAD-LE maintaining strong performance across different visual backbones.
  • Unified motivation: We clarified how both DeltaAD variants mitigate cross-domain visual interference through complementary differential modeling mechanisms.
  • Robustness and memory design: Additional experiments show that DeltaAD-TF maintains competitive performance with only three VisA auxiliary categories, while region-pooled normal memory achieves better localization than patch-level alternatives on MVTec-AD and DTD-Synthetic.

We hope these additional findings and clarifications help address your concerns. If any questions remain, we would be more than happy to discuss them.

Thank you again for your time and consideration.

Best regards,
Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Revision Summary

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for your time and constructive feedback. We have carefully addressed the reviewers' concerns through extensive additional experiments, methodological clarifications, and manuscript revisions. Below, we summarize our main responses and improvements.

For Reviewer 1TwJ, we sincerely appreciate the reviewer's positive assessment of our convincing variance decomposition analysis, effective t-SNE visualizations, comprehensive experiments, strong cross-domain generalization, and model efficiency. To address the reviewer's key concern regarding comparisons beyond CLIP-based methods, we conducted additional experiments with DiAD and MambaAD under the same cross-dataset zero-shot protocol, demonstrating substantially stronger generalization with DeltaAD. We also further clarified the unified differential modeling principle underlying both variants. Regarding auxiliary data robustness, additional experiments demonstrate that DeltaAD-TF consistently outperforms a competitive memory-retrieval baseline across five matched auxiliary settings. Further analyses of memory granularity and challenging histopathology cases provide additional evidence of our design's effectiveness, robustness to varying auxiliary data distributions, and competitive performance on challenging benchmarks.

For Reviewer QsnA, we appreciate the reviewer's recognition of our clear methodological reasoning and the meaningful performance improvements achieved by both DeltaAD variants. A central concern was whether DeltaAD-TF and DeltaAD-LE genuinely share a unified differential modeling principle, given that DeltaAD-TF contrasts normal and anomalous evidence, whereas DeltaAD-LE models query–normal residual relations. We clarified that both variants follow the same underlying principle of suppressing shared visual interference through differential modeling, despite their different implementations. We further distinguished DeltaAD from MRAD through equation-level comparisons and controlled ablations. Additional experiments on normal-only versus dual-memory retrieval and extended component ablations further validate our design choices.

For Reviewer Rb8g, we appreciate the reviewer's positive assessment of our well-motivated residual modeling, auxiliary data analysis, and strong training-free performance. The primary concern was whether DeltaAD remains competitive with AnomalyVFM, particularly under stronger visual backbones. We conducted comprehensive comparisons under matched CLIP and RADIO backbones, demonstrating that DeltaAD-LE-RADIO outperforms AnomalyVFM-RADIO on multiple industrial and medical evaluation metrics despite using the substantially smaller VisA auxiliary dataset. We also extended both DeltaAD variants to RADIO and DINOv3 across all 15 benchmarks, further validating the generalizability of our differential modeling framework. Furthermore, using AnomalyVFM's synthetic auxiliary dataset improved DeltaAD-TF's average P-AUROC from 87.5% to 90.3%. Additional ablations and complexity comparisons further support our aggregation and relation modeling designs.

We have revised the manuscript to address the relevant concerns. The main changes are summarized as follows:

  • Introduction and Sec. 3.1.1–3.1.2: Clarified the unified differential modeling motivation of DeltaAD-TF and DeltaAD-LE. We also resolved the notation ambiguity in the image-level aggregation formulation.
  • Sec. 2.1–2.2: Expanded the discussion of recent methods, including DiAD, MambaAD, and AnomalyVFM.
  • Sec. 4.3 (Table 2) and Appendix E.11 (Tables 29–30): Extended the evaluation of DeltaAD-TF and DeltaAD-LE to RADIO and DINOv3 backbones.
  • Appendix E.1 (Tables 12–14) and E.3 (Tables 16–17): Extended the main component ablations to additional datasets and evaluated the effects of auxiliary dataset selection and category coverage.
  • Appendix E.6 (Table 23), E.8 (Table 26), and E.9 (Table 27): Provided additional experiments on image-level aggregation, region-pooled versus patch-level memory, and normal-only versus dual-memory retrieval.

Further experiments, including comparisons with DiAD, MambaAD, and AnomalyVFM, evaluations under reduced auxiliary category coverage, and additional HIS benchmark analyses, are provided in our detailed rebuttal responses.

We believe these additional experiments and revisions directly address the major concerns raised during the review process and provide stronger evidence for the effectiveness, robustness, and cross-domain generalizability of DeltaAD. We respectfully hope that our rebuttal and revised manuscript will be fully considered in the final assessment.

Thank you again for your time and consideration.

Best regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.