acceptodds
Under review as a conference paper at ICLR 2027

VD-DeepStack: Bridging Visual Comparison and Language Reasoning for Few-Shot Anomaly Detection

Abstract

Few-shot visual anomaly detection is fundamentally a visual comparison task, requiring fine-grained inspection of a query against normal references. Many recent methods based on large vision-language models (LVLMs) emphasize comparative reasoning through language chain-of-thought. Yet discrete, abstract descriptions may underrepresent dense, fine-grained visual differences, leaving a gap between visual comparison and its expression in language. To address this gap, we propose Visual Difference DeepStack (VD-DeepStack), which explicitly conditions language reasoning on query–reference visual differences. Specifically, we fuse DINO features with the LVLM visual hierarchy to strengthen fine-grained representations, then construct dense difference evidence from residuals between query features and softly matched reference features. The difference-evidence path injects spatially weighted difference vectors into query-image states at multiple decoder depths, while an auxiliary visual-context path provides fine-grained appearance information to support their interpretation. Experiments on 4 industrial and 2 medical anomaly benchmarks demonstrate substantial improvements in few-shot anomaly detection over baselines relying on textual comparative reasoning. These results support mitigating the visual comparison–reasoning gap through the joint design of comparison representations and their integration into the decoder. Code will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
visual anomaly detection, large vision language model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_emn8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VD-DeepStack, a reference-based few-shot anomaly detection (AD) method based on large vision-language models (LVLMs). The key idea is "thinking with visual differences." Previous LVLM-based few-shot AD methods use language chain-of-thought to achieve comparative reasoning. In contrast, VD-DeepStack explicitly conditions language reasoning on query–reference visual differences. Experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The proposed idea of thinking with visual differences is interesting and technically sound. Compared with existing text-based comparative reasoning, visual-difference-conditioned reasoning makes sense. The motivation is clear.
  2. This paper is well-organized and easy to follow. Figure 1 clearly illustrates the core idea.
Critical weaknesses
  1. The main concern is that the evaluation is not sufficiently comprehensive. (1) Several existing LVLM-based AD methods are not compared in Table 1, such as Anomaly-OV (CVPR 2025 [r1]) and AD-Copilot (reference-based as well) [r2]. (2) Many CLIP-based few-shot AD methods are not compared in Table 1, such as WinCLIP++ and UniVAD [r3]. (3) The backbone LVLMs consider only Qwen-family models. More diverse LVLM backbones should be considered to show generality. (4) The cross-domain evaluation in Table 3 compares only three baselines. This is an important experiment, so more baselines shown in Table 1 should be considered.

  2. The value of the reference-based few-shot AD setting is not sufficiently solid. The authors are requested to address this. Specifically, what are its advantages over the zero-shot AD setting (e.g., IAD-R1, Anomaly-OV), which is intuitively more efficient? Furthermore, the performance gap to non-LVLM-based full-shot anomaly detectors is still significant, as shown in Table 1.

  3. Figure 3 illustrates only the case where IAD-R1 is wrong, and VD-DeepStack is correct. It is hard to judge their reasoning quality. It should compare the case where both are correct, but their reasoning quality is different.

Given the above three main concerns, my rating at this stage is weak rejection. My final recommendation would depend on the rebuttal.

[r1] Xu, Jiacong, et al. "Towards zero-shot anomaly detection and reasoning with multimodal large language models." CVPR, 2025.

[r2] Jiang, Xi, et al. "AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison." arXiv preprint arXiv:2603.13779 (2026).

[r3] Gu, Zhaopeng, et al. "Univad: A training-free unified model for few-shot visual anomaly detection." CVPR, 2025.

Questions for authors

In addition to the main weaknesses above, I have the following questions.

  1. What exactly is the "Baseline" in Table 2? The number does not match any MLLM reported in Table 1.

  2. In Table 2, why does the VisA Prec. number drop significantly after RL?

  3. Tables should be placed at the top of the pages.

14 comments
AuthorsOpenReview ↗

Response to Reviewer emn8, Part 1

Response to Reviewer emn8, Part 1

Question W1(1) — Additional LVLM baselines

The main concern is that the evaluation is not sufficiently comprehensive. Several existing LVLM-based AD methods are not compared in Table 1, such as Anomaly-OV (CVPR 2025 [r1]) and AD-Copilot (reference-based as well) [r2].

Response:

Thank you for this valuable suggestion, which helps make our evaluation more rigorous and comprehensive.

We have added AD-Copilot-7B, which also adopts a reference-based setting, as an additional baseline in the main paper. Its results are presented below alongside our models.

Each entry reports detection accuracy / localization accuracy (%), following Table 1.

ModelMVTec ADVisAMVTec 3DMPDD
AD-Copilot-7B86.5/55.873.6/26.164.0/25.174.4/42.6
VD-DeepStack-7B91.0/66.682.1/47.976.3/51.978.6/54.8
VD-DeepStack-8B91.0/70.482.7/54.976.6/55.775.2/49.8

Both VD-DeepStack variants achieve higher detection and localization accuracy than AD-Copilot on all four datasets.

Regarding Anomaly-OV, it addresses zero-shot anomaly detection, whereas our work focuses on the one-shot reference-based setting. These are different evaluation paradigms: one-shot methods receive a normal reference that explicitly defines the expected appearance and provides direct visual evidence for identifying defects. Zero-shot methods lack this additional information, placing them at a substantial disadvantage in a direct comparison and making the comparison unfair. Many few-shot studies organize their main comparisons within the same shot setting, as exemplified by PromptAD and UniVAD (please see our response to W2 in Part 5). We follow this practice and prioritize comparisons under the same one-shot normal-reference setting. We therefore have not included Anomaly-OV in this comparison.

References cited by the reviewer in W1:

[r1] Xu, Jiacong, et al. "Towards zero-shot anomaly detection and reasoning with multimodal large language models." CVPR, 2025.

[r2] Jiang, Xi, et al. "AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison." arXiv preprint arXiv:2603.13779 (2026).

AuthorsOpenReview ↗

Response to Reviewer emn8, Part 2

Response to Reviewer emn8, Part 2

Question W1(2) — Conventional anomaly detectors

Many CLIP-based few-shot AD methods are not compared in Table 1, such as WinCLIP++ and UniVAD [r3].

Response:

Numbered citations [1]–[11] for this question refer to the references in Part 3.

Due to differences in evaluation paradigms, many existing LVLM-based anomaly detection studies, including IAD-R1 [6], JUDO [7], and Anomaly-R1 [9], do not use conventional anomaly detectors as their primary baselines. Conventional methods typically output anomaly scores and require an empirically selected threshold to obtain normal/abnormal decisions, as discussed in AnomalyGPT [8]. LVLM-based methods instead generate these decisions directly. Nevertheless, we have carefully considered your suggestion and recognize its value, as comparing the two under a common setting can further substantiate our results.

We therefore provide additional comparisons with WinCLIP+ [1], PromptAD [2], InCTRL [3], UniVAD [4], IIPAD [5], and AnomalyDINO [11], all reproduced under a one-shot normal-reference budget. For AnomalyDINO, we evaluate both its original DINOv2 encoder and a frozen DINOv3 replacement, keeping the remaining pipeline unchanged. Regarding FiLo++, we checked the official repository linked in its paper but could not locate a released FiLo++ implementation. The repository currently provides the implementation of its predecessor, FiLo. We therefore do not include FiLo++ in the present comparison.

We evaluate three threshold-selection protocols:

  • A: Fixed threshold. A sample is classified as anomalous when its original anomaly score exceeds 0.5, following the fixed-threshold comparison reported in AD-FM [10].
  • B: Global threshold. For each method, we select one threshold that maximizes F1 on a labeled 20% subset of the combined test samples and apply it across all four datasets.
  • C: Dataset-specific thresholds. For each method and dataset, we select a threshold that maximizes F1 on a labeled 20% subset of that dataset’s test samples.

All methods are evaluated on the full test sets, including the samples used for threshold selection. Protocols B and C therefore provide score-based methods with additional labeled target-domain information. VD-DeepStack uses its directly generated decisions without threshold calibration.

Table 1. Image-level accuracy (%) under a one-shot normal-reference budget. Mean denotes the unweighted average across the four datasets.

MethodMVTec ADVisAMVTec 3DMPDDMean
(A) Fixed threshold: 0.5
WinCLIP+27.743.421.141.833.5
PromptAD27.743.421.141.833.5
InCTRL58.957.042.650.052.1
UniVAD79.161.780.267.272.1
IIPAD27.743.421.141.833.5
AnomalyDINO–DINOv271.454.034.565.856.4
AnomalyDINO–DINOv336.944.821.242.936.4
(B) Global threshold selected using 20% of test labels
WinCLIP+77.964.477.957.569.4
PromptAD72.356.679.158.266.5
InCTRL80.573.978.572.176.2
UniVAD88.181.981.677.482.2
IIPAD78.266.074.369.372.0
AnomalyDINO–DINOv279.680.378.772.677.8
AnomalyDINO–DINOv381.978.982.466.577.4
(C) Dataset-specific thresholds selected using 20% of test labels
WinCLIP+77.869.578.957.570.9
PromptAD72.257.179.157.866.6
InCTRL85.977.780.572.579.2
UniVAD90.483.383.179.684.1
IIPAD78.369.178.971.374.4
AnomalyDINO–DINOv283.480.381.667.778.2
AnomalyDINO–DINOv389.678.983.462.978.7
(D) Direct decisions without threshold calibration
VD-DeepStack-7B91.082.176.378.682.0
VD-DeepStack-8B91.082.776.675.281.4

Reference cited by the reviewer:

[r3] Gu et al. UniVAD: A Training-free Unified Model for Few-shot Visual Anomaly Detection. CVPR, 2025.

AuthorsOpenReview ↗

Response to Reviewer emn8, Part 3

Response to Reviewer emn8, Part 3

Question W1(2) — Conventional anomaly detectors (continued)

Many CLIP-based few-shot AD methods are not compared in Table 1, such as WinCLIP++ and UniVAD [r3].

Response:

Threshold selection. Under Protocol A, several conventional detectors perform poorly with a fixed threshold of 0.5. Protocols B and C improve their results by using labeled target data to select a global threshold or separate thresholds for each dataset. Even with this additional information and dataset-specific calibration, both VD-DeepStack variants outperform most compared methods in mean accuracy, including AnomalyDINO–DINOv3.

Obtaining these dataset-specific thresholds in practice requires collecting and labeling target-domain examples. Reusing a threshold across datasets can also reduce accuracy, as shown by transferring thresholds to MVTec 3D while keeping anomaly scores unchanged:

MethodOwn-dataset thresholdTransferred thresholdDrop
IIPAD78.92%70.22% (from VisA)8.70 pp
InCTRL80.49%61.99% (from MVTec AD)18.50 pp

Another point worth noting is that many conventional anomaly detection methods also benefit from potentially privileged settings:

  • PromptAD is retrained for each novel class [2]. It learns new category-specific prompts using that class’s normal samples, producing a separate model for each category. Adapting PromptAD to a single model shared across categories leads to a substantial performance drop, as reported by IIPAD [5].

  • IIPAD partially addresses this limitation by enabling one model to handle multiple categories. However, it first requires few-shot normal samples from all target categories during training [5]. Its one-for-all setting therefore still depends on target-category training: new categories must be incorporated into the training data, rather than handled directly by an unchanged model.

  • UniVAD is a non-end-to-end framework that uses category-specific, empirically set thresholds [4]. Its component-segmentation stage relies on GroundingDINO, where box_threshold determines whether a predicted box is retained, while text_threshold determines which words are associated with that box. These settings vary across categories: the released pipeline uses box/text thresholds of 0.25/0.25 for bottle and 0.20/0.20 for cable.

VD-DeepStack applies one trained model to new categories using one normal reference, without target-category fine-tuning, category-specific component-segmentation settings, or anomaly-score calibration. It directly produces anomaly decisions, localization, and defect descriptions.

References for W1(2), Parts 2–3:

[1] Jongheon Jeong et al. WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation. CVPR, 2023.

[2] Xiaofan Li et al. PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection. CVPR, 2024.

[3] Jiawen Zhu et al. Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts. CVPR, 2024.

[4] Zhaopeng Gu et al. UniVAD: A Training-free Unified Model for Few-shot Visual Anomaly Detection. CVPR, 2025.

[5] Wenxi Lv et al. One-for-All Few-Shot Anomaly Detection via Instance-Induced Prompt Learning. ICLR, 2025.

[6] Yanhui Li et al. IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detection. AAAI, 2026.

[7] Hyunju Kang et al. JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. ICLR, 2026.

[8] Zhaopeng Gu et al. AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models. AAAI, 2024.

[9] Xincheng Yao et al. MMR-AD: A large- scale multimodal dataset for benchmarking general anomaly detection with multimodal large lan- guage models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.

[10] Jingyi Liao et al. AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization. AAAI, 2026.

[11] Simon Damm et al. AnomalyDINO: Boosting Patch-Based Few-Shot Anomaly Detection with DINOv2. WACV, 2025.

Reviewer_6KKfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an anomaly detection method called VD-DeepStack, which leverages DINOv3-enhanced multi-scale visual representations to perform patch-level soft matching between a query image and a normal reference image, thereby constructing dense query–reference difference vectors. The model then employs a DeepStack-style multi-layer residual injection mechanism to directly incorporate visual difference evidence into the query image's visual tokens, while injecting appearance context from both images into the reference and query representations. This design enables the language model to directly leverage fine-grained visual differences when generating anomaly predictions, chain-of-thought reasoning, and defect bounding boxes. The authors evaluate the proposed method on four industrial anomaly detection benchmarks and two medical datasets.

Critical strengths

1.The paper presents a clearly motivated problem with practical significance.

2.The experiments cover multiple domains and two different LVLM backbones, demonstrating the applicability of the proposed method across different settings.

3.The paper provides detailed implementation descriptions and is generally well-written and easy to follow.

Critical weaknesses

1.The paper jointly uses an LVLM and DINOv3, forming a dual-visual-encoder architecture. Although DINOv3 is frozen, it still requires forward passes for both the query and reference images. Combined with four-level local fusion, global patch-wise soft matching, and multi-layer residual injection, the end-to-end inference cost may be substantially higher than that of an LVLM-only baseline.

2.Inconsistent structural reward. In Appendix A.3, the structural reward uses the ground-truth label rather than the model's prediction to determine whether bounding boxes are required. As a result, logically inconsistent outputs, such as predicting "No" while providing a bounding box, can still receive the full structural reward. Although incorrect decisions are penalized separately, this design does not enforce consistency between the predicted decision and localization results. The authors should clarify this design choice.

3.Potential limitations of global soft matching. The proposed method performs global patch-level soft matching without explicit spatial constraints. As a result, when an anomaly involves an incorrect arrangement of otherwise normal components, query patches may still match visually similar reference patches at different locations, potentially leading to missed structural anomalies.

4.The paper does not report the reference-image processing time or the end-to-end inference latency.

Questions for authors

1.The paper jointly uses an LVLM and DINOv3, forming a dual-visual-encoder architecture. Although DINOv3 is frozen, it still requires forward passes for both the query and reference images. Combined with four-level local fusion, global patch-wise soft matching, and multi-layer residual injection, the end-to-end inference cost may be substantially higher than that of an LVLM-only baseline.

2.Inconsistent structural reward. In Appendix A.3, the structural reward uses the ground-truth label rather than the model's prediction to determine whether bounding boxes are required. As a result, logically inconsistent outputs, such as predicting "No" while providing a bounding box, can still receive the full structural reward. Although incorrect decisions are penalized separately, this design does not enforce consistency between the predicted decision and localization results. The authors should clarify this design choice.

3.Potential limitations of global soft matching. The proposed method performs global patch-level soft matching without explicit spatial constraints. As a result, when an anomaly involves an incorrect arrangement of otherwise normal components, query patches may still match visually similar reference patches at different locations, potentially leading to missed structural anomalies.

4.The paper does not report the reference-image processing time or the end-to-end inference latency.

  1. In Figures 5–7, the method is referred to as "VS-DeepStack" instead of "VD-DeepStack". Is this a typographical error?
10 comments
AuthorsOpenReview ↗

Response to Reviewer 6KKf, Part 1

Response to Reviewer 6KKf, Part 1

Question W1 / Q1 — Inference cost

The paper jointly uses an LVLM and DINOv3, forming a dual-visual-encoder architecture. Although DINOv3 is frozen, it still requires forward passes for both the query and reference images. Combined with four-level local fusion, global patch-wise soft matching, and multi-layer residual injection, the end-to-end inference cost may be substantially higher than that of an LVLM-only baseline.

Response:

Important suggestion! The explicit efficiency analysis would strengthen the paper and clarify the practical implications of our design. We therefore provide additional parameter, computational-cost, and inference latency comparisons below.

Although VD-DeepStack introduces several components, their combined parameter count is small relative to the LVLM backbone. Specifically, the comparison, injection, and reader-adaptation modules add approximately 36.4M parameters, while the frozen DINOv3 ViT-L/16 encoder contributes 303.1M parameters. Including both, the complete model adds approximately 339.5M parameters, corresponding to 4.09% of the Qwen2.5-VL-7B backbone’s parameter count.

Table 1. Parameter and computational costs. FLOPs are averaged over the evaluation workload.

MetricIAD-R1*VD-DeepStack-7BIncrease
Total parameters (M)8,292.28,631.7339.5 (+4.09%)
Average FLOPs per case (T)12.7915.382.59 (+20.28%)

We further evaluate runtime using SGLang on a single NVIDIA H200 GPU, processing the same 500 cases, each containing a reference image and a query image. With cached DINOv3 features, VD-DeepStack-7B completes this workload in 60.5 s, compared with 58.7 s for IAD-R1*, an increase of 3.1%. DINOv3 feature extraction for the entire workload separately takes 1.2 s. Adding this stage sequentially gives an estimated uncached total of 61.7 s, or 5.1% above the baseline. Regarding caching, DINOv3 reference features can be reused across queries sharing the same normal reference.

Model and configurationTotal time (s)Throughput (cases/s)Time increase
IAD-R1*58.78.52—
VD-DeepStack-7B, cached DINOv3 features60.58.26+3.1%
VD-DeepStack-7B, including DINOv3 feature extraction61.78.10+5.1%

We acknowledge that LVLM-based anomaly detection generally entails greater computational cost than conventional anomaly detectors. Its motivation is the ability to support generalization across categories and datasets while providing natural-language explanations alongside anomaly decisions. These capabilities have motivated the LVLM-based methods, including JUDO [1], Anomaly-R1 [2], AnomalyGPT, IAD-R1, and etc. VD-DeepStack delivers substantial improvements in anomaly detection and localization at a modest additional cost, requiring only 4.09% more parameters and an estimated 5.1% increase in workload processing time under the evaluated configuration.

References

[1] Hyunju Kang, Woohyun Lee, Jaewon Kim, and Hogun Park. JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. International Conference on Learning Representations (ICLR), 2026.

[2] Xincheng Yao, Zefeng Qian, Chao Shi, Jiayang Song, and Chongyang Zhang. MMR-AD: A large- scale multimodal dataset for benchmarking general anomaly detection with multimodal large lan- guage models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.

AuthorsOpenReview ↗

Response to Reviewer 6KKf, Part 2

Response to Reviewer 6KKf, Part 2

Question W2 / Q2 — Structural reward

Inconsistent structural reward. In Appendix A.3, the structural reward uses the ground-truth label rather than the model's prediction to determine whether bounding boxes are required. As a result, logically inconsistent outputs, such as predicting "No" while providing a bounding box, can still receive the full structural reward. Although incorrect decisions are penalized separately, this design does not enforce consistency between the predicted decision and localization results. The authors should clarify this design choice.

Response:

Thank you for pointing this out. The structural reward was intended primarily to supervise the expected output format: the ground-truth label determines whether the target response should contain bounding boxes, while prediction correctness is rewarded separately. We agree that this term does not explicitly enforce consistency between the predicted decision and the presence of bounding boxes. Consequently, the example identified by the reviewer can receive the full structural reward without receiving the decision-correctness reward.

Before reinforcement learning, our SFT data consistently follows a fixed response schema: an anomalous response contains “Yes” followed by bounding boxes, whereas a normal response contains “No” without bounding boxes. This provides explicit supervision for the relationship between the decision and localization outputs.

To examine whether the issue occurs in practice, we audited the predictions on all four datasets reported in Table 1 and observed no decision–bounding-box inconsistencies. This empirical result is consistent with the response schema learned during SFT, although it does not constitute a formal consistency guarantee of the structural reward itself. We have clarified the intended role of this reward and its distinction from an explicit decision–localization consistency constraint in Appendix A.3.

Question W3 / Q3 — Global soft matching

Potential limitations of global soft matching. The proposed method performs global patch-level soft matching without explicit spatial constraints. As a result, when an anomaly involves an incorrect arrangement of otherwise normal components, query patches may still match visually similar reference patches at different locations, potentially leading to missed structural anomalies.

Response:

Thank you for this insightful comment. We agree that this limitation can occur, particularly for logical anomalies involving an incorrect arrangement of otherwise normal components, such as those included in MVTec LOCO AD. Global soft matching may associate visually similar patches across different locations, weakening the difference signal for such anomalies. We have added this limitation to the appendix subsection “Limitations”.

Here, we distinguish visual anomalies, which involve abnormal appearance, from logical anomalies, which violate expected arrangements or relationships among otherwise normal components. Our work targets visual anomaly detection rather than logical anomaly detection.

Nevertheless, VD-DeepStack retains the LVLM’s native visual representations and augments them with difference evidence and appearance context. Its predictions therefore do not rely exclusively on soft-matched differences. The retained visual representations provide an additional pathway for accessing spatial configuration cues relevant to logical anomalies.

AuthorsOpenReview ↗

Response to Reviewer 6KKf, Part 3

Response to Reviewer 6KKf, Part 3

Question W4 / Q4 — Reference processing and runtime

The paper does not report the reference-image processing time or the end-to-end inference latency.

Response:

Thank you for the comment. Please see Part 1 (W1 / Q1) for the efficiency results. DINOv3 feature extraction for the entire 500-case workload takes 1.2 s. DINOv3 reference features can be reused across queries sharing the same normal reference. Adding this extraction stage to the 60.5 s cached-feature workload time gives an estimated total of 61.7 s.

Using SGLang on a single NVIDIA H200 GPU, this corresponds to a throughput of 8.10 cases/s, or approximately 486 inspection cases per minute. Each case consists of a reference image and a query image, with the model producing anomaly decisions, localization outputs, and natural-language explanations.

Question Q5 — Method name

In Figures 5–7, the method is referred to as "VS-DeepStack" instead of "VD-DeepStack". Is this a typographical error?

Response:

Thank you for pointing this out. “VS-DeepStack” is indeed a typographical error and should read “VD-DeepStack.” We have corrected it in Figures 5–7 and rechecked the manuscript for naming consistency.

Reviewer_7umTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VD-DeepStack for reference-based few-shot visual anomaly detection. It augments a Qwen-based LVLM with DINOv3 features, builds fine-grained query–reference differences through soft matching, and injects these features into multiple language-decoder layers via DeepStack. Experiments on four industrial and two medical datasets show competitive anomaly detection and localization results using Qwen2.5-VL-7B and Qwen3-VL-8B.

Critical strengths
  • An interesting design for visual-difference-conditioned reasoning. The paper highlights a plausible limitation of language-only comparisons for subtle defects. Explicitly supplying dense query–reference differences to the LVLM is a useful and reasonably motivated direction.
  • Encouraging results across benchmarks. The method improves detection and localization relative to several LVLM-based baselines on multiple industrial datasets, with additional evaluation across two backbones, component ablations, and medical datasets.
Critical weaknesses
  • High system complexity without efficiency analysis. The method combines a 7B/8B LVLM, DINOv3 ViT-L/16, feature matching, and multi-depth injection, yet reports no end-to-end inference latency, peak memory, throughput, or computational cost. The performance–efficiency trade-off and practical deployment implications remain unclear.
  • Insufficient comparison with established zero-/few-shot anomaly detectors. The main results omit matched comparisons with approaches such as WinCLIP+, PromptAD, FiLo++, and InCTRL. Appendix B.2 compares frozen CLIP/DINO/SigLIP2 encoders under a common one-shot matching protocol, but not full anomaly-detection methods. The gain over a simple DINOv3-only detector therefore remains uncertain.
  • Incomplete isolation of the proposed mechanism. The system jointly relies on DINO features, soft matching, mask supervision, deep injection, and multi-stage training. Existing ablations do not directly compare the complete model without DINO or single-depth versus multi-depth injection under matched supervision. This complicates attribution of the gains to DeepStack conditioning itself.
  • Improved anomaly detection does not necessarily establish improved visual reasoning. The difference branch already receives explicit anomaly-mask supervision, so better detection may primarily reflect stronger anomaly representations rather than better language reasoning. The overall CoT score improves only from 68.3 to 69.4 over IAD-R1* (Table 4) and relies on a single text-based LLM judge. More direct evidence of visually grounded reasoning is needed.
Questions for authors
  • Could you compare with representative CLIP-/DINO-based zero-/few-shot methods under the same reference protocol? In particular, how much does VD-DeepStack improve over simple DINOv3 patch matching?
  • Could you report end-to-end inference latency, peak memory, and compute relative to the Qwen backbone and DINO-only baselines, including whether reference features can be cached?
  • Could you evaluate the complete model without DINO and compare single-depth against multi-depth difference injection with matched training data and supervision?
  • How do you distinguish an improvement in visual anomaly detection from an improvement in language reasoning? Could human evaluation or another visually grounded assessment address the current CoT evaluation limitations?
  • I would genuinely like to understand what practical needs motivate using a large LVLM as the core of an anomaly detector when conventional methods already support detection and localization. Is the intended added value open-category transfer, defect semantics, natural-language explanations, or interactive analysis? If so, could the paper demonstrate these benefits in representative application scenarios?
14 comments
AuthorsOpenReview ↗

Response to Reviewer 7umT, Part 1

Response to Reviewer 7umT, Part 1

Question W1 / Q2 — Efficiency

High system complexity without efficiency analysis. The method combines a 7B/8B LVLM, DINOv3 ViT-L/16, feature matching, and multi-depth injection, yet reports no end-to-end inference latency, peak memory, throughput, or computational cost. The performance–efficiency trade-off and practical deployment implications remain unclear.

Response:

Important suggestion! The explicit efficiency analysis would strengthen the paper and clarify the practical implications of our design. We therefore provide additional parameter, computational-cost, and inference latency comparisons below.

Although VD-DeepStack introduces several components, their combined parameter count is small relative to the LVLM backbone. Specifically, the comparison, injection, and reader-adaptation modules add approximately 36.4M parameters, while the frozen DINOv3 ViT-L/16 encoder contributes 303.1M parameters. Including both, the complete model adds approximately 339.5M parameters, corresponding to 4.09% of the Qwen2.5-VL-7B backbone’s parameter count.

Table 1. Parameter and computational costs. FLOPs are averaged over the evaluation workload.

MetricIAD-R1*VD-DeepStack-7BIncrease
Total parameters (M)8,292.28,631.7339.5 (+4.09%)
Average FLOPs per case (T)12.7915.382.59 (+20.28%)

We additionally measure peak GPU memory using Hugging Face Transformers with a batch size of 24 and BF16 precision. IAD-R1* and VD-DeepStack-7B consume 21.46 GiB and 23.15 GiB, respectively, corresponding to an increase of 1.69 GiB (+7.88%).

We further evaluate runtime using SGLang on a single NVIDIA H200 GPU, processing the same 500 cases, each containing a reference image and a query image. With cached DINOv3 features, VD-DeepStack-7B completes this workload in 60.5 s, compared with 58.7 s for IAD-R1*, an increase of 3.1%. DINOv3 feature extraction for the entire workload separately takes 1.2 s. Adding this stage sequentially gives an estimated uncached total of 61.7 s, or 5.1% above the baseline. Regarding caching, DINOv3 reference features can be reused across queries sharing the same normal reference.

Model and configurationTotal time (s)Throughput (cases/s)Time increase
IAD-R1*58.78.52—
VD-DeepStack-7B, cached DINOv3 features60.58.26+3.1%
VD-DeepStack-7B, including DINOv3 feature extraction61.78.10+5.1%

We acknowledge that LVLM-based anomaly detection generally entails greater computational cost than conventional anomaly detectors. Its motivation is the ability to support generalization across categories and datasets while providing natural-language explanations alongside anomaly decisions. These capabilities have motivated the LVLM-based methods, including JUDO [1], Anomaly-R1 [2], AnomalyGPT, IAD-R1, and etc. VD-DeepStack delivers substantial improvements in anomaly detection and localization at a modest additional cost, requiring only 4.09% more parameters and an estimated 5.1% increase in workload processing time under the evaluated configuration.

References for W1 / Q2

[1] Hyunju Kang, Woohyun Lee, Jaewon Kim, and Hogun Park. JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. International Conference on Learning Representations (ICLR), 2026.

[2] Xincheng Yao, Zefeng Qian, Chao Shi, Jiayang Song, and Chongyang Zhang. MMR-AD: A large- scale multimodal dataset for benchmarking general anomaly detection with multimodal large lan- guage models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.

AuthorsOpenReview ↗

Response to Reviewer 7umT, Part 2

Response to Reviewer 7umT, Part 2

Question W2 / Q1 — Conventional Anomaly detector comparisons

Insufficient comparison with established zero-/few-shot anomaly detectors. The main results omit matched comparisons with approaches such as WinCLIP+, PromptAD, FiLo++, and InCTRL. Appendix B.2 compares frozen CLIP/DINO/SigLIP2 encoders under a common one-shot matching protocol, but not full anomaly-detection methods. The gain over a simple DINOv3-only detector therefore remains uncertain.

Response:

Thanks for the comments. Due to differences in evaluation paradigms, many existing LVLM-based anomaly detection studies, including IAD-R1 [6], JUDO [7], and Anomaly-R1 [9], do not use conventional anomaly detectors as their primary baselines. Conventional methods typically output anomaly scores and require an empirically selected threshold to obtain normal/abnormal decisions, as discussed in AnomalyGPT [8]. LVLM-based methods instead generate these decisions directly. Nevertheless, we have carefully considered your suggestion and recognize its value, as comparing the two under a common setting can further substantiate our results.

We therefore provide additional comparisons with WinCLIP+ [1], PromptAD [2], InCTRL [3], UniVAD [4], IIPAD [5], and AnomalyDINO [11] (DINO-only detector), all reproduced under a one-shot normal-reference budget. For AnomalyDINO, we evaluate both its original DINOv2 encoder and the DINOv3 replacement, keeping the remaining pipeline unchanged. Regarding FiLo++, we checked the official repository linked in its paper but could not locate a released FiLo++ implementation. The repository currently provides the implementation of its predecessor, FiLo. We therefore do not include FiLo++ in the present comparison.

We evaluate three threshold-selection protocols:

  • A: Fixed threshold. A sample is classified as anomalous when its original anomaly score exceeds 0.5, following the fixed-threshold comparison reported in AD-FM [10].
  • B: Global threshold. For each method, we select one threshold that maximizes F1 on a labeled 20% subset of the combined test samples and apply it across all four datasets.
  • C: Dataset-specific thresholds. For each method and dataset, we select a threshold that maximizes F1 on a labeled 20% subset of that dataset’s test samples.

All methods are evaluated on the full test sets, including the samples used for threshold selection. Protocols B and C therefore provide score-based methods with additional labeled target-domain information. VD-DeepStack uses its directly generated decisions without threshold calibration.

Table 1. Image-level accuracy (%) under a one-shot normal-reference budget. Mean denotes the unweighted average across the four datasets.

MethodMVTec ADVisAMVTec 3DMPDDMean
(A) Fixed threshold: 0.5
WinCLIP+27.743.421.141.833.5
PromptAD27.743.421.141.833.5
InCTRL58.957.042.650.052.1
UniVAD79.161.780.267.272.1
IIPAD27.743.421.141.833.5
AnomalyDINO–DINOv271.454.034.565.856.4
AnomalyDINO–DINOv336.944.821.242.936.4
(B) Global threshold selected using 20% of test labels
WinCLIP+77.964.477.957.569.4
PromptAD72.356.679.158.266.5
InCTRL80.573.978.572.176.2
UniVAD88.181.981.677.482.2
IIPAD78.266.074.369.372.0
AnomalyDINO–DINOv279.680.378.772.677.8
AnomalyDINO–DINOv381.978.982.466.577.4
(C) Dataset-specific thresholds selected using 20% of test labels
WinCLIP+77.869.578.957.570.9
PromptAD72.257.179.157.866.6
InCTRL85.977.780.572.579.2
UniVAD90.483.383.179.684.1
IIPAD78.369.178.971.374.4
AnomalyDINO–DINOv283.480.381.667.778.2
AnomalyDINO–DINOv389.678.983.462.978.7
(D) Direct decisions without threshold calibration
VD-DeepStack-7B91.082.176.378.682.0
VD-DeepStack-8B91.082.776.675.281.4

Discussion and references [1]–[11] for W2 / Q1 continue in Part 3.

AuthorsOpenReview ↗

Response to Reviewer 7umT, Part 3

Response to Reviewer 7umT, Part 3

Question W2 / Q1 — Conventional detector comparisons (continued)

Continued from Part 2: interpretation of the comparison results and references.

Response:

Threshold selection. Under Protocol A, several conventional detectors perform poorly with a fixed threshold of 0.5. Protocols B and C improve their results by using labeled target data to select a global threshold or separate thresholds for each dataset. Even with this additional information and dataset-specific calibration, both VD-DeepStack variants outperform most compared methods in mean accuracy, including AnomalyDINO–DINOv3.

Obtaining these dataset-specific thresholds in practice requires collecting and labeling target-domain examples. Reusing a threshold across datasets can also reduce accuracy, as shown by transferring thresholds to MVTec 3D while keeping anomaly scores unchanged:

MethodOwn-dataset thresholdTransferred thresholdDrop
IIPAD78.92%70.22% (from VisA)8.70 pp
InCTRL80.49%61.99% (from MVTec AD)18.50 pp
  • Another point worth noting is that many conventional anomaly detection methods also benefit from potentially privileged settings:
    • PromptAD is retrained for each novel class [2]. It learns new category-specific prompts using that class’s normal samples, producing a separate model for each category. Adapting PromptAD to a single model shared across categories leads to a substantial performance drop, as reported by IIPAD [5].
    • IIPAD partially addresses this limitation by enabling one model to handle multiple categories. However, it first requires few-shot normal samples from all target categories during training [5]. Its one-for-all setting therefore still depends on target-category training: new categories must be incorporated into the training data, rather than handled directly by an unchanged model.
    • UniVAD is a non-end-to-end framework that uses category-specific, empirically set thresholds [4]. Its component-segmentation stage relies on GroundingDINO, where box_threshold determines whether a predicted box is retained, while text_threshold determines which words are associated with that box. These settings vary across categories: the released pipeline uses box/text thresholds of 0.25/0.25 for bottle and 0.20/0.20 for cable.

VD-DeepStack applies one trained model to new categories using one normal reference, without target-category fine-tuning, category-specific component-segmentation settings, or anomaly-score calibration. It directly produces anomaly decisions, localization, and defect descriptions.

References for W2 / Q1 (Parts 2–3)

[1] Jongheon Jeong et al. WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation. CVPR, 2023.

[2] Xiaofan Li et al. PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection. CVPR, 2024.

[3] Jiawen Zhu and Guansong Pang. Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts. CVPR, 2024.

[4] Zhaopeng Gu et al. UniVAD: A Training-free Unified Model for Few-shot Visual Anomaly Detection. CVPR, 2025.

[5] Wenxi Lv et al. One-for-All Few-Shot Anomaly Detection via Instance-Induced Prompt Learning. ICLR, 2025.

[6] Yanhui Li et al. IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detection. AAAI, 2026.

[7] Hyunju Kang et al. JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. ICLR, 2026.

[8] Zhaopeng Gu et al. AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models. AAAI, 2024.

[9] Xincheng Yao et al. MMR-AD: A large-scale multimodal dataset for benchmarking general anomaly detection with multimodal large language models. CVPR, 2026.

[10] Jingyi Liao et al. AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization. AAAI, 2026.

[11] Simon Damm et al. AnomalyDINO: Boosting Patch-Based Few-Shot Anomaly Detection with DINOv2. WACV, 2025.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response

We sincerely thank the Area Chair and all three reviewers for their time and thoughtful feedback. We appreciate the recognition of our motivation and method [Reviewers 7umT, 6KKf, emn8], experimental coverage and results [Reviewers 7umT, 6KKf], and clear presentation [Reviewers 6KKf, emn8]. Their comments have guided the following experiments and revisions.

  • Computational cost and practical use. We added an efficiency analysis covering parameters, FLOPs, peak memory, runtime, and reference-feature caching (Appendix B.2). The results show modest additional parameter, memory, and runtime costs over the LVLM baseline. Our responses also clarify the practical value of reference-based detection for rapid deployment, transfer to new categories, and natural-language explanations.

  • Broader comparisons and generalization. Expanded comparisons with conventional and LVLM-based detectors, an additional InternVL backbone, and broader medical evaluation strengthen the evidence for effectiveness and generalization (Tables 1 and 3; Appendix B.1). VD-DeepStack outperforms most compared conventional detectors in mean accuracy even with dataset-specific threshold calibration, while generating decisions directly without such calibration.

  • Evidence for the proposed mechanism. We clarified the existing DINO, injection-path, and mask-supervised E-tokens controls (Section 4.4, Table 2) and added a single-depth versus multi-depth comparison under matched training and supervision (Appendix B.3). These results support the contribution of deep visual-difference conditioning.

  • Visually grounded explanation quality. Additional image-grounded evaluation by human raters and a multimodal judge consistently shows higher explanation quality, including on cases correctly classified by both models (Appendix B.7). Qualitative examples further illustrate these improvements (Appendix B.9).

  • Method and presentation clarifications. We clarified the SFT baseline and RL precision–recall trade-off (Section 4.4), the structural reward (Appendix A.3), and global soft matching's limitation on logical anomalies (Appendix B.8). We also corrected naming errors and adjusted table placement.

We hope these responses, additional experiments, and manuscript revisions address the reviewers’ concerns. Please let us know if any concerns remain; we would be happy to address them.

AuthorsOpenReview ↗

Friendly reminder regarding our response before the rebuttal deadline

Dear Reviewer,

Thanks for the valuable and constructive comments. We do not mean to rush or disturb you. Since the rebuttal discussion period will close in less than 30 hours, we are sincerely concerned whether our responses have properly addressed your main concerns. We would greatly appreciate it if you could share your feedback at your convenience. We welcome any further discussion and are honored to engage with you.

Best regards,

The VD-DeepStack Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.