acceptodds
Under review as a conference paper at ICLR 2027

Comparative Evidence Policy Optimization for Multimodal Industrial Anomaly Detection

Abstract

Industrial anomaly detection (IAD) is a critical task for automated quality inspection and intelligent manufacturing. Recent advances in multimodal large language models (MLLMs) have extended IAD beyond conventional anomaly classification and localization toward unified visual reasoning and natural-language explanation. However, our analysis reveals a fundamental evidence-grounding gap in reference-guided anomaly reasoning: MLLMs often fail to reliably ground their predictions in anomaly-relevant regions and underutilize the normal reference. More importantly, weak anomaly grounding and insufficient Query–Reference comparison substantially degrade anomaly reasoning performance. This observation motivates a simple principle: reliable anomaly reasoning should be grounded in defect-relevant visual evidence and explicitly supported by comparison against the corresponding normal reference. To this end, we first introduce Comparative Attention Supervision (CAS), which establishes anomaly-aware visual grounding by encouraging the generated reasoning to attend to defect-relevant Query regions and their corresponding normal Reference context. We further propose Comparative Evidence Policy Optimization (CEPO), which moves beyond outcome-only optimization by explicitly evaluating whether the generated reasoning is grounded in anomaly-relevant Query–Reference comparative evidence. Experiments on MMAD and MMR-AD demonstrate the effectiveness of our approach across diverse anomaly understanding and localization tasks, as well as its generalization to unseen datasets. Further ablations and qualitative analyses highlight the complementary benefits of anomaly-aware grounding and comparative evidence in improving IAD performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Industrial Anomaly Detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wUR8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates reference‑guided multimodal industrial anomaly detection. The authors observe that existing multimodal large language models often fail to ground reasoning on real defect regions and underutilize normal reference images for cross‑image comparison. The proposed method has two stages: Comparative Attention Supervision for supervised fine‑tuning, followed by Comparative Evidence Policy Optimization for reinforcement learning.

Critical strengths

The problem setting of reference-guided multimodal industrial anomaly detection is meaningful; however, it is less compelling for supervised image anomaly detection.

Applying policy optimization to image anomaly detection is somewhat novel.

The proposed method is clearly described.

Critical weaknesses

I believe that reference-guided multimodal industrial anomaly detection is more suitable for training-free or few-shot image anomaly detection settings, rather than supervised image anomaly detection.

The proposed Comparative Attention Supervision (CAS) loss requires pixel‑level anomaly masks for the query image during supervised fine‑tuning. Since full pixel‑level defect annotations are expensive, the paper does not explore semi‑supervised or weak‑supervised settings.

CAS essentially adds multi‑term attention regularization loss on top of standard SFT. The idea of supervising decoder attention toward ground‑truth regions is not fundamentally new.

Comparative Evidence Policy Optimization (CEPO) builds directly upon GRPO, and the main difference lies in constructing an extra cross‑image comparative reward term. It is a incremental novelty.

There is no controlled study isolating the impact of inference‑time min‑p sampling. It is well‑known that min‑p strongly affects multimodal reasoning performance.

Computational overhead analysis is missing.

Equations (1)–(9) all end with a period, which is not standard formatting.

Some tables in the paper contain colored text or backgrounds.

Questions for authors

I believe that reference-guided multimodal industrial anomaly detection is more suitable for training-free or few-shot image anomaly detection settings, rather than supervised image anomaly detection.

I am not convinced that LLM-based approaches are particularly beneficial for image anomaly detection. Compared with conventional vision-based methods, they introduce additional language processing and cross-modal alignment components, yet the paper does not provide convincing evidence that these capabilities lead to substantial gains for image anomaly detection.

2 comments
Reviewer_6sEwOpenReview ↗

Comment from Reviewer 6sEw: I believe Reviewer wUR8’s reasons for rejection are unjustified.

I'm Reviewer 6sEw. I believe Reviewer wUR8’s reasons for rejection are unjustified.

  1. On what basis do you claim that reference-guided multimodal industrial anomaly detection is more suitable for training-free or few-shot settings than for supervised settings? Reference-guided anomaly detection is a modeling approach; it has no inherent restriction to training-free methods. A central research focus in MLLM-based anomaly detection is general anomaly detection: training a model that can subsequently detect anomalies in unseen categories without retraining. Relying solely on training-free approaches is insufficient to achieve this goal. An emerging direction is to post-train MLLMs on large-scale anomaly detection datasets to improve their generalization capabilities. MMR-AD was introduced precisely for this purpose, and its query–reference image pairs explicitly support reference-guided multimodal anomaly detection. The paper under discussion builds on this dataset. There is therefore no contradiction between using reference images and performing large-scale post-training before deployment.

  2. Regarding the requirement for pixel-level annotations, MMR-AD already provides large-scale pixel-level supervision, and this paper investigates how to further improve MLLM-based anomaly detection using that dataset. Previous work primarily applied standard SFT and RL post-training without introducing designs specifically tailored to anomaly detection. This paper addresses a defining characteristic of the task—detecting anomalies through comparison with normal references—and proposes a valuable attention-based design to support that comparison. I consider this a meaningful contribution to MLLM-based anomaly detection. Dismissing the work simply because pixel-level annotations are expensive is not a convincing argument. The training paradigm studied here uses both normal and anomalous samples and differs fundamentally from traditional unsupervised anomaly detection. Rejecting a valuable contribution merely because it does not conform to your preferred problem setting is, in my view, highly irresponsible.

  3. If you believe that the proposed method is “not fundamentally new,” please identify the existing work that already provides the relevant mechanisms and explain the specific overlap. Without concrete references and a substantive comparison, this assessment of novelty is difficult to find convincing.

  4. As explained above, the authors specifically address the comparative nature of anomaly detection. Without changing the model architecture, and while retaining its pretrained knowledge, they introduce an attention-based design that enables more effective comparison between the query image and its normal reference, thereby improving anomaly detection. Your assessment appears to overlook this task-specific contribution and instead reduces the method to “adding an extra loss” or “constructing an extra reward term,” then uses “incremental novelty” as a reason for rejection. This does not adequately engage with what the method is designed to achieve or why that design matters for anomaly detection. I also consider this an irresponsible basis for rejecting the paper.

Thank you for your discussions. My responses are as follows:

  1. Image anomaly detection, particularly industrial image anomaly detection, is a low-level vision task. I am skeptical that high-level LLMs can bring meaningful benefits to this task. In my view, the reported performance improvements are more likely attributable to overfitting. I am personally not enthusiastic about research on LLM-based image anomaly detection. In addition, sufficient work has already been published in this direction.

2-3. The title, “Comparative Evidence Policy Optimization for Multimodal Industrial Anomaly Detection,” and the statement that “reliable anomaly reasoning should be grounded in defect-relevant visual evidence and explicitly supported by comparison against the corresponding normal reference” do not provide me with any meaningful new insights.

The central keyword throughout the paper is evidence, yet it is used inconsistently, appearing as visual evidence in some places and comparative evidence in others. I believe the term evidence should not be used loosely, but should be clearly defined and consistently applied. Even setting aside the concerns about the novelty of the work, I find the writing of this paper insufficiently rigorous.

  1. “Introduce an attention-based design that enables more effective comparison between the query image and its normal reference” is indeed a incremental novelty.
Reviewer_6sEwOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

In this paper, for the problem that MLLMS fail to reliably ground their predictions in anomaly-relevant regions and underutilize the normal reference, the authors study that how to achieve evident-grounded comparative reasoning for Ference-guided IAD. To this end, the authors first introduce Comparative Attention Supervision (CAS), which establishes anomaly-aware visual grounding by encouraging the generated reasoning to attend to defect-relevant Query regions and their corresponding normal Reference context. The authors further propose Comparative Evidence Policy Optimization (CEPO), which moves beyond outcome-only optimization by explicitly evaluating whether the generated reasoning is grounded in anomaly-relevant Query–Reference comparative evidence. Experiments conducted on both MMAD and MMR-AD have demonstrated that the proposed method has achieved remarkable success.

Critical strengths
  1. Well-motivated Problem: I think that the paper addresses a highly critical issue. When applying MLLMs to anomaly detection, they frequently fail to attend to the actual anomalous regions and lack a reasonable and correct comparison mechanism.

  2. Strong Motivation: The motivation is compelling and well-founded. The core focus is on how to enable MLLMs to perform more sufficient and explicit normal-abnormal comparisons, which is critical for enhancing their anomaly detection capabilities.

  3. Innovative Methodology and Insight: The method design is highly innovative. Without altering the underlying model architecture, the authors cleverly translate attention weights into an optimizable loss objective, effectively improving the model's focus on anomalous regions. Previous MLLM-based anomaly detection methods often lacked specific architectural or objective-level designs tailored to the unique characteristics of anomaly detection. This paper makes a solid contribution in this regard and provides valuable insights.

Critical weaknesses
  1. The text in Figure 1(c) is too small to read clearly. It could be enlarged appropriately.

  2. In line 118, do the reference image and query image need to be the same size? A straightforward comparison between normal and abnormal situations is the comparison of corresponding positions, which requires H_q \times W_q and H_r \times W_r to be the same size. Does the method proposed in the paper support the use of reference and query images of different sizes?

  3. Equation 9 is not easy to understand. It is unclear what L specifically measures. It would be better to provide some more clear explanations.

  4. In CEPO, will there be situations where R^task and R^cmp are inconsistent? The model correctly identifies an anomaly (R^task=1), but R^cmp is small; the model unrecognizes an anomaly (R^task=0), but R^cmp is large.

  5. In the experiments, 1-shot reference samples were used. Can your method be applied to multi-shot references? Since CAS is based on the reasoning+answer mode, but the MMAD dataset does not provide reasoning text, how is your method applied in MMAD?

  6. How are Q-ground, R-ground, and Evidence visualized in Figure 3? Are they directly visualized based on the numerical values calculated from s_q, s_r in Equation 4 and E_t mentioned in line 227? How is Ori Evidence visualized? It would be helpful to provide some explanations.

Questions for authors

The questions have been written in the Weaknesses part. What concerns me most about this paper is reproducibility. If the authors can guarantee to open-source the code and the method is fully reproducible, it will further enhance my positive impression of this paper, making me believe that it is fully worthy of acceptance.

Reviewer_CneWOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Comparative Evidence Policy Optimization (CEPO) for reference-guided industrial anomaly reasoning. The method consists of two stages. First, Comparative Attention Supervision (CAS) uses anomaly masks to guide the model's attention toward defective Query regions and the corresponding normal Reference regions during supervised fine-tuning. The second stage introduces Comparative Evidence Grounding (CEG), which combines visual feature discrepancies and attention-based comparison paths to construct an additional reward for policy optimization. The method is evaluated on MMAD and MMR-AD, with improvements reported in anomaly understanding, detection, and localization.

Critical strengths

The idea of explicitly incorporating Query–Reference evidence into policy optimization is interesting, especially for reference-guided anomaly reasoning. The reported results are strong on MMAD, where the 7B model achieves an average accuracy of 89.57%. The MMR-AD experiments also show improvements in average detection and localization performance under leave-one-dataset-out evaluation. Table 4 further shows that the comparative reward provides additional gains over standard outcome-based RL.

Critical weaknesses

W1. The MMAD comparisons involve different training-data protocols across methods, and it is unclear whether the evaluation protocols are consistent. The authors use 20% of MMAD for training and evaluate on the remaining 80%, whereas JUDO [1], for example, uses 293 MMAD images alongside additional Real-IAD and domain-specific QA data. It is also unclear whether the reported baselines were evaluated on the same held-out subset. Given these differences, it is difficult to assess whether the performance advantages over prior methods reported in Table 1 come from CEPO itself or from differences in training data and potentially different evaluation settings.

W2. The ablations do not sufficiently isolate the contribution of the proposed comparative evidence mechanism. Table 3 includes a Query-only grounding ablation during SFT, but there is no corresponding comparison with a Query-only grounding reward during RL. Table 4 shows that adding the comparative reward improves performance over standard outcome-based RL, but the individual CEG components, including the discrepancy map and direct/indirect comparison paths, are not separately evaluated. Given that methods such as Saliency-R1 [2] already optimize visual grounding through reward signals, it is unclear how much of the improvement specifically comes from Query–Reference comparison.

W3. I am also not fully convinced that the attention-based signals reliably reflect comparative reasoning. In Eq. (4), Query and Reference grounding scores are normalized separately, so high Reference grounding does not necessarily indicate substantial attention to the Reference. The direct comparison path in Eq. (9) also measures attention to matched Reference regions without requiring attention to the corresponding Query region. Although Eq. (6) introduces a bridge loss to encourage the final answer to attend to the generated reasoning, this does not establish that the prediction actually depends on the identified comparative evidence. The paper does not directly evaluate this dependence.

Overall, the performance improvements are promising, but the MMAD comparisons are difficult to interpret without more consistent experimental protocols. I also find the evidence for the proposed comparative reasoning mechanism insufficient, particularly given the missing ablations and the reliance on attention-based signals.

References

[1] Kang et al., JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. ICLR 2026.

[2] Gong et al., Saliency-R1: Enforcing Interpretable and Faithful Vision-Language Reasoning via Saliency-Map Alignment Reward. CVPR 2026.

Questions for authors

Q1. Were the MMAD baselines evaluated on the same held-out subset, and was the train/test split performed at the image level or question-answer level?

Q2. How would CEPO compare with an otherwise identical RL setup that replaces the comparative evidence reward with a Query-only anomaly-grounding reward, while keeping the same CAS-SFT initialization, task rewards, and training budget?

Q3. Have the authors tested whether predictions actually depend on the identified Reference evidence, for example by replacing or perturbing the Reference image?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.