acceptodds
Under review as a conference paper at ICLR 2027

Teaching MLLMs to Think Like Quality Inspector via Hierarchical Reasoning

Abstract

Driven by the rapid evolution of foundation models, the paradigm of industrial anomaly detection is shifting from traditional deep learning approaches to Multimodal Large Language Models (MLLMs). While MLLM-based methods have demonstrated impressive capabilities, existing works often fail to satisfy the requirements of real-world industrial production. Current research predominantly focuses on determining merely whether a defect exists, while neglecting that industrial quality inspection and subsequent process adjustments rely heavily on diagnostic details, i.e., which object is involved, why a defect is suspected, and what judgment is made. Inspired by the diagnostic process of human inspectors, we abstract object and defect morphologies into generalized structured knowledge and propose Hierarchical Knowledge Preference Optimization (HKPO). Through hierarchical branch-level and segment-level preference optimization, the model learns to identify objects, reference structured knowledge, and judge defects, thereby ensuring a rational and well-grounded quality inspection process. Extensive experiments on public benchmarks show that our framework lifts a lightweight 8B model 7.15% above its own backbone and 2.71% above a 4 larger 32B counterpart, surpasses the strong proprietary model by 5.46% in average accuracy, and outperforms dedicated anomaly-detection MLLMs by at least 7.95% on anomaly discrimination.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multimodal large language models, industrial anomaly detection, preference optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_z6WFOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ADThinker, a framework for industrial anomaly understanding using Hierarchical Knowledge Preference Optimization (HKPO). The authors construct a knowledge base from Real-IAD and use it to generate preference pairs following three stages: object identification, defect knowledge, and answer prediction. HKPO combines adaptive branch-level weighting with segment-level preference optimization. Experiments on MMAD and SSGD show improvements over the backbone and several existing MLLMs.

Critical strengths

S1. ADThinker achieves strong results on MMAD, particularly in anomaly discrimination, where it substantially improves over its backbone and several existing anomaly-specific MLLMs. The improvement on SSGD is also encouraging.

S2. The paper provides useful comparisons against different post-training strategies and ablations on the proposed optimization components. The results suggest that both branch-level and segment-level optimization contribute to the overall performance.

Critical weaknesses

W1. Sec. 2 argues that existing methods insufficiently ground anomaly reasoning in visual evidence and anomaly semantics. However, JUDO [1] directly addresses both through segmentation-based visual reasoning, domain-knowledge SFT, and domain-oriented GRPO. Despite its close relevance to the paper's motivation, JUDO is completely omitted from both the related-work discussion and the experimental comparisons in Table 1, even though it also reports results on MMAD. This makes it difficult to assess the claimed research gap and how the proposed approach differs from existing work.

W2. The generalization claims seem stronger than what the experiments demonstrate. The authors explicitly claim generalization to unseen defects in Sec. 1, but Real-IAD and MMAD contain similar defect morphologies. The SSGD experiment evaluates unseen object categories, which does not necessarily demonstrate generalization to unseen defect types.

W3. The paper emphasizes learning a hierarchical diagnostic reasoning process, but the evaluation relies on multiple-choice accuracy. There is no direct evaluation of the correctness of the generated intermediate reasoning, making it difficult to determine whether the improvements reflect better diagnostic reasoning or primarily better answer selection.

References

[1] Kang et al., JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. ICLR 2026.

Questions for authors

Q1. Does ADThinker generate the full <context>...</context><answer>...</answer> response during MMAD evaluation? It would be helpful to see actual generated reasoning on held-out samples, particularly logical anomalies, to better understand how the proposed reasoning process works in practice.

Q2. Could the authors clarify the overlap in defect categories between Real-IAD and MMAD/SSGD? In particular, are there evaluation defect types that are not represented in the constructed knowledge base?

Reviewer_9n5SOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ADThinker, an 8B multimodal large language model for industrial anomaly detection that follows a hierarchical diagnostic process involving object identification, defect knowledge reasoning, and anomaly judgment. It further introduces Hierarchical Knowledge Preference Optimization (HKPO), which uses generalized structured knowledge together with branch-level and segment-level preference optimization to improve industrial anomaly reasoning and detection performance.

Critical strengths

The paper provides an intuitive and practically motivated formulation of industrial anomaly diagnosis by organizing the prediction process into object identification, defect knowledge reasoning, and final judgment. This hierarchical view is well aligned with the workflow of human quality inspection and offers a interesting way to incorporate structured defect knowledge into MLLM training.

Critical weaknesses
  1. The manuscript does not clarify how normal images are handled during training. If normal samples are excluded from SFT or HKPO, it is unclear whether this may bias the model toward anomalous predictions and increase false positives on normal samples.
  2. The claimed hierarchical reasoning is not directly validated. Existing experiments mainly report downstream performance and optimization statistics, but do not show the model’s actual reasoning outputs on test samples, making it difficult to verify whether the gains indeed arise from the proposed object–defect knowledge–answer reasoning process.
Questions for authors
  1. How are normal samples used in SFT and preference optimization? If normal samples are absent or underrepresented during training, could this bias anomaly discrimination toward anomalous predictions and increase false positives?
  2. Could the authors provide actual hierarchical reasoning outputs at inference time, showing how ADThinker performs the object–defect knowledge–answer reasoning process on test samples?
  3. How does zero-shot generalization on SSGD manifest for unseen object or defect patterns? Qualitative examples would help clarify how ADThinker uses learned general defect patterns and knowledge to make predictions.
Reviewer_A8rGOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces ADThinker, a Qwen3-VL-8B model trained with 10,000 Real-IAD instruction samples followed by hierarchical knowledge preference optimization on 1,000 selected samples. A 67-item knowledge base supports preference construction, while adaptive branch weights and segment-weighted DPO emphasize different diagnostic errors. The model achieves 81.66 percent average accuracy on one-shot MMAD and is also evaluated with domain knowledge and on SSGD.

Critical strengths
  1. The same-backbone improvement is substantial and spans several tasks. Table 1 reports an increase from 74.51 to 81.66 percent average accuracy, including defect localization from 60.34 to 72.62 and description from 69.15 to 83.21.

  2. Organizing preference errors around object identity, defect knowledge, and final judgment is a sensible task-specific learning bias. It offers a more targeted way to use inspection knowledge than treating every token or rejected response identically.

Critical weaknesses
  1. The main innovation is a task-specific preference weighting scheme, and its incremental contribution is considerably narrower than the overall performance gain. Table 4 assigns 5.35 of the 7.15 percentage-point improvement to answer-only SFT; HKPO contributes another 1.80 and exceeds DPO by 1.30. The normalized segment score in Equations 9 and 10 follows Equation 7 of the cited CHiP. In Table 6, adding adaptive weighting to segment weighting raises the average from 81.12 to 81.66. This does not make the gain unimportant or statistically insignificant, but it places the burden on showing why the branch structure is a useful general learning principle beyond an effective combination of established objectives.

  2. The hierarchy's meaning is not convincingly demonstrated by its supervision. Equation 4 assigns a branch according to the first incorrect object, defect, or answer stage, yet Figure 6's object-branch example contains two generic Missing Parts descriptions with the same [object] placeholder and different final answers. It does not show an explicit object-identification error. If the actual pairs primarily contrast generic text variants and answer labels, the method functions as weighted preference tuning without establishing the claimed diagnostic decomposition. An audit of the branch semantics would directly test the distinctive idea.

  3. The evaluation does not show that the hierarchical training produces better grounded reasoning. Sections 4.1 and 4.3 measure option accuracy and F1, while Figures 4, 5, and 9 report optimization statistics. These results are compatible with improved answer selection from additional knowledge-bearing supervision. A matched SFT control using the preferred reasoning responses, together with an image-specific explanation assessment, would distinguish the value of hierarchical preference learning from simply supplying richer training targets. This distinction is central to the claim of teaching inspector-like reasoning.

Questions for authors

What differences in the actual preference records distinguish an object error from a defect-knowledge error? Are branch labels independently checked?

2 comments
Reviewer_A8rGOpenReview ↗

Response to Authors

Thank you for the targeted revisions. The branch-label audit, full-response SFT+DPO control, and intermediate-diagnosis evaluation substantially address my main concerns. These results provide better evidence that the gains extend beyond richer supervision and final-answer selection alone. The incremental novelty remains modest, and improved diagnostic outputs do not fully establish the claimed reasoning hierarchy, but I find the additional evidence supportive of my current weak-accept recommendation.

AuthorsOpenReview ↗

Thank You for Your Time and Feedback!

Dear Reviewer A8rG,

Thank you for recognizing our revisions and for your continued support. We sincerely appreciate your rigorous and constructive feedback, which has been instrumental in strengthening this work. Your remaining suggestions will be carefully addressed in the final version.

Best regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.