acceptodds
Withdrawn from ICLR 2027

Normal-Quotient Residual Readout for Structured Inspection QA

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Structured industrial inspection QA verifies which candidate defect hypothesis is supported by an image, rather than generating an unrestricted report. These candi dates can come from inspection sheets, taxonomies, retrieval modules, or human operators. Large multimodal language models provide a flexible interface, but their cost, latency, and deployment constraints motivate lighter evidence-grounded alter natives. We ask whether a frozen CLIP backbone can support this structured setting with a better visual readout. Our key observation is that comparing a query image with a matched normal reference exposes residual defect evidence. This evidence is not just scalar abnormality: defect families form coherent residual directions. However, raw residuals also vary with the chosen normal reference. We therefore in troduce a normal-quotient residual readout that removes empirical normal-variation directions and explains the remaining evidence with shared-private defect sub spaces. Candidate hypotheses are scored by how well routed subspaces explain the residual evidence, and the same evidence is reused for patch-wise grounding. On AnomalyCoT and MMAD, our readout substantially improves over vanilla CLIP and reaches a competitive regime on structured localization and description, with much lower latency and memory than 4B–8B MLLMs.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Anomaly detection, Computer Vision, Vision-Language-model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nBiJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Problem Setting

  • Structured inspection QA is required for intuitive use of the inspection system
  • Vision-language foundation models are used to ground inspection results, but their deployment is limited in edge computing environments.
  • As a lightweight alternative backbone, a frozen CLIP model can be considered. However, standard CLIP readouts are dominated by coarse object appearance and often fail to capture weak, localized defect evidence.

Method

  • The proposed method extracts patch-level residuals between a query image and a matched normal reference using a frozen CLIP backbone. It then projects out empirical normal variations estimated from normal-normal pairs (normal-quotient projection), scores candidate text hypotheses by measuring how well they are explained by offline-fitted shared-private defect subspaces, and finally reuses the routed subspace energies at the patch level to achieve decision-aligned grounding.

Claim

  • Normal-referenced CLIP residuals reveal coherent, defect-family directional structures rather than simple scalar abnormality magnitudes.

  • Projecting out empirical normal variations via the normal-quotient operation effectively eliminates reference-dependent nuisance signals and stabilizes decision margins.

  • Shared-private defect subspaces capture intra-family defect variations without suffering from prototype collapse, enabling sensitive scoring of dynamic candidate hypotheses.

  • The routed subspace energies can be reused directly for patch-level localization, achieving decision-aligned grounding without requiring an auxiliary detection head or mask supervision.

  • A frozen 0.15B CLIP backbone equipped with the proposed readout achieves competitive structured inspection QA performance against 4B–8B MLLMs while requiring substantially lower latency (56.30 ms) and peak memory (1.62 GB).

Critical strengths
  1. This paper is generally easy to follow
  2. The authors conduct extensive experiments and analyze the proposed method from multiple perspectives
Critical weaknesses
  1. Despite improvements over the vanilla CLIP baseline, the absolute performance on AnomalyCoT remains limited (41.19% accuracy and 0.1742 IoU). These results raise concerns about whether the proposed method can achieve the level of reliability required for practical industrial inspection. The authors should clarify the intended deployment scenarios and discuss whether the reported performance is sufficient for such applications.

  2. The proposed method relies on the availability of abnormal samples for offline training. However, abnormal data are often scarce in industrial inspection settings, and the available samples may not adequately represent the diversity of defects encountered in practice. This reliance on abnormal support data may limit the applicability of the proposed method in real-world industrial environments

  3. Table 1 compares methods evaluated under different adaptation and supervision settings. While several baselines are evaluated in zero-shot or non-adapted settings, the proposed method is trained offline using four defect samples per family. This discrepancy raises concerns about the fairness of the comparison. In my experience, even a small number of adaptation steps can substantially improve foundation models' performance on a target dataset, particularly when adaptation is limited to a lightweight projection layer, as in this paper. Therefore, evaluating the baselines under comparable adaptation and supervision settings would provide a fairer basis for comparison.

  4. In Section 3.1, the operator Pool performs global average pooling over the patch-level residuals rir_{i}, yielding a single global representation (r). This operation collapses the local residuals into a single vector, making it conceptually similar to a global residual prototype. However, the authors distinguish their approach from prototype-based readouts without clearly explaining how the pooled representation differs from a prototype. This ambiguity may mislead readers regarding the actual distinction between the proposed method and prototype-based approaches. The authors should clarify this distinction, particularly given that (r) is used for option scoring.

  5. The central contribution is claimed to be the identification of a “residual readout bottleneck” in structured inspection QA. However, this bottleneck is neither clearly defined nor convincingly established. If the claim concerns the representational capacity of CLIP, it is difficult to justify, given the authors' own premise that CLIP residuals already contain defect-family directions. If, instead, the bottleneck lies in how these representations are read out, the paper does not sufficiently establish why this constitutes a fundamental problem specific to structured inspection QA, rather than a limitation of the particular readout strategies considered.

This weakens the conceptual clarity and significance of the proposed contribution.

Questions for authors
  1. Why did authors choose CLIP as the backbone for the proposed method ? Have authors considered alternative vision-language encoders, such as SigLIP, SigLIP 2, or other recent VLMs ? It would be helpful to understand whether the proposed method is specific to CLIP or generalizes to other VLM representations.
Reviewer_MkJgOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper adapts frozen OpenCLIP to structured industrial inspection by ranking supplied textual hypotheses. It projects query-minus-normal features away from estimated normal variation and learns shared and defect-family-private subspaces with text-dependent routing. Patchwise projection energies provide evidence maps. Using four abnormal supports per family for offline fitting, the method reports 41.19 AnomalyCoT accuracy and 69.80 MMAD average.

Critical strengths
  1. The method offers an efficient implementation of candidate-based inspection. Its reported 56.30 ms median latency and 1.62 GB peak memory are meaningful advantages when the candidate taxonomy and suitable normal references are already available (Table 3).

  2. The controlled readout comparisons show that the learned representation is useful. Under matched backbone and support data, MMAD improves from 58.08 with quotient-linear scoring to 69.80 with the full readout (Table 5). Table 6 also provides evidence against answer-string memorization or text-only shortcuts, and Table 7 demonstrates partial transfer to families excluded from fitting.

Critical weaknesses
  1. The principal significance question is what additional diagnostic capability the structured QA formulation establishes beyond a supervised, text-conditioned defect classifier. The system learns from labeled defect supports and selects among supplied hypotheses; the benchmark supplies the candidate vocabulary needed to express the correct diagnosis. This interface has legitimate applications, but the paper's value must therefore rest on showing consequential flexibility within it. Table 6 rules out simple memorization, yet its novel-answer split concerns unseen answer strings, which does not by itself establish new defect concepts or robustness to changed candidate composition. Table 7 is stronger: it removes four families and their private subspaces, but retains their hypotheses at inference, and family-macro accuracy falls from 57.14 to 46.07 on AnomalyCoT and from 79.03 to 68.24 on MMAD. The null evaluation likewise does not test deliberately removing the correct hypothesis (Appendix E.3). These results support useful but bounded transfer. They leave insufficient evidence that the QA interface adds a substantial capability over ordinary defect recognition with textual label matching, rather than presenting that capability through benchmark question sheets. Open-ended generation is not necessary; establishing reliable behavior when candidate descriptions, distractors, or coverage change would directly substantiate the proposed interface's value.

  2. The efficiency comparison with general-purpose MLLMs does not resolve the more relevant question of value relative to lightweight task-specific alternatives. Section 4 distinguishes four abnormal supports per family for offline fitting from four normal ICL examples for MLLMs. Appendix D.3 usefully supplies abnormal demonstrations, but gives one abnormal image per family through online ICL, so neither supervision budget nor adaptation is matched. The resulting comparisons demonstrate a competitive deployment option, but cannot isolate how much advantage comes from the readout rather than supervised specialization. The matched linear controls in Table 5 establish a benefit over those controls; they do not establish the best accuracy-cost trade-off achievable by a comparably fitted lightweight classifier or retrieval-based hypothesis scorer. This matters because most MMAD improvement appears with shared-private modeling: quotienting alone changes the linear baseline from 58.06 to 58.08, whereas the unconstrained subspace readout reaches 68.88. A stronger matched specialist comparison would test whether this construction offers a distinctive practical advantage. The MLLM results are also task-dependent: the best tested Qwen2.5-VL generation protocol reaches 67.10 MMAD versus 69.80 for this method, while Qwen remains stronger on AnomalyCoT (Table 9). Low inference cost is a real strength, but does not by itself establish the significance of the proposed mechanism.

Questions for authors

What inspection decisions become possible or more reliable through dynamic hypothesis scoring than through supervised defect recognition with textual label matching? What happens when plausible distractors are added or the correct defect hypothesis is absent?

Reviewer_Qt4MOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a normal-quotient residual readout for structured industrial inspection QA. It removes normal-reference variation from CLIP residual features and uses shared-private defect subspaces for candidate scoring and localization.

Critical strengths

The normal-quotient formulation addresses a relevant problem in reference-based industrial inspection. The shared-private subspace decomposition improves several inspection QA metrics over point-prototype and linear readouts.

Critical weaknesses
  1. The proposed method combines feature subtraction, nuisance-subspace projection, and shared-private representation learning. These are established techniques, and the main algorithmic novelty is not sufficiently distinguished from existing methods.
  2. There is a potential inconsistency in the training procedure. The normal-variation basis U0 is estimated from residuals involving the trainable projector Wp, but U0 is then fixed while Wp continues to be optimized. It is unclear whether the estimated nuisance subspace remains valid after Wp changes.
  3. The method depends on category-matched normal reference images at inference time. This assumption limits its applicability to inspection scenarios where suitable references are unavailable.
  4. Patch-wise subtraction assumes correspondence between the query and reference feature locations.
  5. Reference selection still affects the results. Table 15 reports an AnomalyCoT accuracy of 38.20% under alternative references, compared with 41.19% under the default reference.
  6. The robustness experiments cover only moderate transformations. The reported 28-pixel translation already reduces MMAD performance by 3.20 points, while severe layout changes remain untested.
  7. The proposed method is fitted offline using labeled defect examples, whereas several MLLM baselines rely on inference-time prompting or in-context examples.
  8. The localization results remain weak in absolute terms. The reported IoU of 0.1742 is insufficient to establish reliable defect localization for practical inspection.
Questions for authors
  1. How is the fixed normal-variation basis U0 kept consistent with the trainable projector Wp?
  2. Can the method handle severe geometric misalignment between the query and normal reference?
  3. Can the authors provide comparisons using matched abnormal-support supervision for CLIP-based and MLLM baselines?
  4. How does the method behave when the correct defect hypothesis is absent from the candidate list?
  5. Can the authors report localization performance on unseen defect categories and products?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.