Normal-Quotient Residual Readout for Structured Inspection QA

Structured industrial inspection QA verifies which candidate defect hypothesis is supported by an image, rather than generating an unrestricted report. These candi dates can come from inspection sheets, taxonomies, retrieval modules, or human operators. Large multimodal language models provide a flexible interface, but their cost, latency, and deployment constraints motivate lighter evidence-grounded alter natives. We ask whether a frozen CLIP backbone can support this structured setting with a better visual readout. Our key observation is that comparing a query image with a matched normal reference exposes residual defect evidence. This evidence is not just scalar abnormality: defect families form coherent residual directions. However, raw residuals also vary with the chosen normal reference. We therefore in troduce a normal-quotient residual readout that removes empirical normal-variation directions and explains the remaining evidence with shared-private defect sub spaces. Candidate hypotheses are scored by how well routed subspaces explain the residual evidence, and the same evidence is reused for patch-wise grounding. On AnomalyCoT and MMAD, our readout substantially improves over vanilla CLIP and reaches a competitive regime on structured localization and description, with much lower latency and memory than 4B–8B MLLMs.