- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies an interesting problem in which industrial anomaly detectors can achieve strong benchmark performance (e.g., high AUROC and AP scores) while relying on irrelevant visual factors, such as illumination, reflection, or benign texture variations, rather than the intended inspection criteria, such as cracks or contamination. The paper first exposes this issue using matched counterfactual response probes involving two controlled factors and identifies three different effects: the target defect effect, the tolerated-factor effect, and their interaction. The authors then propose a lightweight specification-conditioned counterfactual repair method called SC-CGR, which aims to align these responses with predefined constraints while preserving existing detection performance. Finally, the authors evaluate their method across several industrial datasets. The experiments show that the proposed method substantially improves inspection-contract satisfaction, although some categories reveal a trade-off between inspection-contract satisfaction and the original anomaly-detection performance.
Critical strengths
Strength 1: One strength of this paper is its clear and mathematically grounded formulation of the problem. The authors translate the high-level goal of making anomaly detectors follow inspection specifications into explicit and measurable constraints. For example, the authors use factorial decomposition to systematically separate the target effect, the tolerated-factor effect, and the interaction between the two, which are then incorporated into the proposed repair objective. The authors also clearly explain how this objective can be optimized. Overall, the framework connects diagnosis, optimization, and evaluation in a consistent way.
Strength 2: The paper evaluates its method across several industrial anomaly-detection datasets and compares it with several repair baselines, including pairwise ranking, pairwise + invariance, and binary supervision. I particularly appreciate the ablation studies and the reported cases where simpler methods outperform the proposed approach. These comparisons help illustrate both the strengths and limitations of the method.
Strength 3: I think the topic of this study is important and interesting for anomaly detection, as strong anomaly-detection performance does not necessarily mean that models follow the intended inspection behavior. For example, the paper presents cases where anomaly detectors with high AUROC/AP scores still violate the specified counterfactual response constraints.
Critical weaknesses
Weakness 1: While the paper has a clean mathematical framework, I am not quite sure how well this framework fits actual industrial needs. In Equation (7), the proposed feasible response region includes ,, which aims to achieve a near-zero factorial interaction between the target effect and the tolerated-factor effect, as defined in Equation (4). However, it is unclear whether this requirement comes from actual industrial inspection standards or was manually defined by the authors. While score-level robustness can be useful, the paper does not clearly justify why near-zero factorial interaction is necessary for the specific industrial inspection tasks considered. Correct accept/reject decisions do not necessarily require this constraint, which raises questions about whether it imposes additional requirements beyond the actual inspection specifications.
Weakness 2: The paper seems to have an incomplete evaluation of real-data performance retention. While Tables 2 and 3 compare the repaired detector against the original detector using AUROC/AP, Tables 4 and 6 do not include the unrepaired baseline. This is particularly concerning for the harder categories in Table 6, where the repaired detectors sometimes achieve relatively low real-data AP. Since preserving the original detector's performance is a central goal of the paper, I would like to see direct before-and-after repair comparisons of AUROC/AP across all evaluated categories under the same experimental protocol. This would help clarify how much real anomaly-detection performance is sacrificed to satisfy the proposed inspection contracts.
Weakness 3: It is unclear to me how to choose between the proposed repair method and simpler baseline repair methods for new tasks. I appreciate that the authors acknowledge that the full factorial objective does not consistently outperform simpler methods. While Sections 5.7 and 6 provide some qualitative explanations based on response geometry, it remains unclear how these observations can be generalized to guide the selection of a repair method for a new task.
Questions for authors
Question 1: Can the authors provide further justification for all the constraints in Equation (7), particularly the near-zero interaction requirement? Can the authors explain how these constraints align with or are derived from actual industrial specifications or expert judgments? Can the authors further demonstrate that satisfying these constraints improves decision-level inspection correctness?
Question 2: Can the authors provide the original and unrepaired baseline results for Tables 4 and 6?
Question 3: Can the authors provide more concrete, preferably quantitative, guidance on how the pre-repair response geometry can predict whether the full factorial objective will outperform simpler pairwise or invariance methods? If this turns out to be very challenging, can the authors elaborate on their proposed heuristics and provide some empirical evidence of their effectiveness in selecting repair methods?