acceptodds
Withdrawn from ICLR 2027

Specification-Conditioned Counterfactual Repair for Industrial Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Industrial anomaly detectors can achieve strong benchmark performance while relying on visual mechanisms that do not match the intended inspection logic. For example, a detector may respond strongly to a tolerated appearance change, overlook the specified defect, or depend on interactions between the two. We study this problem as post-training repair under an explicit inspection specification rather than as learning from a new set of anomaly labels. Our approach uses matched counterfactual probes to separate the detector's responses to the target defect, tolerated variation, and their interaction, and evaluates these responses against a user-defined behavioral contract. Based on this formulation, we introduce Specification-Conditioned Counterfactual Gap Repair (SC-CGR), which adjusts a fitted detector through a lightweight head on frozen features while using typed replay and ranking constraints to preserve trusted behavior on real images and clean controls. Across five industrial anomaly-detection datasets, we systematically screen a 71-category inventory to obtain 37 feasible inspection contracts, revealing semantic response violations that are not reflected by the area under the receiver operating characteristic curve or average precision. Evaluation of the 13 contracts taken to the repair stage shows that SC-CGR substantially improves strict contract satisfaction on MVTec AD bottle and metal nut and generalizes to counterfactual probes from held-out Real-IAD bottle-cap anchors. The broader evaluation also identifies cases where pairwise constraints alone are sufficient and cases where enforcing the specification conflicts with retaining real-data behavior. These results show that specification-conditioned counterfactual responses provide a practical interface for auditing and repairing fitted industrial inspection systems beyond conventional anomaly-detection accuracy.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
industrial anomaly detection, post-training model repair, counterfactual evaluation, behavioral specification, semantic robustness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HLdiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies an interesting problem in which industrial anomaly detectors can achieve strong benchmark performance (e.g., high AUROC and AP scores) while relying on irrelevant visual factors, such as illumination, reflection, or benign texture variations, rather than the intended inspection criteria, such as cracks or contamination. The paper first exposes this issue using matched counterfactual response probes involving two controlled factors and identifies three different effects: the target defect effect, the tolerated-factor effect, and their interaction. The authors then propose a lightweight specification-conditioned counterfactual repair method called SC-CGR, which aims to align these responses with predefined constraints while preserving existing detection performance. Finally, the authors evaluate their method across several industrial datasets. The experiments show that the proposed method substantially improves inspection-contract satisfaction, although some categories reveal a trade-off between inspection-contract satisfaction and the original anomaly-detection performance.

Critical strengths

Strength 1: One strength of this paper is its clear and mathematically grounded formulation of the problem. The authors translate the high-level goal of making anomaly detectors follow inspection specifications into explicit and measurable constraints. For example, the authors use factorial decomposition to systematically separate the target effect, the tolerated-factor effect, and the interaction between the two, which are then incorporated into the proposed repair objective. The authors also clearly explain how this objective can be optimized. Overall, the framework connects diagnosis, optimization, and evaluation in a consistent way.

Strength 2: The paper evaluates its method across several industrial anomaly-detection datasets and compares it with several repair baselines, including pairwise ranking, pairwise + invariance, and binary supervision. I particularly appreciate the ablation studies and the reported cases where simpler methods outperform the proposed approach. These comparisons help illustrate both the strengths and limitations of the method.

Strength 3: I think the topic of this study is important and interesting for anomaly detection, as strong anomaly-detection performance does not necessarily mean that models follow the intended inspection behavior. For example, the paper presents cases where anomaly detectors with high AUROC/AP scores still violate the specified counterfactual response constraints.

Critical weaknesses

Weakness 1: While the paper has a clean mathematical framework, I am not quite sure how well this framework fits actual industrial needs. In Equation (7), the proposed feasible response region includes ∣Δ12sϕ∣≤ε|\Delta_{12}s_{\phi}|\leq\varepsilon,, which aims to achieve a near-zero factorial interaction between the target effect and the tolerated-factor effect, as defined in Equation (4). However, it is unclear whether this requirement comes from actual industrial inspection standards or was manually defined by the authors. While score-level robustness can be useful, the paper does not clearly justify why near-zero factorial interaction is necessary for the specific industrial inspection tasks considered. Correct accept/reject decisions do not necessarily require this constraint, which raises questions about whether it imposes additional requirements beyond the actual inspection specifications.

Weakness 2: The paper seems to have an incomplete evaluation of real-data performance retention. While Tables 2 and 3 compare the repaired detector against the original detector using AUROC/AP, Tables 4 and 6 do not include the unrepaired baseline. This is particularly concerning for the harder categories in Table 6, where the repaired detectors sometimes achieve relatively low real-data AP. Since preserving the original detector's performance is a central goal of the paper, I would like to see direct before-and-after repair comparisons of AUROC/AP across all evaluated categories under the same experimental protocol. This would help clarify how much real anomaly-detection performance is sacrificed to satisfy the proposed inspection contracts.

Weakness 3: It is unclear to me how to choose between the proposed repair method and simpler baseline repair methods for new tasks. I appreciate that the authors acknowledge that the full factorial objective does not consistently outperform simpler methods. While Sections 5.7 and 6 provide some qualitative explanations based on response geometry, it remains unclear how these observations can be generalized to guide the selection of a repair method for a new task.

Questions for authors

Question 1: Can the authors provide further justification for all the constraints in Equation (7), particularly the near-zero interaction requirement? Can the authors explain how these constraints align with or are derived from actual industrial specifications or expert judgments? Can the authors further demonstrate that satisfying these constraints improves decision-level inspection correctness?

Question 2: Can the authors provide the original and unrepaired baseline results for Tables 4 and 6?

Question 3: Can the authors provide more concrete, preferably quantitative, guidance on how the pre-repair response geometry (Δ1,Δ2,Δ12)(\Delta_1, \Delta_2, \Delta_{12}) can predict whether the full factorial objective will outperform simpler pairwise or invariance methods? If this turns out to be very challenging, can the authors elaborate on their proposed heuristics and provide some empirical evidence of their effectiveness in selecting repair methods?

Reviewer_Atp7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether an industrial anomaly detector follows an intended inspection rule, beyond achieving good AUROC or AP. It uses matched four-image probes to measure responses to a target defect, a tolerated variation, and their interaction. SC-CGR trains a small head on frozen features to satisfy these response constraints while retaining useful detection behavior. The evaluation audits 37 contracts across five datasets and studies repair on 13. Results show successful repair in several cases, including held-out anchors, but also cases where simpler objectives suffice or detection performance suffers.

Critical strengths
  • The paper addresses a meaningful gap in anomaly detection evaluation. High ranking accuracy does not tell us whether a detector responds appropriately to defects and tolerated changes. The matched-probe formulation provides a concrete way to examine this distinction.

  • The evaluation includes useful negative results. The pairwise-sufficient cases and the retention failures make the paper more informative than a study limited to successful repairs.

Critical weaknesses
  • The assumptions behind the counterfactual probes need better justification. The method assumes that the edits isolate the intended defect and tolerated variation, and that the tolerated variation should leave the defect response unchanged. This is not self-evident: a reflection edit could change defect visibility even if reflection itself is acceptable under the inspection rule. The paper provides too few concrete examples and generation details to assess these assumptions.

  • The motivation for explicit interaction alignment is not yet convincing. In the released implementation, the pairwise-invariance baseline suppresses the effect of the tolerated factor both with and without the defect. This already constrains the interaction between the two factors. The reported advantage of SC-CGR therefore does not establish that the baseline lacks interaction information; it may instead reflect differences in loss shape, weighting, or alignment with the evaluation thresholds. The paper should distinguish these explanations.

  • Figure 1 appears to be AI-generated and needs more careful preparation. The generic visual style, crowded layout, and small text make it less useful than it should be as the main overview of the method.

Questions for authors
  • How were the edits generated and validated? What justifies requiring an unchanged defect response under these edits?

  • Since the implemented pairwise-invariance baseline already constrains interaction effects, what explains the benefit of explicit interaction alignment?

Reviewer_MLxKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper deals with Industrial anamoly detectors, which are statistical/AI models that are trained to check whether images have anamolies or defects. Such detectors are judged on whether they can catch these defects. However, when processing these defects, there may be two situations: one where a serious/harmful defect may require rejection of a product, second a harmless nuisance that is not a defect and is thus tolerated. The authors argue that existing detectors conflate these errors and do not distinguish between them, as perhaps they have not been trained to do so.

In order to distinguish them the authors come up with the idea of matched counterfactual response probes, where starting from a so-called anchor image, they produce 3 other responses, first a synthetically induced defect in the image matching the anchor image, second a nuisance response on the original image, and finally the nuisance applied on top of the defect. The idea is that the defect should be identified, while the nuisance should not affect its identification. At the same time the nuisance by itself should not be identified. The authors then develop a repair technique (called SCCGR) to handle these errors using an appropriate optimization function. This repair should maintain the original detectors behavior on good data while correcting these defects, and not being affected by the nuisance.

To test this idea the authors run experiments on 5 datasets which together contain 71 categories. They explain their results on a few in detail. Their results show that indeed fitted detectors suffer from the above errors. But the defects are artificially introduced if I understand correctly. Second, the counterfactual approach does identify and repair the errors in several synthetic examples. They show that repair also fails in some cases. Finally they conclude that their approach for checking compliance or effectivity of the detectors can be used in tandem with other existing approaches for ranking quality.

Critical strengths

On the positive side, I believe the paper addresses an interesting statistical problem with important practical applications in industrial contexts where AI is used. The formulation is simple at its core and hence possibly more usable in practice. The ideas are validated by experiments, which even if synthetic seem convincing to some extent.

Critical weaknesses

On the negative side, the paper is really written for an expert audience and does not seem accessible to a wider AI audience. Many important terms are not defined (e.g, anchor, hard anchor, retention data, trusted retention data, typed retention, clean control ("benign controls" elsewhere), factorial, response contract, inspection contract, diagnosed gap, hard-anchor active set, and too many more to state here). Even the term SCCGR is defined differently and with different variants in different places.

Further even the introduction does not make it accessible. From a technical point of view the idea is simple which is not to say its not effective. But the effectivity is not measured on practical examples. As the introduced defects and nuisances seem synthetic, there seems no real case-study. More importantly it is not exactly clear how these defects or nuisances were introduced. Finally, even some parts of the experiments section are impossible to understand. What does``Metal nut tests a harder surface-defect/reflection contract.'' mean? Harder than what? tests what? Many subsections have such a leading sentence that left me perplexed.

Overall, as a non-expert, it took me a huge amount of time to parse the terminology (maybe a separate and clear preliminaries section is needed), and I am still not sure I understand a lot of this. Despite this I feel the paper is less about a new theoretical or deep idea but more about a practical issue. Given this I feel the experiments and analysis thereof need to be even more rigorous.

Questions for authors
  1. If only 13/37 (or is it 71?) benchmarks go into repair does this mean that the approach of this paper is only examined on 13? What happens if the repair is done on others?

  2. On these 13, how is the comparison wrt other similar methods done? What were the baselines and were they run on all 13?

  3. How were the defects and nuisance/tolerance introduced? Were some of them real nuisances like glare or appearance? or are they all synthetic?

  4. The method in the paper replaces the requirements of Eq. 6 with penalty terms in the loss (Eq. 12). Are there formal guarantees that the trained detector actually increases its score for the defect and leaves it unchanged for the nuisance? And is that always possible while keeping the old behavior on good data?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.