acceptodds
Withdrawn from ICLR 2027

What Remains Normal? Clean Images Miss Useful Near-Defect Normal Patches for Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Normal-only industrial anomaly detectors use patches from clean training images as normal references or reconstruction targets. This assumes that clean patches are sufficient for the normal regions encountered at test time. We test that assumption directly. On MVTec AD, admitting ground-truth-normal patches from real defect images to a DINOv2 memory candidate pool raises pixel average precision (P-AP) from 73.34 to 76.95 while keeping the encoder, test-time score, and number of stored references fixed. Patches within two patch cells of the annotated defect recover 94.70% of this gain. We then ask whether useful patches of this kind can be exposed using clean training images alone. BoundarySupport inserts a procedural synthetic defect to alter surrounding context, excludes every token intersecting the nominal insertion or a detected RGB change, and learns only from pixel-preserved neighboring patches. Across three paired seeds, the same principle improves P-AP in all six memory/reconstruction–dataset settings across MVTec, VisA, and Real-IAD. Matched controls identify the altered-context feature itself as the useful normal evidence: with synthetic input or selected positions fixed, altered-context features outperform their clean-view counterparts as both reconstruction targets and memory references. On MVTec memory, the final score change is also spatially selective, with larger reductions on normal patches next to defects than on mid-distance or far-normal patches in all 15 categories.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
industrial anomaly detection, anomaly localization, normal representation, memory-based detection, feature reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_b53oOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Problem. The paper questions an implicit assumption of normal-only industrial anomaly detection: that patches from clean training images are sufficient as normal references (memory-based detectors) or reconstruction targets (reconstruction-based detectors) for the normal regions seen at test time. The authors argue that, because ViT patch tokens depend on the surrounding image context, a pixel-unchanged normal patch pp can receive a feature F(x~)pF(\tilde{x})_p under altered context that differs from its clean-view feature F(x)pF(x)_p. Normal patches next to a defect can therefore have features that never appear in clean training images.

Contribution 1: fixed-budget diagnostic. On MVTec AD with a frozen DINOv2 memory detector, the authors add ground-truth-normal patches from real defective images to the memory candidate pool, keeping the encoder, the test-time score, and the bank size BB fixed. On matched held-out images, P-AP rises from 73.34 to 76.95. An expanded bank with 5.3% more references reaches 76.96, so the gain is attributed to which references are stored rather than how many. Restricting the added candidates to patches within two cells of the annotated defect recovers 94.70% of the gain. Two controls on leather and tile address alternative explanations: inpainting the visible defect before re-encoding, and transferring candidates across defect types. Both leave part of the gain in place.

Contribution 2: BoundarySupport. A clean-only method that inserts procedural synthetic defects (CutPaste-Scar, Poisson paste, local warp + paste) into clean images to change their context. It excludes every token that overlaps the insertion mask or any detected RGB change, and uses only the pixel-preserved neighboring tokens (a ring RR of radius r=2r = 2) as additional normal evidence:

  • Memory branch (ProCon readout): selected altered-context features compete with clean candidates and the bank is reselected to exactly the original size BB.
  • Reconstruction branch (frozen-encoder Dinomaly): the same positions receive an auxiliary cosine target from the altered view, weighted by 0.05, while the original clean objective is unchanged.

Validation.

  • Main results. MVTec AD, VisA, and Real-IAD, both branches, three paired seeds. P-AP improves in all six detector–dataset settings (+1.76 to +5.01) and in 329/342 category–seed pairs. Image-level metrics stay roughly unchanged or decrease slightly.
  • Matched controls. Replacing the altered-context feature with the clean-view feature at the same inputs or positions lowers P-AP; for reconstruction, it falls below the baseline.
  • Spatial analysis. For the MVTec memory branch, the score change is larger on normal patches near real defects than on mid- or far-distance normal patches, in all 15 categories.
  • Additional analyses. Failure cases are reported, along with a trade-off between localization and defect-score magnitude.
Critical strengths

1. The fixed-budget diagnostic is well designed and asks a question the literature has not asked.

The oracle changes only which candidates are available. The encoder, the test-time score, and the number of stored references stay fixed, and clean and oracle arms are evaluated on the same held-out images with donor images excluded from both. The expanded-versus-fixed-budget comparison (76.96 vs. 76.95 P-AP) further separates reference count from reference composition. Prior work on contaminated training data (SoftPatch, InReaCh, FUN-AD) asks which patches should be removed from a candidate pool. This paper asks the opposite question: whether a clean pool is missing useful normal evidence. The near-defect concentration (94.70% of the gain within two cells, and an 11.73× / 4.95× over-selection of near-defect patches by k-center in leather / tile) gives this question a concrete spatial answer.

Why it matters: this diagnostic is the main reason I consider the paper a meaningful contribution. It shows that the deficit exists under controlled conditions, independently of whether BoundarySupport fully explains or fixes it. My concerns about its scope (two dominant categories, the partial gain left after inpainting) are listed under Weaknesses and limit, but do not remove, this value.

2. The method is cheap to adopt and reported transparently within its tested scope.

  • No inference cost. BoundarySupport leaves the test-time input, the scoring path, and the bank size unchanged, so it adds no inference cost and could in principle be attached to an existing detector.
  • Seed and category consistency. Results are paired over three seeds, and sign consistency is reported at the category–seed level (171/171 pairs for memory, 158/171 for reconstruction).
  • Pre-fixed Real-IAD configuration. The Real-IAD configuration was fixed before its outcomes were inspected, which reduces the risk of tuning on the evaluation set.
  • Ablation-backed design choice. The strict exclusion of detected RGB changes is supported by a direct comparison with excluding only the nominal mask: +4.94 vs. +0.70 P-AP on MVTec, and +2.09 vs. −2.40 on VisA.
  • Failure cases reported. Failure categories (e.g., VisA cashew, PCB3, Real-IAD rolled strip base) are reported rather than omitted.

Why it matters: this reporting makes the narrow empirical claim credible. Within two DINOv2-based detectors on three industrial benchmarks, the P-AP gain is consistent and not an artifact of seed or category selection. That narrow claim is what I can support. The broader claims about generality and mechanism are addressed under Weaknesses.

Critical weaknesses

1. There is no comparison with established anomaly detection methods, so the significance of the gains cannot be judged.

Every result is a paired change relative to one memory detector (ProCon readout on frozen DINOv2) and one reconstruction detector (Dinomaly, trained per category). The paper does not compare, in absolute numbers under a common protocol, with:

  • memory-bank methods (e.g., PatchCore, and PatchCore + BoundarySupport, since the method is presented as a plug-in);
  • reconstruction/distillation methods (e.g., RD4AD, and Dinomaly in the multi-class unified setting it was designed for);
  • synthetic-anomaly methods, which are the closest methodological relatives (e.g., DRAEM, DeSTSeg, GLASS, and RealNet (CVPR 2024)).

The last group matters most. These methods also insert synthetic anomalies into clean images and train on them; BoundarySupport differs mainly in using the insertion as context and learning only from the preserved neighbors. Without a head-to-head comparison, it is unclear whether this use of the synthetic image is more effective than using the synthetic anomaly as supervision (as in RealNet or GLASS), or whether the two are complementary. The generator choice is also narrow. All three interventions are copy/paste-type operations (Appendix C), and the only generator ablation (Table 18) is a seed-0, reconstruction-only study on ten development categories. Generators such as DRAEM's Perlin-noise texture blending, MemSeg, or RealNet's diffusion-based SDAS are not tested. This is a non-trivial question for the method itself: a generator that changes pixels over a wide area could leave the ring RR nearly empty under the strict RGB-change rule (Eq. 4).

Why it matters: the paper presents BoundarySupport as a method contribution, but without baselines a reader cannot tell whether it advances the state of the art or only improves two specific detectors.

2. The evidence supports a narrower claim than the paper makes, in both generality and mechanism.

  • Single representation. All experiments use DINOv2 ViT-B/14 (without registers for memory, with registers for reconstruction). The Discussion states that "normal-pool completeness is representation-dependent," but no other representation is tested: no CNN backbone (e.g., WideResNet-50 as in PatchCore), no other ViT family (e.g., CLIP), and no layer-wise comparison. The motivating argument is that ViT self-attention mixes global context into patch tokens. A CNN with a limited receptive field is therefore the natural control for this argument, and it is missing.
  • Single domain. The title refers to anomaly detection in general, but every experiment is industrial (MVTec AD, VisA, Real-IAD). Medical anomaly detection (e.g., the BMAD benchmark: brain MRI, liver CT, retinal OCT) is a natural test of the hypothesis. There, the normal appearance of tissue depends strongly on the surrounding anatomy, and lesions often deform the tissue next to them.
  • Pixel AP only. The gains are confined to P-AP and P-F1. I-AUROC decreases in 4 of 6 settings in Table 1 (Real-IAD reconstruction: −0.52±0.04-0.52 \pm 0.04), P-AUROC decreases in 4 of 6, and mean defect scores decrease (e.g., −4.01-4.01 for MVTec reconstruction, Table 25). The paper calls this a trade-off but does not explain why it happens.
  • The diagnostic rests on two categories. 52.8% of the oracle's macro gain comes from leather (+15.58) and tile (+13.07); the other 13 categories average +1.97. These two texture categories are also the only ones used for the distance sweep, inpainting, cross-defect, and enrichment analyses. After inpainting, only about 75% (leather) and 47% (tile) of the gain remains, so visible-defect signal accounts for part of it.
  • The matched controls do not isolate feature identity. In Control 4 (Table 15), the clean-view features F(x)pF(x)_p come from clean training images and are already contained in ZcleanZ_{\text{clean}}. That arm is therefore essentially the baseline (73.36 vs. 73.34 on MVTec; 52.19 vs. 52.23 on VisA), and the "27/27 categories" result largely restates the main comparison. In Control 3 (Table 14), the clean-view target trains the decoder to undo the context change and falls 7.13 points below the baseline, so the reported +12.07 gap combines this harm with the +4.94 gain. A control that perturbs F(x)pF(x)_p by the same magnitude in a random direction, or that alters context far from the retained patch, is missing.

Why it matters: the paper's main contribution is conceptual: that clean pools miss normal evidence and that altered-context features supply it. At the current scope, the evidence supports a narrower claim: adding synthetic-context neighbor features improves the P-AP of two DINOv2-based detectors on three industrial benchmarks. That is closer to a targeted augmentation result than to a general principle.

Questions for authors

Clarifications (answers could change my assessment of the current results)

  1. Patch labels in the oracle. How is a 14×14 token labeled normal: is any token that contains a defect pixel treated as defective, or is a majority/threshold rule used? If tokens containing a few defect pixels can count as normal, the near-defect "normal" candidates would include defect pixels, which bears directly on the 94.70% near-band result.
  2. Reconstruction protocol. The reconstruction branch is trained per category for 5,000 iterations, whereas Dinomaly was proposed for the multi-class unified setting. Why was this protocol chosen, and how do the baseline numbers compare with published Dinomaly results? Does the gain hold in the multi-class setting?
  3. Image-level and defect-score decreases. Why do I-AUROC and mean defect scores decrease? For example, does the reconstruction branch learn to reproduce content near altered context in a way that also partially reconstructs real defects? An analysis of the images whose image-level ranking degrades would help.
  4. Register vs. non-register encoders. Why does the memory branch use DINOv2 without registers while the reconstruction branch uses DINOv2 with registers? Darcet et al., which the paper cites for global information in patch tokens, show that registers change where this information is stored. Does the oracle gain differ between the two encoders?

Suggestions (additional experiments that would strengthen the paper)

  1. Baseline comparison. Please report absolute results under a common protocol against memory-bank (PatchCore), reconstruction/distillation (RD4AD, multi-class Dinomaly), and synthetic-anomaly methods (DRAEM, DeSTSeg, GLASS, RealNet), and show BoundarySupport as a plug-in on at least one of them.
  2. Backbone. Please repeat the fixed-budget diagnostic and BoundarySupport with a CNN backbone (e.g., WideResNet-50) and another ViT family. Either outcome is informative: if the oracle gain disappears with a CNN, this supports the context-dependence explanation; if it persists, the explanation needs revision.
  3. Synthesis generators. How does BoundarySupport perform with DRAEM-style Perlin-noise blending, MemSeg, or RealNet's SDAS as the intervention gg? Please report the resulting ring size ∣R∣|R| per generator, since generators that change pixels over a wide area may leave few preserved tokens.
  4. Attribution control. Can you add a control that perturbs F(x)pF(x)_p by the same magnitude in a random direction, and one that alters context far from the retained patch? This would separate "a context-specific feature" from "any feature-space perturbation."
  5. Representation analysis. At the same position pp, how do F(x)pF(x)_p and F(x~)pF(\tilde{x})_p relate to the clean bank and to the real near-defect normal features found by the oracle (e.g., nearest-neighbor distances, or a PCA/UMAP projection)? If the selected altered-context features land near the oracle's near-defect features, this would directly link the method to the diagnostic.
  6. Medical domain. Does the effect hold on medical anomaly detection benchmarks (e.g., BMAD), where normal tissue appearance depends strongly on the surrounding anatomy?
  7. Diagnostic beyond leather and tile. Can the distance sweep, inpainting, and cross-defect analyses be reported for all 15 MVTec categories, and for VisA?
Reviewer_f6dHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether normal training images can sufficiently cover the features of normal regions near defects. The core perspective is that a local pixel remaining normal does not mean its representation, after contextual interaction, is still adequately covered by clean training features. The authors diagnose this issue through an oracle experiment with a fixed number of references, and propose BoundarySupport: synthesizing contextual variations in normal training images, excluding locations where pixel modifications occur, and using the preserved neighborhood features as normal references or reconstruction targets. Experiments validate the pixel localization benefits on the memory and reconstruction branches across three datasets.

Critical strengths

s1: The problem studied in this paper has practical significance. Industrial anomaly detection relies on normal training samples, and insufficient feature coverage of normal regions can directly impact localization performance. This paper shifts the focus from how to utilize existing references to what is missing in the normal references themselves; the research storyline is clear, and its contributions can be summarized with concise viewpoints.

s2: The fixed-budget diagnosis and feature identity control are highly targeted. The Oracle experiment fixes the encoder, the number of references, and the scoring method, which can rule out the explanation that the benefits merely come from storing more references. The memory and reconstruction controls further fix candidate locations or synthetic inputs, only replacing reference or target features, which helps identify the effective components. The results across three datasets and three paired seeds also support the reproducibility of the localization benefits.

Critical weaknesses

w1: The experiments mainly compare the results of ProCon and Dinomaly before and after adding BoundarySupport, lacking direct comparisons with other representative methods, nor validating the effects on other detectors. Therefore, the relative advantages and applicable scope of the proposed method remain unclear. The authors should supplement method-level comparisons under comparable settings.

w2: The paper does not compare with the directly related work, RAD [1]. RAD improves the retrieval and matching of normal references, while BoundarySupport supplements normal reference features; the two have a clear connection. The authors should clarify their differences and complementarity, and verify whether the main findings and benefits of this paper still hold under retrieval methods like RAD.

w3: The reconstruction training cost is approximately 2.5 times that of the baseline, but there is a lack of comparison under the same computational budget. Furthermore, the P-AP on Real-IAD improves by 4.20% yet the I-AUROC drops by 0.52%. The authors should clarify the benefits brought by the additional computation, as well as the trade-off between localization improvement and image-level detection.

w4: There are also some formatting issues in the paper. For example, the right side of Table 2 exceeds the text margin and needs to be adjusted according to the template requirements.

[1] Xingwu Zhang, Guanxuan Li, Paul Henderson, Gerardo Aragon-Camarasa, Zijun Long. “Is Task-Specific Training Necessary for Anomaly Detection?” ICML 2026.

Questions for authors

See the weaknesses.

Reviewer_BwrHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that anomaly detection models trained on normal images may not learn how normal patches appear in the context of defects. It proposes BoundarySupport, which creates synthetic defects in normal images and uses normal patches near defects as additional training features for anomaly detection.

Critical strengths

The paper investigates a relevant problem in anomaly detection and reports improvements on several datasets. The idea of using normal patches near defects is interesting, but the experiments do not clearly establish why this approach helps.

Critical weaknesses

My main concern is the premise of the paper which rests on the idea providing more normal patches near defects provides more useful data to an anomaly detection model. The authors report an increase in pixel AP from 73.34 to 76.95 when adding normal patches near defects. However, this does not show that the gain comes specifically from learning how normal patches appear near defects. It can be attributed to generally higher feature diversity. A control experiment with equally diverse changes that do not simulate defects would help isolate the proposed explanation.

The novelty of the paper seems limited, as adding synthetic anomalies has been shown to be an effective approach in several previous works such as DRAEM (Zavrtanik et al., ICCV 2021), CutPaste (Li et al., CVPR 2021), and GLASS (Chen et al., ECCV 2024). The paper uses synthetic defects to obtain nearby normal features, but it is unclear what is new beyond this choice. A comparison with existing synthetic-anomaly methods, or a matched baseline that uses synthetic defects without the proposed patch selection, would help establish the contribution.

Questions for authors

Can the authors clarify why it is better to learn from the normal patches surrounding a synthetic defect than to use synthetic defects themselves for anomaly detection supervision, as existing methods do? Can the authors compare BoundarySupport against existing synthetic-anomaly methods, or a matched baseline that uses synthetic defects without selecting nearby normal patches? The experiment supporting the main claim uses only leather and tile classes from one dataset. Can the authors evaluate this claim across more classes and datasets? Can the authors include a control experiment to determine whether the gains come specifically from normal patches near defects, rather than from greater feature diversity or other changes in the surrounding features?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.