acceptodds
Under review as a conference paper at ICLR 2027

SPLIT: Structure-Preserving Budget Allocation for Visual Anomaly Detection

Abstract

Industrial anomaly detectors increasingly exploit multi-layer features from frozen backbones, but conventional layer-wise heads entangle detector budget with feature structure: a head's width is fixed by the backbone's feature dimension, so trainable budget can only be changed by changing which layers are consumed. We introduce SPLIT ( Structure-Preserving budget aLlocation for vIsual anomaly deTection), a latent reparameterization of multi-layer anomaly heads whose latent dimension controls trainable budget at a fixed feature structure. Separating the two exposes a budget-dependent budget-sharing trade-off: under a matched total budget, a single shared latent code concentrates a scarce budget and wins at the smallest budgets, while per-layer codes remove cross-layer latent coupling at the cost of more budgets—take over once each head receives a workable width. The asymmetry between backbones is consistent with their cross-layer representation structure, and feature composition remains a third, independent factor. Across five industrial benchmarks and two frozen backbones, SPLIT is competitive with prior work, using only 0.002M latent parameters on DINOv3 and 0.008M on CLIP.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Structure-Preserving, Latent Reparameterization, Budget Allocation, Industrial Anomaly Detection, Mapping Networks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ovQnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces parameter-efficient linear probing heads for foundation model-based AD. Current AD methods based on foundation models (e.g., DINO, CLIP) mostly train lightweight projection layers on pre-trained features to estimate anomaly segmentations from limited labeled samples. While adopting projection layers provides an efficient and effective approach to few-shot AD, this design still faces challenges when we need to reduce the total number of learnable parameters because the weight dimension is tightly tied to the number of feature dimensions. To overcome this challenge, this paper decomposes the original weights of the linear head W as W=g(W*, z), where W* is orthogonally initialized and fixed during training, and g denotes some non-linear function. Experimental results on widely used AD benchmarks demonstrate that it can reduce the total number of parameters by up to 10 times while minimizing the effect on accuracy.

Critical strengths

S1: The core design philosophy is simple and easy to understand.

S2: Experiments cover broader AD benchmarks, including RealIAD, VisA, BTAD, and MVTecAD. Additional ablation results and technical details are well presented in the main paper and the supplementary.

Critical weaknesses

W1: Contributions. As pointed out in L102, UniADet have already achieved highly-efficient AD with pretrained backbones (~0.02M trainable parameters). While the proposed method achieves a 10x reduction in the total number of trainable parameters, it is still unclear whether such parameter efficiency can yield practical benefits for AD.

W2: Novelty. As stated in L119-121, the authors acknowledge that the weight generation mechanism itself is not a main contribution in this paper, and they adopt a well-established scheme for generating head weights. This highlights the paper's lack of novelty, and the resulting mechanism seems highly engineered. Specifically, they need to tune the training budget for different backbones (e.g., d=512 for DINO, d=2048 for CLIP as in L289). Also, as shown in Fig.3, the best-performing budget differs across backbones.

W3: Accuracy. As shown in Tab.1, the proposed method lagged behind previous competitors like UniADet in most metrics. Also, in their comparison between "shared" vs. "per-layer" code generation strategies, the accuracy with a limited budget is much lower than that of the compared methods in Tab.1.

Questions for authors

Q1: What practical benefit does the proposed method bring? As in W1, explain the motivation and why we need to pursue training memory efficiency beyond lightweight linear probing. During inference, this method likely requires the same total number of floating-point operations. Also, when we use more capable and bigger foundation models (e.g., DINOv3-7B), the inference and training cost is mostly dominated by the backbone forward pass rather than the projection layer.

Q2: Can we test other backbone models? As in W2, we see that the best-performing d is different across backbones. It is better to add other backbone models used in AD, including MAE [1], SigLIP [2]. These additional experiments might help us understand sensitivity to backbone choices.

Q3: Parameter reduction. Recall that the training budget C can be determined with C_b=2LD for baselines and C_p=Ld for the proposed method (L67-74), D=1024 (L34), and d=2048 for DINOv2. Does this therefore imply C_b=C_p (i.e., no improvement in terms of memory efficiency) with per-layer coding?

Q4: Presentation details. What is the definition of sigma (L154), and also for g (L161, L195)? It is better to provide a clear definition of them when they first appear in the manuscript.

References:

[1] He+, "Masked Autoencoders Are Scalable Vision Learners", CVPR'22, https://ieeexplore.ieee.org/document/9879206

[2] Zhai+, "Sigmoid Loss for Language Image Pre-Training", https://ieeexplore.ieee.org/document/10377550

2 comments

W1 / Q1 — practical benefit.

We reframe the benefit as decision guidance and diagnosis, not compute: (i) at a fixed budget, structure and basis choices change accuracy by 5–25 points (Sec. 4.4; Table 12, Appendix A.8); (ii) the failure of a shared code is diagnosable and fixable (independent codes restore per-layer performance: SigLIP shared 31.9 → IW 41.3; a tied basis explains the DINOv3 exception); (iii) we document boundaries (source-selection reversal on DINOv3, augmentation-shortcut at long training, unstable deep-heavy allocation) so practitioners do not mis-attribute failures to capacity. We quantify the cost of wrong choices and make no SOTA claim.

W2 — novelty and backbone sensitivity / Q2 — other backbones.

We added SigLIP and MAE (four backbones). The preferred per-head width is backbone-dependent (CLIP keeps improving to 2048, DINOv3 peaks at 512, and SigLIP/MAE gain little beyond 512), and the structure ranking changes with the target (DINOv3). This variability is now the object of study, not a hidden tuning knob: the paper gives a budget-aware rule (d* vs C/L) and a diagnostic, and states its failure modes.

W3 — accuracy behind UniADet.

We concede the main table is competitive rather than superior and no longer claim otherwise. The table is a feasibility check under one harness; the new analyses (Secs. 4.3–4.5; Table 12, Appendix A.8) are within-structure comparisons at matched budgets. We also report a score-implementation sensitivity analysis (Table 9, Appendix A.5): the original SPLIT numbers are conservative because float32 softmax saturates, and raw logit scores change some image-level cells (e.g., Real-IAD CLIP 69.6 → 76.9). We do not use this to re-rank against baselines, whose raw score maps are unavailable; we flag it as a limitation of the original evaluation.

Q3 — parameter reduction with per-layer coding.

Correct: C_p = Ld. With d=2048 on CLIP, C_p = 8192 = C_b = 2LD, so there is no reduction; with DINOv3 d=512, C_p=2048 vs 8192, a factor of four. The revised paper reports both and makes no universal efficiency claim.

Q4 — definitions of σ and g.

Added at first use (Sec. 3.1): σ is the coordinate-wise activation (identity for SPLIT/SPLIT-M, tanh for SPLIT-D); g(W,z)=σ(W̃ z) is the generator map.

Thank you for your rebuttal with additional details and results. Yet, I think their response merely addressed my concerns.

For W1/Q1: In my opinion, the decision guidelines in this paper should be tied to computational efficiency. As this paper focuses on industrial AD, it should be encouraged to explain what this method brings in terms of actual benefit for practical use.

For W2/Q2: Thank you for providing an additional ablation result. Nevertheless, I still have a concern about the backbone-specific selection of the C/L, and there is a gap in the optimal d*.

The authors mostly addressed the other technical comments (W3/Q3/Q4) in their response. Please highlight the revised parts in the manuscript so reviewers can easily see where the authors made changes.

Overall, I think this paper needs to address the points above before publication. Therefore, I'm inclined to maintain my score.

Reviewer_jeuqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SPLIT, a latent reparameterization of multi-layer linear segmentation heads for anomaly detection on frozen backbones (CLIP, DINOv3). Both the motivation and the results are weak: the trainable parameters of a standard linear head are already negligible compared with the frozen backbone, so reducing them brings little practical benefit, and the method often underperforms the compared approaches.

Critical strengths

Very limited.

Critical weaknesses
  1. Weak motivation. The method aims to reduce trainable parameters, but a directly trained multi-layer linear head already has only about 8K parameters (L×2×D), which is negligible. The frozen backbone dominates both FLOPs and parameter count, so reducing trainable parameters from roughly 8K to 2K (or a few hundred at the smallest budgets) has little practical significance.

  2. The main configuration is essentially a plain linear head. As the authors acknowledge, the additive generator used in all main experiments is functionally equivalent to a plain projection (just fewer parameters in some settings).

  3. No demonstrated computational benefit. It is unclear how much training or inference computation and speed the reduced trainable parameters actually save. The paper's own measurements show that memory and latency are dominated by the backbone, and training cost is not reported.

  4. Even if there is no computational benefit, SPLIT often underperforms the compared methods.

Questions for authors

See Weaknesses.

1 comment

W1/W4 — weak motivation; underperforms; efficiency is negligible.

We agree and no longer claim efficiency. The frozen backbone dominates parameters, FLOPs and latency; at C=2048 all heads have the same trainable count, and after materializing θ=Wz inference is identical (R2-Q1; Tables 19–20, Appendix A.15). On CLIP, d=2048 gives C=8192, equal to a direct head's 2LD, so there is no reduction there; only DINOv3 at d=512 gives 4x. The contribution is therefore not “fewer parameters” but a structural study: choosing the wrong structure or basis at a fixed budget costs 5–25 pixel-AUPR points with identical trainable counts (SigLIP C=256: 15.3 per-random vs 40.1 per-PCA; C=512: 30.6 vs 42.5), and the paper explains why shared codes fail (basis/coordinate mismatch, not capacity) and how to diagnose it.

W2 — the main configuration is a plain linear head.

Correct: the additive term is a null direction, and we say so (Sec. 3.3). SPLIT is used as an experimental tool to vary capacity at fixed layers; we compare against the plain/direct head (mean fusion) and report that at C=2048 the direct head is stronger on three of four backbones. The scientific content is the matched-budget structure/basis comparison and the mechanism, not the generator.

W3 — no computational benefit.

We agree on runtime cost: inference latency and peak GPU memory are identical, and matched trainable counts are equal. There is a narrow checkpoint/startup advantage: the per-layer frozen basis is 16× smaller (4 MB vs 64 MB at C=2048) and correspondingly cheaper to regenerate from its seed. It is not an inference-throughput or peak-memory advantage—materializing the heads makes both artifacts 32 KB—so we describe it as a checkpoint/startup difference, not a deployment-throughput claim. We also make no training-cost savings claim.

Reviewer_3bC9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SPLIT, a latent reparameterization framework for visual anomaly detection. It aims to decouple the trainable parameter budget from the selection of backbone feature layers. The core idea is to generate layer-wise anomaly heads from low-dimensional trainable latent codes via frozen projections. This allows the number of trainable parameters to be controlled by the latent dimension without removing feature layers. This work further investigates how to allocate a fixed total budget across layers. By comparing per-layer latent codes with shared latent codes, the study identifies a budget-dependent trade-off: shared codes perform better under an extremely tight budget, whereas per-layer codes yield superior performance once each layer is allocated sufficient capacity. The authors also examine performance differences between CLIP and DINOv3, attributing such discrepancies to cross-layer feature similarity measured by CKA. The experimental results demonstrate that SPLIT achieves performance comparable to existing methods while utilizing only a tiny number of trainable parameters.

Critical strengths
  1. Decoupling the trainable budget from the selected feature layers. By generating the detection heads from low-dimensional latent codes, SPLIT can vary the number of trainable parameters while keeping the selected backbone feature layers unchanged. This provides a clean way to study the effect of parameter budget independently from feature-layer selection, addressing an important confounding factor in conventional layer-wise anomaly detection heads.
  2. The matched-budget analysis provides a valuable empirical insight rather than simply claiming one allocation strategy is universally optimal. The comparison between shared and per-layer latent codes is rigorously parameter-matched. The shared strategy performs best under the minimal tested budget, while per-layer allocation is preferable once C exceeds approximately 512. In contrast to evaluations with equal latent dimensions (which lead to different total trainable parameters between models), this analysis offers a more nuanced view of budget allocation.
Critical weaknesses
  1. The CKA analysis does not sufficiently support the proposed mechanistic explanation. The paper relates cross-layer representation similarity to the preference for shared versus per-layer latent codes, but the evidence remains primarily correlational. CKA measures representation similarity rather than task-specific functional equivalence or the substitutability of different layers for anomaly detection and pixel-level localization. Moreover, CLIP and DINOv3 differ in pretraining objectives, representations, and other architectural/training factors, so the current experiments cannot isolate cross-layer similarity as the factor responsible for the observed allocation behavior. Thus, the CKA analysis is useful as an explanatory observation, but does not by itself establish the proposed mechanism.
  2. Insufficient comparison with alternative budget-control baselines and justification of performance gains. While the paper demonstrates that SPLIT can achieve competitive performance under an extremely low trainable parameter budget, it lacks direct, matched-budget comparisons against simpler parameter-reduction alternatives, such as fixed feature sub-space selection or standard low-rank matrix factorization (e.g., standard PCA-based or bottleneck projections) applied to detection heads. Without such comparisons, it remains ambiguous whether SPLIT's performance stems from its specific latent reparameterization design or simply from structural regularization. Consequently, the current evaluation proves the feasibility of the parameterization scheme rather than establishing its superiority or practical necessity over existing, simpler alternatives.
Questions for authors

Can the authors provide evidence that the relationship between CKA and budget allocation generalizes beyond CLIP and DINOv3? For example, does cross-layer CKA consistently correlate with the relative benefit of shared versus per-layer encoding across additional backbones? 2. How does SPLIT compare with alternative low-rank, simpler alternative approaches or fixed-subspace detection heads under matched trainable parameter budgets?

1 comment

W1 — CKA is correlational and does not establish the mechanism.

We agree, and we removed CKA from the mechanism. The revised paper (i) reports CKA only as a descriptive statistic (Sec. 4.5; Appendix A.11; Figure 4 in Appendix A.17), explicitly stating that representation similarity is not functional use; (ii) replaces it with two functional tools: a function-preserving reparameterization audit (Table 13, Appendix A.9) and layer-wise ablations (Table 4); and (iii) gives a mechanism at the parameterization level: the shared code's per-head functional rank equals mean fusion's (both 1024; joint ranks 2048 vs 1024), so the gap is not capacity; gradients arriving through independently initialized bases are nearly orthogonal and their summed update has low norm, whereas a tied basis aligns them (Sec. 4.5). The 2x2 basis-tying/code-tying ablation isolates the interaction (Table 3). CKA does not predict which layers are used: the ablations order layers differently, and on DINOv3 the per-layer advantage is not significant across categories (bootstrap 95% CI [-0.7,+5.3], p=0.128).

W2 / Q2 — missing matched-budget comparisons with simpler alternatives.

We added exactly these baselines at the same trainable count and rank (Secs. 4.3–4.4; Tables 1, 2 and 12, with Table 12 in Appendix A.8): frozen random projection (SPLIT), PCA projection, fixed feature-subspace (variance-selected channels), mean fusion (a tied basis on the averaged feature: a direct head at C=2048, a rank-C/2 head at smaller budgets; also the frozen-bottleneck case), concatenation, and a single full-width layer-23 head. Key results:

  • At C=256, the random-basis per-layer head is the weakest structure (CLIP 15.0 vs mean 29.5; SigLIP 15.3 vs 24.1); a PCA basis at the same budget raises it to 37.2 (CLIP) / 40.1 (SigLIP), and it matches or beats mean on all four backbones (Table 12, Appendix A.8).
  • At C=2048 the basis effect is smaller and backbone-dependent (PCA +2.8 SigLIP / +3.3 CLIP, -3.7 DINOv3; Table 2), and concatenation is nearly basis-insensitive on three backbones.
  • At the same trainable count, a trainable bottleneck would need k≈2 at C=2048, which is not a meaningful head; we therefore compare the frozen version at k=C/2, which is exactly mean fusion. A full-width layer-23 head reaches 43.6/39.4/37.7/45.6, so mean fusion improves over it by only +2.3/+2.3/-0.5/+3.8. We report this to avoid overstating the multi-layer premise. This shows the original random-basis comparison confounded structure with basis; we no longer claim SPLIT's reparameterization is superior.

Q1 — does the CKA–allocation relation generalize?

It does not, and we now report that as a negative result. With four backbones (CLIP, DINOv3, SigLIP, MAE) and two targets (MVTec, BTAD), the structure ranking is backbone- and target-dependent: DINOv3 reverses on BTAD (per 44.4 vs mean 48.5), and source-validation selection picks the wrong structure on DINOv3. Hence the paper offers a diagnostic and a budget-aware decision procedure, not a universal CKA-based rule.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.