acceptodds
Under review as a conference paper at ICLR 2027

Modality-Agnostic Anomaly Detection with Deterministic, Decomposable Uncertainty Quantification

Abstract

Anomaly detection (AD) increasingly relies on multi-modal data, yet existing methods use fixed-modality architectures, suffer domain interference from shared reconstruction paths, and output only deterministic predictions without reliable uncertainty. We propose a unified framework combining a Multi-Modal Feature Extraction Backbone (MM-FEB) and a Deterministic Uncertainty Quantification Network (DUQNet). MM-FEB unifies arbitrary modality combinations (RGB, depth, infrared) into a shared feature space and applies a hierarchical multi-scale compression structure that suppresses anomaly-related noise while preserving normal patterns, mitigating cross-modal interference. DUQNet extracts region-level features and trains lightweight MLPs to directly regress predictive uncertainty, decomposing it into three interpretable sources: epistemic model uncertainty, aleatoric proposal uncertainty, and task uncertainty, without requiring calibration data or sampling. Experiments on nine public datasets spanning industrial, synthetic, and medical domains show that our method achieves 99.089% on MVTec-3D, 96.913% on Eyecandies, and 97.468% with a state-of-the-art 51.404% on the challenging BraTs benchmark, where the low reflects the small and ill-defined tumor boundaries rather than weak localization. Our method consistently outperforms state-of-the-art detectors, while reducing the pixel-level Expected Calibration Error (ECE) to 0.049 on BraTs and 0.054 on MVTec-3D, corresponding to an average 37.3% relative reduction compared with the best competing method. These results support practical applications such as adaptive model selection, input quality assessment, and task supervision at low computational cost. The code is available at https://anonymous.4open.science/status/iclr2027-B601.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Anomaly Detection, Multi-modal Learning, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_P6FFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a unified framework for multimodal anomaly detection and deterministic uncertainty estimation. The framework consists of a Multi-Modal Feature Extraction Backbone (MM-FEB), intended to map heterogeneous inputs such as RGB, depth, infrared, and medical images into a shared feature space, and DUQNet, which predicts region quality and three performance gaps. These gaps are constructed from three paired comparisons: single versus dense prompts, tiny versus large models, and unsupervised versus supervised mask selection. The paper interprets them as proposal, model, and task uncertainty, respectively. Experiments are reported on nine datasets from industrial, synthetic, and medical domains.

Critical strengths
  • Interesting construction of actionable quality signals. The comparisons between single and dense prompts, tiny and large models, and unsupervised and supervised selection can be viewed as estimates of three counterfactual performance gains. These signals could be useful for requesting additional user input, routing difficult samples to a more expensive model, or triggering manual review.
  • Predicting these gains with lightweight MLPs has the potential to reduce inference cost compared with running multiple model variants or performing repeated stochastic inference.
  • The empirical evaluation covers a broad range of industrial, synthetic, and medical datasets and includes both image-level and pixel-level metrics.
  • The reported anomaly-detection results are generally competitive, with particularly promising pixel-level results on BraTS.
Critical weaknesses
  • The key architectural components are insufficiently specified. Figure 1 contains a Prior Generator, Modality Prior Modulation (MPM), coordinate prompts, an image encoder, a prompt encoder, a mask decoder, and SamScore, but the manuscript does not define several of these components in enough detail to understand or reproduce the method. In particular, it is unclear how the modality priors are generated, whether explicit modality labels are required, how multiple priors modulate the shared representation, how missing modalities are encoded, and how modality order is handled after channel padding. The manuscript also does not clearly describe how the fused representation is converted into the final anomaly score and mask. Equations (1)–(3) mainly describe standard parallel convolutions and a bottleneck. They do not by themselves establish that anomalous information is suppressed while normal information is preserved. The paper should provide a complete end-to-end algorithm, tensor dimensions, all training objectives, the exact implementation of the Prior Generator and MPM, and a targeted ablation against simpler concatenation or averaging baselines.
  • Claim 2 is inconsistent with its own equations. Claim 2 states that the three gaps, ΔXP(l)\Delta XP(l), ΔA(l)\Delta A(l), and ΔΘ(l)\Delta \Theta(l), are mutually non-negative and sum to the total predictive gap. However, Eq. (17) only establishes $ \Delta XP(l) + \Delta A(l) = MLP_{XP}(l) - MLP_{SAM}(l), $ and does not contain ΔΘ(l)\Delta \Theta(l). The text and the equation therefore define different decompositions. The authors should clarify which version is used by the method and experiments, and provide the correct definition if ΔΘ\Delta \Theta is intended to be part of the total gap. In addition, independent training on different targets does not imply statistical independence or orthogonality, and independently regressed IoUs do not guarantee non-negative differences for every sample. The paper should report the frequency of negative gaps and the empirical correlations among the three components.
  • The three gaps are interesting, but their interpretation as standard uncertainty types is not yet established. The proposed quantities have a clear operational interpretation as prompt-refinement gain, large-model upgrade gain, and mask-selection improvement gain. This is potentially useful. However, these quantities are not automatically equivalent to aleatoric, epistemic, and task uncertainty in the standard probabilistic sense. For example, the performance difference between a large and a tiny model may reflect architecture, optimization, capacity, or training randomness rather than epistemic uncertainty alone. Similarly, improvement from a denser prompt does not necessarily measure irreducible data uncertainty. The paper would be more convincing if it described these quantities as actionable performance or error proxies, or provided stronger theoretical and empirical evidence connecting them to the claimed uncertainty categories. Direct evaluation is also needed: prediction error for each gap, correlation with the realized gain, negative-gap frequency, and the ability of each signal to support model routing or prompt acquisition.
  • The central “arbitrary modality combinations with one parameter set” claim is not sufficiently tested. The reported experiments use fixed modality combinations within each dataset. There are no controlled evaluations with randomly missing modalities, held-out modality combinations, unseen modalities, or permuted modality order. It is also unclear whether all reported results are obtained from one checkpoint shared across all nine datasets or from dataset-specific checkpoints and hyperparameter searches. Since modality agnosticism is a central claim, the paper should report the exact number of trained checkpoints and provide a modality-combination matrix comparing single-modality, full-modality, missing-modality, and unseen-combination settings.
  • The theoretical appendix and several reported numbers require revision. Beyond Claim 2, parts of the theoretical appendix appear disconnected from the implemented method. Claim 1 differentiates empirical coverage defined with a hard indicator; this derivative is zero almost everywhere and undefined at its boundaries, so the stated gradient argument does not follow. Prediction intervals and the corresponding coverage loss are also not introduced in the main method. Claim 4 discusses joint optimization using an undefined LFEB, whereas the limitations section states that the MLP estimators are trained post hoc. These statements should be reconciled or removed. There are also directly checkable numerical inconsistencies. On VisA, MF1P increases from 44.4 to 47.2, which is 2.8 percentage points or approximately 6.3% relative, rather than the stated 10.8%. From Table 4, the mean AUCI values for Baseline and +MM-FEB are approximately 71.079 and 73.380, rather than 72.379 and 75.086. The corresponding mean MF1P values are approximately 23.928 and 24.438, rather than 27.302 and 28.257. The paper would benefit from a complete audit of its theoretical statements, reported averages, dataset descriptions, and references.
Questions for authors
  • What are the exact architectures, inputs, outputs, and training objectives of the Prior Generator and MPM?
  • Should Claim 2 follow its textual statement or Eq. (17)? Is ΔΘ part of the total predictive gap?
  • How are dense prompts generated, and are the tiny/large models and alternative prompt conditions actually run for every training sample?
  • What are the MAE and correlation between each predicted gap and its realized performance gain? How often is each gap negative?
  • Are all results produced by a single checkpoint shared across the nine datasets? Is SMAC3 run once globally or separately for individual datasets?
  • Why does adding DUQNet, which is described as an uncertainty estimator and later as post hoc, substantially change anomaly-detection performance in Table 4?
Reviewer_HutrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents works that are already published as proposed by the authors.

Critical strengths

None

Critical weaknesses

The paper does not introduce anything new. The most problematic issue is that it presented previously published work as novel. The proposed DUQNet is already published under the name UncertainSAM by Kaiser et al. in ICML 2026. Even the lower part of Figure 1. of the manuscript the presents the "proposed" DUQNet is almost identical with Figure 2 of the UncertainSAM paper without any reference. This is unacceptable.

Questions for authors

No questions since the article does not contain novel material.

Reviewer_RTWmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies anomaly detection across heterogeneous modalities with a shared model and interpretable uncertainty estimates. Sections 3.2 and 3.3 combine MM-FEB, which uses channel padding and hierarchical multi-scale compression, with DUQNet, which predicts IoU differences associated with model size, prompt refinement and mask selection. Tables 1 to 5 and Appendix C report detection and calibration comparisons across nine datasets, component ablations, and theoretical claims concerning coverage, decomposition, compression and optimization.

Critical strengths
  1. The channel-padded input interface and shared multi-scale feature path in Section 3.2 and Eqs. 1 to 3 provide a concrete architecture for processing different supported modality combinations. This design supports parameter sharing across heterogeneous sensor configurations and gives the proposed integration practical application value.

  2. The IoU-gap estimators in Section 3.3 and Eqs. 4 to 7 connect uncertainty outputs to identifiable interventions involving model scaling, prompt refinement and mask selection. Their stated inference procedure uses deterministic MLP evaluations, making these intervention targets accessible without Monte Carlo sampling, although this architectural property does not establish the accuracy of the estimates.

Critical weaknesses
  1. The manuscript does not establish a substantive methodological advance beyond combining verified existing mechanisms. UniMMAD: Unified Multi-Modal and Multi-Class Anomaly Detection via MoE-Driven Feature Decompression, CVPR 2026 already contains the channel padding and hierarchical multi-scale compression underlying MM-FEB, while UncertainSAM already uses 512-dimensional IoU regressors and the same model, prompt and selection gaps described in this paper’s Section 3.3. The remaining candidate contribution is their integration for anomaly detection and the normalization in Eq. 8, but Table 4 does not isolate their value beyond a direct combination.

  2. The empirical and theoretical evidence does not establish the claimed value of the integrated system. The verified result correspondences leave the performance and component effects in Tables 1, 2 and 4 unattributed to a clearly documented implementation, while the definitions in Appendices C.1 and C.2 do not support the coverage and three-source additivity claims. Missing supervision and calibration protocols in Sections 3.3, 4.1 and 4.2 and Appendix B further prevent determining whether the reported gains validate the proposed uncertainty mechanism.

Questions for authors
  1. What experiment records explain the exact result correspondences identified in Tables 1, 2 and 4 and establish which reported values were obtained from the submitted implementation?
  2. What is the precise implemented coupling between MM-FEB and DUQNet, including the detector’s training objective and how the regional IoU estimates affect final anomaly predictions, as depicted in Figure 1 and discussed in Sections 3.2, 3.3, 4.3 and 5? Please identify the mechanism that differs from directly combining UniMMAD and UncertainSAM and the controlled comparison that isolates its contribution, including the role of Eq. 8 and the ablation in Table 4.
  3. What corrected uncertainty guarantees hold for the implemented estimator and training objective in Appendices C.1 and C.2? The coverage indicator in Eq. 10 has zero derivative almost everywhere, and Eq. 17 already exhausts the defined total using the prompt and task terms, leaving the claimed coverage convergence and three-term additivity unjustified.
  4. Which masks and prompts, from which dataset partitions, were used to construct the IoU training targets and generate test-time predictions in the procedure described in Sections 3.3 and 4.1 and Appendix B? This clarification is necessary to assess whether the proposed method and the baselines use comparable supervision.
  5. How can the pixel-level ECE comparisons in Table 2 be reproduced from Eq. 8 and Section 4.2, including the confidence definition, correctness target, regional-to-pixel mapping, binning and evaluation split? Please also identify the evaluation supporting the CFM ECE of 0.0840.084 in Section 4.4, which has no corresponding entry in Tables 1 or 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.