- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a unified framework for multimodal anomaly detection and deterministic uncertainty estimation. The framework consists of a Multi-Modal Feature Extraction Backbone (MM-FEB), intended to map heterogeneous inputs such as RGB, depth, infrared, and medical images into a shared feature space, and DUQNet, which predicts region quality and three performance gaps. These gaps are constructed from three paired comparisons: single versus dense prompts, tiny versus large models, and unsupervised versus supervised mask selection. The paper interprets them as proposal, model, and task uncertainty, respectively. Experiments are reported on nine datasets from industrial, synthetic, and medical domains.
Critical strengths
- Interesting construction of actionable quality signals. The comparisons between single and dense prompts, tiny and large models, and unsupervised and supervised selection can be viewed as estimates of three counterfactual performance gains. These signals could be useful for requesting additional user input, routing difficult samples to a more expensive model, or triggering manual review.
- Predicting these gains with lightweight MLPs has the potential to reduce inference cost compared with running multiple model variants or performing repeated stochastic inference.
- The empirical evaluation covers a broad range of industrial, synthetic, and medical datasets and includes both image-level and pixel-level metrics.
- The reported anomaly-detection results are generally competitive, with particularly promising pixel-level results on BraTS.
Critical weaknesses
- The key architectural components are insufficiently specified. Figure 1 contains a Prior Generator, Modality Prior Modulation (MPM), coordinate prompts, an image encoder, a prompt encoder, a mask decoder, and SamScore, but the manuscript does not define several of these components in enough detail to understand or reproduce the method. In particular, it is unclear how the modality priors are generated, whether explicit modality labels are required, how multiple priors modulate the shared representation, how missing modalities are encoded, and how modality order is handled after channel padding. The manuscript also does not clearly describe how the fused representation is converted into the final anomaly score and mask. Equations (1)–(3) mainly describe standard parallel convolutions and a bottleneck. They do not by themselves establish that anomalous information is suppressed while normal information is preserved. The paper should provide a complete end-to-end algorithm, tensor dimensions, all training objectives, the exact implementation of the Prior Generator and MPM, and a targeted ablation against simpler concatenation or averaging baselines.
- Claim 2 is inconsistent with its own equations. Claim 2 states that the three gaps, , , and , are mutually non-negative and sum to the total predictive gap. However, Eq. (17) only establishes $ \Delta XP(l) + \Delta A(l) = MLP_{XP}(l) - MLP_{SAM}(l), $ and does not contain . The text and the equation therefore define different decompositions. The authors should clarify which version is used by the method and experiments, and provide the correct definition if is intended to be part of the total gap. In addition, independent training on different targets does not imply statistical independence or orthogonality, and independently regressed IoUs do not guarantee non-negative differences for every sample. The paper should report the frequency of negative gaps and the empirical correlations among the three components.
- The three gaps are interesting, but their interpretation as standard uncertainty types is not yet established. The proposed quantities have a clear operational interpretation as prompt-refinement gain, large-model upgrade gain, and mask-selection improvement gain. This is potentially useful. However, these quantities are not automatically equivalent to aleatoric, epistemic, and task uncertainty in the standard probabilistic sense. For example, the performance difference between a large and a tiny model may reflect architecture, optimization, capacity, or training randomness rather than epistemic uncertainty alone. Similarly, improvement from a denser prompt does not necessarily measure irreducible data uncertainty. The paper would be more convincing if it described these quantities as actionable performance or error proxies, or provided stronger theoretical and empirical evidence connecting them to the claimed uncertainty categories. Direct evaluation is also needed: prediction error for each gap, correlation with the realized gain, negative-gap frequency, and the ability of each signal to support model routing or prompt acquisition.
- The central “arbitrary modality combinations with one parameter set” claim is not sufficiently tested. The reported experiments use fixed modality combinations within each dataset. There are no controlled evaluations with randomly missing modalities, held-out modality combinations, unseen modalities, or permuted modality order. It is also unclear whether all reported results are obtained from one checkpoint shared across all nine datasets or from dataset-specific checkpoints and hyperparameter searches. Since modality agnosticism is a central claim, the paper should report the exact number of trained checkpoints and provide a modality-combination matrix comparing single-modality, full-modality, missing-modality, and unseen-combination settings.
- The theoretical appendix and several reported numbers require revision. Beyond Claim 2, parts of the theoretical appendix appear disconnected from the implemented method. Claim 1 differentiates empirical coverage defined with a hard indicator; this derivative is zero almost everywhere and undefined at its boundaries, so the stated gradient argument does not follow. Prediction intervals and the corresponding coverage loss are also not introduced in the main method. Claim 4 discusses joint optimization using an undefined LFEB, whereas the limitations section states that the MLP estimators are trained post hoc. These statements should be reconciled or removed. There are also directly checkable numerical inconsistencies. On VisA, MF1P increases from 44.4 to 47.2, which is 2.8 percentage points or approximately 6.3% relative, rather than the stated 10.8%. From Table 4, the mean AUCI values for Baseline and +MM-FEB are approximately 71.079 and 73.380, rather than 72.379 and 75.086. The corresponding mean MF1P values are approximately 23.928 and 24.438, rather than 27.302 and 28.257. The paper would benefit from a complete audit of its theoretical statements, reported averages, dataset descriptions, and references.
Questions for authors
- What are the exact architectures, inputs, outputs, and training objectives of the Prior Generator and MPM?
- Should Claim 2 follow its textual statement or Eq. (17)? Is ΔΘ part of the total predictive gap?
- How are dense prompts generated, and are the tiny/large models and alternative prompt conditions actually run for every training sample?
- What are the MAE and correlation between each predicted gap and its realized performance gain? How often is each gap negative?
- Are all results produced by a single checkpoint shared across the nine datasets? Is SMAC3 run once globally or separately for individual datasets?
- Why does adding DUQNet, which is described as an uncertainty estimator and later as post hoc, substantially change anomaly-detection performance in Table 4?