- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses zero-shot multi-label OOD detection with pre-trained vision–language models. It argues that multi-class scores are mismatched to multi-label data and proposes a probabilistic formulation in which an input is OOD if it is associated with none of the ID labels. Under a noise-contrastive estimation framework, the detection score becomes a product of per-label posteriors, for which a Monte Carlo estimator and a consistency bound are given. On COCO and VOC, the method reports average FPR95 of 17.46% and 17.51%, improving over RenyiNeg by 6.82 and 9.03 points.
Critical strengths
- The multi-label setting is genuinely under-explored, and the formulation in terms of image–label associations is well suited to its non-exclusive structure. It provides a clearer statement of the detection target than max- or sum-based aggregation over class scores.
- The NCE derivation is technically clean. The label prior cancels, the posterior reduces to a tractable per-label expression, and the Monte Carlo approximation is stated explicitly, which makes the estimator easy to reproduce.
- The empirical evaluation is broad and internally consistent. Improvements over NegLabel and RenyiNeg are observed across two ID datasets, four OOD datasets, three backbones, and two input resolutions, and the VOC FPR95 reduction from 26.54% to 17.51% is substantial.
Critical weaknesses
- The score in Eq. 7 is not, in general, a probability. The identity Pr(∃y: C_y=1) = 1 − ∏_y q(C_y=0 | x) requires the association events to be conditionally independent given x, which is violated by co-occurring labels and by the shared image evidence on OOD inputs. The assumption is not stated, so the quantity should be presented as a noisy-OR surrogate rather than a calibrated probability.
- The theoretical results do not establish the claimed fundamental mismatch. The bound in Theorem 1 is informative only when ε is small relative to τ log T; with τ=0.004 and typical cosine gaps of 0.01–0.05, the bound exceeds one. Theorem 2 describes generic softmax saturation and in fact shows that NegLabel benefits from secondary labels, which supports a conclusion of diminishing returns rather than structural incompatibility.
- The experiments do not connect the theory to the observed behavior. The natural test—FPR95 and AUROC stratified by the number of ground-truth labels per image—is absent, as is an ablation of the aggregation function. The state-of-the-art claim is also selective: on COCO, JointEnergy (97.26/10.91) and MaxLogit (96.44/14.16) outperform the proposed method (96.39/17.46), so the claim should be restricted to training-free zero-shot methods.
Questions for authors
NA