- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies online test-time out-of-distribution (OOD) detection with frozen vision-language models. The authors focus on a central difficulty in adapting to unlabeled mixed ID/OOD streams: incorrect assignments can contaminate the adaptation state and reinforce subsequent detection errors. To address this issue, the paper proposes OSGB, an optimization-free framework that maintains separate ID and OOD Gaussian banks in frozen CLIP feature space. Positive class names and negative semantic concepts guide candidate selection, while several rules control bank updates. Experiments cover ImageNet-1K and other benchmarks, with strong reported performance.
Critical strengths
- The paper addresses a practical challenge in online OOD detection: learning from an unlabeled stream without allowing unreliable assignments to corrupt future predictions. The proposed method keeps CLIP frozen and updates sufficient statistics to identify reliable samples instead of using gradient-based test-time adaptation.
- OSGB achieves substantial improvements across several evaluated settings. On ImageNet-1K, it outperforms the AdaNeg and TTL baselines. The proposed method also performs well on other benchmarks.
- The appendix includes global Gaussian baselines, bootstrap ablations, and an analysis of OOD-bank purity and coverage. Code is also provided to reproduce the results.
Critical weaknesses
- The definition of adaptive-z3 is unclear. Does it refer to Equation 1?
- The complete detector performs strongly, but the additional contribution of semantic Gaussian decomposition appears modest. In Table 7, replacing global Gaussian banks with semantic banks changes the average FPR95 from 6.42% to 6.11% and AUROC from 98.33% to 98.34%. Without uncertainty estimates, it is difficult to assess whether this improvement justifies the additional components.
- The main ImageNet experiments use randomly ordered streams with a single fixed OOD source and predominantly ID samples. Achieving strong performance requires accumulating a substantial number of samples: the average FPR95 is 22.09% after 2K samples, compared with 6.11% over the full stream. This setting does not reflect real-world streams, in which different OOD sources may appear over time.
- It is unclear why TTL’s performance in Table 1 is lower than that reported in its original paper, whereas its performance in Table 2 matches the original results.
- Figure 1(a) is broken.
- Table results is far from the experiment section.
Questions for authors
- How is adaptive-z3 defined, and does it correspond to Equation 1?
- Are the gains from semantic Gaussian decomposition statistically significant across multiple runs, and do they hold across other datasets?
- How does OSGB perform when OOD sources change over time or when only a small number of samples are available for adaptation?
- Why is TTL’s performance in Table 1 lower than that reported in its original paper, while its performance in Table 2 matches the original results? Were the evaluation settings or implementation details different?