acceptodds
Under review as a conference paper at ICLR 2027

OptLabel: Label Optimization for Negative-Label-Based OOD Detection

Abstract

Negative-label-based methods have emerged as a strong paradigm for CLIP-based out-of-distribution (OOD) detection. However, existing methods are largely text-driven, defining positive labels directly from in-distribution (ID) class names and selecting negative labels based on textual dissimilarity from ID classes. Consequently, their positive and negative labels may poorly represent the ID distribution, and OOD samples close to the ID boundary remain underrepresented. To address these limitations, we propose OptLabel that optimizes the representation of positive and negative labels under the ID-supervised setting. OptLabel treats the positive and negative labels from existing zero-shot negative-label-based methods as a semantic prior and refines their representation using the available ID data and the near-boundary OOD samples synthesized by applying Gaussian perturbation to ID samples in the CLIP embedding space. We show that, in the high-dimensional CLIP space, the placement of these synthesized OOD samples has an intuitive angular interpretation, and is thus easily controllable. OptLabel is a lightweight solution by keeping the CLIP frozen and requiring no image generation, external models, or auxiliary datasets. On the ImageNet-1K benchmark, OptLabel outperforms SynOOD, the strongest ID-supervised baseline, by 2.41 AUROC and 6.51 FPR95 points. The source code is available in the Supplementary Material.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Out-of-distribution detection, vision-language models, CLIP, negative labels, label embedding optimization, near-boundary OOD synthesis, high-dimensional geometry

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EqvNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OptLabel, a framework for ID-supervised OOD detection based on negative labels in CLIP's embedding space. The method consists of two stages: first, it optimizes positive label embeddings using cross-entropy on labeled ID data; second, it freezes the optimized positive labels and optimizes negative label embeddings using both ID features and synthetic OOD features. The synthetic OOD features are generated by applying isotropic Gaussian perturbations to normalized ID embeddings, followed by renormalization. The paper provides a geometric analysis of the resulting angular displacement and argues that this simple synthesis strategy can generate useful near-boundary OOD supervision without additional filtering. Experiments verify the advantages of the proposed method.

Critical strengths
  1. The paper addresses a relevant problem in negative-label-based OOD detection and reports consistent improvements over existing methods.
  2. The method is simple and efficient, requiring neither CLIP fine-tuning nor external OOD data or image generation models.
Critical weaknesses
  1. The methodological novelty is limited, as the main components, including Gaussian feature perturbation, label embedding optimization, and two-stage training, are largely based on established techniques. The method primarily combines these existing ideas, and the geometric analysis does not sufficiently establish a substantial methodological advance over prior work.
  2. The negative labels are optimized against Gaussian-perturbed ID features, whose distribution may differ substantially from that of real OOD samples. The paper provides limited evidence explaining why the learned separation should transfer to unseen OOD distributions.
  3. Perturbing every ID feature without considering local density or class structure may generate samples that still belong to the ID distribution. Treating all such samples as OOD could introduce conflicting supervision, especially in regions where ID classes overlap.
  4. Directly optimizing negative-label embeddings in CLIP's feature space may weaken their original semantic meaning. The paper does not examine whether the optimized negative labels retain meaningful semantic information or simply become learned discriminative prototypes.
Questions for authors
  1. What is the key methodological contribution of OptLabel beyond combining existing techniques? In particular, how does the proposed approach fundamentally differ from prior feature-space OOD synthesis and label optimization methods?
  2. What evidence supports the assumption that a detector trained with Gaussian-perturbed ID features can generalize to real, unseen OOD distributions, given the potential distribution gap between synthetic and real OOD samples?
  3. How do the authors ensure that perturbing every ID feature does not generate samples that still belong to the ID distribution? Could such samples introduce conflicting supervision, particularly in overlapping or high-density ID regions?
  4. Do the optimized negative-label embeddings retain their original semantic meaning, or do they effectively become learned discriminative prototypes? Can the authors provide evidence that the semantic information remains important after optimization?
2 comments

We sincerely thank the reviewer for taking the time to review our work and for providing insightful comments. We have made our best effort to address all the comments and would greatly appreciate any further questions or feedback from the reviewer.

Response to Question 1

Our methodology goes beyond primarily combining existing techniques/ideas. Each component differs from existing techniques in several major ways:

On 2-stage training: A straightforward way of applying label optimization, which is close to the idea of classifier prototype learning, to our negative-label based setting would be the joint-optimization of positive and negative labels in a single loss. However, this joint-optimization is outperformed by our 2-stage training that first optimizes positive labels then optimizes negative labels while freezing optimized positive labels, effectively preventing distorting the optimized positive-label representations (see Table 4, lines 462-465). Prior to this study, it is not clear which order of optimization works best.

On label optimization: SynOOD trains an additional projection layer over CLIP's image encoder and optimizes only a subset of negative labels. In comparison, by optimizing the label embeddings generated by the text encoder without introducing an additional layer, we preserve CLIP's original representations and generalization capabilities. Prior prompt-tuning methods optimize the context tokens of a class label in CLIP's input space while keeping class label itself fixed, for example, in the prompt "a photo of a [dog]", the context tokens "a photo of a" are replaced with learnable vectors. This context token is shared for all classes but the actual class (e.g., [dog]) is not optimized. We directly optimize individual class label's embeddings in CLIP's output space, examining a larger optimization space and allowing the thousands of negative labels to be updated efficiently.

On OOD synthesis: Our analysis theoretically guarantees that in high dimensional space (like CLIP's) the synthesized OOD sample from every ID image moves away from all ID samples with high probability, therefore, there is no need to consider local density or additional filtering. A more detailed explanation is given in our response to your Question 3. This property of Gaussian perturbation in high dimensional space was previously unknown in the OOD synthesis literature, for example, NPOS uses a filtering step to remove certain synthesized samples. Our method differs from recent OOD synthesis methods, like SynOOD, and few-shot methods LoCoOp, and Local-Prompt, that derive OOD supervision from non-ID concepts or crops in ID images, limiting their OOD coverage to the background content present in the ID train images (lines 90–93, 147–149), as demonstrated empirically on the ImageNet subsets (Table 2, lines 399–404). Unlike these methods, we synthesize OOD samples directly in the embedding space, which generalizes better to the unseen OOD distribution, as explained in the next response.

Response to Question 2

The generalization to real, unseen OOD distributions is supported along the following lines:

First, several prior studies, e.g., VOS, NPOS, SynOOD, have reported that OOD samples near the ID boundary are sufficient for learning ID/OOD separation, and OOD samples far from the boundary (i.e., in the interior of OOD regions) provide no additional useful information. Similar to these studies, our sampling is not to uniformly cover the entire real OOD region, but to synthesize challenging near-boundary OOD samples that help learn an effective separation.

Second, our negative labels (for representing OOD space) is initialized using the WordNet-based labels suggested by prior zero-shot methods, which represents a wide range of semantic OOD concepts and provides a broad coverage of the OOD space. Table 4 supports the importance of this semantic initialization: replacing it with random initialization increases average FPR95 from 21.72 to 28.51.

Third, empirical evidence for this generalization comes from our experiments across diverse ID/OOD settings, where the test OOD data are real and unseen to our label optimization and OOD synthesis, and these settings cover both near-boundary and far-boundary OOD datasets (Tables 1–2). Our method consistently achieves substantial improvements over prior methods in the same ID-supervised setting. See additional empirical evidences in our response to Weakness 3 of Reviewer PN3Z.

Response to Question 3

At first glance, it seems that perturbing every ID image may generate synthesized OOD samples that move closer to other ID samples, potentially introducing conflicting supervision. To address this concern, NPOS applies a filtering step based on k-NN distances to ID samples to remove problematic synthesized samples close to ID data. One of our contributions is the theoretical analysis that shows that this additional filtering is unnecessary in a high dimensional space like CLIP's.

Theoretically, we showed that for a synthesized OOD sample z~i\tilde z_i from perturbing an ID anchor ziz_i, and for any other ID sample zjz_j, the approximation sim(z~i,zj)≈cos⁡θ sim(zi,zj)\mathrm{sim}(\tilde z_i, z_j) \approx \cos\theta \,\mathrm{sim}(z_i, z_j) (see Equation 6) holds in high dimension space with high probability, where the error term O(sin⁡θsin⁡ϕij/d−1)O\left(\sin\theta\sin\phi_{ij}/\sqrt{d-1}\right) diminishes with high dimension dd. See Equation 11 and Appendix A.2 for the derivation of Equation 6. Essentially, this result says that the similarity between the synthesized sample z~i\tilde z_i and any other ID sample zjz_j is reduced by the factor cos⁡θ\cos\theta (which is between 0 and 1), compared to the similarity between the two ID samples ziz_i and zjz_j, thus, the synthesized sample is not closer to zjz_j than the two ID samples ziz_i and zjz_j. This theoretically shows that in the case of high dimension dd, a synthesized sample moves away from all of ID samples, instead of moving away from one while getting closer to another. A sufficiently large angular displacement is needed to move synthesized samples beyond the ID region. We selected the angle θ\theta empirically as shown in Figure 4 and θ=30∘\theta=30^\circ works well for all tested datasets and settings.

The intuition for a synthesized sample z~i\tilde z_i not only moving away from its ID anchor ziz_i but also from the other ID sample zjz_j is that the Gaussian perturbation places the synthesized sample randomly on an angular shell at the angle θ\theta from its ID anchor ziz_i (see the red circle in Figure 2). In a high dimensional space, there are many possible directions on this shell, making a random perturbation unlikely select a direction that would bring the synthesized OOD sample closer to another ID sample zjz_j, because random directions in high-dimensional spaces tend to be nearly orthogonal. In contrast, this property does not hold in low-dimensional spaces. For example, in 2D space, there are only two possible directions along the unit circle, and a small Gaussian perturbation has a large chance of bringing the synthesized sample closer to another ID sample.

The above result is also supported empirically on the ImageNet-1K benchmark. Figure 4 shows that keeping all synthesized OOD samples without filtering (100% kept) achieves the lowest (best) average FPR95, and filtering synthesized samples (based on their kk-NN distances to ID data as performed in NPOS), consistently leads to larger (worse) average FPR95 across all tested values of θ\theta and all filtering ratios (keeping 50% to 90% of synthesized OOD samples). The same trend holds when using 2N2N synthesized samples (Appendix Figure 7).

In summary, our theoretical analysis makes three contributions: (i) it simplifies significantly the OOD synthesis process in high dimension case by removing the need for considering local density or filtering synthesized samples, (ii) it improves OOD detection by keeping all synthesized samples (i.e., data for learning), and (iii) it contributes new knowledge to the OOD synthesis literature.

Response to Question 4

Retaining the semantic meaning of individual negative labels is not required by the OOD detection problem. As defined by the OOD detection problem, it is sufficient to detect whether a sample is OOD; it is not required to tell exactly what negative label corresponds to the OOD sample. In other words, our negative labels are used as learned discriminative prototypes as the reviewer called it.

Reviewer_PN3ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OptLabel for CLIP-based OOD detection under ID supervision. The method sequentially optimizes positive and negative text embeddings using ID data and Gaussian-perturbed synthetic OOD features, while keeping CLIP frozen. A geometric analysis characterizes the perturbations via angular displacement. Experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  • This paper is easy to follow. The proposed method is simple and efficient, optimizing label embeddings without updating CLIP or requiring an additional structure.

  • The experiments on challenging OOD benchmarks speak to the effectiveness of the proposed method.

Critical weaknesses

I have the following concerns:

  • The term “label optimization” may be misleading, since the method keeps label names fixed and directly optimizes their text embeddings in CLIP’s output space. In addition, the authors should discuss the differences from prompt tuning (adopted in LOCOop) and classifier-prototype learning.

  • OptLabel uses the full ID training set, whereas LoCoOp and Local-Prompt use only 16 labeled images per class. In my view, this comparison is not fair and the authors are suggested to evaluate OptLabel under the same few-shot budgets to establish a fair comparison. I wonder the comparison results.

  • The geometric analysis characterizes feature displacement but does not guarantee that perturbed features are semantically OOD. The authors are encouraged to provide further empirical evidence and some theoretical analysis.

  • The selection of the perturbation angle is unclear. The authors should do some sensitive analysis about this key hyperparamter.

Questions for authors

please see the weaknesses above

2 comments

We sincerely thank the reviewer for taking the time to review our work and for providing insightful comments. We have made our best effort to address all the comments and would greatly appreciate any further questions or feedback from the reviewer.

Response to Weakness 1

By label optimization, we mean label embedding optimization or label representation optimization. We will clarify this in the revision.

Prompt-tuning methods (Related Work, lines 137–148), such as LoCoOp, optimize continuous context tokens in the input space of CLIP's text encoder. For example, in the prompt "a photo of a [dog]", the context tokens "a photo of a" are replaced with learnable vectors, which is often shared across all classes; it does not modify the embedding for the actual class [dog]. This requires a forward and backward pass through CLIP's text encoder at every training step. In contrast, our method directly optimizes each positive and negative label embedding in CLIP's output space, without invoking the text encoder (lines 144–147). This provides two key advantages: (i) each of the thousands of negative labels can be optimized individually, rather than learning only shared context tokens; and (ii) training is highly efficient, as the CLIP text encoder is not invoked. Also, unlike prompt-tuning methods such as LoCoOp, our objective is specifically to improve the SOTA negative-label-based OOD detection paradigm in the ID-supervised setting, and all our design choices are motivated by this objective, making our approach fundamentally different in its motivation and design.

Classifier-prototype learning focuses on ID class label representations. A straightforward adaptation of classifier-prototype learning to our OOD detection would be to jointly optimize positive and negative labels using a single loss, which, as showed in Table 4, yields suboptimal results. To address the specific challenges for OOD detection, our optimization differs in several important ways: (i) adds the large set of negative-label embeddings from existing zero-shot methods as initialization for OOD labels, (ii) synthesizes near-boundary OOD samples, and (iii) optimizes through a two-stage optimization with distinct objectives to ensure quality representation of ID labels before optimizing the representation of negative labels.

Response to Weakness 2

In this work, we focus on the ID-supervised setting where the full ID training dataset is available. This is an established setting adopted by several previous studies, including SynOOD, CASOD, and NPOS, which are already included as baselines in our experiments. Zero-shot and few-shot methods, such as LoCoOp and Local-Prompt, are therefore not our direct baselines, and we included them as references to show the performance gains achievable, thus, justifying the ID-supervised setting, when full ID data is available. Furthermore, as shown in Table 1, methods such as LoCoOp and Local-Prompt, even with access to few-shot ID data, still perform worse than zero-shot negative-label-based methods such as NegRefine and CSP; NegRefine, operating in the zero-shot setting, outperforms all these few-shot methods across all datasets in Table 1. Our primary objective in developing OptLabel is to improve this stronger negative-label-based paradigm in the ID-supervised setting, rather than to develop a few-shot method.

Response to Weakness 3

On theoretical analysis, instead of guaranteeing that all synthesized samples are truly semantically OOD, which is a difficult task, we show that, in high dimensional space, synthesized samples move away not only from their original ID samples but also from their local ID neighborhoods with high probability, with the displacement controlled by θ\theta. Please refer to our response to Question 3 of reviewer EqvN for detailed discussion on this guarantee. An appropriate θ\theta setting generates samples that are sufficiently far from ID data to be likely OOD, yet close enough to the ID boundary to be informative for learning the separation of ID/OOD. The Figure 4 presents the ablation on θ\theta.

As additional empirical evidence beyond the demonstrated performance, we measure the cosine similarity of each synthesized OOD sample, each ID training sample, and each ID test sample, to its 20th nearest neighbor in the ID training set (ImageNet-1K), excluding the sample itself for ID train to ID train similarities. The results are:

  • Synthesized OOD to ID train: 0.7481 (std. 0.0528)
  • ID train to ID train: 0.8596 (std. 0.0606)
  • ID test to ID train: 0.8581 (std. 0.0617)

Clearly, synthesized OOD samples have substantially lower similarity to the ID training data, suggesting that they lie outside typical local ID neighborhoods and can serve as surrogate OOD samples.

Response to Weakness 4

We used a fixed value of θ=30∘\theta = 30^\circ (see Implementation details, line 352) across all experiments, including the ImageNet-1K benchmark (Table 1), the ImageNet-subset and Spurious OOD benchmarks (Table 2), and all CLIP architectures (Appendix Table 7), without any OOD dataset-specific or architecture-specific tuning.

We already provided a sensitivity analysis over θ∈10∘,20∘,30∘,40∘,50∘\theta \in \\{10^\circ, 20^\circ, 30^\circ, 40^\circ, 50^\circ \\} in Figure 4 and repeated this analysis using 2N2N synthesized OOD samples in Appendix Figure 7. Considering the setting where 100% of synthesized OOD samples are kept (no filtering applied), the best performance was achieved at θ=30∘\theta = 30^\circ. For θ∈20∘,30∘,40∘\theta \in \\{20^\circ, 30^\circ, 40^\circ\\}, the average FPR95 varied by less than 1 percentage point. Even the worst setting (θ=10∘\theta = 10^\circ, FPR95 of 24.60) outperformed the strongest baseline, SynOOD (average FPR95 of 28.23), demonstrating the relative robustness of our method to this parameter.

Reviewer_cWGgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies ID‑supervised OOD detection for CLIP‑based negative‑label methods. Existing text‑driven negative‑label approaches suffer from misalignment with real in‑distribution data and poor handling of hard near‑boundary OOD samples. The authors propose OptLabel, a lightweight two‑stage framework that optimizes positive and negative label embeddings with CLIP frozen. Stage‑1 aligns positive labels to ID data via cross‑entropy. Stage‑2 optimizes negative labels using real ID data plus virtual near‑boundary OOD embeddings synthesized by Gaussian perturbation in CLIP’s embedding space. High‑dimensional geometric analysis validates the synthesis scheme and removes the need for sample filtering. Evaluations on ImageNet‑1K and other benchmarks show OptLabel outperforms prior ID‑supervised baselines notably on challenging near‑boundary OOD.

Critical strengths
  1. Clearly identifies key limitations of text‑based negative‑label OOD detection.
  2. Lightweight: only optimizes label embeddings; CLIP is frozen, no image generation or external datasets.
  3. Comprehensive experiments on clean low‑contamination OOD benchmarks, achieving SOTA, especially on hard near‑boundary OOD.
Critical weaknesses
  1. Lacks qualitative failure‑case analysis and ablation over different zero‑shot prior sources
  2. The novelty of virtual near‑boundary OOD embeddings is ordinary.
  3. The angle θ\theta maybe sensitive for different ID/OOD data
Questions for authors

Performance depends heavily on high‑quality initial label priors from NegRefine; random initialization may degrade results.

1 comment

We sincerely thank the reviewer for taking the time to review our work and for providing insightful comments. We have made our best effort to address all the comments and would greatly appreciate any further questions or feedback from the reviewer.

Response to Weakness 1

Here is failure case analysis using NINCO, the most challenging OOD dataset in our evaluation. With the FPR95 threshold (i.e., at least 95% of ID test samples are correctly detected as ID), we calculate the misdetection rate for each NINCO class (i.e., the percentage of its samples incorrectly detected as ID) and identify the ImageNet-1K ID classes that these misdetected samples most frequently match, based on their highest-scoring positive labels. The table below lists the five NINCO classes with the highest misdetection rates.

NINCO classMisdetection rateMatching ImageNet classes
Windsor chair100.0%rocking chair
gray silverfish99.0%centipede, cockroach
chicken quesadilla97.0%burrito
cat-faced spider96.0%barn spider
eel-like salamander (Amphiuma means)94.3%eel, worm snake

Although our method achieves substantial improvements on FPR95 over previous methods on this dataset, it still struggles to distinguish these OOD samples from ID samples due to their high visual similarity to ID images of certain ImageNet-1K ID classes (e.g., Windsor chairs vs. the ID class rocking chair).

Regarding zero-shot priors, our method does not depend on a specific zero-shot prior (like NegRefine). Its key important property is to provide a substantial improvement over the chosen prior, as shown by the following average FPR95 across the four OOD datasets:

MethodAverage FPR95 ↓
CSP34.47
NegRefine30.12
OptLabel (random init.)28.51
OptLabel (CSP init.)22.06
OptLabel (NegRefine init.)21.72

As you can see, OptLabel substantially improves upon both zero-shot priors, CSP and NegRefine: reducing FPR95 from 34.47 to 22.06 for CSP, and reducing FPR95 from 30.12 to 21.72 for NegRefine. Notably, the 4.35-point performance gap between CSP and NegRefine shrinks to just 0.34 points after our label optimization. Even with random initialization, OptLabel outperforms both zero-shot priors, while semantic initialization provides further improvements.

Response to Weakness 2

Our main novelty to OOD synthesis is a high-dimensional probabilistic analysis that proves that synthesized samples concentrate at a controllable angular distance θ\theta, and using this property we are able to provide the theoretical guarantee that, in high dimensional space like CLIP's, all synthesized samples move away not only from their ID anchors but also from other ID samples with high probability. This guarantee enables controllable angular displacement and motivating a simpler, more efficient synthesis strategy without additional k-NN filtering, as used in NPOS. More importantly, it also improves OOD detection performance by keeping all synthesized samples. This theoretical guarantee is proved in Equations 5–6 and Appendix A, also summarized in our response to Question 3 of Reviewer EqvN. To our knowledge, this high-dimensional characterization of Gaussian OOD synthesis was previously missing from the literature.

Recent methods commonly derive OOD supervision from non-ID concepts or crops contained in ID images (e.g., SynOOD, LoCoOp, and Local-Prompt), limiting their coverage to the background content present in ID images (lines 90–93, 147–149). We empirically demonstrated this limitation for SynOOD on the ImageNet subsets, where fewer ID classes provide less diverse OOD content, leading to degraded performance (Table 2, lines 399–404). In contrast, we synthesize OOD samples directly in the embedding space, avoiding this limitation.

Response to Weakness 3

We used a fixed θ=30∘\theta = 30^\circ across all experiments (see Implementation Details, line 352), without any dataset-specific or architecture-specific tuning. This fixed value performed well across diverse settings, including ImageNet-1K with OOD datasets ranging from far-boundary (iNaturalist) to near-boundary (NINCO), ImageNet-10/20/99, WaterBirds vs. Placesbg, and different CLIP architectures (Tables 1, 2, and Appendix Table 7). Moreover, we already provided a sensitivity analysis in Figure 4, showing that the average FPR95 varied by less than 1 point for θ∈20∘,30∘,40∘\theta \in \\{20^\circ, 30^\circ, 40^\circ\\} (when 100% of synthesized OOD samples were kept). Even the worst setting (θ=10∘\theta = 10^\circ, FPR95 of 24.60) outperformed the strongest baseline, SynOOD (average FPR95 of 28.23), demonstrating the relative robustness of our method to this parameter.

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your thoughtful review and valuable feedback.

We have provided detailed responses to your questions and concerns. We would greatly appreciate it if you could take a look at our responses and let us know whether they adequately address your concerns or if any further clarification is needed.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.