acceptodds
Under review as a conference paper at ICLR 2027

Semantic Anchor Learning with Conflict-Aware Evidential Inference for CLIP-Based Few-Shot Open-Set Recognition

Abstract

Prompt learning for vision-language models has shown strong performance in few-shot classification and has recently been extended to few-shot open-set recognition through unknown-text modeling. Existing methods construct surrogate unknown semantics using manually selected open words, generated pseudo samples, or statistically derived unknown directions, but they provide limited control over the geometry between known and unknown prompts and typically make open-set decisions from a single response score. We propose Semantic Anchor Learning with Conflict-Aware Evidential Inference (SAEC), a CLIP-based framework that couples unknown semantic construction, boundary regularization, and evidential decision making. SAEC first factorizes auxiliary token embeddings into a sparse dictionary and composes its primitives into synthesized unknown semantic anchors. It then regularizes the shared embedding space with unknown-aware probability regularization, explicit known–unknown separation, and intra-unknown dispersion. At inference time, SAEC converts competing known and unknown similarities into a Dempster–Shafer-inspired mass representation with a residual ignorance term. Across eight standard few-shot open-set benchmarks, SAEC improves AUROC over the directly comparable UTL baseline on all eight settings, with an average gain of 1.35 points and a maximum gain of 3.58 points, while achieving the best closed-set accuracy on six settings among the reported baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Few-Shot Open-Set Recognition, CLIP, Prompt Learning, Semantic Anchor Learning, Evidential Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3wQYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary
  • Objective: The manuscript addresses Few-Shot Open-Set Recognition using vision-language models by three main modules: 1) constructing useful representations for unknown (unseen) categories, 2) regularizing the prompt embedding space, and 3) making decisions beyond single-score response confidence.
  • Methodology: Keeps both textual and visual CLIP encoders completely frozen, then introduces a 3-stage framework:
    1. SDSA (Syntax-Decoupled Semantic Anchor Synthesis): Factorizes 1,000 auxiliary label token embeddings into a sparse dictionary and composes synthetic unknown semantic anchors using learnable composition weights with Gaussian perturbation.
    2. USAR (Unknown-Aware Semantic Anchor Regularization): Trains known prompts and unknown anchors using joint unknown label smoothing, and intra-unknown dispersion.
    3. CEI (Conflict-Aware Evidential Inference): Maps similarities via Softplus function to in-distribution and out-of-distribution evidence, constructs Dempster-Shafer masses with prior ignorance, and derives a decision score.
  • Key Findings: Evaluated across 8 standard FSOR benchmarks under the target 16-shot protocol (CIFAR-10, C10+10, C10+50, CIFAR-100, TinyImageNet, TinyImageNet-Hard, MiniImageNet, ImageNet-200) using CLIP ViT-B/16. Outperforms the baseline UTL across all 8 settings in open-set AUROC (+1.35 points average gain; max +3.58 on C10+10) while achieving top closed-set classification accuracy on 6 of 8 benchmarks with 13,192 trainable parameters.
Critical strengths
  • The manuscript provides an end-to-end integration of structured unknown anchor synthesis, geometric prompt-space regularization, and evidential belief-plausibility score discounting under a fully frozen CLIP backbone. The mathematical derivation in Eq. 16 & 17 proves strict monotonicity, offering a principled alternative to heuristic single-confidence thresholding.
  • The extensive results shown that the proposed framework consistently outperforms existing methods across 8 standard benchmarks with low parameter overhead (13,192 parameters). The qualitative t-SNE visualizations (Figure 3) and score density distributions (Figure 4) confirm that USAR module effectively disperses unknown anchors throughout open space while CEI separates known and unknown score distributions.
Critical weaknesses
  • There is a problematic issue in Figure 1 (under the list of contributions); it appears without any accompanying figures.
  • The paper provides a progressive ablation (Table 4) rather than a full factorial ablation, failing to isolate the individual contributions of each element forming the overall objective (loss function), making it hard to prove whether both hinge losses are equally necessary. Furthermore, Appendix B (Table 7) reveals that varying the number of synthesized unknown anchors from 1 to 100 changes C100 AUROC by a negligible 0.01% (87.84% vs 87.85%), raising concerns that generating 50 dictionary-composed anchors adds unnecessary architectural complexity over a simpler negative prompt vector.
  • Main evaluations are limited to a single visual backbone (CLIP ViT-B/16). Additionally, benchmark results in Tables 1 and 2 represent single point estimates without reporting multi-seed standard deviations or statistical significance tests (e.g., paired t-tests or Wilcoxon signed-rank tests) across support set samplings. Additionally, it lacks of evidence for the contribution of CEI, while the paper explicitly concedes that CEI does not perform Dempster's combination rule, nor does it provide calibrated epistemic uncertainty.
Questions for authors
  1. Can you provide a full factorial ablation isolating SDSA, Lsep\mathcal{L}_{sep}, Ldis\mathcal{L}_{dis}, and CEI individually? Specifically, what is the AUROC impact of Lsep\mathcal{L}_{sep} vs. Ldis\mathcal{L}_{dis} independently, and how does CEI perform on an unregularized prompt space?
  2. Given that Nu=1N_u=1 anchor achieves virtually identical AUROC to Nu=100N_u=100 anchors on C100 (87.84% vs 87.85% in Table 7), what is the technical justification for fixing Nu=50N_u=50 and building a 100-atom sparse dictionary?
  3. Does SAEC generalize to CNN-based CLIP backbones or larger Vision Transformers?
  4. Can you report mean AUROC/ACC and standard deviations across at least 5 independent random 16-shot support set splits?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3wQY

We thank the reviewer for recognizing SAEC’s framework, parameter efficiency, and mathematical analysis. We have expanded the ablations, added multi-seed statistics, and clarified our conclusions.

Q1: Figure 1 presentation.

A1: We corrected the figure insertion and placement. Figure 1 is now fully displayed and explicitly referenced in the text.

Q2: Component contributions, factorial ablation, and CEI without geometric regularization.

A2: The revision includes progressive and targeted ablations:

  • Table 4: Replacing PCA with SDSA, adding USAR, and adding CEI progressively improves AUROC on C10+10, T200, M100, and IN200.
  • Table 6: Comparing PCA, Random, Direct, and SDSA while retaining USAR and CEI examines anchor construction. SDSA achieves the highest mean AUROC and ACC on both benchmarks, although its AUROC advantage over Direct on T200 is only 0.10 points.
  • Table 5: A 2×22\times2 ablation examines separation and dispersion while retaining SDSA, unknown label smoothing, and CEI. Results are mean ± sample standard deviation (%):
ConfigurationC10+10 AUROCC10+10 ACCT200 AUROCT200 ACC
Base92.42±3.3696.20±0.8288.75±1.0289.10±1.84
Sep Only92.80±3.1496.17±0.7189.45±0.3689.20±0.99
Dis Only92.63±3.4996.03±0.1889.61±0.7289.25±1.48
Full USAR93.29±3.1996.88±0.8390.03±1.1490.31±1.72

Separation alone improves AUROC by 0.38/0.70 points on C10+10/T200; dispersion alone improves it by 0.21/0.86 points. Combining them gives gains of 0.87/1.28 points and the highest mean ACC. Their contributions differ across datasets; these results support using both without claiming equal necessity.

CEI: Table 7 compares CEI with Max-Known, Evidence-Margin, Evidence-Ratio, ID-Belief, and Pignistic without additional representation learning. CEI achieves the highest reported mean AUROC on all four settings, averaging 90.85%, versus 90.35% for ID-Belief. Alternative scores use four-run statistics and CEI uses five-run statistics, making this supplementary rather than fully seed-matched evidence.

These studies do not constitute a full factorial analysis across all components, and we do not claim to isolate every interaction. Base removes both geometric losses but retains unknown label smoothing and CEI. It therefore evaluates the framework without geometric regularization, but does not isolate CEI’s gain over alternative scores in that space. This specific matched comparison remains untested.

CEI uses a belief–plausibility product for open-set scoring; it neither applies Dempster’s combination rule nor claims calibrated epistemic uncertainty. Table 9 additionally shows higher mean residual ignorance mass for incorrect, low-confidence, and OOD samples relative to their respective comparison groups. This characterizes behavior rather than establishing calibration.

Q3: Why use 50 anchors and a 100-atom dictionary?

A3: We agree that Table 11 shows little AUROC benefit from increasing anchor count on C100: Nu=1,50,100N_u=1,50,100 yield 87.84%, 87.85%, and 87.61%. However, 50 anchors give higher ACC than one (88.42% vs. 87.72%). We retain 50 as a shared configuration considering both metrics, without claiming universal superiority or an optimal count.

Dictionary size K=100K=100 specifies the reusable basis, whereas Nu=50N_u=50 specifies the synthesized prompts. Table 6 evaluates dictionary-based construction but does not establish the necessity of exactly 100 atoms. We keep K=100K=100, α=1\alpha=1, and σ=0.1\sigma=0.1 fixed; dictionary-size sensitivity remains unevaluated.

Q4: Generalization to other CLIP backbones.

A4: SAEC operates through prompts, anchor embeddings, and image–text similarities while freezing both encoders. It is structurally compatible with CLIP architectures providing aligned embeddings and a compatible text-prompt interface. However, experiments only evaluate ViT-B/16. We distinguish this compatibility from demonstrated cross-backbone generalization, which remains future work.

Q5: Five independent runs and uncertainty reporting.

A5: Tables 1–2 now report mean AUROC/ACC and sample standard deviations across seeds {0,1,2,3,4}\{0,1,2,3,4\}. Each seed controls class partitioning, 16-shot support sampling, and trainable-parameter initialization. SAEC and UTL use identical partitions and support sets within each seed.

SAEC improves mean AUROC on all eight settings, averaging 1.35 points, with a maximum gain of 3.58 points. All 40 seed-wise AUROC differences are positive. Mean ACC exceeds UTL on all eight settings and ranks highest among compared methods on six. These results support consistency across sampled conditions; we do not claim statistical significance without a paired test.

We thank the reviewer again for helping clarify the component evidence and evaluation scope.

Thanks for your response. I decided to keep my rating.

Reviewer_ut8AOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SAEC, a CLIP-based framework for few-shot open-set recognition that combines semantic anchor synthesis, embedding-space regularization, and evidential inference. It constructs unknown anchors using a learned dictionary of auxiliary token embeddings, regularizes known–unknown prompt geometry, and integrates competing similarity scores for open-set decisions. Experiments across eight FSOR benchmarks demonstrate consistent improvements over the UTL baseline while maintaining competitive closed-set accuracy.

Critical strengths
  1. The paper addresses the important problem of unknown-class modeling in CLIP-based few-shot open-set recognition.

  2. The framework integrates semantic anchor construction, prompt-space regularization, and open-set decision making within a lightweight architecture.

  3. Evaluations across eight benchmarks show consistent improvements over UTL, supported by progressive ablations and hyperparameter analyses.

Critical weaknesses
  1. The authors should quantitatively assess their alignment with held-out unknown samples, beyond the t-SNE visualizations.

  2. Equation (3) enforces sparse coding, but Equation (4) uses unconstrained anchor weights. Comparisons with random dictionaries and directly learnable anchors are needed.

  3. The authors should compare CEI with known-class confidence, known–unknown similarity margins, and ratio-based scoring under identical representations.

  4. Table 4 reports ACC improvements despite Equation (18) retaining the same known-class argmax. The authors should clarify the evaluation protocol and explain these differences.

Questions for authors

I would appreciate further quantitative evidence on the alignment of synthesized anchors with unseen classes, the necessity of the learned dictionary, and the advantages of CEI over simpler scoring strategies. The ACC improvements in Table 4 also require clarification given Equation (18). If these concerns are not adequately addressed with convincing experiments and explanations, I may consider lowering my score.

2 comments
AuthorsOpenReview ↗

Response to Reviewer ut8A

We thank the reviewer for the constructive feedback and for recognizing the relevance of unknown-class modeling, SAEC’s lightweight design, and its benchmark results. We have added quantitative anchor-alignment analysis, alternative anchor constructions, and scoring-function comparisons, and clarified the closed-set ACC protocol.

Q1: Quantitative alignment with held-out unknown samples.

A1: We agree that t-SNE alone is insufficient. Appendix C, Table 8 now reports three quantitative metrics. For normalized image features z(x)z(x) and encoded unknown text anchors aja_j, we compute:

s(x)=max⁡1≤j≤Nuz(x)⊤aj.s(x)=\max_{1\leq j\leq N_u}z(x)^\top a_j.

UAS and ID-UAS are the means of s(x)s(x) over held-out unknown and known-class test samples, respectively. Their difference is:

UAG=UAS−ID-UAS.\mathrm{UAG}=\mathrm{UAS}-\mathrm{ID\text{-}UAS}.

A positive UAG indicates higher average maximum anchor similarity for unknown than known samples. SDSA obtains:

BenchmarkUASID-UASUAG
C10+10−0.053469±0.020700-0.053469\pm0.020700−0.073579±0.022368-0.073579\pm0.0223680.020110±0.0061260.020110\pm0.006126
T200−0.015007±0.035299-0.015007\pm0.035299−0.019678±0.034671-0.019678\pm0.0346710.004671±0.0028600.004671\pm0.002860

SDSA achieves the highest mean UAS and UAG among PCA, Random, Direct, and SDSA on both benchmarks. These results provide evidence of relative compatibility with held-out unknown samples, rather than establishing absolute semantic alignment. All metrics are computed on test samples without additional training or checkpoint selection.

Q2: Sparse dictionary learning versus unconstrained composition weights.

A2: Equations (3) and (4) describe different stages. The sparse codes in Eq. (3) reconstruct auxiliary token embeddings and guide dictionary learning. The task-specific weights in Eq. (4) combine the learned dictionary atoms into unknown anchors. We leave these composition weights unconstrained to permit flexible combinations; sparsity in dictionary learning does not require sparse downstream compositions.

Table 6 compares PCA, Random, Direct, and SDSA while retaining USAR and CEI. Direct learns anchors without dictionary-based composition.

ConstructionC10+10 AUROC (%)T200 AUROC (%)
PCA92.85±3.4592.85\pm3.4589.28±1.1889.28\pm1.18
Random92.76±3.6092.76\pm3.6088.60±1.1988.60\pm1.19
Direct92.73±3.3392.73\pm3.3389.93±0.9789.93\pm0.97
SDSA93.29±3.1993.29\pm3.1990.03±1.1490.03\pm1.14

SDSA also achieves the highest mean ACC on both benchmarks: 96.88% on C10+10 and 90.31% on T200. These results support dictionary-based construction in the evaluated settings. However, its AUROC advantage over Direct on T200 is only 0.10 percentage points, so we describe the benefit as modest rather than claiming the dictionary is indispensable.

Q3: CEI versus simpler open-set scores.

A3: Table 7 compares CEI with Max-Known, Evidence-Margin, Evidence-Ratio, ID-Belief, and Pignistic. These scores use the learned representations without additional training or feature adaptation.

CEI obtains AUROC values of 93.29%, 94.45%, 90.03%, and 85.62% on C10+10, C10+50, T200, and T200-H, respectively. Its average is 90.85%, compared with 90.35% for ID-Belief and 90.31% for Max-Known. CEI has the highest reported mean AUROC on all four settings.

The alternative scores retain their four-run statistics, whereas CEI uses the revised five-run main results. We therefore treat this as supplementary evidence rather than a fully seed-matched comparison.

Q4: Why does ACC change if Eq. (18) preserves the known-class argmax?

A4: We agree that changing only the open-set score for a fixed checkpoint and test set must leave closed-set ACC unchanged. CEI does not alter known-class predictions.

Table 4 is a progressive ablation: UTL baseline, +SDSA, +USAR, and +CEI. SDSA and USAR change training and the learned text representations, which can change known-class predictions. CEI only changes inference-time open-set scoring.

Accordingly, ACC is identical before and after adding CEI: 96.88%, 90.31%, 95.51%, and 82.34% on C10+10, T200, M100, and IN200, respectively, while AUROC improves. Section 4.3 now explicitly distinguishes representation learning from open-set scoring.

Additionally, Table 9 shows higher mean residual ignorance mass for incorrect versus correct ID predictions, low- versus high-confidence ID samples, and OOD versus correctly classified ID samples. This supplements the analysis of CEI’s behavior without establishing formal uncertainty calibration.

Thank you for your response. I stand by my original score.

Reviewer_RfevOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose SAEC for CLIP based few-shot open-set recognition. The proposed method learns a sparse dictionary from auxiliary token embeddings then synthesizes unknown anchors through learnable compositions and trains shared prompts with unknown-label smoothing, known–unknown separation, and unknown-anchor dispersion. Both CLIP encoders remain frozen. At inference, the strongest known and unknown similarities are mapped through Softplus into evidential masses, and the product of known-class belief and plausibility is used as the open-set score. Under the reported 16-shot protocol, SAEC improves AUROC on all eight settings.

Critical strengths

Overall I think the method's key feature on parameter-efficient is important and addresses a relevant problem. SAEC learns only the shared context and unknown-anchor composition weights for each target task and looks to be flexible method. The main comparison is encouraging and appropriately qualified. The paper clearly labels all-data methods in Table 3 as contextual comparisons rather than directly comparable few-shot baselines. These are useful results, although the uncertainty and ablation issues below limit their interpretation.

Critical weaknesses

This paper reports point estimates without a stated number of training repetitions or uncertainty across support-set draws and optimization seeds. This is especially relevant to gains of +0.32 and +0.49 AUROC points, although their small size alone is not a reason to dismiss them. The selection rule for the reported hyperparameters is also unspecified. For example, IN200 AUROC changes from 97.79 at gamma= 0.5 to 94.29 at gamma=1 in Table 8. Please report paired repetitions with UTL, the sources of randomness, the checkpoint rule, and how hyperparameters were chosen.

The other concern is reproducibility. this paper does not specify the number of independent runs, whether the tabulated values are single run results or averages or provide standard deviations or confidence intervals. It also does not explain how the few-shot support sets are sampled across runs or whether SAEC and UTL are evaluated on identical sampled sets. This is consequential because several key improvements are below one AUROC point, and some component gains are approximately 0.1 points or less. Without controlled randomly seeded evaluations, I cannot determine whether these differences reflect consistent benefits of the proposed method or variation in sampling and optimization. Please report the run count, sampling and seed protocol, per-run results, and uncertainty over the improvements over UTL. These info will greatly benefit the community on the contribution of the proposed method

Questions for authors

Echoing my reproducibility concern, how many independent support-set draws and training runs underlie the results? Please provide paired results against UTL, the checkpoint and hyperparameter selection rules, and the configurations needed to reproduce the corrected ablations.

1 comment
AuthorsOpenReview ↗

Response to Reviewer Rfev

We thank the reviewer for recognizing SAEC’s parameter efficiency and for the constructive comments on statistical reliability and reproducibility. We have added multi-seed results and clarified the evaluation, checkpoint, hyperparameter, and ablation protocols.

Q1: Number of independent support-set draws and training runs.

A1: The revised main comparisons use five independent runs with seeds {0,1,2,3,4}\{0,1,2,3,4\}. Each seed controls the known/unknown class partition, 16-shot support-set sampling, and trainable-parameter initialization. Tables 1–2 report the mean and sample standard deviation across these runs, accounting for variation in class selection, support sampling, and initialization.

Q2: Paired evaluation against UTL and uncertainty in the improvements.

A2: For each seed, SAEC and UTL use identical class partitions and 16-shot support sets, the same CLIP ViT-B/16 backbone, and a common evaluation protocol.

SAEC improves mean AUROC over UTL on all eight settings by 2.26, 3.58, 0.64, 1.03, 0.97, 0.32, 0.49, and 1.48 percentage points, respectively, averaging 1.35 points. All 40 seed-wise AUROC differences across the eight settings are positive. Mean closed-set ACC is also higher on all eight settings.

The reported sample standard deviations characterize each method’s run-to-run variability; they are not confidence intervals for the paired improvements. We therefore interpret these results as evidence of consistency across sampled conditions, without claiming statistical significance.

Q3: Checkpoint selection.

A3: All configurations are trained for 50 epochs, and the final-epoch checkpoint is evaluated. No checkpoint is selected using validation or test performance. For each method and run, both ACC and AUROC are evaluated using that same final checkpoint. This rule is applied consistently across seeds and configurations and is explicitly documented in Appendix B.2.

Q4: Hyperparameter selection.

A4: Appendix B.3 clarifies the selection protocol, with sensitivity results in revised Tables 11–15. We vary individual parameters while holding the others fixed, except that the margins are varied jointly with m=δm=\delta. We examine NuN_u on C100, γ\gamma on IN200, and ε\varepsilon, C0C_0, and (m,δ)(m,\delta) on C10+50.

We adopt one shared configuration across all eight settings: Nu=50N_u=50, γ=0.5\gamma=0.5, ε=0.1\varepsilon=0.1, C0=10C_0=10, and m=δ=0.2m=\delta=0.2, without dataset-specific retuning. On C100, Nu=5N_u=5 gives slightly higher AUROC than Nu=50N_u=50 (87.89% vs. 87.85%), whereas Nu=50N_u=50 gives higher ACC (88.42% vs. 87.90%). We retain 50 considering both metrics, rather than claiming it maximizes AUROC everywhere. Dictionary size K=100K=100, sparsity coefficient α=1\alpha=1, and perturbation scale σ=0.1\sigma=0.1 remain fixed.

These sensitivity analyses use the corresponding test sets rather than an independent validation set, as now disclosed in Appendix B.3. Sharing parameters avoids dataset-specific retuning, but does not make test evaluation independent of parameter selection.

Q5: Configurations for reproducing the corrected ablations.

A5: Appendix C specifies the reference configuration and targeted changes:

  • SDSA (Table 6): Compare PCA, Random, Direct, and SDSA while retaining USAR and CEI. Direct learns unknown anchors without dictionary-based composition.
  • USAR (Table 5): Retain SDSA, CEI, and unknown label smoothing. Base removes both geometric losses; Sep Only adds separation; Dis Only adds dispersion; Full USAR includes both.
  • CEI (Table 7): Compare Max-Known, Evidence-Margin, Evidence-Ratio, ID-Belief, and Pignistic with CEI using the learned representations, without additional training or feature adaptation.

Other settings remain fixed. Alternative scoring functions retain their four-seed statistics, whereas CEI uses the revised five-seed main results. We treat this as a supplementary comparison rather than a fully seed-matched evaluation.

We also added residual ignorance mass analysis (Table 9), with explicit sample-group definitions and aggregation across seeds. The revision specifies the backbone, trainable parameters, optimizer, and principal hyperparameters.

We thank the reviewer again for helping improve the experimental transparency. The implementation, configurations, and model weights are prepared for release; we will make the repository public after double-blind review restrictions are lifted and include its link in the final manuscript.

Comments on the paper

1 comment

Dear reviewers,

The rebuttal period is ending soon. Please read the authors rebuttal and check if they addressed your concerns. Please acknowledge the rebuttal and adjust your scores accordingly.

Thanks.

-AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.