- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper focuses on fine-grained open-set recognition from the perspective of label smoothing in spherical representation learning. By analyzing four existing representation learning objectives, the authors identify a shared alignment–uniformity structure and propose SmoothOP, which adaptively adjusts the smoothing coefficient for each sample based on its prominence, estimated from classifier confidence margins. Experiments on the SSB datasets demonstrate improvements in fine-grained OSR performance.
Critical strengths
-
The paper is well structured and generally clearly written, with a logical presentation of the motivation and methodology. Especially the progression from the alignment uniformity analysis to the proposed adaptive smoothing mechanism.
-
The proposed method is intuitively motivated and can be easily incorporated into existing representation learning frameworks without substantial architectural modifications.
-
The experimental evaluation is relatively comprehensive, covering multiple datasets, and contrastive learning objects. Additional ablation studies and feature geometry analyses provide further insights into the proposed approach.
Critical weaknesses
-
As a standalone submission, a) although this paper is generally well written, too many methodological details and design choices rely heavily on the SpHOR[1], requiring readers to frequently refer to that work to fully understand the proposed framework, both in the main text and the Appendix. b) In my view, the paper is closer to an incremental investigation of the label-smoothing strategy adopted in existing representation learning methods, particularly SpHOR, building on the observations of [2]. The main methodological contribution is to replace a fixed smoothing coefficient with a sample dependent one.
-
The alignment–uniformity decomposition has already been extensively studied in prior literature. In particular, in OSR field [3] provides a closely related analysis in their subsection “Rethinking the Properties of Contrastive Learning,” where they explicitly show that label-dependent similarity weights affect only the alignment term, while the uniformity term remains independent of them. This observation substantially overlaps with the theoretical motivation of the present paper, raising concerns about the novelty of its theoretical contribution.
-
While the motivation is reasonable, its conceptual novelty appears limited, especially considering existing research on instance-dependent adaptive label smoothing. In particular, [4] already adapt the smoothing coefficient per sample according to the model’s predictive uncertainty. The paper does not sufficiently establish what constitutes a substantial methodological advance beyond these prior works.
-
The empirical improvement is not sufficiently consistent to support some of the paper's stronger claims.
[1] SpHOR: A representation learning perspective on open-set recognition for identifying unknown classes in deep neural networks. CVPR 2026.
[2] Open-set recognition: A good closed-set classifier is all you need. ICLR 2022.
[3] Contrastive open set recognition. AAAI 2023
[4] Adaptive label smoothing with self-knowledge in natural language generation. EMNLP 2022.
Questions for authors
In this section, my main concerns are related to several details of the experimental evaluation.
-
The experiments cover only three fine-grained datasets from the Semantic Shift Benchmark [2], while the ImageNet benchmark included in the original SSB is omitted. Could the authors explain this choice? I would also like to see evaluations on conventional OSR benchmarks, such as MNIST, SVHN, CIFAR, and TinyImageNet, to better assess the generalizability of the proposed method beyond fine-grained recognition.
-
In Table 3, applying SmoothOP to ConOSR on CUB reduces closed-set accuracy from 75.89% to 75.42%. Similarly, Table 4 shows a slight deterioration in intra-class compactness under the same setting. Could the authors provide further analysis of this degradation?
-
I would like the authors to discuss OpenAUC[1]. Furthermore, I am not fully convinced by the justification for excluding OOSA in Appendix A. In particular, the authors argue against using surrogate unknown validation data for threshold selection, yet such data are used to tune the proposed method's hyperparameter . Could the authors further justify their position and clarify this distinction?
[5] Openauc: Towards auc-oriented open-set recognition. NeruIPS 2022.
8 comments
Official Response to Reviewer 4ohG (Part 2/5)
CW2: Theoretical Contribution.
We agree that the decomposition itself is not new, and we thank the reviewer for this pointer. We revised the paper to make this clear. The decomposition builds on Wang & Isola [5], and ConOSR [3] and SpHOR each show it for their own objective. We extend and show that it holds across four OSR objectives in one template (Prop. 1, Table 1), and we rewrote contribution statement 1 (lines 94–100) to say this. Our theoretical contribution is what follows from the decomposition, and it motivates our proposed method, SmoothOP.
(1) The extension to label smoothing. ConOSR [3] shows that label-dependent similarity weights enter only the alignment term. However, it does not link this result to label smoothing: ConOSR and SpHOR both use a fixed smoothing coefficient, so the question of how the coefficient changes the objective does not arise. In Prop. 2 (with Props. 4–5 in App. E), we show that increasing the smoothing coefficient moves target mass from positives to negatives monotonically and by a quantified amount, while the uniformity term stays unchanged. This is what makes the per-sample coefficient of our proposed method interpretable: each value of has a known, one-directional effect on that sample’s alignment. This is the basis of the design principle in point 2.
(2) A theoretical lens for the role of label smoothing, which gives a design principle. *Labels play a different role in classifiers and in representation learning objectives, so label smoothing must also be interpreted differently (see also our response to CW 1(b) and 3). In a classifier, the label is the target of the predicted output, and label smoothing [6] is commonly interpreted as entropy regularization of that output [7], which reduces overconfidence and improves calibration [8]. With classes, label smoothing replaces the one-hot label with , where is the uniform distribution and is the smoothing coefficient. For a classifier with predicted class probabilities , the loss splits into two terms:
The first term fits the label. The second term penalizes confidence: a larger pushes toward higher entropy, i.e., less overconfidence. Existing adaptive methods follow this output-confidence view [4]. In a representation learning objective, the label instead sets the relation between each anchor and its positives and negatives (Eq. 4), and the goal is the geometry of the representation space, not a calibrated output. Props. 1–2 show that, in this setting, increasing smoothing reduces alignment, rather than regularizing output entropy. We therefore propose this alignment view of label smoothing as a design principle for adaptive smoothing in representation learning objectives. It turns the design into a concrete question: which samples need a stronger pull toward their class? Our proposed method answers it with prominence: samples with high prominence receive more smoothing, and samples with low prominence receive less smoothing and a stronger pull toward their class. (revised in lines 80–86) This is our theoretical contribution, and we now state it explicitly in the paper (lines 74-79). We also added a Related work section on classifier-based adaptive label smoothing methods. (Lines 134-139)*
[3] Xu, B., et al, Contrastive open set recognition. AAAI'23.
[4] Lee, D., et al, Adaptive label smoothing with self-knowledge in natural language generation. EMNLP'22.
[5] Wang, T., et al, Understanding contrastive representation learning through alignment and uniformity on the hypersphere. ICML'20.
[6] Szegedy, C., et al, Rethinking the Inception architecture for computer vision. CVPR'16.
[7] Pereyra, G., et al, Regularizing neural networks by penalizing confident output distributions. ICLRW'17.
[8] Müller, R., et al, When does label smoothing help? NeurIPS'19.
(continued in Part 3/5)
Official Response to Reviewer 4ohG (Part 3/5)
CW3: Conceptual Novelty of Instance dependent label smoothing.
We agree that instance-adaptive label smoothing exists, and we compare with [4] directly. At a high level, most adaptive methods, including [4] and ours, can be described as varying the smoothing coefficient per sample. The substance lies in the design decisions:
(1) A different setting and framework. [4] adapts the target of a classifier and follows the output-confidence view: it treats low output entropy as overconfidence. Our proposed method adapts the label in a representation objective, where smoothing controls alignment (see our responses to CW 1(b) and 2).
(2) Adaptive smoothing is task-specific, so rules do not transfer between tasks. Each adaptive label smoothing method rests on a design principle built for its own task, and its signal encodes the inductive bias of that task. CaliMatch uses validation accuracy per confidence level and is designed for semi-supervised learning, where unlabeled data are available; FedLAS uses the feature norm to calibrate output probabilities, not to reject unknowns; and [4] uses prediction entropy to counter overconfidence in text generation. A signal that is meaningful under one framework is not necessarily meaningful under another, so these rules cannot be assumed to transfer to OSR. The inductive bias of OSR is geometric: known classes must form compact, well-separated clusters, so that the remaining space is reserved for unknowns [11]. We therefore propose prominence, a signal that sets the alignment of each sample to produce this geometry (Sec. 4). In fine-grained OSR, unknown classes are visually close to known ones, so the regions between neighbouring known classes are the most at risk. Prominence, the margin between a sample’s own class and its strongest competing class, acts exactly on these regions: it gives samples near a class boundary less smoothing and a stronger pull.
Why prominence, and not entropy. Entropy measures how spread the output is, and it does not use the ground-truth label. It therefore cannot tell whether a confident prediction is correct: a confidently correct sample and a confidently misclassified sample both have low entropy, and both receive near-maximal smoothing under ALS [4]. Prominence uses the ground-truth label. The confidently misclassified sample has low prominence, so it receives weak smoothing and a stronger pull toward its own class, while the confidently correct sample receives strong smoothing.
Evidence. App. C.4 (Table 9) replaces only the rule that sets the coefficient with that of [4] or FedLAS, and keeps everything else identical. Our proposed method gives the best average OSCR on both splits for both objectives. We use the non-mixup variants because ALS/FedLAS doesn’t define a Mixup rule.
Average OSCR (%) over CUB, Aircraft and Cars and four post-hoc scores; mean over 3 seeds. Best in bold, second best underlined.
| Objective | Split | LS | ALS [4] | FedLAS [10] | Ours |
|---|---|---|---|---|---|
| SupCon | Easy | 71.04 | 70.75 | 72.03 | |
| SupCon | Hard | 65.27 | 65.78 | 66.32 | |
| SpHOR-Lite | Easy | 71.19 | 71.74 | 72.91 | |
| SpHOR-Lite | Hard | 65.70 | 65.40 | 67.18 |
(3) Extending it to Mixup. (Refer CW 1(b))
[4] Lee, D., et al, Adaptive label smoothing with self-knowledge in natural language generation. EMNLP'22.
[9] Bae, J., et al, CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning. ICCV'25.
[10] Bahavan, T. T. N., et al, FedLAS: Feature-Modulated Bidirectional Label Smoothing for Neural Network Calibration. ECCV'26.
[11] Scheirer, W. J., et al, Toward open set recognition. TPAMI'13.
CW4: Empirical results.
We revised the wording of these claims so that each one precisely reflects the empirical results. The changes are in dark blue in the revised manuscript (e.g. Table 2 Caption, Lines 500-502).
(continued in Part 4/5)
Official Response to Reviewer 4ohG (Part 1/5)
Overall Response: We thank the reviewer for their comments and questions. We grouped the critical weaknesses and questions by topic and answered each group below. We have also revised the paper. All edits are in dark blue, and we refer to them by line number.
CW1(a): Paper is not self-contained.
We agree with this comment. We have revised the paper and made it more self-contained. We clarified that the paper uses the Supervised Contrastive Learning framework (with other methods such as SpHOR/ConOSR) (Lines 183-189) and provided a generalized equation for it. (“A common template”, Lines 244-264).
CW1(b): Incremental investigation and lacks methodological contribution.
We respectfully disagree that our work is an incremental investigation of the label smoothing used in SpHOR. ConOSR/SpHOR use label smoothing as a fixed training choice; they do not ask how it acts on a representation objective, or how it should vary across samples. Our work answers both questions, and the answers are not specific to SpHOR. We clarify this below, together with the differences from [4].
(1) Not specific to SpHOR. Our proposed method integrates into four objectives, SupCON, ConOSR, SpHOR, and SpHOR-Lite (SpHOR without Mixup), with no change to the architecture or the objective and at negligible cost (Table 7), and it improves their OSR performance in most settings (Table 2).
(2) A design principle. An adaptive smoothing method is a control mechanism: it uses an input signal to vary the smoothing coefficient per sample, and the coefficient in turn controls a property of the learned model. To design it, one must know what the coefficient controls, and which input should set it. Our proposed method contributes along both axes. Most adaptive label smoothing methods are designed for classifiers [4, 9, 10]. In a classifier, the label is the target distribution that the predicted class probabilities must match, and smoothing acts as entropy regularization of the output, which mitigates overconfidence [7]. In a representation objective, the label plays a different role: it decides which samples are positives and which are negatives for each anchor (Eq. 4). Building on the alignment–uniformity decomposition, which we unify across these objectives (Prop. 1), we show that label smoothing acts as an alignment dial (Prop. 2; details in our response to CW 2). We therefore propose this alignment view of label smoothing as a design principle for adaptive label smoothing in representation learning.
(3) A signal designed for OSR. Once we know that smoothing controls alignment, the input must answer: which samples need more alignment? Adaptive smoothing methods answer such questions with a task-specific signal that encodes the inductive bias of their task (See CW 3 answer). The inductive bias of OSR is geometric: known classes must form compact, well-separated clusters, so that the remaining space is reserved for unknowns [11]. We therefore propose prominence, a signal that adjusts the alignment of each sample to produce this geometry. To our knowledge, it is the first adaptive label smoothing method designed for OSR. We explain the design and its differences from [4] in our response to CW 3.
(4) Mixup. A Mixup sample has two classes, so a per-sample coefficient is not defined for it. To our knowledge, no prior adaptive label smoothing method defines one; ours is the first. We compute the prominence of the mixed sample as the Mixup-weighted combination of the prominence of its two constituent samples, and apply the resulting single coefficient once to the mixed target (Eqs. 10–11). In our ablation, we show this form of blending outperforms blending the two smoothing coefficients directly in 11 of 12 settings (App. D.3, Table 12).
[4] Lee, D., et al, Adaptive label smoothing with self-knowledge in natural language generation. EMNLP'22.
[7] Pereyra, G., et al, Regularizing neural networks by penalizing confident output distributions. ICLRW'17.
[9] Bae, J., et al, CaliMatch: Adaptive calibration for improving safe semi-supervised learning. ICCV'25.
[10] Bahavan, T. T. N., et al, FedLAS: Feature-modulated bidirectional label smoothing for neural network calibration. ECCV'26.
[11] Scheirer, W. J., et al, Toward open set recognition. TPAMI'13.
(continued in Part 2/5)