- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Sparse Similarity-Aware Label Smoothing (SSA-LS) to improve confidence calibration. Unlike conventional label smoothing, SSA-LS assigns smoothing probability only to the top-k similar classes based on pretrained feature representations. Experiments on CIFAR-100, TinyImageNet, and ImageNet-1k show improved calibration while maintaining comparable classification accuracy.
Critical strengths
-
The paper addresses an important problem in confidence calibration and provides a simple method that can be easily applied to existing classifiers.
-
The proposed method shows consistent improvements in ECE across multiple datasets and model architectures.
-
The ablation studies on sparsity, class similarity, and feature representations provide useful insights into the proposed method.
-
The method can also be applied to existing label smoothing approaches without changing the model architecture.
Critical weaknesses
-
The methodological novelty is limited. Similarity-based label smoothing has already been studied in previous work, such as Liu and JaJa (2021) [1]. The main difference appears to be selecting only the top-k similar classes.
-
The most closely related similarity-based label smoothing methods are not included in the main experiments. Direct comparisons are needed to show the benefit of the proposed sparsification.
-
The method assigns the same soft label to all images in the same class. However, natural images often contain multiple objects, which may cause different levels of uncertainty. The current method does not directly consider this image-specific ambiguity.
-
The ImageNet-1k experiments only train a small MLP on frozen CLIP features. This limits the evidence for the effectiveness of SSA-LS in large-scale end-to-end classifier training.
[1] Liu, C., & JaJa, J. (2021). Class-Similarity Based Label Smoothing for Confidence Calibration. In Artificial Neural Networks and Machine Learning – ICANN 2021, pp. 190–201. Springer.
Questions for authors
-
What is the main difference between SSA-LS and the class-similarity-based label smoothing method proposed by Liu and JaJa (2021) [1], beyond selecting the top-k classes?
-
Can the authors compare SSA-LS with existing similarity-based label smoothing methods under the same experimental settings?
-
How does SSA-LS handle images containing multiple objects when only one class label is given? Can the authors explain how class-wise smoothing reflects the uncertainty of individual images?
-
Can the authors provide results using end-to-end training on ImageNet-1k, rather than only training an MLP on frozen CLIP features?
[1] Liu, C., & JaJa, J. (2021). Class-Similarity Based Label Smoothing for Confidence Calibration. In Artificial Neural Networks and Machine Learning – ICANN 2021, pp. 190–201. Springer.