- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper theoretically and empirically investigates the effects of a shared temperature on classification scores and learning dynamics in ProCo, a probabilistic contrastive learning method based on the vMF distribution. Specifically, through a high-dimensional analysis of the vMF score, the authors demonstrate that a shared temperature does not necessarily imply a shared angular scale, as the effective angular gain depends on the class-specific mean resultant length . The paper further analyzes how this gain affects decision boundaries and feature gradients, and compares several approaches to equalizing the gain across classes. Experiments on multiple long-tailed image classification datasets are conducted to validate the theoretical predictions and examine their effects on representation learning.
Critical strengths
S1: Presentation. This paper is well organized and clearly written, with a logical progression from problem formulation to theoretical analysis and empirical validation.
S2: Theory. The theoretical analysis is carefully developed and sufficiently detailed. In particular, the uniform approximation guarantees for both the score and its derivative in high-dimensional settings are valuable.
S3: Focus. The distinction between a shared temperature and class-dependent effective angular gains provides an interesting perspective. The experimental design, which separates the different effects induced by temperature modifications, is also well motivated.
Critical weaknesses
W1: The novelty and significance of the theoretical contribution are somewhat limited. The central observation that the effective angular gain takes the form can be derived relatively directly from the expected feature representation defined in the original ProCo formulation. Although the rigorous high-dimensional approximation guarantees are valuable, it remains unclear to what extent the main finding provides fundamentally new insights beyond those already implied by the existing formulation.
W2: The motivation and benefits of angular gain equalization are not sufficiently established. While the paper demonstrates that equalizing the gains affects decision boundaries and gradients, it does not provide a theoretical justification for why such equalization should improve generalization. Moreover, the empirical benefits depend on experimental conditions and the treatment of class priors. Consequently, it remains unclear under what conditions gain equalization is desirable or beneficial.
W3: The practical significance and generalizability of the proposed approach remain limited. The main representation-learning experiments focus on the probabilistic contrastive objective of ProCo, and the observed performance improvements are relatively modest. Furthermore, complete gain equalization brings the leading score geometry close to that of a conventional cosine-prototype classifier. Therefore, controlled comparisons with simpler cosine-based learning approaches are important for establishing the added value of the proposed intervention. The current results do not sufficiently clarify what practical advantages the proposed approach offers over such simpler alternatives.
Questions for authors
Q1: Novelty of the theoretical contribution: The effective angular gain appears to follow relatively directly from the expected feature representation in the original ProCo formulation. Could the authors clarify what fundamentally new insights are provided by the high-dimensional analysis that cannot be obtained from the existing formulation or a straightforward first-order approximation?
Q2: Motivation for angular gain equalization: While gain equalization improves performance under certain experimental conditions, it also leads to substantial degradation depending on the treatment of class priors. Under what conditions do the authors expect gain equalization to be beneficial? Furthermore, is equalization intended as a practical approach to improving generalization, or primarily as an analytical intervention to understand the role of angular gains?