- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes AdaDim, a self-supervised learning framework that adaptively combines the SimCLR and VICReg losses based on the effective rank of the embedding space. Additionally, they introduce a regularization term that encourages higher mutual information between the encoder representation space and projection head embedding space . The paper's main claim is that maximizing representation dimensionality or minimizing mutual information independently is suboptimal. Instead, the authors argue that downstream performance benefits from balancing these two quantities.
Critical strengths
None
Critical weaknesses
- Presentation and clarity. The paper suffers from several presentation issues that make it difficult to follow. References and citations are inconsistently formatted throughout the manuscript, and several figures lack proper subfigure captions and labeling. Furthermore, the methodology section does not explicitly define the individual loss functions used in the proposed framework, instead referring the reader to the appendix. This makes it difficult to understand the proposed method from the main paper alone.
- Fairness and validity of experimental comparisons. The empirical evaluation raises several concerns regarding the fairness and validity of the reported comparisons. In Table 4, the authors compare AdaDim against multiple SSL frameworks using ResNet-18. However, the hyperparameter settings deviate from the original training configurations of several baselines, including the use of smaller batch sizes, which can substantially degrade the performance of evaluated baselines. More concerningly, Table 5 reports ImageNet-1K results for ReSA [1] that do not match those reported in the official ReSA manuscript. Specifically, ReSA reports top-1 accuracies of 71.9% and 73.4% for ResNet-50 trained for 100 and 200 epochs, respectively, using a batch size of 256. In contrast, the authors report 70.8% and 71.6% for the corresponding configurations. This discrepancy is substantial and is not adequately explained in the manuscript.
[1] Weng, Xi, et al. "Clustering properties of self-supervised learning.", ICML, 2025.
Questions for authors
I have no questions that could materially affect my assessment.