- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates the potential conflict between supervised and self-supervised learning objectives when jointly optimizing a shared feature extractor, particularly in open-set recognition. The authors argue that auxiliary self-supervised learning can improve feature coverage and unknown-class detection while simultaneously weakening the discriminative representations required for known-class classification. To address this issue, the paper proposes GradMix, an attribution-guided data augmentation strategy. GradMix uses LayerCAM to identify image regions emphasized by the self-supervised objective and partially replaces these regions with content from other images in the minibatch.
Critical strengths
- The proposed method is intuitive and relatively easy to integrate. Using attribution maps to redirect SSL toward alternative visual evidence is a direct idea.
- The experimental results indicate improvements over conventional augmentation strategies. For instance, the ImageNet100 linear-probing results show that GradMix improves SimCLR top-1 accuracy from 48.1% to 54.8% and MoCo v1 from 40.31% to 47.75%.
- The paper attempts to investigate the underlying representation-learning mechanism.
Critical weaknesses
- Some claims about state-of-the-art performance are inconsistent with the reported results. Table 4 states that GradMix outperforms existing methods on four of the six OSR protocols. However, based on the reported numbers, GradMix achieves the highest AUROC on only three protocols: MNIST, CIFAR+10, and CIFAR+50.
- The benchmark and evaluation paradigm are quite old-school. Please consider more modern setting beyond the classification task, which is extensively studied several years ago. Moreover, the selected classification tasks in your main exp are also quite naive!
- The interaction between patch mixing and self-supervised positive-pair semantics is insufficiently analyzed.
- The paper is low-formatting. i. Figures are quite unclear; ii. the methodology part don't have a clear overview to present your holistic achitecture; iii. The presentation is verbose in introduction.
- Heavy computation cost: GradMix requires gradient-based attribution maps, potentially from multiple convolutional layers, and additional augmented-view processing.
Questions for authors
See weaknesses.