- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
"Complementary label learning" basically "not this class" labels stop working beyond 10-20 classes because each label almost rules out nothing, and the information gain is very little. This paper first groups similar images using some embedding model and, for each group, restricts the set of possible labels to only four label options. It then has a VLM pick out the one that doesn't fit per image. With this approach, the information gain is much more. The accuracy on CIFAR-100 rises from about 5% to 47%.
Critical strengths
- This tackles a real limit of complementary label learning by scaling it from 20 classes to 100+ classes
- Honest ablations over diff learners, encoders, vlms, prompts etc with careful theory that doesn't overclaim
Critical weaknesses
- Swapping the VLM for a random pick within the same four classes loses only 1 to 2 points. This shows that the design choice that matters more is the label set and not the VLM.
- The paper offers a very weak practical use because strong vision models already label CIFAR-scale data very well, and nothing is tested beyond 200 classes.
- From my reading, the gain mostly comes from image clustering not the labels, and obvious simple baselines are missing. There is no comparison with a strong vision model labeling the images directly or with a simple classifier trained on the same image features using a few "true" labels.
- The method isn't really tested as it would ideally be used. Instead of training on VLMs' actual answers,the authors generate the labels from each image's true class, so the results are likely better than what the method would get in practice.
Questions for authors
- What would the results be when training on VLMs actual answers?
- What would the simple baseline in which you train on a few, possibly true, labels be? That seems like the most appropirate baseline.