- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies unsupervised adaptation of vision-language models when the unlabeled training data are long-tailed. MARS first uses frozen zero-shot visual anchors to correct unsupported changes in pseudo-label probabilities, then refines a student with a mixture of teacher distillation and hard pseudo-label supervision whose weight depends on estimated class frequency and prediction margins. The evaluation covers nine benchmarks at several imbalance ratios, with additional backbone, head/tail, and larger-scale analyses.
Critical strengths
The paper highlights a useful failure mode: adapting to unlabeled long-tailed data can hurt well-recognized head classes when the adaptation procedure pushes predictions toward a uniform class distribution. I like that MARS tries to preserve useful zero-shot structure while still leaving room to correct it, rather than simply balancing pseudo-label counts. The reported gains across nine datasets and several imbalance ratios, together with the head/tail analyses, make this a promising direction for practical VLM adaptation. The component ablations also suggest that both stages contribute to the overall result.
Critical weaknesses
-
Could you help me pin down which BPA training path produced the reported results? Eq. (5) and Algorithm 1 use weak-view targets to supervise strong views, but the submitted minimal code returns the weak view first and then uses that first view in train_bpa, discarding the strong one. This may just be a difference in the minimal release, so I'm not assuming the tables used that path. Once that is clear, could we compare corrected and uncorrected targets in one existing setting with the same model, views, and training budget? Table 4 shows that the stage helps; this would tell us what the target correction itself adds.
-
I'm wondering what the MSR gate adds beyond making the teacher's predictions sharper. With the gate fixed, Eq. (10) amounts to learning from a mixture of the teacher's soft prediction and its own top-1 label, so the target still favors the same class even when that label is wrong. Could we keep the BPA teacher and student setup fixed, compare the adaptive gate with a sample-independent gate having the same average weight each epoch, and look at what happens to incorrect teacher labels in rare predicted classes? That seems like a fairly direct way to connect the gains to the proposed handling of noisy supervision.
-
I got a bit stuck on how the final student handles a new image. In the minimal code, MSR trains and evaluates on cached BPA image features, but the returned student's raw-image path rebuilds visual prompts from text prompts that MSR has updated; freezing the projection weights alone doesn't freeze those visual prompts. Is the intended model the frozen BPA image branch paired with the final student text embeddings? Making that path explicit, and checking that it reproduces the cached-feature logits on one small batch, would help connect the reported evaluation to the new-image prediction in Eq. (11).
Questions for authors
Could you walk me through the three points above: the BPA view path and matched target-correction comparison; what the adaptive MSR gate adds under a matched average weight; and the frozen visual branch used for new-image inference? These are the clarifications that would most affect my current assessment.
2 comments
Dear Reviewer j2Lk,
Thanks for your time and effort in reviewing this paper. As the author-reviewer discussion period ends in about 24 hours, we would be grateful if you could take a look at our rebuttal and let us know whether your concerns have been addressed. We are happy to clarify any remaining questions before the deadline.
Regards from the authors.
Dear Reviewer j2Lk,
Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author-reviewer discussion period ends in 48 hours, we kindly request that you review the rebuttal and confirm whether your concerns have been addressed. Thanks a lot.
Regards from the authors.