- Rating
- 3
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Transductive CLIP uses a batch of unlabeled test images jointly to improve their classifications. It uses relationships across the test batch such as clusters of similar images or evidence that certain classes are common, to refine predictions. The paper claims that higher accuracy in transductive CLIP can hide worse discrimination between the classes actually present. A method may improve by suppressing classes absent from the test batch of images.
The paper separates these effects using the batch's true support. It measures ordinary accuracy over the full vocabulary, and accuracy after restricting the same fitted scores to the true support. The results show that MOON’s 10.32% overall gain combines worse discrimination with a larger reduction in absent-class obstruction. Another finding is that similar final accuracies can reflect opposing behaviors. MOON gains from reducing absent-class obstruction, while Adapt improves discrimination but introduces more obstruction. Experiments are done on CUB-200, FGVC-Aircraft, Oxford-IIIT Pets, Flowers-102, and Food-101.
Critical strengths
- This paper has carefully controlled context experiments to show that a transductive method can improve overall accuracy while becoming worse at distinguishing the classes actually present.
Critical weaknesses
The paper misses some citations to related work:
-
Ren et al. Delving into the Openness of CLIP, Findings of ACL 2023 https://aclanthology.org/2023.findings-acl.610.pdf It measures accuracy on fixed target-class images while adding non-target classes to the candidate vocabulary. The authors show that competing class descriptions can substantially degrade recognition even though the underlying image–text scores remain unchanged.
-
Zhou et al. Bayesian Test-Time Adaptation for Vision-Language Models, CVPR 2025 https://arxiv.org/pdf/2503.09248v1 This overlaps with the broader claim that adaptation can improve classification through changes to class plausibility, independently of improving the image–class matching model.
Questions for authors
Could you elaborate on the unique contributions of your work compared to existing work (see below):
-
Ren et al. Delving into the Openness of CLIP, Findings of ACL 2023 https://aclanthology.org/2023.findings-acl.610.pdf It measures accuracy on fixed target-class images while adding non-target classes to the candidate vocabulary. The authors show that competing class descriptions can substantially degrade recognition even though the underlying image–text scores remain unchanged.
-
Zhou et al. Bayesian Test-Time Adaptation for Vision-Language Models, CVPR 2025 https://arxiv.org/pdf/2503.09248v1 This overlaps with the broader claim that adaptation can improve classification through changes to class plausibility, independently of improving the image–class matching model.
We thank the reviewer for the suggested references. We will add them to the related-work discussion, while clarifying that neither addresses the central research question of our paper.
1. Missing references
1a. Ren et al. (ACL 2023).
Ren et al. examine how expanding CLIP's candidate vocabulary affects recognition accuracy. Their results demonstrate that adding competing classes can hurt predictions.
However, our paper is not about whether candidate classes can hurt CLIP accuracy. We study what accounts for the accuracy gains achieved by existing transductive CLIP methods after adaptation. We quantify whether those gains arise from improved discrimination among represented classes or reduced obstruction from absent classes, using an exact decomposition of the accuracy change.
Ren et al. do not analyze this transductive adaptation problem or the resulting gain decomposition.
1b. Zhou et al. (CVPR 2025).
Zhou et al. develop a Bayesian test-time adaptation method that incorporates class-associated prior information. Its relevance is that class-related information can influence adapted predictions.
However, using class priors in an adaptation algorithm is not the same as measuring the sources of its observed accuracy gains. Our analysis is independent of the adaptation mechanism and determines how the final predictions change in terms of within-support discrimination and absent-class obstruction. Zhou et al. do not provide this evaluation.
Our contribution is distinct from 1a and 1b: we introduce an exact decomposition of transductive CLIP accuracy gains, a support-only control that isolates the contribution of the predicted class set, and controlled experiments establishing how these contributions vary across methods, semantic contexts, and support sparsity.
For example, we show that MOON's +10.32 pp accuracy gain consists of −3.09 pp in within-support discrimination and +13.41 pp in obstruction relief—behavior that neither cited study investigates.
We will include the references for completeness, but they do not diminish the novelty of the research question, diagnostic framework, or empirical findings presented in our paper.
2. Additional evidence of practical significance.
We extended our experiments to full-vocabulary support (S = C), where absent-class obstruction relief is exactly zero and all accuracy changes reflect within-support discrimination. The results are reported in the table in our response to Reviewer r9xL, Section 2. These results demonstrate that methods benefiting from sparse support may underperform frozen CLIP when all classes are represented, highlighting the decomposition's practical value for understanding deployment conditions and guiding method selection.