acceptodds
Withdrawn from ICLR 2027

Text as a Witness: Assessing Visual Evidence for Test-Time Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Textual prototypes provide class semantics, while test images reveal appearance in the deployment environment. Test-time adaptation must decide how strongly visual estimates built from inferred assignments should refine a classifier anchored by text. We introduce Witness-Assessed Visual Evidence (WAVE), which organizes this exchange around text as a reference for visual evidence and visual estimates as proposals for adapting class representations. Individual Evidence Assessment (IEA) uses the text class set to assess the response patterns of individual image–class assignments. Collective Evidence Assessment (CEA) assesses whether their aggregate favors its assigned class over its strongest competitor. The two scores jointly determine a proposal's influence, while the original soft assignments determine its visual statistics. Updated class statistics then revise assignments within the batch, closing the feedback loop around fixed text prototypes. WAVE supplies evidence assessment to a statistical inference procedure and operates on frozen embeddings without target labels or encoder backpropagation. Our analysis characterizes the residual geometry underlying IEA and connects proposal influence to regularization and textual decision regions. Across 14 datasets and 11 settings, combining both assessments improves mean accuracy over the stronger individual component by 2.77% on class-sequential streams and 2.15% in small batches sampled from one to four candidate classes. Controls reveal substantial benefits from overall attenuation, with additional gains from class-specific assessment in evaluated ImageNet settings. These findings support regulating the contribution of visual proposals while exposing the limitations of the textual reference.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
transfer learning, domain adaptation, domain generalization, vision language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_v9EcOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how strongly visual estimates built from inferred (pseudo-label) assignments should refine a text-anchored classifier during test-time adaptation of CLIP. The proposed method, WAVE, gates each class's visual contribution through two text-anchored assessments, individual evidence assessment (IEA), which scores an image–class assignment by the entropy of its projected class-similarity profile residual against the candidate's text profile, and collective evidence assessment (CEA), which checks whether the assignment-weighted visual centroid still favors its proposed class over competing text classes. The two scores jointly determine a class-specific fusion weight between the text prototype and the visual estimate, and updated statistics feed back into assignments within the batch.

Across 14 datasets and 11 test-distribution settings, WAVE consistently improves over its base solver and alternative configurations of the assessments.

Critical strengths

1. The paper clearly poses the question of how much influence visual evidence should receive and designs a coherent mechanism to answer it. As the authors frame it, assigned mass to a visual proposal measures how much data supports it numerically, but does not establish the semantic basis for using it to revise a class representation, and WAVE answers this with two complementary checks on the individual assignments and their aggregate. The experiments show consistent gains over StatA-hard, the single assessments, and a rerun ADAPT in nearly all evaluated settings.

Critical weaknesses

1. The two assessments can only see what the text can express, but the information adaptation needs most lies outside the text. Both scores are computed from similarity to the frozen text prototypes, so visual information that the text cannot describe never enters the assessment. This is a problem because test-time adaptation exists precisely to add what the text lacks, such as target-domain appearance and visual distinctions the text prototypes do not capture.

Questions for authors
  1. Robustness to text quality. Since the whole mechanism depends on the text prototypes, it would be useful to see results under different text quality, for example different prompt templates or weaker text encoders, to show how the assessments behave when the text reference degrades.
  2. More backbones. The evaluation uses a single CLIP backbone (ViT-B/16). Results on more backbones, for example ResNet-50 or other CLIP variants, would show whether the conclusions hold beyond one encoder.
  3. Combination with other methods. The evidence assessment is proposed as a general rule for controlling visual influence. It would be useful to see it applied on top of other TTA methods that also build visual estimates from pseudo-labels, to show whether the gains transfer beyond the solver used in the paper.
Reviewer_r81LOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a training-free TTA method called WAVE, which is built on StatA. The method uses text prototypes to assess individual image–class assignments and the visual prototypes obtained by aggregating image features. These two assessments control how much the visual information contributes to the updated class statistics. The method performs adaptation within each test batch using frozen CLIP features. The experiments cover 14 datasets and 11 different settings.

Critical strengths
  1. This study examines the reliability of visual information during the adaptation process during testing, which is of practical significance. Incorrect pseudo-labels can affect the estimation of category prototypes. Evaluating this information using text prototypes is a reasonable approach. Additionally, this method does not require training or backpropagation of CLIP.
  2. The experiments cover 14 datasets and 11 settings, including different batch sizes and category distributions. The experimental scope is quite comprehensive. Moreover, the paper provides separate modules, StatA-soft, different anchor strengths, and Constant Gate as controls, which help to understand the specific role of the proposed module.
Critical weaknesses
  1. The comparison with existing methods is insufficient. Apart from StatA, which is the framework used to develop WAVE, the paper only compares with one additional TTA method, ADAPT. IEA-only, CEA-only, and Constant Gate are internal controls and cannot replace comparisons with other published methods. Why were more relevant methods not included? Although the experiments cover many datasets, I think the limited number of competing methods makes it difficult to fully assess the advantages of WAVE.
  2. The paper does not evaluate the case where batch size = 1, and the experimental setup of ADAPT also needs clarification. The "Online" setting in this paper processes 128 images each time, and WAVE adapts within the current batch and then clears the adaptation status. However, the original online ADAPT processes one image each time and retains historical samples through the knowledge base. I am concerned whether this comparison retains the original working mechanism of ADAPT. Please clarify its cache capacity, parameter settings, and whether the cache is cleared between batches. At the same time, I hope the author can supplement batch size = 1 experiments. If WAVE cannot work in this setting because it requires multiple images for neighborhood construction and collective evaluation, it should be clearly listed as a limitation of the method.
Questions for authors

For this paper, my assessment is weak reject. The paper studies a meaningful problem and provides several useful experiments for its method design. However, the comparison with other published methods is limited, and the batch-size setting and implementation of ADAPT need clarification. At the same time, the additional benefit of the proposed assessment modules needs stronger experimental support. I will also look at the opinions of other reviewers to see if I have missed anything. The specific issues can be found in “Weaknesses”.

Reviewer_Q2AYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WAVE, a TTA method that assesses the reliability of visual evidence. It introduces Individual Evidence Assessment (IEA) and Collective Evidence Assessment (CEA) to evaluate image-level and class-level visual evidence, respectively, and uses these assessments to regulate visual prototype updates within the StatA framework. Experiments across 14 datasets and 11 test settings demonstrate improvements over existing baselines.

Critical strengths
  • The paper addresses the problem of unreliable pseudo-labels contaminating visual prototypes during test-time adaptation.

  • Experiments cover diverse datasets and test-distribution settings, with ablations and diagnostic analyses examining the contributions and limitations of the proposed components.

Critical weaknesses
  • WAVE builds on the StatA framework and introduces two evidence assessment modules to regulate visual prototype updates. The methodological contribution appears incremental.

  • IEA's residual entropy performs poorly in distinguishing correct and incorrect predictions on some datasets, raising concerns about its effectiveness as a visual evidence reliability measure.

  • The improvements may largely result from reducing visual influence overall, rather than the proposed class-specific evidence assessment.

  • The adaptation state is reset after each batch, so the experiments do not establish whether WAVE can handle continuous distribution shifts while retaining information across batches.

  • End-to-end latency, peak memory consumption, and the overhead of the proposed modules should be reported to better establish its practical efficiency.

  • The paper does not discuss several relevant recent methods, including PTA [1] and L-TTA [2].

[1] Prototype-Based Test-Time Adaptation of Vision-Language Models. ICML 2026

[2] Long-tailed Test-Time Adaptation for Vision-Language Models. ICLR 2026

Questions for authors

Please see the concerns raised in the Weaknesses section. Addressing these points would help me better assess the work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.