Text as a Witness: Assessing Visual Evidence for Test-Time Adaptation

Textual prototypes provide class semantics, while test images reveal appearance in the deployment environment. Test-time adaptation must decide how strongly visual estimates built from inferred assignments should refine a classifier anchored by text. We introduce Witness-Assessed Visual Evidence (WAVE), which organizes this exchange around text as a reference for visual evidence and visual estimates as proposals for adapting class representations. Individual Evidence Assessment (IEA) uses the text class set to assess the response patterns of individual image–class assignments. Collective Evidence Assessment (CEA) assesses whether their aggregate favors its assigned class over its strongest competitor. The two scores jointly determine a proposal's influence, while the original soft assignments determine its visual statistics. Updated class statistics then revise assignments within the batch, closing the feedback loop around fixed text prototypes. WAVE supplies evidence assessment to a statistical inference procedure and operates on frozen embeddings without target labels or encoder backpropagation. Our analysis characterizes the residual geometry underlying IEA and connects proposal influence to regularization and textual decision regions. Across 14 datasets and 11 settings, combining both assessments improves mean accuracy over the stronger individual component by 2.77% on class-sequential streams and 2.15% in small batches sampled from one to four candidate classes. Controls reveal substantial benefits from overall attenuation, with additional gains from class-specific assessment in evaluated ImageNet settings. These findings support regulating the contribution of visual proposals while exposing the limitations of the textual reference.