acceptodds
Withdrawn from ICLR 2027

Are We Measuring the Right Angles? Spherical Test-Time Adaptation of Vision-Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

18 views

Abstract

Test-time adaptation (TTA) for vision–language models (VLMs) aims to improve robustness under distribution shifts by exploiting unlabeled test data during inference. Existing CLIP-TTA methods either optimize prompts through backpropagation or estimate Euclidean statistics from normalized CLIP features, despite CLIP performing classification through cosine similarity on the unit hypersphere. We propose STRA, a Spherical Test-Time Robust Adaptation framework for closed-form, backpropagation-free online CLIP adaptation. Our central view is that closed-form CLIP-TTA should recover a target-domain angular coordinate system under a pullback-vMF model, rather than fit an ambient Euclidean classifier. STRA instantiates this view through label-free geometry calibration from streaming second-order statistics and text-anchored spherical prototype updates driven by sparse soft pseudo-labels. This design avoids raw-feature caching and the selection bias induced by confidence-filtered geometry, decouples marginal geometry estimation from semantic pseudo-label updates, and couples them only through a shared bilateral angular map, while maintaining a single fixed configuration across datasets and backbones. Extensive experiments across natural distribution shifts, fine-grained recognition, and corruption robustness demonstrate consistent improvements over recent CLIP-TTA baselines with favorable computational efficiency. The code is available at https://anonymous.4open.science/r/STRA. baselines at favorable computational cost.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Test-Time Adaptation, Vision-Language Models, CLIP, Hyperspherical Geometry, Closed-Form Estimation, Distribution Shift

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vUAbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

STRA adapts frozen CLIP models online without backpropagation or raw-feature caching. It estimates the stream’s uncentered second moment, applies a regularized inverse-square-root map to image features and text-anchored class prototypes, and normalizes both before scoring. Sparse pseudo-labels from frozen CLIP update the prototypes. A pullback-vMF model interprets the classifier. Evaluation covers natural shifts, ten recognition datasets, and fifteen corruption types using RN50 and ViT-B/16.

Critical strengths
  • Geometric alignment and consistent gains: Normalizing transformed features and prototypes preserves a cosine-based readout. With ViT-B/16, STRA achieves 67.70%, 71.43%, and 30.43% across the three benchmark groups, exceeding the strongest listed baseline by 1.17, 1.09, and 1.87 percentage points. RN50 results support transfer across backbones.

  • Useful ablations and practical efficiency: Removing either the geometry map or prototype updates reduces accuracy. Table 24 directly compares confidence-selected and all-feature geometry estimation. Reported latency is lower than SCA and ADAPT across all three groups, with memory close to SCA and independent of stream length.

Critical weaknesses
  • Novelty needs clearer positioning: SCA’s official implementation already estimates geometry from all features independently of pseudo-label weights and uses the same cumulative-probability truncation rule. The paper should distinguish its bilateral normalization and prototype treatment from these existing components.

  • Retrofit comparisons are underspecified: Table 5 does not clarify whether spherical re-projection transforms native class prototypes or fitted classifier weights. Transforming already fitted weights can introduce an additional metric factor. Without explicit scoring formulas, the degraded retrofit results do not isolate the benefit of STRA’s estimators and readout.

Questions for authors
  • Which statistical updates are new relative to SCA? How much improvement remains when prototype blending, decay, truncation, and regularization are matched?

  • What exact scoring formulas define the Table 5 retrofits, and which class vectors are transformed?

  • How does STRA handle repeated abrupt domain changes? Does it discount or remove obsolete statistics?

  • Do you have a concrete low-rank or structured implementation, with initial results, addressing the (\mathcal{O}(d^3)) matrix decomposition cost?

  • Please reconcile two reporting discrepancies: Table 4’s unnormalized variant exceeds the frozen baseline at 66.88% versus 59.11% and 27.05% versus 25.50%, contrary to Section 4.3’s description. ADAPT’s Cars accuracy is also 69.08% in Table 2 but 68.19% in Table 26.

Reviewer_ZmJrOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Training-free TTA method called STRA. In this paper, all the input images are used to estimate the feature geometry, and then the category prototype is updated through soft tail labels. Afterwards, the image features and the obtained category prototype are transformed and normalized for classification, and entropy-based gated fusion is used to combine the predictions before and after adaptation. The experiments include two CLIP backbones and many existing methods.

Critical strengths
  1. This method is a training-free approach. It does not require training or backpropagation of the clip. This significantly enhances the efficiency of the method. At the same time, it has been compared with SCA and ADAPT in terms of accuracy, latency, and memory. This comprehensive comparison fully demonstrates the comprehensiveness of the method.
  2. This experiment involves two types of clip backbones and three benchmarks. The scope of the experiment is quite comprehensive. Moreover, its ablation study clearly demonstrates the specific effects of different modules as well as the effects of their combinations.
Critical weaknesses
  1. This paper omits some previously published methods, such as PTA (ICML 2026) and BoostAdapter (NeurIPS 2024). PTA, in particular, is closely related to efficient prototype adaptation, and it also includes results of enhancing text information using GPT. Why weren't these methods compared? The absence of these comparisons prevents a full assessment of the accuracy and efficiency advantages of STRA as a relatively important alternative solution.
  2. This paper does not provide a clear explanation regarding the impact of additional textual descriptions. In this paper, the prototype of the text is constructed using additional textual information, but methods such as Boost Adapter only use this fixed template. After removing these descriptions, I am very concerned about whether STRA can still maintain strong competitiveness. Therefore, I hope to supplement the experimental results using the same fixed textual descriptions. At the same time, mark in your table which methods have additional descriptions, and then inform the source and quantity of these additional descriptions. I believe this will make your article's presentation clearer.
Questions for authors

For this paper, my assessment is weak reject. This is because it provides effective evidence for the design of his method. However, the experiment missed some baseline studies that have already been published. Additionally, the specific use and evaluation of the text information in this paper also need clarification. At the same time, I will also look at the opinions of other reviewers to see if I have missed anything. The specific issues can be found in "Weaknesses".

Reviewer_wHE6OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Spherical Test-Time Robust Adaptation (STRA), a training-free online adaptation method for CLIP-based vision-language models. STRA estimates target-domain feature geometry using streaming second-order statistics and updates class prototypes using text-anchored pseudo-labels. A shared spherical transformation is applied to image features and class prototypes for classification. Experiments on natural distribution shifts, fine-grained classification, and corruption benchmarks demonstrate improvements over existing CLIP test-time adaptation methods.

Critical strengths

The paper considers the spherical geometry of normalized CLIP features and provides a theoretical interpretation of the proposed adaptation strategy.

Critical weaknesses
  1. Limited novelty of individual components. The method combines covariance-based feature calibration, pseudo-label-based prototype updates, and cosine similarity. Although the spherical formulation is interesting, the methodological differences from existing whitening-based and closed-form adaptation methods could be better established.
  2. Limited gains from the complete formulation. Table 4 shows that removing either the prototype update or geometry map still achieves competitive results. The improvements from combining these components are relatively modest, making their individual contributions and interactions less convincing.
Questions for authors
  1. How does STRA compare with directly applying conventional feature whitening or covariance normalization to both CLIP image and text embeddings?
  2. Table 4 shows that removing the prototype update or geometry map results in relatively small performance differences. What is the main advantage of combining these components rather than using either independently?
  3. Since STRA estimates target-domain statistics from an online stream, how does its performance change under abrupt domain shifts or highly imbalanced class distributions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.