- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces a method, GeoRel for online test time adaptation for CLIP but with batch size set to one. GeoRel splits adaptation into two parts - (1) the geometry that compares class evidence, and (2) the reliability of augmented views. On top of ReTA, it adds multiple components such as:
- a shared diagonal-plus-low-rank metric
- trained with attraction term
- margin and redundancy losses
- EMA visual prototypes
- agreement-sensitive reliability score
- aggregation rule
- a two-branch fusion
GeoRel gains about 1 pp over ReTA on the TPT benchmark suite.
Critical strengths
- The two-role framing, which separates comparison geometry from view reliability, offers an interesting perspective on VLM-TTA.
- The authors justify most design choices. They compare shared and modality-specific transforms at equal parameter counts, which shows that the benefit of sharing does not come from added capacity.
- Gains hold across both backbones and both benchmark families.
Critical weaknesses
- GeoRel adds a learned metric with three losses, EMA prototypes, a rank-based view score and two-branch fusion to ReTA. Eq. (2) is proposed at the principle as it separates two reliability roles: M_t determines where class evidence is compared by weighting feature directions, while {γ_i} determine which observations dominate inference. It remains unclear how much of the improvement the principle itself accounts for.
- Some components do not follow directly from the two-role principle. The paper describes EMA as added stabilization mechanism and branch fusion as an operational design choice. Each choice is reasonable engineering, but together they make the link between the principle and the full method less direct.
- Since TTA offers no target validation set, it becomes an important point to understand how to choose hyperparameters and how many hyperparameters to choose to judge the method's effectiveness. With a mean gain near 1 pp on a heavily reused benchmark suite, this protocol determines whether the gain reflects the method or the tuning.
- The paper reports no latency or memory. When comparing with a method like ZERO which reaches competitive accuracy without optimization, one can not omit this study.
Questions for authors
- What does the minimal method (frozen CLIP, learned W, anchored rank weighting) achieve against CLIP, TDA and ReTA?
- Please report a cumulative table (base, +geometry, +reliability, +EMA, +fusion) on Final accuracy for all datasets.
- How do the authors select the new hyperparameters? Do they stay fixed across datasets and backbones?
- Please share also the standard deviation for all methods alongside the reported mean as the gains are in a marginal bracket and are important in this setting.
- What are the per-image latency and memory against ReTA, TDA and ZERO?