- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies TTA under unknown class priors, including class imbalance, class absence, and open-set samples. It proposes an evaluation protocol for these prior shifts and RoAR, a test-time refinement method that uses prototype information while anchoring to the original zero-shot predictions.
Critical strengths
- The paper addresses a meaningful and practically relevant limitation of existing TTA benchmarks. The explicit consideration of class imbalance, class absence, and open-set contamination provides a useful framework for studying how test-time adaptation behaves when the test-pool prior is unknown, rather than assuming that all candidate classes are adequately represented in the pool.
Critical weaknesses
-
The paper is hard to follow. Although the overall idea of using a zero-shot prediction as an anchor and refining it with test-pool-derived prototypes is interesting, the method is presented in a highly technical and fragmented manner. In particular, the roles of the anchor, the two prototype views, the prototype exponent, geometric scores, and the fallback whitened view are not clearly motivated before being introduced. A more intuitive overview or step-by-step description of the refinement process would make the method much easier to understand and evaluate.
-
The novelty of RoAR relative to prior open-set TTA is unclear. The related-work section mainly mentions AdaND and does not adequately cover existing OSTTA methods that identify unknown/OOD samples and prevent their contamination of adaptation. The authors should clarify whether RoAR's main contribution is handling unknown priors more broadly, including class imbalance and class absence, and explain why existing OSTTA methods are insufficient under this setting.
The baseline selection also underrepresents open-set methods. Most baselines (e.g., StatA, TransCLIP, GDA, COSMIC, and SOTA) are general VLM TTA or refinement methods, while only ZS-NTTA appears to directly address noisy/out-of-label samples. Given the central role of unknown-sample screening in RoAR, comparisons with more existing open-set TTA and test-time OOD filtering methods would strengthen the evaluation.
[1]Gao, Zhengqing, Xu-Yao Zhang, and Cheng-Lin Liu. "Unified entropy optimization for open-set test-time adaptation." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
[2]Sreenivas, Manogna, and Soma Biswas. "Efficient open set single image test time adaptation of vision language models." arXiv preprint arXiv:2406.00481 (2024).
[3]Luo, Wei, et al. "ProtoDCS: Towards Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models." arXiv preprint arXiv:2602.23653 (2026).
[4]Feng, Mingxu, Fengqiang Wan, and Yang Yang. "Open-Set Test-Time Adaptation in VLMs via Noise-Immune Self-Purification.”
Questions for authors
1.The paper positions open-set contamination as one of the three central unknown-prior settings. How does RoAR’s screening-and-prototype construction differ fundamentally from existing open-set TTA methods that already identify and filter unknown samples before adaptation? In particular, which component of RoAR should be considered the main methodological novelty beyond combining existing negative-label and visual-clustering ideas?
2.Since open-set contamination is a central setting in the proposed protocol, could the authors clarify why the evaluation does not include more open-set TTA or test-time OOD filtering methods beyond ZS-NTTA? How should RoAR’s reported gains be interpreted relative to those methods?
5 comments
Table a. Known accuracy / H-score by unknown proportion (%)
| Method | 0% (clean) | 25% | 50% | 75% |
|---|---|---|---|---|
| ROSITA | 38.18 | 55.61 / 68.15 | 57.33 / 69.09 | 59.17 / 69.36 |
| UniEnt | 37.38 | 55.25 / 67.38 | 56.92 / 68.17 | 58.68 / 68.49 |
| ZS-NTTA | 57.22 | 56.93 / 54.07 | 56.90 / 60.46 | 57.84 / 63.52 |
| DDE | 48.61 | 51.73 / 60.13 | 52.89 / 61.24 | 54.13 / 62.35 |
| OpenXpert | 39.93 | 47.25 / 60.38 | 58.56 / 67.98 | 60.97 / 68.04 |
| NISP† | 57.94 | 57.55 / 60.18 | 58.65 / 58.61 | 59.75 / 51.74 |
| RoAR | 76.53 | 75.73 / 56.48 | 75.44 / 64.31 | 75.09 / 66.53 |
The H-score is undefined on clean pools because they contain no unknowns.
Table b. Known false rejection rate by unknown proportion (%, ↓)
| Method | 0% (clean) | Per-dataset max at 0% | 25% | 50% | 75% |
|---|---|---|---|---|---|
| ROSITA | 47.20 | 56.91 (DTD) | 15.59 | 12.67 | 9.17 |
| UniEnt | 48.81 | 55.61 (DTD) | 16.29 | 13.43 | 9.93 |
| ZS-NTTA | 23.19 | 85.99 (Aircraft) | 23.38 | 22.43 | 19.34 |
| DDE | 29.53 | 91.43 (EuroSAT) | 24.62 | 22.71 | 19.74 |
| OpenXpert | 50.77 | 61.47 (DTD) | 39.67 | 19.58 | 12.45 |
| NISP† | 21.42 | 72.94 (Aircraft) | 19.97 | 17.37 | 14.70 |
| RoAR | 0.00 | 0.00 | 0.04 | 0.39 | 0.48 |
The compared methods reject 21–51% of the known images on clean pools, and their false rejection rate generally increases as the unknown proportion decreases. ROSITA and UniEnt reach high AUROC on the Places mixture pools (94.9 and 93.4) but still reject many known images on clean pools. A score's ability to separate known from unknown images therefore has to be evaluated together with how well known images are preserved at the actual rejection threshold. RoAR keeps the known false rejection rate at or below 0.5% at every evaluated proportion.
To check the effect of the unknown source, we replaced Places with iNaturalist, a semantically distant unknown source. We mixed all 10,000 iNaturalist OOD images into the full known test split of each dataset and kept the other settings unchanged.
Table c. iNaturalist unknowns (%)
| Method | Known accuracy ↑ | H-score ↑ | Known false rejection ↓ |
|---|---|---|---|
| ROSITA | 55.43 | 66.29 | 17.31 |
| UniEnt | 55.38 | 66.75 | 17.29 |
| ZS-NTTA | 56.56 | 63.12 | 21.72 |
| DDE | 59.60 | 70.66 | 11.74 |
| OpenXpert | 58.39 | 66.30 | 18.34 |
| NISP† | 57.37 | 56.80 | 20.47 |
| RoAR | 74.69 | 78.48 | 0.94 |
With iNaturalist unknowns and the same configuration, RoAR keeps the known false rejection rate below 1% and obtains the highest known accuracy and H-score among all compared methods.
In every cell a rejected known image counts as an error. † NISP uses our implementation that restores the parts missing from the released code. We will add the works suggested by the reviewer to the Related Work section.
Thank you again for your review. We would appreciate any feedback on whether our responses have addressed your concerns. If anything remains unclear or insufficient, we would be glad to provide further clarification or additional results.
Weakness 1
We thank the reviewer for this comment. We will reorganize Section 3 into three steps and state more clearly what each step does and why it is needed.
Weakness 2
Yes. The main contribution of RoAR is handling unknown priors broadly rather than open-set rejection. RoAR handles class imbalance, class absence and unknown contamination with one configuration and without information about the prior (Tables 1–3 of the paper, and Table G.4 when all three occur together). Existing OSTTA methods set their rejection criterion assuming that unknowns are present, so they reject 21–51% of the known images even on pools without unknowns (Q2, Table b). Pools with class imbalance or absent classes also contain no unknowns, and OSTTA methods have no mechanism for handling frequency changes among the known classes. On the ρ=1000 and 50% absent pools the compared OSTTA methods reject 29–52% of the known images, while RoAR rejects none (0.00%).
The submission did not sufficiently cover closely related OSTTA work such as UniEnt, ROSITA, ProtoDCS and NISP. We will add these works to the Related Work section and have added comparisons with the methods that have runnable implementations (Q2). Q1 summarizes the differences and Q2 reports the comparison results.
Question 1
The core methodological contribution of RoAR is not the screening but the refinement rule that combines a fixed VLM anchor with prototype evidence from an auxiliary visual representation. Negative-label scoring and clustering are existing components. RoAR's contribution lies in how they are combined and in the refinement rule built on top of them. Existing OSTTA methods keep updating the model after filtering unknowns, so unknowns that pass the filter and incorrect pseudo-labels can affect later predictions through these updates. RoAR does not update the encoder. It constructs the VLM anchor once per pool and keeps it fixed during refinement. The reference with which the prototype predictions are combined therefore stays the same even as the prototype estimates change. With a single configuration, RoAR maintained known accuracy across the evaluated conditions (presence and proportion of unknowns, class imbalance and class absence).
RoAR differs from existing OSTTA methods in two ways.
Question 2
The OSTTA comparison in the submission was insufficient. In addition to ZS-NTTA from the submission, we evaluated ROSITA, UniEnt, DDE, OpenXpert and NISP. We could not compare with ProtoDCS because no runnable public code is available. We mixed Places images into the full known test splits of ImageNet and the ten cross-domain datasets. Each method used its released implementation and its own input configuration.
At deployment it is not known whether unknowns are present at all, so we also evaluated clean pools without unknowns (0%) with the same configuration. Existing OSTTA methods reject a substantial share of the known images even when no unknowns are present.