- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the problem of open-set test-time adapatation of vision-language models. Through gradient cosine similarity analysis, the author discovered that the gradient directions for the two objectives gradually cease to be highly aligned during the adaptation process. Consequently, the author proposes DSTA, which employs two separate sets of low-rank Adapters to decouple the optimization parameters for the two objectives.
Critical strengths
-
This paper is well organized and clearly written.
-
The technical details are also easy to follow.
-
The experiments are extensive and helpful to validate the effectiveness of the model.
Critical weaknesses
-
As shown in Table 2, the improvements against baselines seem to be mirrror.
-
The pseudo-labeling for OOD samples may be subject to confirmation bias. Please further investigate the influence.
-
The stability argument for the margin-based objective is not sufficiently substantiated.
-
The ratio between ID and OOD samples is fixed, which is overly idealistic, as the actual ratio is more complex.
Questions for authors
Please see the above weaknesses.
6 comments
Response (part 2)
2. Confirmation Bias in OOD Pseudo-Labeling.
Potential confirmation bias is a general challenge in learning from unlabeled data, and not a limitation specific to DSTA. Without ground-truth labels, existing methods necessarily rely on estimated ID/OOD membership or confidence-based sample selection, and therefore remain susceptible to confirmation bias arising from incorrect assignments. This includes the existing open-set TTA methods evaluated as baselines in our work: OSTTA (Lee et al., 2023), SoTTA (Gong et al., 2023), STAMP (Yu et al., 2024), UniEnt (Gao et al., 2024), and CLIPTTA (Lafon et al., 2025).
As in prior open-set TTA methods, addressing confirmation bias is beyond DSTA's methodological focus. Addressing this broader challenge constitutes a separate research direction and falls outside the scope of the present work.
Response (part 3)
3. Theoretical and Empirical Evidence for the Stability of the Margin-Based Objective.
We would like to clarify that both analytical and empirical evidence supporting the stability of the margin-based objective are already in the submitted paper, specifically in Supplementary Sec. A.1 (Eqs. 4–6, Figs. 3–4) and Tab. 3 of the main paper.
Analytically, CLIPTTA's original OCE objective has a gradient magnitude (), which grows with the separation gap and continuously encourages further separation. In contrast, our margin objective has a bounded gradient magnitude that vanishes once (), preventing excessive optimization beyond the desired separation margin (Fig. 3).
Empirically, Fig. 4 of the main paper shows that OCE leads to an unstable separation gap and AUC degradation to 55.7%, whereas the margin objective maintains stable separation and AUC of 99.8%. Tab. 3 further confirms this improvement, with AUC increasing from 55.7% to 99.5% and FPR95 decreasing from 94.2% to 1.4% under identical decoupled LN adaptation.
Response (part 1)
We appreciate the reviewer's positive comments on our work, particularly for highlighting the clarity of the presentation, the accessibility of the technical details, and the experimental evaluation supporting the effectiveness of DSTA. We address the reviewer's concerns point by point below.
1. Performance Improvements over Existing Baselines.
The results in Tab. 2 demonstrate substantial improvements over existing baselines.
On clean benchmarks, DSTA achieves the best aggregate results across all four metrics, reducing FPR95 from 35.3% to 29.3% and improving H-score from 68.6% to 72.0% compared with UniEnt.
Contrary to the reviewer's observation, the gains are particularly substantial on individual benchmarks: compared with CLIPTTA, DSTA reduces FPR95 from 7.6% to 0.1% on CIFAR-100 and from 9.7% to 0.8% on ImageNet, while also improving classification accuracy.
Under corruptions, DSTA further improves aggregate FPR95 by 5.1 points and H-score by 4.6 points over CLIPTTA.