acceptodds
Under review as a conference paper at ICLR 2027

Ordinal Compatibility for Prediction and Adaptation in Vision-Language Models

Abstract

Online test-time adaptation gives each unlabeled sample two roles: it must be classified immediately, and it can contribute to the model state used for subsequent predictions. These roles call for different uses of the same evidence. A sample may require prediction correction while warranting a conservative update to the evolving class distributions. We introduce Prediction and Adaptation via Compatibility Evidence (PACE), a framework that uses ordinal compatibility between fixed semantic knowledge and adaptive distributional estimates to guide both decisions. For each sample, we compare the class rankings induced by text prototypes and online Gaussian models, and convert their rank displacements into a shared compatibility vector. At the class level, compatibility provides a log prior that adjusts dynamic predictions. At the sample level, its posterior-weighted expectation produces a continuous gate that controls the total responsibility assigned to Gaussian updates. This design connects prediction correction and selective adaptation through one relational signal while keeping their operations distinct. PACE requires no encoder updates or backpropagation. Under dataset-specific oracle tuning on full target streams, the full method improves top-1 accuracy over its rank-free counterpart in all 21 evaluations, with an average gain of 1.09 percentage points. The ordinal-only variant achieves a 1.02-point mean gain, identifying compatibility as the main source of improvement. The gate reduces the fraction of update mass assigned to incorrect classes in five of six diagnostic domains. Matched-mass controls favor compatibility selection over uniform attenuation in three of four domains, while a same-state diagnostic shows that corrected predictions can still carry substantial wrong-class allocation mass.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
vision language models, transfer learning, domain adaptation, domain generalization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hRnJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose PACE (Prediction and Adaptation via Compatibility Evidence), a novel framework for online test-time adaptation. To address the inherent tension between the dual roles of streaming unlabeled samples in immediate prediction and online distribution updates, PACE constructs a compatibility vector by leveraging the sequential consistency between semantic knowledge encoded in fixed text prototypes and online Gaussian distribution estimates. At the class level, the compatibility information is incorporated into the logarithmic prior of dynamic predictions to refine classification results. At the sample level, it guides a continuous gating mechanism that adaptively regulates each sample's contribution to Gaussian distribution updates. The proposed method is extensively evaluated across multiple datasets, demonstrating its effectiveness. Furthermore, comprehensive ablation studies systematically investigate the contributions of individual components, providing additional evidence to support the validity and effectiveness of the proposed framework.

Critical strengths

PACE enables category-specific prediction optimization, allowing for adaptive adjustments to predictions based on compatibility evidence, rather than applying a uniform correction across all categories. 1.This design empowers the model to distinguish the reliability of evidence across different categories and adapt more effectively to shifts in data distribution. 2.The paper introduces a continuous gating mechanism that adaptively regulates each sample’s contribution to the updating of the Gaussian distribution, thereby avoiding the need for sample filtering based on rigid, fixed rules. For samples with lower compatibility, the model reduces their weight rather than discarding them entirely; this approach preserves potentially valuable "weak signals" while mitigating the risk that unreliable samples might distort the estimation of the Gaussian distribution.

Critical weaknesses
  1. The authors propose using a shared compatibility vector for both "category-level prediction refinement" and "sample-level Gaussian distribution updating"; however, the interplay between these two mechanisms has not been sufficiently clarified. Furthermore, it remains unclear how these mechanisms influence each other during the online adaptation process. It is recommended that the authors provide further clarification regarding the interaction between these two mechanisms, as well as the specific methodological innovations they represent.
  2. The manuscript contains numerous long, information-dense sentences, often featuring multiple nested clauses and newly introduced concepts. The authors are advised to simplify complex sentence structures, reduce redundant conceptual overlap, and present the proposed method with greater clarity and a more rigorous logical structure, thereby enhancing the overall readability of the paper.
Questions for authors
  1. The reliability of the proposed "class-level compatibility ranking" may depend heavily on the quality of the text prototypes. Can this method still reliably distinguish between classes when category descriptions are ambiguous or when there is high semantic similarity among the text prototypes? Specifically, could semantic overlap or poorly defined text prototypes lead to similar compatibility scores, resulting in ambiguous ranking outcomes? It is suggested that the robustness of this ranking mechanism be examined under such conditions.
  2. As the number of classes increases, the task of "class-level compatibility ranking" may become more difficult, potentially leading to unstable or unreliable ranking results. It is recommended to investigate the scalability and ranking stability of this approach in large-scale classification scenarios.
  3. The model relies primarily on a continuous gating mechanism to regulate the contribution of individual samples to the updating of Gaussian distributions. Is this gating mechanism the sole component responsible for sample selection and the control of the update process? If so, the technical innovations of this approach should be further clarified, and its distinctions from existing confidence-based sample weighting or filtering strategies should be explained.
Reviewer_BUEaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a training-free TTA method called PACE. The method compares the class rankings obtained from fixed text prototypes and online Gaussian models. Their agreement is used to correct the current prediction and control how much each sample contributes to updating the Gaussian statistics. The method processes one image at a time and does not update the CLIP encoders. The main experiments cover 21 dataset and domain settings, with parameters selected separately using the labels of each complete target test stream.

Critical strengths

1.This paper investigates the reliability issues of prediction and statistical update in online TTA, which is of practical significance. After a sample's prediction is corrected, it may still introduce erroneous information to subsequent updates. Therefore, it is reasonable to consider these two decisions separately, and the core idea is quite clear. Additionally, this method does not require training or backpropagation of CLIP.

2.The experiments cover 21 datasets and domain settings, and the experimental scope is quite comprehensive. At the same time, the paper separately evaluates the effects of prediction correction, update gating, and geometric correction, and also incorporates constant gating and confidence-based gating as controls to help understand the specific effects of the proposed method.

Critical weaknesses

1.This paper omits some published methods, such as PTA (ICML 2026) and SCA (NeurIPS 2025). These two methods also accumulate information from online test samples, which is relevant to this paper. Why weren't these methods compared? Additionally, most of the baseline results in Tables 6 and 7 are from existing papers, while the text prompts, sample order, and parameter selection methods are not unified. I hope the authors can supplement the comparison under the same backbone, text prompts, and evaluation protocol. I believe this is important for judging the advantages of PACE.

2.The additional improvement brought by the ranking strategy is relatively limited. In Table 5, after excluding EuroSAT, its average improvement over score distance is only 0.112 percentage points. In Table 3, MaxProb also performs better on VisDA-2017 and SUN397.

Questions for authors

For this paper, my assessment is weak reject. The experimental controls are relatively comprehensive, and considering prediction and statistical updates separately is reasonable. However, the parameters are selected using the target test labels, and the comparisons with related methods need to be supplemented under the same settings. Additionally, the specific advantages of the ranking strategy still need stronger experimental support. At the same time, I will also look at the opinions of other reviewers to see if I have missed anything. The specific issues can be found in “Weaknesses”.

Reviewer_ic7KOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to address two problems in online test-time adaptation: making correct predictions for the current sample and reliably updating the model state for future predictions. The authors compare class rankings from CLIP and an online Gaussian classifier to compute ordinal compatibility, which is used to correct Gaussian scores before combining them with CLIP scores for prediction. The same compatibility is used to compute a sample-level gate that controls how much each sample contributes to updating the Gaussian distributions; furthermore, CLIP and Gaussian probabilities determine the class allocation. The paper presents ablation studies and additional analyses to evaluate its design choices.

Critical strengths
  • The paper presents a training-free approach that separately addresses prediction correction and updates to Gaussian statistics for future predictions. Using the same ordinal compatibility signal for both tasks while keeping their operations distinct is an interesting design choice for online test-time adaptation.

  • The evaluation covers 21 datasets and domains, with extensive ablation studies and additional analyses. The authors separately evaluate the prediction and update mechanisms and provide diagnostics to understand their contributions.

Critical weaknesses
  • The paper uses ground-truth labels from the complete target test stream to select dataset-specific hyperparameters. This raises serious concerns about the validity of the reported improvements, as it risks overfitting to the test sets and producing overly optimistic results. The authors should evaluate the method using an independent validation set for hyperparameter selection or fixed hyperparameters across datasets.

  • The motivation for using CLIP–Gaussian ranking agreement as a reliability signal is not sufficiently justified. Although Gaussian statistics capture the evolving class distributions, it is unclear why agreement with CLIP rankings should indicate reliable predictions or updates. A stronger theoretical or empirical justification of this assumption would strengthen the method.

  • Most baseline results are taken from previously published papers rather than reproduced under the same experimental settings. Differences in preprocessing, hyperparameter tuning, and software environments could affect the comparisons, especially when performance differences are small.

  • The geometric and ordinal correction mechanisms are not sufficiently justified. The method penalizes Gaussian scores when image features deviate from a mixed text–Gaussian anchor, but it is unclear why this deviation should indicate less reliable class evidence. Similarly, when the Gaussian classifier correctly ranks a class but CLIP ranks it poorly, the ordinal correction can penalize the correct Gaussian score.

  • The evaluation is limited to a single model, CLIP ViT-B/16. Although the paper covers multiple datasets, it remains unclear whether the method generalizes to other vision-language models.

Questions for authors
  • How are the initial Gaussian class means and covariance matrices constructed before adaptation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.