acceptodds
Under review as a conference paper at ICLR 2027

Soft Pseudo Supervised Contrastive Learning for Test-Time Adaptation

Abstract

Test-time adaptation (TTA) aims to refine pretrained models for shifted test distributions using unlabeled test inputs. Teacher–student self-training shows state-of-the-art performance by minimizing the cross-entropy loss with the pseudo-labels generated by the teacher. However, cross-entropy-based approaches remain vulnerable to noisy pseudo-labels despite the use of various regularization methods. In this work, we propose Pseudo Supervised Contrastive Learning (PSCL) as an alternative to cross-entropy that is a better suited loss for noisy labels in TTA. PSCL uses teacher's pseudo-labels to pull together same-class pair representations of test samples while pushing the rest apart. We propose two variants of PSCL. Hard PSCL assigns a binary relation according to whether the teacher predicts two pairs to be the same class, whereas Soft PSCL weights each relation by the teacher-estimated probability that the two samples belong to the same class. Specifically, for each anchor, PSCL aligns the student’s similarity distribution over other samples with the teacher-estimated probabilities that those samples belong to the same class as the anchor. Our theory provides a guarantee that, under mild conditions, soft pairwise pseudo-labels better approximate the true pairwise labels than hard pseudo-labels. We show that PSCL, as a sole objective, outperforms state-of-the-art baselines including methods that combine cross-entropy with weak contrastive regularization and other regularization techniques. This result holds across various experiments on the standard continual TTA (CTTA) benchmarks CIFAR-10-C, CIFAR-100-C, and ImageNet-C.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Contrastive Learning, Test-Time Adaptation, Continual Test-Time Adaptation, Source-Free Domain Adaptation, Distribution Shift, Pseudo-Labeling, Teacher–Student Learning, Noisy Labels, Soft Labels

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_E2A5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors use the teacher’s predicted class probabilities to estimate the probability that two images belong to the same class. These estimates provide soft pairwise supervision for the student to learn relationships between samples in feature space, rather than directly using the teacher’s pseudo-labels for classification training.

Critical strengths

The paper is clearly presented and easy to follow. The method is simple, and its main idea is easy to understand. Using soft pairwise targets to preserve uncertainty in the teacher’s predictions is a reasonable design choice. The method also requires neither source data nor additional auxiliary losses.

Critical weaknesses

Proposition 1 compares the expected squared errors of the unnormalized pairwise weights wijw_{ij} and hijh_{ij}, whereas Equation 3 uses the normalized weights πij\pi_{ij}. The proof therefore does not fully establish the connection to the loss used in practice.

Figure 2 uses binned calibration statistics to verify the conditions. However, the binning procedure, the estimation of the true same-class probabilities, and how these statistics correspond to the conditions in the proposition are not sufficiently explained. This analysis provides some intuition, but I find the explanation of why the method is more robust to noise insufficient.

Table 2 provides an ablation study, but lacks a controlled comparison that replaces PSCL with CE while keeping all other settings unchanged. Such a comparison would help isolate the contribution of the loss function from that of other design choices.

Another concrete concern is the experimental comparison. The authors directly use the baseline results reported in GOLD, which uses a batch size of 64 on ImageNet-C, whereas Table 3 of this paper specifies a batch size of 100. The experimental conditions are therefore not fully matched.

Furthermore, the authors have not provided source code for reproducing the results. This makes it difficult to independently verify the implementation and experimental settings, particularly whether the augmentation, buffering, and evaluation procedures are consistent with those used for the cited baseline results.

Questions for authors
  1. How does the error bound for wijw_{ij} in Proposition 1 translate to the normalized weights πij\pi_{ij} used in Equation 3? Could the authors provide an error analysis after normalization, or clarify what additional conditions are needed to connect the existing result to the noise robustness of the actual loss?
  2. As discussed in the weaknesses, could the authors provide an online ablation that replaces Soft PSCL with CE while keeping all other settings unchanged? This would help clarify how much of the improvement comes from the loss function itself.
  3. This paper uses a batch size of 100 on ImageNet-C, whereas the cited GOLD results use a batch size of 64. Could the authors explain this difference in experimental settings?
3 comments

We thank the reviewer for the positive assessment and the precise questions.

[E1] Proposition 1 and the normalized weights (W1, Q1)

We extended Proposition 1 to the row-normalized targets of Eq. 3 (Section 3.1, Appendix A.4).

  • The connection to the normalized weights. As the number of comparison partners grows, the soft and hard row sums approach the true row sum, relative to its size. The true, soft and hard normalized targets then share one denominator, and the comparison of Proposition 1 carries over with no condition beyond Eq. 4.
  • Measurement. Figure 6 measures this on CIFAR-10-C. Details and the link to the gradient of the loss are in [J4].

[E2] Binned calibration statistics (W2)

Appendix A.3 now describes the procedure: how pairs are binned, how the same-class probability is estimated per bin, and how each bin is checked against Eq. 4. The submission's calibration figure is replaced by Figure 4. Figure 4 adds ImageNet-C, separates true-positive and true-negative pairs, and varies the number of bins. Details are in [J5]. On why the loss is more robust to noise, please see [J3].

[E3] Loss-only comparison with cross-entropy (W3, Q2)

We added soft and hard cross-entropy under the same pipeline, changing only the loss. Soft PSCL is more accurate in every case:

  • Online CIFAR-100-C: Soft PSCL 71.57% versus 67.65% for soft and 67.69% for hard cross-entropy (Section 5).
  • Severity study: Soft PSCL ahead of soft cross-entropy by 2.04–2.23 pp at every severity (Figure 2).
  • ImageNet-R with a ViT: Soft PSCL 50.32% versus 47.29% for soft and 45.33% for hard cross-entropy (Table 2).
  • Episodic CIFAR-10-C: Soft PSCL 85.12% versus 83.97% for soft cross-entropy (Table 5).

Details are in [J2].

[E4] Batch size on ImageNet-C (W4, Q3)

Thank you for spotting this. The mismatch was a mistake on our side. We reran ImageNet-C with batch size 64 and the same 5,000 images per corruption as GOLD. Soft PSCL reaches 59.17% mean error with a standard deviation of 0.10 pp, compared with GOLD's 59.3% (Table 1, Section 5). Table 6 is corrected accordingly. The Soft-versus-Hard PSCL ablation in Table 3 still uses batch size 100 for both variants, as its caption now states.

[E5] Source code (W5)

We are preparing the code and will share an anonymous repository link in this discussion. Until then, the settings you mention are documented in the revision:

  • Augmentations: the same transforms as CoTTA and GOLD (Section 4).
  • Buffer: Section 3.3.
  • Evaluation protocol: Section 4.
  • Hyperparameters: Appendix A.9.

We thank the reviewer again for their efforts and the positive assessment.

Thank you for the detailed response and additional experiments. The new controlled comparisons with CE directly address my concern about the contribution of the loss function, and the ImageNet-C rerun with a matched batch size corrects the previous discrepancy. I appreciate these additions, which strengthen the empirical evidence for the method.

One detail in the theoretical analysis still needs clarification. In Eq.3, the denominator of πij\pi_{ij} is ∑k∈A(i)wik\sum_{k\in A(i)}w_{ik}, which includes the current partner jj. However, Appendix A.4 explicitly excludes jj from all row sums. The normalized targets analyzed in the appendix therefore still differ from those used in the actual loss. If this difference becomes negligible as the number of partners grows, could the authors clarify the required conditions or provide an error analysis connecting the result to Eq.3?

Overall, the additional experiments have substantially alleviated my main experimental concerns, and I maintain my recommendation of weak acceptance.

Thanks for maintaining the recommendation.

Yes, partner jj is in the denominator of πij\pi_{ij}, as in the original SupCon definition (Khosla et al., 2020, Eq. 2). In the proof for the normalized targets, the true row sum ∑kAik⋆\sum_{k} A^\star_{ik} is treated as given, although it contains Aij⋆A^\star_{ij}, which we define as the random quantity. The row sums with and without jj differ only by the single term Aij⋆∈{0,1}A^\star_{ij}\in\{0,1\}, so they become almost equal as mm grows:

∑k∈A(i)Aik⋆≈∑k∈A(i)∖{j}Aik⋆for large m.\sum_{k\in\mathcal A(i)} A^\star_{ik} \approx \sum_{k\in\mathcal A(i)\setminus\{j\}} A^\star_{ik} \quad \text{for large } m.

We will highlight in Appendix A.4 that the proof neglects this difference for large enough m, and that the result holds for the targets of Eq. 3 as mm grows. Moreover, our empirical check in Appendix A.4 computes all row sums (true, soft, and hard) as in Eq. 3, including jj. We will mention this as well.

Reviewer_jsExOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the problem of noisy pseudo-labels in continual test-time adaptation (CTTA). The authors propose Pseudo Supervised Contrastive Learning (PSCL), which replaces the commonly used cross-entropy loss with a contrastive learning objective. The method uses teacher predictions to construct pairwise relationships between samples. It includes both hard and soft variants, where the soft variant uses predicted class probabilities to weight sample pairs.

Critical strengths

The paper addresses a relevant problem in TTA. The proposed method is relatively simple and achieves competitive results on standard benchmarks.

Critical weaknesses
  1. I think the novelty is very weak. Teacher-student self-training, pseudo-labeling, and contrastive learning are already widely studied in TTA. Moreover, similar soft pairwise contrastive losses have been well explored. I do not see a clear methodological contribution.
  2. The evaluation relies on old benchmarks such as CIFAR-10-C, CIFAR-100-C, and ImageNet-C, using relatively old CNN architectures. There is no evaluation on modern pretrained models or more realistic distribution shifts. The improvements over GOLD are also very marginal.
  3. Important comparisons, such as soft cross-entropy and standard self-supervised contrastive learning, are missing. The contributions of the EMA teacher, memory buffer, and multiple augmented views are also unclear.
  4. Figure 1 is poorly designed, with inconsistent fonts, overlapping labels, and unclear visual organization. The grayscale presentation makes it even harder to read. There are also grammar and formatting issues throughout the paper.
Questions for authors

see above.

3 comments

We thank the reviewer for the feedback.

[S1] Novelty (W1)

See [J1]. We cite CoMatch and do not claim the soft pair weight. Our contributions are:

  • a derivation of the loss from supervised contrastive learning (Sections 2 and 3, Eq. 1 and Eq. 2);
  • a calibration condition under distribution shift (Proposition 1 and Eq. 4, extended to normalized targets in Section 3.1 and Appendix A.4, see [J4]);
  • controlled evidence that this single loss beats cross-entropy in TTA (Section 5, Figure 2, Table 2, Table 5) and motivate for further PSCL research in TTA.

[S2] Benchmarks, backbones and size of the gains (W2)

  • Natural shift and modern backbone. We added ImageNet-R with a ViT-B/32 (Table 2). Soft PSCL reaches 50.32%, above all published results in the TACT table and above soft and hard cross-entropy [J9].
  • Size of the gains over GOLD. The gains are typical for these benchmarks (Table 1). They are obtained without the source prototypes (Section 4) and the source-data warm-up that GOLD uses [J6].

[S3] Missing comparisons and component contributions (W3)

Much of TTA methods that we benchmark already builds on soft cross-entropy objectives, for example CoTTA.

  • Soft cross-entropy and InfoNCE. We further add a controlled comparison against soft cross-entropy, as defined in Section 4, in the same pipeline, and Soft PSCL achieves better results in the experiments described in [J2]. InfoNCE, a standard self-supervised contrastive loss, improves only about 1 pp over the source model on CIFAR-100-C, while Soft PSCL improves about 7 pp (Section 5, Appendix A.7).

  • EMA teacher, buffer and views. The buffer and the views change accuracy by less than 0.4 pp (Appendix A.5, Figure 7), and the EMA teacher is shared with all cross-entropy baselines (Section 4). Please refer to [J8].

[S4] Figure 1 and writing (W4)

We redesigned Figure 1 to address these issues. We also copyedited the paper and corrected grammar, notation and formatting throughout (blue text).

We thank the reviewer for their feedback.

Thank the authors for the detailed response and the additional experiments. However, my main concerns regarding the technical novelty remain. I think the key limitation is taht the method is still a combination of several techniques in test-time adaptation like soft pseudo-labeling relational distillation, and contrastive learning. I think this remains a weak contribution. Regarding the theoretical contribution, the authors only show that soft estimates are more accurate than hard estimates in Proposition 1, but do not explain why this leads to better TTA performance or why it should outperform soft cross-entropy. Regarding the performances, the proposed method shows marginal performance gains than the previous SOTA method GOLD. Importantly, the results highly rely on hyperparameter selection. Given that the gains are still marginal even after hyperparameter selection, I'm still not convinced that whether the method is effective and robust. Regarding the paper writing, although the author revise the framework figure from grayscale to colorful following my previous review, the presentation is still poorly organized. Section 2 Preliminary is extremely short and only introduces the standard contrastive learning loss, without providing any useful background. And this paper doesn’t include a Related Work section. Overall, I don’t think this paper is qualified to accept to ICLR. Therefore I maintain my original rating of 1.

We thank the reviewer for their response.

Novelty

Pseudo-label distillation is a standard approach in recent published TTA work, including CoTTA and GOLD. TTA works so far used contrastive terms as auxiliary losses such as unsupervised contrastive learning, source prototypes assisted contrastive term or negative filter such as adacontrast. However, we propose a fully pseudo supervised solution with one objective only. This study shows that this loss alone outperforms existing methods on the benchmarks. It gives future research a base to build on for even better performance.

Theory: why soft targets give better TTA

The best case is SupCon with oracle labels, and the closer our targets are to these labels, the better the learning. Let gig_i be the gradient function of anchor ii (Eq. 5 in the paper), decomposed as

gi(π)=gi(π∗)⏟oracle SupCon+ηi(π)g_i(\pi) = \underbrace{g_i(\pi^*)}_{\text{oracle SupCon}} + \boldsymbol{\eta}_i(\pi)

where the error part ηi(π)=1τ∑j∈A(i)(πij∗−πij) zj\boldsymbol{\eta}_i(\pi) = \frac{1}{\tau}\sum_{j\in A(i)}\big(\pi^*_{ij} - \pi_{ij}\big)\, z_j depends only on how far the target π\pi is from the oracle target π∗\pi^*. We show that Proposition 1 extends to the normalized targets, so the soft target is closer to the oracle target than the hard target. It therefore gives a lower gradient error, which yields better optimization. Moreover, the advantage over soft cross-entropy is shown empirically in our extensive experiments.

Gains over GOLD

As we highlighted before, these methods rely on source prototypes and several auxiliary losses. We show that a single loss, without any source information, still outperforms them. We expect that further, similar adjustments will lead to even better results, and we leave this for future work.

Hyperparameter selection

We choose our configuration manually and do not use grid search or search algorithms, even though other methods do (see their published code). We ran a parameter search in only one experiment, the severity test on CIFAR-100-C, for both cross-entropy and PSCL. There, PSCL improves from ~71.5% (the result reported in the benchmark table against the other methods) to ~72%. We believe this speaks in favor of our method.

Related Work section

In ICLR papers it is common to discuss related work within the Introduction rather than in a separate section, which we also do.

Reviewer_jZFDOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Soft Pseudo-Supervised Contrastive Learning (Soft PSCL) for continual test-time adaptation (CTTA), aiming to mitigate the effect of noisy pseudo-labels in conventional cross-entropy-based teacher–student self-training. Rather than supervising individual predictions, the method uses an EMA teacher to estimate the probability that two samples share a class, computed as the inner product of their predictive distributions. It then trains the student encoder with a supervised contrastive objective whose positive weights are given by these soft pairwise targets. Multiple augmented views per sample and a class-conditional memory buffer are used to ensure sufficient positive pairs. Notably, the authors use this loss as the sole adaptation objective, without source data, auxiliary regularization, or confidence thresholding. They also show theoretically that the soft pairwise estimate attains lower expected squared error than the hard estimate when the pairwise calibration error lies within a derived range, and they empirically verify that most pairs satisfy this condition. Experiments on CIFAR-10-C, CIFAR-100-C, and ImageNet-C report state-of-the-art average error rates under the CTTA protocol. Soft PSCL consistently outperforms its hard-label counterpart, with a particularly large margin on ImageNet-C, and outperforms cross-entropy in an episodic setting and in t-SNE-based representation analysis.

Critical strengths
  1. Clear motivation and simple formulation.

The paper addresses a well-motivated problem in TTA: the vulnerability of cross-entropy-based self-training to noisy pseudo-labels. Using pseudo-labels to supervise relations between samples rather than individual predictions is an intuitive and well-aligned response to this problem. Notably, the method uses this contrastive loss as the sole adaptation objective, without source data, source prototypes, or auxiliary regularization.

  1. Promising empirical performance.

Soft PSCL achieves the best average error rates on CIFAR-10-C, CIFAR-100-C, and ImageNet-C under the CTTA protocol, with results averaged over five runs. The controlled comparison with Hard PSCL, under identical hyperparameters, shows that soft supervision lowers the error on every corruption type across all three datasets, with a particularly large gain on ImageNet-C (67.01% → 58.97%). This provides encouraging evidence for the value of soft pairwise supervision.

  1. Theoretical justification with empirical verification.

The analysis gives a clear condition on pairwise calibration error under which soft pairwise estimates have lower expected squared error than hard ones. The authors also verify empirically that most pairs (about 97%) satisfy this condition, which helps connect the theory to the observed performance gains.

Critical weaknesses
  1. Insufficient controlled comparison with cross-entropy.

A central claim is that Soft PSCL is a better base objective than cross-entropy for TTA. The paper compares the two only in an episodic setting and through a t-SNE analysis, and it is unclear whether the cross-entropy baseline there uses the same multi-view augmentation and memory buffer. In the main CTTA setting, there is no comparison that changes only the loss while keeping the teacher–student framework, augmentation, buffer, and protocol fixed. In particular, soft-label cross-entropy under the same pipeline would help determine whether the gains come from relational contrastive supervision specifically or from soft supervision in general.

  1. Fairness and significance of the baseline comparison.

Baseline results are taken from GOLD without variance, and the GOLD numbers come from its source-assisted variant. Given that the improvements over GOLD are small (about 0.2–0.3 pp on CIFAR-10-C and CIFAR-100-C), it is difficult to judge whether they are statistically meaningful. The comparison is further complicated by the additional augmented views used by Soft PSCL (see point 6).

  1. Limited direct evidence of robustness to noisy pseudo-labels.

Although robustness to noisy pseudo-labels is the primary motivation, the paper does not directly measure how performance changes as pseudo-label quality deteriorates. Experiments across corruption severity levels, tracking pseudo-label accuracy during adaptation for Soft PSCL versus cross-entropy, or injecting controlled label noise would provide more convincing support for the central claim.

  1. Weak connection between the theoretical analysis and empirical gains.

Proposition 1 shows that soft pairwise estimates have lower expected squared error than hard estimates under a calibration condition. However, this does not directly imply better contrastive optimization or classification performance, especially since the actual objective uses row-normalized weights and a temperature-scaled softmax. The empirical verification also has limitations. The reported proportions (about 97%) may be dominated by negative pairs with near-zero weights, which the authors themselves note for CIFAR-100-C. The check is performed on binned calibration gaps rather than individual pairs. Most importantly, it is not reported for ImageNet-C, where the gain over Hard PSCL is largest. An analysis that separates positive and negative pairs and includes ImageNet-C would strengthen this part.

  1. Limited novelty relative to relational contrastive methods.

The core idea combines supervised contrastive learning with soft relational targets derived from class probabilities. This is closely related to existing work such as CoMatch, which also aligns an embedding similarity graph with a pseudo-label graph constructed from class probabilities. Likewise, the theoretical result follows from the standard property that calibrated probabilities minimize expected squared error. A clearer discussion of what is new beyond applying this formulation to CTTA would help.

  1. Insufficient ablations and hyperparameter sensitivity analysis.

The framework depends on several components and hyperparameters, including the number of augmented views, the memory buffer, the temperature, and setting wij=1w_{ij}=1 for views of the same sample. Their individual contributions are not analyzed. The hyperparameters also differ considerably across datasets (e.g., temperature from 0.04 to 0.09, buffer size from 0 to 20, and 6 to 10 views), and the authors acknowledge that implicit tuning occurred. Sensitivity analyses would help assess how robust the method is in practice, where labels for tuning are unavailable.

  1. Limited analysis of computational cost.

The objective scales quadratically with the number of samples, and the multi-view augmentation (e.g., 200 × 10 = 2,000 views per batch on CIFAR) and buffer add further overhead. Since adaptation cost matters in online TTA, comparisons of adaptation time, memory usage, and accuracy under matched compute against existing methods would improve the practical relevance of the work.

  1. Scope of evaluation.

All experiments use synthetic corruption benchmarks and CNN backbones. Evaluation on natural distribution shifts (e.g., ImageNet-R or DomainNet) or transformer-based backbones would better support the claim that PSCL is a general replacement for cross-entropy in TTA.

Overall assessment.

The paper presents a simple, intuitive, and well-motivated approach to CTTA, and the consistent improvements of Soft PSCL over Hard PSCL are encouraging. However, I have several concerns regarding the novelty of the proposed objective and the experimental evidence supporting its central claims. In particular, the proposed formulation appears closely related to existing graph-based contrastive methods such as CoMatch, and its methodological contribution beyond adapting such an objective to CTTA remains unclear. Moreover, the current experiments do not sufficiently establish whether relational contrastive supervision is inherently more effective or robust to noisy pseudo-labels than cross-entropy-based self-training under comparable conditions. The improvements over the strongest baseline are also relatively modest, and the theoretical analysis does not directly explain the observed performance gains. Overall, while I find the approach promising, I do not believe the current evidence is sufficient to support acceptance. My assessment could change depending on the authors' responses, particularly regarding the methodological distinction from CoMatch (Q1), controlled comparisons with cross-entropy (Q2), and robustness to pseudo-label noise (Q3).

Questions for authors
  1. Relation to CoMatch.

CoMatch (Li et al., 2021) also constructs a pseudo-label similarity graph from class probabilities and aligns it with an embedding similarity graph through a cross-entropy-based contrastive objective. Given the substantial similarity between these formulations, could the authors clarify the key methodological differences and what constitutes the main novelty of Soft PSCL beyond its application to CTTA?

  1. Controlled comparison with cross-entropy.

Could the authors report hard- and soft-label cross-entropy results under the same CTTA protocol, using a comparable teacher–student pipeline, augmentation strategy, and computational budget? Additionally, did the cross-entropy baseline in the episodic experiment use the same multi-view augmentation and memory buffer settings as Soft PSCL?

  1. Robustness to pseudo-label noise.

How does the performance gap between Soft PSCL and cross-entropy change as pseudo-label quality deteriorates? Evaluating different corruption severity levels (1–5), together with the corresponding teacher pseudo-label accuracy, would help clarify whether the observed improvements are specifically attributable to robustness against noisy pseudo-labels.

  1. Calibration analysis.

How is the pairwise calibration error used to verify Eq. 4 estimated empirically? Could the authors report the proportion of pairs satisfying the theoretical bound separately for true positive and negative pairs, as well as for different ranges of pairwise weights? Extending this analysis to ImageNet-C would be particularly informative, given the large performance gap between Soft and Hard PSCL on this dataset.

  1. Computational efficiency.

What are the adaptation time and peak GPU memory usage of Soft PSCL compared with GOLD and CoTTA under comparable hardware settings? How does the accuracy change when the number of augmentation views is reduced to approximately match the computational budget of these baselines?

  1. Hyperparameter selection and sensitivity.

How were the dataset-specific hyperparameters in Table 3 selected without access to labeled target validation data? How sensitive is the performance to the temperature, number of augmentation views, and buffer size?

  1. Baseline comparison protocol.

The reported GOLD results use source-derived class prototypes, whereas Soft PSCL does not require such additional source information. Could the authors clarify the implications of this difference for the comparison and whether a variant of GOLD without source-derived prototypes is available? Given the relatively small improvements over GOLD, could the authors also provide variance estimates or repeated-run comparisons to better assess the significance of these gains?

3 comments

We thank the reviewer for the very thorough review. Since the reviewer named Q1–Q3 as decisive, we answer those first.

[J1] Novelty and relation to CoMatch (W5, Q1, overall assessment)

We thank the reviewer for highlighting CoMatch (Li et al., 2021). We cite CoMatch. The revised paper gives CoMatch more emphasis and briefly describes what our work adds to the literature (Introduction). Here we describe these additions in full, from least to most important.

  1. Setting and implementation. CoMatch is a semi-supervised method. It uses labeled data, confidence thresholds and two additional cross-entropy losses, and it smooths its pseudo-labels with a memory bank that contains ground-truth labels. Its graph loss serves as a regularizer inside this full system. PSCL is studied as the sole adaptation objective, in a fully unlabeled setting under distribution shift.

  2. A grounded derivation of the loss. We build the loss step by step (Sections 2 and 3). We start from supervised contrastive learning (SupCon), which is the oracle case with true labels (Eq. 1). We then identify same-class membership as the quantity the pair target estimates, and derive the soft weight as a same-class probability under conditional independence (Eq. 2). This explains where the loss comes from and what its targets mean, and it is what allows us to ask when soft targets beat hard ones.

  3. Calibration under distribution shift. Distribution shift was not studied in Comatch's setting, however this is a core problem in TTA. Particularly distribution shifts degrade the calibration of the model (Ovadia et al., 2019). Proposition 1 provides a sufficient condition, under which soft pair targets still have lower expected squared error than hard ones.

  4. Why this study matters beyond CoMatch. This study invites the field to rebuild its well-established refinements on PSCL rather than on cross-entropy in TTA, including the sample filtering and weight regularization of EATA, the weight restoration of CoTTA, and the source prototypes of RMT.

[J2] Controlled comparison with cross-entropy (W1, Q2, overall assessment)

Much of TTA research builds on soft cross-entropy objectives (ex. such as Cotta, Gold) We added soft and hard cross-entropy under the same pipeline for controlled comparison: Both losses are defined in Section 4.

  • CIFAR-100-C, online CTTA (Section 5). With the hyperparameters of Soft PSCL, soft and hard cross-entropy reach 67.65% and 67.69% online accuracy, compared with 71.57% for Soft PSCL.

  • Cross-entropy with its own tuned hyperparameters (Figure 2). When cross-entropy is tuned separately at severity 5, it improves, but Soft PSCL stays ahead by 2.04–2.23 pp at every severity [J3].

  • ImageNet-R with a ViT (Table 2). With the learning rate searched separately for each loss, Soft PSCL reaches 50.32%, compared with 47.29% for soft and 45.33% for hard cross-entropy [J9].

  • Episodic CIFAR-10-C (Section 5, Table 5). the cross-entropy baseline uses the same views and buffer as Soft PSCL, and so does the cross-entropy model in the t-SNE comparison (Figure 5). For the revision we also tuned the EMA momentum (0.9, 0.99, 0.999) separately for each loss. Cross-entropy improves from 81.11% to 83.97%, and Soft PSCL reaches 85.12%. Each with its optimal hyperparameters.

  • Matched compute. Both losses use the same compute and memory at every setting (Table 4, [J7]). In these profiling runs, Soft PSCL is 2.7–3.6 pp more accurate than cross-entropy at every number of views from 1 to 9.

[J3] Robustness to noisy pseudo-labels (W3, Q3, overall assessment)

  • Severity study (Section 5, Figure 2). We ran the full CIFAR-100-C continual sequence at each severity from 1 to 5, comparing Soft PSCL with soft cross-entropy in the same pipeline. Each loss uses its best hyperparameters found at severity 5, kept fixed across severities. The accuracy of the unadapted source model at each severity measures the initial pseudo-label quality.
  • Result. As severity grows, the source accuracy drops. Soft PSCL stays ahead by 2.04–2.23 pp at every severity. Measured relative to the source accuracy, the gap grows from 2.8% at severity 1 to 3.4% at severity 5. The advantage of PSCL therefore grows relative to source model as pseudo-labels get worse.

[J4] Connection between the theory and the loss (W4, first part; also Reviewer E2A5 W1 and Q1)

  • Row-normalized targets (Section 3.1, Appendix A.4). Proposition 1 compares unnormalized targets, while the loss divides each anchor's targets by their row sum. We now extend the result to the normalized targets.
    • One added assumption: as the number of comparison partners grows, the ratios of the soft and hard row sums to the true row sum approach one.
    • What follows: all three normalized targets (true, soft and hard) then share one denominator. The comparison of Proposition 1 carries over, with no condition on the calibration error beyond Eq. 4.
  • Measured on CIFAR-10-C (Figure 6). Both ratios of row sums approach one as the number of partners grows, and so does the ratio between the normalized and the unnormalized error difference.
  • Link to optimization. The gradient of the loss (Eq. 5) is the difference between the student's similarity distribution and the normalized target. Any error in the target therefore enters the gradient directly. With exact targets, PSCL reduces to SupCon with true labels (Introduction). Proposition 1 thus measures how far the training signal is from this oracle signal.

[J5] Calibration analysis (W4, second part; Q4; also Reviewer E2A5 W2)

  • How the calibration error is estimated (Appendix A.3). Appendix A.3 describes how we estimate the miscalibration of each bin, following the binning estimator of the expected calibration error (Guo et al., 2017). We then score each bin by whether its estimate lies within the bounds of Eq. 4 at the bin’s mean pair weight, and report the percentage of pairs that fall into passing bins. We also study how this score changes with the number of bins.
  • New Figure 4 (replaces the submission's calibration figure). The new figure:
    • covers CIFAR-10-C, CIFAR-100-C and ImageNet-C;
    • shows all pairs, true-positive pairs and true-negative pairs as separate lines;
    • varies the number of bins from 1 to 1,000;
    • uses all pairs among 1,000 samples drawn at random across the 15 corruptions.
  • Result.
    • On all three datasets, a high percentage of pairs lies in bins that satisfy the bound, ImageNet-C included.
    • True-positive pairs are lower than negatives but stay at about 85% or more for every number of bins. The result is therefore not driven by negative pairs alone.
  • Bins versus individual pairs. The second row of Figure 4 shows the median and the spread of the number of pairs per bin for each number of bins, which indicates how noisy the estimate of bound satisfaction can be.

[J6] Fairness and significance of the baseline comparison (W2, Q7, overall assessment)

  • Variance. Table 1 reports Soft PSCL as mean ± standard deviation: 13.87 ± 0.14 and 28.43 ± 0.03 over five runs on CIFAR-10-C and CIFAR-100-C, and 59.17 ± 0.10 on ImageNet-C. GOLD reports means only, so a significance test against GOLD is not possible. We therefore compare mean errors.

  • Size of the gains. Soft PSCL improves on GOLD by 0.23, 0.17 and 0.13 pp on CIFAR-10-C, CIFAR-100-C and ImageNet-C. Such differences are typical on these benchmarks (Table 1).

  • Information used by GOLD. GOLD's reported numbers come from its source-assisted variant with source class prototypes (Section 4). In addition, GOLD's released code warms up the model on source data before adaptation, following RMT. The released CIFAR-100-C configuration trains all weights for one epoch on the 50,000 source training images and loads this warm-up checkpoint (GOLD repository, classification/methods/gold.py). The GOLD paper does not mention this step. In RMT's ablation, the warm-up alone lowers the CIFAR-100-C error by about one point (30.1% to 29.0%). Soft PSCL uses no warm-up, no source data at any stage, and no additional losses.

  • GOLD without source prototypes. GOLD's own ablation without its source-prototype loss (Table 2 of the GOLD paper) reports 15.06%, 28.77% and 59.87%. Soft PSCL is lower on all three benchmarks, its not clear if this GOLD variant still uses the source warm-up as well.

  • PSCL with source information. For a comparable study against GOLD, we added the same kind of source-prototype term to PSCL, with one view and no buffer (Appendix A.7). In a single run on CIFAR-100-C, this reaches 71.58% online accuracy (28.42% error). That is below GOLD's 28.6% error, without the additional compute from the warm-up.

[J7] Computational cost (W7, Q5; the views point in W2)

  • Same cost as cross-entropy (Appendix A.6, Table 4). We profiled Soft PSCL and soft cross-entropy in the same pipeline on CIFAR-100-C, with 1 to 9 views and buffer sizes from 10 to 890. Per adaptation step, both need the same compute (within 0.2%) and the same peak memory (within 0.4%). We report FLOPs because they do not depend on the hardware.
  • The quadratic term is negligible (Section 7). The Soft PSCL loss accounts for at most 0.18% of the adaptation FLOPs. The cost comes from the network passes over the views and the buffer, which cross-entropy pays equally when buffer is used. At equal cost, Soft PSCL is more accurate [J2].
  • Fewer views (Appendix A.5, Figure 7). Accuracy changes by at most 0.34 pp between 1 and 8 views, so Soft PSCL can run with a single view.
  • Versus CoTTA. With one view, our compute per batch (3.9 TFLOPs) is about half of CoTTA's as reported in GOLD (7.84 TFLOPs), with similar peak memory (11.6 GB versus 10.88 GB).
  • Versus GOLD. GOLD's reported compute does not include its source warm-up [J6]. With one view, no buffer and a source-prototype term, PSCL reaches 28.42% error on CIFAR-100-C, below GOLD's 28.6% (Appendix A.7).

[J8] Hyperparameter selection, sensitivity and component contributions (W6, Q6; also Reviewer jsEx W3)

  • How the hyperparameters were chosen. Like other TTA methods, we use the target labels to set the hyperparameters. We set them manually, without a parameter search algorithm. We ran a parameter search only for the severity study (Figure 2), separately for PSCL and cross-entropy, so that the two losses are compared at their best settings.
  • Sensitivity (Appendix A.5, Figure 7). We ran a one-at-a-time sweep on a shortened CIFAR-100-C stream:
    • Temperature: between 0.035 and 0.071, accuracy changes by at most 0.92 pp.
    • EMA momentum: from 0.996 to 0.99975, accuracy changes by at most 0.1 pp.
    • Buffer: removing it entirely costs 0.37 pp.
    • Number of views: from 1 to 8, accuracy changes by at most 0.34 pp. The 12-view setting does not fit in GPU memory.
    • Learning rate: the most sensitive hyperparameter.

[J9] Scope: natural shift and transformer backbone (W8)

We added ImageNet-R (Table 2), a natural distribution shift, with a pretrained ViT-B/32. The setup uses all 30,000 images and 200 classes in a single online pass, with the classifier head frozen. Soft PSCL reaches 50.32% online accuracy. This is higher than all results in the published table of TACT (Liu et al., 2025), whose best are TACT-adapt at 48.81% and SHOT at 48.79%. Under the same pipeline, it is also higher than soft cross-entropy (47.29%) and hard cross-entropy (45.33%).

We thank the reviewer very much for the thorough review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

General response to all reviewers

All figure, table, section and appendix numbers refer to the revised PDF. New text is blue and removed text is struck through.

We thank all three reviewers for their detailed and constructive feedback. The revision adds the following new results and clarifications. Each item is answered in full once, in the reply marked in brackets, and the other replies point there.

What we addedWhere in the paperFull answer
Loss-only comparison with soft and hard cross-entropy in the same pipelineSection 4 (loss definitions), Section 5, Figure 2, Table 2, Table 5[J2]
Robustness to pseudo-label quality across corruption severities 1–5Figure 2[J3]
Extension of Proposition 1 to the row-normalized targets used in the lossSection 3.1, Appendix A.4, Figure 6[J4]
Calibration check redone: true positives and negatives separated, ImageNet-C added, number of bins variedSection 5, Appendix A.3, Figure 4[J5]
ImageNet-C rerun with batch size 64, as in GOLDTable 1, Table 6[E4]
Natural distribution shift with a Vision Transformer (ImageNet-R)Table 2[J9]
Hyperparameter sensitivity (learning rate, temperature, EMA momentum, buffer size, number of views)Appendix A.5, Figure 7[J8]
Compute and GPU memory of the full pipelineSection 7, Appendix A.6, Table 4[J7]
A source-assisted PSCL variant with one view and no bufferAppendix A.7[J6], [J7]
Clarity: redesigned Figure 1, cross-entropy losses defined, notation and wording correctedFigure 1, Section 3, Section 4[S4]

Main message. In every controlled comparison we ran, one basic relational loss, with no source data, thresholds or regularizers, is more accurate than cross-entropy. It also reaches lower mean errors than the reported baselines, which build on cross-entropy and add further mechanisms. We see this as a reason to reconsider the base objective in TTA. PSCL is a simple starting point that future work on regularization, efficiency and better relational objectives can build on.

We have uploaded a revised PDF with all new results. We may upload minor editorial updates until the end of the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.