acceptodds
Under review as a conference paper at ICLR 2027

Thinking the Stability-Plasticity Dilemma in Non-Stationary Test-Time Adaptation from a Bayesian Perspective

Abstract

Reliably adapting foundation models (FMs) to non-stationary medical environments remains unsolved, as test distributions evolve dynamically through changing acquisition protocols, patient populations, and disease prevalence. Existing test-time adaptation (TTA) methods often degrade below the frozen source model. We trace this failure to the stability-plasticity dilemma: plasticity for tracking the evolving target and stability for preserving generic knowledge compete for the same parameter space. From a Bayesian perspective, we decompose the predictive posterior on test data into a class prior that encodes stable class-level knowledge and a class-conditional likelihood that carries per-input evidence, with their balance governed by the uncertainty of online evidence. This yields a Bayesian Generalization Error Bound (BGEB) that factorizes target risk into finite-batch uncertainty, prior mismatch, conditional shift, and parameter drift, thereby turning the dilemma into a principled trade-off. Guided by the error bound, we propose an effective algorithm SynerTTA, which rectifies the online prior, restricts updates to a low-displacement subspace, and weights adaptation by instance reliability. Across eight medical benchmarks and three modalities, SynerTTA outperforms thirteen baselines, surpassing the second-best method by 7.75% in accuracy and 5.56% in macro-F1, while achieving the best calibration and robustness under unreliable online evidence.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
test-time adaptation, foundation models, Bayesian error bound, non-stationary clinical streams

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Hq4zOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper revisits the stability–plasticity dilemma in non-stationary test-time adaptation (TTA) and proposes SynerTTA, a framework combining online class-prior rectification, gradient-sensitive parameter masking, and reliability-aware adaptation. The authors derive a Bayesian-factorized generalization bound to motivate these components and evaluate the method across eight medical imaging benchmarks spanning ophthalmology, histopathology, and chest X-rays. The reported results show substantial improvements over several existing TTA approaches, particularly under class imbalance and small-batch adaptation.

Critical strengths

The paper addresses an important challenge in continual TTA: adapting to evolving distributions without compromising previously acquired knowledge. The stability–plasticity perspective is relevant to medical foundation models, where unreliable online updates can have significant consequences. Combining prior adjustment, selective parameter updates, and reliability-aware supervision is a reasonable approach to improving adaptation robustness. The proposed components are relatively lightweight and can be applied to different pretrained architectures. The experiments cover multiple medical modalities, datasets, and distribution-shift conditions. The inclusion of class-imbalance, batch-size, and forgetting analyses provides useful evidence beyond conventional average-accuracy comparisons.

Critical weaknesses
  1. The connection between the theoretical bound and the implemented algorithm is not sufficiently established. The Bayesian-factorized bound provides a useful decomposition of target risk, but several of its terms, including source-reweighted risk and class-conditional discrepancy, are unobservable during deployment. It remains unclear whether the bound provides actionable guidance beyond motivating familiar regularization and pseudo-labeling strategies.

More importantly, the displacement argument for GSPM in Eqs. (10)–(11) assumes a masked gradient-descent update, whereas the experiments use AdamW with persistent optimizer states. Selecting parameters with small gradient norms does not necessarily minimize the resulting AdamW parameter displacement. The authors should clarify how the theoretical argument applies to the actual optimization procedure and provide evidence that the proposed components meaningfully control the quantities identified by the bound.

  1. The posterior correction in Eq. (6) is exact under label shift, assuming calibrated source predictions and a known source class prior. However, the paper explicitly targets settings involving simultaneous changes in class prevalence and image appearance, where these assumptions may not hold.

This is particularly relevant for medical vision-language models, whose similarity-based predictions are not necessarily calibrated class posteriors. The definition and estimation of the effective source prior are also insufficiently explained. Furthermore, using predictive entropy to determine the reliability of online prior estimates may reinforce confidently incorrect predictions under severe distribution shifts.

The authors should evaluate the accuracy of the estimated class priors and investigate how RWLA behaves when label shift and class-conditional shift occur simultaneously.

  1. In the main benchmark, each dataset is processed as an independent stream, and the model is reset before moving to another dataset. Processing a fixed target dataset sequentially does not, by itself, establish that the underlying distribution changes over time. Although the supplementary experiments introduce sequential tasks and mixed-domain streams, their relationship to realistic non-stationary deployment remains unclear. In particular, the sequential-task evaluation activates task-specific label sets, potentially assuming knowledge of the current task identity.

A stronger evaluation would consider continuously evolving streams with unknown domain boundaries and a consistent prediction space, while explicitly measuring adaptation and forgetting across distribution transitions.

  1. Table 2 provides progressive ablations, but these do not establish whether the proposed components offer advantages over simpler alternatives. In particular, selecting low-gradient parameter groups should be compared with random masking, fixed parameter subsets, and learning-rate-controlled updates under comparable parameter displacement. Similarly, the contribution of reliability-aware supervision should be separated from that of prior correction. The final adaptive component provides a relatively modest improvement over the preceding variant, making its independent benefit less convincing. The overall performance advantage is also uneven across datasets. For example, the substantial gain on COVID-19 contrasts with considerably weaker performance than StatA on NCT-CRC. More detailed per-dataset uncertainty estimates and controlled comparisons would strengthen the empirical conclusions.
Questions for authors

How does the GSPM displacement argument extend to AdamW, and can the authors empirically connect the proposed updates to the terms in the generalization bound?

How is the source class prior determined for each foundation model, and how reliable is the online prior estimator when both label and conditional distributions change?

Can the authors evaluate SynerTTA on continuously changing target streams without predefined task boundaries or task-specific label selection?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part 1/2)

Q1:How does the GSPM displacement argument extend to AdamW?***

A1: We sincerely appreciate your constructive comments. The displacement argument is successfully extended to AdamW through an optimizer-independent bound that links the realized step norm to source-relative drift, with paired one-step replays showing that GSPM yields smaller steps than both matched-budget controls. Equations (10)–(11) motivate GSPM’s selection rule under masked gradient descent, where the step norm equals the learning rate times the norm of the retained gradient. The relationship between current gradient norms and realized step size depends on the optimizer. Under AdamW, the step also depends on the running first and second moments and weight decay, so selecting the groups with the smallest current gradient norms does not guarantee the smallest realized step. However, we tested whether GSPM still reduces displacement under AdamW using paired one-step replays on MESSIDOR. Starting from the same model, AdamW state, and batch, and selecting the same number of parameter groups per encoder, GSPM reduced the realized step norm by 13.18% relative to random selection and 21.36% relative to largest-gradient selection. By the triangle inequality, a smaller step from the same starting point tightens the one-step upper bound on source-relative drift, linking this result to (D_t).


Q2: The connection between the theoretical bound and the implemented algorithm is not sufficiently established.

A2: All these connections and experimental evidence show that the theoretical analysis offers a rigorous foundation for SynerTTA design, and each components of SynerTTA actually reduce the bound. The theoretical bound and SynerTTA algorithm are sufficiently established in our paper, and we also conducted supplementary ablation studies for each module of the algorithm. In the specific response, we explain the details.


Q3: How is the source class prior determined for each foundation mode?

A3: Actually, the source class prior can be estimated without accessing the training data, and this work has already been solved by others[1]. Zhu et al. (NeurIPS 2023)[1] shows how to estimate task-specific label bias from labeled downstream data without accessing the original pretraining data.

To emphasize the focus of our method, the source class prior is set to be determined, but it can be replaced by any estimated approach, as mentioned above.

In our implementation, the model-specific reference-prior correction is omitted, and only the target prior is estimated online from unlabeled predictions. Although source prior estimation is ignored, our method still gains better results compared with mainstream TTA. Evaluating this approach for medical foundation models is a direction for future work.

[1] Zhu, Beier, et al. "Generalized logit adjustment: Calibrating fine-tuned models by removing label bias in foundation models." Advances in Neural Information Processing Systems 36 (2023): 64663-64680.


Q4: how reliable is the online prior estimator when both label and conditional distributions change?

A4: The online prior estimator is relatively reliable when both label and conditional distributions change. Experimental results demonstrate the reliability of the online prior estimator when both label and conditional distributions change, in Table 2, page 9. The performance gains involving both shifts provide indirect evidence that the online prior estimates are useful for adaptation.


AuthorsOpenReview ↗

Official Comment by Authors (Part 2/2)

Q5: Using predictive entropy to determine the reliability of online prior estimates may reinforce confidently incorrect predictions under severe distribution shifts.

A5: To be honest, predictive entropy may reinforce confidently incorrect predictions under severe distribution shifts. However, it is a simpler choice for online prior estimation. Especially, ablation experiments indicate that the impact of entropy is not very significant in the distribution shifts.

In future work, we will explore more effective approaches for online prior estimation.


Q6: Evaluation of SynerTTA on continuously changing target streams without predefined task boundaries or task-specific label selection

A6: We evaluated SynerTTA on continuously changing target streams without predefined task boundaries or task-specific label selection, and The experimental results demonstrate that SynerTTA achieved the highest mean online Macro-F1 under gradual drift and the largest mean online accuracy gains over the frozen source model during both transitions in the constructed MESSIDOR stream, as image gamma gradually decreased from 1.0 to 0.7 and then gradually returned to 1.0. In the evaluation, a single MESSIDOR stream has gradual photometric drift. The model carried its state across the stream, received no task identities or boundary signals, and predicted over the same five classes throughout.

MethodDrift Macro-F1 (%)Shuffled Macro-F1 (%)Outward gain (pp)Return gain (pp)
Source54.91 ± 0.2654.91 ± 0.260.00 ± 0.000.00 ± 0.00
RoTTA52.43 ± 0.4353.67 ± 0.2314.67 ± 2.5512.61 ± 1.72
SynerTTA59.93 ± 2.0056.53 ± 1.7518.06 ± 5.8219.76 ± 2.68

Values are mean ± standard deviation over three paired runs. Transition gains are online accuracy differences relative to frozen Source.

SynerTTA achieved the highest mean online Macro-F1 under gradual drift, exceeding Source by 5.02 percentage points, and achieved the largest mean accuracy gains in both transitions. These results show that SynerTTA can adapt to controlled drift without boundary signals or task-specific label selection. The appendix reports the full protocol, trajectories, and held-out forgetting and recovery results.


Q7: Ablations do not establish whether the proposed components offer advantages over simpler alternatives.

A7: We added comparative experiments that establish whether the proposed components offer advantages over simpler alternatives. The experimental results indicate that each component achieved the best performance compared to its corresponding simpler approach. The details can be seen in the specific response and in Appendix F.4.


Q8: Performance advantage on NCT-CRC.

A8: We noticed the better performance of StatA in the NCT-CRC dataset during running the experiments. We also found that some mainstream methods (such as TENT, CoTTA) show much worse performance even compared with the source model. And our SynerTTA also shares some similarities with those methods at the methodological level; for example, TENT and SynerTTA both employ the updating of normalized affine parameters. So we guess these similar points may be the reason of performance degradation, and we plan to explore this meaningful question in our future work.

Dear Reviewer Hq4z,

I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper!

Best regards,

Authors of Submission 16610

I’m raising my rating to 3. The new MESSIDOR experiment addresses a key concern in my initial review by showing that SynerTTA improves over the frozen model under gradual drift without task labels or boundary cues. I still have questions about long-term parameter drift and the accuracy of prior estimates under mixed shifts.

Reviewer_rM5iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates the stability-plasticity dilemma in non-stationary test-time adaptation (TTA) for medical foundation models. It derives a Bayes-factorized generalization bound and proposes SynerTTA, combining online prior correction, gradient-sensitive parameter masking, and reliability-aware adaptation. Experiments across eight medical imaging datasets and three modalities demonstrate improvements over several existing TTA methods. The paper aims to provide a theoretical perspective on balancing adaptation and knowledge preservation.

Critical strengths
  1. Meaningful research problem. The stability-plasticity dilemma under non-stationary distribution shifts is an important challenge, particularly for medical applications involving small batches and class imbalance.

  2. Encouraging empirical results. The experiments cover multiple medical modalities and include ablation studies and robustness evaluations, demonstrating promising performance improvements.

Critical weaknesses
  1. Insufficient theoretical novelty and justification. Several relevant theoretical studies on TTA are not adequately discussed, including [1][2][3]. Moreover, the proposed bound relies on standard generalization techniques, and its connection to the actual adaptation objective and stability metrics remains insufficiently established.

  2. Inconsistencies in experimental reporting. Several reported numerical results and implementation details appear inconsistent. For example, the performance improvements claimed in Table 1 do not match the displayed values, some averages in Table 2 cannot be reproduced from the reported modality scores, and the hyperparameter settings differ between Appendices D.3 and D.5. These inconsistencies weaken confidence in the experimental conclusions.

Reference

[1] Persistent Test-time Adaptation in Recurring Testing Scenarios, NeurIPS 2024.

[2] Lifelong Test-Time Adaptation via Online Learning in Tracked Low-Dimensional Subspace, NeurIPS 2025.

[3] On the Learnability of Test-Time Adaptation: A Recovery Complexity Perspective, ICML 2026.

Questions for authors
  1. What are the essential theoretical advances of BGEB compared with existing TTA analyses? Can the authors demonstrate that the bound is informative under realistic adaptation settings?

  2. How does the proposed generalization bound theoretically justify the specific components of SynerTTA, particularly gradient-sensitive parameter masking and reliability-weighted optimization?

  3. Could the authors clarify the numerical inconsistencies in Tables 1–2 and provide the exact experimental configurations used to reproduce the reported results?

3 comments

Q1: The essential theoretical advances of BGEB compared with existing TTA analyses

A1: We sincerely appreciate your constructive comments. The BGEB in our theoretical analysis advances existing TTA theory by providing a high-probability bound on target classification risk for source-free, non-stationary adaptation that accounts for finite-batch uncertainty and drift from prior updates while distinguishing class-prior estimation error from class-conditional shift. We've discussed those works in Appendix A: RELATED WORK and added them to our references.

Essential theoretical advance of BGEB. Compared with existing methods, BGEB gives a high-probability bound on target classification risk at each step of source-free online adaptation, allowing both class priors and class-conditional distributions to change over time. Its central contribution is to distinguish the effect of class-prior estimation error from that of changes within each class, while accounting for finite-batch uncertainty and drift from earlier updates in the same risk analysis. This connects the model’s ability to respond to the current distribution with the cumulative effects of adaptation over the stream. Existing TTA analyses provide guarantees for different settings and quantities. For example, AdaNPC (Zhang et al., ICML 2023 [4]) bounds the target excess risk of a nearest-neighbor classifier under covariate or posterior shift. Other analyses characterize collapse in recurring environments [1], local stability of projected updates [2], or post-shift recovery and learnability [3].

[1] Persistent Test-time Adaptation in Recurring Testing Scenarios, NeurIPS 2024.

[2] Lifelong Test-Time Adaptation via Online Learning in Tracked Low-Dimensional Subspace, NeurIPS 2025.

[3] On the Learnability of Test-Time Adaptation: A Recovery Complexity Perspective, ICML 2026.

[4] AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation, ICML 2023.


Q2: The bound is informative under realistic adaptation settings

A2: Yes, extensive experimental results clearly show advantages of the bounds under realistic adaptation settings. We tested whether BGEB could explain adaptation behavior under a realistic small-batch prevalence shift. To model a change in disease prevalence during deployment, we constructed a two-stage stream from real MESSIDOR images, with different class proportions in the two stages. The results of the experiment are shown in the specific response and Appendix F.4.1. In this experiment, RWLA tracked them most accurately over the full stream and also achieved the highest pseudo-label accuracy, which proves our bound is informative under this constructed scenario.


Q3:How does the proposed generalization bound theoretically justify the specific components of SynerTTA, particularly gradient-sensitive parameter masking and reliability-weighted optimization?

A3: Every part of our algorithm connects with the theoretical bound BGEB, and we explain the details about gradient-sensitive parameter masking(GSPM) and reliability-weighted optimization in the specific response.


Q4: The numerical inconsistencies in Tables 1–2 and the exact experimental configurations used to reproduce the reported results

A4: Thank you for flagging the typos in Tables 1 and 2. We've double-checked the whole paper and corrected them.

As for Table 1, we recomputed the overall averages as unweighted means across the eight datasets and reproduced the values shown in Table 1. The improvements stated in the abstract and main text were incorrect. Based on the displayed averages, SynerTTA improves Accuracy by 8.51 percentage points over StatA (48.83% vs. 40.32%) and Macro-F1 by 5.71 percentage points over DOTA (41.31% vs. 35.60%). We will correct these comparisons in the text.

As for Table 2, some ablation rows excluded NCT-CRC, while the modality scores for full SynerTTA included it. We have added the missing NCT-CRC ablation results and recomputed every row over the same eight datasets. The revised Overall Avg. gives each dataset equal weight, corresponding to a 3:3:2 weighting of the ophthalmology, histology, and chest X-ray means. The Source and full SynerTTA summaries now match the corresponding per-dataset results in Table 1. Full SynerTTA achieves the highest overall Accuracy (48.83%) and ties the “w/o adapt” variant on overall Macro-F1 at the reported precision (41.31%). We will update the ablation discussion accordingly.

Dear Reviewer rM5i,

I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper!

Best regards,

Authors of Submission 16610

Dear Reviewer rM5i,

Thank you again for your thoughtful and constructive comments. With about 4 hours remaining in the discussion period, we would like to check whether our response has addressed your concerns. We briefly summarize the corresponding response in the revised draft (new content in blue) and all responses:

  • W1 (Insufficient theoretical novelty and justification): We discussed the contribution and novelty of our method from the perspective of theory and experiments. It is the first work that gives a high-probability bound on target classification risk at each step of source-free online adaptation, allowing both class priors and class-conditional distributions to change over time.
  • W2,Q3 (Inconsistencies in experimental reporting): Thanks for your careful review. We've double-checked the whole paper and corrected this typo.
  • Q1 (Essential theoretical advances): Compared with existing methods, the central contribution of our BGEB is to distinguish the effect of class-prior estimation error from that of changes within each class, while accounting for finite-batch uncertainty and drift from earlier updates in the same risk analysis. This connects the model’s ability to respond to the current distribution with the cumulative effects of adaptation over the stream.
  • Q2 (Theoretical bound justify the specific components of SynerTTA): The theoretical bound (Eq.3) provides a concrete rationale for all three modules of the implemented algorithm SynerTTA (Sec.2.4), guiding how we estimate the target prior, select parameters to update, and balance pseudo-label fitting with source-feature anchoring. Table 1 summarizes these connections. All these connections and experimental evidence show that the theoretical analysis offers a rigorous foundation for SynerTTA design, and each components of SynerTTA actually reduce the bound.

We believe these additions address the points you raised and make the paper's claims more precise and better supported. If any concerns remain, we would greatly appreciate the opportunity to respond to them before the discussion closes. If you feel your concerns have been resolved, we would be sincerely grateful if you would consider raising your score.

Thank you again for your time and valuable feedback!

Best regards,

Authors of submission 16610

Reviewer_DwEVOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the stability-plasticity dilemma in non-stationary test-time adaptation (TTA) for medical imaging. From a Bayesian perspective, they decompose the predictive posterior into a class prior and a class-conditional likelihood, leading to a Bayesian Generalization Error Bound (BGEB) that factorizes target risk into finite-batch uncertainty, prior mismatch, conditional shift, and parameter drift. Then, the auhtors propose SynerTTA by integrating online prior rectification (RWLA), gradient-sensitive parameter masking (GSPM), and a reliability-weighted dynamic loss. Experiments on eight medical benchmarks across three modalities demonstrate that SynerTTA outperforms thirteen baselines.

Critical strengths

1, The theoretical analysis for TTA offers a rigorous foundation for understanding why existing methods fail and how to balance trade-offs. The paper provides the a Bayesian decomposition of the stability-plasticity dilemma in non-stationary TTA, deriving a principled generalization bound (BGEB) that explicitly separates four contributing factors.

2,Unlike many heuristic TTA methods, SynerTTA is directly motivated by the BGEB. Each component (prior correction, parameter masking, reliability weighting) targets a specific term in the bound, creating a clean connection between theory and practice.

3, The method achieves substantial and consistent improvements over a wide range of baselines across three distinct medical modalities. The Bayesian calibration analysis provides convincing evidence that SynerTTA produces more reliable probability estimates.

4, The experiments cover multiple realistic challenges—small batches, class imbalance, mixed clinical centers, and cross-dataset streams—making the results practically relevant for real-world clinical deployment.

Critical weaknesses

1, There is a gap between theory and algorithm: While BGEB identifies five terms, SynerTTA only directly addresses prior mismatch and parameter drift. The conditional shift term (Δcond) is ignored entirely, and the finite-batch uncertainty term is only implicitly handled by masking. The paper does not quantify how much of the bound is actually reduced by each component.

2, Selecting parameters with the smallest gradient norms minimizes instantaneous displacement, but may also prevent adaptation of genuinely important parameters. The paper lacks ablation comparing this strategy against random selection or largest-gradient selection, leaving the rationale unconvincing.

3, The use of negative entropy as a reliability indicator is problematic because low entropy does not imply correct pseudo-labels. Overconfident wrong predictions receive high weight, potentially amplifying confirmation bias. No mechanism (e.g., thresholding, consistency regularization) is introduced to mitigate this.

4, There is a lack of hyperparameter sensitivity and computational cost. The update ratio ρ, temperature τ, and learning rate are fixed across all datasets without justification. Additionally, the “lightweight” claim is unsupported—no runtime or memory comparisons with baselines are provided, despite GSPM requiring gradient computation and sorting for all parameter groups.

5, Table 1 reports only single-run means without standard deviations or confidence intervals, making it impossible to assess the significance of the reported improvements.

Minor issue: the texts in Figure 1 are too small to read.

Comments after author's apply: The authors provided more evidence about the effectives of each component in reducing the error gap, and showed the efficiency of the proposed method. Most of the pervious concerns have been addressed. So I raise my score from 3 to 4.

Questions for authors

1, The efficiency and computational cost is not reported. It is expected to compare the efficiency with the existing methods.

2, The terms mentioned in the theoretical analysis are not fully matched with the designed components. It would be good to discuss this gap.

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (part1/2)

Q1: Efficiency and computational cost.

A1: Thank you for your constructive suggestion. We conducted experiments on computational cost, which demonstrate that our SynerTTA has a significant advantage in terms of computational efficiency compared with SOTA methods. In Table R1-1, we report per-batch latency and peak allocated GPU memory on MESSIDOR for SynerTTA, the frozen Source model, TENT, StatA, TNT, and RoTTA, with three runs per method.

Table R1-1 Comparison of computational cost.

MethodMean latency (ms/batch)P95 latency (ms/batch)Peak allocated GPU memory (GiB)
Source13.39 ± 0.1819.24 ± 0.141.29 ± 0.02
TENT131.07 ± 4.78151.54 ± 3.663.96 ± 1.25
StatA72.14 ± 4.42102.11 ± 9.253.35 ± 0.00
TNT1827.88 ± 16.292009.65 ± 47.629.75 ± 2.26
RoTTA277.30 ± 272.972589.22 ± 3241.898.77 ± 0.00
SynerTTA62.76 ± 4.28100.74 ± 14.553.26 ± 2.41

Under this setup, SynerTTA has the lowest mean latency among the evaluated adaptation methods at 62.76 ± 4.28 ms per batch, compared with 72.14 ± 4.42 ms for StatA and 131.07 ± 4.78 ms for TENT. Its peak allocated GPU memory is 3.26 ± 2.41 GiB.


Q2: Hyperparameter sensitivity

A2: The hyperparameter sensitivity has been presented in the appendix, page 24. The results show that our SynerTTA is not sensitive to the hyperparameters. Appendix E (Fig. 3) reports sensitivity to the GSPM update ratio (ρ\rho) and the RWLA prior-correction strength (τ\tau). COVID and MESSIDOR reached their highest accuracy at (ρ=0.45\rho=0.45), while NCT-CRC degraded at larger update ratios. FIVES and MMAC were relatively stable across a broad range of (τ\tau), whereas COVID was more sensitive to prior correction. A learning-rate sweep for SynerTTA remains a gap in the current evaluation.


Q3: The terms mentioned in the theoretical analysis are not fully matched with the designed components. It would be good to discuss this gap. A3: Thanks for your comments. As the specific response, the theoretical bound (Eq.3) provides a concrete rationale for all three modules of the implemented algorithm SynerTTA (Sec.2.4), and the experimental results show that each components of SynerTTA actually reduce the bound in the theoretical analysis. In addition, we didn't ignore the conditional shift term (Δcond)(\Delta_{cond}), and we also supplement ablation experiments that quantify how much of the bound is actually reduced by each component. The details of the connection between the theoretical bound and our SynerTTA have been shown in the specific responese.


Q4: Parameter selection with the smallest gradient norms and ablation comparing this strategy against random selection or largest-gradient selection

A4: The GSPM is designed to reduce the bound of Finite-batch uncertainty (UtU_t) and Class-conditional shift (Δcond\Delta_{\mathrm{cond}}), which often yields a balance between stability and plasticity. Selecting parameters with the smallest gradient norms is an optimal choice for this balance, because the smallest gradient norms provide a relatively conservative update beneficial to stability, although they may prevent adaptation of genuinely important parameters and are harmful to plasticity. Ablation experiments compared against random selection and largest-gradient selection in Table R1-2 show the advantages of our strategy.

Table R1-2. Mask selection experiments on MESSIDOR.

MetricGSPMRandom updatesLargest-gradient updatesFull updates
One-step AdamW displacement (ℓ2\ell_2 norm)0.04150.04780.05280.0554
Signed soft-disagreement gap (percentage points)−1.43−0.39+0.45+0.62
Online Macro-F152.60%43.64%43.69%39.52%

AuthorsOpenReview ↗

Official Comment by Authors (part2/2)

Q5: The use of negative entropy as a reliability indicator is problematic because low entropy does not imply correct pseudo-labels and also potentially amplifies confirmation bias.

A5: Thanks for pointing this problem. Actually, this is our method's limitation. However, from the ablation experiments, the impact of this is not very significant, but thank you for raising the point; we will explore it in our future work.

Dear Reviewer DwEV,

I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper!

Best regards,

Authors of Submission 16610

Reviewer_DwEVOpenReview ↗

Some clarifications about method and results have been provided in the reply.

Thanks for the detailed reply. It is good to see the analysis of computational cost and the proposed method remain efficient compared with existing method. More ablation study have been provided in the revised version.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Specific response for the connection between theoretical bound and algorithm SynerTTA (part 1/3)

Q. Connections between theory and algorithm

A: We sincerely appreciate the reviewer's careful scrutiny and constructive comments. We first address the shared concern here by clarifying how the theoretical bound informs the design of SynerTTA. The theoretical bound (Eq.3) provides a concrete rationale for all three modules of the implemented algorithm SynerTTA (Sec.2.4), guiding how we estimate the target prior, select parameters to update, and balance pseudo-label fitting with source-feature anchoring. Table 1 summarizes these connections. All these connections and experimental evidence show that the theoretical analysis offers a rigorous foundation for SynerTTA design, and each components of SynerTTA actually reduce the bound.

Table 1. Correspondence between the bound in Eq. (3) and the modules of SynerTTA.

Term in the boundSynerTTA moduleFunction
Finite-batch uncertainty (UtU_t)GSPMRestricts the loss class in the complexity term.
Source anchor (StS_t)—Fixed during the parameter update given π^t\hat{\pi}_t because there are many sucessful approaches dealing with.
Current fitting (AtA_t) and model drift (DtD_t)Reliability-aware adaptation objectiveBalances pseudo-label fitting (AtA_t) and source-feature anchoring (DtD_t).
Prior mismatch (Δprior\Delta_{\mathrm{prior}})RWLAUpdates the target prior estimate.
Class-conditional shift (Δcond\Delta_{\mathrm{cond}})GSPM (design motivation)Motivates the choice of updating normalization affine parameters.
  • RWLA in SynerTTA directly targets prior mismatch (Δprior)(\Delta_{\mathrm{prior}}) of the bound. It updates the target prior estimate (π^t\hat\pi_t) in Δprior(t)=DTV(πt,π^t)\Delta_{\mathrm{prior}}(t)=D_{\mathrm{TV}}(\pi_t,\hat\pi_t) through entropy-weighted smoothing. The fixed-step experiment below shows that smoothing reduces mean prior mismatch relative to both baselines.

    Experimental evidence: To further highlight the RWLA's performance, we compare it with the fixed-step RWLA (α=0.05\alpha=0.05) with per-batch replacement and a fixed uniform prior. Table 2 shows that RWLA achieved lower prior TV than per-batch replacement and performed similarly to the uniform prior. Over the full stream, RWLA’s prior smoothing reduced prior mismatch Δprior\Delta_{\mathrm{prior}} and improved pseudo-label accuracy relative to both baselines.

    Table 2. Target-prior estimation and pseudo-label accuracy.

    MetricRWLAPer-batch replacementFixed uniform prior
    Mean prior DTVD_{TV} over the full stream0.25500.49600.3600
    Mean prior DTVD_{TV} over the first 10 batches after the switch0.31770.462390.3200
    Pseudo-label accuracy over the full stream60.13%56.71%51.75%
AuthorsOpenReview ↗

Specific response for the connection between theoretical bound and algorithm SynerTTA (part 2/3)

  • GSPM in SynerTTA restricts the loss class that enters the complexity component(UtU_t) of finite-batch uncertainty of the bound. Starting from the same pre-update model, consider the full adaptation class in which all candidate normalization affine parameter groups are eligible for updates. Under the masked gradient update in Eq. (10), a fixed mask sets the update to zero for masked-out groups. With the same pseudo-labeler class and loss, this gives $\mathcal L_{\mathrm{masked}}\subseteq\mathcal L_{\mathrm{full}} \quad\Longrightarrow\quad \mathfrak R_{B_t}(\mathcal L_{\mathrm{masked}}) \le \mathfrak R_{B_t}(\mathcal L_{\mathrm{full}})$

The inequality follows because Rademacher complexity takes a supremum over the loss class. When the mask is selected from the current mini-batch, the analysis must cover the union of the loss classes induced by all candidate masks. The same comparison applies when that union is contained in the common full adaptation loss class.

Class-conditional shift (Δcond)(\Delta_{\mathrm{cond}}) motivates the choice of normalization affine parameters. Adjusting the scales and biases of normalization layers lets the model adapt its representation of target inputs under class-conditional shift. GSPM selects which of these parameter groups to update. Given πt^\hat{\pi_t}, the source and target class-conditional distributions, Δcond(t)\Delta_{\mathrm{cond}}(t) remains fixed during the parameter update at step tt.

Experimental evidence: We compare it with random selection, largest-gradient selection, and full normalization updates in paired one-step replays and online adaptation on MESSIDOR (B=5B=5). Experimental results in Table 3 show that GSPM improved online Macro-F1 by 8.95 and 8.90 percentage points over random and largest-gradient selection, respectively, and by 13.07 points over full updates. The gains over both matched-budget controls show that GSPM’s choice of parameter groups improves online adaptation at a fixed update budget.

Table 3. Mask selection experiments on MESSIDOR.

MetricGSPMRandom updatesLargest-gradient updatesFull updates
One-step AdamW displacement (ℓ2\ell_2 norm)0.04150.04780.05280.0554
Signed soft-disagreement gap (percentage points)−1.43−0.39+0.45+0.62
Online Macro-F152.60%43.64%43.69%39.52%
AuthorsOpenReview ↗

Specific response for the connection between theoretical bound and algorithm SynerTTA (part 3/3)

  • The reliability-aware adaptation objective balances pseudo-label fitting and source-feature anchoring( AtA_t and DtD_t). The term AtA_t measures empirical disagreement between the adapted model and the pseudo-labeler on the current mini-batch. Equation (12) uses weighted cross-entropy as a surrogate for this fitting term by encouraging the model to assign higher probability to the pseudo-label class. The term DtD_t measures the parameter displacement of the pseudo-labeler relative to the source model. Motivated by this drift term DtD_t, Eq. (12) penalizes the squared distance between the adapted and frozen source models’ features for the same target input. This penalty constrains adaptation at the representation level. Samples with lower entropy under the rectified predictions receive greater weight in pseudo-label fitting, while samples with higher entropy receive greater weight in source-feature anchoring.
    Experimental evidence: We compare it with adaptation without source-feature consistency, fixed-weight consistency, and reliability-dependent weighting in SynerTTA. Experimental results in Table 4 show that source-feature anchoring improves adaptation across the evaluated task groups and that reliability-dependent weighting adds a gain on the first metric in each group.

    Table 4. Ablation of the reliability-aware adaptation objective.

    MethodOphth.Hist.CXR.Overall Avg.
    Source + GSPM + RWLA60.54 / 54.5837.29 / 27.9246.31 / 38.2748.26 / 40.51
    Source + GSPM + RWLA + Consistency (fixed weight)60.72 / 54.9337.81 / 28.8346.66 / 39.5848.61 / 41.31
    SynerTTA60.85 / 55.1238.08 / 28.7646.92 / 39.4148.83 / 41.31
AuthorsOpenReview ↗

Brief introduction to the revised PDF

We thank the reviewers for their feedback. We clarified how the Bayesian Generalization Error Bound (BGEB) informs SynerTTA, corrected the benchmark and ablation reporting, added controlled experiments, and expanded the related work. Revisions are highlighted in blue.

Core Contributions

  1. Diagnosing the stability–plasticity dilemma in non-stationary TTA. Learning accuracy, forgetting, and adaptation-strength experiments show how adapting to the current target stream can erode performance on earlier distributions.
  2. A Bayes-factorized bound on target risk. BGEB separates prior mismatch and class-conditional shift from pseudo-label fitting, finite-batch uncertainty, source risk, and parameter drift. Under a local Lipschitz assumption, we bound forgetting in terms of cumulative parameter path length.
  3. SynerTTA, a theory-guided adaptation method. RWLA tracks the target prior, GSPM selects normalization parameter groups, and the reliability-aware objective balances pseudo-label fitting with source-feature anchoring. SynerTTA leads the eight-dataset benchmark in average Accuracy (48.83%) and Macro-F1 (41.31%).

Experimental Updates During Rebuttal

  • Sections 3.1–3.2, Tables 2–3 (R2). We added NCT-CRC to the benchmark description, completed its missing ablations, and averaged each variant equally over all eight datasets. The revised text reports gains of 8.51 percentage points in Accuracy over StatA and 5.71 points in Macro-F1 over DOTA, and corrects the MESSIDOR, FIVES, and Covid comparisons.
  • Appendix F.4.1, Table 5 (R2). Under an abrupt prior shift on MESSIDOR, fixed-step RWLA reduces full-stream mean total variation distance to 0.2550, versus 0.4960 for per-batch replacement and 0.3600 for a uniform prior. It reaches 60.13% pseudo-label accuracy. Its prior mismatch is close to the uniform prior’s over the first ten batches after the switch.
  • Appendix F.4.2, Table 6 (R1, R3). In paired replays on MESSIDOR, GSPM yields the smallest AdamW displacement and the lowest signed soft-disagreement gap. Its online Macro-F1 is 52.60%, versus 43.64% for random masking and 43.69% for largest-gradient masking at the same budget, and 39.52% for full normalization updates.
  • Appendix F.4.3, Table 7 (R2, R3). Fixed-weight source-feature anchoring improves Accuracy and Macro-F1 across all three modalities. Reliability-dependent weighting further raises overall Accuracy from 48.61% to 48.83%, with both weighting schemes reaching 41.31% overall Macro-F1.
  • Appendix F.5, Figure 8 and Tables 8–9 (R3). We added gradual photometric drift on MESSIDOR with a fixed five-class label space and no task or boundary signals. SynerTTA reaches 59.93% online Macro-F1, 5.02 percentage points above frozen Source, and 56.53% under shuffled order. On the fixed held-out set, its Macro-F1 falls from 71.03% to 58.97% after the shift and to 54.12% after the stream returns to its original appearance.
  • Appendix F.6, Table 10 (R1). On MESSIDOR, SynerTTA has the lowest observed mean and P95 latency among adaptive methods (62.76 and 100.74 ms per batch) and averages 3.26 GiB of peak allocated GPU memory across three runs. Timings include GSPM’s scoring, sorting, and masking.

Discussion Updates During Rebuttal

  • Equation (3), Section 2.5, and Table 1 (R1–R3). We labeled the BGEB terms and mapped them to the method. RWLA addresses prior mismatch. Conditional shift motivates adapting normalization scales and biases, and GSPM restricts the loss class in the finite-batch complexity term. The Rademacher complexity comparison applies when the full loss class contains the union over possible masks under a common setup. The reliability-aware objective uses weighted cross-entropy for fitting and source-feature anchoring to limit representation drift. Given the prior estimate, the source-risk anchor is fixed within each update. For given class-conditional distributions, conditional mismatch is also fixed.
  • Appendix A, Related Work (R2). We compare PeTTA’s use of source feature statistics with per-image anchoring and LCoTTA’s recent-gradient subspace with GSPM’s batch-dependent mask. We also explain how recovery time and stream-level learnability complement BGEB’s per-step risk bound.

We appreciate the reviewers’ suggestions, which helped us clarify the theory and test SynerTTA’s design choices.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.