- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper revisits the stability–plasticity dilemma in non-stationary test-time adaptation (TTA) and proposes SynerTTA, a framework combining online class-prior rectification, gradient-sensitive parameter masking, and reliability-aware adaptation. The authors derive a Bayesian-factorized generalization bound to motivate these components and evaluate the method across eight medical imaging benchmarks spanning ophthalmology, histopathology, and chest X-rays. The reported results show substantial improvements over several existing TTA approaches, particularly under class imbalance and small-batch adaptation.
Critical strengths
The paper addresses an important challenge in continual TTA: adapting to evolving distributions without compromising previously acquired knowledge. The stability–plasticity perspective is relevant to medical foundation models, where unreliable online updates can have significant consequences. Combining prior adjustment, selective parameter updates, and reliability-aware supervision is a reasonable approach to improving adaptation robustness. The proposed components are relatively lightweight and can be applied to different pretrained architectures. The experiments cover multiple medical modalities, datasets, and distribution-shift conditions. The inclusion of class-imbalance, batch-size, and forgetting analyses provides useful evidence beyond conventional average-accuracy comparisons.
Critical weaknesses
- The connection between the theoretical bound and the implemented algorithm is not sufficiently established. The Bayesian-factorized bound provides a useful decomposition of target risk, but several of its terms, including source-reweighted risk and class-conditional discrepancy, are unobservable during deployment. It remains unclear whether the bound provides actionable guidance beyond motivating familiar regularization and pseudo-labeling strategies.
More importantly, the displacement argument for GSPM in Eqs. (10)–(11) assumes a masked gradient-descent update, whereas the experiments use AdamW with persistent optimizer states. Selecting parameters with small gradient norms does not necessarily minimize the resulting AdamW parameter displacement. The authors should clarify how the theoretical argument applies to the actual optimization procedure and provide evidence that the proposed components meaningfully control the quantities identified by the bound.
- The posterior correction in Eq. (6) is exact under label shift, assuming calibrated source predictions and a known source class prior. However, the paper explicitly targets settings involving simultaneous changes in class prevalence and image appearance, where these assumptions may not hold.
This is particularly relevant for medical vision-language models, whose similarity-based predictions are not necessarily calibrated class posteriors. The definition and estimation of the effective source prior are also insufficiently explained. Furthermore, using predictive entropy to determine the reliability of online prior estimates may reinforce confidently incorrect predictions under severe distribution shifts.
The authors should evaluate the accuracy of the estimated class priors and investigate how RWLA behaves when label shift and class-conditional shift occur simultaneously.
- In the main benchmark, each dataset is processed as an independent stream, and the model is reset before moving to another dataset. Processing a fixed target dataset sequentially does not, by itself, establish that the underlying distribution changes over time. Although the supplementary experiments introduce sequential tasks and mixed-domain streams, their relationship to realistic non-stationary deployment remains unclear. In particular, the sequential-task evaluation activates task-specific label sets, potentially assuming knowledge of the current task identity.
A stronger evaluation would consider continuously evolving streams with unknown domain boundaries and a consistent prediction space, while explicitly measuring adaptation and forgetting across distribution transitions.
- Table 2 provides progressive ablations, but these do not establish whether the proposed components offer advantages over simpler alternatives. In particular, selecting low-gradient parameter groups should be compared with random masking, fixed parameter subsets, and learning-rate-controlled updates under comparable parameter displacement. Similarly, the contribution of reliability-aware supervision should be separated from that of prior correction. The final adaptive component provides a relatively modest improvement over the preceding variant, making its independent benefit less convincing. The overall performance advantage is also uneven across datasets. For example, the substantial gain on COVID-19 contrasts with considerably weaker performance than StatA on NCT-CRC. More detailed per-dataset uncertainty estimates and controlled comparisons would strengthen the empirical conclusions.
Questions for authors
How does the GSPM displacement argument extend to AdamW, and can the authors empirically connect the proposed updates to the terms in the generalization bound?
How is the source class prior determined for each foundation model, and how reliable is the online prior estimator when both label and conditional distributions change?
Can the authors evaluate SynerTTA on continuously changing target streams without predefined task boundaries or task-specific label selection?
4 comments
Official Comment by Authors (Part 2/2)
Q5: Using predictive entropy to determine the reliability of online prior estimates may reinforce confidently incorrect predictions under severe distribution shifts.
A5: To be honest, predictive entropy may reinforce confidently incorrect predictions under severe distribution shifts. However, it is a simpler choice for online prior estimation. Especially, ablation experiments indicate that the impact of entropy is not very significant in the distribution shifts.
In future work, we will explore more effective approaches for online prior estimation.
Q6: Evaluation of SynerTTA on continuously changing target streams without predefined task boundaries or task-specific label selection
A6: We evaluated SynerTTA on continuously changing target streams without predefined task boundaries or task-specific label selection, and The experimental results demonstrate that SynerTTA achieved the highest mean online Macro-F1 under gradual drift and the largest mean online accuracy gains over the frozen source model during both transitions in the constructed MESSIDOR stream, as image gamma gradually decreased from 1.0 to 0.7 and then gradually returned to 1.0. In the evaluation, a single MESSIDOR stream has gradual photometric drift. The model carried its state across the stream, received no task identities or boundary signals, and predicted over the same five classes throughout.
| Method | Drift Macro-F1 (%) | Shuffled Macro-F1 (%) | Outward gain (pp) | Return gain (pp) |
|---|---|---|---|---|
| Source | 54.91 ± 0.26 | 54.91 ± 0.26 | 0.00 ± 0.00 | 0.00 ± 0.00 |
| RoTTA | 52.43 ± 0.43 | 53.67 ± 0.23 | 14.67 ± 2.55 | 12.61 ± 1.72 |
| SynerTTA | 59.93 ± 2.00 | 56.53 ± 1.75 | 18.06 ± 5.82 | 19.76 ± 2.68 |
Values are mean ± standard deviation over three paired runs. Transition gains are online accuracy differences relative to frozen Source.
SynerTTA achieved the highest mean online Macro-F1 under gradual drift, exceeding Source by 5.02 percentage points, and achieved the largest mean accuracy gains in both transitions. These results show that SynerTTA can adapt to controlled drift without boundary signals or task-specific label selection. The appendix reports the full protocol, trajectories, and held-out forgetting and recovery results.
Q7: Ablations do not establish whether the proposed components offer advantages over simpler alternatives.
A7: We added comparative experiments that establish whether the proposed components offer advantages over simpler alternatives. The experimental results indicate that each component achieved the best performance compared to its corresponding simpler approach. The details can be seen in the specific response and in Appendix F.4.
Q8: Performance advantage on NCT-CRC.
A8: We noticed the better performance of StatA in the NCT-CRC dataset during running the experiments. We also found that some mainstream methods (such as TENT, CoTTA) show much worse performance even compared with the source model. And our SynerTTA also shares some similarities with those methods at the methodological level; for example, TENT and SynerTTA both employ the updating of normalized affine parameters. So we guess these similar points may be the reason of performance degradation, and we plan to explore this meaningful question in our future work.
Dear Reviewer Hq4z,
I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper!
Best regards,
Authors of Submission 16610
I’m raising my rating to 3. The new MESSIDOR experiment addresses a key concern in my initial review by showing that SynerTTA improves over the frozen model under gradual drift without task labels or boundary cues. I still have questions about long-term parameter drift and the accuracy of prior estimates under mixed shifts.
Official Comment by Authors (Part 1/2)
Q1:How does the GSPM displacement argument extend to AdamW?***
A1: We sincerely appreciate your constructive comments. The displacement argument is successfully extended to AdamW through an optimizer-independent bound that links the realized step norm to source-relative drift, with paired one-step replays showing that GSPM yields smaller steps than both matched-budget controls. Equations (10)–(11) motivate GSPM’s selection rule under masked gradient descent, where the step norm equals the learning rate times the norm of the retained gradient. The relationship between current gradient norms and realized step size depends on the optimizer. Under AdamW, the step also depends on the running first and second moments and weight decay, so selecting the groups with the smallest current gradient norms does not guarantee the smallest realized step. However, we tested whether GSPM still reduces displacement under AdamW using paired one-step replays on MESSIDOR. Starting from the same model, AdamW state, and batch, and selecting the same number of parameter groups per encoder, GSPM reduced the realized step norm by 13.18% relative to random selection and 21.36% relative to largest-gradient selection. By the triangle inequality, a smaller step from the same starting point tightens the one-step upper bound on source-relative drift, linking this result to (D_t).
Q2: The connection between the theoretical bound and the implemented algorithm is not sufficiently established.
A2: All these connections and experimental evidence show that the theoretical analysis offers a rigorous foundation for SynerTTA design, and each components of SynerTTA actually reduce the bound. The theoretical bound and SynerTTA algorithm are sufficiently established in our paper, and we also conducted supplementary ablation studies for each module of the algorithm. In the specific response, we explain the details.
Q3: How is the source class prior determined for each foundation mode?
A3: Actually, the source class prior can be estimated without accessing the training data, and this work has already been solved by others[1]. Zhu et al. (NeurIPS 2023)[1] shows how to estimate task-specific label bias from labeled downstream data without accessing the original pretraining data.
To emphasize the focus of our method, the source class prior is set to be determined, but it can be replaced by any estimated approach, as mentioned above.
In our implementation, the model-specific reference-prior correction is omitted, and only the target prior is estimated online from unlabeled predictions. Although source prior estimation is ignored, our method still gains better results compared with mainstream TTA. Evaluating this approach for medical foundation models is a direction for future work.
[1] Zhu, Beier, et al. "Generalized logit adjustment: Calibrating fine-tuned models by removing label bias in foundation models." Advances in Neural Information Processing Systems 36 (2023): 64663-64680.
Q4: how reliable is the online prior estimator when both label and conditional distributions change?
A4: The online prior estimator is relatively reliable when both label and conditional distributions change. Experimental results demonstrate the reliability of the online prior estimator when both label and conditional distributions change, in Table 2, page 9. The performance gains involving both shifts provide indirect evidence that the online prior estimates are useful for adaptation.