- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RMEMSAFE, a reliability-gated framework for continual test-time adaptation. It uses the frozen source model's normalized predictive entropy to dynamically modulate a parameter anchor and source-expert agreement weighting, while incorporating several auxiliary stabilizers. Experiments on CCC and long-horizon corruption streams demonstrate improvements under certain distribution shifts, particularly in challenging adaptation scenarios.
Critical strengths
-
The paper addresses an important challenge in CTTA: balancing source knowledge preservation and adaptation under distribution shifts. The proposed reliability-gated mechanism is intuitive and easy to integrate into existing frameworks.
-
The experiments cover multiple benchmarks, model architectures, and challenging adaptation scenarios. The matched controls and ablation studies provide insights into the behavior of the proposed method.
Critical weaknesses
-
The core idea of this paper overlaps with prior studies. VCoTTA [1] has already explored using the source model's predictive entropy to adaptively adjust the mixing weights of the source and teacher priors. OSTTA [2] dynamically controls the adaptation process by comparing the prediction confidence of the original source model and the adapted model. Therefore, the authors should better clarify the novelty of their approach and provide experimental comparisons with these closely related methods.
-
As shown in Tab. 1, adding decoupled flip inference reduces the mean error from 61.28% to 59.43%. However, flip inference is an established test-time augmentation technique and is not directly related to the proposed source reliability gating mechanism. In contrast, dynamic gating provides marginal improvements over the fixed gating. These results suggest that most of the performance gains of the full method come from auxiliary components, while the independent contribution of reliability gating is relatively limited. The authors should further justify the effectiveness of the proposed mechanism and provide more comprehensive ablation studies.
-
The use of predictive entropy as a measure of source reliability is not sufficiently justified. When the source model has low accuracy, the proposed reliability metric may fail to accurately reflect its actual performance degradation. The authors should evaluate the effectiveness of the proposed mechanism when the source model makes confidently incorrect predictions.
[1] Variational Continual Test-Time Adaptation
[2] Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization
Questions for authors
-
What are the key methodological differences between RMEMSAFE and prior works such as VCoTTA and OSTTA?
-
Can the authors provide further ablations to isolate the contribution of reliability gating from auxiliary components?
-
How does the proposed gating mechanism behave when the source model makes confidently incorrect predictions?
Official Review of Submission45384 to Reviewer dYSN
Thanks to the author for the detailed rebuttal. While I appreciate the clarifications, my main concerns remain unresolved. Therefore, I will keep my original rating.