Reliability-Gated Source Anchoring for Continual Test-Time Adaptation

Continual test-time adaptation (CTTA) updates a pretrained model on an unlabeled, changing stream while using source information to limit drift. The challenge is to preserve useful source knowledge without letting an uncertain reference restrict adaptation. We propose RMemSafe, a reliability-gated extension of ROID that uses the frozen source's normalized predictive entropy to modulate a parameter anchor and source-expert agreement weighting. As the source posterior approaches uniformity, these explicit source-dependent loss contributions vanish while the base adaptation losses remain active. Matched controls separate this mechanism from the auxiliary stabilizers in the complete method. On CCC-Hard with ResNet-50, runtime gating reduces error by $0.30$ percentage points relative to the same objective with the gate fixed to one, and by $1.24$ points relative to removing the explicit source terms. On two long-horizon stream orderings, it reduces error by $0.58$ and $0.55$ points, with improvement in all $20$ evaluated seeds. Auxiliary stabilizers account for most of the average CCC improvement, and source-term removal remains stronger on the CCC mean. Source-degradation, small-batch, and class-permutation experiments further characterize the method and its limits. Together, these results motivate adjusting source influence to the current stream rather than assuming that retaining or removing it is always preferable.