Federated Test-Time Adaptation without Waiting: Asynchronous Collaboration in Dynamic Environments

Federated Learning (FL) supports privacy-preserving learning by enabling clients to train models collaboratively without sharing raw data. However, deployed models may encounter unseen and dynamically changing test distributions. Test-Time Adaptation (TTA) mitigates this distribution shift by adapting models online using unlabeled test data. Federated Test-Time Adaptation (FedTTA) extends TTA to FL by allowing clients to share adaptation knowledge. Most existing FedTTA methods rely on synchronous collaboration. They update shared information until all participating clients complete local adaptation, forcing the system to wait for stragglers. To circumvent this limitation, we consider asynchronous FedTTA without synchronization waiting. However, asynchronous updates may cause fast clients to dominate the shared state, while erroneous updates can directly affect subsequent clients. To address these problems, we propose an asynchronous FedTTA method for dynamic environments. Our method uses a dynamic shared memory to reduce the cumulative influence of repeated updates from fast clients and isolates shared knowledge from local TTA to limit error propagation. The local adaptation process remains unchanged, making our method compatible with most existing TTA methods. Across the evaluated datasets and architectures, our method reduces measured wall-clock time by 44.6%–64.2% compared with synchronous FedTTA.