- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an asynchronous Federated Test-Time Adaptation (FedTTA) framework to address synchronization delays caused by clients with different processing speeds. The method introduces a dynamic shared memory with residual updates to reduce the influence of fast clients. It also separates shared knowledge from local adaptation to limit error propagation. Experiments on CIFAR10-C, CIFAR100-C, and ImageNet-C show improved accuracy and reduced runtime compared with the evaluated baselines.
Critical strengths
- The paper addresses a practical problem in FedTTA, particularly the challenges of heterogeneous client speeds and unreliable updates in asynchronous collaboration.
- The proposed framework is compatible with different local TTA methods without modifying their original adaptation procedures.
- The experiments cover multiple datasets, model architectures, and TTA methods. The results show consistent performance improvements over the selected baselines.
Critical weaknesses
- The technical novelty is limited. Equation (9) uses an existing delta-rule update, while Equation (6) applies weighted prediction fusion. The authors should clarify what is fundamentally new compared with existing memory-based FedTTA methods.
- Section 5 lacks key ablation studies. The effects of residual updates, environment matching, and knowledge isolation are not evaluated separately. A baseline using prediction fusion without shared memory is particularly important to establish the contribution of cross-client knowledge sharing.
- The proposed residual update does not necessarily prevent fast client dominance when clients provide conflicting information. Figure 3 only considers a fixed 1:8 speed ratio. More experiments with varying speed ratios and conflicting client distributions are needed to support this claim.
- Table 1 only compares against constructed baselines. Direct comparisons with existing FedTTA methods, such as FedTSA, FedCTTA, and BPFedCTTA, would provide stronger evidence of the method's advantages.
- The efficiency evaluation is limited. Section 5.1 simulates client speeds, and Section 5.5 only reports runtime comparisons with synchronous FedTTA. Communication costs, memory overhead, and runtime comparisons with asynchronous baselines should also be included.
- The experimental results contain inconsistencies. For example, Table 1 reports 77.68% for Tent on CIFAR10-C, while Tables 3 and 5 report 77.52% under apparently identical settings. These differences should be explained.
Questions for authors
see weeknesses
3 comments
Response Part 2
Response to W3
Residual updates may not prevent fast-client dominance under conflicting client corrections. Figure 3 only uses a fixed 1:8 speed ratio. Can the authors evaluate varying speed ratios and conflicting client distributions?
We expanded the speed-heterogeneity and conflict evaluations to examine asynchronous collaboration under more challenging conditions. We extended the original 1:8 speed ratio to five settings: 1:1, 1:2, 1:4, 1:8, and 1:16. In addition, we considered conflicting client corrections at three conflict levels: 20%, 30%, and 40%. Detailed experimental settings and results are provided in our response to Reviewer 1, Q2 (Tables B and C).
Response to W4
Table 1 compares only constructed baselines. Can the authors directly compare with existing FedTTA methods, including FedTSA, FedCTTA, and BPFedCTTA?
Our baselines enable controlled comparisons of synchronous and asynchronous collaboration under a unified experimental framework. Regarding baseline selection, we clarify the differences between our research setting and existing FedTTA methods. Our work considers multiple clients performing local adaptation concurrently at different processing speeds, with the server processing each arriving update immediately without waiting for other clients.
Specifically, FedTSA and FedCTTA primarily organize knowledge aggregation around communication rounds, whereas BPFedCTTA adopts sequential client participation and global model updates. These procedures differ from our setting of concurrent adaptation and interleaved asynchronous updates. Directly applying these methods would require modifying their original collaboration mechanisms, potentially affecting performance and comparison fairness.
Therefore, we construct four controlled baselines under a unified experimental framework: Local TTA evaluates the benefits of cross-client collaboration; SFTTA provides a synchronous reference; and AFTTA and FATTA evaluate direct asynchronous aggregation and staleness-aware aggregation, respectively. All methods use the same experimental settings to ensure fair comparisons.
Response to W5
Section 5.1 simulates client speeds, while Section 5.5 compares runtime only with synchronous FedTTA. Can the authors include communication costs, memory overhead, and runtime against asynchronous baselines?
Our method achieves the shortest total runtime and lower communication overhead across the evaluated settings. To further evaluate runtime efficiency and resource overhead, we extended our experiments to include the asynchronous baselines AFTTA and FATTA alongside the synchronous baseline SFTTA, under identical hardware configurations, test data streams, and client speed heterogeneity settings.
We evaluate total runtime, uplink and downlink communication volumes, server processing time, and peak RAM usage, including shared memory transmission in the downlink overhead. The experiments cover four dataset and model configurations using Tent as the local TTA method, with additional evaluations using SAR and DeYO.
Peak RAM usage remains broadly comparable to the baselines. Although shared memory maintenance increases server processing time in some configurations, the overall runtime advantage remains. Detailed results are provided in our response to Reviewer 1, Q3 (Table E), with SAR and DeYO results included in the appendix.
Response to W6
Table 1 reports 77.68% for Tent on CIFAR10-C, but Tables 3 and 5 report 77.52% under apparently identical settings. What explains the inconsistency?
The numerical inconsistency arose from an oversight in synchronizing the main manuscript and appendix, which we have corrected in the revision.
To further verify the stability and reliability of the experimental results, we conducted multiple independent runs using different combinations of random seeds and consistently adopted the results from a complete experimental run with relatively stable performance. In this run, the accuracy on CIFAR10-C using Tent was 77.68%, while the appendix tables still retained the earlier result of 77.52%.
We apologize for this oversight and have carefully checked and unified the corresponding experimental results throughout the revised manuscript and appendix. Further details are provided in our response to Reviewer 1, Q4.
We hope our revisions and responses have adequately addressed the reviewer's concerns, and we sincerely hope the reviewer will consider raising the score. We would be happy to provide further clarification if there are any questions.
Thank you for the authors' response and clarifications. I appreciate their efforts in addressing my concerns, but I will maintain my original score.
Response Part 1
We thank the reviewer for the constructive feedback. We provide further clarifications and experimental evidence to address the concerns raised.
Response to W1
Our novelty lies in combining knowledge updating and prediction isolation to address the specific challenges of asynchronous FedTTA, rather than introducing new techniques. To the best of our knowledge, this is the first work to investigate asynchronous FedTTA with multiple clients performing local adaptation concurrently. We focus on two key challenges: biased knowledge contributions caused by heterogeneous client update frequencies and cross-client error propagation under heterogeneous environments. Although the delta-rule and weighted prediction fusion are not new techniques themselves, our contribution lies in designing complementary mechanisms to address these challenges in asynchronous collaboration.
Existing FedTTA methods have explored knowledge sharing. FedTSA [1] maintains historical feature statistics on the server and computes cross-client aggregation weights through spatiotemporal attention. Latte [2] retrieves class prototypes from other clients based on feature similarity and uncertainty, while separating communication from local adaptation. However, these methods do not primarily investigate heterogeneous upload frequencies and interleaved updates from concurrently adapting clients. Our method addresses these issues through environment-specific shared memory for environment matching and the following two mechanisms.
(1) Reducing the cumulative influence of redundant information from frequent fast-client uploads. In asynchronous FedTTA, faster clients upload more frequently, potentially causing similar information to accumulate repeatedly and giving them excessive influence over shared knowledge. We use the fixed shared model as a common reference and share prediction corrections produced by local adaptation. Eq. (9) adopts the existing delta-rule to update shared memory based on the difference between the newly uploaded correction and the current memory output. When similar information is already represented in memory, the update magnitude decreases, thereby reducing the cumulative influence of redundant uploads.
(2) Preventing erroneous shared knowledge from being repeatedly learned and propagated through local adaptation. If shared knowledge directly participates in local adaptation, erroneous information may be learned by local models and propagated through subsequent uploads. Although Eq. (6) adopts existing weighted prediction fusion, we restrict the fusion result to the current prediction stage, excluding it from local model initialization, adaptation loss computation, and pseudo-label generation. Clients perform local TTA independently, and uploaded corrections are computed from the local adaptation model and the fixed shared model. This prevents erroneous shared knowledge from entering local adaptation and being repeatedly propagated.
[1] Zhang, J., Liu, X., Zhang, Y., Zhu, G., Niu, J., and Tang, S. "Enabling Collaborative Test-Time Adaptation in Dynamic Environment via Federated Learning," KDD, 2024, pp. 4191–4202.
[2] Bao, W., Deng, R., Qiu, R., Wei, T., Tong, H., and He, J. "Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning," ICCV, 2025, pp. 133–143.
Response to W2
Our ablations show that cross-client knowledge sharing provides benefits beyond prediction fusion, while other components also contribute to performance. We conducted controlled ablation experiments on CIFAR10-C and CIFAR100-C using Tent as the local TTA method. We consider five ablation variants to assess the effects of environment-specific shared memory, residual write-in, prediction fusion, local TTA isolation, and cross-client knowledge sharing.
In particular, to address the reviewer's concern regarding the contribution of cross-client knowledge sharing, we introduce the w/o SM variant, which removes cross-client shared memory while retaining prediction fusion between the local adaptation model and the fixed shared model. The results show that this variant achieves lower accuracy than the full method on both datasets, indicating that the performance improvements do not arise solely from prediction fusion and that cross-client knowledge sharing provides additional benefits. The other ablation results further validate the effectiveness of the individual components. Detailed experimental settings and results are provided in our response to Reviewer 1, Q1 (Table A).