acceptodds
Under review as a conference paper at ICLR 2027

Federated Test-Time Adaptation without Waiting: Asynchronous Collaboration in Dynamic Environments

Abstract

Federated Learning (FL) supports privacy-preserving learning by enabling clients to train models collaboratively without sharing raw data. However, deployed models may encounter unseen and dynamically changing test distributions. Test-Time Adaptation (TTA) mitigates this distribution shift by adapting models online using unlabeled test data. Federated Test-Time Adaptation (FedTTA) extends TTA to FL by allowing clients to share adaptation knowledge. Most existing FedTTA methods rely on synchronous collaboration. They update shared information until all participating clients complete local adaptation, forcing the system to wait for stragglers. To circumvent this limitation, we consider asynchronous FedTTA without synchronization waiting. However, asynchronous updates may cause fast clients to dominate the shared state, while erroneous updates can directly affect subsequent clients. To address these problems, we propose an asynchronous FedTTA method for dynamic environments. Our method uses a dynamic shared memory to reduce the cumulative influence of repeated updates from fast clients and isolates shared knowledge from local TTA to limit error propagation. The local adaptation process remains unchanged, making our method compatible with most existing TTA methods. Across the evaluated datasets and architectures, our method reduces measured wall-clock time by 44.6%–64.2% compared with synchronous FedTTA.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Learning, Test-Time Adaptation, Asynchronous Collaboration, Dynamic Environments

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wDSyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an asynchronous Federated Test-Time Adaptation (FedTTA) framework to address synchronization delays caused by clients with different processing speeds. The method introduces a dynamic shared memory with residual updates to reduce the influence of fast clients. It also separates shared knowledge from local adaptation to limit error propagation. Experiments on CIFAR10-C, CIFAR100-C, and ImageNet-C show improved accuracy and reduced runtime compared with the evaluated baselines.

Critical strengths
  1. The paper addresses a practical problem in FedTTA, particularly the challenges of heterogeneous client speeds and unreliable updates in asynchronous collaboration.
  2. The proposed framework is compatible with different local TTA methods without modifying their original adaptation procedures.
  3. The experiments cover multiple datasets, model architectures, and TTA methods. The results show consistent performance improvements over the selected baselines.
Critical weaknesses
  1. The technical novelty is limited. Equation (9) uses an existing delta-rule update, while Equation (6) applies weighted prediction fusion. The authors should clarify what is fundamentally new compared with existing memory-based FedTTA methods.
  2. Section 5 lacks key ablation studies. The effects of residual updates, environment matching, and knowledge isolation are not evaluated separately. A baseline using prediction fusion without shared memory is particularly important to establish the contribution of cross-client knowledge sharing.
  3. The proposed residual update does not necessarily prevent fast client dominance when clients provide conflicting information. Figure 3 only considers a fixed 1:8 speed ratio. More experiments with varying speed ratios and conflicting client distributions are needed to support this claim.
  4. Table 1 only compares against constructed baselines. Direct comparisons with existing FedTTA methods, such as FedTSA, FedCTTA, and BPFedCTTA, would provide stronger evidence of the method's advantages.
  5. The efficiency evaluation is limited. Section 5.1 simulates client speeds, and Section 5.5 only reports runtime comparisons with synchronous FedTTA. Communication costs, memory overhead, and runtime comparisons with asynchronous baselines should also be included.
  6. The experimental results contain inconsistencies. For example, Table 1 reports 77.68% for Tent on CIFAR10-C, while Tables 3 and 5 report 77.52% under apparently identical settings. These differences should be explained.
Questions for authors

see weeknesses

3 comments
AuthorsOpenReview ↗

Response Part 1

We thank the reviewer for the constructive feedback. We provide further clarifications and experimental evidence to address the concerns raised.

Response to W1

The technical novelty is limited: Eq. (9) uses an existing delta-rule update, and Eq. (6) uses weighted prediction fusion. What is fundamentally new compared with existing memory-based FedTTA methods?

Our novelty lies in combining knowledge updating and prediction isolation to address the specific challenges of asynchronous FedTTA, rather than introducing new techniques. To the best of our knowledge, this is the first work to investigate asynchronous FedTTA with multiple clients performing local adaptation concurrently. We focus on two key challenges: biased knowledge contributions caused by heterogeneous client update frequencies and cross-client error propagation under heterogeneous environments. Although the delta-rule and weighted prediction fusion are not new techniques themselves, our contribution lies in designing complementary mechanisms to address these challenges in asynchronous collaboration.

Existing FedTTA methods have explored knowledge sharing. FedTSA [1] maintains historical feature statistics on the server and computes cross-client aggregation weights through spatiotemporal attention. Latte [2] retrieves class prototypes from other clients based on feature similarity and uncertainty, while separating communication from local adaptation. However, these methods do not primarily investigate heterogeneous upload frequencies and interleaved updates from concurrently adapting clients. Our method addresses these issues through environment-specific shared memory for environment matching and the following two mechanisms.

(1) Reducing the cumulative influence of redundant information from frequent fast-client uploads. In asynchronous FedTTA, faster clients upload more frequently, potentially causing similar information to accumulate repeatedly and giving them excessive influence over shared knowledge. We use the fixed shared model as a common reference and share prediction corrections produced by local adaptation. Eq. (9) adopts the existing delta-rule to update shared memory based on the difference between the newly uploaded correction and the current memory output. When similar information is already represented in memory, the update magnitude decreases, thereby reducing the cumulative influence of redundant uploads.

(2) Preventing erroneous shared knowledge from being repeatedly learned and propagated through local adaptation. If shared knowledge directly participates in local adaptation, erroneous information may be learned by local models and propagated through subsequent uploads. Although Eq. (6) adopts existing weighted prediction fusion, we restrict the fusion result to the current prediction stage, excluding it from local model initialization, adaptation loss computation, and pseudo-label generation. Clients perform local TTA independently, and uploaded corrections are computed from the local adaptation model and the fixed shared model. This prevents erroneous shared knowledge from entering local adaptation and being repeatedly propagated.

[1] Zhang, J., Liu, X., Zhang, Y., Zhu, G., Niu, J., and Tang, S. "Enabling Collaborative Test-Time Adaptation in Dynamic Environment via Federated Learning," KDD, 2024, pp. 4191–4202.

[2] Bao, W., Deng, R., Qiu, R., Wei, T., Tong, H., and He, J. "Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning," ICCV, 2025, pp. 133–143.

Response to W2

Section 5 lacks separate ablations of residual updates, environment matching, and knowledge isolation. Can the authors add prediction fusion without shared memory to isolate the contribution of cross-client knowledge sharing?

Our ablations show that cross-client knowledge sharing provides benefits beyond prediction fusion, while other components also contribute to performance. We conducted controlled ablation experiments on CIFAR10-C and CIFAR100-C using Tent as the local TTA method. We consider five ablation variants to assess the effects of environment-specific shared memory, residual write-in, prediction fusion, local TTA isolation, and cross-client knowledge sharing.

In particular, to address the reviewer's concern regarding the contribution of cross-client knowledge sharing, we introduce the w/o SM variant, which removes cross-client shared memory while retaining prediction fusion between the local adaptation model and the fixed shared model. The results show that this variant achieves lower accuracy than the full method on both datasets, indicating that the performance improvements do not arise solely from prediction fusion and that cross-client knowledge sharing provides additional benefits. The other ablation results further validate the effectiveness of the individual components. Detailed experimental settings and results are provided in our response to Reviewer 1, Q1 (Table A).

AuthorsOpenReview ↗

Response Part 2

Response to W3

Residual updates may not prevent fast-client dominance under conflicting client corrections. Figure 3 only uses a fixed 1:8 speed ratio. Can the authors evaluate varying speed ratios and conflicting client distributions?

We expanded the speed-heterogeneity and conflict evaluations to examine asynchronous collaboration under more challenging conditions. We extended the original 1:8 speed ratio to five settings: 1:1, 1:2, 1:4, 1:8, and 1:16. In addition, we considered conflicting client corrections at three conflict levels: 20%, 30%, and 40%. Detailed experimental settings and results are provided in our response to Reviewer 1, Q2 (Tables B and C).

Response to W4

Table 1 compares only constructed baselines. Can the authors directly compare with existing FedTTA methods, including FedTSA, FedCTTA, and BPFedCTTA?

Our baselines enable controlled comparisons of synchronous and asynchronous collaboration under a unified experimental framework. Regarding baseline selection, we clarify the differences between our research setting and existing FedTTA methods. Our work considers multiple clients performing local adaptation concurrently at different processing speeds, with the server processing each arriving update immediately without waiting for other clients.

Specifically, FedTSA and FedCTTA primarily organize knowledge aggregation around communication rounds, whereas BPFedCTTA adopts sequential client participation and global model updates. These procedures differ from our setting of concurrent adaptation and interleaved asynchronous updates. Directly applying these methods would require modifying their original collaboration mechanisms, potentially affecting performance and comparison fairness.

Therefore, we construct four controlled baselines under a unified experimental framework: Local TTA evaluates the benefits of cross-client collaboration; SFTTA provides a synchronous reference; and AFTTA and FATTA evaluate direct asynchronous aggregation and staleness-aware aggregation, respectively. All methods use the same experimental settings to ensure fair comparisons.

Response to W5

Section 5.1 simulates client speeds, while Section 5.5 compares runtime only with synchronous FedTTA. Can the authors include communication costs, memory overhead, and runtime against asynchronous baselines?

Our method achieves the shortest total runtime and lower communication overhead across the evaluated settings. To further evaluate runtime efficiency and resource overhead, we extended our experiments to include the asynchronous baselines AFTTA and FATTA alongside the synchronous baseline SFTTA, under identical hardware configurations, test data streams, and client speed heterogeneity settings.

We evaluate total runtime, uplink and downlink communication volumes, server processing time, and peak RAM usage, including shared memory transmission in the downlink overhead. The experiments cover four dataset and model configurations using Tent as the local TTA method, with additional evaluations using SAR and DeYO.

Peak RAM usage remains broadly comparable to the baselines. Although shared memory maintenance increases server processing time in some configurations, the overall runtime advantage remains. Detailed results are provided in our response to Reviewer 1, Q3 (Table E), with SAR and DeYO results included in the appendix.

Response to W6

Table 1 reports 77.68% for Tent on CIFAR10-C, but Tables 3 and 5 report 77.52% under apparently identical settings. What explains the inconsistency?

The numerical inconsistency arose from an oversight in synchronizing the main manuscript and appendix, which we have corrected in the revision.

To further verify the stability and reliability of the experimental results, we conducted multiple independent runs using different combinations of random seeds and consistently adopted the results from a complete experimental run with relatively stable performance. In this run, the accuracy on CIFAR10-C using Tent was 77.68%, while the appendix tables still retained the earlier result of 77.52%.

We apologize for this oversight and have carefully checked and unified the corresponding experimental results throughout the revised manuscript and appendix. Further details are provided in our response to Reviewer 1, Q4.

We hope our revisions and responses have adequately addressed the reviewer's concerns, and we sincerely hope the reviewer will consider raising the score. We would be happy to provide further clarification if there are any questions.

Thank you for the authors' response and clarifications. I appreciate their efforts in addressing my concerns, but I will maintain my original score.

Reviewer_AaupOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a method for aggregating results of test-time adaptation across a federated learning environment. The method allows to better balance the contributions from fast and slow clients thus reducing the fast client dominance that can skew the overall training model. The core of the method are residual updates that are proportionally scaled to the speed of the client updates. Experimental results demonstrate its effectiveness.

Critical strengths

The paper very clearly articulates the case when overall model can be significantly affected under test-time adaptation. It clearly and formally explains the process of addressing it, and demonstrates experimental results that support the idea.

Critical weaknesses

The included experiments are a bit idealized. If I understand it correctly, the datasets used in testing the proposed method are basically evenly distributed across the clients ("each client sequentially experiences all 15 corruption types...") I would imagine that in a federated learning scenario different clients should be receiving samples specific to their environment and likely from non-overlapping datasets. I am not sure if the described experiments fit into this framework.

Another issue is with the volume of data. In a federated learning environment test data for different clients can be coming in very different volumes compared to other clients. In this case not just the processing speed of the client will have an impact on the accumulated residuals, but also how frequently clients themselves receive a new out of distribution data. I am not sure if the proposed method addresses this use case.

Questions for authors

I would consider the proposed method not as much as an attempt at improving federated leering as improving distributed learning in a more general sense. Would you agree that your method has a much broader application besides federated learning setup? If yes, what constraints does this remove?

2 comments

We appreciate the reviewer's feedback and address the concerns below.

Response to W1

The experiments appear idealized: test data are evenly distributed, and each client sequentially experiences all 15 corruption types. In federated learning, clients may receive environment-specific, non-overlapping datasets. Does the setting reflect this scenario?

Client test-data volumes and out-of-distribution arrival rates may differ substantially. How does the method account for their effects on accumulated residuals, beyond differences in processing speeds?

Our dynamic test streams follow related protocols while isolating the effects of heterogeneous processing speeds. For each corruption type, samples are partitioned without replacement across clients. Experiencing all 15 corruption types indicates changing local environments, not shared test samples.

Similar protocols appear in prior work. ATP [1] uses fixed data volumes across 240 source and 60 target clients, with 200 test samples per target client. CoLA [2], a multi-device TTA method, considers a collaborative setting where each device sequentially experiences all 15 corruption types in ImageNet-C, with different devices following different corruption orders. FedTSA [3] constructs dynamic test environments with 20 clients on CIFAR10-C and CIFAR100-C, simulating distribution shifts by switching corruption types over time and allowing different clients to follow different corruption sequences. Thus, our test streams follow established protocols while additionally modeling asynchronous updates from heterogeneous processing speeds.

Second, regarding differences in client test data volumes and data arrival rates, this work focuses on the FedTTA setting, where clients continuously receive unlabeled test data and perform local adaptation. We assign equal test volumes and simulate asynchronous collaboration through heterogeneous processing speeds. This controlled setting isolates the effects of data volume differences and focuses on how imbalanced upload frequencies affect shared knowledge updates. In practice, data volumes and arrival rates may also vary and affect upload frequencies. We have not specifically evaluated this more complex heterogeneous setting, which represents a potential direction for future research.

[1] Bao, W., Wei, T., Wang, H., and He, J. "Adaptive Test-Time Personalization for Federated Learning," NeurIPS, 2023.

[2] Chen, G., Niu, S., Chen, D., Zhang, S., Li, C., Li, Y., and Tan, M. "Cross-Device Collaborative Test-Time Adaptation," NeurIPS, 2024.

[3] Zhang, J., Liu, X., Zhang, Y., Zhu, G., Niu, J., and Tang, S. "Enabling Collaborative Test-Time Adaptation in Dynamic Environment via Federated Learning," KDD, 2024, pp. 4191–4202.

Response to Q1

Does the proposed method have broader applications in distributed learning beyond federated learning? If so, what constraints of the federated setting can be removed?

We agree that our method has the potential to extend to more general distributed test-time adaptation scenarios. However, we investigate this problem under the federated learning setting because local data retention and independent client adaptation impose specific constraints on knowledge sharing.

The central problem is knowledge collaboration among nodes in asynchronous environments, particularly imbalanced knowledge contributions caused by heterogeneous processing speeds and the effective use of shared knowledge under heterogeneous environments. These challenges may also arise in general distributed learning systems.

In our federated learning setting, raw test data remain on individual clients, which perform local adaptation independently, while the server cannot directly access or redistribute raw samples. Cross-client collaboration therefore relies on uploaded adaptation information, subject to practical constraints on data privacy, communication overhead, and computational resources.

When extending our method to more general distributed learning scenarios, constraints on local data retention and computation placement can be relaxed. For example, when data sharing or task migration is permitted, the server may perform cross-node data sampling, centralized adaptation, or computational task scheduling rather than relying entirely on independent client adaptation. More flexible task allocation can also partially weaken the coupling between node processing speed and knowledge contribution frequency.

However, relaxing these constraints may increase privacy risks associated with data transmission, communication costs, and server-side computational overhead, while changing how shared knowledge is updated and propagated. Therefore, the collaboration mechanism would need to be adjusted according to specific deployment conditions.

We hope this explanation addresses the reviewer's concerns. We would be happy to provide further clarification if needed.

Reviewer_AaupOpenReview ↗

meaningful explanations provided

Thank you for providing these explanations, and particularly the discussion about an applicability of your method for a more general case.

Reviewer_mTiPOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an asynchronous federated test-time adaptation framework designed to eliminate synchronization waiting while addressing fast-client dominance and cross-client error propagation. The method introduces environment-specific shared memory updated through residual write-in and isolates shared corrections from local test-time optimization.

Experiments on CIFAR10-C, CIFAR100-C, and ImageNet-C evaluate the approach across Tent, SAR, and DeYO with ResNet and Vision Transformer architectures. The reported results show consistent accuracy improvements and wall-clock time reductions of 44.6%–64.2% relative to synchronous FedTTA.

Critical strengths
  1. Practical and relevant asynchronous adaptation problem.

The paper addresses an important limitation of synchronous FedTTA in heterogeneous environments, where slow clients delay collaborative adaptation. The proposed architecture provides a plausible mechanism for supporting continuous cross-client knowledge sharing without requiring local TTA algorithms to be modified.

  1. Consistent empirical improvements across multiple settings.

The proposed method outperforms the evaluated baselines across the main dataset, architecture, and local TTA combinations. The experiments include five random seeds, client-speed heterogeneity, fault injection, hyperparameter sensitivity, and additional scalability evaluations. This breadth provides encouraging evidence of practical usefulness.

Critical weaknesses
  1. Insufficient isolation of the proposed mechanisms.

The evaluation does not systematically distinguish the contributions of environment-specific memory, residual write-in, prediction fusion, and isolation from local TTA. Since the baselines use different collaboration mechanisms, the reported improvements cannot be confidently attributed to the proposed residual update and isolation strategies individually.

Furthermore, the residual update in Eq. (9) does not generally ensure frequency-independent client influence when different clients provide conflicting corrections. More controlled experiments are needed to support the proposed explanation of fast-client dominance mitigation.

  1. Incomplete validation of system efficiency and deployment robustness.

The reported wall-clock improvements are evaluated primarily against a synchronous baseline. Equivalent runtime measurements against asynchronous baselines, together with network latency, communication volume, and server-processing overhead, are needed to substantiate the practical efficiency claims.

The method also uploads per-sample feature representations and correction vectors without assessing their potential privacy implications. The fault-injection experiment considers only a single corrupted update, leaving robustness against repeated erroneous contributions insufficiently established.

The data-partitioning protocol in the larger-client experiments and minor inconsistencies between the main and supplementary numerical results also require clarification.

Questions for authors

Q1 — Mechanism ablation: Can the authors provide controlled ablations separating environment-specific memory, residual write-in, prediction fusion, and local adaptation isolation? In particular, how does residual write-in compare with conventional memory averaging when all other components are held fixed?

Q2 — Fairness and stability: Under strongly imbalanced upload frequencies and conflicting client corrections, how does the proposed memory update prevent fast-client dominance? Can the authors quantify client-level fairness and demonstrate robustness to repeated rather than isolated faulty updates?

Q3 — End-to-end efficiency and privacy: Can the authors report matched wall-clock comparisons against AFTTA and FATTA, including uplink/downlink traffic, memory-transfer overhead, and realistic network delays? What privacy protection, if any, is provided for the transmitted sample-level embeddings and correction vectors?

Q4 — Reproducibility: How are test examples allocated across 120- and 180-client experiments, particularly when the number of required sample occurrences exceeds the available images per corruption? Can the authors reconcile the proposed-method results for Model 1 in Table 1 and Appendix Table 3?

4 comments
AuthorsOpenReview ↗

Response Part 1

We thank the reviewer for the constructive feedback. As the concerns in Critical Weaknesses are closely related to those in Questions for Authors, we address the latter point by point below, hoping that these responses will also resolve the former.

Response to Q1

Mechanism ablation: Can controlled ablations isolate environment-specific memory, residual write-in, prediction fusion, and local adaptation isolation? How does residual write-in compare with conventional memory averaging when all other components are fixed?

Our ablation results confirm the contributions of all five components, including the advantage of residual write-in over averaging-based updates. To better evaluate these contributions, we conducted controlled ablation experiments on CIFAR10-C and CIFAR100-C using ResNet-18 with Tent as the local TTA method. We consider five variants: (1) w/o ESM: replacing environment-specific memory with a single shared memory; (2) w/o RW: replacing residual write-in with averaging-based memory updates; (3) w/o PF: removing prediction fusion and using only the predictions of the fixed shared model corrected by shared memory; (4) w/o LTI: allowing shared knowledge to participate in local TTA optimization; and (5) w/o SM: removing cross-client shared memory while retaining prediction fusion between the local adaptation model and the fixed shared model. All other experimental settings remain unchanged.

Table A. Accuracy comparison of the full method and its ablation variants on CIFAR10-C and CIFAR100-C using Tent.

VariantCIFAR10-CCIFAR100-C
w/o ESM74.50 ± 0.6549.50 ± 0.41
w/o RW73.80 ± 0.6048.50 ± 0.55
w/o PF60.52 ± 0.5535.63 ± 0.58
w/o LTI74.20 ± 1.1249.00 ± 0.72
w/o SM72.80 ± 1.2447.00 ± 0.45
Ours77.68 ± 0.1953.42 ± 0.32

As shown in Table A, the full method achieves the highest accuracy on both datasets, while all ablation variants show varying degrees of performance degradation. In particular, w/o RW replaces residual write-in with conventional averaging-based memory updates while keeping all other components unchanged, resulting in a noticeable accuracy drop on both datasets.

AuthorsOpenReview ↗

Response Part 2

Response to Q2

Fairness and stability: Under imbalanced upload frequencies and conflicting client corrections, how does the memory update mitigate fast-client dominance? Can the authors quantify client-level fairness and robustness to repeated faulty updates?

Our additional evaluations show that our method alleviates fast-client dominance, provides more balanced collaboration benefits, and maintains greater stability under conflicting or faulty updates.

First, regarding fast-client dominance, we extend the original 1:8 speed ratio to five settings (1:1 to 1:16) on CIFAR10-C using Tent. Table B shows that the accuracy-gain gap between fast and slow clients widens for AFTTA and FATTA as speed imbalance grows. In contrast, our method maintains a smaller gap, indicating its effectiveness in alleviating fast-client dominance under imbalanced upload frequencies.

Table B. Accuracy gains (%) of slow and fast clients over standalone Tent on CIFAR10-C under different speed ratios.

Speed RatioAFTTA (Slow)AFTTA (Fast)AFTTA (Gap)FATTA (Slow)FATTA (Fast)FATTA (Gap)Ours (Slow)Ours (Fast)Ours (Gap)
1:12.52 ± 0.2432.64 ± 0.1020.122.33 ± 0.2352.41 ± 0.1140.082.42 ± 0.2392.47 ± 0.1340.05
1:21.97 ± 0.2382.78 ± 0.1160.812.16 ± 0.2512.67 ± 0.1650.512.40 ± 0.1072.63 ± 0.1390.23
1:41.65 ± 0.2042.95 ± 0.1131.302.35 ± 0.1393.04 ± 0.1320.692.57 ± 0.2062.80 ± 0.1090.23
1:81.40 ± 0.1583.00 ± 0.2371.601.80 ± 0.1982.80 ± 0.1981.002.10 ± 0.1582.60 ± 0.1580.50
1:161.10 ± 0.1563.01 ± 0.1361.911.50 ± 0.1732.95 ± 0.1301.452.12 ± 0.1192.75 ± 0.1420.63

Regarding conflicting client corrections, we evaluate directional conflicts under the same five speed ratios. We randomly select 20%, 30%, and 40% of the fast client's uploads and replace their correction vectors with vectors opposite in direction to the slow client's corrections while preserving the original norms. Table C shows that AFTTA and FATTA exhibit greater degradation as speed imbalance and conflict levels increase, particularly under 40% conflict. In contrast, our method maintains higher accuracy gains, demonstrating greater stability under conflicting client corrections.

Table C. Accuracy gains (%) of slow clients over standalone Tent on CIFAR10-C under different speed ratios and conflict levels.

ConflictMethod1:11:21:41:81:16
20%AFTTA0.71 ± 0.0520.54 ± 0.0790.59 ± 0.0460.16 ± 0.091-0.08 ± 0.064
20%FATTA0.86 ± 0.0610.79 ± 0.0440.83 ± 0.0730.57 ± 0.0520.49 ± 0.068
20%Ours1.12 ± 0.0471.19 ± 0.0631.08 ± 0.0381.14 ± 0.0560.98 ± 0.071
30%AFTTA0.58 ± 0.0730.44 ± 0.0550.13 ± 0.0840.19 ± 0.062-0.31 ± 0.097
30%FATTA0.73 ± 0.0490.66 ± 0.0670.48 ± 0.0580.51 ± 0.0820.22 ± 0.046
30%Ours0.97 ± 0.0580.85 ± 0.0410.91 ± 0.0760.72 ± 0.0530.75 ± 0.069
40%AFTTA-0.38 ± 0.080-0.88 ± 0.090-1.36 ± 0.050-2.23 ± 0.110-2.91 ± 0.070
40%FATTA-0.14 ± 0.060-0.49 ± 0.040-1.18 ± 0.080-1.61 ± 0.060-2.46 ± 0.100
40%Ours0.16 ± 0.050-0.25 ± 0.070-0.72 ± 0.060-1.29 ± 0.090-1.96 ± 0.080

Second, regarding client-level fairness, prior studies assess fairness through performance differences across clients [1, 2]. The controlled experiment in Section 5.3 of the original manuscript already provides a quantitative basis: we vary only the processing speeds of one fast and one slow client, measure their accuracy gains relative to standalone Tent, and use the gap to assess the balance of collaboration benefits. Table B shows that our method achieves the smallest gap across all speed ratios, indicating more balanced benefits across clients.

Finally, regarding robustness to repeated erroneous updates, we extend the original single-update experiment to continuous fault injection. Four clients operate normally for the first 100 server arrival events; thereafter, every upload from one designated client is replaced with a faulty update through event 500. Table D reports the final-event accuracy: AFTTA and FATTA exhibit substantial drops, whereas our method shows only a minor decrease.

Table D. Final-event accuracy (%) under normal operation and repeated faulty updates on CIFAR10-C using Tent.

MethodNormal Accuracy (%)Faulty Accuracy (%)Accuracy Drop (pp)
AFTTA69.6064.155.45
FATTA68.3163.814.50
Ours68.9568.600.35

[1] Li, T., Sanjabi, M., Beirami, A., and Smith, V. "Fair Resource Allocation in Federated Learning," ICLR, 2020.

[2] Chen, H., Zhu, T., Zhang, T., Zhou, W., and Yu, P. S. "Privacy and Fairness in Federated Learning: On the Perspective of Tradeoff," ACM Computing Surveys, vol. 56, no. 2, Article 39, pp. 1–37, 2024.

AuthorsOpenReview ↗

Response Part 3

Response to Q3

End-to-end efficiency and privacy: Can the authors provide matched wall-clock comparisons with AFTTA and FATTA, including uplink/downlink traffic, memory-transfer overhead, and realistic network delays? What privacy protection is provided for transmitted sample-level embeddings and correction vectors?

Our extended experiments demonstrate advantages in total runtime and communication overhead, and we clarify the current scope of privacy protection.

First, regarding end-to-end efficiency evaluation, we extend the resource overhead experiments to include additional runtime comparisons with the asynchronous baselines AFTTA and FATTA, alongside the synchronous baseline SFTTA. Under identical hardware configurations, test data streams, and client speed heterogeneity settings, we evaluate the total runtime, uplink and downlink communication volumes, server processing time, and peak RAM usage of each method. The transmission overhead of shared memory is included in the downlink communication volume. The experiments cover four dataset and model configurations using Tent as the local TTA method. Additional experiments using SAR and DeYO are also provided in the appendix of the revised manuscript to evaluate runtime efficiency under different local TTA methods.

As shown in Table E, our method achieves the shortest total runtime across all evaluated settings, outperforming both the synchronous baseline SFTTA and the asynchronous baselines AFTTA and FATTA. In terms of communication overhead, exchanging shared information instead of complete models effectively reduces communication costs. Although maintaining the shared memory increases server processing time in some settings, the overall runtime advantage is preserved, while peak RAM usage remains broadly comparable to that of the baselines.

Table E. Comparison of total runtime (s), uplink and downlink communication volumes (MiB), server processing time (s), and peak RAM usage (GiB) using Tent.

Dataset & ModelMethodTotal Time (s)Uplink (MiB)Downlink (MiB)Server Time (s)Peak RAM (GiB)
CIFAR10-C + ResNet-18SFTTA906.88 ± 28.51527.34 ± 0.17263.67 ± 0.0831.74 ± 1.781.78 ± 0.03
AFTTA1148.53 ± 30.46527.34 ± 0.12263.67 ± 0.1152.36 ± 2.001.55 ± 0.02
FATTA859.27 ± 19.55527.34 ± 0.21263.67 ± 0.0681.67 ± 3.091.60 ± 0.01
Ours502.41 ± 8.96222.09 ± 0.073.10 ± 0.0748.13 ± 3.071.72 ± 0.05
CIFAR100-C + ResNet-18SFTTA1191.22 ± 37.97527.34 ± 0.14263.67 ± 0.0931.79 ± 0.411.77 ± 0.05
AFTTA1161.83 ± 21.48527.34 ± 0.23263.67 ± 0.0552.58 ± 1.561.52 ± 0.01
FATTA875.62 ± 26.89527.34 ± 0.11263.67 ± 0.1283.24 ± 1.851.58 ± 0.02
Ours497.93 ± 14.00259.17 ± 0.0825.38 ± 0.5460.27 ± 1.941.67 ± 0.03
ImageNet-C + ResNet-50SFTTA5731.29 ± 41.262917.97 ± 0.341458.98 ± 0.17107.34 ± 0.781.95 ± 0.02
AFTTA5158.57 ± 24.612917.97 ± 0.231458.98 ± 0.11185.97 ± 1.311.63 ± 0.01
FATTA3871.66 ± 18.742917.97 ± 0.411458.98 ± 0.19261.53 ± 1.461.48 ± 0.02
Ours2229.47 ± 11.841283.86 ± 0.16706.10 ± 18.69336.32 ± 2.181.73 ± 0.01
ImageNet-C + ViT-B/16SFTTA9413.18 ± 73.422109.38 ± 0.271054.69 ± 0.1341.34 ± 0.251.86 ± 0.02
AFTTA7638.25 ± 39.722109.38 ± 0.181054.69 ± 0.2183.38 ± 0.491.82 ± 0.01
FATTA5606.48 ± 26.352109.38 ± 0.321054.69 ± 0.09114.16 ± 0.751.57 ± 0.02
Ours3369.92 ± 17.23770.58 ± 0.11402.76 ± 8.90275.73 ± 1.621.93 ± 0.01

Second, regarding privacy protection, our method does not currently incorporate dedicated privacy protection mechanisms for the transmitted sample-level representations and correction vectors. This work primarily focuses on the efficiency of knowledge sharing and adaptation stability in asynchronous FedTTA under dynamic heterogeneous environments, while the design of dedicated privacy protection mechanisms is beyond the scope of this study. Nevertheless, for applications with stricter privacy requirements, existing techniques such as differential privacy, encrypted transmission, and secure computation could be further integrated to enhance privacy protection during knowledge sharing. However, introducing these techniques would require careful consideration of their potential effects on environment matching, adaptation performance, and communication overhead.

AuthorsOpenReview ↗

Response Part 4

Response to Q4

Reproducibility: How are test examples allocated across 120- and 180-client experiments given possible shortages of images per corruption? Can the authors reconcile Model 1 results in Table 1 and Appendix Table 3?

Our test sample allocation uses globally coordinated non-overlapping partitions, while the numerical discrepancies resulted from unsynchronized updates between the main manuscript and appendix.

First, regarding test sample allocation, we adopt a globally coordinated sampling-without-replacement strategy, where test samples for each corruption type are partitioned without replacement to ensure that different clients receive non-overlapping samples. In the client scalability experiments, each client uses 75 samples per corruption type in the 120-client setting and 45 samples per corruption type in the 180-client setting. Accordingly, the total numbers of sample presentations per corruption type are 9,000 and 8,100, respectively, while 10,000 test images are available at severity level 5. We have clarified the corresponding sampling rules and sample allocation details in the revised manuscript.

Second, regarding the numerical inconsistency between Model 1 in Table 1 and the corresponding appendix tables, we found that the discrepancy was mainly caused by the failure to synchronize the experimental results between the main manuscript and the appendix during subsequent updates.

To further verify the stability and reliability of the experimental results, we conducted multiple independent experimental runs using different combinations of random seeds while keeping the model configurations, hyperparameters, and evaluation protocol unchanged. Due to randomness, the results exhibited small variations across different runs, with some accuracy values increasing slightly and others decreasing slightly. However, the overall variations remained small. For example, on CIFAR10-C with Tent as the local TTA method, the accuracy in the earlier experiments was 77.52%. Considering the stability and reproducibility of the results across different runs, we consistently report the results from a complete experimental run with relatively stable performance, in which the corresponding accuracy is 77.68%.

However, after updating Table 1 in the main manuscript, the corresponding entries in the appendix were not updated accordingly and still retained the earlier results, leading to the reported inconsistencies. We apologize for this oversight and have carefully checked and unified the experimental results in the revised manuscript and appendix to ensure consistency.

We sincerely hope that our revisions and detailed responses have addressed the reviewer's concerns. We would be grateful if the reviewer could consider increasing the score in light of these improvements. Should there be any further questions, we would be happy to provide additional clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.