- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a bi-granularity weighted aggregation framework for asynchronous federated learning (AFL). The main idea is to reconstruct pseudo local models for straggling clients using neural blocks from already received client models, allowing their estimated contributions to participate in global aggregation without waiting for their actual updates.
The framework first obtains client-specific activation fingerprints through a one-time warm-up. During asynchronous training, it groups available neural blocks using representation similarity, constructs candidate models through ordered block borrowing, and selects pseudo models through fingerprint-based matching. The reconstructed models are then aggregated together with received local models.
Experiments on SVHN, CIFAR-10, and CIFAR-100 demonstrate competitive performance against several AFL baselines under different levels of data and system heterogeneity.
Critical strengths
1. An interesting approach to straggler compensation in AFL. Rather than simply discarding or downweighting delayed updates, the paper attempts to incorporate estimated contributions from unavailable clients through model reassembly. This provides an interesting perspective on handling stragglers in asynchronous federated learning.
2. Relatively comprehensive experimental evaluation. The paper evaluates different datasets, statistical heterogeneity levels, and asynchronous settings. It also includes sensitivity analyses and ablation studies for candidate construction, block borrowing, source diversity, and fingerprint-based matching.
Critical weaknesses
1. Unclear distinction between model reassembly and block-wise reweighting.
Model reassembly has already been explored in federated learning, such as pFedHR (NeurIPS 2023) and pFedClub (NeurIPS 2024). While this paper applies model reassembly to approximate unavailable straggler models in AFL, the resulting aggregation can also be interpreted as a form of block-wise client reweighting. Without candidate calibration, aggregating reconstructed models is mathematically equivalent to reweighting the available client models at different block positions. Although calibration prevents strict equivalence for the complete method, the authors should clarify the distinction and compare against a simpler block-wise reweighting baseline.
2. Reliability of static client fingerprints throughout asynchronous training.
The method relies on client fingerprints obtained during a one-time warm-up and reused throughout asynchronous training. However, as client models evolve, these fixed representations may no longer accurately reflect their current characteristics. Although the paper evaluates different warm-up epochs, it does not directly investigate representation drift or its impact on pseudo-model matching. Evaluating fingerprint stability over time and comparing static fingerprints with periodically refreshed ones would help validate this assumption.
3. Unclear handling of late-arriving genuine client updates.
The proposed framework incorporates reconstructed pseudo models for straggling clients before their genuine updates arrive. However, the paper does not clearly explain how these genuine updates are handled afterward. If they are aggregated again, the same client training task may contribute more than once. The authors should clarify whether late-arriving updates are discarded, incorporated again, or used for correction, and explain how duplicate contributions are addressed.
4. Lack of theoretical analysis of the proposed aggregation mechanism.
The paper mainly relies on empirical evaluation without providing theoretical analysis of how reconstructed pseudo models affect asynchronous optimization. In particular, reconstruction errors may introduce additional bias into global aggregation, especially under data heterogeneity and stale updates. A theoretical analysis relating reconstruction error to aggregation bias or convergence behavior would help explain under what conditions the proposed method improves upon aggregation using only received client updates.
Questions for authors
-
Since the pseudo models are constructed from neural blocks of already received client models, to what extent can the proposed aggregation be interpreted as block-wise client reweighting? What benefits does model reassembly provide beyond a simpler reweighting strategy?
-
How stable are the client fingerprints obtained during one-time warm-up? Have the authors investigated representation drift during asynchronous training and its effect on pseudo-model matching?
-
After a pseudo model for a straggling client has been included in aggregation, how is the corresponding genuine local model handled when it eventually arrives? Can the same client training task contribute more than once?
-
Can the authors provide theoretical justification for incorporating reconstructed models into asynchronous aggregation, particularly regarding how reconstruction error affects global update bias and convergence?
3 comments
Thank you for your detailed response and additional clarifications.
Regarding W1, I remain somewhat skeptical about the effectiveness of reconstructing pseudo models for straggling clients by assembling neural blocks from other clients. Although the proposed selection and fine-tuning procedures introduce additional adaptation, it is still unclear whether these reconstructed models can adequately represent the actual contributions of straggling clients. I believe this concern has not been fully addressed.
Regarding W2, I appreciate the additional visualizations and explanations. I think the importance of fingerprint stability may depend on the specific task and experimental setting. Therefore, I consider this a relatively minor concern.
Regarding W3, I have a more significant concern about directly discarding late-arriving genuine updates. In practical heterogeneous federated learning systems, some resource-constrained clients may consistently return their updates later than others. Under the proposed strategy, if their pseudo models have already been incorporated into aggregation, all their genuine updates could be discarded. Consequently, the information learned from these clients' local data may never directly contribute to the global model. Since the reconstructed pseudo models are assembled from other clients' models, they may not adequately preserve the unique information of these persistently slow clients. I believe this is a potentially serious limitation of the proposed approach, particularly under substantial system and data heterogeneity.
Indirect Straggler Contributions and Update Timeliness & Client-Specific Guidance for Model Reconstruction.
W3: Indirect Straggler Contributions & Update Timeliness.
Thank you for your insightful observation. First, stragglers can contribute to the global model through an indirect yet meaningful pathway. Specifically, our method leverages client fingerprints extracted from genuine local data distributions to guide similarity-based candidate selection during model reconstruction. Through this mechanism, even persistently slow clients can influence the reconstructed models and subsequent global aggregation via their historical fingerprints. Second, the value of genuine updates depends on both their knowledge and timeliness. Although late-arriving updates may contain unique client knowledge, they also be derived from outdated global model states. In particular, updates from persistently slow clients can introduce stale optimization directions, may hindering the current global optimization process.
In contrast, our method reconstructs pseudo-local models using timely local information, while leveraging fingerprint-based similarity constraints to guide candidate selection. This combination enables the global model to incorporate estimated straggler contributions in a timely manner while favoring information aligned with the current global optimization state.
W1: Client-Specific Guidance for Model Reconstruction.
We further clarify two key aspects of our method: how target-client fingerprints guide candidate selection and the conditions under which such guidance remains effective. First, our matching mechanism captures the sample relationships learned by the target client. Specifically, CKA measures the similarity between the pairwise sample similarity structures induced by two models. Thus, historical fingerprints characterize the representation structure learned by the target client through local training. Although candidate models are assembled from neural blocks of other clients, their selection is guided by the target-client fingerprint, enabling reconstruction to reflect client-specific characteristics. Second, effective candidate selection relies on the ability of historical fingerprints to preserve informative candidate rankings. Clients are warmed up on their respective local data, resulting in distinct feature responses. Consequently, even with an identical candidate pool, different client fingerprints can yield different candidate rankings. Under data heterogeneity, these fingerprints provide client-specific representation references learned from local data, allowing candidate selection to reflect the characteristics of each target client. Furthermore, the results in Appendix E demonstrate sufficient temporal consistency of fingerprints during subsequent training, allowing historical matching scores to remain informative about the relative similarity between candidate models and the current target-client model. This consistency supports the effectiveness of historical fingerprints in guiding candidate selection.
We clarify the distinction between model reassembly and block-wise reweighting, present fingerprint stability visualizations, and explain how our buffer-free framework handles late-arriving updates.
W1&Q1: Model Reassembly vs. Block-wise Reweighting. We thank the reviewer for noting the link between model reassembly and block-wise reweighting. The key distinction from direct block-wise reweighting lies in how we select block combinations and adapt the assembled model. For each straggling client, we use its cache client fingerprint to evaluate and select complete candidate models. Block selection therefore depends on the overall behavior of the assembled model. We then fine-tune the selected candidate on proxy data to jointly adapt blocks from different sources. Our full pipeline therefore combines history-guided block selection with candidate-specific parameter adaptation, providing an optimization mechanism beyond direct block-wise reweighting.
W2&Q2: Fingerprint Stability. Because the fingerprints cached during one-time warm-up are reused to match reconstructed models to straggling clients, we provide additional visualizations to examine fingerprint stability. In APPENDIX E, we provide two visualizations of activation patterns from the same client model at three warm-up settings: Step 0, Default, and Round 40. At each step, we use four fixed proxy samples and extract activations from the same feature layer, maintaining consistent channel indices and fixed per-channel color scales. The first visualization displays 64 selected channels per proxy sample to facilitate inspection of individual responses. The second displays all 512 channels per proxy sample and highlights six locations, with corresponding enlarged patches shown below each panel. At Step 0, the activation maps are generally weak under the shared color scales. Distinct spatial responses emerge under the default setting. Between Default and Round 40, the highlighted channels exhibit similar spatial response patterns, although activation intensities and local details continue to change. These visualizations provide qualitative evidence of pattern persistence. To ensure that we address the concern precisely, could the reviewer explain which representations are being considered, during which stage of training the drift occurs, and what form of representational change is of concern? We would appreciate further clarification on these points, as it would help us better address your concerns.
W3&Q3: Late-arriving Updates. We thank the reviewer for raising this issue. We propose a buffer-free model reconstruction framework for asynchronous federated learning that constructs reconstructed models for straggler models. Once a reconstructed model for a specific training task has been included in aggregation, the actual local model subsequently returned for that same task is excluded from future global aggregation. This prevents the actual model from being aggregated in addition to the reconstructed contributions for the same task.
If a client remains a straggler, its incomplete training task could still be represented in subsequent rounds by newly reconstructed models assembled from the latest received client models. This buffer-free design allows aggregation to proceed without waiting for straggling clients and avoids retaining the excluded late-arriving models in a queue for future aggregation.
W4&Q4: Theoretical analysis. We thank the reviewer for this suggestion. In future work, we will explore how reconstruction error affects global update bias and convergence in asynchronous aggregation.