- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies a checkpoint decision in federated learning, namely which clients should continue training locally and which should pool into a shared model, before either alternative is trained. It models client mean gradients through a Gaussian hierarchy that separates sampling variance from between-client variance, and uses posterior gradient-estimation risk as the decision criterion. With known scalar variances, clients with sampling variance below the between-client variance stay local and the rest pool with inverse-total-variance weights (Lemma 3.1 and Theorem 3.2). The paper proves this partition almost surely admits no blocking coalition, characterizes precision weights as the unique fixed weights preserving participation for at least three pooled clients (Theorem 3.3), extends stability to general covariances when all directions agree (Theorem 3.4), and gives a counterexample where conflicting covariance directions defeat every partition (Theorem 3.5). Under shared quadratic curvature the risk comparison also orders expected loss after Newton updates (Proposition 4.1). The implemented FedBHG estimates the variances once at a checkpoint and continues with ordinary SGD or Adam. Synthetic experiments isolate failure modes of fitted routing, and neural experiments on Fashion-MNIST, EMNIST-Letters, and Shakespeare compare against universal continuations, simpler routes, and personalization baselines.
Critical strengths
- The theory is tight and symmetric in both directions. The stable construction (Theorem 3.2), the weight characterization (Theorem 3.3), and the impossibility counterexample (Theorem 3.5, with exact gains verified in Table 1) together map out where stability can and cannot hold, which is rare for participation analyses in federated learning.
- The empirical work is unusually disciplined about its own limits. The paper states the claim boundary explicitly in Section 5, isolates route error, directional conflict, and weight error in controlled six-client simulations (93.0% no-deviation with correct routes versus 14.5% after one error in Figure 1), reports paired per-cell intervals rather than cherry-picked aggregates (Table 2), and includes misspecification checks showing the fitted spherical model is rejected yet the score retains useful ranking (Section 5.5).
Critical weaknesses
- The gap between the theory and the deployed procedure is substantial. The guarantees assume known variances, the Gaussian hierarchy, and quadratic shared curvature with Newton updates, while FedBHG estimates variances once and trains with ordinary optimizers, so no formal stability carries over to the neural setting. The practical edge of the full Bayesian score over simpler routes is correspondingly modest, since its routing matches sample-size routing in 7 of 20 repeats and variance-only routing in 5 of 20 (Section 5.3).
- The experimental scope is narrow. Only four cells selected from a ten-cell development sweep are studied in depth, with fixed client partitions of 100 clients on three datasets, and some headline gains are small or unresolved, for example +0.051 accuracy points on Shakespeare/CFL. Broader validation across more tasks and partition scales would strengthen the practical claims.
Questions for authors
- The gap between the theory and the deployed procedure is substantial. The guarantees assume known variances, the Gaussian hierarchy, and quadratic shared curvature with Newton updates, while FedBHG estimates variances once and trains with ordinary optimizers, so no formal stability carries over to the neural setting. The practical edge of the full Bayesian score over simpler routes is correspondingly modest, since its routing matches sample-size routing in 7 of 20 repeats and variance-only routing in 5 of 20 (Section 5.3).
- The experimental scope is narrow. Only four cells selected from a ten-cell development sweep are studied in depth, with fixed client partitions of 100 clients on three datasets, and some headline gains are small or unresolved, for example +0.051 accuracy points on Shakespeare/CFL. Broader validation across more tasks and partition scales would strengthen the practical claims.
Client Selection and Aggregation Weights
Thank you for the questions. We address weaknesses W1–W2 and author questions Q1–Q2.
1. Client Selection and Aggregation Weights (W1, Q1)
We added a new four-arm experiment to measure the contributions of client selection and gradient aggregation separately. On Fashion-MNIST, selecting which clients share a model improves accuracy by 3.488 percentage points with sample-count aggregation and 3.103 points with inverse-total-variance aggregation. Holding the selected clients fixed, inverse-total-variance aggregation reduces loss by 0.0473 on Fashion and 0.0709 on EMNIST. The paired intervals for these loss reductions exclude zero (Section 5.3, Appendix G.2).
The experiment compares sharing among all clients in an upstream cluster with sharing only among clients selected by FedBHG. Clients excluded from the shared model train independently. Each selection is tested with both aggregation methods. All four branches use the same regenerated checkpoint, restored optimizer states, reduced learning rate, local training budget, and client shuffle streams. Thus, differences within each pair isolate client selection or aggregation under the tested training protocol.
Precision-based aggregation also improves the Fashion loss outcome of selective sharing. With sample-count aggregation, choosing the FedBHG-selected clients increases loss by 0.0144 (95% paired interval for the increase [0.0054, 0.0234]). With inverse-total-variance aggregation, the same selection reduces loss by 0.0089 (interval for the reduction [0.0028, 0.0151]). The corresponding weighting differences on Shakespeare have intervals including zero. These results show distinct contributions from choosing clients and combining their gradients.
FedBHG's Bayesian client-selection score gives an additional accuracy gain over simpler selection rules on Fashion. With the number of locally trained clients matched, it gains 0.901 points over sample-count selection and 1.108 points over gradient-variance selection. The Fashion loss difference against sample-count selection and the accuracy differences in the other three settings remain unresolved. The clearest score-specific gain is therefore Fashion accuracy; the four-arm results separately show the contribution of selective sharing and precision aggregation.
Theorem 3.2 explains why the Bayesian model selects clients for local or shared gradient estimation under known variances and union-conditioned summary disclosure. For groups of at least three pooling-eligible clients, Theorem 3.3 characterizes inverse-total-variance coefficients as the only fixed positive aggregation coefficients that preserve every member's willingness to share for every observed summary configuration. The theoretical no-deviation result concerns posterior estimation risk. The experiments evaluate prediction after estimated routing and multi-round SGD or Adam training.
2. Baseline Comparisons and Experimental Scope (W2, Q2)
We added a new experiment comparing FedBHG against CFL and IFCA continuing their original training procedures. FedBHG improves both accuracy and held-out loss in three of four settings. On Fashion/CFL and EMNIST/IFCA, its accuracy gains are 3.079 and 1.983 percentage points, with loss reductions of 0.0606 and 0.0547. Shakespeare/IFCA also has positive paired intervals for both outcomes. Both intervals include zero on Shakespeare/CFL (Table 3, Appendix G.2).
Each comparison starts from the same regenerated round-200 checkpoint and restores the model and optimizer states. Five paired seeds per setting run ten additional rounds, with one local epoch per client per round. Continuing CFL/IFCA retains its original learning rate, sample-count aggregation, and clustering or reassignment rule. FedBHG assigns clients to independent or shared training once, fixes the shared group's gradient-aggregation coefficients, and uses one-tenth of the original learning rate. These comparisons evaluate the full continuation procedures, so learning rate and computation are not matched. The four-arm experiment above keeps the reduced learning rate fixed to isolate client selection and gradient aggregation.
The additional comparisons cover four settings selected from a ten-cell development grid, using 100 clients and full participation. Four earlier fresh vision-partition observations test changed client populations. FedBHG improves accuracy in all four and loss in three; one Fashion partition gains 1.45 accuracy points while increasing loss by 0.013. Those results remain in the revised paper. Sections 5.2–5.3 and Appendix G.2 now distinguish the new baseline comparisons, the new four-arm experiment, and the earlier robustness observations. Broader client scales and additional tasks remain for future evaluation.