- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses routing-prediction federated learning, where the server routes each query to the best-matched client at inference time. The authors observe that existing methods treat each client as internally homogeneous, while real client data often come from several latent subpopulations, and call this coexistence of inter-client and intra-client heterogeneity dual heterogeneity. FedSPM models each client as a mixture of C latent components. Each component has a predictive side (a shared embedding with a client-specific head) and a feature side modeled as a density ratio relative to a common nonparametric baseline estimated by empirical likelihood over pooled samples. A federated EM algorithm optimizes the profile log-EL; the intractable Lagrange multiplier system in the M-step is replaced by a surrogate whose critical points coincide with the exact one. Theorem 1 gives an O(1/sqrt(T)) rate under Assumptions 4 and 5. Communication is limited to m×C scalars per round. Experiments cover FMNIST, CIFAR-10, CIFAR-100 with controlled heterogeneity plus Fed-ISIC2019 with 6 clinical centers.
Critical strengths
- The DRM-EL feature model is a real contribution: keeping the baseline nonparametric and estimating it by empirical likelihood over pooled samples gives client distributions a common coordinate system, which makes routing comparisons well defined.
- The M-step surrogate derivation, showing that each multiplier admits a closed form at any critical point, turns an objective requiring a nonlinear solve per step into one that standard autodiff handles.
Critical weaknesses
- Assumptions 4 and 5 introduce bridge error sequences that are estimated numerically from training trajectories rather than derived from any property of the model or data, so the theorem reduces to a conditional statement.
- The routing story relies on component semantics that Remark 1 explicitly disclaims as non-identifiable, and the ablation study is not reported.
Questions for authors
- Can the bridge errors be characterized in terms of quantities checkable before training?
- What is the performance of FedSPM with C=1?
3 comments
Dear Reviewer SJ84,
We greatly appreciate the time and effort you have devoted to reviewing our work. We have posted our responses and uploaded the revised paper to address your comments. Before the discussion closes, we would sincerely welcome your feedback on whether our revisions and explanations have adequately addressed your concerns. We would be pleased to discuss any remaining questions.
Best,
Authors
Dear Reviewer SJ84,
As the discussion deadline is approaching, we would kindly welcome any final thoughts on our responses and revisions. We sincerely appreciate your time and effort in reviewing our work.
Best,
Authors
We thank the reviewer for the thoughtful review and constructive questions. We appreciate your recognition of the DRM–EL feature model and the M-step surrogate derivation as contributions that enable comparable routing scores and tractable optimization. We address your questions and concerns below.
Role and empirical assessment of the bridge assumptions [W1/Q1]. Assumptions 4–5 specify sufficient conditions for transferring optimization progress from the tractable surrogate to the exact profiled objective. Conditional analyses also appear in federated optimization: FedNova uses bounded gradient dissimilarity, while FedEM’s black-box-solver extension assumes quantitative local approximation accuracy. These provide precedents for analyzing convergence under explicit conditions on optimization behavior.
In our setting, the bridge conditions quantify the discrepancy introduced by replacing the implicit EL multipliers. They control gradient norms and one-step objective changes without requiring gradient-direction alignment or uniformly close objective values. Theorem 1 retains the accumulated bridge-error contributions explicitly, identifying when surrogate optimization preserves the standard convergence rate.
We additionally assess these quantities empirically by numerically solving the EL system along FMNIST training trajectories. Sec. 3.2 (Fig. 2) and App. D.4 show that the normalized cumulative errors remain small or decrease, providing direct empirical support for the conditions in the evaluated runs. Deriving sufficient conditions checkable before training remains open.
References:
Component-level non-identifiability does not prevent client-level routing [W2]. Remark 1 states the non-identifiability of individual latent components. It does not prevent us from estimating each client’s marginal feature distribution and performing client-level routing. Equation (11) uses this marginal distribution, obtained by summing over the components, so routing does not require a unique component decomposition. The components instead capture intra-client heterogeneity, providing a more flexible model of the client-level feature likelihood. FedSPM’s improved routing and prediction performance compared with its single-component counterpart, FedDRM, provides empirical support for the practical value of this modeling choice.
Ablations of the main modeling choices [W2]. Beyond the component-count experiment above, we test the distribution model and client-specific prediction rules separately on CIFAR-10. We replace only DRM–EL with a client-specific GMM, or share the component-wise prediction heads across clients, keeping the remaining settings unchanged. Mean accuracies are:
Both replacements reduce performance, supporting the contributions of semiparametric feature modeling and client-specific prediction. Our main experiments also compare shared encoders (1×) with separate encoders (C×): the smaller shared-encoder variant performs better on all three controlled benchmarks, while separate encoders perform better on Fed-ISIC2019. Thus, increased encoder capacity alone does not explain the gains.
Performance with a single component [W2/Q2]. At the modeling level, FedSPM with C=1 reduces to FedDRM. Our component-count ablation in Sec. 3.2 and Fig. 3 reports 53.51% system accuracy with C=1 on FMNIST, increasing to 57.58% with C=2 and 58.82% with C=3. Since the encoders are shared across components, increasing C adds only lightweight component parameters. These results support the benefit of explicitly modeling within-client heterogeneity.