acceptodds
Under review as a conference paper at ICLR 2027

FedSPM: Routing-Enabled Federated Learning under Dual Heterogeneity via Semiparametric Mixture

Abstract

Routing-prediction federated learning has emerged as a new paradigm that reframes inter-client heterogeneity as a resource for system-level intelligence: at inference time, the server routes each external query to the best-matched client for prediction. Existing approaches, however, typically treat each client as internally homogeneous, overlooking latent subpopulations within local data. For example, patients with the same diagnosis at one hospital may exhibit morphologically distinct disease subtypes. The coexistence of inter-client and intra-client heterogeneity, which we call dual heterogeneity, can impair both routing and prediction. To address this challenge, we propose FedSPM, a routing-enabled semiparametric mixture framework that represents each client using client-specific latent components. Each component combines a predictive distribution for classification with a feature distribution for routing. To flexibly model feature distributions while effectively sharing information across clients, FedSPM models their density ratios relative to a common nonparametric distribution estimated via empirical likelihood. We develop a federated expectation-maximization algorithm that optimizes a tractable surrogate and prove convergence of the exact profiled objective at the standard rate when the surrogate errors are properly controlled. Experiments on controlled benchmarks and real-world medical data demonstrate consistent improvements in routing and prediction under dual heterogeneity.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
federated learning, density ratio model, empirical likelihood, routing

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_SJ84OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses routing-prediction federated learning, where the server routes each query to the best-matched client at inference time. The authors observe that existing methods treat each client as internally homogeneous, while real client data often come from several latent subpopulations, and call this coexistence of inter-client and intra-client heterogeneity dual heterogeneity. FedSPM models each client as a mixture of C latent components. Each component has a predictive side (a shared embedding with a client-specific head) and a feature side modeled as a density ratio relative to a common nonparametric baseline estimated by empirical likelihood over pooled samples. A federated EM algorithm optimizes the profile log-EL; the intractable Lagrange multiplier system in the M-step is replaced by a surrogate whose critical points coincide with the exact one. Theorem 1 gives an O(1/sqrt(T)) rate under Assumptions 4 and 5. Communication is limited to m×C scalars per round. Experiments cover FMNIST, CIFAR-10, CIFAR-100 with controlled heterogeneity plus Fed-ISIC2019 with 6 clinical centers.

Critical strengths
  1. The DRM-EL feature model is a real contribution: keeping the baseline nonparametric and estimating it by empirical likelihood over pooled samples gives client distributions a common coordinate system, which makes routing comparisons well defined.
  2. The M-step surrogate derivation, showing that each multiplier admits a closed form at any critical point, turns an objective requiring a nonlinear solve per step into one that standard autodiff handles.
Critical weaknesses
  1. Assumptions 4 and 5 introduce bridge error sequences that are estimated numerically from training trajectories rather than derived from any property of the model or data, so the theorem reduces to a conditional statement.
  2. The routing story relies on component semantics that Remark 1 explicitly disclaims as non-identifiable, and the ablation study is not reported.
Questions for authors
  1. Can the bridge errors be characterized in terms of quantities checkable before training?
  2. What is the performance of FedSPM with C=1?
3 comments

We thank the reviewer for the thoughtful review and constructive questions. We appreciate your recognition of the DRM–EL feature model and the M-step surrogate derivation as contributions that enable comparable routing scores and tractable optimization. We address your questions and concerns below.

  • Role and empirical assessment of the bridge assumptions [W1/Q1]. Assumptions 4–5 specify sufficient conditions for transferring optimization progress from the tractable surrogate to the exact profiled objective. Conditional analyses also appear in federated optimization: FedNova uses bounded gradient dissimilarity, while FedEM’s black-box-solver extension assumes quantitative local approximation accuracy. These provide precedents for analyzing convergence under explicit conditions on optimization behavior.

    In our setting, the bridge conditions quantify the discrepancy introduced by replacing the implicit EL multipliers. They control gradient norms and one-step objective changes without requiring gradient-direction alignment or uniformly close objective values. Theorem 1 retains the accumulated bridge-error contributions explicitly, identifying when surrogate optimization preserves the standard convergence rate.

    We additionally assess these quantities empirically by numerically solving the EL system along FMNIST training trajectories. Sec. 3.2 (Fig. 2) and App. D.4 show that the normalized cumulative errors remain small or decrease, providing direct empirical support for the conditions in the evaluated runs. Deriving sufficient conditions checkable before training remains open.

    References:

    • Wang, J., Liu, Q., Liang, H., Joshi, G., and Poor, H. V. (2020). Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization. NeurIPS.
    • Marfoq, O., Neglia, G., Bellet, A., Kameni, L., and Vidal, R. (2021). Federated Multi-Task Learning under a Mixture of Distributions. NeurIPS.
    • Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. (2020). Federated Optimization in Heterogeneous Networks. MLSys.
  • Component-level non-identifiability does not prevent client-level routing [W2]. Remark 1 states the non-identifiability of individual latent components. It does not prevent us from estimating each client’s marginal feature distribution and performing client-level routing. Equation (11) uses this marginal distribution, obtained by summing over the components, so routing does not require a unique component decomposition. The components instead capture intra-client heterogeneity, providing a more flexible model of the client-level feature likelihood. FedSPM’s improved routing and prediction performance compared with its single-component counterpart, FedDRM, provides empirical support for the practical value of this modeling choice.

  • Ablations of the main modeling choices [W2]. Beyond the component-count experiment above, we test the distribution model and client-specific prediction rules separately on CIFAR-10. We replace only DRM–EL with a client-specific GMM, or share the component-wise prediction heads across clients, keeping the remaining settings unchanged. Mean accuracies are:

    VariantAverage accuracySystem accuracy
    FedSPM (1×)48.5948.39
    FedSPM (1×), client-specific GMM46.2820.40
    FedSPM (1×), shared prediction heads38.9938.96
    FedSPM (C×)48.3248.03
    FedSPM (C×), shared prediction heads40.4540.36

    Both replacements reduce performance, supporting the contributions of semiparametric feature modeling and client-specific prediction. Our main experiments also compare shared encoders (1×) with separate encoders (C×): the smaller shared-encoder variant performs better on all three controlled benchmarks, while separate encoders perform better on Fed-ISIC2019. Thus, increased encoder capacity alone does not explain the gains.

  • Performance with a single component [W2/Q2]. At the modeling level, FedSPM with C=1 reduces to FedDRM. Our component-count ablation in Sec. 3.2 and Fig. 3 reports 53.51% system accuracy with C=1 on FMNIST, increasing to 57.58% with C=2 and 58.82% with C=3. Since the encoders are shared across components, increasing C adds only lightweight component parameters. These results support the benefit of explicitly modeling within-client heterogeneity.

Dear Reviewer SJ84,

We greatly appreciate the time and effort you have devoted to reviewing our work. We have posted our responses and uploaded the revised paper to address your comments. Before the discussion closes, we would sincerely welcome your feedback on whether our revisions and explanations have adequately addressed your concerns. We would be pleased to discuss any remaining questions.

Best,
Authors

Dear Reviewer SJ84,

As the discussion deadline is approaching, we would kindly welcome any final thoughts on our responses and revisions. We sincerely appreciate your time and effort in reviewing our work.

Best,
Authors

Reviewer_yBsxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FedSPM, a federated learning framework incorporating a routing mechanism designed to address two types of heterogeneity: distributional differences across clients and the presence of latent subpopulations within individual clients. Each client is modeled as a semi-parametric mixture model, where each component consists of a predictive distribution for classification and a feature distribution for routing. The feature distribution is defined via a density ratio model (DRM), which is constructed based on a shared non-parametric baseline estimated through empirical likelihood (EL). The authors derive a federated EM algorithm and develop a tractable surrogate function for the DRM component. Under standard assumptions, and by employing local stochastic gradient descent (SGD) with momentum, the algorithm is shown to converge to a stationary point of the profiled objective function. Experiments on both controlled benchmarks and real-world medical datasets demonstrate that this framework consistently improves performance in both routing and predictive tasks.

Critical strengths
  1. A semi-parametric mixture model framework for federated learning with a routing mechanism is proposed. This framework explicitly captures intra-client heterogeneity and allows different clients to employ distinct component distributions, thereby relaxing the shared-component assumption commonly found in mixture-model-based personalized federated learning. By integrating Density Ratio Models (DRM) with a shared baseline estimated via empirical likelihood, the framework enables flexible, non-parametric modeling of feature distributions and facilitates theoretically grounded routing decisions at the server side.

  2. Comprehensive comparisons are conducted against various federated learning baselines, including global models, personalized models, clustering-based approaches, mixture-model-based methods, and routing-based methods. Evaluations are performed on three benchmark datasets—featuring controllable dual-source heterogeneity—as well as a real-world, multi-institutional medical dataset. Ablation studies regarding the number of components and momentum parameters are presented, alongside an analysis of the impact of the degree of heterogeneity. In addition to overall system accuracy and average accuracy, the paper also reports routing accuracy.

  3. The paper provides detailed derivations for the profile empirical likelihood, the E-step and M-step, and the construction of surrogate functions, while clearly decomposing the model into a supervised learning module and a DRM module. Furthermore, it offers an in-depth discussion on model identifiability, modeling choices, and the respective roles of shared parameters versus client-specific parameters.

Critical weaknesses
  1. Since the shared parameters are global, adding a new device necessitates retraining. This implies that the decoupling is not particularly robust; in practice, maintaining the original number of devices becomes less stable, and the security of the distributed system is compromised.
  2. The connection between the objective function and its surrogate objective function is not sufficiently established—essentially, the server-side aggregation is incomplete. Furthermore, no convergence proofs or experimental evidence are provided for these conditions.
  3. The paper lacks comparisons with recent Personalized Federated Learning (PFL) methods based on prototype/contrastive learning or generative models (e.g., FedProto, FedPRP, FedCCL, pFedGM, etc.). Although these methods do not employ routing mechanisms, they also aim to address heterogeneity and could serve as benchmarks for the predictive component of the proposed approach.
  4. There are no ablation studies to evaluate the individual contributions of the various components within FedSPM (i.e., the hybrid model, DRM-EL, and shared representations)—for instance, by removing the DRM module or enforcing the sharing of components across clients. Additionally, metrics such as communication overhead are not reported.
Questions for authors

Question 1. The shared parameters (θ, ν) and the EL reference distribution G are globally coupled across all clients, with G determined by the aggregated samples. Could the authors clarify how FedSPM adapts when new clients join or the set of participating clients changes? Specifically, does the model require retraining from scratch? Furthermore, is the method robust to changes in the number of clients?

Question 2. The convergence analysis covers only the DRM module ϕ=(γ, ξ, ν) and relies on Assumptions 4–5 (i.e., gradient/function value bridging conditions). These are non-standard assumptions supported only by empirical evidence. Additionally, Algorithm 1 does not explicitly specify how the server aggregates the uploaded parameters (θ, γ, ξ, ν)—even though (γ, ξ) are both client-specific and maintained globally.

Question 3. The current benchmarking lacks comparisons with personalized federated learning (PFL) methods based on prototypes, contrastive learning, or generative models—such as FedProto, FedPRP, FedCCL, and pFedGM. Although these methods do not involve server-side routing, they also address personalized prediction in heterogeneous environments and thus serve as essential reference points for evaluating average accuracy. The authors should include at least FedProto and pFedGM as baselines, or explain why they are not comparable, to provide a proper context for the predictive advantages claimed by FedSPM.

Question 4. The paper does not separately evaluate the individual contributions of the hybrid model, the DRM-EL component, client-specific prediction heads, and shared representations. Furthermore, there is no report on communication overhead in comparison to the baseline methods. The authors should provide systematic ablation studies (by removing or replacing individual components) and compare the communication costs of this method against key baselines, in order to demonstrate that the observed performance gains stem specifically from the proposed semi-parametric hybrid design, rather than merely from an increase in model capacity or the routing mechanism itself.

5 comments

We thank the reviewer for the careful reading and constructive feedback. We appreciate your recognition of FedSPM’s flexible client-specific mixture model, experimental comparisons, and detailed algorithmic derivations. We address your questions and concerns below.

  • New clients do not require retraining from scratch [W1/Q1]. FedSPM separates shared representation learning from client-specific distribution modeling. The supervised and DRM encoders provide common features, while each client has its own prediction heads, mixing weights, and component-wise routing parameters. The encoder architectures do not depend on the number of clients, so a new client can reuse the representations already learned.

    When a client joins, it receives the trained encoders and initializes its new prediction heads, mixing weights, and routing parameters. Existing client parameters are retained, and the server extends the DRM bank with the new client’s entries. The reference distribution G is profiled out of the objective (Sec. 2.2) and cancels from Equation (11), so this procedure does not require explicitly reconstructing G from pooled raw data.

    The server then updates the client priors, collects refreshed responsibility summaries, and resumes federated EM from the current model. Because Equation (10) normalizes over all client–component pairs, the expanded DRM block requires further joint updates; the shared encoders can also be refined from their learned values. The required step is adaptation of the trained model, not reinitialization and training from scratch. A departing client can similarly be removed from the active bank, followed by updating the priors and summaries and refining the remaining model.

  • Performance at different client counts [W1/Q1]. We additionally assess whether FedSPM’s predictive advantage persists as the client count changes. We extended FMNIST from eight to 16, 24, and 32 clients using additional color transformations and mild image perturbations, and compared shared-encoder FedSPM (1×) with FedDRM, Ditto, ClusterFL, and FedAvgFT. Entries report system / average accuracy, with mean ± standard deviation.

    Method16 clients24 clients32 clients
    FedAvgFT19.16±1.30 / 32.86±1.3714.80±1.12 / 27.78±1.5017.21±0.33 / 30.74±1.83
    Ditto19.61±0.96 / 41.04±1.1414.30±0.32 / 38.46±2.2117.47±0.30 / 43.81±2.25
    ClusterFL19.56±0.16 / 27.72±0.7715.41±0.44 / 23.85±1.5017.15±1.29 / 23.97±1.14
    FedDRM40.07±1.04 / 43.02±1.0839.31±0.25 / 44.31±0.2940.38±1.55 / 46.19±1.94
    FedSPM (1×)43.78±2.64 / 47.15±2.9640.02±1.81 / 45.23±1.9941.69±1.14 / 47.70±1.46

    FedSPM achieves the highest mean system and average accuracies at every tested client count, providing complementary evidence of its performance across larger system sizes.

  • Objective–surrogate relationship, convergence scope, and aggregation [W2/Q2]. We distinguish why the surrogate is used, what the convergence theorem establishes, and which parameters the server aggregates.

    • Connection to the exact objective and scope of the analysis. Sec. 2.3 derives the closed-form EL multipliers at critical points of the exact DRM objective and shows that those critical points are also critical points of the tractable surrogate. This motivates the replacement, while additional conditions are needed to transfer progress along the optimization trajectory.

      With the E-step responsibilities fixed, the supervised block is weighted cross-entropy optimization. Our new analysis focuses on the DRM block, where EL profiling and multiplier approximation introduce the additional difficulty.

    • Role and empirical assessment of the bridge assumptions. Assumptions 4–5 quantify the discrepancy introduced by replacing the implicit EL multipliers. They control gradient norms and one-step objective changes without requiring gradient-direction alignment or uniformly close objective values. Sufficient-condition analyses also appear in federated optimization: FedNova assumes bounded gradient dissimilarity, and FedEM’s black-box-solver extension assumes quantitative local approximation accuracy [1, 2].

      Theorem 1 retains the accumulated bridge-error contributions explicitly, identifying when surrogate optimization preserves the standard convergence rate. Sec. 3.2 (Fig. 2) and App. D.4 numerically solve the EL system along FMNIST trajectories and show that the normalized cumulative errors remain small or decrease. This provides empirical support for the conditions in the evaluated runs. The guarantee remains conditional, and sufficient conditions checkable before training remain open.

    • Aggregation of client-specific DRM parameters. Each client’s routing parameters γγ and ξξ describe its feature distribution. Equation (10) normalizes each sample across all client–component pairs, so every training client keeps and updates a local copy of all clients’ routing parameters and the shared DRM encoder νν.

      The server averages corresponding parameters across these copies, weighted by each training client’s sample size (Algorithm 1, line 15). For example, estimates of client 1’s routing parameters are averaged together; client 2’s parameters are updated separately. The shared encoders θθ and νν are also aggregated, while the prediction heads (α,β)(α, β) and mixing weights remain local. Thus, each set of routing parameters represents one client’s distribution, while its estimates from all training clients are combined by the server. This clarifies the aggregation step questioned by the reviewer: Algorithm 1 averages local estimates of each client’s routing parameters separately, preserving distinct client distributions while allowing their parameters to be learned jointly.

  • Comparisons with prototype- and generative-model-based PFL [W3/Q3]. Following your suggestion, we evaluated FedProto and pFedGM on the three controlled benchmarks and both medical datasets. These methods are relevant comparators for personalized prediction, so the table focuses on average accuracy, reported as mean ± standard deviation.

    MethodFMNISTCIFAR-10CIFAR-100Fed-ISIC2019RETINA
    FedProto51.34±2.3242.46±0.7520.47±0.5852.96±1.8488.67±1.54
    pFedGM41.28±1.7636.71±0.4217.80±0.1362.73±0.2684.71±0.20
    FedSPM (1×)58.86±0.6248.59±0.2228.84±0.4064.42±0.2490.14±1.81
    FedSPM (C×)58.32±0.1848.32±0.6128.07±0.4665.07±0.5489.67±2.40

    Both FedSPM variants achieve higher mean average accuracy than these baselines on all five datasets. This extends the predictive comparison to the additional PFL families requested by the reviewer.

  • Component contributions, model capacity, and communication cost [W4/Q4]. We test the modeling choices separately and account for both model transmission and responsibility summaries.

    • Mixture structure. This tests whether modeling within-client heterogeneity helps beyond a single population. The component-count ablation in Sec. 3.2 and Fig. 3 reports FMNIST system accuracies of 53.51% with C=1, 57.58% with C=2, and 58.82% with C=3. The encoders are shared, so increasing the component count adds only lightweight component parameters.

    • DRM–EL and client-specific prediction heads. To test the distribution model, we replace only DRM–EL with a client-specific GMM. To test personalization beyond mixture weights, we share component-wise prediction heads across clients. The remaining settings are unchanged. The CIFAR-10 results are:

      Variant on CIFAR-10Average accuracySystem accuracy
      FedSPM (1×)48.5948.39
      FedSPM (1×), client-specific GMM46.2820.40
      FedSPM (1×), shared prediction heads38.9938.96
      FedSPM (C×)48.3248.03
      FedSPM (C×), shared prediction heads40.4540.36

      Both replacements reduce performance. The matched GMM replacement supports the contribution of semiparametric feature modeling, while the shared-head replacement supports the need for client-specific prediction rules.

    • Encoder sharing and capacity. This tests whether the gains require a separate encoder for every component. FedSPM (1×), with encoders shared across components, outperforms FedSPM (C×) in mean average and system accuracies on all three controlled benchmarks and RETINA; FedSPM (C×) performs better on Fed-ISIC2019.

      The smaller shared-encoder variant therefore shows that increased encoder capacity alone does not explain the gains. This comparison concerns sharing across components; both variants retain the global sharing of encoders across clients.

    • Communication cost. We compare the per-client, per-round communication on Fed-ISIC2019, including model-parameter exchanges and additional statistics. Prediction heads remain local; each client uploads one responsibility-summary scalar per component and receives the summaries for all client–component pairs.

      MethodCommunication (MB/client/round)
      FedAvg (1×)22.377
      FedDRM (1×)24.952
      FedEM (C×)67.130
      FedGMM (C×)67.187
      FedSPM (1×)24.965
      FedSPM (C×)74.857

      FedSPM (1×) adds approximately 0.05% communication relative to FedDRM and uses approximately 66.65% less communication than FedSPM (C×). The additional cost of the latter primarily reflects its component-specific encoders.

References

  1. Wang, J., Liu, Q., Liang, H., Joshi, G., and Poor, H. V. (2020). Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization. NeurIPS.
  2. Marfoq, O., Neglia, G., Bellet, A., Kameni, L., and Vidal, R. (2021). Federated Multi-Task Learning under a Mixture of Distributions. NeurIPS.
Reviewer_AmxFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies routing-based federated image classification when data differ both across clients and within each client. It proposes FedSPM, which models each client as a mixture of latent components and uses a semiparametric density-ratio model to route test images to a client. The selected client combines its component classifiers for prediction. A federated EM procedure trains the model, and experiments on constructed image benchmarks and Fed-ISIC2019 report higher system and average accuracy than the compared methods.

Critical strengths
  1. The paper studies an interesting routing-based FL setting that considers both inter-client and intra-client heterogeneity.
  2. The experimental comparisons cover a reasonably broad set of FL baselines, including tests on both controlled benchmarks and a real medical dataset.
Critical weaknesses
  1. The paper does not compare single-client routing with voting or averaging predictions from multiple clients. Without this baseline and its inference cost, the need to select only one client remains unclear.
  2. Equation 11 routes by input density, which need not indicate predictive accuracy. On Fed-ISIC2019, routing accuracy measures return to the source center, not selection of the best classifier.
  3. The paper does not show why Gaussian modeling is inadequate. FedGMM differs in several respects; a matched per-client GMM comparison is needed to isolate the effect of the distribution model.
  4. Synthetic shifts use image transformations and component-specific label permutations, making visible differences predictive by construction. Their relevance to real patient heterogeneity remains untested.
  5. The main benchmarks are artificially modified FMNIST and CIFAR datasets, with only one real medical dataset. This limits the evidence for naturally occurring clinical heterogeneity.
  6. The ablation varies component count only on FMNIST and does not isolate the effects of client-specific predictors, mixture modeling, or the distribution model.
  7. The paper reports the size of responsibility summaries but not full system costs. Runtime, model size, and communication should be measured as clients and components increase.
Questions for authors
  1. Why is single-client routing preferable to voting or averaging predictions from several clients? Are comparable results or inference-cost estimates available?
  2. Does higher routing density predict lower classification error across clients? Can the authors clarify this relationship beyond routing to the source center?
  3. What evidence shows that Gaussian modeling is inadequate here? Which differences between FedSPM and FedGMM could also explain their performance gap?
  4. How do the synthetic transformations and label permutations reflect clinically relevant patient differences? Is there evidence that the learned components capture differences that affect prediction on Fed-ISIC2019?
  5. Are results available that separate the effects of client-specific predictors, mixture modeling, and the distribution model on the harder datasets?
  6. What are the training, inference, and communication costs as the numbers of clients and components increase?
7 comments

We thank the reviewer for the careful reading and constructive suggestions. We appreciate your interest in our study of routing-based federated learning under both inter-client and intra-client heterogeneity. We also appreciate your positive assessment of the breadth of our comparisons with federated learning baselines and the inclusion of both controlled benchmarks and a real medical dataset. We address your questions and concerns point by point below.

  • Single-client routing, alternative inference strategies, and inference cost [W1/Q1]. Single-client routing offers a way to exploit client specialization while limiting each query to one predictive client. We assess its practical value against two alternative inference strategies for baselines without a routing mechanism: soft voting and highest-confidence voting.

    • Comparison with soft and highest-confidence voting. Following your suggestion, we evaluated soft voting, which averages predictive probabilities across clients before taking the argmax, and highest-confidence voting, which selects the prediction from the client with the highest predictive confidence. Both evaluations use the same trained models without retraining.

      The table reports system accuracy. Paired entries report soft voting / highest-confidence voting; identical results are reported once. Routing-capable methods retain their routing-based inference.

      MethodFMNISTCIFAR10CIFAR100Fed-ISIC2019
      FedAvg39.15 ± 1.3135.02 ± 0.3218.17 ± 0.5360.91 ± 0.58
      FedProx39.14 ± 1.3234.94 ± 0.3618.15 ± 0.4960.95 ± 0.47
      FedAvgFT36.38 ± 1.90 / 34.05 ± 1.2933.23 ± 0.32 / 29.38 ± 1.3017.31 ± 0.62 / 15.35 ± 0.3656.74 ± 1.52 / 54.86 ± 1.45
      Ditto34.99 ± 1.38 / 34.46 ± 0.9034.28 ± 0.31 / 31.62 ± 0.8217.25 ± 0.54 / 16.02 ± 0.7359.57 ± 0.13 / 57.53 ± 0.38
      FedBABU35.20 ± 0.93 / 35.00 ± 0.4033.90 ± 0.45 / 33.65 ± 0.4916.84 ± 0.24 / 16.72 ± 0.4159.59 ± 0.09 / 56.98 ± 0.35
      ClusterFL39.68 ± 2.21 / 31.05 ± 2.1133.07 ± 0.36 / 27.23 ± 3.3618.37 ± 0.29 / 15.10 ± 3.0256.92 ± 1.92 / 51.81 ± 2.45
      FedEM35.59 ± 0.76 / 29.82 ± 3.2431.01 ± 0.76 / 27.60 ± 3.4815.91 ± 0.67 / 13.21 ± 0.6453.39 ± 1.08 / 51.68 ± 0.57
      FedGMM31.98 ± 1.9925.36 ± 0.4811.97 ± 0.9658.62 ± 1.19
      FedDRM53.68 ± 1.0147.47 ± 0.0927.15 ± 0.3761.26 ± 0.30
      FedSPM (1×)58.82 ± 0.6148.39 ± 0.2128.33 ± 0.3762.89 ± 0.19
      FedSPM (C×)58.23 ± 0.1748.03 ± 0.5827.54 ± 0.4563.43 ± 0.67

      As shown in the table, soft voting generally outperforms highest-confidence voting. Nevertheless, FedSPM maintains the highest mean system accuracy across all four datasets. These comparisons support its practical performance against the evaluated voting strategies for the baselines.

    • Inference efficiency of our proposed approach. When applied across mm distinct client models, both soft voting and highest-confidence voting require obtaining predictions from all mm clients: soft voting averages their probabilities, while highest-confidence voting compares their predictive confidence. In a distributed deployment, this entails O(m)O(m) client–server exchanges and mm client-model evaluations per query. FedSPM computes routing scores at the server, sends the query to one selected client, and receives its prediction, requiring O(1)O(1) client–server exchanges.

      The communication saving is the practical motivation for single-client routing. It does not imply that single-client prediction is universally more accurate than either alternative voting strategy. Server-side routing computation is accounted for separately in our system-cost analysis below.

  • Justification of the proposed routing mechanism and supporting evidence [W2/Q2]. We clarify why distributional compatibility provides a criterion for selecting a population-specific predictor, and distinguish this rationale from identifying a correct predictor for every individual query.

    In the ideal population setting, each client’s predictor represents its true conditional label distribution and is therefore Bayes-optimal for its source population. Identifying the source client thus identifies an appropriate population-specific classifier. The feature distribution describes what inputs look like, while the conditional label distribution describes how labels relate to those inputs. Feature compatibility therefore helps identify which population-specific prediction rule applies. Equation (11) combines this compatibility with the client prior to select the most probable source.

    Correct source assignment reproduces the source-informed Bayes prediction, so additional classification errors can occur only on incorrectly assigned queries. MAP routing minimizes this assignment error, providing a population-level justification. Overlapping populations can nevertheless leave source uncertainty even with exact models.

    In practice, FedSPM jointly learns the DRM embedding and density-ratio parameters to capture nonlinear relationships in the input space. Together with the nonparametric baseline and client-specific mixtures, this flexibility helps mitigate misspecification. Approximation and estimation errors affect how closely the fitted models realize the population framework, which we have assessed empirically. The empirical results on the semi-synthetic benchmark and real-world medical datasets show the empirical gain of the proposed approach.

  • Matched Gaussian comparison and the role of the distribution model [W3/Q3]. To directly test the effect of Gaussian feature modeling, we conducted the matched comparison you suggested: we replaced FedSPM’s DRM–EL block with a client-specific GMM, keeping all other model components and experimental settings unchanged. The CIFAR-10 results are:

    MethodAverage accuracySystem accuracy
    FedSPM (1×), DRM–EL (our)48.59 ± 0.2248.39 ± 0.21
    FedSPM with a client-specific GMM46.28 ± 0.9420.40 ± 1.07
    FedGMM38.86 ± 0.4225.36 ± 0.48

    Replacing DRM–EL with the GMM reduces system accuracy by 27.99 percentage points, compared with a 2.31-point reduction in average accuracy. The much larger reduction in system accuracy supports the importance of the feature-distribution model for routing: under otherwise matched settings, the Gaussian replacement is substantially less effective.

    FedGMM also differs from FedSPM in its predictive model: it personalizes mixing weights, whereas FedSPM additionally uses client-specific classification heads. These differences can also contribute to their performance gap. The matched replacement above controls for these predictive components and isolates the effect of changing the feature-distribution model.

  • Clinical relevance of controlled shifts and evidence from natural heterogeneity [W4/W5/Q4]. Our controlled benchmarks isolate two statistical challenges: changes in input appearance and changes in the relationship between inputs and labels. The image transformations provide controlled perturbations of appearance, motivated broadly by differences in acquisition conditions. The label permutations stress-test adaptation to concept shift; they are not intended to reproduce specific clinical labeling processes or patient subtypes. These constructions follow the component-level transformations and label permutations in FedGMM (Wu et al., 2023) and the client-level transformations in FedDRM (Wang et al., 2026).

    To assess whether the framework remains useful beyond these constructed shifts, we evaluate Fed-ISIC2019, where heterogeneity arises naturally across six clinical centers without imposing these transformations or label permutations. As shown in Fig. 4, FedSPM (1×) outperforms FedDRM in average, system, and source-center routing accuracies. We additionally evaluate RETINA, a medical dataset of retinal fundus images from multiple institutions, to extend the evidence beyond a single medical dataset. This setting also uses naturally occurring heterogeneity without artificially imposed distribution shifts. The table reports average and system accuracies, with mean ± standard deviation.

    MethodAverage AccSystem Acc
    FedAvg85.11 ± 1.2285.11 ± 1.22
    FedProx85.31 ± 0.8885.31 ± 0.88
    FedAvgFT86.12 ± 0.2083.90 ± 0.80
    Ditto87.39 ± 0.1283.90 ± 1.72
    FedBABU84.98 ± 0.8184.57 ± 0.76
    ClusterFL87.59 ± 1.4179.34 ± 2.63
    FedEM86.65 ± 0.1278.94 ± 3.60
    FedGMM85.45 ± 2.0684.78 ± 1.81
    FedDRM85.92 ± 1.0186.45 ± 0.95
    FedSPM (1×)90.14 ± 1.8189.94 ± 1.52
    FedSPM (C×)89.67 ± 2.4089.54 ± 2.45

    Both FedSPM variants outperform all compared baselines in mean average and system accuracies on RETINA. Together with Fed-ISIC2019, these results support the predictive usefulness of our framework on two real medical datasets with natural heterogeneity. The comparison with FedDRM provides complementary evidence for the mixture formulation, although it compares complete methods and does not fully isolate the contribution of the latent components.

    To assess sensitivity to label conflicts, we varied only the label permutations on FMNIST, keeping client partitions and feature transformations fixed. We compare FedDRM with the shared-encoder FedSPM (1×) under three mappings: none (common across all clients and components), global (component-specific, shared across clients), and local (client–component-specific, the original setting). Results are system accuracy as mean ± standard deviation over three seeds.

    Label permutationFedDRMFedSPM (1×)Gain
    None (common label mapping)65.68 ± 0.5366.96 ± 1.53+1.28
    Global63.48 ± 0.9964.66 ± 0.23+1.18
    Local (original setting)53.68 ± 1.0158.82 ± 0.61+5.14

    FedSPM achieves higher mean accuracy in all settings, retaining a 1.28-percentage-point gain without label permutations, compared with 5.14 points in the original setting.

Reviewer_ZkrfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FedSPM, a routing-enabled federated learning framework that addresses both inter-client and intra-client heterogeneity. It represents each client as a mixture of client-specific latent components, each combining a conditional label model for prediction with a feature distribution for routing. Shared representations enable information sharing, while feature distributions are modeled through density ratios relative to a shared nonparametric baseline estimated using empirical likelihood. The authors develop a federated EM procedure with a tractable surrogate objective and establish a conditional convergence guarantee for the DRM parameter block under additional assumptions linking the surrogate to the exact profiled objective.

Validation includes controlled experiments on Fashion-MNIST, CIFAR-10, and CIFAR-100, alongside a case study on the six-center Fed-ISIC2019 medical dataset. The reported results show improvements over global, personalized, mixture-based, and routing-based baselines in system and average accuracy, with additional analyses of heterogeneity settings, component count, model capacity, and momentum.

Critical strengths
  1. A coherent formulation of routing and prediction under dual heterogeneity. FedSPM models each client using client-specific latent components, each coupling a feature distribution for routing with a conditional label distribution for prediction. This allows it to capture within-client variation while accommodating differences across clients. The formulation is consequential because it extends routing-based personalization to a setting where treating each client as a homogeneous population can be inadequate.

  2. A tractable federated learning procedure with evidence of benefits beyond increased model capacity. The surrogate-based EM procedure makes the empirical-likelihood formulation trainable without repeatedly solving the nonlinear multiplier system, while communicating component-level responsibility summaries rather than per-sample responsibilities. Experiments include both controlled benchmarks and a real medical dataset; notably, the shared-encoder variant improves over FedDRM at comparable encoder capacity. This supports the practical value of the proposed modeling approach and strengthens the case that its gains are not solely attributable to larger models.

Critical weaknesses
  1. The routing objective does not directly establish the claimed selection of client expertise. Equation (11) selects the client most likely to have generated the input, and the medical experiment measures routing accuracy by recovery of the source clinical center. Neither necessarily identifies the client whose predictor is most accurate for that input. Although system accuracy provides useful downstream evidence, the paper does not directly quantify this gap, for example through comparison with an oracle client selector. This limits support for the central interpretation that distribution-based routing reliably selects the most suitable predictive expert.

  2. The experimental setting limits how broadly the reported gains can be interpreted. The controlled benchmarks introduce independent label permutations for each client–component pair, creating a setting particularly favorable to client-specific predictive heads. Moreover, system accuracy for methods without routing is evaluated through majority voting, which can be poorly suited to these conflicting label mappings. The real medical experiment provides useful complementary evidence, but the main-text results do not sufficiently isolate the effects of these design choices. Consequently, it remains unclear how much of the advantage persists under less artificial concept shifts and alternative inference strategies for the baselines.

Questions for authors
  1. Clarification on routing. On Fed-ISIC2019, routing accuracy measures how often Equation (11) selects the image’s source center. When the selected client misclassifies an image, how often does another client classify it correctly? If you have predictions from all clients, this comparison would help assess how closely source matching tracks predictive expertise.

  2. Clarification on the benchmark results. How sensitive is FedSPM’s system accuracy gain over FedDRM to the random label permutation applied to each client–component pair? Do you have results with the same client partitions and feature transformations but a common label mapping? Since both methods support routing, this comparison would help establish whether FedSPM’s advantage persists when the imposed label conflicts are removed.

5 comments

We thank the reviewer for the careful reading and constructive feedback. We appreciate your recognition that FedSPM coherently couples routing and prediction to address both inter-client and intra-client heterogeneity, and that its surrogate-based EM procedure makes the empirical-likelihood formulation tractable. We also appreciate your assessment that the shared-encoder comparison supports benefits beyond increased model capacity, alongside the evidence from controlled benchmarks and the real medical dataset. We address your questions and concerns point by point below.

  • Justification of the proposed routing mechanism and supporting evidence [W1/Q1]. We thank the reviewer for suggesting the oracle comparison. We clarify the probabilistic basis of our routing mechanism, distinguish source-informed prediction from sample-level oracle selection, and compare the predictive potential of the learned client models.

    • Probabilistic justification for routing. In the ideal population setting, each client’s predictor represents its true conditional label distribution and is therefore Bayes-optimal for its source population. Identifying the source client thus identifies an appropriate population-specific classifier. The feature distribution describes what inputs look like, while the conditional label distribution describes how labels relate to those inputs. Feature compatibility therefore helps identify which population-specific prediction rule applies. Equation (11) combines this compatibility with the client prior to select the most probable source.

      Correct source assignment reproduces the source-informed Bayes prediction, so additional classification errors can occur only on incorrectly assigned queries. MAP routing minimizes this assignment error, providing a population-level justification. Overlapping populations can nevertheless leave source uncertainty even with exact models.

      In practice, FedSPM jointly learns the DRM embedding and density-ratio parameters to capture nonlinear relationships in the input space. Together with the nonparametric baseline and client-specific mixtures, this flexibility helps mitigate misspecification. Approximation and estimation errors affect how closely the fitted models realize the population framework, which we assess empirically below.

    • Two oracle comparisons: the sample-level oracle is unavailable at deployment. An oracle comparison depends on what additional information the selector receives. We distinguish knowing the query’s source from knowing which client predicts it correctly, because these assess source-specific prediction and the full predictive potential of the client-model pool, respectively.

      • The source-informed reference assigns each query to its known source client. This setting has already been evaluated through the reported average accuracy, where FedSPM achieves the highest accuracy among the compared methods.
      • The sample-level oracle counts a query as correct whenever at least one client predicts its label correctly. We report this comparison below to assess the accuracy attainable through optimal, label-informed selection.

      Crucially, obtaining all clients’ predictions does not reveal which prediction is correct. The oracle determines correctness using the true query label, which is precisely what the system must predict. Its selection rule therefore cannot be implemented for unlabeled queries at deployment and serves only as a retrospective benchmark.

    • Comparison with oracle routing. Following your suggestion, we evaluated a sample-level oracle that counts a query as correct whenever at least one client predicts its label correctly. This measures the maximum accuracy attainable through label-informed selection from each method’s trained client-model pool. The Fed-ISIC2019 results are reported below.

      MethodSample-level oracle accuracy
      FedAvg60.93 ± 0.72
      FedAvgFT75.77 ± 1.63
      FedProx60.97 ± 0.56
      FedBABU74.88 ± 0.99
      Ditto78.93 ± 0.73
      ClusterFL73.63 ± 3.70
      FedEM76.27 ± 1.04
      FedGMM73.08 ± 1.61
      FedDRM79.35 ± 0.71
      FedSPM (1×)81.48 ± 1.57
      FedSPM (C×)81.35 ± 0.49

      FedSPM achieves the two highest mean sample-level oracle accuracies, indicating that its client predictors collectively cover more correctly classifiable queries than the compared baselines. Both variants also exceed all baseline approaches. These results support greater predictive coverage together with improved practical performance.

  • Empirical gains over other baselines under new routing strategies [W2]. Since most baselines do not provide a routing mechanism, we used majority voting to obtain their system-level predictions. Following your suggestion, we evaluated soft voting, which averages predictive probabilities across clients before taking the argmax, and highest-confidence voting, which selects the prediction from the client with the highest predictive confidence. Both evaluations use the same trained models without retraining.

    The table reports system accuracy. Paired entries report soft voting / highest-confidence voting; identical results are reported once. Routing-capable methods retain their routing-based inference.

    MethodFMNISTCIFAR10CIFAR100Fed-ISIC2019
    FedAvg39.15 ± 1.3135.02 ± 0.3218.17 ± 0.5360.91 ± 0.58
    FedProx39.14 ± 1.3234.94 ± 0.3618.15 ± 0.4960.95 ± 0.47
    FedAvgFT36.38 ± 1.90 / 34.05 ± 1.2933.23 ± 0.32 / 29.38 ± 1.3017.31 ± 0.62 / 15.35 ± 0.3656.74 ± 1.52 / 54.86 ± 1.45
    Ditto34.99 ± 1.38 / 34.46 ± 0.9034.28 ± 0.31 / 31.62 ± 0.8217.25 ± 0.54 / 16.02 ± 0.7359.57 ± 0.13 / 57.53 ± 0.38
    FedBABU35.20 ± 0.93 / 35.00 ± 0.4033.90 ± 0.45 / 33.65 ± 0.4916.84 ± 0.24 / 16.72 ± 0.4159.59 ± 0.09 / 56.98 ± 0.35
    ClusterFL39.68 ± 2.21 / 31.05 ± 2.1133.07 ± 0.36 / 27.23 ± 3.3618.37 ± 0.29 / 15.10 ± 3.0256.92 ± 1.92 / 51.81 ± 2.45
    FedEM35.59 ± 0.76 / 29.82 ± 3.2431.01 ± 0.76 / 27.60 ± 3.4815.91 ± 0.67 / 13.21 ± 0.6453.39 ± 1.08 / 51.68 ± 0.57
    FedGMM31.98 ± 1.9925.36 ± 0.4811.97 ± 0.9658.62 ± 1.19
    FedDRM53.68 ± 1.0147.47 ± 0.0927.15 ± 0.3761.26 ± 0.30
    FedSPM (1×)58.82 ± 0.6148.39 ± 0.2128.33 ± 0.3762.89 ± 0.19
    FedSPM (C×)58.23 ± 0.1748.03 ± 0.5827.54 ± 0.4563.43 ± 0.67

    As shown in the table, soft voting generally outperforms highest-confidence voting. Nevertheless, FedSPM maintains the highest system accuracy across all datasets. More importantly, under distributed inference, both voting strategies require O(m)O(m) client-server communications per query to aggregate predictions from mm clients, whereas FedSPM requires only O(1)O(1) by routing each query to a single client. These results demonstrate that FedSPM's advantage persists under alternative voting strategies while offering more communication-efficient inference.

  • Purpose of controlled benchmarks [W2]. Our semi-synthetic benchmarks follow the mixture decomposition PiX,Y=∑cπicPicX,YP_i^{X,Y}=\sum_{c}π_{ic}P_{ic}^{X,Y}. They serve three purposes. First, they provide stress tests under severe dual heterogeneity, allowing us to evaluate the robustness of FedSPM against existing methods. Second, the explicitly controlled distribution shifts enable systematic evaluation of different types and intensities of heterogeneity. Third, our design follows established practices in prior work, including the component-level transformations and label permutations used in FedGMM (Wu et al., 2023) and the client-level transformations adopted in FedDRM (Wang et al., 2026), ensuring comparability with existing studies.

    Importantly, the baselines also include mechanisms for handling concept shift: FedDRM, FedBABU, Ditto, and FedAvgFT employ personalized classification heads, while FedEM and FedGMM can capture concept shift through client-specific mixing weights. Thus, our controlled benchmarks provide a challenging setting for systematically evaluating the effectiveness of different approaches to handling concept shift.

  • Sensitivity to label permutation & baselines with common label mapping [Q2]. We conducted additional experiments on FMNIST using the same client partitions and feature transformations, while varying only the label permutations. To control for encoder capacity, we compare the best baseline FedDRM with FedSPM (1×), which shares encoders across components.

    We consider three settings: none, where all clients and components retain the original, common label mapping; global, where component-specific label permutations are shared across clients; and local, where label permutations are specific to each client–component pair, as in our original experiments. The none setting directly tests whether FedSPM’s advantage persists after removing the imposed label conflicts.

    The table reports system accuracy as mean ± standard deviation over three seeds.

    Label permutationFedDRMFedSPM (1×)Gain
    None (common label mapping)65.68 ± 0.5366.96 ± 1.53+1.28
    Global63.48 ± 0.9964.66 ± 0.23+1.18
    Local (original setting)53.68 ± 1.0158.82 ± 0.61+5.14

    FedSPM achieves higher mean system accuracy than FedDRM in all settings. Its gain decreases from 5.14 percentage points under the original client–component-specific permutations to 1.28 percentage points under a common label mapping. Thus, the imposed label conflicts affect the magnitude of the gain, while a smaller numerical advantage remains when these conflicts are removed.

Comments on the paper

1 comment

Dear Reviewers,

As the rebuttal period is nearing its close, please review the authors' responses and indicate whether your concerns have been adequately addressed. Based on your assessment, please provide your final rating. If significant concerns remain, please briefly explain them.

Thank you for your time.

Best regards,

AC for this paper

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.