acceptodds
Under review as a conference paper at ICLR 2027

FedSP: Federated Shared-Personalized Learning for Domain Generalization

Abstract

Federated Domain Generalization (FedDG) aims to learn models that generalize to unseen domains from distributed client data without accessing clients' raw data. However, existing federated learning methods typically promote collaboration by aggregating client models into a shared global model, implicitly assuming that knowledge beneficial to one client can be effectively shared across all clients. In the presence of domain shift, client-specific visual variations may become entangled with domain-invariant semantic knowledge, making it challenging for the model to determine which knowledge should be shared and which should be personalized. To address this challenge, we propose Federated Shared-Personalized Learning for Domain Generalization (FedSP), which explicitly decouples shared knowledge from client-specific knowledge during federated learning. FedSP employs category prototype-driven cross-domain semantic alignment in the shared module and residual adaptation of domain-inherent discriminative attributes in the personalized module to learn transferable domain-invariant shared representations while modeling domain-specific semantic variations. Additionally, FedSP introduces a multi-scale uncertainty-constrained learning strategy to enhance the model's adaptability across diverse domain shifts. Extensive experiments on multiple domain generalization benchmarks demonstrate that FedSP consistently improves model generalization to unseen domains and outperforms existing federated learning and domain generalization methods. Further analysis shows that explicitly decoupling shared and personalized knowledge facilitates more effective federated collaboration and enhances model robustness under significant domain shifts.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Domain Generalization;Domain Shift;Semantic Alignment, Feature Adaptation, Uncertainty-Aware Aggregation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XtdoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FedSP, a federated domain generalization method for settings where each client holds data from a single source domain, with the goal of generalizing to unseen domains. Evaluation follows the leave-one-domain-out protocol, with each source domain split across two clients. The method has three modules. In CSAS, the server aggregates class prototypes computed by clients in two stages, within and across domains, based on reliability, and prototype contrastive losses (PCL and CCPL) align features across domains. DSRA adds a bottleneck-MLP residual to encoder features, which is claimed to complement shared semantics with domain-specific discriminative attributes; however, its parameters are also aggregated on the server. MUCL combines client-side regularization via evidential learning with server-side aggregation that mixes sample-size-based weights with a term assigning larger weights to clients on which the global model has higher predictive entropy. Using an ImageNet-pretrained ResNet-50 as the backbone, the paper reports higher average accuracy than existing methods on PACS, Office-Home, Office-Caltech10, and Terra Incognita.

Critical strengths

The paper addresses a well-motivated question in FedDG, namely how to handle shared versus domain-specific knowledge. It organizes the method around three clear questions (what to share, what to adapt, and how to constrain learning), which makes the overall design easy to follow. FedSP achieves the best average accuracy on all four benchmarks. The improvement of about 6 points over the FedAlign baseline on Terra Incognita, a realistic dataset with large domain shifts, is particularly convincing. In the ablation study, adding or removing each module affects performance in a consistent direction, indicating that the components work well together. The hierarchical prototype aggregation (HPA), which uses sample counts, variance, and cross-domain consistency, is a concrete and easily implementable design and forms the core of CSAS, the module with the largest contribution on PACS. The comprehensive hyperparameter table and the stated code release also reflect good attention to reproducibility.

Critical weaknesses

First, the meaning of "personalized" is unclear. Since DSRA parameters are aggregated on the server and end up shared by all clients, there is no personalized component in the usual sense. No constraint or analysis supports the claim that ΔH carries domain-specific information, and its contribution in the ablation is small. Second, the motivation for and analysis of each module are thin. ELR and PEA are not explicitly connected, and PEA's upweighting of uncertain clients runs opposite to the introduction's framing of "client reliability." Third, the ablation is conducted only at the module level with a single seed, so the contributions of PCL versus CCPL and ELR versus PEA are not separated. Fourth, the evaluation setting is limited. There are only two clients per domain, widely used datasets such as DomainNet and VLCS are not included, and neither standard deviations nor the model selection procedure is reported. In addition, the text states margins over the second-best method of 0.54 points on Office-Caltech10 and 5.89 points on Terra Incognita, but these are margins over FedAlign. The margins over the actual second-best methods in the tables (DI and GA) are only 0.07 and 2.95 points, respectively. Finally, the relationship to CLIP-based methods that use domain-specific prompts is not discussed. Combined with the fact that each component builds on existing techniques, this leaves the novelty weakly positioned.

Questions for authors
  1. Definition of "personalized" and the role of DSRA: DSRA parameters are aggregated on the server and end up shared by all clients. What does "personalized" refer to in this paper? Please explain why these parameters are aggregated, and compare performance against keeping them per client without aggregation. Can you also provide analysis showing that ΔH actually carries domain-specific information?

  2. Granularity of the ablation study: The current ablation is conducted at the module level with a single seed. Can you provide results that separately remove PCL and CCPL, and ELR and PEA, over multiple seeds?

  3. Scale of the evaluation and scalability: Please state the total number of clients and how each domain is split. Can you report results with more clients per domain (e.g., 10), on DomainNet (or miniDomainNet) and VLCS, and with standard deviations for the main experiments? Also, since CCPL's communication grows with the number of clients times the number of classes, how do you view its applicability in large-scale settings?

  4. Relationship to CLIP-based Federatd methods: CLIP-based federated methods that use domain-specific prompts (e.g., FedDSPG, ADAPT) also separate shared and domain-specific knowledge. Please clarify how FedSP differs from these methods and what its novelty is relative to them. Are CSAS and DSRA applicable on top of frozen CLIP features?

Reviewer_xRDrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a federated learning method for domain generalization that explicitly shares semantic and domain-specific knowledge. The authors point out that prior work implicitly assumes that knowledge beneficial to one client can be effectively shared across all clients, while ignoring domain shift and client-specific visual variations. From this point, the authors argue that the FL method should determine which knowledge should be shared and which should be personalized.

Critical strengths

The paper evaluates FedSP on PACS, OfficeHome, Office-Caltech10, and Terra Incognita benchmarks and reports improvements over several FedDG baselines. The paper tackles an important problem. The motivation is intuitive, and the empirical results are generally favorable.

Critical weaknesses

The central claim of this paper is to design a method that determines which knowledge should be shared and which should be personalized. In other words, FedSP separates shared semantic knowledge and domain-specific knowledge in its learning process. However, the paper provides no evidence that such disentanglement is actually occurring. The only supporting evidence is improved classification accuracy.

Communication costs need to be considered and compared, since this method requires exchanging both model parameters and prototypes.

Novelty is moderate. Most individual components have prior work. The main contribution of DSRA and MUCL appears to be the combination of these ideas.

  • DSRA appears to be a lightweight residual adaptation module. While effective, its novelty over existing personalized adaptation mechanisms is unclear, and the paper does not sufficiently isolate the performance gain arising from increased model capacity versus the proposed shared-personalized learning strategy.
  • The main contribution is the CSAS module, which addresses the main problem posed by this paper. CSAS prioritizes prototypes based on sample counts, variance, and cross-domain similarity. However, none of these quantities explicitly characterize semantic invariance. As a result, it remains unclear why the resulting global prototype should correspond to "shared semantic knowledge" rather than simply a more stable prototype estimate.
Questions for authors
  1. How can you demonstrate that CSAS actually captures shared semantic information rather than serving as an additional regularization term? The proposed reliability score and inter-domain similarity score appear to promote prototype consistency across domains. However, consistency does not necessarily imply semantic invariance.
  2. What is the motivation for using two augmented views instead of one? Have the authors evaluated whether both augmented views are necessary, and what is the performance impact of removing one branch? And why is only H~ (not H^)used in the aggregation process of equations (1) and (2)?
  3. What is the communication overhead compared to the other method?
  4. For PCC, do clients need access to prototype banks from all participating clients for equation (10)? If yes, how does communication scale with the number of clients?
  5. Equation (17) shows that clients with high uncertainty awareness should give more weight in the aggregation. However, uncertainty can come from the client data, not from how the model learns, including hard, noisy, or mislabeled domains. Why should higher uncertainty always indicate a more informative client rather than a noisy or poorly fitted one?
Reviewer_9pXPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a Federated Shared-Personalized Learning approach for Federated Domain Generalization (FDG), which aims to disentangle shared and client-specific knowledge and leverage both during training. While existing FDG methods primarily focus on aggregating knowledge across clients, the proposed approach aims to separate personalized knowledge to account for domain-specific semantic variations. For this purpose, the authors propose CSAS, DSRA, and MUCL modules. The proposed method is evaluated on several benchmark datasets, including PACS and Office-Home, and demonstrates generally superior performance compared to existing approaches.

Critical strengths
  • The idea of distinguishing between shared and personalized knowledge and leveraging both during training is reasonable and well-motivated.
  • The individual modules are well explained with appropriate mathematical formulations.
  • The paper is easy to follow.
Critical weaknesses
  • The analysis of the experimental results and ablation studies is quite limited. More detailed discussions and analyses are needed to better explain the observed performance improvements and the contributions of individual components. In addition, experimental results should report standard deviations too.
  • The CSAS and DSRA modules do not appear to incorporate explicit architectural designs or loss functions specifically intended to disentangle personalized information from shared knowledge. The authors should clarify how these modules achieve such disentanglement and provide further justification for their effectiveness.
  • The PEA module requires additional inference using the global model on local data, which may not be well-suited to the federated learning setting. The authors should further justify the necessity of this additional inference step and discuss its compatibility with the practical constraints of federated learning.
Questions for authors
  • Fig. 1 conveys general information and does not sufficiently highlight the main contributions of the paper, despite its relatively large size in the paper. The figure should be revised to more effectively illustrate the key motivations and contributions of the proposed method.
  • In CSAS, simply generating augmented views followed by aggregation may not be sufficient to eliminate client-specific information, as such information is likely to remain in the aggregated prototypes. Furthermore, the notation H^\hat{H} is unused in Eqs. 1 and 2.
  • In Eq. 13, the DSRA module appears to reduce the diversity of client-specific features through aggregation rather than explicitly leveraging their diversity. This seems somewhat contradictory to the objective of effectively utilizing client-specific information, as aggregation may suppress the distinctive characteristics of individual clients. The authors should provide a clearer justification for this design choice and explain how it contributes to preserving or leveraging personalized knowledge.
  • In Eq. 15, the PEA module requires the global model to perform predictions on local data. If this inference is performed on the server, it would conflict with the conventional federated learning assumption that local data remain inaccessible to the server. Even if the inference is performed locally, it introduces additional computational overhead, potentially reducing the overall efficiency of the proposed framework. The authors should clarify where this inference is performed and provide a more detailed analysis of its computational cost and compatibility with the federated learning setting.
  • Although the experimental results generally demonstrate performance improvements, the results for the C domain in PACS and the L38L_{38} domain in Terra Incognita exhibit different trends. A more detailed analysis is needed to explain why the proposed method behaves differently in these domains and to identify the factors contributing to these performance variations.
  • L282: range.After -> range. After

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.