acceptodds
Under review as a conference paper at ICLR 2027

pFLAIR: personalized Federated Learning using Attention-Informed model Recombination

Abstract

Personalized federated learning (pFL) has emerged as a key approach for addressing statistical heterogeneity, while retaining the privacy and collaboration benefits of federated learning. However, most existing pFL methods place the bulk of personalization on the client side, often adding extra local modules, inner-loop optimization, or per-client adaptation steps, which can be computationally and memory intensive, and contrary to a core practical motivation of FL: keeping client workloads and on-device model footprints small. This is especially limiting under label skew, the most realistic heterogeneity regime, where clients observe different subsets or frequencies of classes reflecting natural specialization in real-world deployments. We propose pFLAIR, a pFL framework that shifts a substantial part of personalization to the server by learning an explicit, trainable mixing mechanism that determines how client knowledge should be combined. Crucially, unlike several competitive pFL methods that rely on communicating explicit label frequencies or class-conditioned statistics to achieve their best performance, pFLAIR operates without requiring explicit label-distribution statistics. This avoids exposing such statistics to the server, which is the very premise of federated learning. Across heterogeneous settings, pFLAIR consistently outperforms strong pFL baselines, with particularly large gains under challenging label skew, and degrades far more gracefully than competing methods as the number of clients grows, demonstrating a better scalability for federated deployments.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated learning, server-centric, pFL

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_HCc3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary
  • This paper proposes pFLAIR, a pFL method that shifts the personalization workload to the server, arguing that client-side adaptation is costly for resource-limited devices.
  • Instead of local personalization steps, the server learns how to combine client models through a trainable mixing mechanism, and it does so without clients disclosing explicit label-distribution statistics.
  • The reported results show that pFLAIR outperforms existing pFL methods.
Critical strengths
  • The paper addresses an important practical problem of reducing the client-side burden of personalization by shifting it to the server, which is well suited to practical FL settings such as IoT deployments.
  • The proposed mechanism is reasonably well structured, integrating class-wise attention into a coherent, end-to-end trainable pipeline.
Critical weaknesses
  • One of the paper's main claimed advantages is not well supported. Although the paper states that clients upload only compact summaries, each client in fact sends its full model parameters and class-embedding tables. This is generally more information than most pFL methods communicate, which may largely account for its performance gains.
  • More importantly, contrary to the authors' discussion in the appendix, uploading both the model and the class embeddings can raise privacy concerns, as their combination can reveal information about each client's local data. Since Liemb\mathcal{L}_i^{\text{emb}} pulls each row of EipersE_i^{\text{pers}} toward its class features, EipersE_i^{\text{pers}} effectively serves as class prototypes carrying feature-level information. In addition, the claim that competitive pFL methods rely on per-class count information is not always accurate, since methods like FedTGP aggregate prototypes without using class-wise sample counts.
Questions for authors
  • Q1. The paper uses different alignment losses for Limse\mathcal{L}_i^{\text{mse}} and Liemb\mathcal{L}_i^{\text{emb}}, namely MSE and cosine similarity, respectively. Although the authors briefly justify this choice, further clarification would be helpful. Is there an ablation study or a clearer intuition supporting this design?
  • Q2. Beyond the computational cost in the appendix, what is the server's memory requirement? Does the server generate and store a distinct mixed model for each client at every round?
  • Q3. The reviewer appreciates the comparison with many baselines, but would it be more informative to compare against more closely related approaches, such as representation aggregation methods (e.g., FedTGP) or class-wise aggregation methods (e.g., cwFedAvg)?
2 comments

We thank the reviewer for the feedback.

W1. Information communicated.

The compact objects in pFLAIR are those the server uses to learn the mixing: the head descriptors and the class embeddings. The model is uploaded as in all parameter-sharing FL, and we clarify this in the introduction. The additional items are a small embedding table (C×d, the size of the classifier head) and the feedback gradient. The new information (the embeddings) is negligible: on CIFAR-10, EpersE^{pers} has 5,120 floats, about 0.6% of the 878,538 model parameters. As already stated in Appendix C, the total volume is 1.51× FedAvg (Table 7), below GPFL (1.60×) and FedFOMO (3.00×), which both perform worse (CIFAR-10, Dir(0.5): 76.85% and 68.43% vs. 80.28%). The gains therefore cannot be explained by communication volume.

W2. Privacy and prototypes.

As discussed in Appendix G: pFLAIR is an aggregation algorithm, not a privacy mechanism, and we do not present it as a security contribution. Avoiding explicit label statistics (class counts, proportions) is nevertheless a deliberate design choice, as such statistics can be sensitive including from a regulatory standpoin. This is not a formal guarantee, but there is a real gap between sharing exact class statistics and leaving only an indirect, approximate signal that has to be inferred. Three points (from Appendix G):

First, pFLAIR communicates no explicit class-conditioned statistics (no counts, proportions or computed prototypes). The rows of E^pers are learned embeddings, not statistics computed from the data such as per-class feature means. They have the same shape and nature as the classifier head weights that every parameter-sharing method uploads. Methods sharing class-conditioned statistics expose them by construction (Gu & Bai, 2023). Second, the feedback gradient is accumulated over the whole local set, not per example, and gradient-inversion attacks degrade sharply with the number of aggregated samples (Dimitrov et al., 2024; Huang et al., 2021a). Third, client-level differential privacy applies to our uploads as to model updates.

Regarding FedTGP, we agree that it does not require class counts, unlike FedProto. It does, however, exchange labeled class prototypes (one per class, with its class label), which falls squarely under our statement in Appendix G: "pFLAIR communicates no explicit class-conditioned statistics: no per-class counts, proportions, or labeled prototypes, unlike prototype- and representation-guided methods (e.g., FedProto, GPFL) whose best performance relies on sharing such quantities, which are known to leak information about a client's label distribution (Gu & Bai, 2023)."

Q1. Alignment losses.

The two losses play different roles. LembL^{emb} is a classification loss over cosine logits that learns EpersE^{pers}: it pulls fif_{i} toward the embedding of its own class and, through the softmax, pushes it away from the other classes, which separates classes in the local feature space. We've added an ablation replacing it with MSE (no competition between classes). This drops accuracy from 80.28 ± 0.16 % to 70.51 ± 0.18% (CIFAR-10, Dir(0.5), N=20, 10 iterations). Spreading class embeddings also reinforces the potential of the FiLM layer: it is conditioned on EpersE^{pers} (Eq. 2), so well-separated rows yield distinct class-conditioned tokens. Since only angles matter, the geometry is scale-free, and the norm of EpersE^{pers} is not trained to encode class-level quantities.

LmseL^{mse} has a different role: It transfers knowledge from related clients by regressing fif_i towards EattE^{att} : a personnal combination of their embeddings. Separating classes is already done by LembL^{emb}, so no competition between classes is needed here. Since EattE^{att} is a fixed target, only fif_i (hence the backbone) receives its gradient, and EpersE^{pers} is untouched. We use MSE rather than cosine because we want fif_i to match the features of related clients in position and scale, not only in direction: the classifier heads, which are recombined across clients in parameter space, act linearly on fif_i and are therefore sensitive to its scale as well.

Q2. Server memory.

The server computes one mixed model per client per round (Eq. 5) but does not keep them after sending. It holds the N uploaded models and N feedback gradients, i.e., O(N|θ|): for the CNN (878,538 parameters, about 3.5 MB), N=300 takes about 1 GB.

Q3. Closely related baselines.

Both belong to families already in our comparison. FedTGP is prototype-based (like FedProto and FedDBE): it targets model heterogeneity and exchanges class prototypes. cwFedAvg is an adaptive-aggregation method (like FedAMP, FedFOMO and pFedGAT): it weights aggregation per class using class counts estimated from output-layer norms, whereas pFLAIR learns the mixing end-to-end without counts. Our goal was not an exhaustive benchmark, but to compare pFLAIR with representative methods of each family under the standard pFL protocol.

I thank the authors for their rebuttal. I have read it carefully, and I will keep my original score.

Reviewer_WXCjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes pFLAIR, a personalized federated learning framework that learns class-conditioned attention on the server to recombine client models. The method combines classifier-head descriptors, learned class embeddings, and local-loss feedback without requiring explicit label-frequency statistics, and demonstrates strong performance across heterogeneous vision, language, and sensor datasets.

Critical strengths

-The server learns how to combine client models using feedback from their local objectives, directly connecting the aggregation rule to personalized performance. Combining model descriptors with class information offers a reasonable approach to identifying beneficial collaboration under label skew.

-The paper evaluates image, text, and sensor classification under synthetic label skew, natural domain shifts, and user-level heterogeneity. Comparisons with representative pFL baselines and results averaged over ten independent runs provide substantial empirical support for the method.

-Experiments with up to 300 clients and partial participation examine performance beyond the default setting. Attention visualizations under a class mismatch, together with communication and runtime analyses, provide further insight into the method’s behavior and costs.

Critical weaknesses

-The attention ablation replaces learned attention with cosine similarity, changing both the learning mechanism and the use of class information. Another ablation jointly removes FiLM conditioning and personalized embedding learning. A matched learned class-agnostic attention baseline and separate component ablations would better establish the source of the improvements.

-Although the appendices report communication and computation costs, the experiments use relatively small CNN and FastText models. The method requires an additional model-sized gradient upload, while full-model recombination at the server scales quadratically with the number of clients. Runtime and memory measurements with larger backbones and increasing client populations would strengthen the practical scalability claims.

-Uniformly averaging attention across all classes may dilute useful compatibility signals when clients observe only a small subset of the label space. The authors acknowledge this limitation, but the experiments do not systematically vary the number of classes and local class coverage to establish its effect. Such analysis would better characterize the method’s robustness under this form of heterogeneity.

Questions for authors

The fundamental motivation behind "learning entirely on the server" remains unclear. It is hard to see how effective learning can occur purely on the server if no additional server-side data or local client feedback/information is utilized beyond the existing model parameters. Without clear clarification on where the new information/signal comes from during server-side learning, the necessity of this design choice seems insufficiently justified. I strongly encourage the authors to provide a clear rationale and empirical justification for this paradigm.

1 comment

Thank you for the author's response. But, my concern over the method's deployment when the model size is large. Therefore, I will maintain my original score.

Reviewer_Mbf1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a personalized federated learning framework to address label distribution heterogeneity. The approach shifts the primary computational burden of personalized model aggregation to the server, employing a class-aware attention mechanism to learn collaborative relationships among clients. This enables clients with disparate label distributions to selectively share useful knowledge, while simultaneously reducing reliance on complex local personalization updates. Experimental results demonstrate that the proposed pFLAIR is highly competitive across various datasets and heterogeneity settings. However, the paper’s contributions relative to existing literature require further clarification, and the practical advantages regarding efficiency and scalability have yet to be fully validated.

Critical strengths
  • Shifting the core computation of personalized aggregation to the server is a practically significant design choice. Because federated learning endpoints often face computational and resource constraints, this approach may reduce the burden of complex local personalization and offer an interesting way to share knowledge among heterogeneous clients.

  • The experimental results are highly competitive. The method outperforms existing baselines in most test settings, maintaining a clear accuracy advantage. This illustrates the proposed method's potential in complex, heterogeneous environments.

  • The paper provides a robust experimental evaluation, covering visual, textual, and sensor-based data, while examining various degrees of heterogeneity, client scales, and scenarios involving partial client participation. The ablation studies and supplementary analyses provided in the appendix offer valuable insights into the method's practical performance.

Critical weaknesses
  • Existing research in personalized federated learning has already explored adaptive client collaboration and class-aware aggregation. Although this paper introduces class-conditioned attention, it ultimately averages the attention weights across different classes into a unified model-mixing matrix. Consequently, the core innovation and the specific technical insights offered remain insufficiently clear.

  • Although the paper emphasizes reducing the computational burden on clients, the clients are still required to perform additional embedding optimization and provide gradient feedback. Existing analyses primarily highlight the shift of computational tasks from clients to the server, yet there is a lack of compelling evidence as to whether this shift actually yields significant gains in practical efficiency.

  • Experiments involving varying numbers of clients demonstrate a strong ability to maintain accuracy, but as the number of clients increases, the amount of training data available to each client also changes. The current results focus primarily on predictive performance, lacking server-side computation time and end-to-end training overhead at scale.

Questions for authors

See the weaknesses.

Reviewer_3avwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PFLAIR, a personalized federated learning framework designed to reduce client-side personalization costs while maintaining high predictive accuracy. Each client learns a personalized class-embedding table from its local feature representations, aligned with a global attention-weighted embedding table. Clients upload this table together with their backbone and classifier head. At the server, classifier-head parameters are compressed into descriptors and combined with class embeddings through FiLM conditioning. Class-wise multi-head attention then produces client-specific mixing weights, which are averaged to recombine both client models and the global embedding table for the next round. Client feedback is subsequently used to optimize the server-side FiLM and attention modules.

Experiments cover multiple datasets and heterogeneous settings, including label skew, feature shift, real-world distributions, and varying numbers of clients. Although the framework is reasonably well motivated, I have several concerns regarding its practical utility, scalability, and privacy claims.

Critical strengths

S1. This paper addresses a meaningful limitation of pFL, i.e., the additional computational cost of personalization on the client side. Although this problem has been discussed previously, especially for resource-constrained devices, such as in the FLATEC paper, this work opens a new approach to solving it, e.g., by learning per-class representations for each client rather than per-sample representations.

S2. Strong empirical results. PFLAIR achieves the best or second-best accuracy in most experiments across multiple datasets and settings. Thorough ablation studies are also conducted.

Critical weaknesses

W1. Limited applicability beyond classification tasks. This work appears to be limited to classification tasks. The main concept is to share trainable, client-specific, per-class representations across clients. The experiments use small models and focus exclusively on classification. The authors should clearly discuss whether the method can be applied to other tasks and evaluate PFLAIR with larger models, such as ResNet (at least).

W2. Scalability. Based on the algorithm and the main text, one assumption appears to be that the number of clients participating in each communication round equals the total number of clients in the system. This is not typical in FL research. Methods based on representative vectors or prototype learning can be sensitive to the degree of non-IID data, the number of participating clients per round, and the number of classes. If only a small fraction of clients participate in each round, learning a generalizable global representation may be difficult. The authors should discuss and evaluate this issue.

Furthermore, PFLAIR’s communication cost is significantly higher than that of FedAvg. If all clients must participate in every round, the total communication cost becomes substantially larger.

W3. Client-side computational overhead. The main motivation is to design a pFL method for resource-constrained devices. However, PFLAIR still requires substantial additional computation on the client side compared with FedAvg. The authors should clearly discuss this overhead in the main text and explain how it could be further reduced.

Questions for authors

See above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.