- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
- This paper proposes pFLAIR, a pFL method that shifts the personalization workload to the server, arguing that client-side adaptation is costly for resource-limited devices.
- Instead of local personalization steps, the server learns how to combine client models through a trainable mixing mechanism, and it does so without clients disclosing explicit label-distribution statistics.
- The reported results show that pFLAIR outperforms existing pFL methods.
Critical strengths
- The paper addresses an important practical problem of reducing the client-side burden of personalization by shifting it to the server, which is well suited to practical FL settings such as IoT deployments.
- The proposed mechanism is reasonably well structured, integrating class-wise attention into a coherent, end-to-end trainable pipeline.
Critical weaknesses
- One of the paper's main claimed advantages is not well supported. Although the paper states that clients upload only compact summaries, each client in fact sends its full model parameters and class-embedding tables. This is generally more information than most pFL methods communicate, which may largely account for its performance gains.
- More importantly, contrary to the authors' discussion in the appendix, uploading both the model and the class embeddings can raise privacy concerns, as their combination can reveal information about each client's local data. Since pulls each row of toward its class features, effectively serves as class prototypes carrying feature-level information. In addition, the claim that competitive pFL methods rely on per-class count information is not always accurate, since methods like FedTGP aggregate prototypes without using class-wise sample counts.
Questions for authors
- Q1. The paper uses different alignment losses for and , namely MSE and cosine similarity, respectively. Although the authors briefly justify this choice, further clarification would be helpful. Is there an ablation study or a clearer intuition supporting this design?
- Q2. Beyond the computational cost in the appendix, what is the server's memory requirement? Does the server generate and store a distinct mixed model for each client at every round?
- Q3. The reviewer appreciates the comparison with many baselines, but would it be more informative to compare against more closely related approaches, such as representation aggregation methods (e.g., FedTGP) or class-wise aggregation methods (e.g., cwFedAvg)?
2 comments
I thank the authors for their rebuttal. I have read it carefully, and I will keep my original score.
We thank the reviewer for the feedback.
W1. Information communicated.
The compact objects in pFLAIR are those the server uses to learn the mixing: the head descriptors and the class embeddings. The model is uploaded as in all parameter-sharing FL, and we clarify this in the introduction. The additional items are a small embedding table (C×d, the size of the classifier head) and the feedback gradient. The new information (the embeddings) is negligible: on CIFAR-10, Epers has 5,120 floats, about 0.6% of the 878,538 model parameters. As already stated in Appendix C, the total volume is 1.51× FedAvg (Table 7), below GPFL (1.60×) and FedFOMO (3.00×), which both perform worse (CIFAR-10, Dir(0.5): 76.85% and 68.43% vs. 80.28%). The gains therefore cannot be explained by communication volume.
W2. Privacy and prototypes.
As discussed in Appendix G: pFLAIR is an aggregation algorithm, not a privacy mechanism, and we do not present it as a security contribution. Avoiding explicit label statistics (class counts, proportions) is nevertheless a deliberate design choice, as such statistics can be sensitive including from a regulatory standpoin. This is not a formal guarantee, but there is a real gap between sharing exact class statistics and leaving only an indirect, approximate signal that has to be inferred. Three points (from Appendix G):
First, pFLAIR communicates no explicit class-conditioned statistics (no counts, proportions or computed prototypes). The rows of E^pers are learned embeddings, not statistics computed from the data such as per-class feature means. They have the same shape and nature as the classifier head weights that every parameter-sharing method uploads. Methods sharing class-conditioned statistics expose them by construction (Gu & Bai, 2023). Second, the feedback gradient is accumulated over the whole local set, not per example, and gradient-inversion attacks degrade sharply with the number of aggregated samples (Dimitrov et al., 2024; Huang et al., 2021a). Third, client-level differential privacy applies to our uploads as to model updates.
Regarding FedTGP, we agree that it does not require class counts, unlike FedProto. It does, however, exchange labeled class prototypes (one per class, with its class label), which falls squarely under our statement in Appendix G: "pFLAIR communicates no explicit class-conditioned statistics: no per-class counts, proportions, or labeled prototypes, unlike prototype- and representation-guided methods (e.g., FedProto, GPFL) whose best performance relies on sharing such quantities, which are known to leak information about a client's label distribution (Gu & Bai, 2023)."
Q1. Alignment losses.
The two losses play different roles. Lemb is a classification loss over cosine logits that learns Epers: it pulls fi toward the embedding of its own class and, through the softmax, pushes it away from the other classes, which separates classes in the local feature space. We've added an ablation replacing it with MSE (no competition between classes). This drops accuracy from 80.28 ± 0.16 % to 70.51 ± 0.18% (CIFAR-10, Dir(0.5), N=20, 10 iterations). Spreading class embeddings also reinforces the potential of the FiLM layer: it is conditioned on Epers (Eq. 2), so well-separated rows yield distinct class-conditioned tokens. Since only angles matter, the geometry is scale-free, and the norm of Epers is not trained to encode class-level quantities.
Lmse has a different role: It transfers knowledge from related clients by regressing fi towards Eatt : a personnal combination of their embeddings. Separating classes is already done by Lemb, so no competition between classes is needed here. Since Eatt is a fixed target, only fi (hence the backbone) receives its gradient, and Epers is untouched. We use MSE rather than cosine because we want fi to match the features of related clients in position and scale, not only in direction: the classifier heads, which are recombined across clients in parameter space, act linearly on fi and are therefore sensitive to its scale as well.
Q2. Server memory.
The server computes one mixed model per client per round (Eq. 5) but does not keep them after sending. It holds the N uploaded models and N feedback gradients, i.e., O(N|θ|): for the CNN (878,538 parameters, about 3.5 MB), N=300 takes about 1 GB.
Q3. Closely related baselines.
Both belong to families already in our comparison. FedTGP is prototype-based (like FedProto and FedDBE): it targets model heterogeneity and exchanges class prototypes. cwFedAvg is an adaptive-aggregation method (like FedAMP, FedFOMO and pFedGAT): it weights aggregation per class using class counts estimated from output-layer norms, whereas pFLAIR learns the mixing end-to-end without counts. Our goal was not an exhaustive benchmark, but to compare pFLAIR with representative methods of each family under the standard pFL protocol.