- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes FedPHM, a personalized federated learning method designed to reduce communication costs while adapting to differences in client data distributions. It uses the existing PHM parameterization of a fully connected weight matrix as a sum of Kronecker products, shares the basis matrices across clients, and keeps small interaction matrices private. Clients use an equivalent blockwise forward computation and jointly update the shared bases, private interactions, and other public network parameters using local SGD. After every local step, the private interaction matrices are projected onto a spectral-norm ball. Only the shared bases and other public parameters are uploaded and aggregated by client sample count; private interactions remain local across rounds. The paper analyzes which client-specific weight differences the parameterization can represent and discusses the potential benefits of private interactions under a simplified loss model. Its main experiments use a CNN on CIFAR-100, with additional Fashion-MNIST and CIFAR-10 results in the appendix, to evaluate accuracy, upload payload, parameter count, and the effects of individual design components.
Critical strengths
-
Reduced communication through compact shared bases. FedPHM transmits compact PHM basis matrices together with the other public network parameters, reducing the payload associated with large fully connected weights. This directly addresses the upload bottleneck in federated training. In Table 1, FedPHM uploads 2.62 MiB per selected client per round, compared with 9.62 MiB for FedAvg, a reduction of approximately 72.8%. At Dirichlet alpha 0.1, the corresponding accuracies are 54.08% and 34.06%. The method therefore provides a useful way to reduce upload cost while retaining personalized predictive performance.
-
Client personalization with few private parameters. FedPHM allows each client to combine the shared basis matrices using its own interaction matrices, producing weights adapted to its local data distribution. These private parameters are retained and updated across training rounds. With a PHM order of 8, each layer requires only 512 private interaction parameters, keeping local personalization storage costs low. In Table 2, under strong heterogeneity, this configuration achieves 54.03% accuracy, compared with 31.95% for the fully shared PHM-Full baseline, demonstrating the benefit of private interactions for personalized learning.
Critical weaknesses
- Limited methodological novelty. The parameter compression largely comes from the existing PHM parameterization, while separating globally shared and client-private parameters is an established personalization strategy. The principal change in Sections 3.1–3.4 is to share the PHM bases, keep private interaction matrices, and apply a standard spectral-norm projection. I view this as a specific combination and application of existing techniques, with a limited methodological increment. The practical results are useful, but the paper does not yet articulate a sufficiently strong new technical insight to support its methodological contribution in my assessment.
- The experimental evidence leaves important parts of the assessment unresolved. The main external baselines were mostly published in 2021 or earlier, while the original LoRA work cited for the manuscript's FedLoRA configuration appeared at ICLR 2022. Although Table 1 includes a valuable upload matched LoRA-private-base control, the absence of recent applicable personalized or communication-efficient FL comparators limits the assessment of current competitiveness for an ICLR 2027 submission. Separately, Table 3 varies the projection threshold but does not include a genuinely unprojected control. Consequently, the independent effect of spectral projection on accuracy or stability remains unclear.
Questions for authors
-
Please clarify the main methodological advance of FedPHM over the original PHM parameterization and existing shared/private personalization frameworks. What new technical insight is provided by the specific choice to share the bases and keep the interactions private?
-
Most external baselines evaluated in the paper date from 2021 or earlier. How was this baseline set selected to reflect the relevant literature available at submission time? Could the authors add a recent, applicable personalized or communication-efficient FL baseline under the same backbone and data splits, and compare accuracy at matched cumulative upload budgets?
-
Could the authors include an ablation that removes spectral projection entirely while keeping the other hyperparameters and random seeds fixed? Please report accuracy across seeds, training-loss curves, and spectral-norm trajectories of the private interaction matrices. These results would help quantify the contribution of projection to accuracy and training stability.
-
Appendix B.3 lists PHM-Full accuracies of 9.99%, 62.48%, and 23.88% at 10% participation. These values imply a sample standard deviation of approximately 27.20 percentage points, whereas Table 6 reports 32.12 ± 2.72. Could the authors verify the underlying per-seed results and reconcile the inconsistent statistics?
1. Methodological advance and the role of private interactions
FedPHM assigns distinct learning roles to its two PHM factors: Wk=∑i=1nAk,i⊗Si.
Shared bases Si supply reusable transformations; private interaction matrices Ak,i govern how each client combines them across blocks of its effective weight.
The (a,b) block is [W_k]ab = Σ{i=1}^n [A_{k,i}]_ab S_i.
Clients share transformations while keeping block-mixing coefficients local. For PHM order n, each layer needs only n3 private parameters, independent of the shared-basis dimensions. These compact private states can produce full-rank weight differences across clients.
If only one interaction term differs between clients k and ℓ,
then Wk−Wℓ=(Ak,1−Aℓ,1)⊗S1, and rank(Wk−Wℓ)=rank(Ak,1−Aℓ,1)rank(S1).
Full-rank factors can therefore yield full-rank weight differences, enabling layer-wide personalization through small local states.
Our theory uses a quadratic-risk decomposition to separate approximation costs of the shared PHM structure from heterogeneity captured by private interactions. Shared-gradient disagreement is decomposed into data-driven effects and private-state dispersion; spectral projection constrains gradient amplification from private factors.
The PHM-Full ablation, with fixed architecture and parameter count, validates this sharing strategy:
Results are client-matched accuracy (%), reported as mean ± standard deviation over seeds {42,123,456,2026,2027}. Private interactions improve accuracy by 22.60 percentage points at α=0.1 and 5.60 points at α=0.5.
2. Baseline selection and an additional recent comparator
Our original baselines cover fully shared optimization, personalized representations, regularized local training, and parameter-efficient adaptation. They probe FedPHM's key dimensions: global sharing, client-specific parameters, and communication reduction through structured factorization.
We evaluate DA-PFL (2024) under the same CIFAR-100 partitions, evaluation metric, five seeds, 200 rounds, and 20 local steps per selected client:
At 200 rounds, FedPHM improves accuracy by 2.34 percentage points at α=0.1 and 10.07 points at α=0.5, while reducing per-round uplink by approximately 72.8%.
3. Ablation without spectral projection
We conduct an ablation that disables spectral projection while keeping the seeds, data split, network, and hyperparameters unchanged. Models are trained on CIFAR-100 with α=0.1 for 200 rounds.
Accuracy is reported as mean ± standard deviation over five seeds. Projection enforces the theoretical norm constraint with negligible accuracy cost. Unprojected models reach larger spectral norms but achieve similar accuracy; projection primarily provides analysis-driven norm control rather than an accuracy gain.
4. Audit and five-seed extension of Table 6
We revisited the Table 6 experiments and added seeds 2026 and 2027, following the original protocol. This gives five seeds for each method and participation rate. The updated results use CIFAR-10, 10 clients, α=0.1, and five local epochs per selected client.
Entries are client-matched accuracy (%), reported as mean ± sample standard deviation over seeds {42,123,456,2026,2027}. Across all participation rates, the five-seed results show higher accuracy for FedPHM than for FedAvg and PHM-Full.