acceptodds
Under review as a conference paper at ICLR 2027

FedGeo: Fine-Tuning-Free Federated Personalization via Geometric Stability

Abstract

Personalized federated learning addresses performance degradation under label distribution skew. Fine-tuning-based methods require local optimization for unseen clients. Meta-learning-based generation can avoid these updates, but implicit representations and unconstrained generators may produce unstable parameters and lower unseen-client accuracy. We propose Fine-Tuning-Free **Fed**erated Personalization via **Geo**metric Stability (**FedGeo**), a framework for generalization to unseen clients. Given a label prior estimated from local labeled support, FedGeo uses a geometry-preserving encoder and a spectrally constrained affine generator to produce personalized modulation parameters without local gradient updates. Together, distance preservation and bounded parameter sensitivity provide geometric stability. We establish generalization error bounds and conditional convergence guarantees under federated label-skewed distributions. Experiments demonstrate strong performance for both seen and unseen clients across multiple benchmarks. The anonymous source code is available at [https://anonymous.4open.science/r/FedGeo](https://anonymous.4open.science/r/FedGeo).

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
personalized federated learning, unseen-client generalization, fine-tuning-free personalization, label distribution skew, geometry-preserving representations, stable parameter generation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_EcYdOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the unseen client model assignment problem in personalized FL, and proposes a geometry-preserving framework to achieve more reliable performance.

Critical strengths
  • This paper focuses on the unseen client problem in PFL, and proposed a novel and useful framework to address this problem, especially on the alignment of latent representations.
  • On the presentation, Figure 1 and 2 are both illustrative. The writing and format is well done.
  • The supporting materials in appendix are also strong.
Critical weaknesses
  1. I recognize both the validity of the problem and the novelty of the proposed mechanism. My main concern, however, is the overall significance of the contribution relative to recent literature. Fine-tuning-free personalization for unseen clients is a meaningful but already established problem setting, and FedGeo introduces a novel and arguably cleaner design within it. Nevertheless, the methodological advance appears relatively narrow: the proposed geometric structure specifically preserves the Euclidean geometry of client label priors (P(Y)), combined with bounded sensitivity of the parameter generator. Thus, while the mechanism is technically distinct, I am less certain about the magnitude of its conceptual advance over existing unseen-client personalization frameworks.

  2. Given the concern above, the empirical positioning of the paper becomes particularly important. The paper discusses works such as HyperFedZero as closely related methods but does not include them as main experimental comparisons. Rather than evaluating FedGeo largely on its own, I suggest that the authors add these recent and/or closely aligned baselines to demonstrate the novelty and contributions more explicitly. Besides, I also strongly recommend including:

  • On Bridging Generic and Personalized Federated Learning for Image Classification
  • Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions

In its current form, I find the method novel and technically sound, but I am not yet convinced that the paper demonstrates a sufficiently substantial advance over the existing literature on fine-tuning-free personalization for unseen clients.

Questions for authors

I will reconsider my rating given the weaknesses part can be addressed.

2 comments
AuthorsOpenReview ↗

Response to Reviewer EcYd

We thank the reviewer for recognizing the problem setting and the novelty of the mechanism. Each answer opens with a one-sentence summary and then follows the review. In the revised PDF, revisions for this review are in orange and revisions shared with other reviews are in purple.

W1. "I am less certain about the magnitude of its conceptual advance over existing unseen-client personalization frameworks."

FedGeo shows that, for unseen clients, preserving the geometry of P(Y)P(Y) and bounding the generator's sensitivity is enough to outperform richer generators, and it is what makes the transfer certifiable.

Relative to existing frameworks. FedGeo is the only method that is image-free at a new client, has an output size independent of the backbone, and comes with a numerically evaluated guarantee for unseen clients:

PropertyPeFLL [1]HyperFedZero [2]FedRoD (hyper) [3]FLowDUP [4]FedGeo
Image-free at a new client✗✗✓✗✓
Output size independent of backbone✗✗✗✗✓
Guarantee for unseen clients✓✗✗transductive✓
Guarantee evaluated numerically✗–––✓

On the focus on P(Y)P(Y) geometry and bounded sensitivity. We see this focus as the contribution rather than a limitation: these two properties are exactly what the guarantee requires, and the controls below show that they are also what accuracy requires. Distance preservation and the spectral cap give ∥mi−mj∥2≤2∣A∣ κ ∥di−dj∥2\|m_i-m_j\|_2\le\sqrt{2\lvert\mathcal A\rvert}\,\kappa\,\|d_i-d_j\|_2, so an unseen client's risk is at most its nearest training client's risk plus a term linear in their prior distance (Theorem 4.4). With measured constants, this bound is non-vacuous near training priors (18 of 20 pairs under controlled label shift).

Design, not information, makes the difference. Every row below uses the same label prior; only the mapping differs (pathological CIFAR-100, unseen %):

Prior encodingGeneratorUnseen
isometric [d∥Qd][d\Vert Qd] (FedGeo)affine, cap κ\kappa (FedGeo)51.73
raw prior ddaffine, cap 2κ\sqrt2\kappa (direct affine map)51.40
raw prior ddaffine, cap κ\kappa (FedGeo)50.61
RFF (distorts distances)affine, cap κ\kappa (FedGeo)44.47
MLP (distorts distances)affine, cap κ\kappa (FedGeo)36.15
isometric (FedGeo)MLP42.58
isometric (FedGeo)MLP, width 200, per-layer cap45.20
raw prior, FedRoD (hyper)MLP hypernetwork46.10
raw prior, FedAvglogit adjustment47.80

Scalability. Because its output does not grow with the backbone, FedGeo trains with ResNet-18, where PeFLL runs out of memory, and leads the strongest remaining baseline there by 13.74 points on unseen clients. (Details: Related Work, Sec. 4.2, Table 2, App. I.10.)

W2. "I suggest that the authors add these recent and/or closely aligned baselines to demonstrate the novelty and contributions more explicitly."

All three methods are now included, and FedGeo outperforms each of them in every setting.

HyperFedZero and FedRoD (hyper) are now main baselines in Table 1, with the same LeNet-5, splits, 5,000 rounds, and three seeds as every other method (seen / unseen, %):

MethodC10-PathC100-PathC10-DirC100-Dir
HyperFedZero82.64 / 79.8356.72 / 42.7479.51 / 75.6244.63 / 31.86
FedRoD (hyper)88.30 / 82.8060.40 / 46.1082.70 / 78.2048.80 / 33.20
FedGeo89.48 / 86.7863.13 / 51.7384.13 / 81.7450.77 / 38.62
Δ unseen to the stronger of the two+3.98+5.63+3.54+5.42

FLowDUP uses no labels at a new client, like HyperFedZero, so we compare both with our label-free variant FedGeo-EM, which estimates the prior with forward passes only (pathological, unseen %):

MethodLabel-freeBackward-freeCIFAR-10CIFAR-100
FLowDUP✓✓81.12 ± 3.842.18 ± 2.9
HyperFedZero✓✓79.83 ± 3.942.74 ± 2.9
FedGeo-EM✓✓84.35 ± 3.345.69 ± 3.1

HyperFedZero is ported from its official code with the paper's hyperparameters; FedRoD (hyper) and FLowDUP have no released code and are re-implemented from their papers. (Details: Table 1, App. F.5, I.7.)

We hope these comparisons demonstrate the contribution more explicitly and would be happy to provide further details.

References

[1] Scott et al., PeFLL: Personalized Federated Learning by Learning to Learn, ICLR 2024.

[2] Zhou et al., Deploying Models to Non-Participating Clients in Federated Learning without Fine-Tuning: A Hypernetwork-Based Approach (HyperFedZero), ICLR 2026.

[3] Chen & Chao, On Bridging Generic and Personalized Federated Learning for Image Classification (FedRoD), ICLR 2022.

[4] Zakerinia et al., Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions (FLowDUP), ICML 2026.

I thank the author for the rebuttal materials, and they have mostly addressed my concerns. I updated my score.

Reviewer_A4cjOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes FedGeo, a personalization federated learning method that doesn't require fine-tuning for unseen clients, using locally estimated label priors. FedGeo maps each prior through a QR-based isometric encoder and a spectrally constrained generator to make client-specific FiLM parameters. The authors provide generalization and convergence guarantee and empirically demonstrate performances in both seen and unseen clients through moderate-scale baselines.

Critical strengths

The proposed method enables client-side personalization without gradient updates while retaining client-specific modulation inspired by FiLM layer.

Critical weaknesses
  • It does not convincingly isolate why each proposed component is necessary.

    • While the authors motivates each components and provides ablations, but they currently change several factors at once.
    • For example, it looks unclear why the redundant isometric expansion is better than passing the raw prior dd to the same generator.
      • It constructs the representation as E(d)=[d;Qd]E(d)=[d;Qd] with Q⊤Q=IQ^\top Q=I. Thus, along with orthogonality, it directly implies that ∥E(di)−E(dj)∥2=2∥di−dj∥2\Vert E(d_i)-E(d_j)\Vert_2=\sqrt{2}\Vert d_i-d_j \Vert_2.
    • Plus, the proposed encoder is replaced and compared with an MLP or random Fourier features, but does not include the essential raw-prior control.
    • The control study of the spectral restriction is also vague: while the authors replace the constrained affine generator with an MLP, they change linearity, depth, capacity, and spectral control together, which does not isolate the effects..
      • That said, it is unconvincing why the geometric consistency should be preserved through spectrally constrained encoder.
    • Overall, it reads as a collection of plausible components, i.e., label priors, orthogonal projection, spectral control, affine generation, and FiLM. But it lacks sufficient motivation or evidence on why each component is needed.
  • Missing important branch as a precursor

    • It is closely related to test-time personalization for federated learning, but this literature is barely discussed.
    • In particular, Adaptive Test-Time Personalization [1] should have been discussed and compared where possible.
    • While the proposed method requires labeled support, but typical TTA (including TTA+FL) methods use unlabeled test data and perform no gradient update. This distinction should be stated clearly.
  • The experimental protocol uses an incomplete and partly favorable comparison.

    • The authors reported a single epoch fine-tuning comparison in Appendix H.5, but only as point estimates. The very low FedProto and Ditto results require stronger validation and tuning.
      • Moreover, it omits a simple but strong baseline, i.e., FedAvg with fine-tuning [2], which directly studies adaptation to unseen tasks.
    • Still, it shows that many baseline differences, particularly for seen clients, are not significant after Holm correction.
  • The theoretical guarantees are built upon strong assumptions and can become non-informative.

    • Both guarantees depend on the nearest-client prior distance δmin⁡\delta_{\min}, spectral threshold κ\kappa, number of FiLM layers ∣A∣|\mathcal A|, Lipschitz constants, and conditional shift term ϵshift(R)(u,k)\epsilon_{\mathrm{shift}}^{(R)}(u,k).
      • Even a single uncosntrained large values can make bounds vacuous.
    • In additoin, the convergence is simply assumed as $ \mathbb E[\mathcal L_k^{(R)}]-\mathcal L_k^*\leq C_{\mathrm{fed}}/\sqrt{R}$.
      • Then, Theorem 4.6 adopts this assumed rate to an unseen client, which does not prove convergence of FedGeo's federated optimization.
      • To preserve the current statement, the authors should at least provide with empirical estimates of δmin⁡\delta_{\min} and the other constants, along with settings where the bound is numerically non-vacuous.
  • FedGeo depends heavily on labeled support from each unseen client.

    • While the authors stated that the method is fine-tuning-free, but not support-free. It estimates the unseen client's label prior from labeled local examples.
      • For example, FedGeo uses 75% of each client's holdout data as support in the main CIFAR runs. This is much easier than few-shot or unlabeled test-time personalization.
      • Still, it loses 30.46 and 22.21 accuracy points on CIFAR-10/100 when the estimated prior is replaced by a uniform prior. Thus, I presume much of the benefit may depend on accurate label-frequency information.
      • It should have been tested with fewer/noisy labels, missing classes, and no labeled support.
    • In addition, FedGeo scales explicitly with the class count CC. Its practicality for tasks with thousands of labels is unclear.
  • Its empirical scale is limited and small scale for the practical claims.

    • All experiments evaluates on CIFAR-10/100 and FEMNIST with small image-classification backbones.
    • Please consider including broader and widely-used (personalized) FL benchmarks such as pFL-Bench [3], or FLamby [4].
    • Several costs other than accuracy are missing, e.g., support-label cost, client computation, memory, communication, calibration, and robustness to prior-estimation error.

References

    1. Adaptive Test-Time Personalization for Federated Learning (NeurIPS 2023)
    1. FedAvg with Fine Tuning: Local Updates Lead to Representation Learning (2022)
    1. pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning (NeurIPS 2022)
    1. FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings (NeurIPS 2022)
    1. FedL2P: Federated Learning to Personalize (NeurIPS 2023)
Questions for authors

Please check the Critical Weaknesses seciton.

10 comments
AuthorsOpenReview ↗

Response to Reviewer A4cj (1/4): Overview, W1

We thank the reviewer for the detailed review. Each point below opens with a one-sentence answer and then follows the reviewer's sub-points in order. In the revised PDF, revisions for this review are in red and revisions shared with other reviews are in purple.

WeaknessAnswer
W1The added controls isolate each component: distance preservation is what matters; distorting encoders lose 7.26–15.58 points.
W2Now discussed and compared: our label-free variant beats ATP by 1.92 and 3.33 points on unseen CIFAR-10 and CIFAR-100 clients, without any backward step.
W3In a tuned 3-seed study with FedAvg-FT, FedGeo with 0 gradient steps beats every baseline fine-tuned for 144 steps.
W4Every constant is measured, the bound is non-vacuous in 18 of 20 near-prior pairs, and part of Assumption 4.5 is now proved.
W5FedGeo uses only CC label counts; with 20 labels it matches FedAvg-FT on all ~450 labeled images, and it also works without labels.
W6FedGeo runs on ResNet-18, where hypernetworks run out of memory, is best on pFL-Bench, and needs 140× less computation than PeFLL for a new client.

W1. "It does not convincingly isolate why each proposed component is necessary."

The added controls isolate each component and confirm FedGeo's central idea: transfer to unseen clients depends on preserving the geometry of label priors (pathological CIFAR-100):

ComponentRoleEvidence
label priorcarries the client informationa uniform prior returns FedGeo to the FedAvg level (W5)
isometric encoderpreserves prior distancesdistance-preserving encoders are on par; distorting ones lose 7.26–15.58 points
spectral controlbounds parameter sensitivitycertified at no accuracy cost
affine generatorlinearity6.53 points above a matched-cap MLP
FiLMfinal-layer modulationmatches all-layer modulation with 2C2C values per client (Fig. 3)

Raw-prior control. As the reviewer notes, [d∥Qd][d\|Qd] preserves distances up to 2\sqrt2. Prop. B.1 makes the consequence precise: with QQ fixed, [d∥Qd][d\|Qd] followed by the capped affine generator realizes exactly the affine maps of dd with ∥A∥2≤2κ\|A\|_2\le\sqrt2\kappa. The expansion thus costs no expressivity, and the raw prior is itself distance-preserving. The added control (unseen %, 3 seeds):

EncoderPreserves distancesUnseen
QR, [d∥Qd][d\Vert Qd] (FedGeo)✓51.73 ± 2.4
raw prior dd, matched cap 2κ\sqrt2\kappa✓51.40 ± 2.4
raw prior dd, same generator✓50.61 ± 2.7
RFF [1]✗44.47 ± 3.0
MLP✗36.15 ± 3.6

The control thus confirms our core design principle: what transfers is the geometry of the prior. The isometric encoder guarantees it by construction, every distance-preserving encoding shares the benefit, and encoders that distort distances lose 7.26–15.58 points.

Generator factors. To separate linearity from spectral control, we compare the affine generator with a width-200 ReLU MLP whose per-layer cap κ\sqrt\kappa matches the end-to-end bound κ\kappa, each with and without the cap (unseen %):

GeneratorWith capWithout cap
affine (FedGeo)51.7351.73
MLP, width 20045.2045.20

The 6.53-point gap comes from nonlinearity, not from the cap.

Where the constraint acts. We clarify that the spectral cap acts on the generator, not the encoder: the encoder is an exact isometry by construction, as the reviewer derives, and needs no constraint. The cap bounds how far the generated parameters move when the prior changes, ∥mi−mj∥2≤2∣A∣ κ ∥di−dj∥2\|m_i-m_j\|_2\le\sqrt{2\lvert\mathcal A\rvert}\,\kappa\,\|d_i-d_j\|_2, and Theorem 4.4 carries this to unseen-client risk: under label shift, an unseen client's risk is at most its nearest training client's risk plus a term linear in their prior distance. In training, ∥W∥2=1.15≤κ=5\|W\|_2=1.15\le\kappa=5 and the cap never triggers: the learned map stays well inside the certified region, so FedGeo obtains its guarantee at no accuracy cost. (Details: App. A–B, Sec. 5.4, Table 3.)

References

[1] Rahimi & Recht, Random Features for Large-Scale Kernel Machines, NeurIPS 2007.

AuthorsOpenReview ↗

Response to Reviewer A4cj (2/4): W2, W3

W2. "It is closely related to test-time personalization for federated learning, but this literature is barely discussed."

We added the discussion, the ATP comparison, and an explicit statement of the distinction.

Discussion. Related Work now covers test-time personalization, including ATP [1], Tent [2], and FedL2P [3].

Distinction. Test-time methods such as ATP and Tent adapt with unlabeled test data, whereas FedGeo reads the label counts of a client's labeled local data, as in the unseen-client protocol of pFedHN and PeFLL. Regarding gradient updates, we note that ATP itself "performs one step of gradient descent" [1], as does Tent [2]; FedGeo takes none.

Comparison. We ported ATP from its official code. Without labels, our variant FedGeo-EM, which estimates the prior with forward passes only, outperforms ATP (pathological, unseen %):

MethodLabel-freeBackward-freeCIFAR-10CIFAR-100
ATP-batch✓✗82.43 ± 3.242.36 ± 2.6
FedGeo-EM✓✓84.35 ± 3.345.69 ± 3.1
FedGeo (label counts)–✓86.78 ± 3.751.73 ± 2.4

(Details: Related Work, App. I.7.)

W3. "The experimental protocol uses an incomplete and partly favorable comparison."

We replaced the point estimates with a tuned 3-seed study that includes FedAvg-FT, and FedGeo without any gradient step still outperforms every fine-tuned baseline.

Tuning and FedAvg-FT. All methods fine-tune on the same labeled support from the released model for 1, 5, and 10 epochs. The learning rate is chosen from {0.001, 0.01} on seen clients only, and FedAvg-FT follows [4] (pathological CIFAR-100, unseen %):

Fine-tuning epochs (backward steps)01 (≈14)5 (≈72)10 (≈144)
FedAvg-FT–48.40 ± 2.349.70 ± 2.150.60 ± 2.1
Ditto-FT–48.70 ± 2.449.90 ± 2.250.80 ± 1.9
FedProto–43.60 ± 3.046.40 ± 2.648.20 ± 2.3
FedGeo (+ FT)51.73 ± 2.455.12 ± 2.357.48 ± 2.158.36 ± 2.0

With tuning, Ditto and FedProto reach 50.80% and 48.20% after 144 backward steps, still below FedGeo with none. Fine-tuning FedGeo itself adds 6.63 points, so the two approaches are complementary.

Significance. FedGeo targets unseen clients, where its gains over the strongest baseline in Table 1 are largest (3.24–5.63 points). Across all tables, FedGeo has the higher mean in 122 of 123 comparisons. The Holm analysis is our own, stricter addition (PeFLL [5] reports no tests): 97 comparisons have p<0.05p<0.05, and 51 remain significant even when correcting jointly over all 131 tests. (Details: Sec. 5.2, App. I.6, App. J.)

References

[1] Bao et al., Adaptive Test-Time Personalization for Federated Learning (ATP), NeurIPS 2023.

[2] Wang et al., Tent: Fully Test-Time Adaptation by Entropy Minimization, ICLR 2021.

[3] Lee et al., FedL2P: Federated Learning to Personalize, NeurIPS 2023.

[4] Collins et al., FedAvg with Fine Tuning: Local Updates Lead to Representation Learning, NeurIPS 2022.

[5] Scott et al., PeFLL: Personalized Federated Learning by Learning to Learn, ICLR 2024.

AuthorsOpenReview ↗

Response to Reviewer A4cj (3/4): W4, W5

W4. "The theoretical guarantees are built upon strong assumptions and can become non-informative."

We measured every quantity the reviewer lists, identify where the bound is non-vacuous, and now prove part of Assumption 4.5.

Constants. We clarify that none of the listed quantities is unconstrained; each is bounded by design or measured:

QuantityValue
prior distance δmin⁡\delta_{\min}computed from label counts
spectral thresholdmeasured κeff=1.60\kappa_{\mathrm{eff}}=1.60 / 1.151.15 (CIFAR-10 / 100), certified ≤κ=5\le\kappa=5
FiLM layers ∣A∣\lvert\mathcal A\rvert1 (logit modulation)
Lipschitz constantsLℓ≤2/4L_\ell\le\sqrt2/4 for the loss 1−py1-p_y; LfL_f computed from the logits
conditional shift εshift\varepsilon_{\mathrm{shift}}0 under pure label shift

Non-vacuous settings. Under controlled label shift on pathological CIFAR-10, a held-out prior is moved by λ\lambda from a training prior (20 pairs per row):

λ\lambdaδ\deltaRisk boundObserved gapNon-vacuous
0.050.02150.63130.001718/20
0.100.04311.10900.00728/20
0.200.08612.06450.02633/20

For real held-out clients, 7 of 10 CIFAR-10 pairs are non-vacuous. The observed gap always stays below the bound and grows with δ\delta, exactly as the geometric view predicts: transfer is reliable when a new client's prior lies near training priors, and δ\delta can be computed from label counts before deployment. By comparison, PeFLL [1] does not evaluate its bound numerically.

Convergence. Theorem 4.6, now titled "Conditional excess-risk transfer", states what unseen clients inherit from training clients, while FedGeo's own convergence follows from FedAvg:

  • FedGeo trains all its parameters with FedAvg, so the nonconvex FedAvg analysis [2] gives convergence to a stationary point, the same guarantee PeFLL proves [1, Thm. 3.1].
  • The regularity condition of Assumption 4.5 is now proved (Lemma C.1): under label shift, ∣Li∗−Lj∗∣≤B2∥di−dj∥1|L_i^{\ast}-L_j^{\ast}|\le\frac{B}{2}\lVert d_i-d_j\rVert_1.
  • The remaining rate condition is the input of the transfer, following the standard separation of optimization error [3].

(Details: Sec. 4.3, App. C, App. I.10.)

W5. "FedGeo depends heavily on labeled support from each unseen client."

FedGeo uses only the label counts of a client's local data and remains effective with few, noisy, missing, or no labels.

Support. We clarify that the support is not taken from test data. Every client's data is split 75/25 into a local training split and a test split; participating clients train on the 75% split, and a held-out client, which did not join training, uses the same split as its support, as in pFedHN [4] and PeFLL [1]. From it, FedGeo reads only CC label counts and takes no gradient step, whereas PeFLL and fine-tuning use the images and labels.

Uniform prior. A uniform prior removes the client information by design, so FedGeo falls to the FedAvg level (28.64% vs. 28.21% on CIFAR-100). This confirms that the label prior is the personalization signal FedGeo is designed to exploit, and FedGeo exploits it better than a direct correction: FedAvg given the same prior through logit adjustment [5] trails FedGeo in every split (unseen %):

MethodC10-PathC100-PathC10-DirC100-Dir
FedAvg + same prior83.40 ± 3.247.80 ± 2.177.50 ± 2.034.60 ± 1.8
FedGeo86.78 ± 3.751.73 ± 2.481.74 ± 2.438.62 ± 2.1

Fewer, noisy, missing, and no labels (pathological CIFAR-100, unseen %):

SupportUnseenSupportUnseen
full (~450 labels)51.73 ± 2.420% label noise49.27 ± 2.8
50 labels50.48 ± 2.740% label noise45.88 ± 3.2
20 labels49.16 ± 2.82 of 10 classes missing48.35 ± 3.0
10 labels47.12 ± 2.85 of 10 classes missing44.60 ± 3.2
5 labels42.85 ± 3.3no labels (FedGeo-EM)45.69 ± 3.1

Because FedGeo needs only class frequencies, which a few labels already estimate well, it degrades gradually in every condition. With 20 labels, it matches 5 epochs of FedAvg-FT on all ~450 labeled images (49.16% vs. 49.70%), using 22× fewer labels and no backward step.

Thousands of classes. The overhead is 5C2+2C5C^2+2C parameters, or 2C2+2C2C^2+2C in the equivalent direct form of Prop. B.1, which performs on par (51.40% vs. 51.73%). As an extension, a rank-64 generator modulating 512 penultimate features needs about 0.7M parameters for C=10,000C=10{,}000. (Details: App. I.5, I.6.)

References

[1] Scott et al., PeFLL: Personalized Federated Learning by Learning to Learn, ICLR 2024.

[2] Karimireddy et al., SCAFFOLD: Stochastic Controlled Averaging for Federated Learning, ICML 2020.

[3] Bottou & Bousquet, The Tradeoffs of Large Scale Learning, NeurIPS 2007.

[4] Shamsian et al., Personalized Federated Learning using Hypernetworks (pFedHN), ICML 2021.

[5] Menon et al., Long-Tail Learning via Logit Adjustment, ICLR 2021.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Reviewer fakG (1/2): W1 & Q1, W2 & Q2

We thank the reviewer for the precise and constructive questions. Each answer opens with a one-sentence summary and then follows the question. In the revised PDF, revisions for this review are in blue and revisions shared with other reviews are in purple.

W1 & Q1. "Can the authors provide a spectrally matched direct-affine baseline? How much improvement remains when the FiLM placement, optimizer, and training budget are controlled?"

We agree with the equivalence, now stated as Prop. B.1, and it sharpens our claim: since the composite map has the expressivity of a direct affine map, the matched baseline isolates the role of geometry, and it confirms that preserving prior distances drives the result.

Equivalence. With W=[W1,W2]W=[W_1,W_2], W[d;Qd]+b=(W1+W2Q)d+bW[d;Qd]+b=(W_1+W_2Q)d+b and ∥W1+W2Q∥2≤2∥W∥2\|W_1+W_2Q\|_2\le\sqrt2\|W\|_2; conversely, any AA with ∥A∥2≤2κ\|A\|_2\le\sqrt2\kappa is realized by W=A[I;Q]⊤/2W=A[I;Q]^\top/2. FedGeo's maps are thus exactly {d↦Ad+b:∥A∥2≤2κ}\{d\mapsto Ad+b:\|A\|_2\le\sqrt2\kappa\}.

Spectrally matched baseline. All variants use the same logit placement, SGD (lr 0.01, batch size 32), 5,000 rounds, 20% client sampling, and seeds 42–44 (pathological CIFAR-100, %):

VariantPreserves distancesSeenUnseen
FedGeo, [d∥Qd][d\Vert Qd], cap κ\kappa✓63.13 ± 0.751.73 ± 2.4
direct affine on dd, cap 2κ\sqrt2\kappa✓62.88 ± 0.751.40 ± 2.4
RFF encoder✗58.76 ± 0.944.47 ± 3.0
MLP encoder✗52.84 ± 2.136.15 ± 3.6
MLP generator✓60.52 ± 1.342.58 ± 2.8

How much improvement remains. Over the matched direct map, 0.33 points, within one std, as Prop. B.1 predicts. The improvement that remains comes from geometry and linearity: distorting encoders lose 7.26–15.58 points, and nonlinear generators lose 6.53–9.15 points (a width-200 ReLU generator with per-layer cap κ\sqrt\kappa reaches 45.20%). Distance preservation also gives the sensitivity bound a known constant, ∥mi−mj∥2≤2∣A∣ κ ∥di−dj∥2\|m_i-m_j\|_2\le\sqrt{2\lvert\mathcal A\rvert}\,\kappa\,\|d_i-d_j\|_2, which Theorem 4.4 carries to unseen-client risk. This is the core idea of FedGeo: the encoder's role is to preserve prior geometry, and QR is one way to guarantee it. (Details: App. B, Sec. 5.4.)

W2 & Q2. "Can the authors report the actual spectral norms during training and verify whether the prescribed constraint is satisfied under single-step power iteration? How is Assumption 4.5 justified for FedGeo?"

The constraint holds throughout training, and Assumption 4.5 now needs only the training rate, since its regularity condition is proved.

Spectral norms. We log the exact norm by SVD, the power-iteration estimate, and how often truncation triggers (pathological CIFAR-100, κ=5\kappa=5):

Exact ∥W∥2\Vert W\Vert_2 (SVD)Power iterationRelative errorTruncation triggered
1.1537551.1537546.2×10−76.2\times10^{-7}0.00% of local steps

On CIFAR-10, the norm is 1.60. The learned map thus stays well inside the certified region on its own, so the cap provides the guarantee at no accuracy cost (51.73% with or without it). A single warm-started iteration per step is standard in spectral normalization [1], one exact SVD of the trained generator certifies the bound for any run, and all bound evaluations use the measured norm.

Assumption 4.5.

  • Its regularity condition is now proved (Lemma C.1): under label shift, ∣Li∗−Lj∗∣≤B2∥di−dj∥1|L_i^*-L_j^*|\le\frac B2\|d_i-d_j\|_1.
  • For the rate, FedGeo trains all its parameters with FedAvg, so the nonconvex FedAvg analysis [2] gives convergence to a stationary point, the guarantee PeFLL proves [3, Thm. 3.1].
  • Theorem 4.6, now titled "Conditional excess-risk transfer", takes the training-client rate as input and transfers it to unseen clients with an explicit residual, following the standard separation of optimization error [4].

The stability results, including the sensitivity bound, Theorem 4.4, and Prop. B.1, do not need Assumption 4.5. (Details: App. A, App. C.)

References

[1] Miyato et al., Spectral Normalization for Generative Adversarial Networks, ICLR 2018.

[2] Karimireddy et al., SCAFFOLD: Stochastic Controlled Averaging for Federated Learning, ICML 2020.

[3] Scott et al., PeFLL: Personalized Federated Learning by Learning to Learn, ICLR 2024.

[4] Bottou & Bousquet, The Tradeoffs of Large Scale Learning, NeurIPS 2007.

AuthorsOpenReview ↗

Response to Reviewer fakG (2/2): W2 & Q3, Q4

W2 & Q3. "Can the authors clarify the unseen-client adaptation procedures and hyperparameter settings of PeFLL and other hypernetwork baselines, particularly given their low reported accuracies? Why is HyperFedZero excluded from the main comparisons?"

The low accuracies came from one unified budget without the baselines' own new-client steps; Table 1 now runs each hypernetwork with its official budget and procedure, HyperFedZero is now a main baseline, and FedGeo still leads every column.

Procedures and hyperparameters. The data protocol follows PeFLL [1]: 90 training and 10 held-out clients, 5,000 rounds, batch size 32, LeNet-5.

MethodTrainingNew client
PeFLL [1]official code; 50 SGD steps per round, lr 0.002, momentum 0.9; server Adam, lr 0.0002embeds its labeled support, receives its model, recalibrates BN
pFedHN [2]official code; 50 SGD steps per round, momentum 0.9optimizes a new embedding for 20 rounds, recalibrates BN
FedRoD (hyper) [3]Alg. 2 re-implemented; hidden width 32 / 16generates its head from label proportions
HyperFedZero [4]official code; embedding dim. 16, chunk size 576generates weights from each test image
FedAGHN [5]original paperdefines no model for new clients (N/A)
FedGeo1 epoch of SGD per round, lr 0.01CC label counts → 2C2C FiLM values

Accuracy with native budgets (unseen %, unified → native):

MethodC10-PathC100-PathC10-DirC100-DirFEMNIST
PeFLL75.41 → 80.5013.05 → 31.4069.94 → 78.5010.21 → 28.6040.31 → 87.70
pFedHN24.51 → 80.203.16 → 27.4026.80 → 66.203.58 → 17.3023.11 → 82.80
FedRoD (hyper)82.8046.1078.2033.20–
HyperFedZero79.8342.7475.6231.86–
FedGeo86.7851.7381.7438.6288.49
Δ to best hypernetwork+3.98+5.63+3.24+5.42+0.79

HyperFedZero. It is now in Table 1, where FedGeo exceeds it in every setting. Since HyperFedZero uses no labels, we also compare it with our label-free variant FedGeo-EM, which estimates the prior with forward passes only: 84.35% vs. 79.83% on CIFAR-10 and 45.69% vs. 42.74% on CIFAR-100. (Details: App. F.5, I.7.)

Q4. "How does FedGeo perform when the number of labeled support samples for unseen clients is substantially reduced? Can the authors clarify the exact FiLM placement, parameter counts, and normalization settings?"

FedGeo degrades gradually with fewer labels, and we give the exact settings below.

Reduced support (pathological CIFAR-100, unseen %):

Labels per client~450 (full)5020105
FedGeo51.73 ± 2.450.48 ± 2.749.16 ± 2.847.12 ± 2.842.85 ± 3.3

With 20 labels, FedGeo matches 5 epochs of full-model FedAvg fine-tuning on all ~450 labeled images (49.70%).

Placement. FiLM acts on the output logits of the final classifier, z′=(1+γ)⊙z+βz'=(1+\gamma)\odot z+\beta with γ,β∈RC\gamma,\beta\in\mathbb R^C, shared by all samples of a client.

Parameter counts. 5C2+2C5C^2+2C (520 for CIFAR-10, 50,200 for CIFAR-100), independent of the backbone; 2C2C values are generated per client.

Normalization. LeNet-5 has BN after each convolution; its running statistics are aggregated globally, so seen and unseen clients share them. (Details: Sec. 5.1, App. F, App. I.6.)

References

[1] Scott et al., PeFLL: Personalized Federated Learning by Learning to Learn, ICLR 2024.

[2] Shamsian et al., Personalized Federated Learning using Hypernetworks (pFedHN), ICML 2021.

[3] Chen & Chao, On Bridging Generic and Personalized Federated Learning for Image Classification (FedRoD), ICLR 2022.

[4] Zhou et al., Deploying Models to Non-Participating Clients in Federated Learning without Fine-Tuning: A Hypernetwork-Based Approach (HyperFedZero), ICLR 2026.

[5] Song et al., FedAGHN: Personalized Federated Learning with Attentive Graph Hypernetworks, Knowledge-Based Systems 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.