acceptodds
Under review as a conference paper at ICLR 2027

GeoProtoFL: Are Class Prototypes Geometrically Aggregatable in Heterogeneous Federated Learning?

Abstract

Prototype-based heterogeneous federated learning (HtFL) enables clients with different model architectures to collaborate through class-prototype exchange instead of parameter aggregation. Existing methods commonly map client features to a shared prototype dimension and directly aggregate the resulting prototype coordinates, implicitly relying on their coordinate systems being comparable across clients. However, independently trained feature extractors across heterogeneous clients may represent equivalent class structures in different client-specific coordinate frames. Consequently, direct aggregation of unregistered prototypes may distort the resulting global prototypes. To address this issue, we introduce GeoProtoFL, a geometry-aware method designed to make heterogeneous class prototypes geometrically aggregatable. Each client learns a local bi-projection with equiangular tight frame (ETF) anchoring to map pooled backbone features onto a shared spherical prototype interface. The server registers each client prototype configuration to a server reference configuration through closed-form Procrustes registration performing gauge registration, then computes each global prototype as a class-wise Fr\'echet barycenter of the registered prototypes intrinsic aggregation. GeoProtoFL realizes geometric aggregatability through gauge-consistent server-side prototype construction and intrinsic consensus under the spherical metric defined by the shared interface, without requiring public data or homogeneous client architectures. Across standard benchmarks, it consistently improves over representative HtFL baselines, achieving gains of up to 2.61% over the strongest baseline in a given setting while remaining robust across varying statistical heterogeneity, model heterogeneity, and federation-scale settings.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
federated learning, heterogeneous federated learning, model heterogeneity, prototype learning, representation alignment, orthogonal Procrustes, hypersphere embeddings

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_X1KGOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper studies prototype-based heterogeneous federated learning (HtFL), where clients with different backbone widths exchange class prototypes instead of parameters. It argues that pooling features to a shared dimension only makes prototypes communicable—heterogeneous encoders may express the same class structure under different orthogonal coordinate frames ("gauges"), so direct averaging distorts global prototypes. The authors formalise a geometric aggregatability requirement (common interface + gauge consistency + intrinsic consensus) and propose GeoProtoFL: each client learns an ETF-anchored spherical bi-projection that maps pooled features onto a d-dimensional unit hypersphere; the server registers client prototypes by closed-form orthogonal Procrustes and aggregates them class-wise by a weighted Fréchet barycenter. On HtFLlib (CIFAR-10/100, Tiny-ImageNet; pathological and Dirichlet non-IID) GeoProtoFL achieves the highest mean accuracy against seven HtFL baselines, with single-round theory (Theorems 1–3, Propositions 1–4).

Critical strengths
  1. Conceptual contribution. Names and operationalises gauge inconsistency in prototype-based HtFL; reframes ad-hoc prototype refinements (FedTGP / FedORGP / FedSAF) as partial attempts at the same underlying geometric problem. The geometric-aggregatability definition (interface + gauge consistency + intrinsic consensus) is a useful lens for this sub-field.

  2. Methodologically tight end-to-end design with no public data. Architecture-agnostic, closed-form server-side operations (SVD-based Procrustes + Karcher iterations for the Fréchet barycenter); communication dimension decoupled from backbone feature dimension (d = 128 / 256 vs dᵢ = 512). Empirically the strongest method in all evaluated main and robustness settings, with a clean ablation (Table 4 V1→V2→V3→V4) whose ordering matches the theory: spherical interface brings the largest single step, but registration only helps when coupled with Fréchet aggregation.

Critical weaknesses
  1. The orthogonal Procrustes assumption (Assumption 1) is foundational but is not empirically diagnosed. The backbone continues to update while the bi-projection encoder is frozen, so class directions in the projected space can drift (acknowledged in Remark 1 and the conclusion); the paper does not report how much of the residual mismatch is actually captured by the orthogonal component (e.g. the average ratio F_i/E_i from Theorem 1, or the condition number / rank-deficiency of H_i across the three benchmarks and a few federation scales). Without a measurement, the practical robustness of the closed-form registration to non-orthogonal drift remains an open question.

  2. Advantage narrows under sparse participation, and the theory only covers one server round. At N = 100, ρ = 20 % the strong gap to FedORGP collapses to within one standard deviation (37.35 % vs 37.28 %), and Propositions 1–4 + Theorems 1–3 are explicitly per-round with no convergence / generalisation analysis of the federated procedure (App. A.2.5 acknowledges this). The "robust across model heterogeneity and federation scale" claim is therefore carried mostly by the empirical study.

Questions for authors
  1. Could you report the average ratio F_i/E_i from Theorem 1 (or the condition number / rank-deficiency of H_i) across the three benchmarks and a few federation scales? If the orthogonal component dominates, that supports Assumption 1 in practice; if not, please discuss how a small non-orthogonal correction (e.g. a low-rank refinement step after Procrustes) would behave.

  2. Under sparse participation (ρ = 50 %, 25 %, 20 % in Table 2), could you add a curve of client-frame target stability over rounds (e.g. d_S(\bar p^i_{c,t-1}, \bar p^i_{c,t})) and the staleness of the reused gauge R_i? The conclusion acknowledges that "the advantage narrows under sparse participation"; it would help to know whether this is driven by stale gauges or by reduced per-class coverage.

  3. The communication claim is qualitative ("decoupled from dᵢ"). Could you report concrete per-round bytes for GeoProtoFL vs. FedProto / FedTGP / FedORGP / FedSAF on all three benchmarks, with downlink factored in? For small |C_i|, |C_i|\cdot d does not always beat |C_i|\cdot d_i once the head download is included.

12 comments

W1 & Q1: Assumption 1 is foundational but not diagnosed. How much of the residual mismatch does the orthogonal component capture (e.g. the ratio Fi/EiF_i/E_i from Theorem 1, or the condition number/rank deficiency of HiH_i) across the three benchmarks and a few federation scales? If it does not dominate, how would a small non-orthogonal correction behave?

Thank you for this precise question. On all three benchmarks and at three federation scales, one orthogonal map removes more than 99.99% of the client-to-global distance. Most of this distance lies along a direction shared by a client's classes, which makes HiH_i nearly rank one, so we test the class structure separately. We answer in four parts: (1) the fit of the orthogonal component, (2) the conditioning of HiH_i, (3) ∥Fi∥/∥Ei∥\lVert F_i\rVert/\lVert E_i\rVert with known deviations, and (4) a non-orthogonal correction. Parts (3) and (4) are in R-X1KG-W1Q1 (2/2).

(1) Fit of the orthogonal component. Practical setting (Dirichlet, α = 0.1), 3 runs, median over clients over the last 100 rounds.

  • Before / After: squared distance between a client's normalized class prototypes and the global prototypes of the same classes, without / with registration (2 = orthogonal). The per-class angle is in parentheses.
  • Reduction: 1 − After / Before.
  • CiC_i: median number of classes per client.
  • σ2/σ1\sigma_2/\sigma_1: ratio of the two largest singular values of HiH_i (last round).
DatasetNρCiC_idσ2/σ1\sigma_2/\sigma_1BeforeAfterReduction
CIFAR-1020100%51281.6×10−41.6\times10^{-4}1.76 (83°)3.0×10−53.0\times10^{-5} (0.31°)99.998%
CIFAR-10020100%401283.3×10−53.3\times10^{-5}1.91 (87°)5.7×10−55.7\times10^{-5} (0.43°)99.997%
Tiny-ImageNet20100%872561.8×10−61.8\times10^{-6}1.96 (89°)1.3×10−51.3\times10^{-5} (0.20°)99.999%
CIFAR-1004050%36.51282.2×10−52.2\times10^{-5}1.89 (87°)7.9×10−57.9\times10^{-5} (0.51°)99.996%
CIFAR-10010020%301289.7×10−69.7\times10^{-6}1.89 (87°)8.7×10−58.7\times10^{-5} (0.54°)99.995%

The fit changes little with scale (0.43° → 0.51° → 0.54° per class). What changes is how far the global prototypes move between rounds (R-X1KG-W2Q2).

(2) Conditioning of HiH_i. By construction, rank⁡Hi≤Ci<d\operatorname{rank} H_i \le C_i < d. In addition, σ2/σ1\sigma_2/\sigma_1 ≤ 1.6×10−41.6\times10^{-4}, so HiH_i is numerically close to rank one: a client's class prototypes and their global counterparts concentrate around a shared direction.

  • The class structure is tested separately. The Reduction column mostly reflects the shared direction. We therefore remove that direction and compare each client's registered class structure with that of clients of other architectures (CIFAR-100, last 100 rounds; 0 = identical, ≈ 2 = unrelated). Chance is the same comparison with permuted labels. Registration gives 0.530 vs 0.961 (practical) and 0.040 vs 0.420 (pathological), and it is better than chance for each of the 8 architectures under both splits (per-architecture values in Appendix A.6, Table 12).
  • RiR_i is only weakly determined beyond the shared direction. Theorem 1 needs no rank condition, so this does not enlarge the error of the registered prototypes. Proposition 3 bounds its effect on the returned targets.

Continued from X1KG-W1Q1 (1/2).

(3) ∥Fi∥/∥Ei∥\lVert F_i\rVert/\lVert E_i\rVert with known deviations. EiE_i and FiF_i cannot be separated in real runs. We therefore treat the registered last-round prototypes as the latent ones, so that FiF_i is measured, and add a known rotation plus a known anisotropic stretch or per-class noise (levels 0.01–0.2, 10 draws). Selected rows; each column is a median over clients and draws:

DatasetDeviation∥E∥\lVert E\rVert∥F∥\lVert F\rVertErrorBound ∥E∥+2∥F∥\lVert E\rVert + 2\lVert F\rVertGlobal-prototype displacement, with / without registration
CIFAR-10stretch 0.10.1280.0123.8×10−53.8\times10^{-5}0.1540.00° / 90.1°
CIFAR-10noise 0.10.2240.0120.1720.2504.03° / 87.9°
CIFAR-100stretch 0.10.3600.0450.0010.4620.01° / 90.4°
CIFAR-100noise 0.010.0630.0450.0460.1530.32° / 92.3°
CIFAR-100noise 0.10.6270.0450.5850.7344.58° / 90.2°
Tiny-ImageNetstretch 0.10.5310.0335.8×10−45.8\times10^{-4}0.6000.00° / 90.4°
Tiny-ImageNetnoise 0.10.9250.0330.9030.9974.80° / 89.6°

Findings.

  • The ratio range is set by the injected levels. ∥F∥/∥E∥\lVert F\rVert/\lVert E\rVert ranges from 0.02 to 1.2 (pure rotations, where Ei=0E_i = 0, excluded). The bound held in all 5,400 cases.

  • Noise tests the bound. Under per-class noise 0.1, the error reaches 69%–91% of the bound, so the bound is informative. Noise is not amplified (error < ∥E∥\lVert E\rVert in every row), and the global prototypes stay within 5° of the unperturbed ones, against about 90° without registration.

  • A stretch is almost entirely absorbed (error ≤ 0.32% of ∥E∥\lVert E\rVert). This relies on the concentration of the trained prototypes; on spread-out configurations, no rotation can remove a strong stretch.

(4) A non-orthogonal correction. We did not add a non-orthogonal correction, because an unconstrained one would behave badly in two ways:

  • It would copy the reference. With Ci<dC_i < d, an unconstrained linear map can send any CiC_i independent prototypes exactly onto the reference, so registered prototypes would carry no client information. A low-rank refinement fitted on the same prototypes moves toward this solution as its rank grows.

  • It would break the geometry. It would not preserve the geodesic distances used by guidance and aggregation.

After Procrustes, 0.2°–0.5° per class remains (table in (1)). This is small in absolute terms, though not negligible relative to class differences under the practical setting. A refinement regularized toward the Procrustes solution is a natural extension, which we name as future work.

Revision. We have added these measurements to Appendices A.6 and A.7, and one sentence after the orthogonal-gauge model in Section 3.1 and after Assumption 1 (Appendix A.2.1).

W2 & Q2: The advantage narrows under sparse participation (37.35% vs 37.28% at N = 100, ρ = 20%), and the theory covers one server round. Can you show client-frame target stability over rounds and the staleness of the reused gauge, to tell stale gauges from reduced per-class coverage?

Thank you for this question. We agree that the theory covers a single server round and that the margin narrows under sparse participation (Table 2). The measurements below separate the two mechanisms: coverage stays complete, while the global prototypes move more between rounds and reused gauges go stale.

(1) Measurements. CIFAR-100, practical setting (Dirichlet, α = 0.1), 20 participating clients per round as in Table 2, 3 runs, medians over rounds 10–300. The runs reach 43.30% and 37.30%, consistent with Table 2 (43.26% and 37.35%).

A waiting client neither trains nor uploads, so its reused RiR_i is stale only by how far the global prototypes have moved since it last participated. Stale gauges predict an error that grows with gauge age. Reduced coverage predicts classes without an update, or with fewer contributors.

Measures:

  • Target change: angle between a client's targets in consecutive rounds, also for waiting clients.
  • Stale-gauge error: angle between the targets from the held gauge and from a gauge refitted on the client's last upload against the current global prototypes.
  • Gauge age: rounds since the client last participated.
  • Global movement: per-round angle moved by the global prototypes, over the last 100 rounds.
SettingClients per classClasses without an updateGlobal movement per roundTarget change per round, participating / waitingGauge age, median (max)Stale-gauge error, age 1 → age ≥ 5
N = 20, ρ = 100%≈80%0.0107°–1 (1)–
N = 40, ρ = 50%70%0.25°0.30° / 0.30°1 (16)0.13° → 0.21°
N = 100, ρ = 20%60%0.29°0.35° / 0.35°4 (52)0.17° → 0.35°

At N = 20, clients per class = 20 × 40 / 100. All angles are small because the prototypes concentrate around a shared direction (R-X1KG-W1Q1). As a scale, the nearest other class is 0.60° (N = 40) and 0.64° (N = 100) from a global class prototype.

(2) Finding.

  • Coverage does not change. Each client holds 30–40 of the 100 classes, so 20 participants update every class in the median round; the clients behind each class fall only from about 8 to 6.
  • The global prototypes become less stable. With participants changing every round, they move 23× (N = 40) and 27× (N = 100) more per round than with full participation. Targets follow this movement equally for participating and waiting clients, whose gauges are fixed, so the change comes from the global prototypes.
  • Reused gauges go stale as a consequence. The stale-gauge error grows with gauge age (Spearman 0.45 / 0.57 at N = 40 / 100). At N = 100 it levels off at about 0.4° after 15 rounds, below the 0.64° to the nearest other class.
  • The fit of registration changes little with scale (0.43° → 0.51° → 0.54° per class; R-X1KG-W1Q1).

The narrowing is therefore associated with the movement of the global prototypes between rounds and the stale gauges it causes, not with reduced coverage or with a worse fit of registration.

Revision. We have narrowed the scale claim in Section 4.3.

Reviewer_M2DsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies prototype aggregation in heterogeneous federated learning, where different client models may produce class prototypes in incompatible coordinate systems. The authors argue that mapping representations from different client architectures into a shared feature dimension does not necessarily ensure that their prototypes are geometrically comparable. To address this issue, the paper proposes GeoProtoFL, which first maps local representations into a shared spherical space using ETF-anchored projections. The server then applies orthogonal Procrustes registration to align client prototypes, followed by Fréchet aggregation to construct global prototypes. The method is evaluated on CIFAR-10, CIFAR-100, and Tiny-ImageNet under different data distributions, model architectures, and client participation settings. The reported results show improvements over several existing heterogeneous federated learning methods.

Critical strengths
  1. The paper raises a reasonable concern about directly averaging prototypes from independently trained models. Matching feature dimensions does not necessarily mean that the resulting representations are aligned. This is a relevant issue for prototype-based heterogeneous federated learning.
  2. The method is clearly described, and the authors provide theoretical analysis of the registration and aggregation procedures. Although the underlying techniques are not new, the overall formulation is mathematically motivated and reasonably easy to follow.
  3. The authors have carried out a fair amount of experimental work. The evaluation includes several heterogeneous federated learning baselines, different non-IID settings, model architecture variations, client scalability experiments, and component ablations. The full method achieves the highest reported mean accuracy across the main experimental settings.
Critical weaknesses
  1. The proposed method mainly combines existing techniques, including equiangular tight frame (ETF) anchoring, orthogonal Procrustes alignment, and Fréchet aggregation. Similar issues concerning prototype alignment and representation inconsistency in heterogeneous federated learning have also been studied in prior work. It remains unclear what the main methodological novelty is beyond integrating these established techniques.
  2. The theoretical analysis assumes that differences between client prototype configurations can be approximated by orthogonal transformations. However, independently trained models with different architectures may produce more complex, nonlinear differences in their representations. The paper does not provide sufficient empirical evidence to show how well this assumption holds in practice.
  3. Table 4 shows that most of the improvement over FedProto comes from the shared spherical projection interface. The additional gains from gauge registration and Fréchet aggregation are relatively small. This makes it difficult to assess the independent contribution of the proposed server-side geometric aggregation mechanism, which is presented as a central part of the method.
  4. The experiments mainly focus on classification accuracy, without directly examining whether gauge registration resolves the geometric inconsistency described in the motivation. A controlled experiment with known orthogonal perturbations, together with measurements of prototype alignment errors before and after registration, would provide more direct evidence for the proposed mechanism.
  5. The evaluation is limited to CIFAR-10, CIFAR-100, and Tiny-ImageNet, all of which are relatively small and commonly used image classification benchmarks. Given the focus on representation geometry across heterogeneous models, the paper would benefit from evaluation on a larger and more challenging dataset, particularly a cross-domain benchmark such as DomainNet.
  6. Some baseline results differ noticeably from previously published results, and several training configurations are different from those used in the original studies. Although these differences may be explained by changes in the experimental protocol, their impact on the comparisons is not sufficiently discussed. Providing the relevant hyperparameter settings, training curves, and additional comparisons under alternative configurations would help clarify whether the reported improvements are consistent and fairly evaluated.
  7. The reported results are based on the highest test accuracy achieved across communication rounds, which may introduce optimistic model-selection bias. In addition, averaging client-specific test accuracies under different label distributions makes it difficult to interpret performance across heterogeneous settings. Reporting final-round results or selecting checkpoints using a separate validation set would help address these concerns.
Questions for authors
  1. Table 4 shows that the shared spherical interface accounts for most of the improvement over FedProto. The additional gains from gauge registration and Fréchet aggregation are much smaller. What evidence supports the claim that the proposed geometric aggregation, rather than the projection interface, is responsible for the main improvement?
  2. The analysis assumes that differences between client prototype configurations can be approximated by orthogonal transformations. Have the authors measured how well this assumption holds for the heterogeneous architectures used in the experiments? Reporting registration residuals or alignment errors would help clarify this point.
  3. Some baseline accuracies differ noticeably from previously reported results, and the training configurations are not always identical to those used in the original papers. Could the authors explain these differences and clarify how the baseline hyperparameters were selected?
  4. All three datasets are relatively small image classification benchmarks. Would the method remain effective on a more challenging dataset with genuine domain differences, such as DomainNet? Also, why is the best test accuracy across communication rounds used for reporting, rather than selecting the checkpoint through a validation set?
8 comments

W1: The method mainly combines existing techniques (ETF anchoring, orthogonal Procrustes alignment, Fréchet aggregation), and prototype alignment and representation inconsistency in HtFL have been studied before. What is the main methodological novelty beyond integrating these techniques?

Thank you for this question. GeoProtoFL builds on existing techniques: ETF anchoring [1, 2], orthogonal Procrustes alignment [3] and Fréchet aggregation [4]. These techniques are not our contribution. The novelty is the server-side aggregation problem that we formulate, and what follows from it:

(1) We formulate geometric aggregatability. Prior work on representation inconsistency in HtFL acts on the clients or on the targets. It refines the targets or the guidance (Section 2.1), delays alignment [6], or trains all clients toward fixed shared targets [5]. To our knowledge, none of these methods registers client frames at the server before combining prototype or classifier coordinates. Geometric aggregatability states when server-side aggregation is well defined across client-specific frames (gauges), through three requirements: a common interface, gauge consistency and intrinsic consensus.

(2) We prove that class-wise aggregation cannot, in general, be gauge consistent (Proposition 1). The uploads of one class carry no information about the client gauges. In the worst case, the output of any class-wise rule, including coordinate averaging, is therefore at least 90° from the latent prototype in some client's frame. A gauge-consistent rule needs more than the uploads of one class, which is why we register each client's classes jointly.

(3) We derive a framework in which each technique meets one requirement, with per-round guarantees.

  • Common interface → spherical bi-projection with ETF anchoring: one unit sphere for all architectures, without public data. Supported by the ablation (Table 4; R-M2Ds-W3Q1).
  • Gauge consistency → orthogonal Procrustes registration of each client's classes jointly: invariant to the client's frame (Proposition 2), with a bounded error under approximate gauges (Theorem 1). Verified on the trained clients (R-M2Ds-W2Q2) and in training with known client rotations (R-M2Ds-W4).
  • Intrinsic consensus → Fréchet aggregation: minimizes the same geodesic criterion as local guidance (Proposition 4). Supported by the ablation (Table 4; R-M2Ds-W3Q1).

Revision. We have positioned [5] and [6] in Section 2.1, cited [1]–[4] where Section 3 introduces each technique, and aligned the contribution list in Section 1.

[1] V. Papyan, X. Y. Han, D. L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. PNAS, 2020.
[2] Z. Li, X. Shang, R. He, T. Lin, C. Wu. No fear of classifier biases: Neural collapse inspired federated learning with synthetic and fixed classifier. ICCV, 2023.
[3] P. H. Schönemann. A generalized solution of the orthogonal Procrustes problem. Psychometrika, 1966.
[4] H. Karcher. Riemannian center of mass and mollifier smoothing. Communications on Pure and Applied Mathematics, 1977.
[5] G. Zhu, X. Liu, S. Tang, J. Niu. Aligning before aggregating: Enabling communication efficient cross-domain federated learning via consistent feature extraction. IEEE Transactions on Mobile Computing, 2024.
[6] M. Casado-Diez, A. Dopico-Castro, V. Bolón-Canedo, B. Guijarro-Berdiñas. Closing the alignment-maturity gap in federated prototype learning. arXiv:2606.02172, 2026.

W2 & Q2: The analysis assumes that differences between client prototype configurations can be approximated by orthogonal transformations, while independently trained models with different architectures may differ nonlinearly. How well does this hold for the architectures used, e.g. in terms of registration residuals or alignment errors?

Thank you for this question. In short:

  • (1) The assumption does not require the networks to be related by an orthogonal map. It only requires the class prototypes at the shared interface to have the same inter-class geometry;
  • (2) On the trained clients of Section 4, this holds well: one orthogonal map per client removes almost all of the client-to-global distance for every architecture, and the class structure agrees across architectures better than chance;
  • (3) Where it does not hold exactly, Theorem 1 bounds the error, which we tested with known deviations.

(1) What the assumption requires. Assumption 1 does not require the networks to be related by an orthogonal map; they may differ nonlinearly. It concerns only the normalized class prototypes at the shared interface, after each client's backbone and encoder. Two prototype configurations on the sphere differ by an orthogonal map exactly when they have the same inter-class angles. The assumption therefore asks that clients agree on the class geometry, which ETF anchoring encourages only approximately. Any remaining difference is the term EiE_i of Assumption 1: (2) measures how well the assumption holds, and (3) bounds the effect of what remains.

(2) Registration residuals per architecture. CIFAR-100, practical setting (Dirichlet, α = 0.1), 3 runs, last 100 rounds, median over each architecture's clients.

  • Before / After: squared distance between a client's normalized class prototypes and the global prototypes of the same classes, without / with registration (2 = orthogonal).
  • Structure: a test of the class-specific part. After registration, we remove from each client's prototypes the direction shared by all its classes, and measure the squared distance between what remains for the same class in clients with other architectures (0 = identical, ≈ 2 = unrelated).
  • Chance: the same measure with the client's class labels permuted, so that different classes are compared. Structure below Chance means that registration places the same class at the same position across architectures. | Architecture | Before | After | Structure | Chance | | - | - | - | - | - | | 4-layer CNN|1.85| 3.2×10⁻⁵| 0.437| 1.022| | GoogLeNet|1.85| 4.5×10⁻⁵| 0.575| 0.935| | MobileNetV2|2.01| 5.9×10⁻⁵| 0.634| 0.968| | ResNet-18|1.92| 7.8×10⁻⁴| 0.531| 1.094| | ResNet-34|1.85| 4.3×10⁻⁵| 0.315| 0.978| | ResNet-50|1.83| 5.0×10⁻⁵| 0.541| 0.951| | ResNet-101|1.96| 9.4×10⁻⁵| 0.762| 0.907| | ResNet-152|1.85| 8.2×10⁻⁵| 0.619| 0.862| | All clients | 1.91 | 5.7×10⁻⁵ | 0.530 | 0.961 |

Other settings, all clients:

SettingBeforeAfter
CIFAR-10, practical1.763.0×10−53.0\times10^{-5}
CIFAR-100, pathological0.822.4×10−62.4\times10^{-6}
Tiny-ImageNet, practical1.961.3×10−51.3\times10^{-5}

Finding.

  • One map removes almost all of the distance. For every architecture, one orthogonal map per client reduces the distance from 1.83–2.01 to at most 7.8×10−47.8\times10^{-4}. The all-client median is at most 5.7×10−55.7\times10^{-5} in every setting.
  • The class structure is aligned better than chance. Most of the distance in Before lies along a direction shared by a client's classes, so a small After alone does not show that individual classes are aligned. With that direction removed, Structure is still below Chance for every architecture under both splits (all clients: 0.530 vs 0.961 practical, 0.040 vs 0.420 pathological). Under the practical setting, the margin is smallest for ResNet-101 (0.762 vs 0.907).

(3) When the assumption does not hold exactly. Theorem 1 bounds the error of the registered prototypes by ∥Ei∥+2∥Fi∥\lVert E_i\rVert + 2\lVert F_i\rVert, where EiE_i collects any departure from an orthogonal map and FiF_i the error of the server's reference. We applied a known rotation plus a known non-orthogonal deviation (anisotropic stretch or per-class noise, 10 draws) to the trained prototypes of all three datasets.

  • The bound is informative. Under per-class noise of level 0.1, the error reaches 69%–91% of the bound.
  • Registration does not amplify the deviation. The error stays below ‖E_i‖, and the global prototypes move by only 4.0°–4.8°, against about 90° without registration.
  • Limitation. Class-dependent distortions that no single orthogonal map removes remain. Theorem 1 and Corollary 3 bound their effect on the global prototypes.

Revision. We have added one sentence after the orthogonal-gauge model in Section 3.1 and after Assumption 1 (Appendix A.2.1), and the measurements to Appendices A.6 and A.7.

W3 & Q1: Most of the improvement appears to come from the spherical projection rather than from registration and Fréchet aggregation, which are presented as a central part of the method. Can the authors provide stronger evidence for the independent contribution of the server-side mechanism?

Thank you for raising this point. We agree: the spherical interface gives the largest part of the gain over FedProto, and we have corrected the Section 4.2 sentence that attributed this gain to the aggregation rule. The server-side mechanism still contributes on its own, in (1) accuracy and (2) the geometry of the global prototypes.

(1) Accuracy contribution of each component. Practical setting, change in accuracy (points), from Table 4:

Component addedComparisonCIFAR-10CIFAR-100Tiny-ImageNet
Spherical interfaceFedProto → V1+3.21+7.53+7.45
Gauge registrationV3 → V4+0.62+0.41+0.28
Fréchet aggregationV2 → V4+0.64+0.41+0.52
Both server componentsV1 → V4+0.96+1.16+0.43

Finding.

  • The interface gives most of the gain over FedProto. A shared dimension and scale make prototypes communicable across architectures.
  • The server side adds a consistent gain. Registration and Fréchet aggregation together add +0.96 / +1.16 / +0.43 points over the interface alone, on all three datasets.
  • Both server components are needed. Each alone gives a small or inconsistent change over V1, while removing either one from GeoProtoFL lowers accuracy on all three datasets (V3 → V4 and V2 → V4).

(2) Contribution beyond accuracy. The interface alone does not make prototypes comparable. Without registration, local training pulls each client toward the global coordinates, but the server still averages prototypes about 58° from the global ones (squared distance 0.94). Registration aligns them to about 1° (about 3×10−43\times10^{-4}) and lets each client keep its own frame. With known client rotations, the global prototypes are displaced by 62° without registration and by 0.002° with it (table in R-M2Ds-W4).

Revision. We have corrected the Section 4.2 sentence and added the rotation test to the ablation study (Section 4.5).

Reviewer_CCLPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the prototype-based heterogeneous federated learning problem, which allows multiple clients with different model architectures to learn collaboratively through class-prototype exchange. To address the global prototype distortion, this paper introduces GeoProtoFL, a novel framework to enable heterogeneous class prototypes geometrically aggregatable. Extensive experiments on three datasets using HtFLlib demonstrate the good performance of the proposed method.

Critical strengths

(+) This paper formulates geometric aggregatability formally for prototype-based HtFL, which could potentially impact the following research work.

(+) This paper conducts extensive experiments across three vision benchmarks under varying statistical heterogeneity, model heterogeneity, and client participation, which demonstrates the effectiveness of the proposed method.

Critical weaknesses

(-) The analysis of ablation studies is too limited, which causes difficulties in justifying the effectiveness of each proposed component.

Questions for authors

See weaknesses.

4 comments

W1: The analysis of ablation studies is too limited, which causes difficulties in justifying the effectiveness of each proposed component.

Thank you for this comment. We have expanded the ablation analysis in two parts: (1) a component-by-component reading of Table 4, and (2) a new experiment on what registration does.

(1) Accuracy contribution of each component. Practical setting, change in accuracy (points), from Table 4:

Component addedComparisonCIFAR-10CIFAR-100Tiny-ImageNet
Spherical interfaceFedProto → V1+3.21+7.53+7.45
Gauge registrationV3 → V4+0.62+0.41+0.28
Fréchet aggregationV2 → V4+0.64+0.41+0.52
Both server componentsV1 → V4+0.96+1.16+0.43

Finding.

  • Every component adds accuracy on all three datasets. Removing either server-side component from GeoProtoFL lowers accuracy on all three.
  • Interface. It gives the largest step. Within it, removing only the ETF anchoring loss gives the lowest accuracy in the sensitivity study (46.63–46.93%, against 47.28–47.90% with anchoring; Appendix A.5).
  • Server side. Registration makes client prototypes comparable, and Fréchet aggregation combines them under the same geodesic criterion that drives local guidance (Proposition 4).

The ablation in (1) reports accuracy only. To strengthen the ablation study, we additionally analyzed the effect of gauge registration on the global prototypes themselves, in a controlled experiment with known client rotations.

(2) New experiment: training with known client rotations.

  • Setup. After learning its bi-projection, each client multiplies its projected features by a fixed random rotation for the whole run (CIFAR-100, practical setting, 300 rounds, 3 runs). V3 is GeoProtoFL without registration.
  • Global-prototype displacement: angle between the global prototypes and those that the same server rule computes from the de-rotated uploads. † marks values that are zero by construction (no rotation).
  • Client-to-global distance: squared distance between a client's normalized class prototypes and the global prototypes of the same classes (2 = orthogonal); mean over clients at the last round.
VariantRegistrationRotationAccuracy (%)Global-prototype displacement (°)Client-to-global distance, unregisteredClient-to-global distance, registered
V3––47.49±0.300†0.94–
V3–✓47.58±0.1962.00.96–
V4 (GeoProtoFL)✓–47.90±0.150†1.893.3×10−43.3\times10^{-4}
V4 (GeoProtoFL)✓✓47.88±0.270.0022.042.9×10−42.9\times10^{-4}

Finding.

  • Registration makes GeoProtoFL invariant to the clients' frames. The rotations displace its global prototypes by only 0.002° and leave its accuracy unchanged, as Proposition 2(ii) predicts. Its clients keep their own frames (distance 1.89 / 2.04), and registration aligns them to about 1° (3.3×10−43.3\times10^{-4} / 2.9×10−42.9\times10^{-4}).
  • Without registration, the same rotations displace the global prototypes by 62°. Accuracy still holds because local training pulls each client toward the global coordinates, with or without rotations, but the prototypes that the server averages still lie about 58° from the global ones (distance 0.94 / 0.96).
  • The same holds on the trained prototypes of all three datasets (10 random rotations): the displacement is 0.00° with registration, against 88.5°–91.0° without it.

Revision. We have added the table in (2) to the ablation study (Section 4.5) and the test on the trained prototypes to Appendix A.7.

Dear Reviewer,

We appreciate the time you have taken to review our submission. We have posted point-by-point responses to your comments and updated the manuscript accordingly, with all changes highlighted in the revised PDF.

As a gentle reminder, since the discussion period is closing soon, we would appreciate it if you could take a look at our responses when you have a chance. We would be happy to clarify any points or address further questions.

Best regards,

The Authors

Thanks to authors’ rebuttal. More component-by-component ablation studies and new experiments addressed my concerns. I keep my original score to support this paper.

Thank you for confirming that your concerns have been addressed and for supporting the paper. Your comments on the ablation analysis helped us make the paper clearer and stronger, and we sincerely appreciate your time and constructive feedback.

Comments on the paper

1 comment

Dear Reviewers and Area Chair,

We sincerely thank all reviewers for their careful reading and constructive comments, which have helped us strengthen the paper. We have posted point-by-point responses and updated the manuscript, with all changes highlighted in the revised PDF. For convenience, we summarize below how each main concern was addressed.

Reviewer CCLP

  • Limited ablation analysis (W1). We added the gain of each component and a controlled client-rotation test that isolates gauge registration. Removing either server component lowers accuracy on all three datasets (Table 4). With known client rotations, the global prototypes are displaced by 62° without registration and by only 0.002° with it, at unchanged accuracy, as Proposition 2(ii) predicts (Section 4.5, Table 5; Appendix A.7).

Reviewer M2Ds

  • Novelty beyond combining existing techniques (W1). We credit the classical tools where they are introduced (Section 3) and state the contribution as the formulation of geometric aggregatability, Proposition 1 (class-wise aggregation cannot, in general, be gauge consistent), and a framework in which each component meets one of its requirements (Sections 1, 2.1).
  • Whether the orthogonal assumption holds across architectures (W2, Q2). On the trained clients, one orthogonal map reduces the client-to-global distance from 1.83–2.01 to at most 7.8×10⁻⁴ for every architecture, and the class structure is aligned better than chance for all eight architectures. Under known non-orthogonal deviations, the bound of Theorem 1 holds in all 5,400 cases (Section 3.1; Appendix A.6–A.7, Tables 12–14).
  • Independent contribution of the server-side mechanism (W3, Q1). We agree that the interface gives the largest gain and corrected the Section 4.2 sentence. Registration and Fréchet aggregation together add +0.96 / +1.16 / +0.43% on CIFAR-10/CIFAR-100/Tiny-ImageNet over the interface alone, and removing either lowers accuracy on all three datasets (Table 4).
  • Direct evidence under known orthogonal perturbations (W4). Registration removes known rotations: the global prototypes are displaced by 0.00° on the trained prototypes (against 88.5°–91.0° without registration) and by 0.002° during training (against 62°) (Section 4.5, Table 5; Appendix A.7).
  • Datasets with genuine domain differences (W5, Q4). On DomainNet, GeoProtoFL achieves the highest accuracy, +0.84% over the strongest baseline (Table 1).
  • Baseline configurations and selection of the reported round (W6, W7). We list each method's hyperparameters with their sources (Table 9) and state the selection rule (Section 4.1). Under FedORGP's own protocol, our FedORGP reproduces its published results and GeoProtoFL stays ahead. At the final round, with no checkpoint selection, GeoProtoFL remains first in all six settings, by 1.11–3.65% (Appendix A.8).

Reviewer X1KG

  • We thank the reviewer for the follow-up discussion, which led to additional diagnostics of Assumption 1 (Appendix A.6), a partial-participation analysis (Appendix A.9, Section 4.3), and message sizes for all seven baselines, with GeoProtoFL exchanging 2.2–39.9× fewer bytes per round (Appendix A.1, Section 4.4).

These revisions stay within the scope of the original submission, and the main text remains within the 10-page limit. As the discussion period is closing soon, we would greatly appreciate it if reviewers who have not yet had a chance could take a look at our responses. We would be happy to clarify any points or address further questions. Thank you again for your time and valuable feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.