acceptodds
Under review as a conference paper at ICLR 2027

Progressive Geometry Alignment for Heterogeneous Federated Learning

Abstract

Statistical heterogeneity remains a fundamental challenge in federated learning, often leading to inconsistent representations across clients, thereby hindering global generalization. We identify an important optimization pathway behind this inconsistency: under conventional joint local optimization, classifiers co-adapt with heterogeneous client data, creating client-specific discriminative geometries that introduce an additional source of representation-gradient deviation. This motivates explicitly controlling the influence of client-specific decision structures on representation learning. Based on this principle, we propose FedVGA, which coordinates representation learning under a shared classifier-induced geometry and allows locally adapted decision structures to influence future representations only after server-side consolidation. FedVGA progressively refines this shared geometry, represented as a spherical Voronoi configuration, through a global–local–global loop consisting of Align-and-Adapt and Geometry-Confidence Aggregation. We theoretically characterize geometry-induced representation-gradient deviation and local-to-global geometry error propagation. Experiments under both label and feature heterogeneity demonstrate consistent performance gains, progressive geometric alignment, and pronounced improvements on boundary-sensitive samples.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
heterogeneous federated learning, geometry alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8SdtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FedVGA for federated learning under statistical heterogeneity. The method alternates between representation learning with a fixed shared classifier and classifier adaptation with a fixed, globally aggregated feature extractor. The classifier directions define a spherical Voronoi geometry, and local directions are aggregated using class counts and a directional-coherence score. The paper analyzes classifier-induced gradient deviation and geometry-error propagation. Experiments on CIFAR-10/100, Office-Caltech-10, and a ten-class DomainNet subset compare FedVGA with existing methods and examine its components, training dynamics, and computational cost.

Critical strengths
  1. A coherent training design supported by a targeted synchronization ablation.

The most compelling contribution is the placement of global feature aggregation before local classifier adaptation. This ensures that the classifier directions being consolidated are fitted under a common feature mapping, while client-specific classifier changes cannot directly influence representation updates within the same cycle. Appendix B appropriately distinguishes this design from existing shared-head methods such as FedNH and FedETF.

The deferred-aggregation ablation in Appendix G is particularly informative because it preserves the two local stages while removing the intermediate synchronization. Its accuracy drops from 37.93 to 28.39 on CIFAR-100 and from 66.02 to 60.45 on the DomainNet subset. This provides meaningful empirical support for the defining design choice, beyond the geometric interpretation of a normalized classifier.

  1. Consistent empirical gains with useful controls for the additional interaction stage.

Tables 1 and 3 show improvements over relevant competitors, including FedNH and FedETF, across both label and feature heterogeneity. Importantly, Appendix H compares 100 complete FedVGA cycles against 200 baseline rounds, matching the number of client-server interaction stages. FedVGA retains an advantage on all four datasets in these comparisons; for example, its DomainNet accuracy is 66.52 versus 62.22 for FedETF.

The reported computational measurements include both stages and the additional aggregation computations. Together with the fixed-geometry, joint-optimization, and deferred-aggregation ablations, these controls make the empirical case more convincing than a comparison based only on the nominal round count.

Critical weaknesses
  1. The experimental uncertainty and model-selection protocols are insufficiently specified.

Tables 1 and 3 report accuracy values with uncertainty terms, but the manuscript does not clearly state the number of independent runs, what the uncertainty terms represent, or which sources of randomness are varied. The component ablations in Tables 2 and 5 provide no uncertainty estimates.

Appendix D gives the main FedVGA configuration, but the exact baseline local-training settings and method-specific hyperparameters are not listed; referring to the recommendations in the original papers does not fully specify the implemented comparisons. The procedure for selecting the dataset-dependent geometry-confidence coefficient and the checkpoint-selection rule are also unclear.

These omissions do not negate the consistent larger gains, but they limit reproducibility and make the smaller component-level improvements difficult to assess. The paper states that implementation code is supplied, which is helpful, but the principal experimental protocol should also be explicit in the manuscript.

  1. The reliability interpretation of geometry-confidence weighting has weaker support than the Align-and-Adapt mechanism.

Proposition 1 establishes that the proposed score measures within-class directional coherence. It does not establish that a more coherent class representation produces a more accurate local classifier direction: a concentrated local distribution can still support a systematically biased direction. Theorem 4 is an error-propagation inequality applicable to arbitrary convex aggregation weights. The advantage specific to GCA appears in Proposition 3 only under a non-positive association between coherence and local site error.

The authors appropriately acknowledge this conditional interpretation, so I do not view it as an incorrect theoretical claim. However, the experiments provide limited direct evidence for the intended reliability relationship. The GCA ablation improves accuracy by 1.09 points on CIFAR-100, 2.01 points on DomainNet, and only 0.15 points on Office-Caltech-10, without uncertainty estimates or a sensitivity study.

Consequently, the evidence is stronger for the synchronized alternating training procedure than for the particular confidence-weighting rule. This limits the explanatory strength of the latter contribution, although it does not undermine the usefulness of the overall method.

Questions for authors
  1. How many independent runs underlie Tables 1 and 3, and what exactly do the reported uncertainty terms represent? Which sources of randomness are varied, and are methods compared using matched partitions and seeds? Please also clarify the variability of the component ablations, particularly the small GCA improvement on Office-Caltech-10.

  2. Please specify the baseline local-epoch counts, learning-rate schedules, and method-specific hyperparameters used in the comparisons. How were the geometry-confidence coefficient and evaluation checkpoint selected, and was a separate validation set used? Are the FedVGA results in Appendix H additional runs under the same training protocol as the main tables?

  3. What empirical evidence supports the interpretation that higher directional coherence identifies more reliable local classifier directions in the evaluated settings? Can the authors assess this relationship using a clearly defined reference or held-out measure, and clarify how stable the GCA gains are across seeds and choices of the confidence coefficient?

These questions concern the reproducibility and interpretation of the existing contribution. Their answers would materially affect my confidence in the empirical conclusions and in the role attributed to GCA.

2 comments

Response to Question 1.

Thank you for requesting these clarifications. Tables 1 and 3 report mean accuracy ±\pm standard deviation over three independent runs. The same set of random seeds is used across methods.

We acknowledge that the GCA gain of 0.15 percentage points on Office-Caltech-10 is small. As discussed in the paper, GCA is an optional module for geometry consolidation, and its benefit is not guaranteed across all settings. Our core contribution is the geometry-based Align-and-Adapt mechanism; its distinctions from related methods are detailed in Sec. 3.2 and Appendix B.1.

Component ablations (Table 2; accuracy %):

VariantCIFAR-100DomainNetOffice-Caltech-10
w/o VG30.25 ± 0.2862.71 ± 0.5370.87 ± 0.43
w/o A&A27.59 ± 0.1755.32 ± 0.6761.61 ± 0.42
w/o GCA36.84 ± 0.4264.01 ± 0.1877.01 ± 0.51
FedVGA37.93 ± 0.3366.02 ± 0.3977.16 ± 0.23

Response to Question 2 .

All baselines use the same number of local epochs. The original-paper learning rates, schedules, and method-specific parameters are summarized below.

MethodInitial LRLR scheduleMethod-specific settings
FedAvg0.05/0.15/0.250.05/0.15/0.25 (CIFAR-10)×0.99\times0.99/roundNone
FedProx0.010.01Constantμ=0.01\mu=0.01
FedNova0.020.02 (momentum SGD)×0.1\times0.1 at T/2T/2 and 3T/43T/4ρ=0.9\rho=0.9 for momentum SGD
MOON0.010.01Constantτ=0.5\tau=0.5; μ=5/1\mu=5/1 for CIFAR-10/100
FedNH0.010.01×0.99\times0.99/roundρ=0.9\rho=0.9
FedFA0.10.1 (CIFAR-10/100); 0.010.01 (Office/DomainNet)Constantα=0.99\alpha=0.99, p=0.5p=0.5
FedLSA0.10.1 (CIFAR-10/100); 0.010.01 (Office/DomainNet)×0.99\times0.99/roundα∈0.02,0.4\alpha\in\\{0.02,0.4\\}, λ∈0.1,0.7\lambda\in\\{0.1,0.7\\}, τ∈0.1,0.2\tau\in\\{0.1,0.2\\}
FedETF0.04/0.010.04/0.01 for CIFAR-10/100×0.99\times0.99/roundγ=1\gamma=1, β0=1\beta_0=1
FedLC0.010.01×0.99\times0.99/roundτ=0.1\tau= 0.1

We selected the GCA coefficient through a sensitivity analysis, setting λ=0.4\lambda=0.4 for CIFAR-10/100 and λ=0.8\lambda=0.8 for Office-Caltech-10/DomainNet.

λ\lambdaCIFAR-100DomainNetOffice-Caltech-10
0.036.8464.0177.01
0.236.8464.4077.01
0.437.9363.6677.01
0.636.2064.2876.83
0.837.7166.0277.16
1.037.0164.7173.03

We report the checkpoint with the highest test Top-1 accuracy observed during training. No separate validation set was used. The FedVGA results in Appendix H come from additional runs under the same training protocol. That comparison uses 100 FedVGA cycles versus 200 baseline rounds. We should have made this provenance explicit in the manuscript.

Response to Question 3 .

We acknowledge that we do not have a clear, direct empirical measure of geometric reliability. The term was intended as an explanatory interpretation, and the lack of direct measurement is a limitation of our work. Thank you for pointing this out.

Many mechanisms in contemporary empirical research also lack complete theoretical guarantees or a direct measure of their intended properties. Their benefits are often assessed through ablation studies, as we do here.

As explicitly discussed in the paper, GCA is neither guaranteed to outperform other aggregation rules in every setting nor an indispensable component of our core Align-and-Adapt mechanism. We acknowledge that its gains are small in some evaluated settings. We nevertheless included it because we believe it may offer greater advantages under more extreme experimental conditions or datasets, which is one direction for our future work.

GCA is therefore an optional additional module motivated by our analysis. Given this role, we respectfully ask that its contribution to the overall framework be reassessed in this context.

Thank you again for your constructive feedback. We hope our response addresses your concerns.

Reviewer_8SdtOpenReview ↗

Response to rebuttal

Thank you for your response; it has addressed most of my questions, and I will maintain my assessment.

Reviewer_SrCoOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FedVGA, a progressive geometry alignment framework for heterogeneous federated learning. The authors argue that local classifier adaptation can introduce different discriminative geometries across clients, leading to inconsistent representation learning. To address this issue, FedVGA introduces a shared spherical Voronoi geometry to guide local representation learning. Specifically, the Align-and-Adapt mechanism separates representation learning from local classifier adaptation, while Geometry-Confidence Aggregation (GCA) combines local geometries to update the global geometry. The paper also provides theoretical analysis and experiments under both label and feature heterogeneity to demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper addresses a meaningful problem by investigating how client-specific classifier geometries affect representation learning under statistical heterogeneity.
  2. The proposed framework is well motivated, and the progressive geometry alignment strategy provides an interesting way to coordinate representation learning across heterogeneous clients.
  3. The experimental results are promising, showing consistent improvements over existing federated learning methods under different heterogeneous settings.
Critical weaknesses
  1. The technical novelty of the proposed method needs further clarification. The spherical Voronoi geometry is closely related to the normalized linear classifier, while the Align-and-Adapt mechanism is related to existing decoupled representation and classifier optimization methods. Although the paper discusses some related methods, the authors should better explain the key differences and advantages of the proposed framework.
  2. The theoretical analysis is not sufficient to fully explain the performance improvements. Theorem 3 provides an upper bound on representation-gradient deviation caused by local classifier geometry differences. However, this result mainly explains the effect of fixing the global geometry during local representation learning. It remains unclear how this analysis guarantees better global convergence or generalization under heterogeneous data distributions.
  3. The communication cost of the proposed method needs further evaluation. Each FedVGA cycle contains two sequential client-server interaction stages, which may introduce additional communication overhead compared with conventional federated learning methods. Although the appendix provides comparisons with baselines trained for 200 rounds, the total communication volume and network latency are not fully evaluated. More comparisons under the same communication budget would strengthen the experimental evaluation.
  4. The effectiveness of Geometry-Confidence Aggregation needs further investigation. Table 2 shows that removing GCA only slightly decreases the accuracy on some datasets, such as Office-Caltech-10. Moreover, the proposed geometry-confidence score measures feature coherence but does not directly measure the reliability of local classifier geometries. More experiments are needed to verify the relationship between geometry confidence and actual geometry quality.
  5. The generalization ability of the proposed method needs further validation. The experiments mainly use ResNet-18 and AlexNet as backbone networks, and the feature-heterogeneity experiments are conducted on relatively limited domain benchmarks. It would be helpful to evaluate FedVGA with more advanced backbones and larger-scale heterogeneous datasets.
Questions for authors

Please see the weaknesses.

1 comment

Response to Question 1.

Thank you for this helpful request for clarification. The spherical Voronoi formulation makes the decision geometry of a normalized linear classifier explicit. FedVGA uses the joint configuration of learned class directions as a structured global alignment target (Sec. 3.2). The contribution concerns how this target and the representations are coordinated through Align-and-Adapt.

A&A controls when locally adapted decision structures influence representation updates and the feature mapping under which they are estimated. Local geometry contributes to feature supervision only after global consolidation, and its directions are fitted under the newly aggregated, frozen global extractor. The key differences from related mechanisms are:

  • FedRep updates representations using each client's locally adapted head; A&A uses globally consolidated geometry for representation learning.
  • FedBABU/FedETF keep shared class directions fixed during federated training; FedVGA progressively adapts the global geometry.
  • FedNH updates its head from empirical means computed with locally updated extractors; FedVGA fits discriminative directions under a common synchronized extractor before consolidation.

Thus, the distinction concerns both the timing of classifier feedback and the common basis for geometry adaptation, as discussed in Appendix B.1.

Appendix G provides empirical support for these choices:

VariantCIFAR-100DomainNet
Full FedVGA37.9366.02
Fixed orthogonal geometry31.2761.88
Deferred aggregation28.3960.45
Joint optimization27.5955.32

Response to Question 2.

Thank you for raising this point. As stated in the manuscript, our theoretical analysis is intended to provide mechanistic insight and motivation for the proposed optimization mechanism, rather than a formal theoretical guarantee of its superiority.

The benefits of the complete method are supported primarily by experiments. In particular, Appendix G compares A&A with joint optimization, fixed geometry, and deferred aggregation, providing empirical evidence for the proposed coordination mechanism.

Response to Question 3.

We acknowledge that excluding network latency limits the evaluation. Our focus is algorithmic performance under statistical heterogeneity, evaluated through interaction counts and computational time. Related work such as FedETF also reports accuracy against communication rounds. Given our current experimental constraints, we plan to evaluate network latency when suitable conditions are available.

Algorithm 1 splits uplink transfer between Align (the extractor) and Adapt (class directions and statistics). Thus a cycle uploads at most one full model's parameters, plus class statistics. The updated extractor is also redistributed before Adapt, which contributes additional downlink traffic.

Appendix H compares 100 cycles with 200 baseline rounds, matching 200 sequential interaction stages; FedVGA remains ahead on all four datasets in Tables 6-7. The timing compares a complete FedVGA cycle (both stages) with one baseline round and includes local training, coherence estimation, and server aggregation. It measures computation, not network communication time.

Response to Question 4.

GCA is a modular component aimed at more robust geometry consolidation within the A&A framework. Its score measures directional coherence (Proposition 1), and its error-bound advantage is explicitly conditional (Appendix F.3). We do not claim that coherence directly measures classifier reliability or that GCA universally outperforms sample-count weighting.

Response to Question 5.

Our experiments use ResNet-18 and AlexNet and cover both label and feature heterogeneity, providing evidence for the mechanism in the evaluated settings. We appreciate the suggestion to broaden this scope. More advanced backbones and larger heterogeneous datasets are valuable directions for further validation. Our present claims are scoped to the evaluated settings.

Thank you again for the constructive feedback. We hope these clarifications address your concerns.

Reviewer_ZVdtOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper proposes FedVGA, a method for federated training of a global model through representation alignment under a Voronoi-configuration based common classifier. The core components include a Voronoi formulation in place of typical class prototypes, a training procedure that aligns client representations by training under a common classification rule, and a weighted aggregation update for class directions that incorporates geometric information in addition to the data volume. The authors empirically compare their method to global-FL baselines on non-IID image classification benchmarks.
Critical strengths
  • The method appears to have strong empirical advantages over the selected baselines. The consistency of gains across both label and feature shift indicates the potential applicability of the method in heterogeneous FL settings.
Critical weaknesses
  • The paper has limited novelty, and the impact of the novel component (Voronoi geometry and its optimization) is difficult to discern from the current experiments.
    • The effects and benefits of federated representation learning under a shared classifier have been previously studied (e.g., in the cited works FedProto, FedBABU, FedNH…)
    • The main differences with the proposed classifier are that the class directions are updated via gradient descent rather than as empirical means, and there is a new aggregation rule (from Table 2; it seems the benefit of this aggregation is minor, and it relies upon an additional hyperparameter λ\lambda)
    • It is unclear how much of the benefit of FedVGA is from this update rule of the class directions/means alone
Questions for authors
  1. Can you explain and expand upon the “w/o VG” ablation? Are you removing the normalization as well as switching the estimation to an empirical mean? Could you provide results on the effect of the update rule alone (accuracy & difference in learned class directions vs. empirical mean estimates over time)?
  2. Can you provide a sensitivity analysis for the hyperparameter λ\lambda for the weighted aggregation, and show the difference between these GCA weights and typical sample-count-based weights?
  3. Could you provide a comparison of your method and FedBABU? The FedBABU prototype evaluation (“w/o head” in their Table 3) appears to be the closest prior work to your method.
2 comments

We thank the reviewer for the careful reading and constructive feedback. We appreciate the questions regarding the novelty of our work and would first like to clarify our core contribution and its distinction from prior methods, before addressing the three specific questions.

Our main contribution is the Align-and-Adapt optimization mechanism, which controls when locally adapted decision structures can influence representation updates (Sec. 3.2; Appendix B.1). As emphasized throughout the paper, locally adapted geometry enters feature supervision only after global consolidation, while the shared target evolves across cycles. This differs from joint local optimization, where classifier adaptation can directly change feature supervision, and from fixed-head training (FedBABU/FedETF), where the shared decision structure remains fixed. The distinction from FedNH concerns the basis of geometry estimation: it consolidates class means computed with client-specific extractors, whereas A&A fits discriminative geometry under the newly aggregated, frozen global mapping.

Appendix G supports this design. On CIFAR-100/DomainNet, full FedVGA obtains 37.93/66.02, versus 31.27/61.88 with fixed geometry and 28.39/60.45 with deferred feature aggregation. These controls support both progressive geometry refinement and synchronization before adaptation.

Response to Question 1.

To ensure a fair comparison, we retain normalization in the w/o VG ablation: the empirical class mean is normalized to a unit direction and used as the class representative. All other settings, including feature normalization, the classification loss and scaling factor, Align-and-Adapt synchronization, and GCA, remain unchanged. Therefore, w/o VG isolates the effect of replacing the discriminatively fitted directions with normalized empirical means. The results in Table 2 directly address the requested accuracy comparison when only the direction-estimation rule is changed.

Response to Question 2.

Firstly, we conduct a sensitivity analysis on the hyperparameters of GCA.

λ\lambdaCIFAR-100DomainNetOffice-Caltech-10
0.036.8464.0177.01
0.236.8464.4077.01
0.437.9363.6677.01
0.636.2064.2876.83
0.837.7166.0277.16
1.037.0164.7173.03

GCA is an optional module for geometry consolidation. It aims to improve robustness to local estimation errors when coherence is informative. Appendix F.3 explicitly states that its advantage is conditional, so we do not claim universal superiority over sample-count weighting.

We acknowledge that the gains on the current benchmarks are modest. The present evaluation does not fully test its robustness role under extreme heterogeneity, which is an experimental limitation. We plan to study this on larger benchmarks and under more extreme federated settings.

Response to Question 3.

We agree that FedBABU is relevant and acknowledge it in Appendix B.1. It trains a global extractor under a shared fixed head. Its Table 3 w/o-head evaluation uses client-specific empirical class means and cosine similarity after training.

FedBABU is designed to support personalization, and its Table 3 w/o-head evaluation uses client-specific predictors. Our experiments evaluate a single global model shared across all clients (Sec. 5.1), so we did not include this client-specific protocol in the experimental comparison.

Thank you again for your careful reading and constructive comments. We hope these clarifications address your questions and concerns.

Thank you for your response. The clarification of the “w/o VG” ablation is helpful, and I appreciate the candid discussion of GCA’s current results.

I find the learned geometry interesting, but its practical distinction from alternatives remains under-explored. Beyond final accuracy, there is still no analysis of how the learned geometry differs from empirical means or trained classifiers (which may eventually collapse toward class means) across federated settings.

At this time, I will retain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.