acceptodds
Withdrawn from ICLR 2027

OFLBe: Do Our Best to Make One-Shot Federated Learning Better

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

One-shot federated learning (OFL) enables collaborative training across multiple clients through a single round of client-server interaction, substantially reducing communication costs compared to traditional multi-round FL. However, under non-IID data settings, clients perform local training independently without multi-round interactions with the server to align the representation space across clients, leading to severe feature drift. To address this issue, we propose OFLBe, an OFL framework based on consistent anchor semantic space optimization. OFLBe leverages a pre-trained CLIP text encoder to construct shared class-level semantic anchors, providing a unified semantic representation space reference for all clients. During local training, OFLBe introduces an anchor-guided feature alignment loss and a learnable prototype loss to guide cross-client learning toward a consistent representation space with intra-class compactness and inter-class discriminability. Moreover, on the server side, instead of directly averaging the clients’ model parameters, OFLBe aggregates the uploaded local prototypes and the features output by the local models during inference based on the similarity between the uploaded local prototypes and the class-level semantic anchors, thereby maximizing the utilization of knowledge from a single client-server interaction. Experiments on three datasets under various data heterogeneity settings demonstrate that OFLBe achieves the best performance compared with 12 OFL baselines.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Learning, One-Shot Federated Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_n2A5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes OFLBe, a semantic-anchor-guided framework for one-shot federated learning (OFL) under non-IID data distributions. The method constructs shared class-level semantic anchors using a pretrained CLIP text encoder to provide a common reference for independently trained clients. During local training, an anchor-guided feature alignment loss and a learnable-prototype classification loss are jointly optimized. After a single upload of local models and class prototypes, the server performs similarity-weighted prototype aggregation and feature-level ensemble inference. The paper also derives conditional bounds relating anchor discrepancies to intra-client and inter-client representation inconsistency. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet report improvements over existing OFL baselines under different degrees of label heterogeneity.

Critical strengths

1. Well-motivated semantic alignment for one-shot federated learning. The paper addresses an important challenge in OFL: independently trained clients may learn inconsistent representations under heterogeneous data distributions, without subsequent communication rounds to correct them. Constructing shared semantic anchors before local training provides a reasonable and coherent approach to this problem.

2. Promising empirical performance across heterogeneous settings. Table 1 reports consistent accuracy improvements across CIFAR-10, CIFAR-100, and Tiny-ImageNet under different Dirichlet concentrations, with particularly large gains under severe label skew. The additional experiments on client scalability, heterogeneous architectures, and training costs further support the potential applicability of the framework.

Critical weaknesses

1. Insufficiently established novelty and confounded contribution of pretrained semantics. FAFI (ICML 2025) already combines local feature alignment, learnable class prototypes, server-side prototype aggregation, and feature-level ensemble inference. OFLBe follows a closely related framework, mainly introducing fixed CLIP-derived semantic anchors and similarity-based weighting. However, the paper does not sufficiently isolate the benefits of these algorithmic changes from those of the additional pretrained semantic knowledge. A comparison with FAFI equipped with equivalent CLIP anchors is needed to establish the independent contribution of the proposed approach.

2. Experimental inconsistencies and insufficient validation of the proposed mechanism. Tables 1 and 3 report inconsistent baseline results under the apparently identical CIFAR-10 setting with five clients and α = 0.5. In particular, Co-Boosting changes from 70.24% to 55.34%. FAFI also has different values in Tables 1 and 6 under the corresponding Tiny-ImageNet setting. These discrepancies require clarification, especially because many baseline results are imported from previous papers. Moreover, the paper does not directly quantify the proposed representation-inconsistency measures, and the independent benefit of anchor-aware server integration over simpler aggregation strategies remains insufficiently established.

Questions for authors
  1. Could the authors explain the discrepancies across Tables 1, 3, and 6, particularly the 14.90-percentage-point difference for Co-Boosting? Since many baseline results are taken from previous publications, how are the data partitions, training budgets, model initialization, and evaluation protocols verified to be comparable?

  2. How does OFLBe compare against FAFI when both methods have access to the same frozen CLIP semantic anchors? Can the authors distinguish the contribution of pretrained semantic knowledge from that of the proposed training and server-side integration mechanisms?

  3. Could the authors report the actual intra-client and inter-client inconsistency metrics defined in Appendix C? How much improvement is independently provided by anchor-aware prototype aggregation and feature fusion compared with uniform alternatives? Additionally, how do the fusion weights behave when client class supports have little overlap?

  4. How is the cumulative communication cost in Figure 3 calculated? Does it include the uploads of all client models and prototypes, and how does the server-side inference cost scale with the number of participating clients?

Reviewer_yF58OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposed an OFL framework named OFLBe, which is based on consistent anchor semantic space optimization. OFLBe leverages a pre-trained CLIP text encoder to construct shared class-level semantic anchors. They use the similarity to the CSA as the weight in aggregation.

Critical strengths
  1. the manuscript revisit the performance bottleneck of OFL, and provide detailed theorectical analysis.
  2. the paper is well-organized and easy to follow.
  3. the tables and figures are informative and well-explained with the text.
Critical weaknesses

1.OFLBe introduces CLIP for classification tasks. However, the classification tasks are tested in three simple datasets, CIFRA10, TinyImage. Please provide more analysis to explain the performance increment is brought by the architecture/training strategy design, instead of the pretrained model. 2. The manuscript seems to have too much overlap with the baseline FAFI, please carefully revise the manuscript. 3. The problem, intra-model / inter model inconsistency has been studied in many studies. One point that needs clarification is the main advance of this manuscript relative to prior work. Please provide the key distinguishing feature or incremental contribution compared with previous work. As currently presented, if the main difference is to improve performance by introducing CLIP, the novelty appears somewhat limited.

Questions for authors
  1. why introducing a pretrained CLIP could improve the performance?
  2. what is the key distinguishing feature of OFLBe compared with previous work.
Reviewer_tggFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses feature drift in one-shot federated learning (OFL) under non-IID data, diagnosed as inter-client and intra-client representation inconsistency. The proposed OFLBe constructs shared class-level semantic anchors (CSAs) from a pretrained CLIP text encoder, providing all clients with a unified semantic reference before the single communication round. On the client side, an anchor-guided feature alignment loss and a learnable-prototype classification loss pull local representation spaces toward the shared CSA space. On the server side, an anchor-aware aggregation and inference (AAAI) mechanism aggregates uploaded local prototypes by their similarity to the CSAs, and fuses client models' output features at inference time, again weighted by anchor similarity; final predictions come from matching fused features against global prototypes. Experiments on CIFAR-10/100 and Tiny-ImageNet under several Dirichlet non-IID partitions compare against 12 OFL baselines, with additional studies on scalability (up to 100 clients), communication efficiency versus multi-round FL, ablations of each component, model heterogeneity, and backbone choice. Reported gains are large, especially under high heterogeneity (e.g., +24.30 points over the strongest baseline on CIFAR-100 at alpha=0.05).

Critical strengths
  • A simple, effective idea with strong empirical payoffs. Providing a shared class-level semantic anchor space before the single round is an intuitive and, from my point of view, well-motivated way to prevent feature drift at its source rather than patching it at the server. The margins over 12 baselines are consistently large across three datasets and all heterogeneity levels, and the ablation cleanly shows LA, LP and AAAI each contribute. This empirical strength is the primary consideration behind my assessment.
  • Broad experimental coverage. The scalability study up to 100 clients, the model-heterogeneity cases, the backbone comparison, and the communication-efficiency analysis together make the evaluation more complete than most OFL papers I have seen.
Critical weaknesses
  • Fairness of the comparison. OFLBe injects external knowledge through the pretrained CLIP text encoder while all 12 baselines train from scratch on client data alone. The gains — particularly the very large ones at small alpha — may largely reflect CLIP's semantic prior rather than the proposed alignment mechanism. Without a baseline given access to the same pretrained information, or an ablation replacing CLIP anchors with random or one-hot anchors, the contribution of the CSA itself cannot be isolated, and this directly affects how the main table should be read.
  • The AAAI inference mechanism changes the deployment contract. The server appears to keep all M client models and fuse their output features per test sample — closer to a weighted ensemble than a single global model. Storage and compute at inference scale linearly in M, which partially contradicts the efficiency motivation, yet communication cost is only measured for training. Relatedly, constructing CSAs assumes the complete class name set is known and agreed upon by all parties beforehand, a strong assumption in realistic FL that also implies label-space disclosure; neither point is discussed.
Questions for authors
  • [Clarification] Can you add a baseline with access to the same pretrained knowledge (e.g., CLIP-initialized encoders), or an ablation with random/one-hot anchors, so the CSA's contribution is isolated from CLIP's prior?
  • [Clarification] What exactly must be stored and executed at inference time — all M client models, or only aggregated prototypes? Please report inference-time storage/compute, and a prototype-only variant so readers can see how much gain survives deployment as a single model.
  • [Clarification] Is the shared label space (all class names) assumed known to all parties in advance? How robust is OFLBe to incomplete or mismatched class vocabularies?
  • [Clarification] Local prototypes are class-conditional feature means, which are vulnerable to inversion and membership inference. What is the privacy posture of the upload?
  • [Clarification] In Figure 3, multi-round FL is truncated at 80 rounds; how was this window chosen, and does the communication axis include any CLIP-related transmission cost?
  • [Minor] The paper needs another proofread: "a central sever" (Sec. 2.1), "Sever-side" (Fig. 2), "Notably, Although" (Sec. 3.4), "by evaluate the relationship" (Sec. 3.4). The title reads more like a slogan than a description; a more informative title is suggested.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.