acceptodds
Under review as a conference paper at ICLR 2027

Prompt-Guided One-Shot Federated Learning via Synthetic Data Generation

Abstract

Conventional Federated Learning (FL) requires the server to continuously establish wireless connections with clients to communicate learning models, which burdens the network in terms of management and wireless resource constraints, and may facilitate inference attacks. One-shot FL has emerged as a key innovation to reduce the framework's dependence on a consistent wireless connection. However, existing approaches largely treat clients as passive contributors: the algorithm determines how their local data are used, with limited support for heterogeneous user preferences. In this paper, we propose a one-shot FL framework that allows clients to actively control how their local data contribute to the learning process through synthetic data generation. Each client specifies its preference through a natural-language prompt, which guides generation together with its local images and controls how much the synthetic data deviate from the originals. In addition, we address the limited computational capability of client devices by incorporating a latent consistency model-enabled low-rank adaptation to boost the parameter and energy efficiency. Finally, we conduct a series of experiments to evaluate the effectiveness of our proposal against state-of-the-art benchmarks and obtain substantial accuracy improvements across different deep learning models and levels of heterogeneity across three datasets.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Federated learning, One-shot federated learning, Prompt-guided

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xUAHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper improves the accuracy and efficiency of OSFL by utilizing synthetic data and low-rank adaptation. The experimental results show significant improvements compared to the baseline methods.

Critical strengths

None

Critical weaknesses
  1. This paper proposes an OSFL method based on synthetic data, whereas the baseline methods are non-generative. Therefore, this comparison is unfair and does not prove that the proposed method is SOTA. I believe the authors should have compared it with methods such as FedBiP (CVPR'25) and Espresso (CVPR'26). Furthermore, there are also statistical FL methods based on synthetic data, and the authors should have included these in their comparison.
  2. The two contributions proposed in this paper are not novel in the field of FL. They are methods that have existed for some time. Therefore, this paper does not strike me as particularly innovative.
Questions for authors

I believe the paper has the following issues:

  1. This paper proposes an OSFL method based on synthetic data, whereas the baseline methods are non-generative. Therefore, this comparison is unfair and does not prove that the proposed method is SOTA. I believe the authors should have compared it with methods such as FedBiP (CVPR'25) and Espresso (CVPR'26). Furthermore, there are also statistical FL methods based on synthetic data, and the authors should have included these in their comparison.
  2. The two contributions proposed in this paper are not novel in the field of FL. They are methods that have existed for some time. Therefore, this paper does not strike me as particularly innovative.
Reviewer_C6vEOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a one-shot FL framework that allows clients to actively control how their local data contribute to the learning process through synthetic data generation. Each client specifies its preference through a natural-language prompt, which guides generation together with its local images and controls how much the synthetic data deviate from the originals. In addition, it addresses the limited computational capability of client devices by incorporating a latent consistency model-enabled low-rank adaptation to boost the parameter and energy efficiency. Finally, the authors conduct a series of experiments to evaluate the effectiveness of our proposal against state-of-the-art benchmarks and obtain accuracy improvements across different deep learning models and levels of heterogeneity across three datasets.

Critical strengths

S1. Experiments on CIFAR-10, CIFAR-100, and SVHN show improved classification accuracy over several OSFL baselines under Dirichlet label-skew partitions.

S2. The paper evaluates synthetic-image similarity, generation cost, and communication overhead.

Critical weaknesses

W1. There is a lack of a convincing privacy evaluation.

W2. The efficiency claims rely mainly on FLOPs measured in a high-end GPU environment.

Questions for authors

Q1. How are labels assigned to synthetic images, and what evidence can prove that the generated images preserve their original classes, particularly under aggressive visual abstraction?

Q2. Given the reported re-identification rates, what evidence shows that the generated images resist realistic privacy attacks rather than only differentiating visually from the original images?

Q3. Which parameters or components are newly proposed and learned by the method, and which are directly inherited from the pretrained IP-Adapter and LCM-LoRA pipelines?

Q4. How can the strong-convexity and uniform synthetic-discrepancy assumptions in Appendix A be justified for the CNN and ResNet-18 models used in the experiments?

Q5. What are the actual end-to-end generation latency, peak memory consumption, and energy cost on representative edge devices?

Q6. Does the reported communication cost include the initial distribution of pretrained generation models, and how does the comparison change when communication is measured at comparable final accuracy?

Q7. How much of the performance improvement is attributable to prompt-guided synthesis, image conditioning, LCM-LoRA, and client-side filtering, respectively?

Reviewer_pbivOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a synthetic data-based One-Shot Federated Learning (OSFL) framework. The clients control the synthetic data generation process using preference prompts. Each client executes a frozen Stable Diffusion model with an IP-Adapter and LCM-LoRA. IP-Adapter is used to condition on local images and textual prompts via decoupled cross-attention, and LCM-LoRA is used to reduce the number of denoising steps for computational efficiency. Clients then transmit the synthetic samples in a single round to the central server, and the server trains a global classifier via empirical risk minimization.

Critical strengths
  1. The paper shows a novel and practical perspective on synthetic data-based OSFL by enabling clients to control the utility–deviation trade-off of the shared synthetic data via natural-language prompts and inference hyperparameters. Considering real-world federated learning participants often have varying sharing policies and privacy constraints, the demonstrated prompt-diversified synthetic pools can still be used to train a performant global classifier under extreme non-IID splits, highlighting a practical paradigm for user-centric federated learning.
  2. Incorporating LCM-LoRA into the local synthesis pipeline reduces the number of denoising steps from 20 to 2–3. This cuts the FLOP count by 13x to 20x compared to the standard Stable Diffusion while better following appearance-altering prompts. This is important because computational overhead is the primary bottleneck preventing the deployment of generative data condensation in practical federated workflows.
Critical weaknesses
  1. The proposed client architecture is a direct combination of existing pre-trained models, without any federated-learning-specific algorithmic improvements or actual LoRA fine-tuning. Also, deploying a heavy VLM/diffusion pipeline on "resource-constrained devices" to train a 2 MB ConvNet is a significant contradiction of the motivation and challenges the authors outlined. In addition, the high Top-1 Re-ID rates (0.60–0.87) demonstrate that the vast majority of synthetic samples can be directly linked to their corresponding training images. The authors neither advance the algorithmic perspective of the synthetic data-based OSFL nor address its core motivating claims of edge feasibility and privacy preservation.
  2. The experimental evaluation does not include similar diffusion-based OSFL baselines that the authors cited (FedDEO 2024, FedBIP 2025) and fails to disclose details of the data-sharing ratio selection algorithm. In addition, Theorem 1 in Appendix A relies on a strong-convexity assumption which generally does not hold for deep neural networks.
Questions for authors

How exactly do clients select the subset of synthetic images when the sharing ratio is λ<1.0? Is it uniform random subsampling, or a metric-based filtering? If it is random sampling, how would the authors justify the claim that it reflects the "Pareto principle of highly informative samples"? What is the performance of FedDEO and FedBiP evaluated with the same experiment settings (e.g., datasets and client partitions, computational constraints, etc.)? What is the accuracy if clients generate images using only the text prompt “{classname}” with γ=0 (no local image conditioning)? Stable Diffusion runs natively at 512×512. Were CIFAR/SVHN (32×32) images upsampled during actual generation, or passed directly at 32×32 as claimed in the 12.52 GFLOPs calculation? Also, why was ResNet-18 trained for only 1 epoch on the server, and were the baselines in Table 1 trained with 1 epoch as well? Section 3.2 implies LoRA is used to fine-tune the model on clients, whereas Appendix B.2 states the entire pipeline is frozen. Was any local parameter updated/fine-tuned on the clients’ devices? Is there any improvement made to the IP-Adaptor, LCM-LoRA, or Stable Diffusion in the proposed work compared to their original papers? Can the authors provide a better proof of convergence without relying on unrealistic assumptions? Can the authors quantify the extent to which natural-language prompts contribute to the global classifier’s performance? For instance, how would the global classifier’s performance change w.r.t. a Classname-only prompt, a same-content prompt, clients’ heterogeneous prompts (different wording, different ways or language style and tones to express the user requirements), and the IP-Adapter conditioning scale (Future Work) The authors can consider integrating formal Differential Privacy into the latent-consistency generation step or offloading the heavy generative decoder to the central server to potentially support resource-constrained IoT/edge devices and mitigate the 60%–87% Re-ID privacy risk.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.