acceptodds
Under review as a conference paper at ICLR 2027

Partial Participation: Guarantees of Local Step Convergence for Unified Client Selection in the Non-Convex Case

Abstract

Federated learning enables model training across distributed devices without direct data sharing, making it a natural framework for large-scale learning systems. Its scalability is nevertheless constrained by two fundamental bottlenecks: server-side overload caused by the impossibility of involving all devices in each communication round, and channel overload caused by transmitting high-dimensional models. These challenges are traditionally addressed separately: the former through partial participation training paradigm, and the latter through compression of the transmitted information. In heterogeneous settings, however, standard partial participation schemes may lead to biased optimization dynamics and are difficult to analyze rigorously. While partial participation and compression have been extensively studied in isolation, their joint treatment under heterogeneity remains largely unresolved. In this work, we address this gap by developing a novel federated optimization algorithm that operates under partial participation with compressed communication. We establish convergence guarantees under general non-convex assumptions and empirically evaluate the proposed method on image classification benchmarks.

Primary area:
optimization
Keywords:
Federated Learning, Partial Participation, Distributed Optimization, Client/Device Heterogeneity, Non-convex Optimization, Convergence Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_gDc6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses an important problem of optimization of distributed devices in federated learning, and proposes PP-EFLS that incorporates multiple local steps into federated learning optimization framework. The proposed method helps to solve the challenges in the joint treatment of partial participation schemes. The paper presents a nonconvex convergence bound and evaluates the method on CIFAR-10 with ResNet-18 under different levels of data heterogeneity, numbers of local steps, and client-selection strategies. The reported partial-participation experiments show accuracy improvements over FedAvg and FedProx.

Critical strengths
  1. A practically well-motivated problem is studied to investigate the joint treatment in partial participation schemes.
  2. The authors studied additional analytical challenges for introducing multiple local steps and provide theoretical bounds addressing this interaction between participation, heterogeneity and others.
  3. The reported accuracy gains over FedAvg and FedProx are encouraging.
Critical weaknesses
  1. Some parts in theoretical analysis are not straightforward or clear enough for me. Some are listed in the 'Questions For Authors' section below as examples.
  2. The range of baselines is limited. More comparisons with recent federated learning optimization method would significantly strengthen the paper, such as FAST [1] and FedAU [2].

Reference

[1] Zhe Li, Seyedsina Nabavirazavi, Bicheng Ying, Sitharama Iyengar, and Haibo Yang. FAST: A Lightweight Mechanism Unleashing Arbitrary Client Participation in Federated Learning. Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence (IJCAI), 2025.

[2] Shiqiang Wang and Mingyue Ji. A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging. International Conference on Learning Representations (ICLR), 2024.

Questions for authors
  1. The lower bound in inequality (22) appears without detailed justification. It seems that some special selection of parameters could lead to violation of the lower bound. Could you please provide detailed derivation of the bound?
  2. For controlling stochastic terms (on page 15-16), I think it requires a smoothness on each fmf_m. However the assumption only ask for a global smoothness for the objective ff. Is it guaranteed by derivation or are those objectives assumed to be the same function?
  3. The proposed algorithm extends PPBC with multiple local steps. Does LS1 in Table 1-6 exactly recover PPBC? If not, providing evaluation of PPBC and comparison will strengthen the contribution of the proposed algorithm.
  4. Since there are extra uploads for correction in your algorithm, is it fair to compare with baselines FedAvg and FedProx in your main results from Table 1 to 6 with the fixed 100 epochs?
Reviewer_EDTzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies federated learning under partial client participation and heterogeneous data distributions. It proposes Partial Participation with Error Feedback and Local Steps (PP-EFLS), which builds on the PPBC framework and incorporates multiple local SGD steps into a partial participation setting. The authors aim to establish convergence guarantees for non-convex objectives under assumptions on objective smoothness, data heterogeneity, and stochastic gradient variance. Experiments on CIFAR-10 with ResNet-18 evaluate the proposed method under different levels of data heterogeneity, local update steps, and client selection strategies. The results suggest that PP-EFLS can outperform selected baselines, including FedAvg and FedProx, in several partial participation settings.

Critical strengths

The paper addresses a relevant problem in federated learning and attempts to establish non-convex convergence guarantees for partial participation with multiple local updates. The theoretical analysis and experiments across different heterogeneity levels provide some value. However, the extent of the methodological novelty and the practical significance of the proposed approach remain unclear.

Critical weaknesses
  1. The novelty of this paper is unclear. The paper considers the combination of partial client participation and communication compression under heterogeneous data distributions, a setting for which there appears to be substantial related work. However, the authors do not sufficiently discuss or compare their approach with relevant existing studies, making it difficult to assess the originality of their contribution. Furthermore, the proposed method appears to build primarily on PPBC, yet the corresponding reference does not provide clear publication information. Based on my own search, the PPBC work appears to have been submitted to NeurIPS 2025 but was not accepted. The authors should clarify the publication status and provide a complete reference. More importantly, the proposed methodological contribution appears to consist mainly of incorporating multiple local update steps into the existing PPBC framework. The paper does not sufficiently explain what fundamental technical challenges this extension introduces or what novel algorithmic insights are developed to address them.

  2. The proposed method is named Partial Participation with Error Feedback and Local Steps (PP-EFLS), and the paper motivates its approach partly through the challenge of communication compression in federated learning. However, it is unclear how communication compression is actually implemented in the proposed algorithm. In particular, Algorithm 1 does not explicitly specify a compression operator, such as quantization or sparsification, or a corresponding error residual update that would establish the use of a conventional error feedback mechanism. Although the algorithm maintains and updates client-specific gradient surrogates, it is unclear whether these updates serve as error compensation for compressed communications or primarily address the challenges of partial client participation. The authors should explicitly identify the compression mechanism and explain where and how error feedback is incorporated into the algorithm. If communication compression is a core contribution, the paper should also evaluate its effectiveness in terms of compression ratio, communication payload, or transmitted bits. As currently presented, the relationship between the method's name, its claimed communication compression setting, and its actual algorithmic design remains unclear.

  3. After Assumption 4, the paper introduces a constraint on client weights but does not clarify whether it is a theoretical restriction or an algorithmic constraint, nor how it is justified or enforced. Additionally, (\alpha) is used both in this constraint and as the Dirichlet concentration parameter for data heterogeneity, creating ambiguity. The authors should clarify the constraint's role and use distinct notation for these two parameters.

  4. The experiments are limited to a single dataset (CIFAR-10) and model architecture (ResNet-18), making it difficult to assess the generalizability of the proposed method. Moreover, the baselines are limited and relatively outdated. In particular, the authors do not compare their method directly with PPBC, the framework on which their approach is based, making it difficult to evaluate the actual improvement introduced by PP-EFLS.

Questions for authors
  1. Has PPBC been formally published or accepted for publication? If so, where can the published version be found? More importantly, what are the key technical challenges in extending PPBC to multiple local update steps, and what novel insights does PP-EFLS introduce beyond this extension?

  2. Where exactly are communication compression and error feedback implemented in Algorithm 1? What compression operator and error compensation mechanism are used, and how do they reduce communication costs?

  3. What is the motivation for the client weight constraint introduced after Assumption 4? Is it a theoretical requirement or an algorithmic constraint, and how does the proposed method ensure that it is satisfied?

Reviewer_EhT7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends PPBC with multiple local SGD steps between communication rounds. It retains PPBC’s accumulated corrections for biased client selection, derives non-convex convergence guarantees, and evaluates the method on CIFAR-10 with ResNet-18 against FedAvg and FedProx.

Critical strengths
  1. Adding local steps to PPBC is a useful extension.
  2. The paper provides non-convex convergence guarantees and neural network experiments.
  3. The comparison against full-participation baselines matches uplink budgets, accounting for the extra correction uploads.
Critical weaknesses
  1. The method requires periodic all-client uploads, which has practical downsides by stalling training when devices remain offline. Furthermore, unselected but available clients still have to download models and compute gradients, consuming bandwidth and battery.
  2. Novelty beyond PPBC is insufficiently established, particularly relative to existing error-feedback methods with local steps.
  3. Experiments omit PPBC and SCAFFOLD, cover only CIFAR-10, and lack complete computation and communication accounting.
  4. The proof appears to require unstated client-level smoothness. Lemmas 5–6 use smoothness of each fmf_m, while Assumption 1 only states it for ff; local curvature may cancel in the average, suggesting a possible counterexample under the stated assumptions.
Questions for authors
  1. PPBC+ was evaluated on Food101 with FasterViT. Can the authors evaluate PP-EFLS in the same setting, including a direct comparison with PPBC+, to test whether the local-step extension helps beyond CIFAR-10?
  2. What is technically new beyond PPBC and existing error-feedback methods with local steps?
  3. How does it experimentally compare with SCAFFOLD under matched communication and computation budgets?
  4. Do the results require each client objective to be LL-smooth? If not, how are the smoothness steps in Lemmas 5–6 justified?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.