acceptodds
Withdrawn from ICLR 2027

Beyond Parameter Consensus: Communication-Efficient Personalized Federated ADMM with One-Bit Consensus

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Federated learning enables collaborative model training without sharing raw data, but repeated exchanges of high-dimensional model parameters incur substantial communication costs. Under heterogeneous data distributions, reducing these costs while maintaining effective cross-client collaboration and client-specific adaptation remains challenging. To address this challenge, we propose Federated ADMM with One-bit Consensus (FICO), a communication-efficient personalized federated learning framework. FICO relaxes exact parameter consensus to one-bit consensus over low-dimensional model representations, preserving client-specific variations while coordinating clients through a shared one-bit code. We develop a proximal ADMM algorithm that alternates between personalized model optimization and shared consensus code updates. Each client transmits one-bit signs of local cost differences together with a single amplitude scalar, allowing the server to select the shared code using compact messages while full-precision model variables remain local. Theoretical analysis establishes convergence to stationarity under suitable regularity and error conditions. Experimental results demonstrate that FICO achieves a more favorable trade-off between communication cost and personalized model accuracy than the evaluated baselines.

Primary area:
optimization
Keywords:
Personalized Federated Learning, Federated ADMM, Communication Efficiency, One-Bit Consensus, Distributed Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_q5QDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes FICO, a personalized federated learning framework that replaces parameter consensus with one-bit consensus on low-dimensional model projections. Each client keeps a full-precision local model, and clients agree only on a shared sign code of a sketched representation (Eq. 4 to 10), enforced through a margin-based quantization cell. A proximal ADMM scheme alternates local SGD on personalized subproblems with server-side code selection, which reduces to a coordinate-wise closed form after eliminating the auxiliary variables (Lemma 1, Eq. 17 to 19). Clients upload one sign bit per coordinate plus one amplitude scalar encoding their cost-difference preferences (Eq. 20 to 24), giving a per-round cost of K(2m + 32) bits. The theory bounds the average expected squared KKT residual of the continuous variables within the selected cells by an O(1/T) term plus a stochastic-gradient error term (Theorem 1), with vanishing residuals under summable error budgets (Theorem 2). Experiments on five image datasets with 20 clients under Dirichlet heterogeneity report competitive accuracy against eight baselines at roughly 0.2% of the per-round communication of FedADMM-InSa.

Critical strengths
  1. The formulation is clean and the algorithmic derivation is elegant. Reducing shared-code selection to a coordinate-wise binary problem with an exact closed-form minimizer (Eq. 19), and connecting the compressed messages to actual optimization-cost differences rather than plain sign voting as in pFed1BS, is a genuine conceptual step for low-bit federated coordination.
  2. The theory accounts for the parts that usually get waved away. Theorem 1 covers inexact SGD subproblem solves and compressed uplink messages, and the Hamming-penalized code update with the tie tolerance (Eq. 24) is both practical and folded into the analysis. The 99.8% per-round communication reduction against FedADMM-InSa with accuracy ahead of the closest low-bit competitor pFed1BS by 11.98 and 13.13 points on CIFAR-10 and CIFAR-100 at alpha 0.1 (Table 1) demonstrates a useful operating point.
Critical weaknesses
  1. The accuracy-communication trade-off is not yet mapped against full-precision baselines. At alpha 0.5, FICO trails FedAPM on most columns of Table 2, including a 16-point gap on CIFAR-100 (43.93 vs 60.12). FedAPM pays orders of magnitude more communication, so this does not contradict the paper's claim, but a cost-matched comparison (equal total bit budget over training, or FedAPM restricted to the same number of rounds) would make the trade-off curve concrete. This seems fixable with additional experiments rather than a fundamental flaw.
  2. The evaluation scope could be widened. Accuracy is reported as the maximum over rounds rather than the final or converged value, and all experiments use small vision models, while the communication argument matters most for larger models. Reporting final-round accuracy and one larger-scale task would strengthen the evidence, though neither issue undermines the analysis that is present.
Questions for authors
  1. The accuracy-communication trade-off is not yet mapped against full-precision baselines. At alpha 0.5, FICO trails FedAPM on most columns of Table 2, including a 16-point gap on CIFAR-100 (43.93 vs 60.12). FedAPM pays orders of magnitude more communication, so this does not contradict the paper's claim, but a cost-matched comparison (equal total bit budget over training, or FedAPM restricted to the same number of rounds) would make the trade-off curve concrete. This seems fixable with additional experiments rather than a fundamental flaw.
  2. The evaluation scope could be widened. Accuracy is reported as the maximum over rounds rather than the final or converged value, and all experiments use small vision models, while the communication argument matters most for larger models. Reporting final-round accuracy and one larger-scale task would strengthen the evidence, though neither issue undermines the analysis that is present.
Reviewer_UPt2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FICO, a communication-efficient personalized federated learning framework. FICO coordinates clients through a shared one-bit code,uses approximate local cost differences to guide consensus selection, and avoids repeated exchanges of full-precision models to reduce communication costs in both directions. Experimental results show that FICO achieves a more favorable trade-off between communication cost and personalized model accuracy than the evaluated baselines.

Critical strengths
  1. Each client transmits one-bit signs of local cost differences together with a single amplitude scalar instead of repeatedly exchanging full-precision model parameters.
  2. FICO allows clients to maintain personalized models while coordinating clients through a shared one-bit code, providing flexibility to accommodate heterogeneous local data.
Critical weaknesses
  1. The paper emphasizes the reduction in per-round communication cost. However, a lower per-round communication cost does not necessarily translate into lower total communication cost when achieving comparable accuracy. If FICO requires more communication rounds to reach similar performance, its total communication advantage may be diminished.
  2. The experimental evaluation considers a limited range of heterogeneity settings. More comprehensive experiments under diverse heterogeneity scenarios would strengthen the evaluation of the proposed method.
Questions for authors
  1. A more comprehensive comparison with pFed1BS is needed to better demonstrate the advantages of the proposed method and clarify its contributions beyond existing approaches. 2.The paper reports substantial reductions in per-round communication costs. Could the authors compare the total communication cost required to reach the same target accuracy for FICO and the main baselines? This would help determine whether the communication advantage remains significant over the entire training process. 3.The performance of FICO may depend on hyperparameters such as the penalty parameter and the dimension of the low-dimensional model representation. Sensitivity analyses would help clarify the impact of hyperparameter choices on performance and justify the selected values.
  2. The paper lacks ablation studies to verify the effectiveness of the proposed components. Ablation experiments are needed to clarify the contribution of each component to the overall performance.
Reviewer_zcQDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FICO, a personalized federated learning method that replaces parameter consensus with shared sign constraints on low-dimensional model projections. Clients retain full-precision personalized models and coordinate through a shared binary code with a positive-margin constraint. A proximal ADMM algorithm alternates local model optimization, shared-code selection, auxiliary projection, and dual updates. Clients communicate signs of local candidate-code cost differences and one amplitude scalar. The server reconstructs approximate differences and updates the shared code. Theoretical results establish selected-cell stationarity bounds under stated assumptions. Experiments compare FICO with eight baselines on five datasets under two Dirichlet partition settings, reporting competitive accuracy and substantially reduced per-round communication.

Critical strengths
  1. The formulation provides an explicit alternative to parameter consensus. Shared quantization cells allow clients to retain different parameter values while coordinating through a compact representation. The relationship between cell selection and local optimization costs is clearly motivated.
Critical weaknesses
  1. Section 5.1.3 identifies the datasets, model families, training rounds, batch size, and local epochs. However, it does not report the actual sketch dimension m or the numerical values of FICO’s main parameters, including ρ, β, η, χ, τ, ϵtie, and the local learning rate. The missing m is particularly consequential because Equation (27) makes communication directly dependent on it. Without m and the corresponding model dimensions, Figure 2’s communication costs cannot be independently reconstructed.

  2. The algorithm projects the auxiliary variables sk into the selected cell at every round. This does not itself ensure that the actual model projections Hwk satisfy the same cell constraints, because Hwk = sk is enforced through an iterative ADMM procedure. The reported metrics are test accuracy and communication cost. There are no empirical measurements of cross-client sign agreement or the KKT residual introduced in Equation (30).

Questions for authors
  1. What values of m, ρ, β, η, χ, τ, ϵtie, and the local learning rate were used? What sketch dimensions and normalization were used for pFed1BS?

  2. Were primal residuals, sign agreement, margin violations, or KKT residuals recorded? Do the experimental settings satisfy the parameter conditions in Assumption A6?

  3. Were validation-selected or final-checkpoint accuracies recorded? At which rounds were the reported maxima achieved, and what cumulative communication was required?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.