acceptodds
Under review as a conference paper at ICLR 2027

Knowing When to Adopt the Shared Knowledge: Benefit-Calibrated Federated LoRA under Partial Participation

Abstract

Federated low-rank adaptation (FedLoRA) enables communication-efficient fine-tuning without centralizing client data. However, under partial participation, heterogeneous data, and resource constraints, federated collaboration may not benefit all clients simultaneously. Moreover, aggregation may conceal negative transfer, while short-term local signals do not align with long-term benefits reliably. We therefore revisit FedLoRA from a client-benefit perspective and investigate when knowledge should be shared, personalized, or deferred. We define federation benefit relative to local proximal training and propose a benefit-calibrated framework that accumulates evidence across communication rounds and make the decision by Bayesian method. In particular, historical participation is tracked to prevent persistent client under-representation, while each client’s estimated benefit, associated uncertainty, and resource budget jointly determine whether collaboration follows a shared, personalized, or deferred decision. The analysis across multiple datasets shows that the benefits of federation vary substantially across client, motivating client-level evaluation and evidence-based collaboration. Experimental results demonstrate that our method achieves stronger and more consistent performance than existing baselines in most of scenarios.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Low-Rank Adaptation, Personalized Federated Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WdRWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies federated LoRA fine-tuning with partial participation, non-IID data and per-client rank budgets. Its method, BC-FedLoRA, has each selected client train two branches. The federated branch starts from the global adapter, and the proximal branch continues the client's own adapter with a penalty toward the global adapter. Their validation-accuracy gap is treated as a Gaussian observation of the client's benefit. A deployment rule thresholds the posterior at 0.85 to deploy the federated branch (share), the proximal branch (personalize) or the previous model (defer). Clients are selected by fixed priorities: capacity coverage, then participation counts, then label divergence. Experiments use Swin-T on CIFAR-100 and Tiny-ImageNet and DistilBERT on AG News and IMDB, with 12 clients, 5 seeds and seven baselines, including Ditto-LoRA. It reports the best mean client accuracy in 14 of 16 settings.

Critical strengths

None

Critical weaknesses
  1. Limited novelty. The method is a published algorithm plus a deployment rule, and the paper's ablation finds no measurable effect from it. Specifically, the two branches and the upload are Ditto's. Eqs. 6–7 and the upload of every federated branch are Ditto's Algorithm 2 (Li et al., 2021). The paper cites Ditto only for the proximal term (l.245) and presents the paired branches as its own (l.229–230). MaxFL (Cho et al., TMLR 2024), not cited, already has each client pick a global-derived or a local model on held-out data.

  2. The deployment rule is a fixed threshold. With the paper's prior and noise settings (μ0=0\mu_0 = 0, τ0=σb=1\tau_0 = \sigma_b = 1), Eqs. 13–17 reduce to one test: a client shares when ∑b^≥1.041+n\sum \hat b \ge 1.04\sqrt{1+n} and personalizes when ∑b^≤−1.041+n\sum \hat b \le -1.04\sqrt{1+n}. Here b^\hat b is an observed benefit and nn the number of rounds the client has joined. The posterior variance depends only on nn, so the reported uncertainty ignores how much the observations vary.

Furthermore, replacing the rule changes accuracy by less than the variation across seeds. In Table 4, picking the branch at random costs 0.13–0.18 pp, and using only the latest observation costs 0.05–0.21. BC-FedLoRA's standard deviation over seeds is 0.45–1.81 (Tables 1–2).

  1. The paper states no objective beyond the aim at l.176. "Long-term benefit" (Abstract; l.74, 134, 156) is never defined or measured. The method joins three separate problems: personalization (Eq. 17), client selection (Eqs. 2–5) and LoRA with a different rank per client (Eq. 18). The abstract says the decision controls what is shared and uses the resource budget (l.17–24, 87–89). In the method, every selected client uploads (l.311–312), and Eq. 17 has no resource term.

  2. The metrics and baselines do not fit the method. The method measures benefit against the proximal branch (Eq. 8), but the negative-transfer rate (NTR) is measured against Local-only training. The paper itself says this reference drifts on small, skewed data. The four non-personalized aggregation baselines fall points below Local-only on the most skewed split (Table 1). FedAvg-LoRA with local fine-tuning, the method's own federated branch, is missing.

  3. The paper is poorly presented and extremely difficult to read.

Questions for authors
  1. Apart from the deployment rule, does any step differ from Ditto's Algorithm 2 run on LoRA?

  2. Please report Table 4 with standard deviations over five seeds, and paired tests of the rule against random and latest-observation deployment.

  3. Is b^\hat b a fraction or in percentage points, and how often is a defer decision caused by an exact tie? In percentage points, one disagreeing sample among about 38 validation samples crosses the threshold; as a fraction, a client needs a mean benefit above 0.30 after 13 observations.

  4. Please add FedAvg-LoRA with local fine-tuning, and Ditto deploying the better of its two models on validation, with paired per-seed results to show the efficacy of the method.

  5. What objective does the deployment rule optimize? Please also define NTR, and say why its reference is Local-only while the method's benefit uses the proximal branch.

  6. Which model is evaluated for each baseline, and is the classification head aggregated? FedFomo-LoRA stays very close to Local-only in all vision settings: does its weighting collapse onto each client's own model?

Reviewer_hutYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To address the issue of inconsistent client gains in Federated LORA, this paper proposes the BC-FedLoRA. This approach evaluates individual federated gains through history-aware client selection and dual-branch training; it employs a Bayesian model to accumulate observations over multiple rounds and quantify uncertainty, thereby dynamically determining deployment strategies while ensuring that clients continue to participate in global aggregation. Experimental results demonstrate the effectiveness of this method across most settings.

Critical strengths
  1. The paper is written with clarity.

The structure is well-organized, and the methodology is presented clearly, effectively illustrating the relationships between various modules and making the content easy to follow.

  1. The experimental evaluation is comprehensive.

The authors address both visual and textual tasks, account for heterogeneity in data and resources, and validate the effectiveness of their approach through comparisons with multiple baselines.

  1. The three collaborative decision-making designs(SHARE, PERSONALIZE, and DEFER)are highly insightful.

Based on a client’s historical performance and the associated uncertainty, the framework selects between sharing, personalization, and deferral. The DEFER option allows clients to retain their previously deployed models when the posterior evidence is insufficient to favor either shared or personalized adaptation. This three-way decision mechanism provides flexibility for client-level adaptation in heterogeneous federated learning environments.

Critical weaknesses
  1. The modeling assumption of the latent federation benefit (\beta_i) is insufficiently validated.

In Equations (10)–(12), the paper assumes that each client has a latent federation benefit (\beta_i) shared across communication rounds, and continuously updates its posterior distribution using historical observations. However, as the global and personalized models are continuously updated, the actual federation benefit of a client may change systematically over time. The paper lacks an analysis of the validity of this assumption. Therefore, its applicability in dynamic federated training environments remains to be verified.

  1. Insufficient experimental validation of Bayesian historical benefit estimation.

In the ablation study, the full method improves the mean accuracy by only 0.05–0.21% compared to LATEST, which uses only the most recent benefit observation. These improvements are even smaller than the standard deviations across random seeds reported in Tables 1 and 2, making me question the stability of these performance gains. Therefore,, I am curious about the static benefit modeling assumption discussed in Weakness 1: when client benefits change during training, does accumulating all historical observations always outperform the LATEST strategy?

Questions for authors
  1. In Equations (10)–(12), the latent federation benefit βi\beta_i is assumed to be shared across communication rounds. Have the authors examined whether client benefits remain stable throughout training? Could the authors provide evidence to validate this assumption, particularly as the global and personalized models continuously evolve?

  2. In the ablation study, the improvements over LATEST are relatively small (0.05–0.21%), even smaller than the standard deviations reported in Tables 1 and 2. Could the authors provide statistical significance tests to demonstrate the reliability of these improvements? Furthermore, when client benefits change over time, can the proposed Bayesian historical benefit estimation still consistently outperform LATEST?

Reviewer_2FJiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BC-FedLoRA, a method for federated LoRA with partial client participation. It handles variations in client data and resources to help minimize negative transfer for individual clients. The approach consists of four main components: history-aware client selection, paired federated and local proximal adapters, a Gaussian posterior, and a three-option deployment rule. The authors test it on CIFAR100, Tiny-ImageNet-200, AG News, and IMDB, comparing it against local training and other federated or personalized baselines.

Critical strengths
  1. Instead of just looking at the global average performance, the paper looks at whether sharing actually helps or hurts individual clients during partial participation. This is a very practical angle for both personalization and fairness in federated learning.

  2. The method offers a cohesive, end-to-end pipeline. The combination of history-aware client selection, paired adapter branches, posterior evidence tracking, and client-specific deployment rules flows together really well.

Critical weaknesses
  1. Since every selected client trains both a federated branch and a local-proximal branch, they end up doing a second local optimization step every time they join the training. The paper brings up LoRA to highlight efficiency, but while Section 4.1 matches rounds and rank budgets, it doesn't actually report client FLOPs, real-world run time, memory usage, or how much data is transmitted.

  2. The approach assumes that repeated validation gaps act as noisy measurements of a fixed benefit for each client. This is hard to evaluate, especially since the global adapter keeps changing across rounds and clients are sharing a small validation set. Because the paper doesn't show calibration or test if this actually predicts future benefits on held-out data, it's hard to tell if the posterior threshold is a reliable way to make decisions.

  3. Under the DEFER rule, a client keeps its current model if neither of the two new candidates passes the posterior threshold. The issue here is that the rule compares the two new candidates to each other, but not to the model the client is already using. Because of this, a client might end up deferring an update even if one of the new candidates is actually better than what it currently has.

  4. The ablation studies touch on the deployment strategy, but they only cover α=0.1 and just show mean changes without giving the paired uncertainty across different random seeds. Because of this, it's still pretty unclear how big or consistent the posterior policy's actual contribution is.

Questions for authors
  1. For the held-out participation rounds, can the posterior probabilities actually predict future benefits for the clients? Please provide a calibration measure and show the prediction performance across clients and random seeds.

  2. How would the action rates and model performance change if you used a discounted or sliding-window posterior? Would this help it adapt better as benefits shift over time?

  3. When a client triggers the DEFER rule, how often is its current model actually better (or worse) than the two new candidates? Please share the resulting accuracy drop or regret compared to just picking the better candidate.

  4. Could you provide paired per-seed results for the ablation studies? Also, please clarify exactly which variants keep the selection, branch training, and uploads fixed.

  5. How do the client-side FLOPs, peak memory usage, real wall-clock time, and transmitted bytes compare to the baseline methods? Please also compare the accuracy using a matched compute budget.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.