acceptodds
Under review as a conference paper at ICLR 2027

FedGAug: Diagnosing Late-Onset Instability in Graph-Augmented Federated Clinical Prediction

Abstract

Federated learning (FL) lets hospitals train clinical prediction models together without pooling their electronic health records (EHRs). When patient populations differ between sites, however, the aggregated global model can break down after many communication rounds. We present FedGAug, a federated clinical prediction system that pairs a parameter-efficient transformer with a per-patient knowledge graph encoder, where each local graph is enriched with entities retrieved from external biomedical sources. We train it on three non-IID clients with three random seeds each, using asynchronous, staleness-weighted FedAvg. Without regularization, the global model improves steadily to a mean AUROC of at round 12 and shows no forgetting up to that point. It then collapses to by round 15 and stays there until round 20. Evaluating the local and global models separately shows that the local models hold up much better (AUROC around against for the global model during the collapse), which points to aggregation rather than local overfitting as the source of the failure. We then test Elastic Weight Consolidation (EWC) and Knowledge Distillation (KD) as remedies. None of the regularized variants collapses. Over 29 rounds their peak-to-final forgetting stays at or below , compared with for the unregularized model, and EWC with still reaches a mean AUROC of at round 21, when the unregularized model is at . Our results show that late-onset degradation in non-IID federated clinical prediction can originate at the aggregation step, and that continual-learning regularization prevents it over long training runs.

Primary area:
optimization
Keywords:
Federated learning, continual learning regularization, knowledge graphs, clinical prediction

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_TGthOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates late-onset performance degradation in graph-augmented federated clinical prediction under non-IID data distributions. The authors propose FedGAug, which integrates a LoRA-adapted transformer, patient-level knowledge graph representations, and asynchronous federated aggregation, and evaluates Elastic Weight Consolidation (EWC) and Knowledge Distillation (KD) as stabilization strategies. They evaluate the approach on synthetic EHR data from Synthea, partitioned across three heterogeneous clients over 29 communication rounds, and report that the unregularized global model experiences a late AUROC collapse while the regularized variants maintain stable performance. The study contributes empirical observations on late-onset instability, differences between local and global model performance, and the behavior of regularization and monitoring metrics during federated training.

Critical strengths
  1. The paper investigates late-onset instability that would be missed by shorter federated training evaluations, highlighting the importance of monitoring model performance over extended communication rounds.
Critical weaknesses
  1. The experiments lack comparisons with established federated optimization methods and simpler stabilization strategies, making it difficult to establish whether the adopted regularization choices are necessary.
  2. The evaluation relies on a single synthetic clinical dataset with three clients and one non-IID partitioning configuration, leaving the robustness of the findings across data distributions and federated settings unverified.
Questions for authors

see weaknesses

2 comments

We thank the reviewer for the constructive feedback. We are adding matched FedProx, FedCurv and SCAFFOLD runs and simpler stabilizers under the same protocol, and extending the study to more clients and different heterogeneity levels. We will share results as they become available.

Thank you for the response. Since the additional results are not yet available, my concerns remain, and I will keep my rating.

Reviewer_n8XHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies why a federated clinical mortality predictor can fail late in training. The system is a frozen DistilBERT with LoRA adapters over the patient’s code sequence. This is combined with a GNN over a per-patient knowledge graph that is enriched with entities retrieved from an external biomedical source.

Critical strengths

The paper tracks the global and local models round by round for 29 rounds, reports the peak at round 12, the drop to 0.766 and 0.509, the plateau at 0.370, and the partial recovery. It also states its limits (n = 3, inner products not measured, no claim of regularizer ranking). This matters because it gives the community a concrete, reproducible-looking phenomenon, and the transparency makes the evidence easy to scrutinize.

Critical weaknesses
  • The local-vs-global comparison is confounded. Each client starts local training from the issued global model and trains for 3 epochs. A local model that starts from a degraded global model and fine-tunes on its own data will naturally score higher than that global model, and this is also true in a pure local-overfitting scenario.
  • The paper does not measure update inner products, update norms, or staleness, and it admits it did not verify the opposing-direction mechanism. Yet the abstract and conclusion state the origin as aggregation.
  • There is no baselines.
Questions for authors
  • Does the collapse occur under synchronous FedAvg with the same architecture and data? Does it still occur with a staleness cap or gradient clipping?

  • For the local-vs-global comparison: are local models always initialized from the current global anchor? On which data are local and global models evaluated, and is it disjoint from every client’s training data and from the data used for the Fisher estimate and for choosing peak rounds?

  • What is the unit of replication? Are the three clients three separate federations or three evaluation sets of the same federation?

  • Ablate the knowledge graph and retrieval components.

1 comment

We thank the reviewer for the helpful review. Since local models start from the issued global model, the local and global comparison does not, by itself rule out local overfitting, and we will discuss this in the revision.

We are running synchronous FedAvg, a staleness cap and gradient clipping to test whether the collapse persists. Local models are always initialized from the current global model. We will move all evaluation to a separate test set, disjoint from the training data, the Fisher data and the data used to select peak rounds. There is one federation with three clients, each seed is an independent run of it, and we will add variance across seeds. We are also ablating the knowledge graph and retrieval components.

Reviewer_afivOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies why a federated clinical prediction model can suddenly lose performance during training. Using synthetic patient data from three clients, the authors observe that the global model performs worse than local models after aggregation. They test EWC and knowledge distillation to reduce this instability and report more stable performance, although peak accuracy is lower.

Critical strengths

The paper provides a useful diagnostic perspective by tracking local and global model performance separately to reveal late-stage instability in federated training.

Critical weaknesses
  • Limited experimental scope and inconsistencies in the reported results. The experimental evaluation is insufficient to support the breadth of the paper’s conclusions. All experiments are restricted to a single synthetic dataset, one prediction task, and three clients, without examining different client counts, heterogeneity levels, or asynchronous scheduling conditions. Consequently, it remains unclear whether the observed collapse generalizes beyond this particular configuration. The absence of direct comparisons with established drift-mitigation methods under the same protocol further limits the assessment. More concerningly, the central figures and tables contain inconsistencies. Section 4.3 and Table 6 specify 29 communication rounds, whereas Figure 1 appears to include a round-30 point at which the unregularized AUROC abruptly returns to approximately 0.90. Table 2 reports final forgetting of 0.169±0.0960.169 \pm 0.096, but the corresponding final-window bar in Figure 2 appears close to 0.02 and is labeled “1–30.”

  • Insufficiently justified motivation and an incomplete mechanistic explanation. The divergence between local and global performance in Figure 3 is informative, but it does not establish conflicting client updates as the underlying cause. Although Section 5.2 acknowledges that update inner products were not measured, the abstract and conclusion make stronger claims about the source of the failure than the evidence supports. Furthermore, the accompanying derivation omits a term that is directly relevant to asynchronous aggregation. Given the paper’s definition Δk=θklocal−θ(tissued,k)\Delta_k=\theta_k^{\mathrm{local}}-\theta^{(t_{\mathrm{issued},k})}, Equation (8) implies $\theta^{(t+1)}-\theta^{(t)}= \gamma_0s(\tau)\Delta_k + \gamma_0s(\tau) \left(\theta^{(t_{\mathrm{issued},k})}-\theta^{(t)}\right).$ Section 5.2 omits the second term when characterizing the displacement induced by a merge. This term captures the discrepancy between the model issued to the client and the current server model, and therefore cannot generally be neglected in the asynchronous setting.

  • The manuscript falls below the standard of clarity, precision, and consistency expected of an academic paper. The presentation requires substantial editing to communicate the methods and arguments clearly.

Questions for authors
  • Could the authors explain how the omitted staleness term affects their analysis and provide evidence that conflicting client updates cause the observed collapse?
  • Please see the detailed questions and concerns in the weaknesses section.
1 comment

We thank the reviewer for the careful reading. We will report the round count consistently (30 rounds, numbered 0 to 29) and align the final values and forgetting values in the figures and tables.

We will include the staleness term in the revised derivation. Our logs also show that the smallest client contributes most server steps under merging on arrival, so we are running synchronous and staleness capped controls and measuring update cosine similarities to examine this. We will present aggregation as a hypothesis, add matched baselines and broader settings, and revise the manuscript for clarity.

Reviewer_dWYDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates late-onset performance degradation in graph-augmented federated clinical prediction. FedGAug combines a LoRA-adapted DistilBERT encoder with patient-level knowledge graphs enriched through external biomedical retrieval, and trains across three non-IID clients using staleness-weighted asynchronous aggregation. On synthetic Synthea mortality data, the unregularized model reaches an AUROC of 0.907 at round 12 but drops to 0.370 by round 15. The authors compare local and global model performance to investigate the failure and report that EWC and EWC combined with knowledge distillation avoid a comparable collapse within 29 rounds, albeit with lower peak AUROC.

Critical strengths

A useful focus on failures missed by short federated training runs. Rather than reporting only final performance, the paper tracks global AUROC and AUPRC across 29 rounds and identifies a pronounced deterioration after round 12. The comparison with shorter evaluation windows illustrates why late-stage monitoring matters.

An informative local-versus-global comparison and stability trade-off. Figure 3 distinguishes post-training local models from post-aggregation global models, while Tables 2 and 3 report both stability and final predictive performance. The regularized variants avoid the observed collapse, but the paper also acknowledges their lower peak AUROC and the limited evidence for an additional benefit from KD.

Critical weaknesses

The proposed aggregation-interference diagnosis is not established, and the update decomposition omits staleness. Section 5.2 interprets the server update as approximately γ0s(τ)Δk\gamma_0s(\tau)\Delta_k, where Δk=θklocal−θ(tissued,k)\Delta_k=\theta_k^{\mathrm{local}}-\theta^{(t_{\mathrm{issued},k})}. However, Equation (8) gives the exact change γ0s(τ)[Δk+θ(tissued,k)−θ(t)]\gamma_0s(\tau)[\Delta_k+\theta^{(t_{\mathrm{issued},k})}-\theta^{(t)}]. The omitted term matters precisely in asynchronous training. Moreover, update conflicts are not measured, and the local-global gap alone cannot separate non-IID interference from staleness or aggregation of LoRA factors. The causal interpretation therefore remains a hypothesis rather than a demonstrated mechanism.

The EWC importance estimate in Equation (5) is not the standard Bernoulli Fisher information. Writing p=σ(zθ(x))p=\sigma(z_\theta(x)), the proposed squared positive-class score is (1−p)2(∂z/∂θi)2(1-p)^2(\partial z/\partial\theta_i)^2, whereas the model Fisher is p(1−p)(∂z/∂θi)2p(1-p)(\partial z/\partial\theta_i)^2. For p=0.1p=0.1, the former is nine times larger. This discrepancy is particularly relevant to the imbalanced mortality task and undermines the interpretation of the penalty as Fisher-weighted curvature. A comparison with the correct Fisher estimate and an unweighted proximal penalty is needed.

The experimental evidence is too narrow to attribute the effect or establish generality. All results come from one synthetic dataset, one prediction task, and three clients; Clients 1 and 2 have nearly identical mortality prevalences (12.6% and 12.8%). There are no matched runs of FedProx, FedCurv, SCAFFOLD, or synchronous/size-weighted aggregation, nor ablations of the graph retrieval and LoRA components. Appendix Table 5 quotes baseline results from unrelated experiments rather than evaluating them under the same protocol. Furthermore, Equation (1) specifies a sample-size-weighted objective, while Equation (8) uses arrival- and staleness-dependent merging without sample-size weights, complicating interpretation of the small client's influence.

Questions for authors

Can the aggregation hypothesis be tested directly? Please report per-client update counts and staleness distributions, pairwise update cosine similarities around rounds 12-20, and matched controls using synchronous aggregation, sample-size weighting, and aggregation of the effective LoRA weight updates rather than their factors.

How are the data splits and model evaluations implemented? Is the validation set DkvalD_k^{\mathrm{val}} used to estimate EWC importance in Equation (5) also used for the reported AUROC curves? Are the three seeds independent federated runs or separate client-level randomizations? Please also specify which clinical codes are available at the mortality-prediction time point to rule out outcome leakage.

Is curvature weighting necessary for the reported stability? Would a correct label-free Bernoulli Fisher estimate, an unweighted proximal penalty such as FedProx, or KD alone prevent the same collapse under identical settings?

1 comment

We thank the reviewer for the detailed review. We will revise the Fisher estimate and the staleness term as described in our general comment.

Following the reviewer's suggestion, we examined our logs and found that Client 3 accounts for most server steps. Since updates are merged on arrival without size weights, the smallest and fastest client has the largest influence on the global model, which may also explain why the global model recovers only on Client 3. Synchronous FedAvg will help separate this effect from conflicts between client updates, and we will report merge counts, staleness distributions and update cosine similarities.

Each seed is an independent run of the whole federation. The validation set is currently used for both the importance weights and the AUROC curves, and we will move evaluation to a separate test set. To assess the role of curvature weighting, we are running the standard Bernoulli Fisher, an unweighted proximal penalty, and KD alone under identical settings.

Comments on the paper

1 comment

We thank all reviewers for their thoughtful feedback. The reviews share three themes: the need for matched baselines, broader evaluation, and stronger support for our interpretation of the collapse. We are revising the paper accordingly.

As noted by Reviewer afiv, we will report the round count consistently (30 rounds, numbered 0 to 29) and align the final values in the figures and tables. Following Reviewer dWYD, we will use the standard Bernoulli Fisher in Eq. (5), include the staleness term in Section 5.2, and align the objective in Eq. (1) with the merge rule. We will also evaluate on a separate test set, use one name for the unregularized model, and present aggregation as a hypothesis rather than a confirmed cause.

We are running synchronous FedAvg, a staleness cap, matched FedProx, FedCurv and SCAFFOLD baselines, aggregation of the full LoRA update BABA, per client merge and staleness logs, update cosine similarities, component ablations, and settings with more clients and different heterogeneity levels. We will share results that complete during the discussion and include all of them in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.