acceptodds
Withdrawn from ICLR 2027

Federated Timeline Synthesis: An Empirical Study of Generative Knowledge Transfer

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Synthetic patient timelines offer a means of transferring learned information between electronic health record models without pooling raw longitudinal records. We study Federated Timeline Synthesis (FTS), a one-shot protocol in which clients share locally trained autoregressive generators and a server trains a global generator on their synthetic timelines. Building on ETHOS, we evaluate eleven training configurations across nine structured EHR prediction endpoints using three disjoint, harmonized subsets of MIMIC-IV. The experiments distinguish synthetic-only transfer, multi-source pooling, and augmentation of smaller real-data sources. In the smallest-source setting, augmentation with timelines from a larger-source generator increases DRG accuracy from 0.364 to 0.717 and hospital mortality AUROC from 0.835 to 0.893, with higher point estimates on all nine endpoints. In the larger recipient setting, gains are more task dependent, while adding synthetic sources does not consistently improve performance. These results support synthetic timelines as a mechanism for transferring predictive information between generative EHR models, while showing that the benefit depends strongly on the recipient and training configuration. We publish code: https://anonymous.4open.science/r/fts.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
knowledge transfer, federated learning, synthetic data, electronic health records, foundation models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kWfaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors attempts to address the problem of transferring knowledge between EHR models across different institutions under no raw patient records scenario. The method involves training each client's timeline generator separately and uploads it once. The server then samples synthetic timeline to train a global model. The authors conclude that synthetic timelines can greatly improve a small recipients's model but simply pooling doesn't reliably help. However, this work provides only limited comparisons agains its own configurations.

Critical strengths
  1. This paper identifies a practical and important question: how to transfer clinical knowledge between institutions without pooling records
  2. The tasks span hospital, ICD and ED care, which is comprehensive. The representation and scoring rules are clear as well.
Critical weaknesses
  1. There are many possible baselines that the paper did not compare against: DENSE, FedGen, FedAvg... In addition, a simply fine-tune on the shared donor model using recipients' data is good to compare against.
  2. The experiment is not really across institutions. The authors describe that three 'clients' are random patient shards of one institution. Moreover, the tokenizer and vocabulary are shared. Thus, it introduces leakage across clients and removes the real problem to solve. (which is different subpopulation and clinical practices)
  3. The evaluation is far from sufficient. The model is a small 3-layer GPT-2 and every datapoint is generated from one dataset. The checks are not sufficient as well, fidelity checks are only code frequency diagnostics. The authors should add temporal checks in addition.
Questions for authors
  1. How does FTS compare with fine-tuning the shared big generator directly on the recipient’s data?
  2. How would the results change with different sites, e.g using MIMIC-IV and eICU
  3. Why does adding the little source’s synthetic data hurt every task? There seems to be a lack of explanation for it.
Reviewer_AroEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Federated Timeline Synthesis (FTS), a method for transferring predictive information between generative electronic health record (EHR) models through synthetic patient timelines. Clients independently train autoregressive timeline generators and upload them together with the metadata required for synthesis to a server. These generators can generate synthetic timelines for training a global model or augmenting a recipient's real data. Using subsets of MIMIC-IV, the paper evaluates eleven training configurations across nine clinical prediction endpoints. The results show that synthetic-only training preserves useful predictive information but does not consistently match real-data training, while augmenting the smallest recipient with timelines generated by a larger source improves all nine endpoint point estimates. In contrast, pooling additional synthetic sources does not consistently improve performance.

Critical strengths

The paper proposes using synthetic patient timelines as an intermediate medium for transferring predictive information between generative EHR models, distinguishing synthetic-only transfer, recipient augmentation, and multi-source pooling. This provides a useful empirical framework for federated EHR learning.

Critical weaknesses
  1. The proposed method does not include clinical constraints so that the generation quality. Autoregressive token generation and structural formatting do not guarantee clinically plausible timelines, such as valid event ordering or consistency between diagnoses, treatments, and measurements.

  2. The proposed method has limited conceptual and methodological contribution. FTS largely applies an existing autoregressive EHR model within a synthesis-and-retraining protocol. Its main findings are useful but qualitatively unsurprising. The paper documents these limitations without substantially explaining their causes or introducing mechanisms to address them. These limit the methodological novelty and depth of insight.

Questions for authors
  1. Can the authors report systematic assessments of clinical validity of the generated sequences? What mechanisms enforce clinically meaningful constraints during generation, and how does the method protect against disclosure of patient information through uploaded generators, conditioning metadata, or generated timelines?

  2. Do the recipient-augmentation gains and three-source pooling degradation persist when training-token budgets and validation procedures are matched? Are these results consistent across independent training and synthesis runs?

Reviewer_53bnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Federated Timeline Synthesis (FTS), a one-shot framework for transferring predictive knowledge across healthcare institutions through synthetic electronic health record (EHR) timelines, without directly sharing raw patient records. Building on ETHOS, the authors evaluate 11 training configurations across nine clinical prediction tasks using three disjoint subsets of MIMIC-IV.

Critical strengths
  1. Communication Efficiency: The one-shot framework reduces communication overhead and avoids directly sharing raw patient data.
  2. Broad Empirical Evaluation: The paper evaluates 11 training configurations across nine clinical prediction tasks, providing a comprehensive assessment of synthetic data utility.
  3. Practical Insights: Results show that synthetic data can substantially improve prediction performance for data-scarce institutions.
Critical weaknesses
  1. Lack of Privacy and Security Validation: The paper does not conduct privacy attack tests (e.g., membership inference attacks) on the uploaded generator models, making it impossible to verify their security against malicious reverse engineering. Moreover, no formal mathematical privacy guarantees are provided.
  2. Absence of Realistic Heterogeneity Simulation: The experiments merely partition the dataset by data volume, failing to simulate the complex data distribution discrepancies (Non-IID) across different hospitals in real-world settings.
  3. Insufficient Validation of Clinical Logic and Temporal Consistency: The evaluation only verifies the similarity of statistical distributions, lacking in-depth validation of whether the generated timelines conform to medical common sense.
  4. Insufficient Discussion of Server-Side Computational Bottleneck and Scalability: The massive autoregressive generation task is centralized on the server side, yet the main text lacks an intuitive trade-off analysis between communication cost and computational cost compared to traditional federated learning, and does not adequately discuss the computational bottlenecks in practical deployment.
Questions for authors
  1. Beyond AUROC, did the authors evaluate the Expected Calibration Error (ECE) for the sampling-based probability estimates (Eq. 10)? Furthermore, how do sampling variance and inference temperature impact the reliability of the model's confidence in high-stakes clinical decisions?
  2. The results show that adding little synth to the multi-source pool decreases performance across all nine endpoints. What is the root cause of this degradation: the inherent noise of synthetic data from small sources, or optimization conflicts when mixing different distributions?
  3. Given that autoregressive models are prone to exposure bias, how does the length of generated timelines affect downstream task performance? Specifically, did the authors analyze whether the sliding continuation mechanism effectively mitigates error amplification in long-term patient trajectories?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.