PORTER: Portable Representations for Transferable Electronic Health Records

Clinical prediction models often degrade across sites because differences in clinical practice and documentation induce site-specific joint distributions over observed records and outcomes. These shifts arise from the underlying data-generating process and therefore cannot be fully addressed by increasing downstream model capacity alone. We instead focus on constructing representations that support transferable prediction across sites. We propose Portable Representations for Transferable Electronic Health Records (PORTER), a summarize-then-embed pipeline built using a clinical schema designed to reduce site-specific variation, alongside a fine-tuning method to addresses variation unresolvable by schema alone. We design the schema to guide the LLM to preserve predictive information while ignoring site-specific noise. However, a site-independent representation alone cannot capture how predictions should be adapted to a new site. To address this limitation, we build on the site-invariant representation and further adapt its predictions by distilling target-site policies from a small support set. Across three ICU datasets, six transfer directions, and five clinical prediction tasks, schema-only PORTER achieves the best average rank among all 13 methods in zero-shot transfer. Its target-site distillation procedure also achieves the best average rank in the few-shot setting, with the largest gains on tasks that transfer most poorly under zero-shot setting. We also experiment with PORTER on longitudinal EHR data where we find PORTER improves with increasing context length. Our work demonstrates that LLMs can be used not only to encode clinical text, but to generate representations explicitly designed to improve cross-site clinical prediction.