EMR-JEPA: Future-Focused and Policy-Responsive Learning for Clinical Forecasting

Clinical monitoring requires predicting a patient's future from sparse, irregular measurements collected under hospital-specific acquisition policies. We introduce EMR-JEPA, which learns patient-state representations by predicting future latent states rather than raw measurements. We first examine the masking policy: progressively shifting a matched masking budget from the past to the future increases mean out-of-domain frozen-probe AUROC from 0.6273 to 0.6502 across six-hour AKI and sepsis prediction on MIMIC-III, HiRID, and eICU. We then study the representation on which prediction operates. Fixed-count patching groups 16 consecutive measurements into adaptive-duration patient states, preserving finer resolution in densely observed periods. Because these states are defined by measurement count, their number and timing within the forecast horizon are unknown at prediction time and reflect the hospital's acquisition pattern; EMR-JEPA therefore predicts both alongside the states' latent content. Fixed-count patching alone matches fixed-duration patching in mean out-of-domain AUROC (0.650 for both), whereas adding count and time prediction raises it to 0.681 with one weighting shared across hospitals and tasks, and produces the clearest separation between local and long-range attention heads. Fine-tuned on a held-out hospital, EMR-JEPA attains the best average rank among self-supervised baselines for both sepsis and AKI, with the largest margin when target labels are scarce.