acceptodds
Withdrawn from ICLR 2027

EMR-JEPA: Future-Focused and Policy-Responsive Learning for Clinical Forecasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Clinical monitoring requires predicting a patient's future from sparse, irregular measurements collected under hospital-specific acquisition policies. We introduce EMR-JEPA, which learns patient-state representations by predicting future latent states rather than raw measurements. We first examine the masking policy: progressively shifting a matched masking budget from the past to the future increases mean out-of-domain frozen-probe AUROC from 0.6273 to 0.6502 across six-hour AKI and sepsis prediction on MIMIC-III, HiRID, and eICU. We then study the representation on which prediction operates. Fixed-count patching groups 16 consecutive measurements into adaptive-duration patient states, preserving finer resolution in densely observed periods. Because these states are defined by measurement count, their number and timing within the forecast horizon are unknown at prediction time and reflect the hospital's acquisition pattern; EMR-JEPA therefore predicts both alongside the states' latent content. Fixed-count patching alone matches fixed-duration patching in mean out-of-domain AUROC (0.650 for both), whereas adding count and time prediction raises it to 0.681 with one weighting shared across hospitals and tasks, and produces the clearest separation between local and long-range attention heads. Fine-tuned on a held-out hospital, EMR-JEPA attains the best average rank among self-supervised baselines for both sepsis and AKI, with the largest margin when target labels are scarce.

Primary area:
learning on time series and dynamical systems
Keywords:
irregularly sampled time series, self-supervised learning, joint-embedding predictive architecture (JEPA), electronic health records, domain generalization

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_2L1QOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EMR-JEPA, a self-supervised learning method that adapts Joint-Embedding Predictive Architectures (JEPA) to clinical time-series data for clinical forecasting. Instead of reconstructing raw measurement values, the model predicts the latent representations of masked states generated by an exponential-moving-average (EMA) target encoder.

The methodology focuses on two main components. First, the authors explore an asymmetric masking policy. By keeping the total budget of masked tokens fixed, they progressively shift the masking probability from historical states to the future forecast horizon. They show empirically that masking 100% of future states and a smaller portion of past states reduces future latent prediction loss and improves downstream performance compared to standard symmetric masking. Second, to handle irregular clinical sampling, they introduce fixed-count patching, which groups every 16 consecutive measurements into one patient state regardless of the time duration. Because fixed-count states have variable durations, the number of states and their timestamps in the future six-hour window are unknown at inference time. To address this, the authors add a policy token and two auxiliary heads during pretraining to explicitly predict the count and normalized timestamps of future states.

The authors evaluate their approach across three ICU datasets: MIMIC-III, eICU, and HiRID. The experiments test transfer learning and out-of-domain generalization on two six-hour binary classification tasks: sepsis and acute kidney injury (AKI) prediction, comparing against clinical self-supervised baselines like STraTS, DuETT, and SMART.

Critical strengths

The paper adapts JEPA, a state-of-the-art self-supervised learning method, to ICU time-series datasets. The authors do a good amount of experimental work to test different asymmetric masking ratios and propose a fixed-count patching strategy to handle irregular clinical measurements.

Critical weaknesses
  • Limited technical contribution and overclaiming: The paper is mostly a direct application of JEPA to ICU datasets, which lacks enough core machine learning novelty for ICLR. Although the authors spend a lot of writing on the two main components (asymmetric masking and fixed-count patching), these are presented merely as empirical design choices without strong theoretical justification. Furthermore, the title and framing overclaim the scope of the work. Calling the model "EMR-JEPA" is misleading because general EMR data includes many modalities and longitudinal outpatient records (such as EHRSHOT), whereas this work only tests on 48-hour ICU stays with 48 variables. Similarly, the paper claims to do "clinical forecasting," which usually implies regression or multivariate trajectory prediction, but the experiments only evaluate two binary classification tasks.
  • Insufficient and outdated experimental evaluation: The experimental setup does not convincingly show the broad utility of the proposed method. First, the authors still rely on MIMIC-III. Given that MIMIC-IV has been released and stable for several years, evaluating on an outdated benchmark provides limited insight to the community. Second, even though the authors use the YAIB benchmark pipeline, they only pick two downstream binary classification tasks (sepsis and AKI). YAIB and other ICU benchmarks include many more tasks, including length-of-stay or lab value regression, which are completely missing here.
Questions for authors
  1. Why do we need to train a specialized JEPA model from scratch when existing EHR foundation models or general zero-shot embedding models can already capture clinical record representations? How does EMR-JEPA compare against those?
  2. Could the authors provide experimental results on the newer MIMIC-IV dataset?
  3. For the fixed-count patching, the patch size is set to 16 consecutive measurements. Could the authors explain this design choice and provide ablation experiments on different patch sizes?
  4. Given that the core contribution is applying an existing self-supervised architecture to specific ICU cohorts, this work feels much closer to domain-specific venues like MLHC, ML4H, or CHIL. Could the authors clarify what fundamental representation learning insights this paper offers to the broader ICLR community?
Reviewer_PaoNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper focuses on learning representations on clinical observations for downstream tasks. It proposes to group observations with a fixed count instead of a fixed duration to reflect the measurement policy through their density. It learns the representations of measurement policy by predicting patch timestamps and counts. The paper discovers that asymmetric masking can favor the forecasting horizon, which can learn informative representations for future predictions. Three clinical datasets are used for evaluation that the model is trained on two of them and tested on the third dataset. Both transfer learning and OOD scenarios are evaluated.

Critical strengths
  1. The fact that irregular time intervals among measurements carry information is a good observation to inspire the proposed approached.
Critical weaknesses
  1. Limited originality and contribution: Trying different settings of sample patching and predicting masking distribution (sampling counts and timestamps over an observed patient journey) are the main contribution of the paper. It is not clear if the patching based on the count will unavoidably average out health state changes over a longer period of time. embedding semantics.

  2. The paper is poorly presented and organized, which makes it so hard to read.

  • Line 213: Both OOD and ID are not defined.
  • Line 257: It is not clear what "both views" means.
  • Predicting the acqusition pattern is important but the masking scheme based on Eq 2 is not well explained
  • In Section 3.2, the description of the masking strategies are mixed together with discussion of the experimental results.
  • In Section 3.3, the description of the count and timestamp estimation tasks are mixed together with some discussion of the results. -
  • Line 299: RoPE is used without clear definition.
  • LIne 303: s_{pol}^{ctx} is used without clear definition.
  • Section 4.4, 4.5 and 4.6 are supposed to be presenting and discussing the experimental results. Yet most of the results are referring to the details in the Appendix.
  • The format of the tables is not consistent.
Questions for authors

Question 1: In practice, the settings of how frequent and how many clinical events happen within a period of time often depends on particular practices of a hospital. Has this been taken into consideration? If yes, how?

Question 2: Define s_{pol}^{ctx} and explain how to compute that.

Question 3: Explain the rationales regarding how the settings for each experiments are determined.

Reviewer_fdwKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The study aims to extend the popular JEPA idea to EMR (EHR) data. It specifically aims to present a dedicated masking procedure to fit JEPA to predictive tasks run on EHR.

Critical strengths
  • The idea of combining a “fixed-count grouping of consecutive observations with explicit prediction of future state counts and representative times” is novel and unexplored (as far as the reviewer knows).
  • The study contains a series of fairly extensive experiments evaluating the method across various in-domain and out-of-domain settings, tasks, and configurations.
Critical weaknesses
  • Overall, the writing is noticeably inaccessible (e.g., abstract), and the flow of the text is weak in many places. Notably, the true contributions of the proposed architecture (JEPAs) relative to prior work (e.g., encoder-based and masking approaches) remain unclear and often convoluted. This seems to be caused by switching between different concepts and terminologies across the descriptions.
  • The related work should also cover JEPA-based models on EHRs.
  • Several of the reported improvements seem to be within 2nd or 1st digit approximation, with some missing the variation range (like Table 1 and Table 2).
Questions for authors

How do time-to-event methods compare or interact with the proposed method?

Reviewer_KnUdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EMR-JEPA, a self-supervised framework for learning representations from irregular clinical time series. It groups consecutive measurements into fixed-count patient states, predicts future latent representations using an EMA target encoder, and adds auxiliary objectives for predicting future state counts and timestamps. The paper studies increasingly future-focused masking under a matched expected masking budget. Evaluation covers six-hour sepsis and AKI forecasting across MIMIC-III, eICU, and HiRID, including target-domain adaptation, source-only out-of-domain generalization, and additional representation and robustness analyses. The reported results show strong transfer performance and consistent OOD improvements for sepsis, although OOD AKI improvements are less consistent.

Critical strengths
  1. The evaluation distinguishes target-domain adaptation from source-only OOD generalization and rotates the held-out database across three clinical datasets. EMR-JEPA achieves the best reported transfer-learning average rank for both tasks and improves both OOD sepsis metrics on every target database. These results provide meaningful evidence of the representation’s practical usefulness beyond a single dataset.

  2. Fixed-count states have data-dependent future counts and timestamps, making acquisition-pattern prediction a reasonable auxiliary learning objective. Table 2 reports an improvement in mean OOD frozen-probe AUROC from 0.6502 to 0.6814 for the policy-supervised configuration. The paper also acknowledges that acquisition-sensitive representations need not be invariant to observation thinning, which makes its discussion more balanced.

Critical weaknesses
  1. The main ingredients have precedents, while key comparisons—particularly fixed-duration encoding with observation-process supervision and the same backbone trained to predict raw future values—are missing, leaving the distinctive value of the proposed combination unclear.
  2. Increasing future masking simultaneously exposes more historical observations and removes visible future observations, making it unclear which change drives the improvement.
  3. Table E1 specifies (N=32), whereas Section 4.1 and Appendix G.1 specify (N=64) or (128); Table F2 lists history/future observation caps of 512/512, whereas Appendix G.1 specifies 4096/1024 or 2048.
  4. The core masking and encoding ablations use only one pretraining seed and one probe seed, and the label-efficiency results lack seed-wise dispersion, leaving the stability of these findings unestablished.
Questions for authors

See Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.