- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes EMR-JEPA, a self-supervised learning method that adapts Joint-Embedding Predictive Architectures (JEPA) to clinical time-series data for clinical forecasting. Instead of reconstructing raw measurement values, the model predicts the latent representations of masked states generated by an exponential-moving-average (EMA) target encoder.
The methodology focuses on two main components. First, the authors explore an asymmetric masking policy. By keeping the total budget of masked tokens fixed, they progressively shift the masking probability from historical states to the future forecast horizon. They show empirically that masking 100% of future states and a smaller portion of past states reduces future latent prediction loss and improves downstream performance compared to standard symmetric masking. Second, to handle irregular clinical sampling, they introduce fixed-count patching, which groups every 16 consecutive measurements into one patient state regardless of the time duration. Because fixed-count states have variable durations, the number of states and their timestamps in the future six-hour window are unknown at inference time. To address this, the authors add a policy token and two auxiliary heads during pretraining to explicitly predict the count and normalized timestamps of future states.
The authors evaluate their approach across three ICU datasets: MIMIC-III, eICU, and HiRID. The experiments test transfer learning and out-of-domain generalization on two six-hour binary classification tasks: sepsis and acute kidney injury (AKI) prediction, comparing against clinical self-supervised baselines like STraTS, DuETT, and SMART.
Critical strengths
The paper adapts JEPA, a state-of-the-art self-supervised learning method, to ICU time-series datasets. The authors do a good amount of experimental work to test different asymmetric masking ratios and propose a fixed-count patching strategy to handle irregular clinical measurements.
Critical weaknesses
- Limited technical contribution and overclaiming: The paper is mostly a direct application of JEPA to ICU datasets, which lacks enough core machine learning novelty for ICLR. Although the authors spend a lot of writing on the two main components (asymmetric masking and fixed-count patching), these are presented merely as empirical design choices without strong theoretical justification. Furthermore, the title and framing overclaim the scope of the work. Calling the model "EMR-JEPA" is misleading because general EMR data includes many modalities and longitudinal outpatient records (such as EHRSHOT), whereas this work only tests on 48-hour ICU stays with 48 variables. Similarly, the paper claims to do "clinical forecasting," which usually implies regression or multivariate trajectory prediction, but the experiments only evaluate two binary classification tasks.
- Insufficient and outdated experimental evaluation: The experimental setup does not convincingly show the broad utility of the proposed method. First, the authors still rely on MIMIC-III. Given that MIMIC-IV has been released and stable for several years, evaluating on an outdated benchmark provides limited insight to the community. Second, even though the authors use the YAIB benchmark pipeline, they only pick two downstream binary classification tasks (sepsis and AKI). YAIB and other ICU benchmarks include many more tasks, including length-of-stay or lab value regression, which are completely missing here.
Questions for authors
- Why do we need to train a specialized JEPA model from scratch when existing EHR foundation models or general zero-shot embedding models can already capture clinical record representations? How does EMR-JEPA compare against those?
- Could the authors provide experimental results on the newer MIMIC-IV dataset?
- For the fixed-count patching, the patch size is set to 16 consecutive measurements. Could the authors explain this design choice and provide ablation experiments on different patch sizes?
- Given that the core contribution is applying an existing self-supervised architecture to specific ICU cohorts, this work feels much closer to domain-specific venues like MLHC, ML4H, or CHIL. Could the authors clarify what fundamental representation learning insights this paper offers to the broader ICLR community?