- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether laboratory measurements and clinical care events should be represented separately in self-supervised learning from electronic health records (EHRs). The authors argue that conventional approaches may conflate information about patients' physiological states with information about clinical care processes, potentially limiting generalization across hospitals. To address this issue, they propose a coupled architecture with separate encoders for laboratory values and care events, aligned through contrastive learning objectives. The approach is evaluated on three ICU databases (MIMIC-IV, eICU, and HiRID), using eight downstream tasks and ten random seeds. The authors report statistically significant AUROC improvements over a single-stream baseline in 67 of 85 comparisons, with an average improvement of 0.019. Additional linear probing experiments demonstrate improved recoverability of laboratory values, including under cross-site and cross-database transfer. Ablation studies investigate the contributions of architectural separation and different contrastive objectives. Overall, the paper addresses a practically relevant problem and presents a comprehensive experimental evaluation.
Critical strengths
-
The distinction between physiological measurements and clinical care processes is important for learning representations from observational healthcare data. Laboratory measurements provide information about patient physiology, whereas laboratory ordering patterns and medication initiation also reflect clinical decision-making, institutional policies, and healthcare workflows. The latter may differ substantially across hospitals and healthcare systems. Investigating whether these information sources should be modeled separately is therefore a relevant research question with potential implications for the robustness and transferability of EHR foundation models.
-
A major strength is the extensive evaluation across three ICU databases: MIMIC-IV, eICU, and HiRID. The evaluation covers eight downstream prediction tasks, multiple random seeds, and cross-site and cross-database transfer settings. The reported improvements are consistent across most comparisons, suggesting that the proposed approach provides practical benefits over the selected single-stream baseline. The use of ten random seeds further strengthens the reliability of the experimental findings.
-
The linear probing experiments are a particularly positive aspect of the work. Rather than evaluating representations exclusively through downstream predictive performance, the authors investigate how well laboratory values can be recovered from the learned embeddings. The reported improvements in laboratory-value recovery, including under cross-database transfer, provide complementary evidence about the information retained in the representations.
-
The ablation experiments distinguish the contributions of encoder separation, laboratory set encoding, and different contrastive learning objectives. The observation that the cross-encoder contrastive objective primarily benefits downstream prediction, while not consistently improving laboratory-value recovery, is particularly interesting. It suggests that different architectural components influence representation properties in distinct ways.
Critical weaknesses
-
My main concern relates to the methodological novelty and the positioning of the proposed approach relative to existing literature. In particular, the claim that previous EHR Transformer models serialize heterogeneous patient records into a single token sequence is too general and overlooks previously published approaches that explicitly distinguish different types of EHR information. For example, ExMed-BERT (Lentzen et al., IEEE Journal of Biomedical and Health Informatics, 2023, DOI: 10.1109/JBHI.2023.3288768) represents different EHR information types through separate input sequences and corresponding embeddings before combining them within a shared Transformer architecture. Although this differs from the two-encoder design proposed in the present paper, it demonstrates that heterogeneous EHR information has previously been handled through structured, type-specific representations rather than through a single undifferentiated tokenization scheme.
-
The experimental comparison against a conventional single-stream Transformer is relevant but insufficient to establish the advantages of the proposed design over existing approaches for representing heterogeneous EHR data. In particular, the paper should consider architectures that employ type-specific embeddings followed by a shared Transformer, as well as other multimodal fusion strategies such as separate encoders with intermediate fusion or cross-attention. This distinction is important because the proposed approach changes several aspects of the architecture simultaneously: the separation of input streams, the encoding of laboratory values, the representation capacity, and the contrastive training objectives.
-
Another concern is the apparent lack of systematic hyperparameter optimization. The proposed architecture introduces several design choices, including encoder configurations, representation dimensionality, the weighting of contrastive objectives, and potentially contrastive temperature parameters. The performance of contrastive representation-learning methods can be sensitive to these choices. It is therefore important to establish whether the proposed architecture and the single-stream baseline were optimized under comparable computational budgets. The extensive evaluation across ten random seeds is commendable, but repeated training with different random initializations does not substitute for hyperparameter optimization.
-
The central motivation is that physiological measurements and clinical care events capture different aspects of the patient record, with care processes being more dependent on institutional practices. While the experiments demonstrate improved downstream prediction and cross-database transfer, they do not directly establish that these gains arise from reduced dependence on hospital-specific care processes. For example, it has not been tested whether learned representations allow to predict the hospital or site from which data was collected.
-
The linear probing experiments show improved recoverability of laboratory values from the proposed representations. While this is encouraging, the interpretation requires some caution. Because laboratory values are explicitly processed by a dedicated encoder, improved recovery of these measurements is not entirely unexpected. It remains unclear to what extent the observed improvements reflect a more meaningful representation of patient physiology rather than the preservation of information that is directly supplied to the laboratory encoder.
-
The ablation results suggest that different components contribute differently to laboratory-value recovery and downstream prediction. In particular, the cross-encoder contrastive objective appears to improve predictive performance without consistently improving laboratory-value recovery. This is an interesting finding that deserves further investigation.
Additional considerations:
-
Although the reported improvements are statistically significant in most comparisons, the mean absolute AUROC gain of 0.019 is relatively modest. The authors should contextualize its practical relevance and provide appropriate confidence intervals and effect-size analyses.
-
Given the 85 reported comparisons, the paper should clearly explain how statistical significance was assessed and whether appropriate corrections for multiple testing were applied. Dependencies between tasks, datasets, and seeds should also be considered.
-
The proposed architecture introduces additional encoders and contrastive objectives. Reporting parameter counts, training time, and computational requirements would help assess the practical trade-off between model complexity and predictive performance.
Questions for authors
- What is the precise methodological novelty of the proposed architecture relative to existing modality-specific and multi-view EHR representation-learning approaches?
- Can the authors compare their approach against a shared Transformer with modality-specific embeddings, as well as other established multimodal fusion architectures, under comparable model capacities and training budgets?
- How were the hyperparameters selected? Were the proposed model and the baseline subjected to comparable hyperparameter optimization procedures?
- Can the authors provide more direct evidence that the improved cross-site and cross-database performance results specifically from separating patient physiology and institutional care processes?
- To what extent are improvements in laboratory-value recovery explained by the dedicated laboratory encoder rather than by the contrastive learning objectives?
- How were statistical significance tests across the 85 comparisons performed, and were multiple comparisons appropriately accounted for?
- Will the code, preprocessing procedures, and experimental configurations be made publicly available?
2 comments
Response to Reviewer h7GJ Part 2
Q3 (Hyperparameter selection). At submission all arms shared an unsearched default (lr = 2e-3). We ran an equal-budget sweep across four learning rates × three arms × three seeds on eICU and HiRID (72 cells: 54 new runs plus 18 at 2e-3), selecting rates on the in-distribution split only. Every arm preferred a lower rate than 2e-3; under tuned rates the coupled margin over single-stream holds (eICU: +0.0044 → +0.0123; HiRID: +0.0408 → +0.0363, 3/3 seeds each), while decoupled on eICU moves from −0.0213 to −0.0059. This sweep ran under the recovery-model convention on one hyperparameter across two databases, so it checks relative ordering rather than re-estimating the 85-cell +0.019 margin.
Q4 (Site invariance vs. institutional care processes). Promoted to Appendix K (Table 13): on eICU, a multinomial logistic classifier predicting which of the 19 held-out hospitals (with ≥ 100 evaluation rows) a row came from reaches macro AUROC 0.91 on single-stream and 0.93 on the coupled laboratory stream (0.87 on the coupled care stream). None of the representations are site-invariant. At the same time, the recovery gap is largest under cross-database transfer (+0.075 to +0.238 R², Table 2), where source and target hospital IDs come from different health systems and source site codes cannot help the probe. Section 7 and Appendix K state clearly that the design yields better transfer, not site invariance, and we do not attribute the gains to removing site-specific care information.
Q5 (Dedicated encoder vs. contrastive objectives). Appendix G (Table 10) adds a linear visible-input reference—a ridge (alpha = 10) on unmasked reconstruction-slot values, presence indicators, and vital-sign means—reaching 0.406, 0.501, and 0.465 R² (eICU, MIMIC-IV, HiRID) against 0.303, 0.475, and 0.467 for coupled (0.214, 0.412, 0.336 for single-stream). On eICU this reference exceeds both representations on 16 of 20 analytes, and each analyte's gap tracks its cross-predictability (Spearman +0.86, +0.81, +0.77; on eICU three red-cell indices carry 39% of the summed gap, though the correlation stays +0.77, p = 0.0003 without them). We frame the gap strictly as retention of input signal (0.88 vs. 0.67 of the linear reference on eICU), not information beyond the input. In Table 3, encoder separation carries the recovery gain while the cross-encoder loss does not add to it, and replacing the set encoder with a Transformer preserves the gap on all three databases (Appendix M).
Q6 (Multiple testing, effect sizes, and practical utility). Appendix J (Table 12) applies Holm and Benjamini–Hochberg across all 85 downstream cells (paired with the stay bootstrap in Appendix L):
| Contrast | Uncorrected (+ / − / n.s.) | Holm | Benjamini–Hochberg |
|---|---|---|---|
| Coupled − single-stream | 67 / 1 / 17 | 50 / 1 / 34 | 66 / 1 / 18 |
| Decoupled − single-stream | 47 / 11 / 27 | 24 / 5 / 56 | 42 / 10 / 33 |
We agree that the downstream AUROC margin is modest (+0.0112 against STraTS rather than +0.0179 against our baseline, with decoupled tying STraTS at +0.0023), and we no longer present AUROC as the main payoff of separation. Instead, our claim is that preserving a decodable laboratory channel (+0.042 to +0.108 R² in distribution, +0.075 to +0.238 after transfer) comes at no cost in AUROC—and our three controls show that AUROC alone cannot select for this property, since cross-attention (Appendix N) matches the full AUROC margin while retaining no more laboratory signal than single-stream, random routing (Appendix H) matches semantic routing on AUROC while recovering values worse, and STraTS (Appendix P) ties decoupled on AUROC while recovering less than both separated models. For detection alone, the laboratory/care split is not an accuracy lever; among designs that tie on AUROC, the coupled model retains substantially more laboratory signal for linear readout auditing after transfer. We did not evaluate subgroup performance or calibration, we have not shown that this diagnostic improves clinical decisions, and we do not claim site invariance (Section 7, Appendix K).
Q7 (Code, configurations, and compute). The training script, task lists, analysis scripts, and per-seed JSONs are packaged for public release upon publication with an end-to-end synthetic-data runner (Reproducibility Statement); full architectural and training details are in Section 3, Appendix B, and Appendices G–P (with our 90-run scratch replication bound of 0.008 R² vs. the 0.013 R² single-run GPU noise floor in our response to Reviewer BBsx). Table 5 gives allocated and trained parameter counts (within 6% across the three main models), and all arms train for 60 epochs at the same batch size on identical data, matching optimizer steps and tokens seen by construction.
Response to Reviewer h7GJ Part 1
Thank you for the thorough review. Several of your questions exposed real limitations in our initial framing and led directly to the new experiments in Appendices G–P.
We now cite ExMed-BERT in the Introduction and Related Work and revised both passages: rather than saying prior models typically serialize the record, we state that they process laboratory values and care events within one encoder, whether by serializing events into a single sequence or by combining type-specific embeddings before a shared Transformer trunk.
Q1 (Capacity and confounds). See our response to Reviewer Bsjw (Q3) and Appendix O for the dropout, L2, 192-d readout-width, and probed-layer controls (the last of which shows a real sensitivity on MIMIC-IV now disclosed in Appendix O and Section 7). On capacity, matching the decoupled care-event encoder to the coupled model's exact 693,870 trained parameters (+33,024 parameters, Appendix B) leaves downstream AUROC unchanged (−0.0007 across 85 cells), and replacing the laboratory set encoder with a two-layer Transformer (Appendix M) adds 264,831 effective parameters (+40%, bringing decoupled to 1.40× its set-encoder total) without increasing recovery in either arm.
Q2 (Shared Transformer with modality-specific embeddings, and intermediate fusion). Our single-stream baseline is already a shared Transformer with modality-specific embeddings: separate embedders for laboratory values, vitals, care summary, and care-event sequence, plus a learned token-type embedding, before a shared two-layer Transformer (now explicit in Sections 2–3).
To test intermediate fusion, we implemented bidirectional cross-attention between the laboratory token set and care-event sequence after token embedding and before pooling, replacing the output-level contrastive loss so only the meeting point changes (Appendix N, 60 runs). On the 85 unrestricted downstream cells, cross-attention and the contrastive loss are indistinguishable (+0.0010 within-database, 8 up / 5 down; −0.0003 after transfer, 11 up / 17 down):
On laboratory-value recovery, however, cross-attention drops to the single-stream baseline (0.6810, 0.7701, 0.7573 R² vs. single-stream 0.6945, 0.7921, 0.7564; Table 18), trailing the coupled model by −0.048, −0.079, and −0.113 R² (0 of 10 seeds above zero on each database). Comparing both axes (Table 19):
Any cross-stream interaction buys the downstream margin; keeping the representations separate before pooling buys value retention. Because cross-attention has more parameters than all three main arms (818,222 allocated / 793,454 trained vs. 685,614 / 660,846 for decoupled), its lower recovery is not due to capacity.