- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PAID, a multimodal EHR representation-learning framework that uses different admissions of the same patient as positive pairs. A shared Transformer produces a patient token (PAT) for cross-admission identity alignment and an admission token (ADM) intended to retain episode-specific information. Adversarial identity suppression, masked cross-modal reconstruction, and a missingness-aware matching score support this design. Experiments on MIMIC-IV and eICU evaluate retrospective clinical classification, few-shot learning, patient retrieval, and representation specialization.
Critical strengths
Strength 1: The paper makes good use of recurrent admissions from the same patient as positive pairs for contrastive learning. These admissions share patient-level information but may differ in clinical conditions and available modalities. This provides a natural source of supervision without requiring task-specific outcome labels.
Strength 2: The proposed dual-space design is intuitive. By aligning PAT representations across admissions while using adversarial learning to reduce patient identity information in ADM, the framework aims to capture both patient-level consistency and admission-specific differences. The use of availability-masked attention and cross-modal reconstruction also helps address missing modalities in EHR data.
Strength 3: The experimental results are encouraging. PAID achieves strong performance across multiple clinical prediction tasks on MIMIC-IV and eICU. The additional few-shot, patient retrieval, and ablation experiments provide further evidence of the usefulness of the learned representations.
Critical weaknesses
Weakness 1: The experimental comparisons are not fully consistent across methods. As mentioned in Section 4.3, different methods use different labeled training data and downstream predictors. In addition, some few-shot results combine PAID with XGBoost (Appendix A.3). This makes it difficult to determine whether the improvements come from the proposed method or the different experimental settings. A comparison under the same training data and downstream predictor would be more convincing.
Weakness 2: The experiments do not clearly demonstrate that PAT and ADM learn complementary information. In Table 5, PAT alone achieves an acute-task mean AUROC of 0.85961, while combining PAT and ADM only improves it to 0.85977. Although Table 3 shows that the dual-space design improves performance, it remains unclear how much additional admission-specific information ADM provides beyond PAT.
Questions for authors
Question 1: Could the authors compare PAID with a strong baseline using the same labeled training data and downstream classifier? This would help clarify whether the improvements come from the proposed representations or the different experimental settings.
Question 2: Table 5 shows very little improvement when ADM is added to PAT. Could the authors compare PAT and PAT+ADM using the same frozen encoder and downstream classifier to better demonstrate the additional contribution of ADM?
Question 3: In Section 4.4, the authors report a gain of 0.0135 for readmission over the strongest baseline. However, Table 1 shows that MUSE achieves 0.6172, which is higher than SimCLR (0.6067), so the actual improvement is only 0.0030. Could the authors clarify this discrepancy and check the reported performance gains?
2 comments
Reply 2/2 — Complementarity, corrected claims, σ and ablation (Weakness 2; Question 3)
Complementarity, measured on one frozen encoder. Original Table 5 fine-tuned each readout (20+20 epochs, feature-mask refinement, best validation checkpoint, A.8); fine-tuning absorbs readout differences, so the small increments do not establish complementarity. New test: one shared frozen encoder per seed, identical logistic readout, splits and tuning budget, three fixed seeds, full labels, six-task macro AUROC:
| Readout | AUROC (mean±SD) |
|---|---|
| PAT | 0.8579±0.0033 |
| ADM | 0.8593±0.0026 |
| PAT+ADM | 0.8635±0.0008 |
| PAT+ADM+σ | 0.8639±0.0006 |
PAT+ADM − PAT = +0.0056±0.0025, all three seed-level patient-clustered paired CIs positive; ADM alone matches PAT, not a degraded copy. The increment is larger on individual tasks (LOS>3d +0.0075, LOS>7d +0.0070, ICU utilization +0.0058, AKI +0.0049, sepsis +0.0055, all with three positive seed CIs) and grows when modalities are missing (LOS>7d +0.0171 without text, +0.0120 without ECG; LOS>3d +0.0133 without text; ICU +0.0142 without text), exactly the incomplete-coverage regime the dual space targets. σ adds a small further increment (+0.0004). The complementarity claim is rewritten as this consistent, measurable increment.
Corrected headline numbers. The strongest readmission baseline is MUSE (0.6172), not SimCLR; the improvement is 0.6202−0.6172 = +0.0030. Excluding exploratory Next ICU there are 11 clinical endpoints (7 MIMIC + 4 eICU), not 12. The mean MIMIC gain is 0.0040 (max 0.0074, ICU utilization); the PAID-only few-shot count is 18/21, with the three stacked cells decomposed as PAID-only / raw-XGBoost / ensemble. The abstract, §4.4 and conclusion have been revised. For eICU sepsis we compare results with diagnosis inputs removed: under the exact paper protocol (multi-task), removing diagnosis/ICD inputs and direct sepsis text gives 0.9966→0.9279 (downstream_test), 0.9959→0.9094 (held_out), 0.9963→0.9188 (all_unseen); the drop is expected because diagnosis fields partly constitute this retrospective endpoint, and the remaining 0.9188 reflects substantial non-diagnostic signal.
Table 3 and the controlled attribution. Table 3 ablations are real but jointly remove the associated sampling and auxiliary objectives (A.7), so they show system-level joint effects rather than per-loss attribution: removing the dual space (ADM+matcher) costs −0.0568 mean AUROC and removing the adversarial loss −0.0274, neither negligible. The new controlled matrix (below) gives the sharper attribution.
σ: three components and their use. is the outer average over d coordinates of the availability-weighted per-coordinate standard deviation of visible modality tokens; with one visible modality it reduces to , while context and missingness still contribute (A.4). is a learned nonnegative readout of the patient-context token; with , so observed modalities contribute exactly zero. The squared sum (Eq. 6) aggregates these nonnegative sources so any one can down-weight matching and no term cancels another. Eq. 7 subtracts from the identity logit (), down-weighting identity matching when visible evidence is internally inconsistent, context is weak, or modalities are missing. Dispersion is computed before modality-type embeddings, so it reflects content rather than modality identity.
Controlled protocol. The controlled arms disable all auxiliary identity sources simultaneously (: no CheXpert cross-admission target, no prototype, no temporal term), with reconstruction, VICReg, architecture, masks, data/steps/budget and readout fixed; only the pairing factor varies (true/random/synthetic/no-identity), three fixed seeds, paired CIs. The identity head reads a detached PAT via a separate projection, so identity gradients never enter the shared clinical encoder (). Clinical gains are attributable to cross-modal content learning and the dual readout (+0.0056, CIs positive); identity alignment supplies cross-admission consistency and enables the longitudinal next-visit supervision that improves the macro by +0.0115/+0.0133/+0.0143 (full/5%/1%) over no-identity, all CIs positive.
Reply 1/2 — Fair comparison and input-leakage audit (Weakness 1; Questions 1 and 2)
Comparison under the same labeled data and classifier. The original baselines were faithful reproductions of each method's published recipe (e.g., MUSE used Adam, lr 2e-4, batch 512, dropout 0.25, patient-level val fraction 0.1, final checkpoint), so small recipe differences are inherent to faithful reproduction; all methods shared the same pretraining admissions and feature pipeline. What differed was the downstream labeled pools and predictors (§4.3/A.3), so those tables are system-level comparisons. We agree they cannot isolate the representation's contribution, and re-ran the comparison under a fully unified protocol: the same 304,916 pretraining admissions; the same labeled patient pools and fit/validation/test manifests; the same label budgets (full/1%/5%); StandardScaler fit on training data plus the same logistic readout (same solver, regularization, class weights, tuning budget); validation-only model selection; fixed-final-epoch pretraining checkpoints with no test-based selection; fine-tuning with identical per-task heads, a 50-epoch cap and validation selection; three fixed seeds, 2,000-resample patient-clustered paired bootstrap, and all metrics recomputed from saved predictions. PAID uses the same training configuration as the baselines. Test AUROC (mean±SD):
PAID leads both baselines on both tasks, and every PAID−MUSE seed-level paired CI is positive (AKI +0.0089±0.0020; readmission +0.0136±0.0021). The gain concentrates where patient history exists: stratifying AKI by the patient's admission count, the PAID−MUSE gap rises from +0.0036 (single admission) to +0.0066 (two admissions) to +0.0111 (three or more, all seed CIs positive).
Label leakage audit. MIMIC inputs cannot directly reveal an endpoint: the 51 tabular features are demographics plus present-on-admission diagnosis chapter aggregates only (discharge codes, which define the sepsis label, are excluded by construction) plus ED lab/vital aggregates ≤24h; WARD/ICU labs, meds, transfers, procedures and orders are zeroed. The 64 time-series features are mean/std/min/max of ED vital channels only, with no lab channels, so the creatinine trajectory defining AKI is not an input. Readmission labels use only post-discharge admissions. Label code is shared and identical across methods. MIMIC: mortality = in-hospital death; LOS>3d/7d = discharge−admission timestamps; ICU utilization = ICU transfer during stay; readmission = next admission ≤30 days after discharge (eligible subset 12,853); AKI = creatinine +0.3 mg/dL within 48h or ≥1.5× admission minimum, during stay; sepsis = sepsis/septicemia ICD codes or titles. eICU: mortality = hospital expiry; LOS = unit discharge offset; sepsis = sepsis terms in stay diagnosis text/codes; Next ICU = exploratory subsequent-stay endpoint. Full eligibility filters and per-modality windows are in A.2. For eICU sepsis, whose legacy endpoint is a retrospective diagnosis classification, we compare results with diagnosis inputs removed (Reply 2/2).