acceptodds
Under review as a conference paper at ICLR 2027

MLPA: Multimodal Representation Learning via Masked Latent Prediction

Abstract

Integrating multi-modal Electronic Health Records (EHR) is difficult. Clinical streams vary widely in length, missingness is often clinically meaningful rather than random, and labeled outcomes are scarce. High-volume streams such as medication logs can dominate cross-modal attention and crowd out sparser signals, a failure mode we term modality washout. The objectives typically used for multi-modal integration add to the problem. Generative losses spend capacity on reconstructing high-entropy raw inputs, while supervised losses shape representations around a single downstream target. We introduce the Multi-modal Latent Predictive Architecture (MLPA), a self-supervised framework that learns patient representations from incomplete clinical records by predicting masked targets in uni-modal latent space. MLPA routes cross-modal exchange through a shared bottleneck transformer and represents missing modalities with learned empty-state tokens. We evaluate MLPA on 81,664 MIMIC-IV admissions and 188,094 eICU stays. On MIMIC-IV, MLPA matches a multi-modal masked autoencoder with 42.4% fewer parameters. On eICU, it achieves higher mortality AUPRC than the autoencoder (0.430 vs. 0.403). Without labels at pre-training, MLPA outperforms the semi-supervised method MUSE+ on in-hospital mortality in both datasets (eICU AUPRC 0.430 vs. 0.388). In eICU, this advantage holds for stays with one missing modality. MLPA also forms more compact clusters than MUSE+ for outcomes that neither model was trained on. An attention analysis shows that unconstrained cross-attention gives 47% of its attention to medication tokens, while the bottleneck keeps attention balanced across streams.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Self-supervised Multi-modal learning for Extremely Heterogeneous Clinical Data

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_ZuUtOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This paper introduces MLPA, a self-supervised framework for multi-modal EHR with non-random missingness, imbalanced modal tokens and scarce labels. It defines modality washout, where high-token modalities overwhelm sparse ones in cross-modal attention. MLPA adopts frozen unimodal encoders, learnable empty-state tokens, and MBT bottleneck transformer, with a masked latent prediction pre-training objective without negatives or teachers. Evaluated on MIMIC-IV and eICU, MLPA cuts parameters by 42.4% vs MAE and surpasses MUSE+ on in-hospital mortality prediction. MBT balances attention and yields tighter patient representations (lower DBI).

Critical strengths

1.The problem is practically important. Incomplete EHRs, highly imbalanced modality token counts, and limited labels are realistic obstacles in clinical representation learning. The newly formalized modality washout offers a concrete characterization of cross-modal attention bias specific to multi-modal EHR scenarios. 2.The system design is clear and the paper evaluates two ICU datasets with comprehensive representation quality analysis and downstream clinical prediction comparisons against multiple strong baselines.

Critical weaknesses

1.Incomplete ablation studies and limited technical novelty. The proposed MLPA is assembled from existing modules adapted for incomplete EHR, yet the paper lacks component-wise ablation experiments to independently quantify the contribution of MBT bottleneck routing, masked latent prediction and empty-state token. It also does not test simpler bottleneck alternatives (e.g., pooling, fixed latent queries). Without validating the necessity of each individual component, the technical contribution of this work cannot be substantiated, which fundamentally weakens the core claims of the paper. 2.The core claim of modality washout lacks causal validation, and key experimental comparisons are missing. The argument for modality washout relies merely on correlational attention statistics rather than causal intervention tests. Moreover, MLPA shows severe performance degradation under dual-modal missingness on eICU and lacks quantitative comparison against highly related SOTA CRL-MMNAR and simple imputation baselines. These omissions undermine the reliability and generalizability of the empirical evaluation.

Questions for authors

1.The paper proposes three core components: MBT bottleneck routing, masked latent prediction objective, and empty-state token, but lacks component-wise ablation to separate their individual contributions. Please clearly answer: Could you provide full component-wise ablation results to quantify the individual performance gain brought by each module? What performance drop occurs if MBT is replaced with simple pooling or fixed latent queries? 2.The paper’s core claim of modality washout is only supported by attention proportion correlation rather than causal evidence. Please clearly answer: Can you perform intervention experiments by masking the high-token medication modality to verify whether this attention skew directly degrades downstream prediction performance? 3. MLPA suffers severe AUPRC degradation and is outperformed by MUSE+ when two modalities are simultaneously missing on eICU, and comparisons between empty-state token and simple imputation baselines are absent. Please clearly answer: Why does MLPA degrade severely under dual-modality missingness on eICU? Can you report comparisons between empty-state token and simple imputation baselines (zero / mean filling) across diverse missingness patterns? 4. The paper only discusses CRL-MMNAR (EMNLP 2025) qualitatively and omits a matched-parameter contrastive baseline with concatenated modal embeddings. Please clearly answer: Will you add quantitative experimental comparisons against CRL-MMNAR, and also add the contrastive baseline with concatenated modality embeddings under matched parameter counts? 5. There exists an unjustified line break within the paper title, indicating insufficient proofreading before submission. Please fix the improper line break in the title.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ZuUt

Thank you for the detailed questions. We answer each with new experiments; Tables A–E are in the general response.

Q1. Component-wise ablation (Table D, MIMIC-IV; changes in mortality / phenotyping AUPRC vs MLPA).

  • MBT routing replaced by simple per-stream pooling (late fusion): −0.001 / −0.004, and −0.014 / −0.007 with the linear probe. Fixed latent queries (Perceiver): −0.014 / −0.012. Full self-attention over all tokens (early fusion): −0.003 / −0.006.
  • Masked latent prediction replaced by raw reconstruction (MAE), paired over three seeds (Table A): −0.015 to −0.020 on eICU and −0.002 to −0.004 on phenotyping in every seed, and a tie on MIMIC-IV mortality. Replaced by SimCLR at matched size: −0.028 on mortality. Replaced by VICReg: −0.038.
  • Empty-state token replaced by zero token, omission, explicit indicator or mean fill: within 0.004.

The gains come from the bottleneck and the latent objective. MLPA is robust to how missingness is encoded.

Q2. Intervening on the medication stream. We removed medications at test time and thinned them to 25% of their tokens (Table E). The Perceiver loses more mortality AUPRC than the MBT in both cases (−0.384 vs −0.327 removed; −0.123 vs −0.096 thinned), while removing codes costs it less (−0.040 vs −0.063). Unconstrained cross-attention therefore leans harder on the dense stream. The Perceiver also loses most age and sex information (AUROC 0.71 / 0.74 vs 0.96 / 1.00 for the MBT), and restoring age and sex recovers about 40% of its phenotyping deficit. The removal and restoration tests are interventions, not correlations. They support washout of the shortest stream and show that it accounts for a measurable part of the routing gap. Attention shares are now descriptive only.

Q3. Two missing modalities, and imputation baselines.

  • Degradation. The deficit reported in the submission (0.250 vs 0.374) was an artifact of the faulty MUSE+ loader and does not reproduce. Against the corrected MUSE+ (Part 1) and over three seeds, MLPA reaches AUPRC 0.292 vs 0.280 and AUROC 0.820 vs 0.754 on eICU stays with two missing streams (Table C). AUPRC is lower for every model on these stays, where mortality is 6.1% against 10.3% on complete stays.
  • Imputation baselines. On MIMIC-IV, zero filling, mean filling, omission and an explicit indicator are within 0.004 AUPRC of the empty-state token. On eICU, appending an explicit presence indicator to Zf adds 0.002 AUPRC overall and nothing significant on the one- and two-missing subsets.

Q4. CRL-MMNAR and a matched contrastive baseline.

  • CRL-MMNAR, run with the authors' code on our frozen embeddings over three seeds. At full labels MLPA is ahead of it on MIMIC-IV mortality (0.908 vs 0.896 AUPRC) and phenotyping (0.803 vs 0.795). On eICU, CRL-MMNAR scores 0.484 with its own heads and 0.492 with our probe, against 0.485 for MLPA. With 1% of the labels MLPA is ahead by 0.044 on eICU (0.315 vs 0.271) and 0.051 on phenotyping (0.746 vs 0.695).
  • SimCLR on the concatenated modality embeddings, with trainable parameters matched to MLPA (1.83M vs 1.93M), reaches 0.879 / 0.793 against 0.907 / 0.802 for MLPA (MIMIC-IV mortality / phenotyping).

Q5. Title. The line break is fixed in the revised PDF.

On novelty. Each component has precedents. The combination, as a label-free objective for incomplete multi-modal EHR, is new to our knowledge. The new experiments show what each part contributes: the bottleneck and the latent targets carry measurable gains, and the empty-state token is a convenience. Together they give a label-free representation that is ahead of both supervised baselines on MIMIC-IV at full labels, 0.011 AUPRC behind MUSE+ on eICU, and ahead of both with 1–10% of the labels on eICU and phenotyping (Table B). Our reply to Reviewer qqdC expands on this.

We hope this resolves your concerns about the ablations, the washout evidence and the baselines. If it does, we would be grateful if you reconsidered your score.

Thank you for the detailed rebuttal, which almost solved most of my soundness concerns. However, the new results have not eliminated my concerns about technological novelty. Therefore, I have increased my rating from 1 (clear rejection) to 2 (weak rejection).

AuthorsOpenReview ↗

On the claim of novelty

Thank you for the follow-up and for raising your score. Our previous reply presented the contribution as a new combination of components, which undersells it. We agree that the bottleneck, masked latent prediction and missing-modality tokens each exist, and we do not claim them. We claim three findings about incomplete multi-modal EHR:

Shared-query fusion silently loses most of the shortest stream. Age and sex become poorly decodable, yet headline AUPRC drops by only about 0.01. Latent targets beat contrastive objectives and match or beat raw reconstruction, at lower pre-training cost. Explicit missingness handling adds nothing on top of this objective. With trivial handling, the label-free model is competitive with methods built for missing modalities at full labels. With 1% of the labels it is ahead of them on eICU and phenotyping.

Each finding answers a design question that the component papers leave open for this setting, and each changes what a practitioner would build. (The revised introduction now states them as the contributions) If prior work already establishes any of them, we would be grateful for the reference.

Finally, a brief note on how we understand novelty for a paper of this kind. ICLR’s reviewer guide asks whether a submission contributes new knowledge, counting empirical knowledge and knowledge for practitioners, and notes that this does not require state-of-the-art results. On that reading, the question is less whether each module is new than whether the paper establishes something the community did not know, with evidence strong enough to act on. We believe the three findings above meet that bar. Each is supported by controlled comparisons or interventions, and each changes a concrete design decision for incomplete multi-modal EHR. We hope the paper can be assessed on that basis.

If our arguments are convincing to you, please revise your score. We thank you for your efforts and engagement either way.

The evidence follows below.

Details

  1. Shared-query fusion (Table E).

With Perceiver-style latent queries, linear probes on the patient representation recover age and sex at AUROC 0.71 / 0.74. With the bottleneck they reach 0.96 / 1.00. ICD and radiology content is retained equally well (0.93 / 0.85 vs 0.94 / 0.85). Restoring age and sex recovers about 40% of the Perceiver’s phenotyping deficit. The Perceiver also leans harder on the densest stream. Removing medications costs it 0.384 mortality AUPRC, against 0.327 for the bottleneck.

The loss appears under one routing and not the other, with matched objective, data and parameters. It is therefore a property of the fusion design. It is distinct from the optimization-level modality imbalance addressed by Gradient-Blending or OGM-GE. In headline metrics the Perceiver trails by only 0.014 / 0.012 AUPRC, so the loss is easy to miss.

  1. Pre-training targets (Tables A, D).

At matched size, VICReg on the same bottleneck loses 0.038 mortality AUPRC, and SimCLR on concatenated embeddings loses 0.028. Raw reconstruction (MAE) ties on MIMIC-IV mortality. It trails in every seed on eICU (by 0.015–0.020) and phenotyping (by 0.002–0.004). MAE also needs decoders and vocabulary heads that latent prediction does not. Changing decoder or predictor depth moves AUPRC by at most 0.007 on MIMIC-IV.

  1. Missingness handling and label efficiency (Tables B, C, D).

Five encodings of absent streams are within 0.004 AUPRC on MIMIC-IV: learned empty-state token, zero token, omission, explicit indicator and mean fill. On eICU, appending a presence indicator to Zf adds 0.002 AUPRC on all stays and nothing significant on any missingness subset. Zf therefore already encodes which streams are missing. With this trivial handling, MLPA is ahead of MUSE+ and CRL-MMNAR on MIMIC-IV at full labels, and within 0.011 AUPRC on eICU. On eICU stays with two streams missing, MLPA reaches 0.292 vs 0.280 AUPRC and 0.820 vs 0.754 AUROC against MUSE+. With 1% of the labels, it is ahead of MUSE+ and CRL-MMNAR by 0.024 and 0.044 on eICU, and ahead of MUSE+ in all five label draws. On MIMIC-IV phenotyping the leads are 0.059 and 0.051.

Reviewer_Py9UOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MLPA, a self-supervised framework for learning patient representations from incomplete multimodal Electronic Health Records (EHR). It combines frozen modality-specific encoders, a multimodal bottleneck transformer, and learned empty-state tokens. Instead of reconstructing raw inputs, MLPA predicts masked latent representations. Experiments on MIMIC-IV and eICU show competitive performance against a masked autoencoder (MAE) with 42.4% fewer trainable parameters, as well as improved mortality classification compared to MUSE+.

Critical strengths

(1) The paper addresses an important problem in multimodal healthcare learning, particularly missing modalities and heterogeneous clinical data.

(2) The proposed architecture is reasonable, and the comparison with MAE provides useful evidence about latent prediction versus raw-input reconstruction.

(3) The experiments cover two large clinical datasets and investigate missingness patterns, representation quality, and attention allocation.

Critical weaknesses
  1. Evaluation fairness: MLPA and MAE use downstream probes trained through cross-validation on the held-out test set, whereas MUSE+ is trained on the original training set. This makes the comparisons difficult to interpret. Additionally, MUSE+ performs substantially worse than a medications-only baseline on MIMIC-IV (0.638 vs. 0.832 AUPRC), raising questions about its configuration and evaluation.

  2. Limited experimental reproducibility: Each model is pretrained only once, so the reported standard deviations do not account for pretraining variability. This is particularly important given the relatively small improvement over MAE on eICU (0.430 vs. 0.403 AUPRC).

  3. Insufficient architectural ablations: The paper does not adequately isolate the contributions of the bottleneck architecture, latent prediction, and learned empty-state tokens.

  4. Modality washout: The authors use attention allocation to demonstrate modality washout, but higher attention to medication tokens does not necessarily imply that other modalities are being ignored. The paper does not directly demonstrate how this imbalance negatively affects downstream performance.

Missing Citations

Azad et al. (2025), Addressing Missing Modality Challenges in MRI Images: A Comprehensive Review. Computational Visual Media. This survey provides relevant background on approaches for handling missing modalities in medical data.

Xiang et al. (2026), Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding. This work is closely related to self-supervised representation learning from incomplete EHR data and could help clarify MLPA's novelty.

Questions for authors

(1) Could you evaluate all methods under the same downstream training protocol and explain why MUSE+ performs worse than the medications-only baseline?

(2) Is the downstream 10-fold cross-validation grouped by patient? The MIMIC-IV test set contains multiple admissions from some patients, potentially introducing patient-level leakage if folds are not properly separated.

(3) Could you investigate modality washout through modality removal or perturbation experiments rather than relying primarily on attention distributions?

2 comments
AuthorsOpenReview ↗

Response to reviewer Py9U

Thank you. Your question about MUSE+ led us to an error in our baseline, which we report in Part 1 of the general response. Tables A–E are there as well.

W1 / Q1. Evaluation fairness, and why MUSE+ was below the medications-only baseline. You were right to flag this. A data-loader error mapped every medication code to the unknown token in our MUSE+ pipeline, so MUSE+ never saw the medication codes. With the loader fixed and hyperparameters selected on validation patients, MUSE+ reaches 0.903 AUPRC on MIMIC-IV mortality, ahead of the medications-only probe (0.892 on complete records; App. F). Tuning selected a contrastive weight of 0; the original configuration, with the corrected loader and its contrastive loss, reaches 0.887 (Part 1, App. C.5).

All methods now share one protocol: probes trained on training patients, selected on validation patients and evaluated once on test patients, with the same probes applied to every representation (Part 1). We withdraw the claim that MLPA outperforms MUSE+ on mortality in both datasets. At full labels, MLPA is ahead of MUSE+ on both MIMIC-IV tasks and on eICU AUROC, and behind it on eICU AUPRC (0.485 vs 0.496). With 1% of the labels it is ahead on eICU (0.315 vs 0.291) and on phenotyping (0.746 vs 0.687; Table B).

Q2. Patient grouping. The folds were admission-level. With patient-grouped folds, AUPRC changes by at most 0.007 (App. D). The new protocol no longer trains probes on test records at all.

W2. Pre-training variability. We now report three pre-training seeds; MLPA's s.d. is 0.002–0.004 AUPRC. The eICU gain over the MAE holds in every seed: paired AUPRC +0.015, +0.020 and +0.015, AUROC +0.005 to +0.008, all CIs excluding 0.

W3. Ablations. Table D isolates three components on MIMIC-IV:

  • Routing: MBT vs Perceiver queries, early fusion and late fusion.
  • Objective: MAE with matched decoder depth, SimCLR and VICReg.
  • Missingness handling: zero token, omission, indicator and mean fill.

The bottleneck is ahead of every alternative routing, by 0.014 / 0.012 AUPRC over Perceiver queries. The latent objective is ahead of SimCLR and VICReg by 0.028 and 0.038 on mortality. Against raw reconstruction it is ahead on eICU in every seed (+0.015 to +0.020) and on MIMIC-IV phenotyping (+0.002 to +0.004), and ties on MIMIC-IV mortality (Table A). The alternatives to the empty-state token perform the same, so MLPA does not depend on that choice.

W4 / Q3. Washout by removal and perturbation. Attention share is not contribution, and attention is now descriptive only. The new evidence comes from probes and interventions (Table E):

  • Removing medications at test time costs the Perceiver more mortality AUPRC than the MBT (−0.384 vs −0.327), and so does thinning them to 25% (−0.123 vs −0.096). Removing codes costs it less (−0.040 vs −0.063).
  • The Perceiver's representation has largely lost age and sex (AUROC 0.71 / 0.74 vs 0.96 / 1.00) while keeping ICD and radiology information.
  • Restoring age and sex recovers about 40% of the Perceiver's phenotyping deficit.

Downstream, the Perceiver is 0.014 / 0.012 AUPRC behind the MBT (Table D). Washout is real for the shortest stream and accounts for a measurable part of this gap (Section 4.6). Stream removal also separates the objectives: on eICU, MLPA stays ahead of the MAE in 11 of 14 removal patterns (App. F).

Missing citations. Both are added. Azad et al. (2025) is cited in the introduction, with missing-modality work in other domains. Xiang et al. (2026) is discussed in the related work: it learns from incomplete EHR time series by masked reconstruction of observed values under value-level missingness; we contrast it with MLPA's latent targets and whole-modality missingness.

We hope this resolves your concerns about fairness, pre-training variability, ablations and washout. If it does, we would be grateful if you reconsidered your score.

AuthorsOpenReview ↗

Reminder and request for engagement

Dear Reviewer, We hope that our response adequately responds to your feedback. If so please do engage with it and adjust your score if you agree with the points made. A lot of effort has been put into the rebuttals and submission, it would be good to see it acknowledged.

Reviewer_9tHJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

MLPA learns patient representations from health records with missing data types. It uses frozen encoders to process codes, medications, text, and demographics, then combines their information through a small set of shared tokens. During pretraining, the model uses one combined vector to predict the encoded features of hidden input positions. Missing data types are represented by learned placeholder tokens and are not included in the prediction loss.

On MIMIC-IV, MLPA performs about as well as an MAE using the same fusion backbone. On eICU, mortality AUPRC rises from .403 to .430, while trainable parameters fall from 3.3M to 1.9M. The paper also compares MLPA with MUSE+, individual data types, and simple feature concatenation, and examines missing data, attention, and clustering. Since the model uses the full hospital stay, the mortality task is retrospective classification, not early prediction. The main concerns are that MLPA and MUSE+ are evaluated using different supervised training setups, and that the paper does not clearly show whether its design handles missing data better or prevents longer data streams from dominating shorter ones.

Critical strengths
  1. The comparison between MLPA and MAE is clear and well controlled. They use the same frozen encoders, masking, missing-data tokens, and fusion structure. The main difference is what they predict and the decoder they use. On eICU, MLPA improves mortality AUPRC from .403 to .430 with 42.4% fewer trainable parameters, showing a useful gain in both performance and model size.

  2. The paper openly reports where the method works and where it falls short. It reports results for different missing-data groups and explains that their different outcome rates make direct comparisons between groups difficult. It also shows that MLPA falls behind MUSE+ when two data types are missing (.250 vs. .374 AUPRC). The authors clearly state that the mortality task is not early warning, attention patterns alone do not prove how the model works, and the reported variation does not cover repeated pretraining runs.

Critical weaknesses
  1. MLPA and MUSE+ are not evaluated under the same training setup. MLPA/MAE classifiers are trained on nine folds of the held-out set, while MUSE+ uses the original training set. This changes both the labeled data and the classifier, making it hard to tell whether MLPA’s better results come from better representations. MLPA/MAE also get separately trained classifiers for MIMIC subgroups, while MUSE+ does not. The paper does not say whether patients are kept separate across folds, and each model is pretrained only once. The main results should be checked using the same splits, classifiers, and label budgets, with repeated training runs.

  2. The paper does not clearly show why its main design choices are needed. Frozen-target prediction and MBT are existing ideas, but the paper does not compare with the closest method, CRL-MMNAR, or simpler missing-data options such as zero-filling and missingness indicators. The claim about preventing long streams from dominating mainly relies on attention scores, which MBT partly balances by design. This does not prove that useful information from shorter streams is preserved. The Perceiver comparison is also limited to complete MIMIC records and shows no clear downstream advantage. More direct tests are needed to support this claim.

Questions for authors
  1. Are the ten probe folds strictly grouped by patient? Could all classifiers be trained on the original training patients, selected on validation patients, and evaluated on untouched test patients? Does the MUSE+ gap remain when its frozen representations receive the same probe?

  2. What are the paired differences across independent pretraining runs for MLPA versus MAE and MUSE+? The 42.4% reduction counts fusion-stage trainable parameters; what are the parameter, runtime, and memory costs including unimodal pretraining?

  3. Does the actual text encoder use the TF-IDF sequence described in Appendix C.1/Table 12 or the clinical BERT checkpoint in Appendix G.4? Is the demographic encoder a fixed MLP or linear map? Please also reconcile the .8M frozen-encoder MUSE+ count with the .4M fine-tuned variant.

  4. If available, please provide matched results for empty-state tokens versus zero-filling/missingness indicators, whole-modality dropout, and CRL-MMNAR. Is there direct evidence beyond attention shares that the bottleneck retains useful information from short streams? These questions concern the current claims rather than additional clinical tasks.

2 comments
AuthorsOpenReview ↗

Response to reviewer 9tHJ

Thank you for the precise questions; we ran each check you proposed. Tables A–E are in the general response.

Q1 / W1. Protocol and the MUSE+ comparison.

  • Patient grouping. The ten folds were admission-level, not grouped by patient: 13–17% of fold-test records had a same-patient record in the training part. Re-running with patient-grouped folds changes AUPRC by at most 0.007 (App. D), so this did not drive the results.
  • Your proposed design. We adopted it. Every classifier is now trained on training patients, selected on validation patients and evaluated once on untouched test patients. Subgroup scores are slices of the single all-records probe, so MIMIC-IV subgroups no longer get separate probes. Table B repeats the comparison at matched label budgets (1–10%).
  • Same probe for MUSE+. With the corrected data loader (Part 1), MUSE+'s frozen embeddings under our MLP probe reach 0.898–0.899 AUPRC on MIMIC-IV mortality and 0.491–0.495 on eICU, close to its own classifier (0.903 and 0.496). Under this shared probe, MLPA is ahead of MUSE+ on both MIMIC-IV tasks and behind it on eICU AUPRC (0.485 vs 0.493; App. F). Across the 18 seed-matched comparisons with MUSE+, MLPA's only significant loss is eICU AUPRC in one seed. The large gap in the submission came from the loader error, not from the probe.

Q2. Paired differences across pre-training runs, and cost.

  • MLPA − MAE, per seed: eICU AUPRC +0.015, +0.020, +0.015 (all CIs exclude 0); MIMIC-IV phenotyping +0.004, +0.002, +0.002 (all exclude 0); MIMIC-IV mortality +0.001, −0.000, −0.005 (none exclude 0). Seed 0 is the submitted checkpoint; seeds 1–2 were trained for 100 epochs. Comparisons with MUSE+ and CRL-MMNAR are summarized in Part 2.
  • Cost, including uni-modal pre-training: the frozen encoders take 10.5 h (MIMIC-IV) and 11.8 h (eICU) to pre-train and are shared by every method, MUSE+ and CRL-MMNAR included. On top of that, MLPA uses 42% / 32% fewer pre-training parameters and 20% / 38% less peak GPU memory than the MAE, at about the same epoch time and the same inference size (Part 3).

Q3. Documentation.

  • Text encoder: a BERT-style transformer over TF-IDF token sequences, as in App. C.1 and its encoder table (Table 12 in the submission, Table 9 in the revision). The "clinical BERT checkpoint" sentence in App. G.4 was wrong and is corrected.
  • Demographics encoder: a fixed linear map, as in App. G.4. App. C.1 and the encoder table, which said MLP, are corrected. We also corrected the table's hidden sizes and depths to the logged configurations.
  • MUSE+ parameter counts: 0.8M refers to the frozen-embedding model used in all experiments. The 0.4M fine-tuned variant went through the same faulty loader and has been removed, so only the 0.8M model remains.

Q4. Alternatives and direct evidence.

  • Empty-state tokens vs zero filling, omission, mean fill and an explicit indicator: within 0.004 AUPRC on MIMIC-IV (Table D). On eICU, appending the presence indicator to Zf adds 0.002 AUPRC on all stays and nothing significant on the missingness subsets.
  • Whole-modality dropout: it lowers AUPRC by 0.005–0.009 on all eICU stays, on complete stays and on both MIMIC-IV tasks. Its gain on two-missing stays (0.272 → 0.305, one seed) is within MLPA's seed spread there (0.292 ± 0.018). We keep the original model (App. F).
  • CRL-MMNAR: trained on the same frozen embeddings (3 seeds). At full labels MLPA is ahead of it on both MIMIC-IV tasks. On eICU, CRL-MMNAR scores 0.484 with its own heads and 0.492 with our probe, against 0.485 for MLPA. With 1% of the labels MLPA is ahead on eICU by 0.044 AUPRC (0.315 vs 0.271).
  • Retention of short streams, beyond attention: the Perceiver's Zf predicts age decile and sex at AUROC 0.71 and 0.74, against 0.96 and 1.00 for the MBT. Restoring age and sex to the Perceiver recovers about 40% of its phenotyping deficit (Table E).

W2. The Perceiver comparison. The Perceiver is now pre-trained on all admissions, not only complete ones, and compared on downstream tasks: it is behind the MBT by 0.014 (mortality) and 0.012 (phenotyping) AUPRC. The washout claim now rests on the probes and interventions in Table E.

Presentation. We rewrote Sections 4.2–4.6 around these comparisons and fixed the inconsistencies you listed.

We hope this resolves your concerns about the protocol and the design choices. If it does, we would be grateful if you reconsidered your score.

AuthorsOpenReview ↗

Reminder and request for engagement

Dear Reviewer, We hope that our response adequately responds to your feedback. If so please do engage with it and adjust your score if you agree with the points made. A lot of effort has been put into the rebuttals and submission, it would be good to see it acknowledged.

Reviewer_qqdCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

MLPA learns patient representations from incomplete multimodal EHRs by predicting frozen encoder latents through a bottleneck transformer. Experiments on MIMIC-IV and eICU show competitive performance against a backbone-sharing masked autoencoder and MUSE+, with fewer trainable pre-training parameters.

Critical strengths
  1. Addresses a relevant problem using two substantial cohorts with naturally missing modalities.

  2. The backbone-sharing MAE provides an informative comparison between latent prediction and raw-input reconstruction.

  3. Clearly acknowledges retrospective mortality classification, severe-missingness failures, and limitations of attention analysis.

Critical weaknesses
  1. Core design benefits remain insufficiently isolated. Learned empty-state tokens lack comparison against simple missingness-aware alternatives. Moreover, MLPA and MAE use different predictor/decoder depths, complicating attribution to prediction targets alone. The parameter reduction concerns pre-training heads discarded at inference.
  2. Limited methodological novelty. MLPA mainly combines established ideas: frozen-encoder latent prediction, multimodal bottleneck transformers, and learned tokens for missing modalities. Although their integration into incomplete EHR modeling is practically motivated, the paper does not sufficiently establish a distinctive methodological advance over these existing components. More targeted ablations are needed to demonstrate what new benefit arises from their combination.
  3. Inconsistent empirical advantages. MLPA does not consistently outperform the evaluated baselines. On MIMIC-IV, MAE achieves higher phenotyping AUPRC/AUROC (0.828/0.821 versus 0.825/0.815) and slightly higher mortality AUROC (0.989 versus 0.987). Under two missing modalities on eICU, MUSE+ substantially outperforms MLPA in mortality AUPRC (0.374 versus 0.250). These results do not negate MLPA’s strengths, but they suggest that its advantages are setting-dependent. Given the limited methodological novelty, the current experiments do not yet provide sufficiently compelling evidence of a substantial advance.
Questions for authors

see in Critical Weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer qqdc

Thank you for the careful review. The general response (Parts 1–3) holds the tables we refer to below.

W1. Isolating the design.

  • Empty-state tokens vs simple alternatives. Zero token, omission, an explicit indicator and mean fill all land within 0.004 AUPRC of the empty-state token on MIMIC-IV (Table D). On eICU, appending an explicit presence indicator to Zf adds 0.002 AUPRC on all stays and nothing significant on the missingness subsets. MLPA is therefore robust to this design choice; we keep the token because it avoids imputation and per-sample branching.
  • Predictor vs decoder depth. On MIMIC-IV, an MAE with a 2-layer decoder, matching MLPA's predictor, performs like the 4-layer MAE, and a 4-layer MLPA predictor gives no consistent gain (Table D). Depth does not change the comparison there.
  • Parameters. You are right that the savings sit in pre-training heads discarded at inference. Inference size is identical, and epoch time is about the same. What MLPA saves is pre-training parameters (42% MIMIC-IV, 32% eICU) and peak GPU memory (20% / 38%), and its heads do not grow with vocabulary size. The abstract no longer claims 42.4% fewer parameters.

W2. Novelty. Each component has precedents. The combination, as a label-free objective for incomplete multi-modal EHR, is new to our knowledge, and the new experiments show what each part contributes:

  • Latent targets from frozen encoders, read out from one pooled vector: they are ahead of raw reconstruction on eICU in all three seeds (+0.017 AUPRC) and give the most label-efficient representation we tested on eICU and phenotyping (Table B), with no teacher, negatives or collapse regularizer.
  • Bottleneck routing: replacing it with Perceiver queries costs 0.012–0.014 AUPRC, and the Perceiver loses most of the two-token demographic stream (Table E).
  • Empty-state tokens: no effect on accuracy, as above; they avoid imputation and per-sample branching.

Prior methods for incomplete EHR either reconstruct inputs (our MAE; Xiang et al., 2026) or rely on labels (MUSE+; CRL-MMNAR, which reconstructs modality embeddings but trains jointly with supervised heads). MLPA needs no labels. At full labels it is ahead of both supervised methods on MIMIC-IV and 0.011 AUPRC behind MUSE+ on eICU; with 1–10% of the labels it is ahead of both on eICU and phenotyping (Table B).

W3. Consistency of the advantages.

  • MIMIC-IV, MAE vs MLPA. Under the new protocol, MLPA is ahead on phenotyping in all three seeds (AUPRC and AUROC +0.002 to +0.004) and ties on mortality.
  • Two missing modalities. The deficit was an artifact of the faulty MUSE+ loader. Against the corrected MUSE+, MLPA reaches AUPRC 0.292 vs 0.280 and AUROC 0.820 vs 0.754 (Table C).
  • Where the advantages are. At full labels, MLPA is ahead of MUSE+ on both MIMIC-IV tasks and on eICU AUROC, and behind it on eICU AUPRC (0.485 vs 0.496). Its advantages over the supervised methods are consistent in the two settings the paper targets. With 1–10% of the labels it is ahead of both on eICU and phenotyping (Table B). On stays with missing streams it ties or is ahead of both on every metric (Table C). The revised abstract and conclusion state these advantages specifically.

We hope this resolves your concerns about attribution, novelty and consistency. If it does, we would be grateful if you reconsidered your score.

AuthorsOpenReview ↗

Reminder and request for engagement

Dear Reviewer, We hope that our response adequately responds to your feedback. If so please do engage with it and adjust your score if you agree with the points made. A lot of effort has been put into the rebuttals and submission, it would be good to see it acknowledged.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response (1/3): what changed

We thank all four reviewers. Their comments led to substantial new experiments and a revised paper, and the submission is stronger for it. We hope the changes below answer your concerns.. Their concerns overlapped on four points: the evaluation protocol, single pre-training runs, missing ablations, and the evidence for modality washout. We addressed all four with new experiments and a revised PDF (changes in blue). While doing so we found and fixed an error in our MUSE+ baseline that changes one of our headline claims; we report it directly after the summary.

What the new results show. Without labels at pre-training, MLPA is the strongest representation we tested when labels are scarce on eICU and MIMIC-IV phenotyping. It is ahead of the MAE, which shares its backbone, on eICU in all three seeds. At full labels it is ahead of both supervised baselines on MIMIC-IV and 0.011 AUPRC behind MUSE+ on eICU. With two streams missing it ties the supervised baselines on AUPRC and is ahead of MUSE+ on AUROC (0.820 vs 0.754). Parts 2 and 3 give the numbers.

Correction: MUSE+ baseline. A data-loader error mapped every medication code to the unknown token in our MUSE+ pipeline, so MUSE+ never saw the medication codes. This is why it scored below a medications-only probe, as Reviewer Py9U noticed. With the loader fixed and hyperparameters selected on validation patients, MUSE+ reaches 0.903 AUPRC on MIMIC-IV mortality and 0.496 on eICU (3 seeds). Validation selection set its contrastive weight to 0 on all three tasks, so the tuned model trains without MUSE+'s contrastive objective. With its original configuration, contrastive loss included, the corrected MUSE+ reaches 0.887 and 0.488. App. C.5 reports the faulty pipeline (re-run under the new protocol), the original configuration and the tuned one side by side, and we compare against the stronger tuned model throughout. We withdraw the claim that MLPA outperforms MUSE+ on mortality in both datasets. At full labels, MLPA is ahead of MUSE+ on both MIMIC-IV tasks and on eICU AUROC, and behind it on eICU AUPRC (0.485 vs 0.496). The fine-tuned MUSE+ variant (old Table 17) and the clustering analysis (old Section 4.7) used the same pipeline and are removed.

New evaluation protocol. All results below come from one protocol:

  • Probes are trained on training patients, selected on a validation split (10% of training patients) and evaluated once on untouched test patients. Subgroup scores are slices of the single all-records probe.
  • The same two-layer MLP probe, our primary probe as in the submission (5 probe seeds), is applied to every frozen representation, including the MUSE+ and CRL-MMNAR embeddings. The supervised baselines are also reported with their own classifiers. Linear-probe results are in App. F.
  • Three pre-training seeds: seed 0 is the submitted checkpoint (200 epochs), seeds 1–2 were trained for 100 epochs. MLPA's seed s.d. is 0.002–0.004 AUPRC.
  • 95% CIs come from a patient-clustered bootstrap (B = 1000); paired differences use shared resamples.

The submitted protocol cross-validated probes on the test set with admission-level folds. A leakage audit with patient-grouped folds changes AUPRC by at most 0.007 (App. D), so leakage did not drive the old results; the new protocol removes the issue.

New baselines and ablations.

  • CRL-MMNAR (Liang et al., 2025), trained on our frozen embeddings with the authors' code.
  • Joint-embedding objectives: SimCLR on concatenated modality embeddings at matched size, VICReg on the MBT.
  • Routing: Perceiver latent queries, early fusion (full self-attention) and late fusion (per-stream pooling).
  • Depth: MLPA with a 4-layer predictor, MAE with a 2-layer decoder.
  • Missingness handling: zero token, omission, explicit indicator, mean fill, whole-modality dropout.
  • Washout tests: stream-information probes, test-time stream removal and thinning, restoring age and sex.

Claims revised in the PDF.

  • Parameters: inference size equals the MAE's. MLPA saves pre-training parameters (42% MIMIC-IV, 32% eICU) and peak GPU memory (20% / 38%), not inference cost.
  • Attention shares are no longer used as evidence; probes and interventions replace them (Part 3).
  • Fusion gain: Zf ties concatenation on eICU and is ahead of it on MIMIC-IV phenotyping, so we no longer claim a general gain over concatenation.

In the revised PDF: Section 4 is reorganized into comparison with baselines (4.2), label efficiency (4.3), missing modalities (4.4), component ablations (4.5) and washout (4.6); protocol, costs and full tables are in Appendices C–F.

AuthorsOpenReview ↗

General response (2/3): main results, label efficiency, missing streams

Table A. Full labels, test set, mean ± s.d. over 3 pre-training seeds. MLPA, MAE and concatenation use the MLP probe. MUSE+ and CRL-MMNAR use their own classifiers, trained on each endpoint. Concatenation needs no pre-training (one run).

ModelMIMIC pheno AUPRCAUROCMIMIC mort. AUPRCAUROCeICU mort. AUPRCAUROC
Concatenation0.7970.8170.8960.9900.4920.871
MAE0.800 ± .0010.817 ± .0000.910 ± .0010.991 ± .0010.468 ± .0010.864 ± .000
MUSE+ (tuned)0.800 ± .0000.818 ± .0000.903 ± .0030.990 ± .0010.496 ± .0020.865 ± .000
CRL-MMNAR0.795 ± .0000.814 ± .0000.896 ± .0020.989 ± .0010.484 ± .0000.866 ± .002
MLPA0.803 ± .0020.820 ± .0010.908 ± .0040.991 ± .0010.485 ± .0030.870 ± .001
  • MLPA vs MAE, paired per seed: eICU AUPRC +0.015 to +0.020 and AUROC +0.005 to +0.008, CI excluding 0 in all 3 seeds. MIMIC-IV phenotyping +0.002 to +0.004, CI excluding 0 in all 3 seeds. MIMIC-IV mortality: tie.
  • Against all baselines: with the same MLP probe on every representation, MLPA is significantly better in 40, tied in 30 and worse in 2 of 72 seed-matched comparisons (4 baselines × 3 tasks × 2 metrics × 3 seeds). Both losses are eICU AUPRC, against MUSE+ and CRL-MMNAR in one seed each.

Table B. Label efficiency (AUPRC, 5 label draws). MLPA, MAE and concatenation use the MLP probe; MLPA and MAE are pooled over 3 seeds. MUSE+ and CRL-MMNAR are trained on the same labeled patients; MUSE+ (tuned configuration) also receives the remaining training records as unlabeled patients. MUSE+ was not run at 5% for phenotyping.

Task, labelsMLPAMAEMUSE+CRL-MMNARConcat.
eICU mortality, 1%0.3150.2740.2910.2710.283
eICU mortality, 5%0.3850.3580.3770.3560.372
eICU mortality, 10%0.4160.3910.4040.3890.401
MIMIC phenotyping, 1%0.7460.7390.6870.6950.718
MIMIC phenotyping, 5%0.7770.773–0.7490.760
MIMIC phenotyping, 10%0.7850.7820.7570.7640.771
MIMIC mortality, 1%0.7670.7490.7660.6910.747
MIMIC mortality, 5%0.8520.8640.8510.8140.826
MIMIC mortality, 10%0.8700.8770.8610.8410.851

On eICU, MLPA is ahead of every baseline at 1–10% labels in all 5 draws (vs MUSE+ at 1%: +0.024, 95% CI [+0.017, +0.032]; AUROC 0.797, against at most 0.768 for the supervised baselines). On MIMIC-IV phenotyping it is ahead of every baseline at every fraction (+0.059 over MUSE+ at 1%). On MIMIC-IV mortality it ties MUSE+ and is ahead of CRL-MMNAR at every fraction (0.767 vs 0.691 at 1%).

Table C. Missing streams on eICU (AUPRC / AUROC, 3 seeds).

Stays (N; deaths)MLPAMAEMUSE+CRL-MMNAR
All present (24,467; 2,510)0.519 / 0.8740.500 / 0.8670.536 / 0.8740.518 / 0.869
One missing (10,356; 833)0.408 / 0.8650.395 / 0.8590.402 / 0.8560.404 / 0.864
Two missing (2,449; 149)0.292 / 0.8200.299 / 0.8230.280 / 0.7540.293 / 0.806

MUSE+ is ahead on complete stays (AUPRC +0.017). With one stream missing, MLPA has the highest AUPRC and AUROC of the four models. With two missing, it ties the supervised baselines on AUPRC and is ahead of MUSE+ on AUROC (+0.066, 95% CI [+0.039, +0.096]). The two-missing deficit reported in the submission (0.250 vs 0.374) was an artifact of the faulty MUSE+ loader and does not reproduce. When streams are removed at test time, MLPA stays ahead of the MAE in 11 of 14 removal patterns (App. F).

AuthorsOpenReview ↗

General response (3/3): components, washout, cost

Table D. Component ablations on MIMIC-IV (AUPRC, MLP probe). Variants are single runs of 100 epochs (VICReg: 65); the reference is MLPA seeds 1–2, also 100 epochs.

ComponentVariantMortalityPhenotyping
ReferenceMLPA (MBT, latent targets, empty-state tokens)0.9070.802
RoutingPerceiver latent queries0.8930.790
RoutingEarly fusion (full self-attention)0.9040.796
RoutingLate fusion (per-stream pooling)0.9060.798
ObjectiveMAE, 4-layer decoder (seeds 1–2)0.9090.800
ObjectiveMAE, 2-layer decoder0.9080.801
ObjectiveMLPA, 4-layer predictor0.9140.800
ObjectiveSimCLR on concatenated embeddings, matched size0.8790.793
ObjectiveVICReg on the MBT0.8690.759
Missing streamsZero token / omission / indicator / mean fill0.909–0.9110.800
  • The bottleneck is ahead of every alternative routing: Perceiver queries cost 0.014 / 0.012 AUPRC; early and late fusion cost up to 0.006 (up to 0.014 with the linear probe).
  • Depth does not change the objective comparison on MIMIC-IV: halving the MAE decoder or doubling the MLPA predictor moves AUPRC by at most 0.007.
  • SimCLR (matched size) and VICReg are 0.028 and 0.038 AUPRC behind MLPA on mortality.
  • MLPA does not depend on how missing streams are encoded: all four alternatives land within 0.004 of it. On eICU, appending an explicit presence indicator to Zf adds 0.002 AUPRC on all stays and nothing significant on any missingness subset, so Zf already encodes which streams are missing.

Table E. Modality washout tested by probes and interventions (MIMIC-IV). Attention shares do not show what a representation retains; they are now descriptive only (App. F). Differences are computed before rounding.

TestMBT (MLPA)Perceiver
Linear-probe AUROC from Zf: age decile / sex0.963 / 1.0000.709 / 0.736
Same, ICD sections / radiology terms (mean of 10 each)0.94 / 0.850.93 / 0.85
Mortality AUPRC change, medications removed (fixed probe)−0.327−0.384
Mortality AUPRC change, medications thinned to 25%−0.096−0.123
Mortality AUPRC change, codes removed (fixed probe)−0.063−0.040
Phenotyping AUPRC gain from appending age and sex to Zf+0.001+0.006
  • The Perceiver loses most of the shortest stream (age and sex, two tokens) and keeps ICD and radiology information as well as the MBT does.
  • Restoring age and sex recovers about 40% of the Perceiver's phenotyping deficit (0.790 → 0.795, vs 0.802) and none of its mortality deficit.
  • The Perceiver leans harder on the dense stream: removing or thinning medications costs it more than the MBT, while removing codes costs it less.
  • Washout is therefore real for the shortest stream and accounts for a measurable part of the routing gap (Section 4.6).

Cost (MIMIC-IV / eICU; App. C).

MLPAMAE
Trainable parameters, pre-training1.93M / 2.36M3.33M / 3.45M
Parameters at inference1.63M / 1.64M1.63M / 1.64M
Peak GPU memory, batch 643.39 / 1.20 GB4.24 / 1.95 GB
Epoch time, RTX 4090258 / 481 s253 / 471 s

Uni-modal encoder pre-training is shared by every method, including MUSE+ and CRL-MMNAR: 10.5 h on MIMIC-IV and 11.8 h on eICU. The MAE's output heads grow with vocabulary size (8,961 codes, 3,413 medications and 5,000 TF-IDF features on MIMIC-IV); MLPA's predictors output latents of at most 128 dimensions and do not.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.