- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes LongMoE for clinical prediction when visits are irregular and some modalities are missing at each visit. It has four parts:
- Per-modality encoders.
- An imputation step. A missing modality is filled with its last observed embedding, or, if there is none, with an MLP applied to the mean of the observed embeddings at that visit.
- A tokenizer. It applies self-attention across modalities and adds a multi-frequency sinusoidal encoding of the visit time.
- A causal 2-layer Transformer over the visits, followed by a sparse MoE with hard top-1 routing and “shared” experts. Predictions are combined across visits, weighted by router confidence.
Experiments cover ADNI and OASIS-3 (CN/MCI/AD staging) and MIMIC-IV (one-year mortality), across all 2^L − 1 modality combinations. Large gains are reported when modalities are missing, especially in settings without clinical scores.
Critical strengths
1. The full-trajectory masking control (Appendix A.9.1, Table 14).
This is the paper's strongest evidence. When the excluded modality is withheld at every visit, so the model cannot carry earlier observations of it forward, LongMoE still keeps large margins on ADNI (B: 0.862 vs. 0.642; G: 0.852 vs. 0.639; I: 0.875 vs. 0.714). It also leads on three of four sparse single-modality settings on MIMIC-IV. The authors report that the advantage disappears on OASIS-3 rather than leaving that out.
Why this matters: my main concern is that the headline gains come from LongMoE seeing history the baselines do not. Table 14 is the only result that partly addresses that concern, and it suggests the longitudinal design helps even with the information gap reduced. It is the main reason I see a viable paper here. However, its comparison column reuses the same baselines whose numbers look broken (e.g., Lee-MMGRU at 0.5000 ± 0.0000), so it cannot yet support acceptance by itself.
2. A relevant problem with a broad evaluation.
The paper tackles missing modalities and irregular visits together, a combination that is common in clinical cohorts. It evaluates on three datasets covering two task types (neurodegenerative staging and ICU mortality), every one of the 2^L - 1 modality combinations, five seeds and three metrics, and lists every model's hyperparameters (Appendix A.12).
Critical weaknesses
1. Baseline protocol. Section 5 (l. ~298–302) says Flex-MoE and FuseMoE were augmented with visit history, but l. 320–321 says visit-independent baselines lack history and the Table 14 caption says static baselines receive one row per patient. Which is correct? Could all baselines be re-run with the same current-visit masking protocol LongMoE uses (prior visits intact)?
2. History-only controls. Could you report (a) LongMoE with the current visit fully masked, and (b) simple longitudinal baselines on the same history (e.g., last observation carried forward + logistic regression, or predicting the previous diagnosis), in particular for the ADNI B-only and G-only rows where the gains are largest?
3. Clinical variables and leakage. Which ADNI clinical variables are retained after "label-leaking columns" are excluded (l. ~276)? Are prior diagnoses, CDR-SB, or diagnosis-derived fields in the history? How are the OASIS-3 CN/MCI/AD labels defined, and is CDR an input at the evaluated visit? For MIMIC-IV, is the prediction made at admission or discharge of the final visit, are in-hospital deaths included as positives, and could final-visit notes/ICD codes carry end-of-life information (hospice discharge, palliative care codes)? Were ICD-10 admissions dropped or mapped to ICD-9?
4. Baseline and metric anomalies. How do you explain:
- Lee-MMGRU scoring 0.5000 ± 0.0000 AUC on ADNI B, G and B,G, and identical values for C / B,C / C,G / B,C,G (all 0.9704) and I / I,B / I,G / I,B,G (all 0.5771)?
- MulT, Flex-MoE and FuseMoE at ~0.50 AUC and 0.3333 accuracy on OASIS-3 from CDR/MMSE alone, while Lee-MMGRU reaches 0.88?
- Macro-F1 of 0.0000 in Table 11? On ADNI, Lee-MMGRU's 0.2047 is exactly the always-MCI macro-F1, so the metric is computed correctly there; on OASIS-3 no constant predictor can score below ≈0.085, which also rules out the 0.0131 and 0.0240 entries.
- Cross-table conflicts: w/o LE ΔAUC −0.0335 (Table 5) vs. −0.001 (Table 15); LongMoE ADNI F1 0.8064 (Table 5) vs. 0.8843 (Table 9); OASIS-3 F1 0.6537 (Table 16) vs. 0.4886 (Table 11); identical w/o LE and w/o Imp-ZP rows in Tables 15–17?
Questions for authors
-
Baseline protocol. Section 5 (l. 296-299) says Flex-MoE and FuseMoE were augmented with longitudinal history (a mean-pooled summary of preceding visits and a GRU over visits, respectively). The Table 14 caption says the static baselines receive one row per patient. Which is correct? Could you re-run all baselines with the same visit history that LongMoE receives?
-
History-only control. Could you report (a) LongMoE given only prior visits, with the current visit fully masked, and (b) simple longitudinal baselines using the same history, such as carrying the last observation forward plus logistic regression, or predicting the previous diagnosis? These would show how much of the gain comes from the architecture rather than from access to history.
-
Clinical variables and leakage. Which clinical variables are used for ADNI and OASIS-3 (l. 277 says only that "label-leaking columns [were] excluded")? Are prior-visit diagnoses, CDR-SB, or any fields derived from the diagnosis available as history? For MIMIC-IV, when is the prediction made relative to the one-year mortality window? Could notes or ICD codes at the final visit encode end-of-life information, such as hospice discharge or palliative care codes?
-
Baseline and metric anomalies. How do you explain:
- Lee-MMGRU scoring exactly 0.5000 ± 0.0000 AUC on every ADNI combination without C or I, and giving identical results for C, B,C, C,G and B,C,G?
- MulT, Flex-MoE and FuseMoE scoring about 0.50 AUC on OASIS-3 from CDR/MMSE alone, while Lee-MMGRU reaches 0.88 on the same input?
- The macro-F1 values of 0.0000 in Table 11, which are impossible for a 3-class task?
4 comments
LongMoE is best for all 15/15 combinations under both protocols. Its mean advantage over the capacity-matched Temporal Flex-MoE is +0.0099 AUROC under current-visit masking and +0.0168 AUROC under full-trajectory masking.
These results show i) the temporal baseline reaches 0.92 to 0.94 on B-only and G-only under current-visit masking, against 0.49 to 0.64 for the static baselines in Table 1. Most of the headline gap in Table 1 was therefore due to access to history; ii) a consistent but much smaller advantage remains under identical information. We also trained GRU and Transformer controls that share LongMoE's imputer (Reviewer VRho, Q1). These are comparable to LongMoE, which locates the remaining advantage in the history-based imputation and not in the MoE head. Because the remaining methods are intrinsically cross-sectional, longitudinalizing them requires adding a non-unique temporal architecture. They will be labelled as static references and will not be described as history-matched.
Q2: (a) Current visit fully masked On fixed cohorts containing at least five visits, we removed all time-varying modalities from the final visits; is the requested history-only setting.
| Dataset | Natural AUROC | History only () | AUROC, | |||
|---|---|---|---|---|---|---|
| OASIS-3 | 0.7648 | |||||
| MIMIC-IV | 0.8517 |
History alone therefore carries most of the signal on these tasks.
(b) Simple longitudinal baselines We have not run last-observation-carried-forward with logistic regression. The closest existing evidence is the matched GRU and Transformer controls, which receive the same history and imputer as LongMoE and perform comparably (Reviewer VRho, Q1). Together with (a), this indicates that access to history, and not the LongMoE architecture, accounts for most of the gain in the B-only and G-only rows.
Q3:
| Dataset | Clinical inputs | Label | Diagnosis-derived inputs |
|---|---|---|---|
| ADNI | MMSE, ADAS-13, CDR-SB, FAQ | First valid diagnosis (see below) | Diagnosis columns excluded; CDR-SB retained |
| OASIS-3 | MMSE and age at visit | Derived from global CDR at the last visit | CDR global, CDR sum of boxes, CDR sub-scores, and prior labels excluded |
| MIMIC-IV | Laboratory means, discharge ICD categories, discharge summary, age and sex | Death within 365 days after the final discharge | End-of-life codes can be present |
For ADNI, diagnosis columns are not model inputs. The preprocessing uses the first valid diagnosis as the label while retaining later visits, and 77.8% of patients have post-label observations. The submitted ADNI task is therefore retrospective classification of the initial diagnosis from the full record, not staging at the final visit as Section 5 states. This also explains the nearly flat current-visit masking curve on ADNI, since the visit to which the label refers remained in the history.
For OASIS-3, the clinical modality contains only MMSE and age at visit. The description “CDR/MMSE assessments” in Section 5 has been corrected in the uploaded revised paper. CDR is the source of the label and is not an input at any visit.
For MIMIC-IV, prediction in the submitted pipeline is made after the final discharge, and the label is death within the following 365 days. The diagnosis input includes ICD-10 coded admissions, so the description “ICD-9 codes” in Section 5 has been corrected. End-of-life information can be present:
| Evidence | One-year deaths | Survivors |
|---|---|---|
| Z66 (do not resuscitate) at the final admission | 20.2% | 2.1% |
| Z51 (includes palliative care) at the final admission | 12.6% | 0.8% |
Final-admission ICD codes alone reach 0.880 AUROC.
Q4:
Lee-MMGRU: its stage-2 L1 logistic regression was fitted once on the full-modality representation and only masked at evaluation. Across all three ADNI seeds, it selected zero coefficients for every biospecimen and genetics feature; imaging retained 11, 8, and 0 coefficients. Consequently, B/G-only predictions are constant (AUROC 0.5), and adding B or G to C or to I cannot change the prediction, producing the identical rows (0.9704 and 0.5771). We will refit stage 2 separately for each modality combination and add modality dropout.
OASIS-3 baselines: the stored mask order is [MRI, clinical, genetics], whereas MulT, Flex-MoE, and FuseMoE read it alphabetically as [clinical, genetics, MRI]. Under C-only evaluation, the clinical embedding is therefore multiplied by the MRI flag and erased. Lee-MMGRU uses a separate correctly indexed loader, explaining its 0.88 AUROC. The affected static-baseline results are invalid and will be rerun using name-indexed masks, with unit tests for mask alignment in every loader.
Table 11 F1: these values are not macro-F1. They are AD-vs-rest positive-class F1 at a validation-selected 99%-specificity threshold. A value of zero occurs when no positive test sample exceeds that threshold, and the 0.0131 and 0.0240 entries arise in the same way. We will label this metric explicitly and separately report true three-class macro-F1 from argmax predictions.
Thank you for the extensive rebuttal. However, after re-reading the manuscript and your response, I would like to maintain my score.
We thank the reviewer for the valuable feedback. We have addressed all comments and updated the paper accordingly wherever possible.
Questions For Authors:
Q1: Table 14 is correct for the submitted results: TF, MulT, Flex-MoE, and FuseMoE received one cross-sectional row per patient. For a fair history-matched control, we temporalized Flex-MoE, the closest missing-modality MoE baseline, using its own encoders, missing-modality bank, experts, and routing at each visit, followed by a temporal Transformer. It receives exactly the same patients, visit histories, masks, splits, and five seeds as LongMoE, and reuses no LongMoE components. Results are five-seed AUROC on ADNI, mean ± standard deviation. I, B, C, and G denote imaging, biospecimen, clinical, and genetic modalities.
Current-visit masking Only the listed modalities are available at the prediction visit; every model receives the same naturally observed preceding history.
Full-trajectory masking Only the listed modalities are retained throughout the record, giving every model the same restricted longitudinal information.