- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This work introduces TriHAR, a multi-modal routing model for binary ICU length-of-stay prediction using EHR, ECG and CXR data. The framework designs three adaptive routing modules: R1 selects multi-layer intermediate representations from each modality encoder, R2 performs unimodal expert weighting, and R3 models EHR-centric cross-modal interactions. Predictions are fused from two complementary branches, one for unimodal evidence and one for cross-modal interactions. The main contribution is to dynamically pick appropriate encoder depths per patient-modality pair and separate unimodal and cross-modal reasoning with dual-branch MoE routing. The authors validate TriHAR on a filtered MIMIC subset with complete tri-modal inputs. They test over 27 combinations of modality encoders, compare against strong multimodal MoE baselines, and conduct mask-based missing-modality simulations. Additional analysis demonstrates complementary behavior between the two prediction branches.
Critical strengths
- The paper targets a practically meaningful problem in multimodal clinical prediction: existing models fix the encoder’s final-layer representation for fusion, ignoring patient- and modality-specific optimal representation depth and heterogeneous patient-level modal importance. The proposed TriHAR framework modularly separates direct modal evidence and cross-modal interaction evidence via three routing modules, which offers a conceptually clean design.
- The empirical evaluation is comprehensive: the authors test across all 27 combinations of 3 EHR / 3 ECG / 3 CXR encoders, alongside simulated missing-modality (bimodal) settings. They further analyze routing behaviors and demonstrate complementary predictive behaviors between the two prediction branches, providing interpretability insights beyond simple performance tables.
Critical weaknesses
1.Incomplete ablation fails to isolate the source of performance gains. The paper does not include three critical ablation controls: (1) R2+R3 without R1 (fixed top-layer encoder features); (2) concatenation of low/mid/high features as an alternative to R1’s Gumbel-Softmax selection; (3) R3 without EHR-centric cross-modal attention. Without these comparisons, we cannot verify whether the R1 adaptive layer-selection component contributes meaningfully, or whether all gains come purely from the dual MoE branches. This undermines the core technical claim of the work. 2. Limited dataset and evaluation design weaken generalization and conclusion robustness. All experiments rely on a single-site MIMIC subset filtered to only samples with complete EHR+ECG+CXR. Bimodal tests use mask-based simulated missingness rather than real clinically missing data, which has different data distribution. The binary label uses an arbitrary 3-day ICU stay cutoff with no sensitivity analysis. Moreover, evaluation heavily relies on average ranking instead of statistically verified absolute improvements, omits PR-AUC/F1, and does not compare against strong non-deep-learning baselines such as XGBoost. These issues limit confidence in the claimed advantages of TriHAR.
Questions for authors
1.Please provide ablation results for the three missing control experiments: R2+R3 without R1, multi-layer feature concatenation replacing R1, and non-EHR-centric cross-modal attention for R3. Please clearly answer: How much performance changes for each ablation case across all your 27 encoder combinations? Quantify the contribution of each component. 2.The bimodal experiments adopt mask-simulated missing-modal data on fully observed tri-modal patients. Please clearly answer: What distributional gaps exist between your mask-simulated missing data and real-world clinical missingness? Can you additionally run bimodal experiments using naturally-occurring real missing-modality patient samples from MIMIC and report the performance? If yes, how does the model performance differ compared with mask-simulated missing setting? 3. The paper uses average ranking as primary evidence. Please clearly answer: Can you report PR-AUC and F1 metrics and add comparisons against XGBoost/LightGBM baselines? 4.The routing analysis only provides aggregate statistics of expert weight and layer selection. Please clearly answer: Can you provide patient-level case examples to explain why the model selects a particular representation layer or expert for individual patients, instead of only reporting aggregated routing statistics? 5. The LOS prediction task uses a manually defined 3-day threshold for binary labels. Please clearly answer: How do your model’s quantitative results change if you alter this 3-day ICU stay cutoff? 6. The paper claims that TriHAR differs from prior multimodal MoE works such as I²MoE and FuseMoE. Please clearly answer: What is the core algorithmic distinction between your dual-branch routing design and the gating mechanism adopted by I²MoE and FuseMoE? Why cannot the gains observed in your experiments be reproduced by simply applying I²MoE/FuseMoE over multi-scale concatenated features?
5 comments
Author Response to Reviewer hmbm (2/3)
W2-2. PR-AUC, F1, and tree-based models.
We now report PR-AUC, recall, and F1 for all methods. Recall and F1 use the threshold maximizing validation F1 (t = 0.4; Rajkomar et al., 2018).
| Method | ACC | AUROC | PREC | PR-AUC | Recall | F1 |
|---|---|---|---|---|---|---|
| I²MoE | 0.629 | 0.651 | 0.609 | 0.530 | 0.620 | 0.545 |
| FuseMoE | 0.626 | 0.644 | 0.603 | 0.509 | 0.625 | 0.546 |
| DDMF | 0.622 | 0.642 | 0.602 | 0.521 | 0.616 | 0.540 |
| MulT | 0.627 | 0.653 | 0.608 | 0.534 | 0.606 | 0.542 |
| TriHAR | 0.635 | 0.656 | 0.615 | 0.531 | 0.650 | 0.538 |
For tree-based models, we trained XGBoost and LightGBM on the same EHR variables and data splits, and compare them with TriHAR on the highest combination of Table 1 and with the average of the two models' predicted probabilities:
| Method | Input | ACC | AUROC | PREC | PR-AUC | Recall | F1 |
|---|---|---|---|---|---|---|---|
| XGBoost | EHR | 0.657 | 0.668 | 0.639 | 0.555 | 0.554 | 0.541 |
| LightGBM | EHR | 0.658 | 0.673 | 0.638 | 0.565 | 0.579 | 0.554 |
| TriHAR | EHR+ECG+CXR | 0.650 | 0.683 | 0.633 | 0.557 | 0.661 | 0.532 |
| TriHAR + LightGBM (average) | EHR+ECG+CXR | 0.670 | 0.706 | 0.668 | 0.600 | 0.659 | 0.579 |
Thank you for this suggestion, which gave us a useful insight. Gradient-boosted trees are known to be strong on tabular data (Shwartz-Ziv & Armon, 2022), and they perform well on EHR alone here. However, they cannot take raw ECG waveforms or CXR images, so this is not a like-for-like comparison of fusion designs; the gap mainly reflects the tabular encoder, and TriHAR improves over its own EHR encoder on the highest, median, and lowest combinations (AUROC +0.017 to +0.114). More importantly, the two models are complementary: their predicted probabilities are only weakly correlated (r = 0.26 to 0.37), and averaging them improves over LightGBM in all metrics (last row; AUROC 0.689 to 0.706 vs. 0.673 across the three combinations), consistent with Shwartz-Ziv & Armon (2022).
References: Rajkomar et al. (2018). Scalable and accurate deep learning with electronic health records. npj Digital Medicine, 1, 18. Shwartz-Ziv & Armon (2022). Tabular data: Deep learning is not all you need. Information Fusion, 81, 84–90.
W2-3. Sensitivity to the 3-day cutoff.
The class balance changes sharply with the cutoff. We retrained TriHAR with 2- and 5-day cutoffs on the highest, median, and lowest combinations of Table 1 (three runs each):
| Cutoff | Long stay (test set) | Short : long | AUROC (high / med / low) | Mean AUROC |
|---|---|---|---|---|
| 2 days | 56.7% | 0.76 : 1 | 0.646 / 0.642 / 0.625 | 0.638 |
| 3 days | 38.8% | 1.57 : 1 | 0.683 / 0.655 / 0.629 | 0.656 |
| 5 days | 22.0% | 3.55 : 1 | 0.672 / 0.656 / 0.622 | 0.650 |
Sensitivity is moderate: AUROC stays within 0.62 to 0.68, mean AUROC changes by at most 0.018, and the ordering of combinations is unchanged, so our conclusions do not depend on the cutoff. We use 3 days not because it performs best, but because it yields a meaningful prediction target. LoS > 3 days is a standard MIMIC task (Wang et al., 2020; Susetzky et al., 2025), and the alternatives degenerate in opposite directions. At 2 days, close to the median ICU stay, long stay becomes the majority class and the model labels most patients as long stay at the default threshold (recall 0.72 to 0.85). At 5 days, it labels almost all patients as short stay (recall 0.04 to 0.39), and its accuracy equals that of always predicting short stay; higher cutoffs make this imbalance more extreme. The 3-day cutoff keeps long stay a clinically meaningful minority class with enough positives for stable evaluation.
References: Wang et al. (2020). MIMIC-Extract: A data extraction, preprocessing, and representation pipeline for MIMIC-III. ACM CHIL. Susetzky et al. (2025). A holistic time-aware classification model for multimodal longitudinal patient data. MICCAI.
W2-4. Single-site cohort and average ranks.
We agree that our study is limited to a single MIMIC cohort and a single LoS task; to our knowledge, no EHR–ECG–CXR resource independent of these MIMIC databases currently exists, as stated in our conclusion. The 27 encoder combinations therefore evaluate robustness to the encoder choice, not generalization across datasets. Beyond average ranks, we report the absolute mean of all six metrics for every method in W2-2, where TriHAR has the highest ACC, AUROC, PREC, and recall.
Author Response to Reviewer hmbm (3/3)
Q1. Three ablations.
We addressed this in detail in W1-1 to W1-3, with results on all 27 combinations for Fixed-High and Level Concat and on the highest-performing combination for the CXR and ECG anchors.
Q2. Mask-based vs. natural missingness.
We addressed this in detail in W2-1.
Q3. PR-AUC, F1, and XGBoost/LightGBM.
We addressed this in detail in W2-2.
Q4. Patient-level routing cases.
Each test patient follows an R1 path, defined by the levels selected for EHR, ECG, and CXR. Across all 81 models (27 combinations × 3 runs), paths with higher long-stay rates consistently show lower bicarbonate (mean r = −0.55; same direction in 93% of models), higher WBC (+0.53; 91%), higher anion gap (+0.46; 88%), lower systolic BP (−0.43; 88%), higher respiratory rate (+0.39; 82%), and higher BUN (+0.32; 78%), while sex shows no association (0.00). In a representative model (FTTrans / 1D-ResNet / ResNet18), grouping patients by the selected EHR level gives clinically distinct subgroups:
| EHR level | n | Long stay | Bicarbonate | BUN | Creatinine | WBC | Systolic BP |
|---|---|---|---|---|---|---|---|
| High | 641 | 34.9% | 23.7 | 21.0 | 1.20 | 9.7 | 128 |
| Low | 335 | 37.3% | 24.3 | 22.8 | 1.11 | 11.5 | 116 |
| Mid | 525 | 44.8% | 21.1 | 38.9 | 1.99 | 15.8 | 111 |
Comparing two patients from this model makes the pattern concrete. They were chosen by a predefined rule: the correctly classified patient closest to the median clinical profile of the lowest-risk and the highest-risk path.
| Patient A | Patient B | |
|---|---|---|
| Outcome | Short stay | Long stay |
| R1 path (EHR / ECG / CXR) | High / Mid / Mid | Mid / Low / High |
| Long-stay rate of the path | 27.0% | 48.6% |
| R2 / R3 / final probability | 0.26 / 0.38 / 0.32 | 0.73 / 0.41 / 0.57 |
| Age | 50s | 50s |
| Resp. rate, anion gap, BUN | Normal | Elevated |
The two patients are of similar age, but Patient B shows tachypnea, an elevated anion gap, and elevated BUN, the same acuity markers that separate high- and low-risk paths across models. R1 routes the two patients through different depths in all three modalities, and their paths belong to subgroups with different long-stay rates (27.0% vs. 48.6%). Patient B also shows how the branches complement each other: R3 underestimates the risk (0.41), R2 assigns a high probability (0.73), and the fused prediction (0.57) is correct. Which level corresponds to higher acuity is learned per model, so we interpret these findings descriptively rather than causally.
Q5. Changing the 3-day cutoff.
We addressed this in detail in W2-3.
Q6. Difference from I²MoE and FuseMoE.
Both apply a single stage of expert gating on a fixed encoder representation. TriHAR (i) first selects a representation depth per modality and patient (R1), and (ii) separates modality-specific evidence (R2) from EHR-anchored interaction evidence (R3) in parallel branches. As shown in W1-2, Level Concat, which feeds multi-scale concatenated features into the same routing, does not improve over TriHAR, so the gains are not explained by richer multi-scale features alone.
References: Xin et al. (2025). I²MoE: Interpretable multimodal interaction-aware mixture-of-experts. ICML. Han et al. (2024). FuseMoE: Mixture-of-experts transformers for fleximodal fusion. NeurIPS.
We sincerely thank the reviewer for the thorough and constructive review. The requested controls, metrics, and analyses substantially strengthened the paper, and we hope our responses address your concerns. We would be glad to answer any further questions.
Author Response to Reviewer hmbm (1/3)
Thank you for the thorough review and for the insightful comments. Your requests for additional experiments were especially important and valuable to us. We carried out all of the requested experiments and incorporated the main results into the revised paper, which strengthened its contributions and made the evaluation more rigorous. We answer the weaknesses first and then each question; our response is split into three comments (W1 and W2-1; W2-2 to W2-4; Questions).
W1-1, W1-2. R2+R3 without R1, and multi-level concatenation.
We evaluated Fixed-High (R1 removed, top-level encoder features) and Level Concat (R1 replaced by the concatenation of low, mid, and high features) on all 27 combinations:
(Mean over 27 combinations. TriHAR: ACC, AUROC, PREC from Table 1. Recall and F1 at t = 0.4.)
Removing R1 lowers ACC / AUROC / PREC by 0.012 / 0.006 / 0.003, and replacing it with concatenation lowers them by 0.017 / 0.007 / 0.013 while adding 0.07M parameters. The margins are modest, but R1 is the entry point of TriHAR: both R2 and R3 operate on the representations it selects. Without R1, the same R2 and R3 routing performs at the level of the strongest baselines (0.623 / 0.650 / 0.612 vs. 0.629 / 0.653 / 0.609), whereas with R1 TriHAR is the best method in all three metrics. We revised Sec. 4.2.4 to state this role of R1.
W1-3. Non-EHR-centric interaction.
We replaced the EHR anchor of R3 with a CXR anchor (CXR↔EHR, CXR↔ECG, CXR enhancement) and an ECG anchor (ECG↔EHR, ECG↔CXR, ECG enhancement). Results on the combination with the highest TriHAR performance in Table 1 (TabTrans / WaveNet / VGGNet, three runs):
The interaction branch also works with other anchors, while the EHR anchor is better in most metrics on this combination. We keep EHR as the anchor because (i) EHR is the modality most likely to be available for every ICU patient and is shared by all configurations we consider (trimodal, EHR+ECG, EHR+CXR), (ii) it matches clinical practice, where ECG and CXR findings are interpreted in the context of the patient's state, and (iii) it gives the best results on the highest-performing combination.
W2-1. Mask-based bimodal experiments.
We agree that our bimodal setting does not model real-world missingness. It is a controlled evaluation: both bimodal settings use exactly the same 7,504 trimodal patients and splits as the main experiments, and only the modality configuration changes. The excluded modality is masked and never used in training or inference, and the bimodal baselines are native two-input models. Regarding the distributional gap, patient groups defined by naturally available modality pairs differ from our trimodal cohort only moderately (first ICU stay per patient):
Our cohort has a somewhat higher long-stay rate (38.9% vs. 34.9% and 33.8%), with a similar mean age. The gap is larger for patients who lack the third modality (long stay 33.5% for EHR+ECG only and 20.1% for EHR+CXR only), which is the case-mix shift that a naturally bimodal cohort would introduce. Because the trimodal cohort is close to the overall EHR+ECG and EHR+CXR populations, the controlled setting is a reasonable test of TriHAR under different modality configurations, and it isolates the effect of the input modalities from changes in case mix.