TriHAR: Hierarchical Adaptive Routing of Representations, Modalities, and Cross-modal Interactions for Multimodal Clinical Prediction

Conventional multimodal fusion methods typically rely on fixed encoder outputs, overlooking that informative representation depths may vary across modalities and patients. They also inadequately capture patient-specific variations in direct modality and cross-modal interaction contributions. We propose TriHAR, a hierarchical adaptive routing framework that integrates EHR, ECG, and CXR data through three routing processes. TriHAR selects an informative encoder depth within each modality and independently routes the resulting modality-specific predictions and EHR-centered cross-modal interaction predictions. Experiments on ICU length-of-stay prediction across 27 encoder combinations show that TriHAR achieves the best average rank for Accuracy, AUROC, and Precision and remains effective when either ECG or CXR is unavailable. Further analyses reveal that preferred representation depths vary across modalities and encoder architectures, while modality-expert and interaction-expert routing exhibit distinct yet complementary prediction characteristics. These results demonstrate the effectiveness of hierarchical adaptive routing across heterogeneous multimodal clinical representations.