acceptodds
Under review as a conference paper at ICLR 2027

MARKER: Missing-Aware Low-Rank Expert Routing for Incomplete Multimodal Learning

Abstract

Missing modalities commonly arise in real-world applications due to device failures, network instability, or privacy concerns. Recent studies address this issue with prompt-based fine-tuning. However, most of these prompt-based methods struggle with two aspects: (1) they rely on scenario-conditioned prompt sets and require explicit prompt selection for different modality-availability conditions, leading to a less unified adaptation pipeline; (2) most methods are modality-missing-aware but instance-agnostic, while a few recent approaches achieve instance-level adaptivity through retrieval-based mechanisms at the cost of additional components and increased system complexity. To address these issues, we propose MARKER, a Missing-Aware Low-Rank Expert Routing framework for incomplete multimodal learning. Specifically, we insert a LoRA-based mixture-of-experts module into a pretrained multimodal backbone. Trained on data with missing modalities, MAIL dynamically routes each input to a sparse combination of LoRA experts, conditioned on both the missingness pattern and instance-specific features, enabling adaptive prediction without maintaining per-scenario prompt banks. Importantly, this design decouples adaptation from per-pattern prompt specification, providing a unified mechanism across varying missingness conditions. We further conduct ablation studies on routing strategies and layer-wise expert-allocation shapes to assess their impact under different modality-missing scenarios. Extensive experiments on three real-world datasets show that MAIL consistently outperforms 11 strong baselines, demonstrating its effectiveness and robustness for incomplete multimodal learning.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.