TWIN-MoE: Two-stage Intra-Expert Imputation Network for Incomplete Multimodal Learning

Multimodal learning is often deployed under the assumption that all modalities are fully observed, yet real-world data frequently violates this assumption, leaving some modalities missing. Recent Mixture-of-Experts (MoE) approaches address missing modalities either by routing each missingness pattern to a dedicated expert, which scales poorly as the number of modalities grows, or by imputing the missing modality from representations contributed by multiple experts, which requires a costly retrieval mechanism. We argue that such unconstrained inter-expert imputation can be sub-optimal, as it aggregates features from distinct routing-induced semantic regions and may introduce information unrelated to the target modality. Building on a geometric interpretation of sparse MoE routing through expert-specific Voronoi regions, we show that routing induces expert-conditioned representation neighborhoods and derive an imputation-error bound controlled by their local feature dispersion. When these expert-conditioned neighborhoods are more compact than the global representation space, intra-expert aggregation admits a tighter worst-case reconstruction bound than unconstrained global aggregation. Guided by this analysis, we propose TWIN, a two-stage intra-expert imputation framework. Pre-imputation generates a context-conditioned estimate in the encoder embedding space and radially soft-clips it using modality-specific statistics maintained in an observed FIFO buffer, this ensures that the pre-imputed modality stays in empirical observed distribution. Post-imputation refines the missing representation after MoE by aggregating expert-conditioned observed representations within the same sample. Across four missing-modality benchmarks under natural, random, and asymmetric dropout protocols, TWIN achieves state-of-the-art performance while using substantially fewer parameters than competing MoE-based imputation methods.