HiCTE: Hierarchical Cross-Expert Transferability Estimation for Incomplete Multimodal Representation Learning

Incomplete multimodal representation learning aims to construct robust representations from partially observed inputs. Existing expert-based methods learn joint representations through cross-expert interaction, but typically transfer entire observed representations to different target experts. Such coarse-grained transfer overlooks target-specific transferability differences at the latent-dimension level and sample-specific transfer requirements. To address these limitations, we propose the Hierarchical Cross-Expert Transferability Estimation (HiCTE) framework. Specifically, HiCTE models each observed modality in a probabilistic latent space and derives reliability from the latent variance. A global estimator then provides initial transferability estimates for each target expert using the latent information and reliability. These estimates are further refined for each sample through local iterative optimization during training, with gradients propagated through the updates to train the global estimator. Based on these estimates, HiCTE constructs cross-expert representations for each target expert and a residual representation, then integrates them with the self representation through adaptive weighting. Extensive experiments on three multimodal benchmarks demonstrate the effectiveness of HiCTE under different missing protocols.