acceptodds
Under review as a conference paper at ICLR 2027

Disentangled Representation Learning with Dual-Branch Enhancement and Adaptive Cross-modal Interaction

Abstract

Multimodal Sentiment Analysis (MSA) aims to capture human sentiment by jointly leveraging heterogeneous information from text, speech, and vision. While existing feature disentanglement methods partially separate modality-specific and modality-invariant representations, there remain two key limitations: (i) the disentangled representations lack targeted enhancement. Residual redundant temporal dependencies and intra-modal noise impair the discriminative capability, while insufficient cross-modal semantic interaction result in incomplete shared semantics, and (ii) the interaction features generated by cross-modal transformers are typically treated equally, which may amplify redundant and noisy dependencies while diluting the contribution of critical sentiment cues. To address these issues, we propose a disentangled representation learning framework with dual-branch enhancement and adaptive cross-modal interaction. Specifically, we first design a Dual-Branch Masked Representation Enhancement (DBMRE) module, which employs intra-modal masked attention and inter-modal masked attention to enhance modality-specific and modality-invariant representations, respectively. We then propose an Adaptive Cross-modal Interaction Gating (ACIG) mechanism, which dynamically estimates channel-wise importance weights based on the global semantic context of each interaction branch, selectively enhancing sentiment-relevant cross-modal dependencies while suppressing redundant interactions. Furthermore, we introduce multi-level knowledge distillation and dictionary-guided contrastive learning to provide auxiliary supervision at different representation levels. Extensive experiments demonstrate that our proposed framework achieves competitive performance. Furthermore, comprehensive ablation studies and training dynamics analyses validate the synergistic effectiveness of DBMRE and ACIG in enhancing both discriminative representation learning and model convergence.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.