Selective Evidence Loss: Task-Relevant Missingness in Multimodal Sentiment Analysis
Abstract
Real-world multimodal systems can lose information selectively through content filtering and privacy protection. Existing incomplete multimodal learning mainly characterizes missingness by its amount and structure, leaving the task relevance of missing content underexplored. We introduce Selective Evidence Loss (SEL) to study task-relevant missingness under controlled deletion, and operationalize task relevance in multimodal sentiment analysis using contextual affective intensity. SEL causes greater performance degradation than random deletion under matched deletion amounts, and the gap persists after matching affected utterances and within-utterance deletion structure. This vulnerability is not consistently resolved by mechanism-matched training. We propose Selective Relational Distillation and Preservation (SRDP), which provides relational supervision for affected samples while preserving model behavior on unaffected samples. SRDP produces a single deployment checkpoint and requires only the corrupted multimodal input at inference. In the main evaluation, SRDP reduces affected-sample MAE for all ten models, with a model-average reduction of 0.012. These results show that robustness depends on what evidence is lost, highlighting task relevance as a key dimension of missing-information robustness.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.