Counterfactual Sensitivity-Guided Intervention for Multi-modal Domian Generalization

Multimodal domain generalization (MMDG) aims to learn models from multiple source domains that generalize to unseen target domains without access to target data during training. A key challenge in this setting is that the predictive benefits of individual modalities may not persist across domains. Our analysis shows that a modality that supports fused predictions in source domains can become less useful or even detrimental in a target domain, highlighting the need to draw on alternative evidence when modality contributions change. To develop this capability using source data alone, we propose Counterfactual Sensitivity-Guided Intervention (CSI). CSI uses online, sample-specific logit gaps between complete and modality-removed inputs to identify modalities on which the current prediction strongly depends. Preferentially removing these modalities creates training views that encourage prediction from the remaining evidence. An annealed objective progressively increases the contribution of these views to training, while an exponential moving average teacher guides prediction from reduced evidence using stable distillation targets derived from complete inputs. Experiments on two public MMDG benchmarks demonstrate improved unseen-domain generalization and robustness to modality-specific perturbations, with additional loss-landscape analyses indicating flatter minima.