Exploring Modality Competition in Unsupervised Multimodal Anomaly Detection

Unsupervised Multimodal Anomaly Detection (MAD) identifies anomalies by modelling normality from normal multimodal data and leveraging cross-modal complementarity. However, modality competition remain an issue during training: one modality continually dominates the other. This directly hinders normality modelling on the suppressed modality, making it hard to distinguish anomalies on that modality and leading to inaccurate MAD. Worse still, when anomalies in the dominant modality remain undetected due to their similarity to normal features or their small area, the accuracy of MAD further degrades. We approach this problem from a Mixture-of-Experts (MoE) perspective. To better learn multimodal normal features, we propose Hindsight Learning (HL), a novel MoE training paradigm that constructs a post-hoc routing teacher from observable per-expert modal normality reconstruction advantages using Optimal Transport (OT) with expert capacity constraints. The teacher explicitly trains the router to learn which modality token is sent to a particular expert. We then introduce a Debt-aware MoE (Demo) based on HL to mitigate modality competition by preventing starvation in per-modality normality reconstruction utility. Specifically, each modality maintains a virtual debt queue, where debt accrues when its normality reconstruction utility falls below a minimum service objective in the queue. To reduce debt, the router is trained via debt-aware OT to preferentially serve indebted modality. We theoretically prove a long-run modality-service guarantee. Empirical results show Demo surpasses state-of-the-art methods by 2.4% AUROC on MAD benchmarks and remains effective in continual learning.