Mitigating Modality Conflicts in Multi-modal Clustering with Decoupled Neighborhoods

Multi-modal clustering aims to discover underlying cluster structures by exploring information from different modalities. However, existing methods often struggle with conflicts across diverse modalities, where modality-specific information often leads to inconsistent topological relationships. To address this issue, we propose a novel neighborhood decoupling strategy that decomposes multi-modal graphs into a set of edge-disjoint graphs based on cross-modal edge frequencies. This is driven by the insight that edges repeatedly supported by multiple modalities exhibit higher semantic reliability. By stratifying the structural information into a hierarchy of decoupled neighborhoods according to edge frequencies, we introduce a structure alignment objective that enforces a structural ordering and injects the hierarchical topology into the modality-joint representation for clustering. Furthermore, we establish theoretical guarantees showing that optimizing this objective minimizes an upper bound on the $k$-means cost, ensuring compact cluster structures. Extensive experiments are conducted to verify the superiority of the proposed method compared with state-of-the-art approaches on multiple benchmark datasets.