Multimodal Fusion on the Boolean Lattice: From Local Context Dependence to Global Higher-Order Interactions

In multimodal fusion, a modality's marginal contribution can depend on the combination of modalities already present, which defines its context. Viewing fusion as a vector-valued set function over modality subsets shows that this context dependence is equivalent to higher-order interactions of order two or above. We therefore develop a formulation of multimodal fusion on the Boolean lattice of modality subsets, capturing global higher-order interactions by structuring local context dependence. Building on this formulation, we introduce LatFuse, which constructs fused representations for each subset of modalities in a bottom-up manner on the Boolean lattice, using single-modality extensions as reusable units for fusion. Across four benchmarks comprising three or four modalities and spanning multiomics classification, sentiment analysis, and semantic segmentation, LatFuse outperforms the strongest baselines on all task metrics. By constructing every subset state, LatFuse can infer directly on sublattices and is more robust than the baselines under most missing-modality conditions. On synthetic tasks with known interaction structures, LatFuse learns interactions that largely match the true interaction orders, supporting analysis of higher-order interactions.