Geometry-Aware 2D-3D Multimodal Fusion for 3D Segmentation

Multimodal 2D-3D segmentation leverages complementary 2D semantic priors and 3D geometric structures for scene understanding. However, geometric inconsistency across the 2D-3D fusion pipeline remains a key challenge. To address this challenge, we propose GAMFusion, a geometry-aware 2D-3D multimodal fusion framework designed to preserve geometric consistency throughout the 2D-to-3D feature flow. First, we construct an extensible parallel fusion topology that decouples heterogeneous modality pathways and introduces pixel-wise adaptive interaction to mitigate feature interference caused by modality discrepancies. Second, we introduce a corrective feature consistency alignment module that restores local surface structure before voxelization using geometric consistency constraints. Finally, we design a normal-aware 3D contextual fusion mechanism that incorporates surface normal constraints into neighborhood aggregation to reduce cross-surface feature leakage caused by Euclidean distance-based retrieval. Extensive experiments on ScanNet200, S3DIS, and AI2THOR demonstrate the effectiveness of the proposed method, with strong improvements over existing approaches, particularly on boundary-sensitive metrics.