Escaping the Texture Trap: Intervention-Guided Adaptive RGB–Geometry Fusion for Indoor 3D Segmentation

Robust indoor RGB–geometry segmentation can fail when misleading appearance conflicts with valid geometric evidence. We introduce an intervention-guided framework that couples geometry-preserving texture interventions with input-adaptive fusion. Structured Texture Swapping (STS) creates paired clean and intervened observations while preserving labels and point correspondence. A Geometry–Texture Interaction Gated Unit (GT-IGU) predicts point- and channel-wise routing weights from ordered cross-modal evidence. Candidate-Oracle Routing Distillation (CORD) further supervises GT-IGU using label-aware, candidate-relative routing targets. Structure-Aware Texture-Swapping Invariance (SA-TSI) regularizes intervention-induced feature changes over compatible neighborhoods. All teacher and graph operations are training-only. On S3DIS and ScanNetV2, the system improves affected-region performance under synthetic texture conflicts across both backbones. On the reported appearance, geometry, and joint shifts, it improves full-scene mIoU over the evaluated RGB–geometry baselines. Ablations identify paired intervention–adaptive fusion as the primary driver, with CORD and SA-TSI providing further gains at near-baseline inference cost.