TRIROUTE: Routing Visual Evidence Under Domain Shift for Cross-Domain Deepfake Detection

Multimodal models are expected to integrate complementary signals across modalities, yet in practice they often rely on spurious shortcuts from a single modality. This failure is particularly critical in cross-domain audio-visual deepfake detection, where models may ignore visual manipulation evidence and instead rely on easier but unreliable audio cues. A common approach to mitigate such failures is to enforce invariance through regularization or domain-adversarial objectives. However, it remains unclear whether these methods change how models make decisions, or merely reshape representations without altering their usage. In this work, we show that robustness in cross-domain deepfake detection is not primarily a problem of enforcing invariance, but of controlling how information is routed at inference time. We demonstrate that loss-level interventions fail to resolve representation-level entanglement in standard binary decompositions, where modality-specific signals and domain nuisance signals remain mixed, leading to persistent shortcut reliance. We introduce TRIROUTE, a simple three-way factorization that separates shared content, modality-specific task-relevant components, and residual domain-dominant signals. This structural design explicitly constrains the prediction pathway, encouraging the model to rely on visual manipulation evidence when audio cues are misleading. Across cross-domain audio-visual deepfake benchmarks, TRIROUTE yields substantial improvements in the diagnostic FV-RA setting, where correct predictions require reliance on visual evidence. Crucially, we provide multiple lines of evidence that these gains arise from changes in inference-time routing rather than representation invariance. Head ablation, input-level masking, and subspace probing show that predictions depend critically on the visual-unique component, while domain information remains present in the representation. Our results suggest a shift in perspective for multimodal deepfake detection: instead of suppressing spurious features, robustness can be achieved by explicitly structuring the pathways through which decisions are made.