Balancing Multimodal Learning via Selective Knowledge Transfer with Path Discrepancy

Modality imbalance remains a fundamental challenge in multimodal learning, as heterogeneous modalities exhibit disparate learning dynamics and contribute unequally to joint optimization. Existing methods primarily improve learning balance through optimization regulation, while knowledge transfer complements these efforts by strengthening under-optimized modalities with multimodal supervision, typically through representation alignment. However, alignment-based transfer uniformly reduces cross-modal discrepancy without distinguishing knowledge deficiency caused by insufficient modality learning from modality-conditioned variation arising through cross-modal interaction, thereby conflating information that should be transferred with information that should remain distinctive. We propose Path-Discrepancy Guided Selective Transfer (PDST), which exploits transfer-path consistency to determine how different cross-modal knowledge should be utilized for balanced multimodal learning. By comparing direct transfer with its modality-conditioned counterpart under identical source and target spaces, PDST identifies route-stable information as path-invariant knowledge and interaction-induced variation as path-dependent knowledge. The resulting discrepancy-guided mechanism selectively transfers path-invariant knowledge to compensate for insufficient modality learning while preserving path-dependent knowledge to maintain complementary cross-modal information. Extensive experiments on multiple multimodal datasets demonstrate consistent improvements over representative balanced multimodal learning methods, establishing selective knowledge transfer guided by path discrepancy as an effective approach to mitigating modality imbalance.