Path Conflict: Understanding and Mitigating Hallucinations in Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) are often assumed to hallucinate because they under-utilize visual evidence and over-rely on linguistic priors. We show that this explanation is incomplete: hallucinated generations can exhibit stronger, rather than weaker, visual-pathway influence. To understand this counter-intuitive behavior, we introduce Path Conflict, which characterizes directional inconsistency between Image-to-Text (I2T) and Text-to-Text (T2T) pathways during token prediction. We further define Conflict Ratio and Conflict Strength to quantify the frequency and magnitude of such conflicts at the attention-head level. Our analysis reveals that hallucinated samples exhibit substantially stronger Path Conflict, driven primarily by amplified I2T contributions while T2T contributions remain comparatively stable. This suggests that hallucination can arise from misaligned visual influence, rather than simply insufficient visual grounding. Based on this finding, we propose a lightweight, training-free Conflict-aware Intervention that selectively regulates I2T and T2T flows within conflict-prone heads. Experiments on POPE, MCQ-POPE, and CHAIR consistently outperform strong inference-time baselines, including AllPath, while pathway-specific ablations reveal the asymmetric and complementary roles of I2T and T2T. Our results highlight inter-path directional conflict as an important mechanism-level perspective for understanding and mitigating MLLM hallucination.