Seeing Clearly, Remembering Selectively: Mitigating Multi-Agent Hallucination Snowballing via Evidential Modulation

As Multimodal Large Language Models (MLLMs) continue to advance, they are increasingly integrated into multi-agent systems for collaborative visual reasoning and task completion. However, MLLMs may still generate responses inconsistent with visual or contextual evidence, and such hallucinations can propagate across agents, causing hallucination snowballing and undermining system-level robustness. Existing mitigation methods primarily strengthen individual agents' visual grounding, leaving the reliability of information exchanged between agents insufficiently addressed. In this work, we investigate this problem from an evidential perspective by manipulating agentic visual and memory evidence to trace the influence on latent representations. Our analysis indicates that the generation of hallucination within an agent is linked to the disregard of visual evidence, whereas the propagation of hallucination between agents is associated with reliance on memory evidence supported by cross-layer inconsistencies. Based on these observations, we propose Dual Evidential Adaptive Modulation (DELVE), a training-free framework with two complementary modules. Specifically, Pristine Visual Evidence Reinforcement (PVER) derives dimension-level evidential weights to selectively reinforce latent representations with stable and pristine visual evidence of the initial agent, providing a stronger visual reference throughout agent interactions to mitigate intra-agent hallucination. Conflict-Aware Unreliable Memory Suppression (CUMS) exploits cross-layer directional inconsistency as a proxy for potential memory conflict, selectively attenuating inconsistent memory contributions while retaining coherent upstream information, thereby mitigating inter-agent hallucination propagation. Experiments on six hallucination and visual question answering benchmarks, spanning four MLLMs and three communication topologies, demonstrate improved hallucination mitigation while largely preserving general multimodal capabilities.