ASYM-OPSD: Mitigating MLLM Hallucinations via Asymmetric On-Policy Self-Distillation

Hallucination in Multimodal Large Language Models (MLLMs) remains a critical barrier to trustworthy deployment. Existing mitigation strategies, whether inference-time interventions such as contrastive decoding and attention reallocation, or training-based preference alignment, share a fundamental limitation: they rely on coarse-grained, sequence-level signals that cannot precisely localize the specific tokens responsible for hallucinations. We introduce ASYM-OPSD, an asymmetric on-policy self-distillation framework that derives dense, token-level reward signals from hierarchical textual descriptions of the input image that serve as privileged visual anchors. When a teacher model conditioned on anchored input evaluates the student's own rollouts, the per-token log-probability gap between teacher and student reveals which tokens are visually grounded and which are hallucinated. Our pipeline comprises three stages: visual anchor construction, anchor-guided warm-up that repairs hallucinated responses to initialize a teacher model, and reinforcement learning with group-relative token-level advantages and curriculum scheduling on anchor granularity. Experiments on Qwen3-VL-4B and InternVL3.5-8B show that ASYM-OPSD surpasses training-based and inference-time baselines across four hallucination benchmarks, mitigating long-form hallucinations by up to $3.2\times$ without compromising general VQA performance.