Localizing Failures: Hallucination-Grounded Preference Learning for Faithful Generative Super-Resolution

Generative Super-Resolution (SR) can improve perceptual quality but often introduces SR hallucinations—semantically unsupported objects, textures, structures, or text that are inconsistent with the paired Ground Truth (GT). Existing preference optimization methods rely on either image-level signals without spatial information or predefined spatial units that do not directly localize actual SR hallucinations. To address this limitation, we propose a framework comprising Hallucination-Grounded-Marks (HGM) and Hallucination-Selective Preference Optimization (HSPO). HGM uses a VLM to localize candidate regions on a $16\times 16$ grid and verifies them against the GT to produce a cell-level binary hallucination mask, without requiring precomputed object regions or boxes. Since existing perceptual metrics struggle to quantify hallucination severity, we further introduce the Area-weighted Hallucination Score (AHS) as a composite evaluation metric. HSPO exchanges hallucinated and faithful regions among SR candidates generated from the same Low-Resolution (LR) input to construct preference pairs for region-level optimization. This alleviates global preference ambiguity caused by inconsistent hallucination locations while preventing faithful regions from dominating the optimization signal. Experiments on Syn-TestSet and RealSR-4x show lower hallucination scores than C-FLUX and DP${}^2$O-SR while maintaining competitive perceptual quality.