HSS-DR: From Hidden-State Spectral Signatures to Object Hallucination Detection and Repair in LVLMs

Large vision-language models (LVLMs) have demonstrated strong capabilities in multimodal understanding, yet they still hallucinate objects that are absent from the input image. We investigate this problem by analyzing the spatial spectra of visual representations computed before response generation. In late decoder layers, these spectra show a lower proportion of low-frequency energy and a higher spectral centroid for subsequent hallucinated mentions than for grounded mentions. Based on this finding, we propose HSS-DR, a framework for object hallucination detection and repair. We train a lightweight detector that uses spectral features to identify object mentions at risk of hallucination. For each flagged object mention, HSS-DR uses constrained search to find a lower-risk continuation. The same detector evaluates candidate continuations over a short lookahead and guides their selection based on hallucination risk. Experiments across multiple LVLMs demonstrate that HSS-DR consistently reduces object hallucination while preserving general multimodal capabilities.