acceptodds
Withdrawn from ICLR 2027

HalluGuard: Decoupled Automatic Hallucination Detection and Mitigation in MLLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Large Language Models (MLLMs) often exhibit severe hallucinations, where generated text conflicts with visual inputs. Existing evaluation methods heavily rely on costly human annotators or black-box APIs (e.g., GPT-4o). However, these end-to-end evaluators inherently suffer from multimodal entanglement, making them prone to hallucinations themselves and unreliable as judges. To address this, we introduce HalluGuard, a decoupled and interpretable framework for hallucination evaluation and mitigation. Instead of relying on black-box MLLMs, our AutoEval module decomposes the complex hallucination detection process into atomic visual perception tasks (e.g., grounding and segmentation). This white-box paradigm provides accurate, fine-grained evaluation for attributes prone to hallucination, such as object quantity, color, and relative position. Based on this deterministic feedback, we further propose Interpretable Hallucination Mitigation (IHM), an inference-time intervention mechanism that effectively corrects hallucinations without requiring any parameter updates or preference data. Furthermore, leveraging our fine-grained metric, we reveal a critical “Goodhart's Law” phenomenon in current MLLM alignment: models often hack hallucination metrics by generating shorter, evasive responses rather than genuinely correcting visual-textual misalignments. Extensive experiments demonstrate that HalluGuard achieves high human agreement in evaluation and effectively mitigates hallucinations, offering a robust alternative to current end-to-end paradigms.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Automatic Dection

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9aroOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes HalluGuard, a decoupled framework for evaluating and mitigating hallucinations in MLLMs. The main component, named AutoEval, checks generated descriptions through separate visual perception tasks and provides fine-grained feedback. Its inference-time mitigation module uses this feedback to revise hallucinated content without model updates. Several experiments shows high agreement with human judgments and reveal that shorter responses can misleadingly improve hallucination scores.

Critical strengths
  • The AutoEval and IHM frameworks are well described.
  • The paper is easy to understand.
  • The experimental results offer valuable insights.
Critical weaknesses
  • The paper lacks a sufficient methodological contribution.
  • AutoEval is the main component, but its novelty over previous hallucination detectors [1][2] is unclear. Comparisons with existing detection methods are insufficient.

[1] Unified Hallucination Detection for Multimodal Large Language Models, ACL 2024.

[2] ZINA: Multimodal Fine-grained Hallucination Detection and Editing, CVPR 2026.

  • The evaluation is on only 500 images from a single dataset, which is insufficient to adequately assess the generalizability.
  • The author should evaluate on standard benchmarks like POPE, AMBER, HallusionBench, and MMHal-Bench.
Questions for authors

See Weaknesses.

Reviewer_4Cz6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies post-generation hallucination detection and correction in image descriptions produced by MLLMs. The proposed AutoEval framework uses a series of expert models to detect the unconsistencies and IHM method converts them into feedback prompts that instruct the original model to remove erroneous details or correct counts and colors, without updating model parameters.

The paper also examines the relationship between response length and hallucination rates, arguing that shorter, more conservative responses can receive better hallucination scores.

Critical strengths
  1. The modular design supports diagnosis. Bounding boxes, attribute predictions, and identified inconsistencies make the decisions easier to inspect than a single holistic judgment.
Critical weaknesses
  1. Limited novelty and technical advancement. AutoEval and IHM primarily rely on external expert models to verify and revise generated responses, without demonstrating improvements in the underlying MLLM’s visual understanding or intrinsic resistance to hallucination. The method and experiments mainly target image descriptions, leaving their applicability to other task formats, such as general visual question answering, unclear. Furthermore, the technical advances over closely related approaches, particularly Woodpecker and FaithScore, and their resulting practical benefits are insufficiently established.

  2. Limited task setting and evaluation scope. The paper primarily addresses hallucinations involving object existence, counts, colors, and simple spatial relations that can be verified by expert models. How the framework detects hallucinations involving complex semantics, actions, and interactions remains unclear. Although several MLLM families are evaluated, coverage of newer, stronger, and larger models is insufficient to establish broad applicability. Moreover, the experiments use only 500 images sampled from the COCO 2017 validation set, without results on additional hallucination benchmarks or general capability benchmarks. Consequently, cross-dataset generalization and the impact of correction on overall task performance remain insufficiently validated.

  3. Overinterpretation of experimental observations. Regarding the claimed Goodhart’s Law phenomenon, shorter responses accompanied by lower hallucination rates directly demonstrate sensitivity to response length and content coverage. They do not establish that models have learned to exploit the metric or that alignment has failed to improve visual accuracy. For an overall metric measuring whether a response contains at least one error, increasing the number of factual claims raises the probability of observing an error even when per-claim accuracy remains unchanged. Different models may also describe objects and attributes of different difficulty. Without controlling for the number of claims, content coverage, and claim difficulty, the experiments do not support a causal conclusion about metric exploitation.

Questions for authors
  1. What specific technical advances distinguish AutoEval and IHM from Woodpecker and FaithScore? Can the authors provide direct comparisons under matched settings to establish the benefits of these advances?

  2. Can the framework handle general visual question answering beyond image descriptions, particularly questions involving actions, interactions, or complex semantic relations? What modifications would be required?

  3. Does the method remain effective on newer, stronger, and larger MLLMs? In particular, how does IHM perform when the evaluated model is more capable than the expert models used for verification?

  4. Can the authors provide results on additional hallucination benchmarks and datasets beyond COCO, together with general capability evaluations to assess whether correction preserves useful information and task performance?

  5. After controlling for the number of factual claims, content coverage, and claim difficulty, does the observed reduction in hallucination after alignment persist? What evidence distinguishes metric exploitation from improved factual accuracy or more conservative generation?

Reviewer_ZaEvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents HalluGuard, which automatically evaluates hallucinations in image descriptions generated by multimodal large language models by decomposing hallucination detection into interpretable visual tasks, including object localization, counting, color recognition, and relative-position reasoning. It further introduces IHM, an inference-time method that removes or corrects erroneous information without fine-tuning the model or constructing preference data. The main contributions are a white-box, fine-grained evaluation framework built on open-source models; a low-cost hallucination mitigation method; and the observation that alignment training may reduce hallucination scores by shortening descriptions rather than genuinely improving error-correction capabilities. The paper evaluates the approach on several mainstream MLLMs. It also conducts a human evaluation of 300 image-description pairs with three annotators, showing that AutoEval agrees with human judgments more closely than GPT-4o does. IHM also effectively reduces hallucinations, with IHM-Removal, which directly removes erroneous details, demonstrating more stable performance.

Critical strengths
  1. The method decomposes the complex task of hallucination assessment into verifiable subtasks, including object localization, counting, color recognition, and relative-position reasoning. This design improves interpretability and reduces reliance on black-box evaluators such as GPT-4o.

  2. The proposed IHM can correct model outputs without training or parameter updates, making it comparatively inexpensive to deploy.

Critical weaknesses
  1. The pipeline is heavyweight, which limits its practical utility. HalluGuard/IHM jointly relies on Qwen-2.5-72B, Grounding DINO, SAM 2, CLIP, and the original MLLM. Chaining these models substantially increases inference cost and creates multiple points at which errors can propagate and accumulate.
  2. The main improvements from IHM are measured using the evaluation method proposed by the authors. Because the evaluator and the mitigation method share the same detection results, there may be a risk that the method is effectively optimized for this particular evaluator rather than achieving broader reductions in hallucination.
  3. The method may reduce the informativeness of the descriptions. Although the authors claim that IHM modifies only a small number of words, IHM-Removal still reduces hallucinations by deleting details. Without a joint metric for factual accuracy and information completeness, it is difficult to determine whether the improvement comes at the cost of losing useful information.
  4. The comparative evaluation is insufficient. The paper lacks systematic comparisons with existing training-free hallucination mitigation methods, verification-based rewriting approaches, and lighter-weight single-model evaluation schemes. Consequently, the claimed overall advantage in effectiveness, cost, and efficiency is not yet fully substantiated.
Questions for authors
  1. Could the authors evaluate the hallucination mitigation method using additional independent metrics or evaluators? Such results would help establish whether the improvements generalize beyond the proposed evaluation framework.
  2. Could the authors simplify the evaluation and mitigation pipeline, or provide evidence that a lighter-weight configuration retains comparable performance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.