acceptodds
Under review as a conference paper at ICLR 2027

From Diagnosis to Mitigation: Benchmarking and Routed Mitigation for Visual and Knowledge Hallucinations in MLLMs

Abstract

Multimodal large language models (MLLMs) perform tasks that differ in the primary source of information required for correct answers. Some tasks are driven by directly verifiable visual evidence, while others rely more on knowledge and reasoning. We denote them as visual-dominant and knowledge-dominant tasks. Hallucination arises when an MLLM fails to properly balance these two sources of information. To diagnose and evaluate visual and knowledge hallucinations, we construct a new hallucination benchmark, VKHalluciBench. Compared with existing benchmarks, VKHalluciBench adopts a novel taxonomy, provides broad coverage across visual- and knowledge-dominant tasks, and incorporates carefully designed open-ended questions and counter-intuition samples that are prone to induce hallucinations. After a comprehensive evaluation, we observe that most mitigation strategies face a common dilemma, which manifests as a trade-off between mitigating visual hallucinations and knowledge hallucinations. To address this limitation, we further propose a plug-and-play routing-based mitigation framework to enable adaptive hallucination mitigation. Specifically, we design a Diagnostic Signal Probe to provide diagnostic results, and an Adaptive Hallucination Mitigation Router then selects suitable mitigation strategies. Experimental results demonstrate that our method achieves the best performance on both visual and knowledge tasks and introduces only 0.33M trainable parameters, resulting in an overall improvement of 12.1% over the baseline model.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Hallucination Mitigation, Multimodal Large Language Model, Visual Hallucination, Knowledge Hallucination

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_KQZCOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies hallucination mitigation of MLLMs. The authors propose that some tasks are driven by direct visual evidence while others rely on knowledge and reasoning. To diagnose and evaluate visual and knowledge hallucinations, the authors build a benchmark, VKHalluciBench, providing broad coverage across visual- and knowledge-dominant tasks, and incorporating carefully designed open-ended questions and counter-intuition samples that are prone to induce hallucinations. The authors find that most mitigation strategies struggle balancing mitigating visual hallucinations and knowledge hallucinations. To this end, the authors further propose a plug-and-play routing-based mitigation framework.

Critical strengths
  1. Overall the paper is well-written and easy to understand.
  2. The proposed method shows improvement over baseline methods.
Critical weaknesses

Limited Novelty

  1. The finding on the existence of knowledge-driven and vision-driven hallucination is not new. The conflict between visual information and knowledge prior has been studied by HallusionBench [1], VCD [2], etc. The authors define two mutually exclusive types of hallucination, whereas many VQAs rely on both visual information and knowledge. The taxonomy proposed by the authors is merely a reorganization of existing conclusions, rather than something substantially new.

  2. The benchmark is merely a reorganization of existing datasets. HallusionBench [1] and PhD [3] have cover the evaluation of visual information hallucination and knowledge-driven hallucination. Counter-intuitive and open-ended evaluation is not new, either.

Insufficient findings

  1. The observed trade-off, although empirically useful, is still an expected consequence of task-specific optimization instead of a fundamentally new insight.

  2. The introduced diagnostic confidence does not appear to influence the routing decision, limiting the methodological significance of the proposed uncertainty handling.

  3. The 99.04% diagnostic accuracy is merely about recognizing question types. There are strong patterns which can be easily captured by models. For example, predicting "visual" when reading "What color", predicting "knowledge when reading "Solve". Predicting task type does not equal to diagnosing hallucination type, either. For example, errors from math problem solving may come from OCR error. Errors from object recognizing may come from knowledge prior. Therefore, there is no strong evidence that DSP can recognize the real hallucination mechanism.

  4. The 12.1% improvement is just the improvement from vanilla methods, rather than from strong baselines.

  5. The authors assign SEVA for visual-dominant questions and RITUAL for knowledge-dominant questions, which is essentially a two-branch prediction.

[1] Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In CVPR, 2024.

[2] Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In CVPR, 2024.

[3] Phd: A chatgpt-prompted visual hallucination evaluation dataset. In CVPR, 2025.

Questions for authors

See the weaknesses above.

Reviewer_gDh7OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces VKHalluciBench, a benchmark containing 2,000 image entries and 10,725 question-answer pairs across ten subtasks, organized into visual-dominant and knowledge-dominant categories. The evaluation shows that mitigation strategies have different strengths across these categories, with some exhibiting negative transfer. The authors propose Adaptive Diagnostic Guided Mitigation Routing (ADGMR), which uses a frozen text encoder and a small classifier to predict the query category, then selects SEVA or RITUAL accordingly. On LLaVA-v1.5-7B, the overall benchmark score increases from 56.52 to 63.35, corresponding to a 12.1% relative improvement, with additional results on MME. The benchmark and strategy combination are potentially useful, but the reliability of the evaluation and the diagnostic value of the router need stronger support.

Critical strengths
  1. The benchmark offers useful breadth. It covers perception, counter-intuition, commonsense, mathematics, code, and medical questions through a common evaluation interface, combining open-ended answers with positive and negative verification questions.

  2. Several construction choices are thoughtful. The appendix describes ambiguity filtering, code execution for ground-truth outputs, function-name anonymization, and removal of non-visual medical metadata. These measures address concrete sources of label noise and shortcut features.

  3. The evaluation reveals practically relevant differences between strategies. Thirteen MLLMs and sixteen mitigation methods are examined, and the results show that a method effective on visual tasks may perform poorly on knowledge tasks. This is useful when selecting mitigation methods for a mixed workload.

  4. The routing framework is simple and improves the aggregate score. The additional classifier has approximately 0.33M trainable parameters, and ADGMR outperforms the reported fixed strategies on the overall VKHalluciBench and MME scores. The approach makes practical use of complementary existing methods.

Critical weaknesses
  1. The 99.04% diagnostic accuracy is hard to interpret without a clear test split. Appendix B trains the probe on 50 images per subtask, but leaves the evaluation pool and treatment of related image variants unclear. Any evidence that diagnosis and routing still work with source images and question templates held out, beyond a simple rule-based router?

  2. The judge validation appears to be not that much convincing. The GPT-4o prompt accepts answers that do not conflict with the reference, leaving incomplete answers, abstentions, and extra unsupported claims underspecified. Did authors manually verify the evaluation strategy?

  3. What exactly is being diagnosed here? DSP only sees the question, so the 99.04% result measures task classification even though code and medical errors can also arise from failed visual recognition. Calling this task-type prediction would make the scope clearer, as it doesn't show clear generalization ability.

  4. How much does learned routing actually add? Table 5 implies about 63.28 for a selector using ground-truth task categories, close to ADGMR's 63.35, while the gain over the best fixed method is 1.13 points. Reporting this reference from the existing predictions would help separate the benefit of combining experts from that of the learned diagnostic signal.

  5. ACC-I is sensitive to the number of questions per image. An all-questions-correct score can fall as the bundle grows, even without additional evidence inconsistency. A clearer account of this interpretation, the macro-averaging weights, and the aggregation of the three judge runs would help.

  6. The related-work discussion misses some close connections. VDGD/VaLLu [1] studies the gap between visual-recognition and cognitive prompts, HallusionBench [2] addresses entangled visual and language hallucinations, and Mixture of Decoding [3] adaptively selects mitigation strategies. Where do the main additions lie in the taxonomy, benchmark construction, and routing approach relative to these works?

  7. The fuzzy-set result leaves the robustness claim open. Confidence only flags samples, and matching outputs on 20 of 21 cases could reflect shared errors. Were those answers actually correct in the existing evaluations?

[1] Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Utkarsh Tyagi, Oriol Nieto, Zeyu Jin, and Dinesh Manocha. 2025. Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs. International Conference on Learning Representations (ICLR).

[2] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2024. HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14375-14385.

[3] Xinlong Chen, Yuanxing Zhang, Qiang Liu, Junfei Wu, Fuzheng Zhang, and Tieniu Tan. 2025. Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models. Findings of the Association for Computational Linguistics: ACL 2025, pp. 8525-8542. DOI: 10.18653/v1/2025.findings-acl.448.

Questions for authors

please refer to the weaknesses above.

Reviewer_rFfyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces VKHalluciBench to evaluate visual and knowledge hallucinations in MLLMs. It finds that existing mitigation methods perform differently across these task categories and proposes ADGMR to select a method for each question. ADGMR uses a text-based probe to classify the question and selects SEVA for visual tasks or RITUAL for knowledge tasks. The paper reports higher overall accuracy than either method alone on VKHalluciBench and MME.

Critical strengths
  1. The benchmark goes beyond object existence to cover OCR, counterintuitive images, coding, mathematics, and medical knowledge, while reporting both question-level and image-level metrics. This provides a broad evaluation setting for comparing mitigation strategies.
  2. The authors observe that mitigation methods have different relative strengths across task groups and attempt to exploit this through routing. ADGMR has a small parameter footprint and achieves higher reported overall accuracy than either individual method.
Critical weaknesses
  1. The paper reports a 1.13 percentage-point gain over the best individual method in Table 5, but does not compare ADGMR with routing based on the benchmark’s task labels: SEVA for visual tasks and RITUAL for knowledge tasks. The benefit of learning the router over using these labels is therefore unclear.
  2. Appendix B uses 50 images per subtask for probe training, but does not identify the test set for the 99.04% accuracy in Figure 5. It is also unclear whether the main evaluation excludes these training images or whether training and test questions share templates.
  3. The judging prompt in Appendix C treats non-conflict with the reference as correctness, but does not explicitly require the response to answer the question. No agreement with human judgments is reported for open-ended answers.
Questions for authors
  1. Which images and questions are used to test the probe in Figure 5? Does the main evaluation exclude probe-training images?
  2. How does ADGMR compare with a task-label oracle router? This baseline would use the known task labels to select SEVA for visual tasks and RITUAL for knowledge tasks. The per-subtask results, macro accuracy (equal weight per subtask), micro accuracy (equal weight per question), and ACC-I should be reported.
  3. Does the judge mark incomplete answers as correct? Please report agreement with human judgments on a sample of open-ended answers. Does correcting any grading errors change the method rankings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.