- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes to reduce the hallucinations in MLLMs by using the same MLLM as the rectifier. The claim is that MLLMs (only Qwen in the paper) have a generation-verification gap, where they may generate hallucinations, but they themselves are a good verifier if the content is a hallucination, given the input image. This property is employed to use the MLLM to generate privileged information of 'facts' and 'hallucinations' about the image. Later, a student version is trained to that is guided by the teacher (having access to the privileged info) using on-policy distillation. Experiments are conducted mainly with Qwen-VL-8B-Instruct, and cross-backbone experiments also include 4B and 7B variants of Qwen. Generally strong results are reported over SOTA relevant methods.
Critical strengths
A well written paper, that is clear about the problem (which in itself is an important one) and builds towards the method systematically by first showing empirical evidence behind the motivation.
The results (although with Qwen only) are strong across multiple metrics.
Critical weaknesses
The central idea of the paper is not highly convincing. The work is build on the premise that MLLMs hallucinate when they generate, but not as much when they verify. Whereas this promises some performance gain by reusing the base model for verification, it still remains a model that 'hallucinates'. So, verification by a hallucinating model is not ideal. Not just that, the method seems to keep using the model for other steps, e.g., creating atomic claims. This is not convincing. The only gain achieved here is that instead of using external module, we reuse the model itself - no matter how good/bad it is. The results are only on Qwen, which seems to be good at creating atomic claims and fitting in the proposed overall framework. In general, this might not be true.
Whereas the paper is well-written, it suffers a bit from over-sophistications. Eg, the teacher MLLM seems to be the same as the reference MLLM and most of the process is just prompting the model (creating fact/probe view, atomic claim creation and even the guidance signal estimation). Different symbols for reference and teacher model do not help understanding that, nor the provided description of the approach. The main technical contribution is in training the student using an existing concept.
Ultimately, the results are provide for Qwen variants only. That does not convince a reader of the generalizabiity of the main idea of the paper that MLLMs can themselves be used to curb hallucinations - Qwen, may be, but MLLMs in general; that is not well supported.
Questions for authors
- How do the authors verify that the created sets of 'hallucinated' and 'non-hallucinated' concepts are correct?
- It seems to the reviewer that eventually the outputs are generated using only the student model, while the reference/teacher model is no longer needed. That is, for deployment, only student model is sufficient. Is that understanding correct?
- Can authors provide details of training the student model. That is, what computational resources, training data, etc. is required?
- Have the authors tried the same idea with other MLLMs, eg LLaVa?
- Have authors tried their approach on more recent generative hallucination inducing images like the ones generated by GHOST (ICLR'26)?
3 comments
Thank you for your thoughtful responses and further evaluations. Whereas the reviewer still maintains that a hallucinating model rectifying itself with self-verification is not ideal in practice, as the model itself becomes the bottleneck, author responses generally address the reviewer concerns. Hence, the reviewer is willing to update the rating to weak accept.
We sincerely thank you for taking the time to review our response. Your insightful suggestions have played a pivotal role in enhancing the overall quality of our paper. We deeply appreciate your recognition of our efforts to address the concerns and your willingness to update the rating once again.
We greatly appreciate your recognition of our paper's well-written, the importance of the problem we address, and the strong results across multiple metrics.
The reliability of self-verification is directly supported by the analysis in Figure 1 of the manuscript, where ground-truth annotations from AMBER-Gen, supplemented by manual labeling, show that the model correctly identifies 90.2% of hallucinated and 97.4% of non-hallucinated atomic claims.
To further assess whether these claims provide useful supervision, following Reviewer mySS, we directly evaluate the teacher conditioned on self-verified evidence. The teacher improves all eight metrics over vanilla, further supporting the effectiveness of self-verified atomic claims.
We thank you for this helpful suggestion. As you pointed out, the teacher model and the reference model share identical model weights. To avoid potential confusion, we have followed your suggestion and consistently used πT to denote both models throughout the revised manuscript.
To further evaluate the generalizability of our SVG-OPD, we extended our experiments to LLaVA-OneVision-1.5-8B-Instruct. Encouragingly, SVG-OPD consistently improves performance across all evaluated metrics, in line with our findings on Qwen-VL series. We have included these results in Table 3 of the revised manuscript.
Yes. After distillation, the student internalizes the verification-guided knowledge and performs inference independently, without requiring the reference model, teacher model, or additional verification steps. Therefore, SVG-OPD introduces no additional inference-time overhead.
We follow the experimental setup of RLSF-V (ICML 2026) [1] and train on 10K unlabeled image-instruction pairs from the RLAIF-V (CVPR 2025) [2] dataset. Our method is trained on 8 NVIDIA H100 80GB GPUs, with training taking approximately 4 hours for Qwen3-VL-8B. Please refer to the Implementation Details paragraph in Section 4.1 of the manuscript for further information.
Following your suggestion, we additionally evaluate our SVG-OPD on the recent GHOST (ICLR 2026)[3] benchmark, which uses adversarially generated images to induce object hallucinations. The results are as follows:
Our SVG-OPD reduces hallucination rates by 3.75 and 3.04 percentage points, respectively, demonstrating its effectiveness on the adversarially generated images.
[1] RLSF-V: Mitigating Hallucinations in MLLMs via Fuzzy Semantic Self-Feedback (ICML 2026)
[2] Rlaif-v: Open-source ai feedback leads to super gpt-4v trustworthiness (CVPR 2025)
[3] GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs (ICLR 2026)