- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a method to reduce hallucination in MLLMs. The key idea is reflection on the potential outputs. The reflection is done by a dedicated module, trained by a supervisory signal aware of potential hallucinations, where the signal is also constructed with the help of an MLLM (Qwen). The module uses on-policy distillation for its training. Inference does not require privileged information and mitigates the hallucinations considerably. Strong results are shown on four benchmarks, comparing against at least three baselines.
Critical strengths
The reported results are strong. The core idea of reflection makes intuitive sense.
Critical weaknesses
Whereas the paper idea is intuitive, and approach follows naturally, the AI face-lifting of the paper makes it a considerably uninteresting read. Brimming with fancy terminologies and sentences, like typical AI face-lifted/generated papers nowadays, reviewing this paper was not a pleasant experience.
The approach requires a supervisory signal. A high end Qwen3.6-35B is used for that. In effect, the key strength of the approach is coming from this signal - an implicit distillation. The evaluations on weaker models naturally benefit from such supervision. The central intuition of 'reflection' is also not exactly groundbreaking, as it is used in different VLMs, LLMs since 2023. Whereas the details of CogRef - proposed method - may be unique in its use of OPSD and treatment of the problem, the contribution of the paper is of incremental nature.
Questions for authors
- The state-of-the-art in MLLM hallucination now also considers adversarial/synthetic images that cause hallucinations, eg, GHOST (ICLR'26), did the authors try the method on such images? How is the performance?
- What is the author's opinion on the use of Qwen3.6-35B for 'experience construction'. With the assumption of a stronger Oracle for such an experience, which is eventually shaping the feedback for hallucination mitigation, isn't the performance gain mainly attributed to the stronger model. What if the experience construction if faulty?
- In the Tab 4 of appendix, CogRef latency is even lower than vanilla model. On the other hand, Eq(8) shows the loop for final generation that has additional steps, generation of feedback and then the response. The model M is used twice and there is an additional pass to construct F. Could the authors please clarify how the latency remains lower than the original model despite (apparently) more than twice flops involved that require sequential processing.