- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents Rebalancing Attention over Visual Evidence (RAVE), a lightweight, training-free strategy to mitigate hallucinations in think-with-image models. The motivating experiments identifies that visual recency bias is a key factor for hallucinations in think-with-image models. Based on this obversation, the authors proposed RAVE, a training-free strategy that re-distributes the misallocated visual attention to facilitate better integration of semantic information. Experiment results shows RAVE mitigates the model hallucinations in all evaluated benchmarks and baselines. The paper also includes experiments for other general perception and reasoning benchmarks, also ablation studies.
Critical strengths
I really enjoyed reading this work. The paper is well written and easy to follow. The observations are informative and interesting. Proposed method is effective and well examined using comprehensive experiments.
Critical weaknesses
- This paper only focuses on the hallucinations rather than overall visual capabitilieis, I think tool-calls has broader impacts beyond hallucinations. A more general study will be more interesting and appealing.
- There is no confidence intervals in the main experiment results. Since some performance gaps are minor, it important to report such stats.
Questions for authors
- In section 2.1, the author mentioned to apply temperature coefficients T >1 to amplify the attention assigned to all visual tokens. I am wondering why this is the case.
- How non image manipulation tools will affect here? Can RAVE extend to such cases?
I thank the authors for their additional clarification experiments, discussions and comments. All of the points raised earlier have been addressed. I remain in favor of accepting this work.