Enhancing Multi-modal Reasoning via Self-Revisiting Image Evidence

Robust visual reasoning in Vision–language models (VLMs) often requires re-grounding intermediate steps in the underlying visual evidence. Recent approaches typically rely on external image operations such as zooming or cropping to re-access fine-grained details during inference, which requires additional image re-encoding and can disrupt the reasoning trajectory. We argue that VLMs already provide strong internal signals for identifying and reusing visual evidence, and that these signals can be directly leveraged to support image-grounded reasoning. Motivated by this insight, we propose an end-to-end self-revisit framework, SIEVE, that teaches models to re-engage image evidence through internal representations. SIEVE extracts embeddings of salient image regions and injects them into the reasoning chain when additional grounding is needed, enabling later steps to condition on relevant visual cues without external tool calls or re-encoding. We use reinforcement learning to teach the model when to trigger visual revisiting and which region embeddings to retrieve and insert during the reasoning process. Experiments across multiple visual reasoning benchmarks and evaluations of perception, reasoning, and hallucination show that SIEVE consistently improves performance, achieving an average gain of 8% across settings.