- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Faithful-MR1, a two-stage training framework designed to improve faithful visual reasoning in multimodal large language models. The first stage, Anchoring, introduces a dedicated <Focus> token and supervises its attention toward question-relevant image regions using bounding-box annotations. The second stage, Reinforcing, identifies vision-dependent reasoning tokens through counterfactual image masking and encourages stronger visual attention at these positions using an auxiliary reward during GRPO training. Experiments on Qwen2.5-VL-3B and 7B demonstrate consistent improvements over GRPO and VPPO across seven multimodal reasoning benchmarks. Additional ablation studies suggest that both training stages contribute to the improvements.
Critical strengths
- Considers an important question in VLM reasoning, that how to make VLM focus on relevant areas of image.
- The proposed method achieves consistent improvements across seven benchmarks and two model scales. The ablation studies also demonstrate the complementary benefits of the Anchoring and Reinforcing stages.
Critical weaknesses
- The paper encourages stronger visual attention at vision-dependent reasoning tokens by optimizing the ratio between their attention to visual tokens and that of other response tokens. However, this relative attention ratio can be increased either by enhancing attention at vision-dependent tokens or by suppressing attention at other positions. Consequently, an improvement in this metric does not necessarily indicate more effective utilization of visual evidence. Furthermore, the attention reward includes both image patch tokens and the <Focus> token as visual tokens. It remains unclear whether the resulting improvements primarily come from attending to the actual image content or from attending to the learned <Focus> representation. More targeted analyses or interventions would help clarify how the proposed mechanism improves visual reasoning.
- The Reinforcing stage identifies vision-dependent tokens by selecting the top 10% of response tokens ranked by the KL divergence between predictions conditioned on original and masked images. However, this fixed selection ratio lacks sufficient justification. Even when a response is largely insensitive to visual interventions, the method still selects 10% of its tokens as vision-dependent. This raises concerns about whether the selected tokens genuinely represent positions where visual evidence is required. The paper would benefit from an analysis of the KL divergence distribution and ablation studies on different selection ratios or absolute sensitivity thresholds.
- Faithful-MR1 generates shorter reasoning responses than VPPO (325 versus 382 tokens on average), which may influence how frequently visual facts are explicitly reported and subsequently evaluated. Since the Faithful Use metric is conditioned on correctly reported facts, differences in response length and reporting behavior may also affect the results.
- The training dataset is annotated with question-relevant bounding boxes using Gemini-3-Flash, a powerful multimodal model. These annotations identify the visual regions most relevant to solving each problem. This raises a fundamental question: To what extent do the improvements come from the proposed training mechanisms, rather than simply encouraging the model to attend to regions identified by a stronger model? Although the paper includes GRPO and VPPO baselines trained on the same examples, it lacks a simpler baseline that directly increases attention to the annotated regions without introducing the proposed two-stage framework or counterfactual token identification.
Questions for authors
See weakness
4 comments
Response to Reviewer tCzD [W3-W4] (Part 2/2): Fact-use evaluation and region supervision
W3. Response length and fact-reporting effects
Thank you for raising this concern. We agree that response length and reporting behavior can affect Faithful Use, and therefore conduct two complementary analyses.
- Length matching: Across 198 same-question pairs with response-length differences within 10%, average lengths are 355.9 vs. 359.0 tokens, while Faithful Use is 69.92% vs. 65.13% for Faithful-MR1 and VPPO.
- Shared facts: Separately, on 1,274 facts correctly reported by both models, Faithful-MR1 achieves 67.50% vs. 65.46%, using an identical fact denominator.
Faithful-MR1 also achieves higher answer accuracy (45.6% vs. 42.4%) across all 788 questions, independently of the fact-reporting denominator.
Together, these results support our claim that Faithful-MR1 more consistently uses correctly reported visual evidence and achieves better task performance: the Faithful Use advantage persists under length matching and remains directionally consistent when comparing the same facts.
W4. Contribution beyond region annotations
Thank you for raising this attribution question. We add GRPO + Region-Attention Loss on Qwen2.5-VL-3B-Instruct as a controlled baseline using the same training examples and Gemini-3-Flash region annotations as Faithful-MR1. It directly encourages response-token attention to the annotated patches, without the proposed <Focus> token or counterfactual token identification.
| Method | Uses region annotations | Math Avg. | General VQA Avg. | Overall |
|---|---|---|---|---|
| GRPO | No | 36.0 | 48.7 | 39.6 |
| GRPO + Region-Attention Loss | Yes | 38.9 | 48.9 | 41.7 |
| Faithful-MR1 | Yes | 41.6 | 49.9 | 43.9 |
The improvement from 39.6 to 41.7 shows that directly exploiting the region annotations is beneficial. However, with access to the same annotations, Faithful-MR1 reaches 43.9, a further 2.2 percentage-point gain. This supports an additional benefit of the full framework beyond directly increasing attention to regions identified by Gemini-3-Flash.
Follow-up on W1–W4 and Appendix C
Dear Reviewer tCzD,
Our two-part response covers W1–W4. The new figures on visual attention and token selection are in Appendix C of the revised manuscript. We’d welcome your thoughts on whether our response addresses your main concerns and whether there is anything we could clarify further.
Final Note Before the Discussion Closes
Dear Reviewer tCzD,
Thank you for your thoughtful questions. They helped shape the additional analyses in our rebuttal.
With only a few hours left in the discussion period, we would welcome your thoughts on whether our response addresses your concerns. We hope the new evidence will be helpful as you finalize your rating, and we’d be happy to clarify any remaining points.
Response to Reviewer tCzD [W1-W2] (Part 1/2): Attention reward and VD selection
W1. Absolute visual attention and the role of
<Focus>Thank you for the suggestion. We provide two targeted analyses to clarify the attention reward.
First, Figure 5 (Appendix C.1) tracks the auxiliary attention reward and the absolute visual attention at vision-dependent positions, avd, over the first 150 training steps. The reward increases alongside avd, showing that its improvement is accompanied by stronger visual attention at these positions, rather than solely by reduced attention elsewhere.
Second, we exclude
<Focus>from the attention statistics used to compute the auxiliary reward, counting only image-patch attention:<Focus>The results show small changes in both directions, with no consistent degradation. This suggests that directly rewarding attention to
<Focus>is not essential to maintaining this performance level. This ablation retains<Focus>as a learned representation and isolates only its inclusion in the reward; it does not establish that the representation itself is unnecessary.W2. Absolute KL threshold and candidate selection ratio
Thank you for pointing this out. Our manuscript inadvertently omitted the absolute KL threshold (Dt≥0.3), which was already present in the implementation. We apologize for this omission and have corrected Section 3.3 (p. 4), with the clarification marked in blue. We first select the top-10% candidate positions by KL divergence and retain only those satisfying this threshold. Consequently, the final VD set can be empty; the method does not force every response to contribute 10% VD tokens.
Our analysis of training responses shows that 4.81% of response tokens are selected as VD tokens on average, while 11.20% of responses contain no VD tokens. Separately, we examine the KL distribution of 205,128 top-10% candidates from 10,240 responses before thresholding. The CDF in Figure 6 (Appendix C.2) shows that 55.52% have KL < 0.3 and would be excluded by the implemented threshold.
To examine the threshold’s contribution and sensitivity to the candidate ratio, we consider three selection rules:
The first two settings examine the effect of removing the absolute threshold; the first and third examine increasing the candidate ratio while retaining the threshold. The results support the role of the absolute threshold in filtering low-sensitivity candidates, while the small changes between 10% and 20% suggest limited sensitivity to the candidate ratio.