acceptodds
Under review as a conference paper at ICLR 2027

Perception or Reasoning? Diagnosing and Tackling RL Bottlenecks in VLMs

Abstract

Reinforcement learning (RL) has brought strong gains to language model reasoning, but its improvements on visual tasks remain less consistent. One difficulty is that visual task failures can arise from different sources: the same benchmark may be limited by perception for one model and by reasoning for another. We introduce Perception and Reasoning Attribution (PARA), which measures the remaining perceptual and reasoning headroom of each model-benchmark pair by comparing how many current errors are recovered with more accurate visual information versus a stronger reasoner. Across 4 VLMs and 17 multimodal benchmarks, PARA broadly agrees with the expected characteristics of existing benchmarks while revealing substantial model-specific variation. More importantly, it shows that RL gains become smaller as perceptual headroom increases, but larger when reasoning explains more of the recoverable errors, suggesting that current RL exploits reasoning headroom more effectively than perceptual headroom. Motivated by this gap, we analyze how existing RL objectives affect visual perception and find that reward- and token-level signals influence visual attention only indirectly. We therefore propose GRAD, a group-relative attention distillation objective that uses the best rollout in each group to directly guide the visual attention of weaker rollouts. GRAD consistently improves performance across two model scales and seven benchmarks, and further complements existing RL methods.

Primary area:
foundation or frontier models, including LLMs
Keywords:
VLM, Multimodal, Perception, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_Yev4OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why reinforcement learning (RL) improves some vision-language tasks more than others by separating perception and reasoning bottlenecks. It proposes PARA to diagnose perception and reasoning headroom at the model–benchmark level. Based on the analysis, it proposes GRAD, an attention-distillation method that directly improves visual perception during RL.

Critical strengths
  1. PARA quantifies how much model error can be recovered by better perception versus better reasoning, showing that the dominant bottleneck depends on the specific model–benchmark pair, rather than the benchmark category alone. The analysis further reveals that current RL benefits more from reasoning headroom than perception headroom, providing a clear explanation for the uneven gains of visual RL.
  2. Based on the finding that reward- and token-level signals affect visual attention only indirectly, the paper proposes GRAD to distill attention from the best rollout within each group into weaker rollouts. GRAD consistently improves performance across two model scales and seven benchmarks and outperforms standard GRPO and other perception-oriented baselines on average.
Critical weaknesses
  1. PARA does not fully disentangle perception and reasoning. A recovered sample under the perceptual intervention still depends on the target model’s ability to reason over the oracle caption, so PH{PH} may capture both improved visual information and easier text-based reasoning rather than perception alone. A cleaner design would explicitly decompose each sample into image → visual facts → QA, so perception and reasoning can be evaluated independently.
  2. The logic from Sec. 4 to GRAD is not fully justified. Sec. 4 mainly shows cross-benchmark correlations, while GRAD assumes that the highest-reward sibling provides better visual attention, even though the paper itself argues that outcome reward conflates perception and reasoning.
Questions for authors
  1. Could PARA be interpreted more conservatively as measuring a benchmark’s sensitivity to different forms of additional information, rather than directly identifying the true failure source?
  2. Can the authors justify the bottleneck analysis at the sample level rather than only at the benchmark level? A single benchmark may contain both perception-focused and reasoning-focused questions, so an aggregate headroom score may hide substantial within-benchmark variation; a per-sample attribution or grouping by visual-information dependence could provide stronger evidence for the claimed bottlenecks.
  3. The model-specific bottleneck claim may be confounded by model-family differences. Since the evaluated models come from different families, differences in training data, architecture, or prior knowledge may affect the measured headroom, not just perception/reasoning ability. Would comparing different model sizes within the same model family provide a cleaner analysis of how perception and reasoning headroom change with model capacity?
  4. For GRAD, is the highest-reward sibling necessarily the one with better visual attention? A direct grounding or attention-quality analysis would strengthen this assumption.
  5. Missing related works. For example, No-Thinking-RFT[1], ReFine-RFT[2], MoCA[3], and Chain-of-Thought Degrades Visual Spatial Reasoning, and Look Light, Think Heavy [4].

[1]: Li M, Zhong J, Zhao S, et al. To think or not to think: A study of thinking in rule-based visual reinforcement fine-tuning[J]. Advances in Neural Information Processing Systems, 2026, 38: 167356-167407. [2]: Zhu J, Su Y, Liu X. Can Textual Reasoning Improve the Performance of MLLMs on Fine-grained Visual Classification?[J]. arXiv preprint arXiv:2601.06993, 2026. [3]: Wang H, Xu Q, Wang C, et al. Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning[J]. arXiv preprint arXiv:2605.14054, 2026. [4]: Kancheti S S, Kanade A S, Balasubramanian V N, et al. Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs[C]//Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2026: 862-876.

2 comments
Reviewer_Yev4OpenReview ↗

Response to Author's Rebuttal

Thank you for the detailed reply. I maintain my positive rating.

We sincerely appreciate the reviewer's thoughtful feedback and positive assessment of our work. We are grateful for the constructive suggestions, which have helped us strengthen the paper. Thank you again for your time and engagement throughout the discussion.

Reviewer_jnzLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Failure of reinforcement learning for VLMs can be attributed to perception failure and reasoning failure. The authors present a Perception and Reasoning Attribution (PARA) framework to estimate the headroom of perception and reasoning on various benchmarks and discover that RL exploits reasoning headroom more than perceptual headroom, and existing RL objectives indirectly optimize visual attention. Thus, the authors propose GRAD to distill the visual attention of the highest-reward rollout to its siblings. Experimental results show that GRAD outperforms various methods on different benchmarks for Qwen3-VL-2B/4B.

Critical strengths
  • The paper is easy to follow, and the proposed objective is simple but shows consistent improvements across different benchmarks.

  • The proposed PARA framework separates perception and reasoning, providing a useful framework for diagnosing RL.

Critical weaknesses
  • The intuition behind GRAD seems vague. A successful rollout may not have better visual attention; its success may come from other factors, such as chance or reasoning ability.
Questions for authors

Are there any empirical evidence that a successful roll out has a better visual attention.

Reviewer_TiwTOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies whether perception or reasoning limits VLM performance and how these bottlenecks relate to improvements from reinforcement learning. It introduces PARA, which measures how many existing errors can be recovered by providing better visual descriptions or using a stronger reasoner. Based on these measurements, the authors argue that current RL methods exploit reasoning headroom more effectively than perceptual headroom. They then propose GRAD, an auxiliary objective that aligns the visual attention of negative-advantage rollouts with that of the highest-reward rollout in the same group. Experiments on Qwen3-VL-2B and 4B show improvements across seven benchmarks and additional gains when combined with several existing RL methods.

Critical strengths
  1. The paper addresses a useful question. Distinguishing model-specific bottlenecks is more informative than assuming that a benchmark primarily tests perception or reasoning. PARA provides an operational framework for examining this distinction across models and tasks.
  2. GRAD is a practical approach. It reuses existing rollout groups without requiring an external teacher, additional evidence annotations, or changes at inference time.
  3. The reported performance improvements are encouraging. GRAD improves average performance over GRPO and the other evaluated baselines at both model scales. Its compatibility with Vision-SR1, VPPO, and DAPO, together with the component ablations, supports its usefulness as an auxiliary training objective.
Critical weaknesses
  1. The connection between the diagnosis and the proposed remedy is not sufficiently established. PARA identifies errors recoverable with better visual information, but this does not clarify whether the limitation is insufficient perceptual capability or inconsistent use of capabilities the model already possesses. GRAD appears more directly suited to the latter: it uses successful behavior already present in a rollout group to supervise unsuccessful attempts. Groups with uniformly incorrect answers receive no auxiliary signal, even though these may include cases with substantial perceptual difficulties. This does not undermine the reported gains, but it leaves the motivation for the particular intervention incomplete. Perception-focused training, including supervised training on suitable perceptual tasks, is another natural response to the diagnosis. The paper does not sufficiently explain why its findings specifically motivate group-relative attention self-distillation. The benefits of avoiding additional data and teachers are clear, but they establish practicality rather than the connection between the diagnosed bottleneck and the chosen remedy. Consequently, the diagnostic and methodological contributions feel somewhat disconnected.
  2. The methodological novelty appears limited and needs clearer positioning relative to closely related work. The description-based intervention in PARA builds on ideas closely related to the cited work, Your Reasoning Benchmark May Not Test Reasoning (ACL 2026) (https://aclanthology.org/2026.acl-long.826/). For GRAD, CompoDistill (ICLR 2026) (https://proceedings.iclr.cc/paper_files/paper/2026/hash/74ee8b8a0b958199b440194fcf938bec-Abstract-Conference.html) already uses visual-attention distillation to improve visual capabilities, while Reinforcement Learning via Self-Distillation (ICML 2026) (https://proceedings.mlr.press/v306/hubotter26a.html) uses successful rollouts to provide self-distillation signals for failed attempts without an external teacher. The latter two works are not discussed in the current manuscript. GRAD differs from both, particularly in constructing a visual-attention target from the best sibling and applying it to negative-advantage rollouts. However, the contribution appears to be this specific combination and its empirical effectiveness. The paper would benefit from a clearer account of what this design adds beyond the existing ideas.
Questions for authors

If PARA tells me that perception is the bottleneck, why should I use the proposed self-distillation method rather than directly target that weakness—for example, by fine-tuning on perception-focused data? What makes self-distillation particularly suitable here?

1 comment
Reviewer_TiwTOpenReview ↗

final justification

Thanks for the responses. The authors’ replies to my review and their discussion with Reviewer Yev4 addressed several of my concerns. I found the VisNumBench analysis especially helpful. GRAD’s gains over GRPO were larger on questions that PARA had identified as benefiting from better visual information. This gives me a clearer reason to connect the diagnosis to the proposed method. I would be interested to see whether PARA can consistently identify groups of questions that benefit more from GRAD. So far, the authors have shown this connection for one model on one benchmark.

My assessment is around 3.5 out of 4. What tips the decision for me is GRAD’s practicality. It uses successful rollouts already generated during RL to provide visual attention supervision, without requiring additional training data or an external teacher. Given the four-point rating scale, I am raising my final recommendation to strong accept.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.