acceptodds
Under review as a conference paper at ICLR 2027

Region-Level Policy Optimization for Fine-grained MLLM Perception

Abstract

Fine-grained visual perception in multimodal large language models (MLLMs) is commonly improved by raising the resolution, but the added visual tokens inflate vision-encoding and language-model prefilling costs. We show that the two operations underlying fine-grained perception, localizing the region of interest (RoI) and recognizing its content, have different resolution requirements. In a controlled diagnostic, localization tolerates roughly 3 to 4 times stronger token compression than recognition, which motivates localizing from a coarse view and concentrating resolution on the selected evidence. Decoding coordinates with the MLLM can be trained end-to-end from answers, but costs a full model pass per query and depends on grounding ability. A lightweight proposal network distilled from the model's attention is fast, but inherits the noise of its attention targets. The RoI from the proposal network reaches the answer through a discrete region choice, so its faithfulness to the answer cannot supervise the network. We therefore optimize the proposal network with region-level reinforcement learning, which we call Vision-RL2. It treats coherent regions as actions, and a frozen MLLM reader scores each one by how its removal changes the answer likelihood. Complementary subtractive and additive objectives suppress distracting proposals and recover missing evidence, updating only the predictor without region annotations, response sampling, or reasoning trajectories. The refined proposal further enables a sparse encoding that magnifies evidence and excludes background tokens. Across six fine-grained benchmarks and four MLLM backbones, Vision-RL2 improves accuracy over the base model at every token budget and surpasses its largest-budget accuracy with about 4 times fewer visual tokens. Under the same token limit, Vision-RL2 also outperforms previous state-of-the-art methods that fully finetune the base model, while training an order of magnitude fewer parameters.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal large language models, fine-grained visual perception, visual token efficiency, high-resolution image understanding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_trnJOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Vision-RL2, a region-level reinforcement learning framework for improving fine-grained visual perception in multimodal large language models. The paper presents the localization tolerates stronger compression than recognition. It introduces additive region-level optimization objectives to suppress irrelevant regions and recover missing visual evidence.

Critical strengths

S1. Analysis on the impact of resolution for perception and localization tasks separately.

S2. Improving fine-grained visual perception while reducing the computational overhead associated with high-resolution visual inputs is practically important.

Critical weaknesses

W1. Lack of performance vs vision token comparision with the Efficiency based tool-calling based line like [1] and [2]. As, the paper argues about the same token limits, they are SoTA

  • Efficiecny oriented tool-calling baseline for performance vs vision token/latency are missing.

W2. Limited novelty beyond RL-based refinement of SD-RPN.

W3. The experimental comparison do not isolate the effects of the proposed methods from differences in training data.

[1] Yang, Senqiao, et al. "Visionthink: Smart and efficient vision language model via reinforcement learning." Advances in Neural Information Processing Systems 38 (2026): 95187-95227.

[2] Lee, Jewon, et al. "Ergo: Efficient high-resolution visual understanding for vision-language models." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors

Q1. What would be the efficiency–performance trade-offs on downstream tasks after training the baseline methods using the same datasets?

Q2. Can the proposed framework be applied to alternative region proposal networks or different initialization strategies?

Reviewer_zE8iOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper first runs a diagnostic experiment on ZoomBench with Qwen3.5-4B (n=238). It finds that, at matched survival rates, localization tolerates roughly 3–4× more token compression than recognition. On this basis, the authors argue for localizing on a coarse image and recognizing on a high-resolution one.

The method starts directly from the SD-RPN checkpoint (a lightweight RoI predictor self-distilled from response-to-image attention) and fine-tunes it further with "region-level RL": 1. Regions and scoring: The RoI heatmap is binarized and split into connected regions. Each region is treated as an action. A frozen MLLM serves as the reader and computes the log-odds of the gold answer under teacher forcing. Each region's contribution Δ is measured by leave-one-out occlusion. 2. Subtractive group: A noise margin b is estimated from control regions. Predicted regions whose contribution falls below b are suppressed. 3. Additive group: Candidate regions that the predictor missed are taken from attention maps at 6 layers. Their probabilities are increased if their contribution is positive. 4. Sparse visual encoding: At inference, only the foreground tokens inside the bbox are encoded, and resolution is magnified by 1/ρfg\sqrt{1/ρ_{fg}}

Experiments cover 4 backbones and 6 benchmarks. The authors report that the method beats both the base model and SD-RPN at every token budget, that the 9B model has the highest average in the main table, and that only the RPN (3 blocks) is trained.

Critical strengths
  1. The problem is well identified. SD-RPN's core flaw is that it is supervised by an attention proxy that is never checked against the answer. The authors pinpoint that discrete region selection blocks gradient flow, and they address it by treating regions as actions and using answer likelihood under occlusion as the signal. The motivation holds together.
  2. Low cost and fast inference. Only the RPN is trained, on 7K QA pairs for 1 epoch, with no response sampling and no reasoning trajectories. Routing takes only 31–49 ms at inference. The latency breakdown in Fig. 5 is convincing: end-to-end, the method is about 2× faster than ViCrop and Self-decoding.
  3. Broad evaluation. The experiments use 4 backbones, including the encoder-free Gemma-4, and report full token-budget curves. Against pruning methods such as VisionZip and DART (Table 4), the method shows clear gains on document tasks.
Critical weaknesses
  1. Incremental over SD-RPN. The architecture, inference pipeline, and initialization are all taken directly from SD-RPN. In the 576-token setting, RL alone adds only +3.0 (Table 2). Much of the "beats SoTA" result in the main table comes from sparse encoding (+1.5) and SD-RPN's own routing ability. This is a thin contribution for ICLR.
  2. The additive group still depends on attention, which contradicts the paper's main argument. The paper criticizes attention-derived targets for missing weakly attended evidence, yet the additive candidates come from exactly those 6-layer attention maps. Evidence that attention never sees cannot be recovered by this method either. The paper does not analyze the recall upper bound of the additive candidates.
  3. The degradation on MME-RealWorld is not analyzed in depth, even though this is the benchmark closest to real-world high-resolution scenarios.
  4. The ablations are not rigorous enough. They use only the 4B model and a single run. The ablation averages include InfoVQA, while 5K samples of the RL training pool come from the InfoVQA training set. This makes it an in-domain evaluation and inflates the averages. κ is set per model (1.25 vs. 1.0), and the tuning rationale should be stated.
  5. Comparable baselines are missing. Q-Zoom (Shi et al., 2026b, which the paper itself cites), ZoomEye, and FOCUS appear in neither the main table nor the efficiency comparison.
Questions for authors
  1. In Table 1, do the crop tokens count toward the 16,384-token cap? Please report the actual average number of visual tokens per sample and the latency for each method, especially compared with Vision-OPD.
  2. The additive candidates come entirely from attention. What fraction of GT evidence regions never appears among the 6-layer attention candidates? This fraction bounds how far the method can go beyond SD-RPN.
  3. Is the degradation on MME-RW caused by the frozen reader's recognition bottleneck, or by insufficient magnification when the enclosing bbox spans multiple regions?
  4. On which data were the hyperparameters κ, ρ, and K_max tuned? Were any evaluation benchmarks involved?
Reviewer_kCdFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Vision-RL², a region-level optimization method built upon SD-RPN. The proposed method uses the likelihood of the correct answer under a frozen MLLM to identify helpful and distracting regions. It is combined with sparse visual encoding to improve fine-grained perception under a limited visual token budget. Evaluations are performed on multiple benchmarks and backbones to validate the effectiveness of the method.

Critical strengths
  1. Building upon SD-RPN, the proposed method introduces answer-feedback-based region evaluation and combines region removal and supplementation to improve region selection without requiring bounding-box annotations or full-model fine-tuning.
  2. Experimental results demonstrate meaningful improvements. In Table 2, the average accuracy across six benchmarks increases from 66.6 with SD-RPN to 69.6 after region-level optimization, and further to 71.1 with sparse encoding. In Table 1, Vision-RL² achieves an average accuracy of 80.1 on Qwen3.5-9B, outperforming Vision-OPD-9B, while requiring updates only to the region predictor.
Critical weaknesses
  1. The paper does not sufficiently demonstrate the necessity of the reinforcement learning objective, as it does not evaluate simpler supervised learning objectives using the same region contribution scores. This is important to justify the motivation of this paper.

  2. The paper does not report the complete training cost associated with repeated reader evaluations. In addition, the absence of experimental results on Qwen3-VL-8B limits the evidence for the generalization ability of the proposed method to varying models in the popular Qwen3 family.

Questions for authors
  1. Could the authors include a reward-weighted binary cross-entropy or ranking-loss baseline in Table 2, using the same region contribution scores and training setup, to more clearly demonstrate the practical benefits of the proposed reinforcement learning objective?

  2. Could the authors report the total training cost, including SD-RPN initialization training and reader evaluations? Additionally, could the authors include experimental results on Qwen3-VL-8B in Table 1 to further validate the generalizability of the method across the Qwen3 model family?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.