acceptodds
Withdrawn from ICLR 2027

REFINE: Active Visual Reasoning with Spatially Indexed Evidence Refinement

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Active visual reasoning requires models to acquire fine-grained observations and retain them across successive decisions. Appending every crop grows the visual context even when an inspection revisits a previously observed region. We introduce REFINE, a reinforcement-learning framework that organizes acquired evidence into a persistent spatial state. Each region owns a fixed set of slots initialized from a coarse observation; higher-resolution inspection replaces those slots while preserving evidence elsewhere. A frozen answer-only probe measures the change in reference-answer support between initial and final states. This bounded, signed utility augments correctness and format rewards within Group Relative Policy Optimization, favoring useful refinement without rewarding inspection count. Evaluations span visual search, mathematical reasoning, chart understanding, and real-world perception. REFINE-7B achieves 85.26 on V* positional search and 69.2 on RealWorldQA, compared with 82.89 and 66.4 for the GRPO baseline. Its MME-RealWorld perception and reasoning scores are 68.60 and 50.37. Component ablations show complementary benefits from spatial storage and utility-aware learning. REFINE connects selective observation to persistent evidence through local replacement and outcome-grounded policy optimization.

Primary area:
reinforcement learning
Keywords:
Multimodal reasoning, Vision-language models, Active perception, Visual memory, Visual token compression

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RaPCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

In this work, the authors propose an active visual reasoning framework based on reinforcement learning. The methodology contains three parts: 1) Spatially indexed evidence refinement, where evidence is computed for each cell and can be refined using embeddings at increased resolutions. 2) Answer-support utility, which determines whether intermediate steps are useful based on answer correctness. 3) Utility-aware GRPO, where utility is included as part of the reward. Experiments on multiple benchmarks demonstrate the effectiveness of the proposed method compared with vanilla GRPO on Qwen2.5-VL models.

Critical strengths
  • The problem is well-motivated and the proposed method is intuitive. REFINE separate the storage decision from the acquisition decision, which is a interesting and useful conceptual contribution.
  • Experiments cover different model size and benchmarks, as well as comprehensive ablation, demonstrate the effectiveness of proposed method.
Critical weaknesses
  1. The presentation of the paper need major improvement. It's hard to follow and understand when reading first time. For example, Figure 1 is difficult to read without legend and some terms are used before define them, e.g. "cell" in L162.

  2. There are mismatches between the numbers reported in Table 1 and Figure 5. For example, Qwen2.5-VL-7B achieves >76 on V* in Figure 5, while 70.68 is reported in Table 1.

  3. The baselines numbers are significantly lower than the ones reported in the original paper, which makes the conclusion not convincing. For example, Deepeyes reports 90.1 on V* while the Table 1 it's 84.21. Additionally, for zero-shot Qwen2.5-VL-7B, ACTIVE-o3 [1] reports 67.9 while the paper reports 60.6.

  4. Comparisons, citation and discussion with similar RL-based methods are missing, e.g. Adaptive-CoF [2] and Mini-o3 [3], which target at the same problem setting and report better results on some benchmarks like V* than REFINE.

  5. There is no explanation about the training data. It's hard to tell whether the improvement comes from model learns to actively reasoning over visual cues or in-domain training data brings the performance gain.

  6. The experiments are solely based on Qwen2.5-VL, which is kind of outdated. Does the proposed method generalize to latest MLLMs like Qwen3VL / Qwen3.5?

[1] Zhu, M., Zhong, H., Zhao, C., Du, Z., Liu, M., Huang, Z., Li, A., Chen, H., Zou, C., Chen, J. and Yang, M., 2025. ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning. arXiv preprint arXiv:2505.21457.

[2] Zhang, X., Gao, Z., Zhang, B., Li, P., Zhang, X., Liu, Y., Yuan, T., Wu, Y., Jia, Y., Zhu, S.C. and Li, Q., 2025. Adaptive chain-of-focus reasoning via dynamic visual search and zooming for efficient vlms. arXiv preprint arXiv:2505.15436.

[3] Lai, X., Li, J., Li, W., Liu, T., Li, T. and Zhao, H., 2026, April. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search. In International Conference on Learning Representations (Vol. 2026, pp. 76722-76746).

Questions for authors
  1. What is the "consistency reward" in Figure 3? Was it part of the training reward in Eq (5)?

  2. Is Figure 5 produced from REFINE's own training and evaluation runs?

  3. Why are the reported metrics so far below the published numbers?

  4. What training data was used, such as source, size, task mix?

Reviewer_25SvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces REFINE, a framework that trains vision-language models to selectively inspect image regions and retain useful visual evidence. It updates a fixed set of spatial memory slots rather than continually append new crops to the context. The authors also introduce an auxiliary reward based on how much these updates increase the model’s likelihood of the reference answer. Experiments on visual search, mathematical reasoning, and perception benchmarks show improvements over a GRPO baseline, while ablations suggest that both the memory design and the reward contribute to the gains.

Critical strengths

1.The paper addresses a practical problem in active visual reasoning: how to retain useful observations without continually expanding the visual context. The spatial replacement mechanism is intuitive and easy to follow. 2.The answer-support reward provides a reasonable way to distinguish useful inspections from redundant ones, beyond simply rewarding a correct final answer. 3.The ablations separate the effects of spatial storage and utility-aware training, and suggest that the two components work well together. 4.The evaluation covers several task families, with improvements over GRPO on most reported benchmarks rather than only on visual search.

Critical weaknesses
  1. The paper does not establish whether fixed-capacity visual storage improves practical efficiency. Since each update requires rebuilding the decoder prefix, the computational cost may offset the storage benefits.
  2. Important experimental details are missing, including training data composition, crop resolutions, and inspection budgets. It is also unclear whether computational budgets and V score aggregation are consistent across methods.
  3. The answer-support reward may reflect increased confidence in the reference answer rather than improved visual evidence. The current experiments do not sufficiently distinguish these explanations.
Questions for authors
  1. Could the authors report inference latency, peak memory usage, and training overhead against append-only and FIFO baselines, including the cost of prefix reconstruction?
  2. Could the authors provide the missing experimental settings and clarify whether all methods use matched computational budgets and the same aggregation protocol?
  3. Could the authors test relevant, irrelevant, and corrupted crops to determine whether the utility signal tracks improvements in visual evidence rather than confidence alone?
Reviewer_Z4YQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes REFINE, an RL framework that integrates acquired evidence into a spatial storage. The spatial storage is consisted of K regions, each with m+s vectors. It is initialized with pooled vision encoder features, and regions can be updated with high-resolution results of inspection actions. A frozen answer probe comparing initial state and final state of evidence is used to provide GRPO signals. Results show that REFINE outperforms the GRPO baseline for around 2.5%.

Critical strengths
  1. The observation of the problem of growing context and the solution with a fixed-budget evidence pool is interesting. By leveraging this zoom-and-substitute queue-like mechanism, the architecture successfully controls context.
  2. The analysis section is solid, and Figure 3 and 4 are informative, suggesting that training is quite stable and effective.
Critical weaknesses
  1. Limited novelty. Though there are innovations in the structured evidence design, this paper looks similar to the majority of visual search literature: zooming in for fine-grained visual information, trained with RL. There are few things people can learn from this paper.
  2. Not impressive experimental results. When reading Table 1, it is worth pointing out that a naïve GRPO of Qwen2.5-VL is actually similar to the proposed method. The performance gain is between -0.36% and 3.15%. This questions the necessity of the complex spatial storage design and extended inference time.
  3. Experiments are on an ancient model. Qwen3-VL family came out a year before ICLR 2027’s submission due; however, this paper still keeps its experiments with Qwen2.5-VL. It is known to the community that improvements from many methodological designs narrow down or disappear as VLMs get more powerful; as a result, whether the paper is still valuable this year remains unknown.
  4. Writing needs more polishing. Overall, the paper is not easy to read due to notation. This is surprising when the math is simple. Figures are not well-designed. Figure 1 looks like a placeholder, without reporting precise numbers or comparison with other models.
Questions for authors
  1. What happens when two zoom-ins land in the region under one slot? Will one get deserted?
  2. What is the meaning of keeping slots without a zoom-in, as they are pooled features that already exist in I?
  3. If possible, can you show the performance of your method based on Qwen3-VL-8B compared with Qwen3-VL-8B-GRPO?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.