acceptodds
Under review as a conference paper at ICLR 2027

BiVER: Bidirectional Visual Evidence Rewards for Multi-Tool VLM Reasoning

Abstract

Vision Language Models (VLMs) can actively manipulate images through tool calls during reasoning, yet common reinforcement learning objectives provide little direct feedback on the images these tools produce. An irrelevant tool output thus inherits a positive reward whenever the answer is recoverable without it, while an informative one receives no credit when the final answer is wrong. We show that a frozen VLM can assess returned visual evidence along two complementary directions: a *forward* answer-utility score measures support for the reference answer, while a *backward* question-alignment score measures relevance to the question without using the answer. Building on this finding, we propose **BiVER** (**Bi**directional **V**isual **E**vidence **R**eward) for multi-tool VLM reasoning. It scores intermediate tool outputs in both directions and incorporates their trajectory-level aggregate into GRPO. Because BiVER evaluates the image produced by an executed action rather than its name or arguments, it provides a shared reward criterion for heterogeneous image-producing tools without supervised tool-use trajectories or training a dedicated reward model. Trained from Qwen2.5-VL-7B-Instruct with six tools, BiVER improves its initialization by an average of **11.9** points across high-resolution benchmarks and **5.0** points across document, chart, and table benchmarks, achieving the highest average accuracy among the compared models for each benchmark suite.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
VLM Reasoning, RL

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HbWDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a bilateral visual evidence reward, utilizing a frozen VLM to generate forward answer utility + backward question alignment for bidirectional scoring. It directly evaluates the image content output by tools, achieves unified evaluation across heterogeneous tools, aggregates trajectory score embeddings for GRPO optimization, and eliminates the need for tool usage annotations and dedicated reward models, significantly enhancing the reasoning performance of multi-tool VLMs.

Critical strengths
  1. The proposed reward effectively evaluates the results of intermediate tools and can supervise the intermediate steps of trajectories.
  2. The experimental results have shown significant improvement compared to the Qwen2.5 VL 7B.
  3. S_A and S_Q effectively provide assistance in reasoning answers based on tool results, which is of reference significance for tasks related to tool call.
Critical weaknesses
  1. The presentation of the paper is not clear enough. It is recommended to provide some examples or visualizations to aid understanding of the design ideas behind S_A and S_Q.
  2. The authors introduce two new reward dimensions to provide supervision for the effect of tool use process, but GRPO pipeline provides a reward for the entire response. It remains to be discussed whether the overall reward can make the model aware that this reward is related to tool use process.
  3. The evaluator is consistent with the training model. Is it possible that using other models for supervision can better cover the shortcomings of the training model.
Questions for authors

Please see weaknesses above.

2 comments

We thank the reviewer for the positive assessment and constructive suggestions. BiVER aggregates scalar scores (the mean of s_Q and s_A over the tool outputs of a trajectory), not embeddings; the aggregate is standardized within each GRPO group and added to the outcome reward (Eq. 5).

W1: Examples illustrating the design of S_A and S_Q

  • s_Q (backward question alignment): "Does this tool-output image contain evidence relevant to the question?" It never sees the answer, and so stays informative on trajectories that end in a wrong answer.
  • s_A (forward answer utility): "Given this tool-output image, how likely is the reference answer?" It tests whether the evidence suffices to produce the answer.

Table R1: Scores at the two ends of the candidate range (VizWiz, averaged over 1,769 examples; each tool output is scored in isolation).

Tool outputs_Qs_ACrop-only VQA
GT (human-annotated evidence region)−0.25−3.3470.07
Blank (all-black image)−3.54−7.9010.74

The three intermediate conditions (Self / Center / Corner, at 61.21 / 48.39 / 34.08 crop-only VQA) interpolate monotonically in both scores. The two directions thus move together on average; their complementarity shows up on individual examples:

Table R2: Real cases where s_Q and s_A disagree

CaseQuestion / reference answerTool outputs_Qs_AEvaluator's answer
Relevant but insufficient"What flavor is this?" / alfredocenter crop (on the packaging text)+2.12−7.67"Creamy Parmesan" (wrong)
Incidental support"What is this a bottle of?" / watercorner crop (minimum GT overlap)−4.75−0.32"water" (correct)

With s_Q alone the first case would be rewarded despite being insufficient for the answer, and with s_A alone the second would be mistaken for useful evidence; only their combination rewards tool outputs that are both question-directed and answer-sufficient. This is consistent with the reward ablation (Fig. 4, right): averaged over the seven benchmarks, S_Q only reaches 69.0 and S_A only 69.2, while their combination reaches 70.6.

W2: Can a trajectory-level reward make the model aware that it concerns the tool-use process?

The evaluator receives no reasoning text, tool name, arguments, or final answer, and all trajectories within a group share the same question, so differences in Z_i within a group arise solely from differences in the returned images. Over our full training run, 42.6% of groups have identical outcome rewards across all their rollouts (zero outcome advantage, hence no gradient), and 88.8% of those receive a non-zero BiVER signal — that is, for 37.8% of all groups the gradient comes solely from the quality of the tool outputs. Consistently, the mean s_Q of the policy's own tool outputs rises steadily from −1.33 early in training to about 0.06 later on.

W3: Could supervision from a different model better cover the training model's shortcomings?

We deliberately use a frozen copy of the initial policy as the evaluator: the gains then cannot be attributed to distillation from a stronger teacher, and a frozen evaluator cannot be exploited through self-reinforcing drift. Its task is also easier than the policy's — it only needs to rank tool outputs within a group, from a single already-focused image. The same construction yields consistent gains on Qwen3-VL-4B/8B (Appendix E).

Table R3: Frozen evaluators compared (500 VizWiz examples, paired; both measured against the same fixed crop-only VQA labels; best-of-N selects among Self / Center / Corner / Blank, Self baseline 64.12).

Evaluators_A ranking acc.s_Q ranking acc.Best-of-N by s_ABest-of-N by s_Q
Qwen2.5-VL-7B (self-evaluator, ours)92.487.771.2467.78
Qwen2.5-VL-32B (4.6× larger)91.990.070.3469.80

Within-group ranking ability thus depends little on evaluator scale: the 7B self-evaluator matches the 32B model on answer utility (92.4 vs. 91.9), and the 32B model's edge on question alignment (+2.3) is not large enough to be conclusive at 500 examples. The self-evaluator therefore already provides the discrimination needed for within-group ranking; a stronger evaluator is a plug-in component orthogonal to the method, which we will discuss as future work.

We hope these clarifications address the reviewer's concerns, and we would be glad to answer any further questions.

Thanks again for your thoughtful feedback. We have uploaded a revised manuscript and highlight below the updates most relevant to your comments, building on Tables R1–R3 in our earlier response.

W1–W2. How the two reward components shape crop behavior (Sec. 5.4, Table 6). We add a controlled zoom-only analysis on V∗V^* Bench. With question alignment alone (sQs_Q), the policy achieves 96.9% center hit and 94.8% full-box containment, with crops covering 45.6% of the image on average. Compared with alignment alone, answer utility alone (sAs_A) produces smaller crops (9.7% mean area) with lower target coverage (64.4% center hit). BiVER’s crop geometry is close to that of the alignment-only policy. Its mean crop area is 46.5%, and it increases center hit from 85.9% to 95.8% and full-box containment from 21.5% to 94.2% relative to outcome-only training. In the separate six-tool reward ablation (Fig. 4, right), BiVER achieves higher average task accuracy than either single-direction variant.

W2. Benefit from test-time tool access (Sec. 5.4, Table 5). Disabling tools for the same trained BiVER checkpoints reduces the average score across V∗V^*, HR-Bench 4K, and HR-Bench 8K by 5.9 points on Qwen2.5-VL-7B and 5.8 points on Qwen3-VL-8B. This comparison measures how much the trained policies benefit from tool access at inference.

W3. Evaluator choice and backbone generalization (Appendices D and H). Appendix D (Table 9) reports results on Qwen3-VL-4B and Qwen3-VL-8B. Appendix H discusses the use of a larger or external evaluator as a possible extension.

We appreciate your suggestions on how to make the roles of the reward components and the contribution of test-time tool access clearer.

Reviewer_6kAaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The basic premise behind this work is that in VLMs that use thinking-with-images paradigm, the reinforcement learning objectives do not provide feedback on the utility of the tool, i.e., if a tool invocation was wrong, a correct answer attributes positive reward regardless As a remedy, the paper proposes Bidirectional Visual Evidence Reward (BiVER) which adds two additional reward components to standard GRPO objective (i) forward answer-utility score that measures how well the tool output supports the reference answer and (ii) backward question-alignment score that evaluates if the output contains evidence to answer the question through a frozen VLM judge. The paper motivates the need for both components through a case-study and empirically verifies that BiVER improves the performance of Qwen2.5-VL-7B across several benchmarks.

Critical strengths

S1. The paper presents a simple and intuitive approach for evaluating the usefulness of tool-calls in thinking-with-images paradigm, and proposes a reward formulation that lets the VLMs receive feedback if the invoked tool genuinely helped generate the correct answer. The method is validated empirically across several tools and benchmarks indicating that the approach is not specific to certain tools.

Critical weaknesses

W1. The major motivation behind this paper is not entirely novel [1,2,3,4]. While the authors do not claim that as their contribution, the proposed method BiVER is fairly similar in construction to CodeV [1] given both employ a VLM-based judge to evaluate "whether the current tool result provides any useful evidence for answering the question", with the exception of the answer-utility component in BiVER. However, this paper does not discuss how BiVER differs in motivation and how is it situated compared to CodeV given the combination of both forward and backward scores only yields minimal advantage as opposed to just backward only score (70.6 vs. 69.0), which is in principle conceptually similar to CodeV's TAPO.

[1] Hou, X., Xu, S., Biyani, M., Li, M., Liu, J., Hollon, T. C., & Wang, B. (2026, June). CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 21500–21510.

[2] Yang, W., Xia, Y., Huang, J., Lu, S., Chen, Q.-G., Xu, Z., … Chua, T.-S. (2026). Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization. arXiv [Cs.CV]. Retrieved from http://arxiv.org/abs/2604.06777

[3] Su, Z., Li, L., Song, M., Hao, Y., Yang, Z., Zhang, J., … Cheng, Y. (2025). OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning. arXiv [Cs.CV]. Retrieved from http://arxiv.org/abs/2505.08617

[4] Guo, Z., Hong, M., Zhang, F., Jia, K., & Jin, T. (2026, June). Thinking with Programming Vision: Towards a Unified View for Thinking with Images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 33467–33476.

W2. It is unclear why BiVER helps boost the performance of underlying VLMs. Is it due to more tool-calls (even if unnecessary), better aligned tool-calls, or because it discourages unnecessary tool-calls. The tool-call behavior analysis in Sec. 5.2 is limited. An in-depth analysis discussing how the behavior varies with BiVER and each of its components while comparing it to prior methods would better motivate the proposed method.

Questions for authors

None

7 comments

We thank the reviewer for the careful reading and for recognizing that BiVER is simple, intuitive, and validated across several tools and benchmarks.

W1: Relation to CodeV/TAPO and other prior work.

We agree that the lack of supervision on intermediate tool use under outcome-only rewards has been noted in recent work, and we do not claim this observation as our contribution. Our contribution is a richer supervision signal on tool outputs from two complementary directions.

(1) Differences from CodeV [1]. TAPO's tool reward only asks a judge whether a tool output helps answer the question; BiVER scores it from two directions.

CodeV [1]BiVER
DirectionsQuestion side only (any useful evidence?)Question side (s_Q) + answer side (s_A)
Evaluator inputQuestion + tool output + metadata (coordinates, statistics)Question + returned image only
ScoreRubric judgment, fixed weightLog-odds / log-likelihood, standardized per direction within each GRPO group
ToolsCrop-centric; r_tool ≈ 0 without visual searchOne criterion for six heterogeneous tools
PipelineSFT + RLRL only; no curated tool-use trajectories

BiVER also validates both scores before using them as rewards (Section 3): their pairwise agreement with crop-only VQA is 90.6% and 86.2%.

(2) The two directions are complementary. Seven-benchmark average (same six tools and outcome reward): outcome-only 68.6, backward-only (closest to CodeV) 69.0 (+0.4), forward-only 69.2 (+0.6), BiVER 70.6 (+2.0). BiVER is best or tied-best on all seven benchmarks (per-benchmark gains in Table R1 below and Figure 4, right). A question-aligned crop may truncate the answer, which only s_A detects; an answer-supporting view may be incidental, which only s_Q detects (Section 4.2). A question-side-only signal therefore misses part of the benefit.

(3) Other works.

  • Walk the Talk [2] rewards CLIP similarity between a self-generated label and the returned image. This measures consistency with the policy's own intent, not relevance to the question or support for the answer; a well-matched label can describe an irrelevant crop. BiVER anchors both scores to the question and the reference answer.
  • OpenThinkIMG [3] optimizes task success in RL and does not score intermediate visual outputs.
  • CodeVision [4] rewards annotated required tools with tool-specific rules (e.g., IoU for crops). BiVER uses one tool-agnostic criterion and needs only standard VQA annotations.

BiVER's contribution is a tool-agnostic, self-contained, bidirectional reward on the realized visual result of an action. We will clarify this in the Introduction and Related Work of the revised version and discuss [1–4] in detail.

W2: Why does BiVER help?

We acknowledge that Section 5.2 provides too little analysis: Figure 3 compares with DeepEyes, whose action space contains only zoom-in, so it cannot isolate the effect of the reward.

(1) BiVER rewards better tool outputs, not more calls.

  • More calls: S_Q and S_A are means over unique outputs, so extra or repeated calls do not raise the reward (Appendix A).
  • Better-aligned calls (main mechanism): selecting the highest-scoring output raises crop-only VQA from 61.21 to 67.72 (s_A) / 64.22 (s_Q) (Section 3); with identical tools, BiVER adds 3.8 points over outcome-only training on DocVQA-RF (Section 5.3).
  • Unnecessary calls: discouraged indirectly. Z_i is standardized within each group, so a trajectory with less informative tool outputs than its group gets a negative shaping term even if its answer is correct.

(2) Role of each component.

Table R1: Gain over outcome-only training (same six tools; three representative benchmarks, full results in Figure 4, right).

VariantV*DocVQA-RFCharXiv
Backward (s_Q)+1.1+0.7+1.6
Forward (s_A)+0.5+2.0+1.1
BiVER+2.1+3.8+2.4

Backward alignment helps more on locating small regions (V*, CharXiv); forward utility helps more when the view must be readable enough to give the answer (DocVQA-RF). Their combination exceeds either direction alone on all three benchmarks.

We will add more detailed results and comparisons with prior methods in the revised version. We thank the reviewer again and are glad to answer further questions.

I thank the authors for their comments and additional analysis. A few differences between CodeV and BiVER are clarified by the authors in response to W1, but the performance gains from the bidirectional scoring remain modest. Further, W2 is unaddressed. The properties of the proposed reward formulation is discussed, but there is no empirical analysis on how the learned policy's tool-use behavior changes. It remains unclear whether improvements arise from more frequent, better-aligned, or fewer unnecessary tool calls? While the ablation results in W1 indicate that combining forward and backward scores improve downstream performance, they do not show whether the two modules induce complementary tool-use behaviors in the learned policy.

We thank the reviewer for emphasizing the need to measure how the reward changes the learned policy’s tool use. We add a controlled analysis of crop geometry in Sec. 5.4 (Table 6) and expand the discussion of related work [1–4] in Sec. 2.1.

Are the gains associated with more frequent calls, better-aligned calls, or fewer unnecessary calls?

To isolate how reward design shapes executed crop geometry without cross-tool confounds, Sec. 5.4 (Table 6) evaluates controlled zoom-in policies trained under Outcome only, Alignment (sQs_Q), Utility (sAs_A), and BiVER (sQ+sAs_Q + s_A) against human ground-truth target boxes on V∗V^* Bench (Hit: GT target center inside crop; Contain: full GT box inside crop without clipping):

Controlled zoom PolicyGT Center HitFull-Box ContainCrop Area (% Image)
Outcome only85.9%21.5%1.7%
Alignment (sQs_Q)96.9%94.8%45.6%
Utility (sAs_A)64.4%56.0%9.7%
BiVER (sQ+sAs_Q + s_A, zoom)95.8%94.2%46.5%
  • Similar call frequency: On the single-target V∗V^* queries, the four zoom-only variants average approximately 1.0–1.2 calls per query (Sec. 5.4).
  • Higher target coverage: Compared with outcome-only training, BiVER increases center hit from 85.9% to 95.8% and full-box containment from 21.5% to 94.2%, while increasing mean crop area from 1.7% to 46.5%.
  • Benefit from test-time tool access: Disabling tools reduces BiVER-7B’s average score by 5.9 points across V∗V^*, HR-Bench 4K, and HR-Bench 8K (Table 5). The frequency of unnecessary calls remains unmeasured.

Do sQs_Q and sAs_A induce complementary tool-use behaviors in the learned policy?

The single-direction rewards induce distinct crop behaviors. Alignment-only training produces wider crops and higher target coverage, while utility-only training produces smaller crops with lower coverage. BiVER’s crop geometry is close to that of the alignment-only policy. In the separate six-tool reward ablation (Fig. 4, right), combining both signals improves average task accuracy over either single-direction variant. Table 6 thus shows where the two components differ in crop behavior, while Figure 4 shows the downstream benefit of combining them.

We also clarify the relationship to CodeV/TAPO [1] in Sec. 2.1. Both methods evaluate whether a tool output provides question-relevant evidence. BiVER additionally scores how strongly the returned image supports the reference answer and combines the two signals at the trajectory level. The alignment-only variant is our closest internal comparison to TAPO’s question-relevance criterion.

We thank the reviewer again and hope the new behavioral analysis and revised manuscript resolve your concerns.

[1] Hou et al., CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization, CVPR 2026.

[2] Yang et al., Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization, arXiv:2604.06777, 2026.

[3] Su et al., OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning, arXiv:2505.08617, 2025.

[4] Guo et al., Thinking with Programming Vision: Towards a Unified View for Thinking with Images, CVPR 2026.

Reviewer_HF1dOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies reinforcement learning for "thinking with images" VLMs. It argues that existing methods such as DeepEyes reward only the final answer and therefore cannot tell whether the images produced by intermediate tool calls are actually useful: a trajectory that crops the wrong region but still answers correctly from the original image or from prior knowledge is rewarded, while a trajectory that crops the right evidence but then misreads it receives no signal at all. The authors first run an intervention study on VizWiz-VQA-Grounding, replacing the model's crop with a GT evidence box, the model's own box, same-sized center/corner crops, and a blank image, and find that a frozen Qwen2.5-VL-7B can roughly rank these candidates from the question and the tool output image alone. Building on this, they propose BiVER, which uses a frozen copy of the initial model to score each tool output image in two directions: a forward answer-utility score, defined as the length-normalized likelihood of the reference answer conditioned on the image, and a backward question-alignment score, defined as the Yes/No log-odds of whether the image contains the evidence needed to answer the question (without using the answer). The two scores are averaged within each trajectory, standardized separately within each GRPO group, combined with equal weights, standardized again, and added to the DeepEyes outcome reward with weight λ = 0.3. Because only the produced image is scored, not the tool name or arguments, the same reward applies to all six tools: cropping, rotation, flipping, horizontal/vertical line drawing, and point marking. Initialized from Qwen2.5-VL-7B and trained on about 50K prompts (the DeepEyes training set plus 3K DocVQA samples, 70% of them rotated or flipped), BiVER is reported to improve over its initialization by 11.9 points on average on V*, HR-Bench 4K/8K, and by 5.0 points on average on DocVQA-RF, InfoVQA-RF, CharXiv, and TableVQA. The appendix adds results on Qwen3-VL-4B/8B, and the paper includes reward and tool-set ablations and an analysis of tool-use distributions.

Critical strengths
  1. The paper targets a real flaw of outcome-only rewards, and the motivating study is cleanly designed. The authors clearly point out that outcome-only rewards cannot distinguish trajectories whose tools produce useful evidence from those that answer correctly from the original image or from priors, which is a genuine credit-assignment problem in tool-use RL. The motivating study replaces the returned image of the same valid call with five candidates (GT, SELF, CENTER, CORNER, BLANK), keeps the question and the call text fixed, and shows the evaluator only the question and a single tool output image, which rules out confounds such as the number of dialogue turns or call formatting. The scores are validated from three angles (crop-only VQA, pairwise ranking agreement, and downstream selection), so the argument is well supported.
  2. By scoring only the image a tool produces, the reward provides a unified, low-cost process-supervision interface for heterogeneous tools. Since it does not depend on the tool name, the arguments, or the original image, the same scoring applies directly to cropping, rotation, flipping, line drawing, and point marking. It needs no action-specific rule rewards (such as crop-box IoU), no annotated tool trajectories, and no dedicated reward model. Using a frozen copy of the initial model as the evaluator rules out distillation from a stronger teacher as an explanation, and the backward question-alignment score does not use the answer, so it still provides signal on trajectories that end in a wrong answer. The tool-use distribution also supports this: rotation and flipping together account for 46% of calls on document tasks while cropping dominates on high-resolution tasks, which shows that the policy chooses tools by task under a single reward, and the policy composes tools (e.g., rotate first, then zoom in).
Critical weaknesses

1. The paper does not discuss key work in visual CoT, in particular the two challenges that Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026, hereafter ZwZ) raises against the "thinking with images" paradigm.

ZwZ makes two central arguments against tool-calling visual CoT methods such as DeepEyes. Both bear directly on this paper's premise, yet the paper neither cites nor discusses ZwZ:

  • Inference cost. Tool-calling methods require multi-turn interaction and repeated re-encoding of images; ZwZ reports that its single-forward-pass inference is about 10× faster than agentic baselines. The policy in this paper makes up to 5 tool calls with a generation limit of 20K tokens, yet the paper reports no inference latency or token cost at all. If accuracy is comparable, choosing a paradigm that costs several times more requires additional justification.
  • Source of the gains. ZwZ argues that the gains of visual CoT come mainly from CoT and RL (GRPO/DAPO) themselves, and have little to do with acquiring new visual evidence at inference time. Its Table 10 shows that DeepEyes loses only 2.0 points on average when tools are disabled and it answers directly (74.4 → 72.4), and Thyme loses only 0.6 points (74.0 → 73.4). In its Table 3, text-only RL without tools on the DeepEyes training data already brings Qwen3-VL-8B to 88.5 / 84.8 / 80.1 on V* / HR-4K / HR-8K.

This challenge is especially important for this paper, since its central claim is precisely to reward the visual evidence produced by tools. Yet on the same backbone, Qwen3-VL-8B, BiVER (Appendix D) reaches 93.2 / 81.4 / 81.6 on V* / HR-4K / HR-8K (average 85.4), while ZwZ-8B, which uses no tools and a single forward pass, reaches 91.1 / 84.4 / 82.0 (average 85.8). On HR-4K, BiVER is even below the "DeepEyes data + tool-free RL" result above. Although evaluation settings differ across papers (e.g., this paper reports 68.1 for the Qwen3-VL-8B base on HR-8K, whereas ZwZ reports 74.6), these results still indicate that, after introducing six tools, a process reward, and multi-turn calls, BiVER's advantage over text-only RL is unclear.

Moreover, the paper lacks the key experiment for separating the two sources of gains: how much does the trained BiVER drop when tools are disabled and it answers directly? If the drop is similar to that of DeepEyes, the process reward more likely improves the policy's own reasoning rather than its ability to acquire evidence with tools, which would undermine the paper's core narrative.

2. Averaging scores within a trajectory penalizes exploration and composition, undermining the very purpose of active visual search.

The paper averages the scores of all tool outputs in a trajectory, motivated by avoiding "more calls, more reward." However, by Eq. (7) in Appendix A, appending a new output lowers the trajectory score whenever that output scores below the current mean. Consequently, a trajectory that first crops the wrong region and then the right one receives a strictly lower process score than one that crops correctly on the first try, even when both answer correctly and have exactly the same R_base. After within-group z-scoring, this gap translates directly into a gap in advantage.

In search tasks, "it is not here" is itself useful information: ruling out wrong regions is a normal part of multi-step visual search (the setting targeted by, e.g., Mini-o3 and Visual Probe). Under this paper's setup, however, the evaluator sees only a single tool output image, so such exclusion calls necessarily receive low scores, and this kind of exploration is systematically penalized. As a result, the policy is pushed toward finding exactly the right image at every step. But if the model can localize the evidence precisely on its first call, it most likely already knows where the answer is without calling any tool, so what difference remains between such "search" and no search at all? The point of active visual search is precisely to allow trial and error and to converge to the evidence through it, and the current reward design runs counter to this.

The authors' own case study illustrates this tension. In the "rotate 270°, then zoom in" trajectory of Figure 5, the output of the first rotation is the full page, on which the target text is too small, so its support for the answer is likely low and it pulls down the trajectory mean. In other words, the success case the paper uses to showcase tool composition belongs exactly to the kind of trajectory that this aggregation is likely to penalize. The paper explicitly states that the mean gives no unconditional reward for more calls, but does not discuss that it also penalizes conditional, necessary calls.

A more reasonable design would adapt the process reward to question difficulty. For easy questions, where a single precise crop (or even no tool call) suffices, rewarding precise and concise calls is reasonable. For hard questions, however, the location of the evidence is itself uncertain and multi-step exploration is necessary; intermediate "misses" should not be penalized across the board as they are now, but trial and error should be tolerated and the process that eventually converges to the correct evidence should be rewarded. Requiring every step to be correct may be harmless on easy questions, but on hard questions that truly require search, it suppresses exactly the capability the model needs most.

3. The improvements over baselines are limited, and because in-distribution training data is added, it is hard to tell whether the gains come from data or from the algorithm.

Compared with the most relevant baselines, BiVER's advantage is small. On the high-resolution benchmarks, its average is only 2.0 points above ToolsRL (82.5 vs. 80.5), and it is below ToolsRL on V* (91.1 vs. 92.5). On the document/chart benchmarks, its average is only 1.9 points above DeepEyes* (61.6 vs. 59.7), and it is below DeepEyes* on InfoVQA-RF (58.1 vs. 59.7).

More importantly, these comparisons are confounded by differences in training data. Compared with DeepEyes, the paper adds 3,000 DocVQA samples, 70% of which are rotated or flipped, while the evaluation sets DocVQA-RF / InfoVQA-RF are constructed by the authors in the same way (70% rotated/flipped). That is, the training data contains samples from the same distribution as the evaluation, whereas DeepEyes* uses neither this data nor any tool other than zoom-in. Therefore, the largest improvement in Table 2 (5.0 points over DeepEyes* on DocVQA-RF) cannot be attributed to in-distribution data, the larger tool set, or the proposed reward. The tool-set ablation is similarly confounded: as the authors acknowledge, the no-tool configuration also changes the reward.

The only comparison that fixes both data and tool set and varies only the reward is Figure 4 (right): the average over seven benchmarks rises from 68.6 (outcome-only) to 70.6 (BiVER), a gain of only 2.0 points. Adding alignment or utility alone yields gains of only 0.4 and 0.6 points respectively, and on HR-Bench 4K/8K either direction alone is worse than outcome-only. Overall, the current experiments do not show that BiVER's gains come mainly from the proposed bidirectional visual evidence reward rather than from changes in data scale and distribution.

Questions for authors
  1. How much does the trained BiVER drop on each benchmark when tools are disabled and it answers directly? Could the authors add a tool-free, text-only RL baseline on the same training data (in particular, a direct comparison with ZwZ on Qwen3-VL-8B), and report BiVER's average number of tool calls, inference latency, and token cost per sample?
  2. Could the authors bucket training trajectories by the number of tool calls and report the distribution of process scores and advantages in each bucket, to show whether trajectories that explore first and hit later are systematically pushed down? For hard questions that require multi-step search, have the authors considered adapting the process reward to question difficulty, instead of uniformly requiring every intermediate call to be correct?
  3. Compared with DeepEyes and other baselines, the training data additionally includes 3K DocVQA samples, 70% of which are rotated or flipped, constructed in the same way as the evaluation sets DocVQA-RF / InfoVQA-RF and thus from the same distribution. Given this data confound, how can one tell whether BiVER's gains come from adding in-distribution data or from the proposed reward? For example, does BiVER's advantage over the baselines persist if this DocVQA data is removed, or if the same data is added to the baselines?
  4. The base model scores reported in this paper differ noticeably from those in other work, e.g., on HR-Bench 8K, Qwen2.5-VL-7B scores 64.4 (vs. 67.9 reported by ZwZ) and Qwen3-VL-8B scores 68.1 (vs. 74.6 reported by ZwZ). Are there differences in evaluation settings? The paper does not specify settings such as the maximum image resolution and decoding parameters used at evaluation. Could the authors provide these settings and show whether the gap between the base models and BiVER still holds under standard settings?
4 comments

We thank the reviewer for the thorough critique and helpful suggestions.

W1 & Q1: ZwZ Comparison, Inference Cost, and Tool-Disabled Control

We will discuss ZwZ [1] in Related Work, BiVER and ZwZ are complementary:

(1) Paradigm scope.

DimensionZwZBiVER
MechanismOffline R2I distillation into 1-pass attentionOnline multi-turn tools + bidirectional evidence reward
Supervision108B–235B teachers on 74K synthetic pairsFrozen initial policy + VQA labels
Geometric & multi-toolNot supportedComposable 6-tool space

(2) Same-backbone comparison. BiVER-7B beats ZwZ-7B on 3 benchmarks (+3.4 avg.); on Qwen3-VL-8B, ZwZ's harness has a +4.3 higher base (W4), while BiVER-8B gains far more over its base (+9.7 vs. +5.8) and leads +2.1 on V∗V^*.

BackboneMethodModeV*HR-4KHR-8KAvg.Base Avg.Δ\Delta Base
Qwen2.5-VL-7BZwZ-7B1-pass88.575.473.379.172.7+6.4
BiVER-7BTools91.179.177.482.570.6+11.9
Qwen3-VL-8BZwZ-8B1-pass91.184.482.085.880.0+5.8
BiVER-8BTools93.281.481.685.475.7+9.7

(3) Inference cost. Tmax⁡=5T_{\max}=5 and 20K tokens are safety caps; BiVER averages 2.18 calls and ~340 tokens (2.8× 1-pass latency).

MethodCapCallsOut TokensLatency
No-tool00.00~1450.84 s
DeepEyes*51.45~2301.81 s
Mini-o3*123.38~5804.65 s
BiVER52.18~3402.35 s

(4) Tool-disabled & tool-free RL controls. Disabling tools drops BiVER-7B/8B by 5.9/5.8 pts (vs. −2.0 DeepEyes, −0.6 Thyme), and BiVER beats same-data No-tool RL by +4.8/+5.1 pts.

BackboneVariantV*HR-4KHR-8KAvg.Δ\Delta
Qwen2.5-VL-7BBase (direct)76.471.164.470.6—
No-tool RL (50K; Fig. 4L)86.474.072.677.7Tool-free RL
BiVER-7B (no tools)84.873.671.476.6−5.9 vs. tools
BiVER-7B (tools)91.179.177.482.5+4.8 vs. No-tool RL
Qwen3-VL-8BBase (direct)84.374.868.175.7—
No-tool RL (50K)88.077.675.480.3Tool-free RL
BiVER-8B (no tools)87.476.974.679.6−5.8 vs. tools
BiVER-8B (tools)93.281.481.685.4+5.1 vs. No-tool RL

W2 & Q2: Trajectory Mean Aggregation & Difficulty-Adaptive Reward

Eq. (7) reflects a trade-off: deduplicated averaging prevents reward hacking via redundant calls, though setup or exclusion views (e.g., rotating a page before cropping in Fig. 5) can lower the mean. Multi-step use is preserved (~2.2 calls/trajectory; Figs. 3, 5) as (i) correctness dominates (λ=0.3\lambda = 0.3), so solved multi-step runs outweigh failed 1-step tries, and (ii) scores are standardized within rollouts of the same prompt, competing only against peers on the same input rather than 1-step easy items. Still, averaging can under-credit trial-and-error when lucky 1st-try hits co-occur in a group; adapting aggregation to difficulty via group pass rate (concise precision on easy items, miss-tolerant convergence on hard ones) is a natural extension we will discuss in Sec. 4.2 and App. A.

W3 & Q3: 3K DocVQA Protocol and Controlled Disentanglement

Our 3K train DocVQA and validation DocVQA-RF / InfoVQA-RF have 0 question_id overlap.

VariantDataV*HR-8KDocVQA-RFTableVQA
Qwen2.5-VL-7B (Base)—76.464.465.266.4
DeepEyes* (Official)47K89.073.167.767.7
BiVER (w/o 3K DocVQA)47K90.677.270.470.6
BiVER (Ours)50K (+3K)91.177.472.770.9

Without the 3K subset, 47K-only BiVER still beats DeepEyes*, complementing our Zoom-in only vs. Six tools ablation under fixed data and reward (Sec. 5.3, Fig. 4L, Table 4; full results in revision).

W4 & Q4: Base-Model Scores and Evaluation Settings

Inspecting ZwZ's official repo confirms that our evaluation harness is strictly aligned with official DeepEyes, with key differences from ZwZ as follows:

ProtocolOursZwZ
Input max_pixels≈12.85M\approx 12.85\text{M}≈16.78M\approx 16.78\text{M}
ResizeClient smart_resize (factor=28; 2nd factor=32 on Qwen3-VL-8B)1-step AutoProcessor (factor=32 on Qwen3-VL-8B)
Prompt & decodingGreedy (temp=0)Sampling (temp=0.7, top_p=0.8)
HR-8K (Base→\toBiVER)7B: 64.4→\to77.4 (+13.0); 8B: 68.1→\to81.6 (+13.5)16.78M native: 7B 67.8→\to78.6 (+10.8); 8B 74.4→\to84.8 (+10.4)

Our base scores match prior works under the 12.85M greedy protocol, while lower max_pixels, adaptive <think> prompt, and 28→32 resampling penalize tool-free Qwen3-VL-8B on 8K; under ZwZ's 16.78M native setting, base scores align with ZwZ and BiVER's >10-pt gains hold. We will add full settings in the revised version.

[1] Wei et al., Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception, ICML 2026.

The authors' rebuttal has largely addressed most of my concerns. However, I still strongly recommend that the authors include an analysis similar to "ZwZ" (i.e., W1) in the revised manuscript. This would demonstrate that at least half of the performance gain stems from reinforcement learning itself, rather than the "Thinking with Images" paradigm—a confounding factor that must be clarified (doing so will not diminish the significance of the work). Furthermore, the authors' response regarding W2 aligns with my own view: that the model should be able to adaptively allocate its "work" (computational effort) based on the difficulty of the problem. It is a pity, however, that no quantitative experimental results were provided to support this. In summary, I am willing to raise my score to increase the likelihood of the paper being accepted. It would be best if the authors could incorporate these two points into the paper as soon as possible, which would facilitate further discussion among myself, the other reviewers, and the Area Chair in the next phase of the review process.

We sincerely thank the reviewer for the constructive feedback. We have uploaded a revised manuscript that incorporates both recommendations into the main text and appendix:

ZwZ-style source-of-gain analysis in the main text (Sec. 2.2, Sec. 5.4 Table 5, and Appendix E).

  • Sec. 2.2 adds a dedicated discussion of ZwZ and its two challenges to the "thinking with images" paradigm.
  • Sec. 5.4 (Table 5) reports a same-data no-tool RL comparison and a test-time tool ablation for both Qwen2.5-VL-7B and Qwen3-VL-8B. Consistent with your observation, same-data No-tool RL accounts for roughly half of the total gain over the base model (+7.1 of +11.9 pts on 7B; +4.6 of +9.7 pts on 8B). With tools enabled, BiVER scores 4.8 and 5.1 points above same-data no-tool RL on 7B and 8B, respectively. Disabling tools at inference reduces the average scores of the same BiVER checkpoints by 5.9 and 5.8 points, respectively. Inference efficiency (2.18 calls, ~340 output tokens, 2.35 s vs. 0.84 s single-pass), the 47K control without 3K DocVQA, and the 16.78M cross-harness check are included in Appendix E (Tables 10–11).

Offline analysis of mean–max reward aggregation (Sec. 5.4 and Appendix A).

Under uniform averaging, adding a tool output whose score falls below the trajectory’s current mean lowers its aggregate score (Eq. 7). Consequently, an initial miss can reduce the process credit assigned to a trajectory even when a later call finds the relevant evidence.

To examine this effect, we rescore the same constructed trajectories for 1,769 VizWiz queries using the convex combination SD,iada=p^g,meantsD(Vi,t)+(1−p^g),maxtsD(Vi,t)S_{D,i}^{\mathrm{ada}} = \hat{p}_g \\, \mathrm{mean}_t s_D(V_{i,t}) + (1-\hat{p}_g) \\, \mathrm{max}_t s_D(V_{i,t}) with fixed weights p^g\hat{p}_g (Eq. 13 and Table 7). Reducing the weight p^g\hat{p}_g on the mean from 1.0 to 0.15 raises the two-step search trajectory’s average standardized process score from +0.09 to +0.75. Its win rate against the one-step Self crop increases from 34.3% to 54.5%. The offline analysis thus shows that assigning more weight to the best step increases the relative credit for recovering after an initial miss.

Equation 13 proposes using the within-group pass rate to set p^g\hat{p}_g during training. Our reported policies use uniform averaging, and whether adaptive aggregation changes how a trained policy allocates tool calls across prompts of varying difficulty remains an open question.

Aggregation (p^g\hat{p}_g)1-Step Self (ZZ)2-Step Search (ZZ)Center (ZZ)Corner (ZZ)Search > Self Win Rate
Uniform Mean (p^g=1.0\hat{p}_g=1.0)+0.42+0.09+0.16−0.6734.3%
Adaptive Easy (p^g=0.85\hat{p}_g=0.85)+0.37+0.25+0.11−0.7238.7%
Adaptive Med. (p^g=0.50\hat{p}_g=0.50)+0.26+0.53+0.01−0.8045.5%
Adaptive Hard (p^g=0.15\hat{p}_g=0.15)+0.17+0.75−0.07−0.8454.5%

Thank you again for your helpful suggestions, which have substantially strengthened the paper.

Thank you for your reply; it has answered most of my questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers and Area Chair,

Thank you for your careful evaluation and constructive feedback. We summarize the main strengths recognized in the reviews, our clarifications and additional experimental evidence, and the current discussion status.

(i) Strengths highlighted by the reviewers.

  • Motivation and controlled validation: The credit assignment problem in outcome-only rewards is well motivated, and the controlled intervention study validates evidence scoring through crop-only VQA, pairwise ranking, and downstream selection (Reviewer HF1d).
  • Unified supervision across tools: The approach is simple and tool agnostic, evaluating intermediate visual outputs across heterogeneous tools without annotated tool-use trajectories or a dedicated reward model (Reviewers HF1d and 6kAa).
  • Empirical effectiveness: The results show improvements over Qwen2.5-VL-7B and demonstrate the value of both reward components for tool based reasoning (Reviewer HbWD).

(ii) Clarifications.

  • Relationship to prior work (Sec. 2.1). We clarify that the contribution is bidirectional evaluation of the realized visual output: question alignment measures relevance without seeing the answer, while answer utility measures support for the reference answer. We expand the comparison with CodeV/TAPO and other related methods.
  • Process feedback through trajectory-level optimization. BiVER aggregates scalar evidence scores into a trajectory-level reward. Our training diagnostics show that 37.8% of all rollout groups receive a non-zero learning signal solely from differences in visual output quality, despite identical outcome rewards.

(iii) Additional experimental evidence.

  • Separating RL gains from test-time tool benefits (Sec. 5.4, Table 5; Appendix E). Same data no-tool RL accounts for roughly half of the improvement over initialization: 7.1 of 11.9 points on 7B and 4.6 of 9.7 points on 8B. BiVER additionally exceeds these no-tool RL baselines by 4.8 and 5.1 points, respectively. Disabling tools for the same trained BiVER checkpoints reduces accuracy by 5.9 and 5.8 points. These controls distinguish the substantial contribution of RL from the additional benefit of tool access.
  • Direct measurements of learned crop behavior (Sec. 5.4, Table 6; CharXiv follow-up). Under controlled zoom-only policies, call frequency on V∗V^\ast remains approximately 1.0–1.2 calls per query, while BiVER increases full box containment from 21.5% to 94.2% over outcome-only training. We acknowledge that its geometry on V∗V^\ast closely resembles alignment-only training. On CharXiv, evaluating the same policies shows that adding answer utility reduces median crop area from 77.0% to 50.8% and increases crops covering at most half the image from 21.0% to 48.7%. This provides additional evidence that combining the signals changes crop behavior. Separately, the fixed data, six-tool reward ablation improves average accuracy from 68.6% to 70.6%, versus 69.0%/69.2% for either component alone.
  • Data, evaluation, and inference cost controls (Appendix E). Removing the additional 3K DocVQA samples preserves improvements over DeepEyes* on the four reported benchmarks. Under the alternative 16.78M-pixel evaluation protocol, BiVER retains over 10 point gains on HR-Bench 8K for both backbones. We also report the inference trade off: 2.18 calls, approximately 340 output tokens, and 2.35 seconds per query, versus 0.84 seconds for 1-pass inference.

(iv) Current discussion status.

  • Reviewer HF1d increases the rating from 2 to 3 (weak acceptance) and states in the latest reply that our response answers most of their questions. The revised manuscript incorporates the requested source of gain analysis and quantitative aggregation diagnostic.
  • Reviewer 6kAa’s rating remains 2 (weak rejection). The reviewer acknowledges that the CharXiv experiment provides clearer empirical evidence of how combining the two reward components changes the learned policy’s crop behavior. We also clarify the relationship to CodeV/TAPO and provide additional behavioral analyses and controlled experiments.
  • Reviewer HbWD’s rating remains 3 (weak acceptance). Our responses address presentation, trajectory-level process supervision, and evaluator choice; no further reviewer reply appears in the discussion.

We hope these clarifications and additional results address the main questions raised during the discussion and help inform the final assessment. We sincerely thank all reviewers and the Area Chair for their time and consideration.

Best,

The Authors of Submission #24570

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.