acceptodds
Under review as a conference paper at ICLR 2027

VERA: Visual Evidence Reinforcement for Multimodal Reasoning

Abstract

Multimodal reasoning large language models (MRLMs) have made substantial progress, yet their reinforcement learning objectives still rely primarily on final-answer correctness, leaving intermediate visual evidence largely unsupervised. Such outcome-level feedback cannot determine whether a response is grounded in the image, preserves the information needed to solve the task, or reaches a correct answer through unreliable evidence. We introduce VERA, a visual evidence reinforcement framework that externalizes task-oriented visual evidence as a structured description separated from the reasoning process and final answer. VERA evaluates this evidence along two complementary dimensions: visual faithfulness, which requires every visual claim to be supported by the input image, and task sufficiency, which requires the question and evidence alone to recover the target answer. The two evidence rewards are combined with answer correctness and format compliance in group-relative policy optimization, allowing trajectories with similar answer outcomes to receive different credit according to their visual-faithfulness and task-sufficiency assessments. Across five MRLM backbones and eight multimodal reasoning benchmarks, VERA improves average accuracy by percentage points while reducing output-token consumption by –.

Primary area:
reinforcement learning
Keywords:
Multimodal reasoning, Vision-language models, Reinforcement learning, Visual evidence, Faithful reasoning, Multimodal agents, Evidence-grounded generation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_okULOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VERA, a reinforcement learning framework that supervises an explicit, task-relevant visual description in addition to the final answer. Two separately prompted judges assess whether the description is supported by the image and whether the question and description allow an image-blind model to recover the reference answer. These binary rewards are combined with answer correctness and format compliance for DAPO training:

=R_{\mathrm{ans},i} +0.5R_{\mathrm{suff},i} +0.5R_{\mathrm{faith},i} +0.5R_{\mathrm{fmt},i}.$$ The description is generated after the reasoning trace and serves as an auxiliary supervision target. Experiments cover five backbones and eight benchmarks, reporting an average accuracy improvement of approximately 5 percentage points over the starting checkpoints and reduced output-token consumption.
Critical strengths
  1. VERA provides a clear and implementable formulation of two complementary requirements for task-relevant visual descriptions: support from the image and sufficiency for answering the question. Section 3 explicitly distinguishes these properties, and the separate judge interfaces turn them into practical auxiliary training signals.

  2. The reported evaluation covers five backbones and eight benchmarks, with improvements in average accuracy and reductions in output-token consumption. The InternVL3.5-8B result is particularly encouraging. Table 2 also provides meaningful controlled evidence on Qwen3-VL-4B: retaining the response structure while removing both evidence rewards yields an average of approximately 69.93%69.93\%, compared with 75.64%75.64\% for full VERA. Figure 4 additionally characterizes how the evidence rewards change group-relative credit assignment on InternVL3.5-8B.

Critical weaknesses
  1. The incremental contribution over closely related evidence-supervision methods is limited and lacks a direct empirical comparison. Vision-SR1 already decomposes visual perception and language reasoning and rewards visual descriptions through image-blind answer recovery. VEPA explicitly trains question-conditioned evidence using a blind-reader sufficiency reward and answer-leakage checks. CCCaption jointly optimizes informational completeness and image-grounded correctness. I recognize the distinction established in Section 3: VERA requires sufficiency for an individual question, whereas CCCaption targets the completeness of a general image description. Nevertheless, task-conditioned sufficiency is itself already central to VEPA. VERA's remaining contribution appears primarily to combine this type of utility reward with direct image-faithfulness supervision in a joint answer-generation objective. The paper does not compare against closely related reasoning methods such as Vision-SR1 or PRCO. The component ablations support the usefulness of the proposed combination in its own configuration, but do not establish whether this incremental design offers an advantage over the closest alternatives.

  2. The strongest reported improvement lacks a matched training control, limiting attribution of the empirical contribution. InternVL3.5-8B improves from approximately 69.79%69.79\% to 81.23%81.23\%, the largest backbone-level gain in Table 1. The paper does analyze this backbone: Figure 3 reports training dynamics, and Figure 4 recomputes advantages after adding or removing reward components while holding sampled responses fixed. These analyses show how the rewards affect optimization signals, but do not measure the held-out accuracy of a policy retrained without the evidence rewards. The actual reward-removal training ablations in Table 2 are limited to Qwen3-VL-4B, whose gain over its initial checkpoint is approximately 3.91 points. Moreover, in the Qwen2.5-VL-based 7B comparison, Vision-R1 + VERA averages 61.98%61.98\%, below VL-Rethinker at 62.35%62.35\% and ReVisual-R1 at 64.13%64.13\%. Thus, the results demonstrate improvements over the starting checkpoints, but have yet to establish a clear advantage over the closest methods or isolate the source of the strongest gain. This is consequential given the limited methodological novelty.

Questions for authors
  1. Section 3 already makes the distinction between visual faithfulness and task-specific sufficiency clear. Given that Vision-SR1 and VEPA already use image-blind answer recovery to supervise visual descriptions, could you substantiate the additional benefit of VERA's combined objective through a matched comparison against a closely related reasoning method, such as Vision-SR1 or PRCO? A comparison using the same initial checkpoint, training data, rollout/update budget, and evaluation protocol would help establish the empirical value of the incremental design.

  2. Figure 4 provides an analysis of reward-induced advantage changes on fixed InternVL3.5-8B rollouts, while Table 2 provides retraining ablations on Qwen3-VL-4B. Could you extend the latter type of control to InternVL3.5-8B: retrain with answer-correctness and format rewards only, retaining the same response structure, data filtering, rollout/update budget, and answer-verification procedure? How much of the approximately 11.44-point improvement is retained relative to this matched control?

2 comments
AuthorsOpenReview ↗

Methodological Contribution and Comparison with Closely Related Methods (W1 / Q1)

We thank the reviewer for the constructive comments. To address the concerns regarding comparisons with closely related methods and the attribution of performance gains on InternVL3.5-8B, we conducted two additional experiments, presented in Tables A and B.

1. Methodological Contribution and Comparison with Closely Related Methods (W1 / Q1)

We acknowledge that image-blind answer recovery has been explored in prior work, including Vision-SR1 and VEPA, as a means of evaluating the task sufficiency of visual evidence. VERA builds on this direction by jointly incorporating visual faithfulness and task sufficiency as two independent reward signals into reinforcement learning for final-answer generation.

Specifically, Vision-SR1 primarily evaluates visual descriptions through image-blind answer recovery, while VERA additionally employs an independent visual judge to verify whether the factual claims in the caption are supported by the original image. VEPA focuses on improving visual evidence representations through evidence pre-alignment, while PRCO adopts an Observer–Solver framework to optimize visual evidence through downstream reasoning feedback.

VERA integrates answer correctness, visual faithfulness, and task sufficiency within a single think → caption → answer trajectory, without requiring an additional Observer–Solver training architecture. The two evidence judges receive different inputs and evaluate complementary properties of the generated caption.

To further examine the empirical benefit of this design, we conducted additional direct comparisons with Vision-SR1 and PRCO using Qwen3-VL-4B-Thinking. The results are presented below.

Table A. Comparison with closely related methods on Qwen3-VL-4B-Thinking. Each entry reports Avg@3 accuracy (%) / average output tokens.

DatasetOriginalw/o EvidenceVERAVision-SR1PRCO
MMStar71.0 / 149970.8 / 169875.6 / 100575.2 / 176475.1 / 1541
ChartQA85.1 / 149183.1 / 88688.0 / 56187.6 / 210387.1 / 1292
WeMath82.9 / 421079.4 / 287985.9 / 180686.2 / 532685.4 / 4895
MMMath84.0 / 419678.8 / 531285.0 / 352985.8 / 543283.4 / 5121
LogicVista63.8 / 1362559.6 / 715869.1 / 453167.4 / 1622468.2 / 13121
MathVerse74.5 / 563572.7 / 338980.6 / 207974.3 / 654078.4 / 6643
MathVision31.9 / 1266436.8 / 815039.8 / 963035.2 / 1798836.2 / 11215
MathVista80.6 / 375178.2 / 228781.1 / 132880.8 / 511279.6 / 4528
Avg71.73 / 588469.93 / 397075.64 / 305974.06 / 756174.18 / 6045

VERA achieves an average accuracy of 75.64% across the eight benchmarks, outperforming Vision-SR1 and PRCO by 1.58 and 1.46 percentage points, respectively. Compared with Vision-SR1, VERA achieves higher accuracy on six of the eight benchmarks. Compared with PRCO, VERA achieves higher accuracy on all eight benchmarks.

VERA also produces substantially shorter outputs, averaging 3,059 tokens, compared with 7,561 tokens for Vision-SR1 and 6,045 tokens for PRCO.

Furthermore, under the same structured caption-generation protocol, removing the visual-faithfulness and task-sufficiency rewards reduces average accuracy from 75.64% to 69.93%, a decrease of 5.71 percentage points. Since both configurations retain the same think → caption → answer structure, this comparison provides direct evidence of the additional benefit of evidence-level supervision. These results support the effectiveness of VERA's joint reward design and demonstrate its empirical advantages over closely related methods.

AuthorsOpenReview ↗

Attribution of Performance Gains on InternVL3.5-8B (W2 / Q2)

2. Attribution of Performance Gains on InternVL3.5-8B (W2 / Q2)

The original Figure 4 analyzes how different reward components affect group-relative advantages on fixed rollouts. To directly measure their effects on the trained policy, we conducted additional reward-removal training ablations on InternVL3.5-8B.

All configurations use the same training data, structured output format, training budget, and evaluation protocol, with only the corresponding reward components modified.

In particular, the w/o Evidence configuration retains the think → caption → answer structure and uses only answer-correctness and format rewards. This provides a matched training control for evaluating the additional contribution of visual-faithfulness and task-sufficiency rewards.

Table B. Reward ablations on InternVL3.5-8B. Each entry reports Avg@3 accuracy (%) / average output tokens.

DatasetOriginalVERAw/o Ransw/o Rfmtw/o Evidencew/o Rsuffw/o Rfaith
MMStar70.7 / 214880.6 / 146765.2 / 231278.2 / 199677.6 / 233679.7 / 245578.4 / 2992
ChartQA90.0 / 73992.2 / 73584.0 / 83291.5 / 81290.8 / 97891.3 / 92190.3 / 1121
WeMath78.4 / 389788.5 / 359276.1 / 421380.7 / 402978.5 / 401279.9 / 321279.3 / 4433
MMMath77.5 / 728787.8 / 559877.3 / 822179.3 / 892383.2 / 877679.5 / 798578.7 / 8221
LogicVista57.3 / 823180.4 / 435551.5 / 899675.4 / 902367.5 / 733472.5 / 883262.3 / 6736
MathVerse71.2 / 472586.4 / 459565.9 / 492279.6 / 467374.9 / 544284.5 / 577474.0 / 5509
MathVision29.6 / 1200449.0 / 674928.5 / 829739.1 / 999338.3 / 766338.4 / 932235.3 / 9982
MathVista83.6 / 255484.9 / 244474.6 / 332884.1 / 342480.6 / 331182.6 / 388183.1 / 4410
Avg69.79 / 519881.23 / 369265.39 / 514075.99 / 535973.93 / 498276.05 / 529872.68 / 5426

The complete VERA configuration achieves an average accuracy of 81.23%, improving upon the original InternVL3.5-8B checkpoint's 69.79% by 11.44 percentage points.

When both evidence rewards are removed, the w/o Evidence configuration still achieves 73.93%, corresponding to an improvement of approximately 4.14 percentage points over the original checkpoint. This shows that answer-correctness and format rewards alone also contribute to performance improvements.

Under the same output structure and training configuration, introducing visual-faithfulness and task-sufficiency rewards further increases accuracy from 73.93% to 81.23%, yielding an additional improvement of 7.30 percentage points. This difference corresponds to approximately 63.8% of the total 11.44-point improvement over the original checkpoint.

The remaining reward ablations further support the contribution of the complete objective. Removing answer correctness, format compliance, task sufficiency, or visual faithfulness reduces average accuracy to 65.39%, 75.99%, 76.05%, and 72.68%, respectively, all below the complete VERA configuration.

These retraining experiments complement the analysis in the original Figure 4. Figure 4 characterizes how evidence rewards modify group-relative credit assignment on fixed rollouts, while the new Table B directly measures their contribution to held-out accuracy after policy optimization.

In particular, the 7.30-percentage-point improvement over the matched answer-and-format-only control provides direct empirical evidence that the evidence-level rewards contribute substantially to the performance gains observed on InternVL3.5-8B.

We thank the reviewer for highlighting these experimental concerns. We have added direct comparisons with closely related methods and comprehensive reward ablations on InternVL3.5-8B, and will incorporate the corresponding results and analyses into the revised manuscript. We hope these additional experiments clarify the empirical contribution of VERA and address the reviewer's concerns. We would appreciate the reviewer reconsidering the assessment in light of these new results.

Reviewer_dLSKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VERA tackles a real blind spot in multimodal RL: we basically only reward "did you get the final answer right," which can't tell misreading the image apart from bad reasoning after reading it fine. The fix is to make models emit <think> → <caption> → <answer>, where the caption is a stand-alone, task-oriented description of the visual evidence.

Critical strengths
  1. The core split is genuinely clean. Faithfulness vs. sufficiency is a nice orthogonal decomposition, and the "hide the image from the sufficiency judge" trick is a clever way to actually operationalize it rather than just hand-wave. The motivation is backed by real evidence (right answer ≠ sound evidence; longer output ≠ better reasoning), not vibes.

  2. Solid empirical breadth and real ablations. 5 backbones (7B/8B, different families) × 8 benchmarks, and w/o R_suff, w/o R_faith, w/o Evidence all drop meaningfully (~69–70 vs. 75.6 on Qwen3-VL-4B). The counterfactual advantage analysis — replaying trajectories with rewards added/removed and counting sign flips — is a nice mechanistic probe instead of just another leaderboard table.

  3. Unusually honest and well-documented. They openly admit the reward curves are self-reported by the same judge, that the token drop isn't solely due to their rewards, and that the caption comes after reasoning so causality is unproven. Full prompts, parsing rules, reward table, hyperparameters, filtering stats in the appendix. Refreshing.

Critical weaknesses
  1. The causality claim is unproven and the key baseline is missing. They admit the caption is generated after <think>, so it can't actually guide that reasoning at inference. Worse, there's no "same data, same DAPO, no caption field at all (answer + format only)" baseline, and no caption-first variant. So the +5.0 pts vs. base model is tangled up with RL-on-filtered-data, difficulty filtering, structured formatting, and DAPO resampling all at once. You can't attribute it to evidence supervision.

  2. Judge reliability is unchecked and there's zero independent evaluation of evidence quality. Everything rides on one MLLM judge; the diagnostic set's labels are GPT-generated and unverified, and aren't drawn from the trained policy. No hallucination benchmark, no human eval, and every case study in the appendix is hand-written rather than a real rollout. The 48% token drop is suspiciously consistent with a degenerate hack — shorter, vaguer captions mean fewer falsifiable claims means higher faithfulness scores. They even say they didn't audit for judge exploitation. Relatedly, R_suff basically degenerates on ChartQA (answer is directly readable) and is capped by the judge's own math ability on hard problems.

Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Performance Attribution and Caption Generation Order (W1)

We thank the reviewer for the comments. We conducted additional matched training comparisons, policy-generated evidence audits, cross-judge evaluations, and output-length analyses to address the concerns.

1. Performance Attribution and Caption Generation Order (W1)

VERA generates captions after <think>. During training, faithfulness and sufficiency rewards contribute to the group-relative advantage applied to the entire trajectory, including reasoning tokens. Thus, evidence rewards can influence reasoning through subsequent policy updates.

We conducted matched experiments on Qwen3-VL-4B-Thinking using identical training data, difficulty filtering, DAPO, resampling rules, training budgets, and answer verification. No-caption RL generates think → answer with answer and format rewards; w/o Evidence retains think → caption → answer without evidence rewards; Caption-first moves the caption before reasoning.

Table A. Controlled comparisons on Qwen3-VL-4B-Thinking. Each entry reports Avg@3 accuracy (%) / average output tokens.

DatasetOriginalCaption-firstNo-caption RLw/o EvidenceVERA
MMStar71.0 / 149968.4 / 158674.2 / 122970.8 / 169875.6 / 1005
ChartQA85.1 / 149183.8 / 163184.8 / 99283.1 / 88688.0 / 561
WeMath82.9 / 421081.6 / 443883.6 / 321179.4 / 287985.9 / 1806
MMMath84.0 / 419682.5 / 467484.2 / 388678.8 / 531285.0 / 3529
LogicVista63.8 / 1362561.8 / 998064.5 / 688759.6 / 715869.1 / 4531
MathVerse74.5 / 563571.9 / 655875.8 / 396772.7 / 338980.6 / 2079
MathVision31.9 / 1266431.4 / 1044233.9 / 1022136.8 / 815039.8 / 9630
MathVista80.6 / 375179.7 / 423175.8 / 247178.2 / 228781.1 / 1328
Avg71.73 / 588470.14 / 544372.10 / 410869.93 / 397075.64 / 3059

VERA achieves 75.64%, outperforming No-caption RL (72.10%) by 3.54 pp and w/o Evidence (69.93%) by 5.71 pp. The latter comparison retains the same caption structure, directly demonstrating the additional benefit of evidence rewards. Caption-first achieves 70.14%, indicating that moving the caption before reasoning does not automatically improve performance.

These controlled comparisons distinguish the effects of standard RL, caption formatting, and evidence supervision.

AuthorsOpenReview ↗

Judge Reliability and Evidence Quality (W2)

2. Judge Reliability and Evidence Quality (W2)

To evaluate actual evidence quality beyond the original synthetic diagnostic set, we sampled 400 questions (50 per benchmark) and assessed captions generated by the trained policies.

Faithfulness (F) measures image grounding, while sufficiency (S) measures answer recovery from the question and caption. F and S use non-empty captions as denominators; joint F∧S uses all 400 questions, counting missing captions as failures.

Evidence Quality of Policy-generated Captions

Table B. Evidence quality of policy-generated captions. All values are percentages.

Datasetw/o Evidence F / SVERA F / Sw/o Evidence F∧SVERA F∧S
MMStar82.98 / 74.4789.80 / 83.6760.0074.00
ChartQA69.57 / 67.3984.00 / 78.0044.0068.00
WeMath68.75 / 70.8387.50 / 72.9248.0060.00
MMMath64.58 / 75.0083.67 / 81.6344.0068.00
LogicVista82.98 / 65.9696.00 / 82.0060.0080.00
MathVerse60.42 / 77.0879.59 / 79.5940.0058.00
MathVision68.75 / 68.7581.25 / 81.2546.0060.00
MathVista76.00 / 74.0088.00 / 82.0054.0074.00
Overall71.75 / 71.6986.23 / 80.1349.5067.75

VERA improves joint F∧S on all eight benchmarks, increasing the overall rate from 49.50% to 67.75% (+18.25 pp). Faithfulness increases from 71.75% to 86.23%, and sufficiency from 71.69% to 80.13%.

Cross-judge Evaluation

To assess judge dependence, we repeated the evaluation using Kimi K3 and MiniMax M3 alongside the training judge, Qwen3.5-397B-A17B.

Table C. Cross-judge evaluation of joint faithfulness and sufficiency. All values are percentages.

ModelQwen3.5-397B-A17BKimi K3MiniMax M3
Original39.7541.0039.00
w/o Evidence49.5048.7548.25
VERA67.7558.5063.50
VERA − w/o Evidence+18.25 pp+9.75 pp+15.25 pp

All three judges favor VERA over w/o Evidence, with improvements of 18.25, 9.75, and 15.25 pp, respectively.

We also restricted the evaluation to questions where both methods generated valid captions, eliminating differences caused by caption completion.

Table D. Evidence-quality improvements on paired samples with valid captions.

JudgeΔ FaithfulnessΔ SufficiencyΔ Joint F∧S
Qwen3.5-397B-A17B+14.66 pp+7.85 pp+16.75 pp
Kimi K3+7.07 pp+2.62 pp+7.85 pp
MiniMax M3+9.69 pp+10.99 pp+13.87 pp

On the paired valid-caption samples, all three judges continue to report improvements in F, S, and joint F∧S. This confirms that the gains are not explained solely by differences in caption completion rates.

Output-token Reduction and Evidence Quality

To examine whether token reduction reflects shorter or vaguer captions, we decomposed outputs into reasoning, caption, answer, and formatting tokens on the same 400 questions.

Table E. Output-token decomposition on 400 evaluation samples.

ModelThink TokensCaption TokensAnswer TokensFormat TokensTotal TokensMissing Caption (%)Truncation (%)
Original5918.93189.3520.3121.616150.2111.252.00
w/o Evidence4525.78181.5721.2923.054751.704.500.75
VERA3253.61141.2921.4623.623439.991.750.50
Format-only3046.1999.2221.5223.483190.402.750.25

VERA reduces total output tokens by 44.07% relative to the original model and 27.61% relative to w/o Evidence. Approximately 97.0% of the reduction relative to w/o Evidence comes from the think segment, while caption tokens decrease from 181.57 to 141.29. Caption missing and truncation rates also decrease.

AuthorsOpenReview ↗

Judge Reliability and Evidence Quality (W2)

Table F. Caption specificity, factual errors, and evidence coverage.

ModelSpecific FactsIncorrect FactsRequired Fact Coverage (%)Vague Captions (%)Answer Leakage (%)Sufficiency (%)
Original4.620.5270.8320.284.5167.04
w/o Evidence4.690.4078.3918.593.6671.73
VERA4.640.1984.3214.762.8080.15
Faithfulness-only4.330.1170.6229.561.2960.67

VERA maintains a similar number of specific facts (4.64 vs. 4.69) while reducing incorrect facts by 52.5%, improving required-fact coverage by 5.93 pp, and lowering vague captions from 18.59% to 14.76%.

Faithfulness-only produces fewer incorrect facts but substantially lower coverage and sufficiency, indicating that optimizing faithfulness alone may encourage omission of uncertain yet necessary information.

These results show that token reduction mainly occurs in reasoning, while VERA preserves task-relevant visual information with fewer factual errors.

Verification of the Original Synthetic Diagnostic Set

We additionally verified the intended evidence conditions of the original 3,200 synthetic descriptions, constructed from 800 ChartQAPro questions.

Table G. Label verification of the controlled diagnostic set. All values are percentages.

Intended ConditionNF PositiveS PositiveJoint Label AgreementJudge Disagreement
Faithful & Sufficient80092.0090.1282.758.50
Faithful & Insufficient80091.6212.0080.509.00
Unfaithful & Sufficient80010.2592.1282.506.62
Unfaithful & Insufficient8008.0014.0078.759.25

Joint agreement with intended labels ranges from 78.75% to 82.75%, averaging 81.13%. The new policy-output evaluation provides further evidence beyond this synthetic diagnostic set.

Task-sufficiency Rewards and Judge Reasoning Ability

To assess the judge's reasoning limitations, we evaluated direct-reading, simple-calculation, and multi-step tasks under different evidence conditions. Here, C∗C^* denotes a verified complete description containing all necessary observable facts, without calculations or final answers.

Table H. Judge capability analysis under different evidence conditions. All values are accuracy (%).

Dataset / Task TypeQ-onlyQ + w/o Evidence CaptionQ + VERA CaptionQ + C*Q + Image
ChartQA / Direct Reading12.5077.5090.0092.5092.50
ChartQAPro / Direct Reading7.5072.5087.5097.5092.50
ChartQA / Simple Calculation17.5082.5085.0090.0090.00
MathVista / Simple Calculation17.5082.5080.0092.5095.00
WeMath / Multi-step Reasoning10.0065.0070.0077.5077.50
MMMath / Multi-step Reasoning7.5060.0057.5070.0080.00
MathVision / Multi-step Reasoning15.0057.5067.5077.5082.50
LogicVista / Multi-step Reasoning15.0060.0070.0072.5075.00

On direct-reading tasks, providing VERA captions improves answer recovery substantially over Q-only inputs (12.50%→90.00% on ChartQA and 7.50%→87.50% on ChartQAPro).

On harder tasks, the judge still makes errors even with complete evidence (70.00% on MMMath and 77.50% on MathVision). Thus, answer recovery reflects both evidence quality and judge reasoning ability. VERA combines sufficiency with independent faithfulness and answer-correctness rewards.

Example of Incorrect Evidence with a Correct Final Answer

We further examined actual outputs for an L-shaped region problem, where the blue area below a line is three times that above it.

ModelVisual Facts in CaptionFinal Answer
VERAOverall width 10 cm and height 8 cm; upper-right cutout is 4 cm wide and 3 cm tall3217\frac{32}{17}
w/o EvidenceOverall width 10 cm and height 8 cm; upper-right cutout is 3 cm wide and 4 cm tall3217\frac{32}{17}

Both models produce the correct answer, but w/o Evidence swaps the cutout dimensions. The cutout area remains 12 cm212\text{ cm}^2, and the relevant line intersection lies to the left of the cutout, so the final answer is unchanged. This illustrates how answer correctness can conceal inaccurate visual evidence, which VERA explicitly evaluates.

Reviewer_nKFbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The proposed VERA decouples two mutually exclusive error types in multimodal reinforcement learning: recognition errors and reasoning errors. The authors note that outcome rewards alone are insufficient to constrain the quality of intermediate visual evidence, and employ a Qwen3.5-397B large-model judge to provide process rewards along two dimensions: Visual Faithfulness and Task Sufficiency.

Critical strengths
  1. The paper identifies a reward-hacking phenomenon in multimodal reasoning RL training—correct answers supported by incorrect visual grounding—and introduces process supervision over visual evidence. The motivation is intuitive and well-founded.

  2. The downstream evaluation is broad: consistent performance gains are observed across multiple mainstream multimodal reasoning datasets.

Critical weaknesses
  1. The core assumption of "decoupling" visual evidence from reasoning lacks ablation studies. The authors present the decomposition into Visual Faithfulness and Task Sufficiency as the core innovation, yet the experiments entirely omit controls such as no decoupling, direct use of the same reward model, and holistic end-to-end process rewards. Consequently, it remains unclear whether the performance gains stem from the decoupling design itself or merely from the additional dense supervisory signal provided by the large Qwen3.5-397B-A17B model. Visual perception and reasoning are forcibly separated, and causality is inverted: moreover, given the extremely high information density of images, determining which key visual information to extract is itself a problem highly dependent on visual reasoning; the idea of decoupling visual recognition from reasoning is overly taken for granted. In addition, the evidence blocks are generated only after the reasoning is complete, which cannot guarantee that the evidence genuinely guided the preceding reasoning and can easily degenerate into post-hoc rationalization tailored to the judge.

  2. The judge both sets the questions and grades the answers, which is highly problematic. The same Qwen3.5-397B-A17B judge is used throughout, and the ground truth of the diagnostic set is synthesized by GPT-5.6 Sol rather than manually annotated. With no independent human validation and no cross-evaluation via voting among judges of different architectures, reward hacking is entirely plausible. The method's ceiling is also determined by Qwen3.5-397B-A17B (as the authors themselves acknowledge).

Questions for authors
  1. If the exact same Qwen3.5-397B-A17B were used as the judge with identical contextual information, and were asked to directly output a single holistic quality score for the evidence without decoupling, for RL training, how would the resulting performance compare with VERA?

  2. Why are the evidence blocks designed to appear after the reasoning trajectory rather than before or interleaved with it? Ideal reasoning should be evidence-first, which is more causally sound.

  3. When multiple heterogeneous closed-source frontier large models are used as independent third-party judges, is the evidence quality generated by VERA still significantly superior to the baseline?

10 comments
AuthorsOpenReview ↗

Effectiveness of Decoupled Evidence Supervision (W1 / Q1)

We thank the reviewer for the constructive comments. We conducted additional experiments on holistic reward supervision, generation order, and evidence-quality evaluation to address the concerns.

1. Effectiveness of Decoupled Evidence Supervision (W1 / Q1)

VERA evaluates visual evidence through two complementary rewards: Visual Faithfulness (F) and Task Sufficiency (S). The policy retains full access to the image during reasoning. The total reward is: RVERA=Rans+0.5Rfmt+0.5Rfaith+0.5Rsuff.R_{\mathrm{VERA}}=R_{\mathrm{ans}}+0.5R_{\mathrm{fmt}}+0.5R_{\mathrm{faith}}+0.5R_{\mathrm{suff}}. The group-relative advantage is applied to the entire trajectory, allowing evidence rewards to influence reasoning through policy updates.

Comparison with a Holistic Evidence Reward

We conducted matched experiments on Qwen3-VL-4B-Thinking using the same Qwen3.5-397B-A17B judge, initial checkpoint, training data, filtering, DAPO configuration, rollout/update budget, and evaluation protocol. All configurations retain think → caption → answer.

Table A. Reward configurations and judge inputs.

ConfigurationRewardEvidence Judge Input
w/o EvidenceRans+0.5RfmtR_{\mathrm{ans}}+0.5R_{\mathrm{fmt}}No evidence evaluation
Holistic RewardRans+0.5Rfmt+HR_{\mathrm{ans}}+0.5R_{\mathrm{fmt}}+HImage + question + caption
VERARans+0.5Rfmt+0.5F+0.5SR_{\mathrm{ans}}+0.5R_{\mathrm{fmt}}+0.5F+0.5SF: image + caption; S: question + caption

The Holistic Reward baseline uses one judge call to produce H∈0,0.5,1H\in{0,0.5,1}, matching VERA's maximum evidence-reward contribution of 1.0. It receives the combined information available to VERA's two judges. VERA separately evaluates image grounding and image-blind answer recovery.

Table B. Comparison between holistic and decoupled evidence rewards on Qwen3-VL-4B-Thinking. Each entry reports Avg@3 accuracy (%) / average output tokens.

Datasetw/o EvidenceHolistic RewardVERA
MMStar70.8 / 169874.4 / 143275.6 / 1005
ChartQA83.1 / 88687.8 / 77388.0 / 561
WeMath79.4 / 287985.0 / 203385.9 / 1806
MMMath78.8 / 531284.4 / 392985.0 / 3529
LogicVista59.6 / 715867.7 / 502169.1 / 4531
MathVerse72.7 / 338979.7 / 289180.6 / 2079
MathVision36.8 / 815037.5 / 892239.8 / 9630
MathVista78.2 / 228780.9 / 190181.1 / 1328
Avg69.93 / 397074.68 / 336375.64 / 3059

Holistic supervision improves accuracy from 69.93% to 74.68%, confirming the benefit of additional evidence rewards. VERA further reaches 75.64%, outperforming the holistic baseline by 0.96 pp across the eight-benchmark average, with higher accuracy on every benchmark and 9.04% fewer output tokens.

These results support the additional benefit of VERA's input-separated faithfulness and sufficiency evaluation under the same reward model.

AuthorsOpenReview ↗

Evidence Generation Order (W1 / Q2)

2. Evidence Generation Order (W1 / Q2)

VERA generates think → caption → answer. The model can directly use visual information during <think>, while the caption provides a separately evaluated evidence representation. Its reward contributes to policy updates over the entire trajectory.

We evaluated caption-first variants with and without evidence rewards to examine the effect of generation order.

Table C. Generation-order ablations under matched training configurations.

ConfigurationGeneration OrderEvidence RewardAvg@3 (%)Avg Tokens
w/o Evidencethink → caption → answerNone69.933970
Caption-first (w/o Evidence)caption → think → answerNone70.145443
VERAthink → caption → answer0.5F+0.5S0.5F+0.5S75.643059
Caption-first (+ F/S)caption → think → answer0.5F+0.5S0.5F+0.5S72.324533

The caption-first configuration with complete F/S rewards uses:

R_{\mathrm{ans}}+0.5R_{\mathrm{fmt}}^{\mathrm{CTA}}+0.5R_{\mathrm{faith}}+0.5R_{\mathrm{suff}}.$$ Here, $R_{\mathrm{fmt}}^{\mathrm{CTA}}$ validates the `caption → think → answer` structure. The reward weights, judges, training data, optimization settings, and budgets remain matched with VERA. With full evidence supervision, Caption-first achieves **72.32%**, compared with **75.64%** for VERA, a difference of **3.32 pp**. It also generates more tokens (4,533 vs. 3,059). Without evidence rewards, Caption-first achieves 70.14%, close to the standard w/o Evidence baseline (69.93%). These results favor the original generation order under the evaluated settings. Although a post-reasoning caption cannot directly guide the preceding tokens within the same rollout, its reward influences their subsequent policy updates. We also examined actual outputs where correct answers contain different visual evidence. **Table D. An example of correct answers with different evidence quality.** | Configuration | Generated Visual Evidence | Final Answer | | ------------- | ------------------------------------------------------------ | --------------- | | VERA | The blue region is 10 cm wide and 8 cm tall, with an upper-right cutout measuring 4 cm in width and 3 cm in height. | $\frac{32}{17}$ | | w/o Evidence | The blue region is 10 cm wide and 8 cm tall, with an upper-right cutout measuring 3 cm in width and 4 cm in height. | $\frac{32}{17}$ | The w/o Evidence model swaps the cutout dimensions, yet both responses obtain the same correct answer because the cutout area remains $12\text{ cm}^2$ and the relevant line intersection lies to the left of the cutout. This illustrates how answer correctness can conceal factual errors in visual evidence, which VERA explicitly evaluates.
AuthorsOpenReview ↗

Judge Reliability and Independent Evidence-Quality Evaluation (W2 / Q3)

3. Judge Reliability and Independent Evidence-Quality Evaluation (W2 / Q3)

The original diagnostic questions and reference answers come from ChartQAPro. GPT-5.6 Sol generates controlled descriptions, while Qwen3.5-397B-A17B evaluates them; the reward judge does not generate the questions or reference answers.

We additionally evaluated actual policy-generated captions on 400 questions (50 from each of eight benchmarks), including cross-evaluation by Kimi K3 and MiniMax M3.

Evidence Quality of Policy-generated Captions

Faithfulness (F) is evaluated from the image and caption, while sufficiency (S) is measured through answer recovery using the question and caption. F and S use valid captions as denominators; joint F∧S uses all 400 questions, counting missing captions as failures.

Table E. Evidence quality of policy-generated captions.

ModelValid CaptionsMissing Rate ↓Faithfulness F ↑Sufficiency S ↑Joint F∧S ↑
Original355/40011.25%65.35%67.04%39.75%
w/o Evidence382/4004.50%71.73%71.73%49.50%
Holistic Reward394/4001.50%78.68%78.68%61.00%
VERA393/4001.75%86.26%80.15%67.75%

VERA increases joint F∧S from 49.50% to 67.75% (+18.25 pp) over w/o Evidence and from 61.00% to 67.75% (+6.75 pp) over Holistic Reward under the training judge.

Cross-judge Evaluation

Table F. Cross-judge evaluation of joint faithfulness and sufficiency.

JudgeOriginalw/o EvidenceHolistic RewardVERA
Qwen3.5-397B-A17B39.75%49.50%61.00%67.75%
Kimi K341.00%48.75%65.25%58.50%
MiniMax M339.00%48.25%63.75%63.50%

All three judges favor VERA over w/o Evidence, with improvements of 18.25, 9.75, and 15.25 pp, respectively.

Against Holistic Reward, Qwen favors VERA, Kimi favors the holistic baseline, and MiniMax reports nearly identical scores. Thus, the evidence-quality improvements over w/o Evidence generalize across judges, while the comparison with holistic scoring shows evaluator-dependent differences.

Paired Evaluation Controlling for Caption Completion

To exclude differences caused by missing captions, we compared the same questions where both configurations generated valid captions: 382 samples for VERA vs. w/o Evidence and 393 for VERA vs. Holistic Reward.

Table G. Paired evidence-quality comparisons on common valid-caption samples.

JudgeComparisonPaired SamplesΔ FaithfulnessΔ SufficiencyΔ Joint F∧S
Qwen3.5-397B-A17BVERA − w/o Evidence382+14.66 pp+7.85 pp+16.75 pp
Kimi K3VERA − w/o Evidence382+7.07 pp+2.62 pp+7.85 pp
MiniMax M3VERA − w/o Evidence382+9.69 pp+10.99 pp+13.87 pp
Qwen3.5-397B-A17BVERA − Holistic Reward393+7.63 pp+1.53 pp+7.12 pp
Kimi K3VERA − Holistic Reward393−2.80 pp−6.62 pp−6.62 pp
MiniMax M3VERA − Holistic Reward393+0.76 pp−0.51 pp0.00 pp

The paired results confirm that VERA improves F, S, and joint F∧S over w/o Evidence under all three judges, even after controlling for caption completion. The comparison with Holistic Reward retains the same cross-judge variation.

Reviewer_peQQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VERA, a reinforcement learning framework that provides explicit supervision for task-relevant visual evidence in multimodal reasoning models. The policy generates a reasoning trace, a visual-evidence description, and a final answer. Two judge roles assess the description: a visual-faithfulness judge checks its claims against the image, while an image-blind task-sufficiency judge attempts to answer the question using the description. These rewards are combined with answer correctness and format compliance for DAPO optimization.

Experiments cover five backbones and eight benchmarks. The reported results show an average accuracy improvement of approximately 5.0 percentage points across the five backbone-level averages, together with reduced output-token consumption. Component ablations support the usefulness of the combined evidence rewards, and synthetic diagnostics examine the behavior of the two judges.

Critical strengths
  1. Well-motivated separation of evidence criteria. The paper distinguishes whether a visual description is supported by the image from whether it contains enough information to answer the question. Assigning these criteria to judges with different inputs provides an interpretable reward design that addresses both hallucinated observations and missing task-relevant information. This is a useful approach to enriching supervision beyond final-answer correctness.

  2. Broad empirical improvements with a relevant ablation control. VERA improves benchmark-averaged accuracy on all five evaluated backbones, with gains ranging from approximately 1.7 to 11.4 percentage points. The ablation that removes both evidence rewards while retaining the response structure is valuable.

Critical weaknesses
  1. Incomplete alignment between the motivating challenge and the supervision mechanism. The paper motivates VERA by distinguishing failures in visual evidence acquisition from errors in subsequent reasoning. However, the policy generates the evidence description after the reasoning trace, and the aggregated reward assigns a shared advantage to the entire response. The mechanism therefore does not directly assess whether the preceding reasoning actually used the stated evidence or identify where a perceptual error entered the reasoning process..
  2. Incomplete experimental validation of the motivating challenges. The introduction emphasizes visual-recognition errors and correct answers reached through unreliable evidence. However, the experiments lack a before-and-after breakdown of perception versus reasoning errors or targeted tests of visual-shortcut reliance. The results therefore only partially establish whether VERA addresses these specific failure modes beyond improving answer accuracy.
  3. Limited independent validation of evidence quality. Table 3 relies on unverified synthetic labels rather than independently annotated outputs from the trained policy. It therefore provides limited evidence that reward optimization improves actual evidence quality or avoids judge-specific shortcuts.
Questions for authors

Please refer to the Critical Weaknesses section for detailed questions.

3 comments
AuthorsOpenReview ↗

Relationship Between Visual Evidence Supervision and Reasoning (W1)

We thank the reviewer for the constructive comments. We conducted additional matched training comparisons, trajectory-level error analyses, visual counterfactual tests, and independent evidence-quality evaluations to address the three concerns.

1. Relationship Between Visual Evidence Supervision and Reasoning (W1)

VERA follows think → caption → answer. The caption is explicitly supervised through faithfulness and sufficiency rewards:

R=Rans+0.5Rfmt+0.5Rsuff+0.5Rfaith.R = R_{\mathrm{ans}} + 0.5R_{\mathrm{fmt}} + 0.5R_{\mathrm{suff}} + 0.5R_{\mathrm{faith}}.

The group-relative advantage is applied to all generated tokens, including those in <think>. Thus, caption-level rewards influence reasoning through policy updates, even though the caption is generated after the reasoning trace.

We conducted matched comparisons on Qwen3-VL-4B-Thinking using the same training data, difficulty filtering, DAPO, resampling rules, training budget, and answer verification. No-caption RL uses think → answer, while w/o Evidence retains think → caption → answer without evidence rewards.

Table A. Controlled comparisons on Qwen3-VL-4B-Thinking. Each entry reports Avg@3 accuracy (%) / average output tokens.

DatasetOriginalCaption-firstNo-caption RLw/o EvidenceVERA
MMStar71.0 / 149968.4 / 158674.2 / 122970.8 / 169875.6 / 1005
ChartQA85.1 / 149183.8 / 163184.2 / 99283.1 / 88688.0 / 561
WeMath82.9 / 421081.6 / 443883.6 / 321179.4 / 287985.9 / 1806
MMMath84.0 / 419682.5 / 467484.2 / 388678.8 / 531285.0 / 3529
LogicVista63.8 / 1362561.8 / 998064.5 / 688759.6 / 715869.1 / 4531
MathVerse74.5 / 563571.9 / 655875.8 / 396772.7 / 338980.6 / 2079
MathVision31.9 / 1266431.4 / 1044233.9 / 1022136.8 / 815039.8 / 9630
MathVista80.6 / 375179.7 / 423175.8 / 247178.2 / 228781.1 / 1328
Avg71.73 / 588470.14 / 544372.10 / 410869.93 / 397075.64 / 3059

VERA achieves 75.64%, exceeding No-caption RL by 3.54 pp and w/o Evidence by 5.71 pp. The latter comparison retains the same output structure and directly isolates the benefit of evidence rewards. The Caption-first variant achieves 70.14%, indicating that moving the caption before reasoning does not automatically improve performance.

AuthorsOpenReview ↗

Visual Errors, Reasoning Errors, and Visual Shortcuts (W2)

2. Visual Errors, Reasoning Errors, and Visual Shortcuts (W2)

We audited 400 questions (50 per benchmark), with one response per model and question. Incorrect responses were classified by the earliest consequential error in the complete trajectory. We separately recorded correct answers accompanied by incorrect captions.

Table B. Complete-trajectory error decomposition. Each model is evaluated on 400 questions; values are percentages unless counts are shown.

ModelCorrect AnswerVisual-first ErrorReasoning-first ErrorOther FailuresCorrect Answer with Incorrect Caption
Original63.5014.7514.007.7520.25 (81)
w/o Evidence69.7510.5016.753.0018.50 (74)
VERA72.5012.0014.251.2510.25 (41)

The first four categories are mutually exclusive. The final column is separately calculated over all 400 questions.

Compared with w/o Evidence, VERA improves accuracy by 2.75 pp, reduces reasoning-first errors by 2.50 pp, and reduces other failures by 1.75 pp. Visual-first errors increase from 10.50% to 12.00%. Importantly, correct answers with incorrect captions decrease from 74 to 41 cases (−44.6%), directly addressing the concern about correct answers based on unreliable evidence.

Table C. Dataset-level error changes. Each benchmark contains 50 questions. Arrows indicate w/o Evidence → VERA (%).

DatasetVisual-first ErrorReasoning-first ErrorCorrect Answer with Incorrect Caption
MMStar18 → 1214 → 2210 → 8
ChartQA8 → 60 → 424 → 18
WeMath8 → 102 → 020 → 10
MMMath14 → 2010 → 428 → 16
LogicVista10 → 1024 → 2210 → 2
MathVerse6 → 1016 → 1028 → 14
MathVision10 → 1458 → 468 → 6
MathVista10 → 1410 → 620 → 8
Overall10.50 → 12.0016.75 → 14.2518.50 → 10.25

Across all eight benchmarks, VERA consistently reduces correct answers accompanied by incorrect captions, although the changes in visual-first and reasoning-first errors vary by task.

We also examined paired answer transitions on the same 400 questions.

Table D. Paired answer transitions.

ComparisonWrong → CorrectCorrect → WrongBoth CorrectBoth WrongNet Accuracy Change
Original → VERA371253109+9.00 pp
w/o Evidence → VERA143276107+2.75 pp

VERA corrects 14 previously incorrect responses, while only three previously correct responses become incorrect relative to w/o Evidence. These 400-question results use single generations, whereas the main benchmark evaluation reports Avg@3.

To examine visual-shortcut reliance, we constructed 30 counterfactual groups with original images, task-critical visual changes, irrelevant appearance changes, and Q-only inputs. Correct answer change requires both the original and modified answers to be correct and to differ as expected; correct stability requires correct, unchanged answers under irrelevant modifications.

Table E. Visual counterfactual and Q-only evaluation. Each entry reports successful groups for w/o Evidence → VERA.

DatasetGroupsOriginal Image CorrectCorrect Answer ChangeCorrect StabilityQ-only Correct
ChartQA1210→106→910→103→2
MathVista108→96→78→92→2
LogicVista84→44→44→41→1
Total3022→2316→2022→236→5

VERA improves correct responses to critical visual changes from 16/30 to 20/30, while maintaining stability under irrelevant changes (22/30 vs. 23/30). Q-only accuracy remains low (6/30 vs. 5/30), providing further evidence of reliance on task-relevant visual information.

AuthorsOpenReview ↗

Independent Evaluation of Evidence Quality (W3)

3. Independent Evaluation of Evidence Quality (W3)

We evaluated captions generated by the trained policies on the same 400 questions. Faithfulness (F) is judged using the image and caption, while sufficiency (S) is assessed by recovering the answer from the question and caption. F and S use non-empty captions as denominators; joint F∧S uses all 400 questions, counting missing captions as failures.

Table F. Faithfulness and sufficiency of policy-generated captions.

ModelValid CaptionsFaithfulness F (%)Sufficiency S (%)Joint F∧S (%)
Original355 / 40065.3567.0439.75
w/o Evidence382 / 40071.7371.7349.50
VERA393 / 40086.2680.1567.75

F and S are pooled over valid captions; joint F∧S is calculated over all 400 questions. Evaluator: Qwen3.5-397B-A17B.

VERA increases joint F∧S from 49.50% to 67.75% (+18.25 pp), providing direct evidence of improved quality in actual policy-generated captions.

To assess judge dependence, we repeated the evaluation using Kimi K3 and MiniMax M3.

Table G. Cross-judge evaluation of joint faithfulness and sufficiency (%).

ModelQwen3.5-397B-A17BKimi K3MiniMax M3
Original39.7541.0039.00
w/o Evidence49.5048.7548.25
VERA67.7558.5063.50
VERA − w/o Evidence+18.25 pp+9.75 pp+15.25 pp

All three judges report higher joint evidence quality for VERA, with improvements of 18.25, 9.75, and 15.25 pp, respectively.

We further restricted the comparison to the 382 questions for which both methods generated valid captions.

Table H. Paired evidence-quality improvements on 382 common valid-caption samples.

JudgeΔ FaithfulnessΔ SufficiencyΔ Joint F∧S
Qwen3.5-397B-A17B+14.66 pp+7.85 pp+16.75 pp
Kimi K3+7.07 pp+2.62 pp+7.85 pp
MiniMax M3+9.69 pp+10.99 pp+13.87 pp

All three judges continue to report improvements in F, S, and joint F∧S on matched valid-caption samples, indicating that the gains are not explained solely by caption completion rates.

Finally, we examined factual specificity, errors, required-fact coverage, vagueness, and answer leakage.

Table I. Caption factual quality and evidence coverage.

ModelSpecific FactsIncorrect FactsRequired Fact Coverage (%)Vague Captions (%)Answer Leakage (%)Sufficiency (%)
Original4.620.5270.8320.284.5167.04
w/o Evidence4.690.4078.3918.593.6671.73
VERA4.640.1984.3214.762.8080.15
Faithfulness-only4.330.1170.6229.561.2960.67

VERA maintains a similar number of specific facts (4.64 vs. 4.69), while reducing incorrect facts by 52.5%, improving required-fact coverage by 5.93 pp, and lowering the vague-caption rate from 18.59% to 14.76%. Faithfulness-only achieves fewer factual errors but substantially lower coverage and sufficiency, supporting the complementary role of task-sufficiency supervision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.