acceptodds
Under review as a conference paper at ICLR 2027

Bifocal Self-Rewarding: Aligning Answer Sufficiency with Visual Fidelity in VLM Reasoning

Abstract

Outcome-reward reinforcement learning (RL) can improve multimodal reasoning without establishing whether the reasoning used to produce an answer is both sufficient for that answer and faithful to the image. Existing methods address complementary aspects of this problem: some verify the connection between intermediate reasoning and the final answer, while others strengthen the extraction or use of visual evidence. Yet neither direction alone establishes whether the same reasoning chain satisfies both properties. We introduce Bifocal Self-Rewarding, a dual-view framework that evaluates the same reasoning chain along these two dimensions. In the image-stripped view, the current policy uses its own reasoning chain to recover the answer without access to the image, testing whether the chain itself contains sufficient information. In the image-conditioned view, the frozen RL initialization provides a fixed perceptual reference, favoring correct chains that contain concrete visual evidence supported by the image. Using the current policy for sufficiency allows the evaluator to follow changes in the policy's reasoning, whereas freezing the fidelity evaluator provides a fixed reference throughout training. Both rewards are therefore assigned directly to the reasoning chain used to produce the answer, without requiring annotated visual claims or a separately trained judge. Across diverse benchmarks spanning reasoning, perception, extraction, and hallucination, Bifocal consistently improves multimodal reasoning across Qwen2.5-VL and Qwen3-VL backbones from 3B to 8B parameters while preserving visual capabilities.

Primary area:
foundation or frontier models, including LLMs
Keywords:
VLM, Multimodal Reasoning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ku5UOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Bifocal Self-Rewarding, a reinforcement-learning framework for multimodal reasoning that evaluates the same answer-producing chain from two complementary views. The current policy removes the image and attempts to recover the answer from the question, options, and the same reasoning chain, providing a sufficiency signal. A frozen copy of the pre-RL model retains the image and scores how much concrete, image-checkable visual evidence is exposed by the chain, providing a fidelity preference. These signals are combined with answer correctness and auxiliary format and length rewards in a GRPO update. The method is trained on Bifocal-24K, a 19-source mixture covering science, visual understanding, and multimodal mathematics. It is evaluated on 15 multimodal benchmarks with Qwen2.5-VL and Qwen3-VL backbones from 3B to 8B parameters. The paper also includes component ablations, paired perturbation tests, visual-evidence diagnostics, and a VMCBench chain audit. The reported average score improves over the corresponding base model in all four reported backbone settings, although the gains vary across benchmarks.

Critical strengths

1. Clear problem formulation

The paper identifies a meaningful failure mode in multimodal RL: outcome correctness alone can reward reasoning chains that are incomplete or visually unsupported. It makes the distinction between answer sufficiency and visual fidelity explicit: a chain can contain enough information to recover an answer while relying on unsupported visual claims, and it can contain accurate observations without carrying enough information to solve the task. This gives the two-view design a clear motivation and makes the later diagnostics easier to interpret.

2. Same-chain evaluation

Both reward views operate on the same answer-producing chain. This is a useful design choice because it avoids evaluating a separate description and then assuming that the downstream reasoning uses that description faithfully. The image-stripped recovery pass and the image-conditioned visual pass reuse the same reasoning trace, so the two rewards examine the object that actually produces the final answer. The chain-removal and mismatched-chain controls in Table 3 further test whether the trace contributes answer-relevant information beyond the shared question and options.

3. Well-motivated checkpoint roles

Using the current policy for image-free recovery and a frozen pre-RL checkpoint for visual preference gives the two signals different and understandable roles. The evolving verifier can follow changes in the policy's reasoning language, while the frozen observer provides a stable visual reference during training. The two rewards are detached before the GRPO update, so only the policy is optimized. The ablation results in Table 2 support the claim that the components contribute differently: adding the frozen visual branch to the sufficiency-only model raises E-Avg by 2.7 points and R-Avg by 1.1 points in the reported 3B comparison.

4. Broad empirical evaluation

The method is evaluated across 15 benchmarks, four model configurations from 3B to 8B, and several groups of related baselines covering outcome-driven RL, self-verification, description-based methods, and perception-enhanced RL. The benchmark suite spans science and mathematics, general visual understanding, chart and text extraction, and hallucination-oriented tasks. The paper also combines aggregate results with component ablations, paired perturbation tests, training diagnostics, and an output-chain audit, providing evidence about both performance and the proposed mechanism.

Critical weaknesses

1. The link between the visual preference and the Fidelity objective needs further explanation

The paper defines Fidelity as whether visual claims in a reasoning chain are supported by the image. The frozen model assigns a 1–5 visual preference score, which Eq. 9 converts into a group-relative reward for correct responses scoring above the group's upper median. Section 4.3 and Appendix A.6 explicitly distinguish this proxy from claim-level truth. However, the rating guide in Appendix B.3 emphasizes the amount and specificity of image-checkable information without explicitly instructing the judge to check each visual claim or lower the score for contradictory claims. Table 3 shows that factual edits lower the score in 59 of 60 controlled synthetic chains, whereas local edits in 38 real chains produce 3 decreases, 33 ties, and 2 increases. This limited sensitivity means that a local factual change can leave the preference score unchanged. The authors should explain how rewarding this preference encourages image-supported visual claims and how far this mechanism can realize the formal Fidelity objective.

2. Direct evidence for improved formal Fidelity is limited

The paper evaluates the trained models on 15 benchmarks, but the main results primarily report final-answer, extraction, and hallucination metrics rather than a broad chain-level measurement of formal Fidelity. Direct F/S evaluation appears only in the VMCBench analysis, where an external rubric is applied to saved chains. This leaves a gap between the benchmark gains and the claim that the method improves visual fidelity: the audit provides evidence about jointly valid solutions on its audited items, but it does not establish how formal Fidelity changes across the other benchmarks or the full set of model outputs, and the training preference and external F score are separate quantities. The paper should distinguish results supporting task performance, evidence presentation, and formal Fidelity, and state the Fidelity claim at the scope supported by the available audits and paired diagnostics.

3. The VMCBench task selection and adjudication need clearer reporting

Appendix C.4 states that the audit uses 200 items from four tasks selected after examining answer accuracy, while Section 5.6 reports that the increase in jointly valid solutions comes from VizWiz and MathVista and that VQAv2 and GQA tie. The audit begins with automated F/S judgments and reviews borderline cases before reporting final counts. The task composition affects how the 126-versus-115 comparison should be read, and the adjudication stage can affect which outputs reach both required scores, F=2 and S=2, yet the appendix does not explain the task-selection rationale, who performed the review, how borderline cases were identified, or how many initial scores changed. The authors should briefly report these existing selection and adjudication details so that the selected-subset audit is transparent.

Questions for authors
  1. Section 4.3 and Appendix A.6 identify the frozen score as a proxy preference, and Table 3 shows that most local factual edits in real chains leave that score unchanged. What is the authors' explanation for how the reward constructed from this preference in Eq. 9 encourages image-supported visual claims? In particular, how does the scoring rule distinguish a chain with several concrete but partly incorrect details from one with well-supported details, and what limitations does this imply for the formal Fidelity objective?

  2. Across the 15 main benchmarks, which existing results do the authors view as evidence that the trained policy improves formal visual Fidelity, rather than only final-answer performance or the exposure of visual evidence? If the direct F audit is intended as the main evidence, please clarify how its scope in VMCBench relates to the broader Fidelity claim.

  3. Appendix C.4 states that the four audited tasks were selected after examining answer accuracy and that borderline judgments received item-level review. What was the rationale for choosing these tasks, who performed the review, how were borderline cases identified, and did any initial F/S scores or joint-validity counts change after review?

3 comments
AuthorsOpenReview ↗

How the Visual Preference Supports Fidelity

Thank you for the careful review and precise account of the motivation, same-chain design, and checkpoint roles. We address each question together with its corresponding weakness.

Question 1. Image-supported claims and the visual reward

Submitted Eq. 9 gives additional credit to responses with correct original answers whose traces receive higher image-conditioned ratings within the same question. The observer is the instruction-tuned starting checkpoint, frozen during RL. It receives the image, question, and answer-producing trace and returns a 1-5 rating. Among eight candidates, the upper median is the fifth ascending rating. Correct responses above it receive a bonus increasing with their relative rating.

Submitted Appendix B.3 elicits concrete, image-checkable observations. The observer evaluates the complete trace against the image using its instruction-tuned visual capability. Submitted Table 3(a) tests whether image support affects this preference with textual specificity fixed.

For OCR item ocr_synth_000, the question is "What is the reference code printed on the document?" The image contains CT-7586-X. The supported trace is The image text relevant to the question reads "CT-7586-X". I read those characters directly from the image. The paired trace replaces the code with TX-8654-F, also nine characters, preserving every surrounding character. With the image/question fixed, raw ratings are 5 and 1. Across 60 controlled OCR/table pairs, ratings decrease in 59 and tie in one. Replacing the image while fixing the complete supported text lowers all 60 ratings. These interventions show sensitivity to image agreement beyond detail richness.

The generated-trace procedure edits one numeric option-string occurrence, preserving all other text. AI2D_TEST/59601 asks, "How many stages are shown in the life cycle?" Choices are A=6, B=none of the above, C=4, D=5. The correct and original answers are C, four stages. The trace states "The image is divided into four distinct stages of a ladybug's life cycle" and "The image explicitly shows four stages". Its concluding sentence says "The question directly asks for the number of stages, and the answer is clearly 4." The edit replaces that final 4 with 6, retaining both count observations. The observer receives the original image/question and each trace. Both ratings are 5. This pair is among the 38 generated-trace edits with 3 decreases, 33 ties, and 2 increases. Revised Appendix D identifies the edited span explicitly.

For the partly incorrect versus well-supported traces in your question, the image-conditioned observer supplies the judgment and Eq. 9 allocates credit above the group threshold. If both original answers are correct, equal raw ratings give equal visual bonuses. A local incorrect statement can therefore retain a high rating, as the life-cycle pair shows. The training signal is a whole-trace preference with experimentally demonstrated image sensitivity. Its five-level resolution means that individual factual edits may leave its score unchanged. Formal Fidelity concerns image support for visual claims, with output evidence assessed separately below.

Submitted Table 2 tests the training contribution. Adding the frozen visual branch under the same co-evolving sufficiency verifier raises R-Avg from 63.5 to 64.6 and E-Avg from 74.6 to 77.3. R-Avg averages MMMU/MMStar/AI2D, and E-Avg averages TextVQA/OCRBench. The external audit in Question 2 checks decisive factual support in the resulting solutions. Revised Figure 6(a,b) illustrates the excerpts, edits, and saved ratings. Panel (a) uses an OCR input schematic, and both panels list image-swap summary counts.

AuthorsOpenReview ↗

Fidelity Evidence, Audit Scope, and Adjudication

Question 2. Which results support formal Fidelity?

Within the 15-benchmark suite, submitted Figure 4(a) gives direct qualitative evidence about the trace's visual claims on AI2D. The question asks, "What feeds on crickets?" Perceval and Bifocal both answer B, lizards. Perceval additionally claims that the red-bellied black snake feeds on crickets, an unsupported image relation. Bifocal cites the direct cricket-to-frilled-neck-lizard arrow. Thus identical correct final answers accompany different factual support in their traces.

HallusionBench and POPE provide answer-level grounding evidence. At 3B, Bifocal/Vision-SR1 score 69.4/66.0 and 84.9/83.9, respectively (submitted Table 1). These assess visual consistency and hallucination in answers.

The main quantitative chain-level evidence is the external audit in submitted Section 5.6/Table 4. It compares Vision-SR1 and Bifocal, both 3B, on the same 200 VMCBench questions. External F=2 requires a decisive image-supported claim without a decisive contradiction. Correct solutions meeting F=2 increase from 125 to 134, or 62.5% to 67.0% of all attempts. A separate text-only S check tests whether stated observations and necessary reasoning determine the stated option. Correct solutions with F=S=2 increase from 115 to 126, or 57.5% to 63.0%. VizWiz contributes eight additional jointly valid solutions and MathVista three, with ties on VQAv2 and GQA.

The quantitative Fidelity conclusion concerns the production of correct, evidence-supported solutions on these four complete 50-item DEV subsets under the decisive-claim rubric. The AI2D case gives qualitative chain-level evidence within the main suite, and the hallucination metrics support answer-level grounding. The external F/S audit measures chain quality separately from the 1-5 training preference. Revised Section 5.6 and Appendix C.4 clarify this scope, and revised Figure 6(d) summarizes the checks, correctness condition, and 200-item denominator.

Question 3. Selection and review details

Task selection. We chose four VMCBench subtasks complementing the main suite. VizWiz/VQAv2 cover recognition, MathVista visual mathematics, and GQA relations. This follow-up scope was selected after examining answer accuracy. Each contributes its complete 50-item DEV subset, with identical questions for both methods. Submitted Table 4 reports all four, including the equal joint counts on VQAv2/GQA.

Who reviewed. Initial judgments used qwen3.8-max-0902 with thinking enabled and temperature zero. The image-conditioned F request presents two anonymized traces in randomized order. Separate text-only S requests assess each trace independently. Requests contain the question, choices, trace, and stated answer, excluding the gold label. An AI assistant inspected saved images, traces, and judge explanations item by item. The authors confirmed final decisions under the stated rubric.

Borderline criteria. Score-1 decisions received review when the rationale and cited evidence left a potentially decisive observation or inference unresolved. F review checked image support for observations explicitly stated in the trace. S review checked whether the stated observations and necessary relation already determined the answer, including an omitted elementary arithmetic expansion. The same criteria applied to both models.

Changes. Initial automated joint counts were 115 for Vision-SR1 and 118 for Bifocal. Six Bifocal F judgments changed from 1 to 2, raising its joint count to 124. Two S judgments then changed from 1 to 2, giving 126. Vision-SR1's score distributions and joint count remained 115 after review. Table 4 reports the final adjudicated counts. Revised Appendix C.4 now provides these selection/review details. Thank you again for the careful assessment and specific suggestions.

Thank you for the clarification and response. I have no further questions, and I will maintain my "weak accept" rating.

Reviewer_DB4POpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Bifocal Self-Rewarding for VLM reinforcement learning. The current policy tests answer sufficiency by recovering an answer from the same reasoning chain without the image, while the frozen initialization scores image-conditioned visual evidence. Both signals contribute to a GRPO update. Experiments on four Qwen2.5-VL/Qwen3-VL backbones and 15 benchmarks report improved performance, with component ablations, paired interventions, and output-chain audits. No additional evaluator is required at inference.

Critical strengths
  1. The paper clearly distinguishes answer sufficiency from visual fidelity and evaluates both on the same answer-producing chain. The co-evolving sufficiency verifier and frozen visual observer provide complementary signals with clearly defined roles.
  2. The empirical evaluation covers four backbones and 15 benchmarks, with consistent improvements in overall averages. Importantly, the additional verification passes are only required during training, introducing no extra inference cost.
Critical weaknesses
  1. Presentation quality. The figures and writing are somewhat rough. Some figures are difficult to interpret without reading the text closely, and parts of the exposition are unclear or inconsistent in terminology and notation. A more careful pass on figure design and writing would make the paper considerably easier to follow.
  2. Limits of answer-sufficiency evaluation. The image-free verifier still receives the question and options, allowing prior knowledge and option cues to aid answer recovery. Thus, recovery success does not fully establish that the reasoning chain itself contains sufficient visual evidence.
  3. Visual preference does not guarantee factual fidelity. The rubric emphasizes concrete visual details, but detail richness does not guarantee correctness. Table 3 reports unchanged ratings in 33/38 real-chain fact substitutions, raising questions about whether the visual reward genuinely improves factual grounding.
  4. Insufficient evidence for the robustness of improvements. The 8B model improves over PRCO by only 0.2 points in average performance.
Questions for authors
  1. How do the authors distinguish information contributed by the reasoning chain from prior knowledge and option cues in the sufficiency evaluation?
  2. Can the frozen visual scorer reliably identify decisive factual errors when the overall detail and length of a reasoning chain remain unchanged?
  3. Why does visual-reward activation fall below 1% in late training, and does the visual branch still provide meaningful optimization signals at that stage?
6 comments
AuthorsOpenReview ↗

Figures, Terminology, and Presentation

Thank you for recognizing the same-chain design and complementary model roles. Several concerns relate to explanations and controls already in the submitted manuscript. We clarify these and identify the revisions below.

Weakness 1. Figures, terminology, and exposition

Submitted Sections 3.2 and 4.2-4.3 and Figure 2 already specify the two model roles. We explain their correspondence to the figures below.

Submitted Figure 1(a) organizes methods by answer-sufficiency verification and visual-evidence supervision, including which text receives feedback. Bifocal assesses both properties on the answer-producing reasoning trace. Panel (b) illustrates different failure cases.

Submitted Figure 2 shows eight rollouts per question. Green Stage 2 gives the current policy the question, reordered choices, and sampled trace without the image. First-answer and recovery correctness determine Eq. 8's sufficiency reward. Yellow Stage 3 gives the frozen starting checkpoint the image, question, and same trace for a 1-5 rating. Equation 9 assigns relative visual bonuses among responses with correct original answers. These signals, format and length terms, and within-group standardization drive GRPO with KL regularization. Both checks operate during training. Inference uses the trained policy alone.

Submitted Figure 3(a) shows the 19-source Bifocal-24K mixture. Panel (b) tracks training/validation accuracy and sufficiency. Panel (c) plots bonus counts out of 512 rollouts and CoT length on separate axes. Counts rise then fall while accuracy remains high and length bounded. Question 3 explains this curve.

Submitted Figure 4 compares four saved 3B traces.

  • (a) AI2D asks what feeds on crickets, with lizards correct. Vision-SR1 invents a cricket-to-snake link and answers snake. Perceval answers lizards but also claims the snake feeds on crickets. Bifocal uses the direct cricket-to-lizard arrow and answers lizards.
  • (b) ChartQA asks for the average of the smallest gray and largest light-blue bars, 1 and 42. Vision-SR1 identifies both, then substitutes 7 for 42 and returns 4%. Bifocal preserves the values and returns 21.5.
  • (c) TextVQA asks the material accepted by the red bin, marked PLASTICO. Vision-SR1 answers red, and PRCO switches to the blue PAPEL bin and answers Papel. Bifocal retains the red-bin referent and answers plastic.
  • (d) ChartQA asks for the sum of the two middle bars, Pakistan=1350 and Uzbekistan=762. Vision-SR1 selects Turkey/Uzbekistan and returns 1513. PRCO selects Pakistan/Brazil and returns 4268. Bifocal selects the queried bars and returns 2112.

Submitted Figure 5 probes image use. The particle question compares masses 44/46 u at equal speed, 1400 m/s, so sample B has higher kinetic energy and temperature. The knife question asks where the bottom-row handles point, with downward correct. Local blurring measures changes in gold-answer log-probability. S(R) sums positive sensitivity inside the marked region, and C(R) gives its image-wide share. Bifocal has the highest regional sensitivity in the particle case and concentration on the queried knife row. Panel (b) aggregates layerwise attention changes over 992 fixed inputs relative to pre-RL, separating prompt and teacher-forced answer queries.

Revised Figure 6, on page 10, adds the diagnostic examples. Panels (a,b) show the OCR input schematic/life-cycle image, trace excerpts, edited values, raw ratings, and image-swap summary counts. Panel (c) shows the producer question, option remapping, three recovery inputs, and aggregate outcomes. Panel (d) summarizes external F/S checks and reviewed counts.

The submitted appendix figures retain their order in the revision. Submitted Figure 6 becomes revised Figure 7, illustrating the two views on ChartQA. Submitted Figures 7-11 become revised Figures 8-12, extending local-blur probes to the shrew, bar-sum, options-value, rabbit, and grackle questions. Submitted Figure 12 becomes revised Figure 13, separating attention by benchmark/layer block. Submitted Figures 13-15 become revised Figures 14-16, giving complete particle, chart-average, and cricket traces.

Policy means the trainable model. Verifier means that same model reused for image-free recovery. Observer, visual scorer, fidelity evaluator, and visual/perceptual anchor name the frozen instruction-tuned pre-RL checkpoint in its rating role. Visual reward, visual bonus, and fidelity reward denote submitted Eq. 9. Raw ratings are 1-5, and submitted Table 4's external F/S scores are 0/1/2. Both branches assess trace t, the text producing the answer. The separate description is d.

We have polished transitions and terminology in revised Sections 3.2 and 4.2-4.3 and added the examples in Figure 6 and Section 5.7, referenced from Sections 5.4 and 5.6.

AuthorsOpenReview ↗

Evidence for Sufficiency, Grounding, and Transfer

Weakness 2. Contribution of the reasoning trace

The concern does not discuss the matched chain-removal and mismatch controls already in submitted Section 5.4, Table 3(b), and Appendix D. Submitted Section 3.2 explicitly acknowledges question/option cues, and Eq. 2 defines sufficiency conditional on that context.

The controls use 120 questions, 40 each from AI2D, MMStar, and MMMU, balanced between initially correct and incorrect responses. Each item shares its question, permuted options, target mapping, trained model, and greedy decoder across three image-free conditions.

Trace suppliedAll 120Initially correct 60Initially incorrect 60
Original61556
Empty352510
Mismatched342212

The empty condition measures recovery from the question, choices, and model priors. The mismatch supplies another selected item's trace from the same benchmark. The original trace recovers 26/27 additional answers, or 21.7/22.5 percentage points, over these controls. For initially correct responses, recovery is 55/60 versus 25/60 and 22/60. These paired differences establish the trace's additional answer-relevant information under shared priors and option cues.

Weakness 3. Visual preference and factual grounding

The review discusses the 33/38 local-edit result without considering the component comparison and external output audit when assessing training benefit. Submitted Table 2 tests the visual branch, and Section 5.6/Table 4 check the resulting solutions. Submitted Section 4.3 and Appendix A.6 distinguish formal Fidelity, the 1-5 training preference, and the external 0/1/2 F/S scores.

Submitted Table 3(a) tests image support with textual specificity controlled. Supported values score higher in 59/60 OCR/table pairs, with one tie and no reversals. Replacing the image while fixing every text character lowers all 60 ratings. The 38 generated-trace edits test a different intervention, one local option-string substitution within a complete trace, detailed in Question 2.

Equation 9 compares eight candidates for the same question. A correct original answer receives a visual bonus when its rating exceeds the upper median. Submitted Table 2 tests the contribution under the same sufficiency configuration.

3B configurationR-AvgE-Avg
Pure GRPO60.475.3
Sufficiency, no visual branch63.574.6
Bifocal, frozen observer64.677.3
Co-evolving observer62.475.2

R-Avg averages MMMU/MMStar/AI2D, and E-Avg averages TextVQA/OCRBench. Adding the frozen branch raises them by 1.1/2.7 points. The observer comparison also tests the checkpoint choice.

For the output audit, external Qwen3.8-Max assesses F with the image and S through separate text-only requests, excluding the gold label. Benchmark correctness is checked separately. F=0 denotes a decisive visual contradiction, F=1 absent or unverifiable relevant evidence, and F=2 a supported decisive claim without a decisive contradiction. S=0 denotes no useful premise, a bare answer, or contradictory reasoning. S=1 denotes a missing decisive premise/inference or unresolved material error. S=2 requires observations and necessary reasoning that determine the stated option without the image.

Both methods use the same 200 attempts, covering four complete 50-item task subsets. After item-level review, correct solutions with F=2 increase from 125 to 134. Correct solutions with F=S=2 increase from 115 to 126, or 57.5% to 63.0% of attempts. The audit supports producing more correct solutions whose decisive visual evidence and reasoning jointly satisfy these criteria. The preference diagnostics, component comparison, and output audit address separate parts of this mechanism.

Weakness 4. Evidence across scales and tasks

The 0.2-point 8B margin is one part of submitted Table 1's comparison across four backbones, two model families, and 15 benchmarks. PRCO is the strongest compared prior method by Avg at 3B/7B/8B. Bifocal leads Avg at each scale.

ScalePRCO AvgBifocal AvgBelow-base scores, PRCO/Bifocal
3B64.164.83/0
7B69.770.63/2
8B73.273.41/0

At 8B, Bifocal/PRCO score 54.7/51.9 on MathVerse, 35.2/33.6 on MathVision, 77.5/75.0 on RealWorldQA, and 88.7/86.8 on AI2D. Across these 45 scale-benchmark comparisons, Bifocal has two below-base scores versus PRCO's seven. On 3B ChartQA, base/PRCO/Bifocal score 83.8/80.9/84.9. These comparisons, together with submitted Table 2, support the design through transfer and capability retention across the evaluated settings.

AuthorsOpenReview ↗

Answer Recovery and Factual Sensitivity

Question 1. Distinguishing trace information from prior and option cues

Submitted Appendices A.2/B.2 already specify image removal, option permutation, and target-label remapping. Submitted Section 5.4/Table 3(b) provide matched controls for the trace's contribution. These operations have complementary purposes.

  1. The policy answers from the image, question, and choices, producing its reasoning trace and first answer.
  2. For recovery, we remove the image, retain the question/trace, randomly reorder the options, and remap the target label. The prompt contains the reordered choices and trace. The target is held outside the prompt for scoring.
  3. The current policy re-answers from these text inputs. Equation 8 gives full sufficiency credit when both the first answer and recovery are correct.

The question defines what must be determined, and the choices define the answer space. Recovery tests whether the trace expresses information supporting correct answering after image removal. Random reordering reduces dependence on the original answer letter. The matched original/empty/mismatched conditions isolate the trace's added contribution under shared priors and options, recovering 61/120, 35/120, and 34/120 answers.

For the saved AI2D_TEST/70405 example, the question is "How many organisms in this diagram are producers?" Original choices are A=6, B=5, C=2, D=4. The image has four Producer boxes, and the first answer is D. The trace states four distinct boxes labeled "Producer" at the bottom and "there are four producers in the diagram". Image-free recovery uses A=4, B=6, C=5, D=2, making A the target. The original condition supplies this trace, the empty condition removes it, and the mismatch supplies AI2D_TEST/452603's trace answering "What is located on the top?" by identifying a leaf's rachis. All three share the producer question and reordered choices. Training uses random permutation, while this diagnostic shares a fixed cyclic permutation across conditions. Revised Figure 6(c) shows this input construction and the aggregate outcomes.

The rationale follows Visionary-R1's caption-conditioned answer reward, Vision-SR1's image-free recovery from its description, and PRCO's Solver-based caption utility. Bifocal applies recovery directly to the answer-producing trace. Option remapping addresses original-label dependence, while the paired controls measure additional trace information.

Question 2. Factual changes with matched detail

Submitted Table 3(a) already includes decisive-value controls with matched textual detail. These demonstrate factual sensitivity on controlled inputs. Generated traces show coarser responses to individual edits.

InterventionPairsRating decreasesTiesIncreases
Controlled value edit605910
Controlled image swap606000
Generated-trace value edit383332
Generated-trace image swap3813241

The controlled pairs fix the image, question, sentence template, and claim count while replacing the decisive value, using equal-length replacements where available. OCR item ocr_synth_000 asks, "What is the reference code printed on the document?" The image contains CT-7586-X. Its complete supported trace is The image text relevant to the question reads "CT-7586-X". I read those characters directly from the image. The edit substitutes TX-8654-F, also nine characters, leaving all surrounding text fixed. Raw ratings are 5 and 1. The image-swap test instead fixes every text character. Revised Figure 6(a) shows the input schematic, trace excerpts, and raw ratings.

The generated-trace procedure replaces one numeric option-string occurrence with an equal-length distractor, retaining every other character. AI2D_TEST/59601 asks, "How many stages are shown in the life cycle?" Choices are A=6, B=none of the above, C=4, D=5. The correct and original answers are C. Its trace states "The image is divided into four distinct stages of a ladybug's life cycle" and "The image explicitly shows four stages". The final sentence says "The question directly asks for the number of stages, and the answer is clearly 4." The edit changes that final 4 to 6 while preserving both observations. Both ratings are 5. Revised Figure 6(b) shows this edited span and the retained observations.

Thus a local incorrect statement can retain the whole-trace rating. The 33 ties, including 27 at 5-to-5, characterize this discrete preference's local resolution. The controlled value/image tests establish image sensitivity, while the component comparison and external audit assess its training contribution and the resulting solutions.

Reviewer_6773OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Bifocal Self-Rewarding, a RL framework for VLMs. The framework evaluates the same reasoning chain from two perspectives: a) it uses the current policy to perform image-free answer recovery in order to measure answer sufficiency; b), it uses a frozen model to conduct image-conditioned scoring in order to measure visual fidelity. Both types of rewards are integrated into GRPO. Experiments across four backbone models and 15 benchmarks show overall performance improvements.

Critical strengths

The paper addresses an interesting issue: answer correctness does not necessarily imply that the reasoning is genuinely grounded in visual information. The proposed framework is intuitive, easy to implement, and incurs no additional inference cost. The authors conduct extensive evaluations across multiple models and benchmarks, together with component ablations, and provide largely positive empirical evidence.

Critical weaknesses
  1. Limited novelty. The main components are relatively close to existing methods: Vision-SR1 has already adopted image-free answer recovery; ACRE verifies the complete reasoning chain through repeated answering; and PeRL-VL jointly considers visual faithfulness and sufficiency. I hope authors provide more discussion / comparation on these related works.

  2. Possible answer leakage in sufficiency verification. The reasoning chain may already explicitly contain the final answer. Consequently, even when the visual evidence in the reasoning chain is insufficient or the reasoning is invalid, an image-free verifier may recover the correct answer simply by reading the conclusion. Shuffling option labels does not eliminate this issue. Answer-masking validation experiments are needed to determine whether this reward truly measures reasoning sufficiency.

  3. Insufficient evidence for improved reasoning fidelity. Although the number of jointly valid solutions in the VMCBench audit increased from 115 to 126, the proportion of jointly valid solutions among correct answers decreased from 82.1% (115/140) to 79.7% (126/158). This suggests that the improvement in joint validity may mainly result from higher answer accuracy rather than from reasoning becoming more faithful under the condition of correct answers. A controlled comparison on questions that both models answer correctly would provide stronger evidence.

Questions for authors

See weakness. I will raise my score if concern got addressed.

7 comments
AuthorsOpenReview ↗

Relation to Vision-SR1, ACRE, and PeRL-VL

Thank you for highlighting these related methods. Submitted Sections 2 and 3 and Appendix F discuss Vision-SR1 and ACRE. We clarify the differences and add PeRL-VL below.

Vision-SR1 generates a separate visual description and tests answer recovery from it without the image. Submitted Section 3 explains how the subsequent answering trace can omit, misread, or alter evidence from an otherwise sufficient description. Bifocal evaluates the actual answering trace, aligning feedback with the sampled trajectory receiving policy-gradient credit.

ACRE reuses the answering trace, shuffles options, and rewards consistent, correct answers. Its Section 4 retains the image/video during the second pass, allowing direct access to visual evidence while reanswering. Bifocal also shuffles options, then removes the image during recovery so that image-derived information available to that pass must travel through the trace. A separate image-conditioned check assesses the same trace against the image.

PeRL-VL uses a see-think-answer format. A VLM judge, mainly GPT-4o, scores the separate description for faithfulness and sufficiency, combining description and answer rewards through addition or gating. Text-reasoning SFT precedes perception RL. The paper also includes a self-scoring ablation. Bifocal instead assesses the evidence and its use within the actual answering trace through both checks. The current policy performs recovery, and its frozen pre-RL initialization supplies the visual preference.

The theoretical distinction concerns both the evaluated text and access to evidence. A trace can determine an answer using a false visual premise, while accurate observations can omit information needed to solve the task. Moreover, replacing an unseen image while fixing the question, options, and trace leaves image-free recovery unchanged even when factual support changes. Together with the description-to-trace gap, this motivates checking sufficiency and visual support on the same answering trace through complementary image access. The current verifier follows evolving reasoning conventions, while the frozen observer provides a stable visual reference.

The table compares the main training formulations. The first feature concerns direct training feedback on the actual answering trace.

MethodChecks answering traceImage-free answer recoveryImage-conditioned visual rewardNo external fidelity model used
Vision-SR1✗✓✗—
ACRE✓✗✗—
PeRL-VL (GPT-4o)✗✗✓✗
Bifocal✓✓✓✓

The dash indicates no separate fidelity-scoring component. PeRL-VL's visual reward evaluates its description, while Bifocal's evaluates its answering trace. Submitted Table 2 tests Bifocal's visual branch and checkpoint roles through ablations.

We could not locate public checkpoints for ACRE and PeRL-VL, so we have not conducted large-scale benchmark evaluations of those two methods.

AuthorsOpenReview ↗

Answer Recovery After Masking Conclusions and Numerical Results

Thank you for suggesting this direct control. We tested recovery after masking answer declarations and numerical results, extending submitted Section 5.4 and Table 3(b).

We use the audit's 200 questions and saved outputs, with 50 each from MathVista, VizWiz, GQA, and VQAv2. All conditions use the trained iteration-367 Bifocal-3B recovery model, the same shuffled choices per question, greedy decoding, and no image. Masks use text and candidate options without gold labels and were fixed before supplemental inference. Identical inputs reuse their first saved output. Scoring follows the training code's exact extracted-label comparison with the remapped gold label.

Original excludes the separate final-answer field. Commitment-masked removes explicit answer declarations, option commitments, and boxed conclusions, retaining observations and calculations. Commitment+value-masked also masks numerical candidate-answer values and fraction-format variants in quantitative reasoning traces, protecting numbers given in the question. This can also remove intermediate results. Selected spans become [ANSWER DECLARATION REMOVED]. Empty provides the question and choices alone.

Vision-SR1 applies image-free recovery to a separate description. We apply the same masks to its saved descriptions. The rows identify input text sources, and Bifocal-3B generates all second-pass answers. These are supplemental recovery outputs on the audit's saved questions and texts. Each cell gives strict label-correct recoveries out of 200 and the percentage.

Text sourceOriginalCommitment-maskedCommitment+value-maskedEmpty
Bifocal trace116 (58.0%)112 (56.0%)106 (53.0%)59 (29.5%)
Vision-SR1 description83 (41.5%)83 (41.5%)81 (40.5%)59 (29.5%)

Bifocal's retained traces continue to support recovery after masking. With the broader mask, strict label-correct recovery is 53.0%, exceeding empty input by 23.5 percentage points. The within-source comparisons show that information retained after the specified answer removals contributes to recovery under the fixed model and scoring rule. The Vision-SR1 row provides the corresponding saved-description control.

We further examine traces that were actually modified and whose first answers were correct. The two masks modify 50/81 Bifocal traces and 2/27 Vision-SR1 descriptions, respectively. Among the modified Bifocal traces, 35/56 have correct first answers, satisfying the first-answer condition for full sufficiency credit. Each row below compares original, masked, and empty input on its own fixed subset.

Actually modified Bifocal traces with correct first answersItemsOriginalMaskedEmpty
Commitment masking3527 (77.1%)23 (65.7%)11 (31.4%)
Commitment+value masking5645 (80.4%)33 (58.9%)23 (41.1%)

On these actually modified subsets, masked traces yield 12 and 10 more correct recoveries than empty input, respectively, giving advantages of 34.3 and 17.9 percentage points, calculated before rounding. The retained information therefore continues to support recovery within the correct-first-answer cases relevant to the reward.

A saved calculation example makes the intervention concrete. MathVista item 507 gives diameter AB, AC=8 inches, and BC=15 inches, and asks for the circle's radius. Shuffled choices are A 17, B 8.5, C 23, and D 9.5. The trace identifies right triangle ACB and uses the Pythagorean theorem to obtain diameter 17 and radius 8.5. Value masking removes both computed numbers. Its relevant excerpt becomes

By calculating, we find that AB is [ANSWER DECLARATION REMOVED] inches. Since the radius is half the diameter, the radius of the circle is [ANSWER DECLARATION REMOVED] inches.

The retained givens and geometric relations permit recomputing sqrt(8²+15²)/2=8.5. The saved original and masked recoveries both output <answer>B</answer>. Empty input also answers B on this item. This illustrates successful recovery after masking both numerical results, with the aggregate comparisons above measuring the trace's added contribution.

A saved recognition example illustrates that contribution at item level. VizWiz item 6695 asks "What brand is this popcorn?" Choices are A Jolly Time, B Act ii, C Pop Secret, and D Orville Redenbacher's. We mask "the answer is unequivocally ACT II" and retain the observation that "ACT II" is prominently displayed on the box. Original and masked inputs recover B, while empty input outputs the incorrect "Jolly Time." The retained observation supports selecting the matching shuffled option.

AuthorsOpenReview ↗

Matched Questions Clarify the Gain in Jointly Valid Solutions

Thank you for recommending this matched-question comparison. It directly addresses whether the audit gain persists when final-answer correctness and question identity are held fixed. On the 130 questions both models answer correctly, the paired breakdown gives 113 jointly valid Bifocal solutions (86.9%) and 110 Vision-SR1 solutions (84.6%). This is a descriptive gain of three jointly valid solutions on the same correct-answer set.

We use the explicit-answer criterion and F/S rubric of submitted Table 4 and Appendix C.4. Joint validity requires a correct final answer, image-supported decisive evidence without a decisive contradiction (F=2), and observations and reasoning sufficient to select the stated answer without the image (S=2).

Questions grouped by explicit answer correctnessItemsJoint-valid Vision-SR1Joint-valid Bifocal
Both models correct130110113
Bifocal alone correct28013
Vision-SR1 alone correct1050
Neither model correct3200
All audited questions200115126

The gain has two identifiable components. Bifocal provides three additional jointly valid solutions on the shared-correct questions. The model-only correct groups contribute a further 13-5=8. Together, 3+8=11 accounts for the overall increase. The matched comparison therefore supplies evidence of better joint validity at fixed final-answer correctness, alongside additional valid solutions on questions where Bifocal alone answers correctly.

The reported 82.1% and 79.7% conditional rates use different question sets. Vision-SR1's 140 explicitly correct answers combine the 130 shared questions with 10 model-only questions. Bifocal's 158 combine the shared questions with 28 model-only questions. The requested comparison fixes the question set and separates the shared-correct result from these composition effects.

Our interpretation is that Bifocal increases the yield of solutions satisfying answer correctness, visual support, and sufficiency together on this selected audit. The shared-correct analysis supports that interpretation under matched correctness, and the full 200-question endpoint captures the usable output yield, increasing from 115/200 (57.5%) to 126/200 (63.0%). This paired analysis directly examines the relation between answer gains and joint validity raised in your question.

Thank you for your constructive suggestions. We hope these clarifications and additional analyses address your concerns, and we would be grateful if you could consider raising your rating of the paper based on this additional evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.