- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Bifocal Self-Rewarding, a reinforcement-learning framework for multimodal reasoning that evaluates the same answer-producing chain from two complementary views. The current policy removes the image and attempts to recover the answer from the question, options, and the same reasoning chain, providing a sufficiency signal. A frozen copy of the pre-RL model retains the image and scores how much concrete, image-checkable visual evidence is exposed by the chain, providing a fidelity preference. These signals are combined with answer correctness and auxiliary format and length rewards in a GRPO update. The method is trained on Bifocal-24K, a 19-source mixture covering science, visual understanding, and multimodal mathematics. It is evaluated on 15 multimodal benchmarks with Qwen2.5-VL and Qwen3-VL backbones from 3B to 8B parameters. The paper also includes component ablations, paired perturbation tests, visual-evidence diagnostics, and a VMCBench chain audit. The reported average score improves over the corresponding base model in all four reported backbone settings, although the gains vary across benchmarks.
Critical strengths
1. Clear problem formulation
The paper identifies a meaningful failure mode in multimodal RL: outcome correctness alone can reward reasoning chains that are incomplete or visually unsupported. It makes the distinction between answer sufficiency and visual fidelity explicit: a chain can contain enough information to recover an answer while relying on unsupported visual claims, and it can contain accurate observations without carrying enough information to solve the task. This gives the two-view design a clear motivation and makes the later diagnostics easier to interpret.
2. Same-chain evaluation
Both reward views operate on the same answer-producing chain. This is a useful design choice because it avoids evaluating a separate description and then assuming that the downstream reasoning uses that description faithfully. The image-stripped recovery pass and the image-conditioned visual pass reuse the same reasoning trace, so the two rewards examine the object that actually produces the final answer. The chain-removal and mismatched-chain controls in Table 3 further test whether the trace contributes answer-relevant information beyond the shared question and options.
3. Well-motivated checkpoint roles
Using the current policy for image-free recovery and a frozen pre-RL checkpoint for visual preference gives the two signals different and understandable roles. The evolving verifier can follow changes in the policy's reasoning language, while the frozen observer provides a stable visual reference during training. The two rewards are detached before the GRPO update, so only the policy is optimized. The ablation results in Table 2 support the claim that the components contribute differently: adding the frozen visual branch to the sufficiency-only model raises E-Avg by 2.7 points and R-Avg by 1.1 points in the reported 3B comparison.
4. Broad empirical evaluation
The method is evaluated across 15 benchmarks, four model configurations from 3B to 8B, and several groups of related baselines covering outcome-driven RL, self-verification, description-based methods, and perception-enhanced RL. The benchmark suite spans science and mathematics, general visual understanding, chart and text extraction, and hallucination-oriented tasks. The paper also combines aggregate results with component ablations, paired perturbation tests, training diagnostics, and an output-chain audit, providing evidence about both performance and the proposed mechanism.
Critical weaknesses
1. The link between the visual preference and the Fidelity objective needs further explanation
The paper defines Fidelity as whether visual claims in a reasoning chain are supported by the image. The frozen model assigns a 1–5 visual preference score, which Eq. 9 converts into a group-relative reward for correct responses scoring above the group's upper median. Section 4.3 and Appendix A.6 explicitly distinguish this proxy from claim-level truth. However, the rating guide in Appendix B.3 emphasizes the amount and specificity of image-checkable information without explicitly instructing the judge to check each visual claim or lower the score for contradictory claims. Table 3 shows that factual edits lower the score in 59 of 60 controlled synthetic chains, whereas local edits in 38 real chains produce 3 decreases, 33 ties, and 2 increases. This limited sensitivity means that a local factual change can leave the preference score unchanged. The authors should explain how rewarding this preference encourages image-supported visual claims and how far this mechanism can realize the formal Fidelity objective.
2. Direct evidence for improved formal Fidelity is limited
The paper evaluates the trained models on 15 benchmarks, but the main results primarily report final-answer, extraction, and hallucination metrics rather than a broad chain-level measurement of formal Fidelity. Direct F/S evaluation appears only in the VMCBench analysis, where an external rubric is applied to saved chains. This leaves a gap between the benchmark gains and the claim that the method improves visual fidelity: the audit provides evidence about jointly valid solutions on its audited items, but it does not establish how formal Fidelity changes across the other benchmarks or the full set of model outputs, and the training preference and external F score are separate quantities. The paper should distinguish results supporting task performance, evidence presentation, and formal Fidelity, and state the Fidelity claim at the scope supported by the available audits and paired diagnostics.
3. The VMCBench task selection and adjudication need clearer reporting
Appendix C.4 states that the audit uses 200 items from four tasks selected after examining answer accuracy, while Section 5.6 reports that the increase in jointly valid solutions comes from VizWiz and MathVista and that VQAv2 and GQA tie. The audit begins with automated F/S judgments and reviews borderline cases before reporting final counts. The task composition affects how the 126-versus-115 comparison should be read, and the adjudication stage can affect which outputs reach both required scores, F=2 and S=2, yet the appendix does not explain the task-selection rationale, who performed the review, how borderline cases were identified, or how many initial scores changed. The authors should briefly report these existing selection and adjudication details so that the selected-subset audit is transparent.
Questions for authors
-
Section 4.3 and Appendix A.6 identify the frozen score as a proxy preference, and Table 3 shows that most local factual edits in real chains leave that score unchanged. What is the authors' explanation for how the reward constructed from this preference in Eq. 9 encourages image-supported visual claims? In particular, how does the scoring rule distinguish a chain with several concrete but partly incorrect details from one with well-supported details, and what limitations does this imply for the formal Fidelity objective?
-
Across the 15 main benchmarks, which existing results do the authors view as evidence that the trained policy improves formal visual Fidelity, rather than only final-answer performance or the exposure of visual evidence? If the direct F audit is intended as the main evidence, please clarify how its scope in VMCBench relates to the broader Fidelity claim.
-
Appendix C.4 states that the four audited tasks were selected after examining answer accuracy and that borderline judgments received item-level review. What was the rationale for choosing these tasks, who performed the review, how were borderline cases identified, and did any initial F/S scores or joint-validity counts change after review?
3 comments
Fidelity Evidence, Audit Scope, and Adjudication
Question 2. Which results support formal Fidelity?
Within the 15-benchmark suite, submitted Figure 4(a) gives direct qualitative evidence about the trace's visual claims on AI2D. The question asks, "What feeds on crickets?" Perceval and Bifocal both answer B, lizards. Perceval additionally claims that the red-bellied black snake feeds on crickets, an unsupported image relation. Bifocal cites the direct cricket-to-frilled-neck-lizard arrow. Thus identical correct final answers accompany different factual support in their traces.
HallusionBench and POPE provide answer-level grounding evidence. At 3B, Bifocal/Vision-SR1 score 69.4/66.0 and 84.9/83.9, respectively (submitted Table 1). These assess visual consistency and hallucination in answers.
The main quantitative chain-level evidence is the external audit in submitted Section 5.6/Table 4. It compares Vision-SR1 and Bifocal, both 3B, on the same 200 VMCBench questions. External F=2 requires a decisive image-supported claim without a decisive contradiction. Correct solutions meeting F=2 increase from 125 to 134, or 62.5% to 67.0% of all attempts. A separate text-only S check tests whether stated observations and necessary reasoning determine the stated option. Correct solutions with F=S=2 increase from 115 to 126, or 57.5% to 63.0%. VizWiz contributes eight additional jointly valid solutions and MathVista three, with ties on VQAv2 and GQA.
The quantitative Fidelity conclusion concerns the production of correct, evidence-supported solutions on these four complete 50-item DEV subsets under the decisive-claim rubric. The AI2D case gives qualitative chain-level evidence within the main suite, and the hallucination metrics support answer-level grounding. The external F/S audit measures chain quality separately from the 1-5 training preference. Revised Section 5.6 and Appendix C.4 clarify this scope, and revised Figure 6(d) summarizes the checks, correctness condition, and 200-item denominator.
Question 3. Selection and review details
Task selection. We chose four VMCBench subtasks complementing the main suite. VizWiz/VQAv2 cover recognition, MathVista visual mathematics, and GQA relations. This follow-up scope was selected after examining answer accuracy. Each contributes its complete 50-item DEV subset, with identical questions for both methods. Submitted Table 4 reports all four, including the equal joint counts on VQAv2/GQA.
Who reviewed. Initial judgments used qwen3.8-max-0902 with thinking enabled and temperature zero. The image-conditioned F request presents two anonymized traces in randomized order. Separate text-only S requests assess each trace independently. Requests contain the question, choices, trace, and stated answer, excluding the gold label. An AI assistant inspected saved images, traces, and judge explanations item by item. The authors confirmed final decisions under the stated rubric.
Borderline criteria. Score-1 decisions received review when the rationale and cited evidence left a potentially decisive observation or inference unresolved. F review checked image support for observations explicitly stated in the trace. S review checked whether the stated observations and necessary relation already determined the answer, including an omitted elementary arithmetic expansion. The same criteria applied to both models.
Changes. Initial automated joint counts were 115 for Vision-SR1 and 118 for Bifocal. Six Bifocal F judgments changed from 1 to 2, raising its joint count to 124. Two S judgments then changed from 1 to 2, giving 126. Vision-SR1's score distributions and joint count remained 115 after review. Table 4 reports the final adjudicated counts. Revised Appendix C.4 now provides these selection/review details. Thank you again for the careful assessment and specific suggestions.
Thank you for the clarification and response. I have no further questions, and I will maintain my "weak accept" rating.
How the Visual Preference Supports Fidelity
Thank you for the careful review and precise account of the motivation, same-chain design, and checkpoint roles. We address each question together with its corresponding weakness.
Question 1. Image-supported claims and the visual reward
Submitted Eq. 9 gives additional credit to responses with correct original answers whose traces receive higher image-conditioned ratings within the same question. The observer is the instruction-tuned starting checkpoint, frozen during RL. It receives the image, question, and answer-producing trace and returns a 1-5 rating. Among eight candidates, the upper median is the fifth ascending rating. Correct responses above it receive a bonus increasing with their relative rating.
Submitted Appendix B.3 elicits concrete, image-checkable observations. The observer evaluates the complete trace against the image using its instruction-tuned visual capability. Submitted Table 3(a) tests whether image support affects this preference with textual specificity fixed.
For OCR item ocr_synth_000, the question is "What is the reference code printed on the document?" The image contains CT-7586-X. The supported trace is
The image text relevant to the question reads "CT-7586-X". I read those characters directly from the image.The paired trace replaces the code with TX-8654-F, also nine characters, preserving every surrounding character. With the image/question fixed, raw ratings are 5 and 1. Across 60 controlled OCR/table pairs, ratings decrease in 59 and tie in one. Replacing the image while fixing the complete supported text lowers all 60 ratings. These interventions show sensitivity to image agreement beyond detail richness.The generated-trace procedure edits one numeric option-string occurrence, preserving all other text. AI2D_TEST/59601 asks, "How many stages are shown in the life cycle?" Choices are A=6, B=none of the above, C=4, D=5. The correct and original answers are C, four stages. The trace states "The image is divided into four distinct stages of a ladybug's life cycle" and "The image explicitly shows four stages". Its concluding sentence says "The question directly asks for the number of stages, and the answer is clearly 4." The edit replaces that final 4 with 6, retaining both count observations. The observer receives the original image/question and each trace. Both ratings are 5. This pair is among the 38 generated-trace edits with 3 decreases, 33 ties, and 2 increases. Revised Appendix D identifies the edited span explicitly.
For the partly incorrect versus well-supported traces in your question, the image-conditioned observer supplies the judgment and Eq. 9 allocates credit above the group threshold. If both original answers are correct, equal raw ratings give equal visual bonuses. A local incorrect statement can therefore retain a high rating, as the life-cycle pair shows. The training signal is a whole-trace preference with experimentally demonstrated image sensitivity. Its five-level resolution means that individual factual edits may leave its score unchanged. Formal Fidelity concerns image support for visual claims, with output evidence assessed separately below.
Submitted Table 2 tests the training contribution. Adding the frozen visual branch under the same co-evolving sufficiency verifier raises R-Avg from 63.5 to 64.6 and E-Avg from 74.6 to 77.3. R-Avg averages MMMU/MMStar/AI2D, and E-Avg averages TextVQA/OCRBench. The external audit in Question 2 checks decisive factual support in the resulting solutions. Revised Figure 6(a,b) illustrates the excerpts, edits, and saved ratings. Panel (a) uses an OCR input schematic, and both panels list image-swap summary counts.