acceptodds
Under review as a conference paper at ICLR 2027

Seeing while Reasoning: Advancing Visual Faithfulness in Small Multi-modal Language Models with Verifiable Perception-aware Intermediate Process Optimization

Abstract

Reinforcement learning with verifiable rewards (RLVR) advances multimodal chain-of-thought (MCoT) reasoning, but prevailing objectives emphasize answer correctness or output-level vision-language consistency, leaving intermediate reasoning weakly supervised for visual faithfulness. This work analyze direct text-to-visual attention using Visual Attention Mass (VAM) and identify two systematic behaviors. Visual Attenuation denotes the abrupt reduction in visual attention when generation transitions from captioning to reasoning. Perception Sensitivity captures the positive association between stage-level VAM and the correctness of the corresponding intermediate stage. Motivated by these findings, we operationalize the Seeing while Reasoning principle and construct MCoT-ViG, a structured MCoT dataset pairing four-stage rationales with controlled visual variants and object-level grounding annotations. We further introduce Perception-aware Intermediate Process Optimization (PaIPO), which combines stage-wise process rewards for intermediate supervision with two complementary perception rewards. Caption Perception Reward (CPR) measures caption sensitivity to controlled visual variants, whereas Reasoning Grounding Reward (RGR) verifies object references against annotated target regions. Experiments across five benchmarks and three Qwen3-VL model scales demonstrate that PaIPO consistently improves multimodal reasoning and visual faithfulness. Relative to the backbones, PaIPO achieves absolute improvements of +3.53 to +6.14 in VRC-Bench Step Score and attains edited-image accuracies of 55.92% to 65.69% on VFaith-Bench. Ablations confirm the complementary contributions of CPR and RGR, while analysis of reward dynamics and post-training VAM demonstrate faster convergence and more sustained visual attention throughout intermediate reasoning, respectively. Code and dataset will be available to facilitate reproducible research.

Primary area:
reinforcement learning
Keywords:
Multimodal Large Language Models, Reinforcement Learning with Verifiable Rewards, Visual Faithfulness, Process Reward, Visual Grounding, Perception-aware Optimization

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_xSRaOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies visual faithfulness during intermediate reasoning in small multimodal language models. It introduces Visual Attention Mass (VAM), which measures the average direct attention allocated by generated text tokens to visual tokens. Analyses of Qwen3-VL models identify a reduction in direct visual attention after caption generation and modest positive correlations between VAM and intermediate reasoning quality.

Motivated by these observations, the authors construct MCoT-ViG, using Visual-CoT source records to generate four-stage rationales—Summary, Caption, Reasoning, and Conclusion—with object-grounding annotations and edited image variants. They propose Perception-aware Intermediate Process Optimization (PaIPO), combining:

  1. Caption Perception Reward (CPR): reference-caption semantic alignment and original-versus-edited-image likelihood discrimination;
  2. Reasoning Grounding Reward (RGR): bounding-box localization and object-identifier consistency;
  3. Step Reward (SR): reference-based reasoning alignment, ordering, and terminal entailment.

These rewards supplement answer and format rewards under GRPO or DAPO. Experiments cover five benchmarks and Qwen3-VL models at 2B, 4B, and 8B. PaIPO consistently improves reasoning and visual-faithfulness metrics over the backbones, step-reward baselines, and matched PAPO reimplementations. Reward ablations support the usefulness of combining CPR and RGR.

Critical strengths
  1. Controlled comparisons support the usefulness of the proposed perception rewards.

The paper compares methods under reportedly matched data and training settings, across three model scales and two optimizers. Table 2 further holds SR constant while introducing CPR or RGR, with both additions improving reasoning and faithfulness measures. This is consequential because it provides evidence that the proposed supervision contributes beyond ordinary step-based RL.

  1. The evaluation connects intermediate-process improvements to independent behavioral outcomes.

Improvements are not limited to VAM or training rewards: PaIPO also improves answer accuracy, edited-image accuracy, and SPD-Faith scores. The combination of process-level and behavioral evaluations strengthens the argument that the method has practical value rather than merely optimizing an attention diagnostic.

Critical weaknesses
  1. The validity of counterfactual likelihood discrimination is insufficiently established in the main text.

CPR assumes that higher caption likelihood under the original image indicates genuine visual support. However, original-conditioned sampling already favors the original image, and edited images may introduce unintended distribution shifts. Moreover, the aggregation emphasizes tokens with large likelihood differences without explicitly restricting them to the modified attributes. If these differences are driven by sampling bias or editing artifacts, CPR could reward image discrimination rather than faithful perception.

  1. The proposed diagnostics and grounding reward do not establish sustained visual use during reasoning.

VAM captures direct attention rather than the total influence of visual evidence, while RGR measures box accuracy and identifier consistency. Neither establishes that later reasoning correctly uses the localized objects or their attributes and relations. The paper appropriately acknowledges the correlational nature of VAM, but the “Seeing while Reasoning” interpretation remains stronger than the evidence currently supports.

Questions for authors
  1. How is the CPR indifference margin calibrated, and what evidence shows that it removes original-conditioned sampling bias?

Please specify whether calibration uses held-out data and whether the margin varies across model scales or training checkpoints. Does the reward remain discriminative when captions are sampled under the edited image rather than the original image?

6 comments
AuthorsOpenReview ↗

Author Response to Reviewer xSRa (Part 1)

We sincerely thank you for the positive assessment and the constructive comments on the reliability of counterfactual visual discrimination and the interpretation of our visual-grounding diagnostics. We address the two concerns below.

W1, Q1. Reliability and Calibration of Counterfactual Visual Discrimination

We address this concern through three complementary analyses, focusing on the specificity of counterfactual likelihood differences, the calibration of the indifference margin, and the validity of bidirectional counterfactual discrimination.

1. Are counterfactual likelihood differences primarily attributable to the edited attributes?

We conduct a token-level attribution analysis on sampled rollout Captions. For each sample, we align the recorded attribute transformations with the corresponding descriptions in the generated Caption and divide the tokens into two groups: edited-attribute-related tokens (TiattrT_i^{\mathrm{attr}}) and other Caption tokens (TiotherT_i^{\mathrm{other}}). Following Eq. (22), we calculate the token-level counterfactual evidence ei,te_{i,t}. We then decompose the aggregated score in Eq. (23) into the contributions of the two token groups: $ C_i^{\mathrm{attr}} = \frac{ \sum_{t\in T_i^{\mathrm{attr}}} |e_{i,t}|^\gamma e_{i,t} }{ \sum_{t\in T_i^{\mathrm{cap}}} |e_{i,t}|^\gamma+\epsilon }, $ $ C_i^{\mathrm{other}} = \frac{ \sum_{t\in T_i^{\mathrm{other}}} |e_{i,t}|^\gamma e_{i,t} }{ \sum_{t\in T_i^{\mathrm{cap}}} |e_{i,t}|^\gamma+\epsilon }. $ Accordingly, Dicf=Ciattr+CiotherD_i^{\mathrm{cf}}=C_i^{\mathrm{attr}}+C_i^{\mathrm{other}}. We further quantified the proportion of absolute counterfactual evidence contributed by the edited-attribute tokens: $ R_i^{\mathrm{attr}}= \frac{ \sum_{t\in T_i^{\mathrm{attr}}} |e_{i,t}|^{\gamma+1} }{ \sum_{t\in T_i^{\mathrm{cap}}} |e_{i,t}|^{\gamma+1}+\epsilon }. $ We categorize the sampled Captions into three groups: (1) Aligned, where the Caption correctly describes the original value of the edited attribute; (2) Omitted, where the edited attribute is not mentioned; and (3) Inconsistent, where the Caption mentions the edited attribute but inconsistent with the original image.

Table 1. Token-level attribution of counterfactual likelihood differences across model scales.

BackboneRollout GroupNDcfD_{\mathrm{cf}}CattrC_{\mathrm{attr}}CotherC_{\mathrm{other}}RattrR_{\mathrm{attr}}
2BAligned2,1230.32 ± 0.16+0.28 ± 0.14+0.04 ± 0.060.84 ± 0.12
2BOmitted1,0250.04 ± 0.09-+0.04 ± 0.09-
2BInconsistent373−0.16 ± 0.15−0.21 ± 0.14+0.05 ± 0.080.76 ± 0.13
2BAmbiguous479----
4BAligned2,3740.33 ± 0.15+0.29 ± 0.14+0.04 ± 0.060.88 ± 0.11
4BOmitted1,0160.03 ± 0.05-+0.03 ± 0.05-
4BInconsistent337−0.19 ± 0.14−0.23 ± 0.13+0.04 ± 0.070.79 ± 0.12
4BAmbiguous273----
8BAligned2,4930.35 ± 0.14+0.31 ± 0.13+0.04 ± 0.050.89 ± 0.10
8BOmitted1,0070.02 ± 0.05-+0.02 ± 0.05-
8BInconsistent245−0.21 ± 0.14−0.25 ± 0.12+0.04 ± 0.050.84 ± 0.11
8BAmbiguous255----

The results show consistent attribution patterns across all three model scales. For Aligned Captions, edited-attribute tokens account for 84%–89% of the absolute counterfactual evidence. For Omitted Captions, the mean DcfD_{\mathrm{cf}} remains relatively small, suggesting limited counterfactual evidence from unrelated caption content. For Inconsistent Captions, edited-attribute tokens exhibit negative contributions, accounting for 76%–84% of the absolute evidence, indicating that these tokens are more strongly supported by the edited images. These findings demonstrate that counterfactual likelihood differences are predominantly driven by edited-attribute tokens with consistent patterns across model scales.

AuthorsOpenReview ↗

Author Response to Reviewer xSRa (Part 2)

2. How is the indifference margin calibrated against original-conditioned sampling bias?

Sampling Captions under the original image may introduce an inherent likelihood preference toward the original visual condition. To account for this effect, we calibrate the counterfactual indifference margin before policy optimization using an independent set of 1,000 held-out samples and the pretrained Qwen3-VL-4B-Instruct model.

Specifically, for each held-out sample, the frozen model generates a Caption conditioned on the original image. We then retain the identical generated token sequence and evaluate its conditional likelihood under both the original and controlled images using teacher forcing. We calculate the token-level likelihood differences and aggregate them into DcfD_{\mathrm{cf}} following Eqs. (22)–(23), using the same counterfactual temperature and confidence exponent as in our reward formulation. Based on the empirical distribution of DcfD_{\mathrm{cf}} across these 1,000 held-out samples, we set mcf=0.10m_{\mathrm{cf}}=0.10 as a fixed indifference margin.

Although this calibration is performed exclusively with the pretrained 4B model before reinforcement learning. We analyze the post-training rollout Captions across the three model scales to examine the applicability of this margin. As shown in Table 1, Aligned Captions exhibit substantially larger positive counterfactual likelihood differences (0.32–0.35), whereas Omitted Captions exhibit much smaller mean differences (0.02–0.04). As supplementary material, we refer to the counterfactual likelihood dynamics reported in Appendix D.2 (Figure 7). Under GRPO, PaIPO-G develops a larger counterfactual NLL gap than PAPO-G during the later stages of training. Under DAPO, PaIPO-D establishes visual discrimination earlier than PAPO-D, although their gaps become comparable toward the end of training. These results indicate that the indifference margin does not prevent the policy from progressively strengthening its dependence on the original visual evidence during optimization.

AuthorsOpenReview ↗

Author Response to Reviewer xSRa (Part 3)

3. Does counterfactual discrimination remain effective when captions are sampled under edited images?

To examine whether counterfactual discrimination depends on original-image-conditioned sampling, we conduct an offline evaluation using the trained PaIPO checkpoints. Specifically, for each held-out image pair, we independently sample Captions conditioned on the edited image, and then evaluate the conditional likelihood under both the edited and original images using the same frozen policy checkpoint.

For edited-image-conditioned Captions, we reverse the likelihood comparison in Eq. (22): $ \log\pi_{\bar\theta}(c_{i,t}^{rs}\mid q_i,I_i^{rs},c_{i,<t}^{rs}) - \log\pi_{\bar\theta}(c_{i,t}^{rs}\mid q_i,I_i,c_{i,<t}^{rs}). $ We categorize the generated Captions into three groups: (1) Edited-aligned, where the Caption correctly describes the edited attribute value; (2) Omitted, where the edited attribute is not mentioned; and (3) Inconsistent, where the Caption describes the inconsistent attribute value rather than the edited value. Ambiguous cases are excluded from the primary attribution analysis. We perform the same token-level attribution analysis to examine whether the edited-attribute tokens remain the primary source of counterfactual likelihood differences.

Table 2. Bidirectional counterfactual likelihood discrimination under edited-image-conditioned sampling.

BackboneRollout GroupNDcfrsD^{rs}_{\mathrm{cf}}CattrrsC^{rs}_{\mathrm{attr}}CotherrsC^{rs}_{\mathrm{other}}RattrrsR^{rs}_{\mathrm{attr}}
2BEdited-Aligned2,0940.28 ± 0.13+0.22 ± 0.14+0.06 ± 0.060.75 ± 0.13
2BOmitted9730.06 ± 0.07-+0.06 ± 0.07-
2BInconsistent547−0.19 ± 0.09−0.23 ± 0.13+0.04 ± 0.070.80 ± 0.14
2BAmbiguous386----
4BEdited-Aligned2,1470.33 ± 0.15+0.29 ± 0.14+0.04 ± 0.060.83 ± 0.15
4BOmitted1,0040.03 ± 0.05-+0.03 ± 0.05-
4BInconsistent514−0.19 ± 0.14−0.23 ± 0.13+0.04 ± 0.070.77 ± 0.19
4BAmbiguous335----
8BEdited-Aligned2,2530.32 ± 0.17+0.29 ± 0.13+0.03 ± 0.080.85 ± 0.14
8BOmitted1,0140.04 ± 0.05-+0.04 ± 0.05-
8BInconsistent493−0.24 ± 0.08−0.28 ± 0.14+0.04 ± 0.080.80 ± 0.13
8BAmbiguous240----

These results indicate that the direction of counterfactual likelihood discrimination changes consistently with the sampling image condition. Captions describing the edited attributes receive greater likelihood support from the edited images, whereas Captions describing inconsistent attribute values exhibit greater likelihood support from the original images. These findings demonstrate that the proposed counterfactual discrimination mechanism generalizes effectively across different image-conditioned sampling settings.

Our method does not incorporate edited-image conditioned rollouts into policy optimization for two considerations. First, introducing edited-image-conditioned rollouts would approximately double the rollout generation and associated likelihood evaluation costs, substantially increasing the training time. Second, the current reward formulation is anchored to the original task input and its corresponding reference Caption. Incorporating edited-image-conditioned rollouts would require additional edit-specific reference supervision and corresponding modifications to the reference-alignment and counterfactual reward components. Nevertheless, we consider extending the training framework to incorporate edited-image-conditioned rollouts a promising direction for further investigation, which we leave for future work.

Reviewer_t8ngOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies visual faithfulness in multimodal chain-of-thought (MCoT) reasoning. It argues that models should continue to engage with visual evidence throughout intermediate reasoning. The authors introduce Visual Attention Mass (VAM) analysis, observing that direct text-to-vision attention decreases when generation transitions from captioning to reasoning, and construct MCoT-ViG, which contains four-stage rationales, controlled image edits, and box-level grounding annotations. Building on this setup, the proposed PaIPO combines a counterfactual caption-perception reward, a reasoning-grounding reward, and a step reward, and can be instantiated with either GRPO or DAPO.

Results on several reasoning and faithfulness benchmarks are encouraging, particularly for robustness to edited images and intermediate-step quality. However, the current evidence does not fully support the paper's broader central claim: VAM is only correlational, the need for sustained visual re-engagement is not analyzed across task types, and the reliability of the generated counterfactual edits requires stronger validation.

Critical strengths
  • This paper addresses an important issue. Answer-level RL rewards do not necessarily ensure that multimodal reasoning remains grounded in visual evidence. The attempt to introduce more verifiable process-level supervision for both perception and reasoning is meaningful.

  • The combination of CPR and RGR is interesting. Using controlled visual perturbations for caption verification together with explicit object-level grounding provides a concrete way to supervise intermediate visual behavior. The framework is also evaluated with both GRPO and DAPO, suggesting some degree of optimizer-level generality.

  • The experiments include multiple model scales, several benchmarks, and component-wise ablations, and the reported improvements are generally consistent. That said, the current evaluation does not convincingly disentangle whether the gains come from genuinely sustained visual reasoning, rather than from the structured output format, image-editing artifacts, or the object-centric supervision setup

Critical weaknesses
  • VAM provides correlational rather than causal evidence for the proposed failure mode. The positive association between VAM and Step Score is interesting, but higher attention mass does not necessarily mean that the model causally relies on visual evidence. Conversely, lower direct attention does not rule out the possibility that visual information is already represented in earlier hidden states and remains available to the language model. The observed correlation may also be confounded by question difficulty, response length, object saliency, or task type. Although the paper acknowledges this limitation, the main motivation would be substantially stronger with causal analyses, such as stage-specific visual-token masking or ablation, controlled image perturbations during reasoning, or analyses that control for response length and task difficulty.

  • The counterfactual caption reward may be sensitive to image-editing artifacts rather than the intended visual attribute. CPR relies on Qwen-Image-Edit to generate controlled visual variants and assumes that the original and edited images differ only in the specified local, answer-preserving attribute. In practice, generative editing may also introduce unintended changes in object identity, boundaries, texture, illumination, or surrounding context. As a result, the likelihood difference used by CPR may partly reflect sensitivity to editing artifacts or distribution shift, rather than genuine recognition of the edited attribute. The paper would benefit from quantitative evaluations of edit locality and fidelity, non-target edit controls, or validation on naturally paired or fully controlled images.

  • The claimed generality is limited by the object-centric, box-annotated training distribution. MCoT-ViG is constructed from Visual-CoT samples with target-object bounding boxes and is specifically designed around attribute editing and object-level grounding. This setup is well aligned with the proposed CPR and RGR rewards, but it remains unclear how well the method transfers to reasoning settings where box-level object references are insufficient or unnatural, such as chart understanding, OCR and document reasoning, dense geometric reasoning, fine-grained parts and boundaries, multi-image correspondence, or video reasoning. More targeted out-of-domain evaluations would better support the broader claim of improved multimodal reasoning and visual faithfulness.

Questions for authors
  • It remains unclear when sustained visual re-engagement is actually necessary. The central “Seeing while Reasoning” principle is motivated by an aggregate decrease in VAM from the Caption stage to the Reasoning stage. However, reduced direct visual attention is not necessarily harmful for every task. For simple recognition or answer-selection questions, the model may already have extracted sufficient visual evidence during early perception, after which language-based reasoning may be appropriate. In contrast, tasks involving multiple objects, spatial relations, fine-grained comparison, or counterfactual reasoning may genuinely require repeated visual verification. A task-level stratification by visual dependency, together with PaIPO gains for each category, would make the claim much more convincing by showing where sustained visual grounding is necessary and where it provides only limited benefit.

  • All experiments are conducted on the Qwen3-VL family, which limits the evidence for model-level generalization. PaIPO is evaluated across 2B, 4B, and 8B Qwen3-VL-Instruct models, and the results are consistent across these scales. However, these models share the same overall architecture, visual encoder design, pretraining recipe, tokenization, and prompting conventions. It is therefore unclear whether the reported visual-attention behavior and the benefits of CPR/RGR are specific to Qwen3-VL or would transfer to other MLLM families with different vision-language interfaces and attention patterns. Evaluation on at least one additional open model family would make the generality claim more convincing.

6 comments
AuthorsOpenReview ↗

Author Response to Reviewer t8ng (Part 1)

We sincerely thank you for the careful evaluation of our manuscript and for your constructive comments. We address each of your concerns below, with additional experimental analyses and clarifications where appropriate.

W1, Q1. Necessity of Sustained Visual Attention and Task-type Analysis

Our VAM analysis reveals two patterns during four-stage reasoning: Visual Attenuation, where direct visual attention progressively decreases in later reasoning stages, and Perception Sensitivity, where higher stage-level VAM is positively associated with the correctness of the corresponding reasoning stages. These observations motivate PaIPO, which introduces CPR and RGR to provide complementary visual supervision at different reasoning stages: coarse-grained visual attribute supervision at the Caption stage and fine-grained object-level grounding supervision at the Reasoning stage. We acknowledge that the VAM–reasoning quality association is correlational and does not independently establish a causal relationship.

To investigate the contribution of direct visual access at different reasoning stages, we conducted stage-specific visual-token erasure experiments (Table 1). Starting from the full-image reasoning condition (63.4 VRC Accuracy and 81.2 Step Score), removing visual-token access from the beginning of Summary causes substantial drops of 49.2 and 63.0, respectively. The impact progressively decreases when visual access is removed at later positions. Importantly, removing visual access from the beginning of Reasoning still reduces VRC Accuracy by 6.0 and Step Score by 7.7, while erasure from the middle of Reasoning leads to decreases of 3.2 and 4.1 points. These results indicate that direct visual access remains beneficial during intermediate reasoning, although its contribution varies across stages and diminishes toward final answer generation.

Table 1. Effect of Stage-specific Visual token Erasure on Multi-stage Reasoning Performance.

Erasure PositionVRC Acc. ↑Δ Acc.Step Score ↑Δ Step
Full image reasoning63.4—81.2—
Summary start14.2−49.218.2−63.0
Summary middle31.7−31.740.7−40.5
Caption start43.0−20.455.1−26.1
Caption middle51.7−11.766.2−15.0
Reasoning start57.4−6.073.5−7.7
Reasoning middle60.2−3.277.1−4.1
Conclusion start62.7−0.779.8−1.4
Conclusion middle63.4−0.081.1−0.1

However, this intervention evaluates the behavioral contribution of visual-token availability under our masking protocol, establishing a causal relationship between visual attention and reasoning quality remains challenging.

Furthermore, alongside the observed decline in direct attention to the original visual input, we observe substantial inconsistencies across reasoning stages, raising concerns that reduced visual attention may lead models to rely excessively on previously generated textual descriptions during subsequent reasoning. We analyze 21,666 four-stage reasoning trajectories to investigate the consistency of intermediate reasoning (Table 2). Among the 4,150 trajectories with correct Captions, 3,522 (84.9%) exhibit incorrect Reasoning. Similarly, among the 3,258 trajectories with correct final answers, 2,696 (82.7%) still contain incorrect intermediate Reasoning. These findings reveal substantial inconsistencies between the correctness of intermediate reasoning stages and final predictions. In particular, a correct initial visual description or final answer does not necessarily imply that the intervening reasoning process is correct. This motivates maintaining task-relevant visual grounding throughout reasoning and providing complementary supervision for visual perception and grounding, rather than relying excessively on previously generated language or final-answer optimization alone.

Table 2. Distribution of Stage-wise Correctness Patterns across Four-stage Reasoning Trajectories.

Correctness PatternNCorrectness PatternN
AC-SIC-CIC-RC91AC-SIC-CC-RC2
AC-SIC-CIC-RIC1,235AC-SIC-CC-RIC136
AIC-SIC-CIC-RC143AIC-SIC-CC-RC28
AIC-SIC-CIC-RIC8,036AIC-SIC-CC-RIC973
AC-SC-CIC-RC469AC-SC-CC-RC0
AC-SC-CIC-RIC1,104AC-SC-CC-RIC221
AIC-SC-CIC-RC787AIC-SC-CC-RC598
AIC-SC-CIC-RIC5,651AIC-SC-CC-RIC2,192

AC/AIC: Answer Correct/Incorrect; SC/SIC: Summary Correct/Incorrect; CC/CIC: Caption Correct/Incorrect; RC/RIC: Reasoning Correct/Incorrect.

AuthorsOpenReview ↗

Author Response to Reviewer t8ng (Part 2)

Regarding the task-dependent benefits, we provide the detailed results using the eight native task categories of VRC-Bench (Table 3). PaIPO-D consistently outperforms Step DAPO across all eight categories, with improvements ranging from 1.0 to 3.6 in Accuracy and 0.5 to 5.1 in Step Score. More pronounced improvements are observed in Complex Visual Perception (+3.6 Accuracy, +5.1 Step Score), Medical Imaging (+3.1, +3.7), and Visual Reasoning (+2.2, +4.1), which are more closely aligned with our emphasis on visual perception and object-level grounding. Although Chart & Diagram Understanding and OCR & Document Understanding are not the primary task categories targeted by our optimization design, PaIPO-D still achieves consistent improvements in both categories. These results demonstrate that PaIPO benefits diverse visual reasoning tasks, with more substantial improvements in tasks that closely align with the visual grounding objectives.

Table 3. Category-wise Performance Comparison on VRC-Bench.

Task CategoryDAPO Acc. ↑PaIPO-D Acc. ↑Δ Acc.DAPO Step ↑PaIPO-D Step ↑Δ Step
Visual Reasoning58.861.0+2.278.182.2+4.1
Math & Logic Reasoning60.562.2+1.780.183.4+3.3
Social & Cultural Context72.374.3+2.084.386.2+1.9
Medical Imaging68.071.1+3.183.887.5+3.7
Chart & Diagram Understanding70.773.2+2.586.188.6+2.5
OCR & Document Understanding77.879.9+2.189.489.9+0.5
Complex Visual Perception59.863.4+3.678.783.8+5.1
Scientific Reasoning66.668.7+2.184.688.7+4.1
Overall64.466.7+2.382.085.7+3.7

W2. The Counterfactual Visual Discrimination of CPR

PaIPO employs complementary reward signals to supervise different aspects of multi-stage visual reasoning. Specifically, CPR provides coarse-grained supervision for visual attribute sensitivity at the Caption stage, while RGR provides fine-grained object-level grounding supervision during Reasoning through localization accuracy, target coverage, and cross-step identifier consistency. Complementing these visual rewards, SR provides reference-based step-level supervision to encourage semantic correctness, step correspondence, and trajectory consistency. Together, these rewards connect visual perception, object-level grounding, and semantic reasoning within a unified optimization framework.

To validate the visual specificity of CPR, we conduct token-level counterfactual attribution analyses across three model scales under both original-image and edited-image conditioned sampling (Tables 4 and 5). Under original-image-conditioned sampling, condition-aligned Captions exhibit positive counterfactual likelihood differences (Dcf=0.32−0.35)(D_{\mathrm{cf}}=0.32-0.35), whereas Inconsistent Captions exhibit negative differences. Importantly, attribute-related tokens account for 76%–89% of the counterfactual evidence, while contributions from other tokens remain relatively small. The same pattern can be observed under edited-image-conditioned sampling. Edited-Aligned Captions receive positive likelihood support (Dcfrs=0.28−0.33)(D_{\mathrm{cf}}^{rs}=0.28-0.33), whereas Inconsistent Captions receive negative support, with attribute-related tokens accounting for 75%–85% of the counterfactual evidence. These results demonstrate that CPR captures image-dependent visual attributes, with the counterfactual discrimination signal primarily attributable to the targeted attributes rather than linguistic plausibility or an intrinsic preference for image condition.

AuthorsOpenReview ↗

Author Response to Reviewer t8ng (Part 3)

Table 4. Token-level attribution of counterfactual likelihood differences across model scales.

BackboneRollout GroupNDcfD_{\mathrm{cf}}CattrC_{\mathrm{attr}}CotherC_{\mathrm{other}}RattrR_{\mathrm{attr}}
2BAligned2,1230.32 ± 0.16+0.28 ± 0.14+0.04 ± 0.060.84 ± 0.12
2BOmitted1,0250.04 ± 0.09-+0.04 ± 0.09-
2BInconsistent373−0.16 ± 0.15−0.21 ± 0.14+0.05 ± 0.080.76 ± 0.13
2BAmbiguous479----
4BAligned2,3740.33 ± 0.15+0.29 ± 0.14+0.04 ± 0.060.88 ± 0.11
4BOmitted1,0160.03 ± 0.05-+0.03 ± 0.05-
4BInconsistent337−0.19 ± 0.14−0.23 ± 0.13+0.04 ± 0.070.79 ± 0.12
4BAmbiguous273----
8BAligned2,4930.35 ± 0.14+0.31 ± 0.13+0.04 ± 0.050.89 ± 0.10
8BOmitted1,0070.02 ± 0.05-+0.02 ± 0.05-
8BInconsistent245−0.21 ± 0.14−0.25 ± 0.12+0.04 ± 0.050.84 ± 0.11
8BAmbiguous255----

Table 5. Bidirectional counterfactual likelihood discrimination under edited-image-conditioned sampling.

BackboneRollout GroupNDcfrsD^{rs}_{\mathrm{cf}}CattrrsC^{rs}_{\mathrm{attr}}CotherrsC^{rs}_{\mathrm{other}}RattrrsR^{rs}_{\mathrm{attr}}
2BEdited-Aligned2,0940.28 ± 0.13+0.22 ± 0.14+0.06 ± 0.060.75 ± 0.13
2BOmitted9730.06 ± 0.07-+0.06 ± 0.07-
2BInconsistent547−0.19 ± 0.09−0.23 ± 0.13+0.04 ± 0.070.80 ± 0.14
2BAmbiguous386----
4BEdited-Aligned2,1470.33 ± 0.15+0.29 ± 0.14+0.04 ± 0.060.83 ± 0.15
4BOmitted1,0040.03 ± 0.05-+0.03 ± 0.05-
4BInconsistent514−0.19 ± 0.14−0.23 ± 0.13+0.04 ± 0.070.77 ± 0.19
4BAmbiguous335----
8BEdited-Aligned2,2530.32 ± 0.17+0.29 ± 0.13+0.03 ± 0.080.85 ± 0.14
8BOmitted1,0140.04 ± 0.05-+0.04 ± 0.05-
8BInconsistent493−0.24 ± 0.08−0.28 ± 0.14+0.04 ± 0.080.80 ± 0.13
8BAmbiguous240----

These analyses primarily establish the attribute sensitivity of Caption supervision. At the subsequent Reasoning stage, RGR complements this supervision by grounding object references through localization and cross-step identifier consistency, while SR further evaluates the semantic correctness and coherence of intermediate reasoning trajectories. This complementary design enables PaIPO to jointly supervise what the model perceives, how visual objects are grounded in its reasoning, and how intermediate reasoning steps are semantically organized.

Reviewer_iP6jOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates visual faithfulness in multimodal chain-of-thought (MCoT) reasoning and proposes Perception-aware Intermediate Process Optimization (PaIPO) to improve the utilization of visual evidence during intermediate reasoning. The authors first introduce Visual Attention Mass (VAM) to analyze visual attention across four reasoning stages and identify two phenomena: Visual Attenuation, where visual attention decreases when transitioning from captioning to reasoning, and Perception Sensitivity, where stronger visual attention is positively associated with intermediate reasoning quality.

Motivated by these observations, the authors construct MCoT-ViG, a dataset of 10,240 structured reasoning examples augmented with controlled visual variants and object-level grounding annotations. PaIPO then introduces three complementary rewards: Caption Perception Reward (CPR), which combines semantic consistency with counterfactual visual sensitivity; Reasoning Grounding Reward (RGR), which evaluates spatial grounding and object-reference consistency; and Step Reward (SR), which evaluates semantic and sequential reasoning quality. These rewards are integrated into GRPO and DAPO for policy optimization.

Experiments on Qwen3-VL models at 2B, 4B, and 8B scales demonstrate consistent improvements across multimodal reasoning and visual faithfulness benchmarks. Additional ablations and attention analyses further support the effectiveness of the proposed framework.

Critical strengths

S1. The paper addresses an important and practically relevant limitation of multimodal reasoning.

The observation that accurate final answers do not necessarily imply visually faithful intermediate reasoning is well motivated. The stage-wise analysis of visual attention provides an intuitive perspective on how visual information is utilized throughout MCoT generation. The proposed Seeing while Reasoning principle highlights a meaningful direction for improving visual grounding beyond the initial perception stage.

S2. The proposed dataset provides structured supervision for both perception and reasoning.

MCoT-ViG integrates four-stage rationales, controlled visual variants, and object-level grounding annotations into a unified training framework. The construction pipeline includes multiple quality-verification procedures, and the authors additionally report human–verifier agreement. This dataset design supports complementary supervision signals for caption perception and reasoning grounding, providing value beyond conventional answer-level supervision.

S3. The experimental evaluation demonstrates consistent improvements and meaningful component contributions.

The experiments cover three model scales, two reinforcement learning optimizers, and five benchmarks. The proposed method consistently improves visual faithfulness and reasoning accuracy over the considered baselines. The ablation studies further demonstrate the complementary contributions of CPR and RGR, while the post-training attention analyses provide additional evidence of changes in visual information utilization.

Critical weaknesses

W1. The methodological novelty is relatively limited, and the connection between the motivation and the proposed optimization is indirect.

The proposed framework primarily combines existing techniques, including reference-based process supervision, counterfactual image comparisons, IoU-based grounding rewards, and group-relative reinforcement learning. Although the integration is reasonable, the individual components are not fundamentally new. More importantly, the paper motivates PaIPO through Visual Attenuation, but the proposed rewards do not directly optimize attention allocation or explicitly require the model to revisit visual evidence during reasoning. Therefore, the connection between the observed attention phenomenon and the actual optimization mechanism remains somewhat indirect.

W2. The counterfactual visual supervision may not adequately capture the intended reasoning faithfulness.

The controlled visual variants are constructed by modifying answer-irrelevant attributes while preserving the original answer. However, this design raises questions about whether sensitivity to such changes is an appropriate proxy for visually faithful reasoning. A model may correctly ignore irrelevant visual modifications while still performing faithful reasoning. Conversely, a model may become sensitive to visually distinctive but task-irrelevant features without improving its reliance on evidence necessary for the answer. The paper does not sufficiently disentangle counterfactual sensitivity from task-relevant visual grounding.

W3. The evaluation does not fully establish generalization beyond the prescribed reasoning format and supervised object annotations.

All training examples follow a fixed four-stage reasoning structure, and RGR relies on explicit object-level bounding-box annotations. This setting may encourage models to learn a specific reasoning format and grounding representation rather than general-purpose visual reasoning capabilities. Moreover, the experiments are limited to Qwen3-VL models, and the main training data contain relatively short reasoning trajectories. It remains unclear whether the proposed method generalizes to more flexible reasoning formats, longer reasoning chains, and tasks without predefined grounding annotations.

Questions for authors

Q1. Why are answer-preserving visual modifications preferred over answer-changing counterfactuals for CPR?

The current CPR primarily measures the difference in caption likelihood between the original image and a variant with modified answer-irrelevant attributes. Would counterfactual transformations targeting answer-relevant visual evidence provide a stronger supervision signal? An ablation comparing answer-preserving and answer-changing transformations would clarify whether the improvements come from learning general visual sensitivity or from attending to genuinely task-relevant evidence.

Q2. How does PaIPO perform when explicit grounding annotations or the predefined four-stage reasoning format are unavailable?

RGR relies on annotated bounding boxes, while SR depends on reference reasoning trajectories. Could the authors investigate weaker supervision settings, such as automatically predicted grounding regions, incomplete annotations, or unstructured reasoning outputs? These experiments would help clarify the practicality and scalability of PaIPO beyond the current controlled training setting.

6 comments
AuthorsOpenReview ↗

Author Response to Reviewer iP6j (Part 1)

We sincerely thank you for the careful evaluation of our manuscript and constructive comments. In response to your suggestions, we provide detailed responses to each of your concerns below.

W1. Motivation and Complementary Reward Design of PaIPO

Our work starts from an empirical observation that, as multi-stage chain-of-thought reasoning progresses, models allocate progressively less direct attention to the original visual input. To systematically characterize this phenomenon, we introduce VAM and identify two consistent patterns across Qwen3-VL models. Visual Attenuation shows that direct visual attention is concentrated in the Caption stage and decreases substantially in subsequent Reasoning and Conclusion stages. Perception Sensitivity reveals a positive correlation between stage-level VAM and intermediate reasoning quality. In particular, Reasoning-stage VAM exhibits a stronger correlation with the VRC-Bench Step Score than Caption-stage VAM across all three model scales. These findings suggest that maintaining attention to visual evidence is associated with better intermediate reasoning quality, although the observed relationships are correlational rather than causal.

We consider this attention pattern potentially problematic for multi-stage reasoning. Since subsequent reasoning steps build upon previously generated textual representations, inaccurate or incomplete visual descriptions produced in the Caption stage may propagate as unsupported premises throughout the subsequent reasoning trajectory. Meanwhile, the diminishing direct attention to the original visual input may weaken the visual grounding of subsequent reasoning steps, making them more susceptible to inheriting inaccurate or incomplete visual descriptions from earlier stages. This motivates us to investigate how visual supervision can be maintained across different stages of the reasoning process.

To address this issue, we propose PaIPO, which extends existing policy optimization strategies with two complementary visual process rewards. Caption Perception Reward (CPR) provides coarse-grained visual supervision at the Caption stage by combining semantic reference alignment with counterfactual visual discrimination, encouraging descriptions to remain faithful and sensitive to the original visual evidence. Reasoning Grounding Reward (RGR) provides fine-grained, object-level supervision at the Reasoning stage through localization-weighted object matching and cross-step identifier consistency, encouraging subsequent reasoning to maintain explicit grounding in visual entities. Together, CPR and RGR encourage the model to preserve visual grounding from initial perception to intermediate reasoning, which motivates our `Seeing while Reasoning' principle.

We validate the rationale and effectiveness of this design through several complementary experiments. First, reward ablations demonstrate the complementary contributions of CPR and RGR: their combination, even without SR, improves the VRC-Bench Step Score by 5.02 and 3.85 and VFaith-Bench Perception by 4.72 and 4.87 on the 2B and 4B models, respectively. Second, post-training VAM analysis (Appendix D.3, Figure 8) shows that PaIPO maintains higher direct visual attention during both Caption and Reasoning and reduces the attention attenuation between these stages, consistent with our design objective. Third, the main benchmark results further demonstrate improvements in intermediate reasoning quality and visual faithfulness across different model scales. Finally, we also evaluate the improvements in caption perception and reasoning grounding in Appendices D.4 and D.5, respectively.

AuthorsOpenReview ↗

Author Response to Reviewer iP6j (Part 2)

W2, Q1. Answer-preserving Counterfactual Edits and Counterfactual Discrimination of CPR

Our answer-preserving counterfactual edits are designed to isolate attribute-level visual sensitivity. Specifically, we modify only attributes that are neither explicitly queried nor required to derive the reference answer, while preserving object identities, spatial relations, and other visual content. This controlled design allows CPR to evaluate whether generated Captions are sensitive to localized visual changes without introducing conflicting answer supervision.

To validate this mechanism, we conducted two additional analyses in Tables 1 and 2. Under both original-image and edited-image conditioned sampling, the direction of counterfactual likelihood discrimination consistently follows the image condition. Specifically, captions describing attributes consistent with the conditioning image receive greater likelihood support from that image, whereas captions describing the alternative attribute values are more strongly supported by the corresponding alternative image. Moreover, edited-attribute related tokens account for the majority of the counterfactual evidence, with RattrR_{\mathrm{attr}} ranging from 0.75 to 0.89. These findings demonstrate that the discrimination signal primarily captures sensitivity to the targeted visual attributes rather than an intrinsic preference for either image condition.

Table 1. Token-level attribution of counterfactual likelihood differences across model scales.

BackboneRollout GroupNDcfD_{\mathrm{cf}}CattrC_{\mathrm{attr}}CotherC_{\mathrm{other}}RattrR_{\mathrm{attr}}
2BAligned2,1230.32 ± 0.16+0.28 ± 0.14+0.04 ± 0.060.84 ± 0.12
2BOmitted1,0250.04 ± 0.09-+0.04 ± 0.09-
2BInconsistent373−0.16 ± 0.15−0.21 ± 0.14+0.05 ± 0.080.76 ± 0.13
2BAmbiguous479----
4BAligned2,3740.33 ± 0.15+0.29 ± 0.14+0.04 ± 0.060.88 ± 0.11
4BOmitted1,0160.03 ± 0.05-+0.03 ± 0.05-
4BInconsistent337−0.19 ± 0.14−0.23 ± 0.13+0.04 ± 0.070.79 ± 0.12
4BAmbiguous273----
8BAligned2,4930.35 ± 0.14+0.31 ± 0.13+0.04 ± 0.050.89 ± 0.10
8BOmitted1,0070.02 ± 0.05-+0.02 ± 0.05-
8BInconsistent245−0.21 ± 0.14−0.25 ± 0.12+0.04 ± 0.050.84 ± 0.11
8BAmbiguous255----

Table 2. Bidirectional counterfactual likelihood discrimination under edited-image-conditioned sampling.

BackboneRollout GroupNDcfrsD^{rs}_{\mathrm{cf}}CattrrsC^{rs}_{\mathrm{attr}}CotherrsC^{rs}_{\mathrm{other}}RattrrsR^{rs}_{\mathrm{attr}}
2BEdited-Aligned2,0940.28 ± 0.13+0.22 ± 0.14+0.06 ± 0.060.75 ± 0.13
2BOmitted9730.06 ± 0.07-+0.06 ± 0.07-
2BInconsistent547−0.19 ± 0.09−0.23 ± 0.13+0.04 ± 0.070.80 ± 0.14
2BAmbiguous386----
4BEdited-Aligned2,1470.33 ± 0.15+0.29 ± 0.14+0.04 ± 0.060.83 ± 0.15
4BOmitted1,0040.03 ± 0.05-+0.03 ± 0.05-
4BInconsistent514−0.19 ± 0.14−0.23 ± 0.13+0.04 ± 0.070.77 ± 0.19
4BAmbiguous335----
8BEdited-Aligned2,2530.32 ± 0.17+0.29 ± 0.13+0.03 ± 0.080.85 ± 0.14
8BOmitted1,0140.04 ± 0.05-+0.04 ± 0.05-
8BInconsistent493−0.24 ± 0.08−0.28 ± 0.14+0.04 ± 0.080.80 ± 0.13
8BAmbiguous240----

We also evaluate adaptation to answer-changing visual interventions using VFaith-Bench, which is already included in the submission, where the reference answers are updated according to the edited images. PaIPO improves Edit Accuracy by 1.60%–1.96% over matched PAPO variants across all three model scales. It also improves Difference Reasoning Faithfulness on SPD-Faith by 3.68–4.41 over the original backbones. These results provide complementary explanation that PaIPO improves reasoning under modified visual evidence. We acknowledge that directly incorporating answer-changing counterfactuals into training is a promising direction, but would require additional edited-image-conditioned rollouts and corresponding answer and intermediate-reasoning supervision, which we leave for future work.

AuthorsOpenReview ↗

Author Response to Reviewer iP6j (Part 3)

W3, Q2. Generalization under Weaker Supervision and Alternative Reasoning Formats

PaIPO supports modular reward configurations with different supervision requirements. Specifically, CPR and SR provide visual perception and reasoning supervision, respectively, while RGR incorporates explicit object-level grounding annotations. As shown in Table 3, removing RGR while retaining CPR and SR improves VRC-Bench Step Score by 3.88 and 2.34, and VFaith-Bench Edit Accuracy by 2.91 and 2.54 on the 2B and 4B models, respectively. These results demonstrate that CPR and SR provide effective process supervision under a box-independent reward configuration.

Table 3. Performance without Explicit Object-level Grounding Reward.

BackboneConfigurationVRC Acc. ↑Step Score ↑Edit Acc. ↑Perception ↑
2BBaseline56.1669.1451.2214.30
2BSR + CPR58.3273.0254.1317.48
4BBaseline63.3781.1861.3726.40
4BSR + CPR65.6783.5263.9129.04

We have also investigated the adaptability of PaIPO to alternative reasoning formats and model architectures through preliminary experiments on Qwen3-VL-Thinking and LLaVA-NeXT-7B, as summarized in Table 4. For Qwen3-VL-Thinking, which generates free-form reasoning within <think> and final responses within <answer>, we extract the corresponding Caption and Reasoning content from <answer> to compute CPR and RGR. PaIPO consistently outperforms DAPO across all four metrics, improving Perception by 4.2 and 4.1 and Edit Accuracy by 3.0 and 2.3 on the 2B and 4B models, respectively. These experiments demonstrate the compatibility with alternative output protocols and long-form thinking generation. For LLaVA-NeXT-7B, we first perform supervised fine-tuning (SFT) to establish reliable instruction following and structured reasoning generation, followed by reinforcement learning. PaIPO further improves over DAPO by 9.6 in VRC-Bench Step Score, 2.9 in Edit Accuracy, and 3.4 in Perception, demonstrating the effectiveness across different model family.

Table 4. Generalization to Unstructured Chain-of-Thought Reasoning and Diverse Model Family.

Model/MethodVRC Acc. ↑Step Score ↑Edit Acc. ↑Perception ↑
Qwen3-VL-2B-Thinking57.470.950.813.8
 + DAPO58.272.851.514.3
 + PaIPO59.874.554.518.5
Qwen3-VL-4B-Thinking64.581.561.225.5
 + DAPO66.282.762.426.3
 + PaIPO68.084.564.730.4
LLaVA-NeXT-7B36.235.831.010.5
+ SFT41.545.332.412.5
 + DAPO44.551.832.612.8
 + PaIPO45.261.435.516.2

For Thinking models, the reward extraction was restricted to the <answer> segment, leaving the free-form <think> process without direct stage-specific supervision. Consequently, these experiments do not fully reflect the central objective of this paper of maintaining visual attention throughout the reasoning process. For LLaVA-NeXT, an additional SFT initialization was necessary to establish reliable instruction following and reward computation, resulting in a model-specific training pipeline that differs from the unified setting adopted in our main experiments. We therefore present these results as supplementary exploratory results of PaIPO's adaptability rather than as part of the primary controlled evaluation.

Regarding weaker supervision, recent vision models have demonstrated strong segmentation capabilities, offering the possibility of automatically generating grounding regions and deriving bounding boxes from predicted masks. A key challenge is to integrate these capabilities with multimodal CoT models and develop robust reward mechanisms that accommodate incomplete grounding annotations and reference-free reasoning trajectories, which would extend PaIPO to more flexible supervision settings. Our Thinking-model experiments provide an initial step toward this direction, while directly optimizing the entire free-form thinking process remains an important avenue for future investigation.

We hope that our responses and additional experimental results adequately address your concerns. Thank you again for your careful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.