- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies visual faithfulness during intermediate reasoning in small multimodal language models. It introduces Visual Attention Mass (VAM), which measures the average direct attention allocated by generated text tokens to visual tokens. Analyses of Qwen3-VL models identify a reduction in direct visual attention after caption generation and modest positive correlations between VAM and intermediate reasoning quality.
Motivated by these observations, the authors construct MCoT-ViG, using Visual-CoT source records to generate four-stage rationales—Summary, Caption, Reasoning, and Conclusion—with object-grounding annotations and edited image variants. They propose Perception-aware Intermediate Process Optimization (PaIPO), combining:
- Caption Perception Reward (CPR): reference-caption semantic alignment and original-versus-edited-image likelihood discrimination;
- Reasoning Grounding Reward (RGR): bounding-box localization and object-identifier consistency;
- Step Reward (SR): reference-based reasoning alignment, ordering, and terminal entailment.
These rewards supplement answer and format rewards under GRPO or DAPO. Experiments cover five benchmarks and Qwen3-VL models at 2B, 4B, and 8B. PaIPO consistently improves reasoning and visual-faithfulness metrics over the backbones, step-reward baselines, and matched PAPO reimplementations. Reward ablations support the usefulness of combining CPR and RGR.
Critical strengths
- Controlled comparisons support the usefulness of the proposed perception rewards.
The paper compares methods under reportedly matched data and training settings, across three model scales and two optimizers. Table 2 further holds SR constant while introducing CPR or RGR, with both additions improving reasoning and faithfulness measures. This is consequential because it provides evidence that the proposed supervision contributes beyond ordinary step-based RL.
- The evaluation connects intermediate-process improvements to independent behavioral outcomes.
Improvements are not limited to VAM or training rewards: PaIPO also improves answer accuracy, edited-image accuracy, and SPD-Faith scores. The combination of process-level and behavioral evaluations strengthens the argument that the method has practical value rather than merely optimizing an attention diagnostic.
Critical weaknesses
- The validity of counterfactual likelihood discrimination is insufficiently established in the main text.
CPR assumes that higher caption likelihood under the original image indicates genuine visual support. However, original-conditioned sampling already favors the original image, and edited images may introduce unintended distribution shifts. Moreover, the aggregation emphasizes tokens with large likelihood differences without explicitly restricting them to the modified attributes. If these differences are driven by sampling bias or editing artifacts, CPR could reward image discrimination rather than faithful perception.
- The proposed diagnostics and grounding reward do not establish sustained visual use during reasoning.
VAM captures direct attention rather than the total influence of visual evidence, while RGR measures box accuracy and identifier consistency. Neither establishes that later reasoning correctly uses the localized objects or their attributes and relations. The paper appropriately acknowledges the correlational nature of VAM, but the “Seeing while Reasoning” interpretation remains stronger than the evidence currently supports.
Questions for authors
- How is the CPR indifference margin calibrated, and what evidence shows that it removes original-conditioned sampling bias?
Please specify whether calibration uses held-out data and whether the margin varies across model scales or training checkpoints. Does the reward remain discriminative when captions are sampled under the edited image rather than the original image?
6 comments
Author Response to Reviewer xSRa (Part 2)
2. How is the indifference margin calibrated against original-conditioned sampling bias?
Sampling Captions under the original image may introduce an inherent likelihood preference toward the original visual condition. To account for this effect, we calibrate the counterfactual indifference margin before policy optimization using an independent set of 1,000 held-out samples and the pretrained Qwen3-VL-4B-Instruct model.
Specifically, for each held-out sample, the frozen model generates a Caption conditioned on the original image. We then retain the identical generated token sequence and evaluate its conditional likelihood under both the original and controlled images using teacher forcing. We calculate the token-level likelihood differences and aggregate them into following Eqs. (22)–(23), using the same counterfactual temperature and confidence exponent as in our reward formulation. Based on the empirical distribution of across these 1,000 held-out samples, we set as a fixed indifference margin.
Although this calibration is performed exclusively with the pretrained 4B model before reinforcement learning. We analyze the post-training rollout Captions across the three model scales to examine the applicability of this margin. As shown in Table 1, Aligned Captions exhibit substantially larger positive counterfactual likelihood differences (0.32–0.35), whereas Omitted Captions exhibit much smaller mean differences (0.02–0.04). As supplementary material, we refer to the counterfactual likelihood dynamics reported in Appendix D.2 (Figure 7). Under GRPO, PaIPO-G develops a larger counterfactual NLL gap than PAPO-G during the later stages of training. Under DAPO, PaIPO-D establishes visual discrimination earlier than PAPO-D, although their gaps become comparable toward the end of training. These results indicate that the indifference margin does not prevent the policy from progressively strengthening its dependence on the original visual evidence during optimization.
Author Response to Reviewer xSRa (Part 3)
3. Does counterfactual discrimination remain effective when captions are sampled under edited images?
To examine whether counterfactual discrimination depends on original-image-conditioned sampling, we conduct an offline evaluation using the trained PaIPO checkpoints. Specifically, for each held-out image pair, we independently sample Captions conditioned on the edited image, and then evaluate the conditional likelihood under both the edited and original images using the same frozen policy checkpoint.
For edited-image-conditioned Captions, we reverse the likelihood comparison in Eq. (22): $ \log\pi_{\bar\theta}(c_{i,t}^{rs}\mid q_i,I_i^{rs},c_{i,<t}^{rs}) - \log\pi_{\bar\theta}(c_{i,t}^{rs}\mid q_i,I_i,c_{i,<t}^{rs}). $ We categorize the generated Captions into three groups: (1) Edited-aligned, where the Caption correctly describes the edited attribute value; (2) Omitted, where the edited attribute is not mentioned; and (3) Inconsistent, where the Caption describes the inconsistent attribute value rather than the edited value. Ambiguous cases are excluded from the primary attribution analysis. We perform the same token-level attribution analysis to examine whether the edited-attribute tokens remain the primary source of counterfactual likelihood differences.
Table 2. Bidirectional counterfactual likelihood discrimination under edited-image-conditioned sampling.
| Backbone | Rollout Group | N | ||||
|---|---|---|---|---|---|---|
| 2B | Edited-Aligned | 2,094 | 0.28 ± 0.13 | +0.22 ± 0.14 | +0.06 ± 0.06 | 0.75 ± 0.13 |
| 2B | Omitted | 973 | 0.06 ± 0.07 | - | +0.06 ± 0.07 | - |
| 2B | Inconsistent | 547 | −0.19 ± 0.09 | −0.23 ± 0.13 | +0.04 ± 0.07 | 0.80 ± 0.14 |
| 2B | Ambiguous | 386 | - | - | - | - |
| 4B | Edited-Aligned | 2,147 | 0.33 ± 0.15 | +0.29 ± 0.14 | +0.04 ± 0.06 | 0.83 ± 0.15 |
| 4B | Omitted | 1,004 | 0.03 ± 0.05 | - | +0.03 ± 0.05 | - |
| 4B | Inconsistent | 514 | −0.19 ± 0.14 | −0.23 ± 0.13 | +0.04 ± 0.07 | 0.77 ± 0.19 |
| 4B | Ambiguous | 335 | - | - | - | - |
| 8B | Edited-Aligned | 2,253 | 0.32 ± 0.17 | +0.29 ± 0.13 | +0.03 ± 0.08 | 0.85 ± 0.14 |
| 8B | Omitted | 1,014 | 0.04 ± 0.05 | - | +0.04 ± 0.05 | - |
| 8B | Inconsistent | 493 | −0.24 ± 0.08 | −0.28 ± 0.14 | +0.04 ± 0.08 | 0.80 ± 0.13 |
| 8B | Ambiguous | 240 | - | - | - | - |
These results indicate that the direction of counterfactual likelihood discrimination changes consistently with the sampling image condition. Captions describing the edited attributes receive greater likelihood support from the edited images, whereas Captions describing inconsistent attribute values exhibit greater likelihood support from the original images. These findings demonstrate that the proposed counterfactual discrimination mechanism generalizes effectively across different image-conditioned sampling settings.
Our method does not incorporate edited-image conditioned rollouts into policy optimization for two considerations. First, introducing edited-image-conditioned rollouts would approximately double the rollout generation and associated likelihood evaluation costs, substantially increasing the training time. Second, the current reward formulation is anchored to the original task input and its corresponding reference Caption. Incorporating edited-image-conditioned rollouts would require additional edit-specific reference supervision and corresponding modifications to the reference-alignment and counterfactual reward components. Nevertheless, we consider extending the training framework to incorporate edited-image-conditioned rollouts a promising direction for further investigation, which we leave for future work.
Author Response to Reviewer xSRa (Part 1)
We sincerely thank you for the positive assessment and the constructive comments on the reliability of counterfactual visual discrimination and the interpretation of our visual-grounding diagnostics. We address the two concerns below.
W1, Q1. Reliability and Calibration of Counterfactual Visual Discrimination
We address this concern through three complementary analyses, focusing on the specificity of counterfactual likelihood differences, the calibration of the indifference margin, and the validity of bidirectional counterfactual discrimination.
1. Are counterfactual likelihood differences primarily attributable to the edited attributes?
We conduct a token-level attribution analysis on sampled rollout Captions. For each sample, we align the recorded attribute transformations with the corresponding descriptions in the generated Caption and divide the tokens into two groups: edited-attribute-related tokens (Tiattr) and other Caption tokens (Tiother). Following Eq. (22), we calculate the token-level counterfactual evidence ei,t. We then decompose the aggregated score in Eq. (23) into the contributions of the two token groups: $ C_i^{\mathrm{attr}} = \frac{ \sum_{t\in T_i^{\mathrm{attr}}} |e_{i,t}|^\gamma e_{i,t} }{ \sum_{t\in T_i^{\mathrm{cap}}} |e_{i,t}|^\gamma+\epsilon }, $ $ C_i^{\mathrm{other}} = \frac{ \sum_{t\in T_i^{\mathrm{other}}} |e_{i,t}|^\gamma e_{i,t} }{ \sum_{t\in T_i^{\mathrm{cap}}} |e_{i,t}|^\gamma+\epsilon }. $ Accordingly, Dicf=Ciattr+Ciother. We further quantified the proportion of absolute counterfactual evidence contributed by the edited-attribute tokens: $ R_i^{\mathrm{attr}}= \frac{ \sum_{t\in T_i^{\mathrm{attr}}} |e_{i,t}|^{\gamma+1} }{ \sum_{t\in T_i^{\mathrm{cap}}} |e_{i,t}|^{\gamma+1}+\epsilon }. $ We categorize the sampled Captions into three groups: (1) Aligned, where the Caption correctly describes the original value of the edited attribute; (2) Omitted, where the edited attribute is not mentioned; and (3) Inconsistent, where the Caption mentions the edited attribute but inconsistent with the original image.
Table 1. Token-level attribution of counterfactual likelihood differences across model scales.
The results show consistent attribution patterns across all three model scales. For Aligned Captions, edited-attribute tokens account for 84%–89% of the absolute counterfactual evidence. For Omitted Captions, the mean Dcf remains relatively small, suggesting limited counterfactual evidence from unrelated caption content. For Inconsistent Captions, edited-attribute tokens exhibit negative contributions, accounting for 76%–84% of the absolute evidence, indicating that these tokens are more strongly supported by the edited images. These findings demonstrate that counterfactual likelihood differences are predominantly driven by edited-attribute tokens with consistent patterns across model scales.