acceptodds
Under review as a conference paper at ICLR 2027

VAttnPO: Visual Attention-Guided Policy Optimization

Abstract

Multimodal reinforcement learning with verifiable rewards commonly assigns one sequence advantage to an entire response. This outcome-level signal distinguishes overall response quality but leaves changes in visual-evidence use across generated tokens implicit. We introduce VAttnPO (Visual Attention-Guided Policy Optimization), which uses answer-conditioned visual attention to refine token-level credit while preserving the verifier-derived trajectory-mean advantage. A frozen old policy constructs a visual reference by teacher forcing the known correct answer and produces a visual distribution for each rollout token. VAttnPO measures their agreement in visual-attention space using Jensen–Shannon divergence, rather than reading guidance from teacher output-token distributions or introducing an attention-matching objective. Trajectory centering, relative-position weighting, and credit balancing turn this agreement into a zero-mean, position-aware correction to the GRPO sequence advantage. Token intervention, visual-index permutation, and trajectory analysis provide diagnostic evidence linking this signal to model behavior, visual correspondence, and generation position. On Qwen3-VL-8B-Instruct, VAttnPO raises six-task image AVE from 70.97 with GRPO to 73.50 and scores higher on every reported structured, video, and MME metric. Gains over GRPO also hold on Qwen2.5-VL-7B-Instruct and InternVL3-8B-Instruct.

Primary area:
reinforcement learning
Keywords:
reinforcement learning with verifiable rewards, token-level credit assignment, visual attention

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MWQBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VAttnPO refines GRPO's shared sequence advantage using token-level visual attention. Within each update, the frozen old policy teacher-forces the correct answer to construct a visual-patch reference and extracts attention from sampled responses. Jensen–Shannon similarity to the reference is centered within each response, weighted by relative position, and balanced into a zero-mean correction to the sequence advantage. The authors report that the six-image-task average on Qwen3-VL-8B improves from 70.97 with GRPO to 73.50. Behavioral diagnostics and component controls examine the visual reference and credit construction, with further evaluations on document, reliability, video, and cross-backbone tasks.

Critical strengths
  1. The method provides a precisely constrained way to use visual correspondence for token credit. Centering and balancing preserve the trajectory-mean advantage while distributing credit according to the answer-conditioned reference. The wrong-answer and shuffled-reference controls in Table 4 support the importance of the reference's content and spatial alignment.

  2. The main experiments demonstrate broad task-level improvements under shared training settings. All six image tasks improve over GRPO, with a 2.53-point average gain and a 2.23-point margin over OPD, the strongest compared baseline by average score (Table 1). Shared training data and optimization budgets support this empirical comparison.

Critical weaknesses
  1. The empirical benefit over existing visual-credit methods remains unclear. The baselines cover generic RLVR and distillation but omit VPPO [1] and AT-RL [2], which also use token-level visual signals to refine multimodal policy updates. Neither is discussed in the related work. This leaves VAttnPO's additional empirical benefit over this directly relevant class of methods untested.

  2. The rationale for monotonically increasing position weights remains incomplete. Figure 1(c) reports a non-monotonic correct–incorrect alignment gap, largest at the terminal token, while Section 4.4 uses an increasing sigmoid. The no-position and position-shuffled controls support the usefulness of position modulation, but do not explain why later positions should systematically receive larger weights. It remains unclear how the diagnostic supports this particular position prior beyond the terminal-token effect.

[1] Spotlight on Token Perception for Multimodal Reinforcement Learning.

[2] Credit Where It's Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning.

Questions for authors
  1. How does VAttnPO's answer-conditioned, zero-mean credit correction differ from VPPO and AT-RL? Is a comparison with either method available under the same training data and rollout budget?

  2. Why use monotonically increasing position weights when the diagnostic alignment gap is non-monotonic? After excluding the terminal token, do the existing analyses still support stronger modulation of later positions?

I would consider raising my rating if these concerns are satisfactorily addressed.

1 comment

W1 & Q1: Comparisons with existing visual-credit methods.

We agree that VPPO and AT-RL are previous studies directly comparable to this one. We have also included a comparison with VGPO. VPPO uses token visual dependency for trajectory-level advantage shaping and token-level gradient filtering. AT-RL applies cluster-based soft weighting based on cross-modal connectivity. VGPO combines visual attention compensation with dual-grained advantage re-weighting to counter temporal visual forgetting. In contrast, VAttnPO uses agreement with an answer-conditioned visual reference and relative-position weighting to construct a zero-mean additive correction.

MethodVisual signalCredit assignmentCore motivation
VPPOVisual dependency measured through image perturbationScale trajectory advantages and select tokensPrioritize visually dependent tokens and trajectories
AT-RLCross-modal attention connectivitySoft-weight token clustersReinforce perceptual anchors while retaining other tokens
VGPOHidden-state similarity to a visual prototypeReweight tokens and trajectories with late-stage compensationStrengthen visual focus and mitigate temporal visual forgetting
VAttnPO (Ours)Agreement with an answer-conditioned visual referenceApply relative-position weighting and construct a zero-mean additive correctionRefine within-response credit while preserving the trajectory-mean advantage

We have supplemented our evaluation with VPPO, VGPO, and AT-RL. All four methods, including VAttnPO, are implemented within the GRPO framework using the same backbone Qwen3−VL−8B−InstructQwen3-VL-8B-Instruct, training data, and rollout budget.

MethodVStarHRBench 4KMMVPMMMU-ProMuirBenchBLINKAVE
VPPO83.7778.7581.0054.6264.1568.3371.77
VGPO83.7778.8881.3354.9764.5468.6472.02
AT-RL84.2979.0081.6754.9164.9668.5472.23
VAttnPO (Ours)87.4380.6282.3355.9565.6269.0773.50
MethodChartQAOCRBenchDocVQAInfoVQAPOPEHallusionBenchCountBenchQA
VPPO83.1289.5095.4983.3588.2471.8289.94
VGPO83.1689.4095.5283.0888.2272.4590.35
AT-RL83.2089.6095.6283.0988.2672.6690.14
VAttnPO (Ours)83.8490.1095.59 pd83.6488.6773.8291.17
MethodMVBenchLongVideoBenchTempCompassVideo-MMEMME-perMME-rea
VPPO71.3263.270.1068.51725.1672.9
VGPO71.4863.470.2068.61735.5661.8
AT-RL71.5363.670.4968.61734.0673.6
VAttnPO (Ours)72.1563.671.5868.61755.0683.9

VAttnPO achieves an AVE of 73.50, exceeding VPPO, VGPO, and AT-RL by 1.73, 1.48, and 1.27 points, respectively, with improvements on all six image tasks. Across the complete evaluation, VAttnPO scores higher than VPPO on all 19 individual metrics. At the reported precision, it exceeds VGPO on 18 metrics and matches it on Video-MME. It also exceeds AT-RL on 16 metrics, matches it on LongVideoBench and Video-MME, and scores 0.03 points lower on DocVQA. These comparisons further support the effectiveness of our visual attention-guided credit assignment.

W2 & Q2: Rationale for relative-position weighting.

The non-monotonic alignment gap in Figure 1cc is consistent with our motivation for relative-position weighting. Early CoT tokens may attend to different image regions, and fluctuations in their agreement with the answer-conditioned visual reference are therefore expected. As reasoning progresses, we observe that correct responses tend to become more aligned with this reference. This motivates weaker guidance early and stronger guidance in the latter half of the response.

To examine whether the gains persist when visual credit is confined to reasoning tokens, we additionally evaluate an Only CoT variant. It retains relative-position weighting but applies visual credit correction only within CoT.

Credit AssignmentVStarHRBench 4KMMVPMMMU-ProMuirBenchBLINKAVE
Baseline83.2577.5081.0054.2861.5468.2370.97
Only CoT86.3980.5082.0055.0365.2068.9673.01 +20˙4+2\.04
Full Completion (Ours)87.4380.6282.3355.9565.6269.0773.50 +25˙3+2\.53

Only CoT improves over GRPO by 2.04 points, showing that position-weighted visual credit remains effective when tokens outside CoT receive no visual credit correction. Extending credit assignment to the full completion provides a further 0.49-point improvement. These results establish that the gains persist within the reasoning process itself and are not confined to contributions from the terminal token.

Reviewer_dat1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

VAttnPO uses correct-answer-conditioned visual attention to refine token-level advantages in multimodal RL. Attention similarity is centered within each trajectory, weighted by position, and balanced into a zero-sum correction to the GRPO advantage. The paper reports improvements across multiple benchmarks and three backbones.

Critical strengths
  • Using internal visual statistics for fine-grained credit assignment is a promising idea, without requiring region annotations or an additional reward model.

  • The algorithm is clearly specified, including its mean-preservation property and numerical handling.

  • The experiments cover diverse tasks and include several useful component and correspondence controls.

Critical weaknesses
  • The mapping from attention similarity to token credit relies on hand-designed choices whose robustness is insufficiently established. Tokens serving different reasoning functions need not attend to the same regions as a short correct-answer reference. Thus, greater reference similarity does not automatically imply greater positive credit. The method additionally depends on layer selection, attention pooling, similarity construction, trajectory centering, position weighting, and balancing. Several choices have reasonable mathematical motivations, but the current ablations mainly support the selected configuration; they do not establish robustness across plausible alternatives. This is my central concern, rather than evidence that extensive manual tuning was necessarily performed.

  • The diagnostics only indirectly support the actual training signal.The intervention study uses a different attention statistic and query index from training, while masking complete KV columns measures influence on the original prediction rather than a beneficial visual contribution. Likewise, trajectory-level correct–incorrect separation does not directly validate the within-trajectory differences retained after centering. Direct behavioral validation of the final correction would strengthen the credit-assignment interpretation.

  • The empirical advantage needs stronger verification and attribution.Main results use one training seed, and key configurations for the distillation baselines are insufficiently documented. Furthermore, comparisons with RLSD/RLCSD change both the signal source and its processing. A probability-based signal under the same centering, weighting, and balancing pipeline would help isolate the contribution of visual attention.

Questions for authors
  • How robust are the gains to reference-token selection, layer/head aggregation, position-function choice, and correction strength? Do they persist after excluding formatting and terminal tokens?

  • Can the final correction (d_{i,t}) be directly linked to beneficial behavioral effects, controlling for position and token type?

  • Can you provide multi-seed results, complete baseline configurations, and a matched comparison using probability-based credit signals?

5 comments

We thank the reviewer for the questions and suggestions.

W1 & Q1: Robustness to design choices.

We have conducted additional experiments on attention layers 12 and 28 and position functions x20x^{20} and x1/20x^{1/20}, with results reported below. These additional results support the intermediate-layer choice discussed in Appendix I.1: layer 20 achieves the highest AVE among the tested layers. Both alternative position functions retain gains over GRPO, while the default sigmoid performs best on all six benchmarks.

Attention LayerPosition WeightVStarHRBench 4KMMVPMMMU-ProMuirBenchBLINKAVE
Baseline-83.2577.5081.0054.2861.5468.2370.97
L20sigmoid87.4380.6282.3355.9565.6269.0773.50 +25˙3+2\.53
L12sigmoid81.1576.7579.6753.1263.8867.4970.34 −06˙3-0\.63
L28sigmoid83.7777.8881.0054.6864.1968.0271.59 +06˙2+0\.62
L20y=x20y=x^{20}85.8779.3881.6754.8665.1868.7072.61 +16˙4+1\.64
L20y=x1/20y=x^{1/20}84.2978.1580.6754.2264.8368.0271.70 +07˙3+0\.73

To examine whether the gains persist when visual credit is confined to reasoning tokens, we additionally evaluate an Only CoT variant. It retains relative-position weighting but applies visual credit correction only within CoT.

Credit AssignmentVStarHRBench 4KMMVPMMMU-ProMuirBenchBLINKAVE
Baseline83.2577.5081.0054.2861.5468.2370.97
Only CoT86.3980.5082.0055.0365.2068.9673.01 +20˙4+2\.04
Full Completion (Ours)87.4380.6282.3355.9565.6269.0773.50 +25˙3+2\.53

Only CoT improves over GRPO by 2.04 points, showing that position-weighted visual credit remains effective when tokens outside CoT receive no visual credit correction. Extending credit assignment to the full completion provides a further 0.49-point improvement. These results establish that the gains persist within the reasoning process itself and are not confined to contributions from the terminal token.

W2 & Q2: Direct behavioral validation

Using the same data as the pilot study, we compute the final di,td_{i,t} exactly as in training. Within each response, we match higher- and lower-credit tokens for position, lexical type, and total visual attention mass. Weakening visual access at higher-credit tokens reduces the correct answer’s log-probability margin over other options by 0.0097 nats. This effect exceeds that at matched lower-credit tokens by 0.0316 nats. The 95% confidence intervals for both the high-credit effect and the high–low difference lie entirely above zero. These results show that higher-credit tokens provide positive support for the correct answer and contribute more than matched lower-credit tokens, directly linking the final correction to beneficial visual contributions.

W3 & Q3: Reproducibility and Baseline Comparisons

Consistent improvements across training seeds. We additionally trained GRPO and VAttnPO with seed 43 under the same training configuration and evaluation protocol.

MethodSeedVStarHRBench 4KMMVPMMMU-ProMuirBenchBLINKAVE
GRPO42.0083.2577.5081.0054.2861.5468.2370.97
GRPO43.0083.2577.3880.6754.2261.5868.2870.90
VAttnPO (Ours)42.0087.4380.6282.3355.9565.6269.0773.50
VAttnPO (Ours)43.0087.4380.3882.0055.8365.6969.1773.42

VAttnPO improves over GRPO on all six benchmarks in both runs, with nearly identical aggregate gains. These results support the reproducibility of the reported improvement.

Explicit baseline configurations. GKD and OPD use a frozen Qwen3-VL-32B-Instruct teacher. OPSD, RLSD, and RLCSD use teachers based on the Qwen3-VL-8B-Instruct backbone. All methods use a batch size of 256 and 400 optimization steps, with remaining method-specific hyperparameters following their respective official defaults. We will explicitly document these settings in Appendix B.

Matched comparison with probability-based credit. We additionally compare probability-based and attention-based credit under the same centering, position weighting, and balancing. Attention-based credit outperforms probability-based credit on all six benchmarks, improving AVE by 1.80 points. This matched comparison supports the benefit of visual attention as a token-credit signal beyond the shared credit-processing procedure.

Credit TypeVStarHRBench 4KMMVPMMMU-ProMuirBenchBLINKAVE
Baseline83.2577.5081.0054.2861.5468.2370.97
Probability-based85.3477.8880.3354.9764.8166.8671.70 +07˙3+0\.73
Attention-baesd (Ours)87.4380.6282.3355.9565.6269.0773.50 +25˙3+2\.53

Thank you for the additional experiments. The second-seed results and the new controls strengthen the paper. However, my main concern remains unaddressed. The gains vary substantially with the choice of attention layer. For a trillion-parameter model with roughly 100 layers, selecting a layer through repeated RL runs could be costly. Can you recommend a practical layer-selection rule that does not rely on downstream test scores? Can the position-weighting schedule remain fixed across backbones, and how much additional training would you expect the selection process to require?

Thank you for your response and thoughtful feedback.

Inference-based layer selection.

Our probe (a layer-wise visual-attention diagnostic) selects the attention layer through frozen-model inference on a small diagnostic dataset. The previously reported L12 and L28 training runs were layer-sensitivity ablations, not part of the selection procedure. Following the pilot study setup in Section 3, we compare the visual attention of correct and incorrect responses against the answer-conditioned reference on training-distribution prompts. We select the layer with the largest wrong-minus-correct JSD gap, indicating the strongest separation between correct and incorrect responses in their visual-attention alignment. The experiments below show that probe scores broadly track downstream performance after RL with different attention layers across all three backbones. These results support using the probe to predict relative layer performance and guide selection on a new MLLM backbone, without training every candidate configuration.

Appendix I.1 reports frozen-model diagnostic scores for several layers of each backbone. We select five of these layers per backbone, train each configuration for 100 steps, and evaluate it on MuirBench and VStar. The tables below report their diagnostic scores alongside the corresponding training results. Bold marks the layer selected by the diagnostic.

(a) Qwen3-VL-8B-Instruct

LayerDepthΔℓlate\Delta^{\mathrm{late}}_{\ell}∗95% CI∗MuirBench†VStar†
L1234%3.6×10−33.6\times10^{-3}[7.0×10−4,6.6×10−3][7.0\times10^{-4},6.6\times10^{-3}]62.7781.15
L1646%1.0×10−21.0\times10^{-2}[6.3×10−3,1.4×10−2][6.3\times10^{-3},1.4\times10^{-2}]63.2782.72
L2057%1.1×10−2\mathbf{1.1\times10^{-2}}[6.5×10−3, 1.5×10−2]\mathbf{[6.5\times10^{-3},\,1.5\times10^{-2}]}63.5883.77
L2469%6.7×10−36.7\times10^{-3}[3.2×10−3,1.0×10−2][3.2\times10^{-3},1.0\times10^{-2}]63.3582.72
L2880%6.9×10−36.9\times10^{-3}[3.2×10−3,1.1×10−2][3.2\times10^{-3},1.1\times10^{-2}]63.2382.20

(b) Qwen2.5-VL-7B-Instruct

LayerDepthΔℓlate\Delta^{\mathrm{late}}_{\ell}∗95% CI∗MuirBench†VStar†
L415%3.5×10−43.5\times10^{-4}[−4.1×10−3,4.5×10−3][-4.1\times10^{-3},4.5\times10^{-3}]53.6573.82
L933%1.1×10−31.1\times10^{-3}[−4.1×10−3,5.6×10−3][-4.1\times10^{-3},5.6\times10^{-3}]53.9374.35
L1452%4.8×10−3\mathbf{4.8\times10^{-3}}[−1.5×10−3,1.1×10−2]\mathbf{[-1.5\times10^{-3},1.1\times10^{-2}]}54.2375.92
L1867%3.5×10−33.5\times10^{-3}[−8.2×10−4,7.3×10−3][-8.2\times10^{-4},7.3\times10^{-3}]54.1275.39
L2281%2.8×10−32.8\times10^{-3}[−3.3×10−3,8.9×10−3][-3.3\times10^{-3},8.9\times10^{-3}]54.0075.39

(c) InternVL3-8B-Instruct

LayerDepthΔℓlate\Delta^{\mathrm{late}}_{\ell}∗95% CI∗MuirBench†VStar†
L415%1.3×10−31.3\times10^{-3}[−1.1×10−3,3.3×10−3][-1.1\times10^{-3},3.3\times10^{-3}]55.8169.10
L933%2.8×10−32.8\times10^{-3}[−3.2×10−4,5.5×10−3][-3.2\times10^{-4},5.5\times10^{-3}]55.9669.63
L1452%4.3×10−3\mathbf{4.3\times10^{-3}}[−2.0×10−3,1.1×10−2]\mathbf{[-2.0\times10^{-3},1.1\times10^{-2}]}56.1570.68
L1867%4.3×10−34.3\times10^{-3}[1.6×10−4,8.7×10−3][1.6\times10^{-4},8.7\times10^{-3}]56.0870.68
L2281%3.9×10−33.9\times10^{-3}[−7.7×10−4,8.6×10−3][-7.7\times10^{-4},8.6\times10^{-3}]56.0470.16

∗ Frozen-model inference: Δℓlate\Delta^{\mathrm{late}}_{\ell} is the wrong-minus-correct JSD gap, with its 95% bootstrap confidence interval.

† Evaluation scores of models trained with the corresponding attention layer.

Across all three backbones, the layer selected by the diagnostic achieves the best or tied-best performance on both benchmarks among the tested layers. This agreement between diagnostic selection and post-training performance supports the effectiveness of inference-based layer selection.

Fixed position weighting schedule across backbones.

We use the same sigmoid position weighting schedule (PWS) as in the paper and keep it fixed across all three backbones. Each backbone uses the attention layer marked in bold above. We compare No PWS and Fixed PWS while keeping all other settings unchanged.

The table reports the average score (AVE) across the six image benchmarks, with GRPO as the baseline.

BackboneBaselineNo PWSFixed PWS
Qwen3-VL-8B-Instruct70.9772.3573.50
Qwen2.5-VL-7B-Instruct61.6462.2462.55
InternVL3-8B-Instruct60.4660.8861.03

With the same PWS, our method outperforms the GRPO baseline on all three backbones. The additional gains over No PWS on every backbone further support reusing this schedule without backbone-specific tuning.

In summary, these results support probe-based layer selection and keeping PWS fixed across backbones. Selecting a layer requires only frozen-model inference on a small diagnostic dataset, with no additional RL training. The layer-comparison training experiments validate the probe and are not part of the selection procedure.

Reviewer_aUL1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes VAttnPO (Visual Attention-Guided Policy Optimization), a token-level credit assignment method for multimodal reinforcement learning with verifiable rewards (RLVR). Unlike standard GRPO, VAttnPO leverages visual attention to provide fine-grained training signals. It constructs a visual attention reference by teacher-forcing the ground-truth answer and measures the alignment between this reference and each generated token's visual attention using Jensen–Shannon divergence. The divergence act as psudo score to modify the reward/advantage. The authors compare with multiple baselines on various benchmarks

Critical strengths
  1. The idea of comparing the attntion regions with and without gold label seems reasonable and to my knowledge, it is novel in the field of VLM + RLVR domain.
  2. The paper provides extensive empirical evaluations across diverse multimodal benchmarks and multiple VLM backbones.
Critical weaknesses
  1. The baseline methods are wierd. The authors spend manny effort compare with OPD and Self-distilled RL. However, this paper is clearly in the RLVR domain, where labels are provided and there is no other teacher model other than the model itself. So I don't see the point of comparing with OPD related methods. In the mean time, the paper missed many GRPO variants that are specially designed for VLM settings. For instance VPPO and VGPO they are all methods use token-visual dependancy as signals. The authors should considering replace OPD related baselines with them.

  2. There is a question for the method as well. The gold label for the visual alignment is the attention score with only the final answer while the visual attention for the trained model use the attention across tokens in the COT. There is a mismatch. I think it is likely that the at each token the VLM should focus on different parts of the image? So the idea of the method is questionable.

Questions for authors

In the weakness

3 comments

W1 & Q1: Comparisons with existing visual-credit methods.

Thank you for this suggestion. We agree that VPPO and VGPO are previous studies directly comparable to this one. We have also included a comparison with AT-RL. VPPO uses token visual dependency for trajectory-level advantage shaping and token-level gradient filtering. VGPO combines visual attention compensation with dual-grained advantage re-weighting to counter temporal visual forgetting. AT-RL applies cluster-based soft weighting based on cross-modal connectivity. In contrast, VAttnPO uses agreement with an answer-conditioned visual reference and relative-position weighting to construct a zero-mean additive correction.

MethodVisual signalCredit assignmentCore motivation
VPPOVisual dependency measured through image perturbationScale trajectory advantages and select tokensPrioritize visually dependent tokens and trajectories
AT-RLCross-modal attention connectivitySoft-weight token clustersReinforce perceptual anchors while retaining other tokens
VGPOHidden-state similarity to a visual prototypeReweight tokens and trajectories with late-stage compensationStrengthen visual focus and mitigate temporal visual forgetting
VAttnPO (Ours)Agreement with an answer-conditioned visual referenceApply relative-position weighting and construct a zero-mean additive correctionRefine within-response credit while preserving the trajectory-mean advantage

We have therefore supplemented our evaluation with VPPO, VGPO, and AT-RL. All four methods, including VAttnPO, are implemented within the GRPO framework using the same backbone Qwen3−VL−8B−InstructQwen3-VL-8B-Instruct, training data, and rollout budget.

MethodVStarHRBench 4KMMVPMMMU-ProMuirBenchBLINKAVE
VPPO83.7778.7581.0054.6264.1568.3371.77
VGPO83.7778.8881.3354.9764.5468.6472.02
AT-RL84.2979.0081.6754.9164.9668.5472.23
VAttnPO (Ours)87.4380.6282.3355.9565.6269.0773.50
MethodChartQAOCRBenchDocVQAInfoVQAPOPEHallusionBenchCountBenchQA
VPPO83.1289.5095.4983.3588.2471.8289.94
VGPO83.1689.4095.5283.0888.2272.4590.35
AT-RL83.2089.6095.6283.0988.2672.6690.14
VAttnPO (Ours)83.8490.1095.5983.6488.6773.8291.17
MethodMVBenchLongVideoBenchTempCompassVideo-MMEMME-perMME-rea
VPPO71.3263.270.1068.51725.1672.9
VGPO71.4863.470.2068.61735.5661.8
AT-RL71.5363.670.4968.61734.0673.6
VAttnPO (Ours)72.1563.671.5868.61755.0683.9

VAttnPO achieves an AVE of 73.50, exceeding VPPO, VGPO, and AT-RL by 1.73, 1.48, and 1.27 points, respectively, with improvements on all six image tasks. Across the complete evaluation, VAttnPO scores higher than VPPO on all 19 individual metrics. At the reported precision, it exceeds VGPO on 18 metrics and matches it on Video-MME. It also exceeds AT-RL on 16 metrics, matches it on LongVideoBench and Video-MME, and scores 0.03 points lower on DocVQA. These comparisons further support the effectiveness of our visual attention-guided credit assignment.

W2 & Q2: Position-Aware Visual Guidance across CoT Stages.

We agree that different CoT tokens may attend to different image regions and need not all match the answer-conditioned reference. Figure 1cc makes this observation more specific: deviations from the answer-conditioned reference are more pronounced early, while alignment tends to increase toward later stages. This trend motivates our position weights. We apply minimal guidance early, allowing intermediate steps to attend to different image regions, and progressively strengthen it in the latter half of the inference process.

After careful consideration, I still decide to hold my score.

Dear Reviewer aUL1,

We sincerely appreciate the time and effort you have spent reviewing our submission. We hope that our rebuttal has addressed your concerns adequately.

If you have a chance, we would be very grateful for any further feedback you might have, or any clarification you may need from our side. We would be happy to engage in additional discussion if that would be helpful.

Thank you again for your thoughtful review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of updates in the revised manuscript

We thank all the reviewers and the AC for their valuable contributions during the discussion period. Below, we summarize the updates in the revised manuscript. All updates are highlighted in blue in the revised PDF.

Additional Results and Analysis:

  • Results with VPPO, VGPO, and AT-RL: Section 5.1, Tables 1–3
  • Further ablation on credit variants: Table 4(a), Appendix F.1
  • Further ablation on position and layer design: Table 4(b), Appendix F.1
  • Results under a second training seed: Appendix F.3, Table 10
  • Results on layer selection across three backbones: Appendix I.1

Writing improvements:

  • Expanded related work (VPPO, VGPO, AT-RL): Section 2.3
  • Added the motivation for position modulation: Sections 3 and 4.4
  • Clarified the layer-selection procedure: Section 4.2

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.