acceptodds
Under review as a conference paper at ICLR 2027

Not All Tokens Contribute Equally: Token-Level Reinforcement Learning for Multimodal Hallucination Mitigation

Abstract

Reinforcement learning with verifiable rewards (RLVR) is increasingly used to reduce hallucination in multimodal large language models (MLLMs). A key limitation is that the RLVR reward is defined only at the sequence level, so every token in a response shares the same advantage, without explicitly distinguishing its contribution to the error. Two common failure modes motivate our approach. The model may attend insufficiently to the image and rely on language priors, or make a wrong choice at an uncertain reasoning step. Neither failure mode is directly observable, but the model provides two relevant internal signals during training. The visual attention ratio of a token provides a proxy for visual grounding, and the predictive entropy indicates where the model is uncertain. Building on these two signals, we take a token-level view of RLVR and propose **VIEW** (**V**isual-grounding and **I**nformation-**E**ntropy **W**eighting), which reshapes the learning signal at the token level. VIEW first rescales each token's advantage using visual attention, then focuses the policy gradient on high-entropy tokens where reasoning may fork. Both signals are used only during training, and VIEW leaves standard autoregressive inference unchanged. Extensive experiments across hallucination, visual-reasoning, and general multimodal benchmarks show that VIEW improves the Qwen3-VL-8B average over GRPO and PAPO.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multimodal large language models, hallucination mitigation, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VUyMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies hallucination mitigation in multimodal large language models from the perspective of token-level credit assignment. The authors propose VIEW, a two-stage reinforcement learning method consisting of Visual-Attention Grounding (VAG) and Entropy-Guided Refinement (EGR). VAG adjusts token-level advantages according to visual attention, while EGR focuses policy updates on tokens with high predictive entropy. Both components are applied during training without changing the inference process. The authors evaluate VIEW on Qwen3-VL and Qwen2.5-VL models across seven benchmarks covering hallucination, visual reasoning, and general multimodal understanding. The paper also includes comparisons with existing methods, ablation studies, and analyses of the proposed token-level signals.

Critical strengths

(i) The paper focuses on hallucination in multimodal large language models, which is an important research problem that directly affects the reliability of MLLMs.

(ii) The paper is well organized and easy to follow. The figures complement the text effectively, particularly Fig. 1 and Fig. 2, which help illustrate the motivation and the proposed two-phase training framework.

Critical weaknesses

(i) The motivation and novelty of VIEW are not clearly established. The paper focuses on insufficient visual grounding and uncertain reasoning, but does not explain why these two error types are particularly important among the many causes of multimodal hallucination. The motivation mainly relies on prior studies and illustrative examples in Fig. 1, without analyzing these error types in the evaluated tasks. Moreover, several closely related methods are not discussed. PEPO [1] and ToR [2] have already combined visual perception signals with token entropy for token-level reinforcement learning, while VEPO [3] jointly uses visual sensitivity and entropy for token selection. PGPO [4], SRPO [5], and DyCo-RL [6] have also explored token-level optimization based on visual dependency or perception–reasoning relationships. The paper does not clearly explain what VIEW contributes beyond using an attention-based weighting signal and arranging these ideas into two training phases.

[1] Li, Yunheng, et al. "Rethinking token-level policy optimization for multimodal chain-of-thought." arXiv preprint arXiv:2603.22847 (2026).

[2] Lu, Jinda, et al. "Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs." arXiv preprint arXiv:2603.25077 (2026).

[3] Jin, Senjie, et al. "Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection." arXiv preprint arXiv:2606.03937 (2026).

[4] Ye, Zekai, et al. "Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models." arXiv preprint arXiv:2604.01840 (2026).

[5] Jiang, Bingqing, and Difan Zou. "Structured Role-Aware Policy Optimization for Multimodal Reasoning." arXiv preprint arXiv:2605.07274 (2026).

[6] Lin, Hangui, et al. "DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoning." arXiv preprint arXiv:2606.08035 (2026).

(ii) Several assumptions and claims behind VIEW are not supported by the reported analyses. First, the paper uses visual attention as a proxy for visual grounding and predictive entropy to identify critical reasoning steps. However, Appendix D reports a visual-attention AUROC of only 0.500 for the Qwen2.5-VL base model, compared with 0.632 after VIEW training. EGR also selects the top 20% highest-entropy tokens without examining whether these tokens actually correspond to incorrect reasoning decisions. Second, Appendix C.4 states that VAG also amplifies the suppression of negative-advantage tokens. As a result, correctly grounded tokens with high visual attention may receive stronger penalties than erroneous tokens with lower attention, but the paper does not explain how this addresses the token-level credit assignment problem raised in its motivation. Finally, Section 4.4 argues that visual-attention weights vary less among high-entropy tokens to motivate two-phase training. Although Table 2 compares the individual components and reversed training order, it does not compare the proposed strategy with simultaneous VAG+EGR optimization. The reported results therefore do not establish whether separating the two components is more effective than optimizing them together.

Questions for authors

(i) Please clarify why insufficient visual grounding and uncertain reasoning were selected as the two main failure modes, and provide further evidence of their importance in the evaluated tasks.

(ii) Please discuss the differences between VIEW and the closely related methods, particularly what new contribution VIEW provides beyond existing token-level weighting and selection strategies.

(iii) Please provide further evidence that visual attention and predictive entropy can reliably identify the tokens associated with the corresponding errors.

(iv) Please analyze how VAG affects correctly grounded tokens in negative-advantage responses, and explain how this weighting mechanism improves token-level credit assignment.

6 comments
AuthorsOpenReview ↗

Response to Reviewer VUyM (1/3)

To Reviewer VUyM

Thank you for your comments. We address each concern below.

W1 & Q1. Why visual grounding and reasoning uncertainty are selected

Visual grounding and reasoning uncertainty motivate separate interventions because an answer can fail at either stage. A model may reason fluently from a misidentified object, or describe the image correctly and then draw an incorrect conclusion. VAG directs learning toward image-dependent content across the response. EGR then concentrates refinement on uncertain predictions. We added an independent human study to test whether both error types occur in the evaluated response pool. The study covers 118 distinct responses from 32 images, each classified by ten independent annotators. The table counts 1,180 judgments, rather than independent responses.

Table 1. Human evaluation: 1,180 judgments on 118 responses from 32 images

CategoryJudgmentsPercentage
No identified error76865.08%
Visual only17214.58%
Reasoning only13211.19%
Mixed322.71%
Other383.22%
Insufficient evidence383.22%
Total1180100%

The human judgments identify both visual-only and reasoning-only errors, together with mixed cases. Counting mixed errors in both relevant totals gives 204 visual-error judgments and 164 reasoning-error judgments. These are distinct targets for optimization. Ten annotators independently reviewed every response without discussion or AI assistance. 98 of 118 responses received unanimous classifications and 111 received agreement from at least six annotators.

This task-specific evidence explains the choice of the two training targets. The paper’s component and order ablations then test their effectiveness: visual weighting benefits the base model, and applying VAG before EGR gives the strongest MathVerse result among the compared variants. The motivation is therefore supported by observed response errors and by the behavior of the corresponding training components.

W1 & Q2. VIEW’s contribution relative to token weighting and selection methods

VIEW contributes a specific allocation rule and training schedule: bounded visual-attention weighting across the full response, followed by entropy-selected refinement. VAG converts each token’s share of attention to the image into a positive bounded weight. EGR subsequently removes visual weighting and updates high-entropy positions. The table compares this design with the six closely related methods in terms of signal definition, token weighting or selection, and training organization.

Table 2. Comparison with six token-level training methods

MethodSignal and updateVIEW design choice
PEPO [1]Hidden-state similarity and entropy, combined through smooth token weighting.Visual-attention mass ratio; dense bounded weighting followed by a separate hard-mask phase.
ToR [2]Image/empty-image sampled-token log-probability differences and entropy-based token sets.Attention-based continuous coefficients are used across the response before entropy refinement.
VEPO [3]Original/perturbed-image distribution divergence, entropy gap, and entropy for token selection.No perturbed-image branch; visual weighting and entropy selection have separate training phases.
PGPO [4]Visual-dependency estimates with response normalization and gated advantage weights.A direct attention ratio and a bounded linear coefficient, followed by sparse refinement.
SRPO [5]Structured perception/reasoning segments and advantage-sign-aware intervention weights.No prescribed segment labels; a positive bounded multiplier preserves the response advantage sign.
DyCo-RL [6]Attention-geometry changes for dynamic roles and role–attention consistency weighting.A visual-attention mass ratio rather than dynamic role assignment, with an explicit dense-to-sparse schedule.

The distinction is tested experimentally through component, order, and joint-refinement comparisons. The paper’s Table 2 evaluates VAG, EGR, VAG→EGR, and EGR→VAG. Our added Joint comparison keeps both mechanisms active during refinement, directly testing the decision to remove visual weighting at that stage. We also implemented the public VEPO-D selector [3] alongside EGR and Joint from the same VAG initialization. This is an adaptation of the VEPO-D selector to the shared continuation setting: VEPO-D fixes selected positions before a policy update, whereas EGR recomputes entropy selection during the update.

AuthorsOpenReview ↗

Response to Reviewer VUyM (2/3)

Table 3. Qwen3-VL-8B-Thinking: VEPO-D, EGR, and Joint after a shared VAG stage

BenchmarkVEPO-D (%)EGR (%)Joint (%)VEPO−EGR: pp [95% CI]VEPO−Joint: pp [95% CI]
POPE88.333388.511188.3556−0.1778 [−0.4889, 0.1333]−0.0222 [−0.3556, 0.3111]
HallusionBench72.984974.047873.6935−1.0629 [−3.0278, +0.8227]−0.7086 [−2.6019, +1.0831]
VideoHallucer77.087076.260976.6522+0.8261 [−0.3465, 1.9882]+0.4348 [−0.7525, 1.5941]

The VEPO-D selector adaptation produces benchmark-dependent differences. The comparison therefore quantifies the alternatives under a common continuation setting. EGR has the lowest measured continuation time among these three branches, while the separate two-seed EGR–Joint experiment further examines the accuracy–cost tradeoff. VIEW’s contribution rests on its stated weighting rule, full-response-to-selected-position schedule, and the experiments testing those choices.

W2 & Q3. Evidence for attention weighting and high-entropy token selection

We tested the visual signal directly in Qwen3, rather than extrapolating its behavior from the Qwen2.5 base-model AUROC. Identical object descriptions are paired with images in which the object is present or absent, providing a label for the relevant noun tokens. The table reports the base model and both VIEW training stages. The table reports AUROC [95% CI] over 32 object-present/absent image pairs. q(t) is the current-token attention used by VAG; q(t−1) is attention at the preceding prediction position. Stage differences are assessed using the same paired images.

Table 4. Qwen3 visual grounding across training stages

Training stageAttention at q(t)Attention at q(t−1)Negative entropy
Base Qwen3-VL-8B-Thinking0.839 [0.735, 0.923]0.583 [0.464, 0.698]0.569 [0.429, 0.718]
After VAG0.846 [0.742, 0.931]0.591 [0.476, 0.706]0.578 [0.438, 0.726]
After VAG→EGR0.849 [0.747, 0.932]0.595 [0.481, 0.707]0.571 [0.432, 0.719]

Table 5. Visual weighting versus uniform weights

Weighting rulePOPE (%)MMStar (%)
Uniform weights88.7072.33
Bounded visual-attention weights88.8073.07

The Qwen3 results show visual-evidence discrimination before and after training. Current-token attention AUROC is above 0.8 at all three stages, with intervals above 0.5. The paper’s Qwen2.5 generated-content analysis measures a different setting: VIEW has AUROC 0.632 [0.490, 0.750], and unsupported content is more frequent in the lowest-attention portion. The paper’s weighting ablation separately tests training utility: bounded visual weighting gives 88.80% POPE and 73.07% MMStar accuracy, compared with 88.70% and 72.33% for uniform weights.

Table 6. Entropy-selection fraction under DAPO+EGR

Selected fraction ρPOPE (%)MMStar (%)
0.088.1772.67
0.188.7472.87
0.288.7973.60
0.388.3871.93
0.488.3373.07

Table 7. Qwen3-VL-8B-Thinking: three token selectors at a fixed 20% fraction

BenchmarkHighest-entropy 20%Random 20%Lowest-entropy 20%High−random: pp [95% CI]High−low: pp [95% CI]
POPE88.7288.4288.37+0.300 [0.011, 0.589]+0.356 [0.033, 0.689]
VideoHallucer77.0476.3577.04+0.696 [−0.531, 1.909]+0.000 [−1.231, 1.196]

The two ablations answer complementary questions: how many positions to refine and which positions to choose. The paper’s Table 3 varies the selection fraction under DAPO+EGR. 20% gives the highest POPE and MMStar accuracy among the tested fractions. At a fixed 20% fraction, high-entropy selection reaches 88.72% on POPE versus 88.42% for random selection, a gain of 0.300 percentage points [0.011, 0.589]. On VideoHallucer, high-entropy selection reaches 77.04%, versus 76.35% for random selection and 77.04% for low-entropy selection. The high−random interval includes zero. All three selectors use a shared VAG initialization and the same refinement budget. The POPE result supports choosing uncertain positions at a fixed update budget.

W2 & Q4. Negative advantages: weighting, actual updates, and visual support

VAG changes the relative allocation of a response-level update through bounded weights. With mₜ=0.5+0.5sₜ∈[0.5,1], weighting preserves the advantage sign. Within a negative-advantage response, higher-attention tokens receive more weight than lower-attention tokens. With the other loss terms fixed, their local gradient contribution is no larger than under unweighted DAPO. Appendix C.4’s “amplifies” refers to this within-response comparison. We measured actual probability changes to examine the effect after gradients are combined through shared parameters.

AuthorsOpenReview ↗

Response to Reviewer VUyM (3/3)

Table 8. Negative-advantage responses: measured log-probability changes

ComparisonMean difference ×10⁻⁴95% CI ×10⁻⁴
VAG − ordinary DAPO+0.3505[−2.5446, 4.4448]
VAG − response-mean weights+0.9851[−1.4214, 4.0549]
VAG − permuted token weights−0.9868[−4.3163, 1.2723]

The actual-update comparison uses identical responses and initial parameters. Response-mean weighting assigns every token the response’s average VAG weight. Permuted weighting keeps the weights but changes their token positions. The analysis covers 14 negative-advantage responses. The table reports changes in natural-log probability, with values scaled by 10⁻⁴.

Ten annotators reviewed ten prefixes, giving 86 visually supported judgments out of 100. For the five negative-advantage prefixes, VAG yields a higher human-support-weighted mean log-probability change than response-mean weighting across every allocation compatible with the aggregate annotations. The difference ranges from +0.000600 to +0.001379 per token. The comparison holds the mean response weight fixed and measures the effect of assigning weights to particular positions. The result supports reduced suppression relative to response-mean weighting for these prefixes.

W2. Simultaneous refinement and training cost

We added a direct comparison of entropy-only refinement and simultaneous visual-weighting-plus-entropy refinement. Starting from the same VAG model, EGR applies entropy selection alone, whereas Joint retains visual weighting at the selected positions. The comparison is repeated under two continuation seeds with matched data and generation randomness. The table reports all three hallucination benchmarks for both runs.

Table 9. Qwen3-VL-8B-Thinking: EGR versus Joint under two continuation seeds

SeedBenchmarkEGR (%)Joint (%)Joint−EGR: pp [95% CI]
1POPE88.511188.3556−0.156 [−0.456, +0.144]
1HallusionBench74.047873.6935−0.354 [−2.254, +1.528]
1VideoHallucer76.260976.6522+0.391 [−0.738, +1.561]
2POPE88.533388.3111−0.222 [−0.522, +0.089]
2HallusionBench72.364973.6935+1.329 [−0.699, +3.384]
2VideoHallucer76.173976.3478+0.174 [−1.015, +1.374]

Table 10. Refinement time in the measured first run

MethodTime (hours)
EGR3.5130
Joint9.2203

EGR reduced refinement time by 61.9% relative to Joint in the timed run. Joint did not provide a consistent accuracy improvement across the evaluated benchmarks. Together with the component and training-order ablations, these results support using entropy-only refinement after VAG.

References

[1] Yunheng Li et al. Rethinking token-level policy optimization for multimodal chain-of-thought. arXiv:2603.22847, 2026.

[2] Jinda Lu et al. Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs. arXiv:2603.25077, 2026.

[3] Senjie Jin et al. Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection. arXiv:2606.03937, 2026.

[4] Zekai Ye et al. Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models. arXiv:2604.01840, 2026.

[5] Bingqing Jiang and Difan Zou. Structured Role-Aware Policy Optimization for Multimodal Reasoning. arXiv:2605.07274, 2026.

[6] Hangui Lin et al. DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoning. arXiv:2606.08035, 2026.

Reviewer_kF37OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies token-level credit assignment for reinforcement learning with verifiable rewards in multimodal LLMs. The authors observe that standard GRPO/DAPO assigns the same sequence-level advantage to every token in a sampled response, even though individual tokens may contribute differently to hallucination or reasoning errors. The proposed VIEW framework introduces two model-internal token-level signals. Visual-Attention Grounding (VAG) reweights token updates based on the proportion of decoder attention directed toward visual tokens, while Entropy-Guided Refinement (EGR) focuses policy-gradient updates on high-entropy tokens associated with uncertain decisions. These components are applied sequentially, with visual grounding followed by entropy-based refinement. The method is evaluated on hallucination, visual-reasoning, and general multimodal benchmarks.

Critical strengths
  • The method is directly aligned with a clear limitation of sequence-level RLVR. If every token receives the same response-level advantage, differences in grounding and uncertainty are ignored. VIEW modifies precisely this aspect of the optimization procedure.

  • The method is practical in the sense that it uses signals already available from the policy model during training and leaves standard autoregressive inference unchanged.

  • The paper makes a meaningful effort to validate visual attention as a grounding-sensitive signal. Unsupported tokens are more frequent in the lowest-attention quintile, and attention provides above-random discrimination after VIEW training.

Critical weaknesses
  • Visual attention remains a relatively weak proxy for grounding. The reported token-level AUROC is around 0.63, which is above random but still indicates substantial overlap between supported and unsupported tokens. I therefore view the signal as grounding-sensitive rather than as a reliable estimator of token correctness.

  • The entropy component is less directly validated. High entropy indicates uncertainty, but it does not necessarily identify a token that causally determines a downstream reasoning error. The paper does not directly show that the selected high-entropy positions are disproportionately responsible for incorrect reasoning trajectories.

  • The strongest fully controlled evidence is concentrated on one main backbone. Additional matched experiments on substantially different architectures would strengthen the claim that the underlying credit-assignment principle generalizes broadly.

Questions for authors

I think the paper demonstrates that visual-attention and entropy-based weighting can be useful training signals more convincingly than it demonstrates that they recover the true token-level causes of hallucination or reasoning failure.

4 comments
AuthorsOpenReview ↗

Response to Reviewer kF37 (1/2)

To Reviewer kF37

Thank you for your comments. We address each concern below.

W1 & W2 & Q1. Why attention and entropy are useful training signals

VIEW uses the answer reward to determine the direction of learning, visual attention to distribute update weights, and entropy to choose refinement positions. VAG uses a positive bounded coefficient, mₜ=0.5+0.5sₜ, where sₜ is normalized visual attention within a response. EGR concentrates subsequent updates at uncertain next-token predictions. The attention signal’s usefulness therefore depends on its relationship to visual evidence and on the resulting training behavior. Entropy’s role is assessed through selection ablations and direct interventions.

Table 1. Qwen3 visual grounding across training stages

Training stageAttention at q(t)Attention at q(t−1)Negative entropy
Base Qwen3-VL-8B-Thinking0.839 [0.735, 0.923]0.583 [0.464, 0.698]0.569 [0.429, 0.718]
After VAG0.846 [0.742, 0.931]0.591 [0.476, 0.706]0.578 [0.438, 0.726]
After VAG→EGR0.849 [0.747, 0.932]0.595 [0.481, 0.707]0.571 [0.432, 0.719]

The main Qwen3 model already contains a visual-evidence signal before VIEW training: attention AUROC is 0.839 in the base model, 0.846 after VAG, and 0.849 after VAG→EGR, with all three 95% intervals above 0.5. We evaluate identical object descriptions on 32 pairs of object-present and object-absent images, using POPE labels for the noun tokens. This controls the text and directly tests whether attention responds to image support. q(t) is attention at the token position used by VAG; q(t−1) is attention at the preceding position that predicts that token. The cited 0.500 measures Qwen2.5 base-model attention on generated response content. The added experiment tests the main Qwen3 model with wording held fixed. These are different evaluation settings. The Qwen3 result supports the premise of VAG: a measurable association with visual evidence is available for reward-guided weighting from the start, without requiring attention to serve as a binary correctness label.

Table 2. Entropy-selection fraction under DAPO+EGR

Selected fraction ρPOPE (%)MMStar (%)
0.088.1772.67
0.188.7472.87
0.288.7973.60
0.388.3871.93
0.488.3373.07

Table 3. Qwen3-VL-8B-Thinking: three token selectors at a fixed 20% fraction

BenchmarkHighest-entropy 20%Random 20%Lowest-entropy 20%High−random: pp [95% CI]High−low: pp [95% CI]
POPE88.7288.4288.37+0.300 [0.011, 0.589]+0.356 [0.033, 0.689]
VideoHallucer77.0476.3577.04+0.696 [−0.531, 1.909]+0.000 [−1.231, 1.196]

The two ablations answer complementary questions: how many positions to refine and which positions to choose. The paper’s Table 3 varies the selection fraction under DAPO+EGR. 20% gives the highest POPE and MMStar accuracy among the tested fractions. At a fixed 20% fraction, high-entropy selection reaches 88.72% on POPE versus 88.42% for random selection, a gain of 0.300 percentage points [0.011, 0.589]. On VideoHallucer, high-entropy selection reaches 77.04%, versus 76.35% for random selection and 77.04% for low-entropy selection. The high−random interval includes zero. All three selectors use a shared VAG initialization and the same refinement budget. The POPE result supports choosing uncertain positions at a fixed update budget.

AuthorsOpenReview ↗

Response to Reviewer kF37 (2/2)

W3. Applicability beyond the main backbone

We extended VIEW to LLaVA-1.5-7B, which combines a LLaMA/Vicuna language model with a CLIP vision encoder. This provides an actual training comparison in a model family different from Qwen. GRPO, DAPO, and VIEW use the same initialization, training data, and LoRA training budget. The table compares these methods with the pretrained base model on MMStar and on AMBER’s discriminative task, which assesses hallucination through judgments about image content. Each benchmark uses common generation and scoring settings across the four models.

Table 4. LLaVA-1.5-7B: MMStar and AMBER accuracy

ModelMMStar accuracy (%)AMBER discriminative accuracy (%)
Base LLaVA32.2079.73
GRPO31.9379.40
DAPO32.2079.85
VIEW32.4080.04

VIEW achieves the highest accuracy among the four models on both benchmarks: 32.40% on MMStar and 80.04% on AMBER. MMStar accuracy is 32.20% for the base model and DAPO, and 31.93% for GRPO. AMBER accuracy is 79.73%, 79.85%, and 79.40%, respectively. These experiments demonstrate that both VIEW stages can be applied in the LLaMA/Vicuna–CLIP architecture, extending the training evaluation beyond the Qwen family. On MMStar, the VIEW−GRPO and VIEW−DAPO differences are +0.47 [−0.46, 1.39] and +0.20 [−0.67, 1.13] percentage points.

The paper also evaluates VIEW across two Qwen generations with different visual processing. Qwen2.5-VL uses vision-encoder window attention and time-aligned MRoPE. Qwen3-VL introduces DeepStack visual-feature integration, interleaved MRoPE, and explicit video timestamps. The paper’s seven-benchmark average improves from 62.20 to 68.48 on Qwen2.5-VL-7B and from 71.62 to 74.69 on Qwen3-VL-8B-Thinking. Together with the LLaVA results, these experiments demonstrate the applicability of VIEW across the evaluated model generations and families.

AuthorsOpenReview ↗

A gentle discussion-phase reminder

Dear Reviewer kF37,

Thank you very much for your great efforts in reviewing this paper and for your recognition of our work!

As the end of the discussion period is approaching, we are eagerly anticipating your feedback to ascertain if our responses have adequately addressed your concerns. Your feedback will be instrumental in enhancing the quality of our manuscript. We would appreciate the opportunity to address any additional concerns you may have before the discussion period ends.

Thank you once again for your time and consideration!

Sincerely,

The Submission 4012 Authors

AuthorsOpenReview ↗

A Gentle Reminder Before the Discussion Deadline

Dear Reviewer kF37,

Thank you again for your careful review and constructive suggestions.

As the discussion deadline approaches, we would greatly appreciate your feedback on our responses. We would be happy to clarify any remaining questions.

Thank you for your time and consideration.

Best regards,

The Submission 4012 Authors

Reviewer_9KuGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes VIEW, a two-stage modification of DAPO for multimodal reinforcement learning. Visual-Attention Grounding first rescales each response token's sequence-level advantage using its attention to visual tokens. Entropy-Guided Refinement then updates only the highest-entropy 20% of response tokens. Both signals are used during training, leaving inference unchanged. Experiments use Qwen3-VL-8B-Thinking on seven benchmarks, with additional Qwen2.5-VL results, component/order ablations, and a content-span audit of the attention proxy. VIEW obtains a reported seven-benchmark average of 74.69, compared with 73.61 for GRPO and 73.76 for PAPO-D.

Critical strengths
  1. The intervention is practical and clearly specified. VIEW uses internal attention and entropy signals without an external perception judge during training or additional inference machinery. The explicit separation of the two phases makes the method easy to understand and potentially reuse.

  2. The evaluation contains useful positive evidence. Under a shared Qwen3 backbone, dataset, and two-epoch budget, VIEW improves the reported average over GRPO and both PAPO variants. The equal-budget ordering comparison also favors VAG→EGR over EGR→VAG on MathVerse and MME.

Critical weaknesses
  1. The added value of VAG for hallucination mitigation remains unclear. The ablations provide some evidence that VAG is useful, especially for reasoning, but do not establish a consistent benefit on the paper's central hallucination tasks. EGR alone performs similarly on POPE, and the stage-matched comparison is missing for HallusionBench and VideoHallucer. Differences in optimization settings also limit what can be attributed to VAG from the comparison with GRPO. Given the small reported hallucination gains and the lack of uncertainty estimates, stronger evidence is needed that VAG improves hallucination mitigation beyond the existing training recipe.

  2. The evidence for the proposed grounding explanation is limited. The attention audit is informative, but it uses a small sample from a different backbone and mainly examines the model after training. It therefore provides limited evidence that the attention weights identify visually grounded tokens during the main training process. Attention may still be a useful heuristic, but the current analysis does not firmly connect the observed improvements to better grounding at the token level.

Questions for authors
  1. Targeted comparison: Can you extend the stage-matched DAPO→EGR control to HallusionBench and VideoHallucer, and assess uncertainty for the small hallucination gains? This would directly test whether VAG improves the target behavior beyond the entropy intervention.
  2. Clarification: Is the checkpoint in Figure 7 VAG-only, as its labels suggest, or final VIEW, as Appendix D describes? What evidence supports extending the Qwen2.5 post-training diagnostic to the main Qwen3 training setting?
8 comments
AuthorsOpenReview ↗

Response to Reviewer 9KuG

To Reviewer 9KuG

Thank you for your comments. We address each concern below.

W1 & Q1. The contribution of visual weighting: stage comparison and uncertainty

We compared VIEW with DAPO→EGR on HallusionBench and VideoHallucer to evaluate visual weighting before entropy refinement. DAPO→EGR applies ordinary DAPO before EGR. VIEW applies VAG before EGR. Both pipelines therefore include the entropy-refinement stage. The table compares the two trained models under the same evaluation settings and reports accuracy (%) and paired differences with 95% confidence intervals.

Table 1. Qwen3-VL-8B-Thinking: DAPO→EGR versus VIEW on hallucination benchmarks

BenchmarkDAPO→EGR controlVIEWVIEW − DAPO→EGR, pp [95% CI]
HallusionBench72.6372.72+0.089 [−1.413, +1.648]
VideoHallucer76.3077.65+1.348 [0.130, 2.579]

On VideoHallucer, VIEW reaches 77.65%, compared with 76.30% for DAPO→EGR—a gain of 1.348 percentage points with a paired 95% confidence interval of [0.130, 2.579], entirely above zero. With EGR present in both pipelines, VIEW’s additional gain supports the contribution of visual weighting before entropy refinement to video-hallucination performance. On HallusionBench, the paired difference is +0.089 [−1.413, +1.648] percentage points.

Table 2. Qwen3-VL-8B-Thinking: component and training-order ablations on MathVerse

Training variantMathVerse (%)
Baseline59.90
VAG (batch normalization)65.99
VAG (response normalization)65.36
EGR only (top 20% entropy)65.36
EGR → VAG64.85
VIEW: VAG → EGR67.13

The MathVerse results from the paper’s Table 2 further support both components and the VAG→EGR order. VIEW achieves 67.13%, the highest accuracy among all six variants: 1.77 percentage points above EGR alone (65.36%) and 2.28 points above the reversed EGR→VAG order (64.85%). VAG with response normalization improves the base model from 59.90% to 65.36%. Applying EGR afterward raises accuracy to 67.13%. These comparisons support visual weighting before entropy refinement on MathVerse, and the VideoHallucer result provides further evidence of improved video-hallucination performance.

W2 & Q2. Figure 7 and the extension of grounding analysis to Qwen3

Figure 7 analyzes the final VIEW model based on Qwen2.5-VL-7B after both VAG and EGR training. We also provide the following labeled analysis for the main Qwen3-VL-8B-Thinking model.

We also tested whether the main Qwen3-VL-8B-Thinking model’s visual attention distinguishes image-supported object descriptions before training, after VAG, and after VAG→EGR. For each object, an image containing it and an image without it receive identical descriptive text. POPE labels determine whether the object noun is visually supported. Fixing the text lets us test the relationship to image evidence while controlling wording. The table reports AUROC [95% CI] over 32 image pairs. q(t) denotes attention at the object-token position used by VAG; q(t−1) denotes attention at the preceding position that predicts that token.

Table 3. Qwen3 visual grounding across training stages

Training stageAttention at q(t)Attention at q(t−1)Negative entropy
Base Qwen3-VL-8B-Thinking0.839 [0.735, 0.923]0.583 [0.464, 0.698]0.569 [0.429, 0.718]
After VAG0.846 [0.742, 0.931]0.591 [0.476, 0.706]0.578 [0.438, 0.726]
After VAG→EGR0.849 [0.747, 0.932]0.595 [0.481, 0.707]0.571 [0.432, 0.719]

Visual attention distinguishes supported from unsupported object descriptions at every evaluated Qwen3 stage: AUROC is 0.839, 0.846, and 0.849, with all three 95% intervals above chance (0.5). The association is present before training and remains observable after both stages, providing a measurable visual-evidence signal for VAG. The component and weighting ablations further test the benefit of using this signal for optimization. Figure 7 and the paper’s Table 4 provide a complementary analysis of Qwen2.5 generated responses: VIEW has attention AUROC 0.632 [0.490, 0.750], and unsupported content accounts for 26.8% of the lowest-attention 20% of tokens versus 10.9% of the remaining tokens.

AuthorsOpenReview ↗

A gentle discussion-phase reminder

Dear Reviewer 9KuG,

Thank you very much for your great efforts in reviewing this paper and for your recognition of our work!

As the end of the discussion period is approaching, we are eagerly anticipating your feedback to ascertain if our responses have adequately addressed your concerns. Your feedback will be instrumental in enhancing the quality of our manuscript. We would appreciate the opportunity to address any additional concerns you may have before the discussion period ends.

Thank you once again for your time and consideration!

Sincerely,

The Submission 4012 Authors

Reviewer_9KuGOpenReview ↗

Official Comments by Reviewer 9KuG

Thank you for the additional experiments and clarifications. I appreciate your efforts to address my concerns. Three things still aren't clear to me.

  1. the VideoHallucer gain. POPE and HallusionBench are basically tied with DAPO→EGR, so the case for VAG on hallucination comes down to VideoHallucer. But the same VAG→EGR pipeline gets quite different scores across your responses: 77.65 in the reply to me, 77.04 and 76.26/76.17 in the replies to VUyM, and 76.61 in the paper. Against DAPO→EGR's 76.30, that's anywhere from −0.13 to +1.35. Could you explain what differs between these runs, or show results over a few seeds? Same concern for the main table, which is single-run with gaps of about one point.

  2. grounding. The Qwen3 audit helps, but it shows the attention signal is already in the base model (AUROC 0.839) and barely moves after training (0.849). So it backs up using attention as a weight, but not the claim that training improves grounding, or that this is where the gains come from.

  3. a few points from my review weren't addressed. VIEW and GRPO still use different optimization settings, and there's no plain DAPO row in the main table, so it's hard to tell how much of the gain over GRPO comes from VIEW itself. The new audit is also small (32 image pairs).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.