- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies hallucination mitigation in multimodal large language models from the perspective of token-level credit assignment. The authors propose VIEW, a two-stage reinforcement learning method consisting of Visual-Attention Grounding (VAG) and Entropy-Guided Refinement (EGR). VAG adjusts token-level advantages according to visual attention, while EGR focuses policy updates on tokens with high predictive entropy. Both components are applied during training without changing the inference process. The authors evaluate VIEW on Qwen3-VL and Qwen2.5-VL models across seven benchmarks covering hallucination, visual reasoning, and general multimodal understanding. The paper also includes comparisons with existing methods, ablation studies, and analyses of the proposed token-level signals.
Critical strengths
(i) The paper focuses on hallucination in multimodal large language models, which is an important research problem that directly affects the reliability of MLLMs.
(ii) The paper is well organized and easy to follow. The figures complement the text effectively, particularly Fig. 1 and Fig. 2, which help illustrate the motivation and the proposed two-phase training framework.
Critical weaknesses
(i) The motivation and novelty of VIEW are not clearly established. The paper focuses on insufficient visual grounding and uncertain reasoning, but does not explain why these two error types are particularly important among the many causes of multimodal hallucination. The motivation mainly relies on prior studies and illustrative examples in Fig. 1, without analyzing these error types in the evaluated tasks. Moreover, several closely related methods are not discussed. PEPO [1] and ToR [2] have already combined visual perception signals with token entropy for token-level reinforcement learning, while VEPO [3] jointly uses visual sensitivity and entropy for token selection. PGPO [4], SRPO [5], and DyCo-RL [6] have also explored token-level optimization based on visual dependency or perception–reasoning relationships. The paper does not clearly explain what VIEW contributes beyond using an attention-based weighting signal and arranging these ideas into two training phases.
[1] Li, Yunheng, et al. "Rethinking token-level policy optimization for multimodal chain-of-thought." arXiv preprint arXiv:2603.22847 (2026).
[2] Lu, Jinda, et al. "Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs." arXiv preprint arXiv:2603.25077 (2026).
[3] Jin, Senjie, et al. "Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection." arXiv preprint arXiv:2606.03937 (2026).
[4] Ye, Zekai, et al. "Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models." arXiv preprint arXiv:2604.01840 (2026).
[5] Jiang, Bingqing, and Difan Zou. "Structured Role-Aware Policy Optimization for Multimodal Reasoning." arXiv preprint arXiv:2605.07274 (2026).
[6] Lin, Hangui, et al. "DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoning." arXiv preprint arXiv:2606.08035 (2026).
(ii) Several assumptions and claims behind VIEW are not supported by the reported analyses. First, the paper uses visual attention as a proxy for visual grounding and predictive entropy to identify critical reasoning steps. However, Appendix D reports a visual-attention AUROC of only 0.500 for the Qwen2.5-VL base model, compared with 0.632 after VIEW training. EGR also selects the top 20% highest-entropy tokens without examining whether these tokens actually correspond to incorrect reasoning decisions. Second, Appendix C.4 states that VAG also amplifies the suppression of negative-advantage tokens. As a result, correctly grounded tokens with high visual attention may receive stronger penalties than erroneous tokens with lower attention, but the paper does not explain how this addresses the token-level credit assignment problem raised in its motivation. Finally, Section 4.4 argues that visual-attention weights vary less among high-entropy tokens to motivate two-phase training. Although Table 2 compares the individual components and reversed training order, it does not compare the proposed strategy with simultaneous VAG+EGR optimization. The reported results therefore do not establish whether separating the two components is more effective than optimizing them together.
Questions for authors
(i) Please clarify why insufficient visual grounding and uncertain reasoning were selected as the two main failure modes, and provide further evidence of their importance in the evaluated tasks.
(ii) Please discuss the differences between VIEW and the closely related methods, particularly what new contribution VIEW provides beyond existing token-level weighting and selection strategies.
(iii) Please provide further evidence that visual attention and predictive entropy can reliably identify the tokens associated with the corresponding errors.
(iv) Please analyze how VAG affects correctly grounded tokens in negative-advantage responses, and explain how this weighting mechanism improves token-level credit assignment.
6 comments
Response to Reviewer VUyM (2/3)
Table 3. Qwen3-VL-8B-Thinking: VEPO-D, EGR, and Joint after a shared VAG stage
| Benchmark | VEPO-D (%) | EGR (%) | Joint (%) | VEPO−EGR: pp [95% CI] | VEPO−Joint: pp [95% CI] |
|---|---|---|---|---|---|
| POPE | 88.3333 | 88.5111 | 88.3556 | −0.1778 [−0.4889, 0.1333] | −0.0222 [−0.3556, 0.3111] |
| HallusionBench | 72.9849 | 74.0478 | 73.6935 | −1.0629 [−3.0278, +0.8227] | −0.7086 [−2.6019, +1.0831] |
| VideoHallucer | 77.0870 | 76.2609 | 76.6522 | +0.8261 [−0.3465, 1.9882] | +0.4348 [−0.7525, 1.5941] |
The VEPO-D selector adaptation produces benchmark-dependent differences. The comparison therefore quantifies the alternatives under a common continuation setting. EGR has the lowest measured continuation time among these three branches, while the separate two-seed EGR–Joint experiment further examines the accuracy–cost tradeoff. VIEW’s contribution rests on its stated weighting rule, full-response-to-selected-position schedule, and the experiments testing those choices.
W2 & Q3. Evidence for attention weighting and high-entropy token selection
We tested the visual signal directly in Qwen3, rather than extrapolating its behavior from the Qwen2.5 base-model AUROC. Identical object descriptions are paired with images in which the object is present or absent, providing a label for the relevant noun tokens. The table reports the base model and both VIEW training stages. The table reports AUROC [95% CI] over 32 object-present/absent image pairs. q(t) is the current-token attention used by VAG; q(t−1) is attention at the preceding prediction position. Stage differences are assessed using the same paired images.
Table 4. Qwen3 visual grounding across training stages
| Training stage | Attention at q(t) | Attention at q(t−1) | Negative entropy |
|---|---|---|---|
| Base Qwen3-VL-8B-Thinking | 0.839 [0.735, 0.923] | 0.583 [0.464, 0.698] | 0.569 [0.429, 0.718] |
| After VAG | 0.846 [0.742, 0.931] | 0.591 [0.476, 0.706] | 0.578 [0.438, 0.726] |
| After VAG→EGR | 0.849 [0.747, 0.932] | 0.595 [0.481, 0.707] | 0.571 [0.432, 0.719] |
Table 5. Visual weighting versus uniform weights
| Weighting rule | POPE (%) | MMStar (%) |
|---|---|---|
| Uniform weights | 88.70 | 72.33 |
| Bounded visual-attention weights | 88.80 | 73.07 |
The Qwen3 results show visual-evidence discrimination before and after training. Current-token attention AUROC is above 0.8 at all three stages, with intervals above 0.5. The paper’s Qwen2.5 generated-content analysis measures a different setting: VIEW has AUROC 0.632 [0.490, 0.750], and unsupported content is more frequent in the lowest-attention portion. The paper’s weighting ablation separately tests training utility: bounded visual weighting gives 88.80% POPE and 73.07% MMStar accuracy, compared with 88.70% and 72.33% for uniform weights.
Table 6. Entropy-selection fraction under DAPO+EGR
| Selected fraction ρ | POPE (%) | MMStar (%) |
|---|---|---|
| 0.0 | 88.17 | 72.67 |
| 0.1 | 88.74 | 72.87 |
| 0.2 | 88.79 | 73.60 |
| 0.3 | 88.38 | 71.93 |
| 0.4 | 88.33 | 73.07 |
Table 7. Qwen3-VL-8B-Thinking: three token selectors at a fixed 20% fraction
| Benchmark | Highest-entropy 20% | Random 20% | Lowest-entropy 20% | High−random: pp [95% CI] | High−low: pp [95% CI] |
|---|---|---|---|---|---|
| POPE | 88.72 | 88.42 | 88.37 | +0.300 [0.011, 0.589] | +0.356 [0.033, 0.689] |
| VideoHallucer | 77.04 | 76.35 | 77.04 | +0.696 [−0.531, 1.909] | +0.000 [−1.231, 1.196] |
The two ablations answer complementary questions: how many positions to refine and which positions to choose. The paper’s Table 3 varies the selection fraction under DAPO+EGR. 20% gives the highest POPE and MMStar accuracy among the tested fractions. At a fixed 20% fraction, high-entropy selection reaches 88.72% on POPE versus 88.42% for random selection, a gain of 0.300 percentage points [0.011, 0.589]. On VideoHallucer, high-entropy selection reaches 77.04%, versus 76.35% for random selection and 77.04% for low-entropy selection. The high−random interval includes zero. All three selectors use a shared VAG initialization and the same refinement budget. The POPE result supports choosing uncertain positions at a fixed update budget.
W2 & Q4. Negative advantages: weighting, actual updates, and visual support
VAG changes the relative allocation of a response-level update through bounded weights. With mₜ=0.5+0.5sₜ∈[0.5,1], weighting preserves the advantage sign. Within a negative-advantage response, higher-attention tokens receive more weight than lower-attention tokens. With the other loss terms fixed, their local gradient contribution is no larger than under unweighted DAPO. Appendix C.4’s “amplifies” refers to this within-response comparison. We measured actual probability changes to examine the effect after gradients are combined through shared parameters.
Response to Reviewer VUyM (3/3)
Table 8. Negative-advantage responses: measured log-probability changes
| Comparison | Mean difference ×10⁻⁴ | 95% CI ×10⁻⁴ |
|---|---|---|
| VAG − ordinary DAPO | +0.3505 | [−2.5446, 4.4448] |
| VAG − response-mean weights | +0.9851 | [−1.4214, 4.0549] |
| VAG − permuted token weights | −0.9868 | [−4.3163, 1.2723] |
The actual-update comparison uses identical responses and initial parameters. Response-mean weighting assigns every token the response’s average VAG weight. Permuted weighting keeps the weights but changes their token positions. The analysis covers 14 negative-advantage responses. The table reports changes in natural-log probability, with values scaled by 10⁻⁴.
Ten annotators reviewed ten prefixes, giving 86 visually supported judgments out of 100. For the five negative-advantage prefixes, VAG yields a higher human-support-weighted mean log-probability change than response-mean weighting across every allocation compatible with the aggregate annotations. The difference ranges from +0.000600 to +0.001379 per token. The comparison holds the mean response weight fixed and measures the effect of assigning weights to particular positions. The result supports reduced suppression relative to response-mean weighting for these prefixes.
W2. Simultaneous refinement and training cost
We added a direct comparison of entropy-only refinement and simultaneous visual-weighting-plus-entropy refinement. Starting from the same VAG model, EGR applies entropy selection alone, whereas Joint retains visual weighting at the selected positions. The comparison is repeated under two continuation seeds with matched data and generation randomness. The table reports all three hallucination benchmarks for both runs.
Table 9. Qwen3-VL-8B-Thinking: EGR versus Joint under two continuation seeds
| Seed | Benchmark | EGR (%) | Joint (%) | Joint−EGR: pp [95% CI] |
|---|---|---|---|---|
| 1 | POPE | 88.5111 | 88.3556 | −0.156 [−0.456, +0.144] |
| 1 | HallusionBench | 74.0478 | 73.6935 | −0.354 [−2.254, +1.528] |
| 1 | VideoHallucer | 76.2609 | 76.6522 | +0.391 [−0.738, +1.561] |
| 2 | POPE | 88.5333 | 88.3111 | −0.222 [−0.522, +0.089] |
| 2 | HallusionBench | 72.3649 | 73.6935 | +1.329 [−0.699, +3.384] |
| 2 | VideoHallucer | 76.1739 | 76.3478 | +0.174 [−1.015, +1.374] |
Table 10. Refinement time in the measured first run
| Method | Time (hours) |
|---|---|
| EGR | 3.5130 |
| Joint | 9.2203 |
EGR reduced refinement time by 61.9% relative to Joint in the timed run. Joint did not provide a consistent accuracy improvement across the evaluated benchmarks. Together with the component and training-order ablations, these results support using entropy-only refinement after VAG.
References
[1] Yunheng Li et al. Rethinking token-level policy optimization for multimodal chain-of-thought. arXiv:2603.22847, 2026.
[2] Jinda Lu et al. Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs. arXiv:2603.25077, 2026.
[3] Senjie Jin et al. Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection. arXiv:2606.03937, 2026.
[4] Zekai Ye et al. Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models. arXiv:2604.01840, 2026.
[5] Bingqing Jiang and Difan Zou. Structured Role-Aware Policy Optimization for Multimodal Reasoning. arXiv:2605.07274, 2026.
[6] Hangui Lin et al. DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoning. arXiv:2606.08035, 2026.
Response to Reviewer VUyM (1/3)
To Reviewer VUyM
Thank you for your comments. We address each concern below.
Visual grounding and reasoning uncertainty motivate separate interventions because an answer can fail at either stage. A model may reason fluently from a misidentified object, or describe the image correctly and then draw an incorrect conclusion. VAG directs learning toward image-dependent content across the response. EGR then concentrates refinement on uncertain predictions. We added an independent human study to test whether both error types occur in the evaluated response pool. The study covers 118 distinct responses from 32 images, each classified by ten independent annotators. The table counts 1,180 judgments, rather than independent responses.
Table 1. Human evaluation: 1,180 judgments on 118 responses from 32 images
The human judgments identify both visual-only and reasoning-only errors, together with mixed cases. Counting mixed errors in both relevant totals gives 204 visual-error judgments and 164 reasoning-error judgments. These are distinct targets for optimization. Ten annotators independently reviewed every response without discussion or AI assistance. 98 of 118 responses received unanimous classifications and 111 received agreement from at least six annotators.
This task-specific evidence explains the choice of the two training targets. The paper’s component and order ablations then test their effectiveness: visual weighting benefits the base model, and applying VAG before EGR gives the strongest MathVerse result among the compared variants. The motivation is therefore supported by observed response errors and by the behavior of the corresponding training components.
VIEW contributes a specific allocation rule and training schedule: bounded visual-attention weighting across the full response, followed by entropy-selected refinement. VAG converts each token’s share of attention to the image into a positive bounded weight. EGR subsequently removes visual weighting and updates high-entropy positions. The table compares this design with the six closely related methods in terms of signal definition, token weighting or selection, and training organization.
Table 2. Comparison with six token-level training methods
The distinction is tested experimentally through component, order, and joint-refinement comparisons. The paper’s Table 2 evaluates VAG, EGR, VAG→EGR, and EGR→VAG. Our added Joint comparison keeps both mechanisms active during refinement, directly testing the decision to remove visual weighting at that stage. We also implemented the public VEPO-D selector [3] alongside EGR and Joint from the same VAG initialization. This is an adaptation of the VEPO-D selector to the shared continuation setting: VEPO-D fixes selected positions before a policy update, whereas EGR recomputes entropy selection during the update.