- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper builds a visual attribution for each rollout by weighting its attention to image tokens according to how much the final answer relies on each reasoning step. It averages these attributions over correct and incorrect rollouts to form success and failure prototypes, and takes their difference as a contrastive signal of which visual regions separate success from failure. Pilot studies claim that this signal tells held-out correct rollouts apart from incorrect ones, that perturbing the regions it selects hurts accuracy much more than random perturbation, and that it identifies the response tokens that help or hurt the answer. Building on this, VEP-RL slightly rescales GRPO advantages according to how closely a rollout matches the prototypes, and shifts token-level credit toward the response tokens the prototypes select. The method is evaluated on three open vision-language backbones across many in-domain and out-of-domain benchmarks. The paper also includes ablations, a multi-seed comparison against DAPO, and a local theoretical analysis.
Critical strengths
- I think the main strength is that the annotation-free signal is supported by proper controls, not just by attention visualizations. The pilot study on held-out rollouts shows that the prototype contrast clearly separates correct from incorrect responses, while shuffling the outcome labels brings it down to chance. The intervention experiment shows that replacing the visual states selected by the contrast hurts accuracy much more than replacing random ones, and the paired confidence intervals rule out noise. The leave-one-out variant in the main ablation keeps most of the gain, so the effect does not come from a rollout being compared against a prototype it helped build. Taken together, these results convinced me that the signal captures something real about which visual regions the model relies on.
- I also find the comparison with the self-trained baselines convincing, because they share the training data, hyperparameters and evaluation protocol with VEP-RL, so the gap is not confounded by setup differences. The multi-seed experiment on the newest backbone shows that VEP-RL beats DAPO in every seed with a small spread, which suggests the improvement is not a lucky run. The training-cost table also shows that the extra overhead over GRPO is modest, lower than that of a comparable visual-perception method, since the extra computation only runs in groups that contain both correct and incorrect rollouts. This makes the method easy to adopt in practice.
Critical weaknesses
- My main concern is that the mechanism behind the gain is very mild and was never varied. Both reweighting steps change the advantages only slightly, and the training-cost table shows that the guidance is active in only a minority of rollout groups. Even so, the reported gains over GRPO and DAPO are large. The strength coefficients, selection ratios and attention layers are all fixed, and multiple seeds are reported for only one backbone. The shuffle and contrast ablations are also single runs with small margins. I am therefore not convinced that the VEP selection itself, rather than some other difference in the training recipe, is what causes the gains.
- I also think the comparisons with public baselines are not directly comparable. The evaluation-protocol part shows that several external methods are scored with a single greedy response, some are trained on much less data, and one runs with thinking disabled. The authors' models, by contrast, are averaged over several sampled responses, so the claimed margins are hard to interpret. The setup is also reported inconsistently. One backbone is described as using greedy decoding with several responses per question. The main table lacks the base-model rows and one GRPO run that was trained. It is also unclear whether VEP-RL is built on GRPO or on a DAPO-style recipe.
Questions for authors
- Can you report a lambda_traj/lambda_tok sweep (e.g., 0, 0.05, 0.1, 0.2) and at least three seeds on a second backbone, including the DAPO baseline, the Table 8 shuffle controls and the 'w/o prototype contrast' row? Please also state exactly how VEP-RL's base recipe differs from your GRPO and DAPO runs, given that all three share clip 0.20/0.28 in table 4. A flat lambda curve, or shuffle controls whose seed intervals overlap VEP-RL, would lower my assessment.
- Beyond model-internal proxies, do D+-selected visual tokens overlap annotated evidence regions more than Saliency-R1 or TAM selections do? And does the VEP similarity AUROC (Table 9) survive after controlling for response length and total visual attention? Appendix D.1 itself notes that the method 'does not identify annotated or causal evidence'.
7 comments
Responses to Reviewer D5R1 (Part 2/5)
W1 (3). The strength coefficients, selection ratios and attention layers are all fixed
Attention layers
Our layer selection is motivated by prior findings that later-middle layers contribute more strongly to correct object-token predictions [2], and that multi-layer attention aggregation improves performance, with limited gains from adding more layers [3]. We therefore select three evenly spaced depths spanning the middle-to-late decoder layers: 45%, 65%, and 85%. To examine whether performance depends on these specific depths, we compare alternative layer selections and all-layer aggregation on Qwen3-VL-8B. The results are presented below.
| Decoder depths | Acc. (%) ↑ |
|---|---|
| 25%, 45%, 65% | 76.38 |
| 45%, 65%, 85% | 78.34 |
| 55%, 75%, 95% | 78.17 |
| All layers | 76.12 |
As shown in the table, our selected configuration achieves the highest accuracy. Moving to 55%, 75%, and 95% reduces accuracy by only 0.17 points, whereas the earlier configuration and all-layer aggregation reduce accuracy by 1.96 and 2.22 points, respectively.
Visual-token intervention ratio
We choose 10% based on the intervention analysis in Figure 7: answer accuracy under intervention decreases as the ratio increases to 10% and plateaus thereafter. We train VEP-RL on Qwen3-VL-8B with intervention ratios of 0%, 5%, 10%, 15%, and 20%, keeping trajectory-level reweighting at and token-level reweighting at . At 0% intervention, no response tokens are selected for reweighting, so this setting corresponds to trajectory-only guidance. The results are presented below.
| Intervention Ratio | Acc. (%) ↑ |
|---|---|
| 0% | 77.15 |
| 5% | 77.71 |
| 10% | 78.34 |
| 15% | 77.97 |
| 20% | 77.79 |
As shown in the table, 10% achieves the highest accuracy of 78.34%. Increasing the ratio to 15% and 20% reduces accuracy by 0.37 and 0.55 points, respectively.
Response-token selection ratio
Our token selection is motivated by prior studies showing that RLVR gains can depend on a sparse subset of token decisions [4, 5]. We rank response tokens by their intervention scores and redistribute credit toward up to the top 5% with positive scores. To evaluate this choice, we compare selection ratios of 1%, 3%, 5%, 10%, and 15% on Qwen3-VL-8B, keeping trajectory-level reweighting at and token-level reweighting at . The results are presented below.
| Selection ratio | Acc. (%) ↑ |
|---|---|
| 1% | 77.53 |
| 3% | 77.98 |
| 5% | 78.34 |
| 10% | 77.67 |
| 15% | 76.90 |
As shown in the table, 5% achieves the highest accuracy of 78.34% among the tested ratios. Reducing the ratio to 3% lowers accuracy by 0.36 points, while increasing it to 10% and 15% lowers accuracy by 0.67 and 1.44 points, respectively.
Q1 (4). Please also state exactly how VEP-RL's base recipe differs from your GRPO and DAPO runs, given that all three share clip 0.20/0.28 in table 4.
W2 (5). It is also unclear whether VEP-RL is built on GRPO or on a DAPO-style recipe.
We'd like to clarify that VEP-RL is built on standard GRPO, as stated in Section 4. It adds only VEP-guided advantage reweighting at the trajectory and token levels. The GRPO and DAPO baselines follow their respective official training recipes, using VeRL as stated in Appendix B.1.
The clip-high value of 0.28 reported for GRPO and VEP-RL is a typo, and both use 0.20/0.20. We apologize for this typo and will correct Table 4 and the corresponding text. All three methods use ViRL-39K, two training epochs, and AdamW with a learning rate of . The training settings used in our GRPO, DAPO, and VEP-RL runs are summarized below.
| Setting | GRPO | DAPO | VEP-RL |
|---|---|---|---|
| Clip-low / clip-high | 0.20 / 0.20 | 0.20 / 0.28 | 0.20 / 0.20 |
| Dynamic sampling | No | Yes | No |
| Overlong reward shaping | No | Yes | No |
| KL regularization | Disabled | ||
| VEP-guided advantage reweighting | No | No | Trajectory and token levels |
Responses to Reviewer D5R1 (Part 3/5)
Q1 (5). at least three seeds on a second backbone, including the DAPO baseline, the Table 8 shuffle controls and the 'w/o prototype contrast' row? Shuffle controls whose seed intervals overlap VEP-RL would lower my assessment.
Appendix B.2.2 (Table 7) reports results across three training seeds on Qwen3.5-9B. Following your suggestion, we conducted experiments on a second backbone, Qwen2.5-VL-7B, using three training seeds (42, 43, and 44), and report the 14-benchmark macro accuracy. The 95% CIs are computed across training seeds; paired denotes VEP-RL minus each comparison method under the same seed. The results are presented below.
| Method | Seed 42 Acc. (%) | Seed 43 Acc. (%) | Seed 44 Acc. (%) | Mean Acc. ± Std (%) ↑ | 95% CI of Acc. (%) | Paired Δ Acc. ± Std (pp) ↑ | 95% CI of Δ Acc. (pp) |
|---|---|---|---|---|---|---|---|
| DAPO | 65.29 | 65.50 | 65.11 | 65.30±0.20 | [64.82,65.78] | +3.62±0.04 | [3.52,3.72] |
| Token Shuffle | 67.83 | 67.77 | 67.54 | 67.71±0.15 | [67.33,68.09] | +1.20±0.10 | [0.96,1.44] |
| Trajectory Shuffle | 67.31 | 67.55 | 67.19 | 67.35±0.18 | [66.89,67.81] | +1.57±0.06 | [1.41,1.72] |
| w/o Prototype Contrast | 68.22 | 68.11 | 68.06 | 68.13±0.08 | [67.93,68.33] | +0.79±0.16 | [0.38,1.19] |
| VEP-RL (Ours) | 68.95 | 69.08 | 68.72 | 68.92±0.18 | [68.46,69.37] | — | — |
As shown in the table, VEP-RL outperforms all comparison methods in every seed, and the 95% CI of its mean accuracy does not overlap with those of either shuffle control. The mean paired gains over Token Shuffle and Trajectory Shuffle are 1.20 and 1.57 percentage points, with 95% CIs of [0.96, 1.44] and [1.41, 1.72], respectively, both above zero. Removing prototype contrast reduces mean accuracy by 0.79 percentage points, with a paired 95% CI of [0.38, 1.19]. These results support the contributions of VEP-guided RL and prototype contrast.
W2. I also think the comparisons with public baselines are not directly comparable. The evaluation-protocol part shows that several external methods are scored with a single greedy response, some are trained on much less data, and one runs with thinking disabled. The authors' models, by contrast, are averaged over several sampled responses, so the claimed margins are hard to interpret. The setup is also reported inconsistently. One backbone is described as using greedy decoding with several responses per question. The main table lacks the base-model rows and one GRPO run that was trained. It is also unclear whether VEP-RL is built on GRPO or on a DAPO-style recipe.
W2 (1). some are trained on much less data
We would like to clarify that the strongest baseline on each backbone is trained on the same ViRL-39K dataset as VEP-RL. Under the fair evaluation protocol, VEP-RL outperforms VGPO on Qwen2.5-VL-7B, VPPO on Qwen3-VL-8B, and DAPO on Qwen3.5-9B by 2.68, 1.93, and 3.24 percentage points, respectively. For public baselines trained on smaller datasets, we evaluate their released checkpoints and state the training-data differences in Appendix B.2.1.
W2 (2). one runs with thinking disabled.
We would like to clarify that thinking mode is disabled for all evaluated methods, including VEP-RL, on Qwen3-VL-8B and Qwen3.5-9B. When describing VAD in Appendix B.2.1, we retained this detail from its official evaluation documentation. We will clarify that the same setting applies to all methods.
W2 (3). One backbone is described as using greedy decoding with several responses per question.
The originally reported results for GRPO, DAPO, and VEP-RL on Qwen3.5-9B were obtained by sampling eight responses per question with temperature 1.0 and top-p=1.0. The same sampling settings were also used for the other backbones. We apologize for the typo in Appendix B.2.1 and will correct the temperature from 0 to 1.0.
Responses to Reviewer D5R1 (Part 1/5)
We’d like to thank the reviewer for acknowledging the novelty and effectiveness of our proposed approach. We address your raised insightful points as follows.
We address your concerns about the coverage and strength of VEP guidance through the following analyses and experiments.
We would like to clarify that VEP guidance covers the majority of effective rollout groups, namely groups with nonzero advantages. The percentages in Table 6 are calculated over all sampled groups, including those with zero advantages. We support this through the following analysis and training statistics.
GRPO Advantage Analysis: GRPO computes advantages by normalizing rewards within each group. When all responses receive identical rewards (typically all correct or all incorrect), their advantages become zero. VL-Rethinker [1] also reports this vanishing-advantage problem: for Qwen2.5-VL-72B, the proportion of effective rollout groups decreases from approximately 40% to below 20% after training.
Empirical Validation: We count effective groups and VEP-guided groups throughout training for all three backbones. The results are presented below.
As shown in the table, only 24.8–35.2% of all sampled groups have nonzero advantages, while VEP guidance covers 72.2–78.9% of these effective groups. These results show that VEP guidance covers the majority of effective rollout groups across all three backbones.
The two reweighting steps act multiplicatively on each token’s advantage. With λtraj=λtok=0.1, trajectory-level multipliers range from 0.9 to 1.1. When 5% of response tokens are selected, their token-level multiplier is 1.095 (1+0.1×0.95), compared with 0.995 for the remaining tokens. The combined multiplier can therefore reach 1.1×1.095=1.2045, corresponding to a 20.45% increase in advantage magnitude.
We then vary each coefficient separately on Qwen3-VL-8B, keeping the other fixed at 0.1, to examine how the reweighting strength affects accuracy. All results are Acc. (%).
The GRPO baseline (λtraj=λtok=0) achieves 74.48% accuracy. As shown in the table, removing trajectory or token reweighting reduces accuracy by 1.57 and 1.19 percentage points, respectively. Increasing either coefficient to 0.2 also lowers accuracy, to 76.54% and 75.10%, respectively. Our setting (0.1, 0.1) achieves the highest accuracy among the tested settings.