- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies token-level credit assignment in multimodal RLVR. The authors propose Residual Visual Credit Optimization (RVCO), which uses visual perturbations to identify image-dependent tokens and redistributes the sequence-level learning signal toward them under a fixed credit budget. RVCO combines entropic routing with a residual path that preserves nonzero credit for all tokens. Experiments across multiple VLMs and benchmarks show consistent, though generally modest, improvements over prior multimodal RLVR baselines.
Critical strengths
-
I think the method is clearly formulated. RVCO cleanly separates credit budget, routing sharpness, and support preservation.
-
The evaluation is quite broad, covering multiple VLM families, model scales, reasoning benchmarks, and robustness settings.
-
The training overhead is modest, and I think acceptable. RVCO leaves the reward and advantage estimator unchanged and introduces only modest training overhead.
Critical weaknesses
-
The methodological novelty is kind of limited. The core routing operator is based on an existing LML projection, while the main contribution lies in combining several relatively standard components.
-
The gains over the strongest baselines are modest and the main evaluation relies on accuracy@8. Given that RL typically improves the pass@k performance of the model, I think acc@8 is not enough to clearly establish whether RVCO improves the underlying policy or mainly increases response diversity.
-
The conservation ablation is not fully controlled. Replacing exact mass conservation also changes the overall scale of the token weights, so the observed gain cannot be attributed to conservation alone.
-
Furthermore, the visual credit signal is only indirectly validated. Sensitivity to a perturbed image does not necessarily imply that a token is meaningfully visually grounded. I hope the authors could provide more detailed and direct validation of the visual credit signal.
Questions for authors
-
Could the authors report multi-seed results with variance estimates?
-
Does RVCO still outperform the baselines under accuracy@1 or greedy decoding?
-
Is there a direct evaluation showing that high-credit tokens correspond to visually grounded decisions?
-
Could the authors provide a scale-matched ablation for exact conservation? The current unit-mean variant also changes the global scale of the token weights by , making it unclear whether the performance difference comes from conservation itself or from a different effective update magnitude.
2 comments
I thank the authors for their detailed rebuttal and additional experiments, which address several of my empirical concerns. However, my concern regarding the methodological novelty still remains, as the proposed approach largely builds on existing techniques. I appreciate the authors' efforts but I will maintain my original rating.
We thank reviewer X5wk for recognizing RVCO's explicit formulation, broad evaluation, and modest training overhead.
RVCO's novelty lies in the constrained credit geometry, not the LML solver itself. It converts intervention-based evidence into a jointly optimized, budget-preserving, full-support token field while leaving the verifier and advantage estimator unchanged. The matched comparisons in reviewer 7qTe-W1 establish gains beyond conventional perception-aware weighting at nearly identical update magnitudes.
RVCO improves response correctness, not merely best-of-eight success. Three independent Qwen3-VL-4B training seeds yield 44.2 ± 0.3 versus 42.8 ± 0.6 sampled pass@1, alongside 45.4 ± 0.4 versus 44.1 ± 0.5 greedy accuracy. Acc@8 improves from 61.9 ± 0.4 to 63.9 ± 0.3. The 8B comparison independently shows consistent single-response gains.
These results distinguish training-seed variability from decoding variance and demonstrate improvements beyond response diversity.
Conservation fixes the allocation budget; the observed improvement is not explained by gradient amplification. Scale-matched unit-mean normalization is mathematically identical to RVCO, as shown in Appendix A.6.
The controlled 8B experiment in reviewer 7qTe-W2 instead compares distinct allocation rules under identical total credit mass. RVCO achieves 66.8 Acc@8 with a 1.04 relative pre-clip norm, versus 65.5/1.02 for shuffled weights and 63.0/1.05 for entropic-only routing. Similar update scales but different performance directly support structured credit allocation.
Independent annotations establish that high-credit positions preferentially correspond to verified visual decisions. Across 200 held-out trajectories, annotated visual facts receive 1.49× response-mean credit, compared with 0.84× for nonvisual spans. Removing their supporting regions produces a 0.31-nat/token likelihood gap versus 0.04 for nonvisual reasoning.
This separates meaningful evidence dependence from generic image corruption sensitivity and provides external validation of the credit signal without introducing token-level annotations into training.