- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the Visually-bypassed Shortcut Learning problem in MLLM post-training, where multimodal models become less sensitive to image perturbations after reinforcement learning despite improved task accuracy. The authors evaluate five models on three VQA datasets and propose the Visual Information Evaluation metric, which measures reward retention under progressive image degradation. Additional experiments examine alternative RL algorithms, process-based rewards, and image captioning.
Critical strengths
- The paper addresses an important question: whether improvements in answer accuracy correspond to improvements in visual grounding.
- Experiments cover several model families and scales, with supplementary results using an alternative RL algorithm.
- Multiple perturbation families and intermediate checkpoints provide complementary views of behavioral changes during training.
Critical weaknesses
- Reduced visual sensitivity does not necessarily indicate harmful visual degradation. If questions can be directly answered accurately from linguistic priors, this is a problem or characteristic of the dataset itself, and reduced image dependence can even be a reasonable adaptation to the dataset. The key experiment should separate RL training data from an independently validated, image-essential probing set and test whether training impairs visual performance there. Appendix F’s model-defined subset and Appendix D’s detector-transfer results do not establish this distinction.
- The task coverage is limited relative to the claims. The main experiments cover three VQA datasets. This is insufficient to establish VSL as an inherent or broadly task-independent property of multimodal RL. Experiments on representative post-training tasks with stronger visual requirements, such as visual mathematical reasoning, chart/document understanding, or spatial grounding, would strengthen the conclusions.
- VIE’s practical value for model training remains unclear. The paper does not show that using VIE improves training decisions or downstream performance. A useful experiment would filter or reweight training data using VIE and compare against unfiltered data, size-matched random selection, and a simple question-only-based selection baseline under comparable compute. Evaluation should use an independent image-essential benchmark.
- The reward-variance mechanism is insufficiently justified. For binary correctness rewards, Var(R)=p(1-p), so equal success probabilities imply equal reward variance regardless of the answer pathway. Lower reward variance also does not automatically imply lower policy-gradient variance or preferential optimization by GRPO.
Questions for authors
How do you verify that a perturbation removes information necessary for the particular question, rather than information that a correctly grounded model can safely ignore?
7 comments
Response to Reviewer W2YX (Part2)
W3: VIE's practical value for model training remains unclear.
A3: (1). The purpose of VIE is to tell users, during typical RL training such as GRPO, at which step the decline in visual sensitivity has become unacceptable, even though continued training may still improve benchmark scores. Monitoring this trade-off requires a low-cost detector that does not modify the training process. VIE detects the step at which the model's dependence on visual information has dropped to a non-negligible degree, and signals that developers should adjust the training strategy or the training data, rather than blindly continuing RL training.
(2). In our current design, VIE monitoring does not affect training at all. Different domains and methods may require different training strategies, so we keep the monitor independent of the training procedure; this ensures that VIE remains usable under any condition, and can be applied to training runs with different models, datasets and RL methods.
(3). The reviewer's suggestion is nevertheless reasonable, and we conduct the proposed experiment. Using the pre-RL model, we compute the per-sample VSL-Score of each training sample and remove those samples whose reward is stably retained under visual degradation. We compare against unfiltered training, a size-matched random selection, and a question-only-based selection that removes the same number of samples answerable without the image. All variants use the same number of training steps.
| Training data | Acc ↑ | VS ↑ |
|---|---|---|
| Before RL | 0.3213 | 0.6690 |
| RL, unfiltered | 0.4218 | 0.4200 |
| RL, size-matched random | 0.4196 | 0.4235 |
| RL, question-only selection | 0.4247 | 0.4458 |
| RL, VIE-based filtering | 0.4304 | 0.5169 |
After VIE-based filtering, the trained model retains substantially more visual dependence and its accuracy on OK-VQA is essentially unchanged (0.4304 vs. 0.4218; the small gain may be due to run-to-run randomness). Question-only selection helps partially, because it removes samples that are answerable without the image but not samples whose reward is stably retained under partial degradation. This shows that VIE can also inform training decisions, beyond monitoring. We will add this experiment to the paper.
W4: The reward-variance mechanism is insufficiently justified.
A4: (1). We agree with the reviewer. For a binary reward, Var(R) = p(1 − p), so two pathways with the same success probability have the same reward variance. Moreover, under GRPO, the advantage is normalized by the reward standard deviation within each group, so a difference in reward variance alone does not imply preferential optimization. The formulation in the paper is imprecise, and we will revise it.
(2). The reward-standard-deviation gap reported in Table 7 arises because the per-sample success probability differs between the two subsets: on prior-answerable samples, the model answers consistently and p is close to 1. The mechanism we intend is not about reward variance, but about the gradient signal that the two pathways provide, which is what the other rows of Table 7 measure: gradient SNR (1.5× vs. 1.0×), gradient-direction consistency (0.41 vs. 0.19) and convergence speed (∼110 vs. ∼370 steps).
(3). To separate the two, we repeat the analysis of Table 7 on subsets of S_lang and S_vis matched in their pre-RL success probability p:
| Statistic | S_lang | S_vis | S_lang (p-matched) | S_vis (p-matched) |
|---|---|---|---|---|
| Reward std σ_R | 0.17 | 0.26 | 0.22 | 0.22 |
| Gradient SNR (relative) | 1.5× | 1.0× | 1.4× | 1.0× |
| Gradient-direction cosine consistency | 0.41 | 0.19 | 0.38 | 0.20 |
| Steps to 90% of final reward | ∼110 | ∼370 | ∼140 | ∼350 |
Once p is matched the reward variance becomes equal. The gradient-level asymmetry, however, persists: the linguistic pathway still produces a higher gradient SNR, more consistent gradient directions across rollouts, and faster convergence. The optimization preference is therefore explained by the policy-gradient signal, not by the reward variance.
Response to Reviewer W2YX (Part3)
Q1: How do you verify that a perturbation removes information necessary for the particular question?
AQ1: We examined this question from several angles.
(1). We use multiple perturbation families to study how stable the model's answers are when visual information is unavailable. For the same perturbation family, we strictly vary the degradation strength, from the weakest setting, which may still retain most of the visual information, a pure noise image that contains no usable visual content. For information-free perturbations, every piece of information necessary for the question is removed by construction, so the question of necessity does not arise. Under Gaussian noise substitution, the post-RL model still answers more questions correctly (Table 1: 0.2737 → 0.3111). This multi-level statistical analysis shows that the unchanged answers are not explained by sufficient residual visual information.
(2). For partial perturbations such as blur, we do not assume at the sample level that every perturbation removes necessary information. VS is compared on the same samples under the same perturbation before and after RL, so for a sample where blur leaves sufficient information, the comparison is controlled; the change in VS reflects a change in the model rather than in the information available. Additionally, in Appendix D, under randomly mismatched images, which contain no information relevant to the question, the post-RL model keeps its answer far more often (behavioral shortcut index 0.05 → 0.19), showing that it indeed relies on textual priors.
(3). To verify necessity directly, we curated 500 counterfactual image pairs in which the question stays the same but the correct answer changes. In these pairs, the information that distinguishes the two images is, by construction, necessary for the question.
| Metric | Before RL | After RL |
|---|---|---|
| Prior-locked rate ↓ | 4.6% | 15.8% |
| VS on counterfactual pairs ↑ | 0.412 | 0.368 |
After RL, the model gives the same question-determined answer to both images more than three times as often (4.6% → 15.8%). Under this setting, the lower VS after RL indicates both reduced reliance on visual information and greater reliance on linguistic priors.
Thank you again for your valuable suggestions. We value the opportunity to discuss our work with the research community and welcome any further questions or comments.
Response to Reviewer W2YX (Part1)
We really appreciate your time and insightful comments. Thank you so much for acknowledging notable performance gain and detailed analysis of our work. The response to your concerns are summarized as follows.
W1: Reduced visual sensitivity does not necessarily indicate harmful visual degradation.
A1: We thank the reviewer for this important distinction. We agree that, for a sample that can be answered from the question alone, answering from linguistic priors is not wrong in itself. The question that matters is whether the reliance learned on such data carries over to samples where the image is indispensable. We address this in three ways.
(1). Image-essential probe set, separate from the RL training data and independent of the model. Following the reviewer's suggestion, we train on OK-VQA as before and evaluate on data in which the image is verified to be essential without using the model under study: (a) MMStar, a human-curated benchmark whose samples are selected to be vision-indispensable; (b) a VQAv2 subset of 1,000 questions that two text-only LLMs both fail to answer from the question alone; and (c) the 500 counterfactual pairs described below, in which the answer changes with the image by construction.
While accuracy on the training distribution increases, accuracy decreases on every image-essential set, and the prior-locked rate increases. The reduced visual reliance is therefore not confined to samples where priors suffice: it transfers to, and impairs, performance on data where the image is necessary.
(2) VQAv2 is itself designed to suppress language priors: each question is paired with complementary images that lead to different answers. If reduced visual sensitivity were merely a reasonable adaptation to prior-answerable data, it should be weakest on such a dataset. Yet VS decreases on VQAv2 for all five models.
(3) Reliance on priors is driven by the optimization, not only by the dataset. On the same OK-VQA data, process-based rewards reduce the VS decline by 69% compared with outcome-only rewards. If lower visual dependence were simply a property of the data, changing the reward function on identical data would not change it. This indicates that the pathway the model adopts is shaped by the RL objective, which is the core claim of VSL.
W2: Task coverage is limited relative to the claims.
A2: (1). We thank the reviewer for this suggestion, which we had also considered. Appendix G already extends the study beyond VQA to image captioning on COCO Captions, where there is no question text to exploit. The same signature appears: the benchmark reward (CIDEr) rises from 1.08 to 1.19, while image grounding decreases (CLIPScore 0.71 → 0.68, VS_gen 0.58 → 0.42), and VIE detects the shortcut-reliant checkpoint with an AUC of 0.76.
(2) Following the reviewer's suggestion, we add experiments on visual mathematical reasoning, a task with strong visual requirements. We train Qwen3-VL-4B with GRPO on Geometry3K, in which answering requires reading lengths, angles and relations from the diagram, using the same protocol as the main experiments:
The VSL signature is present: accuracy increases while VS decreases under all three perturbations, and VIE separates the two checkpoints. The average VS drop is clearly smaller than on OK-VQA , comparable to the low-prior CLEVR setting. This matches the prediction stated in Appendix H that tasks strictly anchored in fine-grained visual structure, such as geometric reasoning, should exhibit milder rather than absent VSL, and it is consistent with our finding that VSL severity scales with the strength of task-specific linguistic priors.