- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies visual claims in multimodal reasoning traces, including those that reach correct final answers. It introduces a fixed-rollout counterfactual diagnostic that distinguishes visual sensitivity from claim persistence, showing that the two can diverge under certain RL optimizers. The proposed Persistence-Aware Credit Gating (PACG) uses residual-calibrated likelihood changes under image corruption to attenuate positive credit for persistent visual claims. Experiments across several benchmarks and backbones show performance improvements.
Critical strengths
-The paper addresses an important credit-assignment problem, that a correct final answer does not guarantee that intermediate visual claims are supported.
-PACG is a reasonably simple and conservative modification. It attenuates positive credit on selected visual claims while preserving negative advantages and other tokens.
-The optimizer-dependent findings, including the different behavior of GRPO versus DAPO and VPPO, are informative.
Critical weaknesses
-There are some novelty concerns regarding SRPO. SRPO already uses signed log-likelihood contrasts between original and masked images to modulate perception-token credit. While Table 3 ablated the residual calibration and positive-only gating within PACG, it does not show their incremental benefit over SRPO. A direct comparison using the same backbone and training settings with SRPO would help the reader to identify its contribution and real value. In addition, the overall improvements on all benchmark are quite minor compared to the baseline of DAPO and VPPO.
-The persistence metric measures whether a claim becomes less likely after the image is corrupted, but this does not mean the model would change or stop making that claim in a new response. The corruption may also affect unrelated image regions rather than the evidence for that claim. The existing controls and free-generation audit are helpful, but a stronger test would be that removing the relevant evidence and check whether the model changes its claim.
Questions for authors
See the weaknesses.
3 comments
Thank you for the response. I think my main concerns on novelty and persistence metric are solved. I will increase my score.
The response to Reviewer WnSx
Thank you very much for your careful evaluation and positive feedback. We greatly appreciate your time and efforts reviewing our manuscript, and we are glad that our revisions have addressed your core concerns regarding novelty and the persistence metric, We wish you all the best!
The response to Reviewer WnSx
Dear Reviewer,
I hope this message finds you well. We hope our responses have addressed your concerns. Your further comments are highly valuable for guiding the revision of our manuscript. Should you have any remaining questions or concerns, we welcome further discussion and look forward to your reply.
Thank you for your time and effort in reviewing our paper!