- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how vision-language models can correct cached visual references while preserving the intended referent. It proposes Referent-Preserving Policy Optimization (RP²O), which jointly trains reference revision and downstream reasoning using separate credit signals derived from answer rewards. Controlled experiments on Qwen2.5-VL-7B show improved reference recovery and joint reference–answer correctness, while maintaining answer accuracy comparable to answer-reward GRPO.
Critical strengths
The controlled evaluation clearly distinguishes reference recovery from answer correctness, using shared starting states and frozen-executor comparisons to support the reported gains.
Critical weaknesses
-
The methodological novelty is still unclear to me. The design builds on ideas from ReMax (ICML 2024), RLOO (ACL 2024), and local credit assignment methods such as GiGPO (NeurIPS 2025). However, the main RL comparison is against standard GRPO. I would like to see these closer alternatives adapted to the same interface and compared under matched budgets. Without that, it is hard to tell how much the proposed estimator adds.
-
There is a gap between rewarding correct answers and recovering correct references. The executor can still access the original image and description, so it may answer correctly while leaving the cached reference wrong. If keeping and repairing the reference yield the same answer reward, the objective cannot distinguish them. The unchanged-state baseline does not change the expected policy gradient, so it does not by itself resolve this issue. The paper needs a clearer explanation of why this training setup reliably encourages reference recovery.
-
The ablations do not fully explain where the gains come from. In Table 2, removing the revision control still gives 71.40% JR, compared with 72.58% for the full method. This suggests that much of the improvement remains without this component. I would like to see separate tests of the within-edit execution credit, sampling structure, and advantage normalization, along with comparisons against simpler baselines. These would help establish which parts of the method are actually needed.
-
Joint training seems to involve a robustness tradeoff that needs more discussion. In Table 9, with corrupted caches, the RP²O executor reaches only 17.34% average answer accuracy, versus 42.96% for the GRPO executor. Given the same RP²O-edited references, the GRPO executor also achieves higher answer accuracy with nearly identical JR. This raises the concern that joint training may make the executor more dependent on the cache. I would like to understand what happens when editing fails and whether this tradeoff is necessary for the recovery gains.
-
Releasing the implementation would improve reproducibility and make the method easier to evaluate and adopt. While not essential for acceptance, it would strengthen the overall impression of the work.
Questions for authors
See the above weakness.