- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces Verifiability-aware Policy Optimization (VeriPO), a reinforcement-learning algorithm which assigns token-level credit by combining per-segment verifiable rewards with a token-level reverse-KL signal from a frozen vision-language teacher.
Critical strengths
-
The method tackles an important problem of credit assignment for reasoning tokens in RLVR for visual document QA.
-
Results show strong improvements across document QA and parsing benchmarks. The ablations support the benefit of combining rule rewards and teacher supervision, and examine the effects of routing and teacher quality.
Critical weaknesses
-
The benefit of credit assignment is not clearly seen from the ablation analysis. The rule-only ablation removes the teacher signal while retaining zero rule-reward weight on reasoning tokens. This is different from standard RLVR, where the final-answer reward also supervises reasoning tokens. A matched baseline that passes the outcome reward advantage to reasoning tokens is needed to show the benefit of VeriPO's credit assignment.
-
The efficiency comparison also needs to outline the 72B teacher cost. A compute-matched distillation baseline would better distinguish the benefit of VeriPO from distillation.
Questions for authors
Please refer to the Weaknesses section above.