- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Self-Verifiable Policy Optimization (SVO), a framework designed to improve reasoning–answer consistency in multimodal reasoning while reducing dependence on expensive external verification. The method introduces a co-evolving discriminator trained to distinguish policy-generated verification outputs from judge-aligned references. The discriminator provides a dense consistency reward and gates answer-correctness rewards during GRPO training. Periodic updates using a frozen external judge allow the discriminator to adapt to the evolving policy. Experiments across four vision-language backbones and seven multimodal math benchmarks demonstrate improvements in reasoning–answer consistency (RAC) and consistency-constrained accuracy (CCA) over conventional outcome-based RL methods.
Critical strengths
- Written in a clear and accessible manner.
- Extensive empirical evaluation with consistent improvements.
Critical weaknesses
- Limited innovation
- Significant additional inference required
- Unreliable co-evolution; I suspect the two might end up "hacking" each other
Questions for authors
Please take a look at "weakness."