- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether reinforcement learning with verifiable rewards (RLVR) can train a binary verification judge directly from an existing language model. The proposed approach applies GRPO with a binary reward determined by whether the model's final CORRECT/WRONG verdict matches the ground-truth label. The authors evaluate this approach on GAIA-derived agent trajectory auditing and MuSiQue multi-hop factual verification. The experiments report improvements over the original models, including substantial gains on trajectory auditing across multiple backbones and some transfer to real out-of-distribution agent failures. Additional analyses examine false-alarm and miss rates, task difficulty, training data scale, and reward design.
Critical strengths
- Empirical improvements across different model backbones.
The paper reports substantial improvements in trajectory verification accuracy across Qwen and Llama models.
- Analysis of different verification errors.
The paper separately reports false-alarm and miss rates and acknowledges cases where accuracy improvements mainly result from changes in decision bias rather than improved error detection.
Critical weaknesses
- A lack of innovation is a flaw sufficient to warrant the rejection of a manuscript.
Questions for authors
Why did you write such a long "Related Work" section?
Dear Reviewers,
The authors have replied to your initial comments. Please have a look and engage in the discussion.
Thanks!
Your AC