- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Mirror-Critique, a framework for training generative verifiers to improve mathematical reasoning at test time. The approach generates reference-guided critiques using an instruction-tuned model, filters them based on judgment correctness, and uses the resulting data for supervised fine-tuning followed by reinforcement learning with verifiable rewards. At inference time, the trained verifier generates multiple critiques per candidate solution and aggregates their judgments for weighted voting and selective abstention. Experiments on five mathematical reasoning benchmarks with Qwen2.5-Math 1.5B and 7B solvers demonstrate improvements in answer accuracy and an honesty metric over majority voting and several learned-verifier baselines.
Critical strengths
- Practical training pipeline. The paper presents a straightforward framework combining reference-guided critique generation, supervised fine-tuning, and verifier reinforcement learning. The approach leverages existing reference solutions and open-source models without requiring manually written critiques.
- Improvements across multiple benchmarks. The proposed method improves overall accuracy from 55.7% to 58.2% for the 1.5B solver and from 61.8% to 63.0% for the 7B solver compared with majority voting. The evaluation across five mathematical reasoning benchmarks demonstrates the effectiveness of the proposed approach.
- Joint consideration of accuracy and abstention. Beyond answer selection, the paper investigates selective abstention through an explicit utility-based honesty metric. The comparison between Mirror-SFT and Mirror-Verifier provides evidence that reinforcement learning can improve both answer selection and abstention decisions.
Critical weaknesses
- High inference cost for relatively modest improvements. The proposed method generates 16 critiques for each of 16 candidate solutions, requiring 256 additional critique generations per question. However, the improvements over the strongest learned-verifier baselines are only 0.9 and 0.7 percentage points for the 1.5B and 7B solvers, respectively. The absence of compute-matched comparisons makes it difficult to assess the practical efficiency of the approach.
- Insufficient ablation studies and limited methodological novelty. The framework combines several existing techniques, including critique-based SFT, balanced sampling, difficulty filtering, and RLVR. Although the paper compares Mirror-SFT with Mirror-Verifier, it does not isolate the individual contributions of these components. In particular, the necessity of reference-guided critique initialization over direct outcome-supervised RLVR remains unclear.
- Limited robustness analysis and evaluation of verification quality. The reported improvements over strong baselines are relatively small, without statistical significance analysis or uncertainty estimates. Furthermore, the synthetic critique quality assessment uses only 30 examples per dataset, and the honesty evaluation relies on a fixed abstention threshold despite different verifier score distributions. These limitations weaken the evidence for robust improvements in verification reliability.
Questions for authors
- Inference efficiency: Can the authors provide compute-matched comparisons with majority voting and existing verifier baselines? Specifically, under the same token or FLOP budget, would generating more solver candidates or using a scalar reward model achieve comparable performance?
- Ablation studies: What is the performance of a verifier trained directly with RLVR without critique-based SFT initialization? Additionally, how much do balanced sampling and difficulty-based filtering individually contribute to the final improvements?
- Robustness and calibration: Are the reported improvements statistically significant across multiple runs or bootstrap evaluations? When abstention thresholds are independently optimized on held-out validation data, does Mirror-Verifier still outperform competing methods at matched coverage levels?