- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Stepwise Think-Critique (STC), a framework in which a single model alternates between reasoning steps and self-critiques. Each critique includes an explanation and a binary correctness label. The model is first fine-tuned on synthetic interleaved reasoning–critique data, then trained with GRPO using answer correctness, critique consistency, and format rewards. The critique reward is derived from final-answer correctness and applied only to critique tokens. Experiments on mathematical reasoning benchmarks show improved self-critique performance while maintaining reasoning accuracy comparable to the variant trained without the critique reward.
Critical strengths
-
STC integrates reasoning and step-level critique into a single model, providing explicit correctness judgments and explanations without a separate verifier.
-
The ablations show that the critique reward improves self-evaluation while largely preserving reasoning accuracy. They help distinguish the contributions of the reasoning and critique objectives.
-
The comparison with seven external PRMs uses the same reasoning trajectories and allows per-dataset oracle thresholds for the PRMs. This provides a useful assessment of STC’s self-critique performance.
Critical weaknesses
-
The main evidence for step-level critique quality relies on GPT-5-mini labels rather than human annotations. The reported agreement with ground truth is measured at the final-answer level, which does not establish the reliability of its judgments on intermediate steps.
-
The PRM comparison uses only STC-generated trajectories, with 30 problems per benchmark. Evaluation on independently generated trajectories or an established process-verification benchmark would help assess whether the reported advantage extends beyond STC’s own outputs.
-
The practical benefit of improved critique remains unclear. Adding the critique reward does not improve reasoning accuracy over the ablated model, and the paper does not demonstrate that the critiques reliably support correction or answer selection. Experiments are also limited to a single 1.5B backbone.
Questions for authors
-
Could you validate the intermediate-step labels on a human-annotated subset? Agreement on final-answer correctness may not reflect the judge’s reliability on individual reasoning steps.
-
How does STC perform on trajectories generated by other models? This would clarify whether its advantage over external PRMs extends beyond its own outputs.
-
Can STC’s critiques improve error correction or answer selection? Such an experiment would help establish their practical value beyond the reported critique metrics.