- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The key premise of this paper is to investigate if language models that have the ability to reason can also act as their own verifier in terms of identifying and discarding `bad' reasoning steps. This would then result in reducing the reliance on an external learned verifier. Towards this goal, the paper introduces self-step rejection (SSR) to commit to or reject a sampling step given a reasoning prefix. Through experiments, it is shown that SSR improves (macro-)average accuracy while not significantly increasing token usage over multiple mathematical reasoning benchmarks and three reasoning models.
Critical strengths
-
The challenges associated with same-prefix candidate rejection - i.e., a setting where an external verifier is not used- is described in detail, and the paper further details how the proposed SSR overcomes these challenges, which are mainly related to rollout budgets and sparsity of the recoverability tail.
-
The accuracy performance of SSR achieved across three reasoning models as shown in Figure 1 is impressive, while not incurring significantly high overhead in terms of token usage. At the same time, SSR is shown to achieve comparable accuracy to full-solution scaling baselines at a fraction of token usage.
Critical weaknesses
- The clarity of the explanation of the graphs in Fig. 2 can be significantly improved. The explanation in Sec. 2.2 and Appendix D.1 basically verbalize the shapes of the graphs, but fail to provide context in terms of anchoring the graphs to perhaps a specific example text/ paragraph.
Questions for authors
-
Please see Weaknesses, above.
-
Is there a corresponding ablation on the two other reasoning models similar to that for Qwen3-14B in Table 3? Similarly for Tables 4, 5 in the Appendix. In Table 5, are the numbers also computed over three runs like in Tables 4, 6,7?
3 comments
Response to Reviewer dn8C (Part 2)
Response to Q1
See Response to W1.
Response to Q2
Yes, we provide the Table 3,4,5 results for the other two models below, all averaged over three evaluation runs.
For ablation study, we mainly performed no policy refinement for the other two models.
| Method | A25 | Olym-E | Olym-H | HMMT25 | HMMT26 | Avg. | Tok.× |
|---|---|---|---|---|---|---|---|
| R1-Distill-Llama-8B: Single | 33.3 | 36.0 | 5.0 | 30.0 | 21.2 | 25.1 | 1.00 |
| R1-Distill-Llama-8B: No policy refinement | 35.6 | 39.7 | 8.0 | 31.1 | 25.3 | 27.9 | 1.21 |
| R1-Distill-Llama-8B: Full SSR | 37.8 | 42.7 | 10.3 | 32.2 | 29.3 | 30.5 | 1.40 |
| Method | A25 | Olym-E | Olym-H | HMMT25 | HMMT26 | Avg. | Tok.× |
|---|---|---|---|---|---|---|---|
| gpt-oss-20b: Single | 70.0 | 79.0 | 27.0 | 50.0 | 57.6 | 56.7 | 1.00 |
| gpt-oss-20b: No policy refinement | 75.6 | 80.3 | 37.3 | 54.4 | 63.6 | 62.3 | 1.18 |
| gpt-oss-20b: Full SSR | 78.9 | 81.7 | 45.7 | 58.9 | 68.7 | 66.8 | 1.28 |
SSR deployment behavior.
| Metric | R1-Distill-Llama-8B | gpt-oss-20b |
|---|---|---|
| Candidate rejection rate | 27.0% | 23.7% |
| Candidate attempts per committed step | 1.23 | 1.31 |
| Committed steps per problem | 7.81 | 5.98 |
| Gate evaluations per problem | 9.6 | 7.8 |
| Steps reaching the -attempt cap | 0.7% | 0.9% |
| Problems exhausting the -rejection budget | 3.9% | 3.3% |
Wall-clock time per evaluation run, with generation on six H200 GPUs.
| Method | R1-Distill-Llama-8B wall-clock time (h) | gpt-oss-20b wall-clock time (h) |
|---|---|---|
| Single pass | 0.27 | 0.13 |
| SSR | 0.39 | 0.16 |
| ESC () | 1.59 | 0.56 |
| Majority@8 | 1.86 | 0.79 |
| USC | 1.91 | 0.80 |
We will add the results to the paper.
Thank You
Thank you for your detailed response. I have updated my score to reflect this.
Response to Reviewer dn8C (Part 1)
We sincerely thank the reviewer for their careful review and valuable feedback.
Response to W1
We thank the reviewer and will add annotation for Figure 2 accordingly.
Figure 2(a). The problem requires xi<1; the reference answer is 11 because negative terms are unbounded. The trajectory gradually reinterprets the constraint. It first notes that negative xi are unbounded and wonders whether ∣xi∣<1 was intended (0–887 tokens, V^: 0.89→0.84). It tests n=1,2 and reaches contradictions (→0.75), then proposes an "implicit constraint ∣xi∣<1" (→0.59). V^ rises temporarily when it re-reads the original statement (→0.72), falls again when it concludes the problem is "mistranslated" and adopts ∣xi∣<1 (→0.59), and declines to 0.19 as it derives n≥20 under the wrong constraint; it finally answers 20. No single paragraph produces a statistically resolved drop. The degradation accumulates as the model commits step by step to a misreading, which is what root-relative first-passage labels are designed to capture. We note in the caption that this trajectory was selected as a failure case on a problem the model usually solves (root 0.89).
Figure 2(b). The shared prefix is at 887 tokens of the same trajectory, where the model starts testing small n. The two low-value candidates (7/16 and 8/16) both state that the problem "should be ∣xi∣<1"; high-value candidates (15/16 and 16/16) instead carry out the case analysis and derive ∑∣xi∣−∣S∣=2⋅min(P,N) without changing the constraint. The low-value tail thus corresponds to candidates that commit to the misreading early.
We will revise §2.2–2.3 and the caption accordingly, and add the tables below to Appendix D.1.
Evaluations of the Figure 2(a) trajectory (Qwen3-14B, OlymMATH-HARD-2: xi<1, ∑i∣xi∣=19+∣∑ixi∣, minimum n; reference answer 11). V uses 64 continuations at the root and 32 elsewhere; intervals are 95% normal approximations. Dt=V0−Vt. The barrier (1−τ)V0=0.623 is first crossed at 1,735 tokens. Each row summarizes the text between the previous boundary and this one.
Candidates from the Figure 2(b) prefix (887 tokens into the trajectory in the preceding table), 16 continuations each.