- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
Large reasoning models often overthink easy questions and still spend a full budget on questions they cannot solve. This paper argues that difficulty is the wrong signal for test-time compute. What matters is whether further reasoning can still pay off under the current model. The authors propose Budget-Efficient Thinking (BET), which learns this distinction with a supervised cold-start followed by GRPO. It produces three behaviors: a short answer on easy questions, an early abstention on questions the model cannot solve, and a longer solution only when more reasoning can still change the outcome. Across three models and seven benchmarks, it reports a large drop in thinking tokens, with accuracy held or slightly improved, including on tasks outside the training domain.
Critical strengths
1. A new way to allocate test-time compute. The paper separates how hard a question looks from whether the current model can still solve it. Easy questions need little reasoning, unsolvable ones should be dropped early, and hard but solvable ones still deserve a long trace. This is the most consequential point for my assessment, because it moves efficient reasoning beyond compressing everything or scaling budget by difficulty. The experiments show that BET abstains mainly on questions the base model cannot solve while keeping most of the reasoning budget for solvable ones. The ablations show that abstention and shorter reasoning come from different reward terms, rather than from a single global length penalty.
2. The efficiency gain is broad and not explained by refusal alone. On MATH500 and AMC-23, almost no questions fall into the never-solved group, yet BET cuts thinking tokens by roughly half without losing accuracy. The saving therefore also comes from shorter reasoning on questions the model can answer. The same trade-off holds on GPQA, MuSR, and LSAT-AR, even though training uses only mathematics, with tokens falling by about half and accuracy maintained. This matters because it suggests that BET learns when further reasoning pays off, rather than a math-specific shortcut or an aggressive refusal policy.
Critical weaknesses
1. The evidence for better compute allocation rests heavily on abstention. BET declines to answer often, with final fold rates of roughly 25–35% on Omni-Math and GPQA-Diamond (Fig. 4(c)). About half of the Omni-Math token reduction comes from questions the base model never solves. Without the fold reward, BET falls behind DR.SAF in both accuracy and tokens on Omni-Math and MATH500. The headline saving therefore mostly reflects abstention, not better budget allocation among the questions the model attempts.
2. The cold-start relies on the signal the paper argues against. The paper's central claim is that solvability depends on the current policy and cannot be read from difficulty labels. Yet Stage 1 sets the solvability field from the dataset difficulty label, and marks a question unsolvable after a single failed sample. By the paper's own argument, one sample mislabels many solvable questions, and these fold demonstrations make up about 35% of the cold-start data. After this stage, accuracy drops sharply (63.3% to 35.3% on AIME-25), so RL starts from a weakened policy with a strong difficulty-based fold prior. Without a run that removes the cold-start or labels it with multiple rollouts, the three behaviors cannot be clearly attributed to the solvability-aware reward.
Questions for authors
Clarification requests
-
Among the cold-start fold demonstrations, how many questions are solved at least once by the base model in 16 rollouts? And how does BET perform without the cold-start, or with fold labels based on multiple rollouts and no difficulty label?
-
On MATH500, only a small fraction of questions are never solved by the base model, yet removing the fold reward nearly doubles thinking tokens (3.3K to 6.2K). Why does abstention affect length on questions that the model rarely folds on? The answer would clarify whether the reward terms act separately, as the paper claims.
-
How were δ, β, and p selected? Table 9 is computed on the test benchmarks.
-
The related work omits several relevant studies on adaptive reasoning. The authors should consider discussing the following papers.
- Think Only When You Need with Large Hybrid-Reasoning Models
- SuCo (Sufficiency-guided Continuous Adaptive Reasoning)
- AdaCoT (Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning)
- AdaptThink (Reasoning Models Can Learn When to Think)
Suggestion for future work
A fold is most useful when it hands a query to a stronger model. Evaluating BET in such a cascade would show whether its abstentions route the right queries upward, so that the full system answers more questions correctly while spending less compute overall.
7 comments
Response to Reviewer 41st (2/6)
W2 / Q1. The cold start relies on the signal the paper argues against: the Solvability field comes from difficulty labels, a single failed sample marks a question unsolvable, these fold demonstrations make up about 35% of the data, and accuracy drops sharply after Stage 1. How many fold demonstrations are solved at least once by the base model in 16 rollouts? How does BET perform without the cold start, or with fold labels from multiple rollouts and no difficulty label?
R2. Thank you for this comment. The paper's claim is that the criterion for abstention must come from current-policy rollouts, and the Stage 1 labels do not carry this criterion: D.1 states that "Stage 1 supplies format and behavioral coverage, and Stage 2 supplies the criterion," and ŝ(x) and ĉ∗(x) are estimated online at every step of Stage 2 (§3.2). The accuracy drop after Stage 1 is also recovered in Stage 2, ending above vanilla (§4.6). For your three questions, we re-check the fold demonstrations and add two cold-start experiments.
- Of the fold demonstrations, 2,890 queries (41.7%) are solved at least once by the base model in 16 rollouts. The 16 rollouts serve only to decide whether a query has been solved. In an independent evaluation, the base model answers 24.0% of these queries, so a single sample fails on about 76% of them; these are exactly the queries that single-sample labeling most often gets wrong, and the reason §3.2 uses online estimation instead. In the same evaluation, the Stage-1 checkpoint answers 8.7% and BET 28.6%, above the base model; since all these queries are in the training set, these numbers measure how far Stage 2 rewrites the cold-start labels.
- Without a cold start, the policy never abstains. The vanilla policy emits neither the
<predict>block nor<Unsolvable>, so the Stage 2 reward cannot be computed on it. The closest setting is the format-only cold start SFT-format: the same 20K queries and template, the single base sample of each query as its demonstration with incorrect samples kept as they are, no fold demonstrations, and a constant Solvability. After Stage 2 it never folds, with results close to w/o R_fold in Table 8, so the abstention action must be introduced by the cold start. - A cold start labeled by 16 rollouts without difficulty labels reaches the same result after Stage 2. We built this cold start as you proposed (Stage-1-R16): the 4,041 queries on which all 16 rollouts fail become fold demonstrations, every other query takes one correct rollout as its demonstration, Solvability is the 16-rollout pass rate, and Budget is the mean length of the shortest 30% of correct rollouts (the definition of ĉ∗ in Stage 2), with no difficulty labels. To test whether the criterion comes from Stage 2, we also add a cold start from the opposite end with random folds (SFT-random): the 6,931 fold demonstrations are kept but assigned to random queries, independent of sampling outcome and difficulty. The table summarizes these experiments, with per-benchmark results in Table 14 of the revised App. D.4; accuracy and tokens are Math and General averages, and fold rates on Omni-Math are grouped by the vanilla pass rate ŝ₀.
Response to Reviewer 41st (3/6)
R2 (continued)
| Policy (Qwen3-4B) | Math: Acc / Tok | General: Acc / Tok | Fold @ ŝ₀=0 | Fold @ ŝ₀>0 |
|---|---|---|---|---|
| Vanilla | 73.6 / 10.5K | 62.4 / 6.6K | – | – |
| (1) Original cold start, SFT only (Stage-1-only) | 56.9 / 9.1K | 54.9 / 5.6K | 71.7% | 9.3% |
| (2) SFT-format, SFT only | 71.1 / 10.2K | 58.3 / 6.4K | 0 | 0 |
| (3) SFT-random, SFT only | 54.1 / 7.8K | 44.9 / 4.9K | 25.3% | 24.5% |
| (4) Stage-1-R16, SFT only | 70.4 / 9.3K | 57.6 / 5.7K | 66.2% | 4.7% |
| (1) + Stage 2 (full BET) | 74.9 / 4.6K | 63.5 / 2.9K | 85.7% | 0.7% |
| (2) + Stage 2 | 73.7 / 9.1K | 62.1 / 5.4K | 0 | 0 |
| (3) + Stage 2 | 74.5 / 4.8K | 63.2 / 3.0K | 83.6% | 1.1% |
| (4) + Stage 2 | 75.1 / 4.6K | 63.4 / 3.0K | 86.1% | 0.7% |
- The label quality affects only the starting point and the speed of convergence; the final behavior is set by Stage 2. After SFT alone, Stage-1-R16 reaches 70.4 in Math accuracy, 13.5 points above the original cold start. The original labels every query the base model fails once as a fold, so the 2,890 solvable hard queries above have no long-reasoning demonstrations, which accounts for most of the drop on AIME-25 (63.3% to 35.3%); 16 rollouts keep correct long reasoning for them. After the same Stage 2, the two stay within one standard deviation on all four benchmarks with standard deviations in App. C, and Stage-1-R16 converges about 20 steps faster. After SFT alone, the folds of SFT-random are unrelated to solvability; after Stage 2 they likewise concentrate on never-solved queries, within run-to-run standard deviation of full BET, though convergence is slower (about 220 vs. 160 steps).
Taken together, the cold start supplies the abstention action but does not transmit the fold criterion. Without a cold start the policy does not abstain; with one, whether the fold labels come from a single sample, 16 rollouts, or random assignment, the fold behavior and efficiency after the same Stage 2 agree within run-to-run noise, and the label quality affects only the starting point and the speed of convergence. BET's final fold criterion therefore comes from the online estimation of Stage 2 on current-policy rollouts, not from the difficulty labels of the cold start.
Revision. New App. D.4 gives the setup of this ablation, per-benchmark results (Table 14), and the re-check of the fold demonstrations.
Response to Reviewer 41st (1/6)
Thank you for the detailed review and for recognizing that the paper "moves efficient reasoning beyond compressing everything or scaling budget by difficulty" and that its efficiency gain is "not explained by refusal alone." We group your comments into six parts and respond to each in turn.
R1. Thank you for this analysis. BET allocates budget by the expected return of further reasoning under the current policy (§3.1), and nice fold and hero call come from the same judgment: compute beyond the boundary is reclaimed, and worthy queries within it receive deeper reasoning. Folds are always scored as wrong (§4.2), so accuracy gains come only from answered queries. To separate abstention from allocation, we evaluate on all seven benchmarks the queries vanilla solves, on which BET almost never folds: on these queries, BET cuts tokens by 47% on average (57% on all queries) and raises accuracy by 1.3 points on average.
The table splits each benchmark into easy, worthy, and unsolvable queries by 16 vanilla rollouts (§3.5); solved queries are the easy and worthy ones (Table 13 in the revision). Token ratios are relative to vanilla, and entries average five evaluations (Qwen3-4B, δ=0.10).
Revision. App. B adds the savings, allocation, and accuracy on solved queries across the seven benchmarks (Table 13).