acceptodds
Under review as a conference paper at ICLR 2027

Nice Fold or Hero Call: Learning Budget-Efficient Thinking under Policy-Dependent Solvability

Abstract

Large reasoning models (LRMs) improve problem solving through extended reasoning, but often misallocate test-time compute. Existing efficiency methods reduce cost by compressing reasoning traces or conditioning budget on perceived difficulty, yet read the resulting pass rate as a difficulty score, leaving the zero-return regime unmodeled. As a result, they overspend on queries beyond the model's capability while compressing hard-but-solvable ones that need deeper reasoning. In this work, we formulate adaptive reasoning as a computational investment under uncertainty, where budget follows the expected return of reasoning rather than perceived difficulty. To instantiate this principle, we propose **B**udget-**E**fficient **T**hinking (**BET**), a two-stage framework that combines behavioral cold-start with GRPO under an investment-cost-aware reward. By aligning solve-or-fold decisions with rollout-derived solvability, BET learns three behaviors: (1) *short solve*, answering easy queries concisely; (2) *nice fold*, abstaining early when continued reasoning has near-zero expected return; and (3) *hero call*, preserving sufficient compute for hard-but-solvable queries. Across seven benchmarks and three base models, BET reduces reasoning tokens by **54%** while improving up to **3.2%** accuracy, and transfers zero-shot to scientific QA and logical reasoning with comparable efficiency gains. Code is available at https://anonymous.4open.science/r/BET-CCEC/

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Language Models, Test-Time Compute Allocation, Reinforcement Learning, Adaptive Reasoning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_41stOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

Large reasoning models often overthink easy questions and still spend a full budget on questions they cannot solve. This paper argues that difficulty is the wrong signal for test-time compute. What matters is whether further reasoning can still pay off under the current model. The authors propose Budget-Efficient Thinking (BET), which learns this distinction with a supervised cold-start followed by GRPO. It produces three behaviors: a short answer on easy questions, an early abstention on questions the model cannot solve, and a longer solution only when more reasoning can still change the outcome. Across three models and seven benchmarks, it reports a large drop in thinking tokens, with accuracy held or slightly improved, including on tasks outside the training domain.

Critical strengths

1. A new way to allocate test-time compute. The paper separates how hard a question looks from whether the current model can still solve it. Easy questions need little reasoning, unsolvable ones should be dropped early, and hard but solvable ones still deserve a long trace. This is the most consequential point for my assessment, because it moves efficient reasoning beyond compressing everything or scaling budget by difficulty. The experiments show that BET abstains mainly on questions the base model cannot solve while keeping most of the reasoning budget for solvable ones. The ablations show that abstention and shorter reasoning come from different reward terms, rather than from a single global length penalty.

2. The efficiency gain is broad and not explained by refusal alone. On MATH500 and AMC-23, almost no questions fall into the never-solved group, yet BET cuts thinking tokens by roughly half without losing accuracy. The saving therefore also comes from shorter reasoning on questions the model can answer. The same trade-off holds on GPQA, MuSR, and LSAT-AR, even though training uses only mathematics, with tokens falling by about half and accuracy maintained. This matters because it suggests that BET learns when further reasoning pays off, rather than a math-specific shortcut or an aggressive refusal policy.

Critical weaknesses

1. The evidence for better compute allocation rests heavily on abstention. BET declines to answer often, with final fold rates of roughly 25–35% on Omni-Math and GPQA-Diamond (Fig. 4(c)). About half of the Omni-Math token reduction comes from questions the base model never solves. Without the fold reward, BET falls behind DR.SAF in both accuracy and tokens on Omni-Math and MATH500. The headline saving therefore mostly reflects abstention, not better budget allocation among the questions the model attempts.

2. The cold-start relies on the signal the paper argues against. The paper's central claim is that solvability depends on the current policy and cannot be read from difficulty labels. Yet Stage 1 sets the solvability field from the dataset difficulty label, and marks a question unsolvable after a single failed sample. By the paper's own (1−s)K(1-s)^K argument, one sample mislabels many solvable questions, and these fold demonstrations make up about 35% of the cold-start data. After this stage, accuracy drops sharply (63.3% to 35.3% on AIME-25), so RL starts from a weakened policy with a strong difficulty-based fold prior. Without a run that removes the cold-start or labels it with multiple rollouts, the three behaviors cannot be clearly attributed to the solvability-aware reward.

Questions for authors

Clarification requests

  1. Among the cold-start fold demonstrations, how many questions are solved at least once by the base model in 16 rollouts? And how does BET perform without the cold-start, or with fold labels based on multiple rollouts and no difficulty label?

  2. On MATH500, only a small fraction of questions are never solved by the base model, yet removing the fold reward nearly doubles thinking tokens (3.3K to 6.2K). Why does abstention affect length on questions that the model rarely folds on? The answer would clarify whether the reward terms act separately, as the paper claims.

  3. How were δ, β, and p selected? Table 9 is computed on the test benchmarks.

  4. The related work omits several relevant studies on adaptive reasoning. The authors should consider discussing the following papers.

  • Think Only When You Need with Large Hybrid-Reasoning Models
  • SuCo (Sufficiency-guided Continuous Adaptive Reasoning)
  • AdaCoT (Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning)
  • AdaptThink (Reasoning Models Can Learn When to Think)

Suggestion for future work

A fold is most useful when it hands a query to a stronger model. Evaluating BET in such a cascade would show whether its abstentions route the right queries upward, so that the full system answers more questions correctly while spending less compute overall.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 41st (1/6)

Thank you for the detailed review and for recognizing that the paper "moves efficient reasoning beyond compressing everything or scaling budget by difficulty" and that its efficiency gain is "not explained by refusal alone." We group your comments into six parts and respond to each in turn.

W1. The evidence for better compute allocation rests heavily on abstention: fold rates are about 25–35% on Omni-Math and GPQA-D, about half of the Omni-Math saving comes from questions the base model never solves, and without the fold reward BET falls behind DR.SAF on Omni-Math and MATH500. The headline saving mostly reflects abstention, not better allocation among attempted questions.

R1. Thank you for this analysis. BET allocates budget by the expected return of further reasoning under the current policy (§3.1), and nice fold and hero call come from the same judgment: compute beyond the boundary is reclaimed, and worthy queries within it receive deeper reasoning. Folds are always scored as wrong (§4.2), so accuracy gains come only from answered queries. To separate abstention from allocation, we evaluate on all seven benchmarks the queries vanilla solves, on which BET almost never folds: on these queries, BET cuts tokens by 47% on average (57% on all queries) and raises accuracy by 1.3 points on average.

  • BET learns a sharp boundary. BET folds on 85.7% of the queries where all 16 vanilla rollouts fail, on 5.6% in the lowest-solvability bin, and almost never once solvability exceeds 0.2 (Fig. 3(b)); during training, its fold rate on queries vanilla solves falls from 9.3% to 0.7% (Fig. 4(c)).
  • This boundary lets BET keep its reasoning on worthy queries and answer more of them. BET compresses easy and unsolvable queries to 0.44 and 0.08 of the vanilla budget while keeping 0.70 on worthy ones (Fig. 3(c)), and gains a net 48 correct answers on worthy queries, the most of any method (Fig. 4(b)). Baselines without this boundary still spend 0.76–0.99 of the vanilla budget on unsolvable queries, and DR.SAF compresses worthy queries to 0.61 and loses a net 3 correct answers (Fig. 4(a)(b)).
  • Removing the boundary also degrades allocation on solvable queries, so the comparison should be made with full BET. Without the fold gate, the Omni-Math token ratio rises from 0.44 to 0.74 on easy queries and from 0.70 to 0.88 on worthy ones, and accuracy falls from 55.6% to 53.4% (Fig. 7, Table 8); without solvability calibration, compression no longer follows the current policy's solvability, and Omni-Math accuracy falls 0.7 points below vanilla (Fig. 5(b)). The boundary thus matters beyond the folded queries (mechanism in R3). w/o R_fold is a diagnostic variant with the boundary removed; full BET is more accurate than DR.SAF with fewer tokens on the Math and General averages of all three backbones (Tables 1 and 5).

The table splits each benchmark into easy, worthy, and unsolvable queries by 16 vanilla rollouts (§3.5); solved queries are the easy and worthy ones (Table 13 in the revision). Token ratios are relative to vanilla, and entries average five evaluations (Qwen3-4B, δ=0.10).

BenchmarkFold rateToken cut: allToken cut: solvedToken ratio: easyToken ratio: worthyAcc. change on solved
Omni-Math25.3%56%39%0.440.70+1.4
MATH5007.2%51%46%0.480.75+1.2
AMC-23061%61%0.270.53+1.5
AIME-2526.7%56%40%0.450.76+1.9
GPQA-D27.3%42%19%0.670.93+1.3
MuSR11.4%80%79%0.150.25−0.7
LSAT-AR9.6%50%44%0.450.67+2.7
Average15.4%57%47%0.420.66+1.3
  • On queries unrelated to abstention, BET also compresses heavily and leaves more of the budget to worthy queries. AMC-23 has no never-solved queries, so BET never folds there, yet it cuts tokens by 61% and raises accuracy by 1.5 points. On all seven benchmarks, BET keeps more budget on worthy than on easy queries (0.66 vs. 0.42 on average), and accuracy on solved queries rises on six of them. Abstention contributes most where many queries are beyond reach, as on GPQA-D, where nearly a third are never solved. BET's savings come mainly from allocation on solvable queries, with abstention reclaiming the compute beyond the boundary; both are set by the same boundary, consistent with your observation on MATH500 and AMC-23 in Strength 2.

Revision. App. B adds the savings, allocation, and accuracy on solved queries across the seven benchmarks (Table 13).

AuthorsOpenReview ↗

Response to Reviewer 41st (2/6)

W2 / Q1. The cold start relies on the signal the paper argues against: the Solvability field comes from difficulty labels, a single failed sample marks a question unsolvable, these fold demonstrations make up about 35% of the data, and accuracy drops sharply after Stage 1. How many fold demonstrations are solved at least once by the base model in 16 rollouts? How does BET perform without the cold start, or with fold labels from multiple rollouts and no difficulty label?

R2. Thank you for this comment. The paper's claim is that the criterion for abstention must come from current-policy rollouts, and the Stage 1 labels do not carry this criterion: D.1 states that "Stage 1 supplies format and behavioral coverage, and Stage 2 supplies the criterion," and ŝ(x) and ĉ∗(x) are estimated online at every step of Stage 2 (§3.2). The accuracy drop after Stage 1 is also recovered in Stage 2, ending above vanilla (§4.6). For your three questions, we re-check the fold demonstrations and add two cold-start experiments.

  • Of the fold demonstrations, 2,890 queries (41.7%) are solved at least once by the base model in 16 rollouts. The 16 rollouts serve only to decide whether a query has been solved. In an independent evaluation, the base model answers 24.0% of these queries, so a single sample fails on about 76% of them; these are exactly the queries that single-sample labeling most often gets wrong, and the reason §3.2 uses online estimation instead. In the same evaluation, the Stage-1 checkpoint answers 8.7% and BET 28.6%, above the base model; since all these queries are in the training set, these numbers measure how far Stage 2 rewrites the cold-start labels.
  • Without a cold start, the policy never abstains. The vanilla policy emits neither the <predict> block nor <Unsolvable>, so the Stage 2 reward cannot be computed on it. The closest setting is the format-only cold start SFT-format: the same 20K queries and template, the single base sample of each query as its demonstration with incorrect samples kept as they are, no fold demonstrations, and a constant Solvability. After Stage 2 it never folds, with results close to w/o R_fold in Table 8, so the abstention action must be introduced by the cold start.
  • A cold start labeled by 16 rollouts without difficulty labels reaches the same result after Stage 2. We built this cold start as you proposed (Stage-1-R16): the 4,041 queries on which all 16 rollouts fail become fold demonstrations, every other query takes one correct rollout as its demonstration, Solvability is the 16-rollout pass rate, and Budget is the mean length of the shortest 30% of correct rollouts (the definition of ĉ∗ in Stage 2), with no difficulty labels. To test whether the criterion comes from Stage 2, we also add a cold start from the opposite end with random folds (SFT-random): the 6,931 fold demonstrations are kept but assigned to random queries, independent of sampling outcome and difficulty. The table summarizes these experiments, with per-benchmark results in Table 14 of the revised App. D.4; accuracy and tokens are Math and General averages, and fold rates on Omni-Math are grouped by the vanilla pass rate ŝ₀.
AuthorsOpenReview ↗

Response to Reviewer 41st (3/6)

R2 (continued)

Policy (Qwen3-4B)Math: Acc / TokGeneral: Acc / TokFold @ ŝ₀=0Fold @ ŝ₀>0
Vanilla73.6 / 10.5K62.4 / 6.6K––
(1) Original cold start, SFT only (Stage-1-only)56.9 / 9.1K54.9 / 5.6K71.7%9.3%
(2) SFT-format, SFT only71.1 / 10.2K58.3 / 6.4K00
(3) SFT-random, SFT only54.1 / 7.8K44.9 / 4.9K25.3%24.5%
(4) Stage-1-R16, SFT only70.4 / 9.3K57.6 / 5.7K66.2%4.7%
(1) + Stage 2 (full BET)74.9 / 4.6K63.5 / 2.9K85.7%0.7%
(2) + Stage 273.7 / 9.1K62.1 / 5.4K00
(3) + Stage 274.5 / 4.8K63.2 / 3.0K83.6%1.1%
(4) + Stage 275.1 / 4.6K63.4 / 3.0K86.1%0.7%
  • The label quality affects only the starting point and the speed of convergence; the final behavior is set by Stage 2. After SFT alone, Stage-1-R16 reaches 70.4 in Math accuracy, 13.5 points above the original cold start. The original labels every query the base model fails once as a fold, so the 2,890 solvable hard queries above have no long-reasoning demonstrations, which accounts for most of the drop on AIME-25 (63.3% to 35.3%); 16 rollouts keep correct long reasoning for them. After the same Stage 2, the two stay within one standard deviation on all four benchmarks with standard deviations in App. C, and Stage-1-R16 converges about 20 steps faster. After SFT alone, the folds of SFT-random are unrelated to solvability; after Stage 2 they likewise concentrate on never-solved queries, within run-to-run standard deviation of full BET, though convergence is slower (about 220 vs. 160 steps).

Taken together, the cold start supplies the abstention action but does not transmit the fold criterion. Without a cold start the policy does not abstain; with one, whether the fold labels come from a single sample, 16 rollouts, or random assignment, the fold behavior and efficiency after the same Stage 2 agree within run-to-run noise, and the label quality affects only the starting point and the speed of convergence. BET's final fold criterion therefore comes from the online estimation of Stage 2 on current-policy rollouts, not from the difficulty labels of the cold start.

Revision. New App. D.4 gives the setup of this ablation, per-benchmark results (Table 14), and the re-check of the fold demonstrations.

Reviewer_vutnOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper frames adaptive reasoning as a compute investment under policy-dependent solvability. BET combines behavioral SFT cold-start with GRPO using current-policy rollouts to estimate success probability and efficient solution cost. Its asymmetric reward encourages concise correct solutions on easy queries, preserves reasoning depth on hard-but-solvable queries, and rewards abstention when all group rollouts fail. On seven benchmarks and three backbones, BET reports substantial reductions in think tokens with maintained or improved accuracy, including zero-shot transfer from math training to scientific QA and logical reasoning.

Critical strengths
  1. The paper separates difficulty from current-policy solvability and gives the distinction an operational role in training. The asymmetric fold reward, query-adaptive efficiency shaping, and solvability/budget calibration have complementary ablations, which support the proposed mechanism.
  2. The evaluation spans seven benchmarks, three model backbones, and eight baselines. At δ=0.10, reported math-average think-token savings are 48–57% across backbones while math accuracy increases by 1.3–3.2 percentage points.
Critical weaknesses
  1. The K=16 all-failure event is only a proxy for near-zero solvability. Appendix G.4 usefully reports nested-budget flips (19.31% for K=8→16, 0.43% for K=16→24, and none for K=24→32) and acknowledges the residual low-solvability tail. However, it does not report how many later-discovered successes are actually suppressed by BET’s fold decisions or the resulting missed-answer rate. This limits the strength of the “preserves hard-but-solvable reasoning” claim.
  2. The paper does not specify whether the pre-reasoning Budget prediction is enforced as a decoding cap or is a calibration/training signal only. Reported cost counts tokens inside the think block, so the efficiency result is meaningful but does not establish total completion-token or latency savings under an explicit runtime budget controller.
Questions for authors
  1. Among queries with s_hat=0 at K=16, how many produce a correct rollout at K=24 or 32, and how often does BET fold on these queries? Please report the resulting missed-answer count or rate; the aggregate flip rate alone does not show its decision impact.
  2. Is the predicted Budget field used to impose a hard inference-time token limit? If not, please clarify that BET learns shorter generation/abstention rather than runtime budget enforcement, and report total completion tokens or latency if available.
4 comments
AuthorsOpenReview ↗

Response to Reviewer vutn (1/2)

Thank you for the detailed review and for recognizing that the paper gives this distinction an operational role in training, with complementary ablations of the reward terms. We have added the corresponding measurements for both of your comments.

W1 / Q1. The K=16 all-failure event is only a proxy for near-zero solvability. G.4 reports nested-budget flips but not how many later-discovered successes are suppressed by BET's fold decisions. Among queries with ŝ=0 at K=16, how many produce a correct rollout at K=24 or 32, and how often does BET fold on these queries? Please report the missed-answer count or rate.

R1. Thank you for this question. By your definition, the missed-answer count is 0 for BET at δ=0.05 and 1 at δ=0.10. The nested sampling of G.4 covers all queries of MATH500, AIME-25, AMC-23, and GPQA-D for Qwen3-4B and DeepSeek-7B; among the zero labels at K=16, only 0.43% gain a correct rollout at K=24, which is a single query, and none are added at K=32 (Fig. 5(c)). On this query, BET at δ=0.05 enters reasoning in all five evaluations and answers it correctly once, while BET at δ=0.10 folds in one of the five. Over all evaluations of the K=16 zero-label queries, the missed-answer rate at δ=0.10 is about 0.43% × 1/5 ≈ 0.09%. The revision adds this count and the answers of BET on this query to G.4.

  • This query is answered correctly only 2 times in 30 evaluations across six settings, and BET at δ=0.05 is one of the two settings that solve it. Vanilla, which never abstains, fails all five times, and BET at δ=0.10 fails in its four attempts; the other correct answer comes from CODA. Queries whose first success appears only after K=16 have very low accuracy under every method, and BET mostly attempts them, performing on par with the other methods.
  • To test the relation between folds and solvability on a larger sample, we compare the 56 queries recovered when K grows from 8 to 16. These queries fail all rollouts at K=8 and succeed by K=16, so their solvability is low but observable. Each method answers each query five times on the corresponding backbone; the table gives the correct answers and mean think tokens on the two flip intervals. No query flips between K=24 and 32.
Method8→16: correct (of 280)8→16: Tok16→24: correct (of 5)16→24: Tok
Vanilla208.8K07.4K
Length-Penalty147.0K06.9K
CODA258.4K17.6K
DR.SAF195.4K04.6K
BET (δ=0.10)256.7K04.5K
BET (δ=0.05)277.5K16.0K
  • On these 56 low-solvability queries, BET at δ=0.05 answers correctly 27 times, the most of any method. This is above vanilla's 20 and CODA's 25, and BET at δ=0.10 reaches 25 with 80% of CODA's tokens. Length-Penalty and DR.SAF also compress these queries and fall below vanilla. A fold at test time is the policy's choice when expected return is very low, not a direct mapping from the K=16 zero label; increasing K recovers fewer than 1% of zero labels, which is the basis for choosing K=16 in §4.7.

Revision. App. G.4 adds the count of these queries and how each method answers them on each flip interval (Table 16).

AuthorsOpenReview ↗

Response to Reviewer vutn (2/2)

W2 / Q2. The paper does not specify whether the Budget prediction is enforced as a decoding cap or is only a calibration signal, and reported cost counts only think tokens. If it is not a hard cap, please clarify that BET learns shorter generation and abstention rather than runtime budget enforcement, and report total completion tokens or latency.

R2. Thank you for pointing this out. Budget is not used as a decoding cap in training or evaluation, and BET's savings come from the shorter generation and abstention it learns. This design follows from the role of Budget in the reward; the revision states it where §3.3 introduces budget calibration, and adds total completion tokens and latency.

  • Budget is the planning signal for allocation; truncating at it would cut the reasoning that hero call needs and would make the budget unlearnable. Eq. 2 requires the policy to estimate its own solvability and cost before generating (§3.1), and Budget is this estimate; R_CAL calibrates it toward b∗(x) = ĉ∗(x) / L_max (Eq. 6). ĉ∗(x), the mean length of the shortest 30% of the current policy's correct rollouts, marks where the return of further reasoning starts to diminish, not a length cap for correct solutions, and R_EFF does not penalize correct solutions longer than it (Eq. 5). ℓ penalizes under-declaration twice as heavily as over-declaration (A.3), so under prediction error the declared budget stays above b∗(x), with a median b̂ / b∗ of 1.71 on Omni-Math. Under truncation, an under-declared budget would appear only as a failure, and the reward could not distinguish a reasoning error from a low budget. Without a cap, 89% of the attempting trajectories of full BET (δ=0.10) end within their declared budget (App. A.5), and about three quarters of the over-budget attempts are on worthy queries, exactly where truncation would hurt.

The table reports think tokens, total completion tokens (the <predict> block, thinking, and the final answer), and latency. Latency is measured on the same vLLM server as in A.2 (1×RTX PRO 6000) under 8 concurrent continuous requests, as the mean time per request from submission to the last token.

Policy (Qwen3-4B)Omni-Math: think / total completion / latencyGPQA-D: think / total completion / latency
Vanilla8,584 / 9,070 / 89.3 s6,519 / 6,807 / 61.8 s
BET δ=0.103,847 / 3,920 / 34.2 s3,820 / 3,881 / 33.5 s
BET δ=0.054,988 / 5,064 / 45.9 s4,607 / 4,670 / 41.6 s
  • Total completion tokens fall together with think tokens, and latency falls further. At δ=0.10, BET reduces total completion tokens by 57% and latency by 62% on Omni-Math, and by 43% and 46% on GPQA-D. In autoregressive decoding, every token reads the KV cache of the entire preceding context, and in concurrent serving the requests of a batch share the time of each decoding step; BET mainly removes the longest generations, cutting think tokens on unsolvable Omni-Math queries from 8.9K to 0.75K (Fig. 3(c)), and the mean time per output token falls from 9.8 ms to 8.7 ms.

Revision. §3.3 states that the policy learns to stay within its declared budget as it generates; new App. A.5 includes the table above (Table 12) and per-trajectory budget adherence (Table 11).

Thank you for your valuable comments. If our responses address your concerns, we would appreciate it if you would consider raising your score.

Thank you, the author answered my question and I maintain my rating.

Thank you for reading our rebuttal and for your kind reply. We are glad that our responses have answered your questions. Your questions have made the paper more precise, and the revision adds the queries whose zero labels flip under nested sampling to G.4 (Table 16) and the budget adherence, completion tokens, and latency to A.5 (Tables 11 and 12). We sincerely appreciate your careful reading and the time you have given to our work.

Reviewer_RTvAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that adaptive reasoning should allocate test-time computation according to policy-dependent solvability rather than perceived difficulty. The proposed Budget-Efficient Thinking framework first performs behavioral cold-start fine-tuning, teaching a model to issue a predicted solvability and budget and then choose among short solve, hero call, and an explicit fold. In the RL stage, it derives a group success statistic from 16 rollouts and combines a solve-or-abstain reward, a short-correct-solution bonus, and calibration losses for the predicted solvability and budget. The paper reports accuracy and think-token results on seven benchmarks and three base models.

Critical strengths
  1. The paper identifies a meaningful limitation of difficulty-aware reasoning control: a difficult query may still be worth extended reasoning, whereas another query of similar difficulty may be beyond the current policy’s capability. The distinction between difficulty and policy-dependent solvability is intuitively useful.
  2. BET provides a coherent behavioral formulation for efficient reasoning. The combination of short solve, hero call, and explicit folding gives the model a more expressive decision space than uniform length penalties.
  3. The experiment covers multiple backbones, in-domain mathematical reasoning, and zero-shot transfer benchmarks. The paper reports both accuracy and reasoning-token cost, assigns no accuracy credit to folds, and includes reward ablations and hyperparameter sensitivity analyses.
Critical weaknesses
  1. Stage 1 may introduce a strong and noisy prior toward abstention that is not isolated in the experiments. it is unclear how much of BET’s final behavior comes from this heuristic initialization rather than the proposed online solvability estimation in Stage 2.

  2. The claimed Pareto advantage is not established under matched baseline trade-offs. Reporting two BET operating points does not establish Pareto dominance over these methods. The authors should provide matched-budget comparisons or complete accuracy-token curves, with comparable hyperparameter sweeps for the tunable baselines.

Questions for authors
  1. When all KK trajectories initially declare abstention, Algorithm 2 overrides one trajectory to attempt before profiling. Is s^(x)\hat{s}(x) then computed from this modified group? If so, how should the (1−s)K(1-s)^K false-zero probability and the boundary-reliability analysis be revised? Please report the frequency of such overrides and their success rate.

  2. At inference time, is the predicted budget b^\hat{b} converted into an actual decoding constraint, such as a length cap or stopping rule?

  3. How much of BET's final fold behavior and efficiency gain is already induced by the Stage 1 cold-start data? Please provide full Stage-1-only results on the main benchmarks and compare with a matched SFT baseline using the same data and output format but without the single-sample Nice Fold labels or heuristic budget targets.

8 comments
AuthorsOpenReview ↗

Response to Reviewer RTvA (1/6)

Thank you for the detailed review and for recognizing the distinction between difficulty and policy-dependent solvability and the three behaviors. We respond to your four comments and the ethics flag in turn: the first two are addressed with new training experiments, and the last two with analysis and the corresponding statistics.

W1 / Q3. Stage 1 may introduce a strong and noisy prior toward abstention that is not isolated from the online solvability estimation of Stage 2. Please provide full Stage-1-only results and compare with a matched SFT baseline using the same data and output format but without the single-sample Nice Fold labels or heuristic budget targets.

R1. Thank you for this comment. App. D.1 states the division of labor between the two stages: "Stage 1 supplies format and behavioral coverage, and Stage 2 supplies the criterion." Stage 1 teaches the policy to write the <predict> block and to emit <Unsolvable>; the criterion for abstention, ŝ(x) and ĉ∗(x), is estimated online at every step of Stage 2 from 16 rollouts of the current policy, not taken from the single sample of the cold start (§3.2). To separate the two stages experimentally, we add the full Stage-1-only results on seven benchmarks and the matched SFT baseline you suggested, together with two further cold starts as controls.

  • The four cold starts share the same 20K queries and output template and differ only in their labels. (1) The original cold start labels folds with a single base sample and sets Solvability and Budget from difficulty labels and heuristic targets (D.1). (2) SFT-format, the matched baseline you suggested, imitates the single base sample of every query, keeping incorrect samples as they are, with no fold demonstrations, a constant Solvability, and the realized length as Budget. (3) SFT-random keeps the 6,931 fold demonstrations but assigns them to random queries, independent of sample correctness. (4) Stage-1-R16 labels folds and Solvability from 16 base rollouts, without difficulty labels. Relative to (1), (2) removes both the abstention action and the information on which queries to fold, (3) removes only the latter, and (4) replaces it with more accurate offline labels. Each is followed by the same Stage 2 with unchanged hyperparameters, steps, and data.
AuthorsOpenReview ↗

Response to Reviewer RTvA (2/6)

R1 (continued)

The table reports the four cold starts after SFT alone and after Stage 2; per-benchmark results are in Table 14 of the revised App. D.4. Accuracy and tokens are averaged over Math (four benchmarks) and General (three benchmarks); fold rates are measured on Omni-Math and split by the vanilla pass rate ŝ₀ over 16 rollouts into never-solved (ŝ₀=0) and solvable (ŝ₀>0) queries.

Policy (Qwen3-4B)Math: Acc / TokGeneral: Acc / TokFold @ ŝ₀=0Fold @ ŝ₀>0
Vanilla73.6 / 10.5K62.4 / 6.6K––
(1) Original cold start, SFT only (Stage-1-only)56.9 / 9.1K54.9 / 5.6K71.7%9.3%
(2) SFT-format, SFT only71.1 / 10.2K58.3 / 6.4K00
(3) SFT-random, SFT only54.1 / 7.8K44.9 / 4.9K25.3%24.5%
(4) Stage-1-R16, SFT only70.4 / 9.3K57.6 / 5.7K66.2%4.7%
(1) + Stage 2 (full BET)74.9 / 4.6K63.5 / 2.9K85.7%0.7%
(2) + Stage 273.7 / 9.1K62.1 / 5.4K00
(3) + Stage 274.5 / 4.8K63.2 / 3.0K83.6%1.1%
(4) + Stage 275.1 / 4.6K63.4 / 3.0K86.1%0.7%
  • Stage-1-only carries a coarse prior toward abstention, but the efficiency gain and the sharp fold boundary appear only after Stage 2. Stage-1-only already folds more on never-solved queries (71.7%) but still folds on 9.3% of solvable ones; against the matched baseline (2), it is 14.2 points lower in Math accuracy while using only 11% fewer tokens. The noise of this prior lies in low-pass-rate queries: of the 6,931 fold demonstrations, 2,890 (41.7%) are solved at least once by the base model in 16 rollouts. In an independent evaluation, the base model answers 24.0% of them, the Stage-1 checkpoint 8.7%, and full BET 28.6%, so Stage 2 rewrites these labels. The 57% token reduction, the accuracy gain, and the separation of folds into 85.7% and 0.7% all appear only after Stage 2.
  • (2) shows that the abstention action must be introduced by the cold start. SFT-format never folds after Stage 2: the policy never emits <Unsolvable>, and policy gradients reinforce only sampled actions, so this action receives no learning signal. The result is close to w/o R_fold in Table 8 (53.8 / 7.7K vs. 53.4 / 7.5K on Omni-Math), with Math tokens only 13% below vanilla.
  • (3) and (4) show that the fold criterion comes from Stage 2, while the cold-start labels affect only the starting point and the speed of convergence. After SFT alone, the folds of (3) are unrelated to solvability (24–26% across strata on Omni-Math); after the same Stage 2, they concentrate on never-solved queries, and the Math / General averages are within 0.4 / 0.3 points of full BET, on the order of run-to-run standard deviation. (4) keeps correct long traces for the 2,890 solvable hard queries above and is 13.5 points above (1) in Math accuracy after SFT alone; after Stage 2, it stays within one standard deviation of full BET on all four benchmarks with standard deviations in App. C. Convergence takes about 160 steps for (1), 220 for (3), and 140 for (4).

Taken together, the prior of Stage 1 and the online estimation of Stage 2 separate cleanly. Stage 1 supplies the abstention action: without fold demonstrations, (2) does not fold even after Stage 2. The single-sample prior that comes with it is coarse, with (1) 14.2 points below the matched baseline (2) in Math accuracy at only 11% fewer tokens after SFT alone. Nor does this prior determine the outcome: replacing the fold labels with random assignment (3) or more accurate offline labels (4) leaves the fold boundary and the efficiency gain after Stage 2 unchanged. BET's efficiency gain and fold boundary therefore come from the online estimation of Stage 2 on current-policy rollouts.

Revision. New App. D.4 gives the setup of this ablation, full results on seven benchmarks (Table 14), and the re-check of the fold demonstrations.

AuthorsOpenReview ↗

Response to Reviewer RTvA (3/6)

W2. The claimed Pareto advantage is not established under matched baseline trade-offs, and two BET operating points do not establish Pareto dominance. Please provide matched-budget comparisons or complete accuracy–token curves, with comparable hyperparameter sweeps for the tunable baselines.

R2. Thank you for this suggestion. The Pareto claim of the paper rests on pointwise dominance: on the Math and General averages of three backbones, in 47 of the 48 comparisons with the eight baselines, a BET operating point is no worse than the baseline in both accuracy and tokens (Tables 1 and 5), e.g., DEER at 75.1 / 8.3K against δ=0.05 at 75.9 / 5.8K on Qwen3-4B. The only exception is the General average of KWTQ on DeepSeek-7B, which buys fewer tokens with only 36% of vanilla accuracy. To test this conclusion over the full trade-off range of the baselines, we add two settings for each of the four tunable baselines, forming complete accuracy–token curves as you suggested.

  • The new settings vary the parameter that controls this trade-off in each original work, within the original range. (1) DR.SAF varies the compression reward δ_comp of its Eq. 4, for which the original work gives no value; the 0.2 of Table 1 was chosen by the validation sweep of A.4, and we add 0.1 and 0.4. (2) CODA varies the easy-side penalty strength α (default 0.2, ablated from 0 to 0.4); we add 0.1 and 0.4. (3) DEER varies the exit threshold λ (default 0.95, range 0.8–0.97); we add 0.85 and 0.97. (4) Length-Penalty varies α (0.05–0.4 reported); the 0.1 of Table 1 was chosen by the validation sweep, and we add 0.05 and 0.2. Each new setting of a trained method is retrained for 300 steps as in §4.1, and BET takes the three values of δ in Table 9. The table marks the Table 1 setting of each baseline.
Method (Qwen3-4B, Acc / Tok)SettingOmni-MathMATH500AIME-25
DR.SAFδ_comp = 0.154.6 / 6.4K90.0 / 5.3K64.7 / 10.6K
DR.SAFδ_comp = 0.2 (Table 1)54.1 / 5.1K89.8 / 4.4K64.0 / 9.1K
DR.SAFδ_comp = 0.452.4 / 4.0K88.7 / 3.5K61.3 / 7.3K
CODAα = 0.155.3 / 8.0K89.3 / 5.9K64.0 / 13.0K
CODAα = 0.2 (Table 1)55.0 / 6.9K89.1 / 5.1K64.7 / 11.8K
CODAα = 0.454.0 / 5.5K88.5 / 4.1K62.7 / 9.2K
DEERλ = 0.9754.9 / 8.4K89.2 / 6.1K64.7 / 12.0K
DEERλ = 0.95 (Table 1)55.3 / 8.1K89.4 / 5.2K65.3 / 10.3K
DEERλ = 0.8553.6 / 6.0K88.6 / 4.0K62.0 / 7.8K
Length-Penaltyα = 0.0553.8 / 7.5K88.4 / 6.0K62.0 / 12.3K
Length-Penaltyα = 0.1 (Table 1)52.9 / 6.6K87.4 / 5.4K60.0 / 10.9K
Length-Penaltyα = 0.250.6 / 5.2K85.6 / 4.3K56.7 / 8.8K
BETδ = 0.0256.2 / 6.1K90.8 / 4.7K66.7 / 9.4K
BETδ = 0.0555.9 / 5.0K90.8 / 4.2K66.0 / 7.7K
BETδ = 0.1055.6 / 3.8K90.0 / 3.3K64.7 / 6.1K
  • All 36 baseline points lie below the BET curve, and the most compressive setting of every baseline pays with accuracy below vanilla. Each of the 36 points (four methods, three settings, three benchmarks) is matched by a BET point with higher accuracy at no more tokens, e.g., DR.SAF at δ_comp = 0.4 reaches 52.4 / 4.0K against 55.6 / 3.8K for BET at δ=0.10. On Omni-Math, the most compressive settings reach 50.6% to 54.0%, all below the vanilla 54.5%, while BET reaches 55.6% at 3.8K.
  • The difference lies in where the savings come from. Over the seven baseline settings with regime statistics, 74% to 90% of the savings come from easy and worthy queries, against about half for BET. Moving DR.SAF from δ_comp = 0.2 to 0.4 places nearly 90% of the added compression on these two regimes, and its net loss of correct answers on worthy queries grows from 3 to about 45, consistent with its reward: δ_comp acts only on groups with accuracy above 90%, and δ_ext also rewards longer reasoning on groups below 10% (its Eq. 4). Without an action for zero-return queries, these methods move along their curves mainly by compressing solvable queries; the analysis of §4.5 holds on the full curves.

Revision. App. A.4 states the coefficients of each baseline and their sources; App. B adds these accuracy–token curves (Figure 8) and their conclusion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear AC and reviewers,

We thank the three reviewers for their time and detailed comments. BET starts from the view that the reasoning budget should follow whether further reasoning can still pay off under the current policy, rather than how hard a query looks. We are glad that the reviewers found this distinction valuable: Reviewer RTvA finds that "the distinction between difficulty and policy-dependent solvability is intuitively useful" and that the three behaviors give "a more expressive decision space than uniform length penalties"; Reviewer vutn notes that the paper "gives the distinction an operational role in training" and that the reward terms "have complementary ablations, which support the proposed mechanism"; Reviewer 41st finds that the paper "moves efficient reasoning beyond compressing everything or scaling budget by difficulty" and that the efficiency gain "is broad and not explained by refusal alone."

Revised manuscript. We have uploaded a revised PDF with all changes in blue; the main text grows from 9 to at most 10 pages. New tables and figures are numbered after the existing ones (Tables 11–17, Figure 8), so every table and figure number of the submission is unchanged. The changes are listed below, with the corresponding comments in the last column.

#Location (page)ChangeComment
1New App. D.4 (p. 22)Ablation of the cold-start labels: full Stage-1-only results on seven benchmarks, the matched SFT baseline and two further cold starts, each also after the same Stage 2 (Table 14); re-check of the fold demonstrations with 16 rolloutsRTvA W1/Q3, 41st W2/Q1
2App. A.4 (p. 16); App. B (p. 18)Coefficients of each baseline and their sources; two further settings for each of the four tunable baselines, forming accuracy–token curves with BET at three values of δ (Figure 8)RTvA W2
3§3.2 (p. 4), App. G.4 (p. 29); App. E (p. 24)The zero-label probability is at most (1−s)^K; App. E analyzes the forced attempt and reports the frequency and success rate of overrides and how abstention evolves during training (Table 15)RTvA Q1
4§3.3 (p. 5); new App. A.5 (p. 17)§3.3 states that the policy learns to stay within its declared budget as it generates; A.5 reports per-trajectory budget adherence (Table 11), total completion tokens, and latency (Table 12)RTvA Q2, vutn W2/Q2
5App. G.4 (p. 30)Queries whose zero label flips under nested rollout budgets, and how each method answers them (Table 16)vutn W1/Q1
6App. B (p. 19)Savings, allocation, and accuracy on the queries vanilla solves, on all seven benchmarks (Table 13)41st W1
7App. F.1 (p. 26)How removing the fold gate changes the way the whole policy reasons, and why41st Q2
8App. F.2 (p. 27); Table 9 (p. 28)How δ, β, and p were chosen; sensitivity on a held-out set (new Held-out columns of Table 9)41st Q3
9§2 (p. 3)LHRM, AdaptThink, AdaCoT, and SuCo41st Q4
10New App. G.5 (p. 30); App. I (p. 31)BET's folds as the routing signal of an edge–cloud cascade (Table 17, three values of δ); App. I points to G.541st Suggestion
11End of main text (p. 10)Ethics Statement: ICLR Code of Ethics; no human subjects, personal data, or new data collection; sources, licenses, and decontamination of data and models; impact of more efficient reasoning, with a pointer to App. I. Not counted toward the page limitRTvA ethics flag

Our responses to each comment are posted under each review. We are happy to answer further questions during the discussion period, and we hope the reviewers will reassess the paper in light of the revision and the responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.