acceptodds
Under review as a conference paper at ICLR 2027

Reasoning Just Enough: Learning Latent Sufficiency Boundaries for Adaptive Exit

Abstract

Large reasoning models often continue generating after sufficient information for a correct answer has already emerged, increasing inference cost and sometimes degrading accuracy. We introduce SPARE (Sufficiency Probing for Adaptive Reasoning Exit), a dynamic early-exit framework that detects a sample-dependent latent sufficiency boundary. SPARE derives anchor-relative supervision from verified answer-related expressions and trains a lightweight linear probe on multi-layer hidden states to identify answer-sufficient reasoning prefixes. GraphRACE then performs compute-efficient offline calibration through shared stopping-event evaluation, behavioral consolidation, and risk-aware Pareto selection, yielding a frozen persistence-based stopping policy. At inference time, SPARE requires only the trained probe and calibrated policy, without correctness feedback, online search, or backbone modification. Across multiple reasoning models and mathematical and scientific benchmarks, SPARE largely preserves accuracy and often improves it—by more than 15 percentage points in the strongest case—while reducing net token usage by approximately (30%)–(50%). These results show that latent sufficiency signals can support reasoning that is not maximal, but just enough.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, efficient reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_ia2pOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SPARE learns when a frozen reasoning model has enough information to produce a useful final answer. It identifies answer-related anchors in complete trajectories, verifies an anchor with a readout, and generates dense weak supervision from nearby reasoning-unit endpoints. A linear probe predicts sufficiency from concatenated hidden states. GraphRACE calibrates a threshold-and-persistence stopping policy by reusing shared stopping-event readouts and selecting representative candidate policies. At inference, the frozen policy triggers one guided final answer without online correctness verification. Experiments on four backbones and 758 evaluation examples report substantial net completion-token savings with competitive and often improved accuracy, including a 16.4-point MATH improvement for the 2B backbone.

Critical strengths
  1. The anchor-guided supervision is efficient: multiple weak labels are derived from a single verified anchor, avoiding intermediate-answer evaluation at every step. The controlled truncation study motivates the sufficiency-boundary hypothesis, and the fixed and random anchor comparisons support the labeling strategy.

  2. The stopping framework is practical: a lightweight linear probe and an offline-calibrated policy require no model modification or online verification, GraphRACE reduces calibration cost, and the token accounting includes readout overhead. The reported accuracy–efficiency results are promising.

Critical weaknesses
  1. The largest accuracy gain is not attributed. The 2B MATH result improves from 65.80% to 82.20% with about 50% fewer tokens, but the Full-Reasoning Readout control covers only 4B and 9B; DEER shows a similar gain, so adaptive stopping is not separated from answer elicitation or avoiding budget exhaustion. A matched 2B readout control, generation and scoring details, an error breakdown, and paired uncertainty are needed.

  2. The supervision assumptions and GraphRACE are under-validated. The reported AUROC of 0.851 does not measure weak-label agreement or boundary-location error, and the boundary analysis covers one model and 200 MATH problems. The assumption that pre-anchor reasoning is insufficient and post-anchor reasoning is sufficient is also not directly tested. GraphRACE's components are not isolated against simpler calibration under matched budgets.

  3. Generalization and presentation need work. All four models are Qwen-based and the tasks are mathematical and scientific, so other model families and reasoning tasks are needed.

Questions for authors
  1. Can you explain the source of the 2B MATH gain and report a matched Full-Reasoning Readout control on 2B, including correct-to-wrong and wrong-to-correct transitions, paired uncertainty or additional seeds, and a breakdown of capped, unparseable, and incorrect baseline outputs? How does DEER's similar gain affect the attribution?

  2. Beyond the reported detector AUROC, how well do the constructed weak labels agree with actual readout correctness, and how far are anchors and exits from the empirical boundaries? Can you also isolate GraphRACE's components against simpler calibration under matched budgets?

  3. Can you evaluate SPARE on non-Qwen model families such as Llama or Gemma, and on reasoning tasks beyond math and science?

3 comments

R1. The source of the 2B gains and the DEER comparison [CW1, Q1]

To examine empty-final effects, the table reports MATH examples defined by a nonempty original Vanilla final answer. FullReader applies SPARE's readout protocol after full reasoning (revised Table 12).

ModelSamplesVanilla accuracyFullReader accuracySPARE accuracySPARE token savings vs. FullReader
DeepSeek-1.5B49084.69%84.69%84.49%35.49%
Qwen3.5-2B34093.24%94.71%94.71%53.93%
Qwen3.5-4B47495.57%95.99%95.99%35.50%
Qwen3.5-9B48995.91%95.71%96.11%43.19%

On all MATH500 examples, 2B accuracy is 65.80%/85.20%/82.20% for Vanilla/FullReader/SPARE. Relative to Vanilla, SPARE's correct-to-wrong and wrong-to-correct proportions over all 500 examples are 1.40% and 17.80%.

Both DEER and SPARE dynamically select exits. DEER uses trial answers and confidence, while SPARE uses a hidden-state probe and persistence. In the main table, 2B DEER/SPARE accuracy is 71.37%/70.32%, with mean token costs of 15,313/10,074, illustrating the quality–cost trade-off between stopping signals.

R2. Weak supervision, sufficiency structure, and GraphRACE validation [CW2, Q2]

The supervision audit randomly samples 50 trajectories and 200 pre/post endpoints from the actual 4B training population, yielding readout accuracies of 43%/99%. On the same Math200 diagnostic, hidden-state and fitted position probes achieve AUROCs of 0.851 and 0.650. In the original nine-checkpoint analysis, 126 trajectories exhibit a strict single incorrect-to-correct transition and 12 differ at one checkpoint.

We also added four-model six-candidate simple calibration.

ModelSimple-calibration accuracyNet token savingsGraphRACE accuracyNet token savings
DeepSeek-1.5B64.38%37.56%64.12%39.03%
Qwen3.5-2B70.18%46.94%70.32%46.94%
Qwen3.5-4B87.60%24.56%87.20%33.90%
Qwen3.5-9B89.71%25.21%89.71%41.55%

GraphRACE adds 16.34 percentage points of token savings at equal accuracy on 9B, and 9.34 points with a 0.40-point accuracy difference on 4B. Sections 4.1–4.3 now clarify correctness, anchor labels, exit positions, and selection within the fully evaluated, empirically feasible candidate pool; Table 5 reports the budget controls.

R3. Additional tasks, model backgrounds, and greedy deployment [CW3, Q3]

We evaluated four models on 500 examples from two BBH tasks and all 164 HumanEval+ problems using frozen mathematical probes and policies. Code is graded by official execution tests.

ModelBBH accuracyToken savingsHumanEval+ Plus pass@1Token savings
DeepSeek-1.5B37.20%→40.40%51.38%59.76%→60.37%31.59%
Qwen3.5-2B56.60%→67.60%68.32%59.76%→64.02%33.80%
Qwen3.5-4B73.80%→73.60%40.67%87.80%→91.46%31.61%
Qwen3.5-9B78.00%→77.80%47.33%74.39%→90.24%59.55%

Code Plus pass@1 improves on all four models, with 31.59%–59.55% token savings. DeepSeek's R1 distillation covers reasoning patterns and post-training conditions different from native Qwen3.5. We plan to extend evaluation to Llama, Gemma, and other families.

We also generated from scratch on all 758 examples at temperature 0 in vLLM with continuous batching, concurrency 8, and CUDA Graphs. The three-model online results are as follows (new Table 4).

ModelVanilla→SPARE accuracyActual token savingsGPU-time speedup
DeepSeek-1.5B48.94%→58.84%47.49%1.91×
Qwen3.5-4B61.21%→80.87%48.54%1.80×
Qwen3.5-9B74.93%→85.88%52.25%2.05×

All three models improve accuracy and reduce tokens under greedy generation, showing benefits under fixed decoding and measured runtime acceleration.

Reviewer_ia2pOpenReview ↗

Follow-up on the Authors' Response

Thank you for your response! Regarding Q1, we agree that the accuracy improvement of the 2B model can be primarily explained by Answer Elicitation, rather than being directly attributed to Adaptive Stopping. Regarding Q2, based on your additional experiments, we acknowledge that GraphRACE is beneficial overall. Regarding Q3, your experiments demonstrate cross-task generalization. However, the proposed method is still evaluated exclusively on Qwen-based models, leaving the question of cross-model-family generalization unanswered. We would therefore be inclined to raise our score from 2 to 2.5, if such a rating were available. However, ICLR does not offer a 2.5 rating.

Thank you for your follow-up and for highlighting the remaining question about model families. We trained and calibrated SPARE for NVIDIA Llama-3.1-Nemotron-Nano-4B-v1.1, a Llama-based backbone, using the same pipeline. We evaluated both Vanilla and SPARE on all 758 problems with greedy decoding in vLLM. Token counts include answer readout.

DatasetVanilla → SPARE accuracyToken savings
MATH50090.60% → 90.80%28.32%
GPQA Diamond48.99% → 47.47%27.25%
AIME 2025 + 2026 (60 problems)56.67% → 56.67%22.13%
Overall (758 problems)77.04% → 76.78%26.87%

Summed across batches, end-to-end online generation time fell from 10.30 to 6.94 GPU-hours, a 32.65% reduction. These results extend SPARE’s accuracy–efficiency evaluation to the Llama family.

Reviewer_W7dnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to train an early-stopping probe with fewer answer checks compared to existing work. Their method finds points where an answer appears in a reasoning trace, checks whether the model can answer from that point, and uses earlier and later points as negative and positive training examples. A classifier trained on hidden states then predicts when reasoning can stop. The paper also proposes a method to reduce the cost of choosing a stopping threshold. Experiments report fewer reasoning tokens and lower supervision and calibration costs than the evaluated baselines.

Critical strengths

S1. The research question is useful. Training an early-stopping probe (as done in existing works) by generating and checking answers at many intermediate points is expensive. Reducing this cost while maintaining useful stopping behavior is a practical goal.

S2. The results show substantial token savings while achieving the stated goal.

Critical weaknesses

W1. I am not convinced that the methodological contribution is sufficiently significant relative to existing work. Hidden-state probes and learned early stopping have already been studied in works such as [1] and [2]. The main additions here are assigning many training labels from one answer check and reducing the cost of searching for the optimal point. These are useful practical changes, but don't bring much methodological novelty.

The labeling shortcut also makes assumptions that are not always valid. An answer first appearing at one point does not mean the model could not answer earlier, or that it will remain correct later. Assigning negative labels before that point and positive labels afterward can therefore introduce errors. The probe evaluation in the appendix is useful, but it does not directly show how often these assigned labels are wrong or how much those errors affect performance.

[1] Reasoning Models Know When They’re Right [2] LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning

W2. The experiments do not clearly establish whether the benefit comes from better stopping decisions or different choices of stopping thresholds. For example, on Qwen3.5-4B, SPARE achieves 87.20% accuracy with 7,392 tokens, between LYNX at confidence 0.95 (86.68%, 6,973 tokens) and confidence 0.97 (87.60%, 7,788 tokens). This looks like a similar accuracy–cost tradeoff. I would like to see Pareto curves under matched tuning budgets.

If the main benefit is cheaper preparation, the important comparison is against existing methods using smaller preparation budgets. Could LYNX achieve similar results by checking only a few endpoints per trace? Could a smaller threshold search achieve similar results to GraphRACE? The reported reductions in answer checks are meaningful. But these comparisons are needed to show that the proposed method adds value beyond simply reducing the amount of labeling and tuning.

Questions for authors

Q1. Have you compared against LYNX trained with fewer directly checked endpoints per trace, under the same answer-check budget?

Q2. Have you directly checked a sample of the labels assigned before and after the answer anchor? How often are earlier points already sufficient, and how often do later points produce incorrect answers?

Q3. Can you provide accuracy–token Pareto curves for SPARE and the strongest baselines under matched tuning budgets?

3 comments

R1. Supervision and calibration contributions [W1]

SPARE combines sufficiency learning and efficient calibration in a complete pipeline: one verified anchor expands to up to 61 supervised endpoints, a lightweight hidden-state probe makes online decisions, and GraphRACE calibrates exits through shared stopping events and behavioral representatives. Its value is measured jointly by accuracy and computation.

Under the same training pool, calibration pool, template, and scorer, auxiliary readout calls and output tokens for training and calibration assets are 1/4.23 and 1/7.18 of LYNX. The four-model training-verification-budget and simple-calibration controls in Table 5 further support the efficiency of these designs.

R2. LYNX under the same training-verification budget [W2, Q1]

Each LYNX run receives as many directly verified labels as SPARE's actual training-verification count, using cue endpoints selected without consulting their outcomes, the official MLP, and the original conformal calibration set.

ModelVerification budget per training runSPARE accuracySPARE token savingsLimited-budget LYNX accuracyLYNX token savings
DeepSeek-1.5B131564.12%39.03%58.97%37.03%
Qwen3.5-2B127670.32%46.94%56.07%54.69%
Qwen3.5-4B130587.20%33.90%74.67%44.18%
Qwen3.5-9B129589.71%41.55%78.23%58.09%

SPARE achieves higher accuracy on all four models. Limited-budget LYNX trades larger accuracy reductions for additional token savings on 2B, 4B, and 9B. Table 5 and Appendix B.4 give the settings and separate cost accounting.

R3. Weak-label audit [W1, Q2]

Using a fixed seed, we uniformly sampled 50 of the 1,095 actual 4B training trajectories and randomly selected two pre-anchor and two post-anchor endpoints each, performing 200 checks. Actual pre/post readout accuracy is 43%/99%, quantifying positive-label accuracy and pre-anchor label noise. Exit rules are subsequently calibrated on answer quality and cost. Appendix A.4 now reports the sampling and audit; Section 4.2 clarifies anchor labels as weak supervision.

R4. Simple calibration and the accuracy–cost balance [W2, Q3]

Four-model simple calibration uses the same probe, Math667, and selection rules, fully evaluating six threshold candidates within the original GraphRACE query budget.

ModelSimple-calibration accuracyNet token savingsGraphRACE accuracyNet token savings
DeepSeek-1.5B64.38%37.56%64.12%39.03%
Qwen3.5-2B70.18%46.94%70.32%46.94%
Qwen3.5-4B87.60%24.56%87.20%33.90%
Qwen3.5-9B89.71%25.21%89.71%41.55%

GraphRACE saves more tokens at equal accuracy on 9B. On 4B, a 0.40-percentage-point accuracy difference accompanies 9.34 additional points of token savings. Table 5 reports both budget controls; Appendix B.4 gives their actual query costs.

I really appreciate the authors’ time and efforts during the rebuttal. The new experiments are helpful. As a reviewer, I mainly look for new understanding/perspective of an existing problem, an important new problem overlooked earlier, or a novel/interesting solution to an existing problem. For an incremental method, I look for substantial gains from addressing limitations or incorrect assumptions in earlier approaches.

To me, this work provides a useful optimization of existing approaches, but the methodological contribution remains limited. The comparisons do not sufficiently establish substantial gains over strong baselines. I appreciate the paper’s clear presentation and practical improvements, but these do not change my overall assessment. I will maintain my score.

Thank you for considering the new experiments. SPARE’s contribution is to reduce the answer verification needed to learn and calibrate an adaptive exit policy. Each verified anchor provides up to 61 weak labels, and GraphRACE reuses a readout across policies that stop at the same point on the same example.

With the same number of training verifications on Qwen3.5-4B, SPARE achieves 87.20% accuracy versus 74.67% for low-budget LYNX. On Qwen3.5-9B, relative to DEER, SPARE raises accuracy from 89.31% to 89.71% and reduces mean token usage from 8,157 to 6,248. We hope these comparisons clarify the benefits of the design for both preparation and inference.

Reviewer_qhBtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SPARE, an adaptive early-exit framework that detects when a reasoning model has accumulated sufficient information to answer correctly. Its main contributions are anchor-guided weak supervision for learning sufficiency signals, a lightweight linear probe over multi-layer hidden states, and GraphRACE, an offline policy-calibration procedure that combines shared stopping-event evaluations, behavioral clustering, and risk-aware Pareto selection. At inference time, a frozen persistence-based policy triggers a concise answer readout without modifying the backbone or requiring correctness feedback.

The authors motivate the framework through a controlled truncation study on 200 MATH problems and evaluate it on four reasoning models spanning 1.5B–9B parameters across MATH-500, GPQA-Diamond, AIME25, and AIME26. The reported results show approximately 34%–47% overall token savings while largely preserving or improving accuracy. Comparisons with four early-exit baselines, ablations of anchor supervision and stopping policies, readout controls, detector diagnostics, and calibration-cost analyses assess the framework’s effectiveness and efficiency.

Critical strengths
  1. The truncation study gives useful evidence for sufficiency transitions. In Section 3, 138 of the 143 trajectories with changes in answer correctness show a monotonic or nearly monotonic transition. Incorrect-to-correct changes are also much more common than the reverse (148 versus 18). These results support the idea that reasoning often reaches a point where the model can answer correctly. This finding is useful for early-exit research beyond SPARE itself.

  2. The stopping method is lightweight and achieves meaningful token savings. SPARE uses a linear probe on hidden states already available during generation, without repeatedly generating trial answers or changing the backbone. The reported token costs also include the final answer readout. Across four models, it saves about 34%–47% of tokens while mostly maintaining or improving accuracy. On Qwen3.5-9B, it achieves both the highest overall accuracy and the lowest token cost among the compared methods. These results show the practical value of the approach.

Critical weaknesses
  1. The evaluation covers a limited range of tasks. MATH-500 is drawn from the same distribution (MATH) as the probe-training and calibration data. AIME25/26 have only 30 questions each. GPQA-Diamond is the only benchmark outside math, and SPARE slightly lowers accuracy on three of four models. More evidence is needed to support the claim that the method preserves accuracy across different domains.

  2. The accuracy gains need stronger controls and more reliable estimates. The largest improvement comes from Qwen3.5-2B, which sometimes repeats its reasoning until reaching the token limit without giving a final answer. However, the Fixed Token and Full-Reasoning Readout controls do not include this model. It is therefore unclear how much of the gain comes from detecting sufficient reasoning rather than helping the model finish its answer. Also, the main experiments use only one random seed and report no confidence intervals, so improvements of just a few correct answers are difficult to judge.

Questions for authors
  1. Could you report results across multiple random seeds, or avg@k results on AIME, together with confidence intervals?

  2. Could you evaluate the method on at least one additional benchmark with several hundred questions from a different distribution than MATH?

  3. For each model–benchmark pair, what percentage of Vanilla trajectories reach the 32K limit without producing a scorable answer? Could you also provide Fixed Token 8K/16K and Full-Reasoning Readout results for DeepSeek-1.5B and Qwen3.5-2B, and report each method’s accuracy separately on the subset where Vanilla terminates normally?

2 comments

R1. Tasks beyond the mathematical distribution [CW1, Q2]

Using frozen mathematical policies, we added four-model evaluation on 500 examples from two BBH tasks and all 164 HumanEval+ problems, covering logic, language, and program generation.

ModelBBH accuracyToken savingsHumanEval+ Plus pass@1Token savings
DeepSeek-1.5B37.20%→40.40%51.38%59.76%→60.37%31.59%
Qwen3.5-2B56.60%→67.60%68.32%59.76%→64.02%33.80%
Qwen3.5-4B73.80%→73.60%40.67%87.80%→91.46%31.61%
Qwen3.5-9B78.00%→77.80%47.33%74.39%→90.24%59.55%

BBH includes 250 examples each from logical deduction and language disambiguation. HumanEval+ uses official EvalPlus execution tests. BBH accuracy changes are −0.20 percentage points on both 4B and 9B, with over 40% token savings. Code Plus pass@1 improves on all four models. New Table 3 reports these results; Appendix A.2 gives the settings.

R2. Repeated sampling and greedy decoding [CW2, Q1]

We added three-seed average results on AIME25/26.

ModelVanilla accuracySPARE accuracyNet token savings
DeepSeek-1.5B25.56%24.44%36.13%
Qwen3.5-2B12.78%21.11%35.03%
Qwen3.5-4B52.78%66.11%21.85%
Qwen3.5-9B67.22%73.33%32.72%

To address stochastic-sampling variability, we additionally fixed temperature at 0 and generated from scratch on all 758 examples with greedy decoding.

ModelVanilla accuracySPARE accuracyActual token savings
DeepSeek-1.5B48.94%58.84%47.49%
Qwen3.5-4B61.21%80.87%48.54%
Qwen3.5-9B74.93%85.88%52.25%

All three models improve accuracy while reducing tokens under greedy generation, showing benefits without stochastic decoding (new Table 4).

R3. Empty final answers, budgets, and full-reasoning readout [CW2, Q3]

We completed FullReader controls on all 758 examples for each model. FullReader and SPARE share the answer-readout protocol, counting the actual prefix and readout cost. The table reports the MATH subset defined by a nonempty original Vanilla final answer (revised Table 12).

ModelSamplesVanilla accuracyFullReader accuracySPARE accuracySPARE token savings vs. FullReader
DeepSeek-1.5B49084.69%84.69%84.49%35.49%
Qwen3.5-2B34093.24%94.71%94.71%53.93%
Qwen3.5-4B47495.57%95.99%95.99%35.50%
Qwen3.5-9B48995.91%95.71%96.11%43.19%

The original Table 2 reports fixed 8K/16K controls on 4B and 9B. SPARE achieves higher accuracy and lower token cost than the 16K budget on both models.

Thank you for reviewing our paper. In response to your comments, we added BBH and HumanEval+ evaluations, repeated sampling and greedy online results, and matched full-reasoning readout controls for all four models. We would appreciate any feedback on whether these additions address your main concerns and would be happy to clarify any remaining points.

Reviewer_GE9YOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents SPARE—Sufficiency Probing for Adaptive Reasoning Exit—a post-hoc early-exit method for large reasoning models. The basic idea is to find a verified answer-related “anchor” in each completed reasoning trace, label sampled units before that anchor as negative and units at or after it as positive, then train a linear probe over hidden states from four decoder layers. A second component, GraphRACE, chooses a model-specific threshold and persistence rule by reusing stopping-event evaluations, clustering policies with similar behavior, and balancing accuracy, token savings, and harm. At inference time, SPARE checks the probe at reasoning-unit boundaries and, once the score remains above threshold for enough consecutive units, closes the reasoning block and asks for a concise final answer. The reported results cover DeepSeek-R1-Distill-Qwen-1.5B and three Qwen3.5 models on MATH-500, GPQA-Diamond, AIME25, and AIME26. Token use falls by roughly 34%–47%, while overall accuracy changes range from a 0.39-point drop to an 11.22-point gain relative to full generation

Critical strengths
  1. The problem matters and it is relevant because reasoning models often keep generating after they already have enough information to answer correctly. A stopping method that does not require retraining the backbone could be genuinely useful, especially for local or controlled serving setups where hidden states are available.

  2. The anchor-based labeling scheme is clever. One verified anchor can produce as many as 61 training labels: up to 30 before the anchor, the anchor itself, and up to 30 after it. That is much cheaper than forcing the model to produce and score an answer at every possible exit point.

  3. I found the cost comparison with LYNX particularly helpful. Under the paper’s matched setup, SPARE uses 4.23 times fewer auxiliary readout calls and generates 7.18 times fewer readout tokens. That gives the supervision strategy a concrete practical advantage, not just a conceptual one.

  4. On Qwen3.5-2B, overall accuracy rises from 59.10% to 70.32% while the compression rate drops to 53.06%. On Qwen3.5-9B, SPARE has both the best overall accuracy and the lowest token cost among the methods in the main table.

  5. The train, calibration, and test splits are kept separate. Probe training and policy calibration use disjoint MATH development sets, and the final policies are frozen before formal benchmark evaluation. Since the policy is calibrated only on MATH data, the GPQA and AIME results also give at least some evidence of transfer across tasks.

  6. The reported cost includes the retained reasoning prefix, the stopping-time answer readout, and method-specific auxiliary generations. The methods also share the same post-truncation answer template and evaluator, which makes the exit-policy comparison easier to interpret.

  7. The paper includes fixed-budget and fixed-threshold comparisons, random-anchor and close-only variants, a LYNX confidence sweep, detector diagnostics, anchor-reliability statistics, and qualitative examples. Those additions make the empirical story much stronger than the main table alone.

  8. The manuscript gives pseudocode, deterministic segmentation and anchor-extraction rules, decoding settings, probe-training hyperparameters, calibration folds, and the final thresholds used for each model

Critical weaknesses
  1. The weak labels do not always match the paper’s own definition of answer sufficiency. Equation 5 defines a prefix as sufficient when its readout is correct. Yet Equation 7 accepts an anchor readout if it matches either the gold answer or the completed trajectory’s answer. That second answer may itself be wrong. Once such an anchor is accepted, the anchor and sampled post-anchor units are labeled positive. In that case, the probe may be learning answer stability or agreement with the model’s final trajectory rather than correctness sufficiency. The authors should report how many anchors come from gold references versus self-references, how accurate each group is, and what happens under gold-only, correct-readout-only, and self-reference-free training

  2. A simpler explanation remains possible: the probe may mostly be detecting how far the model has progressed. Labels are created entirely from whether a unit appears before or after an anchor. Anchors later than 60% of the trace are discarded, and exits are forbidden before 32 reasoning units. The detector logits also rise steadily with relative progress, while the random-anchor control remains surprisingly competitive in some settings. A token-position baseline, a reasoning-unit-count baseline, and detector metrics computed within narrow progress bins would help separate genuine sample-specific sufficiency from generic sequence progress

  3. The controlled study uses 200 MATH examples from one model, Qwen3.5-4B, and checks only nine fractions of each trajectory. An “approximate boundary” can contain one mismatched checkpoint, which is permissive for a sequence with only nine observations. The results show that correctness often improves as reasoning proceeds, but they do not yet establish a stable unit-level boundary across models, tasks, or decoding samples. A denser unit-level study on multiple backbones and benchmarks would make this claim much more convincing. I believe the evidence for a general latent sufficiency boundary is still fairly narrow.

  4. All stochastic generations use a single seed, and the headline tables contain only point estimates. AIME25 and AIME26 have just 30 problems each, so several reported differences amount to one example. Multiple decoding seeds, paired bootstrap intervals, paired correctness tests, and confidence intervals for token savings and conditional harm would make it much easier to judge which gains are stable

  5. Qwen3.5-2B improves by 11.22 points overall and by 16.4 points on MATH-500. Yet the Full-Reasoning Readout, Close-Only, and Random Anchor experiments are reported only for Qwen3.5-4B and 9B. Since the 2B model often appears to run into long loops or fail to emit a clean final answer, part of the gain may come from forcing a concise answer rather than from detecting sufficiency. The same controls should be run on Qwen3.5-2B and DeepSeek-1.5B, along with context-limit rates, trigger rates, and full correctness-transition matrices.

  6. The procedure fully evaluates only a small number of cluster representatives instead of all candidate policies. Similar stopping profiles do not necessarily imply similar answer accuracy or harm, and the paper does not compare the chosen policy with the one that exhaustive evaluation would have selected. Appendix B.5 also says that no additional policies were eliminated after the Net20 filter during the pilot. In practice, most of the savings seem to come from clustering and representative selection rather than from successful risk-aware racing. Exhaustive evaluation of the 120-policy space would let the authors report selection agreement, policy regret, missed feasible policies, and sensitivity to the cluster count and pilot budget.

  7. “Risk-aware” currently means empirical screening, not a statistical guarantee. The same 667-example calibration set is used to compare and select many policies, and the five folds are repeatedly consulted during that selection process. Requiring nonnegative accuracy change in every fold is a sensible heuristic, but it does not control out-of-sample premature-exit risk or account for multiple comparisons. The paper should either tone down the wording or add nested holdouts, confidence bounds, or another mechanism that supports a genuine generalization claim.

  8. Fixed Token is tested only at 8K and 16K. Fixed Threshold uses one percentile. That makes it hard to tell whether SPARE is beating a genuinely competitive calibrated budget policy at a matched token cost. A stronger comparison would sweep fixed token budgets, absolute and relative position rules, position-only classifiers, answer-mention heuristics, and conclusion-cue heuristics.

  9. I would like to see variation in the selected layers, number of layers, pre- and post-anchor window sizes, anchor-position limit �0��0�, minimum unit count �1��1�, persistence length �2��2�, gold versus self-reference matching, and anchor verification. A shuffled-anchor or shuffled-label control would also be informative. Exact implementation details for DEER, RPDI, and REFRAIN are comparatively thin, while only LYNX receives a broad sensitivity analysis.

  10. Hidden-state probes have already been used for intermediate answer verification and early exit, while LYNX already combines a hidden-state classifier with calibrated stopping. The paper itself also discusses answer-related anchors and neuron-based early exits in prior work. The clearest new pieces here seem to be the inexpensive expansion from one anchor to many weak labels and the GraphRACE calibration heuristic. Those are meaningful contributions, but the paper should avoid presenting hidden-state sufficiency detection or adaptive early exit more broadly as entirely new.

  11. The evaluation does not yet support broad architectural generality. All four models are Qwen-based or Qwen-derived: one DeepSeek-R1-Distill-Qwen model and three Qwen3.5 sizes. The tasks are also limited to exact-answer mathematics and multiple-choice science, where anchor matching and automatic scoring are unusually convenient. Either add a genuinely different model family and a less structured task, or narrow the claims to open-weight reasoning models with verifiable answers.

  12. The reported metric does not include the cost of exposing hidden states from four layers, online reasoning-unit segmentation, probe execution, synchronization, memory traffic, or possible batching inefficiencies. Those costs may be small. Still, the paper should measure them. Wall-clock latency, throughput, peak memory, and probe overhead per generated token would make the practical case much stronger. Completion-token savings are not the same as end-to-end serving savings.

  13. The paper would benefit from a direct limitations section. There are no obvious human-subject or sensitive-data concerns here. The main risk is deployment under distribution shift: a prematurely stopping model may silently omit necessary reasoning, especially in medical, legal, scientific, or other high-stakes settings. The current experiments do not support those uses, and the paper should say so plainly.

Questions for authors
  1. How can you tell that the probe is detecting answer sufficiency rather than simply how far the model has progressed? The labels are tied to positions before and after an anchor, late anchors are removed using �0��0�, and exits cannot occur before �1��1�. Hidden states also encode sequence position and general reasoning progress. Could you add token-count, unit-count, and position-only baselines, then report probe performance within narrow progress bins or after regressing out absolute and relative position.

  2. How much noise comes from allowing an anchor readout to match the model’s completed answer even when that answer is wrong? Once such an anchor is accepted, the anchor and its post-anchor units are labeled positive. That seems closer to learning answer stability than correctness sufficiency. Please report the share and accuracy of gold-reference versus self-reference anchors, and include gold-only, correct-readout-only, and self-reference-free ablations.

  3. Does GraphRACE choose nearly the same policy that exhaustive evaluation would choose? The method clusters policies based on their stopping profiles and evaluates only representatives, but two policies with similar stopping positions could still have different correctness or harm. Please report agreement with exhaustive selection, as well as regret in accuracy, token savings, and conditional harm. It would also help to clarify that the current “risk-aware” criterion is empirical fold-level screening unless you can provide an out-of-sample guarantee.

  4. How stable are the results across seeds, model families, and tasks that do not have clean answer anchors? The current study uses one stochastic seed, four Qwen-based or Qwen-derived backbones, and benchmarks with automatically verifiable answers. AIME25 and AIME26 also contain only 30 examples each. Could you provide multi-seed uncertainty estimates and, ideally, results on a different architecture and a less structured task where exact anchor matching is harder?

3 comments

R1. Supervision and label audit [CW1, Q2]

Equation 5 defines actual readout correctness yy. Equations 7–8 derive training labels y~\tilde y from verified anchors, providing up to 61 endpoints per check. Reference correctness is the primary acceptance signal. Consistency supervision also covers incorrect original answers reproducible from earlier prefixes, supporting reductions in further computation that leaves the conclusion unchanged. The final policy is calibrated on actual correctness and cost.

The proportions of retained anchors accepted solely through consistency are 5.83%, 2.58%, 1.46%, and 1.31% across the four models. Using a fixed seed, we uniformly sampled 50 of the 1,095 actual 4B training trajectories and randomly selected two pre-anchor and two post-anchor endpoints each. These 200 checks yield readout accuracies of 43%/99%.

Sections 4.1–4.2 now clarify the labels and exit positions; Appendix A.4 adds the random audit.

R2. Position-based explanations [CW2, CW8, Q1]

On the same Math200 diagnostic, the hidden-state probe and the fitted classifier using absolute token and unit positions achieve AUROCs of 0.851 and 0.650. The Fixed Token 8K/16K controls in Table 2 show higher accuracy and lower token cost for SPARE than the 16K budget on both 4B and 9B. On 9B, SPARE improves accuracy by 5.54 percentage points at a cost close to the 8K budget. Appendix D now includes the fitted position control.

R3. Boundary structure [CW3]

Equation 6 defines the onset of a stably correct suffix under the specified readout protocol; an earlier correct readout and entry into that suffix are distinct events. Among 143 trajectories whose correctness changes, 126 exhibit a strict single incorrect-to-correct transition across nine checkpoints, 12 differ at one checkpoint, and five are non-monotonic. Another 51 are always correct and six always incorrect. The 138 boundary-compatible trajectories predominantly exhibit strict transitions. Adjacent checkpoints contain 148 incorrect-to-correct transitions and 18 reverse transitions. SPARE makes exit decisions using a probe and persistence rule.

Section 3.2 now states the nine-checkpoint setting alongside the 138/143 denominator. This structure motivates adaptive stopping, whose practical value is evaluated through frozen-policy accuracy and computation.

R4. Repeated sampling [CW4, Q4]

We added three-seed average results on AIME25/26.

ModelVanilla accuracySPARE accuracyNet token savings
DeepSeek-1.5B25.56%24.44%36.13%
Qwen3.5-2B12.78%21.11%35.03%
Qwen3.5-4B52.78%66.11%21.85%
Qwen3.5-9B67.22%73.33%32.72%

The three-model greedy online evaluation also improves accuracy and reduces tokens; R10 below reports the deployment results.

R5. The 2B gains and empty-final attribution [CW5]

We extended FullReader to all 758 examples for each model: retain the full reasoning prefix and replace the original final-answer generation with SPARE's concise readout, counting prefix and readout tokens. The following MATH subset, defined by a nonempty original Vanilla final answer, tests stopping when a final response already exists (revised Appendix C.1, Table 12).

ModelSamplesVanilla accuracyFullReader accuracySPARE accuracySPARE token savings vs. FullReader
DeepSeek-1.5B49084.69%84.69%84.49%35.49%
Qwen3.5-2B34093.24%94.71%94.71%53.93%
Qwen3.5-4B47495.57%95.99%95.99%35.50%
Qwen3.5-9B48995.91%95.71%96.11%43.19%

On all 758 examples, 2B Vanilla/FullReader/SPARE accuracy is 59.10%/73.61%/70.32%; SPARE saves 45.03% of tokens relative to FullReader.

R6. GraphRACE search and empirical risk [CW6, CW7, Q3]

Sharing readouts for identical stopping events is exact; representative selection approximates candidate coverage with fewer checks. The pilot supplies shared readout evidence, and final selection uses fully evaluated representatives. The four models use 4,000, 4,984, 4,037, and 4,571 queries, reducing the exhaustive cost of 120 candidates after stopping-event deduplication by 65.0%–73.9%.

We added four-model six-candidate simple calibration using the same probe, Math667, and selection rules, within the original GraphRACE query budgets. On 9B, both methods achieve 89.71% accuracy, while GraphRACE increases token savings from 25.21% to 41.55%. On 4B, simple calibration/GraphRACE accuracy is 87.60%/87.20%, with savings of 24.56%/33.90%, demonstrating the calibration design's quality–cost balance.

Section 4.3 and Appendices B.2/B.5 clarify that risk-aware selection uses overall and per-fold empirical accuracy, then balances accuracy, net savings, and conditional harm within the fully evaluated, empirically feasible candidate pool. Table 5 reports the four-model control.

R7. Hyperparameters and baseline details [CW9]

The paper specifies layers, training windows, γ, minimum unit count, persistence, and readout configuration. Original Table 2 and the fixed-anchor ablation examine central designs; the new position control, limited-budget LYNX, and six-point calibration further test position-based explanations and preparation costs. Thresholds and persistence are calibrated only on Math667 and frozen for formal evaluation.

Section 5.1 and Appendix A.2 describe baseline stopping signals, the shared post-truncation readout, and token accounting, including actual auxiliary generations.

R8. Contribution and preparation efficiency [CW10]

SPARE combines sufficiency learning and efficient calibration in a complete pipeline: one verified anchor expands to up to 61 supervised endpoints, a lightweight hidden-state probe makes online decisions, and GraphRACE calibrates exits through shared stopping events and behavioral representatives. Its value is measured jointly by accuracy and computation.

Under the same training pool, calibration pool, template, and scorer, auxiliary readout calls and output tokens for training and calibration assets are 1/4.23 and 1/7.18 of LYNX. The four-model training-verification-budget and simple-calibration controls in Table 5 further support the efficiency of these designs.

R9. Cross-task evaluation and model training backgrounds [CW11, Q4]

Using probes and policies frozen on mathematics, we added four-model evaluation on 500 examples from two BBH tasks and all 164 HumanEval+ problems (new Table 3).

ModelBBH accuracyToken savingsHumanEval+ Plus pass@1Token savings
DeepSeek-1.5B37.20%→40.40%51.38%59.76%→60.37%31.59%
Qwen3.5-2B56.60%→67.60%68.32%59.76%→64.02%33.80%
Qwen3.5-4B73.80%→73.60%40.67%87.80%→91.46%31.61%
Qwen3.5-9B78.00%→77.80%47.33%74.39%→90.24%59.55%

BBH covers logical deduction and language disambiguation, and HumanEval+ uses official EvalPlus execution tests. BBH accuracy changes are −0.20 percentage points for both 4B and 9B, while code Plus pass@1 improves on all four models.

DeepSeek's distillation from R1 reasoning trajectories covers reasoning patterns and post-training conditions different from native Qwen3.5. We plan to extend evaluation to Llama, Gemma, and other model families.

R10. Three-model vLLM greedy online evaluation [CW12]

To address sampling uncertainty and actual overhead, we integrated SPARE into the production-grade vLLM engine with continuous batching, CUDA Graphs, and concurrency 8. At temperature 0, both arms generate from scratch on all 758 problems for each of the three models. Timing includes hidden-state extraction, segmentation, probe decisions, scheduling, and answer readout; token accounting includes readout and actual generation past detected boundaries.

ModelVanilla accuracySPARE accuracyActual token savingsVanilla→SPARE GPU time (GPU·h)Speedup
DeepSeek-1.5B48.94%58.84%47.49%5.35→2.801.91×
Qwen3.5-4B61.21%80.87%48.54%9.96→5.541.80×
Qwen3.5-9B74.93%85.88%52.25%12.34→6.012.05×

New Table 4 reports this three-model online evaluation. Accuracy and efficiency gains persist under fixed greedy decoding and translate into measured acceleration in the serving engine.

R11. Deployment quality [CW13]

The end of Section 5.4 now recommends selecting an operating point for target-task quality requirements and rechecking early-exit quality when the input distribution changes.

Thank you for your detailed review. Our replies above address the supervision labels, the role of reasoning position, the readout control, repeated sampling, and measured inference costs. If you have a chance to review our responses and the revised manuscript, we would welcome any feedback on questions that still need clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.