acceptodds
Under review as a conference paper at ICLR 2027

Supervise What You Certify: Stop-Harm Prediction for Early Stopping in Reasoning Models

Abstract

Reasoning models gain accuracy by generating long chains of thought, but incur substantial inference cost and often continue reasoning after reaching a correct answer. Early stopping aims to remove redundant computation while limiting accuracy loss. Existing learned stoppers use supervision based on current-answer correctness, answer agreement, or later answer stability. These targets do not directly capture lost-correct events: stopping yields a wrong answer although full reasoning would answer correctly. The probability of this event upper-bounds the accuracy drop, yet the loss used for certification differs from these supervision targets. We call this event *stop harm* and introduce StopHarm to supervise it directly at each checkpoint, aligning the learning target with the policy loss at the actual stopping point. StopHarm learns a lightweight predictor from paired stopping and full-reasoning outcomes offline, and stops online when the predicted stop harm stays below a certified threshold. Independent request-level certification bounds lost-correct risk, guaranteeing an accuracy drop of at most with confidence under i.i.d. sampling. We prove that even Bayes-optimal proxy predictions can discard stop-harm information that no score-only post-processing can recover. Across three reasoning models and five benchmarks, StopHarm reduces generated tokens by **25.4–38.3%** with average accuracy drops of only **0.27–0.73** points and **1.71–1.77** single-request speedups.

Primary area:
foundation or frontier models, including LLMs
Keywords:
early stopping, lost-correct risk, conditional mutual information, test-time compute allocation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_v4YcOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when to stop reasoning to reduce inference cost. For this goal, authors propose StopHarm, a early-stopping framework for LRMs that directly supervises the event where early stopping produces an incorrect answer while full reasoning would be correct. The method first trains a predictor on paired early-stop and full-reasoning outcomes, then uses a separate calibration set to select a stopping threshold. The paper also establishes theoretical results characterizing the information and decision value of direct stop-harm supervision. Experiments across different reasoning models and benchmarks demonstrate substantial token savings with relatively small average accuracy degradation.

Critical strengths
  1. The key insight of this paper is interesting. StopHarm directly predicts whether early stopping would lose a correct full-reasoning answer, which is also the risk being controlled. The theoretical analysis explains why existing supervision targets, such as correctness and answer agreement, may fail to capture this risk.

  2. The experiments were carried out in a comprehensive manner, and the experimental results can support the core claims of the article.

Critical weaknesses
  1. Please focus on Section 5.1, Table 2, and Appendix E.2. The risk guarantee requires calibration and deployment data to follow the same distribution (i.e., i.i.d.). However, StopHarm calibrates only on MATH and evaluates on other benchmarks, where the guarantee does not directly apply. Moreover, the main results do not separately report lost-correct and recovered-correct risks, making it difficult to assess the actual risk being controlled.

  2. Although the paper discusses LearnStop, which also controls lost-correct risk, it is not included in the main baseline comparisons. While the ablation studies are valuable, a direct comparison under matched seetings would better establish the practical benefits of StopHarm over this closely related approach.

Questions for authors
  1. StopHarm requires a separate calibration dataset to certify its stopping threshold before deployment. I recalled the recent work Anytime Safe PAC Efficient Reasoning (Yu et al., ICML 2026), which provides anytime-valid risk control without a calibration set by leveraging online feedback. Although that work seems to consider a different but similar setting, could the authors discuss whether a similar online certification approach could be developed for StopHarm? what additional feedback would be required?

  2. Could the authors provide a direct comparison with LearnStop? This would help isolate the practical improvement attributable to direct stop-harm supervision.

  3. Could the authors report the empirical lost-correct and recovered-correct risks separately for each model and benchmark? If your time is limited, you can also present only some of the experimental results. Since these two risks can offset each other in the overall accuracy change, reporting accuracy alone does not directly reveal the lost-correct risk that StopHarm aims to control.

4 comments
AuthorsOpenReview ↗

Response Part 1/2

We sincerely thank the reviewer for finding our key insight interesting, for noting that StopHarm directly predicts the event which is also the risk being controlled, and for considering our experiments comprehensive and able to support the core claims. Below, we address the scope of the i.i.d. guarantee, the comparison with LearnStop, online certification, and the LC/RC decomposition in turn.


W1 — i.i.d. guarantee: scope and target-domain certification

We agree: Proposition E.1 requires i.i.d. calibration and deployment requests, so in Table 2 it covers only MATH500, and the other four benchmarks test transfer.

Theory (Appendix E.4). As the probe is frozen, Proposition E.1 applies unchanged to target calibration data (Corollary E.1). Without target data, LC is at most BαB\alpha under a bounded shift BB in the probe's score and stop-harm trajectories (Proposition E.2). With neither, no calibration-based method can certify an arbitrary unseen task (Proposition E.3), so transfer must be shown empirically.

StopHarm transfers. With the probe and threshold fit on MATH only, it saves 30.4% of tokens on the other four benchmarks of Table 2 with no average accuracy loss (+0.5 pp). Where a formal guarantee is needed, we keep the probe and certify only the threshold on 700 GSM8K or ARC-Challenge requests (α = 5%):

ModelBenchmarkMATH threshold: LC / TROwn threshold: LC / TR
Qwen3-4BGSM8K1.67 / 56.432.88 / 64.45
Qwen3-4BARC-Challenge5.38 / 80.324.27 / 71.59
DeepSeek-7BGSM8K1.29 / 46.983.56 / 62.36
DeepSeek-7BARC-Challenge1.45 / 34.392.73 / 54.94

LC: lost-correct rate (%); TR: token reduction (%); fresh trajectories, so values differ slightly from Table 2.

One scalar threshold carries the guarantee to a new task. With its own threshold, every benchmark keeps LC below α (15 of 16 policies across α ∈ {1, 3, 5, 10}%). Recertification fits the threshold to each task: it brings Qwen3-4B/ARC below α while still saving 71.6% of tokens, and adds 8–21 pp of TR in the other three cases. One training run thus serves many tasks; only the threshold is task-specific, and it can also be certified online (Q1).


W2/Q2 — Direct comparison with LearnStop

At the same accuracy and LC, StopHarm saves more tokens than LearnStop: on Qwen3-14B/GSM8K, both reach about 95.5% accuracy with 0.8% LC, but StopHarm reduces tokens by 47.3% versus 26.9%; on MATH500, StopHarm has higher accuracy, lower LC, and larger token reduction on all three models.

LearnStop shares StopHarm's MATH training and calibration splits (1,400 / 700), test problems, reference trajectories, and grader. It is trained and calibrated separately under two checkpoint settings: its own sparse grid (few trial answers) and StopHarm's paragraph checkpoints. Its cost includes all trial-answer generations.

ModelMethodMATH500GSM8KAIME2024AIME2025GPQA Diamond
Qwen3-4BStopHarm89.2/3.0/24.593.8/1.2/55.760.0/0.0/6.853.3/0.0/5.958.1/3.5/34.1
LearnStop: sparse grid86.0/6.4/18.593.6/1.1/18.756.7/3.3/3.453.3/0.0/1.642.9/18.7/74.8
LearnStop: paragraph86.2/6.4/−19.292.5/2.2/−5.956.7/3.3/−69.056.7/0.0/−61.239.9/23.7/84.1
Qwen3-14BStopHarm87.8/4.4/29.395.5/0.8/47.366.7/0.0/6.360.0/0.0/4.964.6/5.1/45.7
LearnStop: sparse grid85.0/7.2/29.295.5/0.8/26.956.7/10.0/15.460.0/0.0/5.249.5/18.7/69.6
LearnStop: paragraph87.4/5.0/−5.195.7/0.6/10.860.0/6.7/−34.260.0/0.0/−43.553.0/15.2/62.7

Each cell: Acc. / LC / TR (%); LC is over all test requests, and negative TR means higher cost than Full-CoT.

On DeepSeek-7B, StopHarm saves more tokens than sparse-grid LearnStop on four of five benchmarks (e.g., 30.9% / 34.4% vs. 9.7% / 4.2% on AIME2024 / AIME2025, with accuracy within one problem); on GSM8K, it gives up 2.5 pp of TR for 2.5 pp higher accuracy and lower LC (1.2% vs. 3.9%).

StopHarm supports fine-grained checking without extra trial answers. LearnStop must generate a trial answer at every check, whereas StopHarm reads the hidden states and statistics of normal decoding and answers only once it stops. At paragraph granularity, LearnStop therefore raises cost by 34.2%–69.0% on both AIME benchmarks for all three models; on DeepSeek-7B/AIME2025, its average number of trial answers grows from 9.1 (sparse grid) to 334.5.

Changing only the supervision target isolates its contribution. LearnStop supervises the correctness of the current answer (LearnStop §3.1–3.4). In the paper's ablation (Table 3), which fixes features, probe, checkpoints, and certification, replacing correctness with stop-harm supervision keeps accuracy and LC unchanged on DeepSeek-7B/AIME2025 (36.7% / 3.3%) while raising token reduction from 5.9% to 34.4%.

AuthorsOpenReview ↗

Response Part 2/2

Q1 — Online certification extension

Yes. Inspired by Yu et al. (Anytime Safe PAC Efficient Reasoning), we have implemented online certification for StopHarm: it keeps the trained probe and certifies the threshold from random audits during deployment instead of a held-out calibration set.

Required feedback: StopHarm's training label on randomly audited requests. The label Lj=YF(1−Yπj)L_j=Y^{F}(1-Y^{\pi_j}) is 1 when full reasoning answers correctly but stopping at candidate threshold πj\pi_j does not. An audit therefore runs full reasoning once and grades it together with each candidate's stopped answer.

Procedure. Each candidate threshold keeps a score that starts at 1, rises after each audit in which stopping there causes no harm, and falls when it does. A candidate is certified once its score and those of all more conservative candidates reach 1/δ1/\delta; each request uses the most aggressive certified candidate, or full reasoning if none is certified. Because the score can be checked after every audit, with i.i.d. requests, independent audits, and correct labels,

Pr⁡ ⁣(∀t≥1,  RLC(πt)≤α)≥1−δ.\Pr\!\left(\forall t\ge 1,\;R_{\mathrm{LC}}(\pi_t)\le\alpha\right)\ge 1-\delta.

Results. We replay cached outputs (1,000 resampled streams of 10,000 requests per model and benchmark).

  • The guarantee holds at every step: across all three models and five benchmarks, online certification selects an over-budget policy in 0.3%–2.6% of streams, below δ = 5%, whereas repeating a standard binomial test on the same audits reaches 8.0% / 12.2% on MATH500 (Qwen3-4B / Qwen3-14B).
  • The savings remain: after paying for audits, it still saves 6%–53% of tokens, with accuracy at most 1.1 pp below the offline policy and higher on MATH500.
ModelCertificationMATH500GSM8KAIME2024AIME2025GPQA Diamond
Qwen3-4BOffline (paper)89.2/24.593.8/55.760.0/6.853.3/5.958.1/34.1
Online90.6/13.293.6/52.059.9/6.256.3/17.057.0/15.0
Qwen3-14BOffline (paper)87.8/29.395.5/47.366.7/6.360.0/4.964.6/45.7
Online90.8/13.695.0/52.866.6/9.459.9/16.763.5/6.7

Each cell: Acc. / TR (%); online values are means over replay streams, and TR includes audit costs.

StopHarm can thus be certified offline when a calibration set exists, or online when labels arrive with requests, with the same probe and label.


Q3 (also W1, part 2) — LC/RC decomposition and matched-cost comparison

Below are LC and RC for every model and benchmark. At matched cost, supervising stop harm roughly halves LC. Within each model, the four supervision targets share training data, reference trajectories, features, probe, checkpoints, stopping rule, and grader, and are compared at the operating points with the closest test TR. Averaged over the 15 model–benchmark pairs, LC falls from 4.7%–5.1% (correctness, consistency, last-switch) to 2.5%, the lowest in every pair, while RC rises to 2.1% (vs. 1.4%–1.5%): learning the certified event itself protects the answers that full reasoning gets right.

ModelSupervision targetGSM8KMATH500GPQA DiamondAIME2024AIME2025
Qwen3-4BCorrectness55.3/3.2/0.323.9/3.0/0.433.5/8.6/4.56.4/0.0/0.06.1/0.0/0.0
Consistency54.8/1.3/0.324.7/6.4/0.434.1/7.1/4.07.3/0.0/0.06.1/0.0/0.0
Last-switch56.3/1.3/0.324.3/6.2/0.233.8/7.1/2.56.4/0.0/0.05.6/0.0/0.0
StopHarm55.7/1.2/0.524.5/3.0/0.434.1/3.5/5.66.8/0.0/0.05.9/0.0/0.0
DeepSeek-7BCorrectness54.8/5.1/0.727.1/6.6/0.037.8/9.6/8.630.1/16.7/0.035.1/10.0/0.0
Consistency54.4/2.4/0.528.0/5.4/0.038.7/9.6/12.130.9/10.0/0.034.7/13.3/0.0
Last-switch54.9/3.0/0.626.9/5.4/0.037.4/8.1/13.129.9/10.0/0.031.7/10.0/0.0
StopHarm54.3/1.2/0.827.9/4.8/0.238.1/6.1/17.230.9/3.3/0.034.4/3.3/0.0
Qwen3-14BCorrectness47.1/1.3/0.229.2/5.2/0.445.5/7.6/5.16.5/0.0/0.04.5/0.0/0.0
Consistency48.0/0.9/0.329.0/8.2/0.445.7/8.1/4.56.5/0.0/0.05.8/0.0/0.0
Last-switch48.0/1.1/0.329.3/8.0/0.446.6/9.6/5.17.0/0.0/0.03.3/0.0/0.0
StopHarm47.3/0.8/0.229.3/4.4/0.445.7/5.1/6.16.3/0.0/0.04.9/0.0/0.0

Each cell: TR / LC / RC (%). LC (RC) counts requests that Full-CoT answers correctly (incorrectly) but early stopping does not (does), so ΔAcc = RC − LC.

Revision. The revised paper (changes in blue) states the scope of the guarantee (Sections 3.2 and 5.1) and adds the transfer theory (Appendix E.4) and the recertification results (Section 5.6 and Appendix F.11). It also adds LearnStop to Table 2, with both checkpoint settings in Appendix F.10, and online certification in Appendix E.5.


We hope these responses address the reviewer's concerns, and we are happy to discuss any further questions.

I thank the authors for their thoughtful responses. After reviewing the responses, I think that my questions have been resolved. So I am delighted to maintain my positive assignment.

Thank you very much for your positive assessment and for confirming that our responses have resolved your questions. Your constructive comments have substantially improved our paper.

Reviewer_3kFAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes StopHarm, an early-stopping method for reasoning models that directly predicts whether stopping would lose a correct final answer. It combines this prediction with risk calibration to reduce inference cost.

Critical strengths

The main idea is simple and well motivated. Aligning the training target with the actual stopping risk is useful. The experiments show substantial token savings while controlling accuracy loss.

Critical weaknesses

The main methodological novelty is relatively limited. The paper builds on existing early-stopping predictors and risk certification methods, with the primary contribution being the alignment of the supervision target with the certified loss.

Training requires full reasoning trajectories and stop-and-answer labels at multiple checkpoints, which may introduce substantial offline computational cost. The paper does not clearly quantify the cost of preparing these labels relative to the inference savings.

Questions for authors

What is the total offline data-generation cost, and how many inference requests are needed to recover this cost through early stopping?

How well does the certified stopping policy transfer across task distributions or model updates?

How do the reported speedups translate to throughput and latency under realistic continuous batching?

4 comments
AuthorsOpenReview ↗

Response Part 1/2

We sincerely thank the reviewer for the careful review and for recognizing that our main idea is simple and well motivated, that aligning the training target with the actual stopping risk is useful, and that our experiments show substantial token savings while controlling accuracy loss. Below, we address novelty, offline cost, transferability, and serving gains under continuous batching in turn.


W1 — Novelty: directly learning stop harm

StopHarm does more than align the supervision target with the certified loss: it changes what the early stopper learns from "is the current answer correct?" to "what would be lost by stopping now?". This separates currently incorrect states: requests that further reasoning would correct are protected, whereas requests that full reasoning also answers incorrectly can be stopped without losing any correct answer, a source of savings that proxy targets cannot exploit. This distinction cannot be obtained by combining existing predictors with certification: even a Bayes-optimal proxy score may lose it, and no post-processing can recover it (Theorem 1). Changing only the supervision target (Table 3) raises the average token reduction of DeepSeek-7B from 18.0%–25.6% to 38.3% and lowers its accuracy loss from 2.45–3.29 to 0.28 pp.


W2/Q1 — Offline cost: a one-time investment recovered within thousands of requests

The offline cost is small, and on tasks with a correctness oracle most of it comes from serving itself. Preparation takes under 10 GPU·h per model on a single GPU and is recovered after 3,296–6,249 requests. Of these tokens, 82%–91% are full reasoning traces, which a deployment generates anyway when it serves requests with Full-CoT. In practice, many tasks come with a correctness oracle, such as reference answers or unit tests; there, these served requests already provide StopHarm's labels, and only one short forced answer per checkpoint is added, recovered after 285–1,113 requests. The base model stays frozen and no external annotator is needed. One probe serves all five benchmarks (Table 2) and keeps saving after model updates (Q2), so requests from all tasks count toward break-even.

ModelPrep. tokens (extra beyond serving)GPU·hAvg. TRAvg. output (tokens/req)Break-even (req): all / extra only
Qwen3-4B11.7M (2.09M)4.3525.38%7,409.806,249 / 1,113
DeepSeek-7B7.8M (0.67M)3.9738.33%6,178.043,296 / 285
Qwen3-14B10.1M (1.06M)9.8926.71%7,079.625,319 / 563

Preparation: one full reference trace plus one forced answer per checkpoint for 1,400 training + 700 calibration problems; extra = forced answers only, which share the served trace's prefix. TR and output length are means over the five benchmarks of Table 2; break-even = ⌈tokens ÷ (output × TR)⌉, from unrounded values.

This is at the low end of related work. StopHarm uses the same offline data as LYNX (NeurIPS 2026). JET (ICLR 2026), ShorterBetter (NeurIPS 2025), and AnytimeReasoner (NeurIPS 2025) train on 14K–40K problems, and ShorterBetter, AnytimeReasoner, and TERMINATOR (NeurIPS 2026) self-report about 64, 240, and 1,416 GPU·h on various GPUs.

The cost can be halved further: with half the problems (700 + 350), preparation tokens fall by about 50%, while inference tokens are still reduced by 15.5%–17.2% at an accuracy drop of at most 1.5 pp.

ModelTrain + cal. problemsPrep. tokens saved (vs. 1400 + 700)Avg. TR (%)Avg. accuracy drop (pp)
Qwen3-4B1400 + 700—25.380.27
Qwen3-4B700 + 70038.87%18.580.49
Qwen3-4B700 + 35050.98%15.540.34
Qwen3-14B1400 + 700—26.710.73
Qwen3-14B700 + 70037.07%19.211.26
Qwen3-14B700 + 35050.43%17.211.46
AuthorsOpenReview ↗

Response Part 2/2

Q2 — Transferability: reuse across tasks and model updates

The probe transfers across tasks, and the whole policy survives model updates: on ARC, recertifying only the threshold restores the guarantee at 60.44% token reduction; after fine-tuning, the frozen policy still saves 23.64% / 14.56% of tokens within 1 pp of Full-CoT accuracy.

Across tasks. Table 2 is already a transfer test: the probe and threshold are fit on MATH only and applied directly to the other four benchmarks (e.g., Qwen3-4B saves 55.67% / 34.08% of tokens on GSM8K / GPQA Diamond, with LC 1.21% / 3.54%). When target data are available, Proposition E.1 (Appendix E.2) conditions on the frozen probe, so recertifying one scalar threshold restores the guarantee without retraining. We test this on ARC-Challenge (multiple-choice science) with Qwen3-4B (α=0.03, δ=0.05):

SettingProbe trainingThreshold certificationGuarantee on ARCΔAcc (pp)TR (%)
A: direct transferMATHMATHNo−3.2475.00
B: recertify thresholdMATHARCYes−2.3960.44
C: retrainARCARCYes−1.5951.91

Recertifying a single threshold (B) adds the target-domain guarantee and keeps most of the savings; retraining (C) narrows the loss further but needs labeled trajectories for the new task.

Model updates. We run one round of SFT on Qwen3-4B with 700 MATH traces from OpenThoughts-114k-math (checkpoints M30, M80; M0 = original), keeping the original probe, stopping rule, and α=0.05 threshold.

ModelΔAcc vs. own Full-CoT (pp)LC (%)TR (%)
M0−0.271.5525.38
M30−0.831.8123.64
M80−0.941.5914.56

Averaged over the five benchmarks of Table 2.

Without retraining or recalibration, the frozen policy keeps LC below α and accuracy within 1 pp of each updated model's Full-CoT.


Q3 — Continuous batching: higher throughput and lower latency

Under vLLM continuous batching, StopHarm reaches 2.21–2.37× the saturated throughput of Full-CoT and cuts high-load P95 latency by 61.4%–68.1%. The two are complementary: batching schedules across requests, whereas StopHarm shortens each request and frees its slot for queued ones. The probe reuses decoding features and generates no extra answers.

Loads. Saturation (all 1,319 requests submitted at once) measures throughput. Poisson arrivals measure latency at two rates, set to 0.5× and 0.9× Full-CoT's throughput on a separate development workload: at the low rate (0.157 req/s) Full-CoT does not queue, isolating per-request savings; at the high rate (0.283 req/s) it starts to queue, adding savings from freed slots.

LoadMetricFull-CoTStopHarmPaired gain
SaturatedThroughput (req/s)0.318–0.3270.725–0.7532.21–2.37×
Low arrival rateP95 latency (s)100.07–103.7044.38–46.8853.1%–57.2% lower
High arrival rateP95 latency (s)127.90–149.5544.76–49.3361.4%–68.1% lower

Qwen3-4B, one L20, full GSM8K test set (1,319 problems), identical vLLM configuration (max concurrency 16, CUDA Graph, chunked prefill, prefix caching). Ranges: three request sequences per load, paired by sequence. Latency: scheduled arrival to full answer, including queuing and probe decisions.

  • Saturated: 54%–57% fewer tokens per request more than double throughput; correct-answer throughput also rises 2.19–2.34×.
  • Low rate: neither method queues, so the latency gain comes from shorter generation alone.
  • High rate: Full-CoT's mean wait rises to 7.46–15.10 s, while StopHarm's stays at 0.012 s; StopHarm's P95 barely moves from the low rate, so the gain widens.

The speedup is not bought with accuracy: accuracy stays within 1.29 pp of Full-CoT, and the lost-correct rate is 1.36% (18/1,319) under the reference protocol, below α = 5%.

Revision. The revised paper (changes in blue) adds the offline cost analysis (Appendix F.8), the ARC-Challenge recertification with its theory (Section 5.6 and Appendix E.4), and the continuous-batching evaluation (Appendix F.9).


We hope these responses address the reviewer's concerns, and we are happy to discuss any further questions. If they are helpful, we would be grateful if the reviewer could take them into account in the final assessment.

Dear Reviewer 3kFA,

Thank you for your constructive feedback and for finding our idea simple and well motivated. In our rebuttal, we:

  • clarified the novelty: StopHarm learns what stopping would lose rather than whether the current answer is correct, which no post-processing of a proxy score can recover (Theorem 1);
  • quantified the offline cost: under 10 GPU·h per model, recovered after 3.3K–6.2K requests;
  • showed transfer: recertifying one threshold restores the guarantee on ARC-Challenge, and the frozen policy stays within 1 pp of Full-CoT after model updates;
  • evaluated continuous batching in vLLM: 2.2–2.4× throughput and 61%–68% lower P95 latency.

We hope these clarifications address your concerns. We would sincerely appreciate it if you could take our response into account and consider a more positive evaluation of our work. We are happy to answer any further questions.

Dear Reviewer 3kFA, as the discussion period is about to close, we would be very grateful if you could let us know whether our responses on novelty, offline cost, transferability, and continuous batching have addressed your concerns. Thank you again for your time and valuable feedback.

Reviewer_78PLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose StopHarm, an early stopping method for CoT reasoning. It directly supervises the certified loss, modeling the possibility of leading to inaccuracy when stopping at each step. It learns a lightweight predictor from paired stopping and full-reasoning outcomes offline and stops online when the predicted stop harm stays below a certified threshold. The experiment shows that StopHarm achieves a trade-off between generating speed and accuracy, and it's comparable to previous methods.

Critical strengths
  1. I like the idea; it’s straightforward: directly supervise the stop harm, modeling the possibility of leading to inaccuracy when stopping at each step.

  2. The theoretical analysis provides a complete justification for directly supervising stop harm by showing that proxy targets can lose information relevant to stopping decisions.

Critical weaknesses
  1. I’m concerned that this method is not actually practical. Although StopHarm achieves a speedup, it still drops accuracy a little bit. For example, on MATH500, accuracy decreases significantly by 2.6%-5.4%.

  2. The performance is not competitive enough compared with other methods. For example, on MATH500 using Qwen3-14B, both BLADE and LYNX achieve higher accuracy using fewer tokens than the paper’s method. The paper does not explain this result.

  3. The paper's method is trained and calibrated on MATH and then used to test on other math datasets as well. But StopHarm shows the largest accuracy decrease on MATH500 compared to other datasets, without explanation. Does the author have any theory and empirical guarantee that the calibration learned in one data source domain can generalize to other domains?

Questions for authors

Please see the questions in the weaknesses noted above.

4 comments
AuthorsOpenReview ↗

Response Part 1/2

We sincerely thank the reviewer for finding our idea of directly supervising stop harm straightforward and our theoretical analysis a complete justification for it. All three comments concern the accuracy drop on MATH500, so we first address their common cause and then each point.

Common cause: strict grading on MATH500

MATH500 is the only benchmark with free-form expression answers, and Table 2 grades it strictly, counting some mathematically equivalent answers as losses. We clarify this under-explained setting and add two analyses: (i) regrading with all stopping decisions fixed, and (ii) relabeling, retraining, and recalibrating under the updated grader.

Grading setup. AIME, GSM8K, and GPQA Diamond have integer, numeric, or option answers, which the original and updated graders judge identically. The original grader (earlystop/grading.py in the supplementary material) compares the normalized last \boxed{} with the reference without symbolic-equivalence fallback (documented in the README), so \boxed{3/2} is marked wrong against \frac{3}{2}. Lenient grading appeared only as a robustness check of the mechanism analysis (Appendix Table F.4), not in the main results.

Strict grading penalizes early stopping more than full reasoning. Regrading with stopping decisions fixed (regraded answers manually verified) raises Full-CoT accuracy by 1.0 / 1.4 pp but StopHarm accuracy by 2.4 / 2.4 pp on the two Qwen models; 5 / 6 of the 15 / 22 answers counted as lost are mathematically equivalent to the reference. Early-stopped answers are more often written as decimals or unformatted fractions, which strict matching counts as losses. This factor alone reduces the accuracy loss from 2.6 / 4.0 pp to 1.2 / 3.0 pp.

Retraining under the updated grader. Labels are regenerated, and all three methods are retrained and recalibrated with the original recipe.

ModelMethodSubmission Table 2 (strict)Updated grader (retrained + recalibrated)
Qwen3-4BStopHarm2.6 / 24.502.8 / 36.12
Qwen3-4BLYNX3.0 / 33.513.6 / 32.68
Qwen3-4BBLADE3.0 / 32.902.6 / 31.02
DeepSeek-7BStopHarm5.4 / 33.952.2 / 26.73
DeepSeek-7BLYNX6.0 / 28.223.6 / 30.82
DeepSeek-7BBLADE5.4 / 27.144.8 / 45.38
Qwen3-14BStopHarm4.0 / 29.352.6 / 34.42
Qwen3-14BLYNX3.8 / 37.392.8 / 31.72
Qwen3-14BBLADE1.6 / 32.893.8 / 35.06

Each cell: accuracy loss relative to Full-CoT (pp) / token reduction, TR (%). Full-CoT accuracy under the updated grader is 92.8% / 94.6% / 93.2% (91.8% / 92.4% / 91.8% in the submission) for Qwen3-4B / DeepSeek-7B / Qwen3-14B.

StopHarm's lost-correct rates become 3.8% / 2.2% / 3.0%, all below α=0.05. No baseline beats StopHarm on both accuracy and TR for any model: StopHarm has the smallest loss on Qwen3-14B and DeepSeek-7B and the largest TR on Qwen3-4B.

The other four benchmarks are unaffected, so their Table 2 numbers stand. Table 2 trains, calibrates, and tests under strict grading and is therefore internally consistent; strict grading merely yields a more conservative MATH500 operating point (TR 24.5% / 29.4% → 36.1% / 34.4% for the two Qwen models at the same α). Transferring the retrained methods directly to GSM8K as in Table 2 gives consistent conclusions:

ModelStopHarmLYNXBLADE
Qwen3-4B0.91 / 66.300.15 / 23.452.20 / 54.58
DeepSeek-7B0.23 / 46.430.76 / 41.020.68 / 52.24
Qwen3-14B0.91 / 53.430.30 / 15.010.61 / 23.48

Each cell: accuracy loss (pp) / TR (%).

StopHarm keeps the loss within 1 pp (0.5–0.7 pp in Table 2) at 46.4%–66.3% TR, the largest TR on both Qwen models (3.6× LYNX and 2.3× BLADE on Qwen3-14B).

AuthorsOpenReview ↗

Response Part 2/2

W1 — Practicality: the loss is set by α

The accuracy change is governed by the user-specified risk budget α; changing α only requires recertifying a threshold, not retraining the probe. Results under the updated grader:

αMATH500 · Qwen3-4BMATH500 · Qwen3-14BAverage over 5 benchmarks × 3 models
0.0050.0 / 0.000.4 / 3.07−0.01 / 1.35
0.010.2 / 4.810.4 / 4.570.22 / 4.29
0.021.0 / 16.031.2 / 18.890.54 / 11.19
0.031.6 / 23.491.8 / 24.510.78 / 17.53
0.05 (default in the paper)2.8 / 36.122.6 / 34.420.72 / 25.43
0.106.2 / 54.856.2 / 60.492.71 / 38.14

Each cell: accuracy loss (pp) / TR (%).

A smaller α yields a smaller loss while retaining substantial TR. On MATH500 the loss stays below α at every level: tightening α from 0.05 to 0.03 cuts it from 2.8 / 2.6 to 1.6 / 1.8 pp at 23.49% / 24.51% TR; at 0.02 it is 1.0 / 1.2 pp at 16.03% / 18.89% TR (0.54 pp averaged over all benchmarks and models). The accuracy drop is thus an operating point chosen by the user, not an inherent cost of StopHarm.


W2 — Competitiveness: StopHarm leads after regrading

Among the 15 cells of Table 2, MATH500 on Qwen3-14B is the only one where both baselines lead, and it is likewise affected by strict grading; after regrading, StopHarm has the smallest loss there (see the first table). Elsewhere its advantage is clearer: on Qwen3-14B it attains the highest accuracy on GPQA Diamond with 2.1× BLADE's TR, and the best accuracy and TR on the five-benchmark average.

Qwen3-14BStopHarmLYNXBLADE
GPQA Diamond64.7 / 45.6663.6 / 2.4961.6 / 21.40
Five-benchmark average74.9 / 26.7174.6 / 20.6472.3 / 25.84

Accuracy (%) / TR (%).


W3 — MATH500 drop and cross-domain generalization

MATH500 is the only in-distribution test set, so the certified policy spends its risk budget there by design. For a new domain, recertifying one threshold restores the guarantee without retraining.

Why MATH500 drops most. The larger drop is not specific to StopHarm: after regrading, LYNX and BLADE lose even more on MATH500 (3.3 / 3.7 pp vs. 2.5 pp for StopHarm, averaged over models). Besides strict grading (see above), two properties of the setting, shared by all methods, explain it:

  1. The budget is spent where it is certified. Calibration picks the cheapest threshold whose LC is certified below α on MATH, so MATH500 uses the budget for savings, and the other benchmarks inherit a threshold that is conservative for them. Certifying the threshold on GSM8K itself (frozen probe, fresh trajectories) raises TR from 56.4% / 47.0% to 64.5% / 62.4% and the loss from 1.6 / 0.9 to 2.6 / 2.9 pp (Qwen3-4B / DeepSeek-7B), on par with MATH500.
  2. MATH500 offers little harmless saving. Stopping cannot cause LC on requests that full reasoning already gets wrong. On MATH500 these supply only 5%–14% of the savings (49%–71% on GPQA Diamond), so nearly all savings come where the LC budget binds.

Theory (Appendix E.4). Proposition E.1 guarantees LC ≤ α on the calibration distribution. Beyond it:

  • With target data: recertifying only the threshold of the frozen probe restores LC ≤ α exactly (Corollary E.1).
  • Without target data: under bounded shift in the probe's score trajectories and stop outcomes, LC ≤ Bα for a density ratio B (Proposition E.2).
  • With neither: no calibration-based method can certify an arbitrary unseen domain (Proposition E.3), so transfer must be shown empirically.

Evidence: StopHarm transfers. (i) Direct transfer: with the probe and threshold fit on MATH only, StopHarm saves 30.4% of tokens on the other four benchmarks of Table 2 with no average accuracy loss (+0.5 pp vs. Full-CoT). (ii) Recertification on ARC-Challenge (multiple-choice science, α = 0.03):

Setting (probe / threshold)Qwen3-4B: LC / TRDeepSeek-7B: LC / TR
A: MATH / MATH (direct transfer)4.52 / 75.000.68 / 22.24
B: MATH / ARC (recertify threshold)3.33 / 60.442.39 / 46.29
C: ARC / ARC (retrain)2.43 / 51.911.77 / 47.36

LC: lost-correct rate (%); TR: token reduction (%).

Even without ARC data, direct transfer (A) saves 75.00% / 22.24% of tokens. Recertifying only the threshold (B) then fits it to ARC: Qwen3-4B's LC falls from 4.52% to 3.33%, and DeepSeek-7B's TR doubles from 22.24% to 46.29%. B is on par with retraining (C), so the MATH-trained probe itself transfers.

Revision. The revised paper (changes in blue) adds the grading setup and the updated-grader results (Appendices F.1 and F.7), states the scope of the guarantee (Sections 3.2 and 5.1), and adds the transfer theory (Appendix E.4) and the recertification results (Section 5.6 and Appendix F.11).


We hope these responses resolve the reviewer's concerns and are happy to address any further questions. We would be grateful if they could be reflected in the final assessment.

Dear Reviewer 78PL,

Thank you for your constructive feedback and for liking our idea of directly supervising stop harm. In our rebuttal, we:

  • showed that strict grading on MATH500 counts mathematically equivalent answers as losses; under the updated grader, StopHarm leads the baselines with the smallest accuracy loss on DeepSeek-7B and Qwen3-14B and the largest token reduction on Qwen3-4B, while keeping its lost-correct rate below α;
  • showed that the loss is controlled by the risk budget α, e.g., 1.0–1.2 pp on MATH500 at α = 0.02;
  • added transfer theory and ARC-Challenge results: recertifying one threshold restores the guarantee on a new domain without retraining.

We hope these clarifications address your concerns. We would sincerely appreciate it if you could take our response into account and consider a more positive evaluation of our work. We are happy to answer any further questions.

Dear Reviewer 78PL, as the discussion period is about to close, we would be very grateful if you could let us know whether our responses on MATH500 grading, the risk budget α, and cross-domain transfer have addressed your concerns. Thank you again for your time and valuable feedback.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their constructive feedback. We have addressed all concerns with new experiments, theory, and revisions (marked in blue), and Reviewer v4Yc has confirmed that their questions are resolved.

Key outcomes of the rebuttal

  • The MATH500 drop is controllable, and StopHarm leads after regrading (78PL). Strict grading of MATH500's free-form answers counted equivalent answers as losses. Under the updated grader, StopHarm keeps its lost-correct rate below α on all three models and leads the baselines; tightening α lowers the loss to 1.0–1.2 pp without retraining.
  • The gains come from supervising stop harm directly (3kFA, v4Yc). Changing only the supervision target roughly halves the lost-correct rate at matched cost (2.5% vs. 4.7%–5.1%), and at the same accuracy as LearnStop on Qwen3-14B/GSM8K, StopHarm saves 47.3% vs. 26.9% of tokens.
  • The guarantee extends beyond the calibration task (78PL, 3kFA, v4Yc). New theory shows that recertifying one threshold restores the guarantee on a new task without retraining; empirically, 15 of 16 recertified policies keep the lost-correct rate below α, the frozen policy stays within 1 pp of Full-CoT after model updates, and online certification removes the need for a calibration set.
  • The savings are cheap to obtain and hold in serving (3kFA). The one-time offline cost (under 10 GPU·h per model) is recovered within 3.3K–6.2K requests, and under vLLM continuous batching StopHarm achieves 2.2–2.4× throughput and 61%–68% lower P95 latency.

Why StopHarm matters

For early stopping, only one failure matters: losing an answer that full reasoning would get right. This event bounds the accuracy drop and is what certification controls, yet existing learned stoppers are trained on proxy questions such as is the current answer correct? StopHarm learns the question that is actually certified, what would be lost by stopping now? This changes decisions exactly where it matters: when full reasoning would fail anyway, stopping loses nothing, so StopHarm stops where a correctness-based stopper keeps reasoning; when further reasoning would fix the answer, StopHarm keeps going.

  • Idea: the reviewers found directly supervising stop harm "straightforward" (78PL), "simple and well motivated" (3kFA), and "interesting" (v4Yc).
  • Theory: the gap is fundamental: even Bayes-optimal proxy scores can discard stop-harm information that no post-processing can recover, "a complete justification" for direct supervision (78PL).
  • Results: across three models and five benchmarks, StopHarm cuts generated tokens by 25.4%–38.3% with average accuracy drops of only 0.27–0.73 pp; the reviewers found the experiments "comprehensive" (v4Yc), with "substantial token savings while controlling accuracy loss" (3kFA).

StopHarm needs only a lightweight probe on the frozen model, generates no extra answers during decoding, and lets users set the bound α on the accuracy drop in advance. More broadly, it shows that for risk-controlled early stopping, what is supervised matters as much as how it is certified. We thank the Area Chair and reviewers again and hope these results can be taken into account in the final assessment.

We believe our responses have addressed the reviewers’ concerns, and no further questions or concerns have been raised in the discussion; we would be glad to clarify anything further. We are also grateful for the reviewers’ suggestions, which have made the paper more complete and concrete: the revised paper now specifies when the guarantee holds and how to restore it on new tasks, and quantifies what StopHarm costs offline and saves in serving, further demonstrating its practical value. We thank the Area Chair and reviewers again and hope our responses and the revised paper can be taken into account in the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.