acceptodds
Under review as a conference paper at ICLR 2027

It Has the Answer and Keeps Going: How Fine-Tuning Breaks the Decision to Stop

Abstract

Fine-tuning on long reasoning traces is how most open reasoning models are built, and its students often fail to finish: ours runs to the budget on half of its AIME 2025 generations. The cause is a learned stop decision, which fine-tuning can break while leaving reasoning intact. Serve a Qwen3-8B student without its eleven-token, task-free system turn and its computed accuracy moves points while its delivered accuracy falls : it boxes the correct answer early and keeps going. One principle explains every failure we find: a distilled student stops only under conditions that accompanied every stop it was shown. Three causes follow, each confirmed by controlled intervention. (i) An end-of-turn token dropped from supervision is silenced ( of generations) while a standard harness scores both checkpoints alike. (ii) A stop tied to the training prompt fails at the two closing tokens; editing their probabilities restores termination, and training on two formats prevents it. (iii) Every demonstration stops on a converged answer, so a failing student, whose candidate answers never settle, stops after only of its wrong answers (base: ), a pattern of released long-CoT models share. Trained on stops alone, the student collapses; mixed with original data, a moderate dose halves wrong-and-capped generations at no significant accuracy cost on one training seed, and stronger doses abandon solvable problems. Terminator, one guard per cause, terminates non-termination at inference: on every student we trained and benchmark we tested, capping falls to zero and delivered accuracy rises ( points under the training format, without the system turn; also pooled over released checkpoints) while computed accuracy never falls; budget forcing, which also forces a commitment, does as well, as the principle predicts. Reported termination is partly a property of the harness: four standard defaults span points for one checkpoint.

Primary area:
foundation or frontier models, including LLMs
Keywords:
reasoning models, long chain-of-thought, supervised fine-tuning, distillation, termination, overthinking, stopping behavior, evaluation harness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vzFcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies non-termination in long-reasoning models after supervised fine-tuning. It distinguishes between computed accuracy, where a correct answer appears somewhere in the generation, and delivered accuracy, where the model also closes its reasoning and terminates naturally. The paper attributes non-termination to three causes: the end-of-turn token may be excluded from supervision; stopping may become dependent on the training prompt format; and the training data may contain stops only after converged answers, leaving the student unable to stop from unsuccessful or unsettled states. The authors investigate these failure modes through training ablations, prompt perturbations, token-probability interventions, and evaluations across several model families. They further propose TERMINATOR, a collection of data, format, and decoding guards intended to prevent these failures. The distinction between computing an answer and successfully delivering it is useful, and the paper contains several careful controlled experiments.

Critical strengths
  • The paper identifies an important evaluation blind spot: an answer-extraction-based evaluator may mark a generation as correct even when the model fails to terminate and never cleanly delivers the answer to the user.
  • Several controlled experiments are informative. In particular, the end-of-turn supervision ablation provides a clean demonstration of Cause (i), while the closing-token probability intervention offers useful evidence that prompt-format sensitivity can be localized near the stopping decision in the authors' models.
  • The paper examines multiple model initializations, prompt formats, benchmarks, and released checkpoints. It also reports a substantial amount of implementation detail, negative evidence, and limitations in the appendix.
Critical weaknesses
  • The manuscript has severe readability and presentation problems. Several important concepts are expressed using uncommon or paper-specific terms, without sufficiently clear explanation before they are used. The extensive use of italics, boldface, small caps, and other visual annotations makes the paper difficult to read. Some passages resemble compressed notes written by someone who already knows the experiments, rather than an explanation intended for readers without prior knowledge of the authors' experimental setup. The figures are an even larger problem. For example, the introductory figure is visually cluttered, contains too much text, mixes several kinds of information, and is difficult to understand even after reading its long text. The authors need to take the readability of the paper seriously, ensure that it is easy for humans to read, and manually revise the entire manuscript.
  • The practical significance of Cause (ii) is unclear. The paper acknowledges that the required conditions are nearly mutually exclusive among the 31 released checkpoints examined, and none clearly exhibits this proposed failure. The experiments show that Cause (ii) can occur in the authors' controlled setting, but not that it is prevalent in practice. Its status as a central cause, and the substantial main-text space devoted to it, therefore seem disproportionate to the evidence.
  • Cause (iii) is an interesting hypothesis, but the evidence is insufficient to establish causality. Position matching and probing do not fully separate missing failure-state supervision from difficulty, confidence, or distribution shift. A clean training control should hold the remaining data fixed, add natural examples of stopping after failure, and demonstrate stable recovery across training seeds. The current evidence supports an association, but not yet the full causal claim.
  • A central result of TERMINATOR is guaranteed by construction. The decode guard appends closing syntax and forces an end-of-turn token at a fixed position, so reducing capping to zero is not an empirical finding. Delivered accuracy also partly improves because the wrapper rewrites an intermediate output into an evaluator-compatible answer. This may be a useful serving heuristic, but it should not be presented as recovery of the model's learned stopping ability.
  • Table 4 overstates the completeness of the evidence. Each intervention is evaluated on only a subset of models and benchmarks, leaving many cells empty, while different rows report incompatible quantities such as absolute results, format gaps, relative changes, and pooled results. This is both difficult to interpret and insufficient to support the table's broad claim that every guard removes its corresponding failure.
Questions for authors
  • Given that Cause (ii) was not observed as the proposed failure among the 31 released checkpoints, what evidence justifies presenting it as one of the three central causes rather than as a pipeline-specific case study?
  • Can the authors provide a controlled, replicated training experiment that adds realistic stopping demonstrations from unsuccessful or unconverged states while holding the remainder of the training data fixed? What outcome would distinguish the proposed explanation for Cause (iii) from difficulty or confidence effects?
  • How much of TERMINATOR's reported gain remains under a comparison with simple fixed-position truncation followed by extraction and delivery of the last available answer?
  • Why are so many intervention-model-benchmark cells in Table 4 missing, and which are genuinely inapplicable rather than simply not evaluated? Do the headline claims still hold under a complete evaluation matrix and consistent reporting convention?
4 comments

Thank you for recognizing the evaluation blind spot and controlled interventions (* = significant; n.s. = not significant; old→new names: global response).

W1. … The extensive use of italics, boldface, small caps, and other visual annotations makes the paper difficult to read. … the introductory figure is visually cluttered, contains too much text …

We rewrote the main text, redrew Figure 1 and cut Figure 5 to two panels; pages 1–2 show the change directly.

  • In the main text, paper-specific labels (EXACT, capping, TERMINATOR) are replaced by plain terms defined in Table 1.
  • Emphasis in running prose is almost entirely removed; the abstract is 303→225 words and main prose about a quarter shorter.
  • Revised Tables 4–6 mark levels (%) and paired changes (pp) explicitly, with each training experiment beside its own baseline.

Q1. Given that Cause (ii) was not observed as the proposed failure among the 31 released checkpoints, what evidence justifies presenting it as one of the three central causes rather than as a pipeline-specific case study?

Agreed: Section 4 now presents Cause (ii) as a demonstrated pipeline hazard in controlled students, not a prevalent cause. We keep the section because it carries the localization evidence: deleting an 11-token system turn cuts natural delivery 29.2* pp (computed −0.6 pp, n.s.), and restoring two closing-token probabilities on the same prefixes repairs 7B/8B students (null at 32B). Prevalence is not shown: only 4 of 31 public checkpoints meet its conditions (raw-base fine-tune with a removable system prompt), and none meets our failure criterion.

Q2. Can the authors provide a controlled, replicated training experiment that adds realistic stopping demonstrations from unsuccessful or unconverged states while holding the remainder of the training data fixed? What outcome would distinguish the proposed explanation for Cause (iii) from difficulty or confidence effects?

We ran a controlled, replicated version; the tested augmentation did not restore stopping, so Cause (iii) remains a hypothesis. With all 30,000 base traces byte-identical, three arms (base; +1,500 pre-answer teacher prefixes closed with a give-up stop; +1,500 full traces on the same problems) span three training seeds, three benchmarks and two formats (Table 22). The give-up rows are a proxy: an answer-less prefix need not be a failed attempt, and over 99.5% of their supervised tokens are prefix.

Give-up arm minus base (pp)Result
MATH-500 stop-after-wrong (training format), per seed+0.8, +3.4, +1.4 (95% upper bounds +5.2, +8.2, +6.3; target +10)
Same outcome, pooled over seeds+1.9 (n.s.)
No-system delivery, MATH / AIME / GPQA−13.3* / −5.0* / −3.5*

What would distinguish it: recovery beyond both control arms without delivery loss, plus real failure states and length- and difficulty-matched controls to exclude difficulty or confidence (open).

W4. A central result of TERMINATOR is guaranteed by construction. …

Q3. How much of TERMINATOR's reported gain remains under a comparison with simple fixed-position truncation followed by extraction and delivery of the last available answer?

Agreed: zero unfinished runs is by construction, so we report the guard as a serving policy, not learned stopping. Against bare truncation plus extraction, commitment adds +4.0*/+4.2* pp delivery. Main-student MATH-500, threshold at 75% of budget:

Delivery change (pp)TrainingNo-system
Bare truncation − untouched+0.6+17.5*
Commitment added to truncation+4.0*+4.2*

The first row is the rewriting you describe: delivering an answer already boxed in the thought. Without the system turn it supplies most of the gain (+17.5* of +21.7* pp). Commitment (eliciting an answer in ≤64 tokens when none is boxed) beats truncation on computed accuracy in 17/18 controlled settings (16*; Table 86), though on GPQA a forced four-option guess can contribute. On 500 new MATH problems (training format, one checkpoint, four decoding seeds; threshold frozen), the guard raises delivery by 5.05* pp, commitment adds 3.75* pp over truncation, and truncation alone loses 1.55* computed pp (Appendix A20).

Q4. Why are so many intervention-model-benchmark cells in Table 4 missing, and which are genuinely inapplicable rather than simply not evaluated? …

Blank cells mixed three cases, now separated; the guard's 18 controlled settings are complete, and the "every guard removes its failure" title is gone.

  • Unreported: main/science AIME and GPQA computed changes, now in Table 36.
  • Unevaluated: four 32B AIME/GPQA cells, now run; guard delivery rises in 18/18 settings (16*).
  • Inapplicable: boxed-answer commitment on LiveCodeBench code (na).
  • MATH-500 only: s1-style forcing and the 20 public checkpoints (Table 35).

Thank you for the detailed rebuttal, additional experiments, and substantial revisions. These efforts have alleviated some of my concerns. However, the additional evidence does not fully resolve my concerns about the causal account and the scope of the contribution. Moreover, the presentation still requires further revision. I will maintain my original rating.

AuthorsOpenReview ↗

Clarifying the causal evidence and contribution

Thank you for recognizing the improvements. We would like to clarify how the remaining causal uncertainty relates to the paper’s contribution.

1. The central findings have controlled intervention evidence (§§3–4).

  • Terminal supervision: masking only the chat turn-end target reduces its emission from 81.7% to 0%, while a permissive evaluation stop set can conceal this failure.
  • Prompt-dependent closing: on the same generated prefixes, restoring only two closing-token probabilities reduces unfinished responses from 31.6% to 16.6%; matched interventions on unrelated tokens leave them at 30.0%/32.8%.

These results identify specific sources of failed completion in the tested settings. The 32B null result and absence of the format failure in four eligible public releases delimit the scope of the second finding.

2. The training cause of persistence on incorrect attempts remains open (§5).

The constructed stopping demonstrations are proxies for unsuccessful reasoning states. Their negative result does not settle the realistic failed-state hypothesis or distinguish it from difficulty and confidence effects. Our conclusions in Sections 3–4 do not depend on that hypothesis.

3. The contribution is a diagnostic finding with practical consequences.

Deleting a task-free system turn changes extracted-answer accuracy by only −0.6 percentage points, yet lowers correct, naturally completed responses by 29.2 points. The controlled interventions explain specific failures that answer-only evaluation misses, motivating terminal-supervision checks and separate measurement of correctness and natural completion.

Could you identify the specific causal claim or passage in the revised main text that remains unsupported or unclear? If your concern is primarily the significance of these bounded findings, rather than their evidential support, that distinction would help us focus the remaining discussion.

AuthorsOpenReview ↗

The causal account rests on two single-variable interventions

the additional evidence does not fully resolve my concerns about the causal account and the scope of the contribution

The causal account rests on two interventions, each changing one thing (§§3–4).

  • Removing loss from one token, the chat turn-end, takes its emission from 81.7% to 0.0%.
  • Restoring two closing-token probabilities on the same generated prefix cuts unfinished responses from 31.6% to 16.6%; the same edit on two unrelated tokens gives 30.0% and 32.8%.

The scope is stated in the abstract. The second result replicates in 7B/8B students, is absent at 32B, and no eligible public release fails. The one account without an intervention, persistence on incorrect attempts, is labelled open (§5).

Your review called the first "a clean demonstration". Which causal claim in the revised text remains unsupported?

Reviewer_c8BGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies why fine-tuned reasoning models sometimes fail to stop, even when a correct answer already appears in their reasoning. It examines three possible causes: missing supervision on ending tokens, changes in prompt format, and difficulty stopping when reasoning does not reach a settled answer. The authors propose training checks, changes to the training data, and an inference rule that forces a final response before the token budget runs out.

Critical strengths

S1. The controlled experiments on prompt format are interesting and have useful lessons.

Critical weaknesses

W1. I am not convinced that the contribution is sufficiently significant relative to existing work. Training with varied instructions to improve generalization [1] and ending the thinking stage at a fixed token budget [2] have clear precedents. Prior work also studies poor stopping and repetitive reasoning after training on long reasoning traces [3]. The strongest contribution seems to be the detailed analysis of how prompt changes affect stopping. This is interesting, but the paper does not clearly establish what broader understanding it adds beyond these existing observations and fixes.

[1] Lin et al. Debunk the Myth of SFT Generalization. [2] Muennighoff et al. s1: Simple test-time scaling. [3] Zhang et al. Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement.

W2. The evidence supports a narrower set of findings than the paper claims to present. Missing ending-token supervision, sensitivity to prompt format, and failure to finish unsuccessful reasoning could be separate problems. It is unclear that the experiments establish one common explanation for all three. Also, the closing-token intervention does not repair the 32B model.

W3. Some claims about the repairs are stronger than the results support. The fixed stopping position is selected using MATH-500 evaluation results, so its performance on those same problems is difficult to interpret as an independent test. Also, “no accuracy cost” is too strong. The repair lowers GPQA accuracy by 2.8 points; a confidence interval that includes zero does not establish that accuracy is preserved.

W4. The presentation makes the contribution unnecessarily difficult to assess. The abstract combines several problems, interventions, and results without a clear main message. The paper then frequently switches between models, prompt formats, and definitions of accuracy. Claims like “one principle explains every failure” also make the conclusions sound more established than they are.

Questions for authors

Q1. What evidence supports treating the three failure modes as one general phenomenon rather than separate training and inference problems?

Q2. Have you evaluated the inference repair with a stopping position chosen on a separate validation set? Testing another decoding seed on the same MATH-500 problems does not address whether the selected position generalizes to new problems.

6 comments

Thank you for highlighting our prompt-format experiments' practical value (* = significant; n.s. = not significant; old→new names: global response).

W1. … The strongest contribution seems to be the detailed analysis of how prompt changes affect stopping. This is interesting, but the paper does not clearly establish what broader understanding it adds beyond these existing observations and fixes.

What is new: after SFT, task-free prompt structure can decide whether a turn completes while extracted-answer accuracy barely moves, so answer-only scoring misses it. Lin et al. improve robustness to changed task mappings, s1 forces stops at a budget, and Zhang et al. study sparse EOS supervision and EOS-logit interventions. We hold task wording fixed and test closing tokens on the same prefix, one change per row:

Controlled changeObserved effect
Mask chat turn-end targetEmission falls 81.7%→0%; accepting end-of-text hides it (§3)
Delete 11-token system turnComputed −0.6 pp (n.s.); natural delivery −29.2* pp
Restore only the two closing-token probabilitiesUnfinished 31.6%→16.6%; editing two other tokens: 30.0%/32.8%
Impose no-system closing probabilities on the training formatUnfinished 18.2%→49.0% (500-problem cohort, Table 27)
Restore only at likely closing stepsRecovers the whole gap; constant +1/+2/+3-logit boosts recover at most half (200 problems, Table 62)

The turn-end failure also occurs in real pipelines (TRL 0.24 on over-length examples; an audited Megatron-Bridge pipeline; §3). We claim no better forcing method: on main-student training-format MATH-500, s1-style forcing at the full budget delivers 1.6* pp more than our guard but uses 340* more tokens per query (§6).

W2. … Missing ending-token supervision, sensitivity to prompt format, and failure to finish unsuccessful reasoning could be separate problems. … Also, the closing-token intervention does not repair the 32B model.

Q1. What evidence supports treating the three failure modes as one general phenomenon rather than separate training and inference problems?

We agree: they are separate problems that share a symptom, and "one principle explains every failure" is removed. Turn-end masking (§3) and prompt-conditional closing (§4) respond to different interventions; the second is not shown to be prevalent (0 of 4 eligible public releases meet our failure criterion). At 32B the two-token intervention is null (MATH-500 no-system unfinished 54.2%→54.2%, 120 problems), so the localization does not extend there. For persistence on wrong attempts, 1,500 constructed give-up demonstrations added to an unchanged 30k corpus raised the stop rate of wrong attempts by only +1.9 pp (n.s.; three training seeds; preregistered target +10; Table 22), so its cause stays open.

W3. … The fixed stopping position is selected using MATH-500 evaluation results, so its performance on those same problems is difficult to interpret as an independent test.

Q2. Have you evaluated the inference repair with a stopping position chosen on a separate validation set? …

Yes. Treating MATH-500 as the selection set, we froze its threshold and tested 500 new MATH problems: delivery rises from 81.30% to 86.35% (+5.05* pp, the prespecified endpoint). Frozen before generation: threshold 6,144 of 8,192 tokens, ≤64 answer tokens (Appendix A20). Limits: one checkpoint, four decoding seeds; lexical overlap screening, not pretraining decontamination; boxing can contribute.

Change vs. untouched (pp)TrainingNo-system
Guard, delivery+5.05*+20.10*
Guard, computed+2.20*+3.60*
Bare truncation, computed−1.55*−1.30*

W3. … Also, “no accuracy cost” is too strong. The repair lowers GPQA accuracy by 2.8 points; a confidence interval that includes zero does not establish that accuracy is preserved.

Agreed; the claim is removed, and unified scoring shows the cost is real. That 2.8-point loss (the submitted "best give-up training dose", now suffix-only training with replay) becomes a significant −4.2* pp computed and −4.0* pp delivery loss in the training format; the teacher-prefix variant loses 7.4* pp of delivery (Table 5). Mixed-format replay's computed changes (−1.1/−1.4 pp, n.s.) are inconclusive, not preserved accuracy (Table 55).

W4. … The abstract combines several problems, interventions, and results without a clear main message. … Claims like “one principle explains every failure” also make the conclusions sound more established than they are.

The paper now leads with one message: extracted-answer accuracy can hide unfinished turns. The abstract (303→225 words) and main prose (about a quarter shorter) define metrics first, pair negative with positive results, and separate training experiments (Tables 4–5) from serving policies (Table 6).

Thanks for the effort in rebuttal. I think the paper needs a substantial update to be in human-readable form. I think the appendix should support missing claims in the main paper. The current 82-page total submission makes it hard for the paper to be reviewed by humans. Overall, I remain unconvinced that the proposed repairs offer a sufficiently significant contribution over existing approaches. I will maintain my score.

AuthorsOpenReview ↗

Revised manuscript: 82 → 30 pages and clarified contributions

Thank you for clarifying the remaining concerns. Since your comment, we have further revised the manuscript and updated our point-by-point response.

1. We reduced the submission from 82 to 30 pages, with 10 pages of main text. The appendix now follows the claims in the paper: reproduction settings, controlled stopping experiments, training interventions, and serving comparisons. We removed repeated discussion and peripheral analyses. It retains the closing-token controls and 32B null, the corrected GPQA comparisons, the three-training-seed experiment, individual public-model results, and the fixed-threshold test on 500 new problems.

2. The main contribution is a controlled diagnosis of a failure that answer-only evaluation misses. The decisive comparison is:

TestResult
Delete an 11-token, task-free system turnExtracted accuracy changes by −0.6 pp; natural delivery falls by 29.2 pp (Figure 1).
Restore only the two closing-token probabilities on the same generated prefixesUnfinished responses fall from 31.6% to 16.6%; two unrelated-token controls leave them at 30.0% and 32.8% (Figure 2).

Together with the reverse and gated controls, these experiments show that prompt changes can disrupt turn completion while leaving extracted-answer accuracy nearly unchanged, and localize that failure in the tested 7B/8B students. The 32B null explicitly limits this explanation.

3. Comparisons with existing repairs quantify the delivery–cost trade-off. On main-student training-format MATH-500, s1-style forcing at the full budget delivers 1.6 pp more than our guard and uses 340 more tokens per query (§6). On 500 new MATH problems with a frozen threshold (training format; one checkpoint, four decoding seeds), the guard improves delivery by 5.05 pp over untouched decoding (the prespecified endpoint; 95% CI: +3.80 to +6.30). As a secondary comparison, the answer step adds 3.75 pp over bare truncation (95% CI: +2.80 to +4.75; Table 28); some of this gain can come from boxing an existing answer. The training comparisons also expose costs: the constructed stopping augmentation misses its recovery target in all three training seeds, and suffix-only training with replay loses a significant 4.2 points of training-format GPQA extracted accuracy (Tables 17 and 5).

The practical implication is that robustness in answer accuracy does not ensure robustness in answer delivery. The contribution is the controlled diagnosis and evidence about which repairs address each failure, with their measured gains and costs. The main text presents these findings directly; the shortened appendix supports them.

Reviewer_Km53OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates termination failures after long-CoT SFT, distinguishing answer correctness from successful answer delivery. It studies missing terminal-token supervision, prompt-format dependence, and answer-conditioned stopping through training ablations, closing-token interventions, and evaluations of locally trained and released models. It proposes data checks, format augmentation, and a decoding guard as remedies.

Critical strengths
  1. The closing-token intervention is a substantive diagnostic contribution. In Table 56, restoring the closing-token probabilities reduces capping from 31.6% to 16.6%, whereas the two sham interventions leave it at 30.0% and 32.8%. Together with the reverse and gated interventions, this provides convincing evidence for a localized termination failure in the studied students.

  2. The findings have practical relevance beyond the custom evaluation setup. The standard-serving replication reproduces the termination failure, and format-diverse training removes the main student's format gap without a comparable reduction in accuracy. These results support concrete improvements to training and deployment practice.

Critical weaknesses
  1. The GPQA accuracy results are not consistently reconciled after the scoring correction. Table 26 explicitly states that the letter judge undercounts correct answers by approximately half. Nevertheless, Table 50 retains the corresponding accuracy results, describes performance as near chance, and reports significant repair D losses of 3.0 and 6.1 points. Meanwhile, Table 4 reports a 48.7% EXACT baseline, compared with Table 50's 20.8%. Different judges can explain these numbers, but the corrected repair D comparison is not supplied. Consequently, the reader cannot determine whether the reported accuracy losses survive correction or assess the repair's cross-domain accuracy preservation. This is more than a presentation inconvenience.

  2. The reported remedy results do not permit verification of the broad efficacy claims. Section 6 states that delivered accuracy improves for every student and benchmark, without incurring an accuracy cost. However, Table 4 reports the 20 released checkpoints as a single pooled row, reports decoding gains in delivered rather than computed accuracy, and leaves the decode-guard LiveCodeBench cells blank. Appendix A11 supplies both accuracy measures for selected MATH-500 configurations, but does not provide the corresponding model-level released-checkpoint breakdown. A pooled improvement cannot establish improvement at each checkpoint, and delivered-accuracy gains alone cannot establish that computed accuracy is preserved. The claims should be restricted to explicitly documented comparisons.

Questions for authors
  1. Please rescore the CE and repair D generations from Table 50 using the corrected GPQA judge and report the computed and delivered accuracy under EXACT and NO-SYSTEM, with paired confidence intervals. Do the reported −3.0/−6.1-point losses remain? Which existing tables should be treated as superseded?

  2. Which 20 checkpoints constitute Table 4's pooled result? Please provide their individual before/after computed and delivered accuracies, stopping thresholds, and identify the eight checkpoints with significant delivery improvements mentioned in Section 6.

  3. Which exact student–benchmark configurations support "every student and benchmark" and "no accuracy cost"? Please distinguish unevaluated from unreported cells and provide computed-accuracy changes alongside the AIME/GPQA delivered-accuracy changes.

4 comments

Thank you for recognizing our closing-token controls and practical relevance. Computed accuracy ignores closure/stop; natural delivery also requires a post-thought answer and an unaided stop (* = significant paired difference; old→new names: global response).

Please rescore the CE and repair D generations from Table 50 using the corrected GPQA judge … with paired confidence intervals. Do the reported −3.0/−6.1-point losses remain? Which existing tables should be treated as superseded?

No. Under the corrected judge, the −3.0/−6.1-point accuracy losses become −1.1 and −1.4 pp computed, neither significant; no-system natural delivery rises +24.9* pp. One rule-based judge (boxed letter, boxed option text, or explicit choice; Appendix A13.6) rescored the stored CE and D outputs (198 problems × four decoding seeds per cell; D = mixed-format replay):

Format / metricCE → D (%)Δ (pp)95% interval
Training / computed48.7 → 47.6−1.1−4.5 to +2.4
Training / natural delivery42.2 → 38.8−3.4−6.7 to 0.0
No-system / computed46.5 → 45.1−1.4−5.4 to +2.7
No-system / natural delivery12.5 → 37.4+24.9*+19.7 to +30.1

Training-format delivery (−3.4 pp, interval reaching 0.0) is not shown to be preserved, so we drop "no accuracy cost". A different repair, suffix-only training with replay (the submitted "best give-up training dose"), has a significant training-format GPQA cost: −4.2* computed and −4.0* delivery pp (Table 5). Superseded: submitted Table 50's GPQA accuracies by Table 55 and Section 4.1; submitted Tables 26, 47 and 74 by regenerated Tables 29, 52 and 79.

Which 20 checkpoints constitute Table 4's pooled result? Please provide their individual before/after computed and delivered accuracies, stopping thresholds, and identify the eight checkpoints with significant delivery improvements …

These are the 20 checkpoints pooled in submitted Table 4: guarded delivery rises in 15 (8* in bold), ties in two and declines non-significantly in three (−0.2 to −0.4 pp). All use MATH-500 (budget 16,384, threshold 12,288, i.e. 75%; 500 shared problems, one seed except two for OpenThinker-7B and DeepSeek-R1-Distill-Qwen-7B); §2 states how the panel was selected. Entries are untouched→guarded (%); delivery is natural→guarded (Table 35):

CheckpointComputedDelivered
DeepSeek-R1-0528-Qwen3-8B87.2→91.886.8→91.8
Qwen3-4B-Thinking-250790.4→93.689.8→93.6
TinyR1-32B-Preview93.6→94.089.0→94.0
Nemotron-3-Nano-30B-A3B87.8→89.485.8→89.2
Qwen3-8B93.8→94.493.0→94.4
Qwen3-30B-A3B-Thinking93.2→95.493.0→95.4
s1.1-32B90.8→90.890.8→90.8
Qwen3-14B94.4→93.694.0→93.6
Light-R1-14B-DS94.0→94.293.8→94.0
Light-R1-32B92.0→91.691.4→91.2
Qwen3-32B95.4→95.894.8→95.8
QwQ-32B94.6→95.094.4→95.0
Light-R1-7B-DS91.8→91.691.8→91.6
DeepSeek-R1-Distill-Qwen-32B92.4→92.492.4→92.4
Light-R1-32B-DS93.8→94.293.6→94.2
OpenThinker-7B82.5→83.982.4→83.9
DeepSeek-R1-Distill-Qwen-1.5B80.8→81.480.8→81.4
DeepSeek-R1-Distill-Qwen-7B91.7→91.991.6→91.9
DeepSeek-R1-Distill-Qwen-14B91.8→91.691.4→91.6
OpenReasoning-Nemotron-32B91.2→94.091.0→94.0

Four small computed declines (−0.2 to −0.8 pp) are also non-significant; we claim no per-checkpoint preservation. Tokens fall 3.2%–10.3% in all 20.

Which exact student–benchmark configurations support "every student and benchmark" and "no accuracy cost"? Please distinguish unevaluated from unreported cells and provide computed-accuracy changes alongside the AIME/GPQA delivered-accuracy changes.

We withdraw both phrases. The supported scope is the 18 controlled settings (3 students × 3 benchmarks × 2 formats), where guard delivery improves in 18/18 (16*). AIME/GPQA changes, guard − untouched (training / no-system, pp; all 18 in Table 36):

StudentBenchmarkΔ computedΔ delivery
Main 8BAIME+1.1 / +1.1+2.2* / +9.7*
Science 8BAIME0.0 / +1.7+0.4 / +11.2*
32B studentAIME+1.7 / +16.7*+5.0 / +26.7*
Main 8BGPQA+4.2* / +4.3*+6.4* / +23.1*
Science 8BGPQA+3.0* / +4.3*+4.9* / +25.9*
32B studentGPQA+5.3* / +6.2*+12.1* / +16.5*
  • Previously unreported: main/science AIME and GPQA computed shifts above.
  • Previously unevaluated: four 32B AIME/GPQA cells, now run.
  • Inapplicable (na): LiveCodeBench code (boxed commitment does not apply).
  • MATH-500 only: the 20 public checkpoints and s1-style forcing.

Caveat: computed gains can reflect boxing an existing answer or, on GPQA, a forced four-option guess. No delivery decline is significant (Tables 35–37).

Thank you for the revisions. The corrected scoring and detailed results address my original reporting concerns. However, I remain unconvinced of the contribution's significance: the key prompt-format failure is not observed in eligible public models, the cause of persistence in incorrect attempts remains unresolved, and the serving remedy builds on existing methods. I therefore maintain my score of 2.

AuthorsOpenReview ↗

On the significance of the diagnostic findings

Thank you for confirming that the corrected scoring and detailed results address your reporting concerns. We would like to clarify the practical significance of the controlled findings.

A training defect can escape answer-only validation (§3). In otherwise matched training runs, masking the chat turn-end target reduces its emission from 81.7% to 0%, while extracted-answer accuracy remains similar under permissive stopping rules (79.2% with supervision versus 79.6% without). The underlying supervision omission also occurs in audited pipelines: TRL 0.24 truncation of over-length examples and an examined Megatron-Bridge pipeline. This evidence motivates a concrete check that terminal targets survive data processing.

Prompt-format controls reveal a second way that accuracy can hide completion failures (§4). Removing a task-free system turn changes extracted-answer accuracy by −0.6 percentage points but lowers natural delivery by 29.2 points. Restoring only two closing-token probabilities reduces unfinished responses from 31.6% to 16.6%, versus 30.0%/32.8% for unrelated-token controls. The absence of this failure in four eligible public releases bounds its demonstrated scope.

The contribution is the controlled diagnosis of failures that accuracy-based validation can miss, with actionable checks for training and evaluation. The unresolved training cause of persistence on incorrect attempts and the use of established serving methods remain explicit limitations. We believe these diagnostic findings and their practical implications merit consideration alongside those limits.

AuthorsOpenReview ↗

To Km53, Two of the three reasons concern claims the paper does not make

the key prompt-format failure is not observed in eligible public models

Only four public releases were eligible for that test, and a public release is a finished model. These failures are created in fine-tuning, and one is already in shipped pipelines: in an audited Gemma-4 / Megatron-Bridge pipeline, none of 287,735 assistant turns received loss on its turn-end token, and TRL 0.24 drops that target from over-length examples (§3). Without it, turn-end emission falls from 81.7% to 0.0% while computed accuracy differs by 0.4 points.

the cause of persistence in incorrect attempts remains unresolved, and the serving remedy builds on existing methods

The paper claims neither. It labels that cause open (§5) and says in its abstract that it uses established tools. The contribution is the diagnosis your review called "a substantive diagnostic contribution" with "convincing evidence": deleting an 11-token system prompt moves extracted accuracy by 0.6 points and natural delivery by 29.2, and restoring two closing-token probabilities cuts unfinished responses from 31.6% to 16.6% (§4).

Comments on the paper

4 comments
AuthorsOpenReview ↗

Global Response

Thank you for the feedback. The submission is now 30 pages (down from 82), with a 10-page main text. (* = significant; n.s. = not significant.)

Our contribution is a controlled diagnosis of why correct answers fail to become completed responses, with practical checks where completion breaks. Deleting an 11-token, task-free system turn changes computed accuracy (the extracted answer is correct) by only −0.6 pp (n.s.) yet drops natural delivery (the answer also follows a closed thought and an unaided stop) by 29.2* pp (Figure 1). Restoring only two closing-token probabilities on the same prefixes cuts the main student's unfinished responses from 31.6% to 16.6% (two unrelated tokens: 30.0%/32.8%), replicated in 7B/8B students; a turn-end loss ablation isolates a separate pipeline defect.

Turn-end masking occurs in real pipelines and evades answer-only tests. TRL 0.24 right-truncation drops terminal targets on over-length examples, and an audited Gemma-4/Megatron-Bridge pipeline supervised none of 287,735 assistant turn-end tokens (§3; specific pipeline audits, not prevalence estimates). In our controlled ablation (training-format MATH-500), a permissive stop set hides the collapse in computed accuracy:

Stop set (decoding seeds)With turn-end supervisionWithout
Turn-end or end-of-text (4)79.2%79.6%
Turn-end only (2)78.3%45.5%

Without the target, generation runs past end-of-text and overwrites earlier answers. Four practices follow: assert unmasked turn-end targets after formatting, truncation and packing; report natural delivery beside extracted accuracy; mix prompt formats in SFT; and, under a token budget, elicit an answer instead of truncating.

New experiments address the main concerns:

QuestionEvidence and finding
Does the fixed threshold transfer to unseen problems?With the MATH-500 threshold frozen, on 500 new MATH problems (one checkpoint, four decoding seeds) prespecified training-format delivery rises 81.30%→86.35% (+5.05* pp; 95% interval +3.80 to +6.30); bare truncation alone loses computed accuracy (Appendix A4.3).
What does commitment add beyond forced stopping?Commitment (eliciting an answer in ≤64 tokens when none is boxed) adds +4.0*/+4.2* pp delivery over bare truncation (main student, MATH-500; training/no-system). Computed accuracy improves in 17/18 controlled settings; delivery improves in 20/20 selected public checkpoints (model-average +3.1* pp). Full-budget s1-style forcing delivers 1.6* pp more than our guard for 340* extra tokens/query (main-student training-format MATH-500). Zero unfinished runs is by construction; this is a serving policy, not learned stopping (Tables 6, 23–24; §6).
Do the GPQA repair losses survive consistent scoring?Mixed-format replay's computed losses shrink from −3.0/−6.1 to −1.1/−1.4 pp (n.s.), with +24.9* pp no-system delivery; suffix-only training with replay shows a significant −4.2* pp training-format cost (Tables 5, 18).
Does adding stopping demonstrations fix persistence on wrong attempts?Not with the augmentation we tested. With a byte-identical 30k base and three training seeds, constructed give-up demonstrations miss the preregistered +10 pp target (+1.9 pp pooled, n.s.) and lower no-system delivery (Table 17).

What the revised paper claims: (i) missing turn-end supervision is a reproducible pipeline defect; (ii) prompt-conditional closing is a demonstrated pipeline hazard, localized in 7B/8B students, null for the 32B closing-token intervention, and not shown to be prevalent (0 of 4 eligible public releases meet our failure criterion); (iii) persistence on wrong attempts (stop rate 23.5% vs. 96.0% for the base) is an observed pattern, also in 17/21 testable public checkpoints, whose training cause is open. We removed "no accuracy cost", "every student and benchmark", and "one principle explains every failure". The contribution is the controlled diagnosis and measured repair trade-offs.

Readability (all three reviewers): the appendix follows four evidence groups: reproduction settings, stopping mechanisms, training interventions, and serving comparisons. We removed repetition and peripheral analyses, retaining the requested comparisons and negative results. The abstract is 303→225 words; metrics come first (Table 1); Figures 1 and 5 are simplified; Tables 4–5 cover training and Table 6 covers serving.

Renamed terms (main text): EXACT → training format; capped → unfinished; delivered → natural (unaided) or guarded (serving-policy) delivery; repair D → mixed-format replay; best give-up-training dose → suffix-only training with replay; decode guard (TERMINATOR) → guard (truncation plus commitment).

Blue marks added results, corrected scoring, and substantive changes to claims.

Area_Chair_6MdoOpenReview ↗

Reminder: Reviewer Engagement During the Discussion Period

Dear Reviewers,

As the discussion period is drawing to a close, could you please review the authors' rebuttal and provide your feedback at your earliest convenience?

While I would like to thank Reviwer c8BG for the response, feedback from the other two reviewers is still absent.

Active engagement in discussions with the authors is an essential part of the peer-review process and plays an important role in ensuring the success of the conference. Your participation would be greatly appreciated.

Thank you for your time and contributions.

Best regards,

AC

AuthorsOpenReview ↗

Final Hours of Author-Reviewer Discussion: Authors Available for Any Questions

Dear Reviewers and Area Chair,

With only a few hours left before the Author-Reviewer Discussion window closes (November 18, 23:59 AoE), we wanted to briefly note that we remain online and can respond promptly to any last-minute questions while author replies are still open. Our revised manuscript and point-by-point responses include substantial new experiments and readability rewrites directly addressing each review, and we would be very grateful to know if anything remains unclear. Thank you again for your time and thoughtful feedback.

AuthorsOpenReview ↗

For the AC: what the reviewers credit, and what their final reasons concern

Dear Area Chair,

In brief. All three reviewers credit the paper's central diagnosis in their own words, and Km53 confirms that the original reporting concerns are resolved. The reasons given for keeping the scores concern released models, claims the paper does not make, and presentation, which the revision cut from 82 pages to 30.

The contribution in three lines

  1. A reasoning model can compute the right answer and never deliver it. Deleting an 11-token system prompt moves extracted accuracy by 0.6 points and natural delivery by 29.2.
  2. Two causes are identified by single-variable interventions. Removing loss from the turn-end token takes its emission from 81.7% to 0.0% while computed accuracy differs by 0.4 points. Restoring two closing-token probabilities cuts unfinished responses from 31.6% to 16.6%; the same edit on unrelated tokens gives 30.0% and 32.8%.
  3. The first cause is in shipped pipelines. In an audited Gemma-4 / Megatron-Bridge pipeline, none of 287,735 assistant turns received loss on its turn-end token; TRL 0.24 drops it from over-length examples.

The reviewers on the contribution

  • Km53 (Informativeness 3): "a substantive diagnostic contribution"; "convincing evidence for a localized termination failure".
  • vzFc (Soundness 3): "an important evaluation blind spot"; "a clean demonstration of Cause (i)".
  • c8BG: "interesting and have useful lessons for practitioners".

What the revision resolved

  • Scoring and reporting (Km53): unified GPQA scoring, per-checkpoint results and the complete evaluation matrix. Km53: these "address my original reporting concerns".
  • Validation of the serving threshold (c8BG): threshold fixed in advance, 500 new MATH problems, +5.05 delivery points.
  • Truncation baseline and training experiment (vzFc): +3.75 points over bare truncation; a three-seed test of stopping demonstrations, reported as a negative result.
  • Presentation (rated 1 by all three): 82 pages reduced to 30, with a 10-page main text.

What the final reasons concern

  • "the key prompt-format failure is not observed in eligible public models" (Km53). Only four public releases were eligible for that test. The paper's claim concerns fine-tuning pipelines, and the other failure was found in shipped ones.
  • "the cause of persistence in incorrect attempts remains unresolved, and the serving remedy builds on existing methods" (Km53). The paper claims neither: it labels the first open and says in its abstract that it uses established tools.
  • "concerns about the causal account and the scope of the contribution" (vzFc). The two causes rest on interventions, and the scope (7B/8B yes, 32B no, public releases no) is in the abstract. No unsupported claim was named.
  • "clearer writing" (c8BG). Addressed above; the final comment raises no technical objection.

Limits we state ourselves

The prompt-format failure is shown in 7B/8B students and is absent at 32B. The training cause of persistence on incorrect attempts is open. The serving result is on one checkpoint.

This note is visible to everyone. If we have misstated a reviewer's position, we would welcome a correction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.