- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
This paper studies non-termination in long-reasoning models after supervised fine-tuning. It distinguishes between computed accuracy, where a correct answer appears somewhere in the generation, and delivered accuracy, where the model also closes its reasoning and terminates naturally. The paper attributes non-termination to three causes: the end-of-turn token may be excluded from supervision; stopping may become dependent on the training prompt format; and the training data may contain stops only after converged answers, leaving the student unable to stop from unsuccessful or unsettled states. The authors investigate these failure modes through training ablations, prompt perturbations, token-probability interventions, and evaluations across several model families. They further propose TERMINATOR, a collection of data, format, and decoding guards intended to prevent these failures. The distinction between computing an answer and successfully delivering it is useful, and the paper contains several careful controlled experiments.
Critical strengths
- The paper identifies an important evaluation blind spot: an answer-extraction-based evaluator may mark a generation as correct even when the model fails to terminate and never cleanly delivers the answer to the user.
- Several controlled experiments are informative. In particular, the end-of-turn supervision ablation provides a clean demonstration of Cause (i), while the closing-token probability intervention offers useful evidence that prompt-format sensitivity can be localized near the stopping decision in the authors' models.
- The paper examines multiple model initializations, prompt formats, benchmarks, and released checkpoints. It also reports a substantial amount of implementation detail, negative evidence, and limitations in the appendix.
Critical weaknesses
- The manuscript has severe readability and presentation problems. Several important concepts are expressed using uncommon or paper-specific terms, without sufficiently clear explanation before they are used. The extensive use of italics, boldface, small caps, and other visual annotations makes the paper difficult to read. Some passages resemble compressed notes written by someone who already knows the experiments, rather than an explanation intended for readers without prior knowledge of the authors' experimental setup. The figures are an even larger problem. For example, the introductory figure is visually cluttered, contains too much text, mixes several kinds of information, and is difficult to understand even after reading its long text. The authors need to take the readability of the paper seriously, ensure that it is easy for humans to read, and manually revise the entire manuscript.
- The practical significance of Cause (ii) is unclear. The paper acknowledges that the required conditions are nearly mutually exclusive among the 31 released checkpoints examined, and none clearly exhibits this proposed failure. The experiments show that Cause (ii) can occur in the authors' controlled setting, but not that it is prevalent in practice. Its status as a central cause, and the substantial main-text space devoted to it, therefore seem disproportionate to the evidence.
- Cause (iii) is an interesting hypothesis, but the evidence is insufficient to establish causality. Position matching and probing do not fully separate missing failure-state supervision from difficulty, confidence, or distribution shift. A clean training control should hold the remaining data fixed, add natural examples of stopping after failure, and demonstrate stable recovery across training seeds. The current evidence supports an association, but not yet the full causal claim.
- A central result of TERMINATOR is guaranteed by construction. The decode guard appends closing syntax and forces an end-of-turn token at a fixed position, so reducing capping to zero is not an empirical finding. Delivered accuracy also partly improves because the wrapper rewrites an intermediate output into an evaluator-compatible answer. This may be a useful serving heuristic, but it should not be presented as recovery of the model's learned stopping ability.
- Table 4 overstates the completeness of the evidence. Each intervention is evaluated on only a subset of models and benchmarks, leaving many cells empty, while different rows report incompatible quantities such as absolute results, format gaps, relative changes, and pooled results. This is both difficult to interpret and insufficient to support the table's broad claim that every guard removes its corresponding failure.
Questions for authors
- Given that Cause (ii) was not observed as the proposed failure among the 31 released checkpoints, what evidence justifies presenting it as one of the three central causes rather than as a pipeline-specific case study?
- Can the authors provide a controlled, replicated training experiment that adds realistic stopping demonstrations from unsuccessful or unconverged states while holding the remainder of the training data fixed? What outcome would distinguish the proposed explanation for Cause (iii) from difficulty or confidence effects?
- How much of TERMINATOR's reported gain remains under a comparison with simple fixed-position truncation followed by extraction and delivery of the last available answer?
- Why are so many intervention-model-benchmark cells in Table 4 missing, and which are genuinely inapplicable rather than simply not evaluated? Do the headline claims still hold under a complete evaluation matrix and consistent reporting convention?
4 comments
Thank you for the detailed rebuttal, additional experiments, and substantial revisions. These efforts have alleviated some of my concerns. However, the additional evidence does not fully resolve my concerns about the causal account and the scope of the contribution. Moreover, the presentation still requires further revision. I will maintain my original rating.
Clarifying the causal evidence and contribution
Thank you for recognizing the improvements. We would like to clarify how the remaining causal uncertainty relates to the paper’s contribution.
1. The central findings have controlled intervention evidence (§§3–4).
- Terminal supervision: masking only the chat turn-end target reduces its emission from 81.7% to 0%, while a permissive evaluation stop set can conceal this failure.
- Prompt-dependent closing: on the same generated prefixes, restoring only two closing-token probabilities reduces unfinished responses from 31.6% to 16.6%; matched interventions on unrelated tokens leave them at 30.0%/32.8%.
These results identify specific sources of failed completion in the tested settings. The 32B null result and absence of the format failure in four eligible public releases delimit the scope of the second finding.
2. The training cause of persistence on incorrect attempts remains open (§5).
The constructed stopping demonstrations are proxies for unsuccessful reasoning states. Their negative result does not settle the realistic failed-state hypothesis or distinguish it from difficulty and confidence effects. Our conclusions in Sections 3–4 do not depend on that hypothesis.
3. The contribution is a diagnostic finding with practical consequences.
Deleting a task-free system turn changes extracted-answer accuracy by only −0.6 percentage points, yet lowers correct, naturally completed responses by 29.2 points. The controlled interventions explain specific failures that answer-only evaluation misses, motivating terminal-supervision checks and separate measurement of correctness and natural completion.
Could you identify the specific causal claim or passage in the revised main text that remains unsupported or unclear? If your concern is primarily the significance of these bounded findings, rather than their evidential support, that distinction would help us focus the remaining discussion.
The causal account rests on two single-variable interventions
the additional evidence does not fully resolve my concerns about the causal account and the scope of the contribution
The causal account rests on two interventions, each changing one thing (§§3–4).
- Removing loss from one token, the chat turn-end, takes its emission from 81.7% to 0.0%.
- Restoring two closing-token probabilities on the same generated prefix cuts unfinished responses from 31.6% to 16.6%; the same edit on two unrelated tokens gives 30.0% and 32.8%.
The scope is stated in the abstract. The second result replicates in 7B/8B students, is absent at 32B, and no eligible public release fails. The one account without an intervention, persistence on incorrect attempts, is labelled open (§5).
Your review called the first "a clean demonstration". Which causal claim in the revised text remains unsupported?
Thank you for recognizing the evaluation blind spot and controlled interventions (* = significant; n.s. = not significant; old→new names: global response).
We rewrote the main text, redrew Figure 1 and cut Figure 5 to two panels; pages 1–2 show the change directly.
Agreed: Section 4 now presents Cause (ii) as a demonstrated pipeline hazard in controlled students, not a prevalent cause. We keep the section because it carries the localization evidence: deleting an 11-token system turn cuts natural delivery 29.2* pp (computed −0.6 pp, n.s.), and restoring two closing-token probabilities on the same prefixes repairs 7B/8B students (null at 32B). Prevalence is not shown: only 4 of 31 public checkpoints meet its conditions (raw-base fine-tune with a removable system prompt), and none meets our failure criterion.
We ran a controlled, replicated version; the tested augmentation did not restore stopping, so Cause (iii) remains a hypothesis. With all 30,000 base traces byte-identical, three arms (base; +1,500 pre-answer teacher prefixes closed with a give-up stop; +1,500 full traces on the same problems) span three training seeds, three benchmarks and two formats (Table 22). The give-up rows are a proxy: an answer-less prefix need not be a failed attempt, and over 99.5% of their supervised tokens are prefix.
What would distinguish it: recovery beyond both control arms without delivery loss, plus real failure states and length- and difficulty-matched controls to exclude difficulty or confidence (open).
Agreed: zero unfinished runs is by construction, so we report the guard as a serving policy, not learned stopping. Against bare truncation plus extraction, commitment adds +4.0*/+4.2* pp delivery. Main-student MATH-500, threshold at 75% of budget:
The first row is the rewriting you describe: delivering an answer already boxed in the thought. Without the system turn it supplies most of the gain (+17.5* of +21.7* pp). Commitment (eliciting an answer in ≤64 tokens when none is boxed) beats truncation on computed accuracy in 17/18 controlled settings (16*; Table 86), though on GPQA a forced four-option guess can contribute. On 500 new MATH problems (training format, one checkpoint, four decoding seeds; threshold frozen), the guard raises delivery by 5.05* pp, commitment adds 3.75* pp over truncation, and truncation alone loses 1.55* computed pp (Appendix A20).
Blank cells mixed three cases, now separated; the guard's 18 controlled settings are complete, and the "every guard removes its failure" title is gone.
na).