acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing Harmful Continuation in Answer-Correct Mathematical Long-CoT Training Traces

Abstract

Long chain-of-thought (CoT) traces are widely used as supervision for reasoning-oriented LLM SFT, yet answer-correct traces can still lead to markedly different fine-tuning outcomes. We study post-conclusion continuation in answer-correct mathematical long-CoT data: a continuation where the answer appears sufficiently supported, but the trace continues with additional reasoning that remains in the supervised target. To test its training effect, we use a delete-only editor to construct answer-preserving suffix removal and compare CoT-based SFT on the original and processed traces. We observe improved SFT outcomes on mathematical problems after removing the editor-identified post-conclusion continuation, suggesting that this continuation is harmful to training in our setting. We therefore refer to this empirically supported phenomenon as harmful continuation. Beyond this intervention, we further characterize the removed post-conclusion continuation through uncertainty and hidden-state progress. We observe persistent local uncertainty together with weakened terminal-directional progress, forming an uncertainty–geometry mismatch. Finally, we introduce Harmful Continuation Cut (HCC), a lightweight proxy built on a frozen Qwen2.5-0.5B-Instruct backbone and trained to predict editor-identified boundaries with uncertainty–geometry auxiliary objectives. Our code is available at anonymous.4open.science/r/HCC-76B3.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model, Supervised Fine-tuning, Chain-of-Thought

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4CPKOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies answer-correct mathematical long chains of thought in which the answer is already supported and the trace still continues. A delete-only editor removes that continuation while keeping the prefix and the final answer. The authors compare supervised fine-tuning on the original traces with fine-tuning on the truncated ones, and they call the continuation harmful when that deletion is followed by a better training outcome. They then describe the removed suffix by local uncertainty and by hidden-state progress toward the terminal state of the analyzed trace, and they name the resulting pattern an uncertainty–geometry mismatch. Harmful Continuation Cut is a small model on a frozen backbone, trained to predict the editor’s boundary and deletion labels, with a sequential latent and auxiliary losses on the uncertainty and geometry targets. The comparisons include the original traces, the editor, a heuristic truncation, and HCC. For LLaMA3.2-3B-Instruct trained on {T}R\{T\}_R, the paper also reports Random Span, Random-Fix, and Random-Bin, which match retained length or the global cut-position distribution. The 4,780 traces come from Qwen3-235B-A22B-Instruct-2507 and DeepSeek-R1-V3.2. The editor is Qwen3.5-27B, the cut model is Qwen2.5-0.5B-Instruct, and the students are LLaMA3.2-3B-Instruct and Qwen2.5-Math-7B-Instruct. Pass@1 is reported on MATH500, AMC23, and GSM8K.

Critical strengths
  1. The question is about supervision, not about wrong answers. Traces that already reach the correct answer are usually treated as usable targets. This paper asks whether reasoning that continues after the answer is supported should still be imitated, and the introduction states that not every long tail is harmful and that length is not the quantity being isolated. That question is what the deletion, the diagnostic, and the student training are organized around.

  2. The study carries one set of traces through the deletion, the uncertainty and geometry description, and student fine-tuning, on two generators and two students. The paper states that the fine-tuning effect is weaker and less consistent on Qwen2.5-Math-7B. The writeup therefore includes the case in which the deletion helps less, rather than only the case in which it helps most.

Critical weaknesses
  1. The operational object is the editor’s suffix. The editor marks post-conclusion sentences that can be removed while the answer stays recoverable, and the paper states in Section 3.1 that those sentences are not ground-truth harmful continuation. The training comparison removes those marked sentences and calls the continuation harmful when supervised fine-tuning improves, so that improvement is the effect of this deletion. Section 3.4 then computes uncertainty and geometry on the two sides of that mark and says the mismatch does not define harmfulness. A sentence the editor does not mark is not on the removed side, whatever its entropy or curvature. Appendix D.2 re-annotates the same traces with DeepSeek-V4-Flash. Agreement between the two cuts is agreement between two editors, and the appendix says that neither is a ground-truth oracle. The comparison does not replace the editor partition with a continuation that was located without an editor.

  2. HCC stays on that same partition. The cut head is supervised by the editor boundary c∗c^*, the deletion head by the editor’s sentence labels, and the uncertainty and geometry targets are taken from the two sides of this cut. Training the predictor on one generator and scoring it on the other transfers a predictor of c∗c^*. Supervised fine-tuning on the predicted cut measures deletion at that predicted boundary.

  3. The controls reported for LLaMA3.2-3B-Instruct on {T}R\{T\}_R still put this boundary, or a predictor of it, on one side of the comparison. Random Span keeps the final answer and a contiguous span whose length matches HCC’s retained length on that trace. Random-Fix matches the average retained length of the HCC-processed set. Random-Bin keeps the global distribution of HCC’s cut positions and assigns those positions to different traces. A gap that remains after these matches is not a gap explained by length or by that global distribution. It is still a gap between a training set built from c∗c^*, or from a predictor of c∗c^*, and another span of the same traces. That gap does not identify a removed suffix that neither the editor nor that predictor produced.

Questions for authors
  1. Which observation, using neither the editor’s deletion labels nor the supervised loss after deletion, defines “after the conclusion”? Does HCC fit that observation, or does it fit c∗c^*?

  2. Appendix D.2 compares Qwen3.5-27B with DeepSeek-V4-Flash and states that neither editor is a ground-truth oracle. Which quantity in that comparison is not an editor’s cut?

  3. Section 3.4 says the uncertainty–geometry mismatch does not define harmfulness, and the mismatch is computed on the editor’s two sides. Which sample can reject that characterization without lying on one of those two sides?

  4. The cut head is supervised by c∗c^*. Which term of the HCC objective is not a function of the editor’s labels, or of features computed on the two sides of that cut?

  5. On LLaMA3.2-3B-Instruct trained on {T}R\{T\}_R, Random Span matches HCC’s per-trace retained length, Random-Fix matches the average retained length of the HCC-processed set, and Random-Bin keeps HCC’s cut-position distribution. Which comparison identifies a continuation that neither the editor nor a predictor of the editor produced?

  6. The task results are pass@1 on MATH500, AMC23, and GSM8K. The six MMLU subjects check only that existing capability is not degraded. The students are LLaMA3.2-3B-Instruct and Qwen2.5-Math-7B-Instruct. Can the same comparison be run on a domain other than mathematics, and on a student family other than these two?

2 comments

1. What independent observation defines “after the conclusion”? What does HCC fit?

Thank you for asking us to clarify what we mean by “after the conclusion.” We first clarify the question that motivated this work: For the same problem and the same verified correct answer, why does training a student to imitate the full reasoning trace lead to a different SFT outcome from training it to imitate a truncated trace? Answering this question does not require identifying a unique shortest sufficient derivation for every problem. The editor’s cut is an operational choice. With over 4,000 training examples, it would be difficult to try every cut positions for every example and train and evaluate a separate model for each candidate cut to identify the best one. The editor’s cut allows us to study the supervision value of different reasoning content while preserving the problem and its final answer.

Our evidence has two parts. First, in the editor-marked suffixes, we observe a statistical tendency for local uncertainty to persist while geometric progress toward the trace’s terminal state weakens. Second, we incorporate these signals into HCC, use it to select training prefixes, and test through downstream SFT and ablations whether this selection benefits the student. Our claim is that this continuation pattern, which reduces the value of the supervision, is observable in the mathematical reasoning traces we study, and that using it to select training prefixes improves students’ problem-solving accuracy in the settings we test. HCC’s supervised target is c∗c^*. We incorporate changes in geometry and uncertainty into the loss to assist boundary prediction, and our additional ablations validate our observation. We did NOT independently annotate a unique earliest sufficient position for each problem.

2. Which quantity in the cross-editor comparison is independent of the two editors’ cuts?

The cross-editor agreement metrics in Appendix D.2 are all computed from the two editors’ cuts. We did not independently determine a unique optimal cut for each trace. This experiment tests whether a different editor can locate the candidate redundant suffix initially identified by Qwen3.5-27B. DeepSeek-V4-Flash independently applied the same truncation instructions to the same original traces. The two editors generally selected nearby cuts. This shows that the localization of candidate redundant suffixes is reproducible across editors, providing evidence that the phenomenon extends beyond the judgment of a single editor.

3. What kind of sample can challenge the characterization of an uncertainty–geometry mismatch?

Section 3.4 examines statistical trends in the editor-defined segments. The editor determines which sentences are retained or removed, while sentence entropy and forward progress are computed on the original traces. In Figure 2(a), although mean sentence entropy increases overall, some individual traces shown in gray decrease between adjacent stages. In Figure 3(b), the two groups of points overlap substantially, and some removed sentences exhibit greater forward progress than some retained sentences. Individual samples can therefore deviate from the overall trend even when the partition is fixed.

Our claim is that persistent local uncertainty and weak geometric progress constitute a statistical tendency in editor-marked suffixes. Truncation guided by this statistical tendency is effective.

4. Which loss terms depend on the editor?

HCC’s cut and del losses use the editor’s cut c∗c^* and its sentence-level deletion labels, respectively. All three remaining auxiliary losses can be computed along the original trace without first partitioning it at the cut.

5. Which comparison uses an independently identified suffix?

We added a first-boxed truncation control. The rule reads only the original reasoning text, places the cut at the first occurrence of a correct boxed answer in the reasoning section, removes the subsequent reasoning, and preserves the original final answer. Selecting the cut requires no access to the editor’s labels, HCC’s predictions, or HCC’s retained lengths.

DatasetVanillaFirst-boxed truncationHCC
MATH50038.432.047.6
AMC2315.020.022.5
GSM8K69.962.977.8

Using the first explicit answer alone to determine the cut does not reproduce HCC’s average gain in this setting.

6. Do the results generalize to a non-mathematical training domain and a third student model family?

We added Phi-4-mini-instruct trained one group on R1 traces and the HCC-truncated R1 traces respectively. The HCC group achieved higher accuracy than the original-trace group on all 3 dataset.

DatasetVanillaHCC
MATH50055.257.8
AMC222.535.0
GSM8K75.381.7

Furthermore, the performances on non-mathematical training domain is illustrated in the Figure 6 in the main paper and Table 6 in the Appendix.

Thank you for the response. The First-boxed control and the Phi-4-mini results are useful additions. However, the central concern remains. Your reply confirms that c∗c* is an operational editor cut, that the cross-editor metrics are computed only from editors’ cuts, and that HCC is supervised by c∗c*. The samples that “deviate from the trend” still lie on the editor partition, so they do not provide an independent test of the uncertainty–geometry characterization. First-boxed shows that one editor-free rule is weaker than HCC; it does not show that the paper has identified a post-conclusion continuation independent of the editor. If the claim is limited to the training value of editor-removable suffixes in mathematical long-CoT SFT and to a lightweight proxy for that deletion, the evidence is more consistent. As currently framed, the stronger construct of “harmful continuation” as an independent phenomenon is still not supported. My assessment is unchanged.

Reviewer_8KptOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates harmful continuation in answer-correct mathematical long chain-of-thought (CoT) traces, where reasoning continues after sufficient support for the final answer has already been established. The authors show that removing such post-conclusion continuation using a delete-only LLM editor can improve downstream supervised fine-tuning (SFT) performance. The authors observe that editor-removed continuation tends to exhibit persistent local uncertainty and weakened terminal-directional hidden-state progress, which they describe as an uncertainty–geometry mismatch. Based on these observations, the authors propose Harmful Continuation Cut (HCC), a lightweight boundary prediction model.

Critical strengths
  1. The paper addresses an important limitation of reasoning-oriented SFT that answer correctness alone does not guarantee that an entire reasoning trajectory provides beneficial training supervision. The distinction between useful reasoning and potentially harmful post-conclusion continuation is meaningful, particularly as long-CoT data becomes increasingly common in reasoning-model training.
  2. The paper provides a detailed characterization of editor-identified post-conclusion continuation through both uncertainty and hidden-state geometry. The observation that answer-level uncertainty can increase after sufficient answer support has been established is particularly interesting.
  3. HCC provides a practical approximation to expensive LLM-based trace editing. The experimental results show that HCC-processed traces can substantially improve downstream SFT performance over the original trajectories while approaching the results of the much larger editor.
Critical weaknesses
  1. The relationship between geometric progress and actual reasoning utility remains insufficiently validated. A central insight of the paper is that harmful continuation exhibits weakened hidden-state progress. However, the proposed geometric measures are defined primarily through hidden-state displacement and alignment with the terminal representation of the observed trajectory. It is unclear to what extent these quantities reflect meaningful reasoning progress, rather than general representation changes or trajectory-specific properties.
  2. The proposed method is motivated by the interaction between uncertainty and geometric progress. However, the ablation experiments primarily compare combinations of objectives rather than isolating the contributions of the two diagnostic signals.
Questions for authors

Please refer to the weakness part.

2 comments

1. To what extent does geometric progress reflect reasoning utility?

Forward progress measures how far the hidden state moves toward the terminal state of the same trace when a reasoning sentence is added. A crucial verification step can also produce only a small displacement. Our interpretation concerns the statistical trend across traces and the role of this signal in selecting training data.

Per-token forward progress (Tables 1 and 5)Result
Mean for editor-retained / editor-removed segments1.28 / 0.84
Fraction of traces with lower progress in the removed segment79%
95% confidence interval for the paired difference[−0.47, −0.40]
Removed segment minus the subsequent final-answer segment−0.198

The sentence-level evaluation in Appendix C shows that, as sentences from the retained segment are added, the evaluator’s negative log-likelihood for the correct answer generally decreases. This improvement does not persist as sentences from the editor-marked suffix are added.

We also tested the training value of this signal through downstream student SFT. The geo-only variant retains the editor’s boundary supervision and uses geometry as its sole auxiliary objective. When LLaMA is fine-tuned on R1 data truncated by this variant, its average exceeds those of the heuristic and all truncation controls.

Truncation methodAverage
Vanilla41.1
Heuristic41.7
Random Span29.0
Random-Fix9.0
Random-Bin14.3
Cross-problem length-matched truncation36.9
First-boxed truncation39.9
Geo only44.2

Cross-problem length-matched truncation: We used the LLaMA tokenizer and the actual SFT preprocessing pipeline to count the tokens contributing to the loss in each HCC response, excluding the problem and the shared prompt. Within the same set of R1 traces, we shuffled the matching order with seed 42 and then performed maximum bipartite matching. All samples were matched one-to-one. Each pair had to have different problems and different original responses, and the selected original response had to be long enough to meet the target budget. Only the target length was transferred across problems. Each control sample retained its own problem and response. We truncated the full response from the end according to the matched budget. For each sample, we verified that the problem was intact, the response body was a prefix of the original response, the final \boxed{} answer was complete, and the supervised token counts matched exactly within the pair. We trained directly on the verified token sequences. The total token budget is 13,043,937.

First-boxed truncation: The traces is truncated on the first correct answer in the \boxed{}

2. How much does each of the uncertainty and geometry objectives contribute?

Using the same R1 traces and LLaMA student, we retained the uncertainty objective (ent), the geometry objective (geo), or both. Full HCC outperforms both single-objective variants on all three benchmarks. This shows that the geometry signal helps HCC select prefixes that are better suited to student learning.

Auxiliary objectivesMATH500AMC23GSM8K
Ent only41.017.574.0
Geo only41.617.573.4
Full HCC (Table 2)47.622.577.8

Thanks the author for the detailed rebuttal. I would keep my positive score.

Reviewer_jRMiOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies post-conclusion continuation in answer-correct mathematical long-CoT traces. A Qwen3.5-27B delete-only editor identifies removable suffixes while retaining the original final answer. The paper associates these suffixes with persistent uncertainty and weaker terminal-directional hidden-state progress, and reports improved downstream SFT after their removal. It then introduces Harmful Continuation Cut (HCC), a lightweight boundary predictor using a frozen Qwen2.5-0.5B backbone, direct cut/deletion supervision, sequential latent regularization, and uncertainty/geometry auxiliary objectives. HCC approximates the editor’s boundaries at substantially lower computational cost.

Critical strengths
  1. The paper addresses an important limitation of correctness-only CoT filtering and tests it through an actual data intervention rather than relying exclusively on correlational diagnostics. The cross-source setup, multiple student models, larger-student experiment, subsequent-RL check, and released anonymous implementation are valuable.

  2. The authors provide several useful robustness checks, including cross-editor boundary agreement, positional diagnostics, randomized controls, loss ablations, and downstream comparisons with the large editor. HCC recovers much of the editor’s benefit while being considerably cheaper.

Critical weaknesses
  1. The deletion intervention does not separate continuation content from generic shortening and training-budget effects. HCC/editor traces are dramatically shorter, while the heuristic baseline barely changes length. Random Span, Random-Fix, and Random-Bin can remove answer-supporting prefix reasoning or create incoherent targets; they are therefore not a fair prefix-preserving, length-matched suffix control. With the same epoch configuration, processed and original datasets also expose the student to different numbers of supervised tokens. The evidence consequently does not establish that the identified content, rather than well-chosen shortening, is the causal source of the gain.

  2. Evaluation contamination and training variability are unresolved. The paper samples source problems from OpenR1-Math-220k but reports no decontamination against MATH500, AMC23, GSM8K, or the AIME sets. OpenR1’s public decontamination tooling explicitly screens MATH500/AIME, and public decontaminated releases report removing hundreds of overlapping entries. In addition, all downstream SFT results appear to use one training run; several AMC/AIME changes correspond to very few questions, and results on Qwen2.5-Math-7B are mixed.

Questions for authors
  1. Which exact OpenR1 source split/version was used, and were all 4,780 source problems decontaminated against every reported evaluation set before trace generation? Please provide exact and near-duplicate counts and rerun results after removing any matches. Relevant public evidence includes the OpenR1 decontamination script.

  2. Can the authors compare HCC against a prefix-preserving suffix-truncation control matched per trace for retained length and total supervised-token/update budget, with multiple SFT seeds and confidence intervals? This would determine whether the gain is specific to the proposed boundary rather than shortening or optimization noise.

1 comment

1. Where do the training problems come from, and do they overlap with the evaluation sets?

We sampled records from OpenR1-Math-220k in their original order, deduplicated the problem statements, and then had Qwen3-235B-A22B-Instruct-2507 and DeepSeek-R1-V3.2 solve each problem independently. A problem entered the training set only if both models’ final answers passed verification. In this manner, we obtained the math questions and their corresponding traces. These questions are fixed during the research.

We checked the final training problems against MATH500, AMC23, GSM8K, AIME24, and AIME25 separately. We found no overlap with the training set in any of the five benchmarks. We will release the full dataset.

2. Does HCC still yield gains when the prefix is preserved and the training budget is matched?

We used the LLaMA tokenizer and the actual SFT preprocessing pipeline to count the tokens contributing to the loss in each HCC response, excluding the problem and the shared prompt. Within the same set of R1 traces, we shuffled the matching order with seed 42 and then performed maximum bipartite matching. All samples were matched one-to-one. Each pair had to have different problems and different original responses, and the selected original response had to be long enough to meet the target budget.

Only the target length was transferred across problems. Each control sample retained its own problem and response. We truncated the full response from the end according to the matched budget. For each sample, we verified that the problem was intact, the response body was a prefix of the original response, the final \boxed{} answer was complete, and the supervised token counts matched exactly within the pair. We trained directly on the verified token sequences. The total token budget is 13,043,937.

DatasetCross-problem controlHCC (Table 2)
MATH50035.447.6
AMC2310.022.5
GSM8K65.477.8

With the same set of problems and the same distribution of supervised lengths, HCC, which selects the cut based on each problem’s own trace, outperforms the cross-problem length-matched control on every benchmark.

3. How much variability comes from decoding and training with different seeds?

Decoding. We used seed 42 and greedy decoding (temperature 0) to generate the responses. With the model and inputs fixed, changing only the decoding seed produced identical answers and reasoning traces for every example.

Training. We trained HCC on Qwen traces using seeds 42, 43, and 44. For each seed, we used the resulting HCC model to truncate the R1 traces and then fine-tuned LLaMA3.2-3B-Instruct on those traces. We evaluated all students using greedy decoding at temperature 0.

BenchmarkMetricResult
MATH500Mean accuracy ± standard deviation48.20 ± 0.71
AMC23Mean accuracy ± standard deviation20.83 ± 1.18
GSM8KMean accuracy ± standard deviation76.73 ± 0.77

4. Interpretation onthe mixed results on the math-specialized model?

AMC and AIME contain relatively few problems, so a few answers can shift accuracy. For the 30B student in the paper, HCC solves a few more problems on the AIME sets and a few fewer on AMC23. Reporting the number of correct answers makes the magnitude of these changes clearer.

Qwen3-30B-A3B-Instruct (Table 10)ProblemsPercentage points per problemVanilla correctHCC correct
AMC23402.5037/4035/40
AIME24303.3320/3024/30
AIME25303.3317/3020/30

The gains from trace truncation also appear on the larger MATH500 and GSM8K benchmarks. Within the Qwen family, the effect on the math-specialized model varies with the trace source, while the general-purpose instruction-tuned model improves on all three benchmarks. This supports our interpretation that prior math-specific fine-tuning reduces the model’s sensitivity to further trace truncation.

Studentevaluation metricVanillaHCC
LLaMA3.2-3BMATH50038.447.6
LLaMA3.2-3BGSM8K69.977.8
Qwen2.5-7BMATH50053.862.2
Qwen2.5-7BGSM8K79.485.4

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.