- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies answer-correct mathematical long chains of thought in which the answer is already supported and the trace still continues. A delete-only editor removes that continuation while keeping the prefix and the final answer. The authors compare supervised fine-tuning on the original traces with fine-tuning on the truncated ones, and they call the continuation harmful when that deletion is followed by a better training outcome. They then describe the removed suffix by local uncertainty and by hidden-state progress toward the terminal state of the analyzed trace, and they name the resulting pattern an uncertainty–geometry mismatch. Harmful Continuation Cut is a small model on a frozen backbone, trained to predict the editor’s boundary and deletion labels, with a sequential latent and auxiliary losses on the uncertainty and geometry targets. The comparisons include the original traces, the editor, a heuristic truncation, and HCC. For LLaMA3.2-3B-Instruct trained on , the paper also reports Random Span, Random-Fix, and Random-Bin, which match retained length or the global cut-position distribution. The 4,780 traces come from Qwen3-235B-A22B-Instruct-2507 and DeepSeek-R1-V3.2. The editor is Qwen3.5-27B, the cut model is Qwen2.5-0.5B-Instruct, and the students are LLaMA3.2-3B-Instruct and Qwen2.5-Math-7B-Instruct. Pass@1 is reported on MATH500, AMC23, and GSM8K.
Critical strengths
-
The question is about supervision, not about wrong answers. Traces that already reach the correct answer are usually treated as usable targets. This paper asks whether reasoning that continues after the answer is supported should still be imitated, and the introduction states that not every long tail is harmful and that length is not the quantity being isolated. That question is what the deletion, the diagnostic, and the student training are organized around.
-
The study carries one set of traces through the deletion, the uncertainty and geometry description, and student fine-tuning, on two generators and two students. The paper states that the fine-tuning effect is weaker and less consistent on Qwen2.5-Math-7B. The writeup therefore includes the case in which the deletion helps less, rather than only the case in which it helps most.
Critical weaknesses
-
The operational object is the editor’s suffix. The editor marks post-conclusion sentences that can be removed while the answer stays recoverable, and the paper states in Section 3.1 that those sentences are not ground-truth harmful continuation. The training comparison removes those marked sentences and calls the continuation harmful when supervised fine-tuning improves, so that improvement is the effect of this deletion. Section 3.4 then computes uncertainty and geometry on the two sides of that mark and says the mismatch does not define harmfulness. A sentence the editor does not mark is not on the removed side, whatever its entropy or curvature. Appendix D.2 re-annotates the same traces with DeepSeek-V4-Flash. Agreement between the two cuts is agreement between two editors, and the appendix says that neither is a ground-truth oracle. The comparison does not replace the editor partition with a continuation that was located without an editor.
-
HCC stays on that same partition. The cut head is supervised by the editor boundary , the deletion head by the editor’s sentence labels, and the uncertainty and geometry targets are taken from the two sides of this cut. Training the predictor on one generator and scoring it on the other transfers a predictor of . Supervised fine-tuning on the predicted cut measures deletion at that predicted boundary.
-
The controls reported for LLaMA3.2-3B-Instruct on still put this boundary, or a predictor of it, on one side of the comparison. Random Span keeps the final answer and a contiguous span whose length matches HCC’s retained length on that trace. Random-Fix matches the average retained length of the HCC-processed set. Random-Bin keeps the global distribution of HCC’s cut positions and assigns those positions to different traces. A gap that remains after these matches is not a gap explained by length or by that global distribution. It is still a gap between a training set built from , or from a predictor of , and another span of the same traces. That gap does not identify a removed suffix that neither the editor nor that predictor produced.
Questions for authors
-
Which observation, using neither the editor’s deletion labels nor the supervised loss after deletion, defines “after the conclusion”? Does HCC fit that observation, or does it fit ?
-
Appendix D.2 compares Qwen3.5-27B with DeepSeek-V4-Flash and states that neither editor is a ground-truth oracle. Which quantity in that comparison is not an editor’s cut?
-
Section 3.4 says the uncertainty–geometry mismatch does not define harmfulness, and the mismatch is computed on the editor’s two sides. Which sample can reject that characterization without lying on one of those two sides?
-
The cut head is supervised by . Which term of the HCC objective is not a function of the editor’s labels, or of features computed on the two sides of that cut?
-
On LLaMA3.2-3B-Instruct trained on , Random Span matches HCC’s per-trace retained length, Random-Fix matches the average retained length of the HCC-processed set, and Random-Bin keeps HCC’s cut-position distribution. Which comparison identifies a continuation that neither the editor nor a predictor of the editor produced?
-
The task results are pass@1 on MATH500, AMC23, and GSM8K. The six MMLU subjects check only that existing capability is not degraded. The students are LLaMA3.2-3B-Instruct and Qwen2.5-Math-7B-Instruct. Can the same comparison be run on a domain other than mathematics, and on a student family other than these two?
2 comments
Thank you for the response. The First-boxed control and the Phi-4-mini results are useful additions. However, the central concern remains. Your reply confirms that is an operational editor cut, that the cross-editor metrics are computed only from editors’ cuts, and that HCC is supervised by . The samples that “deviate from the trend” still lie on the editor partition, so they do not provide an independent test of the uncertainty–geometry characterization. First-boxed shows that one editor-free rule is weaker than HCC; it does not show that the paper has identified a post-conclusion continuation independent of the editor. If the claim is limited to the training value of editor-removable suffixes in mathematical long-CoT SFT and to a lightweight proxy for that deletion, the evidence is more consistent. As currently framed, the stronger construct of “harmful continuation” as an independent phenomenon is still not supported. My assessment is unchanged.
1. What independent observation defines “after the conclusion”? What does HCC fit?
Thank you for asking us to clarify what we mean by “after the conclusion.” We first clarify the question that motivated this work: For the same problem and the same verified correct answer, why does training a student to imitate the full reasoning trace lead to a different SFT outcome from training it to imitate a truncated trace? Answering this question does not require identifying a unique shortest sufficient derivation for every problem. The editor’s cut is an operational choice. With over 4,000 training examples, it would be difficult to try every cut positions for every example and train and evaluate a separate model for each candidate cut to identify the best one. The editor’s cut allows us to study the supervision value of different reasoning content while preserving the problem and its final answer.
Our evidence has two parts. First, in the editor-marked suffixes, we observe a statistical tendency for local uncertainty to persist while geometric progress toward the trace’s terminal state weakens. Second, we incorporate these signals into HCC, use it to select training prefixes, and test through downstream SFT and ablations whether this selection benefits the student. Our claim is that this continuation pattern, which reduces the value of the supervision, is observable in the mathematical reasoning traces we study, and that using it to select training prefixes improves students’ problem-solving accuracy in the settings we test. HCC’s supervised target is c∗. We incorporate changes in geometry and uncertainty into the loss to assist boundary prediction, and our additional ablations validate our observation. We did NOT independently annotate a unique earliest sufficient position for each problem.
2. Which quantity in the cross-editor comparison is independent of the two editors’ cuts?
The cross-editor agreement metrics in Appendix D.2 are all computed from the two editors’ cuts. We did not independently determine a unique optimal cut for each trace. This experiment tests whether a different editor can locate the candidate redundant suffix initially identified by Qwen3.5-27B. DeepSeek-V4-Flash independently applied the same truncation instructions to the same original traces. The two editors generally selected nearby cuts. This shows that the localization of candidate redundant suffixes is reproducible across editors, providing evidence that the phenomenon extends beyond the judgment of a single editor.
3. What kind of sample can challenge the characterization of an uncertainty–geometry mismatch?
Section 3.4 examines statistical trends in the editor-defined segments. The editor determines which sentences are retained or removed, while sentence entropy and forward progress are computed on the original traces. In Figure 2(a), although mean sentence entropy increases overall, some individual traces shown in gray decrease between adjacent stages. In Figure 3(b), the two groups of points overlap substantially, and some removed sentences exhibit greater forward progress than some retained sentences. Individual samples can therefore deviate from the overall trend even when the partition is fixed.
Our claim is that persistent local uncertainty and weak geometric progress constitute a statistical tendency in editor-marked suffixes. Truncation guided by this statistical tendency is effective.
4. Which loss terms depend on the editor?
HCC’s
cutanddellosses use the editor’s cut c∗ and its sentence-level deletion labels, respectively. All three remaining auxiliary losses can be computed along the original trace without first partitioning it at the cut.5. Which comparison uses an independently identified suffix?
We added a first-boxed truncation control. The rule reads only the original reasoning text, places the cut at the first occurrence of a correct boxed answer in the reasoning section, removes the subsequent reasoning, and preserves the original final answer. Selecting the cut requires no access to the editor’s labels, HCC’s predictions, or HCC’s retained lengths.
Using the first explicit answer alone to determine the cut does not reproduce HCC’s average gain in this setting.
6. Do the results generalize to a non-mathematical training domain and a third student model family?
We added Phi-4-mini-instruct trained one group on R1 traces and the HCC-truncated R1 traces respectively. The HCC group achieved higher accuracy than the original-trace group on all 3 dataset.
Furthermore, the performances on non-mathematical training domain is illustrated in the Figure 6 in the main paper and Table 6 in the Appendix.