- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper interprets thinking collapse in on-policy self-distillation (OPSD) as over-automatisation: students acquire efficient but inflexible reasoning procedures from privileged-context teachers. Experiments with two model families show that rollout-based contexts can shorten reasoning and reduce accuracy on difficult tasks, including outside the training domain. A synthetic shortest-path task separates context informativeness from length, while pointwise usable-information measurements relate context access and reasoning depth to answer likelihood. Further experiments examine adaptation to unseen problems and susceptibility to malicious context. The paper argues that these observations connect reasoning compression, reduced flexibility, and context reliance within a common cognitive framework.
Critical strengths
-
I find the difficulty-dependent transfer results compelling, particularly the contrast between preserved performance on easy ZebraLogic puzzles and substantial degradation on XL puzzles in Figure 1. This establishes a practically important limitation of OPSD that would be missed by reporting reasoning efficiency or average accuracy alone.
-
The synthetic intervention on context informativeness and padding length is a useful attempt to move beyond qualitative comparisons of privileged contexts. Together with the malicious-context evaluation, it provides concrete evidence that context design affects both reasoning behavior and downstream robustness.
Critical weaknesses
-
I find automatisation an interesting organizing hypothesis, but the main text does not sufficiently distinguish it from learning a premature stopping policy or adopting context-dependent shortcuts. Shorter traces, poorer difficult-task performance, and reduced gains in answer likelihood are compatible with these alternatives, so I am not yet convinced that the cognitive framework explains the observations beyond redescribing them.
-
The padded answer-only controls address length, but they do not match rollout contexts in solution content, reasoning style, redundancy, or alignment with the model’s output distribution. Consequently, the conclusion that self-generation causes “over-practice” seems stronger than the evidence warrants; I would find a comparison against matched solutions from another model substantially more persuasive.
-
The PVI analysis supports association more clearly than mechanism. Equation 4 changes both the supplied context and the sampled reasoning trace, while Figure 4 pools settings that already differ in informativeness and padding. Likewise, Figure 5 measures gold-answer likelihood along each model’s own traces, so its differences do not establish whether unproductive reasoning causes early termination or accompanies an already degraded policy.
-
The proposed connection between collapse and safety vulnerability is incomplete. Figure 6 identifies FS as the most vulnerable condition, although FS does not exhibit the pronounced collapse observed for RC/RU in the primary Qwen experiments. I consider the safety finding valuable independently, but this ordering weakens the impression that lost reasoning flexibility and malicious-context reliance are manifestations of one established mechanism.
Questions for authors
-
What intervention would distinguish over-automatisation from a learned stopping bias in the collapsed students? Testing whether an enforced additional reasoning budget restores accuracy would help determine whether the main impairment concerns termination or the reasoning procedure itself.
-
Does the collapse persist when self-generated and externally generated solution contexts are matched for length, correctness, and reasoning style? This comparison would clarify whether self-generation has a distinctive effect beyond the properties of the resulting text.
-
How does the proposed framework explain FS producing the largest safety degradation while RC/RU produce the strongest thinking collapse? Clarifying whether these outcomes reflect separable processes would make the scope of the central explanation more convincing.
5 comments
Response to Reviewer 91VM (2/3): rollouts from other models and length-matched controls
W2+Q2
(a) The padded answer-only controls address length, but they do not match rollout contexts in solution content, reasoning style, redundancy, or alignment with the model's output distribution. [...] I would find a comparison against matched solutions from another model substantially more persuasive.
We trained two Qwen3-8B students whose teacher has access to a correct rollout from another model, and showed that the closer a rollout is to the model's own output distribution, the stronger the collapse. The teacher's privileged context is a correct rollout from DeepSeek-R1 (RC-R1: its reasoning from the dataset followed by the worked solution) or from Gemma-4-E4B (RC-G4). Both also collapse, but less than with the model's own rollouts: on AIME-24, pass@1 drops by 27.7 points for RC, 18.5 for RC-R1 and 8.9 for RC-G4, and on the hardest ZebraLogic puzzles accuracy falls from 38% for Base to 5–7% for RC and RU, against 15–20% for RC-R1 and RC-G4 (updated Fig. 1). We measured the alignment the reviewer mentions as the likelihood of each rollout under base Qwen3-8B: the model finds its own rollouts the most likely, then DeepSeek-R1's, then Gemma-4-E4B's, the same order as their collapse (new appendix Fig. 17). Both students also cut the hardest problems the most, with an effort elasticity below 1 (W1 (a)). Collapse thus arises whenever the teacher is given a rollout, and grows with how close the rollout is to the model's own reasoning; we revised the abstract, introduction and conclusion accordingly.
(b) Does the collapse persist when self-generated and externally generated solution contexts are matched for length, correctness, and reasoning style?
We added two length-matched controls: a worked solution padded to the length of the model's own rollouts, and the model's own rollout cut to the length of Gemma-4-E4B's. The full-solution + random (FS+R) context pads the dataset's worked solution with random tokens, so it is as long as RC's (median 6.0k vs 5.4k tokens; new appendix Fig. 16), correct and complete. Its student shortens reasoning by 49% on the test set, close to RC's 60%, yet does not collapse on AIME-24 and keeps Base's elasticity (1.06). In the second control, the teacher is given the model's own correct rollout, with its reasoning truncated so that, on every problem, it is as long as Gemma-4-E4B's rollout; the final answer is kept whole. Correctness and length then match RC-G4, and only the author differs. Despite more than halving the context, this student behaves more like RC than like RC-G4, and cutting the rollout reduces collapse only in part: AIME-24 pass@1 drops by 19.4 points, against 27.7 for the full rollout and 8.9 for Gemma-4-E4B's rollout of the same length, and on the hardest ZebraLogic puzzles accuracy is 10%, against 20% for RC-G4. On AIME-24, AIME-25 and ZebraLogic, its accuracy and rollout length lie 53–71% of the way from RC-G4 to RC (new Appendix C.5, Table 5). At equal length and correctness, the model's own reasoning collapses the student about twice as much as another model's, so self-generation has an effect beyond length and correctness.
Response to Reviewer 91VM (3/3): Eq. 4, Fig. 5 and safety results
W3
(a) Equation 4 changes both the supplied context and the sampled reasoning trace, while Figure 4 pools settings that already differ in informativeness and padding.
We clarified that Eq. 4 measures the total effect of the context on the teacher, and added a test on math in which every context is a single point. In OPSD the teacher conditions on the context and reasons before every token it scores, so what the student is distilled toward is the information the teacher extracts by reasoning with the context; Eq. 4 measures exactly this, with the reasoning trace as the route through which the context acts. In the graph task the intervention is on the context alone (information density and padding, with the problem fixed), and Fig. 3 shows the effect of each manipulation separately; Fig. 4 then asks whether one quantity summarises both. On math, each of the nine contexts is one point (new Fig. 5a): the more usable information a context gives the teacher, the more its student shortens (Spearman ρ = 0.75). We therefore present PVI as a measure of how much the context lets the teacher, and hence the student, shorten its reasoning, not of whether the shortening turns into collapse: the padded full solution has the highest PVI of any context, yet does not collapse.
(b) Figure 5 measures gold-answer likelihood along each model's own traces, so its differences do not establish whether unproductive reasoning causes early termination or accompanies an already degraded policy.
We agree that Fig. 5 (now Fig. 5c) is correlational, and two new experiments now separate the two readings. If the reasoning were unproductive because the students end it too early, they would be readier than Base to stop while still reasoning, and would recover when forced to reason longer. Instead, they rank the end-of-thinking token lower than Base at every point of their reasoning, and when forced to reason as long as Base they repeat themselves, gaining no accuracy and little information about the answer (W1; new Appendix C.6–C.7, Figs. 20–21). Unproductive reasoning thus accompanies an already degraded procedure rather than causing early termination.
W4+Q3
Figure 6 identifies FS as the most vulnerable condition, although FS does not exhibit the pronounced collapse observed for RC/RU in the primary Qwen experiments. [...] How does the proposed framework explain FS producing the largest safety degradation while RC/RU produce the strongest thinking collapse?
With the retrained students, the rollout students are now the most vulnerable on Qwen3-8B on JailbreakBench and BeaverTails, and Section 2 explains why collapse and context reliance can still separate. On JailbreakBench with a harmful reference, RC, RU, RC-R1 and RC-G4 raise the attack success rate by 5.7–10.4 points over Base, above FS (+4.0; updated Fig. 6). The framework, however, does not require the two orderings to coincide. Section 2 distinguishes two things OPSD can automatise in the student's procedure: a curtailed reasoning budget, and a readiness to defer to whatever context is supplied. Every OPSD teacher answers from its context, so every student can inherit the second; only rollout contexts transfer the first strongly enough to collapse. A student can therefore defer to context without collapsing, as FS does on Gemma-4-E4B, where it is as compromised as the rollout students (appendix Fig. 22). Collapse and context reliance are thus two separable products of procedural automatisation, which is why their orderings need not match.
Summary
We tested the alternatives the reviewer raised, tracking how ready the students are to stop, forcing them to reason longer and adding length-matched and cross-model contexts, and retrained all students on a single shared training set. The collapsed students are further from stopping than Base throughout their reasoning and repeat themselves when forced to continue, collapse grows with how close the context is to the model's own reasoning, and context reliance is a separable part of the same procedural automatisation. We hope these additions address the reviewer's concerns.
References
[1] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. EMNLP, 2025.
Response to Reviewer 91VM (1/3): stopping policy and budget forcing
We thank the reviewer for finding the difficulty-dependent transfer compelling and the synthetic and malicious-context experiments useful. Following the review, we (i) tested whether the collapsed students learned a stopping bias, by tracking how ready they are to end their reasoning at every step and by forcing them to reason as long as Base; (ii) trained students whose teacher is given correct rollouts written by other models, and a control in which the model's own rollout is cut to the length of another model's; (iii) measured how close each rollout is to the model's own reasoning; (iv) computed the teacher PVI of every math context; and (v) retrained all students on a single training set shared by every context, re-running almost every evaluation in the paper. The new results support the automatisation account over the alternatives, and we address each point below.
W1+Q1
We tested both alternatives directly: the collapsed students do not learn to stop earlier, and their shortening is not tied to any context. A stopping policy would make a student readier than Base to end its reasoning while it is still working on the problem. On the 300 problems of the new test set, we fed each student's own rollouts back to it and tracked, at every position of the reasoning, the rank of the end-of-thinking token
</think>in its next-token distribution (new Appendix C.6, Fig. 20). Students of the model's own rollouts (rollout-correct, RC; rollout-unfiltered, RU) rank it lower than Base, not higher, throughout their reasoning (typically around 84,000th, against 69,000th for Base), and the more a student collapses, the lower it ranks the token. They do not stop prematurely; they stop because they have learned to allocate less computation to the problem. As for the reviewer's second alternative, context-dependent shortcuts would not transfer the way collapse does: students are evaluated without any context, yet they shorten on problems and domains never seen in training (ZebraLogic, MBPP+). This is what automatisation of a procedure predicts (Section 2): a compiled routine runs wherever it is triggered. The framework also makes a prediction of its own: an automatised routine no longer scales its effort with the problem, so a collapsed student should cut the hardest problems the most. We test this with a new measure, the effort elasticity (new Fig. 5b): the slope of a student's log reasoning length against Base's across test problems, with Base's length as a measure of difficulty. A slope of 1 means every problem is shortened by the same fraction, so effort still grows with difficulty; a slope below 1 means the hardest problems are cut the most. Only the students of rollouts fall below 1 (0.76–0.77 for RC, RU and RC-R1; 0.92 for RC-G4), while those of answers (AO) and solutions (FS) keep about 1 (1.00–1.12), including the full solution padded to the length of RC's context (1.06). Neither context length nor a stopping policy, ruled out above, accounts for this pattern. The collapsed students thus reason through a shorter procedure, rather than stopping early or exploiting cues in the context.We ran the suggested intervention: a longer budget does not restore accuracy, because the collapsed students repeat themselves instead of reasoning further. On ZebraLogic, we forced every student to reason at least as long as Base does on each puzzle, replacing every earlier end-of-thinking token with "Wait" (budget forcing, as in s1 [1]; new Appendix C.7, Fig. 21, Table 6). The collapsed students are forced on about 9 puzzles out of 10, and their accuracy falls instead of recovering (RC: from 63.0% to 46.7%). On the puzzles where forcing does not cause a repetition loop, no student gains accuracy. What separates the students is how they use the extra budget. Forced to add 1,000–2,000 tokens past their first stop, RC and RU fall into a loop in 46% and 62% of their rollouts, against at most 3% for the students of answers and solutions forced to add as much on puzzles of the same size. A student that had stopped too early would use the extra budget to keep reasoning; the collapsed students instead gain nothing from it and often loop. This agrees with the direct measurement in (a): they are further from stopping than Base throughout their reasoning, and the end-of-thinking token climbs the ranking only in its last percent (new Appendix C.6, Fig. 20b). The impairment therefore concerns the reasoning procedure itself, not its termination.