- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies training-time CoT supervision through gradient-descent trajectories on linear-regression tasks. A linear-attention model learns one-step transitions and is then applied recursively beyond the training horizon. The analysis characterizes regimes of GD recovery, bounded error, quadratic divergence, and exponential divergence, including non-monotonic effects of increasing the training CoT budget. Experiments with fully trained linear and softmax attention test whether these qualitative behaviors persist beyond the restricted theoretical model.
Critical strengths
-
The paper addresses a concrete question beyond inference-time expressiveness: how finite trajectory supervision affects the stability of longer test-time computation. The non-monotonic dependence on supervision budget is an interesting finding.
-
The experiments with fully trained attention provide useful support beyond the single-matrix theoretical parameterization. The distinction between error accumulation and amplification also helps organize the observed failure modes.
Critical weaknesses
-
The theoretical setting builds substantial structure into both the teacher and the student. The teacher supplies GD state transitions, while the effective student fixes most attention blocks and learns only a matrix through a strongly convex objective. This gives a tractable account of recovering and iterating a prescribed update rule, but supports a narrower interpretation than a general explanation of how CoT supervision teaches reasoning.
-
The broader explanatory contribution remains unclear. The paper extends prior in-context regression and iterative-dynamics analyses to finite teacher trajectories, but the empirical extensions remain within the same regression-trajectory setting. Moreover, the softmax experiment reproduces non-monotonicity without clearly separating the two divergence regimes. Thus, the evidence supports some qualitative predictions more strongly than the full four-regime account or the claimed practical guidance for CoT design.
Questions for authors
-
Which qualitative conclusion is specific to finite teacher-trajectory supervision and cannot be obtained from the closest existing analyses of iterative in-context learning?
-
In the fully trained experiments, how robust is the non-monotonic behavior to changes in teacher step size and trajectory length? Which parts of the four-regime picture should be expected to survive beyond linear attention?
3 comments
Rebuttal (2/3)
W2: The broader explanatory contribution remains unclear. The paper extends prior in-context regression and iterative-dynamics analyses to finite teacher trajectories, but the empirical extensions remain within the same regression-trajectory setting. Moreover, the softmax experiment reproduces non-monotonicity without clearly separating the two divergence regimes. Thus, the evidence supports some qualitative predictions more strongly than the full four-regime account or the claimed practical guidance for CoT design.
Thank you for the comment. For the contribution beyond prior work, please see the reply to Q1 below.
We agree that our experimental results mainly provide strong support for some of the qualitative predictions. Although we do not reproduce all four regimes in the fully trained experiments, we believe that our results still lead to the following practical guidance for CoT design.
- Prepare more teacher CoT for tasks with shorter contexts, or lengthen the contexts to reduce the teacher CoT required. Since the budget required for recovery depends on the context length (), the amount of teacher CoT should not be chosen independently of the context length (Section 5.1, paragraph "Context length determines the CoT budget required").
- With short contexts, be careful when increasing teacher CoT gradually. Because an intermediate budget can worsen long-horizon performance, teacher CoT should be added until the budget exceeds the unstable range, and long test-time reasoning should be avoided if this is not possible. We have added this guidance to Section 5.3. This non-monotonicity appears in both linear and softmax attention and persists across teacher step sizes (Section 6 and the new Appendix G).
- Collect more diverse tasks rather than lengthening the CoT of the same tasks. The value of teacher CoT is determined by the diversity of states rather than the number of steps, and if successive teacher states are redundant, shorter teacher CoT from more diverse tasks is more informative (Section 5.1, paragraph "CoT budget matters more than task diversity or CoT length alone"). In end-to-end trained models, increasing the CoT length at a fixed number of tasks did not reproduce the effect of adding tasks (new Appendix G).
Q1: Which qualitative conclusion is specific to finite teacher-trajectory supervision and cannot be obtained from the closest existing analyses of iterative in-context learning?
Thank you for the question. As summarized in the Related Work (paragraph "Theoretical studies of CoT"), prior analyses of iterative in-context learning mainly study the test-time behavior of trained models and do not characterize the effect of providing finite teacher CoT as training data. We have added Appendix I, "Further Related Work," which compares our work with the closest analyses in detail.
In short, unlike [1], we show that overthinking can occur even under in-distribution generalization and consider the regime where the number of samples and the dimension are of the same order. Unlike [5], where the model is trained only on the final answer, we train the model on finite teacher trajectories. This yields the following conclusions specific to finite teacher-trajectory supervision. First, the learned algorithm can succeed beyond the number of steps seen in the teacher CoT (Result 4.1 and Section 5.1). Second, the number of tasks and the CoT length enter only through the training CoT budget , and the budget required for recovery depends on the context length through the boundary (Section 5.1). Third, a failure to reproduce the algorithm takes two forms, accumulation and amplification (Section 5.2). Fourth, long-horizon performance is non-monotonic in the training CoT budget (Section 5.3).
[1] J. Huang, Z. Wang, J. D. Lee, "Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thought," ICLR 2025.
[2] J. Kim, T. Suzuki, "Transformers Provably Solve Parity Efficiently with Chain of Thought," ICLR 2025.
[3] E. Abbe, S. Bengio, A. Lotfi, C. Sandon, O. Saremi, "How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad," NeurIPS 2024.
[4] A. Javanmard, B. Mirzasoleiman, V. Mirrokni, "Understanding the Role of Training Data in Test-Time Scaling," ICLR 2026.
[5] K. Takanami, C. Pehlevan, "An Asymptotic Theory of Chain-of-Thought in In-Context Learning," NeurIPS 2026.
Rebuttal (3/3)
Q2: In the fully trained experiments, how robust is the non-monotonic behavior to changes in teacher step size and trajectory length?
Thank you for the question. We conducted additional experiments with end-to-end trained linear and softmax attention (Appendix G, Figure 4).
To compare teacher step sizes at the same amount of GD progress, we evaluated the error at the depth satisfying . For every step size, both models show the non-monotonic pattern. The step size shifts the budget at which the error drops and changes the height of the peak, but it does not remove the non-monotonicity.
When the budget was increased through the teacher CoT length at a fixed number of tasks, the abrupt transition to stable inference observed when increasing the number of tasks was not reproduced. For linear attention, the error decreases only gradually with the CoT length, and for softmax attention it remains almost unchanged. This suggests that in end-to-end training the model must also learn how to use the context, and since this information is supplied once per task, a longer CoT cannot compensate for it. In the effective model, this part is fixed by the architecture, so the learned solution depends only on .
These results suggest that the qualitative effect of increasing task diversity holds fairly generally in experiments, whereas the result that performance depends only on the teacher CoT budget may be limited to the effective model. We have included these results in Appendix G.
Q2(continued): Which parts of the four-regime picture should be expected to survive beyond linear attention?
Thank you for the question. Even for softmax attention, a sufficient teacher CoT budget yields stable inference beyond the training horizon, and the non-monotonicity in the budget (stable, then unstable, then stable) persists. However, we cannot confirm whether the unstable region clearly separates into the accumulation and amplification regimes. We attribute this to the fact that the Jacobian of softmax attention depends on the state and changes along the inference trajectory (Section 6). Even so, the transition with respect to the budget is phase-transition-like, which is consistent with the prediction of the effective model.
Rebuttal (1/3)
Thank you for your thoughtful review. We address your questions and concerns point by point below. The manuscript has been revised accordingly, with changes highlighted in red.
Thank you for this comment. We agree that our analysis constrains the attention parameters and formulates the teacher CoT as GD, so it does not provide a universal explanation of CoT supervision in the complex mechanisms of real LLMs. At the same time, the constraints on the student are not arbitrary. As described in Section 3.3 and detailed in Appendix A, each fixed block is justified by one of three considerations: the rotational symmetry of the data, the fact that fixing it entails no loss of generality in our setting, or preventing the model from advancing its inference without consulting the context.
That said, we emphasize that the problem is far from trivial even in this setting. For example, although the student's hypothesis class contains the parameters that reproduce GD, there is no guarantee that training finds them, because the amount of data is finite. Our main results, the phase transitions and the non-monotonicity, are all caused by this finiteness of data and are not built into the model structure by hand.
In addition, reaching an answer by repeatedly applying the same update rule is a structure shared by many algorithms, as discussed in the new Section 3.6, "Connection to CoT in Practice," and whether reasoning generalizes beyond the length of the teacher CoT is one of the central questions about reasoning generalization. Our reasoning setting is therefore not far removed from real tasks.
Furthermore, Section 6 confirms that similar qualitative trends appear when linear and softmax attention are trained end-to-end on a non-convex objective, and the new Appendix G shows that the non-monotonicity in these models is robust to the teacher step size. More importantly, qualitatively similar phenomena have already been observed in prior work based on large LLMs. For example, [1] reports a non-monotonic behavior in reasoning SFT with long CoT, where generalization performance first drops during training and then recovers. We discuss the relation of this observation to our theory at the end of Section 5.3, and we believe our work is one of the first to explain this behavior. A theory that can predict the reasoning of LLMs will be built through a series of studies, and we believe this paper is an important step.
[1] Q. Ren, P. Wang, R. Cai, S. Shao, D. Guo, Y. Xie, Y. Li, Q. Zhang, X. Hu, J. Shao, D. Liu, "Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability," ICML 2026.