acceptodds
Under review as a conference paper at ICLR 2027

Learn Your Own Thoughts: Abstract Token Curriculum

Abstract

Large Language Models (LLMs) have achieved remarkable reasoning capabilities by utilizing chain-of-thought (CoT) as a scratchpad for intermediate stages of thinking. However, CoT techniques require explicit supervision on thinking tokens, which requires rich, task-specific data. In this work, we propose Abstract Token Curriculum (ATC), a novel curriculum learning framework that elicits effective continuous intermediate representations without direct supervision or manual scratchpad design. ATC gradually increases problem complexity through a sequence of distributions, training the model to develop internal abstract `thoughts' in the continuous representation space. This paper provides both theoretical and experimental evidence for the benefits of ATC and its advantages over previous methods for training continuous thoughts. Theoretically, we show that for learning parity functions with single-layer softmax attention using ATC, attention naturally focuses on the CoT tokens in the context that provide the “easiest path” to predicting the next token. Experimentally, we show that ATC learns effective continuous thoughts from answer supervision alone on parity, graph reachability, and multi-digit addition tasks, matching or exceeding continuous-CoT baselines that require intermediate supervision.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
continuous chain of thoughts, curriculum learning, chain of thought reasoning, graph reachability reasoning, parity learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tLySOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Abstract Token Curriculum (ATC), which trains continuous latent thoughts using answer supervision without explicitly supervising intermediate reasoning steps. ATC jointly increases task difficulty and the number of latent thoughts, with additional backtracking and truncated backpropagation for longer reasoning chains. Experiments on parity, graph reachability, and multi-digit addition demonstrate its effectiveness, alongside a theoretical analysis of attention behavior.

Critical strengths
  1. ATC achieves 99.9% accuracy on graph reachability using only answer supervision, while removing either the curriculum or continuous thoughts reduces accuracy to approximately 50%. These results provide evidence that combining curriculum learning with latent computation is effective in this setting.

  2. On graph reachability, truncated backpropagation alone achieves 51.0% accuracy, while adding curriculum backtracking restores it to 97.4%. This also reduces peak memory from 5.78 GiB to 1.55 GiB and training step time from 198.6 ms to 124.7 ms.

Critical weaknesses
  1. In the eight-digit addition experiment, ATC progressively increases the number of input digits, while Coconut receives the full eight-digit problem throughout training and predicts intermediate partial sums. Therefore, the reported 98.7% versus 0% comparison involves substantially different curriculum designs, not just different supervision strategies. Similarly, on graph reachability, ATC advances based on answer accuracy while Coconut uses intermediate-node prediction accuracy. Without comparisons under more closely matched curricula and computational budgets, it is difficult to determine how much of the improvement comes from the proposed training method.

  2. Theorem 1 assumes that the preceding thought already perfectly encodes the correct prefix parity and analyzes only a single gradient update on the key parameters of a one-layer attention model. It does not establish how such representations emerge from answer-only supervision under the actual training procedure. Moreover, the empirical attention analysis measures approximately 50% attention on the last thought, which is also the readout position itself. This does not directly validate the claimed attention mechanism.

  3. The main text ends immediately after the attention analysis, without a discussion or conclusion, while the conclusion is placed at the end of the appendix. Important limitations of the theoretical assumptions and baseline comparisons are not adequately discussed in the main text. There are also reporting and presentation inconsistencies: Section 4 states that the experiments use 20 seeds, while Appendix I.3 reports 10 seeds for addition; some figure captions are minimal, and several cross-references in the appendix are incorrect. These issues make the paper's technical contributions harder to assess.

Questions for authors
  1. Can the authors compare ATC and Coconut under the same input-difficulty curriculum, with matched training budgets? In particular, how much of the addition performance gap is caused by ATC's progressively shortened input problems rather than the difference in intermediate supervision?

  2. How does the correlation-one assumption in Theorem 1 relate to the representations actually learned by ATC? Can the authors directly verify that intermediate thoughts encode the expected partial computations and clarify how the attention analysis distinguishes dependence on the previous thought from self-attention at the readout position?

  3. The experiments use task-specific difficulty schedules based on prefix parity, graph distance, and the number of input digits. How much task-specific knowledge is required to construct these curricula, and what evidence supports applying ATC to tasks without such explicitly available decompositions?

  4. Can the authors clarify the inconsistent number of random seeds reported for the addition experiments and provide a consistent description of the evaluation settings?

6 comments
AuthorsOpenReview ↗

Author response-1 (thread-1)

We thank the reviewer for their time to review and the detailed response. Please see our response below

(W1): In the eight-digit addition experiment, ATC progressively increases the number of input digits, while Coconut receives the full eight-digit problem throughout training and predicts intermediate partial sums.

(Answer): The difference in prompts is exactly the point of the comparison. We highlight that using curriculum with input prompt being the intermediate difficult question is the reason for ATC’ success. The original Coconut method uses the full prompt to an intermediate stage question and changing this would make the baseline a “Coconut” method. We believe the rest of the evaluation is fair in comparison.

(W2): Theorem 1 assumes that the preceding thought already perfectly encodes the correct prefix parity and analyzes only a single gradient update on the key parameters of a one-layer attention model. It does not establish how such representations emerge from answer-only supervision under the actual training procedure. ..

(Answer): Proving how the whole representation emerges is out of scope of the theorem. Our theorem proves that once the last thought encodes the parity of the earlier bits, a single gradient step on the attention keys can direct attention to that thought.

We find sufficient empirical evidence to believe that this step is the key to learning chain of thought. For example, in the graph experiments, the last intermediate thought encodes the answer information and corrupting that thought drastically drops the answer accuracy.

(W3): The main text ends immediately after the attention analysis, without a discussion or conclusion, while the conclusion is placed at the end of the appendix. Important limitations of the theoretical assumptions and baseline comparisons are not adequately discussed in the main text. There are also reporting and presentation inconsistencies: Section 4 states that the experiments use 20 seeds, while Appendix I.3 reports 10 seeds for addition; some figure captions are minimal, and several cross-references in the appendix are incorrect. These issues make the paper's technical contributions harder to assess

(Answer): Thank you for this suggestion and we have included a conclusion in the manuscript since the page limit is now increased to 10. We will highlight the scope of the theorem more clearly in our answer. The seeds number was a typo and we fixed it to 10 and revise the figure captions as well. To clarify, graph experiments used 20 seeds and the addition experiments used 10 seeds. We will make the following changes to give a consistent description of the evaluation settings:

  1. Replace line 428 by mentioning separate seeds for graph and addition tasks.
  2. Graph setup and Table 1: lines 1377–1384 and 343–345: mention the graph results used 20 seeds.
  3. Table 2 and Figure 4, lines 378–379 and 390. We will state that clearly that the addition results use 10 seeds per condition.

(Q1): Can the authors compare ATC and Coconut under the same input-difficulty curriculum, with matched training budgets? In particular, how much of the addition performance gap is caused by ATC's progressively shortened input problems rather than the difference in intermediate supervision?

(Answer:) The addition experiments already match the training budget in epochs. Both methods use the same 400-epoch, model architecture, optimizer settings, and batch size (Appendix I.3).

The difference in prompts is exactly the point of the comparison. ATC presents shorter addition problems at early stages and gradually increases their length. Our Coconut-style baseline keeps the full eight-digit problem in the prompt at every stage and changes the partial sum it is trained to predict. Giving Coconut the same prompts as ATC would change its curriculum and yield a modified baseline. The reported accuracy gap compares the two curricula as designed.

(continued)

AuthorsOpenReview ↗

Author response-1 (thread-2)

(Q2): How does the correlation-one assumption in Theorem 1 relate to the representations actually learned by ATC? Can the authors directly verify that intermediate thoughts encode the expected partial computations and clarify how the attention analysis distinguishes dependence on the previous thought from self-attention at the readout position?

(Answer): We perform several intervention experiments (Section-4 and Figure-5) to highlight this assumption. In graph experiments, we show that just corrupting the intermediate last stage thought can degrade the answer, while preserving it, does not change the answer. So this assumption is highly consistent with our experiments (Figure-5).

In addition to the evidence already in the paper, we ran further probing experiments on the thoughts themselves (https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/ATC_interpretable_thoughts.pdf). For graph tasks, ATC’s thoughts encode the depth-by-depth progression of breadth-first search. https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/token_node.pdf . For addition, an MLP probe accurately decodes the true carry bit at each digit position from the hidden state of the corresponding ATC thought https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/addtion_probing.png

Moreover, we consistently observe that the attention concentration shown in the theorem,which is reflected in the graph experiments. For ATC, 63% of the attention is on the last thought, and for Coconut it is 36%, in both cases above the earlier thoughts and the rest of the prompt (https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/attention_conc.png). Together, the interventions and the attention show that the model relies on the last thought.

(Q3): The experiments use task-specific difficulty schedules based on prefix parity, graph distance, and the number of input digits. How much task-specific knowledge is required to construct these curricula, and what evidence supports applying ATC to tasks without such explicitly available decompositions?

(Answer:) We believe that coming up with a curriculum for a task is substantially easier than designing intermediate supervision or identifying what kind of intermediate state is useful for the model, which ATC avoids. For example providing the BFS tree as the intermediate supervision might be difficult if someone is not familiar with BFS algorithm for solving graph connectivity while distance form source to node is a simple heuristic metric we use for defining the curriculum in this case, which encourages the model to solve the problem by saving the BFS tree levels in intermediate states.

However, one can be fully unaware of BFS algorithm, and the point of ATC is the model grows the useful relevant intermediate representations itself in the hidden thoughts without explicit supervision. Similarly in addition, one can be unaware of the addition algorithm and simply define the curriculum to be the number of digits of the input, which we use for ATC.

So the point of ATC is exactly that the model designer can come up with simple heuristic difficulty measures to define the curriculum, and leave the formation of useful intermediate representations to the model itself. We see that this paradigm is highly successful and importantly avoids the need of direct intermediate supervision for every specific tasks.

(Q4): Can the authors clarify the inconsistent number of random seeds reported for the addition experiments and provide a consistent description of the evaluation settings?

(Answer): Please see response to W3.

If these clarifications and new results address your concerns, we would appreciate it if you would reconsider your score.

Thank you for the detailed clarifications and additional probing experiments, which have addressed some of my concerns. However, my main concern about the fairness of the comparison with Coconut remains, as the different curriculum designs make it difficult to isolate the source of the performance gains. I would be happy to reconsider my score if this concern can be further addressed.

Reviewer_kZhQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper wants to get continuous chain of thought without any supervision on the actual thoughts. The method, ATC, trains in stages: each stage uses a harder input distribution and one more continuous thought (the last hidden state is fed back as the next input). Only the final answer is supervised. In theory, for parity with one-layer attention and a "correlation-one" assumption on the thoughts, they show one key-only gradient step puts 99% of the final-answer attention on the last thought, with probability at least 1/4. They say this resolves the conjecture of Abbe et al. (2024).

Critical strengths
  • The research question is good: can continuous thoughts be learned from answer supervision only. The graph task follows the shortcut-controlled design of Mirtaheri et al., so the answer can't be read from local features.
  • Backtracking + truncated backprop is useful in practice. Table 3 is clear (51.0% without backtracking vs 97.4% with it at b=5, with 0.27x peak memory).
Critical weaknesses
  • I don't think Theorem 1 supports the story. In the one-layer model the target is y=γkϵk+1y = \gamma_k \epsilon_{k+1}, and y^=∑jαjw⊤vj\hat y = \sum_j \alpha_j w^\top v_j where each value only depends on its own token. If αn+k≥0.99\alpha_{n+k} \ge 0.99, y^\hat y is almost a function of γk\gamma_k only, but E[y∣γk]=0E[y \mid \gamma_k] = 0. So the attention pattern in the theorem actually can't predict the target. Also, by Lemma 2 the sign of the last-thought key update depends on BEγkB_E \gamma_k, so the concentration only happens for one value of γk\gamma_k. Values and the head stay at the small init scale, so the loss also doesn't decrease after the step. With one step, keys only, huge η\eta, keys as free per-token parameters, and the thoughts assumed to already be the prefix parity, I don't think this can "resolve the conjecture of Abbe et al."
  • "No intermediate supervision" is not fully true to me. The curriculum is designed so that the answer of stage s is exactly the intermediate state of the hard problem. For addition this is the same label as Coconut baseline. So the human design moved from the scratchpad to the curriculum, which is what the intro says it wants to avoid.
  • I don't know if this is mandatory, but I think it's better to have the conclusion part in the main text, from the view of readers.
Questions for authors
  • For Theorem 1, y^\hat y is almost independent of ϵk+1\epsilon_{k+1}. How does this pattern relate to learning the stage-(k+1)(k+1) target?
  • Are the candidates visible at test time? I.2.2 says dropout is removed at evaluation, but Table 2 says candidates are hidden.
  • What is the No-CoT result on addition, and how was Coconut implemented and tuned?
3 comments
AuthorsOpenReview ↗

Author reponse-1

We thank the reviewer for their time to review and the detailed response. Please see our response below.

(W1): I don't think Theorem 1 supports the story. ....So the attention pattern in the theorem actually can't predict the target....

(Answer:) We resolve a misread or a confusion from the reviewer. The 99% attention due to the key update does not mean y^\hat{y} is only correlated with the last thought. The next input bit ϵk+1\epsilon_{k+1} still affects the prediction through its own value and the softmax weights.

Our theorem shows that the interaction between the last thought and the next input bit gives the last thought’s key a learning signal, even though the last thought alone has zero correlation with the answer (of course since its parity). A single key update can therefore direct attention to the relevant intermediate state. The theorem establishes this selection mechanism, while computing the next parity requires a further step that combines the two inputs.

Connection to Abbe et al: Theorem 1 gives a mechanism for the scratchpad intuition of Abbe et al. Once the last thought stores the parity of the earlier bits, the next answer depends on just two tokens which is the last thought and the next input bit. We show that a single key update can use this interaction to direct attention to the relevant thought. We will revise the wording to state this connection precisely.

(W2): "No intermediate supervision" is not fully true to me. The curriculum is designed so that the answer of stage s is exactly the intermediate state of the hard problem. For addition this is the same label as Coconut baseline. So the human design moved from the scratchpad to the curriculum, which is what the intro says it wants to avoid

(Answer:) We clarify a misunderstanding and we believe the above the statement The curriculum is designed so that the answer of stage s is exactly the intermediate state of the hard problem is not correct. Our curriculum specifies which problems to learn first, but does not specify the intermediate steps for solving them. More generally, useful intermediate states need not be answers to simpler problems at all.

Consider learning to check whether brackets in a string match, starting with short strings and gradually increasing their length. Every training label is just “correct” or “incorrect,” but the model might learn to remember which brackets still need closing. We never supervise that memory or tell the model how to update it.

The curriculum therefore still involves human guidance, but it leaves the intermediate computation, which is much more complicated than the curriculum, to be learned. We will clarify that “no intermediate supervision” means no explicit targets for intermediate reasoning states, rather than no task-specific guidance.

(W3): I think it's better to have the conclusion part in the main text,

(Answer:) Thank you, we will move the conclusion to the main section.

(Q1): For Theorem-1...

The response to W1 clarifies this.

(Q2): Are the candidates visible at test time?

At test time, both candidate nodes are included in the graph reachability prompt. For addition, the model predicts the sum directly from the operands, with no answer candidates in the prompt. We will clarify this distinction in the paper.

If these clarifications and new results address your concerns, we would appreciate it if you would reconsider your score.

Reviewer_kZhQOpenReview ↗

Response to authors

Thanks for the reply and your extra work. I’m still not fully convinced on these responses. For W1: in a one-layer model, once almost all of the attention sits on the last thought, the next input bit barely affects the prediction. There is also no later step in the architecture where the two could be combined. On W2: if I understand correctly, ATC’s stage answers are still the same labels Coconut uses as intermediate supervision. So overall, I’ll keep my score.

Thanks for the reply. We emphasize that our theorem isolates one particular step in how curriculum-based CoT works. Once the most recent thought encodes the parity of the preceding input token, a single gradient update on the keys can concentrate softmax attention on that thought.

Accessing the information on that last thought is necessary for the model to reuse it at the next stage. The next input bit can still affect the attention output. The mechanism through which the model learns to combine that input bit with the stored parity in the last thought, for example through an MLP after attention in a full Transformer, is a separate question beyond the scope of our theorem.

Reviewer_ehW9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a curriculum learning strategy to train Coconut-style latent reasoning tokens (abstract tokens) in language modeling. Rather than an explicit supervision, the authors propose difficulty-based curriculumn learning, particularly based on the computational complexity of synthetic algorithmic tasks. Evaluation on synthetic algorithmic reasoning tasks, including parity, graph reachability, and multi-digit addition shows performance improvements.

Critical strengths

The difficulty of explicit supervision for latent reasoning tokens provides a clear motivation.

Critical weaknesses
  1. The most critical limitation of this paper is that whether such curriculum is applicable and useful in real-world reasoning tasks remains unclear. Unlike synthetic algorithmic tasks such as automaton modeling, real-world language-model reasoning problems often are not available to extract a well-defined computational depth or a natural easy-to-hard curriculum. Such discrepancy can limit the potential applicability of ATC in real world reasoning, contrast to Coconut that can leverage existing language-based reasoning traces for supervision.

  2. The paper lacks sufficient evidence for the role of abstract tokens. Beyond the restrictive theoretical assumptions, the paper even provides limited evidence of whether abstract tokens can encode meaningful states during training. For instance, Theorem 1 assumes that the latest token already presents the correct prefix parity, rather than explaining how such a representation emerges from answer supervision. Moreover, it remains unclear whether this assumption extends to realistic language-model reasoning tasks, raising concerns about whether ATC is useful in practical LLM training.

Questions for authors

Can the authors provide evidence or justification that ATC learns meaningful representation rather than merely learning to attend to an already informative representation as in Theorem 1?

4 comments
AuthorsOpenReview ↗

Author response-1

We thank the reviewer for their time to review and the detailed response. Please see our response below.

(W1) : The most critical limitation of this paper is that whether such curriculum is applicable and useful in real-world reasoning tasks remains unclear..

(Answer) : We review that the main message of the paper is answers to progressively harder questions can be enough for a model to build latent representations that is used in later reasoning. As the reviewer pointed out, building these intermediate questions for language model reasoning problems is still open, and our work remains a precursor to this work. The original Coconut method starts from a pretrained language model and uses discrete reasoning traces during training. In our graph experiments, we train both models from random initialization and without any discrete tokens. ATC matches the strongest Coconut-style baseline at 99.9% accuracy using only final-answer supervision, while Coconut is still supervised with the intermediate target nodes.

(W2): The paper lacks sufficient evidence for the role of abstract tokens. Beyond the restrictive theoretical assumptions, the paper even provides limited evidence of whether abstract tokens can encode meaningful states during training.

(Answer) : We disagree that the paper provides limited evidence for the role of the abstract tokens. In graph reachability, ATC starts from random initialization and generates its own latent thoughts while receiving only final answers as training targets. In Figure 7, we replace the earlier thoughts with noise and show that restoring a single intermediate thought from the original run recovers both the next prediction and the final answer. This has been validated extensively in the graph and parity setups.

In addition to the evidence already in the paper, we ran further probing experiments on the thoughts themselves (https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/ATC_interpretable_thoughts.pdf). For graph tasks, ATC’s thoughts encode the depth-by-depth progression of breadth-first search. https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/token_node.pdf . For addition, an MLP probe accurately decodes the true carry bit at each digit position from the hidden state of the corresponding ATC thought https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/addtion_probing.png

We believe these experiements give sufficient evidence for the role of abstract token.

While the assumption on the theorem is made for the sake of the proof, our experiments convincingly show that the intermediate representations encode the answers for that particular stage and is a consequence of minimising the loss for that stage. Moreover, even if the intermediate stage solution drifts, we fix it through curriculum backtracking.

If these clarifications and new results address your concerns, we would appreciate it if you would reconsider your score.

Thank you for the additional probing experiments, which provide stronger evidence that ATC learns meaningful intermediate representations and largely address my concern in W2.

However, my primary concern in W1 remains unresolved. While the authors present ATC as a difficulty-based curriculum learning approach, the evaluated curricula are not merely organized from easier to harder problems. Rather, the solutions to easier problems directly correspond to intermediate computations required for harder ones, such as prefix parity, partial addition, and successive graph traversal. This distinction is important, as the curricula already encode substantial task-specific structure about how the target computation can be decomposed. Consequently, the results demonstrate that ATC can leverage such structured curricula, but do not establish whether difficulty-based training alone can induce useful latent reasoning.

In realistic mathematical reasoning, easier problems do not necessarily correspond to intermediate steps of harder problems, and such computational decompositions are generally not available in advance. I would therefore like to see evidence that ATC is effective in more realistic reasoning settings, where the curriculum is defined by problem difficulty without explicitly aligning with the intermediate computations of the target task. Such evidence would substantially strengthen the claimed applicability of the approach and allow me to reconsider my assessment.

AuthorsOpenReview ↗

Author response-2

The reviewer asks for a setting, where the curriculum is defined by problem difficulty and where the harder problem does not explicitly reuse the easier problem’s intermediate computation.

An example of such design would be learning a base skill to solve an advanced difficult task. This would for example require a Mathematical setup where a model could first learn to solve quadratic equations and later apply that skill to higher polynomial equations that reduce to quadratics by substitution.

We hypothesize that sufficiently expressive continuous thoughts could support this kind of skill transfer. Testing that hypothesis is a separate research question beyond the scope of this paper and the present rebuttal. However, we believe this is to be a future work of our project and currently outside the scope of this paper and especially within the short time of a rebuttal.

The authors state that their main message is that "answers to progressively harder questions can be enough for a model to build latent representations that is used in later reasoning." However, the evaluated curricula rely on task-specific computational structures, where easier problems directly correspond to intermediate computations of harder ones. This is my key concern since the observed benefits may instead stem from the structures embedded in the curricula.

I acknowledge ATC's contribution within these restricted settings. However, coconut has already demonstrated latent reasoning on more realistic tasks such as GSM8K. Considering that the benefits of ATC are demonstrated only under substantially more restrictive conditions, I do not consider its contribution sufficiently significant over prior work to warrant acceptance at ICLR. I therefore maintain my current assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Clarifications and rebuttal summary

We thank the reviewers for their time and feedback. In our responses, we clarified the scope of Theorem 1, and will update the manucsript accordingly. One common question amongst the reviewers were what was the evidence of the role of abstract tokens.

In addition to the evidence already in the paper, we ran further probing experiments on the thoughts themselves (https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/ATC_interpretable_thoughts.pdf). For graph tasks, ATC’s thoughts encode the depth-by-depth progression of breadth-first search. https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/token_node.pdf . For addition, an MLP probe accurately decodes the true carry bit at each digit position from the hidden state of the corresponding ATC thought https://anonymous.4open.science/r/iclr_27_rebuttal-94D6/addtion_probing.png

We believe these experiements give sufficient evidence for the role of abstract token. We welcome any follow-up questions and would be happy to clarify any remaining concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.