acceptodds
Under review as a conference paper at ICLR 2027

Curriculum Learning Shapes the Computation Learned by Arithmetic Transformers

Abstract

Do transformers learn arithmetic by mirroring digit-wise algorithms, or by exploiting statistical shortcuts? Here we study how an encoder transformer learns to solve addition and find that the most significant digits are learned before the less significant ones, in contrast to how one would compute addition by hand. We exactly solve the learning dynamics of a linear model on the same task and show that it learns a continuous magnitude representation of the sum, despite the task being encoded symbolically. As this representation is learned, it encodes the most significant digits accurately first, since errors in this estimate disproportionately affect less significant digits. Next we apply curriculum learning to the transformer to force it to learn the task in the canonical order, and find that this shifts the model from an estimation-like strategy to digit-wise computation, and improves generalisation performance to tasks with a similar computational structure as addition. These findings demonstrate that different strategies compete inside neural networks, and structuring the training curriculum can fundamentally alter a model's internal strategy.

Primary area:
interpretability and explainable AI
Keywords:
learning dynamics, transformers, arithmetic, curriculum learning, mechanistic interpretability, strategy choice

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A6XbOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how training affects the computational strategies learned by Transformers on arithmetic tasks. The authors first observe that a non-autoregressive Transformer trained directly on addition learns the most significant digits before the less significant ones, opposite to the order used in standard digit-wise addition.

To explain this behavior, the authors analyze a simple two-layer linear network and derive its learning dynamics. They show that the network learns a continuous representation of the sum's overall magnitude. With an ideal decoder, this representation explains why more significant digits become accurate earlier in training. The authors then use attention pattern analysis, activation patching, and long carry-chain tests to show that directly trained Transformers exhibit behavior more consistent with numerical estimation than with standard digit-wise computation.

Finally, the authors introduce a curriculum that gradually increases the number of digits during training. This encourages the model to learn less significant digits first and develop attention patterns associated with digit-wise addition and carry propagation. The experiments show faster training, better generalization to unseen digit combinations, and more efficient transfer to related arithmetic tasks.

Critical strengths

1. A good combination of theoretical analysis and mechanistic experiments. The paper goes beyond observing that Transformers learn digits from most to least significant. The authors use an analytically tractable linear model to explain why a magnitude-based representation may be favored early in training. They then examine the computation learned by Transformers through attention patterns, activation patching, and carry-chain tests. In particular, the causal interventions on attention heads associated with local digit summation and carry propagation provide useful evidence for the digit-wise computation learned under curriculum training.

2. Interesting findings on how training order affects computation and generalization. The paper shows that changing the training curriculum alone, without modifying the architecture or final task, can reverse the order in which digits are learned and lead to attention patterns associated with local summation and carry propagation. The curriculum-trained model also performs better on unseen digit combinations and adapts faster to related tasks. I find this interesting because it suggests that curriculum learning can change not only training efficiency but also the computational strategy a network learns.

Critical weaknesses

1. The connection between the theoretical model and the actual Transformer mechanism is not fully established. The linear model is shown to learn a representation of the overall sum magnitude, but it cannot solve the full addition task on its own. The theoretical explanation of digit acquisition order also relies on an additional ideal decoder. For the Transformer, the evidence for an estimation-like strategy comes mainly from digit learning order, attention patterns, carry-chain behavior, and the decodability of intermediate representations. While these observations provide evidence for estimation-like behavior, they do not establish whether the Transformer uses the continuous magnitude representation identified in the linear model, rather than other heuristic strategies. More direct evidence would be needed to establish whether the Transformer actually learns and uses the magnitude representation identified in the linear model.

2. Algorithmic generalization and the scope of the findings remain limited. Although curriculum learning produces attention patterns closer to those expected from standard digit-wise addition, accuracy still drops sharply once the carry-chain length exceeds a certain threshold. This suggests that the learned digit-wise computation remains limited in its ability to propagate carries beyond the model's effective computational depth. Also, most of the detailed mechanistic analysis is conducted on a small encoder Transformer trained from scratch. The experiments on other architectures and large pretrained models are more limited. The results therefore support the conclusion that curriculum learning can change the learned strategy in this controlled arithmetic setting, but do not yet establish that the same approach broadly improves algorithmic generalization.

Questions for authors

1. The linear model is shown to learn a magnitude representation, but the Transformer experiments mainly show behavior consistent with an estimation-like strategy. Could the authors provide more direct evidence that the Transformer actually learns and uses a similar magnitude representation, rather than other heuristics that produce the same behavior? For example, probing or causally intervening on a candidate magnitude representation might help strengthen this connection.

2. The curriculum-trained model still fails on carry chains beyond a certain length. Could the authors clarify how its learned digit-wise computation differs from the standard carry algorithm, and to what extent its generalization is limited by model depth and training length? In particular, how does increasing model depth affect the maximum carry-chain length the model can reliably handle?

2 comments

Thank you for your feedback and questions. Please find responses below:

“Could the authors provide more direct evidence that the Transformer actually learns and uses a similar magnitude representation, rather than other heuristics that produce the same behavior?”

We have added an experiment where the magnitude is erased from the representations during training. This turns out to significantly slow training, and the model fails to converge in 1 million steps. This is in contrast to erasure of the carry values, or all digit sum information beyond the magnitude, which despite being important at convergence, do not affect learning as much when removed. Results can be seen in figure 18.

More broadly, our claim here is not that the transformer does the same thing as the linear model, rather, the linear model is meant to demonstrate a shortcut that exists in the task (the reason the magnitude is learned has little to do with the architecture, and mainly comes from the largest mode in the input-output covariance). The existence of a statistical shortcut is likely to affect other models, and we find some evidence for this early in training for the transformer, although it may exist in parallel with other strategies. Note also that a similar magnitude heuristic has been observed in Claude Haiku, in parallel with digit-wise computation (Lindsey et al. 2025, "On the Biology of a Large Language Model").

“Could the authors clarify how its learned digit-wise computation differs from the standard carry algorithm, and to what extent its generalization is limited by model depth and training length?”

The maximum observed carry-chain length increases linearly as a function of depth. The paper argued that this is because the model can only perform multiple carry operations sequentially given enough depth. Upon further investigation, this appears to still be true, but in a more nuanced way. A shallow network can learn longer carry-chains, but only if they appear in the data. Natural data rarely contains long chains, so shallow models learn only heads which can look up a carry a few places back. A deeper network, even when learning heads with limited carry lookup range, could still compute a longer carry chain, by applying the heads sequentially. Thus, the learned digit-wise computation here is similar to the standard carry algorithm, but differs in that it can only perform a limited number of operations, and so is limited by the depth of the model given a limited carry-chain length observed in data. See figure 19 for these results.

Thank you for your detailed responses and additional experiments, which have addressed my main concerns regarding the connection between the theoretical linear model and the computation learned by Transformers, as well as the limitations of carry-chain generalization.

In particular, I find the magnitude erasure experiments convincing, as they provide stronger causal evidence for the role of magnitude representations during training and strengthen the paper's contribution to mechanistic interpretability. I also appreciate the additional analysis of how model depth affects carry-chain generalization.

However, I still have some reservations about the broader applicability of these findings, as the effectiveness of the proposed approach has not been sufficiently validated on more complex tasks or at larger scales. (Although the paper examine the accuracy of different digit positions in large pretrained models, the paper do not validate the main methods and mechanisms of the paper on these models, including Curriculum Learning, Magnitude Erasure, Activation Patching, and algorithmic generalization experiments.)

Overall, I appreciate your efforts and the additional evidence provided, but I will maintain my original score of Weak Acceptance (3).

Reviewer_Zw6yOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper examines how different training orders affects what a transformer learns on addition. The authors find that an encoder trained directly on uniformly sampled addition problems learns the most significant output digits first. The authors provide a possible explanation, and then go on to explore different curricula, including staggering increased-length problems.

Critical strengths

I liked the curricula interventions to attempt to tease apart different aspects of transformer behavior.

Critical weaknesses

This paper overall seems reasonable. However, my high-level reaction is: Why are these interesting findings? Okay, sure - a different curriculum changes attention patterns, but on a toy arithmetic problem. I don't find this compelling.

Why is this an interesting research question? What insights do the results reveal on more interesting behavior? What actionable lesson should I take away to structure curricula for other non-toy tasks?

To be blunt, I don't see the contribution of this study to our knowledge of transformer training dynamics, writ large.

Questions for authors

Why is this an interesting question to explore?

2 comments

Thank you for your feedback and question. Please find our response below:

Studying toy settings in mechanistic interpretability has significant precedent. For example, Nanda et al. (ICLR 2023) used modular addition to study grokking, and Quirke & Barez (ICLR 2024) identified how a transformer adds multi-digit numbers. More related examples include Power et al. (ICLR 2021 MathAI workshop), Zhong et al. (NeurIPS 2023), and Chughtai, Chan & Nanda (ICML 2023). Why we believe toy settings are worth understanding:

  • Developing mechanistic interpretability methods is difficult as we do not fully understand how LLMs reason. Therefore, any newly developed method is hard to test, since any strange behavior it observes could both be due to an issue with the method or due to an unexpected way of reasoning from the model. To get around this circular issue, we need ground truth settings, which we can understand in detail. Here, addition lets us test whether changing only the order of training examples steers the model towards a different strategy, and whether that change helps on unseen digit combinations.
  • Competing strategies during learning and curriculum design are both directly relevant to cognitive science. Although we do not make claims here about the applicability of transformer learning to human learning, cognitive science has often taken inspiration from deep learning studies.
  • A simple setting such as addition can still represent something more general. Often findings are first seen in toy settings, before they get discovered in the realistic settings (see e.g. Olsson et al. 2022, Templeton et al. 2024, Kantamneni & Tegmark 2025). In our setting, for example, we see that the presence of a statistical shortcut shifts a transformer away from the expected digit-wise computation, and this may connect to something broader about unexpected LLM behaviors. Note also that frontier models still fail when directly applied to addition. Toy settings can tell us what to look for in more complex settings.
Reviewer_Zw6yOpenReview ↗

Rebuttal

Thanks for your explanation. I will retain my score and let the AC adjudicate.

Reviewer_B1UEOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper starts by extending to encoder-only transformers a surprising result where a transformer learns multi-digit addition from most significant to least significant digit. They then suggest a mechanistic explanation of the phenomenon by solving the fixed point of the learning dynamics of a linear toy model that encodes the direct magnitude of the sum when constrained to have one hidden neuron. This magnitude encoding appears in the presence of an overflow digit in the output. Finally, the paper shows evidence that a similar mechanism of magnitude representation happens in more realistic scenarios by looking at the error patterns and internal representation of transformers (trained by them or proprietary).

Critical strengths

The paper presents an in-depth and precise analysis of a phenomenon, from theory on a toy model that clarifies the mechanistic interpretation to the evidence at scale of this mechanism based on error, internal representation and curriculum intervention analyses. They also provide multiple secondary results in the appendix to strengthen their main text arguments.

Critical weaknesses
  1. The paper is sometimes hard to read: figure results are only briefly described in the text and there are a lot of references to the appendix. For instance:
  • “(Sections B.2 and B.3)” is ambiguous (l.189–191).
  • The seeded variant is only explained in the appendix (which carry, “binary task”).
  • Fig. 4’s grey colour bar doesn’t match its coloured cells.
  • The contrast model at l.270–272 has no figure reference.
  • The paragraph "Behavioural comparison” (l.298) is hard to read, with a lot of text not directly related to the main experiments or that cites without explanations appendix results.
  1. While the toy model is clear, the evidence of things happening similarly at scale is thin.

  2. There's no clear evidence of a causal magnitude representation in the transformer model (Fig. 5: it's only transient, and no causal intervention directly on it is done).

  3. The curriculum learning method is not very aligned with the theory; it's an unrelated trick from the theoretical perspective.

  4. As the authors mention, their methodology (find a mechanistic explanation and fix it) to deal with spurious learning does not scale, as it needs to be done on a case-by-case basis.

Questions for authors

How robust is the result on the linear model? For instance, to multiple hidden neurons?

Could you explain: “although bounded by depth” (l.301–303)?

Does the curriculum also work with a variable number of digits and without zero padding (l.360–361)?

How is your curriculum learning method related to the overflow "hiding" in the mechanism before?

Is the result on Claude HAIKU also reproducible from other models?

3 comments

Thank you for your feedback and questions. Please find responses below:

"How robust is the result on the linear model? For instance, to multiple hidden neurons?"

Because E[yx⊤]\mathbb{E}[yx^{\top}] is rank one, there is only one nonzero singular value. Its input singular vector encodes the magnitude of the sum, and the wider linear model learns this same direction. We have added a paragraph (bottom of appendix B.4) showing this more formally.

"Could you explain: “although bounded by depth” (l.301–303)?"

Yes, we have added figure 19 to show how far a carry can propagate in the curriculum-trained models. The two-layer model reliably solves carry chains only up to the lengths seen during training. Deeper models solve longer carry chains than they saw in training, even when no single head attends all the way back to the digit that initiated the carry. This result is consistent with successive layers passing the carry across several shorter spans, but this would then limit the carry-chain length by the depth of the model and carry length seen in data.

"Does the curriculum also work with a variable number of digits and without zero padding (l.360–361)?"

Yes, we added the test suggested by the reviewer and it still learns the digit-wise strategy without zero padding, although training takes longer, as attention has to be remapped when switching to the next stage. Results have been added to the paper (figure 20).

"How is your curriculum learning method related to the overflow "hiding" in the mechanism before?"

It is motivated by the observation that reversing the digit order moves the learned strategy towards digit-wise computation, which was established using the masked-overflow task. It is not intended as an argument for the theory, rather as a demonstration that the learned strategy here can be shifted solely by changing the training order, without editing the task. Prior work on curriculum learning has showed the benefits to training time, but results on how it may change the learned solution are lacking beyond toy synthetic models like parity and Gaussian mixture models.

"Is the result on Claude HAIKU also reproducible from other models?"

Yes, we've tested on 5 additional models and they are all disproportionately more accurate on the significant digits, although Claude Opus is also somewhat accurate on the least significant digit (see figure 27).

Based on the feedback we have also made some small changes to improve the paper's clarity: we now define the seeded target as the carry out of the ones column, point to Fig. 4c and d for the no-overflow model, and explain why accuracy on long carry chains drops sharply for that model and gradually for the directly trained model. Additionally, we have added a direct causal test for the magnitude representation (figure 18).

Thank you for your precise and succinct answers. My points were answered directly and with convincing additional experiments, especially on how depth, together with the carry lengths seen in training, constrains the development of digit-wise computation (Fig. 19), and the test causally affecting the magnitude representation (Fig. 18).

I am raising my soundness score from 2 to 3 and my rating from 3 to 4. Despite the acknowledged limits on the scope of the methodology (mentioned in my CW 5), the analysis is interesting and thorough. I also want to note that this is the best rebuttal I have seen in terms of the quality and conciseness of the answers.

I have still one question: how do you reconcile describing the converged, directly trained model as estimation-like with Fig. 18b, where erasing the carries after training drops accuracy to about 0 while erasing the magnitude leaves it at the level of the baseline "nothing"?

In figure 18b we remove information about the carry at every output position and layer. Since each output digit is the sum of the respective input values plus the carry, any correct solution needs the carry information, so removing it will disrupt the answer, no matter how the carry is computed.

The wording here is a bit confusing. We do this as a control to show that the effects of erasing the magnitude are not simply due to removing essential information, but 'carry' here is not directly related to digit-wise carry computation. An estimation-like model still needs the carry by the final layer, but could obtain it indirectly, for example by guessing it from nearby digits, consistent with what we saw from the misleading problems (figure 17).

On the other hand, the magnitude, although it determines the overflow digit, says exponentially less about each lower digit (if you know the magnitude up to 99% accuracy you actually know very little about most digits). Note that at the overflow position we removed only the part of the magnitude beyond the overflow digit, so that the overflow information isn't erased. Therefore the finished model doesn't need this magnitude information at convergence, but the surprising observation here is that it fails to learn without it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.