acceptodds
Under review as a conference paper at ICLR 2027

Prime by Prime: A Reference for Emergent Modularity

Abstract

Neural networks can break a task into subtasks and learn to solve each independently, in a different part of the model. But what determines whether this modular organization emerges during learning remains poorly understood. One obstacle is that the relevant subtasks and their internal representations are often unknown. We address this by studying transformers trained to compute and , where is a product of distinct primes. These tasks admit a simple decomposition: represent the inputs through their remainders and , perform arithmetic separately for each prime, and reconstruct the unique answer modulo consistent with the results, as guaranteed by the Chinese remainder theorem. We ask whether transformers discover this decomposition through training. We find that models trained jointly on addition and multiplication (i.e., learning the ring ) consistently do so, with evidence at three levels: (i) errors occur approximately independently across primes, (ii) each prime’s computation occupies its own representational subspace, and (iii) these computations largely rely on distinct groups of neurons. This modular organization critically depends on the training task: addition alone produces entangled or partially modular solutions that vary across seeds, including fused subtasks and asymmetric dependencies between primes. Why training on the ring induces this modular solution, while addition alone does not, remains an open theoretical question and an interesting direction for future work.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, modulo arithmetic, grokking, modularity, entanglement

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_hFE5OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies when neural networks learn modular solutions, using modular arithmetic mod n = p₁⋯pₖ as a testbed where the Chinese remainder theorem (CRT) provides a known subtask decomposition (compute mod each prime independently). The authors define modularity at three levels — behavioural (per-prime errors are independent), representational (each prime occupies its own subspace), and neural (each prime uses its own neurons) — and give ablation-based tests for each. One-layer transformers trained jointly on addition and multiplication (the ring Zₙ) consistently satisfy all three definitions across 20 seeds. Models trained on addition alone do not: they are either fully entangled or partially modular (9/20), exhibiting fused subtasks and asymmetric dependencies, which the authors visualize as subtask dependency graphs. An appendix shows that training on the multiplicative unit group alone does not yield modularity, suggesting the effect depends on the interaction of the two operations. Why the ring induces modularity remains open.

Critical strengths
  • Both the subtask decomposition and the relevant subspaces (Fourier features) are known in advance, so modularity can be tested directly rather than discovered, avoiding search bias.
  • Results are averaged over 20 seeds, replicated on several moduli, and checked on an MLP. The 0/20 fully-modular outcome for addition-only training is a strong negative result.
  • Adding multiplication reliably turns modularity on. This is a useful foothold for future mechanistic work on why modularity emerges.
Critical weaknesses
  • No mechanistic explanation. The title is honest about the descriptive scope, but the central question — why the ring is modular and addition is not — is addressed only by one elimination experiment (Appendix D).
  • Toy scale. All models are one-layer, d=256. Implications for deeper models or real-world data are speculative.
  • Evidence is output-side only. There is no analysis of how the input embeddings expose a mod p, so the claim that the model “implements the CRT” rests mainly on the readout and residual-stream ablations.
Questions for authors
  • How do the embeddings represent a mod p in the ring? Does per-prime structure already appear at the embedding layer, or only after attention/MLP?
  • Neural modularity is noticeably weaker for multiplication (kill mod 7 → 0.37 vs. 0.27 information-free). Is this due to the argmax neuron assignment, or do multiplication neurons genuinely mix primes?
6 comments

Thank you very much for the review, we appreciate that you think this may be a useful foothold on why modularity emerges.

On a mechanistic explanation

We agree that including an answer to why the ring is modular would strengthen the paper. Here is the argument, which we have added as a section to the paper:

  1. Multiplication must learn pure-prime frequencies. Given a modulus nn, and a prime divisor pp, the only Dirichlet characters (multiplicative characters) which are non-constant on x=0mod  n/px = 0 \mod n/p are the pure mod pp characters. To distinguish between these answers, the model must therefore learn some pure characters for each prime divisor.
  2. The pure additive and pure multiplicative subspaces are the same. With (1) this implies addition must learn pure frequencies too.
  3. Forcing the model to use a small amount of the pure subspaces for addition can make the entire model modular. We find that when training a model on modular addition, if you require a small proportion of the embeddings and unembeddings to stay in the pure subspaces, the model becomes fully modular.

With some experiments showing (3) to hold in the paper. The idea broadly comes down to the fact that both operations share an embedding and unembedding layer. By this we mean that given some amod  na \mod n, there is an embedding and unembedding for that amod  na \mod n, which is the same whether aa appears in an addition equation or a multiplication one. But addition and multiplication have different structures, because addition uses additive Fourier frequencies for computations, and multiplication uses multiplicative characters. This "tying" together of the two operations in this manner causes them to influence each other in the way described above.

How do input embeddings represent tokens?

This is a very interesting question. Doing the same technique as before (taking the Fourier transform over the embeddings, and checking what proportion of each vector is in the subspaces spanned by the pure-prime frequencies), we can get the following, averaged over 20 seeds:

Table 1. Share of the input embedding's energy on pure, two-prime and joint Fourier frequencies, ring mod 10011001. Mean ±\pm standard deviation over 2020 seeds at step 80,00080{,}000 (all at 100%100\% accuracy on both operations). The last row is the share for a matrix with no structure, such as the random initialization.

PureTwo-primeJoint
Embedding0.84±0.010.84 \pm 0.010.05±0.000.05 \pm 0.000.11±0.010.11 \pm 0.01
Unstructured0.030.030.250.250.720.72

Here we can see that the model uses the pure-frequency subspaces at the embedding layer, so the modular structure is present at input embeddings as well.

One surprising thing is that if we train an MLP with one-hot (unlearned) embeddings, it will still be modular, for example in here:

Table 2. Pure share of the residual stream for attention-free MLPs (n=1001n = 1001), mean over seeds. Width is 10241024, training fraction 0.50.5, weight decay 0.10.1, learning rate 0.0010.001.

Model++×\times
1 hidden layer, learned embedding (2 seeds)1.001.00
1 hidden layer, one-hot inputs (2 seeds)1.001.00
2 hidden layers, learned embedding (5 seeds)1.000.99

On multiplication being weaker

This question is also very interesting. The evidence points to multiplication genuinely being less modular. To test this, we can change how we check if an answer is correct mod  p\mod p by summing over the logits for a given output class mod  p\mod p instead of taking the argmax. This checks if the model is hiding anything when it takes the argmax. We find that the ablation results do not change meaningfully from the paper when this happens (if anything the results are slightly worse):

Table 3. Kill/keep ablations for multiplication in the ring mod 10011001, scored by the marginal over each prime: for each residue class we sum the model's probabilities over the answers in that class and take the largest. Mean over 2020 seeds at step 80,00080{,}000; baseline accuracy is 1.001.00. Rows are the ablation, columns are held-out accuracy mod each prime; the ablated prime is in bold. Chance: 0.270.27, 0.170.17, 0.150.15 (majority residue) and 0.140.14, 0.090.09, 0.080.08 (uniform).

Readout: mod 7Readout: mod 11Readout: mod 13Residual: mod 7Residual: mod 11Residual: mod 13
Kill 70.141.001.000.231.001.00
Kill 111.000.091.001.000.151.00
Kill 131.001.000.081.001.000.11
Keep 71.000.240.201.000.150.13
Keep 110.291.000.150.281.000.13
Keep 130.270.171.000.310.161.00

On toy scale and model size

Thank you very much for pointing this out, we have added a much more thorough analysis in the appendix, which confirms that our claims hold across a variety of widths, depths, and training parameters.

Table 4. Pure share of the residual stream for different widths and depths (n=1001n = 1001).

Model++×\times
1 block, width 256 (appears in paper)1.001.00
1 block, width 5121.000.99
1 block, width 10241.001.00
2 blocks, width 2561.000.99
4 blocks, width 2560.950.95

And then we have the same measurement for MLPs:

Table 5. Pure share of the residual stream for attention-free MLPs (n=1001n = 1001), mean over seeds. Width is 10241024, training fraction 0.50.5, weight decay 0.10.1, learning rate 0.0010.001.

Model++×\times
1 hidden layer, learned embedding (2 seeds)1.001.00
1 hidden layer, one-hot inputs (2 seeds)1.001.00
2 hidden layers, learned embedding (5 seeds)1.000.99

And we can test different hyperparameters and learning rates as well:

Table 6. Pure share of residual stream across different learning rates and weight decay, modulus n=1001n = 1001. Only runs reaching >0.9>0.9 accuracy included.

Addition (++)

lr \ wd00.010.10.51.0
10−410^{-4}–1.00–0.941.00
3⋅10−43\cdot 10^{-4}1.001.001.001.00–
10−310^{-3}1.001.001.001.00–
3⋅10−33\cdot 10^{-3}–1.001.00––

Multiplication (×\times)

lr \ wd00.010.10.51.0
10−410^{-4}–0.94–0.960.96
3⋅10−43\cdot 10^{-4}0.990.970.990.97–
10−310^{-3}1.000.991.000.98–
3⋅10−33\cdot 10^{-3}–0.981.00––

Initially we responded on the argmax assignment of the probabilities, rather than the argmax of the neuron selectivity scores. Here, we answer the neuron assignment question, which is now in the document as Appendix J.

Concerning the argmax of the neuron selectivity scores, it seems that the gap in modularity between addition and multiplication comes from the answers which are 0mod  p0 \mod p for some prime divisor pp (the zero divisors). Specifically, products which are 00 for some prime are computed outside of the MLP, so ablating MLP neurons does not degrade those answers, and then it does not degrade those primes to chance.

First, we can check if the argmax assignment is causing issues by building our neurons sets differently. Instead of saying a neuron jj is in the set NpN_p using the argmax, we can assign it to NpN_p if it is highly selective to the prime pp, which we define as having a selectivity score above 0.80.8 (selectivity for class kk is defined in the paper as sk=ηk2/∑iηi2s_k = \eta_k^2/\sum_i\eta_i^2, the sum over all the classes, and ηi2\eta_i^2 the between class variance over the total variance, so sks_k close to 11 means that neuron is tuned specifically to that class alone).

When we do this, we find that the neuron ablations do not change meaningfully across the 2020 seeds, so argmax doesn't seem to be what is causing the lower modularity. However, if we only do the mod  p\mod p accuracy tests on answers which are nonzero mod  p\mod p, we find that the model is almost perfectly modular:

Multiplication: accuracy mod pp after ablating neurons (mean over 20 seeds)

All pairs

Ablatedmod 7mod 11mod 13
NpN_p, argmax0.380.220.19
NpN_p, share >0.8>0.80.390.230.20
All neurons0.350.220.18
Chance0.270.170.15

Answer ≢0(modp)\not\equiv 0 \pmod p

Ablatedmod 7mod 11mod 13
NpN_p, argmax0.220.120.11
NpN_p, share >0.8>0.80.220.120.10
All neurons0.200.120.11
Chance0.170.110.09

The full table is here (units mean only answers nonzero mod every prime):

Multiplication: neuron set ablations on all pairs and on pairs whose answer is a unit (mean over 20 seeds; chance on units: 0.17, 0.10, 0.08)

kill NpN_p

Pairsppmod 7mod 11mod 13
All70.381.001.00
All111.000.221.00
All131.001.000.19
Units70.221.001.00
Units111.000.121.00
Units131.001.000.10

keep NpN_p

Pairsppmod 7mod 11mod 13
All71.000.210.17
All110.361.000.18
All130.360.221.00
Units71.000.120.11
Units110.211.000.10
Units130.210.111.00

Next, if we set every MLP neuron to its mean value, we find that naturally the model outputs at chance, but shockingly not on the zero divisors, where it stays unscathed, so the zero divisors are being calculated somewhere in the attention heads or at the input layer:

Every MLP neuron ablated (set to mean): accuracy mod pp by whether the answer is 00 mod pp (mean over 20 seeds)

Op.Answermod 7mod 11mod 13
×\times≡0\equiv 00.950.930.88
×\times≢0\not\equiv 00.210.110.11
++≡0\equiv 00.150.140.09
++≢0\not\equiv 00.140.100.09
Chance≢0\not\equiv 00.170.100.09

So these products are not being computed in the MLP at all.

As another interesting experiment, we can ask if the neurons which are assigned to a specific prime pp under addition are the same as those assigned to pp under multiplication, or if a neuron can be selective for, say, 77 under addition, but 1111 under multiplication. We find that there is almost no crossover though, on highly selective neurons.

Neurons by selective prime (share >0.8>0.8) under ++ (rows) and ×\times (columns) (mean count over 20 seeds)

×\times: 7×\times: 11×\times: 13×\times: none
++: 7137003
++: 11024103
++: 130033034
++: none003273
Reviewer_ghLdOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper reports a number of interesting bservations regarding modularity in learning of a decomposable task with transformers. They use single layer transformers to learn two operations mod nn: $ a+b$ and a×ba \times b. They pick n=p1p2....pkn = p_1 p_2 .... p_k where pip_i are distinct primes. They show that models trained jointly on product and addition show "modularity", in various ways. They find distinctive steps and plateaus in the validation loss and show it corresponds to the model learning mod pip_i. They do not observe such plateaus when learning addition alone. They also find that the residual stream of the jointly trained model shows distinct clusters in PCA, where numbers that have the same remainder mod each pip_i cluster together. They do ablations by projecting onto or away from subsaces in the residual and readout identified as encoding each of the primes and find that this indeed collapses the model accuracy from 100% down to random levels.

Critical strengths
  1. Overall, I really like this kind of interpretability work. It reveals how learning is happening in models.
  2. The ablations clearly show the modularity.
  3. The studies showing the emergence of learning of each prime in sequence, small to large, is quite interesting.
Critical weaknesses
  1. All experiments use n=1001=7x11x13
  2. While I think this effect generalizes, no other example of decomposable task and synergy is studied here.
  3. The model size was fixed. The effect may be size dependent.
Questions for authors
  1. You show addition alone training. How does the product alone look? Same random distribution for residual PCA?
  2. Single layer transformers cannot learn certain tasks, as in Sanford, Hsu & Telgarsky (2023), “Representational Strengths and Limitations of Transformers” (communication-complexity show a single attention layer needs very large width on tasks that require composing information across positions). This may not be relevant to your two element input, but I wonder if it affects learnability.
  3. Did you do model size scaling experiments? We know certain abilities emerge at certain sizes.
5 comments

Thank you very much for your review, we sincerely appreciate it.

On multiplication

We find that the model is not fully modular when trained on just multiplication, but is "partially modular." For example, when trained on multiplication, the ablation results give the following:

Table 1. Kill/keep ablations for multiplication alone mod 10011001 (three seeds, step 150k). Entries are held-out accuracy mod 7, 11, 13. Chance (majority): 0.26–0.27, 0.17–0.18, 0.14–0.15. Unlike the ring, the ablations are not modular.

SeedSiteppKill 7Kill 11Kill 13Keep 7Keep 11Keep 13
0Readout70.260.980.971.000.530.46
110.840.080.290.840.980.98
131.000.750.130.950.970.95
Residual70.880.990.960.830.180.12
111.000.900.980.300.500.24
130.990.980.850.320.190.24
1Readout70.260.970.790.790.770.51
110.910.180.690.901.000.98
130.980.930.150.490.940.98
Residual70.820.990.980.810.180.16
111.000.890.990.300.500.16
131.000.980.890.260.190.32
2Readout70.241.001.000.980.920.97
111.000.171.000.581.000.54
130.861.000.141.000.831.00
Residual70.931.001.000.530.180.16
111.000.801.000.250.870.14
130.980.980.930.270.210.82

The PCAs of residual stream class averages are below, and because we find this so interesting (thank you very much for the question!) we have added it to Appendix E in the paper.

In the PCAs the clusters are much less tight than in the ring, hence the weaker modularity. However, they are more geometrically structured than in the addition alone case. In general, the geometry of the residual stream correlates to which multiplicative features the model learns. For example, you can see that quadratic residues and quadratic non-residues both form clusters, which would be a result of the model learning the Legendre character, which separates the two sets.

On different nn

In Table 3 in the paper, we test n=143=11⋅13n = 143 = 11\cdot13, 221=13⋅17221 = 13\cdot 17, and 595=5⋅7⋅17595 = 5 \cdot 7 \cdot 17, and find that transformers are consistently modular at each.

Model size

We have done some experiments on different widths and depths and included them in Appendix G. We find that models are consistently modular (again, this is the proportion of residual stream in the pure subspaces, so near 1 means fully modular):

Table 2. Pure share of the residual stream for different widths and depths (n=1001n = 1001).

Model++×\times
1 block, width 256 (appears in paper)1.001.00
1 block, width 5121.000.99
1 block, width 10241.001.00
2 blocks, width 2561.000.99
4 blocks, width 2560.950.95

In Appendix G, we also have included further robustness tests, where we show that the results hold across different hyperparameters, and architectures.

On generalizability

In the appendix, we have also added tests of some finite polynomial rings, which also factor so we can test modularity, and find that they are consistently modular as well. To summarize the results, the table below includes the proportion of the residual stream in the pure-factor components, so close to 1 means fully modular, but the appendix includes ablation results as well:

Table 3. Residual stream fraction on pure CRT components. Each ring has 625625 elements and the same additive group (Z/5)4(\mathbb{Z}/5)^4. Training fraction 0.50.5, weight decay 0.10.1, learning rate 0.0010.001, seed 00. One layer transformer.

RingCRT decomposition++×\times
F5[x]/(x4−x)\mathbb{F}_5[x]/(x^4 - x)F5×F5×F25\mathbb{F}_5 \times \mathbb{F}_5 \times \mathbb{F}_{25}1.000.99
F5[x]/(x4−x2)\mathbb{F}_5[x]/(x^4 - x^2)F5×F5×F5[x]/(x2)\mathbb{F}_5 \times \mathbb{F}_5 \times \mathbb{F}_5[x]/(x^2)1.000.99
F5[x]/(x4−1)\mathbb{F}_5[x]/(x^4 - 1)F5 4\mathbb{F}_5^{\,4}1.000.97

On model scaling

Thank you very much for this question. To test this, we trained 12 MLPs at 6 different widths (4, 8, 16, 32, 64, 128), each at 2 depths (1, 2). Again learning the ring mod 1001. This confirms that model capabilities emerge at certain sizes.

At width 4 and 8, nothing was learned for addition (8 learned mod 11 multiplication). At width 16 and 32, mod 7 was learned. 32 with depth 2 learned mod 7 and mod 11. And width 64 and 128 learned all three primes. The figure can be found in Appendix H.

Question 2

Because our inputs have fixed length of basically two inputs (there's also an operation and = sign, but that doesn't matter here), this is the kind of problem that the paper describes as being very learnable for a single layer transformer, so we think you are correct that perhaps it doesn't apply.

It would be interesting to see if there is a connection between modularity and the learnability of summing larger sequences though.

Dear Reviewer,

As the discussion period draws to a close, we would once again like to thank you for your thoughtful review, and valuable comments on our paper. We would be grateful if you could let us know whether any questions remain open. If you believe our revisions have addressed your concerns, we would also appreciate you considering an update to your score.

Reviewer_3NxVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper uses three level of analysis to show that neural networks can exploit the hidden modular structure from tasks (in this case modular arithmetic). And that such modular structure only emerges when the models are trained jointly on addition and multiplication. The paper provides a clean testbed for studying emergent modularity, but with limited mechanistic explanation on how such modularity is emerged.

Critical strengths
  1. The analysis is clean and comprehensive. I like that you have three levels of modularity assessment to independently examine emergent modularity.
  2. Writing is good. I can understand what you are trying to say, unlike many other AI-generated bag-of-words.
Critical weaknesses
  1. The scope of the study is quite limited. Your study is solid, but focuses on the specific decomposition from CRT. My worry is that your observation (i.e., addition with multiplication bring modularity) is a special case in the CRT setting.
  2. Lack of mechanistic explanation of why multiplication + addition are needed for emergent modularity. This goes back to my first point. Since you don't have a wide range of tasks to empirically demonstrate how common your observations are, a theory or validated hypothesis of why multiplication brings modularity would also be interesting. But unfortunately it's not provided, making me questioning how generalizable your observations/conclusions are. Since the emergent modularity has been observed/studied under many different settings, a clean testbed based on CRT in itself seems to be a limited contribution.
Questions for authors
  1. In sec 3, why that the primes are acquired in that specific 7, 11, 13 ordering?
  2. Have you considered tasks with hierarchical structures? There is an active line of work about compositional sparsity, which might be good to look into.
  3. Have you observed any task performance-related gain when modularity emerges? For instance, would the network requires significantly fewer samples to train?
5 comments

Thank you very much for your review and helpful feedback.

On a mechanistic explanation

We agree that including an answer to why the ring is modular would strengthen the paper. Here is the argument, which we have added as a section to the paper:

  1. Multiplication must learn pure-prime characters. Given a modulus nn, and a prime divisor pp, the only Dirichlet characters (multiplicative characters) which are non-constant on the set of all x=0mod  n/px = 0 \mod n/p are the pure mod pp characters. To distinguish between these, the model must therefore learn some pure characters for each prime divisor.
  2. The pure additive and pure multiplicative subspaces are the same in embedding and unembedding space. With (1) this implies addition must learn pure characters too, because addition and multiplication share embedding and unembedding vectors.
  3. Forcing the model to use a small amount of the pure subspaces for addition can make the entire model modular. We find that when training a model on modular addition, if you require a small proportion of the embeddings and unembeddings to stay in the pure subspaces, the model becomes fully modular.

Some experiments showing (3) are now included in the paper, in particular that if you force an addition only model to keep just 1/501/50th of its norm in the pure subspaces in its unembedding and embedding matrices, the model will keep a fraction of 0.880.88 of its residual stream in the pure subspaces. In other words there is a kind of snowballing effect, where slightly nudging the pure subspaces can force the whole model into them.

On performance related gains

First, we find that learning on the ring occurs much faster than addition alone, and with less training data than addition typically requires.

As an example, we can train one model on addition and one on the ring (so, both addition and multiplication) with a train test split of 0.1-0.9. This is much smaller than is seen in the grokking literature, which is typically at least 0.5 train. Because the ring sees twice as many examples, we will show the addition model the examples twice (so basically 0.2 train). Across three seeds, we find that the ring consistently learns quite quickly, while addition doesn't at all:

Table 1. Mod 10011001, trained on 10%10\% of the pairs for 4040k steps. Mean over 33 seeds at the final checkpoint. The ring reaches 100%100\% accuracy on both operations in every seed, by step 1010k–1515k.

TrainTest
Addition alone0.890.07
Ring, addition examples1.001.00
Ring, multiplication examples1.001.00

One likely reason is that we know the ring is modular, and learning mod 7 alone is a much easier task than learning mod 1001, and so it can pick that up with relative ease. Learning mod 11 and mod 13 are then quite simple as well.

To see this a bit clearer, we can pick a modulus which would be too large for a model of this size, say n=15015=3⋅5⋅7⋅11⋅13n = 15015 = 3\cdot5\cdot7\cdot11\cdot13. Despite the overall task being too large, because the ring is fully modular it can solve some of the subtasks (in this case learning 3, 5, and 7 in order), so achieves a much lower loss than the addition alone model. The figure can be found in Appendix H.

Why the 7, 11, 13 ordering?

We believe there are two reasons, both quite simple. First, the model uses Fourier features to compute its answers, and it requires fewer mod 7 features to achieve large margins mod 7 than mod 11, 13, etc.

The second reason is that the model sees more training examples mod 7. For n=1001n = 1001 with a training fraction of 0.5, the addition training set has 0.5×100120.5 \times 1001^2 training samples. Meanwhile, there are 727^2 different mod 7 examples, so the model sees each mod 7 example 0.5×10012/72≈10,0000.5 \times 1001^2/7^2 \approx 10{,}000 times. It sees each mod 11 and mod 13 example about 4140 and 3070 times respectively, so it sees each mod 7 example over 3 times more than the mod 13 examples.

On hierarchical structures

Thank you very much for this question. Compositional sparsity seems to be very related to how our model breaks the task into factors which are solved independently, analogous to the idea of compositionally sparse functions for example in [1].

Regarding hierarchy, we do think of this task as being related to hierarchical structures, in the way that each prime is a subtask of the larger task, and we were particularly thinking of this when working on the partial modularity section.

To be more specific, the modularity ablations in this paper can be thought of as an extremely clean case of unlearning. Our definition of modularity between two tasks is basically that you can unlearn one without any collateral damage to the other, and vice versa. The dependency graphs in our partial modularity are meant to reflect when there is a hierarchy of tasks, in which one task depends on the other, or that a model's computation of one task is downstream of another for some reason, so that there is asymmetric collateral damage during unlearning.

[1] Danhofer et al. "Position: A Theory of Deep Learning Must Include Compositional Sparsity"

On limited scope

It is true that the CRT setting is important for the results, but modular arithmetic has been studied quite extensively in interpretability, so we felt that it was a topic which would be of significant interest.

As a slightly different example to show modularity though, we can consider three finite polynomial rings which have a similar decomposition, and find that the model finds this decomposition again:

Table 2. Residual stream fraction on pure CRT components. Each ring has 625625 elements and the same additive group (Z/5)4(\mathbb{Z}/5)^4. Training fraction 0.50.5, weight decay 0.10.1, learning rate 0.0010.001, seed 00. One layer transformer.

RingCRT decomposition++×\times
F5[x]/(x4−x)\mathbb{F}_5[x]/(x^4 - x)F5×F5×F25\mathbb{F}_5 \times \mathbb{F}_5 \times \mathbb{F}_{25}1.000.99
F5[x]/(x4−x2)\mathbb{F}_5[x]/(x^4 - x^2)F5×F5×F5[x]/(x2)\mathbb{F}_5 \times \mathbb{F}_5 \times \mathbb{F}_5[x]/(x^2)1.000.99
F5[x]/(x4−1)\mathbb{F}_5[x]/(x^4 - 1)F5 4\mathbb{F}_5^{\,4}1.000.97

Other than the CRT, we can train a model to do addition and multiplication mod pp for pp prime. Now there is no factorization, but we can ask whether the model solves the two tasks, addition and multiplication, in a modular way. For example, taking p=197p = 197, we find this to be true: ablating the addition subspace the model gets multiplication answers correct with addition wrong, and vice versa:

Table 3. Kill/keep of the addition and multiplication subspaces of the residual stream; held-out accuracy on each operation.

n=1001n = 1001: ++n=1001n = 1001: ×\timesp=197p = 197: ++p=197p = 197: ×\times
Baseline1.001.001.001.00
Kill ++0.000.010.011.00
Keep ++1.001.001.000.05
Kill ×\times0.000.010.990.02
Keep ×\times1.001.000.061.00

However, this is not true for 1001. The reason for this is that the angle between the mod pp addition and mod pp multiplication subspaces has cosine ≈1/p−1\approx 1/\sqrt{p-1}, because if χ\chi is a multiplicative mod pp character, then ∣χ^∣=1/p−1|\hat{\chi}| = 1/\sqrt{p-1}, so for small primes, this causes a lot of interference and addition and multiplication are non-modular.

Dear Reviewer,

We have updated the paper again. It now includes a discussion of compositional sparsity in the related works section.

As we approach the end of the rebuttal period, we would once again like to thank you for your thoughtful engagement with our paper, and for your comments which have helped us improve it. We would be grateful to know if our revisions have addressed your concerns, and for you to reconsider your score if so.

Reviewer_EujWOpenReview ↗
Rating
4(+2)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper trains 1 hidden layer transformers on a+b mod n and a x b mod n for the same n. They find that when training on the ring (both multiplication and addition are in the dataset simultaneously) a strict Chinese remainder theorem is learned with the correct prime factors, compared to the case of prior work that found training on just a+b mod n results in an approximate Chinese remainder theorem, where networks approximate cosets by 'pretending' numbers factorize the modulus that don't. The paper inspects what is learned at different scales via different definitions of modularity, and also measure how disjoint the circuits being learned are.

Critical strengths
  1. Overall I like the work and think the main result is shocking and impactful: that 1-layer transformers learn the correct prime factors of the CRT when trained on both addition and multiplication simultaneously, compared to when shown either addition xor multiplication (the unit group) alone as the dataset. This is verified at different "scales" of interpretation e.g. at both neural and representational levels of view, which aligns with how recent interpretability case studies on math tasks, including modular addition, have been carried out.

  2. The fact networks suddenly learn a precise Chinese remainder theorem utilizing the correct prime factors is not just surprising, but amazes me. It opens multiple questions about why and how gradient descent is pulling this off, and also opens questions about why this isn't the case when training on groups including modular addition (not rings). This actually leads me to flow into the critical weaknesses of this paper: it fails to contextualize and explain this result in terms of the broader literature that reverse engineers and interprets networks learning math tasks. Consequentially, it fails to explain how different this result is compared to the broader literature, and why the research is significant and important. As the paper currently reads, it appears to be just another network reverse engineered on an arbitrarily chosen math task, but I believe this can be easily fixed during rebuttals and instead become a major strength of the paper, which is why I am writing this up here in strengths.

Critical weaknesses

I want to make it clear that I really like the result in this paper, but on a few fronts, I feel like the paper falls significantly and critically short, despite the main result itself being what I consider amazing. This said, I believe it would be easy to fix this over the rebuttals period with moderate work.

To begin: I am not fully convinced that the main result is robust and not the result of magic training conditions, because it is so surprising to me, that I want to see that it is robust under different training settings and architectural settings (hyperparameters, widths, does it need attention or does it happen with 1-layer MLPs with embedding matrices too, different train test splits, etc?). To be clear and concise: I am not convinced it isn't an artifact of an optimal training setup or optimal hyperparameters.

  1. Thus, I would like to see additional quantitative results that prove the claims are robust over different hyperparameters and architectural choices. For example, multiple heatmaps using the scoring definitions they use in the paper, showing the effect of different learning rates and weight decays on various measures of modularity, and the per-prime accuracies, etc, instead of just one learning rate and one weight decay. The reason I believe this is important is that I want to believe this result is robust over a large region of training parameters and setups, and not just the result of magic hyperparameters / architectural choices. I would also like to see how changing widths and training on higher train/test splits, e.g. both 80%/20% and 95%/5% affects their result.

Secondly, a major flaw with this paper is that their narrative fails to contextualize why the result matters and why the reader should care about the paper. While they cite the first paper showing that networks learn an approximate Chinese Remainder Theorem on modular addition, they then fail to cite at least four papers (that I know about, there could be more) that have found networks to learn the Chinese Remainder Theorem or algorithms abstractly equivalent to it. This includes a paper with a different approach compared with this paper for interpreting modularity in modular addition (which is a large part of their paper). While this is a flaw because these papers should be cited as related work, their absence prevents the authors from being able to compare and contrast their main result with what's currently known, and resultantly, (2) they can't emphasize the magnitude of how shocking their main result is, and (3) also can't justify proposing open problems to resolve it. I break these down into 2 and 3 below.

  1. This paper fails to contextualize its importance and contrast the main result against existing papers finding networks learning the Chinese Remainder Theorem to solve various group tasks in the literature. I've taken the time to write a summary below, because I think it is easy to add this commentary to the paper.

To summarize, multiple related works exists that have found networks (MLPs and transformers alike) learn the Chinese Remainder theorem to fit group tasks. Interestingly, this has been found on a diverse array of group tasks with some of those groups not being cyclic or abelian groups (which means that there is no "modulus" and thus makes finding a Chinese Remainder Theorem-like algorithm surprising due to their being no prime factors). The commonality across the group tasks studied in the literature is that in every case study on networks learning a group task, the individual neurons learn cosets, and the neural representations in the network (manifolds composed by many collaborating neurons) have been found to be mostly disjoint Cayley graphs (with their vertices being cosets). Furthermore, the networks are shown to have learned to intersect these neural representations in a way that is abstractly equivalent to the CRT; by dividing the inputs into simpler subtasks (cosets) and then conquering the computation by linearly superimposing the results (merging information from coset membership) at the logits. Within the network, these subtasks are mostly disjoint, but can be partially disjoint depending on training hyperparameters and architectual choices, (which agrees with this papers finding on modular addition in their partial modularity section). On all these group tasks, the Cayley graphs that are learned are random across random seeds, which means networks are not learning the correct factors when trained on e.g. modular addition (a+b) mod n. This is in bold because it is a major difference from the main result in this paper under review.

The summary above means a Chinese remainder theorem is learned when networks are fitting a group task, but it is an "approximate one" because the correct prime factors are not necessarily learned (as shown by [1, 2] on the cyclic group multiplication i.e. (a+b) mod n). In networks learning higher dimensional abelian groups, (homocyclic groups and elementary p groups, which are generalizations of modular addition to higher dimensions) cosets were shown to always be learned, but again they were random across seeds, and intersected within networks via the "multidimensional Chinese remainder theorem" [3]. In networks learning the dihedral group, which is a non-abelian group, but is composed of two cyclic groups (modular addition), networks were shown to have a bias toward learning cosets (the prime factors of those cyclic groups), but again, they didn't consistently learn them and learned random Cayley graphs and reported the graphs were intersected within the model as a Chinese Remainder Theorem-like algorithm would do it [4]. Finally, in alternating groups, which don't have prime factors or an isomorphism to a polynomial mod n, the same story was yet again found: neurons learn random cosets that correspond to manifolds (the neural representations) which are Cayley graphs, and the overall algorithm learned by the network is abstractly equivalent to the Chinese Remainder theorem (look up coset membership, intersect, and compute the answer via divide and conquer) [5].

Why is this important: all of these prior works show networks learn an algorithm that is approximately the Chinese Remainder Theorem, or, in the cases without primes, make use of the exact idea the Chinese Remainder Theorem uses (coset intersection), but also in all of this work, the Chinese Remainder Theorem always operates on cosets that are randomly learned by stochastic gradient descent. In all of these cases, networks aren't learning cosets corresponding to the correct prime factors, but instead random factors that may not even divide the modulus. Thus, in all of this prior work, it can be said that networks learn a "sloppy" approximation of the Chinese Remainder Theorem. And this makes sense: afterall, why would stochastic gradient descent learn the correct prime factors and thus the correct prime factors? Especially when e.g. [1] gave a proof that the margins can be made to be of linear size by choosing O(log(n)) "random" factors (and random factors obviously often have gcd(random_factor, n) = 1? My point is, this is a correct local minima.

Your work claims that the prime factors are all learned every seed deterministically, not randomly. Obviously, that is a discovery that is different to all of the prior work I summarized above that found Chinese remainder theorem based algorithms were learned to fit group datasets. Why is your result is mindblowing to me, if it occurs over a large array of hyperparameters and different architectural choices and not just a small 'perfect' choice: simply put, stochastic gradient descent is finding the correct prime factors. This feels unreasonably powerful to me. Factorization is supposed to be hard, and some of the prior work I mentioned [1-4] actually talked about this and briefly reasoned that SGD wouldn't be able to pull the prime factors.

So now, many open questions are opened (which to be clear, I dont expect you to answer in this rebuttal, but I think adding them to your paper will greatly strengthen its impact), especially because your discussion is currently only a paragraph, with another paragraph mentioning some possible future directions.

  1. i) Why does SGD learn the prime factors when it is shown both multiplication and addition mod n? Is it because it is a ring (your paper asks this), but maybe it is just because more data is presented (scale is all you need afterall). But a counter-point to it being because of more data is the dihedral group paper I mentioned above, which has two cyclic groups mod n, and thus the dataset is 4x larger than training on just one cyclic group, and 2x larger than training on (a+b) mod n and (a x b) mod n (which is what you train on). So maybe it is because it is because it is a ring, since the case with more data and similar learning complexity, but it is a group, does not pull the prime factors. I believe an entire theory / math paper should be written about this.

ii) But also, maybe it isn't true in general, maybe this is one magic ring where SGD actually happens to be just be unreasonably efficient, this open question alone could spawn multiple future papers because the field is trying to figure out why, what, and how happens in pursuit of safe AI which inspired these studies on math tasks.

iii) how does SGD do this? Note this is different to why. Obviously, there must be a point where the training dynamics realize the factors result in a "good local minimum". Is this because the prime factors give larger margins at the logits (mentioned by [1] or if not in [1] then in [2] or [4], possibly also mentioned by [6] or [7], though I think those works also treated all possible factors, not just correct ones). I believe multiple math / theory papers could be written about this, depending on whether we get a nice result, or different phenomena on different rings.

With all that said, I hope I showed that there's a very interesting way to show how your work builds on existing results and actually finds something that no prior work (to my knowledge) has found: learning what are likely the best factors deterministically. Note, the global minimum is still probably something like learning all numbers and "pretending" they're all factors (see [6] and [7]), but I'm willing to bet that if a composite number has k factors, there's a proof that someone could do that learning the k correct factors gives much better margins at the logits than k random factors [1]).

So you can add this as another open question for future work if you want:

iv) can someone prove that learning the per-prime factors is better than learning k random factors (in terms of loss, or margins, or whatever performance metric they find interesting).

So for the weaknesses above, I have given this paper a weak reject, but I could significantly increase my score depending on how 1 and 2 are addressed by a revision.

[1] McCracken et al "Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks" [cited]

[2] Moisescu et al "On the geometry and topology of representations: the manifolds of modular addition" [not cited]

[3] McCracken et al "Interpreting deep neural networks trained on elementary p groups reveals algorithmic structure" [not cited]

[4] Wei et al "Deep neural networks divide and conquer dihedral multiplication" [not cited]

[5] Hilgert et al "Toward a general understanding of neural representations learned by deep neural networks on group multiplications" [not cited]

[6] Mohadami et al "Grokking modular arithmetic can be explained by margin maximization" [not cited]

[7] Morwani et al "Feature emergence via margin maximization: case studies in algebraic tasks" [not cited]

Other work I feel should be cited that is not currently cited and related precursor work:

[8] Tao et al "(How) can transformers predict pseudo-random numbers?" This work again shows difficulty learning prime factors.

[9] He et al "Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks", the training data here is addition of modular linear polynomials, without any multiplication

[10] Doshi et al "Grokking Modular Polynomials", interestingly it shows some polynomials are unlearnable when the modulus is certain numbers (sometimes prime, sometimes composite), it would be interesting if the idea of augmenting with other problems mod the same prime could make transformers learn them

[11] Wu et al. "Towards a unified and verified understanding of group-operation networks"

Questions for authors

I think I proposed enough in the weaknesses section, but I'm curious about this, consider these bonus questions to look into during the rebuttal. I think they should be included in a final camera ready paper in the appendix, but I understand my weaknesses section will take time to get the quantitative results I'm asking for and make the plots, as well as take time to rewrite your discussion section and related work to better contextualize the paper.all of the papers

  1. What about training on mod p where p is prime? Obviously there are no factors here, so what does it do when you look at your case, which is addition data and multiplication data (the ring) vs the just modular addition or just multiplication cases?

  2. Can you add results with a 1-hidden layer MLP with trainable embedding (not one-hot) see [2, 4] for examples of one, though, also seeing the result for one-hot could be interesting, because the expectation in that case is that all numbers are learned and used as factors (Gromov 2023, which your paper cites) and [6] and [7]. It would be shocking to me if a one hot MLP pulls the prime factors (with only 1 layer). My expectation is a 1-layer MLP finds your result, but the one-hot MLP finds all numbers (including the per-primes).

I am looking forward to our continued discussion and seeing a revised, more complete, paper during rebuttals because I really think you can upgrade the current paper, which really reads as just a "we interpret a math task" and turn it into a great story that not only shows off how different the result is in contrast to what's been found so far, but also opens many directions for future work that are each strongly motivated by the fact this result is shocking. I think this result really needs to be emphasized for how unique it is, and the best way to do that is to show how all the preceding work on this task published results, that do not predict your main result. That is why I like your work, it finds something new and you are currently not advertising why and how it is novel well enough to make it clear how many future papers can be written that build on your work. I sincerely believe this result is of high enough impact that it deserves better writing motivating it and discussing it, as well as more broad quantitative experiments to measure its robustness.

8 comments

We would like to sincerely thank you for your extremely informative and rigorous review.

 

On robustness

Thank you for bringing this to our attention. We agree that the result is striking enough that this should have been emphasized in the paper. In general, and quite surprisingly, we do find the result to be robust across different architectures, widths, depths, train-test splits, and hyperparameters. Each of these results is now in the appendix.

Below we report the proportion of the residual stream, for both addition and multiplication examples, which is carried by the pure frequencies. That is, we average the residual stream after the last block over all examples with the same answer, and report the fraction carried by the pure frequencies after taking the Fourier transform. As noted in the paper, when the model uses only pure frequencies this results in modularity, so scores near 1 indicate that the model is modular. The ablation tests specifically are in the paper as appendix as well. In each case we are training on mod 1001, but as noted in the paper, this works for other moduli as well.

Table 1. Pure share of residual stream across different learning rates and weight decay, modulus n=1001n = 1001. Only runs reaching >0.9>0.9 accuracy included.

Addition (++)

lr \ wd00.010.10.51.0
10−410^{-4}–1.00–0.941.00
3⋅10−43\cdot 10^{-4}1.001.001.001.00–
10−310^{-3}1.001.001.001.00–
3⋅10−33\cdot 10^{-3}–1.001.00––

Multiplication (×\times)

lr \ wd00.010.10.51.0
10−410^{-4}–0.94–0.960.96
3⋅10−43\cdot 10^{-4}0.990.970.990.97–
10−310^{-3}1.000.991.000.98–
3⋅10−33\cdot 10^{-3}–0.981.00––

Table 2. Pure share of residual stream across different training fractions, modulus n=1001n = 1001.

Training fraction0.20.30.40.50.70.80.95
++1.001.001.001.001.001.001.00
×\times0.771.000.991.001.001.000.99

Similarly, below shows the same measurement for a few different widths and depths.

Table 3. Pure share of the residual stream for different widths and depths (n=1001n = 1001).

Model++×\times
1 block, width 256 (appears in paper)1.001.00
1 block, width 5121.000.99
1 block, width 10241.001.00
2 blocks, width 2561.000.99
4 blocks, width 2560.950.95

And then we have the same measurement for MLPs:

Table 4. Pure share of the residual stream for attention-free MLPs (n=1001n = 1001), mean over seeds. Width is 10241024, training fraction 0.50.5, weight decay 0.10.1, learning rate 0.0010.001.

Model++×\times
1 hidden layer, learned embedding (2 seeds)1.001.00
1 hidden layer, one-hot inputs (2 seeds)1.001.00
2 hidden layers, learned embedding (5 seeds)1.000.99

As you can see, models are consistently modular, including across widths, depths, hyperparameters, and in MLPs with learned embeddings or one-hot. The one exception seems to be that multiplication is not fully modular when trained on a transformer with a training fraction of 0.2, which is quite interesting, and the reason why would make an interesting follow up. Also, note that some of the learning rate/weight decay combinations did not reach high accuracy, despite training for up to 300k steps, so were not reported.

We also find the results hold in other rings, for example we can take the following finite polynomial rings, and apply the CRT to them, finding models consistently use pure subspaces. Consider the rings F5[x]/(x4−x)\mathbb{F}_5[x]/(x^4 - x), F5[x]/(x4−x2)\mathbb{F}_5[x]/(x^4 - x^2), and F5[x]/(x4−1)\mathbb{F}_5[x]/(x^4 - 1). Doing the exact same analysis, the model factors these rings via the CRT:

Table 5. Residual stream fraction on pure CRT components. Each ring has 625625 elements and the same additive group (Z/5)4(\mathbb{Z}/5)^4. Training fraction 0.50.5, weight decay 0.10.1, learning rate 0.0010.001. One layer transformer.

RingCRT decomposition++×\times
F5[x]/(x4−x)\mathbb{F}_5[x]/(x^4 - x)F5×F5×F25\mathbb{F}_5 \times \mathbb{F}_5 \times \mathbb{F}_{25}1.000.99
F5[x]/(x4−x2)\mathbb{F}_5[x]/(x^4 - x^2)F5×F5×F5[x]/(x2)\mathbb{F}_5 \times \mathbb{F}_5 \times \mathbb{F}_5[x]/(x^2)1.000.99
F5[x]/(x4−1)\mathbb{F}_5[x]/(x^4 - 1)F5 4\mathbb{F}_5^{\,4}1.000.97

On contextualizing the result, and relevant literature

We would like to thank you for bringing each of the relevant papers to our attention, and apologize for the lack of context and credit in the original draft. We have added this to the related works in the new draft, which further emphasizes the prior work in this field, and how our paper differs from it:

Whether models implement the Chinese remainder theorem has been studied extensively. However, no work has found that models explicitly implement it in the way described in this paper. Hwang et al. (2026) note that standard embeddings do not naturally expose per-prime structure when trained on modular addition, and propose embeddings that do. McCracken et al. (2025b) describe an approximate CRT for modular addition, in which approximate cosets of the additive group are intersected in a manner resembling the CRT to find the correct answer. Importantly, the model may learn frequencies that do not divide the modulus, so this is only approximate. In our paper, not only does the model learn frequencies dividing the modulus, but they correspond to precisely the prime factors of the modulus.

Other works similarly focus on studying cosets, and do not find a direct implementation of the CRT. McCracken et al. (2025a) focus on p-groups explicitly, particularly how models create Cayley graphs whose vertices are cosets of the group, an idea also seen in Hilgert et al. (2026). Wei et al. (2026) describe a similar divide-and-conquer algorithm, again involving approximate cosets. Moisescu-Pareja et al. (2026) point towards a universality hypothesis being true in the case of modular groups, which again differs from our literal interpretation of the CRT.

In contrast to the above work, our result is quite surprising, and goes against the intuition of the literature. When training multiplication with addition at the same time, networks no longer implement an approximate CRT using approximate cosets, but instead consistently implement the CRT explicitly, by factoring the modulus into primes, and solving for each prime completely independently.

On why rings are modular

We agree that including an answer to why the ring is modular would strengthen the paper. Here is the argument, which we have added as a section to the paper:

  1. Multiplication must learn pure-prime characters. Given a modulus nn, and a prime divisor pp, the only Dirichlet characters (multiplicative characters) which are non-constant on the set of all x=0mod  n/px = 0 \mod n/p are the pure mod pp characters. To distinguish between these, the model must therefore learn some pure characters for each prime divisor.
  2. The pure additive and pure multiplicative subspaces are the same in embedding and unembedding space. With (1) this implies addition must learn pure characters too, because addition and multiplication share embedding and unembedding vectors.
  3. Forcing the model to use a small amount of the pure subspaces for addition can make the entire model modular. We find that when training a model on modular addition, if you require a small proportion of the embeddings and unembeddings to stay in the pure subspaces, the model becomes fully modular.

As an example of (3), we train on addition mod  1001\mod 1001, with the requirement that the unembedding and embedding matrices must keep at least 1/501/50th of their norm in each of the pure subspaces (if it goes below at a given step, we grow the norms in those subspaces back to our floor), then once fully trained the residual stream had a fraction of 0.880.88 in the pure subspaces. Increasing that floor to 1/101/10 puts the residual stream entirely in pure subspaces.

A likely explanation is that the pure subspaces are sufficient to solve the problem due to the CRT, so if they also become necessary (as for example this experiment, or cotraining multiplication makes it), then the model only learns those subspaces perhaps due to weight decay or implicit regularization, but this is something to be explored in a future paper.

Even more broadly, there is an interesting phenomenon here, where training two tasks together causes interference which changes how the model implements each task.

On learning mod p rings

This is an interesting question which we also intended on further studying in a follow up. When training mod pp for pp a prime, each operation uses a sparse set of its own characters. Breaking down the residual stream for a model trained mod  97\mod 97, you can check how many characters are required for each operation to be reconstructed in the additive and multiplicative bases:

OperationThresholdAdditive basisMultiplicative basis
addition0.50.5523
0.90.91143
0.990.991448
multiplication0.50.5244
0.90.9439
0.990.994810

Here you see that addition uses a sparse set of additive frequencies, and multiplication a sparse set of multiplicative frequencies. So each operation is implemented almost independently, and the angle between the additive and multiplicative representations has cosine ≈1/p−1\approx 1/\sqrt{p-1}, due to the fact that if χ\chi is a mod  p\mod p Dirichlet character, then ∣χ^∣=1/p−1|\hat{\chi}| = 1/\sqrt{p-1}, so as pp grows these become essentially orthogonal.

We sincerely thank you for your review, and have added the new experiments to the Appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.