acceptodds
Under review as a conference paper at ICLR 2027

Coordinate-Dependent Generalization in Transformers: Controlled Interventions and Prospective Prediction on Modular Arithmetic

Abstract

The same mathematical relation can be learnable or unlearnable for a transformer depending only on the coordinates in which it is presented. We isolate this effect with modular arithmetic tasks whose targets are fixed by theorems, where the presentation can be changed while the underlying relation, the information available to the learner, the training data and the held-out inputs are preserved. Re-expressing both inputs and outputs in discrete-logarithm coordinates turns a nonlinear target into an affine one and raises held-out accuracy from chance to between 0.459 and 0.853 on two theorems, while changing either side alone leaves both at chance. In a second, matched intervention, two presentations that share their relabelled inputs, algebraic degree and agreement with the closest affine map reach 0.977 and 0.006, so the internal structure of the presented target, not these coarser properties, carries the effect. Statistics of the complete presented target, computed before training, correctly predict the outcome on all 18 held-out tasks, ten of them with predictions specified before training. Measurements of the trained networks show that the presentation determines whether a low-complexity solution is available, while weight decay affects whether training selects it. All core results use positional-digit encodings.

Primary area:
interpretability and explainable AI
Keywords:
grokking, modular arithmetic, generalization, input representation, Fourier analysis, weight decay, algorithmic reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_J7GSOpenReview ↗
Rating
3
Confidence
2(−1)
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how the coordinate representation of mathematical tasks affects Transformer generalization. Using controlled experiments on modular arithmetic, the authors show that the same mathematical relation can have very different learning difficulty depending on how it is represented. In particular, applying a discrete logarithm transformation turns nonlinear modular exponentiation into an affine mapping, improving held-out accuracy from nearly chance level to 45.9% and 85.3% on two tasks based on number-theoretic theorems.

To investigate this effect, the authors conduct relabeling experiments that control for algebraic degree, best-affine agreement, and the set of modified inputs. The results show that the internal structure of the presented function matters for generalization, even when these other factors are held fixed. The authors also introduce Fourier Complexity as a measure of function complexity and test its ability to predict generalization on additional tasks.

Finally, the paper examines the functions learned by trained networks and the role of weight decay. The results suggest that the coordinate representation determines whether a low-complexity solution is available, while weight decay helps the model select a solution that generalizes. Additional experiments cover different Transformer architectures, MLPs, LSTMs, numerical bases, and finite fields.

Critical strengths

1. Well-controlled experiments that isolate the effects of different factors. The paper does more than compare generalization across coordinate systems. By independently changing the input and output coordinates, the authors show that a substantial improvement occurs only when both are transformed. The matched relabeling experiments are particularly convincing. Even with the same algebraic degree, best-affine agreement, and modified input set, changing the structure of the relabeled mapping produces a large accuracy gap of 97.7% versus 0.6%. This provides strong evidence that the structure of the presented function, rather than just its coarse algebraic properties, affects generalization.

2. A useful connection between mathematical analysis and generalization prediction. The authors use Fourier analysis to characterize the complexity of target functions under different coordinate systems and derive theoretical relationships between Fourier structure and low-complexity predictors. Importantly, the paper goes beyond fitting correlations to existing results. It evaluates fixed prediction rules on additional tasks, including predictions specified before model training. The experiments across architectures, numerical bases, and weight decay settings also help establish that the main observations are not limited to a single configuration.

Critical weaknesses

1. Fourier Complexity does not fully explain or predict generalization. Although Fourier Complexity correlates strongly with generalization in many experiments, it is not a sufficient predictor. As shown in Appendix D.2, tasks with similar Fourier Complexity can have very different held-out accuracies, ranging from nearly 0% to 78%. Moreover, on some task families not used to select the predictors, Best-affine Agreement performs as well as or better than the Fourier-based metrics. This raises questions about how much additional explanatory value Fourier Complexity provides beyond simpler structural measures. The metric captures useful properties of the target function and provides additional predictive information in some controlled settings, but the consistency of this contribution across different task families remains unclear.

2. The theoretical analysis does not fully explain how Transformers learn generalizing solutions. The theoretical results mainly constrain prediction functions with a given Fourier Complexity. They do not establish a direct relationship between this complexity measure and neural network size, optimization dynamics, or the functions that a Transformer can actually learn. Although the experiments show that weight decay improves generalization while reducing the Fourier Complexity of the learned function, they do not fully explain why optimization selects these solutions. The authors also find that structured non-Fourier features can improve generalization on difficult tasks by a similar amount. This suggests that the observed benefits may come from structured representations more generally, rather than Fourier structure specifically. The proposed explanation of the learning mechanism therefore remains incomplete.

Questions for authors

1. The paper shows that tasks with similar Fourier Complexity can have very different generalization performance, with accuracy ranging from nearly 0% to 78% in Appendix D.2. Could the authors explain which structural properties might account for these differences? In particular, would combining Fourier Complexity with piecewise-affine structure or properties of the digit encoding lead to more reliable predictions?

2. The theoretical analysis mainly characterizes the Fourier structure of target functions, but does not directly explain why Transformer optimization favors certain low-complexity solutions. Could the authors clarify what additional assumptions or evidence would be needed to connect Fourier Complexity to the optimization dynamics that select generalizing solutions? Also, given that structured non-Fourier features can produce similar improvements, how can we distinguish the role of Fourier structure from that of structured representations more generally?

3 comments
AuthorsOpenReview ↗

Response to Reviewer J7GS

Thank you for the accurate summary of the paper and for recognizing the matched controls and the predictions specified before training. Both of your questions led to new analyses, which are now in the revised paper. Line numbers refer to the original submission. Section, table and figure numbers refer to the revised PDF.

Tasks with similar Fourier Complexity can have very different held-out accuracies, ranging from nearly 0% to 78%. Could the authors explain which structural properties might account for these differences?

  • Your reading matches the paper’s own position. Lines 462 to 467 state that Fourier complexity tells which side a task falls on, generalize or fail, but not the borderline cases, and that best-affine agreement predicts as well as any spectral measure.
  • The property that explains the difference is how the non-affine part of the task is arranged. In a piecewise-affine task, the inputs are split among several rules and no single rule covers most of them. In a perturbed-affine task, one rule covers most inputs and a few scattered inputs are exceptions.
  • Best-affine agreement captures this difference and Fourier complexity does not. This accounts for accuracies of 0.009 to 0.025 against 0.55 to 0.78 at the same Fourier complexity of about 7 (Section 5.4, Appendix D.2).

Would combining Fourier Complexity with piecewise-affine structure or properties of the digit encoding lead to more reliable predictions?

  • We ran this test. We combined Fourier complexity, best-affine agreement and a measure of how the exceptional inputs line up with the digits. The combination was fitted on one family of tasks and tested on a different family (Table A13).
  • It predicts better than Fourier complexity alone in two of the three tests (rank correlation 0.872 against 0.646, and 0.820 against 0.533). In the third, it does not beat best-affine agreement alone (0.785 against 0.961).
  • The digit encoding matters by itself. Writing the same functions in base 7 instead of base 10 reverses which of two tasks is easier, in all four comparisons. We predicted this before training (Figure 3b,c, Table A12).
  • So no measure that ignores how numbers are written as digits can fully predict accuracy. Fourier complexity tells whether a simple solution exists. How well it is found depends on the digit encoding and on training.

The theoretical analysis does not directly explain why Transformer optimization favors certain low-complexity solutions. Could the authors clarify what additional assumptions or evidence would be needed to connect Fourier Complexity to the optimization dynamics?

  • The theory is deliberately limited. It bounds how well a function of a given Fourier complexity can match the task (lines 454 to 458). It does not say which function training will find. That link is measured in the paper, not derived.
  • The measurement is this. With weight decay, the learned function becomes simpler during training: its Fourier complexity falls by 3.45 while the weight norm falls by 8.3 and accuracy rises by 0.428. Without weight decay, the complexity stays near 14 (lines 387 to 391). The two changes happen together.
  • A derivation would need two things. First, a result showing that, for this architecture and digit inputs, the solutions with small weights are also the ones with low Fourier complexity, in the spirit of margin-maximization analyses of algebraic tasks (Morwani et al., 2024). Second, an experiment that changes the complexity of the learned function while keeping the weight norm fixed, or the reverse.
  • Our measurements are consistent with the first and do not establish it. Section 6 lists this as the first limitation.

Given that structured non-Fourier features can produce similar improvements, how can we distinguish the role of Fourier structure from that of structured representations more generally?

  • Our experiments do not separate the two, and the paper does not claim that Fourier structure is special. Fourier-based and structured non-Fourier input features help a failing task equally (0.785 and 0.777; lines 367 to 370). Removing signal in a non-Fourier way is as damaging as removing the Fourier components (0.048 against 0.000).
  • Fourier complexity should be read as one convenient way to measure how concentrated the structure of a task is. Other measures, spectral entropy and best-affine agreement, behave similarly (Table 3), and Section 5.3 states this conclusion.

We hope these additions address the two concerns behind your soundness score. If anything remains unclear, we would be glad to clarify it during the discussion.

Thank you for your detailed response and for addressing my concerns about Fourier Complexity and generalization. I also appreciate your clarification that the paper does not aim to explain why Transformer optimization favors certain low-complexity solutions.

However, this clarification differs somewhat from my original understanding of the paper, which makes me wonder whether I may have misunderstood its intended scope and contributions in my initial review.

I still believe that the research questions addressed in this paper and the controlled experiments are very valuable. However, after reading the other reviewers' comments, particularly their concerns about the novelty of the findings in relation to existing work on Grokking and Weak-to-Strong Generalization, I have become less certain about the broader significance of the paper. In particular, it remains unclear to me how well the findings and methodology can generalize beyond the specific arithmetic settings studied here.

Overall, I will maintain my original score of 3 (Weak Acceptance), but lower my confidence from 3 to 2, reflecting my uncertainty about the paper's novelty and broader significance relative to the existing literature.

AuthorsOpenReview ↗

Follow-up to Reviewer J7GS

Thank you for the follow-up and for maintaining your score. We answer the three points you raise. Line numbers refer to the original submission. Section, table and figure numbers refer to the revised PDF.

However, this clarification differs somewhat from my original understanding of the paper, which makes me wonder whether I may have misunderstood its intended scope and contributions in my initial review.

  • Your original understanding was correct, and the scope of the paper has not changed. Your review summarized it as: “the coordinate representation determines whether a low-complexity solution is available, while weight decay helps the model select a solution that generalizes.” This is exactly contribution C3.
  • Our reply only separated two things. The paper shows by experiment that weight decay selects the simple solution: accuracy is 0.060 without it and 0.459 with it (lines 380 to 395; Section 5.2). It does not prove mathematically why. Your review drew the same line in its second weakness, and the submission lists it as a limitation (lines 454 to 458).

However, after reading the other reviewers' comments, particularly their concerns about the novelty of the findings in relation to existing work on Grokking and Weak-to-Strong Generalization, I have become less certain about the broader significance of the paper.

  • Grokking work keeps one way of writing the task and studies how generalization appears inside the model during training. We do the opposite: we keep the model and the training fixed and change only how the task is written. The two are complementary.
  • Our results add to that literature directly. Its standard benchmark (Power et al., 2022) gives each number its own token. When the same numbers are written as digits, two of the nine operations generalize instead of eight, as we predicted before training (lines 419 to 427; Table 2). A published grokking result therefore depends on a choice that had not been tested.
  • Weak-to-strong generalization, as we understand it, studies a model that learns from a weaker supervisor. It concerns the quality of the labels, not the way the task is written, so it does not address our question.
  • The concern was phrased as “better representations lead to generalization”. Two of our results are not predicted by that statement. Giving the model the helpful naming on the input side only, or on the output side only, gives no gain (Table 1; lines 195 to 199). And two tasks with the same algebraic degree, equally close to a simple straight-line rule, reach 0.977 and 0.006 (Figure 2; lines 245 to 252).

In particular, it remains unclear to me how well the findings and methodology can generalize beyond the specific arithmetic settings studied here.

  • We use arithmetic on purpose. It is the setting where the full table of answers is known exactly, so the written form can change while the task and the test inputs stay identical. Arithmetic tasks are also the standard testbed for grokking and for length generalization, so the findings apply where such claims are made.
  • The method carries over to any task whose answers are known: write the same task in two ways, keep the examples and test inputs the same, and compare. The practical lesson carries over too: before a failure is blamed on the architecture, check how the task was written and tokenized (Section 7).
  • Within arithmetic, the effect is not tied to one setting. It holds on two theorems, three model types, four number bases, a second number system and a published benchmark (lines 399 to 427; Appendix F).
  • We do not claim results on natural-language tasks, and Section 6 states this scope.

We hope this resolves the uncertainty about scope and novelty. Thank you again for the careful engagement.

Reviewer_ujSBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how coordinate choices affect generalization on digit-encoded modular arithmetic. It compares residue and discrete-logarithm presentations, tests controlled relabellings, and evaluates whether target-level Fourier statistics predict training outcomes.

Critical strengths

The crossed input/output interventions and matched relabelling experiments provide useful controls. The authors test multiple architectures and radices.

Critical weaknesses

Clarity. Very poorly written paper. It is difficult to read with non-technical terms used without proper definition (or scattered definitions). The tone and style is also informal, and not suited for a scientific paper. The different sections just present numbers, which make the story hard to follow. It moves between Fourier complexity, partial generalization, and learnability without consistently distinguishing them. Numerous results use different success criteria and evaluation subsets, which makes their relationship to the main claims difficult to follow. In Section 5, The predictor is a threshold rule computed from the complete target table. Its perfect classification separates affine targets from strongly non-affine targets, leaving its value beyond a simple affine detector unclear.

Significance. The favorable logarithmic presentation still achieves only partial generalization. The paper does not explain this limitation or why it remains substantially harder than the native affine controls, despite both having minimal target Fourier complexity. There is a scope of overclaiming the result on line 86 : “None of this work isolates the presented structure as the cause.” However, the experiments performed do not isolate the cause. The interventions establish an effect of presentation, but do not isolate Fourier complexity as the mediating property. The representation analyses also fail to identify the computation responsible for the improvement or establish a Fourier-specific mechanism.

Novelty. The novelty beyond established representation effects is insufficiently demonstrated. For example, in related group-learning problems, learned matrix representations that respect group relations improve generalization to unseen group elements and longer words [1]; representation-theoretic decompositions provide low-rank implementations of group-word operations and help explain the solutions learned by neural networks [2]; and distinct internal algorithms for permutation composition exhibit different generalization properties, with training interventions steering which algorithm emerges [3]. The basic importance of representation is established in the literature. The paper therefore needs to demonstrate what its controlled comparisons and diagnostics add. They acknowledge the limitation of their relevance themselves : “The step from bound to network is unproved… Nothing here is a Fourier-specific representation.” In this sense, the relevance of these results to modular arithmetic training of real neural networks is questionable. As such, it is not suitable for publication in ICLR.

[1] Laird, L., Hsu, C., Bapat, A., & Walters, R. (2024). MatrixNet: Learning over symmetry groups using learned group representations. Advances in Neural Information Processing Systems, 37, 32512-32535.
[2] Shutman, M., Louidor, O., & Tessler, R. (2025). Learning words in groups: fusion algebras, tensor ranks and grokking. arXiv preprint arXiv:2509.06931.
[3] Li, B. Z., Guo, Z. C., & Andreas, J. (2025). (How) Do Language Models Track State?. arXiv preprint arXiv:2503.02854.

Questions for authors

I do not have any questions that could materially affect my assessment. I urge the authors to fix their presentation, clarity and novelty issues.

5 comments
AuthorsOpenReview ↗

Response to Reviewer ujSB (1/3)

Thank you for the detailed reading, for acknowledging the crossed and matched controls, and for the three references. Line numbers refer to the original submission. Section, table and figure numbers refer to the revised PDF.

I do not have any questions that could materially affect my assessment. I urge the authors to fix their presentation, clarity and novelty issues.

  • We have acted on all three, and we would be grateful if you could look at the revised PDF.
  • Presentation: the paper now follows the standard order Related work, Approach, Experiments, Analysis, Discussion, and the wording is formal throughout. No claim or result has changed.
  • Clarity: each term is defined where it first appears (Section 3), a small worked example introduces the discrete logarithm (Section 1), and the accuracy measures and thresholds are defined in one place (Section 4.1, Table A28).
  • Novelty: Section 2 discusses the three works you suggested. They study or change the model, while we keep the model fixed and change only how an exactly known task is written.

Very poorly written paper. It is difficult to read with non-technical terms used without proper definition (or scattered definitions). The tone and style is also informal, and not suited for a scientific paper. The different sections just present numbers, which make the story hard to follow.

  • The changes above respond to this comment. In the submission, the terms were defined together in one table (lines 110 to 123); they are now defined in the running text.
  • Each experimental subsection now states its question, what is compared, the result and what the result shows, so that every number is tied to the point it supports.

It moves between Fourier complexity, partial generalization, and learnability without consistently distinguishing them.

  • Lines 136 to 139 separate three things: how much of the task matches its closest affine (straight-line) rule, called best-affine agreement; how few affine pieces the task is made of; and how concentrated its Fourier spectrum is, measured by the Fourier complexity ν\nu, which is 1 for an affine rule and about 23 for a random table.
  • Section 3 of the revision defines these measures side by side. It also states the three separate uses of the spectrum: to describe the task before training, to describe the function a trained network has learned, and to test what the network uses internally.

Numerous results use different success criteria and evaluation subsets, which makes their relationship to the main claims difficult to follow.

  • Each success threshold was fixed before the runs it is applied to. The thresholds differ because the experiments ask different questions.
  • In the coordinate experiment, the threshold was a held-out accuracy of 0.5, fixed before the runs, and every result is reported against it (lines 206 to 208). The second theorem clears it in every run (0.853).
  • In the prediction test, the thresholds were 0.7 for “will generalize” and 0.1 for “will fail”, also fixed in advance (lines 283 to 284).
  • In the matched experiments, only one block of inputs is changed. The other 71% of inputs follow the same simple rule in both versions, and the model learns them either way. Accuracy over all inputs (0.972 against 0.656) therefore hides most of the difference. This is why we also report accuracy on the changed block alone, where the difference is 0.977 against 0.006.

In Section 5, the predictor is a threshold rule computed from the complete target table. Its perfect classification separates affine targets from strongly non-affine targets, leaving its value beyond a simple affine detector unclear.

  • The submission makes this point itself. It presents the rule as a check that sees the whole table of the task (lines 272 to 274). It also notes that the 18 tasks fall into two well-separated groups, ν=1.00\nu = 1.00 or ν≥12.49\nu \geq 12.49, so “the cutoff is not doing the work” (lines 286 to 288).
  • It reports that the simpler measure, best-affine agreement, predicts equally well, and concludes that “what predicts is presented affineness” (lines 290 to 297).
  • What ν\nu adds is tested separately, on tasks built so that best-affine agreement is the same for all of them. There, ν\nu still ranks the tasks by accuracy (rank correlation 0.743 and 0.966), while best-affine agreement cannot (−0.094-0.094). In 10 of 12 matched pairs, the task with the lower ν\nu is learned better, p = 0.019 (lines 298 to 308).
  • The 18-task test gave the claim a chance to fail: the rule was fixed before ten of the tasks were trained.
  • In the revision, contribution C2 and Section 5.4 say that ν\nu “acts as an affine detector” on such tasks, and Table 3 places it beside best-affine agreement and algebraic degree.
AuthorsOpenReview ↗

Response to Reviewer ujSB (2/3)

The favorable logarithmic presentation still achieves only partial generalization. The paper does not explain this limitation or why it remains substantially harder than the native affine controls, despite both having minimal target Fourier complexity.

  • ν=1\nu = 1 means the task is an affine rule, so a simple solution exists. It does not mean that every affine rule is equally easy to learn when numbers are written as digits. The submission reports this gap (lines 206 to 211 and 462 to 464).
  • Following this comment, we added an analysis in Section 4.4 and Appendix C.1. It shows that the gap is caused by the multiplier in the rule, not by the logarithmic naming. In logarithmic form the task is x↦11x mod 690x \mapsto 11x \bmod 690, while the easy control is r↦1+r mod 691r \mapsto 1 + r \bmod 691.
  • We changed the differences between these two tasks one at a time. The modulus, the added constant and the choice of held-out inputs change accuracy by at most 0.016. Changing the multiplier from 1 to 11 lowers it from 0.984 to 0.446 (Table A6).
  • Nine affine rules that differ only in their multiplier reach accuracies from 0.444 to 1.000 (Figure 3a, Table A7). Most errors (75% to 92%) are wrong in a single digit, by ±10\pm 10 or ±100\pm 100, and occur near inputs where the answer wraps around the modulus.
  • A rule with multiplier 11 written in the ordinary, non-logarithmic form is harder still (0.095). So the logarithmic naming helps, and what remains difficult is multiplying by 11 in digits.

There is a scope of overclaiming the result on line 86: “None of this work isolates the presented structure as the cause.” However, the experiments performed do not isolate the cause. The interventions establish an effect of presentation, but do not isolate Fourier complexity as the mediating property.

  • The sentence at line 086 is about the written form of the task (“the presented structure”), which is what our experiments change directly. In the coordinate experiment, the task, the training examples, the test inputs and the training settings are identical, and only the written form differs. The matched experiment also keeps the changed inputs, the degree and the best-affine agreement identical (lines 245 to 252).
  • The submission does not claim that Fourier complexity is the mediating property. It calls ν\nu “our measure of” the structure (lines 247 to 248). It states that one family of tasks “does not separate ν\nu from piece count” (lines 262 to 263), and that “the defensible claim concerns presented affineness” (lines 466 to 467).
  • When only the written form changes and accuracy moves from chance to 0.459 or 0.853, the written form is the cause in the experimental sense. ν\nu is a tool for describing the written form, not a claimed cause.
  • The revised introduction poses the question as “which property of a presentation is responsible”, and Section 4.3 concludes that the structure “can be measured either by its number of affine pieces or by ν\nu”.

The representation analyses also fail to identify the computation responsible for the improvement or establish a Fourier-specific mechanism.

  • Explaining the network’s internal algorithm is not a claim of the paper. We ran these analyses as checks, and their conclusion is stated in the heading at line 363: “The representation carries the function, but not in a Fourier-specific way”.
  • The checks show that Fourier structure is not special (lines 363 to 374):
    • Removing the strongest Fourier components from the network’s internal signal destroys its accuracy (0.000). Removing an equal amount of signal in a non-Fourier way does the same (0.048).
    • Giving a failing task extra Fourier-based input features helps (0.785). Structured features that are not Fourier-based help equally (0.777).
    • A Fourier fit to the training answers alone, with no network, reaches 0.975. This is above the 0.627 obtained from the network’s own signal.
  • Our claims are about the task, not about the mechanism: which written forms make a generalizing solution possible, and whether training finds it. They hold whatever algorithm the network uses. Section 5.3 keeps these checks under the title “Fourier specificity”.
AuthorsOpenReview ↗

Response to Reviewer ujSB (3/3)

The novelty beyond established representation effects is insufficiently demonstrated [1, 2, 3]. The basic importance of representation is established in the literature. The paper therefore needs to demonstrate what its controlled comparisons and diagnostics add.

  • Thank you for these references, which are complementary to our work and are now discussed in Section 2. [1] learns matrix representations that respect group relations, [2] explains learned solutions through representation-theoretic decompositions, and [3] identifies internal algorithms and steers which one emerges. In each, what is studied or changed is the model.
  • We keep the model fixed and change only how an exactly known task is written. This gives four results that the general statement “representation matters” does not give:
    • A helpful naming on one side only, inputs or outputs, does not help. It must be on both (Table 1).
    • Two versions of a task with the same degree, and equally close to an affine rule, are not equally learnable: 0.977 against 0.006 (Figure 2).
    • Even among affine rules, which all have the same Fourier complexity, the multiplier and the number base change accuracy. Switching from base 10 to base 7 reverses which of two tasks is easier, as we predicted before training (Section 4.4).
    • On a published benchmark, two of nine operations generalize when numbers are written as digits, against eight. We predicted this before training (Table 2).
  • “Representation matters” does not say which property matters, and it does not let one predict a result before training. Our comparisons do both. They also show that the obvious candidates, degree and best-affine agreement, are not the answer.

They acknowledge the limitation of their relevance themselves: “The step from bound to network is unproved… Nothing here is a Fourier-specific representation.” In this sense, the relevance of these results to modular arithmetic training of real neural networks is questionable.

  • The two quoted sentences set limits on the theory, which is a mathematical bound, and on claims about the network’s internal mechanism. They are not about the experiments, which all train real networks.
  • The results hold on four model types: a one-layer transformer, a standard two-layer transformer with LayerNorm (0.624 in the coordinate experiment; 0.964 against 0.010 in the matched experiment), an LSTM (0.525) and an MLP (0.391) (lines 399 to 402; Tables 1, A19 and A26).
  • They also hold with a much larger modulus (10007) and with ten times longer training (120,000 steps) (lines 403 to 405). The nine operations of Power et al. (2022) were retrained with numbers written as digits (lines 419 to 427).
  • Because the bound does not explain training, we measure directly which function each trained network has learned (lines 353 to 362), and do not assume it.
Reviewer_UhaNOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper explains the way that the data is represented matters in addition to the architecture, optimizer, etc. The paper considers several modular arithmetic tasks to illustrate this.

Critical strengths

1- Task representation is an important concept in machine learning that has been studied less. The paper provides fruitful intuitions on what the model can and cannot learn.

Critical weaknesses

1- The abstract is written poorly with no clear structure. I couldn't understand the key points of the paper from it.

2- The paper considers a very limited scope, and shows that better representations lead to generalization. I do not understand the significance of these points. It is already known that the feature representations are important for generalization in phenomena such as grokking and weak-to-strong. I.e., if the model lacks the good features, it generalizes poorly. The solution that the papers proposes is to just provide the good features for learning the task to the model.

3- That said, the paper does not explain how we can use any of the findings in the paper on a broader sense: how can we find the good representations on other and general domains?

Questions for authors

Please read above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer UhaN (1/2)

Thank you for your review, and for recognizing that task representation is an important and less studied question. We address your points below. Line numbers refer to the original submission. Section, table and figure numbers refer to the revised PDF.

Three terms are used below. Held-out accuracy is accuracy on inputs the model never saw during training. An affine rule is a straight-line rule such as x↦11xx \mapsto 11x. The discrete logarithm of a number is its exponent with respect to a fixed base number, so renaming numbers by their discrete logarithms turns multiplication into addition.

The abstract is written poorly with no clear structure. I couldn’t understand the key points of the paper from it.

  • The abstract makes three points (lines 014 to 029). In plain terms:
    • The way a task is written decides whether the model generalizes. We take one arithmetic task and rename its inputs and outputs by their discrete logarithms. The task, the training examples and the test inputs stay the same. Held-out accuracy rises from 0.003, which is chance level, to 0.459. On a second task it rises from 0.000 to 0.853. If only the inputs or only the outputs are renamed, nothing improves.
    • Standard measures of task difficulty do not explain this. We build two versions of a task that have the same algebraic degree and are equally close to an affine rule. The model reaches 0.977 on one and 0.006 on the other.
    • Training matters as well. The written form decides whether a simple solution exists, and weight decay decides whether training finds it. Accuracy is 0.060 without weight decay and 0.459 with it.
  • The revised abstract presents the same points in the order problem, approach, findings, implication, and gives the chance level next to the numbers.

The paper considers a very limited scope, and shows that better representations lead to generalization. I do not understand the significance of these points. It is already known that the feature representations are important for generalization in phenomena such as grokking and weak-to-strong.

  • We agree that the importance of representation is known. The paper asks a more specific question: when the same task is written in two ways and the model generalizes on only one of them, which property of the written form is responsible? The literature mentioned studies the features a model learns internally, for one fixed way of writing the task.
  • “Good features help” does not explain our main result. We gave the model the helpful logarithmic naming on the input side only, and separately on the output side only. In both cases accuracy stays at or below 0.003, in all four settings of Table 1 (lines 195 to 199). Accuracy rises only when both sides are renamed, because only then is the rule that connects input to output an affine rule.
  • Standard descriptions of the task do not explain it either. Two versions of a task with the same degree, and equally close to an affine rule, reach 0.977 and 0.006 (lines 245 to 252).
  • This matters for published results. The grokking benchmark of Power et al. (2022) gives each number its own token. When we write the same numbers as digits, two of its nine operations generalize instead of eight. We predicted this before training (lines 419 to 427).
  • The setting is narrow on purpose. Only tasks fixed by theorems let us change the written form while the task and the test inputs stay exactly the same. Within this setting, the effect holds on two theorems, three model types, four number bases and a second number system (lines 399 to 412).

The solution that the papers proposes is to just provide the good features for learning the task to the model.

  • The paper does not propose this. Lines 204 to 205 state that the network does not compute the logarithm; the presentation supplies it, “which makes the intervention a diagnostic rather than a recipe”.
  • The renaming is done by us before training, as an experimental control. It lets us show the model the same task in four written forms, on identical data, and see on which form it generalizes. It is a way of testing a model, not a way of improving one.
  • The revised introduction and Section 4.2 keep this statement, and Section 2 explains how the paper differs from work on learned features.
AuthorsOpenReview ↗

Response to Reviewer UhaN (2/2)

The paper does not explain how we can use any of the findings in the paper on a broader sense: how can we find the good representations on other and general domains?

  • The findings are meant as a check to run when evaluating a model. Lines 480 to 482 state it: before concluding that a model cannot learn a task, measure how simple the task is in the form the model is actually shown.
  • The check works for any task whose full table of inputs and answers can be written down. From that table alone, before any training, we compute how close the task is to an affine rule (Section 3).
  • We already used it outside our own tasks. It predicted which operations of the benchmark of Power et al. (2022) generalize when numbers are written as digits (Section 4.5, Table 2).
  • It also gives a check for benchmark builders. In our benchmark, 138 tasks with different names turned out to be only 129 different tasks, because some share exactly the same table of answers. Duplicate checks based on text cannot see this (lines 150 to 156). We release the detector.
  • Finding a good written form automatically is a different problem. Our results motivate it, but the paper does not claim to solve it. Section 7 states the recommendations for benchmark design, and Section 6 keeps the scope explicit.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the paper and the revision

We sincerely thank the Area Chair and the reviewers for their time and careful assessment of our work. This comment states the paper’s question and findings in brief, addresses the concern that remains after the discussion, and lists the updates in the revised PDF. Line numbers refer to the original submission. Section, table and figure numbers refer to the revised PDF.

  • Research question: a transformer can learn one way of writing an arithmetic task and fail on another way of writing the same task. Which property of the written form decides this?
  • Method: we use arithmetic tasks whose answers are fixed exactly by theorems. We write the same task in different ways, for example by naming each number by itself or by its discrete logarithm, and train the same model on the same examples. Each experiment changes one thing only. The renaming is done by us before training and is never computed by the model, so the study is a diagnostic test, not a method for supplying features.
  • Findings, unchanged from the submission:
    • The written form decides generalization. In the usual form, accuracy on unseen inputs is 0.003, which is chance level. When both the inputs and the outputs are renamed by their discrete logarithms, the same task becomes a simple affine (straight-line) rule and accuracy rises to 0.459. On a second theorem it rises from 0.000 to 0.853. Renaming only the inputs, or only the outputs, gives no gain (Table 1; lines 195 to 199).
    • Standard measures of task difficulty do not explain this. Two versions of a task with the same algebraic degree, and equally close to an affine rule, reach 0.977 and 0.006 (Figure 2; lines 245 to 252).
    • Training matters as well. On the same task, accuracy is 0.060 without weight decay and 0.459 with it (Figure 4; lines 380 to 395).
    • The effect appears on a published benchmark. When the numbers are written as digits, two of the nine operations of Power et al. (2022) generalize, against eight in the published runs. We predicted this before training (Table 2; lines 419 to 427).
  • Implication: when a model fails on an arithmetic benchmark, the failure should be attributed to the model only after the way the task is written, the digit encoding and the training have been controlled.
  • On the concern that remains after the discussion, novelty and scope:
    • What is new: prior work shows that representation matters. This paper shows which property of the written form matters, rules out the obvious candidates (algebraic degree, closeness to an affine rule), and predicts outcomes before training.
    • Relation to grokking: that work keeps one way of writing the task and studies how generalization appears inside the model. We keep the model and the training fixed and change only how the task is written. The benchmark finding above shows what this adds: a published grokking result changes with a choice that had not been tested.
    • Not a restatement of “good features help”: that statement predicts neither the failure of one-sided renaming nor the gap between the two matched tasks.
    • Scope: arithmetic is used because only there is the full table of answers known exactly, so the written form can change while the task stays identical. The same test applies to any task with known answers. The effect holds across two theorems, three model types, four number bases and a second number system (lines 399 to 412; Appendix F).
    • These claims rest on results that were already in the original submission. The added analyses only answer reviewer questions.
  • Updates in the revised PDF:
    • The text is reorganized into Related work, Approach, Experiments, Analysis, Discussion (section map below). The title, data and code are the same, the contributions C1 to C3 cover the same experiments, and no reported result has changed.
    • Two analyses were added at the reviewers’ request: why one affine task is harder than another, in Section 4.4 and Appendix C.1 (Reviewer ujSB), and a test with a different number base plus a combined predictor, in Appendix D.1 and D.2 (Reviewer J7GS).
    • Section 2 discusses the three works on group representations suggested by Reviewer ujSB.

Where each original section is now:

  • Original Section 2, Setup: now Section 3 (Approach) and Section 4.1
  • Original Section 3, Intervention I: now Section 4.2
  • Original Section 4, Intervention II: now Section 4.3
  • Original Section 5, Prediction: now Section 5.4; dilated tasks in Section 4.4
  • Original Sections 6 and 6.1, Network and weight decay: now Sections 5.1, 5.3 and 5.2
  • Original Section 7, Extent of the effect: now Sections 4.2 and 4.5, Appendix F
  • Original Section 8, Related work: now Section 2
  • Original Section 9, Scope and limitations: now Section 6, with the four limitation paragraphs retained

We thank the reviewers again for the constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.