acceptodds
Under review as a conference paper at ICLR 2027

Small Transformers Learn Kepler's Equation with a Number Line and an Eccentricity Register

Abstract

We train a family of 210 small transformers to solve Kepler's equation , then reverse-engineer the algorithm they learn. To our knowledge this is the first circuit-level account of a continuous, transcendental equation. The task is trigonometric, yet the digit embeddings form a number line, not a circle. Attention writes into a single residual-stream direction largely hidden from the readout, the e-register. The MLP reads it and writes the corrections. The output carries harmonics of to . Scale the register and the corrections scale with it; set it to zero and the model tracks its own circular orbit. The line and the register hold across the 210-model family. We release the models and the analysis code as a testbed: the register's variable is known to be , so a method that claims to find hidden computation can be scored against it.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, circuit analysis, transformers, Kepler's equation, number representations, activation patching, hidden computation, small models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vG1YOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper analyzes the internal representations and computational mechanisms of a small Transformer trained to solve Kepler's equation, M=E−esin⁡EM=E-e\sin E. The authors observe a number-line-like structure in the numerical token embeddings and propose that attention reads significant digits and transfers information about the eccentricity ee into a specific vector direction, termed the ee-register, which the MLP then uses to produce nonlinear corrections. They validate the causal role of this direction through removal and scaling interventions and analyze how the final LayerNorm affects the relationship between output contributions and actual intervention effects. The authors also investigate the consistency and limitations of their findings across different model configurations and release 240 model checkpoints and analysis code as an experimental environment for evaluating interpretability methods.

Critical strengths
  1. The paper validates the causal role of internal representations beyond identifying correlations. Rather than merely identifying a direction associated with ee through regression, the authors examine whether this component is used in subsequent computations through removal and scaling interventions, along with random-direction controls. Although these interventions are approximate, they provide more convincing evidence than representation visualization alone.

  2. The paper provides extensive validation across different settings and releases substantial resources. The authors investigate the consistency of the main observations and their exceptions across multiple random seeds, model widths, training durations, and architectural variations. They also release the checkpoints and code for the 240 models analyzed. Although the scope of the task is narrow, these resources provide a solid foundation for reproducing the results and conducting further interpretability research.

Critical weaknesses
  1. The new insights gained are limited relative to the level of detail in the analysis. The paper identifies internal variables and validates their roles through interventions, but this general approach is already well established. In particular, prior work on the harmonic oscillator has also analyzed intermediate representations of internal computations and performed interventions that modify physical parameters. While I acknowledge the detailed analysis of computational pathways in this paper, it remains unclear what new insights into neural network computation are provided beyond adding the Kepler equation as another case study. Furthermore, the presence of an approximate decomposition that does not fully explain the precise computation makes it difficult to assess the scope of the contribution, especially given the strong claim of having "read the algorithm."

  2. The paper lacks sufficient control experiments to assess the significance of the identified mechanism. In an independent exploratory experiment, I found that a GELU MLP with only 1,153 parameters, without attention or LayerNorm, achieved similar median accuracy and exhibited a partially manipulable direction corresponding to ee in its first hidden layer. This does not contradict the paper's analysis of the Transformer, but it highlights the need for a baseline against which to assess how much of the observed phenomenon is expected from general function learning. Even if the authors do not claim that the identified mechanism is specific to Transformers, without comparisons to simpler control models, it is difficult to determine why the mechanism discovered in this case is particularly significant.

  3. The significance of the proposed evaluation environment remains more of a proposal than an empirically demonstrated contribution. Releasing the model and code is useful, but their availability alone does not establish the research value of a new evaluation environment. Although the paper suggests that this environment can be used to evaluate methods for identifying hidden computations, it does not sufficiently compare existing methods to demonstrate which types of failures the environment can distinguish or what improvements it can enable. Given that the current results focus on a single equation and primarily on one-layer models, there is insufficient evidence to establish a broader methodological contribution beyond a detailed case study.

Questions for authors
  1. What is the scientific motivation for choosing the Kepler equation? Beyond the fact that its solutions can be computed numerically with high accuracy, what new mechanisms did the authors expect this equation to reveal compared with previously studied physical or mathematical tasks? Have the authors performed the same regression and intervention analyses on a linear control task such as E=aM+beE=aM+be? If not, how can they distinguish whether the identified ee-register is a feature associated with solving a nonlinear equation or simply reflects a general mechanism for propagating input variables?

  2. Do the findings generalize beyond the specific structure of the trigonometric correction? For example, in tasks involving higher-order polynomial terms such as E=M+eMkE=M+eM^k or interaction terms such as e2M+eM2e^2M+eM^2, is it still possible to identify a single direction corresponding to ee and reproduce counterfactual outputs by scaling that direction? If the current findings are limited to a single equation, please clarify the scope of the claims that the authors consider generalizable.

  3. What additional insights does this study provide beyond the phenomena observable in a simple MLP? In my limited control experiment, a GELU MLP with only 1,153 parameters, without attention or LayerNorm, achieved similar median accuracy. Applying regression using the same 11 functions and performing interventions on independent inputs revealed a partially manipulable direction in the first hidden layer that explained 90–97% of the variation associated with ee. Although differences remain in maximum error and intervention fidelity, what is the central insight of this paper in light of these findings? In particular, please clarify what new knowledge is gained from the detailed analysis of computational pathways beyond identifying variable representations and demonstrating their manipulability.

2 comments

Thank you for the detailed review, and for running your own MLP control. We ran our own version, and it agrees with yours in outline. Our general comment lists the new experiments.

"What is the scientific motivation for choosing the Kepler equation? ... what new mechanisms did the authors expect this equation to reveal compared with previously studied physical or mathematical tasks?"

The closest circuit-level account is modular addition (Nanda et al., 2023), a discrete task. We wanted a continuous one with a known classical answer. Kepler's equation is one of the oldest problems in mathematical physics, with a long line of solution methods, so the learned algorithm can be placed against classical ones (§4.6). We also like it. We expected the trigonometric task to push the model to a circular code, as modular addition does. It does not. The digits form a number line, and no wrapped variant builds a circle (§4.2). We also hoped for a new mechanism, or one that mirrors a classical method more closely. The output approximates the Fourier–Bessel series, as any accurate solver's must, but the MLP builds its corrections from a learned spline basis (§4.4). And the new runs show the main structures are not specific to Kepler. Your point about the harmonic-oscillator work stands: the intervention method is not new.

"Have the authors performed the same regression and intervention analyses on a linear control task such as E=aM+beE = aM + be? If not, how can they distinguish whether the identified e-register is a feature associated with solving a nonlinear equation or simply reflects a general mechanism for propagating input variables?"

Yes, with a = b = 1, with the final LayerNorm on and off. The register appears in 5 of 5 and 4 of 5 seeds. We set a bar before the run. If Kepler's register did something a linear task does not need, the LN-off linear model should pass e straight to the readout and barely use the MLP. The bar was |cos| of at least 0.7 and an MLP ablation that raises error under 3×. We measured 0.487 and 2316×. So we cannot distinguish the register from a general mechanism for propagating an input.

"In tasks involving higher-order polynomial terms such as E=M+eMkE = M + eM^k or interaction terms such as e2M+eM2e^2 M + e M^2, is it still possible to identify a single direction corresponding to e and reproduce counterfactual outputs by scaling that direction?"

We ran E = M + e(M/π)³, which keeps E odd in M and |E − M| ≤ 1, as in Kepler. We did not run the interaction terms. One direction carries 86% to 99% of the write's variation across e in all 10 seeds. Scaling it reproduces counterfactuals only partly. With the final LayerNorm on, zeroing it closes at least half the gap to the model's own e = 0 output in 3 of 5 seeds, and halving it closes 0.245 to 0.739 of the gap to its e/2 output (Kepler: 0.656 to 0.849). These are fractions of a small gap: cubic's do-nothing gap is 0.044 rad against Kepler's 0.257, and in radians its residuals are below Kepler's. With LN off, scaling fails in all 5 seeds. On scope: we would claim neither the number line nor the register as a property of Kepler. The line appears on every task with this input format. The register appears on most seeds but not reliably.

"What is the central insight of this paper in light of these findings? In particular, please clarify what new knowledge is gained from the detailed analysis of computational pathways beyond identifying variable representations and demonstrating their manipulability."

We ran your control with hidden widths 27, 21 and 21 (1,153 parameters), float inputs, 5 seeds. It beats our primary configuration at the median (4.9e-4 vs 1.9e-3 rad) and loses at the worst case (0.238 vs 0.086). In its first hidden layer, one direction carries 72% to 82% of the variation across e, below the 90–97% you report. We guessed the widths, so our networks or measure may differ. That direction is manipulable in 3 of 5 seeds (median gap closed 0.550, against 0.828 for the transformer). Our bar, also set before the run, was that the transformer's register is stronger if at least 4 of 5 MLP seeds fail. Only 2 did. You are right: a manipulable direction is not special to the transformer.

On the central insight, we concede it is narrower than the paper claims. The pathway analysis adds how the model uses e: attention passes it as one direction the readout barely sees, and the MLP turns it into e·sin(nM) corrections. That pattern is not specific to Kepler. And the account is approximate. Replacing attention's write with its 11-term fit raises median error from 1.5e-3 to 5.3e-2, so "read the algorithm" overstates it. We read its structure, not its precision.

On the evaluation environment.

We agree it is a proposal more than a demonstrated contribution. The ridge and PCA comparison in the general comment is a first test, and we read it as weak support.

Thank you again.

Thank you for running the requested control experiments, including the plain MLP baseline and the linear/cubic tasks. The follow-up results demonstrate that the primary algorithmic claims of the paper do not hold: both the number line and the ee-register appear in simpler baselines and linear tasks, reflecting general input-propagation mechanisms rather than properties specific to Kepler's equation or Transformer architectures. Furthermore, the register fails to generalize reliably beyond a single layer, and the proposed evaluation environment remains undervalidated without systematic comparisons across existing interpretability methods to demonstrate its discriminative value. Given that the core contributions are largely invalidated by these findings, I maintain my recommendation for rejection and keep my score at 2.

Reviewer_47CuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper trains a series of transformer models to solve Kepler's equation. Then, the paper reverse engineer the computation inside the neural network to show, how the equation is solved.

Critical strengths
  • The tasks of reverse engineering the circuitry of trained neural networks is generally difficult.
  • Reverse-engineering neural network might uncover interesting phenomenons.
Critical weaknesses
  • One of the main weaknesses is limited novelty. I am familiar with the work of Nanda, 2023 and I was therefore not that surprised by results of this paper. I critically miss section explaining methods used to reverse engineer the network, because describing these methods enable me to transfer the methods to different problems. Here, I have not learned anything.
  • The section 4 about results is not easy to read. Despite my honest try, I have missed the overall goal and got lost in text without understanding details.
  • The biggest omission is motivation. Why is Kepler's equation so important and why is it so different from all other tasks being done.
Questions for authors
  • What is the motivation for the work and why is the Kepler's equation so different from other problems?
  • Taking into the account that neural networks very rarely learns algebraic solution, what makes transcendental equation so special and difficult?
  • Can you describe the methods used to learn the circuitry of the neural network?
  • Why is the result so important for the community?
  • What the result teach me about the neural networks / transformers?
  • How was the architecture of the transformer selected?
2 comments

Thank you for the review. We agree the presentation, §4 especially, is hard to follow, and we lay out the algorithm step by step below. Our general comment lists new experiments that bear on motivation and importance.

"What is the motivation for the work and why is the Kepler's equation so different from other problems?"

The closest circuit-level account is modular addition (Nanda et al., 2023), a discrete task. We wanted a continuous one with a known classical answer. Kepler's equation is one of the oldest problems in mathematical physics, with a long line of solution methods, so the learned algorithm can be placed against classical ones (§4.6). We also like it. We expected the trigonometric task to push the model to a circular code, as modular addition does, and hoped for a new mechanism or one closer to a classical method. Instead the digits form a number line, and no wrapped variant of the task builds a circle (§4.2). And the MLP builds its corrections from a learned spline basis, not from a classical series (§4.4). On "so different": less than we thought. The two main structures we report also appear on E = M + e and E = M + e(M/π)³.

"Taking into the account that neural networks very rarely learns algebraic solution, what makes transcendental equation so special and difficult?"

It is harder for the same model: median error 1.9e-3 rad on Kepler, against 4.9e-4 on the linear task and 1.0e-3 on the cubic one. But the structures we report also appear on those tasks, so we do not claim a transcendental equation needs a special mechanism. The model does not learn an algebraic solution either. It adds learned e·sin(nM) corrections to M, and its output approximates the Fourier–Bessel series.

"Can you describe the methods used to learn the circuitry of the neural network?"

The algorithm, in steps:

  1. The digit embeddings lie on a straight, evenly spaced line (§4.1).
  2. An attention head reads the leading digits of M and e as quantities (§4.3).
  3. Attention writes M toward the output, and writes e into one direction, the e-register, that the readout barely sees (§4.5).
  4. The MLP reads the register and M and computes the e·sin(nM) corrections (§4.4).
  5. The output is M plus those corrections, and its harmonics track the Fourier–Bessel coefficients (§4.6).

You are right that the paper has no methods section. The methods all come from prior work. The embedding analysis follows Nanda et al. (2023): PCA, Fourier and geometry tests on the digit embeddings. The rest comes from other work: the QK and OV factoring of attention (Elhage et al., 2021), mean-ablation (Wang et al., 2023), activation patching (Vig et al., 2020; Meng et al., 2022), and distributed alignment search (Geiger et al., 2024) as a cross-check. We also regress each component's output on an 11-term trigonometric library.

We ran the same register recipe unchanged on the two new tasks. Regress the attention write at the readout position on a library of functions of the inputs. The fit gives one direction per library term. Take the direction for one input, and check that it carries most of the write's variation across that input. Then scale it to zero and to half, and compare against the model's own output with that input set to zero and to half.

"Why is the result so important for the community?"

We concede the case is weaker than the paper argues, since the two main structures are not specific to Kepler. What remains is a causally tested account of a small model on a continuous transcendental equation, which other methods can be checked against. That use is only weakly shown so far.

"What the result teach me about the neural networks / transformers?"

Each point is shown only in small models. On all three tasks, a transformer given decimal digits encodes them as a number line. It usually passes a scalar input like e as one direction in the residual stream. And the final LayerNorm can hide that direction from direct logit attribution.

"How was the architecture of the transformer selected?"

For legibility, while keeping it a transformer. We kept digit tokens as input. With two floats, attention has almost nothing to route (App. B), and binning each value into one token would not resolve the inputs finely enough. We kept the final LayerNorm, which we consider part of a transformer. It improves accuracy (median error 1.9e-3 rad with it, 3.7e-3 without), and on Kepler it hides the register from direct logit attribution. We simplified in two places: one layer, and one scalar output from a linear readout. The primary uses the smallest width that avoids the degenerate low end at width 4 (App. I): width 8, 2 heads, GELU, MSE loss, 1,153 parameters (§3.2). The 210-model family varies width, training steps, loss, output map, activation and head count, to check that results do not hinge on this one choice.

Thank you again.

Thank you very much for the answers, though I have to admit that the answers does not convince me to change my score. I stay with my original objections.

Reviewer_7F2hOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains a family of 210 small one-layer transformers (the primary has 1,153 parameters) to solve Kepler’s equation from digit-tokenized inputs, then reverse-engineers the learned algorithm. The main findings are:

The digit embeddings form a linear “number line” rather than a circle, even when the task is made periodic (wrap study). Attention routes the leading digits, and the MLP computes the e·sin(nM) corrections using a spline-like basis. Attention writes e into a single residual direction (the “e-register”) that is nearly orthogonal to the readout. The MLP consumes this direction, and scaling it approximately sets the model’s effective e.

These properties are verified across the model family via ablations, patching, and distributed alignment search. The authors release all models as a testbed for hidden-computation detection methods, since the hidden variable is known.

Critical strengths
  1. Rigorous causal evidence, replicated across a model family. The mechanistic claims rest on ablation, activation patching, and an independent distributed alignment search cross-check, not on probing alone. Each claim is checked across 210 models with explicit controls, including random directions and models that ignore e. This level of verification is stronger than typical single-model case studies.

  2. A concrete demonstration that standard attribution methods can miss a known hidden variable. Because the identity of the hidden variable is known, the paper can show directly that readout-based methods such as direct logit attribution fail to see the e-register and misrank neurons, and that LayerNorm scaling is the cause. The released models offer a ground-truth setting for testing such methods, which is useful to the interpretability community.

Critical weaknesses
  1. The findings cannot be attributed to the task rather than to the fixed architecture and tokenization.

    • Several results are, by the authors' own account, properties any accurate solver must have: the 10× M/e dependence ratio, the Bessel spectrum, and the 10× per-digit sensitivity falloff.
    • Others follow largely from the setup. A linear digit code is the natural encoding for a magnitude task. In a one-layer model whose readout position is a constant token, e can reach the MLP only through attention. Since e is a scalar, a rank-1 register off the readout direction is the most economical way to pass it.
    • The 210-model family varies only hyperparameters (width, steps, loss, activation, head count), while the architecture, input format, and task stay fixed. Its consistency may therefore reflect these constraints rather than a non-trivial learned algorithm.
    • Two observations support this. Appendix B acknowledges that continuous inputs would remove the number line, which suggests the representation is driven by tokenization. And once the architecture changes, the structure becomes unreliable: the register holds in only 2 of 5 two-layer seeds.
    • Prior work (Zhong et al., 2023) shows that the same task can yield different algorithms under different architectures, yet this is not tested here.

    As a result, it is unclear which findings belong to Kepler's equation and which belong to this specific model setup.

  2. Limited scope, and the testbed claim is under-validated.

    • All results come from tiny one-layer models on a single equation, and relevance to LLMs is argued but not tested.
    • The release is positioned as a benchmark for hidden-computation methods, yet only DLA, readout projection, and DAS are evaluated. Common methods such as linear probes and sparse autoencoders are not tested.
    • The register is identified by regressing onto a library that already contains e, so its discovery relies on prior knowledge of the variable.

    The benchmark's value therefore remains to be demonstrated.

Questions for authors

Clarifications:

  1. Which of the reported structures would you expect any accurate one-layer solver with this input format to exhibit, and which do you consider non-trivial findings?
  2. The e-register is identified by regressing onto a library that already contains e. How would a researcher discover it without prior knowledge of the variable? Does any unsupervised method recover it?
  3. In the three 2-layer seeds where the register fails, where does the information about e go? Is it distributed across layers, or does the model use a different mechanism?

Would change my assessment:

  1. Task vs. architecture vs. tokenization. Can you disentangle which findings come from Kepler's equation itself and which come from the fixed setup? For example, do the number line and the e-register still appear with continuous inputs, other architectures (MLP, RNN, deeper transformers), or another transcendental equation?
  2. Depth. Does the e-register appear reliably in deeper models (e.g., 2–4 layers, with more seeds), supported by a joint multi-layer patch?
  3. Testbed value. Can you evaluate additional interpretability methods on the released models, such as linear probes and a sparse autoencoder, to show the testbed discriminates between methods?

I would consider raising my score if the authors can show that the main mechanisms are attributable to the task rather than to the specific architecture and tokenization, and that they hold beyond one layer.

2 comments

Thank you for the careful review, and for crediting the causal evidence. Our general comment lists the new experiments used in some answers below. We agree with both weaknesses you raise. The answers below cover them, except relevance to LLMs, which we agree is argued and not tested.

"Which of the reported structures would you expect any accurate one-layer solver with this input format to exhibit, and which do you consider non-trivial findings?"

Forced by the task:

  • the 10× M/e ratio
  • the Bessel spectrum and the per-digit falloff, as you note

Not specific to Kepler:

  • the number line. It appears in all 20 new-task seeds.
  • the e-register. It appears in 9 of 10 linear seeds and 3 of 10 cubic seeds. It is not specific to the transformer either. On Kepler, 3 of 5 seeds of an MLP with no attention have it.

Your explanation fits the line and most of the register results. Cubic is the exception, at 3 of 10 seeds.

Tested, with a mixed result:

  • the final LayerNorm hiding the register from direct logit attribution. It holds on Kepler. On cubic the register is hidden with the LayerNorm on, but with it off no seed has a register. On the linear task it fails.

Not tested on other tasks:

  • the wrap study (no periodic variant builds a circle).

So the two headline structures are not specific to Kepler. What we would still call non-trivial is the attribution failure you credit. Its LayerNorm explanation holds cleanly on Kepler only.

"How would a researcher discover it without prior knowledge of the variable? Does any unsupervised method recover it?"

Not without knowing e. The standard unsupervised method would be a sparse autoencoder, which we did not run. We have two partial answers. The top principal component of the attention write's variation across e at fixed M matches the register (cosine at least 0.991 in all 210 family models), but grouping by e assumes you know which input is e. Plain PCA over all inputs has a component near the register (median |cos| 0.858 over the 7 DAS models), but it is never the top component, and we used e to pick it.

"In the three 2-layer seeds where the register fails, where does the information about e go?"

It differs by seed, and none of the three forms a register at either layer. Patching e out of both layers' attention writes jointly leaves most of the e·sin M products. In the two register seeds, the layer-0 patch alone leaves under 7%.

Seede·sin M left, joint patchGap closedBest single MLP ablation (e·sin M left)
097%0.0228% (layer-0 MLP, e digits)
157%0.56775% (layer-0 MLP, e digits)
272%0.44350% (layer-0 MLP, readout)

In seed 0, mean-ablating the layer-0 MLP at the e digits removes 92% of the products. Those are the only digit positions that see both M and e. The ablation also breaks the model (median error 0.27 rad). Seed 1 spreads e across both layers. Seed 2 is in between.

Task vs. architecture vs. tokenization.

Partly. See the general comment for experiments run: we changed the task and the architecture. The MLP takes floats, so it changes tokenization and architecture at once. We did not run a transformer on continuous inputs, an RNN, deeper transformers on the new tasks, or another transcendental equation. The point from Zhong et al. stands.

Depth.

Not resolved. We did not train deeper models or more seeds. The register holds in 2 of 5 two-layer seeds, and the joint patch above does not find it in the other 3.

Testbed value.

Ridge probe and PCA at the attention write of the 7 DAS models, scored on the same swap test DAS uses (medians):

Method|cos| to registerGap closed
Regression (the register)10.866
DAS, same-M sources0.9720.920
DAS, random sources0.8430.840
PCA0.8580.461
Ridge probe0.2630.001
Random direction0.2110.045

The methods disagree, which is what a testbed should show, but we read that as weak support for it. The paper predicted a linear probe would find the register. Ridge decodes e well (R² 0.80 to 0.995) but does not find the direction. That matches a known effect in probing: a probe's weights are a decoder and need not point along the direction the variable is written in (Haufe et al., 2014, "On the interpretation of weight vectors of linear models in multivariate neuroimaging", NeuroImage). DAS with same-M sources is trained on the same swap it is scored on. We have not explained why PCA lands close but steers less. We did not run an SAE.

On raising the score.

We cannot meet either condition. The line and the register appear on other tasks with the same format, and the register holds in only 2 of 5 two-layer seeds.

Thank you again.

Thank you for your candid response and for conducting the additional experiments.

However, the new experimental results confirm my primary concerns. Both the "number line" and the "e-register" appear in these new tasks—as well as in attention-free MLPs—indicating that they reflect general properties of the model architecture rather than mechanisms specific to the Kepler model. Furthermore, the explanation provided for LayerNorm does not hold in contexts outside of Kepler. The results from the ridge regression probes contradict the predictions made in Section 5 of the paper. Moreover, the experimental framework still lacks analysis based on sparse autoencoders (SAEs), nor does it offer a method for feature discovery that does not rely on the "e-register" construct.

Given the current results, I will maintain my original score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Responses and Followup Experiments

Thank you very much for the thoughtful reviews! We ran a few additional experiments to make our responses more concrete, and in general agree with the review feedback.

We ran 20 models: 5 seeds each, changing the task to linear (E = M + e) and cubic (E = M + e(M/π)³) with the final LayerNorm (LN) on and off. We find the line in every seed and the register in 12 of 20.

We also trained 5 seeds of a plain 1,153-parameter MLP (no attention, no LN, float inputs) on Kepler's equation and observe the register in 3 of 5. The MLP has no digit embeddings, so the number line does not apply.

ArmMedian errorMax errorSeeds with registerSeeds on lineMedian gap closedMedian register-readout |cos|
(Paper's models) Kepler, LN on1.9e-30.0865/55/50.8280.065
(Paper's models) Kepler, LN off3.7e-30.1375/55/50.8350.372
E = M + e, LN on4.9e-40.0145/55/50.9350.630
E = M + e, LN off6.8e-40.0154/55/50.8850.487
E = M + e(M/π)³, LN on1.0e-30.0263/55/50.6200.055
E = M + e(M/π)³, LN off2.3e-30.0730/55/5−2.5260.865
Kepler, MLP (no attention, no LN)4.9e-40.2383/5n/a0.550n/a
  • Seeds with register means: one direction carries at least 72% of the write's variation across e, and the zero-scale patch closes at least half the do-nothing gap.
  • A negative gap closed means the patch lands farther from the model's e = 0 output than doing nothing.
  • Values are medians over 5 seeds. Errors are in radians and max error is from the worst seed.
  • The paper's 1.5e-3 median error is seed 0. The 1.9e-3 here is the median over 5 seeds.
  • Register-readout |cos| is the overlap between the register and the direction the readout reads (w_eff in the paper). Low means hidden from direct logit attribution.

We ran the joint two-layer patch on the 3 of 5 existing seeds from the paper where the single-layer e patch failed to find a register. Removing e at both layers still left 57% to 97% of e·sin M products. So those 3 two-layer seeds don't have an obvious e-register.

We ran a ridge probe and PCA on the same 7 models from the paper where we already ran DAS, and scored each direction on the swap test DAS uses. The register itself closes a median 0.866 of the gap on that test. The ridge probe decodes e well (R² 0.80 to 0.995) but closes a median 0.001. PCA has a component near the register (median |cos| 0.858, chosen using e), but it only steers about half as well as DAS (median gap closed 0.461, against 0.840 for DAS with random sources and 0.920 for DAS with same-M sources).

Conclusions

We concede that the e-register is not unique to Kepler or a transformer, but not guaranteed either (cubic with LN off shows it 0 of 5). Its existence is consistent with a mechanism to route the input value. Similarly the line is not unique.

We concede the e-register is not shown beyond one layer. 2 of 5 two-layer seeds have it and the joint two-layer patch doesn't recover it in the other 3.

We agree that as submitted our claim as a testbed is undervalidated. Running ridge and PCA shows they do give different results than DAS. However, we read this as weak support. Ridge's failure matches a known effect in probing: a probe's weights are a decoder and need not point along the direction the variable is written in (Haufe et al., 2014, NeuroImage). DAS with same-M sources is trained on the same swap it is scored on. We have not worked out why PCA lands close but steers less, and SAE was not run.

The paper and the anonymous repository are not updated with these findings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.