acceptodds
Under review as a conference paper at ICLR 2027

Shared Conditional Transformations of Arithmetic Directions in Language Models

Abstract

Mechanistic interpretability seeks to explain model behavior, in part by identifying feature directions in internal representations and testing their effects through interventions. Yet a central question is whether shared structural principles can explain and predict how representations change across contexts. In controlled addition, we find that arithmetic directions exhibit condition-dependent variation described by low-dimensional orthogonal transformations shared across values at certain layers. These transformations predict target-condition directions for values excluded from fitting. At the reporting layer of each of four language models, hidden-state interventions using directions predicted by these shared transformations achieve exact complete target-answer match rates of 71.13%–99.43%, compared with 29.66%–41.68% using untransformed source directions. In three models, these effects approach those of directions estimated directly under the target condition. The shared-relation pattern also recurs after independent refitting in a tens-column setting. We further characterize a more constrained structure in terms of direction-level approximate conditional equivariance: powers of a shared orthogonal operator approximate the transformations associated with cyclic condition shifts. Predictions under this description induce expected output changes at multiple tested layers. Together, these findings suggest that interpretable structure can lie not only in feature directions themselves, but also in the shared, predictable relations organizing their variation across contexts.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, feature directions, linear representation hypothesis, activation steering, representation geometry, arithmetic in language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tAikOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

According to its introduction, this submission aims to analyze how feature directions change depending on context:

a feature direction identified in one input context may not transfer directly to another (line 64)

We ask whether this context dependence has shared structure: when the information under study takes multiple values, do the corresponding directions change according to a common rule as one moves between contexts? Can a rule learned from some values predict directions for the remaining values? (lines 69-72)

The submission studies this issue in a simple arithmetic setting in which two numbers are added. Prior work has found that number values can be probed with high precision in such a setting, but causal interventions aiming to steer model output to counterfactual sums have proven more difficult. The paper positions itself as demonstrating an effective steering method for number representations.

Critical strengths

Within its narrow experimental scope, the proposed method is effective for steering number representations in LMs. More generally, studying potential regularities in how context affects feature directions is an interesting research direction.

Critical weaknesses
  1. The submission's framing doesn't fit its content. In contrast to the expectations raised by the introduction, the experiments are not concerned with context, but what the authors call "conditions", which are simply the ones and tens digits of the addends. There may be a sense in which the digits of the addends can be the "context" of the addition, but that is not how the word is commonly used.

  2. The contributions are too thin. While precise steering of number representations on a single prompt is a focused contribution, this is not enough for a long paper at a top tier conference. To make this a stronger submission, the authors could, for example, demonstrate that their Procrustes-based method for transferring edit vectors across contexts (or conditions) works in other settings or on other tasks.

  3. It is possible that I missed other contributions, but the empirical part of the paper is an agglomeration of statements without much apparent structure to guide the reader. To give only two examples out of many, a typically report of results looks like like

5.3 GENERALIZATION ACROSS VALUES HAS A MODEL-SPECIFIC LAYER RANGE Canonical interventions yield complete target-answer match rates of 94.78%–99.43%

without any apparent relation to the preceding context, making it difficult to understand how this fits into the paper's overall narrative. Another common issue are sentences like

The comparison with Common-GPA tests whether powers of one operator can retain the output effects of condition-specific frames in the same common basis. At Llama L21, Step-power and Common-GPA yield similar match rates (99.21% and 99.10%) with 66 and 594 continuous transfor- mation degrees of freedom, respectively, illustrating that a more compact description can preserve the observed output effects at this layer

This hyper-precise scoping of claims might be valued by AI reviewers, but catastrophically lowers the signal-to-noise ratio for human reviewers.

Questions for authors

none

Reviewer_TKvHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how hidden-state directions for addition change in four LLMs. Orthogonal maps (Canonical), or powers of a single shared operator G (Step-power), predict held-out-value directions under the target condition. Steering with these predicted directions reach very high match rates in three models at post-hoc selected layers.

Critical strengths
  • Conceptual: The idea is clear and appealing: interpretable structure may lie in how hidden-state directions change across different contexts, not only in the directions themselves.
  • Steering: Held-out-value prediction combined with behavioral interventions is a stronger test than geometry alone. The paper also shows that cosine alone can be misleading.
  • Evaluation: Four models and careful evaluation.
Critical weaknesses

Substance

  1. Scope: The title says "arithmetic", but only addition is studied. Also the claim is that this is a general pattern, but maybe the case of numbers/arithmetics is itself especially well-suited for the authors claim. Work that the authors cite shows that LLMs represent numbers with periodic (Fourier/helical) features, and that addition is carried out by rotating these circles (Zhou et al., 2024, Kantamneni & Tegmark, 2025). Since the conditions here are defined cyclically (mod 10), a shared rotation across conditions is close to what these accounts would predict. Most other concepts lack this ordered, cyclic structure, so it is unclear whether the finding carries over. It would help to discuss this, or to test whether G acts within the known Fourier planes.
  2. What is represented? At the final token, the directions may encode the answer rather than a separate "value under condition". For example, (c, k = 13) and (c+1, k = 3) give the same last two answer digits. If their raw centroids nearly coincide, the directions encode the answer. GLM, the only model without single-token answers for all of 0–999, is also the weakest model for every construction, including Direct.
  3. The rule is not tested across conditions: All conditions are used in fitting, so the "rule" is never asked to predict an unseen condition. A held-out-condition experiment would make the equivariance claim more convincing.
  4. The low-dimensionality claim: Rank 12 is close to the maximum rank possible with 15–16 training directions, and it is not varied. Why 12?
  5. Post-hoc layers and stats: Reporting layers are chosen by peak performance of the Canonical method. I see only one prompt template and no confidence intervals. Please add prompt variations and uncertainty estimates from refitting.

Presentation

  • Section 3 is dense. For example, the U_c,tr definition could be one equation, and the text needs more intuition before the formalism.
  • Figure 1 has too much text, and some arrow tips are misplaced. Keep the three-column layout, but make the core idea readable at a glance, and show already in panel A that the task is addition.
  • Minor and my own preference: Typeset method names (Canonical, Step-power, …) with \texttt{} instead of \emph{}.
Questions for authors
  • Do the findings generalize to other arithmetic relationships?
  • How can the answer be distinguished from the proposed value under condition?
  • Does Step-power predict a condition left out of fitting?
  • How do results change with rank r?
Reviewer_fVfvOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In a controlled arithmetic setup of summing two numbers, this work explores how a representation (one digit sum) changes with conditions/context (ten digit sum). This authors provide support for their framing by training condition-specific subspaces and performing interventions in the LMs representation to inject a new condition, number pair and check how the LM behavior changes accordingly.

By training another shared-generator subspace, the authors also show that the change across different condition subspaces can be predicted with a step power (I am not entirely sure on this, check my questions).

Critical strengths
  • Tests conjectures with causal interventions in a controlled setup.
Critical weaknesses
  • The problem setup is very controlled and toy-ish. The authors at least test their findings across a few different setups to establish generality. I am also not sure about the broader impact/applicability of the findings.
  • The alternate hypothesis that a shared structure exists is not given adequate consideration. See my first question for the authors below.
  • The paper is often hard to follow with occasional LLM smells. And I am still not sure how the "Shared-Generator Model" fits with the paper's current framing.
Questions for authors
  1. You do a lot of engineering to defend the framing "interpretable structure changes with context/conditions", by training condition specific subspaces, tuning intervention scalers, .... The alternate hypothesis of hunting for a shared structure do not get equivalent treatment in your paper (only pooled is explored, transfer is also condition-specific). You should try 2 separate subspaces: B10B_{10} is a subspace for capturing the condition cc (the tens digit sum), and B1B_1 is a subspace for capturing the ones digit sum (independent of the condition). Then, you should try to perform the inject the cc and kk separately in the LM latents using those subspaces.

    • You should also check if intervention with B1B_1 is effective across conditions, which will suggest there are indeed some shared structure across conditions.
  2. I am not entirely sure how the "Shared-Generator Model" fits with the paper's current framing, which seems to be that the representation changes with context/condition. But, then your interventions in 3.3 seems to suggest that this change is predictable from the step size of the condition shift alone? Framing-wise these two claims seem to be at odds with each other, and you also don't clarify this properly in the paper. If the change is predictable by another linear transformation (powered accordingly), then it does suggest a shared representation space. All that is needed is to shift the space accordingly, which may be achievable by just adding a few more dimensions to the shared subspace.

  3. On interventions, I am curious to see the scaler values α\alpha, β\beta, and σj\sigma_j youu used in Equation 7 (maybe only for the "canonical" setup for a single LM). In line 791, you mention that you do a grid search over range(0,5, 2.01, 0.5), but you never report the winning combinations.

    • Do you need to calibrate α\alpha, β\beta separately? Or, can you get the same performance with a single scaler γ\gamma such that h′=h+γσj(dj←i,knew−uj,kold)h' = h + \gamma \sigma_j (d_{j \leftarrow i, k_\mathrm{new}} - u_{j,k_\mathrm{old}})?
    • Subtracting uj,koldu_{j,k_\mathrm{old}} (amplified by some scaler) doesn't ensure complete removal of the old information. If you really believe that the information is captured by the subspace spanned by the per-condition basis vectors BcB_c in Section 3.2 (or maybe a general basis set BB from 3.3), then you should just project the old information out using the basis subspace(s). Then (in thory) you shouldn't even need to worry about subtracting uj,koldu_{j,k_\mathrm{old}}.
    • In my opinion, the most "principled" intervention should also make sure that the magnitude of the new and old info have equivalent magnitudes, i.e. ∣dj←i,knew∣≈∣uj,kold∣|d_{j \leftarrow i, k_\mathrm{new}}| \approx |u_{j,k_\mathrm{old}}|. If your interventions are still successful then this would ensure a complete capture of the representation rather than the new signal completely flooding out the old signal residues with an amplified magnitude. I understand that it is pretty common in intepretability interventions to use amplifications, but I think in your very controlled setup it might be worth exploring.
  4. Unless I am getting anything wrong, you are using "direct" intervention as your reference/ceiling. But the true ceiling is the LM baseline performance with natural conterfactual/target inputs, correct? I don't really see the point of this "direct" intervention at all. Is the direct intervention is kind of just a sanity check that the projections and interventions are working as intended?

Nitpicks on results and generalization:

  1. Rank=12 is used throughout the paper, across LMs of different sizes with different dimensions. What is the rationale for this choice?
  2. A single template The sum of {a} and {b} is use throughout. How sensitive is the projections you train to the choice of a different template?
  3. Lines 293 - 297. Not sure what justified these layer sweep choices. Consider adding other layers as well, maybe with sparser step sizes between layers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.