acceptodds
Withdrawn from ICLR 2027

The Deviation Carries the Code: Causal Surgery on Geometric Representations in Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Causal analyses locate language-model arithmetic in geometric structures like the month circle and the number helix. Each characterizes the representation through that structure. We call the remainder orthogonal to that structure the deviation. Recent theoretical work predicts such components exist, but no work has manipulated their content. We test whether models rely on them by editing token embeddings, rewriting only tested components. A matched-norm control substitutes random content, isolating content effects from magnitude effects. Across four open-weight models and a grokked transformer, on month, weekday, hour, letter, and two-digit addition arithmetic, the structure alone never restores the unedited model's accuracy. In month, hour, and letter arithmetic, the deviation alone restores 70 to 102% of that accuracy. Weekday arithmetic varies by model, and addition requires both components. Rotating the circle one step opposes the deviation, which we shrink to half its norm. In Llama-3.1-8B, the answer follows the deviation on 95% of prompts and the circle on 1%. Patched at full norm into three models' residual streams, the deviation wins month and weekday conflicts at every tested layer where the operand position still controls the answer. For detecting such structure, strength-based readouts are uncalibrated, reporting structure in 93.5% of structureless inputs matched to the GPT-2 embedding spectrum, whereas our significance test does so for only 4.0% of those inputs.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal intervention, representation geometry, language models, activation patching

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_nLKVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Interpretability works have found structured representations of semantic information (circular arrangement of moths, weekdays, Engels et al, 2025) and arithmetic operands (Kantamneni and Tegmark, 2025; Feucht et al, 2026). This paper instead studies the reminder, the representation orthogonal to the identified structure. With ablation experiments this paper finds that often the reminder is sufficient for the LM to perform the task. The authors draw the conclusion that LM representations are much richer than their visible geometry.

Critical strengths

The paper does follow a nice framework of separating structure vs deviation (everything but the structure). The paper pits different kind of interventions against each other, such as introducing conflict with rotation on structure vs amplifying the deviation, to understand what actually matters for LM computation.

Critical weaknesses
  • The paper is very poorly written. For example: terms such as "embedding campaign", "zero patch", ... are mentioned in passing without any explicit definition, and it is really up to the reader to figure out what they actually means. The paper also have a heavy LLM smell.

  • I think the core technical and conceptual contribution of this paper is weak and not novel, nor do they provide any actionable insights. Please see my questions for authors below for more details.

Questions for authors

First of all, this paper is very difficult to read. I am familiar with most of the works this paper is based on, but I still had a hard time understanding the contents. How the paper presented the ideas is very tedious and that needs significant improvement before it is worthy of publication. My current assessment is potentially due to my incomplete understanding of some of the technical/conceptual details and I am happy to revise my scores if the authors are able to give satisfactory response and update the paper to improve readability.

  1. Most of your headline scores are on the "embedding campaign". You never explicitly define what that means. Such terms are unconventional and I personally have never seen them in any previous interp papers, so it is up to the reader to work out what it means. How I understood it as (and correct me if I am wrong): you are editing the representation after the input embedding and checking the downstream effects. A few questions on this.

    • (1a) In Figure 2 you report that ablating the structure (deviation-only) leaves performance largely intact for months and letters. The works you cite (Engels, Feucht) find this structures in middle to later layers. And, the LM might have multiple opportunities to "re-build" the structure, as long as it knows that the input is "January". After your edit, have you tried applying the Wemb†W_\mathrm{emb}^\dagger on your edited month embedding to check if it still decodes to "January"? This approach is just the inverse of Logit Lens with input embeddings. If it does decode do the month (i.e "January") then the deviation-only scores are trivial and the insights you are drawing from them don't really mean anything.
    • (1b) Same question for structure-only. And, is the structure even formed that early in the computation so that these interventions are expected to give target effects? In how many cases you are just corrupting the semantic information (i.e: "January", checked with the reverse Logit Lens I suggested) and the LM just can't recover the structure as it doesn't even know what the input is?
  2. In your "activation campaign", you target one layer at a time. Again, the LMs are distributed systems and potentially get multiple opportunities to "re-build" the structure. Have you considered targeting a window of layers? Specifically targetting the window of layers Engels, Feucht papers found for the representation/computation?

  3. Previous literature (see Gur Arich et al, 2026; Cui et al, 2026, Sen Sharma et al, 2025, among others) have also found mix of different mechanisms inside LM computation. When you ablated on structure, maybe the LM can still do the same computation based a redundent computation which got abosrbed in the deviation. Maybe you are also finding that in your rotation experiment on Figure 3a: as you are amplifying the deviation, you are just suppressing one structure with it alternate computation absorbed in the deviation. I am also asking this because of 2 lines in your conclusion.

    • "The results carry a caution and a corrective for intervention-based interpretability": Sure! We knew already that the LM does redundant computations. Interpretability is more of an effort to catalogue all such mechanisms one at a time and understand how they work together. I don't see how your results are a caution or corrective for this. And, if it is, what do you suggest interp researchers should do differently?
    • "The representations ... language models compute are richer than their visible geometry, and this paper supplies instruments for studying the remainder": Really? Maybe I missed something, but where is the "instrument" that you are referring to here? With your ablations/interventions, all you are really saying is that "the reminders matter, and is often enough", which is not a new insight.
Reviewer_ZTYnOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

Causal analyses identify compelling geometric structures in the activations of language models, such as circles for days of the week, and helixes for numbers. While these “idealized structures” explain some corresponding behavior of the model, this is not always the case (ex: in the case of arithmetics, tasks like division, multiplication by 1.5, etc.).

This paper studies what the authors call the “deviation”: the orthogonal complement to the ideal structure’s subspace.

The authors study a range of tasks that all involve some ordinal or periodic concept (days, months, modular addition, etc.). As such, most of these concept representations can be represented using a Fourier basis. The models consider the representations in embedding space and activation space, and decompose it into “structure” and “deviation”:

W \in \mathbb{R}^{n \times d}: input embeddings of operands (eg., days of the week) S \in \mathbb{R}^{n \times s}: structure subspace

W = SS^\top W + (I - SS^\top)W, D = (I - SS^\top)W where D = deviation, aka the orthogonal complement to the ideal structure’s subspace

As such, most interventions boil down to replacing the representations of a token by replacing/deleting/rotating etc. the deviation components. For instance, deleting or scaling the deviations might look like the following: Delete: W’ = SS^\top W Scale: W’ = SS^\top W + \alpha D

The authors conduct these interventions either in input embedding space or model activation space and study the effects on the model’s final task behavior, and find that the deviation does play a significant role (e.g., task performance can be maintained by only keeping the deviation, while preserving only the structure sees a much larger drop in task performance). In short, the paper claims that the “ideal structure” is never sufficient to explain the entire behavior. Unfortunately, I think this is most people’s expectations and the findings are not surprising, and I additionally see many problems with the paper (see Weaknesses section).

Critical strengths

The paper is thorough in the experiments conducted, although I am not sure if these experiments are well motivated (see Weaknesses).

Critical weaknesses

The paper is poorly written, if not AI-generated. It is difficult to follow the technical descriptions of their method and findings, and contains a lot of grammatical errors, to the point where it is difficult to appreciate the scientific content. As such, in its current form it does not meet the bar for ICLR as I do not believe the readers will be able to parse much of the content.

Secondly, I do not find the paper’s findings all that surprising. While the prior works eluded in this paper all find beautiful, “ideal” structures in model activations or embeddings, I do not recall any of them claiming that such structures explain all of the model’s behavior (even on narrow tasks from which the structures can be found). The existence of “competing mechanisms” [1, 2, 3, 4, 5] is well known and studied, but also the identified structures often occupy a very low-dimensional subspace (e.g., 2, 3, or 9 in this paper’s experiments) compared to a massive number of dimensions available in the “deviation” (eg., 3584 for Qwen2.5-7B). This second point makes Figure 2 a very unfair comparison to be looking at, in which the baseline and “deviation” have 3~4k dimensions while “structure only” have 2, 3, or 9 dimensions. If anything, the fact that “structure only” has so few dimensions yet sometimes achieve non-trivial accuracy is more interesting than that the “deviation” achieves high accuracy.

All of this to say, to put frankly I think the paper’s findings are somewhat expected.

Lastly, I believe the causal role of the structures from prior work are demonstrated in the mid-layer activations, as opposed to input embeddings. I understand that Section 3.3 studies mid-layer activations, but much of the text’s emphasis is on the input embeddings. I find the motivation for studying the deviations in input embedding space, as well as the overall motivation of the work difficult to follow.

[1] Yu et al. Characterizing Mechanisms for Factual Recall in Language Models. [2] Gur-Arieh et al. Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context. [3] Nanda et al. Emergent Linear Representations in World Models of Self-Supervised Sequence Models. [4] McGrath et al. The hydra effect: Emergent self-repair in language model computations. McGrath et al. [5] Ortu et al. Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals

Questions for authors

N/A

Reviewer_2fJnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether geometric structures previously identified in language model representations (e.g. circles for months and weekdays or a helix for numbers) sufficiently explain the models' arithmetic behavior over these concepts, or whether information beyond these idealized structures also plays a causal role. To investigate this, the paper decomposes learned representations into an idealized geometric structure and an orthogonal remainder, termed the deviation. The paper then selectively removes, modifies, or restores these components and evaluates their effects on arithmetic tasks involving months, weekdays, hours, letters, and numbers. The experiments cover four language models and a network trained on modular addition. The paper reports that the idealized structures alone are insufficient to reproduce the models' original performance, arguing that previously identified geometric structures provide an incomplete account of the representations underlying language model arithmetic.

Critical strengths

The paper’s key strengths are listed below:

  • Novel and interesting approach. By explicitly decomposing representations into an idealized structure and its deviation, and selectively intervening on these components, the paper introduces an interesting framework for investigating which information the model actually uses.
  • Important research question. Previous work has identified geometric structures in language models and demonstrated their causal relevance for tasks such as arithmetic. However, this does not necessarily establish that these structures are sufficient to explain the model's task behavior. Investigating the role of the remaining representational information is therefore a valuable direction, provided that the identified structures and their deviations are appropriately defined and validated (see critical weaknesses).
Critical weaknesses

The paper’s main weaknesses are listed below:

  • The chosen idealized structures may not fully capture the representations identified in prior work. For cyclic concepts (e.g. months), the authors define the idealized circle using Fourier frequencies k=0,1k = 0, 1, whereas Engels et al. (2025) identified circular representations by fitting probes to hidden activations. For example, if the learned month representations form a circle with uneven angular spacing, the Fourier decomposition may incorrectly assign part of this circular structure to the deviation. Similarly, for letters, the assumed straight-line representation may miss an ordinal structure following a curved trajectory. Although Appendix F provides some evidence of approximate alignment with the fitted circles from Engels et al. (and a stronger theoretical correspondence for the numerical helix), the circle comparison is conducted at the input embeddings rather than the original intervention layers, and does not establish causal equivalence. To address this, I suggest that the authors reproduce the fitted circular representations at the same hidden-state layers used in prior work, compare them with the Fourier-derived structures, and repeat the structure-only, deviation-only, and rotation interventions using both definitions. For letters, the authors could similarly compare their linear approximation against alternative learned ordinal representations.
  • The deviation may still encode the same geometric structure rather than a genuinely separate representation. The Fourier components are orthogonal across concept indices, but not necessarily in the model's embedding space. For example, a higher-frequency component k=2k = 2 could lie in the same two-dimensional plane as the k=1k = 1 month circle and still encode cyclic information. Similarly, the deviation from the assumed straight line for letters could preserve alphabetical order through a nonlinear representation. Thus, strong performance using the deviation alone does not necessarily imply that the model relies on a distinct computational mechanism. I suggest that the authors measure how much of the deviation overlaps with the identified geometric subspaces and test whether removing this overlap affects arithmetic performance.
  • The clarity and presentation of the paper should be improved. For example, the Introduction presents numerous detailed findings before adequately explaining the underlying methodology, making them difficult to interpret. Similarly, the contributions are often expressed in overly dense and convoluted sentences (e.g., ll. 104–106 and 124–126), obscuring the main takeaways. I suggest simplifying the Introduction to emphasize the key ideas and high-level findings.
Questions for authors

A list of questions and suggestions can be found below:

  1. How closely do the chosen idealized structures correspond to the representations identified in prior work? Appendix F provides some evidence of alignment with the circular representations of Engels et al. (2025), but the comparison is conducted at the embedding level rather than the original intervention layers. Could you compare both approaches at the same hidden-state layers? Furthermore, could you check whether the structure-only, deviation-only, and rotation-conflict experiments yield similar results when using circular representations identified through the fitted-probe approach of Engels et al. (2025), rather than the analytically defined Fourier components? For this purpose, you could define the deviation relative to the fitted circular plane (e.g. as the orthogonal remainder after projection onto that plane) and repeat the corresponding interventions.
  2. To what extent does the deviation still contain information belonging to the identified geometric structure? Since orthogonality across Fourier frequencies does not imply orthogonality in the model's representation space, higher-frequency components could still lie within the identified circular plane. Have you examined this overlap, and would the deviation remain causally important after removing such components?
  3. How sensitive are the findings to the choice of idealized structure? For example, the assumed straight-line representation of alphabetical order could miss a nonlinear ordinal representation, while higher Fourier frequencies could also encode cyclic relationships. Have you considered alternative geometric representations and assessed whether the relative importance of the structure and deviation changes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.