acceptodds
Withdrawn from ICLR 2027

Bringing Order to LLMs: The Lie Bracket Is Absent but Trainable

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When language models compute transformations, do they know the rules? We ask this for by investigating two separate aspects: representation of group elements, and representation of the Lie bracket that composes them. We find that pretrained models have neither of the above. Across eight models in five families, no pretrained or instruction-tuned checkpoint shows a measurable bracket, and none represents the composed group element beyond token-identity and last-token cues. LoRA fine-tuning using non-abelian composition installs element decoding well beyond the last-token bound in all eight models, and a partially faithful bracket in seven of eight. The installed structure sits at depths that differ by family (40–83%). Below a threshold on the number of generators, an order-blind predictor suffices. Above it, all eight models beat a baseline that sees only the bag of generator tokens, six of them clearly, but none beats, beyond seed noise, a baseline that knows the signed generator pair and not its order. During fine-tuning, element information emerges before bracket structure, and bracket fidelity does not predict behavioral margin across models. A second finding is methodological. The three statistics that would translate representation into behavior each return a forced positive: cross-model decoder agreement by shared supervision, alignment-based transfer by per-model fidelity under -equivariance, and the causal order-swap correlation by a Baker–Campbell–Hausdorff identity that reproduces the effect () with no model in the loop. Together, these findings separate three questions: representing a group's elements, representing its algebra, and using it.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, probing controls, Lie algebra, so(3), group composition, fine-tuning, representational structure, null distributions

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_y85aOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This study focuses on the question of whether the models learn the rule behind a task or just the answers. The task they use is composing 3D rotations and the rule which is the Lie bracket captures how much the order of rotations matters. They tested 8 models and they found that the untrained (not fine-tuned) models do not show signs of learning the rule while LoRA fine tuning installs partial traces in all but one of the models. They use a baseline that ignores the rule (order blind predictor) and they argue that if a model is better than the baseline then they may be using the rule but even the fine tuned model does not beat the (strong) baseline beyond noise. Authors claim that 3 interpretability checks come out (by mathematical necessity) positive.

Critical strengths

The authors make sure that they compare the results to cases where the right answer is known and make sure to take back any claim that does not hold. This makes their results to be trusted more. In a steering style test the correlation was 0.96 even when there was no model which is a useful warning for future studies.

Critical weaknesses

I did not see a control where the rule is not correct, but the order still plays an important role. The claim they make about llama-3-8B is only based on one model and a single run. The part about learned, but not used is not sufficiently supported, considering that defined tune model learns only a little of the order dependant part so I am not surprised that it is not using the rule. I would want to see how the answers would change if they remove the rule from the model.

Questions for authors
  1. Have you tried measuring the rule score when the rule is scrambled (or replaced with other algebra) and in the case that order does not matter.
  2. I want to know if you’re results would still hold if the move names do not share word pieces?
  3. Have you tried removing the rule from the model? How does the results change?
Reviewer_JdydOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether pretrained and LoRA-fine-tuned LLMs represent 3D rotations (SO(3) elements) and their non-commutative composition rules (the so(3) Lie bracket / cross product) on a synthetic rotation-composition task. The authors claim that base models represent neither, LoRA fine-tuning installs both in 7 of 8 models, and the installed bracket is not detectably used behaviorally.

Critical strengths

The task of understanding whether language models trained to compose transformations can learn to represent the elements being composed or the algebra that composes them is generally interesting.

Critical weaknesses

In my opinion, this paper does not match the minimum expected standards of scientific writing. The abstract reads like unfiltered AI. The introduction reads more like a discussion on the results (extremely hard to follow without any context) and does not deliver on any of the expected features of an introduction (motivation, rough positioning with respect to other works in the area, clear statement of the contributions). Clarity is also greatly undermined by the fact that often notation is used before being properly introduced (e.g., kk in the introduction is never defined, Figure 1 and the Introduction talk about “0-2 of 27 cells”, “7 of 8 carriers”, “sign-blind .386 ceiling”, and “sign-aware .545 ceiling” on pages 1–2, whereas what a cell is or how the task works is not explained until pages 4–5.). Figures 1 and 2 fail to properly convey their messages clearly and would require a substantial re-design. The method section is also extremely cryptic. I am admittedly not very familiar with Lie groups, and the paper provided absolutely no context on it (it took me 5 minutes to catch up with it online, and adding a paragraph in the method section on it would have instantly made the paper more comprehensible to a broader ML audience).

Besides these evident limitations in the presentation, the paper also features some methodological flaws.

  • For sequence lengths in [2,4], most of the target rotation is entirely determined by the commutative multiset, and for lengths [5,8] an order-blind predictor achieves close to perfect accuracy. Moreover, the continuous target log(R) is coarsely discretized into 31 bins per axis. Concluding that language models do not behaviorally use the Lie bracket when the training task and coarse binning barely penalize order-blind approximations is a flaw of the task design.

  • The failure of base models in Section 4.1 is not surprising. The task is performed on meaningless token strings without in-context examples or definition. How would a model be able to spontaneously perform SO(3) group composition in this context?

  • The procedure used to estimate the generator matrices M_i relies on strong, unjustified assumptions and suffers from severe estimation noise. Fitting a single linear transition Zp+1∼AiZp+biZ_{p+1} \sim A_iZ_p+b_i on a small PCA subspace at the readout token assumes the language model updates its state step-by-step like a linear recurrent network rather than composing the sequence all at once via attention. Also, taking the real part of AiA_i when it has negative real eigenvalues is mathematically questionable. Even worse, the resulting matrices are heavily dominated by noise: on the abelian control task (where the true brackets are strictly zero), the fitting procedure injects spurious commutator norms of 0.24–0.43, and the bracket relative error is above 100% (1.05 to 3.00 in Table 20) across all models.

Questions for authors

See Weaknesses.

Reviewer_QPCcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates whether LLMs represent the Lie algebra so(3) when they compose rotations. The authors frame this question as important for understanding whether language models represent "knowledge" of transformation rules when they compute transformations. Models read sequences of named rotation generators and output the binned rotation vector of their product. The authors test eight models from five families and find that the base models represent the group element only at the level that the multiset of generators and the last token supply, and do not represent the Lie bracket that composes them. LoRA fine-tuning installs element decoding in all eight models and a partial bracket in seven. No fine-tuned model uses the bracket in behavior beyond a baseline that knows the signed generator pair but not its order. A methodological section argues that three cross-model and causal statistics are positive by construction.

Critical strengths

The control design is careful, and the ablations in the appendix are thorough. The authors report negative and unexpected results, voided runs and withdrawn claims openly.

Critical weaknesses

Presentation:

The presentation of the paper does not meet the standards of ICLR, and this is the main reason for my rating.

  1. An interpretability paper should be written so that a reader with general knowledge of the field and - specifically - an interest in emergent structured representations can understand from the introduction what the paper is about. This paper assumes familiarity with Lie theory that many interpretability readers do not have. Instead of explaining the concepts, the introduction lists one result after another. A short, accessible introduction to so(3) for readers outside geometric deep learning would strengthen the paper significantly.

  2. What first caught my attention was that the figures appear to not have been thoroughly checked before submission. Figures 2 and 3 in particular are overloaded, with overlapping labels throughout their subfigures. Figure 3 combines four separate claims. Panel B has two y-axes, panel C has overlapping lines, and panel D is a row of dots with labels inside the plot. Figures 1 and 4 also have overlapping labels. They are clearly below the standard expected at ICLR.

  3. Notation and terms appear before they are defined. The introduction uses k=12 and k=10 before Section 3.1 defines k. The introduction and Figure 1 use the terms "carrier" and "cell" before Section 3.2 defines them, and "arm" is never defined.

As a reviewer who works on probing and representational analysis, I had a very hard time understanding the paper: I could not follow large parts of it and cannot fully assess its technical claims. I see this as a consequence of the presentation problems.

Substance

My impression is that the main finding on base models is expected from the setup. The base models receive prompts such as "Compose: Rxz+ Rx+ Rxy+ Rxz- =>". The generator names are invented strings, and the prompt gives no instructions or examples. The authors themselves note that these names "mean nothing to a pretrained model" (Appendix L). Accordingly, the base model cannot do the task (0.000 exact match, Table 14). A model that does not understand the task has no reason to compute rotations, so it is no surprise that no rotation structure appears in its hidden states. This result is a sanity check, but the paper presents it as a main finding, as the first half of the title and the whole of Section 4.1. The test would be informative only if the base models partly solved the task, for example with a few worked examples in the prompt, and still showed no bracket.

Questions for authors

Did you test the base models in a setting where they can partly solve the task, for example with a few worked examples in the prompt?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.