Bringing Order to LLMs: The Lie Bracket Is Absent but Trainable

When language models compute transformations, do they know the rules? We ask this for $\mathfrak{so}(3)$ by investigating two separate aspects: representation of group elements, and representation of the Lie bracket that composes them. We find that pretrained models have neither of the above. Across eight models in five families, no pretrained or instruction-tuned checkpoint shows a measurable bracket, and none represents the composed group element beyond token-identity and last-token cues. LoRA fine-tuning using non-abelian composition installs element decoding well beyond the last-token bound in all eight models, and a partially faithful bracket in seven of eight. The installed structure sits at depths that differ by family (40–83%). Below a threshold on the number of generators, an order-blind predictor suffices. Above it, all eight models beat a baseline that sees only the bag of generator tokens, six of them clearly, but none beats, beyond seed noise, a baseline that knows the signed generator pair and not its order. During fine-tuning, element information emerges before bracket structure, and bracket fidelity does not predict behavioral margin across models. A second finding is methodological. The three statistics that would translate representation into behavior each return a forced positive: cross-model decoder agreement by shared supervision, alignment-based transfer by per-model fidelity under $SO(3)$-equivariance, and the causal order-swap correlation by a Baker–Campbell–Hausdorff identity that reproduces the effect ($r=0.96$) with no model in the loop. Together, these findings separate three questions: representing a group's elements, representing its algebra, and using it.