acceptodds
Under review as a conference paper at ICLR 2027

Tracking States or Tracking Cosets? An Algebraic Account of Learned State Tracking

Abstract

State tracking requires composing a sequence of updates, but accuracy alone does not reveal what a model has learned. We study neural networks trained to predict the running product of group elements. We identify quotient solutions in Transformers, where models recover the quotient class while predicting nearly uniformly among its members. The reciprocal of class size predicts partial accuracy without a fitted parameter, extending parity-based accounts to non-parity quotients. Our baseline Transformers’ predictions change little under prefix reordering beyond the exact-tracking frontier. We prove that, for finite groups under uniform i.i.d. full-group inputs, optimal order-blind exact accuracy converges to the reciprocal of abelianization class size as prefix length grows, consistent with the observed abelianization plateaus. Sequential updates permits more: any partition into right cosets of a subgroup, normal or not, survives sequential updates. In our census of standard Transformers, every recovered coset partition comes from a normal subgroup, whereas parameter-matched recurrent networks pass through both normal and non-normal right-coset stages during training. On , we identify low-dimensional subspaces of the recurrent state that encode non-normal cosets. The coset mean vectors form dodecahedra in three-dimensional subspaces, and swapping the state components in these subspaces makes subsequent coset predictions follow the donor sequence. Our results connect partial accuracy, learning stages, and internal computation through the subgroup cosets that models learn to track.

Primary area:
interpretability and explainable AI
Keywords:
state tracking, mechanistic interpretability, group theory, quotient groups, transformers, learning dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_y7zoOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies state tracking in transformers and uses group composition as a controlled setup. The authors leverage the group structure of the problem, primarily abelianization, (normal) subgroups and cosets, to mechanistically study the transformer's internal computation and connect it to partial accuracy. In particular, they find that transformers do not distinguish different members of the same coset in a group's abelianization.

Critical strengths
  1. The overall question of how do the algebraic properties of a particular problem determine certain mechanisms internal to a model is interesting.
  2. The synthetic group composition task is an appropriate controlled scenario to investigate state tracking.
Critical weaknesses
  1. The paper is very difficult to read and opaque, with many critical choices not sufficiently supported. For example, in section 3.1 the paper exposes two 'recipes', A and B, with which experiments are run. However, it is not clear why do the authors make this choice and what are the authors trying to control for by having both setups. I attempted to resolve this by reading the referenced appendices H and I, but these appendices simply list the specifications of these recipes; while this is great for reproduction, the scientific objective of this control is unclear. Furthermore, appendix I contains many tables with different setups, once again without explaining why they are relevant. If the authors are trying to assess whether their conclusions hold for different hyperparameters, a series of plot would be a better presentation. Additionally, the table contains a column "Status; Location" and reports 'pilot' and 'exploratory' experiments not included in the paper. I believe that the paper needs a major rewrite to make it clearer and understandable.
  2. Overclaims. The authors findings about transformers coset tracking seem limited to a particular training and data sampling regime. As acknowledged by the authors in Appendix G, transformers can indeed achieve perfect state tracking (in distribution), and their "order-blind" results and prediction may only apply when a particular sampling and training process is applied. This introduces confounding factors and makes it less clear when do these findings actually apply. More care is needed to assess what conditions are necessary for the author's predictions to be true.
  3. Theory/Practice mismatch. Following the above, the theory should be adapted to reflect these further limitation. Additionally, a transformer is not an order-blind function, even when positional encoding is removed, despite this property being an assumption of the propositions in Section 4. Therefore, more care is needed to justify why the theory is relevant for the considered cases when the experiments do not satisfy the same theory's assumptions.
Questions for authors
  1. Can the authors clarify what is the significance of Recipe A vs Recipe B? What is this trying to control/ablate? The same question applied to the tables in Appendix H.
  2. Can the authors clarify what are the conditions needed for the theory to apply, and make important requirements (such as the data generating distribution) more explicit and justified? Do these further requirements hinder the significance of the results
  3. The geometric analysis from Section 6 currently covers a GRU. Can it be extended to transformers?
9 comments
AuthorsOpenReview ↗

Recipes and presentation

We thank the reviewer for the review and for the concrete presentation requests.

W1 / Q1 (recipes A and B). In short, recipe A is our main setting, and recipe B is a control that finds out why Liu et al. (2023) reach exact tracking while we do not.

  • Recipe is an architecture with its optimizer and training budget. The inputs, the full group or a generator set, are chosen separately.
  • Recipe A: Transformers trained from scratch on uniform i.i.d. inputs from the full group (§3.1). This is where the plateau appears.
  • Recipe B: It replicates the setup of Liu et al., who obtain exact tracking on non-abelian groups with generator inputs.
  • What the control shows: We vary recipe and inputs independently on D4. Only recipe B with generator inputs reaches exact tracking (99.999% conditional accuracy). Changing only one of the two leaves the model at the plateau: recipe B on full-group inputs gives 49.96%, and recipe A on generator inputs gives 50.02% (Table 7). Hence both the inputs and the recipe matter. On these inputs a position-dependent counting solution exists, and our causal interventions support that the successful models indeed use it (App. O). Architecture and positional encoding also affect whether it is learned (Fig. S7b).

Appendices H and I. Appendix H now opens with a table mapping each configuration to the section that uses it. In Appendix I, every row names the claim it supports and where it is used, and the Status column is removed. The pilot and exploratory runs are now in a separate table with the reason each is unused, such as a single seed, a short run or a missed registered target. We list them for transparency, and no claim relys on them.

Plots. As suggested, a new figure (Fig. S7) replaces the robustness rows of Appendix I. It shows full-state accuracy given a correct quotient for 60 D4 runs across 20 architecture and training variants under full-group inputs, next to the generator-input runs. No variant leaves the plateau.

AuthorsOpenReview ↗

Scope, theory, and geometry

W2 / Q2 (scope and conditions). The plateau prediction is specific to uniform i.i.d. full-group inputs. Our framework also makes distinct predictions for other input sets.

  • Where the plateau applies? It applies under uniform i.i.d. inputs from the full group, our main setting (recipe A, §3.1), and Theorem 2 states this assumption. As a robustness check, we evaluated 20 architecture and training configurations on D4 with full-group inputs, and none leaves the plateau (Fig. S7).
  • Other settings are predicted as well. An algebraic criterion on the input set says when counting can solve the task.Recipe B reaches exact tracking on {r, s}, and in a preregistered test it stays at the plateau on {r, s, e}, as the criterion predicts (§5.3). Recipe B with generator inputs also reproduces the exact tracking of Liu et al. (2023) (Table 7).
  • Why this does not reduce the contribution? In every setting we tested, the same readout identifies the tracked state, and the class size gives its accuracy (§3.4, §5.2, §5.3).

W3 (a Transformer is not order-blind). Thank you for raising this point. We would like to clarify that the theorems are about order-blind predictors, and a new corollary (App. S) extends the bound to models that are nearly order-blind, which the trained Transformers are beyond the exact frontier.

  • New corollary (§4, App. S). If reordering the prefix changes a model's outputs by ε in total variation on average, its mean probability on the true state is at most the order-blind optimum plus ε.
  • Measurement (App. S). Beyond the frontier, reordering changes the outputs by .011–.051, against .22–.50 inside it, and the bound holds in all 28 runs. The bound is on probability, and the accuracy plateau is the measurement in Table 1.

Q3 (geometric analysis for Transformers). Yes. Appendix R analyzes 40 recipe-A Transformer endpoints across 11 groups. In 38 out of 40, we find that the class means have the minimum dimension expected for the abelian quotient, and the fitted transformations approximately match its symmetries. The exceptions are a low-accuracy Q8 endpoint and an H₃(ℤ/5) endpoint with a higher-dimensional representation. Subspace swaps transfer the donor class at the intervention position, while separate single-position interventions generally do not sustain this transfer at later positions. These results establish a causal role in the current quotient readout, while leaving its sequential implementation unresolved (§6.2).

We hope these additions clarify the scope of the theoretical claims and the corresponding empirical evidence.

AuthorsOpenReview ↗

Follow-up: summary of answers to W1–W3 and Q1–Q3

Dear Reviewer y7zo,

Thank you again for your review. As the discussion period ends on November 18, we would be grateful to know whether our response and the revision address your concerns. Below is a detailed summary.

(W1, Q1): Recipe A is our main recipe for studying the abelianization plateau in Transformers. Recipe B, adapted from Liu et al. (2023), tests what it takes to go beyond the abelianization. The initial submission introduced both in §3.1 (“Recipe B, adapted from Liu et al. (2023), uses a 3-layer, width-512 GPT-2 transformer”) and used recipe B in §3.4 (“Quotient tracking also extends beyond the abelianization. With recipe B and odd-permutation inputs, …”). §8 also stated that “Training recipes can determine whether models reach exact tracking or remain at a partial resolution,” and App. G opened with the aim: “We now investigate how models recover state distinctions beyond the abelianization. […] We therefore use generator-restricted inputs to study how models refine quotient-level solutions.”

We also ablated recipe B in App. O of the initial submission: removing position embeddings gives chance accuracy, switching to full-group inputs produces two plateaus and one partial solution, and replacing the optimization block leaves only a short exact prefix (frontiers 12–22), while changing the MLP width keeps exact tracking. Table 7 (App. D) then crosses the two recipes with the two input types: among these cells, only recipe B with generator inputs leaves the 1/2 plateau (99.999% vs. 50.0%). We kept these results in the appendix because recipe B serves as a control, while the main text tests the plateau law under recipe A.

We agree that this rationale was too easy to miss, so the revision states it in §3.1 and §5.3 (with the block and position-encoding ablation in Fig. S7b), and App. H opens with a table linking each configuration to where it is used. In App. I, the coverage tables now list only runs that support a claim, with a column naming where each is used. Pilot and exploratory runs, initially included for transparency, are kept in a separate table as each is unused. A single figure now summarizes robustness across configurations (Fig. S7a: 20 architecture and training configurations, 60 runs on D4).

(W2, Q2): The abelianization plateau prediction for Transformers is stated for uniform i.i.d. full-group inputs, as stated in the initial abstract and §1. The exact tracking in App. G happens outside this condition, and in §4 we noted that “restricted alphabets require separate analysis (App. G).”

In fact, the exact tracking in App. G is not a counterexample: it is a counting shortcut. On {r, s}, the k-th r at position p has exactly p − k reflections before it, so the whole D4 state follows from three parities: the counts of r, of s, and of r at even positions (Eq. 12). Our activation-replacement experiment indicates that the successful models read these counts rather than a running product. Lemma 2 in App. O shows that this shortcut exists exactly when the inputs come from at most two of D4’s four abelianization classes ({r, s} uses cosets {r, r³} and {s, r²s}). Adding e uses a third class {e, r²} and breaks the shortcut: in a preregistered test, recipe B reaches exact tracking on {r, s} but stays at 1/2 on {r, s, e} (Table 21). On full-group D4 inputs, counting recovers only the class, which is where the plateau sits. All of this, including the full proof, is in App. O of the initial submission.

So, to answer your Q2, the input condition does not narrow the significanceof the results : the restricted-input setting is a controlled case in which a model gets past the abelianization, and the same algebra predicts when it can.

(W3): We did not assume order-blindness. The initial submission measured it in 28 runs (§4, p. 5) and noted that these tests do not prove it (“…without proving strict order-blindness or explaining training’s choice”). For clarity, the revision now adds a corollary to the main text that turns this measurement into a bound, with the proof in the new App. S.

(Q3): Our initial submission already included Transformer geometry analysis: “we also find internal geometries reflecting learned abelian quotients in Transformers (App. R).” In App. R, we analyzed 40 Transformer endpoints across eleven groups: “We find that their mean hidden vectors usually occupy a small subspace whose geometry agrees with the abelian quotient. Replacing the activation in this subspace transfers the predicted class at the intervention position.” In 38 of the 40 endpoints, “the retained rank and the fitted maps’ trace and determinant patterns agree with a faithful real representation of minimum dimension.”

If this resolves your concerns, we would appreciate it if you would consider updating your assessment. If not, we are happy to clarify anything further.

Reviewer_HBQEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies partial state tracking in Transformers and recurrent architectures, including LSTMs, GRUs, and Mamba-2, on group-composition tasks. It shows that partial tracking can arise when a model identifies a quotient class (coset) of the group but cannot distinguish individual states within that coset. In Transformer experiments, the authors find that the abelianization quotient often explains the observed accuracy plateaus: models identify the corresponding cosets accurately, while conditional exact accuracy remains near 1/∣H∣1/|H|, where HH is the commutator subgroup.

They also go beyond the abelianization quotient in selected experiments, demonstrating tracking of finer quotient groups, which can be non-Abelian and yield higher exact accuracy. Standard Transformers and Mamba-2 exhibit persistent quotient-tracking solutions. Although the authors do not establish that Transformers are fundamentally unable to track cosets of non-normal subgroups, they do not observe such tracking in their standard Transformer experiments.

In contrast, LSTMs and GRUs can pass through intermediate stages in which they track cosets of non-normal subgroups before reaching exact state tracking. The authors further provide causal evidence that these coset representations are encoded in low-dimensional subspaces of recurrent hidden states.

Critical strengths

1- This work provides a useful perspective on state tracking in Transformers and recurrent architectures, including RNNs and SSMs. Much of the state-tracking literature I am familiar with focuses on whether models can achieve perfect tracking of individual group elements. In contrast, this work investigates why models can exhibit above-random but imperfect tracking, showing how such performance can arise from tracking quotient classes or cosets rather than individual group elements. This offers a valuable contribution to understanding the internal representations underlying partial state-tracking performance.

Critical weaknesses

1- It remains unclear whether and how the identified quotient- and coset-tracking mechanisms could help predict or improve state-tracking performance across architectures. While the paper provides a useful explanation for partial tracking, the implications of these findings for model design and performance are not fully explored.

Questions for authors

Clarification Questions

  1. Did the authors investigate whether models continue to track the same cosets when evaluated on sequences longer than those used during training?

  2. What is the main conclusion of Section 3.4 regarding tracking finer quotient groups? Can models track these quotients as reliably as the abelianization quotient? It would be helpful to compare performance on abelianization and finer-quotient tracking using the exact and quotient frontiers defined in the paper.

Clarification / Future Work Suggestion

  1. The paper focuses on explaining partial state tracking rather than generalization to longer sequences, which is a central concern in much of the recent literature on state tracking with RNNs and SSMs. Do the identified quotient- and coset-tracking stages have any implications for length generalization? Clarifying this connection, even as a direction for future work, would help position the findings in the broader state-tracking literature.
6 comments
AuthorsOpenReview ↗

Response to Reviewer HBQE

We thank the reviewer for the accurate summary and the constructive questions.

W1 (do the mechanisms help predict or improve state tracking across architectures?). Thank you for the insightful question, and our marked revision now adds §5.3 and an Implications paragraph in §8 for this.

Prediction can be made. Under uniform i.i.d. full-group inputs, the conditional accuracy at the plateau is 1/|[G,G]| for each group, which follows from the group alone (Table 1). In recurrent models, incorrect predictions stay in the true coset Hq, which identifies the subgroup H of each stage, while the fraction in the left coset qH matches the algebraic prediction (§5.2, Fig. 3d). This contrast matches how each architecture can compute. Constant-depth Transformer constructions (Liu et al., 2023) go through normal subgroups, computing a quotient first. A5 has no proper normal subgroup, so this route offers no partial stage, and our baseline Transformers stay at chance on A5 (Table 1). A step-by-step recurrent update only needs right cosets, which A5 does have, and the LSTMs and GRUs that learn A5 pass through C3-coset stages on the way (§5.2). Lastly, for restricted inputs, an algebraic criterion says when a counting solution exists. In a preregistered test, recipe B reaches exact tracking on {r, s} and stays at the plateau on {r, s, e} (§5.3).

Improvement. At a plateau the class is already tracked, and what is missing is the order information inside the class (§4). Improving beyond the plateau requires distinguishing states within the tracked class. The recurrent results motivate feedback as a possible way to do this. We find that all 36 parameter-matched LSTM and GRU runs on D4, A4 and S4 reach exact tracking (App. N). Mamba-2, which passes only a linear state with a scalar decay between positions, stops at the abelianization like the Transformers (§5.1). To see this for Transformers, we adapted recirculation (Mozer et al., 2026), where we fed the previous position's final representation back into a recipe-A model on S4. Without feedback the three seeds end at the plateau, with exact accuracy .081–.084 against 1/12 ≈ .083. With four rounds of feedback they end at .140, .271 and .541, while shuffled feedback stays at .084 (App. B.2). No run reaches exact tracking yet, so we present this as a design direction.

Q1 (are the same cosets tracked beyond the training length?). We evaluate to position 128 with a training length of 100. The Transformers lose the coset there. For the D4 baseline, quotient accuracy is .98 at position 90, .85 at 100, .34 at 110 and .25 (chance) at 128, while the ratio of exact to quotient accuracy stays between .48 and .52. The matched LSTMs and GRUs keep exact tracking through position 128, with 35 of 36 runs at 1.0000 and one at .9998. App. N now reports this comparison, next to the existing Table 18 and Fig. S5b.

Q2 (main conclusion of §3.4, and whether finer quotients are tracked as reliably). To answer the question: they are not. Under full-group inputs, we observe no finer quotient. In the five-million-update S4 run, the abelianization is tracked at 99.67%, while S3 class accuracy stays below 34.1%, near the 1/3 expected from parity alone (§5.1). Finer quotients appear under recipe B with restricted inputs, where inputs are sampled only from the 12 odd permutations in S4. With the frontiers you suggest, the S4 example has an S3 frontier of 54 (at .90) and an exact frontier of 6 (at .75). The other two S4 seeds show no S3 stage. However, on these odd-permutation inputs, abelianization class is fixed by position (simple counting of number of tokens gives parity), so its frontier of 100 does not test tracking. We add this discussion into §3.4 in this revision.

C1 (length generalization). Q1 gives the connection we can support. A sequential update of the state, or of a right coset, does not depend on position, and the LSTMs and GRUs keep exact tracking beyond the training length. The Transformers' quotient solutions do not extend past the trained positions. Whether the intermediate coset stages of recurrent models also hold beyond the training length is future work, which §8 now names.

AuthorsOpenReview ↗

Follow-up: coset stages beyond the training length

Dear Reviewer HBQE,

We would like to add to our previous reponse to your Q3 that asked what the quotient and coset stages imply for length generalization (we ran tests to seqence length of 1000), the central concern of the RNN and SSM literature.

First, a length test at the plateau shows whether a model carries its partial state from step to step. Models stuck at the same quotient look identical within the training length but separate beyond it:

  1. our new results show that LSTMs keep the quotient state to ten times the training length even during the learning phase;
  2. Mamba-2 loses it, as Sarrof et al. (2024) and Grazzi et al. (2025) predict for transitions with only nonnegative eigenvalues;
  3. our Transformers lose it almost at once (App. N).

Hence, the expressivity already holds at the first stage of learning, the quotient.

Second, our account gives what a model learns when it fails. Shakerinava et al. (2026) show that stacked diagonal SSMs can express some non-abelian groups, yet often fail to learn them, and they report these runs only as failures (Table 2 in their paper). Our account predicts where such runs stop: at the abelianization, the largest part of the group that a single diagonal layer can represent. A natural next experiment is to train their SSMs on generator inputs, which can move our Transformers past the abelianization (§5.3), and to see whether this unlocks exact tracking.

We have updated the Implications paragraph in §8, reference, and App. N accordingly. If this resolves your concern, we would be grateful if you would consider reflecting it in your rating. If something is still missing, we would be glad to hear what it is.

I thank the authors for answering my questions regarding the details of the experiments, providing additional experimental results and also updating the draft.

It still remains unclear to me what are the practical implications of these results for improving models performance on tasks requiring state-tracking. For example, adjusting a language model training to require it to learn not just based on the exact state accuracy, but also coset tracking, seems to demand a prior knowledge of the group structure in the data. It is not clear to me how one can practically apply this to improve their training.

I maintain my score of 3.

Reviewer_AudXOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Authors argue that algebraic properties of the group structures used to formally model state-tracking problems are visible in and causally-relevant to the solutions that language models learn when directly trained on these problems. In particular, authors hypothesize that models will learn to identify classes of partial solutions to state-tracking problems by learning to represent the cosets of solutions without needing to necessarily pick out a particular solution (eg, they will learn to represent solutions as orbits rather than necessarily learning exact solutions); over the course of training, these representations become more fine-grained, ideally resolving to cases where models can exactly identify a solution; but in cases where they cannot, they will learn to represent solutions "up to equivalence" with other members of the equivalence class, whose size is predictive of the accuracy of the model at that stage of training. Authors observe that empirically, the choice of architecture affects what kinds of cosets can be learned: transformers appear restricted to learning cosets identified by normal subgroups, while (linear) RNNs have no such limitation; this is relevant when the groups in question are simple, i.e., contain no nontrivial normal subgroups, as this limits the intermediate representations that transformers can learn such that there are no "rungs on the ladder" for partial accuracy available to transformers over the course of training. Authors also observe that the activations of states in well-trained models (under SVD) form approximately-geometric representations of the state space in question.

Critical strengths

The connection between the algebraic properties underlying state tracking and the empirical results, both in terms of overall accuracy and the learning dynamics over the course of training, is interesting and compelling. I think the results are neat and, although I've had a similar hypothesis/assumption before for what must be going on inside models re: learning of cosets/quotient groups as partial solutions, i think it's quite nice that the results are investigated and presented here.

Critical weaknesses

I think the paper suffers from two meaningful but fixable problems. The first concerns the presentation: the bulk of the paper focuses on a presentation of the results themselves, but there is not much to say which contextualizes these results in the broader landscape of LLM research. Assuming their results hold, it's unclear what is actionable or useful beyond the recognition that they are true. The second concerns the connection to other literature: I think there is room to relate both the empirical results and theoretical results to two bodies of work: first, concerning the implications that the choices of architecture have on the results, the authors do not spend much time talking about why the various state tracking problems (eg, S3, A5, etc) have been chosen and what their differences mean vis-a-vis architectures; the two main citations for these would be Liu et al. (2023), for Transformers; and Merrill et al. (2024), for (linear/non-linear) RNNs; both works are cited here but not really in connection to the impact that architecture has on the expressibility of these problems. Particularly since the authors find an interesting result regarding whether architectures can learn non-trivial quotient groups as intermediate steps towards a full solution, I think spending more time discussing what theory predicts should be possible and contextualizing the interesting results along those lines would be well worth it. Hu et al. (2023) "Latent State Models of Training Dynamics" may also be relevant tangentially as a discussion of learned intermediate representations of simpler group learning problems. The second body of work that should be engaged with is work done on understanding the geometry of representations. I am less familiar with this, but I know at least that Goodfire has a series of papers on this (https://www.goodfire.com/research/neural-geometry) which seem quite relevant. Engaging with both these areas of research more thoroughly would be well-received and strengthen the paper's contributions to literature.

Questions for authors

Beyond the more conceptual points raised in the Weaknesses section, I have some more low-level questions. First, can the authors clarify exactly what the representation of the input sequences is? Are they sequences of products with their associated values written out (eg, a * b * c * ... * z = k)? If so, are models evaluated on the accuracy at each position? What tokenization scheme is used? Regarding tables and figures, what do the vertical lines at P=4 represent in Fig. 2?

5 comments

We thank the reviewer for the careful reading and the encouraging assessment of the results.

W1 (broader context and what is actionable). Thank you for this point. Our marked revision now contains a section §5.3 and an Implications paragraph in §8, which say what the results let one evaluate and change.

  1. Evaluation: partial accuracy on a state-tracking task can come from a model that tracks a coarser state exactly. Accuracy alone cannot tell this from noisy tracking of the full state, which matters when state tracking and world models are evaluated in language models (Kim and Schuster, 2023, and Vafa et al., 2024). Quotient accuracy shows that the class is tracked, the within-class divergence shows that the output carries almost no finer information, and the class size gives the accuracy with no fitted parameter (Table 1).
  2. Control: two levers can move a model past the plateau. One is the input distribution. Recipe B gives 50.0% conditional accuracy on full-group inputs and 99.999% on generator inputs (Table 7), and no variant leaves the plateau on full-group inputs (Fig. S7). However, the gain on generators depends on the block and position encoding, a phenomenon we do not full understand yet (Table 10). The other is output feedback, where each position's output enters the next position's computation. Parameter-matched LSTMs and GRUs reach exact tracking in all 36 runs on D4, A4 and S4 (Apps. I and N). Recirculation raises a Transformer's exact accuracy on S4 from .081–.084 to .140–.541, while shuffled feedback stays at .084 (App. B.2). However, no run reaches exact tracking, leaving it a design direction for Transformers. Our main results use models trained from scratch. Whether the same coset geometry appears in pretrained language models is a direction we plan to explore.

W2 (literature and choice of groups). We add the points below to §5.2, §7 and §3.1.

  • What theory predicts (new paragraph closing §5.2): Transformers can express tracking for every finite solvable group at constant depth, and not for the non-solvable A5 and S5 under standard complexity assumptions (Liu et al., 2023). Their construction computes a quotient by a normal subgroup first, and an abelian quotient needs one layer of counting. The abelianization is the largest quotient that counting can compute, so the plateau matches the first stage of that construction. A5 has no nontrivial proper normal subgroup, hence the construction has no intermediate stage. Our 4-layer Transformer models are at chance on A5 and track only parity on S5, leaving its A5 factor unresolved (Table 1). A sequential update, on the other hand, only needs right cosets. We find that the LSTMs and GRUs pass through such stages on A5 and S5. Merrill et al. (2024) put diagonal state-space models in the Transformer's class and show that one nonlinear recurrent layer tracks S5. This matches what we see: Mamba-2 stops at the abelianization on D4, A4 and S4 (§5.1), and the LSTMs and GRUs reach exact tracking.
  • Liu et al.'s dihedral, symmetric and alternating experiments use generating sets. Ours use the full group, which is where the plateau appears. With their recipe and generator inputs we reproduce exact tracking on D4 (App. O).
  • Hu et al. (2023) detect training phases with a hidden Markov model over training statistics. Our readout directly identifies which quotient or coset each phase tracks.
  • Representation geometry: the series the reviewer points to finds curved, low-dimensional structure in activations and steers along it (Wurgaft et al., 2026), including circular representations of cyclic concepts (Feucht et al., 2026). Our setting differs. In §6 a group action fixes the geometry, as §7 now notes. The dodecahedral class means are the orbit of A5 on its C3-cosets, and the fitted maps between them are rotations that compose like the group.
  • Why these groups: the non-abelian groups of order ≤ 12 form a complete census. A4 and SL(2,3) share the abelianization C3 with class sizes 4 and 8. A5 is perfect, so the predicted plateau equals chance, and S5 has parity as its only nontrivial proper quotient. The Heisenberg groups compare finite with infinite, and C8 is the abelian control.

Questions. Thank you for raising these questions, and we have clarified them in our markup revision (§3.1).

  • Input representation and tokenization: in the finite-group experiments under recipe A, each token is one group element and the alphabet is the group (|G| tokens). There is no start token, operator nor "=". The output at position t is a distribution over G for the running product. §3.1 now states this.
  • Per-position evaluation: yes. Loss and accuracy are computed at every position of length-100 sequences.
  • Lines at P = 4 in Fig. 2: they mark the exact frontier of the displayed A4 model. We edited the caption to clarify.

Thank you for your response. I think the technical points are addressed well enough, but I am still a little uncertain about the broader implications of the work. Your W1 response does not yet address the broad question of what the results of your study mean for the field. Eg, suppose we take the conclusion as given that models will learn coset representations of these group problems during training. It would be very helpful to be able to understand what the implications of that are.

AuthorsOpenReview ↗

Broader implications to the field

Dear Reviewer AudX,

Thank you for the timely follow-up, and we are happy that the technical points are addressed. Thank you also for pressing on this point. Your question genuinely pushed us to step back from the task-level details and think much more carefully about what these results mean for the broader field. Taking the conclusion as given, that models learn coset representations of these problems, we do see four implications beyond our setting.

  1. A plateau in accuracy is not necessarily a failure to learn. When exact accuracy stalls, the natural reading is that the model has learned little or tracks the state noisily. Our results instead show that the model tracks a coarser state exactly and is uncertain only within it. For language models this becomes a testable hypothesis. In tasks such as tracking objects through swaps, the natural coarser state is whether the number of swaps so far is even or odd, and Li et al. (2025) find that language models trained on such tasks can rely on exactly this parity. A stalled model can hence be checked for whether it has resolved this coarser state, rather than judged by exact accuracy alone.

  2. Apparent state tracking can be a mirage. On restricted inputs, a model can get every answer right by counting instead of composing updates, and our results show that this success does not transfer to unrestricted inputs. Since the algebra says in advance when such a shortcut exists, a benchmark or curriculum can be designed and checked before its accuracy is taken as evidence of state tracking.

  3. Expressivity does not guarantee what a model will learn Expressivity results (Liu et al.; Merrill et al.) concern what state an architecture can represent. Our results also ask how it gets there. Unlike the Transformers we trained, recurrent networks refine through intermediate coset states that are not quotients. Mechanisms that feed states forward may therefore matter beyond expressive power: recirculation, which we test, moves a Transformer past the plateau, and whether chain-of-thought does the same is testable with our readout.

  4. World-model claims need interventions, not just decoding or geometry. It is tempting to read a decodable, well-structured representation as evidence that a model tracks the state. Our results show that this is not enough. Transformers and recurrent networks both learn the geometry the group predicts, but only the latter uses it as a state. Strikingly, in recurrent networks, each input acts on the coset means as a rotation, and these rotations compose like the group, as a genuine state update should. Our intervention results further show that swapping this subspace carries the donor’s state through later inputs. Transformers show the same kind of geometry at the current position, but replacing a position’s hidden state does not carry the donor’s class to later positions: they behave as if recomputing the class from the inputs, an order-free shortcut of the kind described by Liu et al. and for permutations by Li et al. For world-model studies of language models, this gives a direct, actionable test: a representation is a tracked state only if intervening on it changes what the model does later.

These implications come from controlled group tasks. Testing them in language models is the next step, and quotient accuracy makes that test directly measurable. We have added these points to the Implications paragraph in §8.

We hope this directly addresses your question, and we are grateful for the push to articulate the bigger picture. Please let us know if anything remains unclear. Otherwise, we would be very grateful if you might consider reflecting these updates in your rating.

Comments on the paper

1 comment

We thank all three reviewers. The revised manuscript contains the following changes. In the marked manuscript, we use different colors for three reviewers: reviewer AudX (green), reviewer HBQE (orange), and reviewer y7zo (blue). Changes addressing multiple reviewers are shown in purple, and removed text is red and struck through.

  1. §3.1 clarifies the purpose of recipes A and B and gives the rationale for the group selection (y7zo, AudX).
  2. §3.1 states the input format, and the caption of Fig. 2 explains the P = 4 line (AudX).
  3. §3.4 clarifies the conclusion on finer quotients and reports the exact and finer-quotient frontiers (HBQE).
  4. §4 states a bound that connects the theorems to trained models. The corollary, its proof and the reordering measurement are in the new App. S (y7zo).
  5. A new §5.3 names two levers that move a model past the abelianization plateau, namely the input distribution and feedback of each position's output into the next, and §8 adds an Implications paragraph. The last sentence of the abstract and one sentence in §1 now state this (y7zo, AudX, HBQE).
  6. §5.2 closes with a comparison to the expressivity results of Liu et al. (2023) and Merrill et al. (2024). §7 adds Hu et al. (2023) and work on representation geometry (AudX).
  7. Appendix H opens with a map of configurations. Appendix I is reorganized and has a new robustness figure (y7zo).
  8. App. N and Fig. S10 test partial states to position 1000. LSTMs keep the quotient, while Mamba-2 and our Transformers lose it, and recurrent coset stages extend far past the training length during partial learning stage (HBQE, y7zo).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.