acceptodds
Under review as a conference paper at ICLR 2027

What Can Be Identified in a Transformer?

Abstract

Mechanistic interpretability names parts of a network (heads, neurons, residual directions), yet many weight settings compute the same function. We ask which of these parts the function determines, and obtain three results. First, a multi-head attention layer is identified exactly: its function determines precisely the measure that assigns to each query–key form the summed value–output maps of the heads using it, already from inputs of length three, so head permutations and per-head gauges are its only symmetries on generic weights. Second, the symmetries of deployed transformers depend on their architecture: rotary embeddings, QK-normalization and attention sinks constrain the gauges, and normalization placement decides whether the residual stream has a privileged basis (none in pre-norm models, only signed permutations when sublayer outputs are normalized). We verify every prediction, with a negative control, on nine checkpoints of seven families up to 32B parameters in float64. Third, these symmetries sort interpretability claims: claims about a checkpoint must be equivariant, claims about its function invariant, and claims about what training produces must replicate across runs. Across independently trained seeds of Pythia and GPT-2, head indices carry no information about IOI or induction attributions, whereas heads matched by symmetry-invariant signatures, and label-invariant statistics, replicate.

Primary area:
learning theory
Keywords:
Mechanistic Interpretability, Non-Identifiability, Symmetry Groups, Gauge Transformations, Transformer Architectures, Attention Head Relabeling, Representation Learning, Sparse Autoencoders

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PJauOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The authors use a gauge theory formalism to describe which transformations of the heads leave the outputs of the attention mechanism unchanged. The papers main theoretical contribution is Theorem 4.1 which claims that the set of symmetries they identify are all the possible symmetries of the head parameters that leave the output unchanged. They identify how changes in the architecture (RoPE, QK-normalisation, grouped query attention etc.) change the symmetry group. Experimentally they do two things, they test these symmetries by pairing predicted symmetries with plausible but non-predicted "random control" symmetries and then "editing" the weights of several LLMs and seeing whether they can distinguish between outputs of edited and non-edited models for each of the symmetries. Secondly they determine if these symmetries can be used to match heads between training runs.

Critical strengths
  1. I think a complete enumeration of the symmetries inherent in transformer models and developed theory on how to find these for different architectures, as well as a consideration more generally for whether mechanistic interpretability methods are robust to the sorts of across training runs and transformations of the weights is valuable for the field.

The field has been vulnerable to interesting sounding results that end up not surviving controls or that oversell, so an attempt to put this on a more rigorous footing is welcome.

  1. The paper gives concrete recommendations in section 4.4 on how practitioners should interpret and utilise this work.

Ultimately the goal of MI is applied in that we want to use these methods to understand LLMs and (usually) make them safer. Without this the work would be a bit too abstract.

Critical weaknesses
  1. I found this paper quite hard to follow.

I, and I imagine most other MI researchers, don't have much knowledge of gauge theory, and this paper isn't especially sensitive to this. At numerous points notation is introduced without any introduction or explanation. There are a few minor examples of this like using ϵ\epsilon before it is clear you mean the ϵ\epsilon in RMSnorm, or not defining θ\theta, but the worst offender is not defining what GLGL or SHS_H are. I understand these are common gauge theory terms but since most will not know much about that I think the paper would benefit from a bit more explanation and intuition. The paper by Godfrey et al. that you cite is a good example of giving intuition for work like this.

More generally the paper has some odd structural choices. Why is the rounding pitfall so prominently discussed? This only becomes clear towards the end of the paper, and even then bringing it up right at the beginning is odd since it seems like a minor correctness point? This has something to do with how numerical precision interacts with the symmetries you are studying but this isn't adequately explained.

Questions for authors

N/A

3 comments
AuthorsOpenReview ↗

Reply to Reviewer PJau: presentation

Thank you. We agree that the paper assumed too much background. The revised version makes three changes.

Notation is defined where it first appears (Sec. 3). GL(k)GL(k) is the group of invertible k×kk\times k matrices, and the gauge has one GL(dk)GL(d_k) and one GL(dv)GL(d_v) factor per head. SHS_H is the group of permutations of the HH heads, and ⋊\rtimes records that relabelling the heads also relabels their matrices. θ\theta denotes all weights, and ε\varepsilon, the RMSNorm stabilizer, is introduced before it is used. The word "gauge" needs no gauge theory here. It only means a change of weights that leaves the function unchanged, and the text now says this.

A short paragraph before Thm 4.1 gives the intuition. Inserting ShSh−1S_hS_h^{-1} between the query and key projections of a head changes both matrices but not the attention scores, so a statement that depends on the individual matrices cannot describe the function. Thm 4.1 and Cor. 4.2, which the revision credits to Tran et al. (2025) and Kim & Kim (2026), show that for generic weights head relabelling and such insertions are the only changes of this kind.

The rounding pitfall is a point of method for anyone running such checks, and we no longer list it as a contribution. The test has to separate exact symmetries from controls. Some implementations compute normalizations in float32 inside a float64 model. A control that changes one weight by a relative 10−1010^{-10} then gives bit-identical logits, so a broken symmetry looks exact. The revision moves this point from the introduction to the method paragraph of Sec. 5 and explains it there.

AuthorsOpenReview ↗

Reviewer PJau: where your points are in the revised PDF

In the revised PDF, the points of your review are addressed at the following places.

Notation (Sec. 3, p. 3). GL(k)GL(k), SHS_H, ⋊\rtimes and θ\theta are defined at first use. The words "gauge", "orbit" and "generic" are explained in plain terms, and no gauge theory is needed. ε\varepsilon is introduced as the stabilizing constant of RMSNorm, and it no longer appears in the introduction.

Intuition (Sec. 4.1, p. 4). The paragraph "Intuition" before Thm 4.1 explains, with Figure 1, why inserting ShSh−1S_hS_h^{-1} inside a head changes the weights and not the function. Two shorter paragraphs do the same for rotary embeddings (after Prop. 4.3, p. 5) and for the residual basis (after Prop. 4.9, p. 7).

Rounding pitfall. It is no longer in the introduction or in the contributions. The protocol of Sec. 5.1 (p. 8) explains why it matters: a broken symmetry could look exact. Details are in App. B.1 (p. 19).

What you found useful

  • The enumeration of symmetries: Sec. 4.2-4.3 (p. 5-7), and Table A (p. 8), which shows which symmetry holds on which architecture.
  • The recommendations: Sec. 4.4 (p. 7), which now includes a numerical test of a claim on three models, and Table 9 (p. 25), which gives an invariant replacement for common objects, including sparse autoencoder features.
  • Robustness across training runs: Sec. 5.2 and Figure 3.

Thank you for taking the time to address my concerns. While you've addressed the lack of sufficient background knowledge which has improved the legibility of the manuscript, having read the other reviews I agree that this paper needs further work before it is ready for acceptance. As such I maintain my score.

Reviewer_MtBpOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks which parts of a transformer are determined by its function.

  1. For bias-free attention with bilinear scores, the function determines exactly μθ=∑hδMhNh\mu_\theta=\sum_h \delta_{M_h} N_h. Inputs of length three suffice and length two does not (Thm 4.1). Hence, on generic weights, head permutations and per-head GL\mathrm{GL} gauges are the only symmetries (Cor 4.2).
  2. For RoPE, GQA, QK-norm, biases/sinks, SwiGLU and MoE, it gives exact gauges but does not prove completeness. In the residual stream, pre-norm models admit every orthogonal change of basis. Normalizing sublayer outputs leaves only signed permutations among linear changes.
  3. It proposes that checkpoint claims are equivariant, function claims invariant, and training claims replicate across seeds.
Critical strengths
  1. Correct exact symmetries for the architectural features of current LLMs. For RoPE, GQA, QK-normalization, attention biases and sinks, SwiGLU, mixture-of-experts, and the placement of normalization, the paper derives weight transformations that leave the function exactly unchanged. Some of them could matter in practice.
  2. Broad, well-controlled verification The predicted symmetries are checked on nine trained models from seven families. Symmetries and controls are always separated by at least 10 orders of magnitude, so the verdicts are unambiguous. The released function that applies random exact symmetries lets others test their own claims the same way.
Critical weaknesses
  1. Most of the headline result is prior work.

    • For bidirectional attention, (ii)⇒\Rightarrow(iii) of Thm 4.1 is Cor. A.5 of Tran et al. (ICLR 2025), which the paper does not cite.
    • The generic part of Cor 4.2 is their Thm 3.2, up to the length quantifier, the causal mask and one-sided genericity.
    • The length-three criterion, with essentially the same proof, is Prop. 3.1 of the concurrent Kim & Kim (arXiv:2608.03294). The submission describes that paper only as recovering the canonical form "as the output of a query-learning algorithm", which should be corrected.
    • The RoPE commutant appears in Wang & Wang (NeurReps 2025).
    • What remains new is relatively little: the length-two counterexample, Prop 4.9, and the QK-norm and sink refinements.
  2. The theory does not reach deployed models.

    • None of the nine checkpoints satisfies the hypotheses of Thm 4.1, and for GPT-2 the paper's Table 2 shows symmetries outside GG.
    • For deployed models the paper only exhibits symmetries, and its residual-stream statements cover linear maps only. As App. D concedes, the test is necessary, not sufficient.
    • There is no stability result.
Questions for authors
  1. How do Thm 4.1 and Cor 4.2 relate to Cor. A.5 and Thm 3.2 of Tran et al. (ICLR 2025)? What does Thm 4.1 add beyond Prop. 3.1 of Kim & Kim applied to each output coordinate?
  2. Do you agree that length-two inputs already suffice for Cor 4.2 on the open dense subset of UU where no Mh=−MkM_h=-M_k and no non-empty subset of the NhN_h sums to zero?
  3. Can you give a stability bound for Thm 4.1, e.g. in terms of the quantities in App. B.4?

If these questions, in particular the first, are adequately adressed by the authors I would be willing to adjust my rating.

2 comments
AuthorsOpenReview ↗

Reply to Reviewer MtBp: Q1-Q3 and scope

Thank you for the precise mapping. We checked it against the three papers and agree with it.

Q1. Thm A.1 and Cor. A.5 of Tran et al. give (ii)⇒\Rightarrow(iii) of Thm 4.1 without mask, and their Thm 3.2 gives the generic part of Cor. 4.2 up to the differences you list. Prop. 3.1 of Kim & Kim, applied to each output coordinate, gives (i)⇒\Rightarrow(iii). Since it only reads the final token, it also covers the causal mask. Thm 4.1 therefore adds nothing of substance, and our proof follows theirs step by step (Lemmas A.1-A.2 of their supplement). We cited Kim & Kim only for their recovery algorithm and presented the length-three proof as our own. That was a mistake. The revised paper states Thm 4.1 as a known result and credits the proof to Kim & Kim.

What is left in Sec. 4.1 is the description of the fibres off U\mathcal U and the length-two counterexample, which by your Q2 only matters off a generic set. We accept your list of what remains new (the length-two counterexample, Prop. 4.9, the QK-norm and sink refinements). The paper also covers partial RoPE, which is a simple extension. Its experiments check the predictions on real weights and test a criterion for interpretability claims across seeds. We do not count these as new theory.

Q2. Yes. On inputs (v+w,v)(v+w,v) the last row is v∑Aμ(A)+∑Aσ(vAw⊤)wμ(A)v\sum_A\mu(A)+\sum_A\sigma(vAw^\top)w\mu(A). The functions 11 and σ(γt)\sigma(\gamma t) for distinct γ>0\gamma>0 are linearly independent, because the pole iπ/γmax⁡i\pi/\gamma_{\max} belongs to γmax⁡\gamma_{\max} alone (then induct). For generic ww and v=tzv=tz, equal outputs give, with D=μθ−μθ′D=\mu_\theta-\mu_{\theta'}, D(A)=D(−A)D(A)=D(-A) for A≠0A\neq0 and D(0)=−2∑{A,−A}D(A)D(0)=-2\sum_{\{A,-A\}}D(A). If no Mh=−MkM_h=-M_k, a parameter θ′\theta' of the same architecture with HH heads can only keep or flip each head, and the NhN_h of the flipped heads must sum to zero. If no non-empty subset of the NhN_h sums to zero, then D=0D=0 and Cor. 4.2 applies. Length two is thus enough on that open dense subset of U\mathcal U, and length three is only needed for Thm 4.1 without genericity. The revision adds this to Sec. 4.1 and App. A.2, and the abstract no longer suggests otherwise.

Q3. Not globally. The pole argument gives no quantitative bound. Kim & Kim state conditions under which the error of their recovery is controlled by the output error, with a constant that depends on the model and the queries, and the revision cites this. A global bound in terms of the margins of App. B.4 is open, and the revised paper lists it as a limitation. App. E gives a local bound, with a constant that we measure in terms of these margins.

Scope. We agree. Thm 4.1 is about bias-free bilinear scores, and its statement now says so. GPT-2 is no longer given as an example, since it has biases and Table 2 shows bias symmetries outside GG for it. For deployed models the paper only exhibits symmetries (Sec. 4.2-4.3). The revised text treats them as bounds on what the function determines (Sec. 3), states that the residual-stream results concern linear changes of basis, and moves the caveat of App. D ("necessary, not sufficient") into the main text. Completeness for full RoPE without mask is known from Tran et al. (2026), and it follows for partial RoPE from their Thm 4.1 (App. A.4). It remains open for RoPE under a causal mask, QK-normalization, sinks and biases.

AuthorsOpenReview ↗

Reviewer MtBp: where your points are in the revised PDF

In the revised PDF, the points of your review are addressed at the following places. Section, theorem and table numbers are unchanged.

Q1. Thm 4.1 (p. 4) is stated as a known result, and the paragraph "Attribution" gives the source of each implication: it adds nothing of substance beyond Prop. 3.1 of Kim & Kim applied to each output coordinate. App. A.2 (p. 15) credits the proof to Kim & Kim. Their paper is described correctly in Sec. 2 (p. 2). The generic part of Cor. 4.2 (p. 4) is credited to Thm 3.2 of Tran et al., and the next paragraph compares the assumptions. The contribution list (p. 1) now rests on what you identified as new and as strengths.

Q2. The paragraph "Inputs of length two" (p. 5) states the result on the open dense subset you describe. The proof is in App. A.2 (p. 16), the numerical checks are in App. E (p. 27), and the abstract no longer suggests otherwise.

Q3. We have no global bound. The pole argument is not quantitative, and Cor. D.9 of Kim & Kim shows that no model-independent constant exists; the limitations (p. 10) say so and point to their Thm 5.2. What we can give is local (App. E, p. 26): near atoms with a full-rank Jacobian, the change of the outputs is at least 12σmin⁡\frac12\sigma_{\min} times the change of (Mh,Nh)(M_h,N_h). In terms of the quantities of App. B.4, on small random layers σmin⁡\sigma_{\min} scales as the square of the QK separation and linearly with the OV mass and with the σmin⁡/σmax⁡\sigma_{\min}/\sigma_{\max} of the factors. These dependences are measured, not proved. On the states of a trained model the constant is small (Sec. 5.1, p. 9).

The theory does not reach deployed models

  • Thm 4.1 (p. 4) says "bias-free", GPT-2 is no longer an example, and the text says that none of the nine checkpoints has this form.
  • Sec. 3 (p. 3) reads the exhibited symmetries as bounds on what the function determines, and separates a symmetry from a function-preserving rewrite.
  • The residual-stream statements are restricted to linear changes of basis (start of Sec. 4.3, p. 6), and "necessary, not sufficient" is in Sec. 4.4 (p. 7).
  • Completeness for full RoPE is credited to Tran et al. (2026) (p. 5). It also follows for partial RoPE without mask from their Thm 4.1 (App. A.4, p. 17). The open cases are listed in the limitations (p. 10).
  • Sec. 5.1 (p. 9) and Table 10 (p. 27) compare the Jacobian null space with the listed symmetries on small random instances. The dimensions coincide in all 23 cases, including RoPE under a causal mask, QK-normalization, sinks and whole networks. At those weights this excludes other continuous symmetries, linear or not. It is local evidence, not a proof.

Prop. 4.9 (p. 6). The hypotheses are explicit and the conclusion is conditional on them. The proof is repaired (App. A.5, p. 18), and the hypotheses are measured on both OLMo-2 checkpoints (App. E, p. 26): they hold on four attention sublayers, with ∑hNh\sum_h N_h invertible but ill-conditioned.

Reviewer_mBSHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper provides complete identifiability of multihead attention and rotational position embedding model. The paper also investigate the summetry causes by nomalization. Experimently, the paper demonstrates that their recovered symmetry keep the functional of the neural network unchanged across multiple public model checkpoints.

Critical strengths

None

Critical weaknesses
  • Theorem 4.1 and Proposition 4.3 substantially overlap with previous work [1, 2], but the paper provides neither citations nor a clear discussion of the differences. In particular, the all-sequence-length, bidirectional identification in Theorem 4.1 follows from the head-independence result in [1], while the central RoPE symmetry characterization in Proposition 4.3 is already established in [2]. The authors should clarify what the fixed-length-three result, causal extension, and treatment of partial RoPE add beyond these works.

  • The empirical results mainly verify the symmetry predictions on pretrained networks and examine their implications for interpretability across training runs, rather than introduce a new application that uses these symmetries. These experiments provide useful validation, but their additional contribution needs to be made clearer, especially given the substantial overlap in the theoretical results.

  • Minor presentation detail: Figure 1a is cut off at the left border.

[1] Hoang et al., Equivariant Neural Functional Networks for Transformers, ICLR 2025.

[2] Hoang et al., Functional Equivalence in Attention: A Comprehensive Study with Applications to Linear Mode Connectivity, ICML 2026.

Questions for authors
  • Can the authors clearly identify the differences between Theorem 4.1 and Corollary 4.2 and the results in the aforementioned works, including their assumptions, conclusions, and proof ideas?
  • Can the authors provide a more extensive review of existing work on symmetries of Transformer models and clearly explain what this work adds beyond those results
5 comments
AuthorsOpenReview ↗

Reply to Reviewer mBSH: differences with [1, 2], related work, ethics flag

Thank you. [1] and [2] should have been cited and discussed. We no longer claim Thm 4.1, the generic part of Cor. 4.2, or the RoPE gauge for full RoPE as contributions.

Q1. Thm A.1 and Cor. A.5-A.6 of [1] assume that two MHA maps agree on every sequence length, without mask, and conclude that the grouped sums ∑h:Mh=ANh\sum_{h:M_h=A}N_h, our μθ\mu_\theta, coincide. Thm 4.1 reaches the same conclusion from length-three inputs, with or without a causal mask. This refinement is not ours either. It is Prop. 3.1 of the concurrent paper of Kim & Kim, which we had cited only for its recovery algorithm, and our proof follows theirs. The proof in [1] is different and varies the length (Lemma A.4).

Thm 3.2 of [1] gives the generic part of Cor. 4.2. It assumes both parameter settings generic and full rank of the d×dd\times d products, while Cor. 4.2 assumes only θ\theta generic and full rank of the factors. These differences are minor. Cor. 4.2 also describes the fibre off the generic set, which [1] does not.

[2] proves the RoPE gauge (C×)dk/2(\mathbb C^\times)^{d_k/2} and completeness for full RoPE (its Thm 4.2), which we had wrongly called open. Our Prop. 4.3 also covers partial RoPE, where only r<dkr<d_k coordinates are rotated, as in Pythia. The extension is simple, since the gauge splits into scaled rotations on the rotated pairs and GL(dk−r)GL(d_k-r) on the rest. It matters in practice because it decides which part of the key bias can be shifted without changing the function (Prop. 4.6), as Table 2 confirms on Pythia.

The revised paper states Thm 4.1 as a cited result ((ii)⇔\Leftrightarrow(iii) without mask from [1], length three from Kim & Kim), and Prop. 4.3 credits [2] and Wang & Wang.

Q2. The revised related work covers the attention layer ([1]; Kim & Kim, 2026; Wang & Wang, NeurReps 2025 workshop; Morrow, 2026), positional encodings ([2]; Wang & Wang), the residual stream and normalization (SliceGPT, QuaRot, SpinQuant; Theus et al., NeurIPS 2025; App. B of [1]), and the uses of these symmetries in merging and mode connectivity (Ainsworth et al., 2023; Imfeld et al., 2024; Verma & Elbayad, 2024; Zhang et al., 2025; Theus et al., 2025; [2]), weight-space networks ([1]) and representations (Godfrey et al., 2022).

The remaining theoretical results are elementary, and the revised paper says so. They are useful because they give the exact symmetries of architectures in use. QK-normalization moves the query-key gauge into the normalization gains (Prop. 4.5). Under RoPE (in the rotated coordinates) and with attention sinks, shifting the key bias is no longer a symmetry (Prop. 4.6). When sublayer outputs are normalized, as in OLMo-2, the only changes of residual basis that readers, writers and gains can absorb are signed permutations (Prop. 4.9, under hypotheses that the revision states explicitly). This last result is the least immediate. In this sense OLMo-2 has a privileged residual basis, while pre-norm RMSNorm models do not. All these results address the same question, which components of a deployed architecture the function fixes, and the answer depends on RoPE, QK-normalization, sinks and the placement of normalization.

The experiments on nine checkpoints of seven families (end to end up to 7B, layer by layer up to 32B, with minimally different negative controls) verify these predictions. As you note, this is not a new application.

The paper does not propose a new matching method and makes no claim against existing alignment methods. It states a criterion (Sec. 4.4). A claim about a head across training runs only makes sense once heads are aligned by a quantity the symmetries leave unchanged. Sec. 5.2 tests it on IOI and induction attributions. Compared by head index, agreement across seeds is at chance level. After matching heads by their attention patterns, agreement comes back strongly for induction heads and weakly for IOI. That indices disagree is expected. The section measures this against a relabelling null and shows how much agreement survives alignment.

Ethics flag. We agree that the issue is attribution and originality. The revised paper credits [1], [2], Kim & Kim and Wang & Wang in the statements themselves, not only in the related work. Its contributions are those listed above, and Thm 4.1, the generic part of Cor. 4.2 and the full-RoPE gauge are presented as known results. Omitting [1] was our error, and [2] should also have been discussed.

Fig. 1a is fixed in the revision.

AuthorsOpenReview ↗

Reviewer mBSH: where your points are in the revised PDF

In the revised PDF, the points of your review are addressed at the following places. Section, theorem and table numbers are unchanged.

Overlap with [1, 2], and what the length-three result, the causal extension and partial RoPE add

  • Thm 4.1 (p. 4) is stated as a known result, with [1] and Kim & Kim in its header. The paragraph "Attribution" says which implication comes from which paper. The length-three criterion and the causal mask come from Prop. 3.1 of Kim & Kim, and the start of App. A.2 (p. 15) says that the proof follows theirs.
  • Cor. 4.2 (p. 4): the generic part is credited to Thm 3.2 of [1] and Thm 2 of Wang & Wang, and the next paragraph compares the assumptions. What we keep is the description of the fibres off the generic set.
  • Prop. 4.3 (p. 5): the full-RoPE gauge and its completeness are credited to Wang & Wang and [2]. The next paragraph says what partial RoPE adds: a simple extension, complete without mask by Thm 4.1 of [2] (App. A.4, p. 17), which decides which part of the key bias can be shifted (Prop. 4.6, p. 6; Table A, p. 8).

Review of prior work (Q2). Sec. 2 (p. 2-3) covers attention layers, positional encodings, the residual stream and normalization, and the uses of these symmetries. The contribution paragraph of the introduction (p. 1) says what remains ours.

Additional contribution of the experiments

  • Table A (p. 8) shows which symmetry holds on which architecture.
  • Sec. 5.1 (p. 9) reports what the states of a trained model determine locally, and checks that the listed symmetries account for the whole Jacobian null space on small random instances (App. E, p. 26; Table 10, p. 27).
  • Sec. 4.4 (p. 7) applies the numerical test of a claim on three models, and Sec. 5.2 tests the criterion across seeds.

Ethics flag. The credit is in the statements: Thm 4.1, Cor. 4.2, Prop. 4.3, Cor. 4.4 and Prop. 4.7 carry their sources in their headers.

Figure 1a (p. 4) is fixed.

Reviewer_mBSHOpenReview ↗

Reply to the Rebuttal

I thank the authors for their honest reply and for making substantive revisions to reduce the claims in the paper. However, the revised paper now differs too much from the original submission, with multiple main theoretical results no longer being novel and instead being more like extensions of previous work. I have compared the revised paper with the original submission and find that the main theoretical section, Section 4, has been almost completely rewritten, with most of the major claims now recognized as following previous work, including Theorem 4.1, Corollary 4.2, Proposition 4.3, and Proposition 4.7.

These form a large part of the claimed theoretical contribution; hence, the novelty of these partial improvement results is hard to evaluate during the rebuttal phase alone. The contribution section of the paper has also been completely rewritten to reflect the many changes made since the original submission. Due to the large number of changes that need to be reviewed, I think this paper needs another round of review to be assessable, and I do not recommend acceptance of the paper at this stage. I hope that in the future, the authors can be more meticulous and thorough when conducting the required literature review and more carefully attribute credit to existing works.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the changes between the submitted and the revised PDF

We thank the reviewers. The revised PDF keeps the title and the numbering of sections, theorems, figures and existing tables, so the reviews can still be read against it. The changes are the following.

1. Attribution of prior results. Reviewers mBSH and MtBp pointed out that our results on a single attention layer overlap with prior and concurrent work that we did not cite, or cited only in part. They were right, and the credit is now in the statements themselves.

  • Thm 4.1 is stated as a known result. Without mask, (ii)⇔\Leftrightarrow(iii) is Thm A.1 and Cor. A.5-A.6 of Tran et al. (ICLR 2025). (i)⇒\Rightarrow(iii) from length-three inputs is Prop. 3.1 of Kim & Kim (arXiv 2608.03294) applied to each output coordinate. The proof in App. A.2 follows theirs and says so.
  • The generic part of Cor. 4.2 is credited to Thm 3.2 of Tran et al. (2025) and Thm 2 of Wang & Wang (NeurReps 2025 workshop).
  • The gauge of Prop. 4.3 for full RoPE is credited to Wang & Wang and Tran et al. (ICML 2026), who also prove completeness, which we had wrongly called open.
  • Prop. 4.7 is credited to SliceGPT and Theus et al. (NeurIPS 2025).
  • Sec. 2 is rewritten around these papers, and the description of Kim & Kim is corrected.

2. Contributions re-centred. The abstract and the contribution list now rest on what the reviews identified as useful. Sec. 4.1 is shortened and presented as known results. We did not find the theoretical points below in the works above; they are elementary, and the paper says so.

  • Exact symmetries of deployed architectures: partial RoPE, a simple extension of the full-RoPE gauge; the key bias under RoPE and with sinks (Props 4.3, 4.6); QK-normalization (Prop. 4.5); mixture-of-experts, SwiGLU. Grouped queries are in Morrow (2026), on a toy model.
  • The residual basis: under the hypotheses of Prop. 4.9, normalizing sublayer outputs restricts the absorbable linear changes of basis to signed permutations.
  • Verification on nine checkpoints of seven families with minimally different negative controls.
  • A criterion for interpretability claims, its numerical test, and a study across seeds. We propose no new matching method.

On the attention layer itself we add only the fibres off the generic set and the length-two counterexample, which concerns the non-generic case.

3. Scope and corrections.

  • Thm 4.1 now says "bias-free", GPT-2 is no longer given as an example, and the text says that none of the nine checkpoints has the form of the theorem. For deployed models the paper exhibits symmetries, read as bounds on what the function determines.
  • Sec. 3 separates a symmetry of an architecture from a function-preserving rewrite. Table 2 gains a row for the rewrite alone, and App. A.7 states what holds without it for LayerNorm and tied embeddings.
  • Prop. 4.9: the hypotheses are explicit and the proof is repaired. Cor. 4.2: case (c) now covers dk=dd_k=d.
  • The abstract no longer says that every prediction was verified up to 32B. It says end to end up to 7B, and on three or four layers each up to 32B (13 of 176 layers).
  • Cross-seed results: "chance level" replaces "no information", and agreement after alignment is described as strong for induction and weak for IOI, instead of "replicate".
  • Limitations: no global stability bound, with the conditions of Kim & Kim and a local bound in App. E, and the open cases (RoPE under a causal mask, QK-normalization, sinks, biases). The caveat "necessary, not sufficient" moves from App. D to Sec. 4.4.

4. Results added or moved to the main text.

  • Sec. 4.1 and App. A.2: length-two inputs suffice on an open dense subset of the generic set.
  • App. A.4: completeness for partial RoPE without mask, from Thm 4.1 of Tran et al. (2026).
  • Sec. 5.1: Table A, an excerpt of Table 2 showing where the architecture decides the verdict; the robustness margin; and what the states of a trained model determine locally (previously App. B.5).
  • Sec. 4.4: the numerical test of a claim (previously App. D).
  • New App. E and Table 10: on small random instances, the Jacobian null space has exactly the dimension of the listed symmetries in all 23 cases, including RoPE under a causal mask, QK-normalization, sinks and whole networks. This is local evidence, not a proof. App. E also measures the hypotheses of Prop. 4.9 on both OLMo-2 checkpoints: they hold on four attention sublayers, with ∑hNh\sum_h N_h invertible but ill-conditioned. It also tests LayerNorm models with the normalization kept, on the trained weights of GPT-2 and Pythia-160M.

5. Presentation. Notation is defined at first use (GL(k)GL(k), SHS_H, ⋊\rtimes, θ\theta, ε\varepsilon, "gauge", "orbit", "generic"), short paragraphs of intuition accompany Thm 4.1, Prop. 4.3 and Prop. 4.9, the float32 pitfall moves from the introduction to the method part of Sec. 5.1 and is explained there, and Fig. 1a is fixed.

The statements now precede the references, and the supplementary material includes the new scripts and results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.