- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies whether independent computations are carried out by independent attention heads. It trains single-layer Transformers to predict sequences generated by two independent hidden Markov processes. Section 3 describes attention as accumulating token-dependent evidence with a factor-specific decay over distance: QK supplies the source weights, while OV determines what information is written into each factor's space. The authors show that this computation can be distributed across heads even when the factor representations are separate. They introduce effective subspace attention and derive minimum head counts under specific assumptions. Validation includes 240 training runs across five settings, head counts, and seeds, controlled interventions, and two GPT-2 case studies: IOI and neither/nor prediction.
Critical strengths
I appreciate the interpretation of attention as an approximate update of information about HMM hidden states. It connects distance-dependent weighting to QK and token-dependent writes to OV in this setting. This makes the proposed computation more concrete than an attention-map interpretation alone. The known data-generating process and 240 training runs provide a meaningful setting for testing this view.
Comparing positive and negative decay cases helps explain why a single head may not implement a factor on its own. Combining attention with OV writes reveals structure that raw maps can miss, and the interventions add useful evidence. I find the possible connection to the degree of attention superposition interesting. I regard this as a insteresting and useful direction.
I consider Effective Subspace Attention a valuable contribution of this paper. It not only provides theoretical predictions of the computations the model should perform, but also recovers these computations from trained attention heads, allowing the authors to verify whether multiple heads collaboratively implement the theoretically predicted computations.
Critical weaknesses
Is the paper explaining how a known computation is distributed across heads, or proposing a method for discovering unknown computations? The controlled experiments support the first goal more directly. Eq.(5), Δr_n(d)=∑s=1^d ζ_n^(d−s)gn(z_s^(n)) , has the same weighted-sum form as attention. This does not make the theory trivial: the decay rates and update directions come from the generator. However, the paper should distinguish deriving this target computation, showing that attention can express it, and demonstrating that trained models actually learn it. A concrete sequence example would make the contribution much easier to assess. In Appendix L and Table 3, unequal positive and negative decay rates are grouped into a two-head case. Under the stated requirement of exact realization at all distances, +0.3/−0.5 needs three heads, unlike the tested +0.7/−0.5 case. Finite sequence length must also be distinguished from arbitrary distances. The general counting procedure raises another issue: the number of boundary directions of the target updates need not equal the minimum number of head directions, since other directions can also combine to produce the targets. These concerns affect the theoretical extension, not necessarily the five tested configurations.
The real-model application is limited and does not test the full theory. Section 5 studies GPT-2 small on IOI and neither/nor prediction. These examples are useful, but the method uses an answer-logit direction rather than recovering unknown factor subspaces and their update rules. It is therefore unclear which parts of the HMM-based account carry over, or what the method adds beyond existing OV/logit contribution analysis. The authors acknowledge that this is a demonstration; the main claims should reflect that scope. In the neither/nor case, the evidence also does not clearly separate pairing information in the heads' OV weights from information already present in their inputs. I am not requesting a large new benchmark, but a clearer account of what these examples establish.
Theoretical and experimental differences from closely related work are not established clearly enough. Anthropic's Attention Head Superposition (Jermyn et al., 2023), which the paper already cites, demonstrated computations distributed across heads in toy models. He et al. (2026), Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition (Lorsa), decomposes the attention layer as a whole into sparse, interpretable computational units rather than treating native heads as the units. Wang et al., Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning, further studies attention outputs concentrated in low-dimensional residual-stream subspaces. The latter two works are not discussed. These results are related but not equivalent: a low-dimensional output space need not be a task-specific computational unit. The authors should explain which theoretical claims go beyond these results and provide a focused comparison on the same controlled model. In particular, how do effective subspace attention and Lorsa differ in recovering factor updates and separating computations, and how do the factor spaces relate to the low-dimensional attention-output space? Such a comparison should report reconstruction quality, full update errors, and held-out results. I am requesting evidence of the differences, not assuming that the prior methods solve the same problem or requiring a full large-model benchmark.
Questions for authors
The questions below concern the current claims, not optional future projects. Their answers could change my assessment.
Is the main contribution an account of how known factor updates are distributed across heads, or a method for discovering unknown computations? Please use a concrete sequence to explain the prediction task, the information supplied by each token, and the combined head updates. Which factor labels, theoretical directions, or target answers must the analyst know beforehand? What does the theory predict beyond the weighted-sum structure of attention? And if it is about interpretability, could you briefly describe what exactly is being interpreted?
Does the two-head claim require the positive component to decay more slowly than the negative one? How does it change for +0.3/−0.5 and for finite versus unlimited sequence length? Why should the general procedure count the target updates' own boundary directions rather than allow a smaller set of other head directions that can also produce those updates?
I wonder whether there are practical methods to quantify the functional similarity and specialization of attention heads. For example, how can we measure how many distinct functions are represented by a single head, and how many heads collectively contribute to a particular function?
3 comments
Thank you for your detailed response. I generally like this work and will maintain my score. I also encourage you to compare your method with Lorsa in future work.
We realize we could have gone into more detail regarding the comparison with Lorsa, so we do that here.
The main difference is that our work explains which computation attention must implement and how it can be distributed across heads, whereas Lorsa is a concrete decomposition routine that finds the interpretable computational components of attention. You are right to point out that Lorsa also finds that the computational units need not correspond directly to heads -- we do not claim novelty here.
Our novelty lies in studying a setting where the computational parts are pre-specified by the data-generator. This lets us derive the required per-factor updates, the conic characterization of the update rules, and the number of heads required to minimize loss. From there, we can see that even with completely disentangled activations we still find that attention heads can be individually uninterpretable, and that at effective subspace attention is necessary to fully see the implemented computation. Lorsa in contrast lets us discover computation that attention implements, but not why it implemented it. We will add Lorsa (and the Wang et al. paper) to our related works section:
Our work is closest to approaches that explicitly recover computation spanning multiple heads. Parameter decompositions can identify functionally meaningful subcomponents of attention (Christensen & Riggs, 2025), while Bushnaq et al. (2026) find weight-space components spanning heads and separating QK from OV contributions. Wang et al. (2025) show that attention outputs concentrate in low-dimensional subspaces and leverage this structure to improve sparse dictionary learning. He et al. (2026) introduce Lorsa, a sparse decomposition of the full attention module that identifies interpretable units spanning multiple heads. These approaches agree with our work that multiple heads must be taken into account to discover units of computation, but differ in trying to discover computation from trained models. Unlike these approaches, our controlled setting lets us derive the required factor-specific updates analytically and characterize how they can be distributed across heads, even when factor representations are orthogonal.
We thank the reviewer for their continued engagement, and are happy to answer any other questions and provide more clarity.
We also agree with the reviewer that method-specific comparisons could be valuable follow-up work. Specifically, for settings like ours where the implemented computation is analytically derivable from the data-generator, do decomposition methods recover the derived factor-specific computations?
Reviewer ZHUU Response
We would like to thank the reviewer for their thoughtful comments and their close reading of our manuscript. We acknowledge that the clarity of the current manuscript be improved and propose the following outline:
Q1: We consider the main contributions to be 1) the precise link between the spectral properties of the data generator and the minimal function the Transformer can implement to get optimal loss and 2) our use of this precise setting to show that the aggregate contribution of attention heads within specific subspaces can be viewed as interpretable even when individual attention head contributions cannot.
While the existence of uninterpretable heads is well established in the field, their origin is poorly understood. Are illegible heads a consequence of poor training, highly entangled representations, or capacity constraints? We demonstrate that none of the above are necessary for head collaboration. In a toy model of data that demonstrably leads to disentangled representations, head collaboration is still present, even when trained to Bayes optimal loss with sufficient capacity. In LLMs, attention head superposition may arise from dimensionality pressure and polysemic/collaborative heads may form because of training dynamics and just having a large amount of heads. Our minimal head count case is neither over nor under-parameterized.
We will make the positioning of our paper next to the related works highlighted by the reviewer clearer. Specifically we will address how this relates to superposition (namely that it is not superposition because there are no capacity constraints), and that instead of looking for discrete computational units (as is the case with the decomposition and dictionary learning methods) we predict the required update. We can also include example sequences from our process in the appendix, to clarify the structure of our dataset.
In terms of what information must be known beforehand: we do need to know which object (the predictive vector) the Transformer tracks to make its prediction. In terms of how we can disentangle collaborative/polysemantic attention heads in real models, the answer is that we must have some candidate subspace/direction already.
As to what is being predicted by the theory, it is the minimum number of attention heads required for optimal loss as well as which update they must perform. The interpretability claim relates to working out which algorithm the model implements. We want to know: “if trained on data X, what is model M going to learn?” We provide a case where this is fully tractable to compute.
Q2: We commend the reviewer on this question! The +0.3/-0.5 case indeed does have a 3-head requirement, with the theory as written. +0.3/-0.5 requires three heads for all distances while two are sufficient for any finite length context length. We will correct the corollary and appendix to distinguish these cases. As for “Why [..] count the target updates' own boundary directions rather than allow a smaller set of other head directions”, this is a relevant point in general for arbitrary numbers of factors (it vanishes with 2 factors only). The reviewer is correct that for an arbitrary number of factors, the latter can be smaller than the former. We will clarify this in the appendix.
Q3: For a specific function, figuring out the number of heads that contribute to it is relatively straightforward if we can use knowledge of the function to find a specific subspace/direction. We can then work out the effective attention similarly to our GPT-2 experiments where we measure OV-writes onto that target direction.
Knowing which functions a given head it implements, is harder. It's possible that an unsupervised method can be used to find consistent subspaces that various heads write to without knowing the function. Weight-level decompositions (such as SVD) could suggest candidate directions, and methods like Adversarial Parameter Decomposition can – given enough data – potentially discover many mechanisms a head participates in. However, none of those methods give any guarantee that we’ve found everything a head does. We consider this area of research – unsupervised subspace/factor detection – extremely important and are actively working on it. However, that project is an entire research question on its own– and not within the scope of the current manuscript.
Once again, thank you very much for your thorough review, especially catching the +0.3/-0.5 case!