- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper study the phenomenon of head specialization, when different heads specialize to learn different functions during training. This phenomenon has been mostly studied as a function of training time but not as a function of the number of samples in the training set.
The authors try to address this gap. In particular, they first perform an experiment on BERT small trained via masked language modelling. In this experiment, they show how the attention heads specialize to syntactic relations as the size of the training set increases. In particular, they measure the unlabel attachment score (UAS), a metric quantifying the alignment between attention and semantic structure, and show that this observable undergoes a phase transition. They then propose an high-dimensional data model to explain this phenomenology, which is analytically tractable with tools from statistical physics.
In the model, tokens are generated iid from a Gaussian distribution with zero mean and unitary variance, except for a special token, containing a planted spike in the mean. The spike is generated as a linear combination of latent features. The authors then use replica theory to obtained closed-form expression for the generalization error and the training error, as a function of training set versus the embedding dimension.
In particular, they distinguish the case where one of the latent features is kept constant and the other ones are allowed to flip the sign randomly in the sequence with the case where the features are drawn iid from a Gaussian distribution with different covariances. In the first case, depending on the regularization strength, the authors identify the emergence of three different phases: permutation symmetry (PS), where all heads behave in the same way; permutation-symmetry-broken (PSB), where all heads behave differently but they still do not align with the latent feature directions; specialized (SPEC), where each head specialize to one of the latent feature direction. For regularization strengths above the critical value, the model shows a second order phase transition from the PS to the SPEC phases as a function of the size of the training set. For regulation strengths below the critical value, the model exhibits the intermediate PSB phase, a phenomenology closer to what it has already observed in committee machines. This phase transition is qualitatively similar to the one observed in the BERT experiment. In the case of inhomogeneous features, the model shows a fourth phase (r-SPEC), where the heads specialize to the first r strongest features, generating a cascade of phase transitions.
Finally, the authors compare the equilibrium result with previous results on the same data model in the online learning regime.
Critical strengths
The paper addresses an important research question, going in the direction of better understanding what large language models learn from data.
The main strength of the paper is Result 2.1, where the authors derive a set of self-consistent equations whose solutions leads to a closed-form expression for the generalization and training error of a multi-headed attention layer in the high-dimensional limit, where the size of the training set scales proportionally with the input dimension.
The second strength of the paper is that, thanks to these equations, the authors can identify several learning phases and relate them to the generalization performance of multi-headed attention. In particular, they manage to identify and distinguish between the PS, PSB, SPEC and r-SPEC phases and how the model changes behavior between these phases, depending on the size of the training set; they also discuss the connection between this equilibrium analysis with simplicity bias observed in training time and perform the experiment on BERT small trained on masked language modelling, where they show the emergence of a syntactic phase transition when increasing the size of the training set.
Critical weaknesses
I think that the main weakness is the limited connection between the theoretical model and the syntactic specialization in language models. The task is a multi-feature variant of single-location regression, where attention heads are required to identify a special token, containing a planted signal, in a sequence of tokens. However, in the data model, tokens are sampled iid from a Gaussian distribution with an identity covariance matrix in the embedding space. Because of that, the model cannot capture the correlations or the syntactic structure which is precisely the structure that BERT is trying to learn when trained via masked language modelling. Despite the qualitatively similar behavior, it is then hard to conclude that the syntactic transition observed in the BERT experiment in Figure 1 is of the same nature of the specialization transition observed in the synthetic data model and emerging from the theoretical analysis. In particular, there is no a clear connection between UAS and the head-feature overlap m. These two quantities are in principle measuring two different aspects of head specialization and I would therefore appreciate if the authors can better clarify this point.
A related weakness concerns the r-SPEC phase. While the phenomenon of the cascade of phase transition based on the strength of the latent features is mathematically interesting, it is not clear to me whether this phenomenon occurs when also training language models on real language data or whether this is a particular feature of the synthetic data model considered in this work. I would thus appreciate if the authors can better discuss if this phenomenology is expected to occur in realistic learning settings.
In conclusion, I find the theoretical analysis very interesting but I think that the connection with the syntactic transition in language models should be downplayed because it seems more speculative at this point rather than a true connection. Because of that, I am rating the paper with a weak acceptance.
Questions for authors
In the main, the authors write that the theoretical analysis can be equivalently performed for generic convex regularizer and activation function, beyond the softmax. Where is it possible to see it? Because the calculation in the appendix seems restricted to softmax and L2 regularizer.
In the paper, the authors perform the analysis in a mismatched setting, where the number of heads (H) is not necessarily equal to the number of features (F). Is it possible to understand what happens in the regimes: H smaller than F, H equal to F and H larger than F?
I think there are some citations missing: In the section of related works, in the paragraph related to the theory of attention, I think one should add the works:
- Towards a theory of how the structure of language is acquired by deep neural networks, F Cagnetta, M Wyart, Advances in Neural Information Processing Systems 37, 83119-83163
- Mapping of attention mechanisms to a generalized Potts model, R Rende, F Gerace, A Laio, S Goldt, Physical Review Research 6 (2), 023057 At the beginning of page 5, when mentioning the extensive width limit, the authors should add the work:
- Statistical Physics of Deep Learning: Optimal Learning of a Multilayer Perceptron near Interpolation, J Barbier, F Camilli, MT Nguyen, M Pastore, R Skerk, Physical Review X 16 (3), 031014
2 comments
Thank you for the clarification. My concerns have been addressed.
We thank the reviewer for the positive evaluation of our work and for the feedback, which helped improve the manuscript. The suggestions are implemented in the updated version.
W
About token correlations: please see point 1 of our common rebuttal. About the r-SPEC phase: please point 2 of our common rebuttal. About the connection between UAS and the head-feature overlap: this is an important point. The head-feature overlap m measures alignment of an attention head vector with planted latent directions in embedding space. UAS plays an analogous role: assuming that syntax is the relevant latent structure for masked language modeling, it provides a proxy for alignment between learned attention and this structure. First, the underlying structures differ, since syntax involves both token and embedding structure (see point 1 of our summary rebuttal). Then, UAS measures alignment with the syntactic structure through a more complicated function than an overlap (counting correctly attended syntactic pairs across heads and layers). Notably, it involves selecting a best performing head across the dataset and therefore requires specialization to syntactic relations, but does not measure meaningful differentiation between heads.
Q1
The free entropy and saddle point equations of Appendix A.1 are derived for a generic loss ℓ and estimator y^. We do not make use of any particular property of ℓ and y^, encompassing any non-pathological activation function. As to the regularizer, we admit that the equations are written for L2 regularization; yet deriving them for generic regularizer is standard (see Duranthon et al. 2026 for example) and would mainly require to substitute ∣∣K∣∣F2 by the appropriate regularizer in the equations. We corrected this point in the revised version.
O. Duranthon, P. Marion, C. Boyer, B. Loureiro, and L. Zdeborová. Statistical advantage of softmax attention: Insights from single-location regression. ICLR, 2026
Q2
In the non-isotropic Gaussian model (the flipping spike model is less generic for this question), two heads are needed to recover each additional spike. In the underparameterized regime H<2F, heads recover the ⌊H/2⌋ strongest features. In the matching case H=2F, F pairs of heads recover each feature. In the overparameterized regime H>2F, once all spikes have been recovered, H−2F heads are redundant and still have zero alignment with all features. As the number of samples increases, we also observe these heads differentiating from all others without specialization.
Q3
We thank the reviewer for the suggested citations. We added them in the updated manuscript.