- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
This paper investigates the roles of MLP layers in a Transformer model and shows that MLP layers may induce feature clustering behaviors that are different from attention-only models. This work focuses on an analytical toy model that interprets the forward pass through Transformer layers as continuous dynamics. Under these dynamics, the authors show that specific choices of MLP weights result in equilibrium points that cannot be represented in an attention-only model. The paper concludes with experiments demonstrating that the theoretical findings are consistent with the pre-trained models found in practice.
Critical strengths
The approach to understand model behavior through the lens of dynamical systems is very interesting and can be very useful. Furthermore, the main hypothesis posed by this paper is also worthy of further research. I hope that the authors can continue on this work and present more comprehensive results in a paper with better exposition.
Critical weaknesses
The paper is poorly written and has major technical limitations. First of all, there is very little discussion surrounding the main theorems, and this makes interpreting the consequences of the work extremely difficult. Also, the proof sketch is quite messy and hard to map to the various steps in the full proof.
Most importantly, I have serious doubts about the model formulation of the work.
-
The Transformer block definition (1-2) deviates from the standard implementation in popular models such as Llama or Qwen. The choice to route the residual connection around the whole block instead of individually for the attention and MLP layers are baffling to me. Using post-norm instead of pre-norm is also less common but more forgivable.
-
In (7), the projection onto the orthogonal direction makes little sense given that the author decided not include residual connection for the attention layer. And adding a residual connection would require a change to the LHS of (7), which is also nontrivial to fix.
-
The projection operator defined on line 160 is not norm-invariant, which invalidates the assumption on line 180.
Also, the depth of the technical results are quite limited and do not provide substantial insights.
-
I feel that Theorem 4.1 does not add much on top of Theorem 3.1. Specifically, Thm 4.1 only discusses equilibrium points where all tokens are the same. This basically renders the Q and K matrices vacuous.
-
The results only discuss specific weights construction for the MLP. In particular, Thm 3.1 and 4.1 assume that , and are in the neighborhood of diagonal matrices, and Thm 4.2 assumes that is in the neighborhood of a rank-one matrix. I feel that, at minimum, there needs to be additional analysis showing the relevance of these constructions.
-
Only two equilibrium points were studied by each of the theorems. The paper did not consider other possible equilibrium points and thus gave an incomplete picture.
-
The paper only studies the case where all layers share the same weights.
-
There is also no mention of training dynamics. While this may not be necessary to have a strong paper, proving such results could be a valid way to address my prior concerns.
Overall, I found there are significant limitations in the presentation and technical sophistication of this paper. There remains a lot of work to be done to have a strong paper. I recommend clear rejection.
Questions for authors
I have no questions that can help improve the current draft.
6 comments
Reply [2/3]
W4. The projection operator defined on line 160 is not norm-invariant, which invalidates the assumption on line 180.
Answer. We respectfully disagree with the reviewer. The tangent projection in line 160 is the projection onto the tangent plane at of the unit sphere, therefore it is not norm invariant. But we do not need it to be norm-invariant. The unit-norm conclusion in line 180 concerns the token trajectory and follows from the tangency of its velocity.
To see this directly, let . Along the flow dynamic of , which can be rewritten shortly in the form for a suitable function , we have
=2\langle x_\ell(t),\mathsf P_{x_\ell(t)}f(x_\ell(t))\rangle =-2\langle x_\ell(t),f(x_\ell(t))\rangle\,g(t).$$ Since $g(0)=\|x_\ell(0)\|^2-1=0$, this scalar equation gives $g(t)=0$ throughout the solution’s interval of existence and $x_\ell(t) \in \mathbb{S}^{d-1}$. Thus the outer tangent projection preserves the unit-sphere constraint. To clarify wording that may have been misleading, we have added an explanation after line 180 of the revised manuscript showing why $x_\ell(t)$ remains on the unit sphere. **W5. Also, the depth of the technical results are quite limited and do not provide substantial insights. I feel that Theorem 4.1 does not add much on top of Theorem 3.1. Specifically, Thm 4.1 only discusses equilibrium points where all tokens are the same. This basically renders the Q and K matrices vacuous.** **Answer.** We respectfully disagree that Theorem 4.1 adds little to Theorem 3.1. The main contribution of Theorem 4.1 is to show that an attractor created by the MLP for the first token can draw initially distinct tokens to the same non-spectral direction. We agree that the existence of the synchronized equilibrium follows from Theorem 3.1. The stability of that equilibrium in the full product space is the new result. For $L>1$, configurations with identical tokens form a set of zero product surface measure. Theorem 4.1 instead gives an open basin of positive measure in which the tokens converge exponentially to the common direction. This full-prefix stability can fail even when Theorem 3.1 applies. To see why, at the reference MLP weights with $V^0=\operatorname{diag}(-1,-2,-3)$, Theorem 3.1 gives a first-token sink at $u$. At the corresponding synchronized three-token equilibrium, however, a perturbation of the third token in the $e_3$ direction has linearized growth rate $-\frac{\lambda_1+\lambda_2-2\lambda_3/3}{4}=\frac14>0$. Thus the synchronized equilibrium is unstable in this example. The additional hypothesis in Theorem 4.1 controls these later-token perturbations and ensures stability for every finite prefix length and every fixed choice of $Q,K$, with a common radius for parameter perturbations. We have added this counterexample to Remark C.1 in appendix. We have also added a remark after Theorem 3.1 explaining how the MLP changes the attracting directions of the first token, and a remark after Theorem 4.1 explaining how these attracting directions persist across subsequent tokens.Reply [3/3]
W6. The results only discuss specific weights construction for the MLP. In particular, Thm 3.1 and 4.1 assume that , and are in the neighborhood of diagonal matrices, and Thm 4.2 assumes that is in the neighborhood of a rank-one matrix. I feel that, at minimum, there needs to be additional analysis showing the relevance of these constructions.
W8. The paper only studies the case where all layers share the same weights.
W10. Overall, I found there are significant limitations in the presentation and technical sophistication of this paper. There remains a lot of work to be done to have a strong paper.
Answer to W6-W8-W10. We agree that our theory does not yet cover the most general setting of Transformer and a simplified version of Transformer is used for the theoretical study. At the same time, we would stress that this kind of restriction is standard in theoretical studies of attention dynamics. It is common to work in a simplified but still expressive regime that makes rigorous analysis feasible while preserving the key qualitative mechanisms of interest. Several related works in this area follow the same philosophy, for example:
[A] Rigollet, The Mean-Field Dynamics of Transformers, Proceedings of the International Congress of Mathematicians 2026 - Volume 7. (The assumption was used.)
[B] Geshkovski et al. The Emergence of Clusters in Self-Attention Dynamics. NeurIPS 2024. (The assumption was used.)
[C] Geshkovski et al. A Mathematical Perspective on Transformers. Bulletin of the AMS 2024. (The assumption was used.)
[D] Karagodin et al. Clustering in Causal Attention Masking. NeurIPS 2024. (The assumption , and are used.)
[E] Abella et al. Consensus is all you get: The role of attention in transformers. ICLR 2025. (The assumptions symmetric and hemisphere initializations were used.)
[F] Pham et al., Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding. NeurIPS 2025. (The assumption , and symmetric are used.)
Importantly, our setting is already substantially broader than the most basic static case: we analyse the dynamical role of the MLP, and inparticular its interaction with attention, which is already a nontrivial generalization of prior analyses. This role and interaction remains much less understood from the dynamical system point of view in the literature. By contrast, a full theory for completely general Transformer update and no structural restriction, appears highly challenging. To the best of our knowledge, no existing work has established such a result so far, and we view this as an important direction for future work.
W7. Only two equilibrium points were studied by each of the theorems. The paper did not consider other possible equilibrium points and thus gave an incomplete picture.
Answer. Under the assumptions of Theorems 3.1 and 4.1, only two types of equilibria arise. These equilibria have the properties needed to prove our main claims, that the MLP can create new non-spectral attractors for the first token and preserve them across subsequent tokens. Beyond these assumptions, additional equilibria may exist. The study for more general assumption which may lead to more equilibria lies beyond the scope of the present paper and is a direction for future work.
W9. There is also no mention of training dynamics. While this may not be necessary to have a strong paper, proving such results could be a valid way to address my prior concerns.
Answer. Regarding the role of training dynamics in our analysis, we treat the weights as fixed parameters and allow them to vary over the model’s parameter space independently of any particular training procedure. Incorporating training would shift the focus from the asymptotic behavior of tokens at fixed weights to how changes in the weights during training shape token behavior at finite depth. This question lies beyond the scope of the present paper and is a direction for future work.
We hope we have cleared your concerns about our work. We have also revised our manuscript according to your comments, and we would appreciate it if we can get your further feedback at your earliest convenience.
Reply [1/3]
Thank you for your thoughtful review and constructive feedback. We have revised our manuscript in response to your suggestions, with all changes highlighted in blue. Below, we address each of your concerns in detail.
W1. The paper is poorly written and has major technical limitations. First of all, there is very little discussion surrounding the main theorems, and this makes interpreting the consequences of the work extremely difficult. Also, the proof sketch is quite messy and hard to map to the various steps in the full proof.
Answer. We have added a remark after Theorem 3.1 to discuss the changes of attractors when adding MLP. Theorem 3.1 says that adding a GELU MLP can change where the first token converges, even with the same value matrix and the same initial token. Indeed, in the attention alone, almost every initial direction converges to e1 or −e1, the two orientations of the dominant eigenvector of V0. While in attention with the specified MLP, the coupled flow has two additional attracting directions, ±u=±(e1+e2)/2. The attractors are non-spectral because the attention-only velocity is nonzero at both points, even after perturbation. The simulation in Section 6.1 verifies the predicted change in attracting direction numerically in the projected model.
Another discussion has been added after Theorem 4.1 to explain the effect of MLP in Transformer's dynamic. while Theorem 3.1 shows that the MLP can create a new attracting direction for the first token, Theorem 4.1 shows that the same direction can attract an entire causal prefix, even when the tokens start at distinct points. For example, at the reference weights, if all L tokens start sufficiently close to u=(e1+e2)/2, they all converge exponentially to u. If they start sufficiently close to −u, they all converge to −u. These are two separate basins, each leading to one shared limit. The shared limit is non-spectral, meaning attention alone would still move the tokens even once they were synchronized there.
We have also updated the proof sketch in Section 5 to improve readability and clarify how it corresponds to the steps in the full proof.
W2. The Transformer block definition (1-2) deviates from the standard implementation in popular models such as Llama or Qwen. The choice to route the residual connection around the whole block instead of individually for the attention and MLP layers are baffling to me. Using post-norm instead of pre-norm is also less common but more forgivable.
W3. In (7), the projection onto the orthogonal direction makes little sense given that the author decided not include residual connection for the attention layer. And adding a residual connection would require a change to the LHS of (7), which is also nontrivial to fix.
Answer to W2-W3. Regarding the normalization, we follow Rigollet [A] in adopting a post-normalized spherical model for our theoretical analysis. In the attention-only setting, Rigollet shows that pre-norm and post-norm share the same projected attention field, with normalization-dependent factors that change the speed of each token. This common structure allows both schemes to be studied within the same analytical framework, even though their convergence rates can differ.
Regarding the residual connections and the projection in Equation (7). We use a single residual connection around the composed block to obtain a tractable model that retains causal token mixing and a tokenwise MLP. This lets us isolate whether the MLP can create new attracting directions and whether those directions can attract subsequent tokens. In Equation (7), the projected attention vector is the instantaneous change in token direction supplied to the MLP. For a unit token xℓ and attention output aℓ, this interpretation follows from
=P_{x_\ell}a_\ell.$$ This identity motivates the tangent projection as a modeling choice in the reduced system. We agree that giving attention and the MLP separate residual updates would change the dynamics. If the attention substep were modeled as a separate continuous-depth flow, the intermediate token state would satisfy an evolution equation with $\partial_t$ on the left-hand side, in place of the algebraic relation in Equation (7). We conjecture that the MLP can also create non-spectral attractors in that setting and that these attractors can persist across subsequent tokens. We have added a paragraph after Equation (9) to justify the tangent projections in Equations (6)–(7). [A] Rigollet, The Mean-Field Dynamics of Transformers, Proceedings of the International Congress of Mathematicians 2026 - Volume 7.