- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The draft studies the training behavior of multi-head attention. The authors study, under a certain setting, the dynamics of multi-heads, including the redundancy and emerging behaviors. There are two stages of behaviors in their setting: the first is when all the heads tend to learn the easiest representation, i.e., the average, then after certain steps, the heads start to specialize. Based on these observations, the authors propose a closed system of equations tracking the full evolution of head alignments along the SGD trajectory and introduce the Bayes-softmax attention, which, under this specific setting, prescribes the optimal number of heads and the way to normalize them, while standard softmax is generally suboptimal because redundant or poorly specialized heads can introduce persistent noise.
Critical strengths
- The authors propose a tractable framework of a low-dim system of head-signal alignment and orthogonal overleaps, connecting the head specialization and training dynamics. In this framework, they are able to connect the early collective learning and later specialization behaviors.
- Also within the framework, they deserve the Bayes predictor and show how joint normalization across heads and tokens can represent it for a finite support latent distribution, which gives Bays-softmax.
Critical weaknesses
- Regarding Proposition 4.3: For , , and signal , softmax-v keys with fixed weights under Eq. (3) yield vanishing risk as after , through cancellation. This challenges the strong-signal interpretation of Proposition 4.3, though not fixed-signal positivity; the readout constraints and lower-bound quantifiers need clarification.
- And about Lemma A.8: Orthogonality makes the Gaussian noise projections independent, but does not preserve independence after marginalizing over . For example, has mean but cross-covariance . The expectation factorization is therefore unjustified for general , leaving the mean-direction invariance underlying Proposition 3.3 unsupported without additional assumptions.
correct me if I'm wrong for both questions.
Questions for authors
They are mostly about Proposition 4.3 and Lemma A.8. Please see above.
We thank the reviewer for the detailed comments and remarks.
W1, Q: the ill-definiteness of the softmax-v holds for L=2 tokens. For larger L, as examplified by the reviewer, it may not hold. Still, the softmax-v is sub-optimal because it only depends on the difference between its inputs, while the Bayes estimator does not. Being able to dynamically deactivate heads when input scores are low is needed. Notice that the addition of the constant bias in softmax-1 precisely allows to break the invariance of softmax w.r.t. a constant shift, while turning off heads. We added this explanation to the revised article.
More precisely, we show that at L=2 for symmetric data distribution the v-softmax cannot recover anything; and for L≥2, for any finite signal there is a strictly positive data-dependent bound between the softmax-v and the Bayes risk; and this bound does not depend on the number of heads or the parameters of the softmax-v. We corrected Proposition 4.3 and the associated appendices in the revised version of the article.
W2, Q: indeed Lemma A8 requires an additional assumption on Pθ, that the mean direction is an eigendirection of the covariance. It holds for the flipping spike distribution we consider. In the general case when this assumption does not hold, there is a minor change to Proposition 3.3 of the main: the invariant manifold always holds, and at initialization the gradient points towards the mean signal; consequently the heads stay on the invariant manifold and initially move towards the mean signal. We corrected Prop. 3.3 and Lemma A.8 in the revised article. Edit: following discussion with reviewer 76kH we made the 2nd part of Lemma A8 more solid with a rotational-invariance assumption on Pθ. The mean-direction invariance directly follows then.
We hope to have addressed all the reviewer's concerns about the article. Overall, we would like to emphasize that our main results and contributions remain unchanged (model, closed-form description of SGD, fast unspecialized and slow specialization phases, Bayes risk and importance of head normalization).