- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method for eliciting attention sinks in pre-LN style transformer models. Its scope is limited to the first layer, token embeddings, and self-attention outputs. Given pre-defined tokens (BOS, different punctuation), patching is used to help identify the cause of secondary sinks (tokens not in the first position), whether due to a token's embedding or self-attention.
The main contribution of the paper is the patching methodology: a way of establishing sufficient and necessary conditions for identifying pathways (token embedding, self-attention) that contribute to secondary sink formation.
Experimental results show the formation of secondary sinks across a number of LLMs, sink attribution (embedding vs. self-attention) in LLMs, identification of self-attention heads that contribute to sinks, and a study of sink token mixing.
Critical strengths
The main strength of the paper is the methodology proposed in eliciting secondary attention sinks. To the best of my understanding, the patching approaches for establishing sufficient and necessary conditions in sink formation are (1) novel, (2) useful in attributing specific transformer modules to sinks, and (3) could help inform other types of sink formation studies in the future.
Critical weaknesses
The limited scope of the paper - a focus on just the first layer, token embeddings, self-attention - leads to an incremental contribution. Such a limited scope makes it less clear whether the findings made in the paper could be achieved with prior work, in place of patching, where attention sinks are studied in more detail. Namely, existing work (all largely cited in the paper) has focused on how positional encodings, query-key relations, and feedforward blocks lead to attention sinks, and not necessarily limited to first-position tokens. From this perspective, it is also unclear whether token embeddings and self-attention are the primary drivers for secondary sinks, or if other parts of the model are more important.
Additionally, there is an unclear significance of the findings made in the paper. This stands in contrast with prior work, where the focus is not just on discovery of attention sinks, but also, (1) emergence of attention sinks during pretraining, (2) importance of attention sinks for minimizing LM loss, (3) role of attention sinks in limiting context mixing, and (4) how transformer design leads to attention sinks, and alternative designs for sink suppression. Without a more detailed investigation into the emergence, or implications, of secondary attention sinks, it is difficult to appreciate their importance.
Questions for authors
In Sec. 3.2, it is claimed that "the first block is crucial for identifying the patterns that initiate secondary sink behavior", with additional arguments raised in A.6. However, attention sinks can form in later layers, and not necessarily as a consequence of representations formed in the first layer, see [Sun et al. 2026] for evidence of massive activations formed beyond the first layer. As a result, the arguments in A.6 are not convincing, and it would strengthen the paper to experimentally validate these sorts of claims.
As the results in Sec. 5 are anecdotal, are there metrics that could be employed to capture sensitivity of mixing multiple sink tokens, analogous to what is done in Sec. 4 (Eq. 6)? This would make the results more convincing.