acceptodds
Under review as a conference paper at ICLR 2027

Attention Sink Functionality Is Query-Conditional

Abstract

Attention sinks are usually treated as a single phenomenon, yet the literature assigns them opposite roles: near-uninformative rest states in some accounts, functional carriers of information in others. We argue that this conflict is partly a coordinate problem: attention mass, value norm, and Euclidean value geometry do not directly measure what a sink contribution does to the output. We introduce \method brightness, the variance a residual direction induces in the next-token distribution through the unembedding, and use it to audit first-token sink contributions in decoder-only language models. Our central finding is that sink output functionality is query-conditional. The same first-token value direction, in the same head and text, can strongly perturb one query's next-token distribution while remaining quiet for another. In matched-dose event pairs, attention-mass and residual-norm log-ratios stay within , yet readout brightness can differ by more than two orders of magnitude; brightness stratifies which suppressions perturb the local output, with model-dependent strength. The strongest direction-specific evidence comes from Mistral-7B and a seed-split-selected Qwen3-8B layer-28 site, where a strict fp32 rerun gives a local-KL gap and exceeds same-norm random-direction actual interventions by . Llama-3.1-8B is positive but weaker at the pre-specified layer-20 site; a fixed-candidate audit over already-audited layers identifies a stronger layer-31 site. Gemma2-9B further shows that the local effect survives a soft-capped-readout stress test. Finally, a normalization-aware small-perturbation audit separates the Fisher diagnostic from the finite-dose causal effect: the exact direct-readout counterfactual converges to the RMSNorm-corrected Fisher prediction as , while finite-dose interventions measure propagated network effects. These results suggest that single-role accounts of attention sinks partly reflect coordinate choice: output-quiet sink components are not globally irrelevant, but their output function is visible only in an output-readout coordinate.

Primary area:
interpretability and explainable AI
Keywords:
Attention Sinks, Mechanistic Interpretability, Large Language Models, Fisher Information, Causal Intervention

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_WsMhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper raises the following research question: Attention Sinks attract a high degree of attention, do they actually have a significant impact on the model’s output? The paper outlines a rigorous experimental design. The paper did not reach any novel conclusions; it merely arrived at some conclusions that were to be expected.

Critical strengths

None.

Critical weaknesses

(1) The paper lacks novelty and significance.

(2) The paper fails to address the question posed at the beginning of the article.

(3) The discussion in the latter half of the paper strays from the main topic.

(4) The title and abstract of the paper in the PDF do not match those on OpenReview.

(5) The paper is very difficult for human to read.

Questions for authors

Could you read through your own paper before submitting it?

Reviewer_vecBOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The first contribution of this paper is to investigate whether the effects of attention to the first token (a canonical example of a token sink) are query-dependent. A second contribution is to investigate whether the effects of removing some vector (in this case attention output) on the KL are well predicted by the local curvature of the KL (a quantity the authors term the 'Fisher brightness').

Critical strengths

None

Critical weaknesses

The question of whether attention outputs typically thought of as inert (e.g. first token attention) are actually context-dependent is interesting. Unfortunately it is more or less unanswered in favour of uninteresting minutiae regarding the fisher brightness metric the authors introduce.

Another key weakness is readability - the paper is clearly entirely LLM-generated, with no real attempt to make the output human-readable.

The effects measured in the paper - while statistically significant in many cases, are so small as to be basically irrelevant. This is also a problem for the question of whether the fisher brightness is predictive of the effect of intervention: at these magnitudes of KL effect, the local curvature is almost guaranteed to be a good measure of the effect of ablation, so these experiments can't really answer the second question in general.

Questions for authors

No questions

Reviewer_pP6NOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes readout-Fisher brightness to examine the question: do sink tokens that receive heavy attention actually shape a model's predictions? Controlled deletions on Qwen3-8B and Mistral-7B reveal that contributions sharing a source direction and magnitude can still produce very different output changes. Adding Fisher brightness yields small improvements in predicting deletion-induced KL over attention- and norm-based features, with gains that depend on the model and setup.

Critical strengths
  1. Distinguishing attention strength from real output influence is a worthwhile goal.
  2. Interventions experiments are well designed, and within-head controls and same-norm random-direction baselines give stronger evidence than correlations alone.
  3. Grounding the score in Fisher geometry is principled, and the vocabulary-permutation control is a good check.
  4. Reporting is very honest (even with negative results) with confidence intervals and unfavorable results included.
Critical weaknesses
  1. Prior work already shows that attention weights are poor proxies for importance, and the categorical Fisher metric itself is well established, so the conceptual contribution feels incremental.
  2. Because the score relies only on direct readout while deletions pass through subsequent layers, observed effects cannot be cleanly credited to Fisher brightness.
  3. Vocabulary alignment helps significantly only on Qwen (1.76%), and remaining comparisons are inconclusive.
  4. Most convincing evidence comes from a handful of late-layer heads, and similar gains for non-sink positions weaken the sink-specific framing.
  5. Shifts in KL do not translate directly into task performance, and in pruning or KV-cache tests the method fails to outperform simple baselines.
Questions for authors
  1. How much of the variation stems from the clean predictive distribution versus the downstream Jacobian?
  2. Could a baseline enriched with similar nonlinear features reproduce the gains of the permutation control?
  3. What explains the weaker results on held-out layers?
  4. Do these effects hold specifically for first-token sinks, or for residual contributions in general?
  5. Is there evidence of benefit in a practical task such as pruning or KV-cache retention?
Reviewer_6jo9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the effect of attention sinks on the distribution of outputs of decoder-only language models. They define readout-Fisher brightness to audit how the output distribution changes when the first-token sink contribution is deleted, and study relevant contributing features.

Critical strengths

This paper conducted detailed analysis on the distribution change of model outputs, and distinguish different quantities which contribute to the changes. The use of readout-Fisher brightness and experiments on the vocabulary permutations allow us to better understand the underlying mechanism.

Critical weaknesses

(1) The main weakness is that most of the results are based on several selected layers: Qwen L28 and Mistral L27. Although Table 4 in the paper includes results on more layers, the conclusions are weak. As they mention in Section 5.3.2, all eight model/predictor/regime intervals for the increment conditional on shuffle cross zero and many things remain inconclusive.

(2) When studying the distribution change, this paper examines the KL divergence after deleting the “sink contribution”, which is different from removing the sink token itself. Is it possible to conduct a more direct study on the effect of attention sinks, for example, try to remove the sink token?

(3) The paper only studies the output change, but doesn’t include results on final loss/perplexity or generating quality. Hence we do not know how their results are related to the model performance and can be used to guide practice.

Questions for authors

Apart from the questions mentioned above, I have one more question: The experiments were conducted mainly on trained models. Is it possible to track the training process of a model and track the effect of attention sinks during training?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.