acceptodds
Under review as a conference paper at ICLR 2027

Before Attention Sinks Form: Hidden-State Geometry Reveals and Modulates Their Formation

Abstract

Attention sinks are usually studied after they have formed. We ask whether a measurable hidden-state change appears beforehand and whether changing that geometry alters what happens later. To remove network depth as a shortcut, we compare prompts in the same model and at the same layer. When sink timing varies across prompts, simple spectral summaries of the hidden states add information beyond current attention, with the clearest prompt-specific signal in a short window about two to three layers before sink formation. The predictive strength varies across cohorts, so we treat the spectrum as a readout of a pre-sink state change rather than a universal warning statistic. The signal is broader than one measured extreme activation. We then modify only the singular values of the post-layer hidden states after current attention has already been computed. These interventions change later sink behavior in Qwen2.5, GLM-Z1, and GPT-OSS relative to norm-matched random spectral perturbations. In Qwen2.5, the immediate suppressive effect is accompanied by a reduction in next-layer query–key preference for position 0, and the learned spectral directions remain stable when estimated from substantially less training data. Together, the results show that attention sinks are preceded by a short spectral reorganization of hidden states, and directly perturbing this geometry can causally alter subsequent sink formation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
attention sinks, hidden-state geometry, spectral analysis, transformer representations, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3b1eOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper attempts to revisit the previously established relationship between massive activations and attention sinks. The authors argue that the properties associated with attention sinks are not limited to massive activations, but instead involve the distribution of singular values along the principal component directions of hidden states. Their causal intervention experiments largely support this argument.

Critical strengths

The paper provides an alternative explanation for the previously established relationship between attention sinks and massive activations. Indeed, the conventional explanatory paradigm underlying massive activations, which emphasizes individual hidden-state channels, appears rather outdated. In contrast, shifting the focus toward feature directions or principal component directions represents a more advanced perspective.

Critical weaknesses
  1. The paper makes rather limited scientific contributions. Several hypotheses have already been proposed to explain the formation of attention sinks. Most of them share a common underlying idea: Transformer computations produce relatively stable constant terms that facilitate the emergence of attention sinks. Such constant terms may originate from massive activations (https://arxiv.org/abs/2402.17762), bias terms associated with attention parameter matrices (https://aclanthology.org/2026.acl-short.9), or other sources.

    • As discussed below, the present paper makes only a limited improvement within this existing framework and does not substantially expand our scientific understanding.

    • The contribution of this paper may even be conceptually at odds with the massive activation explanation, although the authors argue in Section 4.1 that their explanation based on the hidden-state feature spectrum is more accurate than the massive activation account.

    • However, one can argue that the hidden-state feature spectrum and the explicit hidden-state representation are related through a linear transformation. Consequently, the distinction between them may not be fundamental, and it is difficult to establish whether attributing the phenomenon to the feature spectrum constitutes a genuinely new mechanism. Furthermore, the overall argumentation of this paper closely resembles that of earlier studies on massive activations.

    • Admittedly, there are clear benefits to moving beyond the channel-wise perspective of massive activations and instead adopting a spectral perspective based on eigenvalues and a basis of feature vectors. After all, there is no reason to assume that Transformer computations must be understood in a basis consisting of the original coordinate axes. However, this intuition is not original to the present paper. Relevant prior work includes:

      • Spectral Filters, Dark Signals, and Attention Sinks. ACL 2024.
      • Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin. ICLR 2026.
    • A more scientifically valuable direction would be to explain why the observed spectral structure is preserved through attention-matrix computations. In particular, why should the eigenvectors of attention matrices align with the feature directions of hidden states? Investigating the origin and preservation of this alignment could lead to a broader and more interesting discussion of the underlying mechanism.

  2. Methodological concerns. Why must the experimental framework be restricted to models in which attention sinks emerge only after more than one Transformer layer, with their onset varying depending on the prompt? One can readily imagine that even when attention sinks emerge in the very first Transformer layer, there is still a preceding hidden-state representation, directly connected to the token embeddings, on which the proposed experiments could be performed. Therefore, I find this restriction in the experimental design unnecessary and somewhat redundant.

  3. The manuscript appears to be AI-generated with insufficient manual refinement, and inadequate attention has been paid to human readability. The paper retains numerous characteristics commonly associated with AI-generated scientific manuscripts that make it unnecessarily difficult to read. Substantial revisions throughout the manuscript are needed. For example:

    • Very poor presentation of experimental results. Some numerical results are repeatedly interspersed with the main prose, often in a format that is itself rather unconventional for major AI conferences, such as AP .857 → .913 (∆AP = +.056, [.028, .087]). This presentation substantially distracts readers from the main scientific arguments, making the entire manuscript feel more like an appendix than a coherent research paper. Moreover, this way of presenting numerical results is rather outdated. A more conventional approach would be to organize these results into figures and tables, separating numerical details from the main logical narrative.

    • Even the few figures included in the paper appear to be AI-generated, or at least exhibit many of the shortcomings commonly found in AI-generated scientific visualizations.

    • Excessive use of mathematical notation and equations. Some theoretical details and mathematical derivations could be moved to the appendix, while the main text should focus on the logical arguments and more intuitive explanations. In my view, the mathematical intuition underlying this paper is not particularly complicated, but the authors have presented it in an unnecessarily convoluted manner.

Questions for authors

See weaknesses.

Reviewer_b36UOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This article studies the hidden state changes before the position-0 attention sink appears in a single forward pass. Comparisons between the same model and the same layer show that spectral summary can add short-term onset information to the current attention features. The author further only modifies the hidden state singular values ​​after the current layer calculation is completed, and changes the subsequent sink intensity while maintaining the mean, singular direction and centralization energy. The results support spectral intervenability of model and state dependencies, but not a uniform “towards the sink” direction.

Critical strengths
  1. Prompt-grouped nested CV, fold-local processing, strict within-cell AP, nonlinear attention baseline and scrambling control, more seriously handle deep confounding and evaluation leakage.
  2. Train-only direction, opposite-sign contrast, post-implementation norm matching, and 30 new prompts after GPT-OSS freeze settings make causal evidence stronger than observational correlation.
  3. Proactively disclose Qwen3 post-audit uncertainty, long context support collapse, recurrent onset and distributed activation redundancy to avoid hiding failed settings in the overall average.
  4. GLM-Z1’s T=256, 500-source results rechecked the short window; half-data directional stability and cross-layer propagation diagnosis further supplemented the main experiment.
Critical weaknesses

This paper suffers from two key issues:

  1. Presentation: The use of tables is difficult to follow; large blocks of text have been converted into tabular format without any careful adjustment of the table structure. Furthermore, images contain excessive empty space, while the text within them is rendered so small that some parts are barely legible.

  2. Trivial conclusions: The "attention sink" phenomenon is fundamentally a consequence of the softmax requirement that weights must sum to 1. Consequently, the observation that one can alter the flow of the "sink" by adjusting specific feature directions (singular vectors) is merely a trivial result of linear algebra.

Questions for authors

See weakness

Reviewer_jXcWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies attention sinks before they form in transformer LLMs. There are two linked claims. (1) Observational: a short spectral reorganization of hidden states precedes sink formation. Using same-model, same-layer comparisons that hold network depth fixed, scalar summaries of the centered token-by-hidden-state singular spectrum (log-log slope, spectral entropy, effective rank, leading singular energy, top-5 energy) add predictive power over an attention-only baseline for near-horizon targets, with the clearest within-cell gain in a short window and no measurable gain farther out. (2) Causal: after the source layer has computed its attention, the authors edit only the singular values of the post-layer hidden state, preserving the token mean, singular subspaces, and centered Frobenius energy. These edits change later sink behavior in Qwen2.5, GLM-Z1, and a pre-committed frozen GPT-OSS confirmation, relative to norm-matched random spectral directions. In Qwen2.5 the suppressive effect tracks a drop in next-layer position-0 query-key preference. Cross-layer tracing shows later blocks reshape the perturbation differently in different models, which explains why the downstream effect depends on where the edit is injected.

Critical strengths
  • The same-model, same-layer design is the right control for depth. A naive predictor cannot cheat by learning sink location
  • The methodological rigor is high: prompt-grouped nested CV, fold-local preprocessing, cell-stratified evaluation, prompt-level bootstrap CIs, source-layer attention invariance verified on every causal run.
  • The pre-registered GPT-OSS confirmation with 30/30 fresh prompts hitting the predicted sign gives a strong backing to the approach.
  • Qwen3 example decisively falsifies the “one massive activation causes the sink” theory: both early-sink (layer 3) and late-sink (layer 7) prompt groups have their extreme-activation event at layer 6.
  • Boundary conditions are stated plainly
  • Direction stability tests show the learned direction is not a lucky result of one training sample.
Critical weaknesses
  • Small cohorts: 50 prompts for the main models, 20 for most boundary models (50 for Qwen3.6), and a 30-prompt frozen holdout for the GPT-OSS confirmation.
  • Qwen2.5 and GLM-Z1 interventions report identical empirical p = .0308 despite different test statistics. Is this correct?
  • Causal effect sizes are small in absolute terms and the paper doesn’t say what counts as a meaningful change.
  • GPT-OSS flips the sign (strengthens later sinks) and the paper notes this without really explaining why. What property of GPT-OSS (attention topology, layer index, pretraining data) picks the sign?
  • Only the canonical position-0 sink is studied. Sinks other positions are common and may follow different pre-sink dynamics.
  • Hyperparameter choices sit in main text without main-text sensitivity analysis.
  • Queipo-de Llano et al. 2025 is cited but not compared. It is close enough to the spectral reorganization story that a direct comparison would help place the contribution.
Questions for authors
  • Why do the Qwen2.5 and GLM-Z1 interventions give identical p = .0308?
  • How sensitive is the result to the top-32 cutoff? What about top-8 or top-64?
  • Does the GPT-OSS sign flip survive if you move the source layer earlier or later within the full-attention band?
  • Does the learned Qwen2.5 direction transfer to Qwen3 (same family) or require re-estimation?
  • Could the spectral change be a manifestation of the “compression valley” of Queipo-de Llano et al.? Have you measured the two side by side?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.