- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how attention routing affects the directional concentration (cone geometry) of Transformer representations. With inputs and weights fixed, the authors suppress attention to selected recipients (e.g., the first token, EOT, or padding) before the softmax, and measure whether the cone opens or narrows using the mean pairwise cosine similarity between tokens.
The experiments have three main parts:
- In SD3, suppressing EOT-like tokens opens the image-token cone, while suppressing padding narrows it.
- Layer-wise first-token suppression in LMs such as Qwen3-14B shows that the sign of the effect varies with the intervention and readout layers, and can flip during propagation.
- A fixed-depth reanalysis of 16 LMs finds that signs are mostly consistent across two datasets but differ across models.
On the theory side, the authors derive the value-replacement update induced by suppression and its first-order effect on the cone, and at four sites they decompose finite-amplitude prediction errors into readout and propagation terms. The paper also includes robustness checks on mask strength, exclusion of the suppressed token, and comparison with spectral metrics.
Critical strengths
-
The SD3 sign split is clean and shows that sink suppression does not have a single geometric direction. At the same model, layer, and denoising step, suppressing EOT-like tokens opens the cone, while suppressing padding narrows it. This holds for nearly every prompt, with tight CIs. Padding receives more attention mass yet produces the opposite sign, so the effect is not simply driven by mass. The family-level sign split in Table 3 points the same way. This is a useful corrective to the common assumption that sinks always compress representations, and it is the main reason I find the paper's observations valuable.
-
The experimental design is careful, and the narrow claim is credible. The checks include:
- signs are preserved across mask strengths from 10 to 10⁴;
- excluding the suppressed token leaves the effect essentially unchanged;
- the breadth analysis uses a fixed depth rule rather than per-model cherry-picking;
- cosine concentration is explicitly distinguished from spectral compression.
Critical weaknesses
-
My main issue after reading the paper is that I am not sure what it ultimately wants to tell me. What can be concluded with confidence is that suppressing sinks changes the cone, but whether it opens or narrows depends on the token type, the layer, and the model. This reads more like a record of a phenomenon than a conclusion. The title says the cone is "routing-controlled", yet the paper never explains how it is controlled, and the sign cannot even be determined in advance. The SD3 effect is quite small to begin with and essentially vanishes after step 0. The link between cone changes and model outputs is only correlational. On top of that, the heavy hedging and process-style wording throughout the main text ("historical", "frozen criterion", etc.) make it read like an experiment log and make the main thread harder to follow. For an analysis paper, I don't think stopping at "it depends" is enough.
-
The theory does not explain the most important part of the phenomenon. Eq. 3 and Eq. 4 are essentially softmax renormalization plus the chain rule. More importantly, the first-order prediction is reasonably accurate only at sites where the cone opens, and it largely fails where the cone narrows. For TinyLlama, for example, the linear prediction is close to zero while the observed effect is −0.146, and the response is not even monotonic in the perturbation scale. Of the four diagnostic sites, only OLMo meets the authors' own criterion. In addition, the signs in Table 3 largely split by model family. Based on the authors' own Eq. 22, this is likely related to the direction of each family's sink values, but the paper does not pursue this analysis.
Questions for authors
-
The signs in Table 3 seem to split largely by model family: GPT-2 and Qwen3 are positive, while Pythia, OLMo, TinyLlama and others are negative. I'm curious whether this sign can be predicted directly from quantities in the baseline forward pass, e.g., the first-order term in Eq. 22, or simply the alignment between the sink value and the centroid. If this works reasonably well across the 16 models, the paper would have a much clearer conclusion, and I would reconsider my score.
-
Mean cosine in LMs is often dominated by a few outlier dimensions (Timkey & Van Schijndel, 2021), and suppressing sinks is likely to affect massive activations. If you remove the top-k outlier dimensions, or standardize each dimension before computing cosine, do the sign and magnitude of ΔC still hold? If the effect mostly comes from a few coordinates, the "cone geometry" framing may need to be revised.
-
(Clarification) For SD3 at layer 12, step 0, what is the baseline C of the image tokens, and how much does it vary across prompts and seeds? With only ΔC ≈ 0.002 reported, it is hard for me to tell whether this is a large effect. Also, does this intervention have any visible effect on the final generated images?