- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper studies attention steering, i.e., changing where a head looks, e.g., by adding a bias to the logits of a chosen span of tokens. For a single head at a fixed input, it asks which output changes such an edit can reach at all (reach) and how strongly the output responds to it (cost).
Key contributions are:
-
Exact reachable set of bias edits (Theorem 1, Corollaries 2–3). The theorem characterizes which outputs an edit can reach, for edits that add to the logits a bias that is constant within each group of keys. Because a shared bias rescales all weights in a group by the same factor, proportions inside a group never change, and each group acts as a single "super-token" whose value is a fixed centroid (the attention-weighted mean of the group's value vectors). The row output is therefore always a convex combination of the centroids, and the bias only changes the mixing weights. Conversely, every strictly positive mixture is attained by the closed-form bias , so the reachable set is exactly the relative interior of the convex hull of the centroids.
Consequences: a larger budget never enlarges the reachable set, and groups expose at most output directions.
The authors acknowledge that the underlying geometry is classical (exponential tilting of a categorical distribution); the novelty lies in its application to attention control.
-
Decomposition of interventions by whether they pass through the softmax (Theorem 4). Theorem 4 decomposes any intervention on a head into three routes by which the output can change (through the logits, through the values, or added after the softmax), via a straightforward application of the product and chain rules. Its main takeaway is that the softmax sensitivity enters only the logit route, so attention-steering methods are limited by it, while steering vectors and value/MLP edits are not.
-
Trade-off between attention sharpness and sensitivity (Theorem 6). Theorem 6 formalizes softmax saturation: the more concentrated an attention row, the less it responds to logit edits, with the product of sharpness and sensitivity bounded by approximately 0.35.
-
Empirical validation and practical tools (see Strengths).
Critical strengths
-
Theorem 1 is insightful, and Figure 1 makes it easy to understand. The figure shows the whole result in one picture: each key group is collapsed into a single centroid , and the bias enters only through a -way softmax that mixes these fixed centroids into the output . This makes it immediately clear that the edit changes only the mixing weights, while what can be reached is fixed by the centroids, i.e., by the values rather than by the attention pattern.
-
The finite-edit result derived in Corollary 5 is experimentally verified in Figure 3 ("Path aperture closes the finite-budget gap").
-
Practical consequence of Theorem 6: interpretability and steering pull in opposite directions. Although it is intuitively tempting to look at an attention heatmap and say "this head clearly focuses on token X", those clear, sharp heads are exactly the ones where pushing attention barely works, because their softmax is saturated.
-
The theory may lead to a better head-selection rule for steering. On NQ-Swap (passages edited to contradict the model's memorized answer), steering only the heads chosen by centroid-contrast selection beats PASTA's head selection at matched scope and matched transfer.
-
Formal (Lean) verification of the core results. The main identities and bounds are machine-checked in Lean, and the authors are transparent about what is not formalized. This substantially increases confidence in the correctness of the theory.
Critical weaknesses
-
Theorem 4 is mathematically elementary. Its value lies in the resulting classification of steering methods rather than in the derivation.
-
Limited theoretical novelty. Most results are, in some sense, elementary consequences of well-known facts:
- Theorem 1 follows from the fact that an attention output is a weighted average of value vectors, so, by simple properties of the exponential function, a group-wise bias can only re-mix fixed group averages.
- Theorem 4 is the product and chain rules applied to .
-
Major weakness: presentation.
(a) Simple results presented in a complex way. The paper introduces a lot of terminology (aperture, payload contrast, saturation, certificates) and very dense notation, which makes intuitive ideas hard to follow. The main messages (pushing attention can only re-mix value averages; saturated softmax rows barely respond to logit edits) could be stated much more simply, which would make the contribution clearer and the paper accessible to a broader audience.
(b) Most of the empirical work is relegated to the appendix. As a result, the reader cannot judge the strength of the empirical evidence from the main paper without going through roughly 20 pages of appendix. A compact main-text table summarizing each experiment's setup, baseline, and outcome would help.
Questions for authors
I have no questions.
2 comments
Thank you for the response.
I will maintain my current rating. I think your work is valuable, but the local edits proposed are insufficient to improve the presentation quality. I struggled a lot to understand your work and I think the global edits are necessary. In particular, I think you should remove all the jargon (payload, range, reachable set, saturation) and rewrite the paper with much simpler and cleaner language (see my point 3 a).
Consider this sentence from your paper: "The routing factor in (2) is classical: a groupwise bias is an exponential tilt of the k-atom categorical family with base measure π, so b → w(b) is its natural-to-mean map (Amari, 2016), and the image of that simplex under the value map, the rank ceiling with it, is the family’s mean-map image (Wainwright & Jordan, 2008); the closed-form bias is in Kang et al. (2026, App. A.1) and the b = 0 factorisation in Su et al. (2025), with further antecedents in Appendix L."
There are 7-8 (depending on how you see it) claims here.
Thank you for a careful reading. Your summary states our results more plainly than our abstract does, which is the clearest evidence for your presentation point, and we agree with it. We propose local changes, each shown here as it would appear so you can judge it now. None changes a result, proof, experiment or number.
Novelty. We agree that the derivations are elementary. Theorem 1 is exponential tilting, and Theorem 4 is the chain rule; the paper attributes both. The contribution is what these facts make checkable for attention control, and not their proofs:
We propose to rename Theorem 4 a Proposition, with statement and proof unchanged.
The result we placed too late. One result goes beyond checking the theory, and we put it at the end of Section 5 with no number in the abstract. The standard predictor of a steering edit, the derivative at zero bias, is local; Corollary 2 gives the routing at the budget actually spent in closed form. On development the derivative is steeper at zero bias for 382 of 398 records, yet delivers the smaller gain at the operating budget on 256 of them, which is the saturation Theorem 6 describes. On a pre-registered held-out split (Qwen2.5-3B, NQ-Swap, 398 records, no question shared with development), using the closed form in place of the derivative orders 77.1% of intervention pairs correctly, against 39.8% for the derivative and 60.2% for the development-fixed choice. Prediction error falls from 0.350 to 0.191 logits. In this result the elementary facts predict where a standard tool misranks interventions, and a held-out test confirms it. Scope: one model, layer, budget and task, with the downstream step first order.
Presentation (3a). We propose three local edits:
Empirical evidence (3b). We propose to extend Table 2 with baseline and outcome columns. Every entry below is already in Section 5 or its appendix.
Scope of the revision. The changes are limited to the abstract's central sentences, one introduction paragraph, two lead-in sentences, first-use glosses, one renamed result and two added table columns, offset by shortening the Section 5 prose that repeats the table's numbers. All statements, proofs and experiments stay as reviewed.
We would welcome your view on whether these changes address your concern.