acceptodds
Under review as a conference paper at ICLR 2027

Identifying the Functional Distance Between Attention Heads

Abstract

Attention heads in transformers are routinely compared through the similarity of their attention rows. Most heads place their largest attention weight on a sink token, and analysts must choose whether to keep or remove the sink before comparing, without knowing what either choice measures about a head's effect on the model. We prove that attention rows do not determine how differently two heads act, while the value vectors do. We show this by writing the difference between two head outputs as the value matrix applied to the difference of their attention rows: the rows leave the distance free within an interval, and the Gram matrix of the values fixes it exactly. We then introduce the recipient-slot distance, the output change when one head's output passes through the output projection of another, and prove that this substitution changes the layer output less than removing the head exactly when the distance is below the head's output norm. On GPT-2, Pythia, and Llama, comparisons based only on attention rows agree with the value-aware comparison about as often as chance, and about nine in ten heads change their functional nearest neighbour. Head outputs within a layer are nearly orthogonal, so their distance is set mostly by their norms, which attention rows do not contain. Selecting heads for removal with the value-aware distance costs less perplexity than selection by attention rows in eight of nine settings, and on Llama, where heads share values, substitutions chosen by the recipient-slot distance cost less than every removal rule. We further split every standard dissimilarity exactly into a sink part and a content part. This split shows when keeping or removing the sink reverses conclusions. Under natural axioms, no dissimilarity can be exactly separable and also continuous at the boundary or monotone under processing. Practitioners can compare heads within a layer through their projected values and report sink mass and content distance separately.

Primary area:
interpretability and explainable AI
Keywords:
attention heads, attention sinks, head similarity, interpretability, identification, compositional data, value matrices, head pruning, transformers

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_W5hbOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors of the paper examine the problem of connection between attention patterns and heads divergence. Since a head’s output depends on both its attention row and its value vectors, the authors prove that attention rows alone leave the output distance undetermined, while the Gram matrix of the projected values fixes it. In order to do so, they consider an attention row as a point pp of the probability simplex over DD key positions and the head output as y=V⊤py = V^\top p, where VV holds the projected value vectors. The first position is treated as a sink, so each row splits into a sink mass ss and a content distribution π\pi, p=(s,(1−s)π)p = (s, (1-s)\pi). They show that over value matrices with bounded rows, the output distance ∥V⊤(p−q)∥2\|V^\top(p-q)\|_2 can take any value in [0,B∥p−q∥1][0, B\|p-q\|_1], whereas it equals exactly the Gram distance dG(p,q)=(p−q)⊤VV⊤(p−q)d_G(p,q) = \sqrt{(p-q)^\top VV^\top (p-q)}. Using this identity they decompose Jensen-Shannon divergance by two parts: sink term and non-sink term. Based on the following idea they propose:

  1. Theoretical examination of heads. They state that it should be compared via their projected value vectors (e.g., the Gram distance or the full output distance), since similar attention patterns are not evidence of similar function.

  2. Pattern similarity vs. output similarity. On within-layer head pairs of GPT-2, Pythia-160M and Llama-3.2-1B (WikiText-103), the authors test how well attention-pattern scores predict the distance between head outputs.

  3. Practical examination (like head-prunning). When 10-30% of the heads in each layer are removed or replaced, selection by value-aware distances increases perplexity less than selection by attention patterns in most settings.

Critical strengths
  1. The authors provide the accurate formulation to the problem. They precisely explain why it is necessary to consider the projected values and take in the account sink part. The sink-content decomposition explains precisely when keeping or masking the sink token changes conclusions.
Critical weaknesses
  1. Limited idea. The main proposition to consider differentiate the sink part out of the other tokens is well-examined (even in the context of Gramm matrices) [like to noted in Kobayashi et al. 2020 and Xiangming Gu et al. 2024]. Much of your theory is a formalization of the linearity of the head output. Moreover, the key empirical finding is that output distance is driven almost entirely by head norms, since outputs within a layer are nearly orthogonal. As a result, the proposed “functional distance” mostly measures scale rather than function.

  2. No practical gain for the real models. Experiments use only small models (up to 1B parameters), short windows of 192 tokens, and perplexity on WikiText-103, without the BOS token that models like Llama normally use.

Questions for authors
  1. What can you say about the comparison of your paper to the research of Kobayashi et al. and Xiangming Gu et al.?

  2. What your theoretical results (Th.1 and Th.3 f.e.) could propose to the researchers from the practical point of view?

1 comment
AuthorsOpenReview ↗

Respond to reviewer W5hb

Thank you for the review. Each limit you name now has an experiment. On three larger models, and on Llama-3.2-1B with BOS, replacing heads by partners chosen with SS costs less than the best removal rule at every budget. The theory gives five concrete instructions. Changes are blue in the revised PDF, and the new analyses were chosen after the reviews (Appendix L).

Q1, W1. Kobayashi et al. and Gu et al. Kobayashi et al. measure the norm of each weighted value inside one head. Gu et al. describe the sink as a key bias whose value has a small norm. Neither compares two heads. Norms are the diagonal of the Gram matrix, and the distance between two heads needs the inner products across heads. In case A of new Table 1, two heads have equal patterns and equal norms at every token, and their outputs are orthogonal. The closest measurements are the average disagreement of Li et al. (2018) and the swap measure of Haklay et al. (2026). To our knowledge, no earlier work tests a pattern score against the output distance of the same two heads. On the sink our data agree with both papers, and our test does not show the mechanism of Corollary 2 (Table 30).

W1. Linearity and scale. We agree that Theorem 1(a,b) follows from linearity. The contribution is the measurement on trained heads. You are right that head norms set most of D\mathcal D (0.94 to 0.98). We therefore rescaled every output to unit norm. Pattern scores correlate with this scale-free distance at −0.17-0.17 to −0.03-0.03 and recover 4 to 15 percent of its neighbours (chance 3 to 9). The Gram distance of unit-norm outputs reaches 0.50 to 0.67 (Table 19). Scale is also part of what a head does, and Sandoval-Segura et al. (2025) find that output norms identify inactive heads.

W2. Scope. We add six models of up to 7.6B parameters (Qwen2.5-7B), windows of 1,024 tokens, a second corpus (C4) and BOS at the start of every window (Tables 22 to 24). Gu et al. also compute their sink metric on 64-token sequences without BOS. In every setting, averaged pattern scores recover 0.9 to 12 percent of output neighbours (chance 3 to 9), and head norms recover more than half. The condition S<NhS<N_h holds for 47 to 64 percent of the pairs that share values and for at most 1.6 percent of the others.

W2. Practical gain. The gain is a test that needs no training. From one set of forward passes, S<NhS<N_h tells which head can replace which, and where replacement will fail.

(1) In grouped-query models, the design of Llama 3 and Qwen2.5, SS finds heads that are cheaper to replace than to remove. Replacing 10 percent of the heads of every layer raises perplexity by 4, 2 and 2 percent on Llama-3.2-3B, Qwen2.5-1.5B and Qwen2.5-7B, against 17, 27 and 10 for the best removal rule (Table 23). On Llama-3.2-1B the costs are 6 against 28, and 3 against 5 with BOS. In these five settings substitution costs less at all three budgets.

(2) The test predicts the outcome pair by pair. Substitution costs less than removal for 74 to 85 percent of the pairs that pass and for 8 to 20 percent of the others. Inside key-value groups the shares are 74 to 88 and 45 to 54 percent (Table 25).

(3) The test also says where not to substitute. Without shared values, fewer than 2 percent of pairs pass. On GPT-2 and Pythia the best removal rule is then cheaper in the main protocol and in 16 of 18 cells of the other settings.

On Llama-3.2-1B the gain varies with the setting. On C4, substitution is cheaper at 20 and 30 percent (51 against 61, and 191 against 387) and removal at 10 percent (21 against 37). With 1,024-token windows the two are level at 30 percent (159 against 162) and removal is cheaper below. Perplexity remains our downstream metric, and we present the edits as a diagnostic.

Q2. What the theory proposes in practice. Theorem 1 gives two instructions.

(1) Do not choose heads to merge, prune or match across models by their attention maps. For any pattern score there are values that reverse its ordering (part a), and on nine models these scores recover output neighbours near chance.

(2) Compute the quantities of part (b), the head norms and the Gram blocks of the values. The same forward passes give them. Norms alone find the head with the closest output for 58 to 86 percent of heads (Table 22).

Theorem 3, with Theorem 2, gives two more.

(3) State whether the sink or BOS is kept or masked, and report the sink masses and the content distance as two numbers. The choice reverses 21 and 26 percent of JS verdicts on GPT-2 and Llama, and Theorem 2 gives the term that masking removes.

(4) A better pattern metric will not close the gap. Under the axioms of Theorem 3, the only scores that separate every key are multiples of the Aitchison distance, which correlates with D\mathcal D at 0.06 at most. The missing information is in the values.

Corollary 1 gives the last.

(5) Test S<NhS<N_h before replacing a head, and search inside key-value groups first.

Reviewer_zQZyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper examines whether attention patterns alone determine the function of attention and finds that they do not. This is a straightforward consequence of the way attention works, and is already well known. The paper also introduces distance measures on attention heads that incorporate the output vector - the significance of these is not clear and not discussed.

Critical strengths

None

Critical weaknesses

The central claim of the paper is trivially true and well-known. The additional dense mathematics add very little to the core scientific question. The paper is so LLM-written as to be close to unreadable.

Questions for authors

None

1 comment
AuthorsOpenReview ↗

Respond to reviewer zQZy

Thank you for the review. We answer its points in order. Changes are blue in the revised PDF.

1. "The central claim is trivially true and well-known." We agree. The output of a head depends on its values as well as on its pattern (Kobayashi et al. 2020, Elhage et al. 2021). Theorem 1(a,b) follows from linearity, and the criterion of Corollary 1 is immediate from its definitions. The submission presented these as results. The revision presents them as background in the abstract, the introduction and after each statement.

The known fact leaves open how much pattern similarity reveals about outputs in trained models, and practice still reads pattern similarity as function. Bali et al. (2026) write that they compare head functions through attention score matrices. The closest measurements are Li et al. (2018), who report an average disagreement between heads of 0.007 on attended positions and 0.881 on outputs, and the swap measure of Haklay et al. (2026). Both are aggregates or downstream measures. The paper adds three measurements on pairs of heads.

(a) Patterns against outputs. On GPT-2, Pythia-160M and Llama-3.2-1B, the head with the most similar averaged pattern is the head with the closest output for 2 to 9 percent of heads (chance 3 to 9). The two head norms find it for 65 to 86 percent, because outputs within a layer are nearly orthogonal. We also rescale every output to unit norm. Pattern scores correlate with that distance at −0.17-0.17 to −0.03-0.03 and recover 4 to 15 percent of its neighbours (Table 19).

(b) Where patterns are informative. Among Llama heads that share values, averaged-pattern scores correlate with S/NS/N at 0.12 to 0.50, against −0.04-0.04 to 0.04 for other pairs (Table 26). On four Llama-type models the per-token kept cosine correlates with the output distance at 0.49 to 0.68, and at −0.06-0.06 to 0.01 with head norms controlled (Tables 20, 27 and 29).

(c) When one head can replace another. Sh←gS_{h\leftarrow g} is the change of the layer output when head gg replaces head hh, and NhN_h is the change when hh is removed. The condition S<NhS<N_h holds for 62 percent of the Llama pairs in one key-value group and for at most 1.2 percent of other pairs. On five further grouped-query models the shares are 47 to 64 and at most 0.4 percent. Tested against the loss pair by pair, substitution costs less than removal for 74 to 85 percent of the pairs with S<NhS<N_h and for 8 to 20 percent of the others. Inside key-value groups the shares are 74 to 88 and 45 to 54 percent (Table 25). On Llama-3.2-1B, replacing 10 percent of the heads with SS raises perplexity by 6 percent, against 28 for the best removal rule, and this advantage reverses with longer windows and on a second corpus.

If earlier work reports (a) or (c) for pairs of heads, we would be grateful for the reference.

2. The significance of the distances "is not clear and not discussed." Section 2 now defines functional similarity by the effect of exchanging two heads on the predictions, and new Table 1 gives what each distance measures and its use, with a worked example. The Gram distance is what a pattern comparison measures once the values are added. The complete-output distance compares the vectors that two heads write. SS decides between substitution and removal, and the condition holds where heads share values, the design of Llama 3 and Qwen2.5.

3. "The additional dense mathematics add very little." We agree that the main text carried too many formal statements. Proposition 1 and Theorem 3 are now stated in the appendix with their numbers, and the main text keeps five. Appendix K gives the assumption and the use of each result and reports where the theory is weak on our models: the certificate of Proposition 1 certifies no head, and two bounds are too loose to be informative. One identity does work in the experiments. Theorem 2 gives the term that masking the sink removes, and keeping or masking the sink reverses 21 and 26 percent of verdicts on GPT-2 and Llama.

4. Readability. The abstract and the introduction are rewritten. They give the gap in prior work, two questions, the answer to each question and four numbered contributions. Section 2 defines every object before the first result, and each experiment opens with the test it runs.

Soundness. Every theorem has a complete proof (Appendices A and E to G), and Appendix D lists 13 predictions, including the three that failed. We would be glad to address any step that seems in doubt.

Reviewer_9ocoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Attention heads are often compared by how similar their attention patterns are, i.e., which tokens they attend to, but this manuscript argues that the content (the value vectors) in these positions should also be considered for similarity judgment. Two heads can attend to the same tokens and still pass very different information, or attend to different tokens and pass the same activation. The manuscript proves that attention patterns alone leave the difference between two heads' outputs largely undetermined, whereas the value vectors pin it down exactly. The authors introduce three measures that indicate how different two heads in the same layer are.

  1. Fixed-signature distance (Theorem 1). Each head is summarized by a single typical attention pattern: how much attention tokens give, on average, to each of the first 64 positions in a text, averaged over tokens at positions 64–191 and across many texts. The distance measures how different the two heads' outputs would be if each always uses that typical pattern, given the values it reads. Theorem 1 shows that the patterns alone leave this distance undetermined within a wide range, while the value vectors determine it exactly.

  2. Complete-output distance (Proposition 1). This compares what the two heads actually write into the model on real text. It uses the attention each head computes on each sentence, over all tokens, together with its own value vectors. It answers 'Do these two heads produce the same output?' Proposition 1 bounds how far the fixed-signature distance can be from it.

  3. Recipient-slot distance (Section 4). This measures how much the model's computation changes if one head is replaced by another, i.e., if one head's output is fed into the place where the other head's output normally goes. It answers the question `Can head g stand in for head h?' and is directional: g may replace h, but not the reverse. They show that such a replacement disturbs the model less than deleting the head exactly when this distance is smaller than the head's own output size. When heads share their value vectors, as in grouped-query attention, this condition reduces to a comparison of their attention patterns.

The authors also study the attention sink, the token that receives the most attention across most heads. It shows that keeping or removing the sink before comparing patterns changes what the comparison measures, and can change which heads are judged most similar.

Critical strengths
  1. The paper addresses an interesting problem. Interpretability often compares attention heads by the similarity of their attention patterns. The manuscript points out a basic gap in this practice: where a head attends to does not necessarily determine what it writes, because its output also depends on the input it receives in those positions (the value vectors).

  2. It clarifies what 'similar heads' mean. It distinguished between three questions:

  • Would the two heads produce similar outputs if each used its typical attention pattern?
  • Do o they produce the same output for the same text input?
  • Can any one head replace another?
  1. The measures are tied to concrete interventions. The recipient-slot distance links a measurement to a common operation: replacing one head with another. A simple criterion is proposed for evaluating such a replacement.

  2. The paper sheds light on attention sinks. It explains why keeping or removing the sink token before comparing heads can change, and even reverse, the order of similarity.

  3. A careful experimental protocol. Scores are computed on one set of texts and evaluated on a separate set. Results are repeated over three random seeds.

Critical weaknesses
  1. Hard to follow. The manuscript is written in a very compressed style, packed with too many notations and with little intuition, motivation, or examples from real scenarios. I believe the exposition can be greatly simplified. Specifically:
  • The three distances, which form the core contributions, are introduced in just a single dense paragraph. A numerical example or a table contrasting them would be helpful.
  • Results follow one another without explaining why each is needed or how it serves the main goal.
  • Not self-contained. Problem setting, definitions, the experimental protocol, and many terms are missing from the main manuscript. The reader is forced to look back and forth at the main text and the Appendix.
  1. Limited discussion of previous work. The related-work section is short. The contribution relative to previous work is unclear.

  2. Practical relevance. It is unclear to what extent the theoretical assumptions are satisfied by real models. For example, the main result assumes two compared heads in the same layers read the same values, but this holds only for grouped heads, as in LLaMA.

  3. The attention sink analysis appears disconnected from the core message (that the value vectors matter), and its practical relevance is unclear.

  4. Unclear and unconvincing evaluations. There is no external definition of 'functionally similar'. The reference distance defined by the authors primarily reflects the size of the heads' outputs rather than their behavior. Also, the experimental protocol is under-specified.

Questions for authors

The following questions relate directly to the weaknesses listed above.

  1. The three distances. Can the authors explain more clearly how the fixed-signature, complete-output, and recipient-slot distances differ, what question each one answers, and when each should be used? A short table or a running example comparing the three distances would greatly improve readability.

  2. Motivation. Can the authors better motivate the paper's flow and explain how each theoretical result serves the main research question? In particular, how do the results on the attention sink relate to the central message that value vectors determine the functional distance?

  3. Relevance to real models. Some of the theoretical results rely on assumptions that do not seem to hold in the experiments. Can the authors clarify which results are intended as practical tools and which as conceptual statements, and discuss under what conditions the assumptions hold for real models?

  4. Contribution relative to existing work. Can the authors contrast their contributions with respect to prior work?

1 comment
AuthorsOpenReview ↗

Respond to reviewer 9oco

Thank you for the careful review. Changes are blue in the revised PDF. Theorem statements are unchanged, and the new analyses were chosen after the reviews (Appendix L).

Q1, W1. The three distances, and what "functionally similar" means. Section 2 now gives an external definition. Two heads are functionally similar when exchanging them changes the predictions of the model little (Haklay et al. 2026). Our three distances are measured at the output of one layer, and Section 6 tests them against the change in next-token loss. New Table 1 gives the definition, the use and a worked example for each distance. Use the Gram distance dGd_G to read a comparison of average patterns once the values are added, the complete-output distance D\mathcal D to judge whether two heads write the same vector, and the recipient-slot distance Sh←gS_{h\leftarrow g} to decide whether gg can replace hh. Replacing changes the layer less than removing hh when S<NhS<N_h, the output norm of hh. In the example, case A has equal patterns and orthogonal outputs, and case B has different patterns, equal outputs and S>NhS>N_h.

Q2, W4. Flow, and why the sink. The abstract and introduction now give the gap, two questions and their answers. The sink takes 40 and 51 percent of the attention on GPT-2 and Llama, so every pattern comparison must keep it or mask it, and the choice reverses 21 and 26 percent of JS verdicts. Theorem 2 gives the term that masking removes. The link to the main message is scale. Against D\mathcal D, masking lowers the correlation of cosine and JS by 0.23 and 0.16 on these models (Table 2). Against the distance between unit-norm outputs, kept and masked scores are both at −0.13-0.13 to −0.03-0.03 (Table 19). This suggests that the sink mass informs about the scale of outputs.

Q3, W3. Assumptions. You are partly right. Theorem 1(a) and the sink form of Theorem 1(b) are stated for one value matrix, that is, two rows of one head or two heads of one key-value group. For two heads with their own values, Theorem 1(b) gives a block form, and every experiment uses it. The criterion S<NhS<N_h needs no shared values. Appendix K gives the assumption, the status on our models and the use of each result. Three results give no usable guarantee there: the certificate of Proposition 1 certifies no head, and the bounds of Theorem 1(c) and Proposition 2 are loose. Proposition 1 and Theorem 3 are now stated in the appendix. The criterion holds for 62 percent of the Llama pairs in one key-value group and at most 1.2 percent of other pairs, and for 47 to 64 and at most 0.4 percent on five further grouped-query models (Tables 22 and 24).

Q4, W2. Prior work. That a head output depends on pattern and values is known (Kobayashi et al. 2020, Elhage et al. 2021). The closest measurements are Li et al. (2018), who report an average disagreement between heads of 0.007 on attended positions and 0.881 on outputs, and the swap measure of Haklay et al. Practice still reads pattern similarity as function (Bian et al. 2021, Bali et al. 2026). We add a test of pattern scores against the output distance of the same two heads, a criterion for substitution tested against the loss, and the exact sink and content terms of pattern scores. Section 7 and Appendix J give the difference from each close paper. Three similarities of recent work correlate with the substitution cost at 0.39 at most, against 0.58 to 0.72 for SS (Table 31).

W5. Evaluation. The external criterion is the change in next-token loss, which uses none of our distances. Within layers, SS predicts the cost of a substitution at 0.60, 0.58 and 0.72, against 0.54 to 0.56 for D\mathcal D and at most 0.27 for the five averaged pattern scores of Table 8. Pair by pair, substitution costs less than removal for 74 to 85 percent of the pairs with S<NhS<N_h and 8 to 20 percent of the others. Inside key-value groups the shares are 74 to 88 and 45 to 54 percent, so shared values explain part of the gap (Table 25). You are right that D\mathcal D mostly reflects size, since norms predict it at 0.94 to 0.98. Head norms find the head with the closest output for 65 to 86 percent of heads, averaged patterns for 2 to 9 percent (chance 3 to 9). Against the distance between unit-norm outputs, pattern scores correlate at −0.17-0.17 to −0.03-0.03 and recover 4 to 15 percent of its neighbours (chance 3 to 9). The Gram distance of unit-norm outputs reaches 0.50 to 0.67.

Limits. In the main protocol, no substitution rule beats the best removal rule on GPT-2 and Pythia. On Llama-3.2-1B, replacing 10 percent of the heads with SS costs 6 percent perplexity against 28 for the best removal rule. This advantage reverses with 1,024-token windows (10 and 20 percent) and on C4 (10 percent). On four Llama-type models the per-token kept cosine reaches 0.49 to 0.68 with D\mathcal D, and −0.06-0.06 to 0.01 with head norms controlled (Tables 20, 23, 27 and 29).

Comments on the paper

1 comment
Area_Chair_1fZ4OpenReview ↗

Discussion phase coming to an end

Dear reviewers,

As we are approaching the deadline of the discussion phase, please read the authors’ responses and share your thoughts in the discussion as soon as possible. In particular, please clarify whether the responses have addressed your main concerns. Please also update your review and scores if appropriate.

Your timely input will help us reach a well-informed decision. Thank you for your time and effort!

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.