- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper asks why language models can distinguish different meanings of a word even when their hidden vectors are similar. The authors split the average vectors for two meanings into a shared part and a difference vector. They then study how these parts relate to the model’s output weights. The experiments cover 16 models and examine changes across layers and context lengths. The paper reports that the difference vector is more aligned with output-sensitive directions than a random vector, and that the hidden states become more aligned with these directions at the final layer.
Critical strengths
-
The paper gives a useful way to study why similar vectors can still carry different meanings. Looking at both the shared part and the difference between the vectors makes this question easier to understand. The analysis also connects these differences to the model’s own output weights.
-
The experiments include 16 models with different sizes and architectures. The authors compare different layers and context lengths and use both linear probes and ablations. This gives a broader view than results from a single model or measurement.
Critical weaknesses
-
Equation (1) holds when the two sense centroids have equal norms, but the paper only gives numerical evidence that this is a reasonable assumption. Section 5.3 correctly calls the equation an identity. However, Section 6.1 describes it as predicting how sense separation changes across layers. Both sides are calculated from the same pair of vectors, so their agreement mainly shows that the equal norm approximation works. It does not explain why the vectors change across layers. The authors should show how close the two norms are, or gives explanation of when this approximation is valid and how unequal norms affect it.
-
The result in Figure 2 is largely expected. With more context, the target token receives more clues about its meaning, so the two groups of representations may become easier to separate. It is unclear what this result tells us beyond that. The experiment also changes input length, topic information, and word sense clues together. A comparison with equally long but irrelevant context would help show which factor matters.
-
The introduction says that the final-layer sense centroids have cosine similarity above 0.99 across all 16 models. Table 6 shows otherwise. Only GPT-2 and Pythia-160M exceed 0.99. Figure 2 also shows values of r above 1, which would not give cosine similarity close to 1 under the equal norm assumption.
Questions for authors
See the weakness.