- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper examines the problem of entity copying in the Qwen3-8B language model. It shows that the top half of the layers are responsible for entity copying and that context tokens, both when they are semantically related or unrelated to the entity in question, affect copying through their attention weights. To this end, the paper develops two techniques, called genie-in-a-bottle and attention lobotomy, that control causal factors in the computation of attention, and test them across five prompt templates.
Critical strengths
-
Careful causal procedures for identifying the sets of layers where entity copying takes place and the effect of attention from context tokens.
-
Results on the Qwen3-8B model showing that entity copying occurs at the top half of the layers and that context tokens guide the model to allocate more attention to the correct entity token.
Critical weaknesses
-
Results are restricted to just one model. As such, it is unclear whether the conclusions hold in general for LLMs or reflect peculiarities of the Qwen3-8B model.
-
Weak conclusions for layer analysis. The analysis indicates that entity copying occurs in the top half of the layers. Such localization is neither very specific nor surprising. Can entity copying be localized more precisely? Does the copying location vary with the specific entity, context, or prompt?
-
The examples given in the text use well-known names, such as Einstein or Sheakspear, as entities. Copying such names may be affected by their frequent appearance in training. Did you also use generic names and even made-up names to control for such bias?
Questions for authors
See comments above. Overall, the layer localization is too crude and unsurprising. The analysis of the impact of context tokens is interesting, but conclusions drawn from a single language model are not sufficiently reliable.
5 comments
Results regarding Layer Contributions are Genuinely Novel
While we understand the role of deeper layers in entity copying is not surprising for the reviewer, we would like to point out that, as subjective as “surprising” is, it is actually counterintuitive that entity copying happens late in the model layers. This is because entity copying simply extracts and transfers one token’s information to another — a basic functionality of any single attention layer. The fact that early attention layers are neither necessary nor sufficient is therefore highly informative on how the model divides the labor among its layers.
Additionally, per the reviewer’s request, we have sharpened our analysis of the contributions of layers. It suggests that, as opposed to the complete second half of layers, two disjoint groups of layers in the second half jointly contribute to entity copying, separated by a group of layers with minimal contribution. This is supported by a “valley” of reconstruction success in Figure 3 and a “rise” in Figure 4. We have updated the paper’s abstract, introduction, and Section 3 with those revisions.
Contribution of Prompts and Contexts Have Been Explicitly Illustrated
While we share the reviewer’s interest in understanding more about the impact of specific prompt templates and contexts in entity copying, we would like to point out that it is already demonstrated in Figure 3 and 4. In both figures, experiments on different templates support the same conclusions regarding the contribution of layer groups and context tokens, with only minor variations in success rates among them. Since the objective of the work is to study the contributions of layer groups and context tokens in entity copying, we believe the two figures have sufficiently illustrated what the reviewer asks for.
Generalizable Results on a Different Model
We appreciate the reviewer’s request for additional models to test the generalizability of our claims. We therefore attach our findings on Gemma-4-12B, which broadly supports the same conclusions while having intriguing differences. We will update our appendix with the findings on Gemma.
In summary, results on Gemma also validate the necessity for context tokens to attend to entity tokens jointly, as the removal of their attention causes huge degradation. Differently from Qwen, however, even the entity’s identity, as is measured by the recovery of nonexact names, is hard to extract for Gemma, compared to Qwen. In addition, Gemma also bolsters the role of description tokens in transferring entity information, as appending the correct, incorrect, and “a human” description show the same trend of improved results. Differently from Qwen, however, incorrect descriptions do not show a large discrepancy from correct descriptions, and can also meaningful recover entity identity in some cases.