- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies how entity binding forms in language models. The authors use activation writing to exchange entity states and their estimated Slot and Content components across layers. They find that early-layer pairing typically follows the new receiving position, whereas moving both components together at middle layers better preserves the original pairing. The authors further show that writing entity states separately supports two-hop accuracy close to that obtained with text input, while content mixing reduces two-hop accuracy even when all receiving positions and original states are retained.
Critical strengths
-
The paper investigates how binding forms through state exchanges rather than only analyzing established representations. The comparisons among whole-state, Slot-only, Content-only, and joint exchanges are well targeted, and all three models show distinct early- and middle-layer patterns. The shared-Slot experiment further shows that source-position differences need not be supplied for early pairing to form. These results provide valuable interventional evidence about entity binding and are the main reason for supporting acceptance.
-
The content-mixing experiment includes useful controls. The authors distinguish the reduction in written positions from the changes in state composition caused by pooling. They then compare partner Content additions with norm-matched random perturbations while keeping receiving positions and original states fixed. On Gemma, content mixing reduces two-hop accuracy from 99.0% to 19.75%, while direct-question accuracy remains at 96.75%. With random perturbations, two-hop accuracy remains at 98.5%. This shows that the decline in relational use cannot be explained solely by perturbation magnitude.
Critical weaknesses
-
The experiments do not fully separate the effects of receiving position and surrounding relational context. Moving an entity state also changes its local context, while relation words and sentence structures remain intact. Appendix A.4.2 acknowledges that these factors are not measured separately. Moreover, some middle-layer WHO conditions preserve the original answers even after random replacement, so they cannot establish whether the moved state carries pairing information. The evidence supports a role for the receiving context in binding, but the attribution to position itself and its consistency across query types remain limited.
-
Two-hop failures may arise because the second fact becomes inaccessible, rather than because two independently accessible facts cannot be combined. The direct question tests only the first fact, and mixing the state at the tag position alone is sufficient to impair the two-hop answer. The current comparison therefore does not distinguish these explanations. Scoring choices also affect some results: in Table 25, adding capitalized “Unknown” and applying strict scoring reduces Llama-3.1’s unmixed accuracy from 73.75% to 18.5%. Gemma’s results are more robust, but these issues limit the mechanistic conclusions and cross-model evidence from the two-hop experiments.
Questions for authors
-
Does “receiving position” refer to the actual token position, relative position, or relational role within the sentence structure? Can the existing experiments distinguish these factors?
-
Considering only conditions in which random replacement can change the answers, does the middle-layer preservation of original pairing hold across models and both WHO and WHAT questions?
-
When content mixing causes two-hop failures, can the model still correctly answer a direct question about the second fact, such as “What has the tag?” If this has not been tested, please provide the comparison to distinguish fact-access failure from failure to combine facts.
-
How do the authors explain Llama-3.1’s sensitivity to the capitalized “Unknown” candidate? Does evaluation with a more complete candidate set or free generation still support the effectiveness of separate writes and the recovery from leaving the tag position unmixed?
-
Single-component exchanges may create combinations that are incompatible with the model’s original state structure. What evidence shows that the advantage of joint exchange comes from preserving pairing information rather than differences in state compatibility?