Where Does Entity Binding Come From?

To use factual knowledge, e.g., "Alice has the umbrella," a language model must do more than represent the entities: Alice and the umbrella. It must bind them, tracking which entity goes with which. Prior work showed that binding appears in model activations as identity vectors attached to each entity. But where does this binding come from: is it carried in the content of the representations, or computed from where they sit? We first show that, at early layers, binding is constructed from receiving position: placing the same entity content at different token locations predictably changes which entities are bound together. As computation proceeds, however, moved representations increasingly preserve their original pairing, showing that the established relation becomes carried by the representation itself. We then ask what this mechanism implies for injected knowledge. Writing entity states separately at their corresponding positions supports multi-hop reasoning close to ordinary text, whereas pooling those states sharply degrades relational use. By retaining all receiving positions while mixing entity Content, we show that position alone is not sufficient: two-hop reasoning can fail even while direct factual recall remains available.