acceptodds
Under review as a conference paper at ICLR 2027

A Linear Name-Origin Prior in the Recall Heads: How Language Models Fabricate and Misrecall Facts About People

Abstract

Ask a chat model where a person who does not exist was born, and it answers with a city. We ask what computes that answer. On 402 invented names certified absent from three pretraining corpora, Wikidata and Wikipedia, five open instruction-tuned models, one only when required to answer, name a city of the name's linguistic tradition almost every time (0.99 to 1.00 with related traditions pooled, 0.88 to 0.99 without). We trace the invented fact to the machinery of factual recall running on a name-origin prior. The late heads that write any city answer, a real person's recalled birthplace included, hold in their value and output weights a map from a name's tradition to its demonyms, countries and cities. A linear origin direction at the name tokens carries the prior. Adding the difference between two traditions' means makes the model invent a city of the donor tradition for 0.51 to 0.83 of fresh names (0.98 in Llama when required to answer), never for random directions of the same norm. The same prior costs real people. Those whose birthplace contradicts their name are recalled at a quarter to a half of the rate of fame-matched people. The gap survives equal corpus exposure and holds among people the model knows. Moving the direction shifts the recalled birthplace of two thirds of the people Gemma-2-2B knows, while disturbing half of their known birth years. Shifting the representation also shifts refusal, which can in turn be moved without changing what is said, and whether a model declines depends on the name's tradition. In OLMo 2, whose pretraining corpus we can read, the prior and the map form by about one percent of pretraining. The mechanism yields a forecast of what a model will invent from a name list, at the level of the tradition, and a refusal push scaled by corpus evidence that keeps known answers. A detector that compares an answer with the name's prior fails, because where a name fits the record the prior names the true city. Every pass/fail criterion was fixed before its data, and every outcome, met or not, is reported.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.