acceptodds
Under review as a conference paper at ICLR 2027

A Linear Name-Origin Prior in the Recall Heads: How Language Models Fabricate and Misrecall Facts About People

22 views

Abstract

Ask a chat model where a person who does not exist was born, and it answers with a city. We ask what computes that answer. On 402 invented names certified absent from three pretraining corpora, Wikidata and Wikipedia, five open instruction-tuned models, one only when required to answer, name a city of the name's linguistic tradition almost every time (0.99 to 1.00 with related traditions pooled, 0.88 to 0.99 without). We trace the invented fact to the machinery of factual recall running on a name-origin prior. The late heads that write any city answer, a real person's recalled birthplace included, hold in their value and output weights a map from a name's tradition to its demonyms, countries and cities. A linear origin direction at the name tokens carries the prior. Adding the difference between two traditions' means makes the model invent a city of the donor tradition for 0.51 to 0.83 of fresh names (0.98 in Llama when required to answer), never for random directions of the same norm. The same prior costs real people. Those whose birthplace contradicts their name are recalled at a quarter to a half of the rate of fame-matched people. The gap survives equal corpus exposure and holds among people the model knows. Moving the direction shifts the recalled birthplace of two thirds of the people Gemma-2-2B knows, while disturbing half of their known birth years. Shifting the representation also shifts refusal, which can in turn be moved without changing what is said, and whether a model declines depends on the name's tradition. In OLMo 2, whose pretraining corpus we can read, the prior and the map form by about one percent of pretraining. The mechanism yields a forecast of what a model will invent from a name list, at the level of the tradition, and a refusal push scaled by corpus evidence that keeps known answers. A detector that compares an answer with the name's prior fails, because where a name fits the record the prior names the true city. Every pass/fail criterion was fixed before its data, and every outcome, met or not, is reported.

Primary area:
interpretability and explainable AI
Keywords:
hallucination, mechanistic interpretability, factual recall, activation patching, steering vectors, name-origin bias, fairness, refusal and abstention, sparse autoencoders, large language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_cQeaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks a simple question. When a chat model is asked where a made-up person was born and names a city anyway, what decides which city it picks? The usual story is that the model guesses. The authors argue that it runs its normal fact-recall machinery and reads a guess off where the name seems to come from.

To test this they invent a few hundred people from 11 naming traditions, mostly European plus a few South Asian ones, along with mixed names that take a first name from one tradition and a surname from another. Every name is checked against large training corpora, Wikidata and Wikipedia to make sure no such person exists. Then they ask five open chat models, the largest at 8B, where each person was born. Llama nearly always refuses, so they start its answer for it.

The answers are very consistent. A made-up Baltic name gets a Baltic city almost every time, but different models seldom choose the same city. The heads that write the city are largely the same ones used when the model recalls a real person's birthplace, and in the two Gemma models, switching them off breaks both. If you nudge the model's internal picture of the name toward another tradition, the invented city moves with it, and random nudges don't do this. In one model the nudge also moves real people's birthplaces. The authors then show a real cost: people whose birthplace doesn't match their name are recalled far less often than equally famous people whose birthplace does. Whether a model refuses also depends on where the name is from. In OLMo 2, where the training data is public, the prior shows up very early in training.

They check the work with random-nudge controls, a baseline that looks only at spelling, a second judge model, and a rerun on fresh names. What stands out is that every pass/fail test was set before the data came in, and every result is reported. More than half of the 93 tests failed, and the paper says so plainly.

Critical strengths
  1. The question is clear, and the paper actually follows it all the way through. It starts with what the models do, finds the part of the model that writes the answer, shows that changing the name's representation changes the answer, and ends with a cost to real people. The care that went into making the names is a big part of why I believe the result. Because each name was checked against the training data, the easy objection that the model has seen these people before doesn't hold.

  2. I appreciated how open the paper is. Its tests were fixed in advance, and it reports the ones that failed, with reasons. That makes it easy to tell which claims are solid and which are shaky. The finding that matters most to me is the fairness one. The same habit that produces made-up answers also makes the model worse at remembering real people whose birthplace doesn't fit their name, and that holds up even for people the model otherwise knows well.

Critical weaknesses
  1. The mechanism is thinner than the paper makes it sound. Each tradition is built from just 12 first names and 12 surnames, and the test names reuse those same pieces, which the authors admit in their limitations. So the "origin direction" could simply be recording which familiar name pieces are present. A plain spelling classifier already gets the tradition right about 99% of the time, and just swapping in a name from the other tradition moves the answer at least as often as the steering does. The "shared recall heads" are also less specific than they look. The same heads write the city for a capital-city prompt with no person in it, and replacing them with their average output barely hurts real recall in Qwen and Llama. Steering real people also wipes out about half of the birth years the model knew, which the authors say may mean the whole person is being damaged rather than redirected. I read this as good evidence that models guess a city from what a name looks like, but not yet as evidence of a specific recall circuit running on a linear prior.

  2. Several headline claims rest on tests that failed or only passed after the fact, while the abstract reads as if everything held. 51 of the 93 pre-registered criteria were not met, which the abstract doesn't mention. For OLMo, the falsification test was triggered and saved only by a clause the pre-registration never stated, and the "forms by about one percent of pretraining" result clears its bar by just 0.002. The claim that refusal can be moved without changing what is said rests on small numbers: in one Gemma arm the model refuses everything, so nothing is left to check. Refusal varying by tradition holds up mainly in Qwen, and which traditions get refused changes across models, wordings and checkpoints. These points don't sink the paper, but the abstract and conclusion should match what actually held up.

Questions for authors

Clarifications:

  1. Can you rerun the steering test on names built from first names and surnames that were never used to build the direction?
  2. On the main held-out set, how do random directions at the same layer and size as the actual edit do? The random control there uses a different layer and a smaller size.
  3. For real people whose names mislead the model, where do the wrong answers land if you count all of them, including ones that don't map to any tradition?
  4. In the refusal-steering experiments, how many answers are left to score in each arm?
  5. Was the OLMo context clause written before you saw the few-shot results, and did every checkpoint get the same prompts?
  6. What is the release plan for model answers about real people who have few Wikipedia links and may not be public figures?

Suggestion (future work): testing larger models, non-Latin scripts and relations beyond birthplace and university would show how general this is.

1 comment

Thank you for the detailed response and the new experiments.

Most of my concerns are addressed:

  • The rerun on names built from unseen first names and surnames (Appendix O.2) is the test I asked for. The direction transfers, with a clear loss (0.38 to 0.74 in the free-answer models), and the probe stays above the spelling baseline. So the direction is not only recording familiar name pieces, but part of the original effect was.
  • The path-patching test shows that the origin shift reaches the answer through the shared heads. Calling them a generic city stage and limiting the necessity claim to the Gemmas now matches what the evidence shows.
  • The random control at the same block and norm, the count over all wrong answers, the per-arm refusal counts, and the note that the OLMo clause was written after the result answer my questions. I appreciate that the numbers are reported even where they go against the paper.
  • The changed release plan resolves my ethics concern.

Two things I would still like to see in the final version:

  1. Report the unseen-component steering numbers in the main text next to Table 1. They are the cleaner estimate of how well the direction generalises.
  2. Soften the claim that refusal can be moved without changing what is said. It rests on 13 to 36 answers per arm, and the abstract still states it without qualification.

The limits on model size and script remain, but they are stated. I am raising my rating to 3.

Reviewer_gNybOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates the relationship between names of people and the birthplaces language models assign to them, for fabricated and real people. It finds the following. First, the paper finds that models invent birthplaces belonging to a fabricated name's tradition (e.g. 'Tartu, Estonia' for fabricated name 'Priit Ozolinš'). Second, it finds that the name tokens have a linear direction representing origin which causally determines the city the language model responds with. Third, this paper states that the mechanism used for this is factual recall. Lastly, the paper finds that language models are worse at recalling birth cities of people whose birthplace doesn't match their name tradition (e.g. 'Jorma Kaukonen' is a name from the Finnish tradition but this real person was born in Washington).

Critical strengths
  • the experiments in this paper are documented in detail in the appendix, so I expect they are easy to replicate and verify.
  • the finding that language models are worse at recalling the birth place of real people whose name tradition does not reflect their birth city is novel finding.
Critical weaknesses
  • Many details are not explained in the main text. The paper often refers to a table in appendix A, which is a list of pre-registered criteria and outcomes, but when the main body references this table it often gives the experiment definition and outcome in a single high-level sentence with no motivation of why the experiment was performed or what to take away from the result. Needing to constantly refer to appendices makes this paper difficult to read and understand. This is the main reason I lean towards rejection. One example of this is the paragraph on lines 203-207 ("A pre-registered confirmation") which draws three extra conclusions without explaining what the goal of these experiments is or how conclusions were made.

  • A large part of this analysis depends on fabricated names that match traditions. These names need to 1) not be in the models' training data and 2) match a tradition. I have concerns about both these points. On (1), the main analysis is done on four models whose training data is not public, making it impossible to know whether the fabricated names do not appear in training data. Some analysis is repeated on OLMo 2, which releases its training data, but the paper does not actually check all of OLMo's training data. Namely, the paper looks at the pretraining corpora but not the post-training corpora (even though these are publicly available). On (2), appendix B says names are seeded from 12 given names and 12 surnames per tradition, and the AI use statement says they were assembled "with the assistant's help and checked by the authors". How were these twelve chosen, and how do we know they are representative of the tradition? How do we know they are not biased by the assistant?

Questions for authors
  • Why did you do the main analysis on four models that have proprietary training data instead of making OLMo 2 the main evaluated models (with these four maybe as supporting analyses)? Because we can't know the training data of these models, we can't verify that the fabricated names used in this study do not appear in their training data. Moreover, why did you only repeat some of the analyses you did on the main four models on OLMo and not all?
Reviewer_rKGHOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at how language models invent biographical facts about people who don’t exist, and whether those inventions come from the same mechanisms the models use to recall real facts. The authors build 402 synthetic names from eleven naming traditions, plus 804 mixed-origin variants, and check them against corpora and knowledge bases so the models are unlikely to have seen them. Across five open-weight models, the invented birth cities mostly match the linguistic origin of the name. This points to a systematic name-origin prior rather than random invention.

On the mechanistic side, probing finds origin information that can be read out linearly from name-token representations. Attention-head attribution and ablation show that many of the same components drive both invented and correctly recalled cities. Steering a name’s representation toward another naming tradition often moves the invented birthplace to a city from that tradition. The same steering also changes how the models recall real people’s birthplaces and, in some settings, whether they refuse to answer.

The prior also gets in the way of real facts. The models are much worse at recalling the birthplaces of real people whose names don’t match where they were born than they are for controls of similar fame. OLMo 2 checkpoints show these origin representations and behaviors appearing early in pretraining. The paper also tests forecasting, hallucination detection and correction methods, and reports the ones that failed as well as the ones that worked.

Critical strengths
  1. Strong evidence that fabricated biographies depend on name origin: Across several model families, the birth cities invented for nonexistent people mostly match the linguistic origin of the name. The design is careful: a constructed synthetic-name set, corpus-based absence checks, mixed-origin variants and a separate confirmation set. The consistency across models shows these fabrications are not arbitrary.

  2. Mechanistic analysis backed by causal interventions: The paper goes beyond probing to attention-head attribution, ablation and steering of origin directions in the residual stream. Shifting a name’s origin redirects the invented cities and also affects recall of real people, which is meaningful causal evidence. The analysis of people whose birthplaces don’t match their names shows the practical cost of these priors. The authors also openly report pre-registered criteria that failed, which helps transparency and reproducibility.

Critical weaknesses
  1. Shared mechanism not fully established. Head overlap, ablation effects and steering don’t uniquely point to a shared factual-recall mechanism. The same heads also drive plain capital-city completions, so they may reflect generic geographic-token or answer-extraction functions. Ablation effects are strong in Gemma but weak in Qwen and Llama, which limits generality. Steering shows that origin information affects outputs, but not that the identified heads carry this effect. Path-specific interventions linking name-token representation → heads → output city are needed, especially since the paper’s main contribution is explaining how fabricated facts are computed.

  2. Name-origin prior not isolated from lexical features or broad perturbations. The theoretical result (Sec. 10, App. F) holds only under a feature-bottleneck assumption that is never established on its own, so it formalizes the hypothesis rather than deriving it. Because surface characters and token embeddings already predict origin very accurately, the behavior may come largely from lexical associations, and the added value of deeper representations isn’t isolated. Difference-of-means steering also degrades unrelated attributes (e.g., birth year), which suggests it disrupts entity representations broadly rather than targeting a separable prior. Finally, held-out names reuse given names and surnames from the fitting pools. This weakens the generalization claims and may make the uncertainty estimates look tighter than they are.

Questions for authors
  1. The identified heads also drive capital-city completions with no person names. Can the authors show, with a path-specific intervention, that the name-origin signal reaches the output through these heads? And does disrupting that path selectively affect origin-conditioned fabrication?

  2. The origin patch also degrades unrelated facts (e.g., only about half of known birth years stay correct). How do the authors tell selective prior modification apart from broad disruption of entity representations? Controls such as other name-linked attributes, orthogonalized directions or matched perturbations would help here.

  3. Held-out names reuse given names and surnames from the fitting pools. Do origin decoding and steering still work for names built entirely from unseen components?

  4. Which of these failures bear on the central hypothesis, and which concern only auxiliary applications? Specifically, how should readers weigh the failed origin-removal intervention, the model-dependent ablation effects and the fluency costs?

Comments on the paper

1 comment

I urge all reviewers to respond to the authors before the rebuttal period ends, to ensure a fair chance for the authors to address any outstanding issues.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.