acceptodds
Withdrawn from ICLR 2027

Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

90 views

Abstract

Names are personal identifiers, but they also carry social meaning and are widely used to evaluate how language models treat different people. Such evaluations typically assume that matched names are comparable model inputs. We show that this assumption often fails at the lexical interface: matched names are not necessarily matched inputs. Some names receive direct single-token access, while others are assembled from multiple subwords, creating unequal name-surface support. Across nearly half a million first names and 12 LLM-associated tokenizers, direct lexical access is highly selective, model dependent, and uneven across race- and gender-associated name metadata. We introduce NameTrace, a model-native, fine-grained, pre-behavioral framework for measuring whether unequal name-surface support remains a vocabulary property or becomes visible in task-relevant internal representations. NameTrace measures concept accessibility from the model's own probabilities over task-specific adjective axes with continuous task-aligned weights. On matched atomic and short-fragmented names within the same race/ethnicity–gender-associated strata, support predicts systematic differences in concept accessibility across fellowship, hiring, clinical assessment, and lending. These differences persist across all eight matched strata, extend across model families, and transfer to unseen names. Hidden-state interventions further show that the measured task directions have downstream leverage, shifting later constrained choices. Unequal lexical support is therefore demographically structured at the input and remains visible in task-relevant model computation. NameTrace makes lexical comparability measurable, supporting a broader principle: behavioral comparability begins with lexical comparability.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
name tokenization, lexical comparability, tokenizer bias, internal representations, concept accessibility, fairness evaluation, representation analysis

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xQd7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether socially matched first names are comparable inputs to language models. It first audits 12 tokenizers on about half a million first names. It finds that single-token access is selective and uneven across race- and gender-associated name groups, even after accounting for frequency and length. It then compares matched pairs of atomic and short-fragmented names within demographic strata. Using adjective probabilities read from an intermediate layer, it reports that atomic names receive higher scores on task-relevant concepts such as competence and promise. Finally, it shows that the gaps replicate on held-out names and that steering along a task direction shifts a later binary choice.

Critical strengths

The tokenizer audit is the most useful part of the paper. It is large, clearly reported, and gives evaluators a concrete reason to check lexical comparability when building name-based benchmarks. The matched-pair design is careful, with matching within strata and a development and evaluation split in which measurement choices are fixed in advance. The ethics discussion around the voter-registration data is good.

Critical weaknesses

My main concern is that the evidence for the central claim is weaker than the paper presents. The claim is that lexical support shapes task-relevant internal representations.

The first issue is identification. Names become single tokens because they are frequent in tokenizer training data, so atomic names are very likely more frequent in pretraining data than their fragmented matches. Matching on Florida voter counts does not control for this. Prior work already links lower name frequency to less reliable representations. The authors acknowledge the confound, but the framing throughout still attributes the effect to lexical support. A web-frequency control or a frequency-matched analysis would help.

The second issue is the size and consistency of the effect. The pooled gaps are averages over three models and are carried almost entirely by Qwen3-4B. Llama’s gaps are an order of magnitude smaller, and Ministral’s are near zero or negative. More importantly, when the same score is computed from the model’s actual output logits, the hiring and clinical gaps reverse sign. This makes the practical implication unclear. It is not shown that atomic names are treated better at the point where decisions are made. A direct behavioral comparison on the existing two-choice prompt, without intervention, would address this.

The third issue is that the RQ3 analyses do not test what they are said to test. The cross-name transfer analysis subtracts the mean development gap from evaluation pairs drawn from the same matched pool. This is a split-half replication rather than transfer. The intervention shows that steering along an adjective-derived evaluative direction changes an evaluative choice. That is expected from prior steering work and would hold for any pair of names. It does not show that the gap between atomic and fragmented names runs through this direction. Removing the direction and checking whether the gap closes would be a more direct test.

The readout method also needs more detail and validation. The paper does not specify how intermediate hidden states are mapped to adjective probabilities. Several results suggest the readout is sensitive to layer choice. Examples include the shift from layer 2 to layer 25 in Gemma-3-4B and the sign change in Qwen’s lending gap between layers 18 and 30.

On novelty, An & Rudinger (2023) already showed that tokenization length contributes to first-name bias. The paper would benefit from stating more clearly what it adds beyond broader coverage of modern tokenizers.

Questions for authors
  1. How is an intermediate hidden state converted into a probability over adjectives?
  2. On the two-choice prompt without intervention, does the model select the atomic name more often?
  3. Can you control for pretraining frequency, for example with web-corpus counts?
  4. After the task direction is projected out, does the atomic versus fragmented gap shrink?
Reviewer_8bCxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors of this paper introduce a framework called NameTrace. It shows that similar names might receive unequal lexical support, e.g. one name may be tokenized into multiple subwords while another remains an entire token. The authors then measure how “accessible” task-relevant adjectives are to hidden states of prompts containing names (i think?), and show that it relates to names’ lexical support.

Critical strengths

The overarching idea of this paper might be interesting to some people.

Critical weaknesses

Figure 1 is incredibly wordy and crowded. The purpose of summary figures should be to summarize and ease a reader’s cognitive load, rather than add more to it.

The beginning of the paper was okay in terms of writing but as one progresses through the paper, the content becomes more convoluted, information-dense, and jargon-y.

There are many results but the main narrative or thesis of the paper is very much lost. The narrative is kinda there in the intro and conclusion, but it becomes muddled in the middle of the paper, perhaps buried in paper-specific terminology and findings that don’t sufficiently stitch together in a way that is explicitly communicated to the reader. For example, I wasn’t too sure what the point of the CKA analysis was, or what it really tells us bigger-picture-wise. It just seems to say that models from the same family represent information similarly, which isn’t very surprising. There were multiple points where you show a graph or a plot but it’s not clear what the takeaway should be.

It’s not super clear in the intro (and also later on) what “matched” means for names. Some examples in the section describing how pairs are formed would be helpful.

One of the paper’s key claims (“[lexical] support predicts systematic differences in concept accessibility”) just says each is related to the other, which is unsurprising. Names that are less common and associated with minoritized groups are known to be discriminated against in prior work. Names that are less common are also less common in data used to train a tokenizer. So, it makes sense that the two are related.

There’s a little bit of too much reliance on the Appendix for details around experimental setup. For example, I would’ve liked to know what the different tasks are (beyond just the generic name of the task), and how names are matched.

Questions for authors

What are “weak orthographic cues”?

What does “concept accessibility” actually mean? I feel like there was an attempt at defining it, e.g. lines 107-111, but it doesn’t make much sense.

What model layer is used to report results in section 4.2?

Reviewer_5yStOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that not all names are afforded the same level of privilege, a factor that should be considered in name-permutation-based evaluations (such as those measuring bias). The authors first demonstrate that names common among dominant subgroups (gender/race) are more likely to be tokenized as atomic units rather than fragmented ones. Subsequently, in Sections 4 and 5, they illustrate how this tokenization bias propagates to the representation layer and ultimately influences model decisions.

Critical strengths

The study is highly lucid, and the research questions are articulated clearly. The overall flow of the paper and the logical connections between the research questions are sound.

Critical weaknesses
  • It is unclear what actionable solutions the authors are proposing or how NLP practitioners can practically apply the findings from this study. As the authors note, in NLP evaluations (such as those investigating phenomena like bias), practitioners typically perturb the name in the input to observe changes in model behavior. However, it remains ambiguous how this study proposes to improve model quality or refine bias evaluation methodologies. While the authors successfully identify one mechanism through which models generate bias, they fail to adequately address its practical significance.

  • Several questions remain unanswered, and various aspects are left unexplored. For instance, it is unclear how the tokenization algorithm directly contributes to this problem. A name is typically represented as a single token primarily because it appears frequently in the training data. Consequently, the properties of being a single token and being a "common name" are inherently entangled. The conducted tests only demonstrate that the direction is usable, rather than proving that the tokenization process itself causes the performance gap. The methodology lacks a control experiment that maintains a constant name frequency while varying the tokenization.

  • The data is derived from a single US source, meaning the study is predominantly centered on US naming conventions.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.