acceptodds
Under review as a conference paper at ICLR 2027

How Does "English (US)” Become the Default? Triangulating Structural Bias Towards American English Across the LLM Pipeline

92 views

Abstract

Large language models (LLMs) are increasingly embedded in educational, professional, and public infrastructure, yet widely used platforms expose “English (US)” as a primary English setting despite the global diversity of English. We ask: How does “English (US)” become the default? We study this question as structural bias, examining how geopolitical histories of data curation, digital dominance, and linguistic standardization intersect with the LLM development pipeline. Using British English as a controlled reference, we construct a curated resource of 1,813 matched American English (AmE)–British English (BrE) variants and introduce DiAlign, a dynamic, training-free method for estimating regional alignment from distributional evidence. We triangulate the AmE preference across data exposure → representation → generation, jointly examining pretraining and post-training data, tokenizer behavior and provenance, model prediction cost, and generated language across developer countries, prompt conditions, domains and sources, linguistic categories, and registers. AmE is consistently favored across all six audited pretraining corpora and 21 post-training datasets, is generally represented more compactly by tokenizers, and receives lower prediction cost. It also remains the dominant generation default under neutral English prompting; British-English prompting shifts this preference toward BrE but does not consistently eliminate the AmE default. To our knowledge, this is the first rigorous pipeline-wide study of structural bias across major phases of LLM development. Our findings show that contemporary LLMs privilege AmE as the de facto norm, raising concerns about linguistic homogenization, epistemic injustice, and inequity in global AI deployment, while providing a rigorous basis for targeted component-level intervention.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
structural bias, linguistic variation, LLM evaluation, tokenization, post-training data, regional alignment, language-model bias

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VWnHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper does an end-to-end pipeline evaluation of the difference in how American and British English are treated in different aspects of model construction, from data to tokenization to perplexity to generation. They also introduce DiaAlign, a metric for understanding how much text leans towards either American or British English.

Critical strengths

This is an interesting paper and I appreciate the topic it covers. I like that it examines a wide breadth of systems at every stage of the pipeline to demonstrate that these patterns are broader than just any single system. The analysis is thorough and well written.

Critical weaknesses

Many of your analyzes are in some way entangled, which makes it hard to understand the goal. Data influences tokenization and both influence the perplexity of the final model. I think in general, I would have done some sort of controlled study, systematically ablating each stage in the pipeline to understand its influence. While I understand that is difficult to do on a large scale, one such study with a single, small LLM (say, Qwen 0.6B) would have improved the strength of the work. I think this is my biggest sticking point with the paper. You claim you are examining how "linguistic asymmetries persist across the LLM pipeline," but from my perspective it seems to basically come down to data, which in turn has these downstream impacts. If your argument is that multiple stages of training are biased towards AmEng, I don't believe that point can be made with systems trained on this unbalanced data. If your argument is that unbalanced training data impacts many aspects of performance, then this paper aligns with that argument but that is a less novel argument.

I also think this paper would also be greatly improved by including Indian English (or another largely-spoken dialect) -- British English benefits from many systems that might be hiding how significantly these issues impact performance and Indian English is a better proxy for a very widely spoken but understudied dialect.

Some smaller concerns:

  1. Your method of defining DiaAlign concerns me in a few points, especially since it is one of the primary metrics used across the paper. First, the dependence on n-gram frequency from Google Books feels as though it may result in bias due to the register/domain, especially when examining highly casual/conversational text as is done in a lot of the pre- and post-training corpora. I also am concerned by the choice to upweight samples that contain terms that are part of your curated lexicon. Since this curated lexicon is part of the construction of the test set, it seems like it could introduce issues and should largely be unnecessary if the lexicon is as binary as claimed (with one dialect showing significantly higher usage of some of these words). I similarly don't see a point in downweighting samples that are not highly dialectal since in theory, that should be automatic if the fraction of f_am/f_be is near 1. I would like to see it confirmed that the results hold without this unnecessary weighting. I would also like to see it confirmed that the DiaAlign metric is equally accurate on more diverse data, rather than the formal written register seen in books and news articles.

  2. For the prediction cost and generation samples, I would like to see if the results hold when the input context is in the target dialect. While I understand the point that when the input is "neutral" the output leans American, I am curious whether that is flipped by the user simply speaking in British English, with that raising the likelihood of British orthography enough to overcome the differences.

Questions for authors
  1. Do you find that the tokenization differences are localized more to specific categories? Intuitively, I would imagine "words with sublexical spelling variation" would be the majority of cases, since both -ise and -ize likely appear enough to be tokenized as suffixes. I also notice that you include "jewelry vs. jewellry" in this category rather than as part of group 5, why?
  2. What is Velvet? You bring it up for the first time in the tokenization section but I don't know what that is. Did you mean GPT-4o?
  3. Is there a reason for using GPT-4o tokenizer instead of the GPT-5 tokenzier, considering they are different?
  4. Can you confirm to me that non highly-dialectal text ("My name is X") is not assigned to AmEng by your metric?
1 comment

Thank you! I have updated my score.

Reviewer_TCafOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work aims to study why American English is structurally the effective “default” in LLMs, using British English as a baseline. The analyses specifically look at different components of an LLM pipeline, including pretraining and posttraining data, tokenizer representations, and generation. To aid analyses, the researchers introduce DiAlign, a method for estimating the regional variety of given text. The analyses show that American English is more represented than British in a wide variety of pre- and post-training datasets, uses more tokens on average to represent semantically equivalent British English text, and that when prompted in British English, some—but not all—preference for American English responses is mitigated.

Critical strengths
  1. Broad Analysis. The analysis covers multiple sources (i.e., pre-/post-training data, tokenization) and realizations (i.e., prediction cost, response preference) throughout an LLM pipeline, surfacing broadly consistent patterns rather than datasets/model-specific artifacts.
  2. Social Significance. The aims and results are socially significant, highlighting gaps between American and British English representation according to the set of metrics used and substantiating the preferential treatment of American English as a “default” in model development.
Critical weaknesses
  1. Synthesis of Experiments. As discussed and referenced (e.g., the cited Dodge et al., 2021), prior work has individually studied similar dialect biases in training data, tokenization, and model behavior. While the examination of AmE across different components of the LLM pipeline is principled and systematic, the particular novelty and utility of this pipeline-wide approach is unclear without more thorough synthesis and linking of the experiments and results. For example, the pre-/post-training datasets evaluated appear to be largely disjoint from the evaluated models (many of which are closed-weight), while data/models like DOLMa/OLMo may be more appropriate to study a single, cohesive pipeline.
  2. DiAlign Reliability & Validity. The DiAlign metric is derived from Google Books frequencies, representing only one of many domains of texts that may be highly represented in pretraining data (e.g., social media/forums and the broader internet). Furthermore, the metric is validated only on distinguishing American and British news, similarly representing only one domain. Without broader representation and evaluation with American/British English speakers, the validity of the metric is unclear and results are disconnected from targeted harms.
Questions for authors
  1. How does the DiAlign metric behave on text that is neither markedly American or markedly British? Given that AmE is more highly represented online and the frequencies are derived from Google Books, could it be that the DiAlign metric is itself biased toward American English and driving the results?
  2. Does DiAlign assume that all texts are either American or British English? How are other dialects (e.g., African American English) handled?
  3. Following Blodget et al., (2020), how specifically do these results capture the downstream harms discussed in the Conclusion and what are these harms more specifically? Making the anticipated harms more explicit may help more clearly motivate the study.

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (Blodgett et al., ACL 2020)

Reviewer_u595OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper explores how English (US)” become the default dialect in LLMs. Using British English as a controlled reference, the authors build a resource of 1,813 matched American British word pairs (spelling contrasts like color/colour and vocabulary contrasts) and introduce DiAlign, a training-free method that classifies text as American or British. DiAlign compares how often short word sequences (2–5 words) appear in Google Books' American and British corpora, and reaches 93% accuracy on HuffPost vs. BBC news. The paper traces American preference through (1) Data (2) Representation and (3) Generation.

The paper finds that multiple pre-training corpora prefer American form. Tokenizers often split British words into more tokens and across 10 models, ~65-80% of neutral English prompts answers are classified American.

Critical strengths
  1. The paper studies the issue of structural bias towards American English and understands where in the pipeline the bias originates from. They study data, tokenization, model generation. American preference shows up across pre-training corpus, post-training datasets, tokenizers, models. Even StableLM-2 built by a UK developer uses a similar tokenizer as GPT-4, which shows that some of the inherent biases are carried across regions.

  2. The paper uses controls in the likelihood experiment. The authors compare sentence pairs that differ only in the regional form, keep only pairs with identical token counts so that tokenization can't explain the loss gap. They then check with embedding similarity that the paired sentences really mean the same thing. All checkpoints assign a higher loss to the British version compared to the American version. They also show that American English is clearly overrepresented across pretraining and post-training data, establishing an exposure asymmetry.

Critical weaknesses
  1. The paper shows that American English form is over-represented across the LLM pipeline but it is not clear if that leads to a harmful bias. Model generation is consistent with what it is trained on (Trained on more AmE leads to more generation in AmE). Do British-English inputs receive lower quality responses or do we see other failures with other English dialects that would be worth testing. While the current auditing pipeline described in the paper is useful, it seems like the practical value of it would be in cases where we see these differences affect users.
Questions for authors
  1. Can you report the full confusion matrix for HuffPost/BBC benchmark evals? 96.25% recall, 90.67% precision for AmE seems to imply that DiAlign mislabels British text at a higher rate than the reverse. How does this affect RQ3 result that british prompting does not eliminate AmE deafult?
  2. In Table 4, how much questions are there and could you add Confidence Intervals?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.