acceptodds
Under review as a conference paper at ICLR 2027

What's in a Place Name? Spelling as a Confounder in Coordinate Probes of Language Models

Abstract

Advances in language models have enabled applications in geospatial fields, involving tasks ranging from geocoding to spatial question answering. These applications have raised the question of whether, and to what degree, language models encode spatial information about real-world entities. Language model probes are increasingly taken as evidence of learned representations, but the extent to which spatial information is available from the name alone has not been systematically measured. We show through a series of experiments that probes trained to predict coordinates from place names recover a large share of their signal from the name alone. This share rises with entity granularity, from under a tenth at metros to above two fifths at venues, in the same order across model families and does not diminish as model parameters scale, revealing a confounder in probe-based claims of learned spatial representations.

Primary area:
foundation or frontier models, including LLMs
Keywords:
linear probes, probing classifiers, language models, geographic knowledge, world models, spelling share, random-initialization control, untrained baseline, string baseline, named entities, toponyms, interpretability, model scaling, evaluation methodology

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_M1jxOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Probes of large language models have demonstrated that the models encode geospatial properties with respect to entities such as countries and cities. However, it is unclear how much of the spatial information is directly encoded in the initial entity tokens compared to later model activations. To explore how the initial entity embeddings contribute to the overall spatial encodings, the authors train a linear probe that extracts an entity's coordinates on a globe, and then compare the extracted coordinates against the true coordinates using spherical distance. The authors also design the Spelling Share metric, which uses the error distance between the true coordinates and predicted coordinates to measure the contribution of the initial token embeddings to the final spatial representation.

Critical strengths

Significance - Studying the spatial awareness at varying levels of spatial granularity (i.e. village --> town --> city --> country) is useful to ensure that the conclusions hold for a wide range of spatial categorizations.

Critical weaknesses
  1. Clarity - The writing of this paper is quite confusing due to wording and phrasing issues and missing details. (1) Some of the writing is very difficult to interpret. For example, lines 42 - 44 are discussing how linear probes have been used before. The authors write that the probes have been fitted to "the colour a word names and the state of board." I do not understand what that means. Does the first part refer to using a probe to see what color a model maps a target word to? Also, how does this relate to the second part, which I assume is referring to the state of a boardgame? Also, line 105 states "They probed sentence embeddings beside the same encoder with random weights..." It is unclear what the authors are trying to say. Are the authors saying that two of the same model are probed and compared, with one model being trained and the other untrained? What does beside mean? I should note, these questions are rhetorical and meant to highlight examples where the writing is extremely unclear. There are more instances of this lack of clarity throughout the paper. (2) The methodology is unclear. Details behind design choices are missing and there is seemingly conflicting information. See Questions for Authors for specifics.
  2. Soundness / Significance - It is difficult for me to understand how the Spelling Share metric actually measures an entity name's contribution to the model's spatial understanding of that entity. For me, this calls into question whether the results are meaningful. See Questions for Authors for specifics.
Questions for authors
  1. Where are the probes being placed? Line 207 states that probes are placed at "the residual stream at the last token of the name at layer l, with layer 0 the embedding output." The phrasing is a confusing, but I am interpreting this mean that the probes are placed directly after the token embeddings are obtained in the model during a forward pass. This would be directly after passing tokens to a model and before the embeddings are passed to the first transformer layer. But then, that doesn't fully make sense because the line mentions the residual stream. Line 211 states that "the layer l* of each probe is one whose median ... is lowest," seeming to indicate that probes can be placed at any transformer layer. To me, it seems line 207 and line 211 are giving conflicting information.
  2. How is the Spelling Share measuring an entity name's contribution to the model's spatial encoding of the entity? It seems to be more of an error ratio since it is comparing the error distance of one encoding to a baseline error distance. Also, why does the ratio need to be converted to bits?
Reviewer_qDPJOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Previous studies, like Gurnee and Tegmark (2024), have found that language models encode representations of space and time. This paper attempts to understand the degree to which these representations are encoded within token embeddings rather than other model components. The authors run an experimental setup in which the language model (but not the token embedder) is swapped out with a randomized model. They then compare the accuracy of linear probes for spatial data with accuracy in the unperturbed setting. In doing so they find that you can still recover some probing accuracy in the perturbed setting. The authors interpret this as some kind of 'spelling confound'.

Critical strengths
  • I do like the problem setting and find world models studies to be fascinating. So I think the author has picked an interesting domain to study
Critical weaknesses
  • I really just don't get what the point of this paper is. Why does it even matter if there is a 'spelling confound'?
  • Gurnee and Tegmark report R2 across all layers of their models -- doesn't that alleviate concerns about understanding the influence of the tokenizer?
  • The writing is really hard to follow in the way that AI generated text sometimes is. I don't mean to throw that in as an accusation; I think the authors should rely on whatever tools they think is appropriate. But I constantly found myself getting lost in what the point of all of the little details were. Broadly there just wasn't a clear narrative for me to latch on to.
Questions for authors
  • I think it would be useful to start our discussion with a clarification of what a 'confounder' actually is and why it matters in this case!
Reviewer_bCHoOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper analyzes whether predicting a place’s coordinates from representations of language models is possible. The paper sets up two settings, probing a pretrained language model and probing a language model with random weights, and introduces spelling share, which measures the fraction of the pretrained model probe’s improvement over a random model probing result. The results show that the performance varies across the spatial granularity of the place names.

Critical strengths

None

Critical weaknesses

W1. The paper introduces spelling share to compare coordinate prediction using pretrained language model representations with prediction using the proposed baselines. However, the motivation for this metric needs stronger justification. It is unclear what the reported fraction additionally provides beyond directly comparing the prediction errors of the pretrained model probe and the baselines.

W2. The explanation in the method section is difficult to follow. The relationship between coordinate error, spherical-cap area, and the quantities called bits, bits_T, and bits_C is not explained clearly enough. This makes it difficult to understand what spelling share measures and how the reported fractions should be interpreted.

Questions for authors

Q1. What additional understanding does spelling share provide beyond directly comparing the prediction errors of the pretrained model probe and the baselines?

Q2. Why are coordinate errors converted into bits using the sphere area? How does this conversion affect the interpretation of spelling share and the comparisons across different levels of geographic granularity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.