- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors study the representations models learn when trained on synthetic location data: concretely, they train LMs to solve tasks related to the distance angle between, angle between, etc. different cities. The use the same setup as previous work (Park '26) and extend it in two ways: 1) they intervene on hidden states for different locations, finding that displacing and interpolating the hidden representations gives the expected geographic readout 2) they find that training on new locations (e.g. fictional "Atlantis" cities) performs better if the intial pretraining occurs with long-tailed distributions (sampling from a Zipf distribution) compared to uniform sampling, implying that maintaining representational plasticity through pretraining on long-tailed data facilitates later finetuning.
Critical strengths
- Understanding the dynamics of how and when learned representations generalize is an important and open question. This is a clever synthetic setting and the trade-off between plasticity and structure is insightful.
- The Zipf distribution is empirically observed in natural language (and in many domains), so it seems plausible that the good results we see for autoregressive models across a variety of domains is related to balancing this plasticity and structure tradeoff.
Critical weaknesses
- My main concern is that this paper seems not fully baked. The contributions are incremental to Park '26 (that shares the same setup) and large sections smell of LLM writing (difficult to interpret results dumps lacking clarity or context). While I think this is interesting and promising work, this needs quite a bit of polishing and potentially some additional contributions.
Clarity
- Is the normalized improvement metric provided anywhere other than the Park '26 reference?
- Figure 8: It would be good to give a bit more context in the caption; I had to backtrack quite a bit through the paper to understand what is happening here
- In fact, most of the captions could use a bit more context.
Questions for authors
- In the interpolation, etc. experiments: Is there any interpretable notion of hidden state distance that corresponds to output space distance?
- Is there anything noticably different in the representations between the Zipf training vs the uniform training that could lead to the observered difference in plasticity? Do you see anything interesting in the new representations that you are trying to train in when they "work" vs "don't work"?
5 comments
Thanks for the response. I'm not sure I understand your last point, though. My questions about the representations had more to do with the geometry of the representation space: is the structure of the Zipfian training representations somehow more interpretable in a way that shows clear integration of new entities (where the uniform pretraining does not)?
While the work is interesting, I still think this paper is not fully developed and would need either several more experimental settings or an additional contribution to be merit acceptance.
Putting it simply, the geometry of representations is mostly similar between the pre-trained models (uniform vs zipfian). You can see this in the decoded real-city representations visualized in gray in Fig. 6. You'll notice that the cities are slightly more diffuse in the zipfian (alpha = 1.25) model. [Note: Fig. 6 is post-fine-tuning, but the gray cities look the same as post-pre-training.] Fig. 5(a) shows how this harm to the real-city world model is a trade-off w.r.t. plasticity, although our results don't establish that it is the cause for plasticity.
Our feeling is that our contributions (listed 1-4 in the Introduction, L80-100) are thorough and meaningfully independent from what is investigated in Park (2026). Are any of these listed contributions under-developed? Or could you please say more substantively what kinds of additional contributions are missing?
Thank you for your thoughtful review! We were glad to hear that you found our question important and the trade-off we identify insightful. We’d be happy to address your concerns below.
We appreciate your feedback about the clarity of our writing. We recognize that parts of our paper need more context and clarity, and will continue revising our text. We kept the figure captions very brief to meet the page limit and relied on the surrounding text to explain our findings. Now that we have an additional page, we will extend our captions and make our figures more self-contained.
We used some AI assistance in our writing process, but our paper’s main body is substantially human written.
We do share Park (2026)’s setup, but our work’s central question is about how pre-training shapes a model’s subsequent plasticity (i.e. ability to learn new entities). We do so via 3 contributions: (1) showing how world model representations are used in model computations and how newly learned representations can be reused by existing computations; (2) showing how plasticity declines during pre-training and identifying how pre-training data distributions remedy this; and (3) examining which aspects of the proposed distributions contribute to these benefits and their trade-offs. We feel these goals and investigations are meaningfully different from Park’s.
We describe normalized improvement in App. B.3 (L1020-1037). We will add a brief definition into the main text.
Yes. In the uniform pre-trained model, Euclidean distances between real-city representations at the intervention layer strongly correlate with the model’s generated distance answers (r2=0.91±0.02, mean ± sample SD across three seeds).
We understand that the real-city representations are substantially similar after uniform and Zipfian pre-training. In both cases, models learn world-aligned representations and all pre-training tasks operate on these world-aligned representations (e.g. rather than learning city representations independently per task).
There is a difference in new city (Atlantis) representations between settings. When Atlantis cities are integrated in the world model (e.g. via Zipfian pre-training), these Atlantis representations can be reused by the task circuits learned from pre-training, which enables cross-task generalization. Otherwise (e.g. with uniform pre-training and distance fine-tuning), the model effectively learns Atlantis representations independently from the existing world model, and produces a solution specialized to the fine-tuning examples. This solution generalizes poorly to other tasks and to queries between two Atlantis cities (which is underrepresented in the fine-tuning data; rather than between an Atlantis city and real city). This is consistent with reduced reuse of the existing world model.
===
We hope to have addressed your concerns! Please consider raising the score if so. Please let us know if you have additional concerns or questions.