- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper analyzes the geometry of concept representations in language models, such as circular configurations of weekday representations. More specifically it studies if and how contextualization and in-context learning can change the "default" canonical structures.
Even though the topic falls squarely into my area of expertise, the paper is difficult to understand due to the incoherent writing.
Critical strengths
none
Critical weaknesses
From what I can tell, I see neither much novelty nor technical contributions or insights.
Lack of novelty: It is already known that in-context learning can override semantic priors and that this reflects in internal representations (e.g., the cited "In-context learning of representations" by Park et al. 2025), and it is also known that contextualization affects representational geometry. For example Tiblias et al. find that "Different continuations produce drastically different topologies."
Tiblias, F., Bigoulaeva, I., Niu, J., Balloccu, S., & Gurevych, I. (2026). Hypothesis-driven feature manifold analysis in LLMs via supervised multi-dimensional scaling. Transactions on Machine Learning Research. https://openreview.net/forum?id=vCKZ40YYPr
Lack of contributions or insights: The conclusions section summarizes the paper's findings as follows:
Across two model families, pretrained organization persists at the entity token but gives way down- stream to context-specified relational geometry. The same entities form a cycle or hierarchy on command, and entity identity is causally bound to the specified relation, with clean binding emerg- ing only at scale. Whether this downstream geometry is built or reconfigured remains open; either way, recovered “world-model” geometry must be interpreted relative to context and readout posi- tion.
None of this meets the standards of a top tier conference.
Questions for authors
none
Dear Reviewer, thank you again for your review. As the discussion period ends on November 18, we wanted to check whether our response addressed your concerns, and whether there is anything further we could clarify or analyze in the remaining time.