acceptodds
Under review as a conference paper at ICLR 2027

Scale, Drivers, and Limits of Representational Alignment Across Large Language Models

Abstract

Comparative analysis of large language models offers a means of identifying shared representational structure and systematic differences across them. However, aggregate measures of representational similarity, commonly used in these comparisons, can obscure how agreement varies across geometric scales and which linguistic properties support it. We introduce two complementary methods to resolve where models agree and what supports that agreement. Standard representational similarity analysis (RSA) summarizes similarity across all stimulus pairs, potentially obscuring differences in alignment between nearby and distant inputs. Neighborhood-Weighted RSA (NW-RSA) addresses this limitation by weighting stimulus pairs according to their proximity in representational space, enabling systematic comparison of alignment from local neighborhoods to global geometry on a common scale. Counterfactual RSA (CF-RSA) tests which linguistic properties support alignment by comparing intact representations in one model with representations under controlled linguistic perturbations in another. Using these tools across diverse language models, we find strong agreement in coarse representational geometry but substantial divergence in local neighborhoods. Alignment is observed both across broadly sampled natural text and within linguistic equivalence classes that preserve particular aspects of meaning or structure. Counterfactual analyses reveal selective sensitivity, with changes to logical scope reducing alignment substantially more than the tested changes to hierarchical syntax, event roles, or discourse structure. Alignment also decreases when sentence structure or lexical meaning is disrupted through word lists and jabberwocky, suggesting that the regularities of natural language support stronger convergence. These findings provide a differentiated account of representational convergence in language models and refine claims of representational universality by showing that strong overall similarity need not imply agreement in how models represent finer linguistic distinctions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
representational alignment, large language models, representational similarity analysis, linguistic structure probing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_m8xNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Existing research shows that different LM may provide similar representation structure (not the raw tensor which is uncomparable, but about the proximity structure). However, this paper claims that metrics like RSA only give a global measure, which may "obscure how agreement varies across geometric scales and which linguistic properties support it".

The paper introduces 2 modified metrics: NW-RSA and CF-RSA. NW-RSA assigns more weights to nearby representations, essentially calculating a localized measure. CF-RSA measure the turbulence effect when changing one side of the representation (when g(.) is applied).

Experiment 1 (5.1) shows that broad geometric agreement coexists with weaker agreement among nearby inputs. Global RSA is in general larger than NW-RSA. When the scale of comparison gets larger (less localized and more globalized), so does the NW-RSA.

Experiment 2 (5.2-5.3) shows that representation structure is aligned within most linguistic equivalence class (probably excluding Syntactic group). Such alignment is selectively sensitive to perturbations: only counterfactual changes to logical scope produce particularly large reductions in alignment.

Experiment 3 (5.4-5.5) shows that alignment is relatively robust to paraphrasing but weaker when sentence structure or lexical meaning is disrupted.

Overall, the paper shows that while different models may preserve the global representation structure, this kind of alignment depends on the geometric scale of comparison and the linguistic properties of the inputs.

Critical strengths
  1. The research topic is interesting and thoughtful. The similarity of representation structure across models has similarity on a global scale, but this alone does not tell us what exactly do the models agree on. The paper inspects it from different layer of granularity.

  2. The experimental framework for investigating what exactly the models have representational alignment is very systematic.

Critical weaknesses
  1. The main analysis use the penultimate (second-to-last) layer and mean-pool hidden states over all non-special, non-padding tokens. While appendix E have some additional results on the effect of depth, the exact reason to choose second-to-last layer as the main research target needs a further clarification.
Questions for authors
  1. Regarding results 5.1, the finding is that broad geometric agreement coexists with weaker agreement among nearby inputs. Is it possible that the lower local RSA arise partly from the fact that within a small neighborhood, all sentences have very similar distances, and in this case, small variations in these distances could then produce substantial changes in their rankings? If this is the case, can we still say that the local representation structure across models are different, or it's merely a statistical confounder?

  2. Figure 4 shows that some linguistic equivalence classes have RSA comparable to global RSA, but "Syntactic" class has a significantly lower RSA. Do you think this tells you something about the linguistic property itself? Do you think this has implications that language models are good at identifying certain linguistic property, while others (like syntactic) are not important factor of the model representation.

Reviewer_47AaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a battery of updated representational alignment measures (NW-RSA, CF-RSA) to test whether the core tenet of the platonic representation hypothesis — that sufficiently capable intelligent systems converge to the same representation space — holds up for a set of controlled linguistic stimuli. The broad goal is to assess whether stimulus pairs are organized in a way that preserves local and global representational structure and sensitivity to perturbation in model representations. Across their analysis of 16 models the authors find evidence for stronger alignment globally vs. locally and that logical scope and scrambling perturbations are the main drivers of misalignment in counterfactual stimuli. Overall, I enjoyed reading this paper and found the methods interesting and the results informative. I have some comments regarding specific results and framing that I outline in sections below.

Critical strengths
  • The paper targets a topic that is centrally being discussed in the community and brings to the forefront the question of measurement metrics, which is useful. Separating geometric from linguistic locality is also a useful conceptual distinction to make.
  • The stimuli design draws on well validated and familiar sets that will be legible to both the NLP and computational linguistics community. The manipulations are also well thought out and clearly presented.
  • The authors evaluate their hypotheses under a variety of similarity metrics.
Critical weaknesses
  • I’m concerned that the local–global gap could be at least partly a statistical artifact. Restricting a correlation to near pairs restricts the range of distances, which attenuates correlations, and small distances are also likely to be the noisiest. The union-of-kNN weighting additionally selects pairs that are close in one model but not necessarily the other. The toy simulation shows the measure can increase, but doesn’t address what happens with shared geometry plus noise. In the RSA literature this is usually handled with a noise ceiling (Nili et al., 2014). Here, something like alignment between the same model under different prompts/seeds, or a simulated shared-geometry-plus-noise baseline, would help establish how low local NW-RSA should be expected to fall even for highly aligned systems.
  • It appears to me that the logical-scope result might be influenced by the amount of lexical change in the items. Given that representations are averaged across tokens, the increased drop for the logical scope might simply reflect the difference in lexical scope, no? It is possible I am missing some obvious control that was applied here.
  • Relatedly, I think the paper would benefit from motivating several analysis choices including the choice to mean pool, which layers are attended to, and the correlation distances. Many of these decisions are defensible under the literature, however laying out reasons explicitly would be helpful.
  • The persistent alignment for non-word lists is quite compelling and seems to be convergent with similar results in the field (https://arxiv.org/abs/2601.11432). However, I am wondering whether here too low level factors such as token count and length of sentences oculd influence these factors.
  • How contingent are the results on model scale? In other words, could the authors consider providing more of a model-level breakdown in the main text along with some measure of alignment vs. param count. This seems particularly relevant when qualifying the platonic rep. hypothesis.
Questions for authors
  • Could the authors more clearly delineate their contributions in terms of similarity metrics relative to the other work cited, e.g., the more recent topographic RSA work?
  • How do the authors view this work relative to work that seeks to understand representational ‘concept’ alignment and its relation to model ingredients (https://arxiv.org/abs/2510.01030)
  • What is the noise ceiling for NW-RSA at smaller neighbourhood sizes?
  • Related to a concern regarding motivating analysis choices, how might the results change with alternative pooling strategies such as looking at the last token for the models or after some form of residual layer standardization?
Reviewer_yNYJOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes some novel RSA methods for assessing how representational similarity between models is affected by representational scale and sources of variability in the inputs. The results show that models are generally less aligned at more fine-grained representational scales, and their similarity is sometimes sensitive to particular properties of the input. These results are taken to challenge the platonic representation hypothesis.

Critical strengths
  • The paper is very clearly written and all methods are clearly described. Experiments are well documented and the results are easy to interpret.
  • The neighborhood weighted RSA is interesting and novel.
  • The paper highlights the importance of the data over which representational alignment is evaluated. Much of the work that aims to evaluate representational alignment does so over generic datasets often similar to those that models have been trained on, without evaluating the impact of alternative sources of variability (such as those explored in the 'counterfactual' settings in this work) which may reveal misalignment that's hidden by the use of more generic datasets.
Critical weaknesses
  • The choice to focus on local, neighborhood-weighted similarity is somewhat arbitrary. Why should we care more about the alignment of similar things than we do about the global representational structure of the model? It is unsurprising that local alignment is lower than global alignment, as this will reflect more fine-grained differences, but why should we care more about one than the other? Overall the motivation for this approach is unclear.
  • Related to the previous point, the paper does not present any evidence that focusing on local neighborhood similarity produces substantively different results. For instance, does the rank order of model-to-model similarity across models change as a function of the locality weights? The paper essentially shows that if you zoom in on fine-grained differences there is overall weaker alignment between models, but that seems essentially guaranteed to be true and not especially informative.
  • There is no connection between the NW-RSA results and the 'counterfactual' RSA results. It is somewhat odd to emphasize the benefit of NW-RSA and then abandon it in favor of traditional RSA in the counterfactual analyses.
  • The notion of 'counterfactual' RSA is a bit over-promoted here. This is basically a term to refer to the introduction of additional sources of variability in the data over which alignment is assessed. This is a good thing to do in assessing representational alignment, but this doesn't really need to be presented as if it's a new method.
  • The results are presented as though they pose a challenge to the platonic representation hypothesis, but it's not clear that this is the case at all. The emphasis of the platonic representation hypothesis is that model-to-model alignment increases as model scale (parameter count or training data size) and competence increases. There are no results here showing how these new RSA metrics change as a function of model size or capability. Thus, while more fine-grained measures can reveal discrepancies between models that aren't captured by global measures, there's no evidence that these more fine-grained measures don't also scale with model size/capability.
Questions for authors
  • What is the motivation for looking at local neighborhood similarity? Why should this matter more than global similarity?
  • Are there any instances where NW-RSA provide qualitatively different results than global RSA?
  • Why not use NW-RSA in the counterfactual analyses?
  • Is there any evidence that these new RSA measures don't scale with model size/capability (as global RSA does)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.