- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper tests the Platonic Representation Hypothesis by comparing autoregressive models trained on Danish administrative life histories with independently trained language models representing descriptions of the same event codes. Across seven life-model sizes and two language-model cohorts, the authors measure local neighborhood agreement using mKNN and global similarity agreement using RBF CKA, with permutation calibration that repeats the complete layer search. Local alignment increases with life-model size and correlates with language-model performance, whereas global alignment remains above chance but plateaus along the life-model scaling axis. The paper validates these findings through controls for lexical overlap, coding hierarchies, representation dimensionality, random initialization, and co-occurrence baselines, providing evidence for partial representational convergence in an additional modality.
Critical strengths
The comparison provides a valuable test of representational convergence using unusually independent training sources. The life models learn from confidential administrative histories without access to event descriptions, while the language models represent descriptions without access to those histories. This setting reduces the shared-training-data explanation that complicates other cross-modal comparisons. Although it does not eliminate shared human or institutional structure, it provides a meaningful extension of the empirical evidence for the hypothesis. The study carefully addresses several plausible measurement artifacts and alternative explanations. Repeating layer selection within each permutation is particularly valuable, and the lexical, coding-hierarchy, dimensionality, and co-occurrence controls strengthen the local-alignment finding. The distinction between increasing local agreement and plateauing global agreement also makes the conclusion more informative than a single aggregate similarity result. These controls and the appropriately qualified findings are the main reasons for my positive recommendation.
Critical weaknesses
The conceptual contribution is primarily an extension of an existing hypothesis and evaluation framework to a new domain. The observed local-versus-global distinction broadly follows earlier findings, and the experiments establish geometric agreement without demonstrating functional interchangeability or a concrete consequence of that agreement. Moreover, administrative codes and their descriptions reflect shared human categories and institutional processes, some of which may remain beyond the measured controls. Consequently, the evidence supports partial agreement between the tested representations more directly than it supports a broader interpretation of convergence toward a shared model of reality. The scaling results leave important attribution questions unresolved. Language-model capability is strongly confounded with model size: in the HellaSwag cohort, local alignment correlates approximately as strongly with log-parameters as with benchmark performance. The conclusions also depend on the representation: contextual vectors show increasing alignment, whereas static embeddings on the larger code set peak at an intermediate size, and fixed-layer comparisons give weaker associations.
Questions for authors
-
Do the available language-model results provide evidence that performance predicts alignment beyond model size and family? A size-adjusted or within-family analysis, with appropriate uncertainty estimates, would help establish whether capability contributes explanatory information beyond scaling.
-
Given the different trends for contextual vectors, static embeddings, and fixed-layer comparisons, what is the intended scope of the convergence claim? Is it specifically about the existence of an increasingly aligned representation somewhere within each model, rather than convergence of a consistently defined representation?
-
Have you tested training on chronologically shuffled histories while preserving the events within each individual’s history? The Markov baseline already captures substantial alignment, so this comparison would help determine how much of the remaining advantage depends on temporal structure.