acceptodds
Under review as a conference paper at ICLR 2027

Same World, Different Geometry: A Controlled Test of the Platonic Representation Hypothesis

207 views

Abstract

The Platonic Representation Hypothesis Platonic holds that models trained on different modalities converge, with scale and competence, toward a common representation of the world. Testing whether models converge to the world, and not only to each other, requires knowing the world's true structure, which is unknown for natural images and text. Consequently, existing evidence compares representations only with each other, not with the world, and cannot manipulate what makes them agree. We therefore construct a world whose structure is known: eight Hamiltonian systems with ground-truth latent variables , rendered independently as video and text. The two encoders share no weights or joint objective and are trained in matched pairs that differ only in their prediction targets, so the cause of alignment can be manipulated directly. We show that a shared world is not enough: each encoder recovers the physics, yet the two do not align. Alignment arises only when both are also trained to predict information about . We characterize this phenomenon through the -subspace, the subspace of each latent on which both encoders carry the same linear statistics of . We prove that the latent spaces align on it and show that a single linear map transports information between the spaces. Alignment depends on the information the targets carry about , not on whether they are shared: different targets, each informative about , align the encoders, while an identical target uninformative about does not. Large-scale pretraining does not substitute for this information: models pretrained on unrelated natural corpora decode the physics but share no such subspace. In this setting, convergence is installed by information about , not by observing the same world.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Platonic Representation Hypothesis, Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RRxWOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces a principled way of testing the Platonic Representation Hypothesis (PRH). The PRH states that sufficiently large and well-trained models converge to the same representation of the world. The authors claim that this phenomenon is always checked by assessing whether models align with each other, without any measure of alignment to the world. They address this limitation by using a set of simulated physical systems parametrized by a Hamiltonian with ground truth variables hh, from which text and video are generated in pairs coming from the same hh.

Video and text models are trained independently on each modality using two different regimes: generative task only, and generative training with an additional task that predicts the physical properties of the system. They vary whether the image and text models are trained to predict the same quantities, partially overlapping quantities, or different ones. They show that training the models with an additional task significantly increases their alignment, suggesting that for models to align, seeing the same « world » is insufficient to produce strong alignment, they must also be trained to predict information about it.

Critical strengths
  • Making a controlled test for Platonic Representation is very interesting and, in my opinion, an important contribution: having a parameterized model of the world allows us to precisely measure the impact of the variables on alignment, distinguish whether alignment comes from shared information or a shared objective, and measure precisely which quantities promote alignment.
  • Authors provide an extensive set of experiments to study in depth their proposed test for PRH. They provide ablations, study the impact of the prediction heads, compare to frozen pretrained encoders, and include several physical system families. The set of experiments supports the main findings well.
Critical weaknesses
  • It is not certain that the text and video models considered here have the scale and competence to reach the PRH regime. Although the authors include experiments with frozen pretrained models, they are trained on natural data and evaluated on synthetic data outside of their training distribution. The observed limited alignment may therefore partly reflect this distribution shift.
  • Considering physical systems is a good way of having a controlled way of testing the PRH, but it is a very limited representation of the world. The convergence predicted by the PRH may emerge because models see a world so complex, rich, and diverse that the way they are trained constrains them to converge toward only one shared representation. Since the physical systems considered here cover a restricted part of that world, the training may leave greater freedom in how models represent it, potentially explaining the limited alignment when λ=0\lambda=0.
Questions for authors
  • Table 31 and Figure 9 show that alignment does not increase monotonically with λ\lambda. Could you explain this pattern and provide the alignment metrics more finely between λ=0\lambda=0 and λ=0.1\lambda=0.1?
  • Can you clarify Figure 3? Is the ground truth represented before or after the swap? What are the visual differences between the pairs of images?
Reviewer_AwPXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a controlled investigation of the Platonic Representation Hypothesis (PRH), examining whether models independently trained on different modalities develop aligned representations of the same underlying world. The authors construct a benchmark using Hamiltonian dynamical systems with known ground-truth latent variables, represented through videos and natural-language descriptions. Separate video diffusion and autoregressive text encoders are trained with and without auxiliary supervision to predict physical properties.

The central finding is that independently learning the same physical system does not necessarily produce strong cross-modal alignment, even when both models encode recoverable physical information. Introducing auxiliary supervision substantially improves alignment, increasing CKA from 0.21 to 0.72, mutual-kNN from 0.047 to 0.408, and mapped retrieval recall@1 from approximately 0.04 to 0.84. Experiments with shuffled, asymmetric, and partially overlapping prediction targets suggest that alignment depends on information about the underlying physical variables rather than merely sharing an identical objective.

The authors further introduce the H-subspace to characterize shared linearly accessible information between representations and establish theoretical properties of cross-modal linear transport. The findings are evaluated across multiple dynamical systems, random seeds, model architectures, training objectives, alignment metrics, and held-out physical systems.

Critical strengths

1. A carefully controlled experimental study of an important hypothesis concerning representation learning.

The most significant aspect is the experimental design. Rather than merely measuring the similarity between existing models, this study presents a controlled environment in which the underlying physical state is known and the auxiliary supervision can be systematically varied. The comparison of encoders that are trained independently, have a common but uninformative target, and have different targets that still provide information about the physical system is especially valuable. It enables one to tell apart the effects of the shared training objectives from those of the information about the underlying world. Thus, it offers a useful empirical view of when representational convergence should be expected.

2. Validation based on extensive empirical evidence in addition to a single alignment metric.

The study looks at global similarity, neighborhood agreement, cross-modal retrieval, linear mapping, and representation transfer. The big difference between the supervised and unsupervised conditions is obtained in a number of different dynamical systems and with various random seeds. The addition of nonlinear mapping baselines, partial-target ablations, alternative training objectives, experiments involving different model scales, and the use of external physical systems enhances the empirical contribution. I specifically commend the authors for distinguishing between coarse representational similarity and reliable sample-level correspondence, as different metrics can lead to different conclusions regarding alignment.

Critical weaknesses

1. The theoretical description does not show the mechanism by which auxiliary supervision leads to alignment.

Although the H-subspace analysis is interesting, the main guarantees it provides seem mostly to describe a correspondence that already exists rather than to show when the training process brings about that correspondence. Theorem 7 links exactly matched linear statistics to unit canonical correlations and to the eigenspace of a round-trip operator. Yet the more strong interpretation that a common auxiliary reference leads to an aligned subspace depends on further assumptions, especially the factorization given in Equation (24). The paper does not prove that encoders trained independently necessarily meet this factorization as a result of the auxiliary objective. Furthermore, the experiments use near-unit canonical correlation thresholds, while the theorem is about exact unit correlations.

The importance of this distinction lies in the fact that the theoretical contribution is presented as an explanation of why alignment arises when the system is under supervision. The paper would be considerably strengthened if there were a more clear distinction made between an unconditional mathematical characterization, conditional theoretical predictions, and empirically observed behaviour.

2. The generalizations stated are limited by the evaluation setting and the cross-modal mapping protocol.

The benchmark makes use of simple synthetic physical systems together with structured captions that include explicit numerical observations. While this offers excellent experimental control, it does so at the expense of allowing any conclusions to be drawn regarding more complex representations that are learned from natural multimodal data. In a similar way, the comparisons between the pretrained models involve models that have been trained on unrelated corpora and have unequal decodability with respect to physical parameters across different modalities; as a result, they fail to demonstrate that large-scale pretraining generally cannot lead to alignment.

I am also worried regarding the interpretation of zero-shot cross-modal transfer. In Section 5.3 and Appendix E.1, cross-modal mappings are fitted on 70 per cent of the paired evaluation samples and then evaluated on the other 30 per cent. If the same procedure is applied to the held-out dynamical systems, the retrieved performance that is reported will show that the frozen encoders have been transferred, but it will not necessarily indicate that the previously learned cross-modal map has been transferred without having had paired examples from the new system. It is important that this distinction be made if the paper is to claim that the learned correspondence extends to unseen physical laws.

Questions for authors

1. Regarding cross-modal transfer to the held-out systems.

Are the linear maps employed for retrieval and stitching on the three-body and Coulomb systems trained only on the original training systems, or are they readjusted using paired examples from each of the held-out systems? If they are readjusted, could the authors provide results based on a fixed map that has been learned solely from the original systems? This would significantly clarify their claim of zero-shot transfer.

2. Theoretical guarantee and empirical approximation.

To what extent does minimizing the auxiliary prediction objective lead to the latent factorization assumed in equation (24)? Can the authors make clear whether the H-subspace theorem predicts the emergence of alignment or instead describes the geometry assuming that shared linear statistics already exist? It would also be helpful to determine how the exact unit-correlation results relate to the near-unit correlations that are obtained experimentally.

3. Separating out the increased physical information from alignment.

Table 11 indicates that auxiliary supervision boosts the decodability of physical parameters, raising the figure from 0.79 to 0.96 in the case of vision and from 0.85 to 0.95 for text. Could the authors compare models which have approximately the same level of physical-parameter decodability in order to find out whether auxiliary supervision affects cross-modal alignment beyond just increasing the amount of recoverable information about the physical state? A comparison of a similar kind involving phase information would be especially informative.

Reviewer_8gKDOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper takes as a starting point the platonic representation hypothesis, that AI models can learn similar internal representations when trained on different modalities (e.g. the book vs the movie). In a nice spirit, the authors developed simple toy “worlds”, e.g. a swinging pendulum with described both with simulated video frames and generated text of each frame, in order to test out the ideas. One model learns from the video, another from the text and the test is in the similarity of the resulting internal representations. Interestingly, both learn useful information about the underlying physics of the problem but their representations only seemed to align well when they were trained on the same prediction task (which should also have something to do with the physics). They do this exercise with simulations from 8 simple systems, with the conclusion holding that the internal representations become more similar when they are trained to accomplish a similar task, despite the difference in modalities. The authors also present a seemingly interesting mathematical characterization – though I did not confirm how well it mirrored the experiments performed and it seemed like too many linear assumptions.

Critical strengths

The authors create simulated world paradigm where they know the ground truth, can confirm properties of the underlying physics have been learned, control the exact level and type of noise, adjust the task accordingly, etc. It is a nice toy model setup that I imagine could continue to be useful and be developed further

The suggestion that alignment is enhanced when both models are trained with the same predictive task is interesting.

Critical weaknesses

The simulated worlds are a little simple, there really was not that much that could be learned from the data so alignment is maybe not that surprising, especially when the same task was given to both. I think it is a nice starting point but I dont think it is safe to extrapolate conclusions very much.

Questions for authors

I do not have any good questions for the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.