acceptodds
Under review as a conference paper at ICLR 2027

Is There a Platonic Tree? Calibrating Latent Hyperbolicity in Foundation Models

Abstract

Hyperbolic representation learning rests on a premise we call latent hyperbolicity: standard models are already tree-like because their class geometry scores a low Gromov , a measure of tree-likeness that is zero for a tree. That low value is also what a random cloud of the same dimension and spread scores. We introduce a calibrated framework that reads tree-likeness as its excess over such structureless clouds, together with a second test, whose power we measure, designed to distinguish hierarchical organization from simple clustering. We apply it to 12 vision backbones, 6 datasets and 15 text models. On image features, where the premise is read, the calibrated reading is indistinguishable from a random cloud in most model-dataset cells and small elsewhere. Class centroids do carry structure that a random cloud of the same shape does not, in 44 of 72 cells, but so does a star of clusters. The second test finds no detectable hierarchy among superclasses in the 9 of 12 backbones where it detects a planted one. What it finds in 4 of them is how each cluster is oriented, which disappears when clusters are rotated at random. MERU, trained in hyperbolic space, shows the same structure as its Euclidean twin in a space that stays nearly flat. In text, the result depends on the model's training recipe and size. Across models, the trees agree on which classes group together well above chance, though less than two resampled versions of the same model, and not on distances; the self-supervised models organize classes by direction rather than by distance, which a comparison by distance misses. Read correctly, foundation models organize classes into clusters that they partly share and show no detectable hierarchy among the superclasses where the test has power. Their raw tree-likeness is not evidence for hyperbolic geometry.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Latent hyperbolicity, Gromov hyperbolicity, hyperbolic representation learning, foundation models, Platonic Representation Hypothesis, class geometry

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jUuROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits the claim that foundation models naturally learn tree-like representations. It shows that a low Gromov δ, a common measure of tree-likeness, can also arise in high-dimensional random data. The authors introduce matched random baselines and additional tests to distinguish clustering from deeper hierarchy. Experiments across vision and language models reveal substantial non-random structure at the class-centroid level, but limited evidence of hierarchy among superclass centers. These findings challenge the use of uncalibrated tree-likeness measures as standalone justification for hyperbolic representation learning.

Critical strengths
  1. The paper asks an interesting question: do low tree-likeness scores really mean that models have learned a hierarchical structure? It offers a useful perspective on how to interpret these representations.
  2. The experiments seem carefully designed, with several comparisons and checks to support the main claims. The authors also discuss where their approach may fall short.
  3. The paper covers a wide range of models and datasets and is clear about the limits of its findings. Releasing the code would also make it easier for others to check and build on the results.
Critical weaknesses
  1. The main message could be clearer. The abstract contains many results, numbers, and qualifications, making the key finding difficult to identify. I suggest emphasizing that low raw δ alone is insufficient evidence of hierarchy, and presenting the more conditional findings about superclass structure separately.
  2. Part of the evidence rests on limited data. The original claim is about individual image features, but this part of the analysis uses only 2 datasets with about 1,000 images each. More datasets would make this conclusion more convincing.
  3. The presentation can be difficult to follow at times. The paper introduces several terms with specific meanings, such as “genuine”, “census”, and “hub excess” , which take some effort to keep track of. Although the main figures summarize the findings, many supporting details are spread across the appendix, requiring frequent cross-referencing.
  4. There are some writing issues, e.g., "So, no hub hierarchy therefore means none among the tested superclass hubs" (lines 401–402) and "Of the two constructions, one reproduces the shape exactly. Of the two compared, ..."
Questions for authors
  1. Based on your results, what concrete check would you recommend before a researcher decides to use hyperbolic geometry?
  2. What is the computational cost of the full procedure? Is it practical for other researchers to apply it to their own models?
Reviewer_ACb2OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

A common argument in hyperbolic representation learning is that the features of standard models have a low Gromov δ\delta and must therefore be close to a tree. The authors test this by comparing δ\delta with random point clouds of the same covariance and they score as low. After this calibration, the per-image features mostly look like the random reference, and only one of the published values of Khrulkov et al. stays significant. Class centroids carry structure, however a star of clusters with no hierarchy gives the same signal. A second test looks for hierarchy among 'superclass' centres and finds none in the backbones where it can detect a planted one. In a few backbones it picks up how each cluster is oriented relative to its superclass centre. Finally, class trees from different models agree well on groupings and less well on distances.

Critical strengths
  1. The main point is strong. A low δ\delta is usually compared with the value of an ideal tree, when it should be compared with what a structureless cloud of the same shape gives. In network science, comparing δ\delta with random graphs is an old habit, and it's good to see it used for embeddings. The null has the same covariance as the data, so it controls for dimension and anisotropy. Gaussian clouds already look like hyperbolic discs under δ\delta. The authors also reproduce the published values before calibrating them, and I would accept for the calibration and the reproduction alone.

  2. The authors also look into what the structure left after calibration is made of. I have doubts about the second test (see below), but I agree with the question it asks. They report the test's power and how the result changes with the grouping, and they say that one pre-set expectation was not met.

Critical weaknesses
  1. I am less convinced by the conclusion that there is no hierarchy among superclasses. The test is shown to detect one planted three-level hierarchy, and I could not find how strong its branches are. A planted two-level tree at the real spread is almost never detected. The result supports "no hierarchy as strong as the planted one", which is how §5.3 puts it. I also wonder how far the orientation effect is from hierarchy. Each cluster being oriented relative to its superclass centre looks to me like a trace of the superclass in the shape of each class. The authors note that a hierarchy below the grouping would be removed with the orientations and counted as alignment, and they leave the form of the alignment open (question 1).

  2. I am not sure δ\delta is the right tool for the hierarchy question. High-dimensional clouds concentrate toward equidistance, which is a degenerate star, as the authors point out. Much of what δ\delta sees in these spaces may be that effect, in which case even a calibrated δ\delta is weak evidence for or against hierarchy. On the sample level, "small elsewhere" in the abstract undersells the cells that do differ, for example DINOv2-G on CIFAR-100, which is well below its null. The authors' own sentence, "what fails is the evidence the premise cites, not the possibility of a hierarchy", seems to me the right summary, and I would like the abstract and conclusion to say that.

Questions for authors
  1. How do you interpret the orientation result? Do you see it as a form of hierarchy that the hub test is not built to detect, or as something else?

  2. Do you think a calibrated δ\delta is the right tool to look for hierarchy here, given the dimension confound? Where would you expect it to give a positive result, for example within superclasses?

  3. Why does the Poincaré metric still help the contrastive VLMs a little on nearest-centroid and few-shot tasks, given that the features don't look hierarchical?

  4. The title asks whether there is a Platonic tree. What is your answer?

  5. How were the branch strengths of the planted three-level hierarchy chosen?

Reviewer_zthzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper argues that hyperbolicity is not a sufficient metric for understanding the tree-likeness of representation spaces because a random cloud or a star can have the same or lower hyperbolicity compared to a hierarchy. They introduce a new metric that measures how different a representation is from a random cloud and a star. They measure these two metrics across multiple datasets, models, and backbones and then compare them to the hyperbolicity values and then ask the question whether the tree-likeliness justifies using hyperbolic space for those cases. The paper concludes that foundation model representations contain real non-random semantic structure, but the evidence mostly supports clustering/shared grouping rather than a detectable hierarchy among superclass centers.

Critical strengths

The paper points to an interesting problem with the hyperbolicity metric and tries to complement it with newer metrics. The motivation and the gap that such metrics are needed to address are interesting, and the fact that lower hyperbolicity does not always ensure meaningful latent tree structure. They progressively construct harder alternatives: matched random clouds test whether raw tree-likeness exceeds high-dimensional effects; matched stars test whether clustering suffices; cluster rotations distinguish hub organization from cluster orientation; and planted hierarchies test whether a negative result could simply reflect an insensitive test.

Critical weaknesses

The paper's language is very strange and hard to follow; it feels like a blog post at times and, at other times, uses very unintuitive, vague, and not usually used for research paper words. The reader needs to make a lot of effort to understand the main message of the paper: If you read the abstract, there is a very short introduction, and most of it is about the results, that is hard to interpret; however, no real conclusion can be derived from those detailed results that are presented. I would suggest that the authors substantially change the abstract and the introduction. The name of the paper suggests that the focus is on foundation models; however, it's hard to see that consistency throughout the paper. Another main comment about the paper is that hyperbolic models have been shown to work for certain tasks under a certain hierarchy. However, if you take the model out of context and test it against another new hierarchy, the representations probably do not conform to that new hierarchy because they were embedded in the space with a different one. The point of using hyperbolic learning is to have more control over the representation space of what superclasses you want to contain what classes and sometimes even the hierarchy is not between class-superclass but could be like the hycoclip/meru loss. So the semantic hierarchy matters and not just the hierarchical shape in the representation space.

Questions for authors

How robust are the main conclusions across the different hyperparameters? Can you show what your test would mean for hyperbolic models that do not encode the hierarchy as class-superclass? When measuring tree-likeliness for a star representation, do you consider the semantic tree as well, or do you just look for the tree structure but don't care whether the tree reconstructed from that star makes sense semantically?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.