- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper revisits the claim that foundation models naturally learn tree-like representations. It shows that a low Gromov δ, a common measure of tree-likeness, can also arise in high-dimensional random data. The authors introduce matched random baselines and additional tests to distinguish clustering from deeper hierarchy. Experiments across vision and language models reveal substantial non-random structure at the class-centroid level, but limited evidence of hierarchy among superclass centers. These findings challenge the use of uncalibrated tree-likeness measures as standalone justification for hyperbolic representation learning.
Critical strengths
- The paper asks an interesting question: do low tree-likeness scores really mean that models have learned a hierarchical structure? It offers a useful perspective on how to interpret these representations.
- The experiments seem carefully designed, with several comparisons and checks to support the main claims. The authors also discuss where their approach may fall short.
- The paper covers a wide range of models and datasets and is clear about the limits of its findings. Releasing the code would also make it easier for others to check and build on the results.
Critical weaknesses
- The main message could be clearer. The abstract contains many results, numbers, and qualifications, making the key finding difficult to identify. I suggest emphasizing that low raw δ alone is insufficient evidence of hierarchy, and presenting the more conditional findings about superclass structure separately.
- Part of the evidence rests on limited data. The original claim is about individual image features, but this part of the analysis uses only 2 datasets with about 1,000 images each. More datasets would make this conclusion more convincing.
- The presentation can be difficult to follow at times. The paper introduces several terms with specific meanings, such as “genuine”, “census”, and “hub excess” , which take some effort to keep track of. Although the main figures summarize the findings, many supporting details are spread across the appendix, requiring frequent cross-referencing.
- There are some writing issues, e.g., "So, no hub hierarchy therefore means none among the tested superclass hubs" (lines 401–402) and "Of the two constructions, one reproduces the shape exactly. Of the two compared, ..."
Questions for authors
- Based on your results, what concrete check would you recommend before a researcher decides to use hyperbolic geometry?
- What is the computational cost of the full procedure? Is it practical for other researchers to apply it to their own models?