- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper revisits what converges under the Platonic Representation Hypothesis. The authors argue that prior local–global comparisons also change what is compared: local measures typically compare neighbor identities, whereas global measures often involve distances or correlations. This makes it difficult to determine whether the observed differences arise from the scale or the object of comparison.
The authors introduce a 2×2 framework, using nearest-neighbor overlap and minimum spanning tree edge overlap to measure local relations and global spanning structure, respectively. They then introduce distance-agreement requirements while keeping these relations fixed. Image–text and video–text experiments show robust relational alignment at both scales, whereas stricter distance agreement weakens alignment and its association with increasing model size.
Critical strengths
-
The paper offers a clear and useful refinement of prior conclusions. Separating the scale of comparison from what is compared helps avoid attributing differences between metrics directly to local versus global representational differences. Although the broader topic has been extensively studied, this framework provides a valuable addition to our understanding of representational convergence.
-
The experimental and statistical procedures are carefully designed. The measures share consistent data, preprocessing, and calibration protocols. Repeating the maximum-over-layer-pairs selection during permutation controls score inflation from layer selection. The authors also examine rank correlations between model size and alignment and preserve dependencies arising from model reuse in their tests. The conclusions therefore go beyond the inevitable score decrease caused by distance weighting.
-
The main observations receive support from several complementary analyses. The image–text experiments cover 204 model pairs, with additional checks using alternative distance normalizations, cosine similarity, a locally covariance-based metric, and a video–text setting. These results strengthen the credibility of the observed pattern.
Critical weaknesses
-
It remains unclear whether global skeleton alignment provides evidence beyond local neighborhood agreement. A minimum spanning tree is globally constrained to connect all samples, but its edge overlap may still be dominated by short, local edges. The current results do not sufficiently distinguish these sources. Showing that tree edges outside the local neighborhood relations also exhibit an alignment trend would provide stronger support for the central claim that convergence extends beyond local structure.
-
Metric stability and statistical power under stringent distance requirements need clarification. The rank-correlation analysis rules out a simple uniform-rescaling explanation, but very strict distance weighting may leave a small number of edges dominating the score. Moreover, the neighborhood and spanning-tree measures involve different numbers of relations, so differences in significance rates may not exclusively reflect differences between local and global geometry. The paper should distinguish geometric disagreement from the effects of sparse effective contributions more clearly.
-
The interpretation of scaling trends and generalization across settings needs qualification. Model size does not fully represent capability, and training differences remain potential confounders. For example, the VideoMAE scale series mixes pretrained and fine-tuned checkpoints, while some video representations come from single-frame image models. These settings remain informative, but should not be interpreted as isolating model-scale effects or providing comprehensive validation of temporal representations.
Questions for authors
-
How much of the minimum spanning tree alignment comes from local neighborhood edges? Can existing results or saved edge sets establish whether relations beyond local neighborhoods also contribute to the size-dependent alignment trend? If this has not been analyzed, please clarify the boundary of the claim.
-
Are the metrics sufficiently stable under stringent distance requirements? Are there existing diagnostics on the number of effectively contributing edges, the prevalence of zero or tied scores, and whether these factors affect the size–alignment correlations or the local–global significance comparison?
-
How can model-scale effects be distinguished from training differences? In particular, VideoMAE checkpoints at different scales have different training histories. Which cross-modal conclusions remain supported after accounting for such differences?
4 comments
Response to Reviewer 4vUK, continued 2
3. Theoretical interpretation and controlled verification (Appendices E.4 and E.6).
Decreasing necessarily reduces the weights of edges with nonzero discrepancies, but does not necessarily weaken the ordering of raw alignment scores across model sizes. To formalize this distinction, we introduce cumulative agreement , defined as the normalized number of shared edges whose distance discrepancies are at most .
Proposition 3 establishes that if for every , then for every , where is the raw distance-aware alignment score. In other words, if geometric agreement improves with model capacity such that cumulative agreement increases pointwise, then the raw alignment score is monotonically nondecreasing with model capacity for every fixed .
Using a fixed DINOv2 ViT-Base model and 1,024 images, we independently applied Gaussian perturbations in feature space and image space, approaching the same clean reference as noise decreased (Figures 10–11). In both experiments, preserves its pointwise ordering across noise levels for both mKNN and MST relations, with no violations across the empirical discrepancy breakpoints. Consistent with Proposition 3, raw distance-aware alignment remains monotonic as noise decreases at all five evaluated tolerances, including .
These results show that strict weighting does not inherently disrupt alignment ordering when representations progressively approach a common reference geometry.
4. Local-global significance comparison (Appendix E.5).
Differences in significance rates cannot be attributed exclusively to differences between local and global geometric agreement. At , BH-significant pair counts are 183/204 for mKNN and 82/204 for MST.
The two measures involve different numbers of relations and substantially different effective contributions, potentially affecting statistical power. Since these effects have not been independently quantified, we do not interpret the significance gap alone as evidence of weaker global geometric convergence.
We revised Appendices E.4–E.6 to clarify the theoretical distinction, report stability diagnostics, and identify the limitations of interpreting alignment under extreme distance tolerances.
W3/Q3. Scaling Trends and Generalization
The structure-geometry pattern persists within separate pretrained (PT) and fine-tuned (FT) VideoMAE scale series. Since the original series mixed pretrained Base/Large and fine-tuned Huge checkpoints, we conducted an additional 3×2 VideoMAE experiment covering Base, Large, and Huge at both training stages. Each checkpoint was evaluated against the same 13 language models (78 pairs), using aggregation-aware calibration with 500 permutations.
Mean calibrated relational alignment across 13 language models:
| Measure | Stage | Base | Large | Huge |
|---|---|---|---|---|
| mKNN | PT | 0.0272 | 0.0441 | 0.0507 |
| FT | 0.0756 | 0.1065 | 0.1343 | |
| skeleton overlap | PT | 0.0155 | 0.0259 | 0.0290 |
| FT | 0.0358 | 0.0532 | 0.0706 |
For both measures and both training stages, calibrated alignment increases strictly from Base to Large to Huge for all 13 individual language models, with all 78 pairs significant after BH-FDR correction.
Under stricter distance agreement, these trends become less consistent. At , strict increases remain for 4/13 (mKNN) and 3/13 (MST overlap) language models in PT, versus 13/13 and 8/13 in FT. At , only 35/78 distance-aware mKNN pairs and 0/78 distance-aware MST overlap pairs remain significant, although differences in the number of relations may affect statistical power.
These results show that the structure-geometry pattern persists within separate PT and FT scale series, rather than depending solely on their mixture. Nevertheless, other training differences remain uncontrolled, so we interpret these findings as model-size associations rather than causal scaling effects. Model size should not be equated with capability.
Finally, this additional analysis uses VideoMAE exclusively, while DINOv2 and CLIP serve as single-frame baselines in the original evaluation. Our conclusions therefore support cross-modal relational alignment, but do not establish comprehensive convergence of temporal representations.
We have incorporated these findings in Section 4.5 and Appendix H.3.
Thank you for your detailed rebuttal. As someone who also reviews and responds to rebuttals, I found your response helpful in resolving my questions. I appreciate your efforts in addressing the concerns. Thank you again for your rebuttal.
We are glad that our rebuttal has addressed your concerns. Your questions helped us clarify the interpretation of MST overlap and stringent distance tolerances. The additional VideoMAE experiments allowed us to better distinguish model-size trends from training-stage differences. We sincerely appreciate your thoughtful feedback, which has greatly helped us improve our paper.
Response to Reviewer 4vUK, continued
W1/Q1. Contribution of Local Edges to MST Alignment
In our experiments, all MST edges are contained in the corresponding symmetrized 10-NN graphs (100%). Consequently, no MST edges outside these neighborhoods contribute to the observed overlap or its capacity-dependent trends. We therefore do not claim independent convergence of non-local relations or separately establish between-group convergence.
The local-global distinction in our framework concerns the constraints used to select relations, rather than the lengths of individual edges. While mKNN selects neighbors independently for each sample, MST selects edges under global connectivity and acyclicity constraints. Even locally supported edges can collectively connect the entire representation through multi-edge paths. Therefore, global connectivity does not require individually long-range edges.
Importantly, MST alignment is not mathematically implied by mKNN agreement. To clarify their complementary properties and limitations, we added four one-dimensional representations of six labeled samples in Appendix C.5:
These examples show that neither measure dominates the other. MST overlap can miss changes in local neighborhoods (X1 vs. X2) and penalize within-group rearrangements that preserve local neighborhood structure (X1 vs. X3). Conversely, MST overlap can detect changes in the relative ordering of local clusters that mKNN misses (X1 vs. X4).
Moreover, setting k=5 makes the neighborhood graphs complete and identical in all comparisons, yielding 5-mKNN =1. Nevertheless, MST overlap remains 4/5 for X1 vs. X3 and 3/5 for X1 vs. X4. Thus, even perfect mKNN agreement and complete MST inclusion in the kNN graphs do not guarantee perfect MST overlap. Consequently, convergence in mKNN alone does not establish convergence in MST overlap.
This distinction motivates evaluating MST alignment separately, while not establishing its empirical independence from local neighborhood agreement.
Our main contribution is the controlled 2×2 framework, rather than evidence of independent non-local convergence. We separate relation-selection principles (local neighborhoods versus global spanning constraints) from comparison targets (relational identities versus metric distances). Our findings therefore extend the analysis of relational convergence to globally constrained spanning relations while identifying the distinction between relational structure and metric agreement as the central result.
We revised Sections 3.2 and 4.2 and Appendices C.4–C.5 to clarify this distinction, illustrate the complementary properties of mKNN and MST overlap, and explicitly delimit the scope of our conclusions.
W2/Q2. Metric Stability and Statistical Power Under Stringent Distance Weighting
Stringent distance weighting concentrates alignment scores on a small number of edges, particularly for MST overlap. To quantify this effect, we added diagnostics of effective edge contributions, zero-valued scores, and raw versus calibrated capacity correlations in Appendix E.5.
1. Effective edge contributions and zero-valued scores (Appendix E.5).
The effective number of contributing edges decreases substantially as τ becomes smaller. We quantify this using Neff=(∑ewτ(e))2/∑ewτ(e)2, which measures the concentration of edge weights.
Across 204 vision–language model pairs, median Neff for mKNN/MST decreases from 141.6/8.5 at τ=10−2, to 19.1/2.0 at τ=10−3, and 3.8/1.0 at τ=10−4. At the strictest tolerance, all saved raw scores remain nonzero, but null-threshold calibration produces 18 zero-valued mKNN scores and 69 zero-valued MST scores, introducing ties that may affect rank correlations.
2. Effects on capacity-dependent correlations (Appendix E.5).
Capacity correlations weaken even before calibration, showing that calibration-induced ties and uniform score rescaling cannot fully explain the observed changes. From τ=1 to 10−4, mean within-family Spearman correlations of raw scores decrease:
However, sparse effective contributions can also influence raw scores. Therefore, these diagnostics do not establish geometric disagreement as the sole cause of weakening capacity trends.