- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper analyses the result of an asymmetric alignment metric (cycle-KNN) for latent geometry across modalities. It selects ~50 unimodal models over point clouds, vision and language --- modalities chosen for their increasing abstract nature. They find a to (corresponding to a relative difference between 1% and 6%) in the direction of the more “abstract” modality.
This effect is linked to a density measure D of the latent space, and to the information bottleneck. The paper makes the following claims :
- C1 : The asymmetry of cycle-KNN consistently points towards language.
- C2 : This effect is scale-invariant.
- C3 : The underlying geometric cause is feature density.
- C4 : language geometry is most stable.
- C5 : WRH : cross-modal geometric convergence is directional rather than mutual, and points towards more compact representation and semantic coherence. The empirical target is language geometry.
Critical strengths
S1 : Novelty. To the best of my knowledge, the directionality of alignment has indeed never been investigated significantly in prior work. The paper is transparent on the fact that this metric is not novel, and that what is being studied here is its asymmetry.
S2 : Large empirical sweep. The paper reports an analysis that spans to parameters, across 29 language models, 22 vision and 7 point cloud models, with ablations and selection protocol (e.g. for layer selection). The reported values show that, though small, the asymmetry measure is significant.
S3 : Strong appendices. The appendices provide additional controls, ablations, and sensitivity. They also report unflattering results.
S4 : Claims C1 and C2 are well supported by the empirical findings.
S5 : Claim C4 is a good attempt at falsifying the assumption. I’d be careful with the framing, it currently reads as a confirmation, where it’s only a failed falsification. The fact that language geometry is stable doesn’t mean it’s the only possible one. See the weaknesses for a discussion on whether C4 is sufficiently supported though.
Critical weaknesses
The core of the paper, but also where it falls short, is the explanation and interpretation of the observed effect. Let’s take the WRH.
Directional convergence. The WRH mentions “convergence”. But convergence with respect to what ? This is very obscure, and seems to be convergence as modality become more “abstract” (cf “We assemble 50 independently trained models spanning three modalities along an abstraction hierarchy” l143 144). This notion is not defined, and an attempt at formalising it might not produce a partial order, let alone a pointed one where language could serve as a candidate top. Moreover, this makes the statement of directionality tautological. If there were no directionality in “modality” term (as opposed to, say, scale or training step), there would be no convergence. Moreover, an arbitrary directed graph between three points is transitive 6 out of 8 times; and the largest gap (PC - vision) does not involve language. The number of modalities has to be higher for this claim to be significant.
Geometric explanation. The authors correlate convergence with compactness, or density, as measured by , the average pairwise distance between the embeddings. Correlation (*) is not causation, and a trivial counter example shows that cannot be the geometric explanation behind the asymmetry in cycle-KNN. See counter-example below (**) disproving claim .
Based on the nearest-neighbor aspect of your metric, I’d say that more you could look at something like normalised local density : , where is standardized, or maybe hubness. There might be some link to neural collapse. This is just an intuition of course but I’m curious what the authors think about this discussion.
(*) note : the authors have to report the at the target layer. Currently, the only trace that the are smaller for language then vision then PC is a promise, not a number. You have to show a graph of the diff (or ratio) in D on one axis, and on the other axis, and show that there is indeed a correlation. I’d also like this graph for all pairs of layers, not just for the selected ones, to see if the correlation survives there.
(**) Let , be unitary, normalised embeddings, living in a plane of dimension , orthogonal to . Assume . Now take , and the normalized counterpart. The nearest neighbor geometry of is exactly the same as that of since embeddings will simply occupy a tighter dimensional sphere Therefore, . For large enough, we have .
Semantic coherence is not defined despite being key in the WRH. Maybe something like the proposed above might be a candidate definition ? At least this is how I understand it based on your paper. I might be mistaken.
Conclusion. Any attempt at better formalising the WRH with the current experiment fails to do anything else than restate . Claiming a new representation hypothesis based on such a small and unexplained gap in one metric is a massive leap…
Less critical points :
- How to interpret absolute values of cycle-KNN (S) ? It seems that is not even guaranteed to be . E.g., isolated or OOD points will not be in their neighbors’ neighborhood. This seems to create a confound that might bias the results and explain the reported asymmetry. The paper should report values for per modality.
- equation 2 is stated as a proposition. But, once again, there is no definition of “coherent”. Even though I get the intuitive idea you are trying to convey, this should be formalised for such a statement to even make sense, let alone be proven. This seems to be the core lack in this paper, as this would likely provide the missing geometric characterisation
- equation 3 : once again, is a measure about global geometry, and cycleKNN is about local geometry. (a) A uniform blob and (b) two tight blobs separated by a large gap will share the same but have massively different NN relationships.
- figure 2 : showing the confidence intervals would greatly help visualise the significance of the . This is minor since you give values in the caption.
- figure 4 : CKA is known to have massive confounds like dimensionality. The paper doesn’t report whether it used a normalised version of CKA or whether all dimensionalities are the same. Dimensionality alone could explain the differences.
- The number of points per modality is only 1024 pairs for vision-text. This is smaller or in the same order of magnitude than latent dimensionality. This cannot approximate the latent distribution enough to make stable measures of alignment.
- “WRH differs from PRH in providing an operationally testable endpoint—the most compressed, semantically coherent modality [in the sense of D]” (l400) Once again, this claim is false in the sense of “compressed”, and undefined in the sense of “semantically coherent”.
Questions for authors
Q1 : figure 4 and table 2 : isn’t the instability in point clouds fully explained by architectural choices of the models, or some other factor ? The bottom left corner seems rather stable to me.
Q2 : The number of models reported seems inconsistent. You report both (5, 16, 29) and (7, 22, 29) models for (PC, vision, language). This is likely only a typo. Could you precise which is the correct number ?
4 comments
(2/4) Responses to reviewer 8cW6
(3) Given that cycle-kNN is based on nearest neighbors, would normalized local density or hubness be a more appropriate geometric diagnostic? Could the phenomenon be related to Neural Collapse?
Local density and hubness would be useful complementary diagnostics, but they are not replacements for , because the quantities serve different purposes. Cycle-kNN is already a local statistic: it directly measures whether instance-level neighborhoods can be recovered through another representation space. In contrast, is used only as a global summary of modality-level compactness; the paper does not claim that alone determines the local cycle-kNN topology. The appendix also goes beyond global : among the eight synthetic generators, the folded-manifold experiment explicitly controls local density through fold amplitude, and the directional gap follows the induced neighborhood-density asymmetry for . Together with the row-permutation and -matched Gaussian controls—where compactness without semantic correspondence produces no stable gap—this supports the paper's stated joint account of compactness and semantically coherent local neighborhoods. A hubness analysis could further characterize which points dominate neighbor sets, but its absence does not invalidate the measured directional asymmetry.
The existing class-centroid-collapse control argues against simple within-class collapse as the explanation of our result. Replacing each representation with its class centroids maximizes within-class compactness, yet it produces a qualitatively different signature: both absolute cycle-kNN scores decrease substantially while is inflated to approximately , about an order of magnitude larger than the real effects. The observed asymmetry instead occurs while absolute cross-modal alignment remains high.
(4) Semantic coherence is not defined despite being key in the WRH.
As with “convergence,” “semantic coherence” is not introduced by WRH as a new undefined primitive. We follow the standard PRH-style setting in which semantically corresponding samples are expected to preserve meaningful neighborhood relations across independently trained representation spaces. In our paper, this notion is also given an explicit and independently measurable operational definition in the appendix: , the fraction of a sample's kNN neighbors that share its ground-truth category. Under this definition, language reaches , whereas vision is approximately – at comparable . Thus, the reviewer's suggestion that might define semantic coherence conflates two quantities that the paper deliberately separates: measures global compactness, whereas measures the semantic organization of local neighborhoods. This distinction is empirically necessary: row permutation preserves exactly while destroying semantic correspondence, and -matched random representations likewise fail to produce a stable directional gap.
Equation 2 is a hypothesis/prediction, not a theorem, and it was not intended as a logical derivation of semantic coherence from . The empirical quantities on both sides are independently operationalized: by directional recovery, compactness by , and semantic coherence by . We will relabel the statement and spell out these definitions, but the absence of a theorem proof is not a defect for a falsifiable empirical hypothesis.
(5) Is proposing WRH from a small and insufficiently explained effect measured by a single metric too large a leap?
Calling WRH a “hypothesis” is already the appropriate epistemic framing; hypotheses are proposed from systematic, falsifiable evidence rather than only after becoming laws. The evidence is not one small number: it spans 50 models, three pair types, model-pair sign matrices, k-sensitivity, seven layer protocols, semantic-destruction controls, granularity controls, and independent Information Imbalance agreement on V–L. Moreover, under the aggregation-aware permutation calibration of Gröger et al. (2026), the V→L gap remains , with the predicted sign in 80.7% of 638 model pairs (), while the PC→L gap remains , with the predicted sign in all 203 pairs (). Thus, the directional pattern is not an artifact of reporting uncalibrated maxima over layer pairs. We will distinguish the measured effect from the proposed mechanism more sharply, but the breadth and consistency of the pattern justify proposing WRH.
(3/4) Responses to reviewer 8cW6
(6) What are the values for each modality, and could variation in self-cycle recovery confound the reported asymmetry?
is not guaranteed to equal 1. Because the query itself is excluded from its kNN list, measures the fraction of samples with at least one reciprocal k-nearest neighbor in . Anti-hubs—samples that never occur in another sample's neighborhood—therefore reduce the self-score. More generally, closes its return cycle in the source graph and is upper-bounded by , the fraction of samples with non-zero k-occurrence in that graph. Since the two directions in close their cycles in different source graphs, modality differences in neighborhood reciprocity could in principle contribute to the raw directional gap.
We quantified this effect explicitly. At the selected layers, the self-scores are all high— to , compared with a chance level of —and anti-hubs account for only – of samples:
| Modality | , selected layers | , all-layer mean | Anti-hubs | |
|---|---|---|---|---|
| Point cloud | ||||
| Vision | ||||
| Language |
The selected-layer self-score gap favors the reported direction by for PC→L, and for V→L. We therefore recomputed under three controls: (i) both directions evaluated on the identical set of anchors with non-zero k-occurrence in both graphs; (ii) a stricter common-anchor set requiring a reciprocal neighbor in both graphs; and (iii) each directional score normalized by its source-graph ceiling . We also regressed on the self-score gap and evaluated the intercept at equal reciprocity.
| Direction | Raw | Matched reachable anchors | Ceiling-normalized | Matched reciprocal anchors | Regression intercept |
|---|---|---|---|---|---|
| PC→L | () | ||||
| PC→V | () | ||||
| V→L | () |
All controlled gaps remain positive and statistically significant (Wilcoxon signed-rank ). Thus, differences in neighborhood reciprocity do not create the directional asymmetry: the effect persists when both directions use exactly the same anchors and when the source-graph ceilings are normalized.
(7) Figure 4 does not specify the CKA variant or representation dimensionalities, so dimensionality could explain the reported differences.
The CKA computation in Figure 4 follows the same standard symmetric CKA implementation and representation-comparison protocol used by PRH. It is not the supplementary directional CKA analyzed separately in our appendix.
The stronger claim that dimensionality alone explains Figure 4 is not supported by the results. Figure 4's ordering is reproduced by mutual kNN, which compares the overlap of neighborhood identities and does not require the two representations to have matched feature dimensions: Language Vision Point Cloud . This is the same ordering produced by CKA: Language Vision Point Cloud . Thus, representation dimensionality may affect the absolute magnitude of CKA and should be documented, but it cannot by itself account for an ordering independently recovered by a neighborhood-based metric. We will clarify the CKA implementation and feature dimensions and present mutual kNN as the dimension-mismatch-robust corroboration rather than relying on CKA alone.
(4/4) Responses to reviewer 8cW6
(8) With only paired samples for vision–language—comparable to or smaller than the latent dimensionality—are the alignment estimates stable?
is not an arbitrary sample size: it directly follows the cross-modal evaluation protocol of PRH, whose Appendix C.2 computes nearest-neighbor alignment over 1,024 WIT image–text pairs. Using the same dataset size preserves direct comparability with the work on which our analysis builds. Moreover, the observation that is comparable to the latent dimensionality does not by itself imply that cycle-kNN is unstable. Cycle-kNN is a rank- and neighborhood-based statistic; it does not estimate a full -dimensional density or covariance matrix and therefore does not require as an identifiability condition. Empirically, the direction is evaluated across 638 vision–language model pairs, with 530/638 pairs (83.1%) showing the predicted sign, and remains consistent across , layers, and layer-selection protocols. Subsampling is still a useful finite-sample robustness check, but its absence is not evidence that the reported estimate is unstable.
Q1: Could the apparent instability of point-cloud representations be explained by model architecture or another factor rather than modality?
Architectural diversity may contribute to the lower point-cloud consensus, but it is not an alternative explanation that nullifies a modality-level result: the evaluated modality is instantiated precisely through its current architectures and objectives. Moreover, language and vision pools are also architecturally diverse, yet the same two metrics give the ordered consensus Language > Vision > Point Cloud.
Q2: The manuscript inconsistently reports and point-cloud, vision, and language models. Which count is correct?
The correct counts are 5 point-cloud models, 16 vision models, and 29 language models, for 50 models in total.
(1/4) Responses to reviewer 8cW6
We thank the reviewer for their careful evaluation of our manuscript and for recognizing the novelty of our systematic study of alignment directionality, the breadth of our empirical investigation across model families and scales, the strength and transparency of the appendices and sensitivity analyses, the empirical support for Claims C1 and C2, and the value of C4 as an attempt to falsify the proposed interpretation. We address the reviewer's concerns below:
We DO NOT introduce a NEW definition of “convergence” in this paper. In fact, the meaning of “convergence” is clear: the definition of “convergence” in this paper follows PRH [1] — a positive association between competence/scale and convergence degree observed at the cross-sectional level:
PRH defines convergence as "alignment toward Z increases toward a convergent point as a function of scale" (Sec. 1 [1]), where the axis is competence/scale, not training time. All of PRH's evidence is cross-sectional: 78 vision models aligned by VTAB competence (Fig. 2 [1]), 20 LLMs aligned by language modeling score (Fig. 3 [1]) — no training trajectories, no movement through representation space. Our evidence is structurally identical: 58 independently trained models across 3 modalities, analyzed as a function of model capability rather than training dynamics.
In short, the definition and empirical evidence of "convergence" are already established by PRH's extensive experiments; the focus of this paper is the modal directionality of this "convergence". Our contribution is to exploit cycle-kNN's inherent asymmetry: Δcyc(A,B)=cycle(A→B)−cycle(B→A) reveals cross-modal directional asymmetry that PRH's symmetric metrics cannot capture.
The counterexample establishes that the scalar D is not a sufficient statistic for cycle-kNN; it does not establish that feature density is irrelevant to the asymmetry in semantically organized learned representations. Claim C3 concerns a geometric association in this restricted empirical setting, not a universal identity in which D uniquely determines Δ for arbitrary point configurations.
We separates these claims through several interventions: row permutation preserves D while destroying semantic correspondence, D-matched Gaussians reproduce compactness without reproducing the directional gap, and centroid collapse and random projection produce qualitatively different behavior from the real representations. These controls show that compactness alone is insufficient, while the observed direction is associated with compactness only when accompanied by semantically organized neighborhood structure. We will therefore revise any wording that presents the global scalar D as a unique causal explanation and state the supported conclusion more precisely: D is a modality-level summary of compactness and one component of the proposed mechanism, whereas the directional statistic also depends on local semantic topology that D alone does not encode.
The paper does not claim that a space serves as a more effective reference simply because it is more compact. Rather, it claims that the more compact and semantically coherent space serves as the effective reference (see Section 5).