Mutual neighbor similarity: reliable and rich representational alignment measurements

Every representational similarity measure carries strengths and weaknesses. Mutual kNN (m-kNN) style approaches can report local alignment, but saturate at larger scales. In addition, these ordinal approaches, including kendall's tau, and spearman rank correlation, discard all information-rich geometric details in favor of coarse rank-based descriptions, incurring various geometric failure modes. On the other end, many geometric approaches such as centered kernel alignment (CKA) provide global summary metrics but cannot describe alignment at local or intermediate scales. Furthermore, many of these methods can be easily confounded by factors such as embedding dimension and must be null-calibrated via expensive Monte-Carlo simulations to account for geometric priors and deliver robust estimates of alignment with meaningful baselines. To unify many of these strengths in a single measure while mitigating weaknesses, we introduce mutual neighbor similarity (MNS), a hybrid geometric and ordinal comparison of corresponding embedding space neighborhoods. With an analytically derived and simple to compute null expectation, MNS enables robust comparisons across all scales without saturation at either extreme, while accounting for the structure of point cloud geometries. We construct MNS, describe its invariances, derive its robust form, and run experiments on synthetic data demonstrating its ability to overcome various failure modes of existing approaches while introducing entirely new descriptive abilities. Finally, we present results on large scale real world data suggesting the superiority of our approach for transfer learning tasks.