Comparing and Integrating Different Notions of Representational Correspondence in Neural Systems

Representational similarity metrics emphasize different properties, so their agreement with one scientific criterion need not extend to another. We compare seven widely used metrics using hypothesis-driven tests of within-group correspondence and between-group separation in artificial and biological neural systems. The benchmarks include 35 vision models evaluated on ImageNet, Ecoset, CIFAR10, and CIFAR100, human fMRI responses from the Natural Scenes Dataset, and mouse Neuropixels recordings from the Allen Brain Observatory. Geometry- and tuning-sensitive metrics generally recover stronger model-family and anatomical-group separation than more flexible mappings, but the leading metric changes with dataset and separation criterion. In a population of independently trained seeds, the same metrics separate training objectives beyond seed-to-seed variation, while analyses at six network depths show that intermediate-layer rankings differ from final-layer rankings. Integrating the metrics with Similarity Network Fusion improves most group-separation scores and yields data-driven typologies in which the training paradigm can cut across architecture. These results demonstrate metrics' different abilities in separating neural systems, while also showing that two different neural systems could still share great similarity in some representational aspects.