Neural compatibility: functional similarity with learning

An ongoing problem for computer science and neuroscience is to chart the similarities and differences between how both artificial and biological neural systems represent data and process information. Representational similarity and representational alignment methods seek to quantify and close these gaps. A recent idea is to quantify whether representations from one model are usable by another model by “stitching” activity from one into the other. However, a fundamental property of neural systems is that they learn and adapt to new inputs. We therefore ask what makes two neural systems "compatible" in the sense that one model can efficiently **learn to use** representations from the other. We illustrate with simple examples how "correlation", "stitchability", and "compatibility" between models are in principle distinct ways of conceptualizing similarity between models and brains. Compatibility provides more evidence that representations are most similar across corresponding fractional depths. Despite the shown in-principle distinction, we find empirically that among pretrained image classification models, stitchability and compatibility are strongly correlated. We conclude that 'neural compatibility' is conceptually distinct from other notions of representational alignment, but identifying in-practice impactful differences requires further investigation.