Improving geometric consistency of neuronal functional embeddings

Deep neural networks provide a powerful framework for learning functional embeddings of neurons from their responses to visual stimuli. These embeddings can reveal structure in neuronal function, but biological interpretations require their structure to be reproducible across model fits. We identify three sources of inconsistency in data-driven core-readout predictive models for mouse visual cortex. First, the functional basis of the embeddings spanned by the core is not orthonormal, leading to distorted Euclidean distances when computed naively. Second, learnt receptive-field locations vary across model fits. Third, the image encoders (cores) learn different functional subspaces across different model initializations. Here we show that the first issue can be resolved by whitening the functional basis. Whitening corrects the geometric distortion, improving the consistency of local neighborhoods and global structure across models trained with different random weight initializations. The other two issues are mitigated by introducing a new Retinotopic Factorized Linear readout (ReFL). ReFL improves the consistency of readout locations and makes core functional decomposition more similar. Combining whitening with the ReFL readout yields the most consistent functional embeddings. These results highlight the importance of both the orthonormality of the functional basis and the readout mechanism for the reproducibility of learned neuronal embeddings.