- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper discusses the issue of inconsistency across different random seeds when learning functional embeddings of neurons in the mouse visual cortex using deep predictive models. The authors employ a shared CNN to extract visual features, using the readout weight vector for each neuron as its functional embedding. The authors identify three sources of this inconsistency: the non-orthogonality of the feature basis formed by the core channels, which distorts Euclidean distances, the drift in learned receptive field locations across different seeds, and the emergence of distinct feature decompositions within the core across different seeds. To solve this, the authors propose estimating the channel covariance post-training and applying a whitening transformation, while simultaneously performing an inverse transformation on the readout vectors to ensure that the model's predictions remain unchanged. To address the latter two issues, the authors introduce ReFL readout, which generates spatial masks directly from a neuron's cortical coordinates via a shared linear mapping.
Critical strengths
There are many studies use neuron embeddings from various models to discuss cell types and functional organization. However, few have systematically examined whether these embeddings remain stable across different random seeds. In this paper, the authors provide a comprehensive evaluation of this issue, assessing both global and local consistency.
Critical weaknesses
-
I feel there is a flaw in the whitening approach: the embedding distance after whitening is, in essence, equivalent to comparing the similarity between the predictive responses of two neurons to a set of images, which is expected that this approach would prove more stable across different random seeds since it's primarily determined by the data itself. Given this, a more straightforward approach would be to use the neurons' responses to a collection of images whether derived from model predictions or from experimental recordings, directly as their embeddings. The paper doesn't compare its method against this direct baseline. So, it remains unclear whether whitening actually provides any added value.
-
This paper's analysis focuses largely on the consistency of embeddings across different random seeds. However, it doesn't examine whether greater consistency leads to better alignment with known cell types, morphologies, or transcriptomic labels. A model could be consistently wrong, which I think is an oversight, given that the core motivation of this paper is to provide a reliable foundation for biological conclusions.
Questions for authors
Could part of the functional differences between neurons be encoded in the shape and size of the ReFL masks rather than in the readout vectors?
Consistency is a prerequisite for biological validity and we added the response clustering baseline
Thank you for your thoughtful review and recognizing the importance of evaluating consistency across random seeds for biological reliability.
Below, we address the concerns:
W1 Embeddings indeed provide a data-driven way to compare neuronal functions, just like raw responses. Our contribution is not using embeddings for this purpose, as prior work has done so (Turishcheva et al.,2024; Nellen et al.,2025). Our contribution is recognizing and correcting for the non-orthonormal basis where these embedding lie – Fig. 1 illustrates why this is important. In fact, whitening corrects the flawed distance computations used in prior work and does not require additional data.
As for comparing embeddings with responses, we disagree that it is a direct baseline.
First, raw responses are dominated by stimulus drive (if there was any information in the receptive field) rather than real functional properties. For recorded responses, this means we can only compare neurons that see the same part of the image (have the same receptive fields), although neurons can have similar functions despite different receptive fields. The model separates receptive field position from function, allowing comparisons across all neurons. This is completely independent of whitening. Also, we lack ground-truth receptive fields and can only estimate them using the model.
Second, comparing ground-truth responses requires repeated presentations of the same stimuli sets, to cluster a set of presentations and measure consistency across repeats. However, raw responses do not account for pupil movements, which can change the input received by individual neurons even for the same stimulus.
Third, the response vectors (predicted or recorded) can be extremely high-dimensional - equal to the number of images, thousands for our train set. Computing distances in such high dimensions is problematic (Aggarwal, Hinneburg, Keim 2001). Functional embeddings compress neuronal responses, avoiding extremely high dimensions.
Despite these limitations, we performed the analysis using 9 repeats of 100 test images for 5 mice (2 lacked test sets; test images are shared across mice) and 2,553 neurons with receptive fields within 1 pixel of the population median. For each repeat (100 unique images), we clustered neuronal responses and model predictions and computed consistency metrics across repeats. Model predictions were more consistent, possibly because models account for pupil shifts, pupil dilation, and running speed, which modulate neuronal response amplitudes (Niell & Stryker, 2010).
To compare response and embedding consistency, we predicted 5,000 responses to unique training images from each model seed, clustered responses per seed and measured consistency across seeds. We used the same mice and neurons as before. Embeddings outperform predicted responses on all metrics (ARI, CKA, RSA, kNN), with ReFL outperforming Gaussian.
The plots are in the appendix W.
W2 We agree that high consistency does not guarantee biological validity. However, reproducibility across model initializations is a prerequisite for reliable biological interpretation. For example, deriving cell types from representations that cluster differently across random seeds would be unreliable. Our approach helps ensure that geometric analyses reflect stable representations rather than optimization stochasticity. We do not claim that consistency guarantees correctness. Rather, our claim is that biological conclusions drawn from models require both state-of-the-art predictive performance and stable representations, the former ensures that the model is not “consistently wrong”.
To show the link between consistency and biological validity, we analyzed embeddings before and after whitening on the marmoset retina dataset used in Nellen et al.(2025), with 4 ground-truth cell types. The model was trained on 235 cells, and consistency was evaluated on 145 labeled cells. Whitening improves all consistency metrics. Most notably, whitened embeddings show near-perfect ARI up to 4 clusters followed by a sharp drop at 5, while non-whitened embeddings show a smooth decay with much lower ARIs. Confusion matrices further support this result. Since the default model uses data-initialized receptive fields, adding retinotopy is not meaningful, so we cannot test ReFL on this dataset.
The plots are in the appendix V.
Q1 That's an interesting question. The mask could give some type of pooling across bigger receptive fields. We hypothesize that this may benefit cells exhibiting surround suppression (where a cell response is reduced when stimuli extends beyond the classical receptive field, such cells require integration over larger input size; Allman et al., 1985; Vaiceliunaite et al., 2013), but this hypothesis requires a separate study and currently unavailable experimental data.
We hope our reply addresses your concerns and helps to reconsider your assessment.