acceptodds
Under review as a conference paper at ICLR 2027

Partitioning Neural Co-Variability

Abstract

Individual sensory neurons signal precise stimulus information despite high trial-to-trial variability.. Much of this variability is overdispersed — spike-count variance exceeds the mean — and can be partitioned into a stimulus-driven component and a gain that fluctuates across repeated presentations of the same stimulus. While the structure of neural activity, i.e., the stimulus-driven portion of the spike rate, has been explored widely in modeling population-level activity, the role of how modulatory gain factors are shared between neurons or changes over time has not been as thoroughly explored. Recovering the latent structure of the gains would offer a way to separate the stimulus drive from the shared gain that modulates a population over time. Most current overdispersion models thus treat each neuron's gain as independent across neurons and time. To capture the network-level statistics of the population, we thus present the Poisson matrix-normal latent variable model (PMNLV), which extends single-neuron overdispersion to neural populations and across trials, allowing the stimulus drive and the shared population gain to be estimated jointly rather than assumed independent. The model places a matrix-normal prior on the latent gain, and binned spike counts are drawn as Poisson from a rate that sums a per-neuron tuning term and the latent gain passed through a soft-rectifying link; its only structural assumption is a Kronecker factorization of the gain covariance into a neuron factor and a time factor. We derive a variational expectation-maximization algorithm that recovers both factors together with the tuning curves. On synthetic data the algorithm recovers the tuning curves and the covariance factors, and remains accurate at firing rates as low as 0.3 Hz. Compared against related population and overdispersion methods, PMNLV compares favorably in recovering the underlying covariance structure. Applied to Neuropixels recordings from mouse visual cortex, we find that the population's activity covariance and its estimated gain covariance are spectrally misaligned, indicating that the drivers of gain covariance may operate separately from the activity covariance. The PMNLV framework applies to any simultaneously recorded population in which structured gain covariance is of scientific interest, providing a tool for basic neuroscience that resolves population-level structure without compromising single-neuron statistics.

Primary area:
applications to neuroscience & cognitive science
Keywords:
neuroscience, latent variable, modeling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_u3xCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces PMNLV, a Poisson matrix-normal latent variable model for population-level overdispersion. Binned spike counts are Poisson with a rate equal to a per-neuron tuning term plus a latent gain passed through a softplus link, where the gain has a matrix-normal prior with Kronecker-factored covariance V⊗UV\otimes U (neuron factor UU, time factor VV). Because the marginal likelihood is intractable, the authors derive a variational EM procedure and then a precision-corrected variant (PC-VEM), motivated by the observation that the optimal Gaussian posterior precision is the prior precision plus a diagonal term, so the target lies outside the Kronecker family; they replace the diagonal by its scalar average γ\gamma and treat γ\gamma as a free parameter of the objective. On synthetic data the method recovers tuning curves and covariance factors and is competitive on co-smoothing; on Allen Visual Coding Neuropixels data it reports near-flat single-neuron variance along the visual hierarchy, shared co-variability peaking in VISp, and a spectral misalignment between the activity covariance and the estimated gain covariance.

Critical strengths
  • The generative formulation is well posed and useful. Making the shared gain a matrix-normal latent with explicit neuron and time factors provides an interpretable, scalable handle on population overdispersion that existing single-neuron or low-rank-noise models do not give, and it directly enables the neuron-versus-time decomposition that the results exploit.
  • The real-data finding is concrete and biologically interpretable. The claim that gain covariance is spectrally misaligned from the activity covariance, if robust, would be a substantive empirical contribution, since it suggests that the drivers of shared gain differ from the traditionally studied activity covariance.
  • The observation behind the correction is a genuine insight. Characterizing the optimal Gaussian posterior precision as prior precision plus a data-dependent diagonal is a clean way to show why the Kronecker family cannot reach the target posterior.
Critical weaknesses
  • The precision-correction narrative overreaches and is under-justified. The theoretical analysis is used to motivate a heuristic that changes the objective, and no guarantee or clean characterization of what PC-VEM optimizes is given, yet the framing implies principled consistency. The averaging of a position-, neuron- and time-dependent curvature into a single scalar γ\gamma discards the dependence structure the paper argues is important, and is not defended.
  • The real-data evidence is limited and weakly controlled. Results rest on three sessions, small per-region unit counts, and largely two-condition comparisons; the hierarchy trend, the alignment-index result and the multiple comparisons are not supported by uncertainty quantification adequate for the strength of the conclusions.
  • The synthetic comparison is not always fair. The out-of-distribution claim for the PLDS-simulated data reports a fraction of oracle for PC-VEM without reporting the corresponding fractions for the baselines, and the co-smoothing evaluation varies the observation model across methods in ways that make direct comparison difficult.
Questions for authors
  • Since PC-VEM replaces the true posterior precision with (Σ−1+γI)−1(\Sigma^{-1}+\gamma I)^{-1}, what objective is being optimized? Is there a valid bound or a fixed-point/consistency argument, or is this best understood as a heuristic estimator?
  • How sensitive are U^\hat U, V^\hat V, the covariance-error metrics and the misalignment result to the scalar averaging of DntD_{nt}? Please report a variant in which γ\gamma varies by neuron or time, or where a low-rank correction is used.
  • In the synthetic comparison, what are the baseline fractions of oracle on the PLDS data? Reporting them alongside PC-VEM's 0.97 is needed for a fair statement of the advantage. Or, specifically, the paragraph starting at line 361 is a bit unclear to me. The authors should clarify or explain these numbers in more detail.
2 comments

Response to Weakness 1 (narrative) and Question 1:

Thank you. Section 3.3 will state the E-step constraint set as its own numbered equation, together with both steps. We apply the correction to the constraint set of the E-step. The ELBO, eq. (3.3), is unchanged. What changes is the set of Gaussian posteriors, which in PC-VEM is built from the current neuron and time covariances. Lemma 1 proves that, at every critical point of the ELBO, the posterior precision is the prior precision plus a diagonal. Lemma 2 proves that this diagonal is positive. By the proof of Theorem 1, this form with any positive \gamma is not a Kronecker product of neuron and time covariances. Using one \gamma for the whole diagonal is a heuristic choice.
PC-VEM is an alternating maximization of the ELBO. In the E-step, the constraint set changes only for the variational covariance, by constraining its precision. The posterior covariance is shared across conditions, as in VEM. But its precision is the current prior precision with \gamma added to each diagonal entry. Lemma 1 shows that an optimal Gaussian posterior has precision equal to the prior precision plus a diagonal, and PC-VEM adopts that form with a single \gamma. This observation motivated the change to the optimization constraints and is heuristic. Every positive \gamma gives a positive-definite covariance, and so the ELBO is a lower bound on each iterate by Jensen’s inequality. We do not have a statistical consistency result. For robustness, we will add standard EM algorithm safeguards e.g. step-reduction updates. In all our synthetic experiments the ELBO rose at every iteration, so we expect the safeguard to leave our results unchanged.

Response to Weakness 1 (gamma):

We appreciate this point. It was very helpful, and addressing it has made Section 3.3 clearer. The proof of Theorem 1 uses only that \gamma is positive, so we will restate the theorem for any positive \gamma. The Poisson likelihood contributes one curvature value per neuron and time bin, with nothing linking different entries. All dependence across neurons and time therefore comes from the prior precision. PC-VEM keeps this in its entirety, and the M-step re-estimates every entry of both covariances. A single \gamma does not capture how the curvature varies between entries, and we will state this trade-off in Section 3.3 explicitly. Empirically we also find this important, as in our ground-truth experiments, PC-VEM recovered both covariances more accurately than VEM in every regime.

Response to Weakness 2:

We agree that the current real-data evidence is relatively limited. To address this weakness, we extend our analysis to an additional 9 sessions from the Brain Observatory component of the Allen Visual Coding database matching the stimuli originally analyzed. These 9 sessions contain an average of 58.0 +/- 19.3 (mean +/- SD) neurons in LGd, 48.7 +/- 11.3 neurons in VISp, 45.9 +/- 10.0 neurons in VISam, and 30.9 +/- 14.1 neurons in VISrl, with 15 presentations per stimulus condition (0°, 45°, 90°, and 135° drifting gratings) at 80% contrast. Applying the same analysis, we find that the results are consistent between the original 3 sessions and the additional 9 sessions (see table below). In both, the marginal variance across regions remains relatively flat while the mean absolute gain correlation starts low in LGd, peaks in VISp, and decreases in VISrl and VISam. We also continue to observe misalignment between activity covariance and gain covariance across all 4 regions.

RegionMarginal Variance (Median [Q1, Q3])Mean Absolute Gain Correlation (Median [Q1, Q3])Activity Covariance--Gain Covariance Alignment (Mean ± SEM)
LGd0.561 [0.402, 0.763]0.260 [0.224, 0.306]0.513 ± 0.013
VISp0.469 [0.247, 0.808]0.440 [0.409, 0.472]0.548 ± 0.010
VISrl0.411 [0.242, 0.676]0.367 [0.328, 0.401]0.608 ± 0.017
VISam0.418 [0.246, 0.777]0.362 [0.320, 0.462]0.495 ± 0.016

Response to Weakness 3:

This is a valid point to make an OOD claim. The oracle fraction was PC-VEM baseline which favours our method. We have corrected this using the co-smoothing metric described in the paper compared to a Poisson baseline. The methods cannot be directly compared between observation models but the methods can be evaluated by their ability to report the variability in the data which they all capture with different assumptions. In order to keep the evaluation fair, the observation models were kept to their original form.

Response to Question 2:

The covariance errors did not change. For \hat{U}, the relative Frobenius error was 0.320 ± 0.004 under PC-VEM and 0.318 ± 0.004 under the per-neuron variant (mean ± SEM over 12 draws). For \hat{V}, it was 0.248 ± 0.015 and 0.249 ± 0.015. Both differences are smaller than the standard error across draws. In this test, using one number in place of D_{n,t} doesn’t affect how well either covariance factor is recovered. The misalignment result is computed from \hat{U}, whose error did not change. We will add this comparison to the appendix. We additionally ran a variant in which D_{n,t} varies. Each neuron gets its own weight, equal to its average Fisher information over time bins and trials, scaled to have mean one. As in PC-VEM, a single overall scale is then chosen by maximizing the ELBO. We fit the variant to the (LR, M) regime of Table 1, on the same 12 data draws, finding a negligible difference

MethodεU\varepsilon_U MeanεU\varepsilon_U SEMεV\varepsilon_V MeanεV\varepsilon_V SEM
PC-VEM0.3200.0040.2480.015
per-neuron DD0.3180.0040.2490.015

Response to Question 3:

We fit each method with its own observation model in order to preserve the method comparison. We have corrected this by using the co-smoothing metric described in the paper compared to a Poisson baseline given by each neuron’s average spike count in each time bin over the training trials (nats per observation). The comparison doesn’t depend on the models since each method predicts firing rate for the same held-out neurons and all predictions are scored by the same Poisson LL to the same baseline (see table below). Accordingly the paragraph starting at line 361 will be updated.

ScenarioPC-VEMGPFAPGPFASNP-GPFAGP-LVM
LR LR0.04590.0135-0.02280.00830.0243
LR Matern0.07540.0285-0.01460.06200.0418
Matern Matern0.09160.0213-0.01690.07210.0368
snp0.08250.04110.01680.08210.0547
plds0.17380.07660.02650.14430.1111
Reviewer_d8h7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces the Poisson Matrix-Normal Latent Variable model (PMNLV), which jointly estimates stimulus tuning and latent gain fluctuations in neural populations. It models spike counts with a Poisson likelihood and places a matrix-normal prior on the gain, with its covariance factored into neuron and time components. To address a limitation of a standard matrix-normal variational approximation, the authors derive a precision corrected variational EM algorithm (PC-VEM).

On synthetic data, PC-VEM improves recovery of the covariance factors over standard VEM and performs favorably against several population models on co-smoothing prediction. The authors also apply it to Allen Institute mouse visual cortex recordings from four regions, reporting regional differences in estimated gain co-variability and a misalignment between gain covariance and activity covariance.

Critical strengths
  1. The paper identifies a meaningful gap between single neuron overdispersion models and population models that capture shared gain fluctuations. The matrix normal formulation provides an interpretable separation of neuron and temporal covariance, and the analysis of the variational posterior’s Kronecker structure limitation gives a clear motivation for PC-VEM.

  2. The synthetic experiments evaluate both covariance recovery and predictive performance across multiple covariance regimes, including data generated from other models. The real data analysis also offers a concrete application to an established visual-coding dataset.

Critical weaknesses
  1. The main biological conclusions rely on a narrow real data evaluation: the authors select only three sessions, chosen for having many isolated units, and require at least 20 units per region. This makes it difficult to assess whether the reported regional trends and gain/activity subspace misalignment generalize across sessions or animals. The real data model comparison is also limited to a mean-rate baseline, while comparisons to population models are conducted on synthetic data.

  2. The model’s gains are assumed to have a separable neuron-by-time covariance and to be independent across condition slices. These assumptions make the model tractable and interpretable, but they constrain which forms of population variability it can represent. The synthetic robustness experiments provide some support but limited.

Questions for authors
  1. Can you provide a real data comparison with relevant population models? The current real-data predictive evaluation against a mean-rate baseline does not establish how PMNLV compares with existing methods on the application that motivates the paper.

  2. How sensitive are the estimated covariance factors and the reported biological conclusions to violations of the separable covariance and conditional independence assumptions?

1 comment

Response to Weakness 1:

We agree that evaluating a greater number of sessions is important in assessing the generalizability of our described results. We extend our analysis to an additional 9 sessions from the Brain Observatory component of the Allen Visual Coding database matching the stimuli originally analyzed. These 9 sessions contain an average of 58.0 +/- 19.3 (mean +/- SD) neurons in LGd, 48.7 +/- 11.3 neurons in VISp, 45.9 +/- 10.0 neurons in VISam, and 30.9 +/- 14.1 neurons in VISrl, with 15 presentations per stimulus condition (0°, 45°, 90°, and 135° drifting gratings) at 80% contrast. Applying the same analysis, we find that the results are consistent between the original 3 sessions and the additional 9 sessions (see table below). In both, the marginal variance across regions remains relatively flat while the mean absolute gain correlation starts low in LGd, peaks in VISp, and decreases in VISrl and VISam. We also continue to observe misalignment between activity covariance and gain covariance across all 4 regions.

Marginal VarianceMean Absolute Gain CorrelationActivity Covariance--Gain Covariance Alignment
(Median [Q1, Q3])(Median [Q1, Q3])(Mean ±\pm SEM)
LGd0.561 [0.402,0.763]0.561\ [0.402, 0.763]0.260 [0.224,0.306]0.260\ [0.224, 0.306]0.513±0.0130.513 \pm 0.013
VISp0.469 [0.247,0.808]0.469\ [0.247, 0.808]0.440 [0.409,0.472]0.440\ [0.409, 0.472]0.548±0.0100.548 \pm 0.010
VISrl0.411 [0.242,0.676]0.411\ [0.242, 0.676]0.367 [0.328,0.401]0.367\ [0.328, 0.401]0.608±0.0170.608 \pm 0.017
VISam0.418 [0.246,0.777]0.418\ [0.246, 0.777]0.362 [0.320,0.462]0.362\ [0.320, 0.462]0.495±0.0160.495 \pm 0.016

Response to Question 1:

We present a comparison of our model with other relevant population models on the real data, using the same held-out predictive log-likelihood and 70-30 split as Fig. 3d. The results demonstrate our model’s competitive performance in comparison with other population models. We note that beyond prediction, it is worthwhile to emphasize that our model’s primary contribution focuses on the estimation of structured gain covariance in a population, which is not directly estimated by prior models.

MetricPC-VEMGPFASNP-GPFAP-GPFAGP-LVM
Median PLL+0.069+0.069+0.064+0.064+0.053+0.053+0.029+0.029−0.009-0.009
IQR[+0.041,+0.108][+0.041, +0.108][+0.041,+0.092][+0.041, +0.092][+0.020,+0.080][+0.020, +0.080][−0.015,+0.067][-0.015, +0.067][−0.040,+0.008][-0.040, +0.008]

Response to Weakness 2 and Question 2:

Assumption of independence between slices is reasonable. Interpretation of probabilistic dependence between successive or possibly non-successive trials also could be interpreted as network memory and is not something that we could disambiguate in our model despite this being possibly through synaptic plasticity or gain regulation by other brain regions targeting visual area (Pennartz et al., 2023). We have tested robust to non-separability in two ways. First, by using PLDS to generate the data, we violate our model’s separability assumption then using the co-smoothing metric on each model we show that PC-VEM outperforms related models:

ScenarioPC-VEMGPFAPGFPASNP-GPFA`GP-LVM
LR LR0.04590.04590.01350.0135−0.0228-0.02280.00830.00830.02430.0243
LR Matern0.07540.07540.02850.0285−0.0146-0.01460.06200.06200.04180.0418
Matern Matern0.09160.09160.02130.0213−0.0169-0.01690.07210.07210.03680.0368
snp0.08250.08250.04110.04110.01680.01680.08210.08210.05470.0547
plds0.17380.17380.07660.07660.02650.02650.14430.14430.11110.1111

Additionally, to test robustness, we produce an NTC tensor where the first half have neuron covariance U_1 (low-rank) and the second half have covariance U_2 (Matérn). Splitting the tensor by structure, we fit each subtensor and recover both U_1 and U_2. In all 12 draws, the two estimates differed more than the two halves of any tensor built from a single covariance. We next pool the slices and fit the full tensor once, without the labels. The recovered U_3 is approximately the average of U_1 and U_2. A single fit therefore gives one neuron covariance for all the trials it is given. So if the conditions are known, or are assumed to have different structures, the model can be run on the subtensors. If not, the model still estimates the average over the trials in the fit.

U1U_1 (low-rank)U2U_2 (Matérn)
Correlation of estimate with its own ground truth0.961±0.0020.961 \pm 0.0020.859±0.0040.859 \pm 0.004
Correlation of estimate with the other ground truth0.054±0.0030.054 \pm 0.0030.048±0.0060.048 \pm 0.006
Pooled estimate U3U_3 compared withCorrelation
U1U_10.832±0.0050.832 \pm 0.005
U2U_20.466±0.0090.466 \pm 0.009
Average of U1U_1 and U2U_20.933±0.0020.933 \pm 0.002
Reviewer_JD4oOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a population-level method for estimating neural variability, the Poisson matrix-normal latent variable model (PMNLV). It extends single-neuron overdispersion (modulated Poisson) models to neural populations by placing a matrix-normal prior on a latent gain that is shared across neurons and over time, with the gain covariance factorized (Kronecker) into a neuron factor and a time factor. Spike counts are modeled as Poisson, with a rate that combines a per-neuron tuning term and the latent gain through a soft-rectifying link.

The key contribution is a variational expectation-maximization algorithm that jointly estimates the tuning curves and both covariance factors, rather than assuming gains are independent across neurons and time. The method is validated on synthetic data, where it recovers the tuning curves and covariance factors (including at low firing rates, ~0.3 Hz) and compares favorably with related population and overdispersion methods. It is then applied to Allen Institute Neuropixels recordings from mouse visual cortex, where it reveals across-area differences in shared gain structure and reports that the activity covariance and the estimated gain covariance are spectrally misaligned.

The paper is well written, and the methods and results are clearly laid out.

Critical strengths
  1. A principled extension of overdispersion models to the population level. Most existing overdispersion models treat each neuron's gain as independent. Estimating a shared gain jointly with the stimulus drive, using a variational EM algorithm, is a natural and useful methodological step. It gives the field a tool to ask how gain fluctuations are coordinated across neurons and over time, which single-neuron models cannot address.

  2. Clear presentation and validation on both synthetic and real data. The paper is well written, and the model, inference procedure, and results are laid out clearly. Applying the method to large-scale Allen Institute Neuropixels recordings is commendable, because it shows that the method is practical on real multi-area data and can reveal across-area differences.

Critical weaknesses
  1. Limited scientific insight from modeling population-level variability through a shared gain. A key open question in the field is why the gain in overdispersion models exists and what role it plays in neural computation. People have speculated that it reflects attention or arousal (neuromodulation), but there is little direct empirical evidence for this. For example, Goris et al. (2014) showed that overdispersion gain increases along the visual hierarchy, but the functional significance of that result is still unclear. The present paper estimates shared gain structure more completely, but it does not bring us closer to answering this question. As a result, it is unclear what we learn about neural computation from the across-area differences reported in the Allen data. Even for a methods paper, a discussion of why VISp shows higher population-level gain (and how this fits with the hierarchical trend reported by Goris et al.) would be an important part of the paper's message.

  2. Incomplete validation, and real-data results that are not yet convincing. (a) Synthetic data: There is no test in which the neuron-to-neuron covariance is varied across conditions. Without one, it is unclear whether the method can recover several distinct covariance structures rather than a single shared structure. This matters for any scientific use of the method that compares conditions, states, or areas. (b) Real data: VISp appears to show higher variability at the single-neuron level too, even though that difference is not statistically significant, while the population-level difference is significant. A significant result at one level and a non-significant result at the other does not, by itself, show that the two levels differ: the difference between "significant" and "not significant" is not necessarily itself significant. To claim that the method reveals something single-neuron analyses miss, the authors need a direct statistical comparison of the two levels (e.g., testing the interaction between level and area). Without this, the paper's central empirical claim, that population-level gain gives new insight, is not established.

Questions for authors
  1. Can you add a synthetic experiment in which the neuron-to-neuron covariance structure differs across conditions, and show that PMNLV recovers each distinct covariance structure (and correctly detects that they differ)?

  2. For the Allen Neuropixels results, can you directly test whether population-level and single-neuron-level variability differ across areas, rather than comparing a significant result at one level with a non-significant result at the other? VISp looks numerically higher at the single-neuron level as well, so it is important to show that the population-level effect goes beyond what single-neuron overdispersion already captures.

  3. What is the scientific insight from the across-area differences? In particular, why might VISp show higher population-level gain? How does this relate to the increase in overdispersion gain along the visual hierarchy reported by Goris et al. (2014)? Even a well-motivated speculation in the Discussion would strengthen the paper.

3 comments

Response to Weakness 1:

The scientific insight we aim to convey is based on the observation that areas differ much more in how gain is shared across neurons than in how single neuron gains are modulated. Moreover, we saw that gain sharing peaked in VISp from LGd and decreased further up the hierarchy (Fig. 4b). We note that it is difficult to compare directly to Goris et al. (2014) in terms of brain area due to the difference in species (mouse vs. macaque). In mouse, VISp has distinct temporal-frequency encoding and VISrl and VISam prefer gratings that drift at much higher speeds than VISp, are more direction selective (Marshel et al., 2011), and have large receptive fields (Siegle et al., 2021; D’Souza et al., 2022), indicating that they represent broader features. We can interpret this as single units in LGd are more likely to either be highly robust in their response (low gain) or not (high gain) independently from other units, meaning that the encoding properties are really per-neuron and potentially recommending that correlations between neurons are due to the image statistics rather than the network. Alternatively, in VISp, a much higher correlation in the gain means that regardless of the feature the neurons are tuned to, the propensity to be stable (low gain) or not (high gain) occurs over populations, indicating that the network is imbuing more of the encoding structure as compared to the visual stimulus. We feel that this interpretation opens up avenues for mechanistic modeling to test these hypotheses, and will add a discussion point to this effect.

Response to Weakness 2 (Real data) and Question 2:

We appreciate the opportunity to clarify. Our intent was not to rely on significance levels. We wanted to show that regions differ far more in how much gain is shared between neurons than in how much each neuron’s gain varies. To test this directly, we asked whether pairs of neurons that are equally variable on their own still differ in gain correlation across regions. We grouped neuron pairs by their single-neuron gain variance, using the same groups for all regions, and compared the median absolute gain correlation within each group. LGd pairs had the lowest gain correlation in every group, and VISp pairs had the highest in nine of ten groups (table below). Because the pairs being compared are equally variable on their own, single-neuron variability alone cannot explain the regional differences in gain correlation. We will add this analysis and table to the paper.

AreaAbsolute gain correlation (Median across bins)
LGd0.177
VISp0.401
VISrl0.351
VISam0.391

Response to Weakness 2 (Synthetic Data) and Question 1:

We produce an N×T×C tensor in which slices 1 to C/2 have neuron covariance U_1 (low-rank) and slices C/2+1 to C have U_2 (Matérn). Splitting the tensor by structure, we fit each subtensor and recover both U_1 and U_2. In all 12 draws, the two estimates differed more than the two halves of any tensor built from a single covariance. We next pool the slices and fit the full tensor once. The recovered U_3 is approximately the average of U_1 and U_2. A single fit therefore gives one neuron covariance for all the trials it is given. So if the conditions are known, or are assumed to have different structures, the model can be run on the subtensors. If not, the model still estimates the average over the trials in the fit. The tables below have the results of these tests, which we will include in the manuscript.

U1U_1 (low-rank)U2U_2 (Mat'ern)
Correlation of estimate with its own ground truth0.961±0.0020.961 \pm 0.0020.859±0.0040.859 \pm 0.004
Correlation of estimate with the other ground truth0.054±0.0030.054 \pm 0.0030.048±0.0060.048 \pm 0.006
Pooled estimate U3U_3 compared withCorrelation
U1U_10.832±0.0050.832 \pm 0.005
U2U_20.466±0.0090.466 \pm 0.009
Average of U1U_1 and U2U_20.933±0.0020.933 \pm 0.002

Response to Question 2 (across-area differences and VISp):

We interpret the shared gain differences as a function of the representation properties as features go from simple visual features to higher-order features. LGd is often considered a relay area from retina to cortex, capturing more basic visual properties, whereas the higher visual cortical areas VISp, VISrl, and VISam are considered to capture more complex spatial and temporal patterns. Moreover, VISp and higher visual cortical areas are also affected by behavioral-state effects such as arousal, locomotion and attention, which can modulate responses across neuronal populations in these regions (Pennartz et al., 2023; Akella et al., 2025).

Low level features require direct transmission of representations faithful to the visual stimuli independent of other factors that might impact perception. We feel that the independence of the gain demonstrates this as the trial-by-trial resilience (low gain correlations) implies that whether or not a neuron is consistently responsive to a stimuli does not depend on the consistency of the other neural responses. Similarly, as features begin to coalesce into more complex features that must be combined with other task information and internal priors to form percepts, we would expect that different neurons that might encode different features might have their reliability modulated together (i.e., shared gain). VISp is a prime area for maximal correlation as the visual features are still somewhat localized (smaller receptive fields than VISrl and VISam) while still being a part of the higher cortical pathway. Thus the slight decrease in shared gain in these latter areas might be due to the larger receptive fields (D’Souza et al., 2022) and more specialized tuning (Marshel et al., 2011), that create functional specialization with slightly more independent gain properties.

Response to Question 2 (improved discussion):

In anesthetized macaques, per-neuron gain fluctuations grow from thalamus (LGN) to V1, V2 and MT (Goris et al., 2014). Hénaff et al. interprets this rise as consistent with growing uncertainty. Interestingly, our parallel measure, per-neuron gain variance, changes minimally (Fig. 4a). We would like to note that while a direct relationships between bain areas is difficult between our work (in mouse) and Goris et al. (2014) (Macaque) due to the species differences, we speculate that there is a similarity in the relationship between gain and uncertainty as observed in macaques (Hénaff et al., 2020; Goris et al., 2025). Specifically, we speculate that shared gain in VISp reflects uncertainty shared across a population, which specialized subpopulations in higher areas would share less. Shared uncertainty among unshared tuning properties can be highly beneficial when considering uncertainty at the object level, rather than the feature level. Gain correlations, we feel, might open future avenues to explore such alternative coding. Overall, these speculations form hypotheses that can be directly tested in future experiments. We will add this potential biological connection and future potential to the discussion.

I thank the authors for their detailed responses. However, it remains unclear what insight the method (a population-level decomposition of neural variability) provides, as the proposed interpretation is too broad and speculative. In addition, the split-covariance analysis does not fully address my point in Weakness 2(a). I will therefore keep my assessment largely unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.