- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a flow-matching framework that connects several neural representational distances through constraints on the velocity field. It also applies the method to Fisher information and time-dependent representational distances in the mouse visual cortex.
Critical strengths
-
I find the unification of representational distances the strongest part of the paper. The authors show how constraints on the velocity field recover familiar distances, which is useful because they capture which features of the response distributions each distance cares about. This provides a useful way to think about metric choice in neural data analysis and is the main reason I lean toward acceptance.
-
The continuous-condition applications give the framework practical value. Estimating Fisher information and time-varying distances from limited neural recordings is a relevant problem, and the simulation results are promising. The neural-data examples also show how the method could be used, though my concerns about the evaluation splits still need to be addressed.
Critical weaknesses
-
I am not fully convinced that there is a benefit to the flow matching itself. In the non-gaussian experiments, most baselines assume gaussian responses, so they cannot capture the structure being tested. A conditional normalizing flow would be a useful comparison. Without this, it is hard to tell how much of the gain comes from flow matching itself. In other words, a conditional normalizing flow might also capture the higher-order structure in the gaussian mixture simulations and outperform the gaussian baselines without using flow-matching training.
-
The Allen dataset's test-train splits raise concerns about the reported held-out performance. Wouldn't placing temporally correlated observations from the same trial in both training and test sets make the evaluation optimistic? Also, for the neuropixels dataset, PCA was fitted before splitting, unlike the other datasets. I would like to see trial or block-disjoint evaluation with training-only PCA, since these issues weaken the evidence for generalization to new trials.
Minor clarity issue: 3. Figure 1 is great, but in the caption, is mentioned (line 39) without explaining what it is. Without the definition, that sentence is meaningless until a reader gets to Section 3 line 124 where it is defined as "velocity class", which is, by itself, still not clear what it means. Only after I got to Figure 2, I understood what it meant. The readability could be improved.
-
Line 88: was introduced without defining what p and x are.
-
Table 1: "Auxiliary tau-dependent distance". What is this? What is this ?
Questions for authors
Can HSIC or CKA also be derived from the author's framework?
We thank the reviewer for recognizing the value of our unification and the practical relevance of our continuous-condition applications, and for the constructive feedback. Please see our response below.
Q3.1:
I am not fully convinced that there is a benefit to the flow matching itself. In the non-gaussian experiments, most baselines assume gaussian responses, so they cannot capture the structure being tested. A conditional normalizing flow would be a useful comparison. Without this, it is hard to tell how much of the gain comes from flow matching itself. In other words, a conditional normalizing flow might also capture the higher-order structure in the gaussian mixture simulations and outperform the gaussian baselines without using flow-matching training.
R3.1:
Thank you for this suggestion. We have now compared flow matching with a conditional neural spline flow (Durkan et al., 2019; https://arxiv.org/abs/1906.04032) trained by maximum likelihood, using matched datasets and splits across five runs (Supplementary Figure S16, page 52, lines 2766–2775). Flow matching generally had lower estimation error. It is worth emphasizing that neural spline flow serves as one example method, and broader comparisons with other non-Gaussian estimators remain for investigation in future.
Our main contribution is the theoretical framework and for neuroscience applications. In neuroscience applications, popular estimators such as the Wishart process (for fisher information) impose explicit distributional assumptions that may lead to wrong result (Figure 4D to F). This is where unconstrained flow-based methods may have an advantage.
Q3.2:
The Allen dataset’s test-train splits raise concerns about the reported held-out performance. Wouldn’t placing temporally correlated observations from the same trial in both training and test sets make the evaluation optimistic? Also, for the neuropixels dataset, PCA was fitted before splitting, unlike the other datasets. I would like to see trial or block-disjoint evaluation with training-only PCA, since these issues weaken the evidence for generalization to new trials.
R3.2:
This is a good point. We agree that the original split could give optimistic held-out performance. We therefore re-ran all four methods across all 32 Neuropixels sessions using whole-presentation (“trial”) splits within each condition (12 training and 3 test trials when 15 were available), with the 70-component PCA fitted only on training responses (lines 1887–1892). Mean held-out likelihood is lower for all four methods, but their ranking is unchanged: unconstrained flow, affine flow, GKR, then Bin+LW. This preserves the conclusion of the likelihood comparison. Figure S14 shows the results (page 50, lines 2661–2666).
Q3.3:
Minor clarity issue: 3. Figure 1 is great, but in the caption, [V] is mentioned (line 39) without explaining what it is. Without the definition, that sentence is meaningless until a reader gets to Section 3 line 124 where it is defined as “velocity class”, which is, by itself, still not clear what it means. Only after I got to Figure 2, I understood what it meant. The readability could be improved.
Line 88: [pθ(x)] was introduced without defining what p and x are.
Table 1: “Auxiliary tau-dependent distance”. What is this? What is this [τ]?
R3.3:
We clarified all three definitions, with changes marked in red: V is the set of allowed velocity fields, such as all fields affine in x (Figure 1 and Section 3, lines 38–39 and 125); x is a neural population response vector, and pθ(x) is its density conditional on a continuous stimulus or behavioral variable θ (lines 88–89); τ is an additional observed variable, such as time within a trial, at which conditions are compared (Table 1, lines 217–218 and 231).
Q3.4:
Can HSIC or CKA also be derived from the author’s framework?
R3.4:
CKA operates at a different level from the condition-to-condition distances studied in our framework. Our framework estimates distances between condition-specific response distributions to construct an RDM. Given two populations’ responses to matched conditions, CKA compares their centered kernel similarity matrices. This comparison across representations is not directly covered by our framework.
HSIC is also not directly covered by our current framework because it addresses a different comparison task. Our framework compares condition-specific response distributions, pa(x) and pb(x), defined on the same neural response space, so our results concern distributional dissimilarity across conditions. HSIC instead measures statistical dependence between paired variables X and Y, which may take values in different spaces.
We thank the reviewer for highlighting this distinction and have added it to the Discussion (page 10, lines 493–498).