acceptodds
Under review as a conference paper at ICLR 2027

Topographic Soft Matching Distance (TSMD): A Framework for Comparing the Spatial Organization of Neural Tuning

22 views

Abstract

Neural populations are spatially organized according to their functional tuning, but how consistently this organization recurs across individuals remains unclear. Existing work typically begins with a predefined tuning variable (e.g., orientation preference, category selectivity), and asks whether its spatial organization appears similar across individuals, often through qualitative comparison of the resulting maps. This becomes limiting for responses to complex natural stimuli, where tuning is high-dimensional and its relevant axes may not be known or easily interpretable. Here, we introduce Topographic Soft-Matching Distance (TSMD), a framework for quantifying shared spatial organization in high-dimensional neural representations without requiring predefined tuning features. TSMD extends soft-matching distance khosla2024soft using fused Gromov–Wasserstein optimal transport, identifying correspondences between neurons that preserve both tuning similarity and relative spatial geometry. It tests for shared topographic organization by varying the relative weighting on matching neurons with similar tuning and preserving pairwise spatial distances, then comparing the resulting matches against a permutation null that shuffles tuning profiles across fixed spatial locations. After validating TSMD on synthetic orientation maps, we apply it to neural recordings across species and cortical areas. We find shared topographic organization across individuals in macaque and , but little evidence in mouse or macaque inferotemporal recordings. In human fMRI data collected during naturalistic viewing, TSMD reveals conserved large-scale semantic maps across individuals that cannot be explained by local smoothness alone. Finally, comparisons of developmental models of topography reveal shared global organization across independent realizations of the same self-organizing model despite differences in their precise layouts, but no reliable correspondence beyond local smoothness between models with different map-forming rules. Together, these results establish TSMD as a general framework for testing the conservation of functional topography across brains and computational systems.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Optimal Transport, Neural Tuning, Representational Similarity, Deep Neural Networks

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 4.00 over 3 reviews, above 99% of the others.
Reviewer_5KoCOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for comparing the spatial organization of two neural representations. Unlike conventional representational similarity measures such as CKA, RSA, and Procrustes distance, the proposed Topographic Soft-Matching Distance (TSMD) jointly considers the similarity of neural tuning profiles and their relative spatial organization.

The proposed method is based on fused Gromov–Wasserstein optimal transport, which allows the comparison of representations while accounting for both functional tuning and pairwise spatial relationships.

The authors evaluate the proposed method using synthetic data and biological datasets, including the Allen Brain Observatory dataset and the THINGS Ventral Stream Spiking Dataset. The method is also applied to computational models of cortical topography.

The paper is generally well written, and the proposed method is intuitive and easy to understand. However, the reviewer has several concerns regarding its methodological properties and the interpretation of some experimental results, as discussed below.

Critical strengths
  1. The proposed measure, presented in Section 2.2 (page 3), is conceptually simple, intuitive, and easy to understand. The combination of tuning similarity and relative spatial geometry provides a straightforward framework for comparing topographic representations.

  2. The application of TSMD to biological datasets provides useful empirical support for the proposed framework. In particular, the comparisons across different species and cortical regions demonstrate its potential applicability to real neural recordings.

  3. The use of synthetic experiments and permutation-based null models provides a systematic means of evaluating the proposed measure under different spatial organization conditions.

Critical weaknesses

The reviewer does not identify any fundamental conceptual weaknesses in the proposed framework. However, several concerns should be addressed.

  1. Normalization of the tuning and geometric costs

TSMD combines a tuning cost and a geometric cost using a weighting parameter α\alpha. However, the effective contribution of each term depends on their relative numerical scales.

Therefore, it is important to clarify how the authors normalize the spatial distances and the tuning costs. In particular, the authors should explain whether the conclusions are robust to changes in the relative scales of these two terms.

This issue is especially relevant because the proposed evaluation integrates the similarity over different values of α\alpha.

  1. Additional experiment on spatial scaling Related to the first concern, the reviewer suggests adding an experiment to Fig. 2 in which the original spatial representation is expanded or compressed.

In particular, it would be useful to investigate both uniform scaling and anisotropic scaling, where the expansion rates differ across spatial axes.

While uniform scaling preserves the relative geometric structure, anisotropic scaling changes the pairwise spatial relationships. Examining how TSMD responds to these transformations would help clarify its geometric properties and sensitivity to spatial distortions.

  1. Interpretation of the SOM experiments

The objective of a self-organizing map (SOM) is to preserve the topological relationships of high-dimensional data in its low-dimensional representation, rather than the absolute locations of individual representations. Consequently, different initializations can naturally produce different locations of the best-matching units (BMUs).

The authors report significant TSMD correspondence between independently initialized SOMs. However, the reviewer would appreciate a clearer explanation of what additional insights can be gained from these experiments.

In particular, does the observed correspondence indicate the conservation of a specific global spatial organization, or could it primarily reflect common statistical properties of maps generated by the same learning rule?

Clarifying this distinction would help establish the significance of the SOM experiments.

As an additional experiment, the reviewer suggests comparing representations generated by different dimensionality-reduction methods, such as SOM and t-SNE, using the same high-dimensional dataset. Since these methods employ different objectives and produce different spatial organizations, such a comparison could provide further insight into what TSMD measures. It would also help demonstrate the practical usefulness of TSMD for researchers interested in comparing representations generated by different methods.

Questions for authors
  1. Spatial sampling in the within-monkey experiments

In the within-monkey experiments, the electrodes from a single cortical region are divided into two disjoint subsets. Since these subsets may sample different portions of the cortical organization, how does their spatial sampling affect the measured TSMD similarity?

In particular, would the conclusions remain valid if the two subsets were spatially separated rather than randomly interleaved?

Although the authors provide subsampling and checkerboard controls, further clarification of how these controls address differences in spatial coverage would be helpful.

  1. Practical interpretation of TSMD

What practical insights can be gained from this study by researchers who need to compare two different neural representations?

In particular, how should practitioners interpret the magnitude of TSMD similarity or its AUC? Can the authors provide guidance on determining whether an observed difference reflects meaningful variation in spatial organization rather than differences in spatial scale, sampling density, or noise?

5 comments

We would like to thank reviewer 5KoC for finding the measure conceptually simple and intuitive, and for noting its applicability across species and cortical regions. We address each of their questions below.

Normalization of the tuning and geometric costs Great point. Pairwise distances lie on a different scale compared to tuning cost, which lies in [0,2][0, 2] for our mean-centered, unit-normed tuning vectors. To circumvent this, we rescale each population’s sqaured pairwise distance matrix by its median off-diagonal entry, and now state this explicitly in Sec. 2.2 L148. We also experimented with max scaling. In our data, max scaling compressed most distances toward zero, and we observed less reliable convergence under this normalization. With median rescaling, a typical cortical distance is one, the same numerical scale as the tuning cost for uncorrelated response profiles (1−r=1)(1-r=1). The geometric penalty then reflects squared discrepancies in these normalized distances and \alpha controls the relative weight of these geometric and tuning penalties

More importantly, we would like to note that rescaling the geometric term by a constant does not change the set of couplings traced over α\alpha, as it simply reparametrizes α\alpha. Consider the loss Lα\mathcal{L}_\alpha from Sec. 2.2, and write G(T)G(\boldsymbol{T}) and C(T)C(\boldsymbol{T}) for its geometric and tuning costs. If the geometric term is rescaled by some constant c>0c > 0, we have

αc G(T)+(1−α) C(T)=(cα+1−α)(α′G(T)+(1−α′) C(T)),α′=cα1−α+cα.\alpha c\,G(\boldsymbol{T}) + (1-\alpha)\,C(\boldsymbol{T}) = (c\alpha + 1 - \alpha)\left(\alpha' G(\boldsymbol{T}) + (1-\alpha')\, C(\boldsymbol{T}) \right), \qquad \alpha' = \frac{c\alpha}{1 - \alpha + c\alpha}.

Note that α′∈(0,1)\alpha^\prime \in (0, 1) is positive and independent of T\boldsymbol{T}. Therefore, the optimal coupling at α\alpha under scale cc is the optimal coupling at α′\alpha^\prime in the original scale. Since we normalize each population by its own median, the relative scale of the populations is fixed, which makes TSMD invariant to rescaling either population’s coordinates (e.g., differences in unit or array size). This holds for mean, median or max scaling alike. We use the median because it is robust to a few outlying pairwise distances, which can dominate the mean or maximum. We also note that while the normalization preserves the family of couplings across the full α\alpha sweep, switching the normalizations (e.g., from median to mean) can change the AUC because the same couplings occur at different α\alpha values. We now state this explicitly in Sec. 2.2.

We tested this sensitivity directly in the CNeuroMod analysis by repeating the full sweep with median and maximum scaling. With median scaling, the observed and null curves separated at lower values. Under both normalizations, however, the observed AUC significantly exceeded both the full-permutation and smooth-patch nulls for all six participant pairs. Thus, the choice of scaling affected where the separation appeared, while the inference of shared topography held under both choices. We report median scaling in Fig. 7 and maximum scaling in Appendix Fig. A9.

Additional experiment on spatial scaling. Related to the first concern, the reviewer suggests adding an experiment to Fig. 2 in which the original spatial representation is expanded or compressed.

Thank you for the suggestion. We have added this experiment to Fig. 2B. We now scale the coordinates of each neuron ii in Xbase\boldsymbol{X}_{\text{base}} as ri=(xi,yi)↦(sxxi,syyi)\boldsymbol{r}_i = (x_i, y_i) \mapsto (s_x x_i, s_y y_i), keeping its tuning fixed. Uniform scaling (say sx=sy=λs_x = s_y = \lambda) multiplies all squared distances in a population by λ2\lambda^2. Our median rescaling (see response to the first point) removes this factor, so TSMD is invariant to uniform scaling of either or both populations. Anisotropic scaling (sx≠sys_x \neq s_y) changes relative pairwise distances, so neither α\alpha nor the median rescaling can absorb it. On our synthetic orientation map (Fig. 2), with sx/sy=2s_x/s_y = 2, we find that TSMD similarity stays nearly constant at small α\alpha, and decreases mostly at larger α\alpha, where the geometric term dominates. It remains well above the permuted map across α\alpha. We note that we intend TSMD as a test for conserved topography, rather than a raw distance. A moderately stretched map keeps the same neighboring neurons, so it should count as being conserved.

Interpretation of the SOM experiments

We thank the reviewer for highlighting this distinction. Our experiment does not test whether independently trained SOMs place the same tuning at corresponding cortical locations. Rather, our question is whether independent SOMs nevertheless reproduce some of the “relative spatial arrangement of tuning”. For example, two maps could contain equally smooth orientation and ocular-dominance domains, yet differ in how those domains are arranged relative to one another. TSMD asks how closely units can be matched by their tuning while preserving the distances between them within each map. This comparison does not require any unit or pinwheel to occupy the same absolute location across runs.

We use the patch-permutation control to go beyond the observation that SOMs produce smooth maps. It retains tuning neighborhoods within patches while rearranging the patches on the sheet. Across 15 independent SOM pairs, the original maps showed greater correspondence than this control (q=0.010q=0.010; Fig. 8C). Both the SOM and the elastic net produced smooth maps with orientation pinwheels, yet we found no evidence of correspondence beyond the patch control when comparing the two models (q=0.319)(q=0.319). We therefore interpret the SOM result as evidence that some spatial relationships among tuning domains recur across runs beyond what within-patch organization alone predicts.

We also acknowledge that the patch null does not preserve the complete spatial correlation structure so we cannot exclude the contribution of all generic spatial statistics to our evidence of shared topography. We now discuss this in the limitations section of our paper (L526-531).

Comparison against t-SNE

We thank the reviewer for this suggestion. Our focus here was to compare models of cortical development, particularly whether different learning rules produce similar spatial arrangements of tuning on a simulated cortical sheet. We note that the conventional application of t-SNE places supplied observations (e.g., images) in space, whereas our developmental models learn tuning at locations on a cortical sheet.

We see the use of TSMD to compare two-dimensional embeddings produced by methods such as PCA and t-SNE as a promising direction for future work. TSMD is well suited here because it can compare the relative positions of input examples without requiring the two embeddings to share a coordinate system (which is often arbitrary for these methods). We have noted this potential application in the Discussion.

Spatial sampling in the within-monkey experiments

Thank you for raising this. TSMD compares populations sampling the same or homologous cortical regions, either within or across individuals. We interleave the within-monkey split so that both halves sample the same region of the cortex, and test whether that region’s spatial distribution of tuning is significantly organized against our permutation null. Spatially separated halves would sample different parts of the cortical map, which need not share the tuning or the spatial organization of that tuning. To rule out correlated signals from neighboring electrodes driving the result, the checkerboard split assigns alternate electrodes to each half, and random subsampling controls for recording density. The cross-monkey comparison is the strongest test across separately sampled populations, and V1\mathrm{V}1 and V4\mathrm{V}4 remain significant there (see Sec. 3.2.2).

Practical interpretation of TSMD

We intend TSMD as a tool to test for shared spatial organization rather than as a raw distance. Since it is difficult to choose a value of α\alpha a priori, we sweep α\alpha and summarize the curve by its AUC. Although A(α)A(\alpha) is expressed in units of tuning correlation, its magnitude and AUC depend on the distribution of tuning similarities across populations and, for α>0\alpha>0, on the distribution and normalization of spatial distances within each population. We therefore encourage practitioners to interpret it relative to a permutation/patch null constructed for each dataset which preserves a population’s tuning distribution, sampling density, and noise, and only breaks the tuning-location pairing. We also encourage practitioners to plot the observed and null A(α)A(\alpha) curves alongside the AUC to show where and by how much they diverge as the match gives greater weight to spatial distances. We note that a larger raw AUC in one dataset does not by itself imply more strongly conserved topography.

We handle spatial scale separately. Since each population's pairwise squared distances are rescaled by its median, TSMD is invariant to uniform spatial rescaling of either population's coordinates (see also our response on the normalization of the two costs). We demonstrate that the null test recovers known differences across species, with no topography in mouse V1\mathrm{V}1 (salt-and-pepper), and significant topography in macaque V1\mathrm{V}1. We also document the invariance of TSMD to coordinate isometries in Appendix A7. We have added a sentence to this effect in Sec. 2.3.

Reviewer_ejFrOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper provides a nice explanation of the Fused Gromov-Wasserstein distance, which combines (through alpha, 1-alpha weights) the soft-matching distance for tuning curve similarity and the relative distances (Euclidean or otherwise) of functionally-matched neurons. It argues that this metric can be used to identify meaningful patterns in spatial organization, particularly when taken in comparison to a tuning-curve-permuted null. This tool is applied to mouse V1 (confirming lack of structure), macaque V1, MT, IT (largely confirming structure, with some subtleties), human fMRI, and model-generated data. A few points could be explained more clearly, but overall the paper does an impressive job of conveying information, and presents an interesting idea with applications across a range of settings.

Critical strengths

Overall, the paper conveys a lot of information well in a small page count. The applications are broad, including multiple animals, brain regions, data types, and settings, with a diversity of results and interpretations across settings. I found the paper thoughtful, informative, and well-written.

Critical weaknesses
  1. The central metric is not novel to this paper, though its application to neural populations is (to my knowledge). The results are compared only to a permutation null rather than other metrics (besides the alpha = 0 and alpha nearly 1 cases).

  2. The retinotopy argument is noticeably more muddled than the rest of the explanation of the method. I understand the desire to leave it until after the first results are shown, but I would love to see it pulled into the beginning of section 2 and brought up to the level of the rest of page 2, which is admirably clear. The argument that local distances matter, but that these distances should be interpreted wrt retinotopy - this deserves significant attention upfront. It also seems that the retinotopy issue is handled differently in the model-generated case (map pooling), which raised further confusion.

Questions for authors
  1. The AUC similarity metric introduced in 2.3 seems somewhat arbitrary (why would all alphas be equally weighted?) and doesn't seem to appear in the paper. It is labeled as the "TSMD similarity" but this phrase appears as a y axis across the figures as the single-alpha similarity. Is this AUC metric a leftover from an earlier draft of the paper or am I missing something?

  2. The VGG layer sweep on line 322 sounds very intriguing - are there any interesting trends in which layers provide the best fits?

  3. Fig 7 shows a small separation between observed and null, and only at high alpha. Panel C does show null rejection, but can we get a little more context for why the small separation is still meaningful, and that these high alphas are the relevant values?

Not questions but a few minor points:

  • Spatial organization is a somewhat ambiguous term - it would be helpful to specify in the abstract that spatial distances are within tissue rather than, e.g. between receptive fields in world coordinates. The mention of bio-informed distance metrics on line 141 is an interesting point, but I believe the results all use 2D Euclidean distance, so either a preview of this fact after line 141 or a specification per-experiment would be nice.
  • The sentence on line 32 about the developmental models is accurate to the paper's content but very hard to interpret before reading the full paper. Please streamline.
  • Typo on line 121 "limitation"
  • By page 6, I was having to remind myself of which end of the alpha scaled was SMD and which was all geometry - this could be annotated on x axes as an aid to the reader.
  • Line 186 would be slightly more consistent with section 2.0 if it used N_x rather than N.
  • Fig 5C was a bit odd and not directly referenced, could use a little more support (but the paragraph starting on line 357 was thoughtfully written)
  • Fig 7 text is way too small
  • Line 484 typo "is complex" --> "is a complex"
5 comments

We would like to thank reviewer ejFr for noting the breadth of our applications and that the paper conveys information with brevity. We address each of their questions and concerns below.

The central metric is not novel to this paper, though its application to neural populations is (to my knowledge). The results are compared only to a permutation null rather than other metrics (besides the alpha = 0 and alpha nearly 1 cases).

We agree that the fused Gromov-Wasserstein (FGW) objective is not, in itself, a novelty of this paper. It is an established optimal transport tool whose mathematical properties have been studied in detail (Vayer et al., 2018, 2020). Our contribution is to introduce FGW as a tool for comparing neural representations that accounts for both how individual neurons encode information and how those neurons are arranged relative to one another in cortical tissue.

Regarding other metrics, standard measures such as RSA, CKA, Procrustes, Soft-matching etc all define similarity in terms of neural responses alone and do not incorporate cortical distances into their objective. Therefore, each would assign exactly the same score to the observed data and our shuffled control in which the same tuning profiles are reassigned to different cortical locations. TSMD brings tuning similarity and preservation of cortical distances into a single matching objective, allowing us to test whether the spatial arrangement of tuning is conserved across individuals. Related work has used a joint objective for cortical surface alignment (Thual et al., 2022). Here we use it to quantify and test for shared topography.

Consequently, there is no other metric for us to compare with. We should also clarify that we do compare against tuning correlation as a function of cortical distance, which is widely used as a signature of topographic organization (Fig. 5C). We observe this signature in both V4\mathrm{V}4 and IT. As we note in the introduction (L82-88), such statistics capture only whether nearby neurons share similar tuning, or local smoothness, and as a result cannot by itself tell whether two individuals share the same spatial arrangement of tuning.

The retinotopy argument is noticeably more muddled than the rest of the explanation of the method. I understand the desire to leave it until after the first results are shown, but I would love to see it pulled into the beginning of section 2 and brought up to the level of the rest of page 2, which is admirably clear. The argument that local distances matter, but that these distances should be interpreted wrt retinotopy - this deserves significant attention upfront.

Thank you for this suggestion. We chose to keep the full treatment of retinotopy after the first macaque results, since the confound is easiest to motivate once the reader has seen a within-monkey effect that it could explain. We agree, however, that the method itself belongs upfront. We have moved the factorized encoding model into the methods (Sec. 2.4), and kept the retinotopy control and its results after the macaque analysis (Sec. 3.2.2).

It also seems that the retinotopy issue is handled differently in the model-generated case (map pooling), which raised further confusion.

We use the same principle in the V1\mathrm{V}1 recordings and the developmental models. In the recordings, we estimate feature tuning separately from receptive-field position and compare the feature tuning across animals. In the models, retinal position helps shape the map during development. Once the map has formed, we compare only orientation, ocular dominance, and spatial-frequency tuning in the TSMD analysis because we wanted to focus on the columnar organization. In both cases, shared retinal position alone cannot drive a tuning match. We also note that pooling simply sets the resolution at which we compare the model maps and is not a correction for retinotopy. We have clarified this in Section 3.5.

The AUC similarity metric introduced in 2.3 seems somewhat arbitrary (why would all alphas be equally weighted?) and doesn't seem to appear in the paper. It is labeled as the "TSMD similarity" but this phrase appears as a y axis across the figures as the single-alpha similarity. Is this AUC metric a leftover from an earlier draft of the paper or am I missing something?

We agree that the labeling was unclear. “TSMD similarity” on the y-axis denotes A(α)A(\alpha), the coupling-weighted mean tuning correlation at a given α\alpha. The AUC summarizes this curve over the tested range. It is hard to decide a priori how much weight each α\alpha should receive. For example, does a practitioner care more about pure spatial organization, or just the tuning vector distribution? And in what proportion? We think this is a non-trivial task to decide. Hence, we use AUC as a summary rather than as a metric. Our point in using the AUC is to define a statistical test of significance against the null over KK resamples. The AUC enters the Wilcoxon test behind reported pp-value, as well as the observed-minus-null AUC panels (Figs. 7C, 8C, A5).

We also note that the numerical AUC depends on how the two costs are scaled. We therefore interpret the observed AUC relative to the null constructed from the same dataset using the same scaling. In CNeuroMod, for example, the curves separate at lower α\alpha values with median scaling than with maximum scaling, yet the observed AUC significantly exceeds both nulls for all six participant pairs under either normalization. We have clarified this interpretation in Sec. 2.3 and encourage readers to examine the observed and null curves alongside the AUC to see where and by how much they diverge.

The VGG layer sweep on line 322 sounds very intriguing - are there any interesting trends in which layers provide the best fits?

We observe that the best-fitting layer follows the ventral-stream hierarchy, and this pattern replicates across both monkeys. V1\mathrm{V}1 is best fit by conv2_2, and V4\mathrm{V}4 and IT by conv4_3. Held-out correlation in V1\mathrm{V}1 declines monotonically with network depth, while in IT, rises towards the later layers. This matches well-established studies which study correspondence between CNN depth and ventral stream (Yamins et al., 2014; Khaligh-Razavi & Kriegeskorte 2014; Cadena et al., 2019).

Fig 7 shows a small separation between observed and null, and only at high alpha. Panel C does show null rejection, but can we get a little more context for why the small separation is still meaningful, and that these high alphas are the relevant values?

We agree that the differences appear modest but note that the particular (\alpha) at which the curves separate depends on how cortical distances are scaled relative to the tuning cost, so we attach no special significance to a numerical cutoff. For this reason, we tested the paired difference in area under the full curves, using the same distance scaling for the observed and shuffled maps. The effect is small but consistent: for all six participant pairs, the observed AUC significantly exceeded both the full-permutation null and the patch null, which preserves local smoothness. This consistency supports our conclusion that the spatial arrangement of semantic tuning is shared across participants beyond what local smoothness alone can explain.

We also found that the original Fig. 7 used maximum scaling of squared cortical distances, whereas the other figures used median scaling. This helps explain why the gap was less apparent in Fig. 7: maximum scaling gives cortical distances less weight at a given . For consistency, we now show the median-scaled results in Fig. 7, where the curves separate much more clearly at lower values. We include the original maximum-scaled figure in Appendix A8 to show that the conclusion holds under both normalizations.

Spatial organization is a somewhat ambiguous term - it would be helpful to specify in the abstract that spatial distances are within tissue rather than, e.g. between receptive fields in world coordinates. The mention of bio-informed distance metrics on line 141 is an interesting point, but I believe the results all use 2D Euclidean distance, so either a preview of this fact after line 141 or a specification per-experiment would be nice.

Thank you for pointing out this ambiguity. We revised the abstract to clarify that spatial distances refer to the physical separation of recording sites within cortical tissue and we have avoided the term ‘spatial organization’ here. We also define spatial organization explicitly in the introduction (L104-L106), i. e., the extent to which neurons can be matched by their tuning while preserving their spatial relationships. Here, we also note that spatial distances are independent of the populations’ absolute locations, i.e., TSMD does not require two neural populations to share a registered coordinate system. This is by virtue of the fact that the geometric term requires only intra-population distances dX(i,i′)d_{\boldsymbol{X}}(i, i^\prime) and dY(j,j′)d_{\boldsymbol{Y}}(j, j^\prime) for neural populations X\boldsymbol{X} and Y\boldsymbol{Y} (see Sec. 2.2). We mention bio-informed distance functions d(⋅)d(\cdot), e.g., geodesic distance on the cortical surface, as an appealing choice if a practitioner cares about it. We would like to emphasize that our framework is flexible to the choice of distance function, and we merely state that there are other possibilities that a practitioner could take into consideration, depending on their application of interest. That said, we have now added a clarification in Sec. 2.2 stating that all our experiments use a Euclidean distance function in the geometric term.

The sentence on line 32 about the developmental models is accurate to the paper's content but very hard to interpret before reading the full paper. Please streamline.

Thank you for this suggestion. We simplified the sentence in the abstract to clarify the distinction between independent runs of the same model and models with different developmental rules.

Typo on line 121 "limitation"

Thank you, we have fixed this.

By page 6, I was having to remind myself of which end of the alpha scaled was SMD and which was all geometry - this could be annotated on x axes as an aid to the reader.

Good point. We now annotate the two ends of the α\alpha axis in the caption of every figure that sweeps α\alpha (α=0\alpha=0: tuning only as in soft-matching; α→1\alpha \rightarrow 1: geometry only).

Line 186 would be slightly more consistent with section 2.0 if it used N_x rather than N.

These are indeed different quantities. NN on L216 denotes the number of neurons in a base sheet Xbase\boldsymbol{X}_{\text{base}}, whose transformed copies have the same NN neurons. On the other hand, NxN_x and NyN_y in Sec. 2 denote possibly different neuron counts in a pair of representations.

Fig 5C was a bit odd and not directly referenced, could use a little more support (but the paragraph starting on line 357 was thoughtfully written)

Fair point. We now explicitly discuss naïve tuning correlation as a function of distance, which is what most work in the field uses. We also explicitly reference Fig. 5C directly in the main text as the comparison to this standard measure.

Fig 7 text is way too small

We have increased the font size in Fig. 7.

Line 484 typo "is complex" --> "is a complex"

Thanks for catching this, we have fixed it.

References:

[1] Yamins, D. L., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 111(23), 8619-8624.

[2] Khaligh-Razavi, S. M., & Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS computational biology, 10(11), e1003915.

[3] Cadena, S. A., Denfield, G. H., Walker, E. Y., Gatys, L. A., Tolias, A. S., Bethge, M., & Ecker, A. S. (2019). Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS computational biology, 15(4), e1006897

Reviewer_tLYROpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces Topographic Soft-Matching Distance (TSMD) to test whether neural populations share a global spatial organization of tuning across individuals, going beyond simple local clustering. Using fused Gromov-Wasserstein optimal transport, TSMD jointly matches high-dimensional tuning curves and pairwise cortical distances, evaluating significance against permutation nulls without needing predefined tuning labels. Across synthetic maps, mouse/macaque recordings, human fMRI, and developmental models, the authors show conserved global topography in macaque V1/V4, human semantic cortex, and across independent SOM seeds, but not in mouse V1, macaque IT, or between different map-forming rules (SOM vs. elastic net).

Critical strengths

The paper tackles a well-motivated question about whether the spatial layout of neural tuning is conserved across individuals, and the introduction sets up the problem very clearly. I also appreciate the breadth of the experiments, spanning synthetic maps, mouse and macaque recordings, human fMRI, and developmental models. Importantly, the authors include thoughtful controls, such as factoring out receptive fields and testing against patch-permutation nulls, to rule out simple alternative explanations. Together, these analyses make a solid empirical case for how the framework behaves across very different biological and model systems.

Critical weaknesses
  1. The contribution relative to existing methods needs clearer positioning. The paper acknowledges that its matching objective is fused Gromov-Wasserstein. A closely related approach, FUGW (Thual et al., NeurIPS 2022), also combines functional features and cortical geometry for brain alignment, but is not discussed or compared. Clarifying this relationship would help distinguish the contribution of the matching formulation from that of the spatial-weight sweep, similarity summary, and null-testing procedure. The additional methodological or inferential value of these components remains to be established relative to existing approaches.
  2. The semantic specificity of the human results requires further clarification. The human analysis defines semantic tuning through encoding-model coefficients fitted to transcript embeddings during movie viewing. These embeddings may covary with acoustic, visual, and lower-level linguistic features. The spatial permutation controls assess the organization of the resulting tuning profiles, but do not separate these possible sources of correspondence. The results therefore provide evidence for cross-participant correspondence under the tested null models; additional controls would help determine how specifically this correspondence reflects shared semantic organization.
  3. The spatial null models leave some ambiguity about what drives the observed correspondence. Patch permutations preserve within-patch organization but can alter continuity across patch boundaries and spatial correlations at larger scales. Differences from these controls could therefore partly reflect spatial statistics that the permutations do not preserve. It remains unclear how the procedure behaves on independent maps with comparable multiscale spatial structure but no shared global layout. Evaluating this case would help calibrate false-positive rates and clarify how specifically the reported effects support shared global organization.
Questions for authors
  1. Since TSMD adopts an established FGW objective, what is its methodological contribution relative to prior approaches such as FUGW (Thual et al., 2022)? Please distinguish changes to the matching formulation from contributions in the subsequent analysis, and clarify what additional capability these provide for identifying shared topography.
  2. Does the cross-participant correspondence in the human data remain after accounting for acoustic, visual, and lower-level linguistic features correlated with the transcript embeddings? This distinction matters for determining whether the result specifically supports conserved semantic topography or could largely reflect shared responses to these correlated stimulus features.
  3. Can the spatial null tests reliably distinguish shared global organization from independent maps with comparable spatial statistics across scales? Since patch rearrangement can disrupt these statistics, how do the authors establish that the observed advantage reflects shared topography rather than spatial correlations that the null model does not preserve?
3 comments

We would like to thank reviewer tLYR for recognizing the depth and breadth of our experiments, and for noting that our analyses make a solid empirical case for our claims. We address each of the reviewer’s questions and concerns below.

The contribution relative to existing methods needs clearer positioning. The paper acknowledges that its matching objective is fused Gromov-Wasserstein. A closely related approach, FUGW (Thual et al., NeurIPS 2022), also combines functional features and cortical geometry for brain alignment, but is not discussed or compared. Clarifying this relationship would help distinguish the contribution of the matching formulation from that of the spatial-weight sweep, similarity summary, and null-testing procedure. The additional methodological or inferential value of these components remains to be established relative to existing approaches.

We thank the reviewer for bringing FUGW (Thual et al., 2022) to our attention. FUGW and TSMD share a similar formulation, in that both leverage fused Gromov-Wasserstein. However, they address different problems. FUGW is proposed as an optimization objective for landmark-free registration across subjects, to map one subject’s whole brain responses onto another’s. Moreover, they augment the fused objective by relaxing its marginal constraints, to account for functional areas varying in size across subjects. Instead, TSMD asks whether two populations (particularly responses within individual cortical areas such as V1\mathrm{V}1, V4\mathrm{V}4, IT) share a spatial organization of tuning at all. For this, we sweep over α\alpha, compare against a permutation null, and test across resamples. We now discuss FUGW in the introduction (Sec. 1).

The semantic specificity of the human results requires further clarification. The human analysis defines semantic tuning through encoding-model coefficients fitted to transcript embeddings during movie viewing. These embeddings may covary with acoustic, visual, and lower-level linguistic features. The spatial permutation controls assess the organization of the resulting tuning profiles, but do not separate these possible sources of correspondence. The results therefore provide evidence for cross-participant correspondence under the tested null models; additional controls would help determine how specifically this correspondence reflects shared semantic organization.

We agree that our results did not isolate semantic specificity of the response profiles. To test which properties of the movies drive the correspondence, we performed two additional controls (see new Appendix A8.1).

  • Does share topographic organization persist after accounting for visual, acoustic and simple linguistic features? We modeled visual content using 3232 PCA-whitened components of SlowFast-R50 features, acoustic content using 2020 mel-frequency cepstral coefficients (MFCCs), and basic transcript properties using word count, non-space character count, and speech presence. Using ridge regression, we estimated the contribution of these covariates to both the transcript features and each parcel’s BOLD response. We then refitted the encoding model to predict the residual responses from the residual transcript features (128128 PCA-whitened components). We selected ridge penalties using season 66 and fitted the final coefficients on all 287287 shared runs. The resulting tuning profiles showed greater cross-participant correspondence than both the full-permutation and smooth-patch nulls in all six participant pairs. For the 5050-parcel patch null, the mean observed-minus-null AUC difference ranged from (0.0421)(0.0421) to (0.0640)(0.0640) across pairs ((qFDR<3.1×10−5)((q_{\mathrm{FDR}}<3.1\times10^{-5}); Appendix Fig.A7).
  • Is shared topography also detectable directly in the measured responses? We applied TSMD using each parcel’s BOLD time course across the same 287287 runs, standardized within each run. All six participant pairs again exceeded both nulls. For the 5050-parcel patch null, AUC differences ranged from (0.0275)(0.0275) to (0.0385)(0.0385) (qFDR<1.6×10−4q_{\mathrm{FDR}}<1.6\times10^{-4}; Appendix Fig. A8). This establishes that shared response topography is detectable independently of the transcript embeddings and encoding model. Because the BOLD time courses reflect responses to the entire movie, this analysis does not identify which stimulus features account for that topography.

As a further test of robustness (see response to reviewer ejFr), we also repeated all analyses using median instead of max normalization of squared spatial distances and again found significant evidence of conserved topography across all 66 participants. We have revised the manuscript to clarify that tuning to features of the spoken dialogue shows a similar spatial organization across participants, even after accounting for the measured visual, acoustic, and linguistic covariates. The direct BOLD analysis supports the broader finding of shared movie-evoked response topography. We also acknowledge that the nuisance features capture only some properties of the movie, so the extent to which this shared organization specifically reflects semantic tuning remains an open question (Sec 3.4).

The spatial null models leave some ambiguity about what drives the observed correspondence. Patch permutations preserve within-patch organization but can alter continuity across patch boundaries and spatial correlations at larger scales. Differences from these controls could therefore partly reflect spatial statistics that the permutations do not preserve. It remains unclear how the procedure behaves on independent maps with comparable multiscale spatial structure but no shared global layout. Evaluating this case would help calibrate false-positive rates and clarify how specifically the reported effects support shared global organization.

We thank the reviewer for raising this point. To test maps with local smoothness but no shared global arrangement of tuning, we permute a synthetic orientation map and blur it with Gaussian kernels of increasing width (σ∈{0,0.5,1,2,4,8})(\sigma \in \{0, 0.5, 1, 2, 4, 8\}). This procedure restores local smoothness at a level of spatial scales, while the global layout stays unrelated to the original map. At σ=0\sigma=0, the original and null curves coincide (p=0.21)(p=0.21), as expected. For every σ>0\sigma > 0, TSMD between the original and blurred maps never exceeds a re-permuted null (p=1p=1; one-sided Wilcoxon signed-rank test) over K=15K=15 resamples, and the gap grows with increasing σ\sigma. Local smoothness alone therefore does not produce fall positives under TSMD. We have now added these results to Appendix A2.

Thank you to the authors for the thorough response and the additional analyses in the revised manuscript.

The revisions adequately address my main concerns. Specifically, the added discussion in Section 1 clarifies the distinction between FUGW-based cortical registration and TSMD's mesoscale hypothesis-testing framework. In Section 3.4 and Appendix A8.1, the nuisance-residualized encoding analysis and the direct BOLD time-course evaluation demonstrate that the human cross-participant correspondence persists beyond basic visual, acoustic, and speech covariates. Finally, the synthetic smooth-map control in Appendix A2 (Fig. A2), the nearest-neighbor similarity retention metrics in Appendix A8.1, and the added Limitations discussion in Section 4 properly calibrate the behavior and inferential scope of the spatial null models.

I have two minor suggestions for the final revision:

Harmonize the claims on semantic specificity in Section 3.4: There is still a slight tension between lines 427–428, which state without qualification that semantic tuning organization is conserved, and lines 456–458, which note that the extent to which the shared organization reflects semantic tuning remains open. Please align the wording in lines 427–428 (as well as in the Abstract and Discussion) with the more nuanced interpretation in the second paragraph.

Surface the patch-null continuity metric and check covariate counts: Reporting the percentage of nearest-neighbor similarity retained by the patch null (currently in Appendix A8.1, line 1118) briefly in the main text of Section 3.4 would help clarify how well local continuity is preserved. Please also verify the exact covariate dimensions in the text (the rebuttal text notes 50 visual components and 13 MFCCs, whereas Appendix A8.1 reports 32 and 20).

With these concerns addressed, I am raising my score.

Comments on the paper

1 comment

We thank all the reviewers for their generally positive feedback and constructive critiques. We have made a number of changes to the manuscript, which we hope alleviate most concerns. We have uploaded a new draft with all our changes highlighted in red, which we summarize below:

New analyses

  1. Local smoothness control (Appendix A2; tLYR): We test maps that are locally smooth but share no spatial organization with the original, by permuting a (synthetic) orientation map and then blurring it. These maps never score above the permutation null, indicating that local smoothness alone doesn’t produce false positives under TSMD.

  2. Anisotropic scaling (Fig 2B; 5KoC): We scale the spatial coordinates of a synthetic map anisotropically. With this, we find TSMD similarity decreases mostly at large α\alpha, but remains well above the permuted map.

  3. Semantic tuning (Sec. 3.4, Figs. A7–A8; tLYR): We regress visual features, acoustic features, speech presence, and text length out of both the dialogue embeddings and the BOLD responses, and refit the encoding model on the residuals. Shared topography remains significant across all six participant pairs against both nulls (qFDR<3.110×10−5)(q_{\mathrm{FDR}} < 3.110\times 10^{-5}). Shared topography also holds for raw BOLD responses without an encoding model (qFDR<1.610×10−4(q_{\mathrm{FDR}} < 1.610\times 10^{-4}; Fig. A8).

Clarification to methods

  1. Relation to prior work (Sec. 1; tLYR): We now discuss Thual et al. (2022), who use an unbalanced fused Gromov–Wasserstein objective to register cortical surfaces across subjects. Registration assumes that such a correspondence exists, while TSMD tests whether one exists at all.

  2. Retinotopy and the factorized encoding model (Secs. 2.4 and 3.3; ejFr): We moved the description of the factorized encoding model into the methods section, and kept the retinotopy control and its results after the macaque analysis. We also clarify in Sec. 3.5 that, as in the macaque analysis, retinal position is excluded when computing tuning similarity for the developmental models, so that shared retinotopy cannot drive the result.

  3. Distance decay comparison (Sec. 3.2; ejFr): We now describe Fig. 5C in the main text. It shows the standard distance-decay statistic, i.e., the tuning correlation between electrode pairs as a function of cortical distance. This agrees with our within-monkey results. However, it can only be computed within a single animal, since electrodes in different monkeys do not share a registered coordinate frame.

  4. Choice of distance (Sec. 2.2; ejFr): We clarify that the intra-population distances can be any biologically appropriate distance, such as geodesic distance on the cortical surface. All our experiments use squared Euclidean distance.

  5. Reading the α\alpha-axis (ejFr): We now state the two ends of the α\alpha-axis in the caption of every figure that sweeps α\alpha (α=0\alpha=0: tuning only; α→1\alpha \rightarrow 1: geometry only).

  6. Distance normalization (Sec. 2.2; 5KoC): Pairwise distances live on a different scale than the tuning cost, which lies in [0,2][0, 2]. We now explicitly state that we rescale each population’s squared distance matrix by its median off-diagonal entry. This is a key pre-processing step that allows the optimization routine to converge.

  7. Interpreting TSMD (Sec. 2.3; 5KoC): We now explain how to interpret TSMD. A practitioner should read TSMD through the observed-minus-null curve rather than its raw values. We propose TSMD as a tool to test for shared topographic correspondence rather than as a standalone distance, since raw values are hard to compare across species and datasets.

Other corrections

  1. Neuron coordinates now lie in RD\mathbb{R}^D rather than R2\mathbb{R}^2, since the human analysis uses 3D coordinates.

  2. Sec. 2.3 now refers to “neurons” rather than “electrodes”, since our test can be applied to any recorded unit.

  3. We corrected the description of layer selection. We select one layer per region and monkey, the one with the highest noise-corrected held-out correlation averaged across electrodes. The earlier text read as if we fit a separate layer per electrode, which is incorrect.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.