- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method for comparing the spatial organization of two neural representations. Unlike conventional representational similarity measures such as CKA, RSA, and Procrustes distance, the proposed Topographic Soft-Matching Distance (TSMD) jointly considers the similarity of neural tuning profiles and their relative spatial organization.
The proposed method is based on fused Gromov–Wasserstein optimal transport, which allows the comparison of representations while accounting for both functional tuning and pairwise spatial relationships.
The authors evaluate the proposed method using synthetic data and biological datasets, including the Allen Brain Observatory dataset and the THINGS Ventral Stream Spiking Dataset. The method is also applied to computational models of cortical topography.
The paper is generally well written, and the proposed method is intuitive and easy to understand. However, the reviewer has several concerns regarding its methodological properties and the interpretation of some experimental results, as discussed below.
Critical strengths
-
The proposed measure, presented in Section 2.2 (page 3), is conceptually simple, intuitive, and easy to understand. The combination of tuning similarity and relative spatial geometry provides a straightforward framework for comparing topographic representations.
-
The application of TSMD to biological datasets provides useful empirical support for the proposed framework. In particular, the comparisons across different species and cortical regions demonstrate its potential applicability to real neural recordings.
-
The use of synthetic experiments and permutation-based null models provides a systematic means of evaluating the proposed measure under different spatial organization conditions.
Critical weaknesses
The reviewer does not identify any fundamental conceptual weaknesses in the proposed framework. However, several concerns should be addressed.
- Normalization of the tuning and geometric costs
TSMD combines a tuning cost and a geometric cost using a weighting parameter . However, the effective contribution of each term depends on their relative numerical scales.
Therefore, it is important to clarify how the authors normalize the spatial distances and the tuning costs. In particular, the authors should explain whether the conclusions are robust to changes in the relative scales of these two terms.
This issue is especially relevant because the proposed evaluation integrates the similarity over different values of .
- Additional experiment on spatial scaling Related to the first concern, the reviewer suggests adding an experiment to Fig. 2 in which the original spatial representation is expanded or compressed.
In particular, it would be useful to investigate both uniform scaling and anisotropic scaling, where the expansion rates differ across spatial axes.
While uniform scaling preserves the relative geometric structure, anisotropic scaling changes the pairwise spatial relationships. Examining how TSMD responds to these transformations would help clarify its geometric properties and sensitivity to spatial distortions.
- Interpretation of the SOM experiments
The objective of a self-organizing map (SOM) is to preserve the topological relationships of high-dimensional data in its low-dimensional representation, rather than the absolute locations of individual representations. Consequently, different initializations can naturally produce different locations of the best-matching units (BMUs).
The authors report significant TSMD correspondence between independently initialized SOMs. However, the reviewer would appreciate a clearer explanation of what additional insights can be gained from these experiments.
In particular, does the observed correspondence indicate the conservation of a specific global spatial organization, or could it primarily reflect common statistical properties of maps generated by the same learning rule?
Clarifying this distinction would help establish the significance of the SOM experiments.
As an additional experiment, the reviewer suggests comparing representations generated by different dimensionality-reduction methods, such as SOM and t-SNE, using the same high-dimensional dataset. Since these methods employ different objectives and produce different spatial organizations, such a comparison could provide further insight into what TSMD measures. It would also help demonstrate the practical usefulness of TSMD for researchers interested in comparing representations generated by different methods.
Questions for authors
- Spatial sampling in the within-monkey experiments
In the within-monkey experiments, the electrodes from a single cortical region are divided into two disjoint subsets. Since these subsets may sample different portions of the cortical organization, how does their spatial sampling affect the measured TSMD similarity?
In particular, would the conclusions remain valid if the two subsets were spatially separated rather than randomly interleaved?
Although the authors provide subsampling and checkerboard controls, further clarification of how these controls address differences in spatial coverage would be helpful.
- Practical interpretation of TSMD
What practical insights can be gained from this study by researchers who need to compare two different neural representations?
In particular, how should practitioners interpret the magnitude of TSMD similarity or its AUC? Can the authors provide guidance on determining whether an observed difference reflects meaningful variation in spatial organization rather than differences in spatial scale, sampling density, or noise?
5 comments
Interpretation of the SOM experiments
We thank the reviewer for highlighting this distinction. Our experiment does not test whether independently trained SOMs place the same tuning at corresponding cortical locations. Rather, our question is whether independent SOMs nevertheless reproduce some of the “relative spatial arrangement of tuning”. For example, two maps could contain equally smooth orientation and ocular-dominance domains, yet differ in how those domains are arranged relative to one another. TSMD asks how closely units can be matched by their tuning while preserving the distances between them within each map. This comparison does not require any unit or pinwheel to occupy the same absolute location across runs.
We use the patch-permutation control to go beyond the observation that SOMs produce smooth maps. It retains tuning neighborhoods within patches while rearranging the patches on the sheet. Across 15 independent SOM pairs, the original maps showed greater correspondence than this control (; Fig. 8C). Both the SOM and the elastic net produced smooth maps with orientation pinwheels, yet we found no evidence of correspondence beyond the patch control when comparing the two models . We therefore interpret the SOM result as evidence that some spatial relationships among tuning domains recur across runs beyond what within-patch organization alone predicts.
We also acknowledge that the patch null does not preserve the complete spatial correlation structure so we cannot exclude the contribution of all generic spatial statistics to our evidence of shared topography. We now discuss this in the limitations section of our paper (L526-531).
Comparison against t-SNE
We thank the reviewer for this suggestion. Our focus here was to compare models of cortical development, particularly whether different learning rules produce similar spatial arrangements of tuning on a simulated cortical sheet. We note that the conventional application of t-SNE places supplied observations (e.g., images) in space, whereas our developmental models learn tuning at locations on a cortical sheet.
We see the use of TSMD to compare two-dimensional embeddings produced by methods such as PCA and t-SNE as a promising direction for future work. TSMD is well suited here because it can compare the relative positions of input examples without requiring the two embeddings to share a coordinate system (which is often arbitrary for these methods). We have noted this potential application in the Discussion.
Spatial sampling in the within-monkey experiments
Thank you for raising this. TSMD compares populations sampling the same or homologous cortical regions, either within or across individuals. We interleave the within-monkey split so that both halves sample the same region of the cortex, and test whether that region’s spatial distribution of tuning is significantly organized against our permutation null. Spatially separated halves would sample different parts of the cortical map, which need not share the tuning or the spatial organization of that tuning. To rule out correlated signals from neighboring electrodes driving the result, the checkerboard split assigns alternate electrodes to each half, and random subsampling controls for recording density. The cross-monkey comparison is the strongest test across separately sampled populations, and and remain significant there (see Sec. 3.2.2).
Practical interpretation of TSMD
We intend TSMD as a tool to test for shared spatial organization rather than as a raw distance. Since it is difficult to choose a value of a priori, we sweep and summarize the curve by its AUC. Although is expressed in units of tuning correlation, its magnitude and AUC depend on the distribution of tuning similarities across populations and, for , on the distribution and normalization of spatial distances within each population. We therefore encourage practitioners to interpret it relative to a permutation/patch null constructed for each dataset which preserves a population’s tuning distribution, sampling density, and noise, and only breaks the tuning-location pairing. We also encourage practitioners to plot the observed and null curves alongside the AUC to show where and by how much they diverge as the match gives greater weight to spatial distances. We note that a larger raw AUC in one dataset does not by itself imply more strongly conserved topography.
We handle spatial scale separately. Since each population's pairwise squared distances are rescaled by its median, TSMD is invariant to uniform spatial rescaling of either population's coordinates (see also our response on the normalization of the two costs). We demonstrate that the null test recovers known differences across species, with no topography in mouse (salt-and-pepper), and significant topography in macaque . We also document the invariance of TSMD to coordinate isometries in Appendix A7. We have added a sentence to this effect in Sec. 2.3.
We would like to thank reviewer 5KoC for finding the measure conceptually simple and intuitive, and for noting its applicability across species and cortical regions. We address each of their questions below.
More importantly, we would like to note that rescaling the geometric term by a constant does not change the set of couplings traced over α, as it simply reparametrizes α. Consider the loss Lα from Sec. 2.2, and write G(T) and C(T) for its geometric and tuning costs. If the geometric term is rescaled by some constant c>0, we have
αcG(T)+(1−α)C(T)=(cα+1−α)(α′G(T)+(1−α′)C(T)),α′=1−α+cαcα.
Note that α′∈(0,1) is positive and independent of T. Therefore, the optimal coupling at α under scale c is the optimal coupling at α′ in the original scale. Since we normalize each population by its own median, the relative scale of the populations is fixed, which makes TSMD invariant to rescaling either population’s coordinates (e.g., differences in unit or array size). This holds for mean, median or max scaling alike. We use the median because it is robust to a few outlying pairwise distances, which can dominate the mean or maximum. We also note that while the normalization preserves the family of couplings across the full α sweep, switching the normalizations (e.g., from median to mean) can change the AUC because the same couplings occur at different α values. We now state this explicitly in Sec. 2.2.
We tested this sensitivity directly in the CNeuroMod analysis by repeating the full sweep with median and maximum scaling. With median scaling, the observed and null curves separated at lower values. Under both normalizations, however, the observed AUC significantly exceeded both the full-permutation and smooth-patch nulls for all six participant pairs. Thus, the choice of scaling affected where the separation appeared, while the inference of shared topography held under both choices. We report median scaling in Fig. 7 and maximum scaling in Appendix Fig. A9.
Thank you for the suggestion. We have added this experiment to Fig. 2B. We now scale the coordinates of each neuron i in Xbase as ri=(xi,yi)↦(sxxi,syyi), keeping its tuning fixed. Uniform scaling (say sx=sy=λ) multiplies all squared distances in a population by λ2. Our median rescaling (see response to the first point) removes this factor, so TSMD is invariant to uniform scaling of either or both populations. Anisotropic scaling (sx=sy) changes relative pairwise distances, so neither α nor the median rescaling can absorb it. On our synthetic orientation map (Fig. 2), with sx/sy=2, we find that TSMD similarity stays nearly constant at small α, and decreases mostly at larger α, where the geometric term dominates. It remains well above the permuted map across α. We note that we intend TSMD as a test for conserved topography, rather than a raw distance. A moderately stretched map keeps the same neighboring neurons, so it should count as being conserved.