- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose a way to measure the similarity between the internal representations of neural networks using probability distributions, rather than directly comparing activations or their similarity matrices. They compare the neural network representations through the prior predictive distributions induced by Gaussian linear readouts.
The neural network representations are compared using several divergences and distances between Gaussian distributions, including KL, Jensen–Shannon, Bhattacharyya, Hellinger, total variation, and Wasserstein. I found the idea very interesting.
To evaluate the proposal, the authors used six architectures, with multiple model instances, and three criteria including layer retrieval, layer-order consistency, and stability across image subsets.
Critical strengths
The paper is easy to read and the formulation is mathematically well motivated. The code is available and the implementation details contribute to reproducibility helping the reader.
I found the experimental evaluation to be reasonably broad, including multiple architectures, training configurations, and image subsets.
I liked the way stability is evaluated with respect to the choice and number of input images, particularly relevant for representation similarity methods.
The analysis of correlations between different similarity measures shows a very interesting way to understand redundancy among existing metrics.
Critical weaknesses
To me the novelty of the paper is limited as the central idea of comparing representations through Bayesian predictive distributions was already introduced in the authors' earlier work.
The previous paper also included experiments with ImageNet representations. If I understand correctly, the contribution here is an extension of the evaluation and the family of distances and does not present sufficient novelty to justify the paper.
I think the experimental conclusions are stronger than the reported results support. The results show only two of the six proposed variants outperform RSA in layer retrieval and this should be better evaluated in the paper. Seems that all proposed variants perform worse than several existing methods in the layer-order experiment.
I think the baseline comparison is not fully controlled. For instance, in the GULP case, it appears to use a regularization configuration that can lead to degenerate behavior, while CKA and distance correlation rely on biased estimators. A question for the authors, is the method's hyperparameter selected using the evaluation data?
Questions for authors
Which Gram matrix is actually implemented? the formulation presented in the main text or the centered formulation in the appendix? Will be interesting to understand how this choice affects the invariance properties.
Does the claimed advantage in layer retrieval remain after applying rank transformations? How much of the improvement can be attributed to the nonlinear scaling of the distances?
Please verify whether the statistical results in Figure 10 appear to have incorrectly assigned metric labels. This affects the interpretation of the significance tests supporting the layer-retrieval results.
Please verify the equations because there are mathematical and implementation ambiguities concerning the Gram matrix definition, the claimed invariances, and the expression for Jensen–Shannon divergence.
2 comments
Responses to Questions
-
We appreciate the reviewer flagging the inconsistency regarding double-centering. We implemented BayesCompare both with a double-centered Gram matrix in the feature dimension (appendix formulation) and without any double-centering (main text formulation). Our intention was to present results without double-centering. However, the initial results in the paper are computed with double-centering in the feature dimension (appendix formulation).We have verified that this preprocessing step does not alter the categorical conclusions, likely because most features have near zero-mean activations. All relevant formulations and figures have now been updated to match the main text (without double-centering). If the reviewer is interested, we can include the double-centered results in the Appendix, though both approaches yield highly similar outcomes.
-
BayesCompare metrics demonstrate the strongest performance in the layer retrieval task prior to rank transformation. After applying the rank transformation, Jaccard achieves the highest score (mean SS = 0.9797), with BayesCompare’s TVD closely following (mean SS = 0.9762), as reported in Table 1. This indicates that BayesCompare retains strong layer retrieval capability after rank transformation, while Jaccard shows substantial improvement specifically due to the transformation. This non-linear rank transformation appears particularly advantageous for Jaccard, as it moves from being the lowest-performing metric in the original setting to the highest after transformation.
-
We thank reviewer for their attention and we acknowledge this oversight. There has been a mixup in the names of the x-axis labels which is now fixed. We confirm that the results shown in Fig 10 in Appendix (now Fig. 11 in Appendix) are consistent with the results shown in Figure 3.
-
We confirm that the definitions regarding Gram matrices have been corrected, and the invariance claims should now hold. We have also revised the definition of Jensen-Shannon Distance for clarity. If there are any further ambiguities regarding Gram matrix implementations, we would be happy to address them specifically.
We thank the reviewer for their careful evaluation and constructive feedback.
Responses to Weaknesses
Novelty: While we acknowledge that [44] provides the theoretical foundation for our method, its scope remains purely theoretical, showing only demonstration-style examples without any formal evaluation. In contrast, in our work,
Conclusions and Supporting Evidence: We appreciate the chance to further defend our results. We report results for the different metrics one could apply in BayesCompare to provide guidance on which metric one should use within BayesCompare. We do not claim that all metrics work better than existing ones. Practically speaking, we expect that users of BayesCompare will choose their metric wisely, such that the performance of worse metrics does not matter. With the metrics we recommend in our discussion, all qualitative advantages we report can indeed be realized simultaneously.
We were also careful to point out that no single metric outperforms all others across every task consistent with Klabunde's ReSi [23]. Also, we acknowledge the good performance of the rarely used Jaccard distance. Nonetheless, the layer monotonicity and functional similarity findings varied strongly by architecture (even within vision modality). For instance, CKA yields the best results for monotonicity in EfficientNet but is outperformed in ResNet50. Within this cohort of varyingly successful metrics, BayesCompare seems like a good general purpose solution. It shows the best performance or closely follows the best performer in all of the experiments. Additionally, it comes with properties that are practically relevant, such as having differentiable proper metrics in the space of representations, which are highly stable across image samples. These properties should make BayesCompare particularly well suited for optimizing distances between representations (e.g. stimulus optimizations for explainable AI or teacher student setups for network training).
Baseline Control: For GULP, we included a regularized version because the unregularized variant (λ=0) is equivalent to a linear transformation of ridge-CCA, which is generally excluded from our comparisons (as noted in Section 2) due to its greater flexibility and suboptimality, as established in the literature. The original GULP paper [5] also notes that low regularizations primarily capture matrix rank, while higher regularizations can cluster layers by depth. Thus, the hyperparameter settings for GULP we evaluate correspond to the recommendations of the original authors.
Thank you for you suggestion of unbiased CKA. We included the unbiased CKA as a control for Exp1 and Exp2, and we are waiting on the Exp3 to finish computing. We, for now, added these results to the Appendix and we plan to include its results in the main text once all experiments are computed. Our results showed that while unbiased CKA shows an intermediate performance for Exp1, layer retrieval, it indeed shows the best performance in Exp2, layer monotonicity, in almost all architectures.
Evaluation data question The aim of our experiments is to evaluate the performance of BayesCompare over the range of possible hyperparameters i.e. metric and noise levels. We do not first optimize the hyperparameter values on one dataset and then evaluate the performance on another. We simply aim to show the users of BayesCompare which of the hyperparameter settings would yield a favorable result in certain use cases, such as layer retrieval, monotonicity or consistency. Therefore, the whole experimental suite can be considered as evaluation.