- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors consider the robustness of the model similarity metrics SVCCA and CKA. The authors approach this study by framing the problem as an adversarial optimization; can models be trained to exhibit both a desired functional, behavior and a desired similarity score to other models. The authors approach this problem experimentally by inducing representational similarity/difference in models that are functionally different/similar (respectively.
Critical strengths
- The paper addresses a timely problem, as the proliferation of models necessitates a study of model relationships.
- The ability to arbitrarily inflate and deflate CKA while maintaining a desired functional profile is an interesting finding.
Critical weaknesses
- CLAIM SCOPE: The claim scope of the paper should be tightened to reflect the actual set of experiments run. For instance, claims about representation fragility should be restricted to claims about SVCCA and CKA. The title of the paper should be updated to "On the fragility of SVCCA and CKA for model similarity, and associated discussion of representation metric fragility should be scoped to SVCCA and CKA specifically instead of making general claims.
- CLAIM CONSISTENCY: The abstract's claims are not consistent throughout the paper. In particular, L017 seems to contradict the finding in L365.
- EXPERIMENTAL DESIGN: The paper does not report the self similarity of different instances of baseline models on any plots. This makes it difficult to contextualize whether the differences in CKA and SVCCA are actually significant or not. This is particularly troublesome in Figure 5, where it is difficult to understand how the the CKA scores between the functionally unaligned models compare to the gap in CKA scores between functionally aligned models.
- EXPERIMENT REPORTING: Figure 4 needs to report the accuracy of the adversarial models, otherwise, there is no way to know if the adversarial models maintain the desired functional, behavior.
- NIT: The paragraph starting at L117 should explicitly state that this paper considers the deterministic model setting, as replications over model inference are not considered in the abstract formulation for model similarity.
- NIT: The resolution of figure 4 should be improved.
Questions for authors
N/A
Response to ejee
We thank reviewer ejee for their helpful comments. In this comment, we address the individual concerns raised by ejee.
Claim scope: This concern was raised by multiple reviewers and is addressed in our main comment above. In summary—we have run additional experiments on SVCCA and CKNNA and found that our approach generalizes well to these metrics.
Claim consistency: Thank you for noting this inconsistency regarding the SVCCA results which has since been resolved. We will revise the claims in the updated version of the paper.
Comparison against additional baseline models. We agree it is important to contextualize representational similarity against natural variation between independently trained base models and thank the reviewer for raising this concern. Motivated by this comment, we conducted additional benign-to-benign evaluations across ResNet-18, ResNet-50, and ResNet-152 on CIFAR-10, CIFAR-100, MNIST, and Tiny-ImageNet. For each architecture–dataset configuration, we measured CKA and prediction agreement between two independently trained checkpoints, using the same 512 test samples for each comparison.
Across datasets, benign-to-benign CKA varies substantially: 0.9935 on MNIST, 0.9598 on CIFAR-10, 0.7878 on CIFAR-100, and 0.7445 on Tiny-ImageNet. These results demonstrate that independently trained benign models naturally exhibit relatively high representational similarity in our evaluated settings. In contrast, our adversarial optimization can drive CKA toward zero while maintaining comparable task accuracy, even under parameter constraints. This establishes that the low CKA values observed in our experiments are not merely a consequence of natural variation between independently trained models, but can be deliberately induced through optimization. We also specifically investigate benign baselines for ResNet-50 on CIFAR-10, which matches the architecture used in Figure 5. For this setting, independently trained benign models exhibit a mean penultimate-layer CKA of 0.9598 ± 0.0047. This provides a reference for our functional-unalignment experiments which shows that while independently trained benign models exhibit high CKA and strong task performance, our optimization can maintain high CKA even as the adversarial model's task performance deteriorates.
Finally, we evaluated 100 pairs of independently randomly initialized models (which are untrained) per architecture. These models also exhibit high CKA, with means of 0.9434, 0.9622, and 0.9614 for ResNet-18, ResNet-50, and ResNet-152, respectively.
Accuracy reporting in Figure 4. We have an updated version of
Figure 4(Figure 34) that now includes the model accuracies.Additional points: We will address the other clarifications and readability concerns raised by the reviewer in camera-ready version. Thank you again for your helpful feedback.