acceptodds
Under review as a conference paper at ICLR 2027

On the Fragility of Representational Similarity Metrics

Abstract

Representational similarity metrics are widely used tools that measure correspondences between machine learning (ML) models' learned representations. We show that these metrics can be brittle and raise important questions about whether they provide meaningful notions of model similarity. To do this, we introduce an adversarial optimization framework that manipulates the parameters and thus the representation space of an adversarial target model relative to a reference benign model. Using this framework, we show that we can inflate or suppress representational similarity metrics, such as Centered Kernel Alignment (CKA) and Singular Vector Canonical Correlation Analysis (SVCCA), in the adversarial model while keeping its parameters and task performance close to those of a benign model. We can also make the adversarial and benign models functionally unaligned but keep their representational similarity high and parameters similar. Across five datasets and eight architectures, our findings show that high functional similarity between models does not necessarily imply high representational similarity between them, and vice versa.

Primary area:
interpretability and explainable AI
Keywords:
Representational Similarity, Representation Learning, Adversarial Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ejeeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors consider the robustness of the model similarity metrics SVCCA and CKA. The authors approach this study by framing the problem as an adversarial optimization; can models be trained to exhibit both a desired functional, behavior and a desired similarity score to other models. The authors approach this problem experimentally by inducing representational similarity/difference in models that are functionally different/similar (respectively.

Critical strengths
  • The paper addresses a timely problem, as the proliferation of models necessitates a study of model relationships.
  • The ability to arbitrarily inflate and deflate CKA while maintaining a desired functional profile is an interesting finding.
Critical weaknesses
  • CLAIM SCOPE: The claim scope of the paper should be tightened to reflect the actual set of experiments run. For instance, claims about representation fragility should be restricted to claims about SVCCA and CKA. The title of the paper should be updated to "On the fragility of SVCCA and CKA for model similarity, and associated discussion of representation metric fragility should be scoped to SVCCA and CKA specifically instead of making general claims.
  • CLAIM CONSISTENCY: The abstract's claims are not consistent throughout the paper. In particular, L017 seems to contradict the finding in L365.
  • EXPERIMENTAL DESIGN: The paper does not report the self similarity of different instances of baseline models on any plots. This makes it difficult to contextualize whether the differences in CKA and SVCCA are actually significant or not. This is particularly troublesome in Figure 5, where it is difficult to understand how the the CKA scores between the functionally unaligned models compare to the gap in CKA scores between functionally aligned models.
  • EXPERIMENT REPORTING: Figure 4 needs to report the accuracy of the adversarial models, otherwise, there is no way to know if the adversarial models maintain the desired functional, behavior.
  • NIT: The paragraph starting at L117 should explicitly state that this paper considers the deterministic model setting, as replications over model inference are not considered in the abstract formulation for model similarity.
  • NIT: The resolution of figure 4 should be improved.
Questions for authors

N/A

1 comment
AuthorsOpenReview ↗

Response to ejee

We thank reviewer ejee for their helpful comments. In this comment, we address the individual concerns raised by ejee.

Claim scope: This concern was raised by multiple reviewers and is addressed in our main comment above. In summary—we have run additional experiments on SVCCA and CKNNA and found that our approach generalizes well to these metrics.

Claim consistency: Thank you for noting this inconsistency regarding the SVCCA results which has since been resolved. We will revise the claims in the updated version of the paper.

Comparison against additional baseline models. We agree it is important to contextualize representational similarity against natural variation between independently trained base models and thank the reviewer for raising this concern. Motivated by this comment, we conducted additional benign-to-benign evaluations across ResNet-18, ResNet-50, and ResNet-152 on CIFAR-10, CIFAR-100, MNIST, and Tiny-ImageNet. For each architecture–dataset configuration, we measured CKA and prediction agreement between two independently trained checkpoints, using the same 512 test samples for each comparison.

Across datasets, benign-to-benign CKA varies substantially: 0.9935 on MNIST, 0.9598 on CIFAR-10, 0.7878 on CIFAR-100, and 0.7445 on Tiny-ImageNet. These results demonstrate that independently trained benign models naturally exhibit relatively high representational similarity in our evaluated settings. In contrast, our adversarial optimization can drive CKA toward zero while maintaining comparable task accuracy, even under parameter constraints. This establishes that the low CKA values observed in our experiments are not merely a consequence of natural variation between independently trained models, but can be deliberately induced through optimization. We also specifically investigate benign baselines for ResNet-50 on CIFAR-10, which matches the architecture used in Figure 5. For this setting, independently trained benign models exhibit a mean penultimate-layer CKA of 0.9598 ± 0.0047. This provides a reference for our functional-unalignment experiments which shows that while independently trained benign models exhibit high CKA and strong task performance, our optimization can maintain high CKA even as the adversarial model's task performance deteriorates.

Finally, we evaluated 100 pairs of independently randomly initialized models (which are untrained) per architecture. These models also exhibit high CKA, with means of 0.9434, 0.9622, and 0.9614 for ResNet-18, ResNet-50, and ResNet-152, respectively.

Accuracy reporting in Figure 4. We have an updated version of Figure 4 (Figure 34) that now includes the model accuracies.

Additional points: We will address the other clarifications and readability concerns raised by the reviewer in camera-ready version. Thank you again for your helpful feedback.

Reviewer_M9NLOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates whether representational similarity and task performance can be decoupled while networks remain close in parameter space. It optimises a target network against a fixed reference using a similarity objective, a task loss, and a parameter distance constraint. The experiments report low CKA with similar classification accuracy, and high CKA with low accuracy. They cover five datasets and eight architectures. SVCCA changes less under the reported optimisation, while CKA responds to the proposed research questions well.

Critical strengths
  1. The research topic is interesting. Parameter proximity alongside representation similarity and task performance is a useful experimental question. The paper reports results with a base and adversarial model pair and examines interventions at different target layers.

  2. The study includes mainstream networks with multiple datasets and three runs for the main experiments. The architecture and constraint ablations provide investigations for assessing how broadly the observed behaviour occurs.

Critical weaknesses
  1. Model selection and coverage. Figure 2 uses ResNet-18, Figure 3 specifies ResNet-152 in its caption, and Figure 5 uses ResNet-50, without explaining the choice for each setting. Thus, comparing the unconstrained and constrained examples in Figures 2 and 3 also changes the architecture. Section 3.1 further describes Figure 3 as ResNet-18, contradicting its caption. These make the comparison unfair. A common-architecture comparison of the core settings, or reporting performance on all three architectures, is needed to assess consistency.

  2. Feature dimensionality and task redundancy. The image tasks have 10, 100 or 200 classes, while the released default configurations use 512-dimensional penultimate features for ResNet-18 and 2048-dimensional features for ResNet-50/152. The paper does not report the feature dimensions and effective ranks for each experiment or vary the representation bottleneck. Consequently, it does not establish how much the accuracy-preserving CKA reduction depends on directions unused by the classifier. Please test reduced feature dimensions and simple feature-transformation baselines under comparable accuracy and parameter-distance budgets. This would address the role of redundancy more directly than adding architectures alone.

  3. Strength and scope of the parameter constraint. The final distances of 24.4 and 29.7 on CIFAR-10 and CIFAR-100 do not show whether changes are small in the target layer or classification head. Moreover, the released penalty uses named_parameters(), excluding BatchNorm running statistics. Please define the aggregation consistently, report layer-specific relative changes, control these model states, and show the similarity-accuracy tradeoff as the distance budget tightens. The current results do not establish how restrictive the constraint is for task-preserving changes.

  4. Classification-head controls for RQ2. Similarity is measured before the classifier, but the optimisation setup does not freeze or share the classification head; the released optimiser includes its parameters. Freezing the feature extractor and permuting the classifier's output classes preserves the hidden representation exactly while degrading classification. This also preserves maximal SVCCA for identical nondegenerate representations. A baseline that changes only the head under the same distance budget, and an experiment with a fixed shared head, are needed to distinguish the reported effect from damage to the downstream classifier. Without these controls, RQ2 and its metric comparison do not isolate a failure to measure representation similarity.

Questions for authors
  1. Which architecture actually produced Figure 3, and why do the main experiments switch ResNet variants?

  2. How do CKA and accuracy change over tighter budgets, and how much do the target layer and classifier change individually?

  3. Can changing only the classification head reproduce RQ2 under the same distance budget? What happens when both networks use the same fixed classifier? Please report the changes in the head and feature extractor separately.

5 comments
AuthorsOpenReview ↗

Response to M9NL

We thank reviewer M9NL for their helpful suggestions. Here we address individual concerns raised by them.

Model selection and coverage: This concern was raised by multiple reviewers and is addressed in depth in our main comment above. In summary—we selected the models presented in the paper to showcase the generalization of our findings across architectures and will clarify this choice in the camera-ready version of the paper.

Strength and scope of the parameter constraint: Per the reviewers' suggestions, we have extended our analysis to examine component-wise relative parameter changes and BatchNorm running-statistic differences across CIFAR-10, CIFAR-100, and Tiny-ImageNet using ResNet-152 and ResNet-18. Relative change is defined as

∥θa−θb∥2∥θb∥2×100%.\frac{\|\theta_a-\theta_b\|_2}{\|\theta_b\|_2}\times 100\%.

As shown in Table 3 and Table 4, ResNet-152 exhibits small overall relative parameter changes across all three datasets when we train for functional alignment and low cka. While a slightly higher percent of changes concentrate in the layer before CKA is measured and in the classification head, parameter changes remain distributed throughout the whole model. This confirms that parameter modifications are not concentrated to a particular region of the model.

ResNet-18 exhibits larger relative parameter changes than ResNet-152, particularly in its BatchNorm running statistics. Importantly, our original regularizer operates on named_parameters() and therefore does not directly constrain BatchNorm running means and variances. We measured these inference-time buffers separately and will explicitly clarify this distinction and include the corresponding results in the revised manuscript.

Table 3: Relative parameter changes between benign and adversarial ResNet-152 models across three datasets. Relative change is computed as ∥θa−θb∥2/∥θb∥2×100%\|\theta_a-\theta_b\|_2 / \|\theta_b\|_2 \times 100\% for each component. This is between the models used to present results for current Figure 3 in our paper.

ComponentCIFAR-10CIFAR-100Tiny ImageNet
All parameters0.64%0.63%0.37%
Classification head4.57%2.49%1.99%
Final residual stage (layer4)1.24%1.19%0.58%
Other parameters0.04%0.03%0.02%
BN running statistics (float)5.16%4.15%3.27%

Table 4: Relative parameter changes between benign and adversarial ResNet-18 models across three datasets. Relative change is computed as ∥θa−θb∥2/∥θb∥2×100%\|\theta_a-\theta_b\|_2 / \|\theta_b\|_2 \times 100\% for each component. This is between the models used to present results for Figure 35.

ComponentCIFAR-10CIFAR-100Tiny ImageNet
All parameters4.25%6.13%4.35%
Classification head8.95%7.81%7.29%
Final residual stage (layer4)3.42%5.91%3.44%
Other parameters5.99%6.83%6.61%
BN running statistics (float)91.61%93.84%94.28%

Feature dimensions: We appreciate this suggestion for exploring how feature dimensions may impact the ability to degrade CKA while maintaining accuracy. We are actively running experiments on this and plan to share results shortly in a follow-up comment.

AuthorsOpenReview ↗

Role of feature dimensions in the optimisation

"...does not establish how much the accuracy-preserving CKA reduction depends on directions unused by the classifier...": We would like to thank the reviewer for posing this very interesting question. To understand the asked for behavior, we trained 3 different ResNet50 models on CIFAR10. We obtained a benign model (with around 89% accuracy), an adversarial model using our unconstrained framework, where the classifier head is not fixed and a second adversarial model with a fixed head (from the benign model). Both these adversarial models have around 87% accuracy.

To understand the dependance of our results on the directions in feature space that are important or unimportant for classification, we perform a singular value decomposition of the features for the penultimate layer. As expected, since there are 10 classes, we find 10 non-trivial singular values. We project the features onto i) the 9-dimensional subspace spanned by the right singular vectors corresponding to the non-trivial singular values (important subspace), and ii) the subspace spanned by the right singular vectors corresponding to the remaining trivial singular values (unimportant subspace). We then compute the CKA between the projected features and find that:

  • The CKA between the feature representations of the benign and adversarial models in the important subspaces is high in both the malleable and frozen head settings (~ 0.86).
  • The CKA between the unimportant feature representations of the benign model and the important subspace of the adversarial model is high (~0.81) in both cases.
  • The CKA between the important feature representations of the benign model and the unimportant subspace of the adversarial model is very low in both cases.
  • The unimportant subspaces have very low CKA between them in both cases.

Thus, what our optimisation does to reduce CKA while maintaining classification accuracy is to align the features that were unimportant to the benign model to become important to maintain accuracy. It keeps the overall CKA low by ensuring that the erstwhile important features are extremely unaligned with the unimportant subspace of the adversarial model. As noted by the Reviewer, the redundancy is indeed important to achieve our goals. We will add this discussion and further experiments across datasets to the updated version of the paper.

Dear Reviewer M9NL,

Thank you once again for your review. We would greatly appreciate a response to our rebuttal and hope that we have addressed any lingering concerns about the paper.

Regards, The Authors

Reviewer_utUYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines whether representational similarity scores reliably reflect similarities in model behavior. It proposes an adversarial optimization framework that changes a target model’s parameters relative to a fixed reference model, jointly optimizing a representation similarity metric, classification loss, and, in some experiments, parameter distance. The authors seek two types of counterexamples: models with comparable task accuracy but low representational similarity, and models with high representational similarity but substantially different task accuracy. Experiments span five datasets and eight architectures. The results show that CKA can be driven toward low or high values even under parameter constraints, whereas SVCCA is less responsive to the tested optimization procedures. The paper concludes that representational similarity scores alone are insufficient evidence of functional similarity.

Critical strengths

The paper offers a clear counterfactual framework for testing similarity metrics. By jointly optimizing representational similarity, task loss, and parameter distance, the authors examine both comparable task accuracy with dissimilar representations and similar representations with different task performance. This provides a more direct test of what a similarity score means than an observational comparison alone, and offers a method that could be applied to other metrics. The evidence for CKA covers several settings and includes a meaningful parameter constraint. The authors report substantial changes in CKA across multiple datasets, architectures, and target layers. Figure 3 further shows low CKA at the target layer while the models’ parameters become much closer and their test accuracies remain comparable. This strengthens the CKA-specific finding beyond a single architecture or a fully unconstrained experiment.

Critical weaknesses

The claim about the fragility of representational similarity metrics is broader than the evidence supports. The SVCCA experiments do not reproduce the main CKA result: SVCCA changes little when task performance is maintained and generally decreases when performance deteriorates. Nevertheless, the abstract groups CKA and SVCCA together as examples of metrics that can be inflated or suppressed. With only two metrics tested and no thorough SVCCA-specific optimization sensitivity analysis, the evidence supports a narrower conclusion about CKA under the tested conditions, rather than a general conclusion about representational similarity metrics.

Questions for authors

Can the authors provide a more direct measure of functional agreement and clarify the control for Figure 5? What is the prediction agreement, or output distance, between models described as functionally aligned? At which layer is CKA measured in Figure 5? Would changing only the classifier while leaving that layer’s representation unchanged produce a similar high-CKA, low-accuracy result? How was the effectiveness of SVCCA optimization assessed? Were the loss weights, learning rate, and dimensionality-reduction settings tuned specifically for SVCCA, and was optimization convergence checked? The answer would help distinguish resistance of the metric from a failure of the tested optimization procedure to change it. Section 3.1 states that Figure 3 uses ResNet-18, whereas the Figure 3 caption identifies ResNet-152. Which architecture was actually used, and were the reported accuracy, parameter-distance, and CKA results all obtained with that architecture? This clarification is necessary to interpret and reproduce the parameter-constrained experiment.

3 comments
AuthorsOpenReview ↗

Response to utUY

We thank Reviewer utUY for their helpful feedback. Here, we address individual concerns they raised.

Claim scope: This concern was raised by reviewer ejee as well and is addressed at length above in the shared concerns. To summarize, we revisited SVCCA and added experiments on CKNNA. In both cases, our updated results show both metrics being sensitive to our optimization. To make SVCCA work, we resolved an issue where we were not handling gradients correctly. Otherwise, we did not need to disentangle differences in optimization parameter settings, as they were held constant across attacks.

Better measure of functional alignment: We appreciate the helpful suggestion to test functional alignment by measuring agreement (or disagreement) between models' classification of the same set of test images. We tested this alternative measure on the models presented in Figure 3 of the paper (minimizing CKA with constrained parameters while maintaining functional alignment). Here, the benign/adversarial CIFAR10 models agree on 96% of test images, CIFAR100 models agree on 87%, and Tiny ImageNet models agree on 87%. We can add this metric as a suggested alternative for functional alignment and report results for other settings in an updated version of the paper.

Figure 5 Clarification. CKA was measured at the penultimate layer for Figure 5.

Architecture inconsistencies. Thank you for pointing this out. The architecture used to produce Figure 3 in the main body is ResNet152, and we have updated the paper body accordingly. For completeness, we have included results for a ResNet 18 version of this figure in Appendix B (Figure 35).

Classification head controls: Please see the common response for our experiments regarding this. In summary, we find that changing only the classifier does indeed lead to a low accuracy, high CKA result. In addition, both i) keeping the classifier fixed and only changing the representation, and ii) modifying both jointly (as our initial experiments did), also lead to functional unalignment while maintaining representation alignment.

Dear Reviewer utUY,

Thank you once again for your helpful review. We believe we have addressed all of your concerns in the updated paper, common rebuttal and the rebuttal specific to you. Please let us know if any concerns remain.

Regards, The Authors

Dear Reviewer utUY,

Thank you once again for your review. We would greatly appreciate a response to our rebuttal and hope that we have addressed any lingering concerns about the paper.

Regards,
The Authors

Comments on the paper

4 comments
AuthorsOpenReview ↗

Common Response: Part 1

We first thank the reviewers for their helpful feedback. We are glad the reviewers agree that our paper addresses a timely and interesting research question (M9NL, ejee), that there is a clear need for robust similarity metrics to characterize model relationships (ejee), and that our methodology directly tests the meaning of these similarity metrics (utUY). We also appreciate that the reviewers found our core results compelling—e.g. inflating and deflating CKA while maintaining functional characteristics (ejee) and manipulating CKA under parameter constraints (utUY)—and acknowledged the breadth of our evaluation across many modalities, architectures, and optimization constraints (utUY, M9NL).

Note: We have added to the manuscript a rebuttal section (Appendix B: Rebuttal Figures) with only relevant rebuttal figures at the very end after the current appendix. The main manuscript will be updated at a later stage.

We now turn our attention to concerns raised by the reviewers. In this comment, we focus on reviewers' shared concerns and respond directly to reviewers addressing their individual concerns.

Claim Scope (Reviewers utUY and ejee): Reviewer utUY was concerned that the main claims of our paper did not extend to SVCCA, while reviewer ejee noted that our claims were limited only to CKA and SVCCA, not representational similarity metrics broadly. To address these concerns, we have run additional experiments on SVCCA and CKNNA and found that our optimization framework readily extends to both metrics.

SVCCA. Per utUY's suggestion, we investigated why our original optimization failed on SVCCA and found that the SVCCA implementation we were using did not propagate gradients correctly through the metric. We fixed this issue and rerun the SVCCA experiments reported in Tables 1 and 2 (a wide selection with sufficient coverage to validate our main findings), holding all else fixed. As is clear from the tables below, SVCCA is vulnerable to our optimization attacks in both the min and max directions, targeting both functional alignment and unalignment. Our attack does not target specific characteristics of SVCCA, so we expect that further adjustments to the training procedure would only strengthen the attack.

CKNNA. We also ran experiments on the CKNNA metric using ResNet-152 trained on CIFAR-100 in the constrained min direction targeting the penultimate layer. Our results show that this metric too can be pushed towards 0 when up to 400 layers are constrained under our optimization framework. We have included a figure with these results as Figure 36 in Appendix B, a new Rebuttal Figures section of our manuscript.

Targeting multiple layers at once. On top of this, we also ran experiments in which we attempt to minimize CKA in multiple target layers at once. We test this on ResNet-152 architecture trained on CIFAR-10 and find that we can successfully deflate CKA to below 0.5 in three target layers when up to 240 other layers are constrained. We can deflate CKA below 0.5 in 2 of the 3 target layers when up to 467 layers are constrained, with a smaller drop in the third target layer. See Figure 37 in Appendix B of our manuscript.

Generalization to other metrics. With these updated results, it is clear that our optimization methodology applies readily to three widely-used representational similarity metrics. In general, our optimization framework can be applied to bounded differentiable metrics, but assessing the fragility of specific metrics requires additional analysis.

Clarification of model selection in Figures 2, 3, and 5 and Section 3.1 (Reviewers utUY and M9NL): Section 3.1 states that Figure 3 uses ResNet-18, whereas its caption identifies ResNet-152. We apologize for this mistake. The figure was produced with ResNet-152, and we will correct the text of Section 3.1 in the camera-ready version. The reported accuracy, parameter distance, and CKA results in Figure 3 were all measured on ResNet-152. We also now have an additional Figure 35 where we reproduce Figure 3 with ResNet-18.

AuthorsOpenReview ↗

Common Response: Part 2

Classification-Head Modifications (Reviewers M9NL and utUY): We thank the reviewers for identifying the need to disentangle changes to the feature extractor from changes to the downstream classifier. We conducted additional experiments on CIFAR-10 using ResNet-18, comparing head-only, and fixed-head, and joint attacks under matched parameter-distance budgets of 5, 10, 20, and 40, as well as unconstrained head-only and fixed-head attacks. BatchNorm running statistics were held fixed.

Without parameter-distance constraints, the head-only attack achieves 10.00% accuracy with CKA = 1.0000 (head/other parameter distances: 145.61/0). Most importantly, the fixed-head attack reduces accuracy from 87.30% to 6.96% while maintaining CKA = 0.96 (head/other parameter distances: 0/316.91), thus establishing that both changes to the feature extractor and the classifier head can result in RQ2 being answered in the affirmative.

For the original joint attack, accuracy falls to 10.00% at budgets 20 and 40, with head vs. other parameter distances of 11.23/8.77 and 32.89/7.11, respectively. At budgets 5 and 10, the classifier remains unchanged, and accuracy is 87.37% and 85.47%, respectively.

In the constrained setting, the head-only attack reduces accuracy to 10.00% while maintaining CKA = 1.00 (as expected because the feature extractor remains unchanged), with no changes in other parameters. In contrast, when both models share the same fixed classifier, accuracy drops by a few percentage points to 85.89–87.89%, while CKA ranges from 0.97 to 0.99 and there is no change in the classifier parameters.

These controls confirm that classifier-only changes can reproduce the high-CKA, low-accuracy result under matched distance budgets. However, the unconstrained fixed-head experiment further demonstrates that substantial functional degradation can also arise from changes to the feature extractor alone while maintaining high representational similarity.

Table 1: Unconstrained Functionally Aligned SVCCA minimization and maximization penultimate layer.

DatasetObjectiveAvg. L2 weight distance to benignSVCCAAccuracy (%)
Epoch 1Epoch 50Epoch 1Epoch 50Epoch 1Epoch 50
MNISTMin23.0034.920.340.2260.1399.58
Max23.0233.250.470.9152.6299.14
CIFAR-10Min28.0640.030.320.2438.8187.83
Max28.0639.650.350.8243.2684.75
CIFAR-100Min31.6443.650.340.249.1864.64
Max31.6443.660.360.7510.4568.22
AuthorsOpenReview ↗

Common Response: Part 3

Table 2: Constrained Functionally Aligned SVCCA experiments using ResNet-18 CIFAR-10 as the number of layers constrained (NN) increases. We report metrics at the beginning and end of model training (at epoch 50). Results are averaged over three independent runs per setting.

LayerNNMinMax
Avg. L2SVCCAAccuracy (%)Avg. L2SVCCAAccuracy (%)
Early18.64 → 0.000.46 → 0.2242.36 → 90.548.75 → 208.300.80 → 0.8541.40 → 90.41
108.88 → 0.320.46 → 0.3840.57 → 90.179.61 → 190.680.80 → 0.8443.54 → 90.38
2010.23 → 0.150.45 → 0.4041.79 → 90.0311.27 → 168.100.80 → 0.8542.17 → 90.24
3011.03 → 0.090.46 → 0.4141.53 → 89.1412.07 → 107.720.80 → 0.8542.36 → 90.28
4016.36 → 0.070.47 → 0.4241.32 → 86.1717.78 → 89.310.80 → 0.8543.70 → 90.12
5023.36 → 0.050.46 → 0.4338.98 → 85.6425.24 → 74.290.80 → 0.8444.01 → 90.29
6026.95 → 0.040.46 → 0.5133.78 → 87.8429.06 → 77.040.80 → 0.8443.71 → 90.42
6226.23 → 0.040.46 → 0.5233.86 → 88.1128.27 → 74.520.80 → 0.8443.77 → 90.26
Middle18.62 → 0.000.39 → 0.1641.17 → 90.058.64 → 197.060.65 → 0.7444.02 → 90.21
108.74 → 0.000.38 → 0.1641.29 → 90.069.72 → 180.980.65 → 0.7443.40 → 90.15
2010.17 → 0.000.40 → 0.1642.03 → 89.7011.27 → 158.800.65 → 0.7443.77 → 90.12
3011.15 → 0.070.39 → 0.5440.50 → 88.9411.93 → 92.670.65 → 0.7443.91 → 90.12
4016.46 → 0.020.40 → 0.6141.03 → 87.2217.65 → 70.540.65 → 0.7444.13 → 90.21
5023.43 → 0.020.39 → 0.6140.27 → 86.8125.09 → 62.190.65 → 0.7443.64 → 90.28
6027.00 → 0.000.39 → 0.6733.63 → 89.3028.91 → 69.970.65 → 0.7444.87 → 90.08
6226.28 → 0.000.40 → 0.6734.36 → 89.5628.12 → 68.300.65 → 0.7444.16 → 90.28
Penultimate18.60 → 0.010.32 → 0.2437.57 → 87.568.68 → 138.500.34 → 0.8141.72 → 83.73
108.73 → 0.000.32 → 0.2439.30 → 87.439.70 → 108.850.34 → 0.8142.38 → 83.94
2010.17 → 0.000.32 → 0.2438.55 → 87.7111.26 → 77.060.34 → 0.8142.25 → 83.75
3010.97 → 0.000.32 → 0.2437.98 → 87.9812.02 → 59.880.34 → 0.8241.32 → 84.30
4016.32 → 0.010.32 → 0.2437.67 → 88.1117.71 → 42.590.34 → 0.8142.05 → 83.87
5023.30 → 0.010.32 → 0.2537.25 → 89.7525.16 → 51.000.34 → 0.8242.54 → 84.33
6026.90 → 0.020.32 → 0.9832.37 → 89.6128.99 → 48.120.34 → 0.8141.84 → 84.32
6226.18 → 0.020.32 → 0.9832.41 → 89.7628.20 → 47.810.34 → 0.8143.22 → 83.80
AuthorsOpenReview ↗

Updates To the Paper

We thank the reviewers for taking their time to provide valuable reviews and raising important questions which led us to understand the problem in greater depth. We are glad that the reviewers found the topic interesting (M9NL, ejee) and the experiments clear across multiple datasets and architectures (utUY,M9NL). We have tried to answer all of the concerns raised by the reviewers in the rebuttal and have updated the figures and writing in our paper to reflect the same. Here we present a concise list of changes to the paper:

List of changes

  • Main Body (Changes are highlighted in blue):
    • Figure 4: Added accuracy for both benign and adversarial models
    • Section 3.3: Updated to reflect that our framework can be extended to other representation similarity metrics like SVCCA and CKNNA.
  • Appendix We added the Rebuttal Figures( Appendix B) section which now reflects:
    • Figure 33: Plots for Constrained Functionally Unaligned SVCCA experiments on differnt datasets using ResNet-18
    • Figure 34: This figure is same as figure 4.
    • Figure 35: Effect of our adversary on CKA of ResNet-18 when all layers are constrained across different datasets. This mirrors the experiment in Figure 3 for a different classifier architecture.
    • Figure 36: Fragility of CKNNA for ResNet-152 on CIFAR100 under our optimisation approach
    • Figure 37: Effectiveness of our framework when multiple layers are targeted at the same time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.