I believe that while the spirit of the paper is appreciable, it has major weaknesses that prevent me from recommending acceptance. These weaknesses, elaborated below, are particularly present in terms of (i) novelty, (ii) strength of the evidence and its interpretation, (iii) experimental soundness.
Novelty
The paper overlaps both in methodology and results with multiple prior works that are ignored or downplayed. For example, Ramasesh et al. [1] already perform layerwise CKA analysis, studying task semantics and their consequences on performance. Mirzadeh et al. [2] and Wenger et al [3] study forgetting and the role of model width, and Graldi et al. [4] directly connect scale, representation dynamics, and feature evolution via CKA measures. Most notably, however, are the similarities with Davari [5]: this prominent prior work already shows how internal representation influences performance-forgetting. A more diligent discussion would clarify the paper’s contribution, and — I am afraid — highlight the limited novelty proposed in the work.
Evidence and Interpretation
The main limitation of the results to me is the recurring comparison of similarity metrics across different setups, dimensions, and absolute values. In particular:
- It is very hard to interpret and judge if a “5% similarity survives” is actually a lot or not. Especially as the results are mostly aggregates across layers, disregarding the dimensionality of the layers, and thus it is hard to compare how the actual representation is evolving.
- Figure 2 is misleading in depicting accuracy and RSA on the same 0-1 axis as if the two quantities would be meaningfully related. It is misleading as it invites comparing the two absolute values while this is not possible. If one were to compare the relative RSA and relative accuracies from the baselines, I think the results would be even less convincing than they already are, and the surviving similarities even more questionable and hard to interpret.
- I don't find solid enough results and insights to interpret the surviving similarity as “the representation survives”: it could just be background noise fitted spuriously that hardly provides any meaningful and semantic representation in the model.
- For the reason above, I think it is very hard to interpret Figure 3: each block might have a different dimension, and therefore it is unclear to me if and how one should compare similarities across layers without confounding the similarity with the actual dimensionality varying. I think one should for sure compare mainly the relative scores (which is only done at times), but even then a more thorough (possibly statistical) analysis of the role of dimensionality would be needed.
Recent work on measuring geometric change as a proxy for the usefulness of model representations (Göring et al. [6]) further questions how CKA-style metrics can be interpreted in this context.
Experiments
At least some of my previous comments above could be dismissed with a strong, fair, and interventional experimental plan. This is however missing. Indeed, the only experiment proposed is the “relearning speed”, which, however presents significant limitations
- Firstly, it is completely arbitrary to set the threshold at 70% (note that in a plot a wrong label with 75% appears); unclear if the results would change (and how) when this threshold is higher. For example: would the advantage also reflect in final saturation performance?
- Secondly, but most importantly, the experimental setup does not guarantee a fair evaluation of the various baselines: hyperparameters are not tuned separately for each run, but rather kept constant, and therefore it is impossible to claim whether the shown relearning results are due to surviving information, or rather to better/worse tuning. The bare minimum would be to separately tune the runs on a common reasonable LR grid.
- Even with these further results, however, I am afraid the current experiments on re-learning would not be enough to convincingly show a cause-and-effect relationship between surviving representation and forgetting. Examples of stronger evidence are Ramasesh [1] and Davari [5].
References
[1] Vinay Venkatesh Ramasesh, Ethan Dyer, and Maithra Raghu. Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics. In International Conference on Learning Representations (ICLR), 2021.
[2] Seyed Iman Mirzadeh, Arslan Chaudhry, Dong Yin, Huiyi Hu, Razvan Pascanu, Dilan Gorur, and Mehrdad Farajtabar. Wide Neural Networks Forget Less Catastrophically. In Proceedings of the 39th International Conference on Machine Learning, PMLR 162:15699–15717, 2022.
[3] Jonathan Wenger, Felix Dangel, and Agustinus Kristiadi. On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective. arXiv preprint arXiv:2310.00137v2, 2024.
[4] Jacopo Graldi, Alessandro Breccia, Giulia Lanzillotta, Thomas Hofmann, and Lorenzo Noci. The Importance of Being Lazy: Scaling Limits of Continual Learning. In Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:20326–20367, 2025.
[5] MohammadReza Davari, Nader Asadi, Sudhir Mudur, Rahaf Aljundi, and Eugene Belilovsky. Probing Representation Forgetting in Supervised and Unsupervised Continual Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16712–16721, 2022.
[6] Niclas Alexander Göring, Charles London, Abdurrahman Hadi Erturk, Chris Mingard, Yoonsoo Nam, and Ard A. Louis. Disentangling Feature Learning from Generalization in Neural Networks. In High-dimensional Learning Dynamics (HiLD) Workshop at ICML, 2025.