- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces the concept of Neural Compatibility to evaluate relationships between neural representations. The authors argue that existing representation similarity measures (e.g., CKA, RSA) and model stitching methods mainly focus on whether representations are similar or directly usable by another model, while overlooking whether a downstream system can effectively utilize the representation after learning and adaptation. The paper defines compatibility as the performance after stitching followed by further fine-tuning, and provides theoretical examples showing that activity correlation, stitchability, and compatibility can produce different rankings of representations. Experiments on ResNet models further investigate the relationship between stitchability and compatibility.
Critical strengths
-
The paper raises an interesting question: whether neural representations should be evaluated based on their adaptability, rather than only similarity or immediate usability. This perspective is potentially valuable for understanding information transfer, modular neural networks, and representation reuse.
-
The experiments attempt to validate the existence of compatibility in real neural networks and analyze the relationship between stitching and adaptation, providing initial observations for studying representation transfer and adaptability.
Critical weaknesses
-
The motivation for introducing compatibility is not sufficiently differentiated from existing concepts such as transferability, fine-tuning efficiency, and representation transfer. Since compatibility essentially measures whether a representation can be effectively utilized after adaptation, its unique scientific value requires further justification.
-
The definition of compatibility heavily depends on the adaptation process, including the optimizer, learning rate, number of training steps, trainable parameters, and data size. Therefore, compatibility may not be an intrinsic property of a representation, but rather a property determined jointly by the representation, learning algorithm, and adaptation budget. The paper does not provide a sufficiently clear and stable definition.
-
The theoretical examples mainly demonstrate that different metrics can produce different rankings, but they are highly constructed and do not show whether such differences commonly occur in real neural networks. Moreover, the experiments suggest that ranking reversals between stitchability and compatibility are relatively rare, raising questions about the practical importance of compatibility.
-
The experiments are mainly conducted on ImageNet-pretrained ResNet models, which limits the generality of the conclusions. It remains unclear whether compatibility can extend to more complex foundation models, such as Transformers, large language models, multimodal models, or biological neural representations.
Questions for authors
-
What is the fundamental difference between compatibility and existing notions of transferability or fine-tuning performance? If compatibility ultimately measures adapted task performance, why should it be considered a new concept rather than a form of transfer evaluation?
-
Since compatibility depends on the specific adaptation strategy and budget, how should compatibility be fairly compared across different representations?
-
Why are ranking reversals between stitchability and compatibility relatively uncommon in real ResNet experiments? If compatibility captures an important new property, why is this phenomenon not observed more frequently?
-
Can compatibility be extended to unsupervised learning, self-supervised learning, or adaptation processes in biological neural systems? Is the current supervised fine-tuning based definition too restrictive?
Rebuttal Reviewer 4
W1: Addressed in general response.
W2: We believe it is essential that the measurement is explicit about the learning algorithm and budget. The sensitivity of the measurement ensures conclusions are drawn with learning algorithms and budget in mind similar to how a researcher should consider the implications of the chosen decoder family when comparing representations with decoder probing.
W3: Addressed in general response.
W4: Addressed in general response.
The answers to your questions:
[answers:] Our results so far have shown that rank order violations are rare between ResNets, likely because ResNets have highly similar block structures that are repeated over skip connections and share a similar training history. We theorize that more architectural differences between models will yield more variation in neural compatibility, since their mechanisms for feature extraction influence representational format. are more inclined to specific features. For example, position information is implicit in CNN architectures while injected in ViT architectures.
Yamini Bansal, Preetum Nakkiran, and Boaz Barak. Revisiting model stitching to compare neural representations. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 225–236. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/01ded4259d101feb739b06c399e9cd9c-Paper.pdf