- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work conducts a comprehensive empirical study on the probing quality of different layers and modules of vision transformers and LLMs. The conclusion is, in general, the optimal layer id and activation varies a lot across models and tasks, but most ViT prefers the last layer residual stream and the second normalization layer at 60% of LLMs are fairly good choice.
Critical strengths
The empirical study has a board and comprehensive sweep on vision transformer. And this paper also touched the LLM as well. For ViT, the linear probing is an effective way to evaluate the feature and representation learning, which is an important topic requiring detailed understanding.
Critical weaknesses
- The analysis on LLM is not comprehensive enough. Especially authors only focus on very toy multi lingual text classification task.
- Authors didn't provide enough in-depth analysis on the reason why the certain position is a better feature for the task. This makes this study fully empirical so that the conclusion hard to generalize.
Questions for authors
See weakness In addition, what do you think about the necessity of studying the probing potential of LLMs? Also, does the conclusions above relevant to model architecture (e.g. prenorm vs postnorm) and training objective?
4 comments
-
Impact of training strategies
Q2:
Also, does the conclusions above relevant to model architecture (e.g. prenorm vs postnorm) and training objective?The literature mainly relies on pre-norm architectures, which is why we chose to focus on them. While we agree that we cannot claim our results extend to post-norm models, such a study is out of the scope of this paper. Regarding the training objective, however, our comprehensive experiments cover different training strategies: supervised (ViT), self-supervised (DINOv2/v3), autoregressive (AIMv2) and contrastive (CLIP). This suggests that the benefits of a good probing strategy hold consistently across training methods.
-
Utility of LLM probing
Q1:
In addition, what do you think about the necessity of studying the probing potential of LLMs?
We thank the reviewer for this question. Linear probing is a protocol for evaluating how reusable frozen embeddings are: a linear classifier on top of frozen features measures how much task-relevant information the representation exposes without any finetuning. Studying where to probe is therefore, more broadly, studying where a model’s most reusable embeddings lie. This question is actively studied for LLMs, as shown by Skean et al. (ICML 2025), who analyse representation quality layer by layer. Our work identifies a simple fixed strategy, LN2@60, that improves embedding quality across a wide range of tasks and models without any retraining. To show that this goes beyond linear probing, we added MTEB experiments on STS, reranking, and clustering tasks, which involve no trained classifier at all: they rely directly on cosine similarity and k-means over the embeddings. The fixed strategy improves performance on STS, reranking, and clustering, as stated earlier. The choice of location thus also matters when embeddings are reused as they are; beyond linear probing, probing strategies transfer effectively to these settings
Thanks for the reply. Regarding the LLM benchmark selection, I understand that you followed the previous work, but this does not imply that is correct.
- Previous work might be not perfect as well
- The "proper" eval shifts with time. GenAI research is moving fast and crazy. For example, in LLM, frontier labs hill climb different benchmarks every year (or even every month),
Just want to share my view, the new eval looks better and helped to improve this paper, so I raised my score to weak accept.
Thank you!
We thank the reviewer for their insightful feedback and interesting perspective. We agree that appropriate evaluation evolves quickly for LLMs. We are glad that the additional evaluation helped improve the paper, and we thank the reviewer for the increased score.
Extend LLM experiments
W1:
The analysis on LLM is not comprehensive enough. Especially authors only focus on very toy multi lingual text classification task.We thank the reviewer for this feedback. Our LLM experiments are conducted on the MTEB benchmark, which is commonly used in the literature, whether in research papers such as "Layer by Layer: Uncovering Hidden Representations in Language Models" by Skean et al. (ICML 2025), "KV-embedding: Training-free text embedding via internal KV re-routing in decoder-only LLMs" by Tang et al. (ACL 2026), "Your mixture-of-experts LLM is secretly an embedding model for free" by Li et al. (ICLR 2025), or to evaluate industrial embedding models, such as Gemini Embedding and EmbeddingGemma. For consistency with the vision experiments, we focused on 24 classification tasks spanning 31 languages and 14 domains. To address the reviewer's concerns, we provide additional experiments beyond classification on 3 new types of tasks (29 additional tasks in total): STS (Semantic Textual Similarity), reranking, and clustering. We observe that our conclusions hold: the fixed strategy improves performance on STS (+2.9 for Qwen and +4.6 for Llama), reranking (+1.5 and +7.8), and to a lesser extent for clustering (+0.5 and +0.4). These new results are in the revised version section 3.3, Tables 3, 4 and 5 and below. Due to the time constraints of the rebuttal period, we were not able to run the oracle experiment but our current results highlight the interest of an effective and efficient probing strategy. If the reviewer believes this if of importance, we will add the oracle performance in the final version of the paper.
Explanation of why our strategy are trustable
W2:
Authors didn't provide enough in-depth analysis on the reason why the certain position is a better feature for the task. This makes this study fully empirical so that the conclusion hard to generalize.We thank the reviewer for this question. We first note that the additional experiments above show that our conclusions generalize to held-out models and tasks. We further discuss the reason why a certain position is a better feature for the task in the two parts below.
Why Act for VFMs. Act is the output of the nonlinearity in the expanded feed-forward block (dimension 4d). The same location was singled out by Ghiasi et al. (2022), who visualize ViT features and find that the outputs of the GELU layers in the feed-forward block are “significantly more interpretable than other layers”. They hypothesize that “the network exploits these high-dimensional spaces to store relatively disentangled representations”, whereas “compressing the features into a lower dimensional space may result in the jumbling of features”. Our results are consistent with this view: Act outperforms FC1, which has the same dimension but lies before the nonlinearity (see discussion in Appendix, Section C.1.7 and Table 23). This suggests that the nonlinearity, rather than the dimension alone, makes Act a better representation to probe. When averaged over patches, these activations summarize which features are present across the image.
Why LN2 at intermediate depth for LLMs. On depth, our finding is consistent with Skean et al. (2025) and with Lad et al. (2025), who describe the final layers of LLMs as specialized for next-token prediction, whereas intermediate layers build more general features. On the module, LN2 is a per-token normalized version of RC1. To test whether this normalization explains LN2’s advantage, we applied the same per-token normalization before pooling to all modules (Appendix C.2.3, Table 26 and below). Normalization improves most other representations, but no representation module outperforms LN2. Per-token normalization therefore explains part of LN2’s advantage, but not all of it, and the remaining part is an open question that we discuss in Appendix C.2.3.