acceptodds
Under review as a conference paper at ICLR 2027

Are In-Context Images Worth 10 Dimensions?

Abstract

There has been significant work on understanding the In-Context Learning capabilities of Large Language Models, especially on the induction circuit. For a few-shot classification task, the induction circuit leverages linear representations of each labeled example in-context in order to classify an unlabeled query. However, few works focus on how those linear representations are built in the first place. Leveraging the expressivity of the vision modality compared to text, we uncover a Shared Discriminative Geometry (SDG) inside Large Vision Language Models (LVLMs). It is a low-dimensional space, shared across all image classification tasks, in which in-context images are compressed into linearly separable representations later used to perform classification. We observe that this is the result of the model performing a dimensionality reduction of vision representations in early layers. In order to explain this phenomenon: (1) We show analytically that linear self-attention can perform a dimensionality reduction by projecting in-context data onto its principal components, with each layer implementing one gradient descent step toward this objective. (2) We provide evidence that trained LVLMs reduce the dimensionality of vision representations in early layers via a similar mechanism.

Primary area:
interpretability and explainable AI
Keywords:
In-Context Learning, Vision-Language Models, Mechanistic Interpretability, Representation Space Geometry

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_NqE4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work, a mechanistic study, investigates how vision-language models (VLMs) structure image representations during few-shot classification. Specifically, the authors uncover a Shared Discriminative Geometry (SDG)—a low-dimensional subspace of hidden-state directions that consistently underpins classification across distinct vision tasks. Causal intervention experiments confirm that these directions directly steer model outputs.

Critical strengths
  1. The paper discovered a useful observation about information hidden by the default similarity measure. Raw dot-product classification on the forerunner states performs poorly, while a learned compact metric substantially improves classification.
  2. The evidence from the main three-shot settings supports a functional contribution of the identified directions.
  3. The authors examine randomized support labels, different shot counts, four-way/four-shot training, and probes trained on individual domains.
Critical weaknesses
  1. The experimental design fails to validate the proposed dimensionality-reduction mechanism. Candidate heads are selected solely by how well their outputs align with the learned SDG, whereas the theoretical algorithm explicitly requires computing the similarity error (X^TX – Z^TZ). Furthermore, ablating these heads degrades classification accuracy even when the probe retains all 128 dimensions. This indicates that the ablation indiscriminately damages general discriminative features, suggesting that the model actually relies on a higher-dimensional representation rather than the proposed low-rank subspace.
  2. The paper needs to re-consider the novelty of the work and make comparison against the previous transformer–PCA research. For example, in the paper Learning Spectral Methods by Transformers already constructs transformers whose forward passes approximate PCA through power iterations, while this paper builds a linear-attention development. So, it is rational to add the comparison with prior studies.
  3. The three main intervention settings do not support the claim that the forerunner tokens are necessary representations. In the appendix, the paper showed that removing top 15 SDG, removing top 15 directions from forerunner and removing all 128 directions will reduce the prediction accuracy. This shows that the directions matter somewhere in the distributed computation but provides weaker evidence that the specific forerunner state is necessary.
Questions for authors

Appendix B.1 states that support and query images are distinct within an episode. I did not find a clear account of held-out images, held-out classes, or the separation between projection-training and evaluation episodes. Could the authors explain more about how to avoid the label leakage?

3 comments
AuthorsOpenReview ↗

Response to Reviewer NqE4

Dear Reviewer NqE4,

We sincerely appreciate your detailed comments on our work. Your thorough review and helpful recommendations will greatly enhance the manuscript.


[Weakness 1a]: Head selection via XTX−ZTZX^TX - Z^TZ

It is true that we select candidate heads based on the OV circuit, whereas using the QK circuit similarity error should be a valid convincing selection criterion. So why the OV circuit?

First, we do not have an exact estimation of the X nor the Z subspace used by the model making the estimation of XTX−ZTZX^TX - Z^TZ finicky, especially because it is a difference. The OV matrix in Eq (9) has a simpler theoretical formulation that is easier to identify.

Second, by selecting heads by their OV score and ablating only their cross-image attention, we validate 2 theoretical predictions at the same time. Selecting heads based on XTX−ZTZX^TX - Z^TZ and ablating the attention used for selection would focus both steps on the QK side.

We think that using XTX−ZTZX^TX - Z^TZ would make the paper’s argument hard to follow and the experiment less reliable. As stated in the paper, our contribution is not identifying exactly the theoretical circuit, but showing the observed dimensionality reduction is possible and that the main mechanism driving it is sample-to-sample attention.

[Weakness 1b]: Ablation degrades all 128 dimensions.

We appreciate your excellent intuition on that question. This needed an additional experiment for clarification.

As you note, the paper ablation reduces accuracy both with the top 10 dimensions and with all 128 dimensions. The 128D drop reflects another phenomenon we already observe in the model without ablation. The accuracy of the remaining SDG dimensions (dimensions 11 to 128) degrades across layers from 90.9% at the input layer to 53.5% at ℓ∗−1\ell^\ast-1. This explains why, in the ablation, the remaining 118 dimensions do little to compensate for the loss of accuracy in the top 10 dimensions. (see Appendix F.2)

[Weakness 2]: Transformer PCA Literature.

We understand your concern and the novelty of our theoretical circuit deserves clarification, specifically regarding the paper "Learning Spectral Methods by Transformers" on arXiv. The authors construct a more sophisticated transformer circuit involving 3 independent steps (Symmetrization, Power Iterations, Removal of Principal Eigenvectors) using MLPs and Attention. Their transformer is capable of constructing principal directions of input data. Our theoretical circuit is much simpler. It differs by only projecting in-context data onto its principal components using only one attention layer for the update. We added a discussion of the relevant PCA literature to the Related Work section of the revised paper.

[Weakness 3]: Forerunner SDG Necessity.

If we reformulate your concern: In Table 10, ablating the SDG dimensions across all tokens costs 11.4 accuracy points, whereas doing the same only for the forerunner token costs 0.5 points. Our interpretation in the paper is that other tokens may carry the SDG information and restore it at the forerunner between layer ℓ∗\ell^\ast and the last layer. However, we agree that in the current paper there is no evidence to support that interpretation.

That is why we designed a simple experiment to show the necessity of the forerunner SDG for the in-context classification task. To that end, we ablate Text SDG only for the forerunner token but from layer ℓ∗\ell^\ast to the last, to prevent any copying of SDG information onto the hidden states. This has a catastrophic effect on model accuracy (-27.9% in 3-shot vs +1.8% for the control random subspace ablation). Details in Appendix E.1.

[Question 1]: Label leakage.

This is a fair question. We made sure that a query image is never in the support set and that we sample train and evaluation episodes with different seeds. However, in the main paper setup, the probe training and probe evaluation image pool are the same. Our probe plays the role of an oracle in the image pool (152,929 images from 10 datasets). This design choice was made to disentangle probe transferability and is important to support some of the paper’s conclusions. When showing that low-dimensional SDG accuracy at the input layer is lower than at layer ℓ∗\ell^\ast, we show that our oracle probe found a projection at ℓ∗\ell^\ast that separates classes better than the one it found at the input layer. If we use different images or classes at evaluation, observing that accuracy gap could still leave open whether a different input-layer projection could better separate classes.

Thank you for highlighting this point. We agree that SDG probe transferability is a key claim of our paper, we report results on transfer to unseen classes and domains in our response to Reviewer TEPr Question 1 or Appendix C. The Text SDG transfers well to unseen classes, reaching 90.6% NCC accuracy, just 1.3 percentage points below an oracle probe trained on those classes.

I thank the authors for the detailed rebuttal. Most of my previous concerns are addressed. Therefore, I am happy to raise my score to weak accept.

Thank you for your thoughtful review and for taking the time to reconsider your assessment. Have a great day!

Reviewer_HnJyOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies in-context few-shot image classification (ICL) in large vision-language models (LVLM). The authors find that class information is mainly encoded in an approximately 10-D subspace shared by visual and forerunner tokens, which they call Shared Discriminative Geometry (SDG). Classification in this subspace can outperform the original model, while removing key dimensions will reduce accuracy. Their analysis shows that inter-sample attention first compresses visual representations while preserving class information, then transfers it to the forerunner token. The authors also provide an explanation for this process and identify the attention heads responsible for dimensionality reduction. Experiments across models and tasks show that SDG benefits both image and text classification.

Critical strengths
  1. The paper identifies an intersting mechanism and demonstrates it with good empirical evidence. Contrary to the common view that the forerunner token takes visual information for classification, the authors show that it barely separates classes in the original feature space. Instead, classification relies on an approximately 10-dimensional SDG. Further ablations isolate the roles of visual SDG, textual SDG, and visual-to-text attention. Together, these results demonstrate that SDG is essential to classification rather than an engineering-driven setup.

  2. The authors show that NCC classification depends on inter-sample similarities. The show that linear non-causal attention can preserve these similarities through gradient-based updates with local exponential convergence. Guided by this analysis, they use BDE to identify attention heads. When inter-sample attention in these heads is removed, the classification accuracy gains from using SDG disappear. Although this does not establish that trained LVLMs implement the exact theoretical circuit, the agreement between theory and intervention provides a useful explanation of how SDG may emerge.

Critical weaknesses

The main concern is that the central claims rely heavily on linear probes trained on top of a frozen LVLM. These probes define the Text and Vision SDG, estimate the effective dimensionality, identify important attention heads, and determine the directions. Although the authors provide controls across subspaces, training settings, and models, the evidence remains tied to the learned projections. This raises an issue with the ablations - removing dimensions identified by a learned projection does not necessarily mean removing the corresponding directions used by the model itself, since the projection may mix or rotate the original features. The results therefore support the discriminative value of SDG, but do not fully establish that it is the internal mechanism underlying the model’s classification ability.

Questions for authors
  1. See Critical Weaknesses
  2. Are the projection directions consistent with the model's native subspace?
  3. Table 2 shows that ablating the SDG improves performance on matching tasks. Is the SDG necessary only for classification tasks, or If removing SDG improves matching performance, does this mean that SDG is useful only for certain tasks, rather than being a general basis for classification?
2 comments
AuthorsOpenReview ↗

Response to Reviewer HnJy

Dear Reviewer HnJy,

Thank you for your thoughtful review and for highlighting the agreement between our theoretical predictions and the ablation results. Your well-reasoned critiques and helpful suggestions provide valuable guidance for improving the manuscript.


[Question 1 and 2, Weakness 1]: Are the projection directions consistent with the model's native subspace?

It is true that finding a probe with high accuracy does not guarantee that the model relies on the probe’s subspace [1]. This is a limitation for most interpretability work based on probes. This is why the paper first starts with an ablation in page 5 before proceeding into the analysis in order to show that the geometry at least contributes to the model’s ability to classify.

However, we provide in Appendix E.2 independent evidence that the model may use the SDG natively. First, to get closer to the model's native subspace, we look at attention head weights. Specifically we look at how accurately attention heads look at in-context examples. At ℓ∗\ell^\ast, Text SDG accuracy is 91.5% (xq⊤Φℓ∗⊤Φℓ∗xi)\bigl(x_q^\top \Phi_{\ell^\ast}^\top \Phi_{\ell^\ast} x_i\bigr) and the accuracy of the top 5 best heads is 85.4% (xq⊤WQ⊤WKxi)\bigl(x_q^\top W_Q^\top W_K x_i\bigr). Now, if the model did not use the learned Text SDG natively, removing it from the QK bilinear similarity would have a small impact on the head classification performance. However the top 5 heads’ accuracy drops to 52.3% (chance 33.3%).

Head similarity at ℓ∗\ell^\ast (3-shot, chance 33.3%)Top-5 headsAll heads
Unchanged85.460.2
Text SDG removed52.343.9
Random 128-dim subspace removed85.360.2

[Question 3]: Is SDG useful only for certain tasks?

We really appreciate the reviewer’s interest in Table 2. This is the result we’re most excited about.

In-context capabilities are known to emerge during pretraining, and the SDG is observable in Base model checkpoints before instruction tuning. While classification provides a test bed for studying the SDG, its emergence during pretraining suggests that it may have a broader function. We think that more generally, the SDG helps the model to attend and retrieve context relevant to the query.

This is hard to test but we designed a small prototype experiment to investigate that. We boost the SDG on LongBench, a long-context, text-only retrieval benchmark involving retrieving the relevant Wikipedia article given an abstract. Across 500 tasks, average accuracy increased from 88.4% to 91.8%. It is impressive that a geometry learned on an image classification task transfers to such a different setup.

While we do not claim that the SDG is the general mechanism for retrieving query-relevant context, we are excited and plan to pursue this direction by studying how to adapt our meta-training to long-context language tasks such as coding. For more details, see Appendix H.1. We will add a reference to this discussion in the main text.


Once again, thank you for the time you have devoted to our paper and for your excellent review. It has greatly improved the quality of our work. We look forward to further discussion with you.

[1] Elazar, Y., Ravfogel, S., Jacovi, A., and Goldberg, Y. (2021). Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals. TACL.

Dear Reviewer HnJy,

Can you please take a look at the authors' response and provide your further input? Thank you.

AC.

Reviewer_cEMPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how Large Vision-Language Models (LVLMs) form image representations for in-context few-shot classification. In a 3-way, 3-shot setting, the authors compare pooled visual-token representations with text-side forerunner states at the position where a label will be generated. They train layer-specific projections and use nearest-centroid classification to identify a low-dimensional Shared Discriminative Geometry (SDG), with an effective dimension of about 10 in the main model. Layer-wise analyses and interventions are used to argue that visual information is compressed and then transferred to text-side states; an idealized linear-attention construction gives a Gram-matrix-matching account of dimensionality reduction. The evidence combines learned probes, behavioral interventions, and a simplified theoretical model. The authors acknowledge that the theory does not directly identify the full circuit in trained LVLMs.

Critical strengths
  1. The paper studies how the representations needed for visual in-context learning are formed, a less explored question than how the model uses demonstrations at the final prediction stage.

  2. It combines layer-wise representation analyses with targeted interventions and a mathematically analyzable model, and tests the SDG hypothesis across multiple tasks and LVLMs. This gives the proposed mechanism more empirical grounding than a probe result alone, while leaving room to sharpen the causal interpretation.

Critical weaknesses
  1. The text-side analysis is affected by causal-prefix asymmetry. A support forerunner at position i is conditioned on a different prefix of earlier images and labels than a later support forerunner, while the query forerunner sees all nine support examples. The block design randomizes class order within each three-example block, which mitigates a fixed class-position correlation, but it does not equalize the prefix length or context across support states. As a result, Text SDG/NCC may reflect sequence position and preceding context in addition to class-relevant visual information, and class centroids pool states measured under different histories. Robustness across support permutations and position- or prefix-matched analyses would be needed to support a stronger representation-level interpretation.

  2. The mechanistic interpretation and cross-task generalization are stronger than the current uncertainty reporting supports. Ablating attention from layer-15 vision tokens to layer-16 text tokens reduces accuracy by 5.2 percentage points. This supports a functional contribution of that path, but by itself does not establish that it is the principal or necessary route. The main results are generally reported as task averages (e.g., 1,000 tasks for the 3-shot evaluation), without systematic confidence intervals, paired tests, or multiple training seeds for the key differences. The cross-task effects are also heterogeneous: the stated trend that SDG ablation helps matching tasks has an exception on NaturalBench, where ablation lowers accuracy by about 2 points. These results call for more qualified mechanistic and task-level claims.

Questions for authors
  1. How stable are Text SDG/NCC results when the same support images are evaluated under multiple permutations? Could you report position-stratified or prefix-matched analyses to separate class information from causal context and support position?

  2. Could you report task-level confidence intervals or paired tests, and seed variability where applicable, for the principal representation and intervention effects? In particular, how uncertain is the 5.2-percentage-point visual-to-text attention ablation effect, and what does it establish beyond a contribution of that edge?

  3. For the SST-2 transfer result, please make the sampling protocol easy to verify: which data split and sampling pool were used, how many examples or episodes were evaluated, and whether class balance or example difficulty was controlled? If these details are in the supplement, please point to the relevant section or table.

  4. Please sharpen the theory’s novelty relative to PCA and prior similarity/Gram-preserving embeddings and transformer gradient-descent analyses. Is the main new result the specific Gram-matching objective, its exact realization as a linear-attention update, and/or the convergence guarantee under the stated assumptions?

  5. Table 2 appears to show that SDG ablation improves three matching benchmarks but decreases NaturalBench accuracy by about 2 points. Is this exception robust to task-level uncertainty, and should the broad classification-versus-matching interpretation be qualified?

4 comments
AuthorsOpenReview ↗

Response to Reviewer cEMP

Dear Reviewer cEMP,

We sincerely appreciate your thorough review and valuable feedback on our manuscript. Below, we provide our responses to the points you have kindly raised.


[Question 1, Weakness 1]: Stability under permutations.

We thank the reviewer for this excellent question. In our protocol P(class | position) is uniform at every position. Using a symmetric bilinear similarity and optimizing for probe accuracy should reduce reliance on positional information for NCC classification. However, it is true that adding an experimental control on how position affects classification accuracy is relevant to the paper.

Position-stratified: For each position in the prompt, we classify the forerunner state at that position by NCC in the Text SDG subspace at layer ℓ∗\ell^*. The class centroids are computed from all other examples of the episode, both before and after it. Accuracy per position is 92.6±1.6%92.6 \pm 1.6\% (mean ±\pm std across positions), so Text SDG shows little dependence on position. Interestingly, your intuition is right for the raw forerunner token space (no projection) that does depend on position: 6.8% at position 0 and 24.8% for block 1, against about 44% afterwards. The SDG projection removes that dependence.

Support-order permutations: We evaluate the same support images and query under 5 support orders (all following the block design) and measure the query Text SDG NCC accuracy at layer ℓ∗\ell^*. Accuracy is 91.3±0.4%91.3 \pm 0.4\% (mean ±\pm std across orders), so Text SDG results are stable under permutations of the support examples. Of course, some positional variation is inherent to causal LLMs and cannot be fully eliminated. We added those results in Appendix B.3.

[Question 2, Weakness 2]: Task-level confidence interval and visual-to-text attention ablation effect

Thank you for raising this point. We agree that Table 1 should report task-level uncertainty, since its ablation results are central to our interpretation of vision-to-text transfer. We reran all Table 1 ablations on 5000 tasks. The table now reports the accuracy change with its 95% confidence interval in the revised paper.

  • (1) Text SDG at ℓ∗−1\ell^*-1: −3.5±0.6-3.5 \pm 0.6 (old was −3.2-3.2)
  • (2) Vision SDG at ℓ∗−1\ell^*-1: −6.5±0.7-6.5 \pm 0.7 (old was −6.4-6.4)
  • (3) Attention from vision at ℓ∗−1\ell^{\ast}-1 to text at ℓ∗\ell^*: −5.2±0.7-5.2 \pm 0.7 (unchanged)
  • Text SDG at ℓ∗\ell^*: −11.2±0.9-11.2 \pm 0.9 (old was −11.4-11.4)

Specifically, the vision-to-text attention ablation effect is now 5.2 ± 0.7 points. Overall numbers changed only a little and the paper’s conclusions should be unaffected.

[Question 3]: SST-2 transfer protocol.

We revised the SST2 transfer protocol in Appendix H.2 by adding all the asked details. Feel free to check the reproducible code (included in the supplementary) if any details remain unclear. In short: we sample 5,000 2-way episodes from the biggest split (training split) with class balance controlled and no difficulty control. The Text SDG is meta-trained only on image episodes.

[Question 4]: Transformer PCA Literature.

We understand your concern and the novelty of our theoretical circuit deserves clarification, specifically regarding the paper "Learning Spectral Methods by Transformers" on arXiv. The authors construct a more sophisticated transformers circuit involving 3 independent steps (Symmetrization, Power Iterations, Removal of Principal Eigenvectors) using MLPs and Attention. Their transformer is capable of constructing principal directions of input data. Our theoretical circuit is much simpler. It differs by only projecting in-context data onto its principal components using only one attention layer for the update. We added a discussion of the relevant PCA literature to the Related Work section of the revised paper.

Overall, our main theoretical contribution is to show that a single linear-attention layer can implement an exact gradient step of the Gram-matching objective and to prove convergence under the stated assumptions.

[Question 5, Weakness 2]: Should the classification-versus-matching interpretation be qualified?

To check whether the classification-versus-matching interpretation is robust to task-level uncertainty, we compute p-values for the four following hypotheses: (i) SDG Ablation hurts classification, (ii) SDG Boost helps classification, (iii) SDG Ablation helps matching, and (iv) SDG Boost hurts matching. For each hypothesis, we use an exact one-sided paired McNemar test on the per-task predictions, pooling all tasks of the same type.

InterventionClassificationMatching
SDG Ablationhurts, p=3.2×10−15p = 3.2 \times 10^{-15}helps, p=9.4×10−3p = 9.4 \times 10^{-3}
SDG Boosthelps, p=6.8×10−9p = 6.8 \times 10^{-9}hurts, p=3.6×10−4p = 3.6 \times 10^{-4}

All four hypotheses are supported (p<0.01p < 0.01). The NaturalBench change is within task-level noise (two-sided p=0.62p = 0.62). We will clarify this in Section 6.

Dear Reviewer cEMP,

Can you please take a look at the authors' response and provide your further input? Thank you.

AC.

Thank you for the detailed response and the additional experiments. The position-stratified and support-order analyses substantially alleviate my concern about the robustness of Text SDG NCC accuracy. The updated confidence intervals, the clarified SST-2 evaluation protocol, and the explanation of the theoretical contribution also address several of my original questions.

I retain some reservations about the strength of the interpretation. I view the vision-to-text attention ablation as evidence for a functional contribution of this path, while its status as the principal transfer route remains less firmly established. Similarly, the pooled tests support an overall trend across the evaluated matching tasks, with the NaturalBench effect remaining inconclusive. I encourage the authors to reflect these qualifications in the manuscript, particularly in Sections 4.2 and 6.

Overall, the rebuttal has alleviated my main concerns, and I maintain my weak acceptance recommendation.

Thank you for your thoughtful feedback and support. We are glad that the additional experiments and clarifications have addressed your main concerns. As promised, we have revised Section 6. We have also revised Section 4.2 to address your remaining reservations. Have a great day!

Reviewer_TEPrOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper dives deeper into how large vision language models (LVLMs) maintain a representation of in-context samples for image classification. It is motivated by a surprising observation - Using a nearest centroid classifier on image tokens works, but building it on the forerunner tokens leads to near random accuracy. The core insight from the paper is that early layers help do dimensionality reduction and the later layers transfer them to text for ICL. The aper proposes this concept of a Shared Discriminative Geometry (SDG) subspace, which is a subspace shared across all classification tasks. The paper claims that this is a low dimensional space and makes classes linearly separable.

Empirical and theoretical analyses are provided to support the paper’s claims. The paper runs experiments over a combination of 10 different image classification datasets (~2000 classes), in a 3-way, 3-shot classification setup.

Critical strengths
  1. The paper is well motivated. It studies an important problem of trying to bring more insight on how in-context learning works and explains design choices like the focus on image tokens vs text tokens. This is helpful to understand the usefulness of the problem and potentially could guide future research on building separable visual representations.
  2. The paper focuses on diving deeper into the core insights which I really appreciated. It talks about how the text SDG formation is a two step circuit, involving compression followed by knowledge transfer. This helps understand the individual insights better which accumulate to the main claim.
Critical weaknesses
  1. One of the core insights in the paper is that the SDG is low dimensional across all image classification tasks. I think this claim needs further corroboration. 1a. The current setup involves 10 datasets which generally cover different domains each. Since the SDG is claimed as a shared subspace, it is unclear how the size of the universe of classes would affect the effective dimensionality. Currently the paper reports ~2000 classes, and it would be interesting to understand whether this claim will hold if you had something like ImageNet-21k which has ~21k classes. Would SDG still be low dimensional and retain its ability to separate classes using NCC? 1b. Another potential way to further corroborate that SDG is low dimensional is to increase the number of classes in the support set. The paper currently focuses on 3 way and 4-way classification, but would the SDG be low dimensional even as the number of classes increase (e.g. 20-way, 30-way classification) is unknown.
  2. The way that the paper measures whether the dimensionality in SDG is 10 is through participation ratio. However, the participation ratio is computed over the learned probe weights, and not the model activations. In its current form it is an indication of how many dimensions the similarity function depends on and also is dependent upon the training of the probe adapters. Furthermore, my understanding of participation ratio is that it does not give a hard count as it is dominated by the larger eigenvalues. So in this paper the useful SDG dimensionality may not exactly be 10, as the causal experiments show that effective dimensions may be 15-20. This makes the paper title and introduction imprecise.
Questions for authors
  1. Since the compressed representation of the SDG depends on the training, what does the transfer to unseen classes look like? Given that the paper claims the SDG is shared across all image classification tasks, it would be helpful to understand how the learned probes can transfer to unseen classes at test time. The paper does touch upon it a bit in B.3, but a more thorough investigation is needed given the broad claim of the paper.
2 comments
AuthorsOpenReview ↗

Response to Reviewer TEPr

Dear Reviewer TEPr

We sincerely appreciate your careful and insightful review. Your constructive comments and suggestions are invaluable and will meaningfully improve the quality of our manuscript. We address each of your points in detail below.


[Weakness 1.a]: Training on ImageNet-21k.

If we fit straightforwardly the SDG using 21k classes from ImageNet-21k, then the 3-way classification task would be too easy. It is easy to find 10 directions that discriminate well a car from a dog and a flower from a plane. That is why in our evaluation protocol, we first choose randomly one of the 10 datasets then sample a task from it. Here it is much harder to find 10 shared dimensions that discriminate well a car from another car and at the same time a dog from another dog.

However, it is true that we could have used in the paper the ImageNet-21K hierarchy to create fine-grained tasks by sampling classes from the same label branch, resulting in a meta-training way more diverse. This is why we created synthetically 130 fine-grained domains from ImageNet-21k, and trained the SDG on it. The resulting Text-SDG effective dimensionality remains low (10.34)

The training setup of the paper transfers remarkably to the ImageNet-21K setup. At layer ℓ∗\ell^*, the main-paper Text-SDG reaches 80.5%, within only 1 accuracy point of the Oracle SDG trained on ImageNet-21K (81.5%)! Details were added in Appendix C.2 with additional tests on a finer-grained setup of 211 domains.

[Weakness 1.b]: Would the SDG be low dimensional even as the number of classes increase.

Your intuition was right. We trained the SDG while increasing the number of ways up to our hardware limit at 20 ways (3-shot). When increasing the number of classes the effective dimensionality of the Text SDG increases. (See Appendix B.2)

SettingText SDG effective dimension
3-way10.22
5-way11.44
10-way13.5
20-way15.26

When the number of classes grows 6.7×, the SDG’s effective dimension grows 1.5×, from 10.22 to 15.26. On one hand, the geometry stays low-dimensional through 20-way, even with 60 high dimensional images in context. On the other hand it weakens the 10 dimensional claim, which brings us to your next point.

Weakness 2: This makes the paper title and introduction imprecise.

We agree that not a single fixed round number can describe the size of the geometry. As stated in the paper participation ratio was chosen out of practicality (see Fig. 8 for the SDG eigenvalue spectra). Appendix B.2, C.3 and G, as well as the experiment requested above, all point to this conclusion. We wrote the paper with this in mind and made sure that abstract, introduction and all sections were extra clear and precise about it. However, we agree that most of the confusion comes from the "catchy" title, and propose to change it: Are In-Context Images Worth a Few Dimensions?

[Question 1] What does the transfer to unseen classes look like.

Transferability to unseen classes: As for ImageNet-21k, the SDG transfers well. We split the classes of the 10 datasets into seen and unseen halves, train the SDG on seen classes only, and evaluate it on unseen classes. At ℓ∗\ell^*, it reaches 90.6%, only 1.3 points below an oracle SDG trained on the unseen classes (91.9%). (see Appendix C.1)

Transferability to unseen domains is interesting and strengthens the mechanistic claim of the paper: We trained a probe only on the Aircraft dataset domain. The probe at layer 8 has a bad transferability to unseen domains: 85.7 on aircraft -> 56.2 on the 9 other datasets.

However, after the dimensionality reduction claimed in the paper the probe at vision layer ℓ∗−1\ell^*-1 transfers much better to unseen domains: 87.1 on aircraft -> 81.0 on the 9 other datasets.

Then we use the same ablation protocol as in the paper (ablate cross image attention for the 15 dimensionality reduction heads selected) and the transferability vanish: 85.9 on aircraft -> 64.0 on the 9 other datasets.

We detail all those experiments and repeated the same protocol for the texture dataset in Appendix C.3. Thank you for this excellent question, which motivated an experiment linking cross-domain transfer to the paper theoretical circuit.


Thank you once more for the time and attention you have devoted to reviewing our manuscript. We value the chance to continue this discussion.

Dear Reviewer TEPr,

Can you please take a look at the authors' response and provide your further input? Thank you.

AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.