acceptodds
Under review as a conference paper at ICLR 2027

Read Where the Model Reads: Causal Anatomy of ViT Classification and Its Instruments

Abstract

In language models, the logit lens interprets intermediate representations by applying the final unembedding directly to the current token state. This is natural because the current token stream ultimately reaches the unembedding. However, in Vision Transformers (ViTs), the classifier reads only the CLS stream, while image information resides in the patch tokens. Despite this structural difference, direct projection of intermediate representations through the final classifier is routinely used to interpret ViTs. We show that this routine can be misleading. Across five ViT backbones, we first examine the causal anatomy of classification and find that class evidence develops within the patch stream and is written into CLS only through a few terminal layers, which we call the *read window*. Motivated by this anatomy, we introduce the read-through lens (RTLens), a logit lens variant that passes intermediate patch states through the read window before applying the final classifier. We further derive an anatomy-based attention map that exactly decomposes patch contributions to the prediction through the read window. Across all five backbones, RTLens predictions match the model's causal behavior far better than direct projection, and our map more accurately localizes the evidence the model actually uses. Together, these results support a simple principle for interpreting ViTs, *read where the model reads*.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Vision Models, Logit Lens

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_6C4jOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the causal anatomy of ViT classification. It finds that class evidence develops within the patch stream and is written into [CLS] token only through a few terminal layers, called the read window. Motivated by this, the paper introduces RTLens, which reads intermediate patch states through the read window rather than directly through the final head, and an attention map that attributes the prediction margin to messages delivered through that window.

Critical strengths
  • The paper is in general easy to follow.

  • The proposed principle "read where the model reads", is simple and compelling. RTLens directly follows the measured causal path and improves over the naive logit lens across multiple ViT backbones.

  • The paper presents some clear and interesting findings supported by empirical evidence.

  • The experimental evaluation is careful and broad. The paper studies five ViT-Base backbones, includes a ViT-Large scale probe, uses distribution-preserving interventions, and provides extensive study in the appendix.

Critical weaknesses
  • The paper is strong overall, but the main causal anatomy is still fairly coarse. The analysis identifies important token-group routes such as P←P and C←P, but it does not resolve the mechanism at the level of individual heads, MLPs, features, or circuits. The authors acknowledge that the two-stage anatomy is not the complete classification circuit, but many of the claims are phrased quite strongly around “where the model reads". In practice, the results show the dominant token-group communication path, but not a full mechanistic explanation of how class evidence is represented, transformed, or selected.

  • The qualitative examples are helpful but somewhat selective. For the patch-label maps and attribution maps, more randomly sampled examples, failure cases, and side-by-side comparisons across model families would make it easier to understand when the method works well or poorly.

  • The paper does not fully explain why different models have different read-window structures. DINOv2, DINOv3, and CLIP are dominated by the final layer, while supervised ViT and DeiT use a two-layer read, and ViT-L appears to use a wider terminal region. The empirical finding is interesting, but it would be good to provide a deeper explanation of what architectural or training factors determine the width and timing of the read.

  • The paper's conclusions seem to be tied to CLS-based classification readouts only. The results in appendix show that under mean-pool readout on DINOv2, blocking P← P is highly damaging while blocking C ←P is almost irrelevant.

Questions for authors

See Weaknesses.

Reviewer_Vr3sOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how ViT classifiers causally route image evidence to their classifier. It finds a two-stage anatomy: patch-to-patch computation forms evidence, then patch-to-CLS attention transfers it through a narrow terminal “read window.” It further finds that pre-read CLS states can be decodable while their image-specific content is causally near-inert. Then it introduces the RTLens, which transports intermediate patch states through the read window before applying the frozen classifier, and a read-window-scoped attention map that exactly decomposes the patch-delivered component of the prediction margin.

Critical strengths
  1. A consequential causal finding. The pre-read CLS result directly demonstrates that decodability need not imply causal use: cross-image replacement of pre-read CLS content is much less damaging than zeroing, despite substantial decodability. Together, this gives a clear mechanistic explanation for the proposed “read where the model reads” principle. This is the primary reason I admire the paper as a contribution.

  2. The proposed tools are derived from the causal anatomy and validated through causal interventions. RTLens is parameter-free and improves agreement with neuron-clamp effects near the read window across all five backbones. The attention map is an exact, gradient-free decomposition of the patch-delivered margin component. This transparent, intervention-based validation supports acceptance despite the limitations below.

Critical weaknesses

The primary results establish consistency across diverse training regimes, but within a narrow architectural family: all five primary models are ViT-B variants sharing the same broad encoder template. In particular, CaiT would be a natural stress test because its class-attention layers make the terminal read explicit. This limits the breadth of the paper’s central anatomy claim. Also, the paper does not give a direct, matched comparison of RTLens and JLens on decoding or neuron-level causal agreement, nor explain why CLIP and DINOv3 are omitted. Since JLens is presented as an independent lens that approaches the same transport near the read window, clarifying the empirical distinction matters for assessing the incremental value and scope of RTLens.

Questions for authors

Could the authors add a targeted causal-screen experiment (the token-route intervention identifying the read window) on CaiT or another architecturally differentiated CLS-based ViT? A positive or negative result would materially strengthen or qualify whether the proposed anatomy generalizes beyond the shared ViT-B template.

And could the authors add a matched RTLens-versus-JLens experiment on the same evaluated states and models, reporting at least layerwise decoding accuracy and/or patch-neuron causal agreement? Please also briefly summarize in the main text how the pre-read CLS mechanism differs from the register-token mechanism.

Addressing these concerns with additional evidence in the rebuttal would make me inclined to increase my score by one point.

4 comments
AuthorsOpenReview ↗

Follow-up on Our Rebuttal Responses

Dear Reviewer Vr3s,

Thank you again for taking the time to review our paper and provide constructive suggestions.

Following your suggestions, we have conducted additional experiments on CaiT and a direct comparison between RTLens and JLens. We have incorporated the results and their interpretations into our rebuttal responses and revised manuscript.

With approximately 25 hours remaining in the discussion period, we would appreciate it if you could review our responses and share your thoughts before the deadline.

  • If you have any remaining concerns or questions, please let us know. We would be glad to address them as promptly as possible during the remaining discussion period.
  • If you feel that your concerns have been adequately addressed, we would greatly appreciate it if you could consider updating your rating or confidence level, as appropriate.

Best regards,
The Authors

Thank you again for taking the time to review our paper and provide constructive suggestions.

With approximately 9 hours remaining in the discussion period, we would appreciate it if you could review our responses and share your thoughts before the deadline.

  • If you have any remaining concerns or questions, please let us know. We would be glad to address them as promptly as possible during the remaining discussion period.
  • If you feel that your concerns have been adequately addressed, we would greatly appreciate it if you could consider updating your rating or confidence level, as appropriate.

Best regards,
The Authors

Sorry for my delayed response, and thank you for the thoughtful responses and additional experiments. The new results on CaiT and the matched comparison between RTLens and JLens have addressed my main concerns and further strengthened the paper.

I am therefore raising my rating from 3 to 4.

Dear Reviewer Vr3s,

Thank you for your response. We are glad that we were able to address your concerns. We appreciate the time you took to review our responses and your constructive feedback throughout the discussion.

Best regards, The Authors

Reviewer_J8dcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that logit lens is not a good enough interpretability mechanism to be directly applied to inspect ViT internals. The authors propose RTLens: they first find a "read window" (given a model) by empirically measuring which layers are causally important for transferring information from patch tokens to CLS (in the models examined, usually near the final layers). They then compute intermediate representations with respect to this read window, instead of directly applying the final readout as in logit lens. Evaluation of 5 models show that RTLens can recover the class prediction from model internal representations better than conventional logit lens and achieve a higher agreement with the class label promoted by interventions. The authors also designed a read window based attribution method and show that it achieves better spatial localization of the image regions contributing to the predicted class.

Critical strengths

S1: The paper provides a strong motivation of taking the information flow in ViTs into the consideration of constructing the interpretability mechanism. Empirical evidence in section 3 shows that for the 5 models assessed, classification-relevant information is concentrated in the final layers, and less dense in the earlier layers, which is observed consistently across different ViT backbones. These findings provide a strong motivation for the design of RTLens.

S2: This work uses controlled causal interventions to evaluate the proposed interpretations: RTLens is assessed on whether the predicted decision agree with the observed effects of neuron interventions. The substantial improvement in agreement (near the read window) provides evidence that the proposed method better reflects/approximates model internal behaviour.

Critical weaknesses

W1: The proposed methods have limited evidence showing generalizability/stability.The experiments uses ImageNet-1k only, and the read window is empirically found for each model. It is not very clear whether the window would change across different datasets, tasks, or input distributions (it is already shown that it changes across different backbones); this stability is important since the proposed interpretability mechanism is heavily dependent on this empirically identified window.

W2: The experiments reveal limited practical advantages (especially in the main text). The authors acknowledged that the proposed methods are not always superior than existing methods (such as GAE for deletion faithfulness). But what are some practical benefits that we can obtain by applying RTLens? A more concrete downstream application could better demonstrate how these improved metrics (localization, agreement, etc.) lead to more useful interpretations or actionable insights about model internals. I found some of the qualitative analysis in the appendix more relevant, but maybe there is not enough space to discuss those results in depth in the main text.

Questions for authors

(1) Related to W1: How robust is the localization of the read window (W)? In the current writing it shows that different model architectures can have different W (L11, v.s. L10-11), do the authors believe that they will always be found near the last layers? How much does this pattern depend on the dataset? The evaluations are done on ImageNet-1k, would the read window (since they are empirically identified) change under different dataset?

(2) Related to W2: Can the authors discuss more on downstream applications of the proposed interpretability methods? Especially on how they advance the community's understanding of ViT internals?

4 comments
AuthorsOpenReview ↗

Follow-up on Our Rebuttal Responses

Dear Reviewer J8dc,

Thank you again for taking the time to review our paper and provide constructive feedback.

We have provided detailed responses to your concerns and conducted additional experiments to evaluate the stability of the read window. We have also further clarified the practical applications of our proposed methods.

With approximately 25 hours remaining in the discussion period, we would appreciate it if you could review our responses and share your thoughts before the deadline.

  • If you have any remaining concerns or questions, please let us know. We would be glad to address them as promptly as possible during the remaining discussion period.
  • If you feel that your concerns have been adequately addressed, we would greatly appreciate it if you could consider updating your rating or confidence level, as appropriate.

Best regards,
The Authors

Thank you again for taking the time to review our paper and provide constructive suggestions.

With approximately 9 hours remaining in the discussion period, we would appreciate it if you could review our responses and share your thoughts before the deadline.

  • If you have any remaining concerns or questions, please let us know. We would be glad to address them as promptly as possible during the remaining discussion period.
  • If you feel that your concerns have been adequately addressed, we would greatly appreciate it if you could consider updating your rating or confidence level, as appropriate.

Best regards,
The Authors

Dear authors,

I believe the additional explanations and experiments provided resolve most of my main concerns. I will adjust the score accordingly.

Dear Reviewer J8dc,

Thank you for your response. We are glad that we were able to address your concerns. We appreciate the time you took to review our responses and your constructive feedback throughout the discussion.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on Our Rebuttal Responses

Dear Reviewers,

Thank you for taking the time to review our paper and provide valuable feedback.

As we are now halfway through the rebuttal period, we would appreciate it if you could let us know of any remaining concerns or questions regarding our responses. We would be happy to provide further clarification.

Best, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.