acceptodds
Under review as a conference paper at ICLR 2027

Controlling Information Flow in Vision Transformers with Variational Bottlenecks

Abstract

We make the information transmitted by attention in vision transformers a measurable and controllable quantity. By inserting variational information bottlenecks on attention-mediated writes to the residual stream, we train models under an explicit information cost, producing a spectrum from independent patch processing to fully expressive global attention. The same mechanism is compatible with both supervised classification and DINO-style self-supervised learning, allowing us to study how useful visual representations emerge as progressively more information is exchanged across patches. On ImageNet-100, we characterize how performance and representation quality vary with information rate, and how communication is allocated across depth, attention heads, and spatial locations. Under tight information constraints, only a small number of attention heads become active, exposing simple visual computations that precede richer global representations. These results provide a direct way to control internal information flow and yield vision transformers whose computations are more tractable for mechanistic analysis.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability, rate-restricted communication, attention heads, computer vision

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LmBKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies attention-mediated information flow in vision transformers by placing a variational bottleneck on each attention head’s update before it is written to the residual stream. The authors train separate ViT-Tiny models with different information penalties and examine how accuracy, patch-level predictions, and representations change as the constraint is relaxed. Under tight constraints, only a few heads remain active. The paper analyzes these heads and finds that some respond to repeated patches within an image. It also applies the same approach to DINO training.

Critical strengths
  • The paper is well written and easy to follow.
  • The motivation is reasonable. Restricting attention writes during training offers a way to study simpler models whose internal computations may be easier to inspect.
  • The copy-paste experiment is a useful step beyond visualizing attention maps. It provides evidence that selected heads respond to repetition elsewhere in the image while the queried patch remains unchanged
Critical weaknesses
  1. This paper treats information retained in a head update as a measure of communication between patches. However, the update can also contain information about the receiving patch itself. Neither the KL cost nor the reported mutual information cleanly isolates information arriving from other patches. This makes the paper’s central interpretation less precise than its language suggests.
  2. The repetition-sensitive heads are an interesting finding, but their role in the model’s predictions is not established. The copy-paste experiment shows that their messages respond to repeated content. It does not show whether those messages are needed for classification, or whether an unrestricted ViT uses a comparable computation.
  3. The most detailed head-level analysis concerns strongly constrained ViT-Tiny models on ImageNet-100. It is unclear how well these findings carry over to less constrained or larger models. The bottlenecks also add substantial parameters, and removing the KL penalty does not fully recover the unmodified ViT’s accuracy.
Questions for authors
  • How much of the measured information in a head update comes from other patches rather than the receiving patch?
  • What happens to predictions when a repetition-sensitive head is disabled? Is similar behavior present in an unrestricted ViT?
  • Do the head-level findings persist with a larger backbone or a more varied dataset?
1 comment

We appreciate the feedback on our submission, which led to some exciting improvements to the manuscript.

W1/Q1, cross vs local information: The attention output is what’s restricted, and it’s a function of all patches (including the receiving patch). When the information that attention writes -- the only way information can pass between patches -- is restricted, then total cross communication is restricted. We found several places where our prose could be misinterpreted and revised them to be more clear to this point. The guiding intuition is that under information restriction, processing of the receiving patch's information will be done by the MLPs where it is not penalized. The attention heads will then add only essential updates that are themselves a function of both the receiving patch and the rest of the image (as in the repetition detection, which is necessarily a function of both).

W2/Q2, role of repetition head in prediction: If ablated, the accuracy matches that of the local patch processing, 70% -> 69%. Thus with just one head active and about 1 nat KL for every 10 patches, we get a 1% boost and about a tenth of the total effect of having an unrestricted ViT, which is remarkable. Thanks to the simplicity of the information-starved models, we have other interventions as well: we can steer in the latent space of the active head’s information bottleneck, which is effectively one-dimensional. The results are interesting; please take a look at our new Fig. 5 if you have a chance. In short, while the head's channel is ~simple, the effect on downstream prediction is highly patch dependent. We looked and did not find strongly matching behavior in existing ViTs (by correlating attention maps), nor was it particularly present in ViT-IB models with more communication. We assume that with less/no pressure toward economical messages, the behavior is smeared across many heads.

W3/Q3 larger backbone, more varied dataset: For the revision, we have also trained a series of ViT-S and Vit-B backbones; intriguingly, we found similar repetition and foreground/background detection heads in their lowest-info models too. In terms of parameter count, the relative size of the VIB modification shrinks from +40% params for ViT-tiny to only +10% for ViT-B. We recover a reasonable spectrum of accuracy, similar head sparsification, and head behavior in both backbones that matches the repetition and the foreground/background heads in the ViT-tiny series.

W3 removing the penalty does not fully recover ViT accuracy: This is a fair point. The boost in expressivity from the added MLP layers appears to be more than offset by breaking the linear structure of the writes to the residual stream. With a biased opinion, to be sure, we think the added interpretability is worth the cost, though we agree that this is an encumbrance to shed in future work.

Reviewer_w3GXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces VIB into attention-head outputs in Vision Transformers to investigate how information constraints affect model performance, representations, and internal computation. Experiments on ImageNet-100 cover supervised classification and DINO self-supervised learning, with analyses of information allocation and attention-head behavior across different information budgets.

Critical strengths
  1. The paper is well-motivated and addresses an interesting question about information exchange and internal computation in Vision Transformers.

  2. The method is simple and well-structured, enabling information costs to be controlled and analyzed across attention heads and patches.

  3. The paper is clearly written and well organized, with extensive experiments and effective visualizations. The analysis of attention-head behavior under strong information constraints is particularly interesting.

Critical weaknesses

1.The measurement of cross-patch information lacks a clear distinction. I(Δ; Z) includes information from both the receiving patch and other patches, without directly isolating newly acquired cross-patch information.

2 The mechanistic interpretation remains insufficiently validated. Figures 3 and 4 demonstrate repetition-sensitive head responses and cross-model similarities, but do not directly establish their contributions to classification decisions.

  1. Controlled analysis of the bottleneck architecture is limited. VIB increases model parameters from approximately 5.5M to 7.6M and introduces stochasticity. The current experiments do not sufficiently separate these effects from information restriction.

  2. Models under different information budgets are trained independently. The results characterize differences across constrained models rather than the temporal emergence of computational mechanisms during training.

Questions for authors
  1. How do the authors distinguish local information from newly acquired cross-patch information in attention outputs?

  2. Can the authors directly intervene on repetition-sensitive heads to verify their contributions to classification predictions?

  3. Can controlled experiments separate the effects of information restriction, additional parameters, and stochasticity?

  4. How do independently trained models support claims about the order in which computational mechanisms emerge?

1 comment

We thank the reviewer for the close reading of our paper and the push for more clarity. We have added a steering analysis of the repetition tagging's effect on downstream classification as a consequence, which we think strengthened the finding considerably.

W1/Q1 Cross patch vs local information: The attention output is what’s restricted, and it’s a function of all patches (including the receiving patch). When the information that attention writes -- the only way information can pass between patches -- is restricted, then total cross communication is restricted. We found several places where our prose could be misinterpreted and revised them to be more clear to this point. The guiding intuition is that under information restriction, processing of the receiving patch's information will be done by the MLPs where it is not penalized. The attention heads will then add only essential updates that are themselves a function of both the receiving patch and the rest of the image (as in the repetition detection, which is necessarily a function of both).

W2/Q2 Intervene on repetition: Great idea! Our setup and the low information regime make this particularly doable. Specifically, the average pooling at the end means we can interpret a patch’s change in voting independently of the rest of the image, the singular active attention head (and its position in the final layer; either would be sufficient) means that no other patches are affected by the ~repetition update to a patch, and the posterior collapse grants us a single dimension to sweep in the IB latent space. We have added a new Figure 5 based on this intervention, and found that explaining which patches are tagged is far simpler than explaining how their downstream classification changes. The change in logits appears to be highly patch-dependent, which made sense after some head scratching. The extra context is combined with the rich per-patch statistics that already does well (69%) to allow for patch-conditional changes. Some patterns that stand out above the spread -- with repetition, there's an intuitive shift toward tile roof, honeycomb, and window screen, and also with repetition, the patch-specific top votes are downweighted. We think this strengthened the story considerably and appreciate the suggestion.

W3/Q3 Effect of additional parameters vs information restriction: For the revision, we have also trained a series of ViT-S and Vit-B backbones; intriguingly, we found similar repetition and foreground/background detection heads in their lowest-info models too. In terms of parameter count, the relative size of the VIB modification shrinks from +40% params for ViT-tiny to only +10% for ViT-B. We agree that the modification clearly has a nontrivial effect, since the β=0\beta=0 limit does not match (underperforms) the unmodified ViT architecture, and are running matched experiments to add to the appendix.

W4/Q4 “order” of mechanism emergence: We now see that this was confusing. Our framing was in reference to the spectrum from restricted to unrestricted information, where update behavior emerges in an order that reveals prioritization. Thus, we take the repetition tagging to be a foundational mechanism because it seems to emerge “first” as attention update information is increased. We have rewritten the relevant passages in the manuscript to refer to these as prioritized under information restriction instead of as emerging first.

Finally we want to check in on the ethics review question -- you entered “yes”: was that a mistake?

Reviewer_V8JsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a new method for controlling the information flow in ViT models by introducing variational bottlenecks after attention layers (but before residual updates) across the entire ViT. By controlling the weight of the resulting KL loss, the authors show that the amount of information passed between patches can be controlled (from patch-independent updates only, to full global attention sharing). Experiments are run on ImageNet-100 with a ViT-Tiny model, both on image classification as well as DINO-style self-supervised learning. These experiments provide a thorough analysis of their method and the insights it can provide, such as repetition-sensitive heads, how SSL handles restricted information, and more.

Critical strengths
  1. The method is well explained and well-explored in experiments, showing useful insights from the introduction of these information bottlenecks. Testing on both classification and SSL shows that this is generalizable to multiple types of settings. This also raises interesting future possibilities of applying this to other transformer architectures such as LLMs.

  2. The amount of detailed figures provide a lot of visualization and understanding of the method, and make the insights more easily understandable.

Critical weaknesses
  1. The main critical weakness I find is the additional overhead is quite significant (40% parameter overhead from introducing all the bottlenecks, and about twice the needed training time compared to the non-bottleneck versions). It is not clear to me that this method is scalable at all, or even details about whether bigger bottlenecks are needed for larger models. Additionally, the requirement to test this across many beta values does add significantly to the cost of using this method. Thus, it seems hard to determine whether the insights, or the method of analysis, would generalize.
Questions for authors
  1. Do you have any results about the scalability of this approach? For example, if you were to train a ViT-S or ViT-B model on the same setup, do the insights from the ViT-Tiny model transfer (are the insights similar)? And how feasible would those trainings be?

  2. Did you perform any tests about applying and fine-tuning pre-trained models with this method? If these can be added to already-trained models and achieve interesting results, then that could help reduce computational costs significantly, and allow for a much more detailed exploration of many existing models and setups in the literature.

1 comment

We are grateful for the feedback and are glad that aspects of the method and presentation were well-received.

Regarding scalability, we have optimized a series of ViT-S and ViT-B models for the revision and added key results. Importantly, the additional overhead shrinks with the larger models in terms of relative size of the parameterization: the relative size of the VIB modification shrinks from +40% params for ViT-tiny to only +10% for ViT-B. Intriguingly, we found similar repetition and foreground/background detection heads in their lowest-info models too.

Zooming out, we agree that scalability is probably the most significant challenge to overcome for more impactful insights. Fine-tuning an existing model is a good idea and one that we should validate at a scale where we can compare to models trained from scratch. We are also pursuing other, lighter-touch modifications that still allow for tracking information flow but scale up more gracefully and/or port into pre-trained models more easily. That said, we feel the current contents are an important step in the direction of interpretable information flows in transformers, with scaling suitable for a future study.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.