acceptodds
Under review as a conference paper at ICLR 2027

Causal Analysis of Robot Foundation Models Enables Efficiency Gains

Abstract

Vision-language-action (VLA) models are a promising approach to general purpose robot manipulation. While the internal computation of vision-language models, which compose the backbone of VLAs, has been studied extensively, less is known about how their perceptual and semantic representations are transformed into action in VLA models. We study this question mechanistically in three frontier VLA models (, MolmoAct2, and MolmoBot) by analyzing the causal pathways through which the input image and the instruction shape actions. Across the three models, activation patching shows that causally relevant information influences the action expert (AE) through a narrow band of mid-to-late cross-attention layers, which we term the action-conditioning interface. The content of this information changes with token and layer within the model. Early-to-mid layer representations of the target noun token determine which object is selected, whereas mid-to-late layers carry the spatial information in text tokens that redirects the movement when patched. Where spatial information is stored in prompt tokens various based on the architecture. In MolmoAct2 the model forms task-conditioned representations at the text positions that follow the instruction, whereas additionally stores this information at image token positions. We show that these findings can be translated directly into improving model efficiency. We find that restricting MolmoBot's action expert conditioning to 3/36 layers reduces inference FLOPs by up to 33% while maintaining comparable task success with the default model across more than 3,000 evaluation episodes (46.7% vs. 46.9%), and the same principle carries over to real-robot deployment of .

Primary area:
applications to robotics, autonomy, planning
Keywords:
mechanistic interpretability, vision-language-action models, robot manipulation, activation patching, causal analysis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TYmpOpenReview ↗
Rating
2
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how visual and linguistic information influences action generation in three vision-language-action models. It uses activation patching with counterfactual changes to object identity and scene layout to study representations within the VLM backbone and conditioning at the VLM–action-expert interface. The authors observe different intervention effects across token positions and layers, and identify a narrow band of interfaces with relatively strong influence on predicted trajectories.

Critical strengths
  1. The connection between mechanistic analysis and architectural simplification is valuable. The paper goes beyond reporting intervention effects and evaluates whether the identified interfaces support more efficient policies in complete closed-loop tasks.

  2. The counterfactual design is useful for interpreting behavioral changes. Varying object identity and location helps distinguish transferable object-selection effects from spatially specific effects more carefully than changing the instruction in a fixed scene alone. Studying three architectures also reveals how attention masks affect where instruction-dependent information can appear.

Critical weaknesses
  1. The additional insight from early-layer noun patching is not sufficiently established. Replacing an early representation of "apple" with that of "banana" may behave similarly to modifying the instruction internally, after which downstream computation locates the requested object in the recipient scene. Because backbone patching changes all subsequent computation, successful transfer establishes behavioral influence but does not directly identify where object selection is computed. The cross-layout result is informative, but comparisons with direct instruction changes or input-embedding replacement would help establish what additional insight the layer-wise analysis provides. Although the paper acknowledges limitations of information localization, it remains unclear how much the depth-dependent effects reflect changes in the representations themselves versus differences in how downstream computation propagates or uses the patched signal.

  2. Activation patching is an established technique, and the paper itself discusses prior work applying activation interventions and attention knockout to VLAs. The contribution therefore rests primarily on the new mechanistic insights obtained through these tools and their value for architectural design. However, some findings, such as early noun representations influencing object selection through downstream computation, appear largely expected. Although the empirical evaluation is substantial and the measured efficiency gains are useful, I remain unconvinced that the paper establishes a sufficiently strong conceptual advance beyond applying existing interpretability tools to motivate a sparse interface.

Questions for authors

see weaknesses

3 comments

We thank the reviewer for their thoughtful review and feedback. We address the weaknesses and questions below.

Comparison with direct instruction change

“The cross-layout result is informative, but comparisons with direct instruction changes or input-embedding replacement would help establish what additional insight the layer-wise analysis provides”

We agree that patching the noun at the earliest layers is equivalent to changing the instruction. In fact we already include the baseline you suggest. E is measured against the trajectory produced by directly changing the instruction, and patching the noun at layer 0 gives E = 1.00 in all three models.

The interesting question is what happens as the input representations are transformed with depth. What type information they carry (semantic, visual or geometric), and whether it is causally relevant to the action. While the patching effect at layer 0 is expected, patching internal representations across our counterfactual scenes produced several findings were unexpected and which significantly advanced our understanding of VLA models:

  • Only one camera is causally relevant. Scene layout is read mainly from the exterior camera in MolmoAct2 (E = 0.90, vs. 0.56 for the wrist camera), and from the wrist camera in π0.5 (0.96 vs. 0.02) and MolmoBot (0.90 vs. 0.25).
  • Image tokens are causally relevant only up to mid depth. Patching the dominant camera stays near its full effect through layer 12 in MolmoAct2, 7 in π0.5 and 14 in MolmoBot. It then declines and reaches zero by layers 17, 15 and 24.
  • The noun is used only early. Its effect stays near E = 1 through layer 11 in MolmoAct2, 3 in π0.5 and 7 in MolmoBot. It is effectively zero by layers 15, 7 and 12.
  • The noun token only holds the target object, not its position. In comparison C the target we change the target location but not the instruction. Patching the noun then has almost no effect at any depth in our three tested models
  • Where the target location object location lives ? It is stored by later text positions, We should in Appendix B.3 that in MolmoAct2 four tokens retain 89% of the all-text effect at layer 18 (Figure 7). In π0.5, bidirectional attention also places this information in the image tokens.
  • Where the target object location is computed. The location appears in text positions at layer 15 in MolmoAct2 and layer 19 in MolmoBot. In Pi0.5 it appears in image tokens from layer 7 and in text positions from layer 10. This is due to the bidirectional attention of Pi0.5

“Because backbone patching changes all subsequent computation, successful transfer establishes behavioral influence but does not directly identify where object selection is computed.”

We added an additional mediation experiment to test this directly (Appendix B.4). We patch the noun at layer 0 while holding the later text positions at their original values. In MolmoAct2 and MolmoBot the patching effect unsurprisingly disappears (E = 1.00 → 0.00 and 0.03). Copying only those downstream positions into an otherwise original run (other tokens held constant) reproduces the patching effect. This establishes that object selection is not computed at the noun itself. The noun passes the requested object to later text positions (or unused image tokens in the case of Pi0.5). After this depth (layer 16 in MolmoAct2, 10 in MolmoBot and 8 in Pi0.5) the action depends only on those later text positions.

What is the contribution beyond prior work ?

“I remain unconvinced that the paper establishes a sufficiently strong conceptual advance beyond applying existing interpretability tools to motivate a sparse interface.“

We respectfully disagree that the paper only applies existing tools to motivate a sparse interface. While activation patching is an existing tool, the insights we glean from it significantly improve on the understanding of VLA models, while also demonstrating compelling efficiency improvements.

Prior causal work on VLAs is limited. Prior work shows that individual representations can be steered or knocked out. However this work does not localize where the action expert reads the VLM backbone, what information is exchanged or how that information forms and flows through the network. We provide an expanded selection of exciting novel contributions of our paper:

  • The action-conditioning interface (Section 2). In three VLAs with different architectures and conditioning mechanisms, the action expert reads the VLM backbone through a small number of mid to late layers. The transmitted information is the target location (Section 4.2).
  • How that information forms (Section 4). We agree that the early noun patching selecting the object is expected. However we find significantly more surprising phenomena in the later representations. -The noun’s causal role collapses from E ≈ 1 to near zero by layer 15 in MolmoAct2, 14 in MolmoBot and 7 in π0.5, after which the target object location is carried in later text representations.
    • Scene layout is read mainly from one camera: the exterior camera in MolmoAct2, the wrist camera in π0.5 and MolmoBot.
    • The target location sits in very few tokens. In MolmoAct2, four tokens retain 89% of the all-text effect at layer 18 (Figure 7).
    • π0.5’s bidirectional attention writes the target object location into image tokens, which the Molmo models cannot do.
  • We leverage this understanding to create improved models (Section 3). Keeping only the action-conditioning-interface reduces MolmoBot’s computation and latency by 33% without reducing success rate. We find the same principle carries over to Pi0.5 on a real robot and actually leads to a significant improvement in completion time.

It also justifies existing VLA designs and makes predictions for improvements. Two existing VLA models, GR00T N1 and SmolVLA utilize slightly different architecture, conditioning on a single mid-layer and on the first half of its layers respectively. Both choices were made heuristically. Our experimental findings explain why conditioning on intermediate layers works. It also shows that neither of these VLAs are an optimal. To improve performance SmolVLA could likely drop its early backbone connections while GR00T N1 would likely gain performance from conditioning on a few layers around the action conditioning interface instead of one.

The paper gives the first account of the path from perception to action in VLAs. We show clearly that this information is useful towards improved architecture designs. The localization of relevant information across tokens and layers provides a guide for further interpretability studies on specific parts of the model.

Thanks for authors detailed response. I will increase my score.

Reviewer_ogGyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how visual and language information influences action generation in VLA models. The authors use activation patching on three models (π0.5, MolmoAct2, and MolmoBot) to examine the roles of different tokens and layers. They find that the action expert mainly relies on a small number of mid-to-late layers for conditioning. Based on this observation, they simplify the connections between the VLM and action expert. On MolmoBot, this reduces inference computation by about 33% with little change in task success. They also test the idea on a real robot using π0.5.

Critical strengths

The paper provides a useful analysis of how different layers affect action generation across three VLA models, and the authors further use these findings to simplify the VLM–action expert interface. On MolmoBot, this reduces inference computation by about 33% while maintaining a similar task success rate.

Critical weaknesses

The layer-wise patching results reveal causal effects at individual locations, but do not fully establish the causal dependencies between early semantic representations, later spatial representations, and action expert conditioning. Therefore, the proposed information-transfer mechanism remains only partially explained.

Questions for authors
  1. Why does the action-conditioning effect consistently concentrate in a narrow band of mid-to-late layers across different VLA architectures? Can the authors provide a deeper mechanistic explanation for this localization?
  2. Is the identified action-conditioning interface a necessary computational bottleneck in the original dense VLA, or merely an effective intervention and pruning location? Could the authors clarify this distinction through direct ablations without additional training?
2 comments

We thank the reviewer for their thoughtful review and feedback. We address the weaknesses and questions below.

Causal dependencies

“The layer-wise patching results reveal causal effects at individual locations, but do not fully establish the causal dependencies between early semantic representations, later spatial representations, and action expert conditioning.”

We agree that patching individual locations shows where information matters, but not necessarily how one location depends on another. To test the dependencies directly, we added a mediation experiment (Appendix B.4). We patch the target noun at layer 0 (comparisons A and B) or both camera images (comparison C), and hold the later text positions at their original values at every subsequent layer. This allows us to measure what positions the signal transfers through to reach the action.

In MolmoAct2 and MolmoBot, holding the later text positions reduces the noun’s effect from E = 1.00 to 0.00 and 0.03, and the images’ effect from 1.03 and 0.95 to 0.00. In π0.5, holding the image tokens removes the noun’s effect (0.01), while holding the other text removes it only partly (0.38), consistent with bidirectional attention routing the instruction into the image tokens. In the other direction, copying only those downstream positions into an otherwise original run reproduces the effect from the depth at which the noun’s own effect fades in Figure 4.

The semantic and visual signals therefore reach the action through the later prompt positions which are read by the interface. Combined with the results in Figure 4 and the formal causal model in Appendix A, this establishes the chain from early semantic and visual representations, through later geometric representations, to action-expert conditioning.

Explaining the action conditioning interface

“Why does the action-conditioning effect consistently concentrate in a narrow band of mid-to-late layers across different VLA architectures? Can the authors provide a deeper mechanistic explanation for this localization?”

We have restructured the paper to make the answer to this question more clear. Section 4 focuses on explaining the formation of the action conditioning interface using our counterfactual analysis. Overall, the action expert reads the backbone as soon as the relevant information is formed. We know that the expert reads the target location (Section 4.2). This information is only formed in the backbone at mid-to-late depths.

  • In the early layers the target object identity, visual information are stored separately. The object information is held in by the noun and the layout by the image tokens (Figure 4). The interface does not transmit either of these (Section 4.2), which is why early connections have almost no effect in Figure 2.
  • The target location forms at mid depth. It appears in the later text positions from layer 15 in MolmoAct2 and layer 19 in MolmoBot, (Figures 4 and 6). The interface follows in each model: layers 21–27 and 23–25
  • Later layers are not read. Past the interface, patching the backbone has no effect on the action (|E| < 0.001 beyond layer 26 in MolmoAct2, despite the model having 36 layers).

All three models show the same ordering despite different conditioning mechanisms and architectures. This is why the band is consistent across the different architectures. Our real-robot results strongly support these conclusions. Restricting π0.5 to layers 14–16 preserves success (67 vs. 64 of 100 trials) and shortens successful trials by 24% (median duration 34.9 s to 26.5 s, averaged over seven tasks; Section 3.2). The connections outside the interface may therefore add little beyond what the interface already carries.

“Is the identified action-conditioning interface a necessary computational bottleneck in the original dense VLA, or merely an effective intervention and pruning location? Could the authors clarify this distinction through direct ablations without additional training?”

We add an additional experiment in the appendix to address exactly this question. (Appendix B.1, Figure 5a). We replace the conditioning at every connection except the interface window with the donor’s, so that only the interface still carries the original input. We find that in MolmoBot and π0.5, patching all but the three action-conditioning-interface connections recovers less than a third of the full patching effect. The interface is therefore the dominant route in the dense model.

Thank you for your detailed response. Your explanations have adequately addressed my questions and concerns. After considering your clarifications, I have decided to maintain my original rating of 3 (Weak Accept).

Reviewer_ULXSOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

Pretty interesting and insightful paper! This paper uses activation patching to investigate how visual and linguistic information influences action generation in MolmoBot, MolmoAct2, and π0.5. The authors find that causal influence on the action expert is concentrated in a small subset of mid-to-late-layer connections and use this finding to simplify MolmoBot’s conditioning interface. Retaining only 3 of 36 connections achieves an average success rate of 46.7% across three simulation tasks, compared with 46.9% for the original model, while reducing both computational cost and measured inference latency by approximately 33%. Real-robot experiments with π0.5 provide preliminary evidence that restricting the conditioning interface can preserve task performance.

Critical strengths
  1. Useful separation of allocation from compression. The formulation distinguishes how much bandwidth each modality receives from how its activations are compressed, exposing a practical limitation of uniform keep ratios. Server-side probing reuses received activations rather than requiring additional modality uploads.

  2. Substantial experimental coverage and informative controls. The paper reports 12-seed results, improvements in 29 of 36 compressor–dataset–budget configurations, and controls for equal payload, alternative contribution signals, freezing, and split depth. The reported 5×-compression gains of 15.4 and 12.4 percentage points on CREMA-D and MVSA are substantial.

Critical weaknesses
  1. Some interpretations of information content exceed what the interventions establish. The null effect in comparison A cannot distinguish an interface transmitting only target location from one jointly transmitting object identity and scene layout that together specify the same target location. Consequently, the interpretation that the action expert reads positional rather than semantic information requires additional evidence or more qualified wording.

  2. The incremental mechanistic contribution and advantage of causally informed layer selection remain insufficiently established. VLA-Trace already reports a narrow visual bottleneck in π0.5, while GR00T N1 and SmolVLA employ intermediate-layer conditioning or backbone truncation; although these works are cited, the additional mechanistic findings are not clearly distinguished. The layer-12 control demonstrates that conditioning location matters, but does not establish the additional benefit of causal localization over budget-matched layer-selection heuristics.

Questions for authors
  1. Mechanistic interpretation: Can additional controls rule out an interface jointly transmitting object identity and scene layout? Otherwise, could the claim that the action expert reads positional rather than semantic information be qualified?

  2. Distinction from prior work: What additional mechanistic evidence do the A/B/C patching experiments provide beyond VLA-Trace’s bottleneck findings? Please also discuss the relationship to How Do World Action Models Use Their Futures?, particularly regarding donor transplantation, K/V pathway interventions, and interface localization.

  3. Layer-selection validation: When matching the number of connections, backbone exit depth, and training budget, how does causally informed selection compare with simple depth-based rules, random selection, or representation-similarity-based selection?

  4. Metric interpretation: Equation (4) measures trajectory displacement projected onto the reference direction. Could full-trajectory error or target-selection accuracy be reported to verify that high EE corresponds to the intended behavioral transfer?

2 comments

We thank the reviewer for their thoughtful review and feedback. We are glad that you found our results insightful! We address the weaknesses and questions below.

What does the action-conditioning interface transmit ?

“The null effect in comparison A cannot distinguish an interface transmitting only target location from one jointly transmitting object identity and scene layout that together specify the same target location.”

We agree that a full-interface patching A is not sufficient to conclude the interface transmits purely geometric information. However this is addressed by the token-restricted backbone patches (Figure 4). We have added Section 4.2 which explicitly explains how we make the conclusion that the action-conditioning interface transmits geometric information. This does not rely on the data from any new experiments, but is fully explained by the existing experiments in the original paper.

Consider comparison A with a concrete example. In the recipient, the apple is on the left and the instruction is to pick up the apple. In the donor, the objects are swapped and the instruction is to pick up the banana. Both of these scenes have the same target object location (left)

Assume for the sake of contradiction that the action expert read the scene layout from image positions and the object identity from text positions. If we patch only the image tokens this would give the action expert the donor’s layout, where the apple is on the right, so the reach would switch to the right (E ≈ 1). Additionally if we patch only the noun that would give it the donor’s target object, the banana, so the reach would again switch (E ≈ 1).

However in our experiment we see neither of these effects. At each model’s interface layers, patching the image tokens, the noun, or all text in comparison A gives |E| ≤ 0.05. This means that the expert does not combine a scene layout from the image tokens with a target object from the text. Furthermore, in our experiments on MolmoAct2 and MolmoBot the image tokens at the action conditioning layers have no effect even in comparison C (where the target location is swapped visually). The action expert therefore does not read the layout from image positions at all. Text positions carry the entire effect.

What is the contribution beyond prior work ?

“VLA-Trace already reports a narrow visual bottleneck in π0.5“ VLA-Trace finds a Pi0.5 bottleneck near layer 14 which generally agrees with the interface we find (layers 14–16). However this work does not localize where the action expert reads the backbone or what it reads and how that information forms. Furthermore VLA-Trace terms this a visual-bottleneck, however our results show that this is actually geometric information stored in visual tokens rather than pure visual information (See label “image tokens instruction-dependent“ in Figure 4). We also contribute:

  • The action-conditioning interface (Section 2): In three VLAs with different conditioning mechanisms, the action expert reads through a small number of mid to late connections to the VLM and what it reads is the selected target location.
  • A map of how visual, semantic and geometric information moves through the backbone (Section 4) (Figure 4): Through our counterfactual analysis we reveal several non-obvious and important findings about VLAs:
    • The early noun selects the object, as expected, but its causal role collapses from E ≈ 1 to near zero by layer 15 in MolmoAct2, 14 in MolmoBot and 7 in π0.5, after which later text positions carry target object position.
    • Scene layout information comes mainly from one camera: the exterior camera in MolmoAct2, the wrist camera in π0.5 and MolmoBot.
    • Pi0.5’s bidirectional attention writes the instruction-conditioned target into its image tokens
  • We can use this information to improve policies (Section 3). Keeping only the interface cuts MolmoBot’s compute and latency by 33% at matched success, and the approach carries over to π0.5 on a real robot (where we achieve fast task completion time). Overall, we see this as a significant advance over prior causal analyses of VLAs. Beyond showing that representations can be manipulated, it explains how perception reaches the action expert and uses this to build a faster model.

“GR00T N1 and SmolVLA employ intermediate-layer conditioning or backbone truncation; although these works are cited, the additional mechanistic findings are not clearly distinguished.“

These models show that intermediate conditioning can work. GR00T N1 conditions on a single mid-layer and SmolVLA on the first half of its layers. In the respective papers both choices were made heuristically. Our experimental findings explain why conditioning on intermediate layers works. It also shows that neither of these VLAs are optimal. To improve performance SmolVLA could likely drop its early backbone connections while GR00T N1 would likely gain performance from conditioning on a few layers around the action conditioning interface instead of one.

“Please also discuss the relationship to How Do World Action Models Use Their Futures?,”

This concurrent work is methodologically parallel to ours and was published after our work was already submitted to ICLR. We will add this to the related-work section as concurrent work. In general we apply different techniques (key/value swap, versus patching) to different models (VLAs versus WAMs). Furthermore we use our insights to propose sparser models and validate them in closed loop.

Additional experiments

“When matching the number of connections, backbone exit depth, and training budget, how does causally informed selection compare with simple depth-based rules, random selection, or representation-similarity-based selection?“

We agree that more baselines were necessary. We fine-tuned additional restricted MolmoBot policies with the same 50K-update budget and hyperparameters as the original policies. These results are added to Table 1 in the paper. We first train a series of single layer models to emphasize the importance of the action conditioning interface. Models trained on L1, L12, L24 and L35 get 4.3%, 17.8%, 44.5% and 38.0% respectively, task success across 3000 trials. This clearly shows the importance of conditioning from the interface. We train an additional baseline on a representation-similarity-based selection and random selection of three layers, matching backbone exit depth. We select randomly L0,3,20 and select L5,13,22 using CKA. We find these models perform near but not better than the L23-25 model which is exactly what our analysis predicts given they both share a layer near the action conditioning interface.

“Metric interpretation: Equation (4) measures trajectory displacement projected onto the reference direction. Could full-trajectory error or target-selection accuracy be reported to verify that high (E) corresponds to the intended behavioral transfer?”

We have added Appendix B.1 (Figure 5b) which repeats the interface analysis using the RMS displacement of the full trajectory, which counts change in every direction; the peaks are unchanged. We keep E because it measures transfer toward the counterfactual target specifically, but the plots would be virtually identical in RMS displacement.

Comments on the paper

1 comment

We thank all three reviewers for their constructive feedback, which have substantially improved the paper.

In response to specific requests, we add the following: Table 1 now includes additional baselines, Appendix B.1 including additional patching experiments on every connection except the interface (Figure 5a), Appendix A gives a formal statement of our causal model. Appendix B.4 adds a mediation experiment, which patches the early noun or image tokens while holding later positions at their original values. We ran an additional 150 trials in the real world on Pi05 which we have added to Table 2 and have added additional details and visuals about our experiment setup in the Appendix D.

Additionally following the comments of several reviewers we have restructured the paper to better emphasize our contributions. Section 2 presents the discovery of the action-conditioning interface. Section 3 builds more efficient VLA policies Section 4 uses our counterfactual analysis to better explain the action conditioning interface. The core findings and experiments are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.