- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether target-location information can be causally transferred through internal representations of vision-language-action (VLA) policies to produce predictable changes in robot behavior. The proposed protocol transplants selected key-value caches between counterfactual observations while keeping the physical execution state fixed. Experiments across two VLA models and two manipulation tasks demonstrate bidirectional relative endpoint biases that exceed the mean absolute endpoint effects of four predefined equal-norm controls.
Critical strengths
The experimental design carefully separates internal interventions from changes in the physical environment, providing a controlled setting for examining causal effects. The use of bidirectional predictions and equal-norm controls helps distinguish target-conditioned directional effects from generic perturbation effects. The evaluation across two VLA models and two tasks, together with a frozen experimental protocol and code-independent offline reconstruction, strengthens the reliability of the empirical findings.
Critical weaknesses
The research question and the experimental design used to address it are clear. However, I would appreciate a clearer explanation of the new scientific understanding gained from establishing these causal effects. In particular, what do these findings reveal about VLA internal representations and action generation that prior probing or activation-intervention studies did not establish? While more rigorous causal evidence is valuable in itself, the implications of this evidence for our understanding of how these policies generate behavior could be articulated more explicitly.
I would also appreciate clarification on how these findings could inform existing methods for analyzing or controlling VLA models. This does not necessarily require proposing a new control algorithm or improving task success. Rather, explaining the additional value these findings offer to existing methods, or providing a simple illustrative application, would help clarify the significance of the work.
The finding that internal interventions can produce predictable directional biases in robot behavior is interesting. It also raises the question of whether these findings could support more flexible interventions, including human-in-the-loop robotic control. For example, could behavior be adjusted finely in response to user-specified changes in target location? Although the paper acknowledges its limited scope, a more explicit discussion of how the demonstrated effects relate to these broader possibilities, and what limits such extensions, would be helpful.
The evaluation across two models and two tasks is a strength. Nevertheless, the restricted experimental environments and intervention scope leave open how broadly the findings generalize. Building on the paper’s discussion of its controls, a more direct comparison with closely related activation-steering and interpretability studies would help clarify the specific evidential advantage of the proposed protocol.
Questions for authors
Please see the weaknesses.
2 comments
Thank you to the authors for their rebuttal. However, as the response was submitted on the last day, I have not had sufficient time to fully understand and assess it. So I will defer further discussion and evaluation to the discussion phase with the AC.
Cross-scene cache transfer and 10 mm corrections
What does this establish about action generation? A probe could decode the pad’s location even if the policy ignored it. Here, moving only the pad in the donor image predicted which way a cache swap would shift the held object’s x endpoint. In all 114 eligible states across both policies and tasks, the two swaps produced opposite shifts relative to their unedited recipients. Each exceeded the mean absolute effect of four equal-norm edits (§5.2). The target-induced cache difference was sufficient to bias action generation.
Häon et al. steer direction and speed, and Hong et al. use contrastive activations for feedback control. Grant et al. reproduce a source trajectory by injecting π0.5 action-expert activations, including on a physical robot. We tested π0.5’s multimodal prefix against two references in the same simulated recipient scene. One moved the pad in the recipient image. The other replayed the source’s actions. We compared the resulting planar object displacements at +24, each relative to its center baseline.
We matched 14 disjoint pairs on stapler, pad color and arm and used one instruction per pair. Five met the opposite-side pad criterion. Testing both recipients in four directions gave 40 comparisons, 34 with distinguishable references. With a same-scene donor, 33/34 edits landed at least 5 mm closer to the local visual response than to replay. With a cache difference from the other scene, 15 favored the local response, 16 favored replay and 3 were ambiguous. Only 3/34 showed a local advantage at least 5 mm greater than both the zero-displacement margin and the median of four equal-norm controls. A steering method can use these two references to check whether its edit follows a new scene. In these five pairs, visual cues and prepared movements both remain plausible.
Could an operator request a 10 mm correction? We calibrated an x shift relative to the unedited +24 endpoint on 12 simulated FastWAM/stapler states. In 13 held-out states, a nominal 10 mm donor offset gave 7/13 hits each way (x error and unintended y shift each within 5 mm). Calibrated offsets of 16.7 mm left and 17.1 mm right gave 10/13 each way. Also requiring height within 5 mm gave 9/13 leftward and 7/13 rightward hits. One leftward request gave 42 mm left and 16 mm sideways. We would reobserve the object before another correction.