Causal Transfer of Scene Information in Robot Policies

An internal edit can move a robot toward a donor’s target without identifying what scene information it transferred. We study whether a controlled scene factor can be transported through a policy’s internal state into predictable physical behavior. Using target location, we generate counterfactual observations, replace key–value caches at selected recipient layers with donor caches, and execute all branches from a common simulated physical state. Our primary frozen confirmation compares the true cache difference with four fixed channel-sign transformations that preserve each selected layer’s K/V perturbation norm within 2% while changing its direction. Across two manipulation tasks, FastWAM and π0.5, 128 new model rollouts yield 117 reachable and 114 visually eligible states; all four task–model cells meet the joint criterion for bidirectional relative endpoint effects exceeding the equal-norm controls, with control-adjusted means of 51–72 mm. A separate non-target spatial comparison supports three cells, with the fourth limited by a pre-specified control-visibility requirement. In bounded two-axis development pilots (four model-visited states per policy family), own-axis effects exceeded cross-axis responses for every tested state–axis pair; these pilots are descriptive rather than independent confirmation. Independent offline reconstruction reproduces the primary effects over 2,340 branches. Together, these results show that, under the tested cache intervention and short simulated horizon, the direction of target-conditioned internal differences—not only their magnitude—can be transferred to produce predictable relative object-motion biases.