Seeing What Matters for Action: Bridging Visual Priors and Robotic Control in Vision-Language-Action Models

Pretrained visual foundation models provide rich semantic and spatial priors for Vision-Language-Action (VLA) policies, and recent methods often transfer these priors by directly aligning external visual features with VLA representations. However, direct alignment leaves two gaps between generic perception and robotic control: it does not explicitly prioritize visual information relevant to action prediction, and frame-wise supervision leaves cross-frame relational structure only indirectly constrained. We propose ActSight, a lightweight framework for action-sensitive, temporally structured visual transfer in VLA policies. ActSight consists of two complementary components. Control-Oriented Visual Refinement weights external visual supervision by each token’s influence on the action objective, guiding lightweight residual refinement that preserves the VLA's learned representation while incorporating control-relevant visual knowledge. Temporal-Aware Representation Alignment complements frame-wise supervision by explicitly preserving relational structure across neighboring observations, encouraging the adapted representation to retain how visual states change during interaction. Across multiple manipulation tasks, ActSight improves control performance over direct feature alignment and visual-enhancement baselines. These results demonstrate the effectiveness of organizing visual transfer around action relevance and temporal relational structure.