Rethinking Dense Spatial Features: Compact Spatial Queries for World Action Models
Abstract
World Action Models (WAMs) jointly model future visual observations and robot actions, transferring video priors to robotic manipulation. However, their image- derived representations leave interaction-relevant 3D geometry implicit; dense 3D features can provide complementary spatial cues, but may also introduce manipulation-irrelevant redundancy and additional learning and computational costs. We therefore systematically investigate whether WAMs benefit from ad- ditional spatial conditioning, which representations and supervision targets are effective for action generation, and how compact an effective spatial representa- tion can be. To test a compact and manipulation-aligned alternative within this study, we develop QuestWAM, which learns latent spatial queries through super- vision from future 3D bounding boxes, interaction points, and semantics, organiz- ing spatial information around where task-relevant objects will be and where the robot should interact with them. These queries condition action generation with- out requiring future annotations or dense 3D reconstruction at inference. Quest- WAM achieves 93.14% and 92.18% success on RoboTwin 2.0 Clean and Random, respectively, and improves LIBERO-Plus success from 49.43% for original Fast- WAM to 53.48%. Compared with online VGGT conditioning at the same token budget on RoboTwin 2.0 Clean, QuestWAM improves success by 1.00 percentage point. Controlled comparisons favor structured supervision over dense-feature distillation and latent queries over decoded attributes. Spatial interventions fur- ther show that the queries encode scene-dependent future geometry. These find- ings suggest that compact and manipulation-aligned spatial conditioning improves WAM action generation, with its value depending on how relevant information is selected and organized, beyond representational richness alone.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.