acceptodds
Under review as a conference paper at ICLR 2027

SpaVLA: From Explicit 3D Spatial Understanding to Robotic Manipulation

Abstract

Vision–language–action (VLA) models inherit rich semantic knowledge from pretrained vision–language models (VLMs), yet developing 3D spatial understanding and using it to guide action generation remain challenging. We propose SpaVLA, a spatially enhanced VLA model that connects geometric perception to action generation through explicit, manipulation-relevant 3D spatial understanding. To enhance the 3D spatial capabilities, we first construct ManiSpatial-565K, which pairs robot manipulation data with 3D spatial question-answering (QA) annotations to provide fine-grained, manipulation-relevant supervision. To support reasoning about these spatial relationships, we introduce a 3D-aware module that incorporates geometric representations into SpaVLA's VLM backbone. We then introduce a progressive training strategy to align this enhanced spatial understanding with robotic control. By conditioning the action expert on fine-grained spatial reasoning generated by the VLM, SpaVLA connects geometric perception to action generation, enabling the VLA model to act on manipulation-relevant 3D spatial relationships. To evaluate how well this spatial understanding supports manipulation, we further introduce SimplerEnv-Spatial, a 20-task benchmark for spatial referring and reasoning. SpaVLA achieves 48.6% on MMSI-Bench and 95.6% on MindCube, with manipulation success rates of 84.4% on Google Robot under SimplerEnv's Visual Matching protocol and 65.6% on SimplerEnv-Spatial. Real-world experiments further demonstrate spatial instruction following under object rearrangement, destination relocation, and illumination changes. The dataset and code will be publicly available.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.