acceptodds
Under review as a conference paper at ICLR 2027

What Should a World-Action Model Imagine? 4D Geometry as an Intermediate Representation for Control

Abstract

World Action Models (WAMs) imagine future observations to guide control, yet it remains unclear what a model should imagine for action. Most existing WAMs predict RGB futures, where physical structure and motion are entangled with texture, illumination and observer motion. We systematically study 4D geometry as a predictive intermediate representation for control. Rather than asking whether geometric supervision is useful, we ask when and how an imagined geometric future should participate in action generation. We represent the observable 4D state with world-frame pointmaps and scene flow, which jointly describe where visible surfaces are and how the same physical points move, together with wrist-view depth for complementary local geometry. Our controlled studies reveal several consistent findings. First, geometry is most useful when explicitly imagined and made directly accessible to action, outperforming feature alignment and auxiliary-only geometric supervision. Second, geometric and RGB futures are complementary rather than substitutable: removing either condition degrades control. Third, the value of 4D depends critically on its form and interface: world-frame pointmaps and scene flow outperform projective depth-and-optical-flow cues; and geometric supervision should be selected by view-specific reliability. These findings translate into substantial robustness gains. The resulting WAM improves a matched LingBot-VA baseline from 73.73% to 82.73%, with the largest gains under sensor noise, lighting, background, and viewpoint changes.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.