LEON: Operator-Structured Transitions for World Action Models
Abstract
Latent world models for embodied agents model how system states evolve under action. In robot manipulation, World Action Models (WAMs) couple such predictions to policy learning as supervision or future conditions for action generation. Yet their latent transitions typically rely on general-purpose predictors, leaving dynamical structure to be learned implicitly from data. We derive a latent-transition formulation from controlled mechanical evolution. A common finite-dimensional projection of observable dynamics yields a shared operator family affine in physical control and an explicit projection residual. Guided by these results, we introduce the Latent Evolution Operator Network (LEON), which adapts pretrained visual features into learned observables, learns latent control coefficients from the WAM context, and combines factorized operator evolution with complementary additive updates. We instantiate LEON in prediction-supervised policy learning and future-conditioned action generation while preserving the original prediction targets and policy interfaces. Across LIBERO, LIBERO-Plus, and RoboTwin 2.0, LEON improves average success rates over the corresponding WAM baselines and matched transition controls. Ablations support the joint representation and transition design, and controlled-system diagnostics show improved state-range extrapolation. These results establish dynamics-derived latent-transition structure as an effective inductive principle for latent WAMs.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.