GPWM: Generative Particle World Models for 3D Prediction and Planning

We introduce GPWM a generative world model that learns 3D physical dynamics directly from particle trajectories. Its shared representation and architecture covers rigid, deformable, cloth, and granular materials, in both passive physics and robotic interaction. GPWM is an object-aware multi-resolution transformer that combines flow matching with diffusion forcing to enable streaming generation far beyond the training horizon. We train and evaluate GPWM on ParticleWorlds, a 25.5-million-frame corpus spanning diverse geometries, physical properties, interactions, and robot embodiments. When trained within individual domains, GPWM reduces prediction error by 29-53% relative to the strongest applicable prior learned-dynamics models. Going beyond the domain-specific setting of prior methods, a single Unified GPWM jointly models heterogeneous physical regimes within one set of weights. GPWM further generalizes to unseen geometries, physical properties, object counts, and rollout horizons. Beyond prediction, GPWM enables contact-rich planning. An object-space goal reward is backpropagated through the learned interaction dynamics to the latent noise, steering actuator motion and the contacts it establishes. Executed open-loop in simulation, the resulting plans outperform unguided generation and planning with prior learned dynamics models on pushing, picking, and granular manipulation.