Phys4D: Injecting Physical Structure into Video Diffusion via Lifted RGB-D-Motion 4D Interfaces

Recent video diffusion models generate increasingly realistic videos, yet visual plausibility alone does not imply physical understanding. Physical laws govern evolving 3D scene states, while videos observe only their 2D projections. We introduce Phys4D, a structure-focused, appearance-constrained physics injection framework that transfers simulation-derived physical structure into pretrained video diffusion models while preserving their real-video generative prior. Phys4D operationalizes physics-aware video generation through a prior-preserving RGB-D-motion interface. Our key principle is simulation teaches physics, not appearance: simulation provides dense structural supervision, including depth, motion, and masks, while adaptation remains anchored to the pretrained real-video prior rather than drifting toward simulator-specific visual statistics. To this end, Phys4D exposes geometry and motion with lightweight prediction heads, injects local physical structure through gated geometry-motion supervision, and aligns sequence-level evolution using a simulation-grounded trajectory objective. Supported by a scalable coupled-physics simulation system, Phys4D improves external RGB physics scores and controlled geometry-motion diagnostics across multiple video generation backbones while maintaining visual quality. The simulated diagnostic split tests unseen templates within supported physical regimes. We plan to release code and datasets upon acceptance. Project page: https://sensational-brioche-7657e7.netlify.app