PhysAtObj: Learning Physics-Grounded 3D Object Interaction from Multi-view Videos

Modeling dynamic 3D scenes with interacting objects is crucial for understanding physical environments. However, existing methods struggle to extrapolate future frames because they fail to capture underlying physical dynamics. In this paper, we introduce PhysAtObj, a new framework that learns object interaction dynamics directly from multi-view videos without requiring explicit physical properties. The core of PhysAtObj is an object interaction module that identifies dynamic contact regions and directions, and then learns comprehensive impulse-induced and time-agnostic velocity changes during interactions. By relying on the physical states of interacting entities rather than absolute timestamps, the learned dynamics generalize robustly beyond the observed training window. Extensive experiments on PhysInOne and our newly introduced synthetic and real-world datasets demonstrate that PhysAtObj achieves state-of-the-art performance in future frame extrapolation, significantly outperforming baselines.