Differentiable Cross-Object Contact Reconstruction from Real-World Collision Videos

Reconstructing real-world collisions from video requires identifying both how objects deform and how they exchange momentum through contact. Real-world recordings expose these coupled dynamics under occlusion, reconstruction errors, and mismatch between physical behavior and simulation models. We present an object-aware differentiable framework that jointly fits object-level material parameters and inter-object contact parameters from calibrated multi-view video. Separated Material Point Method states preserve each object's constitutive response, while contact geometry derived from reconstructed surfaces drives two-way impulse exchange. Differentiating through a finite projected contact solver connects surface and image observations to material response, friction, and optional adhesion. We also introduce Contact4D, a calibrated multiview dataset of real-world collisions between objects in free fall across diverse material pairings, with persistent instance labels and reconstructed surface observations. Comparisons with adapted reconstruction baselines show improved agreement with observed motion, geometry, and appearance. Matched ablations demonstrate the contribution of inter-object contact and the treatment of ambiguous surface observations, while numerical audits assess the consistency of contact derivatives. Together, the method and dataset support joint material and better contact reconstruction from real-world collision videos.