SPUD: Skeletal Physics-based human pose estimation Using Diffusion

We propose SPUD as a diffusion-based framework for physics-based human pose estimation (PHPE), regularized by a novel kinetics prior and a novel SKEL-based pose prior. We also present a fencing motion-capture video dataset, synchronized with force plates, to allow for quantitative evaluation of kinetics estimation for athletic human movements. PHPE methods aim to improve realism by enforcing the laws of dynamics. However, inverse dynamics optimization without force annotations is challenging, as it is an underconstrained problem. To address this, we turn to diffusion models and their expressiveness in learning complex distributions, as recent studies have demonstrated impressive results for solving inverse problems via variational diffusion sampling. We build on these frameworks for inverse kinematics, and extend them to inverse dynamics by training a novel kinetics prior on simulation data. Furthermore, for improved compatibility with downstream biomechanics analysis, SPUD is the first PHPE method to infer human dynamics for the SKEL human model. Lastly, we conduct extensive experiments on various inverse kinematics and inverse dynamics tasks to demonstrate SPUD's versatility, and on datasets with disparate human movements to demonstrate improved adaptability. This includes a force-annotated fencing dataset we collected, which can facilitate more robust evaluations for future PHPE research.