TriFactorHOI: Structured Diffusion for Controllable and Contact–Consistent Human–Object Interaction Synthesis

Recent diffusion models have advanced human–object interaction (HOI) generation. However, generating coordinated human–object motion from sparse waypoints while following user-specified routes and maintaining stable contacts remains challenging. We propose TriFactorHOI, a structured diffusion framework that organizes HOI generation through object path, part, and contact factors. For trajectory control, a reference path interpolated from sparse waypoints serves both as a generation condition and as the baseline for planar object translation, with learned residuals accommodating interaction-related deviations. A part-based spatiotemporal denoiser preserves separate temporal and part axes to model within-part dynamics and cross-part human–object coordination. Hand–object and foot–ground guidance refine contacts during denoising. After sampling, a full-sequence foot-refinement network predicts support probabilities and correction confidence for world-space foot anchoring and temporal inverse kinematics. Experiments on FullBodyManipulation demonstrate improved waypoint adherence, motion-distribution quality, and contact stability. With the reference-path condition held fixed, residual prediction reduces waypoint error and FID relative to absolute planar translation. Tests on unseen 3D-FUTURE objects without retraining further support generalization to new object geometries. The code and processed datasets will be made publicly available.