HoAM-4D: Holistic Aware Mapping for 4D Mesh Animation and Reconstruction

While generative 3D modeling has advanced rapidly, lifting static 3D assets and monocular videos into coherent 4D mesh sequences remains challenging, as end-to-end feed-forward frameworks often lack explicit video–mesh alignment, while view-dependent optimization methods struggle to model authentic 4D motion, resulting in inconsistent image observations or disordered trajectories. To address these challenges, we present HoAM-4D, a multi-stage framework that introduces holistic vertex awareness through 4D kinematic flow and video-to-mesh mapping for coherent 4D mesh generation. Specifically, to improve 4D motion modeling, HoAM-4D projects both the 3D mesh and video observations into 3D space and predicts geometry-motion and image-motion displacement fields. To address video–mesh misalignment, it further predicts image-to-canonical-mesh and image-to-synchronous-mesh correspondence fields. Beyond visible regions, we extend the video-to-mesh mapping to occluded areas through skeleton-based conduction. Finally, we leverage both the input video and 4D motion as supervision to optimize holistic-aware mesh deformation from the first-frame canonical mesh, enforcing articulated structural, temporal, and appearance consistency. Extensive experiments demonstrate that HoAM-4D outperforms existing methods in geometric accuracy, motion coherence, and video-aligned rendering quality.