- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes FUHA, a feed-forward human image animator using a compact learned representation of per-frame pose/expression. A frozen DINOv3 encoder and Q-Former compress a driver image into two 512-dimensional motion tokens. A separate appearance encoder and transformer decoder synthesize the reference person in the driver's configuration through a frozen RGB tokenizer. A triplet objective pulls driver embeddings toward embeddings of rendered keypoints and away from another pose of the same identity, while reconstruction and identity losses train the renderer. Keypoint estimators are used during training/data preparation, but not at inference. A single model handles face, upper-body, and full-body settings. Validation includes extensive reconstruction/cross-person comparisons, component ablations, retrieval/interpolation probes, a diffusion-decoder variant, and timing. The faster variant reaches 38.9 model FPS at 512x512 on a 4090 with the reported optimized inference setup. Upper-body self-reenactment and cross-person identity scores are strong, while cross-person/full-body perceptual video quality is more mixed.
Critical strengths
-
This paper proposes a useful quality/throughput regime for unified learned-motion animation. FUHA extends a stateless, single-pass renderer across face, hands, upper body, and full body. In upper-body experiments it combines strong self-reenactment scores and high cross-person identity similarity with a much faster inference regime than the evaluated multi-step alternatives.
-
This paper presents several complementary tests to support the usefulness of the learned motion representation. The contrastive-loss ablation causes a major degradation, motion retrieval improves body-pose similarity over global encoder embeddings, and the frozen embedding can condition a separate diffusion renderer.
Critical weaknesses
- The methodological novelty is limited, with the contribution primarily consisting of a combination of existing system components(X-UniMotion, IM-Animation).
Questions for authors
See weaknesses.
2 comments
Thank you again for your review. As the discussion period closes on Nov 18, we would be glad to answer any remaining questions.
Thank you for the review and the positive assessment.
FUHA starts from what current animation methods leave open. Compact learned motion latents mostly cover the face. Body methods take skeletons from pose estimators, which carry little expression and fail under occlusion. The two whole-body embeddings, X-UniMotion and IM-Animation, rely on crops, synthetic or generated identity pairs, and multi-step diffusion. FUHA is designed to close these gaps in one framework: one embedding for face, hands and body, no per-frame preprocessing, identity separation learned from real video, and single-pass rendering.
The table compares FUHA with these two methods:
At inference, FUHA takes only the reference image and the driver frame: no pose estimator, no face or hand crops, nothing to localize per frame. X-UniMotion reports that one global descriptor misaligns gaze, lips and fingers. FUHA's single embedding nonetheless gives the lowest upper-body self-reenactment hand and expression errors (AKD-H 48.5 vs. 65.6, AED 0.315 vs. 0.467; Table 1).
Keypoints enter only in training, as the triplet's positive, and reconstruction trains the embedding end to end, so it is not limited to what they encode: driving FUHA with keypoint images instead raises self AKD-H from 48.5 to about 85 (reply to spE1, Q1). The same loss keeps identity out using real video only. In retrieval, the embedding returns the query's own person in 0.1% of queries, against 9.5% for DINOv3 (Table 6); without the loss, cross-ID falls from 0.701 to 0.057 (Table 4).
Finally, the embedding is not tied to its renderer. Kept frozen, it also drives Wan-Animate (FUHA-W, App. A.3), whereas X-UniMotion trains its tokens jointly with its generator.
Some ingredients are known on their own, such as DINOv3 features, Q-Former bottlenecks and triplet losses. Our contribution is how they are combined to address these gaps, and Table 4 and the new controls show that each part contributes. We will add this discussion to the revision.