- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a diffusion-based framework for personalized talking portrait generation. Its two main components are group-to-individual appearance alignment (AG2I) and geometry–motion fusion (FG&M). These components combine feature decomposition, individual-guided modulation, and multi-scale motion modeling. Experiments report improvements over six diffusion-based baselines, supported by component ablations, a user study, and a lightweight distilled variant.
Critical strengths
This paper addresses a talking head problem: preserving distinctive appearance details while generating coordinated facial motion. The reported quantitative improvements span visual quality, identity similarity, and lip synchronization. The component ablations also suggest complementary roles for appearance alignment and motion fusion. Including perceptual evaluation and a distilled model broadens the assessment beyond image reconstruction quality alone.
Critical weaknesses
The method is insufficiently specified, with questionable objective formulations. Although the task takes a portrait image and audio, both appearance and motion representations are defined using a frame sequence. It remains unclear where these frames originate during inference and how audio enters the computational pipeline. The operational distinction between “individual” and “group” features is also unclear; decorrelation alone does not establish the claimed semantic separation. More concretely, the entropy loss in Eq. (19), repeated in Eq. (80), is positive entropy. Minimizing it with a positive coefficient encourages concentrated scale weights, contrary to the stated goal of preventing scale collapse. The negative squared-distance term in Eq. (13) also permits unbounded separation unless additional constraints are imposed. These issues prevent a reliable assessment of the implemented method.
The evaluation does not sufficiently establish generalization or explain the reported gains. The hybrid dataset is divided 7:3, but the paper does not specify whether identities and source videos are disjoint, or how overlap among constituent datasets is removed. Consequently, performance on unseen identities remains uncertain. The substantial reconstruction gains require a clearly defined reference-image selection and evaluation protocol. Baseline training descriptions do not adequately distinguish pretrained initialization from retraining or specify comparable inference settings. There is also a concrete reporting inconsistency: the AniPortrait and MoDiTalker result rows are exchanged between Tables 1 and 3. Finally, the ablations remove entire modules without comparable-capacity replacements, so they do not isolate the proposed mechanisms from additional model capacity.
Questions for authors
What exactly is available during inference, and how are the frame sequences, audio features, and group representations obtained without access to the target video?
What loss signs, coefficients, and feature constraints are actually implemented for Eqs. (13) and (19)? Can corrected formulations and corresponding ablations be provided?
Are evaluation identities and source videos excluded from training, and which AniPortrait/MoDiTalker results are correct?