- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a novel framework for cross-species portrait animation, to metigate the leakage of species-specific feature during motion and expression transferring. There are three major contributions:
- A cross-species portrait-motion alignment framework which disentangles portrait features into rigid head motion represented with dense pose-flow, and facial deformation represented with shared implicit-keypoint tokens.
- A species-routed expression encoder, preserving capacity for anatomical differences while providing a shared expression-conditioning interface by species-specific LoRA adapters.
- An expression-level adversarial domain adaptation (Ex-ADA) to mitigate species leakage.
The authors conduct major experiments on datasets containing three different species (human, cat and dog) and visualize some results on unseen species. Qualitative/quantitative evaluation and a user study demonstrate better performance over previous methods.
Critical strengths
-
The proposed method is technically sound. How to disentangle motion, expression and identity-specific feature is one of the biggest challenges in motion/expression transferring for generation. The integration of species-specific LoRA adapters and adversarial expression alignment provides a reasonable approach under the settings considered in this paper.
-
The qualitative and quantitative results are promising.
Critical weaknesses
My main concerns are related to the generalizability, scalability and architectural limitation, most of which the authors have acknowledged in the limitations section. Although in the appendix the authors show results with unseen species as the source, it seems that framework cannot directly handle unseen species as driving inputs, since no corresponding adapter exists for an unseen species label and the classifier could be incompatible. Furthermore, the experiments are conducted on only three species, which is a relatively limited number. Considering the design of LoRA tuning and classifier are directly related to the number of labels, it remains unclear whether the proposed method can maintain its performance when trained on a larger and more diverse set of species.
A minor concern is about the quantitative evaluation. For A to H, the authors calculated AED of SMIRK features, APD of SMIRK features and CSIM of ArcFace features for expression, pose and identity, respectively. I understand that it is difficult to directly evaluate the results regarding these similarities since there is no ground truth nor golden standards for evaluation for generation tasks. And I agree that measuring feature distances using relevant pretrained feature extractors is a reasonable approach. However, it would be more convincing if the authors could report results obtained using multiple different feature extractors for each evaluation aspect. This would help mitigate potential biases introduced by relying on a single pre-trained extractor for each metric. Some candidates for consideration: AdaFace, 3DDFA, EMOCA, Action Units, etc.
Questions for authors
Please see the weaknesses
Follow-up on Author Responses
Dear Reviewers,
Thank you again for your time and thoughtful feedback. We would appreciate your feedback on whether our responses address your concerns. If any questions remain unresolved, we would be happy to provide further clarification.
If our responses have resolved your concerns, we would be grateful if you could consider updating your assessment and score accordingly.
Thank you for your consideration.
Best regards,
The Authors